From f254f5ac2b804b353a711264bb178f43fdf07867 Mon Sep 17 00:00:00 2001 From: Ysopwnable <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 10:21:53 +0800 Subject: [PATCH 001/159] feat(android): add opt-in package summaries and retained manifest views Add explicit package summary delivery and exact retained manifest pages while preserving complete MCP defaults, one-shot CLI output, provider requests, source identity and session lifetime. Return distinct linked Evidence and preserve partial/null facts; reject unavailable retention before engine work. Validation: 27 relevant SDK/domain/Android boundary checks, focused test-only correction rerun, full supported compiler/lint and docs checks, real NewPipe SDK pairs, authentic old Evidence continuation, four actual Codex trials with failures/regressions retained, and all 39 exact-head CI checks. Package replies shrink 119,527 to 8,659 bytes; no model-token or time gain is established. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- docs/android-analysis.md | 21 ++ experiments/analysis-efficiency/README.md | 5 + .../android-package-summary.md | 166 ++++++++++ experiments/analysis-efficiency/measure.mjs | 44 ++- .../android/AndroidPackageInspection.ts | 43 +++ src/contracts/analysisViewToolContracts.ts | 2 +- src/contracts/android/androidToolContracts.ts | 16 +- src/domain/analysisView/analysisView.ts | 30 +- src/domain/analysisView/androidPackageView.ts | 136 ++++++++ .../analysisView/javascriptApplicationView.ts | 2 +- src/domain/android/androidAnalysis.ts | 6 + src/server/createServer.ts | 1 + src/server/registerAndroidTools.ts | 21 +- .../mcp/androidPackageSummaryMcp.test.ts | 295 ++++++++++++++++++ 14 files changed, 777 insertions(+), 11 deletions(-) create mode 100644 experiments/analysis-efficiency/android-package-summary.md create mode 100644 src/application/android/AndroidPackageInspection.ts create mode 100644 src/domain/analysisView/androidPackageView.ts create mode 100644 tests/boundary/mcp/androidPackageSummaryMcp.test.ts diff --git a/docs/android-analysis.md b/docs/android-analysis.md index 5a4a2f879..d98d9b5da 100644 --- a/docs/android-analysis.md +++ b/docs/android-analysis.md @@ -9,6 +9,27 @@ This family is present on repository main and in npm 4.1.0. Check the [released package boundary](installation.md#released-package-and-main) and the connected server's tool list before selecting it. +## Optional package delivery + +The MCP `inspect_android_package` request keeps complete output when `detail` is +omitted or `"complete"`. Select `detail: "summary"` explicitly to receive all +package, SDK and permission declarations, engine identity and class/resource +counts without decoded manifest text or duplicate raw provider replies. A +retaining session keeps the complete original Evidence and returns a distinct +linked `inspect_analysis_view` summary. This changes delivery, not JADX analysis. + +Use the returned `normalized_result.parent_evidence_id` with +`inspect_analysis_view`, source kind `retained-evidence`, and +`view: {kind: "android", facet: "manifest", offset: 0, limit: 4096}` to read the +exact decoded manifest without reopening the APK. Offsets and limits count UTF-16 +code units; follow `coverage.next_offset`. The page preserves `source_status` and +`reported_total_bytes`: exhausting retained text does not make a partial decoded +manifest complete. Complete source Evidence remains exportable under the existing +session lifetime. Servers without retention reject summary before starting JADX. +One-shot CLI output remains complete. Older servers require the existing complete +request after schema discovery. APK authenticity and runtime behavior remain +unverified. + ## Supply tools Use a **full JDK 17 or newer** (including its compiler module) and diff --git a/experiments/analysis-efficiency/README.md b/experiments/analysis-efficiency/README.md index 79bac97e0..8afef3a66 100644 --- a/experiments/analysis-efficiency/README.md +++ b/experiments/analysis-efficiency/README.md @@ -37,6 +37,11 @@ raw MCP replies from observed client execution/discovery and usage. Optional trial capture retains tool/usage items while excluding account and ambient state; existing ephemeral trials keep their default behavior. +Follow-up: [Android package delivery and retained manifest](android-package-summary.md) +records real NewPipe SDK pairs, unchanged old requests/Evidence, exact retained +text, and actual Codex outcomes including client errors and a failed discovery. +Reply bytes fall; model-token/time gains are not established. + Follow-up: [resource counters at payload completion](resource-counters.md) adds optional owned counter observations while preserving unknown whole-scope final counters, default receipt shape and the unchanged CPU/memory safeguards. diff --git a/experiments/analysis-efficiency/android-package-summary.md b/experiments/analysis-efficiency/android-package-summary.md new file mode 100644 index 000000000..448202d87 --- /dev/null +++ b/experiments/analysis-efficiency/android-package-summary.md @@ -0,0 +1,166 @@ +# Android package summary and exact retained manifest + +Continues [#2016](https://github.com/morluto/rea/issues/2016) and +[#2438](https://github.com/morluto/rea/pull/2438). The recurring NewPipe question +asks for APK/package/version identity, all case-sensitive MainActivity matches +and all reported onCreate entries, including explicit null DEX descriptors. +Complete package replies delivered decoded manifest text and duplicate upstream +calls unrelated to those selected facts. + +## Compatibility and ownership + +MCP `inspect_android_package` accepts explicit `detail: "summary"`; omitted and +`"complete"` requests preserve the complete output and provider request. The +one-shot CLI stays complete. Summary selection happens in the application +workflow, independently of the engine protocol, using the existing retained +analysis-view machinery. No third-party plugin API or persistent analysis cache +is introduced. Servers lacking retention reject summary before provider work. + +The complete original Evidence stays under its original ID and session lifetime. +A distinct linked derived view returns all package/SDK/permission declarations, +engine identity, class/resource counts and signature-verification status. +Manifest metadata retains the producer's partial/complete status, nullable +reported byte total, and observed UTF-16 length. Text and duplicate raw calls +remain in the original record, rather than being rewritten under its ID. + +Use `inspect_analysis_view` with the parent's ID and +`view: {kind: "android", facet: "manifest", offset: 0, limit: 4096}` to read exact +retained decoded text. Follow `coverage.next_offset`; surrogate boundaries use +the existing text-page implementation. Exhaustion describes the retained text, +not global completeness when the producer says partial. APK signatures and +runtime behavior remain unverified. Engine artifact SHA-256 identifies the JAR; +views require the authenticated APK subject and never substitute the engine +hash for a missing target identity. Older servers require schema discovery and +the original complete request; automatic fallback is not introduced. + +## Frozen runtime and real SDK pairs + +Source runtime: `28c84e78cf2fbfde4aceed345b773c59eb889f68`; later test-only schema +typing corrections do not relabel these receipts. Compiled dist/bridge file-table +SHA-256: `5332dda13981f043e146b3601ebec1f380d9f698280ca53d75786b638a912e11`. +The frozen old runtime is `8bc1601ca51e417ca472d90ea0367e9418a1a0f2`, digest +`9e7380f537301bf0cfe2926f5619011af848af7c220750e75189498e65d825e1`. + +Real NewPipe 0.29.1 bytes, acquisition and digest are pinned in `corpus.json`. +Actual configured JADX headless MCP 0.7.1/Java runs use the same selected +provider settings and search-path digests, one worker and 768 MiB Java heap. +Node is 24.18.0; actual SDK client/server are 2.3.1. Processes and engines are +fresh per trial; host page caches are not flushed. SDK schema mode is selected +in both treatments. Heavy runs are serial, with one CPU, unchanged 3 GiB early +stop/4 GiB hard memory, no swap, and 192 tasks. + +All four candidate workflows use three successful public MCP calls. Only the +first package delivery changes. Both pairs pass settings and selected-fact +comparison, including all declarations, manifest status/length, engine identity, +original limitations, matching class rows and onCreate entries. Original replies +and IDs remain separate. A fresh old-runtime complete workflow also passes +comparison against the candidate complete request; both complete package replies +are exactly 119,527 compact client-result JSON bytes. This is scoped compatibility +evidence, not a guarantee for every historical release/client/host. + +| Trial | Package reply bytes | SDK workflow seconds | Payload CPU seconds | Kernel peak bytes | +| -------------- | ------------------: | -------------------: | ------------------: | ----------------: | +| sdk-complete-1 | 119,527 | 24.059 | 25.259 | 1,225,076,736 | +| sdk-summary-1 | 8,659 | 23.694 | 24.894 | 1,226,870,784 | +| sdk-summary-2 | 8,659 | 24.407 | 25.611 | 1,228,296,192 | +| sdk-complete-2 | 119,527 | 24.509 | 25.740 | 1,229,844,480 | + +Summary replies are 8,659 bytes, about 92.8% smaller; all three replies together +fall from 149,577 to 38,709 bytes. This is a delivery result, not a model-token +or engine-work saving. The SDK timing differences are small. CPU and memory do +not establish a reduction. No provider setting or analysis scope changes. +Transport roundtrip includes server/client work and is not an engine-only timer. + +The complete catalog is 2,506,213 compact JSON bytes versus 2,497,635 on the old +runtime: about 0.34% growth, including intervening main changes. Discovery cost +remains visible alongside selected-delivery savings. + +## Authentic old Evidence continuation + +A real package record from old runtime 8bc160 imports into the candidate and +supports retained summary, exact manifest pages, inline portable summary and +export through actual public SDK tools. The new session explicitly selects an +absent JADX JAR; this workflow performs no package analysis. Eleven successful +calls preserve the original record, original ID, all declarations and all 25,222 +manifest UTF-16 units. Manifest UTF-8 SHA-256: +`fee192fe2a6d7892c22cba115184c74c7e2c42ce464e8bb37be6289f2c16881d`. +The exported original record is deeply equal to the imported record. + +## Actual Codex task outcomes + +Four serial trials run in complete/summary/summary/complete order on the same +frozen candidate. Requested configuration: Codex CLI 0.161.0, `gpt-5.6-terra` high, +three exact selected tool names and the same optional context/discovery capture +settings. The requested alias does not certify the server-resolved model. The +original analyst question is unchanged; equal treatment instructions explicitly +choose package detail and explain the summary carrier. Temporary configuration +points to the frozen modified REA MCP; no global configuration is edited. + +| Trial | Input | Cached | Uncached | Output | Seconds | MCP calls | Usage records | Facts | +| ---------------- | ------: | -----: | -------: | -----: | ------: | --------: | ------------: | ----- | +| model-complete-1 | 39,267 | 23,040 | 16,227 | 1,690 | 94.053 | 3 | 3 | Pass | +| model-summary-1 | 128,417 | 93,440 | 34,977 | 3,633 | 150.401 | 8 | 9 | Pass | +| model-summary-2 | 54,437 | 37,120 | 17,317 | 2,185 | 123.677 | 3 | 4 | Pass | +| model-complete-2 | 21,332 | 9,984 | 11,348 | 667 | 20.774 | 0 | 2 | Fail | + +Three answers pass the original selected-fact gate against the actual SDK +producer. Each trial preserves APK bytes and removes its isolated profile. +The first summary trial makes five successful calls and three rejected requests: +an invented `case_sensitive` argument, empty class-search input and empty +class-inspection input. It then repeats successful package/search work. These +errors and correction turns remain part of measured task cost; strict existing +schemas correctly reject them. The second summary trial uses the intended three +successful calls, but still does not establish improved total/uncached input or +latency against the factually acceptable first complete control. + +The final complete control's first client executor filters `ALL_TOOLS` for the +same three exact names and receives an empty array. It makes zero MCP calls, +reports all requested facts unknown and fails the factual gate. Identical name +filters worked in the other trials. Stderr contains no explanatory error; the +client visibility failure remains unexplained and is not attributed to REA or +summary delivery. Its short duration is not comparable completed work. The +original failed receipt is retained rather than replaced by a successful trial. + +These measurements demonstrate no real-model token or speed gain. Smaller raw +MCP envelopes do not establish smaller model context; observed usage also +includes discovery, execution context, corrections and extra responses. Usage +records count captured usage items rather than asserting exact remote request +contents or equating MCP calls with model rounds. The complete default and +opt-in choice remain unchanged; this is useful delivery/retention capability, +not a general performance recommendation. + +All model runs have verified payload-completion witnesses with only the observer +remaining and zero memory/task limit events at that phase. Kernel peaks are +602,472,448–1,444,945,920 bytes; the failed no-analysis control is included in that +range and is not evidence of lower analysis memory. Whole-scope final counters +remain unknown and the existing full resource grader stays false. No broader +CPU, memory, OS/provider or model claim is made. + +## Resource interpretation + +Every SDK run and portable continuation has a verified owned +payload-exited-observer-alive witness, with only the observer remaining and no +memory/task limit events at that phase. Kernel peak and CPU include that opt-in +observer. Whole-scope terminal counters remain unknown; the existing complete +physical resource grader is not changed or called successful. No OOM stress or +resource-limit increase is used. + +## Verification and scope + +Twenty-seven relevant converted-schema, SDK, domain and existing Android +failure/cancellation/cleanup checks pass. A three-case focused rerun passes after +a test-only schema typing correction. The 1 MiB manifest protocol fixture verifies +original default/full output, reconstruction after source removal, unchanged +exported Evidence and one engine launch. Other cases preserve nulls and partial +status, Unicode boundaries and tagged malformed/missing-identity failures, and +reject unavailable retention or cancelled requests before provider work. These +fixtures verify production boundaries; actual JADX claims use the real NewPipe +runs above. Full supported single-thread compiler and unchanged lint pass under +the same memory safeguards. No other Android OS/device/provider lane is inferred. + +Raw SDK/model replies, private profiles and resource receipts stay outside Git. +Reproduce SDK pairs using the existing `measure.mjs` with `--artifact newpipe.apk`, +`--schemas selected`, identical explicit provider configuration and frozen runtime, +and alternating `--delivery complete` / `--delivery summary`. Preserve failed +attempts and run `compare.mjs` on both pair orders. Model findings require their +own identical analyst facts and explicit delivery controls. diff --git a/experiments/analysis-efficiency/measure.mjs b/experiments/analysis-efficiency/measure.mjs index 3e34d935e..bfac3adb6 100644 --- a/experiments/analysis-efficiency/measure.mjs +++ b/experiments/analysis-efficiency/measure.mjs @@ -666,7 +666,40 @@ const workflows = { } }, "newpipe.apk": async () => { - const pkg = await call("inspect_android_package", { path: values.path }); + const packageInput = { + path: values.path, + ...(values.delivery === "summary" ? { detail: "summary" } : {}), + }; + const definition = catalogDefinitions.get("inspect_android_package"); + if ( + definition === undefined || + !new Ajv2020({ strict: false, validateFormats: false }).validate( + definition.inputSchema, + packageInput, + ) + ) + throw new Error( + "Selected Android delivery is not advertised by this runtime", + ); + const pkg = await call("inspect_android_package", packageInput); + const result = normalized(pkg); + const report = values.delivery === "summary" ? result.summary : result; + if ( + values.delivery === "summary" && + (result.kind !== "summary" || + result.parent_operation !== "inspect_android_package") + ) + throw new Error("Runtime returned no retained Android package summary"); + const { manifest, ...declarations } = report; + const manifestObservation = + values.delivery === "summary" + ? manifest + : { + status: manifest.status, + reported_total_bytes: manifest.reported_total_bytes, + text_code_units: manifest.text.length, + text_included: false, + }; const found = await call("search_android_classes", { path: values.path, query: "MainActivity", @@ -677,9 +710,12 @@ const workflows = { }); return { artifact_sha256: targetSha(pkg), - package_name: normalized(pkg).package_name, - version_name: normalized(pkg).version_name, - version_code: normalized(pkg).version_code, + package_name: report.package_name, + version_name: report.version_name, + version_code: report.version_code, + package_declarations: declarations, + manifest_observation: manifestObservation, + package_limitations: pkg.limitations, matches: normalized(found).matches, coverage: normalized(found).coverage, onCreate: normalized(cls).methods.filter( diff --git a/src/application/android/AndroidPackageInspection.ts b/src/application/android/AndroidPackageInspection.ts new file mode 100644 index 000000000..3fbaa2f1b --- /dev/null +++ b/src/application/android/AndroidPackageInspection.ts @@ -0,0 +1,43 @@ +import type { ExecutionOptions } from "../AnalysisProvider.js"; +import type { AndroidAnalysisService } from "./AndroidAnalysisService.js"; +import { + requireAnalysisViewRetention, + summarizeRetainedAnalysis, + type AnalysisViewRetention, +} from "../analysisView/AnalysisViewService.js"; +import type { AnalysisError } from "../../domain/analysisErrorBase.js"; +import { AnalysisCancelledError } from "../../domain/analysisErrorCore.js"; +import { androidPackageInspectionInputSchema } from "../../domain/android/androidAnalysis.js"; +import type { Evidence } from "../../domain/evidence.js"; +import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js"; +import { err, type Result } from "../../domain/result.js"; + +/** Preserve the complete provider request; select retained delivery explicitly. */ +export const inspectAndroidPackage = async ( + service: AndroidAnalysisService, + rawInput: unknown, + retention: AnalysisViewRetention, + options?: ExecutionOptions, +): Promise> => { + const operation = "inspect_android_package"; + if (options?.signal?.aborted === true) + return err(new AnalysisCancelledError(operation)); + const parsed = androidPackageInspectionInputSchema.safeParse(rawInput); + if (!parsed.success) + return err( + analysisInputErrorFromIssues(operation, parsed.error.issues, rawInput, { + cause: parsed.error, + }), + ); + if (parsed.data.detail === "summary") { + const ready = requireAnalysisViewRetention(operation, retention); + if (!ready.ok) return ready; + } + const executed = await service.execute( + operation, + { path: parsed.data.path }, + options, + ); + if (!executed.ok || parsed.data.detail !== "summary") return executed; + return summarizeRetainedAnalysis(executed.value, retention); +}; diff --git a/src/contracts/analysisViewToolContracts.ts b/src/contracts/analysisViewToolContracts.ts index 682c49e6b..a4004e85b 100644 --- a/src/contracts/analysisViewToolContracts.ts +++ b/src/contracts/analysisViewToolContracts.ts @@ -16,7 +16,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [ ...toolContractMetadata("inspect_analysis_view"), kind: "application", description: - 'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_evm_interface, inspect_managed_artifact or inspect_web_network_capture Evidence without repeating analysis. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. For layout, JavaScript, native, inventory, WASM, EVM and managed artifact sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Native views select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', + 'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_evm_interface, inspect_managed_artifact, inspect_android_package or inspect_web_network_capture Evidence without repeating analysis. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. For layout, JavaScript, native, inventory, WASM, EVM, managed artifact and Android package sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Native views select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', inputSchema: inspectAnalysisViewInputSchema, outputSchema: evidenceResultOf(analysisViewResultSchema), examples: [ diff --git a/src/contracts/android/androidToolContracts.ts b/src/contracts/android/androidToolContracts.ts index df2cccf59..b555fcbea 100644 --- a/src/contracts/android/androidToolContracts.ts +++ b/src/contracts/android/androidToolContracts.ts @@ -1,7 +1,10 @@ +import { z } from "zod"; import { androidInputSchemas, + androidPackageInspectionInputSchema, androidResultSchemas, } from "../../domain/android/androidAnalysis.js"; +import { androidPackageSummaryViewResultSchema } from "../../domain/analysisView/analysisView.js"; import type { ToolContract } from "../toolContractTypes.js"; import { evidenceResultOf } from "../toolOutputSchemaPrimitives.js"; import { toolContractMetadata } from "../toolEffects.js"; @@ -12,14 +15,21 @@ export const ANDROID_TOOL_CONTRACTS = [ name: "inspect_android_package", ...toolContractMetadata("inspect_android_package"), description: - "Inspect a local APK's package, SDK and permission declarations, decoded manifest and class/resource counts. Does not execute the APK or verify signatures. Requires a caller-supplied JADX MCP JAR and JDK; uses one worker in an isolated temporary workspace.", + 'Inspect a local APK\'s package, SDK and permission declarations, decoded manifest and class/resource counts. Omitted or complete detail preserves the complete result. Select detail: "summary" for all declarations, engine identity and counts without decoded manifest text or duplicate raw calls; retains the complete source and returns a distinct linked inspect_analysis_view summary. Read the exact retained manifest with view kind android, facet manifest, offset and limit (UTF-16 code units), using normalized_result.parent_evidence_id. Does not execute the APK or verify signatures. Requires a caller-supplied JADX MCP JAR and JDK; uses one worker in an isolated temporary workspace.', kind: "android-provider", - inputSchema: androidInputSchemas.inspect_android_package, + inputSchema: androidPackageInspectionInputSchema, outputSchema: evidenceResultOf( - androidResultSchemas.inspect_android_package, + z.union([ + androidResultSchemas.inspect_android_package, + androidPackageSummaryViewResultSchema, + ]), ), examples: [ { title: "Inspect an APK", input: { path: "/tmp/Example.apk" } }, + { + title: "Inspect APK declarations with retained manifest", + input: { path: "/tmp/Example.apk", detail: "summary" }, + }, ], }, { diff --git a/src/domain/analysisView/analysisView.ts b/src/domain/analysisView/analysisView.ts index 4ab9c77da..784d5dbed 100644 --- a/src/domain/analysisView/analysisView.ts +++ b/src/domain/analysisView/analysisView.ts @@ -56,6 +56,12 @@ import { wasmSummarySchema, } from "./wasmArtifactView.js"; import type { WebNetworkApiSurfaceAccumulator } from "./WebNetworkApiSurfaceAccumulator.js"; +import { + projectAndroidPackageView, + androidPackageSummarySchema, + androidViewRequestSchema, + androidManifestViewItemSchema, +} from "./androidPackageView.js"; /** Public name of the selected-view workflow. */ export const INSPECT_ANALYSIS_VIEW_OPERATION = "inspect_analysis_view" as const; @@ -169,6 +175,7 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [ wasmViewRequestSchema, evmViewRequestSchema, managedViewRequestSchema, + androidViewRequestSchema, apiSurfaceViewSchema, ]); @@ -283,6 +290,7 @@ const viewResultBase = { "inspect_wasm_artifact", "inspect_evm_interface", "inspect_managed_artifact", + "inspect_android_package", ]), parent_digest: digestSchema, view_digest: digestSchema, @@ -340,6 +348,13 @@ export const managedArtifactSummaryViewResultSchema = z.strictObject({ summary: managedSummarySchema, }); +/** Exact package declarations with retained decoded manifest text. */ +export const androidPackageSummaryViewResultSchema = z.strictObject({ + ...summaryViewResultBase, + parent_operation: z.literal("inspect_android_package"), + summary: androidPackageSummarySchema, +}); + /** Projected facts for one selected view of retained analysis Evidence. */ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ z.strictObject({ @@ -362,6 +377,7 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ wasmSummarySchema, evmSummarySchema, managedSummarySchema, + androidPackageSummarySchema, ]), }), z.strictObject({ @@ -410,6 +426,13 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ ...managedViewContextSchema.shape, parent_operation: z.literal("inspect_managed_artifact"), }), + z.strictObject({ + kind: z.literal("android"), + view: androidViewRequestSchema, + item: androidManifestViewItemSchema, + ...viewResultBase, + parent_operation: z.literal("inspect_android_package"), + }), z.strictObject({ kind: z.literal("inventory"), view: inventoryViewSchema, @@ -480,6 +503,10 @@ export type UnsignedAnalysisView = | Omit< Extract, "view_digest" + > + | Omit< + Extract, + "view_digest" >; /** Attach the content digest that must change when projected bytes change. */ @@ -525,7 +552,7 @@ const unsupportedParent = ( new AnalysisUnsupportedTargetError( INSPECT_ANALYSIS_VIEW_OPERATION, parentArtifactPath(parent), - `Parent Evidence operation ${JSON.stringify(parent.operation)} does not support analysis views. Use completed binary layout, JavaScript application, native function, artifact inventory/inspection, WASM, EVM, managed artifact, or web network capture Evidence.`, + `Parent Evidence operation ${JSON.stringify(parent.operation)} does not support analysis views. Use completed binary layout, JavaScript application, native function, artifact inventory/inspection, WASM, EVM, managed artifact, Android package, or web network capture Evidence.`, ); const parentArtifactPath = (parent: AnalysisViewParent): string => { @@ -554,6 +581,7 @@ const retainedViewProjectors = new Map([ ["inspect_wasm_artifact", projectWasmArtifactView], ["inspect_evm_interface", projectEvmInterfaceView], ["inspect_managed_artifact", projectManagedArtifactView], + ["inspect_android_package", projectAndroidPackageView], ["inspect_artifact", projectArtifactInventoryView], ["inventory_artifact", projectArtifactInventoryView], ]); diff --git a/src/domain/analysisView/androidPackageView.ts b/src/domain/analysisView/androidPackageView.ts new file mode 100644 index 000000000..caf948c10 --- /dev/null +++ b/src/domain/analysisView/androidPackageView.ts @@ -0,0 +1,136 @@ +import { z } from "zod"; +import type { AnalysisError } from "../analysisErrorBase.js"; +import { androidResultSchemas } from "../android/androidAnalysis.js"; +import { analysisInputErrorFromIssues } from "../inputIssueProjection.js"; +import { err, ok, type Result } from "../result.js"; +import { + analysisViewInputError, + type AnalysisViewParent, + type AnalysisViewRequest, + type UnsignedAnalysisView, +} from "./analysisView.js"; +import { completeWithinViewCoverage } from "./analysisViewCoverage.js"; +import { analysisTextPage } from "./analysisViewText.js"; + +const packageSchema = androidResultSchemas.inspect_android_package; + +/** All package declarations remain inline; decoded manifest text stays retained. */ +export const androidPackageSummarySchema = packageSchema.extend({ + manifest: packageSchema.shape.manifest.omit({ text: true }).extend({ + text_code_units: z.number().int().nonnegative(), + text_included: z.literal(false), + }), +}); + +/** Exact decoded text; offsets and limits count UTF-16 code units. */ +export const androidViewRequestSchema = z.strictObject({ + kind: z.literal("android"), + facet: z.literal("manifest"), + offset: z.number().int().nonnegative().default(0), + limit: z.number().int().positive(), +}); + +export const androidManifestViewItemSchema = z.strictObject({ + text: z.string(), + unit: z.literal("utf16-code-units"), + source_status: packageSchema.shape.manifest.shape.status, + reported_total_bytes: packageSchema.shape.manifest.shape.reported_total_bytes, +}); + +/** Project an authenticated package record without reopening the APK or engine. */ +export const projectAndroidPackageView = ( + parent: AnalysisViewParent, + view: AnalysisViewRequest, +): Result => { + if (view.kind !== "summary" && view.kind !== "android") + return err( + analysisViewInputError([ + { + path: ["view"], + reason: "invalid_value", + message: + "Android package Evidence supports summary and manifest views.", + }, + ]), + ); + const parsed = packageSchema.safeParse(parent.normalizedResult); + if (!parsed.success) + return err( + analysisInputErrorFromIssues( + "inspect_analysis_view", + parsed.error.issues.map((issue) => ({ + ...issue, + path: ["source", "normalized_result", ...issue.path], + })), + { source: { normalized_result: parent.normalizedResult } }, + { cause: parsed.error }, + ), + ); + // Engine artifact_sha256 identifies the JAR, never the APK. A missing + // authenticated APK subject cannot be repaired using the engine identity. + if (parent.subject == null) + return err( + analysisViewInputError([ + { + path: ["source", "subject"], + reason: "invalid_value", + message: + "Android package views require the recorded APK subject identity.", + }, + ]), + ); + const report = parsed.data; + const shared = { + parent_evidence_id: parent.evidenceId, + parent_operation: "inspect_android_package" as const, + parent_digest: parent.evidenceId.slice(3), + artifact: { + path: parent.subject.local_path, + sha256: parent.subject.digest.sha256, + }, + limitations: [...parent.limitations], + unknowns: [ + "APK signatures were not verified; package declarations do not establish authenticity or runtime behavior.", + ...(report.manifest.status === "partial" + ? [ + "The decoded manifest is partial; exhausting retained text does not establish complete manifest coverage.", + ] + : []), + ], + }; + if (view.kind === "summary") { + const { text, ...manifest } = report.manifest; + return ok({ + kind: "summary", + view, + ...shared, + summary: { + ...report, + manifest: { + ...manifest, + text_code_units: text.length, + text_included: false, + }, + }, + coverage: completeWithinViewCoverage(1, 1), + }); + } + const page = analysisTextPage( + report.manifest.text, + view.offset, + view.limit, + "Manifest", + ); + if (!page.ok) return page; + return ok({ + kind: "android", + view, + ...shared, + item: { + ...page.value.item, + source_status: report.manifest.status, + reported_total_bytes: report.manifest.reported_total_bytes, + }, + coverage: page.value.coverage, + }); +}; diff --git a/src/domain/analysisView/javascriptApplicationView.ts b/src/domain/analysisView/javascriptApplicationView.ts index 5e5f9bb88..f4c09a68d 100644 --- a/src/domain/analysisView/javascriptApplicationView.ts +++ b/src/domain/analysisView/javascriptApplicationView.ts @@ -314,7 +314,7 @@ export const projectJavaScriptApplicationView = ( ...shared, }); } - if (view.collection !== "modules") + if (view.kind !== "page" || view.collection !== "modules") return err( jsIncompatible( "Section and symbol collections apply to inspect_binary_layout Evidence.", diff --git a/src/domain/android/androidAnalysis.ts b/src/domain/android/androidAnalysis.ts index e27be2b20..ca4d6db59 100644 --- a/src/domain/android/androidAnalysis.ts +++ b/src/domain/android/androidAnalysis.ts @@ -45,6 +45,12 @@ export const androidInputSchemas = { }), } as const; +/** Optional retained delivery is separate from the engine's APK request. */ +export const androidPackageInspectionInputSchema = + androidInputSchemas.inspect_android_package.extend({ + detail: z.enum(["complete", "summary"]).optional(), + }); + /** Supported static Android operations. */ export type AndroidOperation = keyof typeof androidInputSchemas; /** Validated Android requests carry their selected operation. */ diff --git a/src/server/createServer.ts b/src/server/createServer.ts index 12ad6418b..e445dcfab 100644 --- a/src/server/createServer.ts +++ b/src/server/createServer.ts @@ -515,6 +515,7 @@ const registerConfiguredAnalysisTools = ( toolLogger, recordEvidence, withAdmittedAnalysis, + evidenceById, ); registerJebTools( server, diff --git a/src/server/registerAndroidTools.ts b/src/server/registerAndroidTools.ts index 722bd6695..a8c29a187 100644 --- a/src/server/registerAndroidTools.ts +++ b/src/server/registerAndroidTools.ts @@ -1,6 +1,8 @@ import type { EvidenceMcpServer } from "./EvidenceMcpServer.js"; import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js"; import { AndroidAnalysisService } from "../application/android/AndroidAnalysisService.js"; +import { inspectAndroidPackage } from "../application/android/AndroidPackageInspection.js"; +import type { EvidenceLookup } from "../application/EvidenceInputResolver.js"; import { summarizeRetainedAndroidSearch } from "../application/android/AndroidSearchEvidenceSummary.js"; import { toolContract } from "../contracts/toolContracts.js"; import type { Logger } from "pino"; @@ -18,6 +20,7 @@ export const registerAndroidTools = ( logger: Logger, recordEvidence?: EvidenceWriter["recordEvidence"], withAdmittedAnalysis?: WithAdmittedAnalysis, + evidenceById?: EvidenceLookup, ): void => { const handler = createContractToolHandler( server, @@ -44,10 +47,26 @@ export const registerAndroidTools = ( withAdmittedAnalysis, ); const packageContract = toolContract("inspect_android_package"); + const packageService: ContractToolService<"inspect_android_package"> = { + execute: (_operation, input, options) => + inspectAndroidPackage( + service, + input, + { recordEvidence, evidenceById }, + options, + ), + }; + const packageHandler = createContractToolHandler( + server, + packageService, + logger, + recordEvidence, + withAdmittedAnalysis, + ); server.registerTool( packageContract.name, toolRegistrationOptions(packageContract), - handler(packageContract), + packageHandler(packageContract), ); const searchContract = toolContract("search_android_classes"); server.registerTool( diff --git a/tests/boundary/mcp/androidPackageSummaryMcp.test.ts b/tests/boundary/mcp/androidPackageSummaryMcp.test.ts new file mode 100644 index 000000000..960886b02 --- /dev/null +++ b/tests/boundary/mcp/androidPackageSummaryMcp.test.ts @@ -0,0 +1,295 @@ +import { unlink } from "node:fs/promises"; +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import { Ajv2020 } from "ajv/dist/2020.js"; +import { expect, it, onTestFinished, vi } from "vitest"; +import { AndroidAnalysisService } from "../../../src/application/android/AndroidAnalysisService.js"; +import { inspectAndroidPackage } from "../../../src/application/android/AndroidPackageInspection.js"; +import { inspectAnalysisView } from "../../../src/application/analysisView/AnalysisViewService.js"; +import { androidResultSchemas } from "../../../src/domain/android/androidAnalysis.js"; +import { + androidPackageSummaryViewResultSchema, + analysisViewResultSchema, +} from "../../../src/domain/analysisView/analysisView.js"; +import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js"; +import type { JsonValue } from "../../../src/domain/jsonValue.js"; +import { createServer } from "../../../src/server/createServer.js"; +import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; +import { createTestBinarySession } from "../../fixtures/binarySession.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { + createJadxProtocolFixture, + verifyJadxFixtureCleanup, +} from "../../fixtures/android/jadx.js"; + +it.skipIf(process.platform === "win32")( + "retains complete APK Evidence and reads exact manifest without its source or another engine launch", + async () => { + const { provider, apk, launches } = + await createJadxProtocolFixture("large-manifest"); + const harness = await createApplicationMcpHarness({ + serverOptions: { androidAnalysis: provider }, + }); + onTestFinished(harness.close); + const { client, session } = harness; + const catalog = (await client.listTools()).tools; + const definition = catalog.find( + (tool) => tool.name === "inspect_android_package", + ); + if (definition?.outputSchema === undefined) + throw new Error("Missing APK schema"); + const outputSchema: Record = definition.outputSchema; + const ajv = new Ajv2020({ strict: false, validateFormats: false }); + expect(ajv.validateSchema(definition.inputSchema)).toBe(true); + expect(ajv.validateSchema(outputSchema)).toBe(true); + expect( + ajv.validate(definition.inputSchema, { path: apk, detail: "summary" }), + ).toBe(true); + expect( + ajv.validate( + catalog.find((tool) => tool.name === "inspect_android_class")! + .inputSchema, + { + path: apk, + class_name: "fixture.Target", + detail: "summary", + }, + ), + ).toBe(false); + const reply = await client.callTool({ + name: "inspect_android_package", + arguments: { path: apk, detail: "summary" }, + }); + expect(reply.isError).not.toBe(true); + expect( + ajv.validate(outputSchema, reply.structuredContent), + JSON.stringify(ajv.errors), + ).toBe(true); + const receipt = parseEvidence(reply.structuredContent); + const view = androidPackageSummaryViewResultSchema.parse( + receipt.normalized_result, + ); + const parent = session.evidenceById(view.parent_evidence_id); + if (parent === undefined) + throw new Error("Complete package Evidence not retained"); + const report = androidResultSchemas.inspect_android_package.parse( + parent.normalized_result, + ); + expect(parent.operation).toBe("inspect_android_package"); + expect(parent.parameters).toEqual({ path: apk }); + expect(view.artifact.sha256).toBe(parent.subject?.digest.sha256); + expect(view.artifact.sha256).not.toBe(report.engine.artifact_sha256); + expect(receipt.evidence_id).not.toBe(parent.evidence_id); + expect(receipt.evidence_links).toEqual([parent.evidence_id]); + const { manifest, ...declarations } = report; + expect(view.summary).toMatchObject(declarations); + expect(view.summary.manifest).toEqual({ + status: manifest.status, + reported_total_bytes: manifest.reported_total_bytes, + text_code_units: manifest.text.length, + text_included: false, + }); + expect(receipt.raw_result).toBeNull(); + expect(parent.raw_result).not.toBeNull(); + for (const detail of [undefined, "complete"]) { + const full = parseEvidence( + ( + await client.callTool({ + name: "inspect_android_package", + arguments: { + path: apk, + ...(detail === undefined ? {} : { detail }), + }, + }) + ).structuredContent, + ); + expect(full.operation).toBe("inspect_android_package"); + expect(full.normalized_result).toEqual(parent.normalized_result); + expect(full.parameters).toEqual(parent.parameters); + expect(full.raw_result).not.toBeNull(); + } + const before = JSON.stringify(parent); + await unlink(apk); + let text = ""; + let offset = 0; + do { + const paged = await client.callTool({ + name: "inspect_analysis_view", + arguments: { + source: { + kind: "retained-evidence", + evidence_id: parent.evidence_id, + }, + view: { kind: "android", facet: "manifest", offset, limit: 700000 }, + }, + }); + expect(paged.isError).not.toBe(true); + const page = analysisViewResultSchema.parse( + parseEvidence(paged.structuredContent).normalized_result, + ); + if (page.kind !== "android") throw new Error("Unexpected manifest page"); + text += page.item.text; + expect(page.item.source_status).toBe(manifest.status); + expect(page.item.reported_total_bytes).toBe( + manifest.reported_total_bytes, + ); + offset = page.coverage.next_offset ?? manifest.text.length; + } while (offset < manifest.text.length); + expect(text).toBe(manifest.text); + const portable = inspectAnalysisView({ + source: { kind: "inline", evidence: parseEvidence(JSON.parse(before)) }, + view: { kind: "summary" }, + }); + if (!portable.ok) throw portable.error; + expect(portable.value.normalized_result).toEqual(view); + expect(JSON.stringify(session.evidenceById(parent.evidence_id))).toBe( + before, + ); + expect( + session + .exportEvidenceBundle() + .records.find((record) => record.evidence_id === parent.evidence_id), + ).toEqual(parent); + expect(launches).toHaveLength(1); + await harness.close(); + await verifyJadxFixtureCleanup(launches); + }, + 30000, +); + +it("preserves partial/unknown declarations and Unicode text while rejecting missing APK identity and malformed source", () => { + const report = { + engine: { + name: "jadx", + version: null, + artifact_sha256: "b".repeat(64), + source_revision: null, + worker_count: 1, + heap_limit_mib: null, + }, + package_name: null, + version_name: null, + version_code: null, + min_sdk: null, + target_sdk: null, + permissions: [], + manifest: { status: "partial", text: "a😀b", reported_total_bytes: null }, + class_count: 0, + resource_count: 0, + signature_verification: "not_performed", + }; + const evidence = (result: JsonValue = report, hasSubject = true) => + createEvidence( + hasSubject + ? { path: "/original.apk", sha256: "a".repeat(64), format: "apk" } + : undefined, + { id: "jadx", name: "JADX", version: "0.7.1" }, + { + operation: "inspect_android_package", + parameters: {}, + result, + limitations: ["Original limit"], + }, + ); + const parent = evidence(); + const source = { kind: "inline", evidence: parent }; + const summary = inspectAnalysisView({ source, view: { kind: "summary" } }); + if (!summary.ok) throw summary.error; + const overview = androidPackageSummaryViewResultSchema.parse( + summary.value.normalized_result, + ); + expect(overview.summary.package_name).toBeNull(); + expect(overview.summary.manifest.status).toBe("partial"); + expect(overview.limitations).toEqual(parent.limitations); + const page = inspectAnalysisView({ + source, + view: { kind: "android", facet: "manifest", offset: 0, limit: 4 }, + }); + if (!page.ok) throw page.error; + const result = analysisViewResultSchema.parse(page.value.normalized_result); + expect(result).toMatchObject({ + kind: "android", + item: { + text: "a😀b", + source_status: "partial", + reported_total_bytes: null, + }, + coverage: { exhausted: true }, + }); + expect(result.unknowns.join(" ")).toContain("partial"); + for (const offset of [2]) + expect( + inspectAnalysisView({ + source, + view: { kind: "android", facet: "manifest", offset, limit: 2 }, + }), + ).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } }); + for (const invalid of [ + evidence(report, false), + evidence({ ...report, class_count: -1 }), + ]) + expect( + inspectAnalysisView({ + source: { kind: "inline", evidence: invalid }, + view: { kind: "summary" }, + }), + ).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } }); +}); + +it("rejects unavailable retention, invalid selectors and cancellation before provider execution", async () => { + const execute = vi.fn(); + const provider = { + execute, + close: async () => {}, + inspectAvailability: vi.fn(), + }; + const service = new AndroidAnalysisService(provider); + const retention = { recordEvidence: undefined, evidenceById: undefined }; + expect( + await inspectAndroidPackage( + service, + { path: "/missing.apk", detail: "summary" }, + retention, + ), + ).toMatchObject({ + ok: false, + error: { _tag: "AnalysisCapabilityUnavailableError" }, + }); + expect( + await inspectAndroidPackage( + service, + { path: "/missing.apk", detail: "other" }, + retention, + ), + ).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } }); + expect( + await inspectAndroidPackage( + service, + { path: "/missing.apk", detail: "summary" }, + retention, + { signal: AbortSignal.abort() }, + ), + ).toMatchObject({ ok: false, error: { _tag: "AnalysisCancelledError" } }); + expect(execute).not.toHaveBeenCalled(); + const session = createTestBinarySession(() => { + throw new Error("Unexpected deep provider"); + }); + const server = createServer( + { kind: "fixed", analysis: session }, + { androidAnalysis: provider }, + ); + const client = new Client({ name: "android-fixed-summary", version: "1" }); + onTestFinished(async () => { + await client.close(); + await server.close(); + await session.close(); + }); + const [a, b] = InMemoryTransport.createLinkedPair(); + await server.connect(b); + await client.connect(a); + const reply = await client.callTool({ + name: "inspect_android_package", + arguments: { path: "/missing.apk", detail: "summary" }, + }); + expect(parseMcpToolError(reply).error.code).toBe("capability_unavailable"); + expect(execute).not.toHaveBeenCalled(); +}); From a2d8b94dd50dab8d966239dd63ab84718aa8b4b4 Mon Sep 17 00:00:00 2001 From: Ilian Iliev Date: Tue, 13 Oct 2026 05:51:14 +0300 Subject: [PATCH 002/159] feat(fiddler): inspect native SAZ captures Inspect bounded recorded raw HTTP/XML SAZ archives and page retained capture Evidence. Preserve exact stored bytes and provenance, validate ZIP/XML integrity, and require an enforceable decoder memory bound. Verified 356 real recorded sessions against independent byte and metadata oracles, public CLI/MCP parity, retained pages, and related interface checks. Current Fiddler Everywhere GUI and non-Linux producer workflows remain unverified. Co-authored-by: zahhak <1792039+zahhak@users.noreply.github.com> Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com> --- .../skills/reverse-engineer-anything/SKILL.md | 8 +- .../references/runtime-observation.md | 15 + README.md | 2 +- docs/.vitepress/config.ts | 4 + docs/browser-observation.md | 4 + docs/client-configuration.md | 6 + docs/packet-captures.md | 2 +- docs/web-network-captures.md | 113 +++- scripts/lib/catalog-builders.mjs | 5 + src/application/WebNetworkCaptureService.ts | 16 +- .../analysisView/AnalysisViewService.ts | 8 +- src/browser/history/CaptureFailures.test.ts | 8 +- src/browser/history/CaptureFailures.ts | 24 +- src/browser/history/CaptureFormatError.ts | 2 +- src/browser/history/CaptureRelease.ts | 19 +- src/browser/history/HarCaptureAdapter.ts | 33 -- .../history/HistoricalCaptureDecoder.ts | 23 +- .../history/HistoricalCaptureFormatAdapter.ts | 1 + .../history/NodeCaptureAdapter.test.ts | 15 + src/browser/history/NodeCaptureAdapter.ts | 58 ++ ...aptureProcess.ts => NodeCaptureProcess.ts} | 35 +- src/browser/history/SazCapture.ts | 174 ++++++ src/browser/history/SazHttp.ts | 72 +++ src/browser/history/SazMetadata.ts | 83 +++ src/browser/history/SazProjection.ts | 131 +++++ src/cli/webNetworkCaptureCommands.ts | 7 +- src/composition/webNetworkCaptures.ts | 8 +- src/contracts/analysisViewToolContracts.ts | 2 +- .../webNetworkCaptureToolContracts.ts | 17 +- src/domain/analysisView/analysisView.ts | 38 +- .../analysisView/javascriptApplicationView.ts | 26 +- .../webNetworkApiSurfaceView.test.ts | 12 +- .../webNetworkCaptureView.test.ts | 176 ++++++ .../analysisView/webNetworkCaptureView.ts | 212 ++++++++ src/domain/webNetworkCapture.ts | 60 ++- src/domain/webNetworkCaptureCoordinates.ts | 2 + src/generatedPackageMetadata.ts | 2 +- src/server/createServer.ts | 1 + src/server/registerWebNetworkCaptureTool.ts | 36 +- .../tools/historicalCaptureResources.test.ts | 4 +- tests/boundary/browser/sazWebCapture.test.ts | 499 ++++++++++++++++++ website/locales/ar/messages.po | 29 +- website/locales/de/messages.po | 30 +- website/locales/es/messages.po | 30 +- website/locales/fa/messages.po | 29 +- website/locales/fr/messages.po | 30 +- website/locales/id/messages.po | 30 +- website/locales/it/messages.po | 30 +- website/locales/ja/messages.po | 28 +- website/locales/ko/messages.po | 28 +- website/locales/messages.pot | 24 +- website/locales/pl/messages.po | 29 +- website/locales/pt-BR/messages.po | 30 +- website/locales/ru/messages.po | 30 +- website/locales/th/messages.po | 30 +- website/locales/tr/messages.po | 30 +- website/locales/uk/messages.po | 30 +- website/locales/vi/messages.po | 29 +- website/locales/zh-CN/messages.po | 27 +- website/locales/zh-TW/messages.po | 27 +- website/public/guides/browser/index.html | 9 + 61 files changed, 2237 insertions(+), 285 deletions(-) delete mode 100644 src/browser/history/HarCaptureAdapter.ts create mode 100644 src/browser/history/NodeCaptureAdapter.test.ts create mode 100644 src/browser/history/NodeCaptureAdapter.ts rename src/browser/history/{HarCaptureProcess.ts => NodeCaptureProcess.ts} (59%) create mode 100644 src/browser/history/SazCapture.ts create mode 100644 src/browser/history/SazHttp.ts create mode 100644 src/browser/history/SazMetadata.ts create mode 100644 src/browser/history/SazProjection.ts create mode 100644 src/domain/analysisView/webNetworkCaptureView.test.ts create mode 100644 src/domain/analysisView/webNetworkCaptureView.ts create mode 100644 tests/boundary/browser/sazWebCapture.test.ts diff --git a/.agents/skills/reverse-engineer-anything/SKILL.md b/.agents/skills/reverse-engineer-anything/SKILL.md index cd50cff64..35b99d8dc 100644 --- a/.agents/skills/reverse-engineer-anything/SKILL.md +++ b/.agents/skills/reverse-engineer-anything/SKILL.md @@ -2,7 +2,7 @@ name: reverse-engineer-anything description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. metadata: - version: "40" + version: "41" --- # REA @@ -61,7 +61,11 @@ explicit path or endpoint and do not need it. consult the NativeAOT workflow in `references/native-and-artifacts.md` for optional metadata recovery and its supported host/layout boundary. - User-owned browser page already open: `list_browser_targets`. -- Retained HAR/native mitmproxy capture: `inspect_web_network_capture`. Select +- Fiddler Everywhere traffic or a native `.saz` capture: read the + [Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging). + Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`; + connect Everywhere's separate MCP server for live traffic. +- Retained HAR/SAZ/native mitmproxy capture: `inspect_web_network_capture`. Select original record ordinals when useful; inspect producer fields and byte/number sidecars without fetching recorded URLs or claiming live attribution. See [historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream diff --git a/.agents/skills/reverse-engineer-anything/references/runtime-observation.md b/.agents/skills/reverse-engineer-anything/references/runtime-observation.md index 3e16a3e19..2d660d675 100644 --- a/.agents/skills/reverse-engineer-anything/references/runtime-observation.md +++ b/.agents/skills/reverse-engineer-anything/references/runtime-observation.md @@ -26,3 +26,18 @@ passive browser/Electron Evidence exist. Prefer captured-source digest identity; report path/digest disagreement as a mismatch and preserve ambiguity. Explicit path mappings are inference inputs and never broaden filesystem or CDP authority. A bundle observed at runtime does not prove every contained module executed. + +## Fiddler Everywhere traffic debugging + +For large saved captures, request `detail: "summary"`, then page retained Evidence +with `inspect_analysis_view` and `view: {kind: "capture", detail: "preview", offset: 0, limit: 20}`. +Select complete record pages only for needed headers, metadata or payloads; views +read the historical parent without repeating capture decoding. + +Inspect native `.saz` snapshots with `inspect_web_network_capture` and +`format: "saz"`; encrypted archives are unsupported. For live +traffic analysis, connect Everywhere's separate MCP server and use its +advertised session tools with an explicit `sessionsSource` (`LiveTraffic` or +`AgentCalls`). Preserve existing sessions and filters when inspecting a +recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) +for the workflow and Telerik's public setup and traffic-debugging skills. diff --git a/README.md b/README.md index 1e14f3a2d..49173ab4b 100644 --- a/README.md +++ b/README.md @@ -161,7 +161,7 @@ Additional tools and host support depend on the target: | Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) | | JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; [application analysis](https://rea.tools/guides/javascript/) | | Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](https://rea.tools/guides/browser/) | -| Saved network captures | Requests, responses, exposed payloads and source locations | HAR; mitmdump on Linux for native mitmproxy captures; [capture guide](docs/web-network-captures.md) | +| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) | | .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) | | Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) | | Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) | diff --git a/docs/.vitepress/config.ts b/docs/.vitepress/config.ts index 5a357c80f..001c07fbe 100644 --- a/docs/.vitepress/config.ts +++ b/docs/.vitepress/config.ts @@ -37,6 +37,10 @@ export default defineConfig({ link: "/javascript-application-workflows", }, { text: "Browser observation", link: "/browser-observation" }, + { + text: "Web network captures", + link: "/web-network-captures", + }, { text: "Electron observation", link: "/electron-observation" }, { text: "Process capture", link: "/process-capture" }, { text: ".NET assemblies", link: "/managed-code-analysis" }, diff --git a/docs/browser-observation.md b/docs/browser-observation.md index 2e3d8e37c..2f73f4261 100644 --- a/docs/browser-observation.md +++ b/docs/browser-observation.md @@ -8,6 +8,10 @@ exported script's native imports under an explicit URL/import-map context. REA can attach to a user-owned Chrome-family browser through the Chrome DevTools Protocol (CDP) and produce bounded Evidence about an existing page. This is a passive reverse-engineering capability, not a general browser automation or remote-control surface. +For request/response bodies and traffic debugging, see +[Fiddler Everywhere captures and MCP](web-network-captures.md#reading-fiddler-captures-saz). +Inspect native SAZ or HAR 1.2 offline with REA, or connect Everywhere's separate MCP server. + ## Shipped surfaces - `list_browser_targets` / `rea list-browser-targets` discovers all eligible page targets at the endpoint by default, with an optional exact-origin filter. diff --git a/docs/client-configuration.md b/docs/client-configuration.md index c0e629d66..a6e254488 100644 --- a/docs/client-configuration.md +++ b/docs/client-configuration.md @@ -6,6 +6,12 @@ for the reviewed setup transaction and [client IDs](installation.md#supported-ag The versioned verification notes below describe their stated runs; live service acceptance is verified only where explicitly recorded. +## Fiddler Everywhere alongside REA + +Connect Fiddler Everywhere's MCP server separately for HTTP traffic debugging, +preserving your REA client entry. See the +[setup instructions and public skills](web-network-captures.md#fiddler-everywhere-mcp-and-public-skills). + ## GitHub Copilot CLI For GitHub Copilot CLI 1.0.95, `gpt-4.1` model metadata can block ordinary diff --git a/docs/packet-captures.md b/docs/packet-captures.md index d56a80037..e8ce3ae92 100644 --- a/docs/packet-captures.md +++ b/docs/packet-captures.md @@ -4,7 +4,7 @@ in an existing capture. Network, USB or Bluetooth recordings can use the same field/filter interface when the installed TShark supports their encapsulation and dissectors. This tool does not acquire traffic, open an interface or control -a device. HAR/mitmproxy web-capture workflows remain separate. +a device. HAR/SAZ/mitmproxy web-capture workflows remain separate. Supply an existing TShark 4.x installation **on Linux**, with util-linux `prlimit` available at `/usr/bin/prlimit`. Override its absolute path with diff --git a/docs/web-network-captures.md b/docs/web-network-captures.md index d49f1dcc3..e65887a71 100644 --- a/docs/web-network-captures.md +++ b/docs/web-network-captures.md @@ -25,6 +25,36 @@ and original zero-based producer ordinals; duplicate or out-of-range selections are input errors. Every record is validated before selection, so selecting one entry does not hide a malformed entry elsewhere. +For agent discovery over MCP, request `detail: "summary"`. REA retains the +complete selected Evidence in the session and returns identity and counts. +Use `normalized_result.parent_evidence_id` with `inspect_analysis_view`: + +```json +{ + "source": { + "kind": "retained-evidence", + "evidence_id": "" + }, + "view": { "kind": "capture", "detail": "preview", "offset": 0, "limit": 20 } +} +``` + +Preview pages select recorded HTTP headline fields, event names/times when +present, original locations and byte descriptors; they omit headers, XML and +payload content. Choose `detail: "complete"` in that view for exact record +pages, including all sidecars. Offsets follow the parent's selected record +order; producer ordinals remain unchanged. `total_records` counts the original +capture while `retained_records` and page coverage count records in the parent. +An exhausted page does not establish absence from unselected records. + +Views do not reopen the file or rerun its decoder. They remain historical +Evidence if the source changes or disappears; inspect the capture again to +observe new bytes. The inspection tool's default `detail: "complete"` preserves +existing inline delivery. For CLI use, save complete inspection JSON and pass +it as inline Evidence to `inspect-analysis-view`; the same summary and capture +page contracts apply. Large complete records can still exceed MCP's response +budget; export the retained Evidence when a single record cannot fit. + ## Formats and upstream profiles HAR uses unchanged `har-schema@2.0.0` draft-06 schemas, `ajv@8.20.0`, @@ -57,11 +87,80 @@ tested upstream source is tag `v12.2.3`, commit an explicit unsupported-profile diagnostic. Non-Linux native decoding is not supported by this adapter. +## Reading Fiddler captures (`.saz`) + +Fiddler Everywhere saves traffic in **Session Archive Zip (`.saz`)** files. +Inspect native SAZ directly, without installing Fiddler or converting to HAR: + +```sh +rea inspect-web-network-capture /absolute/session.saz saz --json +rea inspect-web-network-capture /absolute/session.saz saz --record 0 --json +``` + +MCP uses the same `capture_path`, `format: "saz"`, and optional +`record_ordinals` fields as the examples above. REA pairs `raw/_c.txt` +requests, `_s.txt` responses, and `_m.xml` metadata by filename stem. Archive +member names identify the source; zero-based ordinals follow each session's +first member in ZIP directory order, not its numeric ID or capture time. + +HTTP fields and ordered duplicate headers are retained, with stored body bytes +in binary sidecars. Bodies are not dechunked or decompressed. Metadata is an +XML node tree; optional `_w.txt` and `_g.txt` files retain opaque WebSocket and +gRPC message bytes. Missing members remain null, distinct from empty bodies. +See Telerik's [SAZ format reference](https://www.telerik.com/fiddler/fiddler-everywhere/documentation/knowledge-base/fiddler-archives). + +The existing ZIP reader verifies read members; expanded archive contents are +limited to 32 MiB in addition to the input/output budgets below. Encrypted +archives and unrecognized raw session members are unsupported. SAZ requires an +enforced decoder memory bound: Node's fixed heap, or the Windows Job Object +commit limit. Bun on hosts without that Job Object limit returns capability +unavailable before starting the SAZ decoder. Non-session +members are inventoried without decoding their content. Original files remain +unchanged. Linux verification includes independently checked public recorded +Mattermost captures from [FOREST](https://github.com/dfrc-korea/FOREST/tree/405c9154658a128ea203a6c10040cec55111ed03/api-results/dataset) +and the Fiddler author's [encoding corpus](https://github.com/ericlaw1979/Clearinet/blob/b6faf78cde1e72042980364f180ceb34b77d1434/Test/data/Encodings.saz), +plus synthetic malformed/limit fixtures. Their exporting product versions are +not authenticated. An interactive export from a current Fiddler Everywhere +installation and its live MCP server remain unverified. Support here is the +bounded documented raw HTTP/XML SAZ layout, not every product/version profile. + +For visual inspection, use Everywhere's **Import Snapshot**, then **Inspectors** +and **Overview**. HAR remains an alternative: select sessions, choose +**Export > HTTPArchive**, and confirm `log.version` is `"1.2"`. Preserve the +original SAZ when exporting. See the [import/export instructions](https://www.telerik.com/fiddler/fiddler-everywhere/documentation/collaboration/exporting-and-importing). + +## Fiddler Everywhere MCP and public skills + +For agent-driven traffic debugging, connect Everywhere's MCP server separately +from REA. Run a current Everywhere release with MCP access (Trial, Pro, or +higher), internet access, and a compatible MCP client. Use +**Settings > MCP Server > Coding agent**, or **Manual setup** with the generated +API key. The default endpoint is `http://localhost:8868/mcp`. See Telerik's +[MCP setup guide](https://www.telerik.com/fiddler/fiddler-everywhere/documentation/agent-tools/fiddler-mcp-server) +and [agent-skills prerequisites](https://www.telerik.com/fiddler/fiddler-everywhere/documentation/agent-tools/agent-skills). + +Telerik's [public skills](https://github.com/telerik/fiddler-agent-tools) cover +installation (`fiddler-download-setup`), connection setup, and traffic analysis: + +```sh +npx skills add telerik/fiddler-agent-tools --skill fiddler-mcp-setup +npx skills add telerik/fiddler-agent-tools --skill fiddler-traffic-debugging +``` + +After setup, reload the agent client and use the connected server's advertised +tool names and schemas. Check status, list sessions, then inspect relevant +session IDs. Set `sessionsSource` to `LiveTraffic` for HTTP(S) traffic or +`AgentCalls` for enriched LLM calls; active filters affect the session list. +For an existing run, skip the skill's pre-capture steps that replace process +filters and clear sessions. The documented MCP session sources do not provide +a saved-snapshot selector; use the SAZ workflow above for saved files. REA does +not register or wrap the Fiddler MCP server. + ## Reading the evidence - `artifact` identifies the caller-selected file and stable snapshot digest. -- `records` preserve producer order/ordinals and original JSON pointers or byte - ranges. No live browser transaction or scenario IDs are invented. +- `records` preserve producer order/ordinals and original JSON pointers, byte + ranges, or SAZ member names. No live browser transaction or scenario IDs are invented. - `reported` retains the producer's fields. HAR container metadata is returned separately, with `log.entries` replaced by `null` and `records_pointer` identifying the record collection. This is an inspection representation, @@ -156,7 +255,7 @@ response fields, including HAR's `status: 0` and an empty `mimeType`, remain unknown, not invented statuses or MIME types. The view makes no handler-identity, schema or causality claim. It performs no -network activity. Native mitmproxy captures return a typed unsupported error. +network activity. Native mitmproxy and SAZ captures return a typed unsupported error. Parents inspected with `sensitive_values`, or whose Evidence does not record `sensitive_value_count`, return a typed unsupported error whose reason is the canonical token `sensitive_values_unretained`. The view would write path @@ -192,14 +291,14 @@ Complete evidence has a 32 MiB input, 96 MiB decoder reply and 64-level nesting budget. Exceeding a budget returns no partial success. Oversized selected captures return an input error identifying `capture_path` and the byte budget; owned reply/output failures remain output or provider errors. -HAR runs in an owned -192 MiB Node old-generation heap with one V8 worker. Native decoding applies +HAR and SAZ run in an owned +192 MiB Node old-generation heap with one V8 worker. Mitmproxy decoding applies Linux limits of 768 MiB address space, 30 seconds CPU and 96 MiB file output; both commands have a 30-second wall deadline and independently supervised process cleanup. Owned decoder diagnostics retain at most 1 MiB across stdout and stderr; the collector drops overflow as it arrives, reports the limit failure and verifies cleanup. Heap limits alone do not establish aggregate RSS limits. -When REA runs on Bun, the HAR decoder runs on Bun, which ignores these V8 heap +When REA runs on Bun, the HAR/SAZ decoder runs on Bun, which ignores these V8 heap flags; its limitations and resource failures then omit the heap sizes. On Windows, the decoder's owned Job Object instead limits its committed memory to 704 MiB, the 192 MiB budget plus a 512 MiB reserve for Bun itself. The @@ -207,7 +306,7 @@ limitations state that limit, and reaching it returns `resource_constraint` with `process_memory_limit_bytes`, whatever exit status Bun reports. Observed decoder memory exhaustion returns `resource_constraint`, with the -selected capture, applicable limits and retained process diagnostics. HAR's +selected capture, applicable limits and retained process diagnostics. HAR/SAZ's old-generation heap is fixed; inherited `NODE_OPTIONS` cannot raise it. Record selection happens after complete decoding and does not reduce that workload. Use a smaller capture exported by its producer or another decoder diff --git a/scripts/lib/catalog-builders.mjs b/scripts/lib/catalog-builders.mjs index 1658c08fa..f97f5ed5c 100644 --- a/scripts/lib/catalog-builders.mjs +++ b/scripts/lib/catalog-builders.mjs @@ -272,6 +272,11 @@ export const providerCatalog = (sources) => { contracts: sources.webNetworkCaptureContracts.WEB_NETWORK_CAPTURE_TOOL_CONTRACTS, }, + { + identity: sources.webNetworkCaptureProvider.SAZ_CAPTURE_PROVIDER_IDENTITY, + contracts: + sources.webNetworkCaptureContracts.WEB_NETWORK_CAPTURE_TOOL_CONTRACTS, + }, { identity: sources.webNetworkCaptureProvider.MITMPROXY_CAPTURE_PROVIDER_IDENTITY, diff --git a/src/application/WebNetworkCaptureService.ts b/src/application/WebNetworkCaptureService.ts index 326016a26..a571fa8c1 100644 --- a/src/application/WebNetworkCaptureService.ts +++ b/src/application/WebNetworkCaptureService.ts @@ -16,6 +16,7 @@ import { jsonObjectSchema } from "../domain/jsonValue.js"; import { err, ok, type Result } from "../domain/result.js"; import { inspectWebNetworkCaptureInputSchema, + captureSensitiveValues, webNetworkCaptureSchema, } from "../domain/webNetworkCapture.js"; @@ -32,21 +33,8 @@ export class WebNetworkCaptureService { ): Promise> { if (options?.signal?.aborted) return err(new AnalysisCancelledError(OPERATION)); - const declarations = - inspectWebNetworkCaptureInputSchema.shape.sensitive_values.safeParse( - typeof rawInput === "object" && - rawInput !== null && - "sensitive_values" in rawInput - ? rawInput.sensitive_values - : undefined, - ); const failure = (error: AnalysisError): Result => - err( - redactExplicitFailure( - error, - declarations.success ? declarations.data : [], - ), - ); + err(redactExplicitFailure(error, captureSensitiveValues(rawInput))); const parsed = inspectWebNetworkCaptureInputSchema.safeParse(rawInput); if (!parsed.success) return failure( diff --git a/src/application/analysisView/AnalysisViewService.ts b/src/application/analysisView/AnalysisViewService.ts index a5b603a86..baaa6b7ca 100644 --- a/src/application/analysisView/AnalysisViewService.ts +++ b/src/application/analysisView/AnalysisViewService.ts @@ -47,14 +47,8 @@ export const inspectAnalysisViewValidated = ( input: InspectAnalysisViewInput, lookup?: EvidenceLookup, ): Result => { - const inline = input.source.kind === "inline" ? input.source.evidence : null; - const inlineCapture = - typeof inline === "object" && - inline !== null && - Object.getOwnPropertyDescriptor(inline, "operation")?.value === - "inspect_web_network_capture"; const accumulator = - input.view.kind === "api-surface" || inlineCapture + input.view.kind === "api-surface" ? new WebNetworkApiSurfaceAccumulator() : undefined; if (input.source.kind === "inline" && accumulator !== undefined) { diff --git a/src/browser/history/CaptureFailures.test.ts b/src/browser/history/CaptureFailures.test.ts index ff7be71a6..047722ac5 100644 --- a/src/browser/history/CaptureFailures.test.ts +++ b/src/browser/history/CaptureFailures.test.ts @@ -4,7 +4,7 @@ import { projectAnalysisError } from "../../domain/analysisErrorProjection.js"; import { analysisErrorProjectionSchema } from "../../contracts/errorSchemas.js"; import { inspectWebNetworkCaptureInputSchema } from "../../domain/webNetworkCapture.js"; import { historicalCaptureFailure } from "./CaptureFailures.js"; -import { HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES } from "./CaptureRelease.js"; +import { NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES } from "./CaptureRelease.js"; import { redactExplicitFailure } from "../../domain/explicitSensitiveFailure.js"; import { OwnedCommandFailure } from "../../process/OwnedCommand.js"; @@ -118,7 +118,7 @@ it.each([ }, }); // The enforced limit is reported only where Bun runs on Windows. - if (HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES === null) + if (NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES === null) expect(JSON.stringify(projected)).not.toContain( "process_memory_limit_bytes", ); @@ -126,12 +126,12 @@ it.each([ expect(projected).toMatchObject({ details: { reported_limits: { - process_memory_limit_bytes: HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES, + process_memory_limit_bytes: NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES, }, }, remediation: { action: expect.stringContaining( - `${String(HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES)} bytes`, + `${String(NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES)} bytes`, ), }, }); diff --git a/src/browser/history/CaptureFailures.ts b/src/browser/history/CaptureFailures.ts index 228541545..51eceb0b5 100644 --- a/src/browser/history/CaptureFailures.ts +++ b/src/browser/history/CaptureFailures.ts @@ -18,9 +18,9 @@ import { } from "../../domain/webNetworkCapture.js"; import { OwnedCommandFailure } from "../../process/OwnedCommand.js"; import { - HAR_CAPTURE_HEAP_ENFORCED, - HAR_CAPTURE_HEAP_LIMITS, - HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES, + NODE_CAPTURE_HEAP_ENFORCED, + NODE_CAPTURE_HEAP_LIMITS, + NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES, } from "./CaptureRelease.js"; const OPERATION = "inspect_web_network_capture"; @@ -78,7 +78,7 @@ export const historicalCaptureFailure = ( ); if ( phase === "decoder" && - input.format === "har" && + input.format !== "mitmproxy" && cause.reason === "process" && (cause.snapshot?.signal === "SIGABRT" || (cause.snapshot?.signal === null && cause.snapshot.exitCode === 134)) && @@ -195,15 +195,17 @@ export const captureMemoryFailure = ( format: input.format, maximum_input_bytes: WEB_NETWORK_CAPTURE_LIMITS.inputBytes, maximum_reply_bytes: WEB_NETWORK_CAPTURE_LIMITS.outputBytes, - ...(input.format === "har" && HAR_CAPTURE_HEAP_ENFORCED + ...(input.format !== "mitmproxy" && NODE_CAPTURE_HEAP_ENFORCED ? { - old_generation_heap_mib: HAR_CAPTURE_HEAP_LIMITS.oldGenerationMiB, - semi_space_heap_mib: HAR_CAPTURE_HEAP_LIMITS.semiSpaceMiB, + old_generation_heap_mib: NODE_CAPTURE_HEAP_LIMITS.oldGenerationMiB, + semi_space_heap_mib: NODE_CAPTURE_HEAP_LIMITS.semiSpaceMiB, } : {}), - ...(input.format === "har" && - HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES !== null - ? { process_memory_limit_bytes: HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES } + ...(input.format !== "mitmproxy" && + NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES !== null + ? { + process_memory_limit_bytes: NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES, + } : {}), ...(cause instanceof OwnedCommandFailure ? { @@ -214,7 +216,7 @@ export const captureMemoryFailure = ( }, { cause, - remediationAction: `Use a smaller capture exported by its producer, or another decoder with sufficient capacity for the original capture. Preserve the original for provenance; a subset does not establish complete-capture coverage.${input.format === "har" && HAR_CAPTURE_HEAP_ENFORCED ? ` REA's HAR old-generation heap is fixed at ${HAR_CAPTURE_HEAP_LIMITS.oldGenerationMiB} MiB; inherited NODE_OPTIONS cannot raise it.` : ""}${input.format === "har" && HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES !== null ? ` REA's HAR decoder process commit limit is fixed at ${String(HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES)} bytes.` : ""}`, + remediationAction: `Use a smaller capture exported by its producer, or another decoder with sufficient capacity for the original capture. Preserve the original for provenance; a subset does not establish complete-capture coverage.${input.format !== "mitmproxy" && NODE_CAPTURE_HEAP_ENFORCED ? ` REA's Node capture old-generation heap is fixed at ${NODE_CAPTURE_HEAP_LIMITS.oldGenerationMiB} MiB; inherited NODE_OPTIONS cannot raise it.` : ""}${input.format !== "mitmproxy" && NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES !== null ? ` REA's Node capture decoder process commit limit is fixed at ${String(NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES)} bytes.` : ""}`, ...(cause instanceof OwnedCommandFailure && cause.snapshot !== null ? { capturedOutput: { diff --git a/src/browser/history/CaptureFormatError.ts b/src/browser/history/CaptureFormatError.ts index fd7a9272d..a31b3057d 100644 --- a/src/browser/history/CaptureFormatError.ts +++ b/src/browser/history/CaptureFormatError.ts @@ -1,7 +1,7 @@ /** Expected malformed, unsupported or oversized producer data at a historical decoder boundary. */ export class CaptureFormatError extends Error { constructor( - readonly reason: "format" | "unsupported" | "input-limit", + readonly reason: "format" | "unsupported" | "input-limit" | "limit", message: string, readonly pointer = "", ) { diff --git a/src/browser/history/CaptureRelease.ts b/src/browser/history/CaptureRelease.ts index d8af643aa..512c2fa6e 100644 --- a/src/browser/history/CaptureRelease.ts +++ b/src/browser/history/CaptureRelease.ts @@ -8,19 +8,19 @@ export const HAR_CAPTURE_PROVIDER_IDENTITY = { version: "har-schema@2.0.0;jsonc-parser@3.3.1;lossless-json@4.3.1", } as const; -/** Fixed V8 heap settings used by the owned HAR decoder process. */ -export const HAR_CAPTURE_HEAP_LIMITS = { +/** Fixed V8 heap settings used by the owned Node capture decoder process. */ +export const NODE_CAPTURE_HEAP_LIMITS = { oldGenerationMiB: 192, semiSpaceMiB: 8, } as const; -/** The HAR decoder runs on this executable; Bun accepts but ignores V8 heap flags. */ -export const HAR_CAPTURE_HEAP_ENFORCED = +/** The Node capture decoder runs on this executable; Bun accepts but ignores V8 heap flags. */ +export const NODE_CAPTURE_HEAP_ENFORCED = javaScriptRuntime(process.versions).id === "node"; /** Windows Job Object commit limit replacing the ignored heap flags under Bun; otherwise null. */ -export const HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES = - javaScriptWorkerMemoryLimitBytes(HAR_CAPTURE_HEAP_LIMITS.oldGenerationMiB); +export const NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES = + javaScriptWorkerMemoryLimitBytes(NODE_CAPTURE_HEAP_LIMITS.oldGenerationMiB); /** Native executable profile verified with unchanged mitmproxy raw-state decoding. */ export const MITMPROXY_CAPTURE_PROVIDER_IDENTITY = { @@ -28,3 +28,10 @@ export const MITMPROXY_CAPTURE_PROVIDER_IDENTITY = { name: "REA offline native mitmproxy adapter", version: "12.2.3", } as const; + +/** Native SAZ member decoding with the existing ZIP and XML package profiles. */ +export const SAZ_CAPTURE_PROVIDER_IDENTITY = { + id: "fiddler-saz", + name: "REA Fiddler SAZ capture adapter", + version: "1;zip.js@2.15.0;xmldom@0.9.12", +} as const; diff --git a/src/browser/history/HarCaptureAdapter.ts b/src/browser/history/HarCaptureAdapter.ts deleted file mode 100644 index 9a5204257..000000000 --- a/src/browser/history/HarCaptureAdapter.ts +++ /dev/null @@ -1,33 +0,0 @@ -import { fileURLToPath } from "node:url"; -import type { HistoricalCaptureFormatAdapter } from "./HistoricalCaptureFormatAdapter.js"; -import { - HAR_CAPTURE_PROVIDER_IDENTITY, - HAR_CAPTURE_HEAP_LIMITS, - HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES, -} from "./CaptureRelease.js"; - -/** HAR's parser/schema packages run in one bounded Node heap, without network or target execution. */ -export class HarCaptureAdapter implements HistoricalCaptureFormatAdapter { - readonly format = "har"; - readonly identity = HAR_CAPTURE_PROVIDER_IDENTITY; - async command(requestPath: string): Promise<{ - command: string; - arguments: readonly string[]; - memoryLimitBytes?: number; - }> { - return { - command: process.execPath, - arguments: [ - `--max-old-space-size=${HAR_CAPTURE_HEAP_LIMITS.oldGenerationMiB}`, - `--max-semi-space-size=${HAR_CAPTURE_HEAP_LIMITS.semiSpaceMiB}`, - "--v8-pool-size=1", - "--single-threaded", - fileURLToPath(new URL("./HarCaptureProcess.js", import.meta.url)), - requestPath, - ], - ...(HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES === null - ? {} - : { memoryLimitBytes: HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES }), - }; - } -} diff --git a/src/browser/history/HistoricalCaptureDecoder.ts b/src/browser/history/HistoricalCaptureDecoder.ts index 155ba229b..f17f45a21 100644 --- a/src/browser/history/HistoricalCaptureDecoder.ts +++ b/src/browser/history/HistoricalCaptureDecoder.ts @@ -4,9 +4,9 @@ import { captureMemoryFailure, } from "./CaptureFailures.js"; import { - HAR_CAPTURE_HEAP_ENFORCED, - HAR_CAPTURE_HEAP_LIMITS, - HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES, + NODE_CAPTURE_HEAP_ENFORCED, + NODE_CAPTURE_HEAP_LIMITS, + NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES, } from "./CaptureRelease.js"; import { projectAnalysisError } from "../../domain/analysisErrorProjection.js"; import type { HistoricalCaptureFormatAdapter } from "./HistoricalCaptureFormatAdapter.js"; @@ -208,10 +208,11 @@ export class HistoricalCaptureDecoder { format: input.format, runtime_attribution: "unknown", limitations: [ + ...(adapter.limitations ?? []), "This is retained producer evidence, without live browser transaction IDs, scenario provenance, execution attribution or deployment authenticity. Recorded URLs are never fetched.", "Every record is decoded before selection; no partial success is returned on malformed input or a resource limit.", "Known authentication headers/cookies and explicitly marked literal values are excluded. Unknown extension names do not establish sensitivity.", - `A 32 MiB input, 96 MiB reply and 64-level nesting budget bound complete evidence. ${harMemoryLimitation()}; native decoding uses a 768 MiB Linux address-space limit. Each owned command has a 30-second deadline with independent cleanup.`, + `A 32 MiB input, 96 MiB reply and 64-level nesting budget bound complete evidence. ${nodeMemoryLimitation()}; mitmproxy decoding uses a 768 MiB Linux address-space limit. Each owned command has a 30-second deadline with independent cleanup.`, ], }); } catch (cause: unknown) { @@ -251,13 +252,13 @@ export class HistoricalCaptureDecoder { } } -/** The HAR decoder runs on this executable; only Node.js enforces its V8 heap flags. */ -const harMemoryLimitation = (): string => { - if (HAR_CAPTURE_HEAP_ENFORCED) - return `HAR uses a ${String(HAR_CAPTURE_HEAP_LIMITS.oldGenerationMiB)} MiB Node old-generation heap`; - return HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES === null - ? "HAR runs on Bun, which does not enforce REA's V8 heap flags" - : `HAR runs on Bun, which does not enforce REA's V8 heap flags, under a ${String(HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES)}-byte Windows Job Object commit limit`; +/** The Node capture decoder runs on this executable; only Node.js enforces its V8 heap flags. */ +const nodeMemoryLimitation = (): string => { + if (NODE_CAPTURE_HEAP_ENFORCED) + return `HAR/SAZ use a ${String(NODE_CAPTURE_HEAP_LIMITS.oldGenerationMiB)} MiB Node old-generation heap`; + return NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES === null + ? "HAR/SAZ run on Bun, which does not enforce REA's V8 heap flags" + : `HAR/SAZ run on Bun, which does not enforce REA's V8 heap flags, under a ${String(NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES)}-byte Windows Job Object commit limit`; }; const replyFailureReason = (cause: unknown): string => { diff --git a/src/browser/history/HistoricalCaptureFormatAdapter.ts b/src/browser/history/HistoricalCaptureFormatAdapter.ts index 811ea56e2..9660fd579 100644 --- a/src/browser/history/HistoricalCaptureFormatAdapter.ts +++ b/src/browser/history/HistoricalCaptureFormatAdapter.ts @@ -3,6 +3,7 @@ import type { InspectWebNetworkCaptureInput } from "../../domain/webNetworkCaptu /** Format-specific process profile keeps mature upstream code behind a replaceable adapter seam. */ export interface HistoricalCaptureFormatAdapter { readonly format: InspectWebNetworkCaptureInput["format"]; + readonly limitations?: readonly string[]; readonly identity: { readonly id: string; readonly name: string; diff --git a/src/browser/history/NodeCaptureAdapter.test.ts b/src/browser/history/NodeCaptureAdapter.test.ts new file mode 100644 index 000000000..c341949db --- /dev/null +++ b/src/browser/history/NodeCaptureAdapter.test.ts @@ -0,0 +1,15 @@ +import { expect, it, vi } from "vitest"; +import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCore.js"; +import { NodeCaptureAdapter } from "./NodeCaptureAdapter.js"; + +vi.mock("./CaptureRelease.js", async (original) => ({ + ...(await original()), + NODE_CAPTURE_HEAP_ENFORCED: false, + NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES: null, +})); + +it("refuses a SAZ worker when the runtime has no enforceable memory bound", async () => { + await expect( + new NodeCaptureAdapter("saz").command("/owned/request.json"), + ).rejects.toBeInstanceOf(AnalysisCapabilityUnavailableError); +}); diff --git a/src/browser/history/NodeCaptureAdapter.ts b/src/browser/history/NodeCaptureAdapter.ts new file mode 100644 index 000000000..f1e98a07b --- /dev/null +++ b/src/browser/history/NodeCaptureAdapter.ts @@ -0,0 +1,58 @@ +import { fileURLToPath } from "node:url"; +import type { HistoricalCaptureFormatAdapter } from "./HistoricalCaptureFormatAdapter.js"; +import { + NODE_CAPTURE_HEAP_ENFORCED, + HAR_CAPTURE_PROVIDER_IDENTITY, + SAZ_CAPTURE_PROVIDER_IDENTITY, + NODE_CAPTURE_HEAP_LIMITS, + NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES, +} from "./CaptureRelease.js"; +import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCore.js"; + +/** Package-backed capture decoders share one bounded offline Node process profile. */ +export class NodeCaptureAdapter implements HistoricalCaptureFormatAdapter { + readonly identity; + readonly limitations; + constructor(readonly format: "har" | "saz") { + this.identity = + format === "har" + ? HAR_CAPTURE_PROVIDER_IDENTITY + : SAZ_CAPTURE_PROVIDER_IDENTITY; + this.limitations = + format === "saz" + ? [ + "SAZ ordinals follow first session-member occurrence in ZIP directory order, not chronology or numeric session ID.", + "HTTP fields use byte-preserving Latin-1 text; start-line separators and header delimiters are not projected. Bodies retain stored bytes without transfer/content decoding. Absent members are null, distinct from empty bodies.", + "XML preserves node order and attribute/text values, not its original byte serialization. WebSocket and gRPC members retain opaque message-file bytes, not decoded protocol events.", + "Non-session archive members are inventoried but their content is not decoded.", + ] + : []; + } + async command(requestPath: string) { + if ( + this.format === "saz" && + !NODE_CAPTURE_HEAP_ENFORCED && + NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES === null + ) + throw new AnalysisCapabilityUnavailableError( + this.identity.id, + "inspect_web_network_capture", + "SAZ decoding requires an enforced decoder memory limit. Run REA with Node.js on this host.", + ); + return { + command: process.execPath, + arguments: [ + `--max-old-space-size=${NODE_CAPTURE_HEAP_LIMITS.oldGenerationMiB}`, + `--max-semi-space-size=${NODE_CAPTURE_HEAP_LIMITS.semiSpaceMiB}`, + "--v8-pool-size=1", + "--single-threaded", + fileURLToPath(new URL("./NodeCaptureProcess.js", import.meta.url)), + requestPath, + this.format, + ], + ...(NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES === null + ? {} + : { memoryLimitBytes: NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES }), + }; + } +} diff --git a/src/browser/history/HarCaptureProcess.ts b/src/browser/history/NodeCaptureProcess.ts similarity index 59% rename from src/browser/history/HarCaptureProcess.ts rename to src/browser/history/NodeCaptureProcess.ts index 1e4c66c76..e94295da4 100644 --- a/src/browser/history/HarCaptureProcess.ts +++ b/src/browser/history/NodeCaptureProcess.ts @@ -3,7 +3,6 @@ import { z } from "zod"; import { readStableArtifact } from "../../artifacts/readStableArtifact.js"; import { WEB_NETWORK_CAPTURE_LIMITS } from "../../domain/webNetworkCapture.js"; import { CaptureFormatError } from "./CaptureFormatError.js"; -import { decodeHarCapture, decodeHarText } from "./HarCapture.js"; const requestSchema = z.strictObject({ snapshot_path: z.string(), @@ -11,24 +10,34 @@ const requestSchema = z.strictObject({ sensitive_values: z.array(z.string().min(1)), }); -/** Decode untrusted HAR in an owned, bounded heap rather than the long-lived MCP process. */ +/** Parse untrusted captures in an owned heap and return one complete bounded reply. */ const main = async (): Promise => { + const format = z.enum(["har", "saz"]).parse(process.argv[3]); const request = requestSchema.parse( JSON.parse(await readFile(process.argv[2] ?? "", "utf8")), ); let reply: unknown; try { - const snapshot = await readStableArtifact( - request.snapshot_path, - WEB_NETWORK_CAPTURE_LIMITS.inputBytes, - ); - reply = { - ok: true, - value: decodeHarCapture( + let value; + if (format === "har") { + const { decodeHarCapture, decodeHarText } = + await import("./HarCapture.js"); + const snapshot = await readStableArtifact( + request.snapshot_path, + WEB_NETWORK_CAPTURE_LIMITS.inputBytes, + ); + value = decodeHarCapture( decodeHarText(snapshot.bytes), request.sensitive_values, - ), - }; + ); + } else { + const { decodeSazCapture } = await import("./SazCapture.js"); + value = await decodeSazCapture( + request.snapshot_path, + request.sensitive_values, + ); + } + reply = { ok: true, value }; } catch (cause: unknown) { reply = cause instanceof CaptureFormatError @@ -41,7 +50,7 @@ const main = async (): Promise => { : { ok: false, reason: "decoder", - message: `HAR decoder failed unexpectedly (${cause instanceof Error ? cause.name : "unknown exception"}).`, + message: `Capture decoder failed unexpectedly (${cause instanceof Error ? cause.name : "unknown exception"}).`, pointer: "", }; } @@ -50,7 +59,7 @@ const main = async (): Promise => { text = JSON.stringify({ ok: false, reason: "limit", - message: "HAR reply exceeds the complete-evidence output budget.", + message: "Capture reply exceeds the complete-evidence output budget.", pointer: "", }); await writeFile(request.reply_path, text, { flag: "wx", mode: 0o600 }); diff --git a/src/browser/history/SazCapture.ts b/src/browser/history/SazCapture.ts new file mode 100644 index 000000000..0d597eb5a --- /dev/null +++ b/src/browser/history/SazCapture.ts @@ -0,0 +1,174 @@ +import { ZipArtifactReader } from "../../artifacts/ZipArtifactReader.js"; +import { + ArtifactReaderFailure, + type ArtifactEntry, +} from "../../artifacts/ArtifactReader.js"; +import { + WEB_NETWORK_CAPTURE_LIMITS, + type WebNetworkCaptureRecord, +} from "../../domain/webNetworkCapture.js"; +import { SAZ_CAPTURE_PROVIDER_IDENTITY } from "./CaptureRelease.js"; +import { CaptureFormatError } from "./CaptureFormatError.js"; +import { decodeSazHttp } from "./SazHttp.js"; +import { decodeSazMetadata } from "./SazMetadata.js"; +import { projectSazRecord, type SazValue } from "./SazProjection.js"; + +/** Decode native SAZ sessions without extracting files, running Fiddler or converting to HAR. */ +export const decodeSazCapture = async ( + snapshotPath: string, + sensitiveValues: readonly string[], +) => { + const reader = new ZipArtifactReader( + snapshotPath, + "zip", + WEB_NETWORK_CAPTURE_LIMITS.inputBytes, + ); + try { + const groups = new Map(); + const inventory: SazValue[] = []; + const paths = new Set(); + let declaredBytes = 0; + for await (const entry of reader.entries()) { + if (paths.has(entry.path)) + throw new CaptureFormatError( + "format", + "SAZ contains duplicate archive member names.", + ); + paths.add(entry.path); + if (entry.encrypted) + throw new CaptureFormatError( + "unsupported", + "Encrypted SAZ archives are unsupported; export an unencrypted copy from Fiddler Everywhere.", + ); + if (entry.kind !== "file" && entry.kind !== "directory") + throw new CaptureFormatError( + "unsupported", + "SAZ archive links are unsupported.", + ); + if (entry.declaredSize !== null) { + if (!Number.isSafeInteger(entry.declaredSize) || entry.declaredSize < 0) + throw new CaptureFormatError("format", "SAZ member size is invalid."); + declaredBytes += entry.declaredSize; + } + if (declaredBytes > WEB_NETWORK_CAPTURE_LIMITS.inputBytes) + throw new CaptureFormatError( + "limit", + "SAZ expanded members exceed the 32 MiB complete-evidence budget.", + ); + inventory.push({ + path: entry.path, + kind: entry.kind, + bytes: entry.declaredSize, + }); + if (entry.kind !== "file") continue; + const match = /^raw\/(\d+)_(c\.txt|s\.txt|m\.xml|w\.txt|g\.txt)$/.exec( + entry.path, + ); + if (match?.[1] !== undefined) { + const group = groups.get(match[1]) ?? []; + group.push(entry); + groups.set(match[1], group); + } else if (entry.path.startsWith("raw/")) { + throw new CaptureFormatError( + "unsupported", + "SAZ contains an unrecognized raw session member.", + ); + } + } + if (groups.size === 0) + throw new CaptureFormatError( + "format", + "Archive contains no SAZ session members.", + ); + const records: WebNetworkCaptureRecord[] = []; + for (const [session_id, entries] of groups) { + const source: Record = { + session_id, + request: null, + response: null, + metadata: null, + websocket: null, + grpc: null, + }; + for (const entry of entries) { + const stream = await reader.open(entry); + const chunks: Buffer[] = []; + let length = 0; + for await (const chunk of stream) { + if (!Buffer.isBuffer(chunk)) + throw new CaptureFormatError( + "format", + "SAZ member stream returned non-byte data.", + ); + length += chunk.length; + if (length > (entry.declaredSize ?? 0)) + throw new CaptureFormatError( + "format", + "SAZ member exceeds its declared size.", + ); + chunks.push(chunk); + } + if (length !== entry.declaredSize) + throw new CaptureFormatError( + "format", + "SAZ member does not match its declared size.", + ); + const bytes = Buffer.concat(chunks, length); + if (entry.path.endsWith("_c.txt")) + source.request = { + member: entry.path, + ...decodeSazHttp(bytes, true), + }; + else if (entry.path.endsWith("_s.txt")) + source.response = { + member: entry.path, + ...decodeSazHttp(bytes, false), + }; + else if (entry.path.endsWith("_m.xml")) + source.metadata = { + member: entry.path, + document: decodeSazMetadata(bytes), + }; + else + source[entry.path.endsWith("_w.txt") ? "websocket" : "grpc"] = { + member: entry.path, + messages: bytes, + }; + } + records.push({ + ordinal: records.length, + location: { + kind: "archive-members", + members: entries.map((entry) => entry.path), + }, + ...projectSazRecord(source, sensitiveValues), + limitations: [], + }); + } + const container = projectSazRecord({ members: inventory }, sensitiveValues); + return { + decoder: { ...SAZ_CAPTURE_PROVIDER_IDENTITY }, + container: { + reported: container.reported, + numeric_literals: container.numeric_literals, + redactions: container.redactions, + records_pointer: null, + }, + total_records: records.length, + records, + }; + } catch (cause: unknown) { + if (cause instanceof CaptureFormatError) throw cause; + if (cause instanceof ArtifactReaderFailure && cause.reason === "limit") + throw new CaptureFormatError( + "limit", + "SAZ archive exceeds the ZIP reader's resource budget.", + ); + throw new CaptureFormatError( + "format", + "SAZ archive or session data is malformed or fails ZIP integrity validation.", + ); + } finally { + await reader.close(); + } +}; diff --git a/src/browser/history/SazHttp.ts b/src/browser/history/SazHttp.ts new file mode 100644 index 000000000..246dd6164 --- /dev/null +++ b/src/browser/history/SazHttp.ts @@ -0,0 +1,72 @@ +import { CaptureFormatError } from "./CaptureFormatError.js"; + +/** Preserve ordered HTTP fields and the stored body; never dechunk, decompress or fetch. */ +export const decodeSazHttp = (bytes: Buffer, request: boolean) => { + const crlfSplit = bytes.indexOf("\r\n\r\n"); + // Once a CRLF separator is found, only an earlier LF separator can win. + // Avoid searching an otherwise unrelated large body a second time. + const lfSplit = bytes + .subarray(0, crlfSplit < 0 ? bytes.length : crlfSplit) + .indexOf("\n\n"); + // The first separator ends the headers. Later separators belong to the + // stored body, which may use different line endings or contain binary data. + const split = + crlfSplit >= 0 && (lfSplit < 0 || crlfSplit < lfSplit) ? crlfSplit : -1; + const boundary = split < 0 ? lfSplit : split; + if (boundary < 0) + throw new CaptureFormatError( + "format", + "SAZ HTTP message has no header/body separator.", + ); + const lines = bytes + .subarray(0, boundary) + .toString("latin1") + .split(split < 0 ? "\n" : "\r\n"); + const first = lines.shift() ?? ""; + const fields = request + ? /^(\S+) +(\S+) +(HTTP\/\d+(?:\.\d+)?)$/.exec(first) + : /^(HTTP\/\d+(?:\.\d+)?) +(\d{3})(?: (.*))?$/.exec(first); + if (fields === null) + throw new CaptureFormatError("format", "SAZ HTTP start line is malformed."); + const headers: { name: string; value: string }[] = []; + for (const line of lines) { + if (/^[ \t]/.test(line)) { + const previous = headers.at(-1); + if (previous === undefined) + throw new CaptureFormatError( + "format", + "SAZ HTTP header continuation has no field.", + ); + previous.value += `${split < 0 ? "\n" : "\r\n"}${line}`; + continue; + } + const colon = line.indexOf(":"); + // HTTP/2 pseudo-headers use the second colon as the delimiter. + const delimiter = colon === 0 ? line.indexOf(":", 1) : colon; + if (delimiter < 1) + throw new CaptureFormatError("format", "SAZ HTTP header is malformed."); + const name = line.slice(0, delimiter); + if (!/^:?[!#$%&'*+.^_`|~0-9A-Za-z-]+$/.test(name)) + throw new CaptureFormatError( + "format", + "SAZ HTTP header name is malformed.", + ); + headers.push({ name, value: line.slice(delimiter + 1) }); + } + const start = request + ? { + method: fields[1] ?? "", + url: fields[2] ?? "", + http_version: fields[3] ?? "", + } + : { + http_version: fields[1] ?? "", + status: fields[2] ?? "", + reason: fields[3] ?? "", + }; + return { + ...start, + headers, + body: bytes.subarray(boundary + (split < 0 ? 2 : 4)), + }; +}; diff --git a/src/browser/history/SazMetadata.ts b/src/browser/history/SazMetadata.ts new file mode 100644 index 000000000..6e311f8e6 --- /dev/null +++ b/src/browser/history/SazMetadata.ts @@ -0,0 +1,83 @@ +import { DOMParser, ParseError, type Node } from "@xmldom/xmldom"; +import type { JsonValue } from "../../domain/jsonValue.js"; +import { WEB_NETWORK_CAPTURE_LIMITS } from "../../domain/webNetworkCapture.js"; +import { CaptureFormatError } from "./CaptureFormatError.js"; + +/** Preserve XML node order, attribute names and text without interpreting producer-specific flags. */ +export const decodeSazMetadata = (bytes: Buffer): JsonValue => { + let xml: string; + try { + const encoding = + bytes[0] === 0xff && bytes[1] === 0xfe + ? "utf-16le" + : bytes[0] === 0xfe && bytes[1] === 0xff + ? "utf-16be" + : "utf-8"; + xml = new TextDecoder(encoding, { fatal: true }).decode(bytes); + } catch { + throw new CaptureFormatError( + "unsupported", + "SAZ metadata must use UTF-8 or BOM-marked UTF-16.", + ); + } + let parseFailed = false; + let document: ReturnType; + try { + document = new DOMParser({ + normalizeLineEndings: (source) => source, + onError: () => { + parseFailed = true; + }, + }).parseFromString(xml, "application/xml"); + } catch (cause: unknown) { + if (!(cause instanceof ParseError)) throw cause; + throw new CaptureFormatError( + "format", + "SAZ metadata is not well-formed XML.", + ); + } + // The pinned parser does not resolve external or DTD-defined entities. + // Inspect its parsed node; text inside comments/CDATA is ordinary evidence. + if (document.doctype !== null) + throw new CaptureFormatError( + "unsupported", + "SAZ metadata DTDs are unsupported.", + ); + if (parseFailed) + throw new CaptureFormatError( + "format", + "SAZ metadata is not well-formed XML.", + ); + if (document.documentElement?.tagName !== "Session") + throw new CaptureFormatError( + "format", + "SAZ metadata must have a Session document element.", + ); + const visit = (node: Node, depth: number): JsonValue => { + if (depth > WEB_NETWORK_CAPTURE_LIMITS.depth) + throw new CaptureFormatError( + "input-limit", + "SAZ metadata exceeds the complete-evidence nesting budget.", + ); + const children: JsonValue[] = []; + for (let child = node.firstChild; child !== null; child = child.nextSibling) + children.push(visit(child, depth + 1)); + const attributes: JsonValue[] = []; + if (node.nodeType === 1) { + const element = node as import("@xmldom/xmldom").Element; + for (let i = 0; i < element.attributes.length; i++) { + const attribute = element.attributes.item(i); + if (attribute !== null) + attributes.push({ name: attribute.name, value: attribute.value }); + } + } + return { + name: node.nodeName, + type: node.nodeType, + value: node.nodeValue, + attributes, + children, + }; + }; + return visit(document, 0); +}; diff --git a/src/browser/history/SazProjection.ts b/src/browser/history/SazProjection.ts new file mode 100644 index 000000000..eba813628 --- /dev/null +++ b/src/browser/history/SazProjection.ts @@ -0,0 +1,131 @@ +import { createHash } from "node:crypto"; +import { isUtf8 } from "node:buffer"; +import type { JsonValue } from "../../domain/jsonValue.js"; +import { + WEB_NETWORK_CAPTURE_LIMITS, + type WebNetworkCaptureRecord, +} from "../../domain/webNetworkCapture.js"; +import { CaptureRedaction, capturePointer } from "./CaptureRedaction.js"; +import { CaptureFormatError } from "./CaptureFormatError.js"; + +export type SazValue = + | JsonValue + | Buffer + | { readonly [key: string]: SazValue } + | readonly SazValue[]; + +/** Project native byte fields with the same credential and explicit-literal exclusions as HAR. */ +export const projectSazRecord = ( + source: SazValue, + sensitiveValues: readonly string[], +) => { + const redactor = new CaptureRedaction(sensitiveValues); + const binary_fields: WebNetworkCaptureRecord["binary_fields"] = []; + const visit = ( + value: SazValue, + pointer: string, + depth: number, + parent?: Readonly>, + ): JsonValue => { + if (depth > WEB_NETWORK_CAPTURE_LIMITS.depth) + throw new CaptureFormatError( + "input-limit", + "SAZ projection exceeds the complete-evidence nesting budget.", + pointer, + ); + if (parent !== undefined && redactor.harCredential(pointer, parent)) + return redactor.credential(pointer); + if (Buffer.isBuffer(value)) { + const base64 = value.toString("base64"); + const sha256 = createHash("sha256").update(value).digest("hex"); + const hidden = + redactor.sensitiveBytes(value) || + sensitiveValues.some( + (literal) => + base64.includes(literal) || + sha256.includes(literal) || + String(value.length).includes(literal), + ); + if (hidden) { + binary_fields.push({ + pointer, + representation: "producer-bytes", + state: "redacted", + content_base64: null, + bytes: null, + sha256: null, + }); + redactor.redactions.push({ + pointer, + reason: "explicit-sensitive-value", + }); + return null; + } + binary_fields.push({ + pointer, + representation: "producer-bytes", + state: "retained", + content_base64: base64, + bytes: value.length, + sha256, + }); + return isUtf8(value) ? value.toString("utf8") : null; + } + if (typeof value === "string") { + // HTTP fields are Latin-1 views of bytes. A UTF-8 declaration must also + // exclude its byte spelling, even when its display is mojibake. + if ( + /^\/(request|response)\//.test(pointer) && + !pointer.endsWith("/member") && + redactor.sensitiveBytes(Buffer.from(value, "latin1")) + ) { + redactor.redactions.push({ + pointer, + reason: "explicit-sensitive-value", + }); + return null; + } + return redactor.text( + value, + pointer, + redactor.harTransportUrl(pointer, parent), + ); + } + if (value === null || typeof value === "boolean") return value; + if (typeof value === "number") + return redactor.text(String(value), pointer) === null ? null : value; + if (Array.isArray(value)) + return value.map((child: SazValue, i: number) => + visit(child, capturePointer(pointer, String(i)), depth + 1), + ); + return Object.fromEntries( + Object.entries(value).flatMap(([key, child]) => { + if (sensitiveValues.some((literal) => key.includes(literal))) { + redactor.redactions.push({ + pointer, + reason: "explicit-sensitive-value", + scope: "property-name", + }); + return []; + } + return [ + [ + key, + visit( + child, + capturePointer(pointer, key), + depth + 1, + value as Readonly>, + ), + ], + ]; + }), + ); + }; + return { + reported: visit(source, "", 0), + binary_fields, + numeric_literals: [], + redactions: redactor.redactions, + }; +}; diff --git a/src/cli/webNetworkCaptureCommands.ts b/src/cli/webNetworkCaptureCommands.ts index 1b902969c..336c79364 100644 --- a/src/cli/webNetworkCaptureCommands.ts +++ b/src/cli/webNetworkCaptureCommands.ts @@ -1,4 +1,5 @@ import { z } from "incur"; +import { WEB_NETWORK_CAPTURE_FORMATS } from "../domain/webNetworkCapture.js"; import { createWebNetworkCaptureService } from "../composition/webNetworkCaptures.js"; import { CLI_COMMANDS } from "../cliCommandNames.js"; import { logCliCommand } from "../cliLogging.js"; @@ -16,10 +17,12 @@ export const registerWebNetworkCaptureCommands = ( const service = createWebNetworkCaptureService(environment); cli.command(CLI_COMMANDS.inspectWebNetworkCapture, { description: - "Inspect retained HAR or native mitmproxy records without fetching recorded URLs", + "Inspect retained HAR, native mitmproxy, or Fiddler SAZ records without fetching recorded URLs", args: z.object({ capturePath: z.string().describe("Absolute local capture path"), - format: z.enum(["har", "mitmproxy"]).describe("Producer capture format"), + format: z + .enum(WEB_NETWORK_CAPTURE_FORMATS) + .describe("Producer capture format"), }), options: z.object({ record: z diff --git a/src/composition/webNetworkCaptures.ts b/src/composition/webNetworkCaptures.ts index 6dc7668d4..017182500 100644 --- a/src/composition/webNetworkCaptures.ts +++ b/src/composition/webNetworkCaptures.ts @@ -1,6 +1,6 @@ import { WebNetworkCaptureService } from "../application/WebNetworkCaptureService.js"; import { HistoricalCaptureDecoder } from "../browser/history/HistoricalCaptureDecoder.js"; -import { HarCaptureAdapter } from "../browser/history/HarCaptureAdapter.js"; +import { NodeCaptureAdapter } from "../browser/history/NodeCaptureAdapter.js"; import { MitmproxyCaptureAdapter } from "../browser/history/MitmproxyCaptureAdapter.js"; /** Compose offline format adapters without import-time reads, installation or process startup. */ @@ -9,7 +9,11 @@ export const createWebNetworkCaptureService = ( ): WebNetworkCaptureService => new WebNetworkCaptureService( new HistoricalCaptureDecoder( - [new HarCaptureAdapter(), new MitmproxyCaptureAdapter(environment)], + [ + new NodeCaptureAdapter("har"), + new NodeCaptureAdapter("saz"), + new MitmproxyCaptureAdapter(environment), + ], environment, ), ); diff --git a/src/contracts/analysisViewToolContracts.ts b/src/contracts/analysisViewToolContracts.ts index a4004e85b..161d9d7b4 100644 --- a/src/contracts/analysisViewToolContracts.ts +++ b/src/contracts/analysisViewToolContracts.ts @@ -16,7 +16,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [ ...toolContractMetadata("inspect_analysis_view"), kind: "application", description: - 'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_evm_interface, inspect_managed_artifact, inspect_android_package or inspect_web_network_capture Evidence without repeating analysis. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. For layout, JavaScript, native, inventory, WASM, EVM, managed artifact and Android package sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Native views select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', + 'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_evm_interface, inspect_managed_artifact, inspect_android_package or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. For layout, JavaScript, native, inventory, WASM, EVM, managed artifact and Android package sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Native views select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', inputSchema: inspectAnalysisViewInputSchema, outputSchema: evidenceResultOf(analysisViewResultSchema), examples: [ diff --git a/src/contracts/webNetworkCaptureToolContracts.ts b/src/contracts/webNetworkCaptureToolContracts.ts index cb919172f..06420f960 100644 --- a/src/contracts/webNetworkCaptureToolContracts.ts +++ b/src/contracts/webNetworkCaptureToolContracts.ts @@ -4,7 +4,11 @@ import { } from "../domain/webNetworkCapture.js"; import type { ToolContract } from "./toolContractTypes.js"; import { toolContractMetadata } from "./toolEffects.js"; -import { evidenceResultOf } from "./toolOutputSchemaPrimitives.js"; +import { + analysisDetailSchema, + analysisDetailOutputSchemaOf, +} from "./analysisDetail.js"; +import { webNetworkCaptureSummaryViewResultSchema } from "../domain/analysisView/analysisView.js"; /** Historical inspection is distinct from live browser capture and runtime authority. */ export const WEB_NETWORK_CAPTURE_TOOL_CONTRACTS = [ @@ -12,10 +16,15 @@ export const WEB_NETWORK_CAPTURE_TOOL_CONTRACTS = [ name: "inspect_web_network_capture", ...toolContractMetadata("inspect_web_network_capture"), description: - "Inspect retained local HAR 1.2 or native mitmproxy capture records without contacting recorded URLs. Returns original ordinals, pointers/byte ranges, preserved producer fields, precise numeric representations and exact exposed binary bytes inline. Excludes known transport credentials and explicitly marked values. Decodes every record before optional ordinal selection; malformed or oversized captures return no partial success. Duplicate HAR members are invalid; prototype-named members such as __proto__ are preserved as ordinary own JSON members without prototype mutation. Uses pinned upstream parsers, owned temporary files and bounded offline processes. Native mitmproxy requires caller-supplied mitmdump 12.2.3 on Linux. Browser transaction IDs, runtime attribution and original bytes of HAR Unicode text remain unknown.", + "Request detail summary for capture identity and counts; then page retained records with inspect_analysis_view kind capture (preview or complete), avoiding repeated decoding. Default detail complete preserves existing inline delivery. Inspect retained local HAR 1.2, native mitmproxy, or Fiddler SAZ capture records without contacting recorded URLs. Returns original ordinals, pointers/byte ranges/archive member names, preserved producer fields, precise numeric representations and exact exposed binary bytes inline. Excludes known transport credentials and explicitly marked values. Decodes every record before optional ordinal selection; malformed or oversized captures return no partial success. Duplicate HAR members are invalid; prototype-named members such as __proto__ are preserved as ordinary own JSON members without prototype mutation. Uses pinned upstream parsers, owned temporary files and bounded offline processes. SAZ preserves HTTP fields, XML metadata and encoded body/message bytes without replay or HAR conversion; encrypted archives are unsupported. Native mitmproxy requires caller-supplied mitmdump 12.2.3 on Linux. Browser transaction IDs, runtime attribution and original bytes of HAR Unicode text remain unknown.", kind: "browser-provider", - inputSchema: inspectWebNetworkCaptureInputSchema, - outputSchema: evidenceResultOf(webNetworkCaptureSchema), + inputSchema: inspectWebNetworkCaptureInputSchema.extend({ + detail: analysisDetailSchema, + }), + outputSchema: analysisDetailOutputSchemaOf( + webNetworkCaptureSchema, + webNetworkCaptureSummaryViewResultSchema, + ), examples: [ { title: "Inspect historical HAR entries", diff --git a/src/domain/analysisView/analysisView.ts b/src/domain/analysisView/analysisView.ts index 784d5dbed..6b3b6bef8 100644 --- a/src/domain/analysisView/analysisView.ts +++ b/src/domain/analysisView/analysisView.ts @@ -55,6 +55,12 @@ import { wasmViewItemSchema, wasmSummarySchema, } from "./wasmArtifactView.js"; +import { + projectWebNetworkCaptureView, + captureViewRequestSchema, + captureViewItemsSchema, + captureSummarySchema, +} from "./webNetworkCaptureView.js"; import type { WebNetworkApiSurfaceAccumulator } from "./WebNetworkApiSurfaceAccumulator.js"; import { projectAndroidPackageView, @@ -177,6 +183,7 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [ managedViewRequestSchema, androidViewRequestSchema, apiSurfaceViewSchema, + captureViewRequestSchema, ]); // The application owns admission and authenticated Evidence parsing. SDK and @@ -216,7 +223,7 @@ const artifactSchema = z.strictObject({ path: z .string() .describe("Exact recorded artifact path, including an empty string."), - sha256: digestSchema, + sha256: digestSchema.nullable(), }); const layoutSummarySchema = z.strictObject({ @@ -290,6 +297,7 @@ const viewResultBase = { "inspect_wasm_artifact", "inspect_evm_interface", "inspect_managed_artifact", + "inspect_web_network_capture", "inspect_android_package", ]), parent_digest: digestSchema, @@ -348,6 +356,13 @@ export const managedArtifactSummaryViewResultSchema = z.strictObject({ summary: managedSummarySchema, }); +/** Exact historical capture summary, with no body or member-inventory payload. */ +export const webNetworkCaptureSummaryViewResultSchema = z.strictObject({ + ...summaryViewResultBase, + parent_operation: z.literal("inspect_web_network_capture"), + summary: captureSummarySchema, +}); + /** Exact package declarations with retained decoded manifest text. */ export const androidPackageSummaryViewResultSchema = z.strictObject({ ...summaryViewResultBase, @@ -377,9 +392,17 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ wasmSummarySchema, evmSummarySchema, managedSummarySchema, + captureSummarySchema, androidPackageSummarySchema, ]), }), + z.strictObject({ + kind: z.literal("capture"), + view: captureViewRequestSchema, + items: captureViewItemsSchema, + ...viewResultBase, + parent_operation: z.literal("inspect_web_network_capture"), + }), z.strictObject({ kind: z.literal("facet"), view: facetViewSchema, @@ -500,6 +523,10 @@ export type UnsignedAnalysisView = > | Omit, "view_digest"> | Omit, "view_digest"> + | Omit< + Extract, + "view_digest" + > | Omit< Extract, "view_digest" @@ -603,10 +630,7 @@ export const projectAnalysisView = ( }, ]), ); - if ( - view.kind === "api-surface" || - parent.operation === "inspect_web_network_capture" - ) { + if (view.kind === "api-surface") { const projected = projectWebNetworkApiSurfaceView( parent, view, @@ -614,6 +638,10 @@ export const projectAnalysisView = ( ); return projected.ok ? ok(sealAnalysisView(projected.value)) : projected; } + if (parent.operation === "inspect_web_network_capture") { + const projected = projectWebNetworkCaptureView(parent, view); + return projected.ok ? ok(sealAnalysisView(projected.value)) : projected; + } if (parent.operation === "inspect_binary_layout") { const layout = binaryLayoutSchema.safeParse(parent.normalizedResult); if (!layout.success) diff --git a/src/domain/analysisView/javascriptApplicationView.ts b/src/domain/analysisView/javascriptApplicationView.ts index f4c09a68d..41a22f8d9 100644 --- a/src/domain/analysisView/javascriptApplicationView.ts +++ b/src/domain/analysisView/javascriptApplicationView.ts @@ -241,22 +241,14 @@ export const projectJavaScriptApplicationView = ( analysis: JavaScriptApplicationAnalysisResult, view: Exclude, ): Result => { + if (view.kind !== "summary" && view.kind !== "item" && view.kind !== "page") + return err( + jsIncompatible( + "JavaScript application Evidence supports summary, module item or module page views.", + ), + ); const shared = parentFields(parent, analysis); const modules = moduleEntries(analysis.graph.nodes); - if (view.kind === "native") - return err( - jsIncompatible("Native views require analyze_function Evidence."), - ); - if (view.kind === "inventory") - return err(jsIncompatible("Inventory views require artifact Evidence.")); - if (view.kind === "wasm") - return err( - jsIncompatible("WASM views require inspect_wasm_artifact Evidence."), - ); - if (view.kind === "evm") - return err( - jsIncompatible("EVM views require inspect_evm_interface Evidence."), - ); if (view.kind === "summary") return ok({ kind: "summary", @@ -291,12 +283,6 @@ export const projectJavaScriptApplicationView = ( coverage: completeCoverage(modules.length, analysis.graph.nodes.length), ...shared, }); - if (view.kind === "facet") - return err( - jsIncompatible( - "facet views apply to inspect_binary_layout Evidence (mitigations or linkage).", - ), - ); if (view.kind === "item") { if (view.collection !== "modules") return err( diff --git a/src/domain/analysisView/webNetworkApiSurfaceView.test.ts b/src/domain/analysisView/webNetworkApiSurfaceView.test.ts index f13e392ff..b57d54b59 100644 --- a/src/domain/analysisView/webNetworkApiSurfaceView.test.ts +++ b/src/domain/analysisView/webNetworkApiSurfaceView.test.ts @@ -85,11 +85,13 @@ it("pages groups independently of record completeness and preserves null digests }); it("rejects native captures and malformed parents with typed errors", () => { const capture = analysisViewCaptureFixture(); - capture.format = "mitmproxy"; - expect(project(capture)).toMatchObject({ - ok: false, - error: { _tag: "AnalysisUnsupportedTargetError" }, - }); + for (const format of ["mitmproxy", "saz"] as const) { + capture.format = format; + expect(project(capture)).toMatchObject({ + ok: false, + error: { _tag: "AnalysisUnsupportedTargetError" }, + }); + } expect( projectAnalysisView( { diff --git a/src/domain/analysisView/webNetworkCaptureView.test.ts b/src/domain/analysisView/webNetworkCaptureView.test.ts new file mode 100644 index 000000000..ff73b29a6 --- /dev/null +++ b/src/domain/analysisView/webNetworkCaptureView.test.ts @@ -0,0 +1,176 @@ +import { expect, it } from "vitest"; +import { createHash } from "node:crypto"; +import { createEvidence } from "../evidence.js"; +import { webNetworkCaptureSchema } from "../webNetworkCapture.js"; +import { projectAnalysisView } from "./analysisView.js"; + +const capture = () => { + const body = Buffer.from("original body"); + const artifact = { + path: "/recordings/input.saz", + sha256: "b".repeat(64), + bytes: 100, + }; + const decoder = { id: "fixture", name: "Fixture", version: "1" }; + const report = webNetworkCaptureSchema.parse({ + artifact, + decoder, + format: "saz", + runtime_attribution: "unknown", + limitations: [], + container: { + reported: { members: [] }, + numeric_literals: [], + redactions: [], + records_pointer: null, + }, + total_records: 9, + records: [7, 2].map((ordinal) => ({ + ordinal, + location: { kind: "archive-members", members: [`raw/${ordinal}_c.txt`] }, + reported: { + session_id: String(ordinal), + time: null, + request: { + method: "POST", + url: "https://example.invalid/a", + body: body.toString(), + }, + }, + binary_fields: [ + { + pointer: "/request/body", + representation: "producer-bytes", + state: "retained", + content_base64: body.toString("base64"), + bytes: body.length, + sha256: createHash("sha256").update(body).digest("hex"), + }, + ], + numeric_literals: [ + { + pointer: "/time", + producer_type: "json-number", + literal: "9007199254740993", + }, + ], + redactions: [], + limitations: [], + })), + }); + const evidence = createEvidence({ ...artifact, format: "file" }, decoder, { + operation: "inspect_web_network_capture", + parameters: { record_ordinals: [7, 2] }, + result: report, + confidence: "observed", + authority: "historical-reference", + }); + return { + report, + parent: { + evidenceId: evidence.evidence_id, + operation: evidence.operation, + normalizedResult: report, + subject: evidence.subject, + limitations: evidence.limitations, + }, + }; +}; + +it("keeps selected-record page coverage distinct from capture coverage and unsafe number absence", () => { + const { parent, report } = capture(); + const summary = projectAnalysisView(parent, { kind: "summary" }); + expect(summary.ok && summary.value).toMatchObject({ + summary: { total_records: 9, retained_records: 2 }, + coverage: { examined: 2, total: 2 }, + }); + const preview = projectAnalysisView(parent, { + kind: "capture", + detail: "preview", + offset: 0, + limit: 1, + }); + expect(preview.ok && preview.value).toMatchObject({ + items: [ + { + ordinal: 7, + reported: { time: null }, + numeric_literals: [{ pointer: "/time", literal: "9007199254740993" }], + }, + ], + coverage: { total: 2, examined: 1, next_offset: 1, exhausted: false }, + }); + expect(JSON.stringify(preview)).not.toContain("original body"); + expect(JSON.stringify(preview)).not.toContain("content_base64"); + const complete = projectAnalysisView(parent, { + kind: "capture", + detail: "complete", + offset: 1, + limit: 1, + }); + expect(complete.ok && complete.value).toMatchObject({ + items: [report.records[1]], + coverage: { total: 2, examined: 1, next_offset: null, exhausted: true }, + }); + const empty = projectAnalysisView(parent, { + kind: "capture", + detail: "preview", + offset: 50, + limit: 1, + }); + expect(empty.ok && empty.value).toMatchObject({ + items: [], + coverage: { + status: "empty", + total: 2, + examined: 0, + next_offset: null, + exhausted: true, + }, + }); +}); + +it("retains explicitly unknown digests and rejects a different subject path", () => { + const { parent, report } = capture(); + const unknown = projectAnalysisView( + { + ...parent, + subject: null, + normalizedResult: { + ...report, + artifact: { ...report.artifact, sha256: null }, + }, + }, + { kind: "summary" }, + ); + expect(unknown.ok && unknown.value).toMatchObject({ + artifact: { sha256: null }, + }); + expect(unknown.ok && unknown.value.unknowns.join(" ")).toContain( + "excluded artifact digest", + ); + if (parent.subject === null) throw new Error("Expected fixture subject"); + const mismatched = projectAnalysisView( + { + ...parent, + subject: { ...parent.subject, local_path: "/different/source.saz" }, + }, + { kind: "summary" }, + ); + expect(mismatched.ok).toBe(false); +}); + +it("rejects invalid ordinal coverage in a borrowed parent", () => { + const { parent, report } = capture(); + const result = projectAnalysisView( + { + ...parent, + normalizedResult: { + ...report, + records: [report.records[0], report.records[0]], + }, + }, + { kind: "summary" }, + ); + expect(result.ok).toBe(false); +}); diff --git a/src/domain/analysisView/webNetworkCaptureView.ts b/src/domain/analysisView/webNetworkCaptureView.ts new file mode 100644 index 000000000..07495466c --- /dev/null +++ b/src/domain/analysisView/webNetworkCaptureView.ts @@ -0,0 +1,212 @@ +import { z } from "zod"; +import type { AnalysisError } from "../analysisErrorBase.js"; +import { analysisInputErrorFromIssues } from "../inputIssueProjection.js"; +import type { JsonValue } from "../jsonValue.js"; +import { err, ok, type Result } from "../result.js"; +import { + webNetworkCaptureRecordSchema, + webNetworkCaptureBinaryDescriptorSchema, + webNetworkCaptureSchema, + type WebNetworkCaptureRecord, +} from "../webNetworkCapture.js"; +import { + analysisViewInputError, + type AnalysisViewParent, + type AnalysisViewRequest, + type UnsignedAnalysisView, +} from "./analysisView.js"; +import { + completeWithinViewCoverage, + pageViewCoverage, +} from "./analysisViewCoverage.js"; + +/** Offsets page the parent's retained record order, preserving producer ordinals. */ +export const captureViewRequestSchema = z.strictObject({ + kind: z.literal("capture"), + detail: z.enum(["preview", "complete"]).default("preview"), + offset: z.number().int().nonnegative().default(0), + limit: z.number().int().positive().default(64), +}); + +const previewSchema = webNetworkCaptureRecordSchema.extend({ + binary_fields: z.array(webNetworkCaptureBinaryDescriptorSchema), +}); +export const captureViewItemsSchema = z.array( + z.union([webNetworkCaptureRecordSchema, previewSchema]), +); + +/** Count/identity overview; bodies and the archive member inventory stay in the parent. */ +export const captureSummarySchema = z.strictObject({ + artifact: webNetworkCaptureSchema.shape.artifact, + decoder: webNetworkCaptureSchema.shape.decoder, + format: webNetworkCaptureSchema.shape.format, + total_records: z.number().int().nonnegative(), + retained_records: z.number().int().nonnegative(), + runtime_attribution: z.literal("unknown"), +}); + +const object = ( + value: JsonValue, +): Readonly> | null => + value !== null && typeof value === "object" && !Array.isArray(value) + ? value + : null; + +/** Select already parsed producer facts without interpreting HTTP or payload bytes again. */ +const preview = (record: WebNetworkCaptureRecord) => { + const source = object(record.reported); + const reported: Record = {}; + const selectedPointers = new Set(); + if (source !== null) { + for (const key of ["session_id", "name", "time", "startedDateTime", "type"]) + if (Object.hasOwn(source, key) && source[key] !== undefined) { + reported[key] = source[key]; + selectedPointers.add(`/${key}`); + } + for (const key of ["request", "response"]) { + const value = source[key]; + if (!Object.hasOwn(source, key) || value === undefined) continue; + const fields = object(value); + if (fields === null) { + // Preserve a reported null/scalar instead of inventing HTTP fields. + reported[key] = value; + continue; + } + const selected: Record = {}; + for (const field of [ + "method", + "url", + "httpVersion", + "http_version", + "status", + "statusText", + "reason", + ]) + if (Object.hasOwn(fields, field) && fields[field] !== undefined) { + selected[field] = fields[field]; + selectedPointers.add(`/${key}/${field}`); + } + reported[key] = selected; + } + } + return { + ...record, + reported: source === null ? record.reported : reported, + binary_fields: record.binary_fields.map( + ({ content_base64: _content, ...field }) => field, + ), + numeric_literals: + source === null + ? record.numeric_literals + : record.numeric_literals.filter(({ pointer }) => + selectedPointers.has(pointer), + ), + }; +}; + +/** Page immutable retained facts; no capture file, provider or network is consulted. */ +export const projectWebNetworkCaptureView = ( + parent: AnalysisViewParent, + view: AnalysisViewRequest, +): Result => { + if (view.kind !== "summary" && view.kind !== "capture") + return err( + analysisViewInputError([ + { + path: ["view"], + reason: "invalid_value", + message: + "Historical captures support summary, capture record pages, and admitted api-surface views.", + }, + ]), + ); + const parsed = webNetworkCaptureSchema.safeParse(parent.normalizedResult); + if (!parsed.success) + return err( + analysisInputErrorFromIssues( + "inspect_analysis_view", + parsed.error.issues.map((issue) => ({ + ...issue, + path: ["source", "normalized_result", ...issue.path], + })), + { source: { normalized_result: parent.normalizedResult } }, + { cause: parsed.error }, + ), + ); + const report = parsed.data; + if ( + report.artifact.sha256 !== null && + (parent.subject == null || + parent.subject.local_path !== report.artifact.path || + parent.subject.digest.sha256 !== report.artifact.sha256) + ) + return err( + analysisViewInputError([ + { + path: ["source", "subject"], + reason: "invalid_value", + message: + "Capture parent subject must identify the report's exact path and SHA-256.", + }, + ]), + ); + const shared = { + parent_evidence_id: parent.evidenceId, + parent_operation: "inspect_web_network_capture" as const, + parent_digest: parent.evidenceId.slice(3), + artifact: { path: report.artifact.path, sha256: report.artifact.sha256 }, + limitations: [ + ...new Set([ + ...parent.limitations, + ...report.limitations, + "Pages cover the records selected into the parent Evidence, in that selected order; total_records counts the original decoded capture.", + ]), + ], + unknowns: [ + "Historical records do not establish live runtime attribution or deployment authenticity.", + ...(report.artifact.sha256 === null + ? ["The explicitly excluded artifact digest remains unknown."] + : []), + ], + }; + if (view.kind === "summary") + return ok({ + kind: "summary", + view, + ...shared, + summary: { + artifact: report.artifact, + decoder: report.decoder, + format: report.format, + total_records: report.total_records, + retained_records: report.records.length, + runtime_attribution: report.runtime_attribution, + }, + coverage: completeWithinViewCoverage( + report.records.length, + report.records.length, + ), + }); + const records = report.records.slice(view.offset, view.offset + view.limit); + return ok({ + kind: "capture", + view, + ...shared, + items: captureViewItemsSchema.parse( + view.detail === "complete" ? records : records.map(preview), + ), + coverage: pageViewCoverage( + view.offset, + records.length, + report.records.length, + ), + limitations: [ + ...shared.limitations, + ...(view.detail === "preview" + ? [ + "Preview selects recorded headline fields and byte descriptors; headers, XML, bodies, message data and other fields remain in complete record pages. Missing preview fields do not establish their absence from the producer record.", + ] + : []), + ], + }); +}; diff --git a/src/domain/webNetworkCapture.ts b/src/domain/webNetworkCapture.ts index ed31d5c70..ec98ca496 100644 --- a/src/domain/webNetworkCapture.ts +++ b/src/domain/webNetworkCapture.ts @@ -10,12 +10,14 @@ export const WEB_NETWORK_CAPTURE_LIMITS = { depth: 64, } as const; +export const WEB_NETWORK_CAPTURE_FORMATS = ["har", "mitmproxy", "saz"] as const; + /** Inspect an explicit retained capture without contacting its recorded destinations. */ export const inspectWebNetworkCaptureInputSchema = z.strictObject({ capture_path: localPathStringSchema.describe( - "Absolute local HAR or mitmproxy capture path.", + "Absolute local HAR, mitmproxy, or Fiddler SAZ capture path.", ), - format: z.enum(["har", "mitmproxy"]), + format: z.enum(WEB_NETWORK_CAPTURE_FORMATS), record_ordinals: z .array(z.number().int().nonnegative()) .min(1) @@ -31,6 +33,21 @@ export const inspectWebNetworkCaptureInputSchema = z.strictObject({ ), }); +/** Read declarations through their owning schema for service and adapter failures. */ +export const captureSensitiveValues = ( + rawInput: unknown, +): readonly string[] => { + const parsed = + inspectWebNetworkCaptureInputSchema.shape.sensitive_values.safeParse( + typeof rawInput === "object" && + rawInput !== null && + "sensitive_values" in rawInput + ? rawInput.sensitive_values + : undefined, + ); + return parsed.success ? parsed.data : []; +}; + const pointerSchema = z .string() .describe("RFC 6901 pointer in the producer's reported object."); @@ -53,21 +70,26 @@ const binaryBase = { pointer: pointerSchema, representation: z.enum(["producer-bytes", "har-base64-content"]), }; +const retainedBinaryDescriptorSchema = z.strictObject({ + ...binaryBase, + state: z.literal("retained"), + bytes: z.number().int().nonnegative(), + sha256: z.string().regex(/^[0-9a-f]{64}$/), +}); +const redactedBinaryDescriptorSchema = z.strictObject({ + ...binaryBase, + state: z.literal("redacted"), + bytes: z.null(), + sha256: z.null(), +}); +/** Exact byte facts shared by complete records and previews that omit content. */ +export const webNetworkCaptureBinaryDescriptorSchema = z.discriminatedUnion( + "state", + [retainedBinaryDescriptorSchema, redactedBinaryDescriptorSchema], +); const binarySchema = z.discriminatedUnion("state", [ - z.strictObject({ - ...binaryBase, - state: z.literal("retained"), - content_base64: z.string(), - bytes: z.number().int().nonnegative(), - sha256: z.string().regex(/^[0-9a-f]{64}$/), - }), - z.strictObject({ - ...binaryBase, - state: z.literal("redacted"), - content_base64: z.null(), - bytes: z.null(), - sha256: z.null(), - }), + retainedBinaryDescriptorSchema.extend({ content_base64: z.string() }), + redactedBinaryDescriptorSchema.extend({ content_base64: z.null() }), ]); const numberSchema = z.strictObject({ pointer: pointerSchema, @@ -84,6 +106,10 @@ export const webNetworkCaptureRecordSchema = z.strictObject({ reason: z.literal("explicit-sensitive-value"), }), z.strictObject({ kind: z.literal("json-pointer"), pointer: pointerSchema }), + z.strictObject({ + kind: z.literal("archive-members"), + members: z.array(z.string()).min(1), + }), z.strictObject({ kind: z.literal("byte-range"), offset: z.number().int().nonnegative(), @@ -156,7 +182,7 @@ export const webNetworkCaptureSchema = ), bytes: z.number().int().nonnegative(), }), - format: z.enum(["har", "mitmproxy"]), + format: z.enum(WEB_NETWORK_CAPTURE_FORMATS), runtime_attribution: z.literal("unknown"), limitations: z.array(z.string()), }); diff --git a/src/domain/webNetworkCaptureCoordinates.ts b/src/domain/webNetworkCaptureCoordinates.ts index 6cba46cd9..617dd7607 100644 --- a/src/domain/webNetworkCaptureCoordinates.ts +++ b/src/domain/webNetworkCaptureCoordinates.ts @@ -22,6 +22,8 @@ export const excludeCaptureCoordinates = ( const location = (record.location.kind === "json-pointer" && sensitive(record.location.pointer)) || + (record.location.kind === "archive-members" && + record.location.members.some(sensitive)) || (record.location.kind === "byte-range" && (sensitive(String(record.location.offset)) || sensitive(String(record.location.bytes)))) diff --git a/src/generatedPackageMetadata.ts b/src/generatedPackageMetadata.ts index 1af9203ad..bada60663 100644 --- a/src/generatedPackageMetadata.ts +++ b/src/generatedPackageMetadata.ts @@ -5,5 +5,5 @@ export const PACKAGE_METADATA = { serverSdkVersion: "2.3.1", clientSdkVersion: "2.3.1", coreSdkVersion: "2.3.1", - skillVersion: "40", + skillVersion: "41", } as const; diff --git a/src/server/createServer.ts b/src/server/createServer.ts index e445dcfab..c213efb0b 100644 --- a/src/server/createServer.ts +++ b/src/server/createServer.ts @@ -617,6 +617,7 @@ const registerConfiguredAnalysisTools = ( toolLogger, recordEvidence, withAdmittedAnalysis, + evidenceById, ); registerJavaScriptRecoveryTool( server, diff --git a/src/server/registerWebNetworkCaptureTool.ts b/src/server/registerWebNetworkCaptureTool.ts index 9f0c9dec8..8b1203745 100644 --- a/src/server/registerWebNetworkCaptureTool.ts +++ b/src/server/registerWebNetworkCaptureTool.ts @@ -1,5 +1,14 @@ import type { EvidenceMcpServer } from "./EvidenceMcpServer.js"; import type { StandardSchemaWithJSON } from "@modelcontextprotocol/server"; +import type { EvidenceLookup } from "../application/EvidenceInputResolver.js"; +import { + requireAnalysisViewRetention, + summarizeRetainedAnalysis, +} from "../application/analysisView/AnalysisViewService.js"; +import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js"; +import { redactExplicitFailure } from "../domain/explicitSensitiveFailure.js"; +import { captureSensitiveValues } from "../domain/webNetworkCapture.js"; +import { err } from "../domain/result.js"; import type { WebNetworkCaptureService } from "../application/WebNetworkCaptureService.js"; import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js"; import { toolContract } from "../contracts/toolContracts.js"; @@ -15,6 +24,7 @@ export const registerWebNetworkCaptureTool = ( logger: Logger, recordEvidence?: EvidenceWriter["recordEvidence"], withAdmittedAnalysis?: WithAdmittedAnalysis, + evidenceById?: EvidenceLookup, ): void => { const contract = toolContract("inspect_web_network_capture"); const registration = toolRegistrationOptions(contract); @@ -30,7 +40,31 @@ export const registerWebNetworkCaptureTool = ( }; const handler = createEvidenceToolHandler( server, - (input, options) => service.inspect(input, options), + async (rawInput, options) => { + const parsed = contract.inputSchema.safeParse(rawInput); + if (!parsed.success) + return err( + redactExplicitFailure( + analysisInputErrorFromIssues( + contract.name, + parsed.error.issues, + rawInput, + { cause: parsed.error }, + ), + captureSensitiveValues(rawInput), + ), + ); + const { detail, ...input } = parsed.data; + const retention = { recordEvidence, evidenceById }; + if (detail === "summary") { + const ready = requireAnalysisViewRetention(contract.name, retention); + if (!ready.ok) return ready; + } + const inspected = await service.inspect(input, options); + return detail === "summary" && inspected.ok + ? summarizeRetainedAnalysis(inspected.value, retention) + : inspected; + }, logger, recordEvidence, withAdmittedAnalysis, diff --git a/tests/acceptance/tools/historicalCaptureResources.test.ts b/tests/acceptance/tools/historicalCaptureResources.test.ts index d4c56d644..64a7b6638 100644 --- a/tests/acceptance/tools/historicalCaptureResources.test.ts +++ b/tests/acceptance/tools/historicalCaptureResources.test.ts @@ -5,7 +5,7 @@ import { fileURLToPath } from "node:url"; import { promisify } from "node:util"; import { expect, it } from "vitest"; -import { HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES } from "../../../src/browser/history/CaptureRelease.js"; +import { NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES } from "../../../src/browser/history/CaptureRelease.js"; import { WEB_NETWORK_CAPTURE_LIMITS } from "../../../src/domain/webNetworkCapture.js"; import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js"; import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; @@ -46,7 +46,7 @@ const capture = (count: number) => // Node.js enforces the decoder's V8 heap flags. Bun ignores them; on Windows // the decoder's Job Object commit limit is the enforced bound instead, and // Bun elsewhere has no bound this capture can reach. -const jobLimit = HAR_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES; +const jobLimit = NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES; it.skipIf(process.versions.bun !== undefined && jobLimit === null)( "reports real HAR memory exhaustion through CLI and MCP and preserves server usability", async () => { diff --git a/tests/boundary/browser/sazWebCapture.test.ts b/tests/boundary/browser/sazWebCapture.test.ts new file mode 100644 index 000000000..69eca33b8 --- /dev/null +++ b/tests/boundary/browser/sazWebCapture.test.ts @@ -0,0 +1,499 @@ +import { createHash } from "node:crypto"; +import { execFile } from "node:child_process"; +import { readFile, unlink, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { promisify } from "node:util"; +import { fileURLToPath } from "node:url"; +import { gzipSync } from "node:zlib"; +import { Uint8ArrayReader, Uint8ArrayWriter, ZipWriter } from "@zip.js/zip.js"; +import { Ajv2020 } from "ajv/dist/2020.js"; +import { expect, it as runtimeTest } from "vitest"; +import { z } from "zod"; +import { parseEvidence } from "../../../src/domain/evidence.js"; +import { webNetworkCaptureSchema } from "../../../src/domain/webNetworkCapture.js"; +import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +import { + NODE_CAPTURE_HEAP_ENFORCED, + NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES, +} from "../../../src/browser/history/CaptureRelease.js"; +const hasMemoryBound = + NODE_CAPTURE_HEAP_ENFORCED || + NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES !== null; + +const it = runtimeTest.skipIf(!hasMemoryBound); + +// Synthetic member fixtures based on Telerik's SAZ layout; not a real-producer golden. +const compressedBody = gzipSync(Buffer.from("stored compressed body")); +const members = (): [string, Buffer][] => [ + [ + "raw/0042_s.txt", + Buffer.concat([ + Buffer.from( + "HTTP/1.1 200 OK\r\nContent-Encoding: gzip\r\nSet-Cookie: server-credential\r\n\r\n", + ), + compressedBody, + ]), + ], + [ + "raw/7_c.txt", + Buffer.concat([ + Buffer.from("POST /bytes HTTP/2.0\r\n:authority: example.test\r\n\r\n"), + Buffer.from([0xff, 0, 0x80]), + ]), + ], + [ + "raw/0042_c.txt", + Buffer.from( + "GET https://user:password@example.test/a HTTP/1.1\r\nAuthorization: Bearer client-credential\r\nCookie: cookie-credential\r\nX-Duplicate: one\r\nX-Duplicate: two\r\nX-Text: unicode-\u03b2\r\n\r\n", + ), + ], + [ + "raw/0042_m.xml", + Buffer.from( + '', + ), + ], + ["raw/0042_w.txt", Buffer.from("opaque websocket message file")], + ["raw/7_g.txt", Buffer.from([0, 1, 2])], + ["[Content_Types].xml", Buffer.from("")], +]; +const archive = async (entries = members(), password?: string, level = 6) => { + const writer = new ZipWriter(new Uint8ArrayWriter(), { + useWebWorkers: false, + level, + ...(password === undefined ? {} : { password }), + }); + for (const [name, bytes] of entries) + await writer.add(name, new Uint8ArrayReader(bytes)); + return Buffer.from(await writer.close()); +}; +const fixture = async (bytes: Buffer) => { + const root = await createTestTempDirectory("rea-saz-capture-"); + const path = join(root, "capture.saz"); + await writeFile(path, bytes); + return path; +}; +const entrypoint = fileURLToPath( + new URL("../../../scripts/rea.mjs", import.meta.url), +); +const cliResponse = async (path: string, extra: string[] = []) => { + try { + const { stdout } = await promisify(execFile)( + process.execPath, + [ + entrypoint, + "inspect-web-network-capture", + path, + "saz", + "--json", + ...extra, + ], + { timeout: 40_000, maxBuffer: 4 * 1024 * 1024 }, + ); + return { ok: true, stdout }; + } catch (cause: unknown) { + if ( + cause instanceof Error && + "stdout" in cause && + typeof cause.stdout === "string" + ) + return { ok: false, stdout: cause.stdout }; + throw cause; + } +}; +const cli = async (path: string, extra: string[] = []) => { + const result = await cliResponse(path, extra); + expect(result.ok, result.stdout).toBe(true); + return parseEvidence(JSON.parse(result.stdout)); +}; + +it("inspects native SAZ through the public CLI and MCP with equal evidence, bytes and archive coordinates", async () => { + const bytes = await archive(); + const path = await fixture(bytes); + const evidence = await cli(path, ["--record", "1", "--record", "0"]); + const capture = webNetworkCaptureSchema.parse(evidence.normalized_result); + expect(capture).toMatchObject({ + format: "saz", + total_records: 2, + runtime_attribution: "unknown", + records: [ + { ordinal: 1, reported: { session_id: "7", response: null } }, + { + ordinal: 0, + reported: { + session_id: "0042", + request: { url: "https://example.test/a" }, + response: { status: "200" }, + }, + }, + ], + }); + expect(capture.records[1]?.location).toEqual({ + kind: "archive-members", + members: [ + "raw/0042_s.txt", + "raw/0042_c.txt", + "raw/0042_m.xml", + "raw/0042_w.txt", + ], + }); + expect(capture.records[1]?.binary_fields).toContainEqual({ + pointer: "/response/body", + representation: "producer-bytes", + state: "retained", + content_base64: compressedBody.toString("base64"), + bytes: compressedBody.length, + sha256: createHash("sha256").update(compressedBody).digest("hex"), + }); + expect(capture.records[0]?.binary_fields).toContainEqual( + expect.objectContaining({ + pointer: "/request/body", + content_base64: "/wCA", + }), + ); + expect(capture.records[1]?.binary_fields).toContainEqual( + expect.objectContaining({ + pointer: "/request/body", + bytes: 0, + content_base64: "", + }), + ); + expect(capture.records[1]?.reported).toMatchObject({ + request: { + headers: [ + { name: "Authorization", value: null }, + { name: "Cookie", value: null }, + { name: "X-Duplicate", value: " one" }, + { name: "X-Duplicate", value: " two" }, + { name: "X-Text" }, + ], + }, + }); + expect(JSON.stringify(capture)).toContain("retained metadata"); + for (const secret of [ + "user:password", + "client-credential", + "server-credential", + "cookie-credential", + ]) + expect(JSON.stringify(capture)).not.toContain(secret); + expect(capture.artifact.sha256).toBe( + createHash("sha256").update(bytes).digest("hex"), + ); + expect(await readFile(path)).toEqual(bytes); + + const { client, call } = await connectLocalToolsMcp(); + const tool = (await client.listTools()).tools.find( + (item) => item.name === "inspect_web_network_capture", + ); + if (tool?.outputSchema === undefined) + throw new Error("Missing capture schema"); + const args = { capture_path: path, format: "saz", record_ordinals: [1, 0] }; + const ajv = new Ajv2020({ strict: false, validateFormats: false }); + expect(ajv.validate(tool.inputSchema, args)).toBe(true); + const response = await call("inspect_web_network_capture", args); + expect(response.isError, JSON.stringify(response)).not.toBe(true); + const fromMcp = parseEvidence(response.structuredContent); + expect(fromMcp.normalized_result).toEqual(evidence.normalized_result); + const bundle = await client.callTool({ + name: "get_evidence_bundle", + arguments: { detail: "complete" }, + }); + const records = z + .object({ result: z.object({ records: z.array(z.unknown()) }) }) + .parse(bundle.structuredContent) + .result.records.map(parseEvidence); + expect( + records.find((record) => record.evidence_id === fromMcp.evidence_id), + ).toEqual(fromMcp); +}, 60_000); + +it("pages retained SAZ Evidence through public MCP and portable CLI after source deletion", async () => { + const path = await fixture(await archive()); + const { call } = await connectLocalToolsMcp(); + const args = { capture_path: path, format: "saz", record_ordinals: [1, 0] }; + const full = await call("inspect_web_network_capture", args); + const fromMcp = parseEvidence(full.structuredContent); + const capture = webNetworkCaptureSchema.parse(fromMcp.normalized_result); + const summaryArgs = { ...args, detail: "summary" }; + const summaryResponse = await call( + "inspect_web_network_capture", + summaryArgs, + ); + expect(summaryResponse.isError, JSON.stringify(summaryResponse)).not.toBe( + true, + ); + const summary = parseEvidence(summaryResponse.structuredContent); + expect(summary.normalized_result).toMatchObject({ + kind: "summary", + parent_evidence_id: fromMcp.evidence_id, + summary: { format: "saz", total_records: 2, retained_records: 2 }, + }); + expect(JSON.stringify(summary)).not.toContain( + compressedBody.toString("base64"), + ); + + // Retained pages must keep their snapshot semantics after the input disappears. + await unlink(path); + const source = { + kind: "retained-evidence", + evidence_id: fromMcp.evidence_id, + }; + const previewArgs = { + source, + view: { kind: "capture", offset: 0, limit: 1 }, + }; + const previewResponse = await call("inspect_analysis_view", previewArgs); + expect(previewResponse.isError, JSON.stringify(previewResponse)).not.toBe( + true, + ); + const preview = parseEvidence(previewResponse.structuredContent); + expect(preview.normalized_result).toMatchObject({ + items: [ + { + ordinal: 1, + reported: { + session_id: "7", + request: { method: "POST", url: "/bytes" }, + }, + }, + ], + coverage: { examined: 1, total: 2, next_offset: 1, exhausted: false }, + }); + expect(JSON.stringify(preview)).not.toContain("content_base64"); + const completeResponse = await call("inspect_analysis_view", { + source, + view: { kind: "capture", detail: "complete", offset: 1, limit: 1 }, + }); + expect(completeResponse.isError, JSON.stringify(completeResponse)).not.toBe( + true, + ); + const complete = parseEvidence(completeResponse.structuredContent); + expect(complete.normalized_result).toMatchObject({ + items: [capture.records[1]], + coverage: { next_offset: null, exhausted: true }, + }); + + const workflow = join(path, "..", "summary-view.json"); + await writeFile( + workflow, + JSON.stringify({ + source: { kind: "inline", evidence: fromMcp }, + view: { kind: "summary" }, + }), + ); + const standalone = await promisify(execFile)(process.execPath, [ + entrypoint, + "inspect-analysis-view", + workflow, + "--json", + ]); + expect( + parseEvidence(JSON.parse(standalone.stdout)).normalized_result, + ).toEqual(summary.normalized_result); +}, 60_000); + +it("keeps CRLF separators inside a body after LF-only HTTP headers", async () => { + const body = Buffer.from("\ufefffirst\r\n\r\nsecond\n\nthird"); + const bytes = await archive([ + [ + "raw/1_c.txt", + Buffer.concat([ + Buffer.from("POST https://example.test/ HTTP/1.1\nX-Test: request\n\n"), + body, + ]), + ], + [ + "raw/1_s.txt", + Buffer.concat([ + Buffer.from("HTTP/1.1 200 OK\nX-Test: response\n\n"), + body, + ]), + ], + ]); + const path = await fixture(bytes); + const capture = webNetworkCaptureSchema.parse( + (await cli(path)).normalized_result, + ); + expect(capture.records[0]?.reported).toMatchObject({ + request: { + headers: [{ name: "X-Test", value: " request" }], + body: body.toString("utf8"), + }, + response: { + headers: [{ name: "X-Test", value: " response" }], + body: body.toString("utf8"), + }, + }); + for (const pointer of ["/request/body", "/response/body"]) + expect(capture.records[0]?.binary_fields).toContainEqual({ + pointer, + representation: "producer-bytes", + state: "retained", + content_base64: body.toString("base64"), + bytes: body.length, + sha256: createHash("sha256").update(body).digest("hex"), + }); + expect(await readFile(path)).toEqual(bytes); +}); + +it("excludes explicit literals from XML, HTTP byte views, member coordinates and binary sidecars", async () => { + const path = await fixture(await archive()); + const result = await cli(path, [ + "--sensitive-value", + "0042", + "--sensitive-value", + "unicode-\u03b2", + "--sensitive-value", + compressedBody.toString("base64"), + "--sensitive-value", + "retained metadata", + ]); + const capture = webNetworkCaptureSchema.parse(result.normalized_result); + expect(capture.records[0]?.location).toEqual({ + kind: "unknown", + reason: "explicit-sensitive-value", + }); + expect(capture.records[0]?.binary_fields).toContainEqual({ + pointer: "/response/body", + representation: "producer-bytes", + state: "redacted", + bytes: null, + sha256: null, + content_base64: null, + }); + const text = JSON.stringify(result); + for (const secret of [ + "0042", + "unicode-", + "retained metadata", + compressedBody.toString("base64"), + createHash("sha256").update(compressedBody).digest("hex"), + ]) + expect(text).not.toContain(secret); +}); + +it("retains folded headers, stored chunk framing and BOM-marked UTF-16 metadata", async () => { + const body = "3\r\none\r\n0\r\n\r\n"; + const path = await fixture( + await archive([ + [ + "raw/1_s.txt", + Buffer.from( + `HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\nX-Folded: first\r\n second\r\n\r\n${body}`, + ), + ], + [ + "raw/1_m.xml", + Buffer.from( + '\ufeff\u03b2]]>', + "utf16le", + ), + ], + [ + "raw/2_c.txt", + Buffer.from( + "CONNECT example.test:443 HTTP/1.1\nHost: example.test:443\n\n", + ), + ], + ]), + ); + const evidence = await cli(path); + const capture = webNetworkCaptureSchema.parse(evidence.normalized_result); + expect(capture.records[0]?.reported).toMatchObject({ + request: null, + response: { + body, + headers: [ + { name: "Transfer-Encoding", value: " chunked" }, + { name: "X-Folded", value: " first\r\n second" }, + ], + }, + }); + expect(capture.records[0]?.binary_fields).toContainEqual( + expect.objectContaining({ + pointer: "/response/body", + content_base64: Buffer.from(body).toString("base64"), + }), + ); + expect(JSON.stringify(capture.records[0]?.reported)).toContain("\u03b2"); + expect(capture.records[1]?.reported).toMatchObject({ + request: { method: "CONNECT", url: "example.test:443", body: "" }, + }); +}); + +it.each([ + "bad-http", + "bad-xml", + "dtd", + "unknown-member", + "encrypted", + "duplicate", + "crc", + "expanded-limit", +])("rejects %s before returning a selected subset", async (variant) => { + let entries = members(); + if (variant === "bad-http") + entries.push(["raw/8_c.txt", Buffer.from("malformed-private-input")]); + if (variant === "bad-xml") + entries.push(["raw/8_m.xml", Buffer.from("")]); + if (variant === "dtd") + entries.push([ + "raw/8_m.xml", + Buffer.from( + ']>&external;', + ), + ]); + if (variant === "unknown-member") + entries.push(["raw/8_unknown.txt", Buffer.from("unknown extension")]); + if (variant === "duplicate") + entries = [ + ["raw/8_c.txt", Buffer.from("GET /a HTTP/1.1\r\n\r\n")], + ["raw/9_c.txt", Buffer.from("GET /b HTTP/1.1\r\n\r\n")], + ]; + let bytes = await archive( + entries, + variant === "encrypted" ? "fixture-password" : undefined, + variant === "crc" ? 0 : 6, + ); + if (variant === "duplicate") + bytes = Buffer.from( + bytes.toString("latin1").replaceAll("raw/9_c.txt", "raw/8_c.txt"), + "latin1", + ); + if (variant === "crc") { + const offset = bytes.indexOf(Buffer.from("HTTP/1.1 200 OK")); + expect(offset).toBeGreaterThan(0); + bytes[offset + 9] = 0x35; + } + if (variant === "expanded-limit") { + const central = bytes.indexOf(Buffer.from([0x50, 0x4b, 0x01, 0x02])); + bytes.writeUInt32LE(33 * 1024 * 1024, central + 24); + } + const path = await fixture(bytes); + const result = await cliResponse(path, ["--record", "0"]); + expect(result.ok, result.stdout).toBe(false); + expect(JSON.parse(result.stdout).category).toBe( + ["encrypted", "unknown-member", "dtd"].includes(variant) + ? "unsupported_provider" + : variant === "expanded-limit" + ? "execution_failure" + : "invalid_input", + ); + if (variant === "expanded-limit") + expect(JSON.parse(result.stdout).code).toBe("unreadable_output"); + expect(JSON.stringify(result)).not.toContain("malformed-private-input"); +}); + +runtimeTest.runIf(!hasMemoryBound)( + "reports SAZ unavailable when its runtime cannot enforce decoder memory", + async () => { + const response = await cliResponse(await fixture(await archive())); + expect(response.ok).toBe(false); + expect(JSON.parse(response.stdout)).toMatchObject({ + code: "capability_unavailable", + }); + }, +); diff --git a/website/locales/ar/messages.po b/website/locales/ar/messages.po index 8a7ea9d40..329f0cd96 100644 --- a/website/locales/ar/messages.po +++ b/website/locales/ar/messages.po @@ -70,7 +70,7 @@ msgstr "بواسطة N0zoM1z0 * 10 أكتوبر 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4660,7 +4660,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "نسخ قراءة البرامج النصية المحملة" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "العودة للأعلى" @@ -11469,7 +11469,7 @@ msgstr "الأدلة" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -11489,7 +11489,7 @@ msgstr "الكود المصدري" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11509,7 +11509,7 @@ msgstr "رخصة MIT" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11529,7 +11529,7 @@ msgstr "انضم إلى المجتمع" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -12461,6 +12461,23 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "التبديل إلى الوضع الداكن" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"لفحص أجسام طلبات HTTP واستجاباتها، استخدم {ph0}Fiddler Everywhere: تسجيلات " +"SAZ وMCP والمهارات العامة{ph1}. صدّر بتنسيق SAZ ?? HAR 1.2 للحصول على أدلة " +"تسجيل في REA دون اتصال، أو اربط خادم MCP في Everywhere لفحص حركة المرور " +"المسجلة مع وكيلك." + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "دليل الإعداد" diff --git a/website/locales/de/messages.po b/website/locales/de/messages.po index 3b4690d00..ee3fec463 100644 --- a/website/locales/de/messages.po +++ b/website/locales/de/messages.po @@ -71,7 +71,7 @@ msgstr "Von N0zoM1z0 · 10. Oktober 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4854,7 +4854,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "Kopieren Lesen Sie die geladenen Skripte" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "Nach oben" @@ -11930,7 +11930,7 @@ msgstr "Anleitungen" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -11950,7 +11950,7 @@ msgstr "Quellcode" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11970,7 +11970,7 @@ msgstr "MIT-Lizenz" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11990,7 +11990,7 @@ msgstr "Zur Community" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -12939,6 +12939,24 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "Zum dunklen Modus wechseln" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"Für HTTP-Anfrage- und Antwortinhalte verwenden Sie {ph0}Fiddler Everywhere: " +"SAZ-Aufzeichnungen, MCP und öffentliche Skills{ph1}. Exportieren Sie SAZ " +"oder HAR 1.2 für die Offline-Auswertung der Aufzeichnung als REA Evidence, " +"oder verbinden Sie den MCP-Server von Everywhere, um aufgezeichneten " +"Datenverkehr mit Ihrem Agenten zu untersuchen." + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Einrichtungsanleitung" diff --git a/website/locales/es/messages.po b/website/locales/es/messages.po index a9d6e8fb6..f40b06204 100644 --- a/website/locales/es/messages.po +++ b/website/locales/es/messages.po @@ -71,7 +71,7 @@ msgstr "Por N0zoM1z0 * 10 octubre 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4827,7 +4827,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "Copiar leer los scripts cargados" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "Volver arriba" @@ -11849,7 +11849,7 @@ msgstr "Guías" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -11869,7 +11869,7 @@ msgstr "Código fuente" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11889,7 +11889,7 @@ msgstr "Licencia MIT" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11909,7 +11909,7 @@ msgstr "Únete a la comunidad" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -12853,6 +12853,24 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "Cambiar al modo oscuro" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"Para examinar los cuerpos de las solicitudes y respuestas HTTP, use " +"{ph0}Fiddler Everywhere: capturas SAZ, MCP y habilidades públicas{ph1}. " +"Exporte SAZ o HAR 1.2 para obtener Evidence de capturas en REA sin conexión," +" o conecte el servidor MCP de Everywhere para inspeccionar el tráfico " +"capturado con su agente." + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Guía de instalación" diff --git a/website/locales/fa/messages.po b/website/locales/fa/messages.po index 65fd093ac..e163a131b 100644 --- a/website/locales/fa/messages.po +++ b/website/locales/fa/messages.po @@ -70,7 +70,7 @@ msgstr "توسط N0zoM1z0 * 10 اکتبر 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4694,7 +4694,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "کپی خواندن اسکریپت های لود شده" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "بازگشت به بالا" @@ -11564,7 +11564,7 @@ msgstr "راهنماها" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -11584,7 +11584,7 @@ msgstr "کد منبع" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11604,7 +11604,7 @@ msgstr "مجوز MIT" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11624,7 +11624,7 @@ msgstr "به جامعه بپیوندید" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -12563,6 +12563,23 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "تغییر به حالت تیره" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"برای بررسی بدنهٔ درخواست‌ها و پاسخ‌های HTTP، از {ph0}Fiddler Everywhere: " +"ضبط‌های SAZ، سرور MCP و مهارت‌های عمومی{ph1} استفاده کنید. برای Evidence ضبط" +" در REA به‌صورت آفلاین، خروجی SAZ ?? HAR 1.2 بگیرید، یا سرور MCP نرم‌افزار " +"Everywhere را متصل کنید تا ترافیک ضبط‌شده را با عامل خود بررسی کنید." + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "راهنمای نصب" diff --git a/website/locales/fr/messages.po b/website/locales/fr/messages.po index 899844e40..f6e6ab357 100644 --- a/website/locales/fr/messages.po +++ b/website/locales/fr/messages.po @@ -71,7 +71,7 @@ msgstr "Par N0zoM1z0 · 10 octobre 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4834,7 +4834,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "Copier lire les scripts chargés" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "Haut de page" @@ -11860,7 +11860,7 @@ msgstr "Guides" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -11880,7 +11880,7 @@ msgstr "Code source" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11900,7 +11900,7 @@ msgstr "Licence MIT" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11920,7 +11920,7 @@ msgstr "Rejoindre la communauté" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -12867,6 +12867,24 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "Passer au mode sombre" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"Pour examiner les corps des requêtes et réponses HTTP, utilisez {ph0}Fiddler" +" Everywhere : captures SAZ, MCP et compétences publiques{ph1}. Exportez en " +"SAZ ou HAR 1.2 pour obtenir des Evidence de capture dans REA hors ligne, ou " +"connectez le serveur MCP d’Everywhere pour inspecter le trafic capturé avec " +"votre agent." + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Guide d’installation" diff --git a/website/locales/id/messages.po b/website/locales/id/messages.po index 39e1ae619..6faa06dad 100644 --- a/website/locales/id/messages.po +++ b/website/locales/id/messages.po @@ -71,7 +71,7 @@ msgstr "Oleh N0zoM1z0 · 10 Oktober 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4785,7 +4785,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "Salin baca skrip yang dimuat" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "Kembali ke atas" @@ -11739,7 +11739,7 @@ msgstr "Panduan" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -11759,7 +11759,7 @@ msgstr "Kode sumber" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11779,7 +11779,7 @@ msgstr "Lisensi MIT" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11799,7 +11799,7 @@ msgstr "Gabung komunitas" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -12740,6 +12740,24 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "Beralih ke mode gelap" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"Untuk memeriksa isi permintaan dan respons HTTP, gunakan {ph0}Fiddler " +"Everywhere: rekaman SAZ, MCP, dan skill publik{ph1}. Ekspor SAZ atau HAR 1.2" +" untuk menghasilkan Evidence rekaman di REA secara luring, atau hubungkan " +"server MCP Everywhere untuk memeriksa lalu lintas yang direkam bersama agen " +"Anda." + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Panduan pemasangan" diff --git a/website/locales/it/messages.po b/website/locales/it/messages.po index 0c1aff1c9..fdebe81d8 100644 --- a/website/locales/it/messages.po +++ b/website/locales/it/messages.po @@ -70,7 +70,7 @@ msgstr "Di N0zoM1z0 * 10 ottobre 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4809,7 +4809,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "Copia leggi gli script caricati" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "Torna in alto" @@ -11797,7 +11797,7 @@ msgstr "Guide" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -11817,7 +11817,7 @@ msgstr "Codice sorgente" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11837,7 +11837,7 @@ msgstr "Licenza MIT" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11857,7 +11857,7 @@ msgstr "Unisciti alla community" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -12801,6 +12801,24 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "Passa alla modalità scura" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"Per esaminare i corpi delle richieste e delle risposte HTTP, usa " +"{ph0}Fiddler Everywhere: acquisizioni SAZ, MCP e skill pubbliche{ph1}. " +"Esporta SAZ o HAR 1.2 per ottenere Evidence delle acquisizioni in REA " +"offline, oppure collega il server MCP di Everywhere per esaminare il " +"traffico acquisito con il tuo agente." + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Guida all’installazione" diff --git a/website/locales/ja/messages.po b/website/locales/ja/messages.po index 8f23848c6..10ed8ee27 100644 --- a/website/locales/ja/messages.po +++ b/website/locales/ja/messages.po @@ -69,7 +69,7 @@ msgstr "によってN0zoM1z0•10 10月2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4440,7 +4440,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "ロードされたスクリプトをコピーして読みます" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "ページ上部へ" @@ -10971,7 +10971,7 @@ msgstr "ガイド" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -10991,7 +10991,7 @@ msgstr "ソースコード" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11011,7 +11011,7 @@ msgstr "MIT ライセンス" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11031,7 +11031,7 @@ msgstr "コミュニティに参加" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -11947,6 +11947,22 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "ダークモードに切り替える" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"HTTP リクエストとレスポンスの本文を調べるには、{ph0}Fiddler Everywhere：SAZ " +"キャプチャ、MCP、公開スキル{ph1}を利用してください。SAZ ??? HAR 1.2 をエクスポートすれば REA でオフラインのキャプチャ " +"Evidence を取得できます。また、Everywhere の MCP サーバーを接続して、エージェントと一緒に記録済みの通信を調べることもできます。" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "セットアップガイド" diff --git a/website/locales/ko/messages.po b/website/locales/ko/messages.po index 62a18cf20..efcf4a708 100644 --- a/website/locales/ko/messages.po +++ b/website/locales/ko/messages.po @@ -68,7 +68,7 @@ msgstr "여N0zoM1z0·10 월 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4422,7 +4422,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "로드 된 스크립트 읽기 복사" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "맨 위로" @@ -10937,7 +10937,7 @@ msgstr "가이드" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -10957,7 +10957,7 @@ msgstr "소스 코드" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -10977,7 +10977,7 @@ msgstr "MIT 라이선스" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -10997,7 +10997,7 @@ msgstr "커뮤니티 참여" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -11911,6 +11911,22 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "다크 모드로 전환" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"HTTP 요청과 응답 본문을 살펴보려면 {ph0}Fiddler Everywhere: SAZ 캡처, MCP 및 공개 스킬{ph1}을 " +"사용하세요. SAZ ?? HAR 1.2로 내보내면 REA에서 오프라인 캡처 Evidence를 얻을 수 있으며, Everywhere의 " +"MCP 서버를 연결하여 에이전트와 함께 캡처된 트래픽을 살펴볼 수도 있습니다." + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "설정 가이드" diff --git a/website/locales/messages.pot b/website/locales/messages.pot index e5f6fea82..ab1a4a17e 100644 --- a/website/locales/messages.pot +++ b/website/locales/messages.pot @@ -131,7 +131,7 @@ msgstr "" #: website/public/examples/dino-lab/index.html:467 website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 website/public/guides/index.html:274 +#: website/public/guides/browser/index.html:499 website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 #: website/public/showcase/aegis/index.html:800 @@ -148,7 +148,7 @@ msgstr "" #: website/public/examples/dino-lab/index.html:468 website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 website/public/guides/index.html:275 +#: website/public/guides/browser/index.html:500 website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 #: website/public/showcase/aegis/index.html:801 @@ -165,7 +165,7 @@ msgstr "" #: website/public/examples/dino-lab/index.html:469 website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 website/public/guides/index.html:276 +#: website/public/guides/browser/index.html:501 website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 #: website/public/showcase/aegis/index.html:802 @@ -182,7 +182,7 @@ msgstr "" #: website/public/examples/dino-lab/index.html:470 website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 website/public/guides/index.html:277 +#: website/public/guides/browser/index.html:502 website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 #: website/public/showcase/aegis/index.html:803 @@ -201,7 +201,7 @@ msgstr "" #: website/public/examples/dino-lab/index.html:475 website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 website/public/guides/index.html:282 +#: website/public/guides/browser/index.html:507 website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 #: website/public/showcase/aegis/index.html:808 @@ -4299,6 +4299,18 @@ msgid "" "workflow{ph1} to follow files and process boundaries." msgstr "" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ captures, MCP " +"and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA capture Evidence, or " +"connect Everywhere's MCP server to inspect captured traffic with your agent." +msgstr "" + #: website/public/guides/browser/index.html:6 msgctxt "page:guides/browser/index.html" msgid "Browser and Network Analysis — REA" @@ -4346,7 +4358,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "" diff --git a/website/locales/pl/messages.po b/website/locales/pl/messages.po index 2c80f47bd..3a6657178 100644 --- a/website/locales/pl/messages.po +++ b/website/locales/pl/messages.po @@ -70,7 +70,7 @@ msgstr "Przez N0zoM1z0 * 10 października 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4754,7 +4754,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "Kopiuj przeczytaj załadowane Skrypty" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "Do góry" @@ -11703,7 +11703,7 @@ msgstr "Poradniki" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -11723,7 +11723,7 @@ msgstr "Kod źródłowy" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11743,7 +11743,7 @@ msgstr "Licencja MIT" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11763,7 +11763,7 @@ msgstr "Dołącz do społeczności" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -12706,6 +12706,23 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "Przełącz na ciemny motyw" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"Aby zbadać treść żądań i odpowiedzi HTTP, użyj {ph0}Fiddler Everywhere: " +"zapisów SAZ, MCP i publicznych umiejętności{ph1}. Wyeksportuj SAZ lub HAR " +"1.2, aby uzyskać Evidence z zapisu w REA offline, lub połącz serwer MCP " +"Everywhere, aby analizować zapisany ruch z agentem." + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Instrukcja instalacji" diff --git a/website/locales/pt-BR/messages.po b/website/locales/pt-BR/messages.po index 0e1b8a3a3..a0741271d 100644 --- a/website/locales/pt-BR/messages.po +++ b/website/locales/pt-BR/messages.po @@ -70,7 +70,7 @@ msgstr "Por N0zoM1z0 * 10 de outubro de 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4793,7 +4793,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "Copiar ler os scripts carregados" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "Voltar ao topo" @@ -11780,7 +11780,7 @@ msgstr "Guias" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -11800,7 +11800,7 @@ msgstr "Código-fonte" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11820,7 +11820,7 @@ msgstr "Licença MIT" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11840,7 +11840,7 @@ msgstr "Participe da comunidade" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -12780,6 +12780,24 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "Alternar para o modo escuro" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"Para examinar os corpos das requisições e respostas HTTP, use {ph0}Fiddler " +"Everywhere: capturas SAZ, MCP e habilidades públicas{ph1}. Exporte SAZ ou " +"HAR 1.2 para obter Evidence das capturas no REA offline, ou conecte o " +"servidor MCP do Everywhere para inspecionar o tráfego capturado com seu " +"agente." + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Guia de instalação" diff --git a/website/locales/ru/messages.po b/website/locales/ru/messages.po index 4349d04ef..9f0486916 100644 --- a/website/locales/ru/messages.po +++ b/website/locales/ru/messages.po @@ -70,7 +70,7 @@ msgstr "Автор: N0zoM1z0 · 10 октября 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4804,7 +4804,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "Копировать читать загруженные скрипты" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "Наверх" @@ -11817,7 +11817,7 @@ msgstr "Руководства" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -11837,7 +11837,7 @@ msgstr "Исходный код" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11857,7 +11857,7 @@ msgstr "Лицензия MIT" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11877,7 +11877,7 @@ msgstr "Вступить в сообщество" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -12822,6 +12822,24 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "Переключить на тёмную тему" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"Для изучения тел запросов и ответов HTTP используйте {ph0}Fiddler " +"Everywhere: захваты SAZ, MCP и общедоступные навыки{ph1}. Экспортируйте SAZ " +"??? HAR 1.2 для получения Evidence захвата в REA без подключения к сети или " +"подключите MCP-сервер Everywhere, чтобы исследовать записанный трафик вместе" +" с агентом." + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Руководство по установке" diff --git a/website/locales/th/messages.po b/website/locales/th/messages.po index 2fab386ea..94e6d1873 100644 --- a/website/locales/th/messages.po +++ b/website/locales/th/messages.po @@ -70,7 +70,7 @@ msgstr "โดย N0zoM1z0 ·10 ตุลาคม 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4674,7 +4674,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "คัดลอกอ่านสคริปต์ที่โหลด" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "กลับขึ้นด้านบน" @@ -11461,7 +11461,7 @@ msgstr "คู่มือ" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -11481,7 +11481,7 @@ msgstr "ซอร์สโค้ด" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11501,7 +11501,7 @@ msgstr "สัญญาอนุญาต MIT" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11521,7 +11521,7 @@ msgstr "เข้าร่วมชุมชน" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -12459,6 +12459,24 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "เปลี่ยนเป็นโหมดมืด" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"หากต้องการตรวจสอบเนื้อหาคำขอและคำตอบ HTTP ให้ใช้ {ph0}Fiddler Everywhere: " +"ข้อมูลที่บันทึกในรูปแบบ SAZ, MCP และทักษะสาธารณะ{ph1} ส่งออกเป็น SAZ ???? " +"HAR 1.2 เพื่อรับ Evidence จากข้อมูลที่บันทึกใน REA แบบออฟไลน์ " +"หรือเชื่อมต่อเซิร์ฟเวอร์ MCP ของ Everywhere " +"เพื่อตรวจสอบทราฟฟิกที่บันทึกไว้ร่วมกับเอเจนต์ของคุณ" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "คู่มือการติดตั้ง" diff --git a/website/locales/tr/messages.po b/website/locales/tr/messages.po index 7de2f7e5a..e4d0ba4a0 100644 --- a/website/locales/tr/messages.po +++ b/website/locales/tr/messages.po @@ -71,7 +71,7 @@ msgstr "Başlangıç N0zoM1z0 * 10 Ekim 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4754,7 +4754,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "Kopyala yüklenen komut dosyalarını oku" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "Başa dön" @@ -11690,7 +11690,7 @@ msgstr "Rehberler" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -11710,7 +11710,7 @@ msgstr "Kaynak kod" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11730,7 +11730,7 @@ msgstr "MIT lisansı" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11750,7 +11750,7 @@ msgstr "Topluluğa katıl" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -12688,6 +12688,24 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "Koyu temaya geç" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"HTTP istek ve yanıt gövdelerini incelemek için {ph0}Fiddler Everywhere: SAZ " +"kayıtları, MCP ve herkese açık beceriler{ph1} rehberini kullanın. REA ile " +"çevrimdışı kayıt Evidence elde etmek için SAZ veya HAR 1.2 dışa aktarın veya" +" kaydedilen trafiği ajanınızla incelemek için Everywhere MCP sunucusunu " +"bağlayın." + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Kurulum rehberi" diff --git a/website/locales/uk/messages.po b/website/locales/uk/messages.po index ddd250d24..a5fae5c80 100644 --- a/website/locales/uk/messages.po +++ b/website/locales/uk/messages.po @@ -70,7 +70,7 @@ msgstr "Автор: N0zoM1z0 · 10 жовтня 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4780,7 +4780,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "Копіювати читати завантажені скрипти" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "Нагору" @@ -11769,7 +11769,7 @@ msgstr "Посібники" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -11789,7 +11789,7 @@ msgstr "Вихідний код" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11809,7 +11809,7 @@ msgstr "Ліцензія MIT" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11829,7 +11829,7 @@ msgstr "Приєднатися до спільноти" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -12773,6 +12773,24 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "Перемкнути на темну тему" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"Для дослідження тіл запитів і відповідей HTTP використовуйте {ph0}Fiddler " +"Everywhere: захоплення SAZ, MCP і загальнодоступні навички{ph1}. Експортуйте" +" SAZ ??? HAR 1.2 для отримання Evidence захоплення в REA без підключення до " +"мережі або підключіть MCP-сервер Everywhere, щоб досліджувати записаний " +"трафік разом з агентом." + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Посібник зі встановлення" diff --git a/website/locales/vi/messages.po b/website/locales/vi/messages.po index 3aa144d2f..d9dd57fb0 100644 --- a/website/locales/vi/messages.po +++ b/website/locales/vi/messages.po @@ -70,7 +70,7 @@ msgstr "Bởi N0zoM1z0 * 10 tháng mười 2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4764,7 +4764,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "Sao chép đọc các tập lệnh đã tải" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "Về đầu trang" @@ -11679,7 +11679,7 @@ msgstr "Hướng dẫn" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -11699,7 +11699,7 @@ msgstr "Mã nguồn" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -11719,7 +11719,7 @@ msgstr "Giấy phép MIT" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -11739,7 +11739,7 @@ msgstr "Tham gia cộng đồng" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -12675,6 +12675,23 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "Chuyển sang giao diện tối" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"Để kiểm tra phần thân yêu cầu và phản hồi HTTP, hãy dùng {ph0}Fiddler " +"Everywhere: bản ghi SAZ, MCP và kỹ năng công khai{ph1}. Xuất SAZ ho?c HAR " +"1.2 để lấy Evidence từ bản ghi trong REA ngoại tuyến, hoặc kết nối máy chủ " +"MCP của Everywhere để kiểm tra lưu lượng đã ghi cùng trợ lý của bạn." + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Hướng dẫn cài đặt" diff --git a/website/locales/zh-CN/messages.po b/website/locales/zh-CN/messages.po index b9350e625..91728965c 100644 --- a/website/locales/zh-CN/messages.po +++ b/website/locales/zh-CN/messages.po @@ -68,7 +68,7 @@ msgstr "通过N0zoM1z0*10十月2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4237,7 +4237,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "复制读取加载的脚本" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "回到顶部" @@ -10700,7 +10700,7 @@ msgstr "指南" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -10720,7 +10720,7 @@ msgstr "源代码" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -10740,7 +10740,7 @@ msgstr "MIT 许可证" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -10760,7 +10760,7 @@ msgstr "加入社区" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -11666,6 +11666,21 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "切换到深色模式" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"要检查 HTTP 请求和响应正文，请使用{ph0}Fiddler Everywhere：SAZ 捕获、MCP 和公开技能{ph1}。导出 SAZ ? " +"HAR 1.2 可在 REA 中离线获取捕获 Evidence，也可以连接 Everywhere 的 MCP 服务器，让智能体检查已捕获的流量。" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "安装指南" diff --git a/website/locales/zh-TW/messages.po b/website/locales/zh-TW/messages.po index 98984f130..875779f14 100644 --- a/website/locales/zh-TW/messages.po +++ b/website/locales/zh-TW/messages.po @@ -68,7 +68,7 @@ msgstr "透過N0zoM1z0*10十月2026" #: website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 #: website/public/get-started/index.html:882 -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 #: website/public/guides/index.html:282 #: website/public/guides/javascript/index.html:503 #: website/public/guides/native/index.html:509 website/public/index.html:1191 @@ -4252,7 +4252,7 @@ msgctxt "page:guides/browser/index.html" msgid "Copy read the loaded scripts" msgstr "複製讀取載入的指令碼" -#: website/public/guides/browser/index.html:498 +#: website/public/guides/browser/index.html:507 msgctxt "page:guides/browser/index.html" msgid "Back to top" msgstr "回到頂部" @@ -10719,7 +10719,7 @@ msgstr "指南" #: website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 #: website/public/get-started/index.html:874 -#: website/public/guides/browser/index.html:490 +#: website/public/guides/browser/index.html:499 #: website/public/guides/index.html:274 #: website/public/guides/javascript/index.html:495 #: website/public/guides/native/index.html:501 website/public/index.html:1183 @@ -10739,7 +10739,7 @@ msgstr "原始碼" #: website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 #: website/public/get-started/index.html:875 -#: website/public/guides/browser/index.html:491 +#: website/public/guides/browser/index.html:500 #: website/public/guides/index.html:275 #: website/public/guides/javascript/index.html:496 #: website/public/guides/native/index.html:502 website/public/index.html:1184 @@ -10759,7 +10759,7 @@ msgstr "MIT 許可證" #: website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 #: website/public/get-started/index.html:876 -#: website/public/guides/browser/index.html:492 +#: website/public/guides/browser/index.html:501 #: website/public/guides/index.html:276 #: website/public/guides/javascript/index.html:497 #: website/public/guides/native/index.html:503 website/public/index.html:1185 @@ -10779,7 +10779,7 @@ msgstr "加入社群" #: website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 #: website/public/get-started/index.html:877 -#: website/public/guides/browser/index.html:493 +#: website/public/guides/browser/index.html:502 #: website/public/guides/index.html:277 #: website/public/guides/javascript/index.html:498 #: website/public/guides/native/index.html:504 website/public/index.html:1186 @@ -11686,6 +11686,21 @@ msgctxt "runtime" msgid "Switch to dark mode" msgstr "切換至深色模式" +#. {ph0}: +#. {ph1}: +#: website/public/guides/browser/index.html:480 +#, python-brace-format +msgctxt "page:guides/browser/index.html" +msgid "" +"For HTTP request and response bodies, use {ph0}Fiddler Everywhere: SAZ " +"captures, MCP and public skills{ph1}. Export SAZ or HAR 1.2 for offline REA " +"capture Evidence, or connect Everywhere's MCP server to inspect captured " +"traffic with your agent." +msgstr "" +"若要檢查 HTTP 請求與回應本文，請使用{ph0}Fiddler Everywhere：SAZ 擷取、MCP 與公開技能{ph1}。匯出 SAZ ? " +"HAR 1.2 可在 REA 中離線取得擷取 Evidence，也可以連接 Everywhere 的 MCP 伺服器，讓代理檢查已擷取的流量。" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "安裝指南" diff --git a/website/public/guides/browser/index.html b/website/public/guides/browser/index.html index 6437873c2..60185ddb6 100644 --- a/website/public/guides/browser/index.html +++ b/website/public/guides/browser/index.html @@ -477,6 +477,15 @@ link.click(); application workflow to follow files and process boundaries.

+

+ For HTTP request and response bodies, use + Fiddler Everywhere: SAZ captures, MCP and public skills. Export SAZ or HAR 1.2 for offline REA capture Evidence, or + connect Everywhere's MCP server to inspect captured traffic with + your agent. +

From bb154d1067b0fc250452b7d468d0b0523031a16f Mon Sep 17 00:00:00 2001 From: Luis <86675342+LuisTaffelli@users.noreply.github.com> Date: Tue, 13 Oct 2026 00:24:05 -0300 Subject: [PATCH 003/159] feat(browser): inspect retained aioquic qlog JSON captures Inspect bounded offline aioquic1.5.0 JSON qlog0.3 and page retained event/transport facts with explicit trace and stream selectors. Preserve producer coordinates, exact numeric lexemes, global pre-selection fact scope and endpoint declaration uncertainty. Verified four real HTTP/3 recordings with an independent interval oracle, public CLI/MCP/schema parity, auth exclusions, source-deletion views and actual decoder heap exhaustion. Unverified producer platforms and qlog-specific lifecycle induction remain documented. Co-authored-by: LuisTaffelli <86675342+LuisTaffelli@users.noreply.github.com> Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com> --- .../skills/reverse-engineer-anything/SKILL.md | 9 +- docs/web-network-captures.md | 114 +++- package.json | 3 +- scripts/verify-qlog-captures.mjs | 215 ++++++++ src/application/WebNetworkCaptureService.ts | 10 + src/browser/history/CaptureFailures.ts | 2 +- .../history/{HarJson.ts => CaptureJson.ts} | 27 +- src/browser/history/CaptureRelease.ts | 8 + src/browser/history/HarCapture.test.ts | 20 +- src/browser/history/HarCapture.ts | 18 +- .../history/HistoricalCaptureDecoder.ts | 30 +- .../history/NodeCaptureAdapter.test.ts | 13 +- src/browser/history/NodeCaptureAdapter.ts | 11 +- src/browser/history/NodeCaptureProcess.ts | 15 +- src/browser/history/QlogCapture.test.ts | 259 +++++++++ src/browser/history/QlogCapture.ts | 503 ++++++++++++++++++ src/cli/webNetworkCaptureCommands.ts | 2 +- src/composition/webNetworkCaptures.ts | 1 + src/contracts/analysisViewToolContracts.ts | 2 +- .../webNetworkCaptureToolContracts.ts | 10 +- src/domain/analysisView/analysisView.ts | 15 + src/domain/analysisView/qlogCaptureView.ts | 248 +++++++++ .../analysisView/webNetworkCaptureView.ts | 90 +++- src/domain/qlogCapture.ts | 76 +++ src/domain/webNetworkCapture.ts | 140 +++-- src/domain/webNetworkCaptureCoordinates.ts | 22 + src/server/registerAnalysisViewTool.ts | 11 +- src/server/registerWebNetworkCaptureTool.ts | 14 +- src/server/toolRegistrationOptions.ts | 12 + tests/acceptance/tools/qlogCapture.test.ts | 457 ++++++++++++++++ 30 files changed, 2206 insertions(+), 151 deletions(-) create mode 100644 scripts/verify-qlog-captures.mjs rename src/browser/history/{HarJson.ts => CaptureJson.ts} (83%) create mode 100644 src/browser/history/QlogCapture.test.ts create mode 100644 src/browser/history/QlogCapture.ts create mode 100644 src/domain/analysisView/qlogCaptureView.ts create mode 100644 src/domain/qlogCapture.ts create mode 100644 tests/acceptance/tools/qlogCapture.test.ts diff --git a/.agents/skills/reverse-engineer-anything/SKILL.md b/.agents/skills/reverse-engineer-anything/SKILL.md index 35b99d8dc..3b508af1c 100644 --- a/.agents/skills/reverse-engineer-anything/SKILL.md +++ b/.agents/skills/reverse-engineer-anything/SKILL.md @@ -65,11 +65,16 @@ explicit path or endpoint and do not need it. [Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging). Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`; connect Everywhere's separate MCP server for live traffic. -- Retained HAR/SAZ/native mitmproxy capture: `inspect_web_network_capture`. Select +- Retained HAR/SAZ/native mitmproxy/aioquic qlog capture: `inspect_web_network_capture`. Select original record ordinals when useful; inspect producer fields and byte/number sidecars without fetching recorded URLs or claiming live attribution. See [historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream - profile and credential exclusions. + profile and credential exclusions. Qlog support starts with aioquic 1.5.0 + JSON 0.3; stream coverage and endpoint loss declarations do not recover bodies + or establish network loss. Prefer detail summary, then kind qlog trace/stream + pages; select one trace, stream ID and direction for contribution/range pages. + Facts retain all-recorded-events-before-selection scope. Other qlog layouts + and PCAP correlation are unsupported. - User-owned Electron runtime already open: `list_electron_targets`. - Recorded packet captures: `inspect_packet_capture` with selected TShark fields, display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see diff --git a/docs/web-network-captures.md b/docs/web-network-captures.md index e65887a71..c447c4783 100644 --- a/docs/web-network-captures.md +++ b/docs/web-network-captures.md @@ -255,7 +255,7 @@ response fields, including HAR's `status: 0` and an empty `mimeType`, remain unknown, not invented statuses or MIME types. The view makes no handler-identity, schema or causality claim. It performs no -network activity. Native mitmproxy and SAZ captures return a typed unsupported error. +network activity. Native mitmproxy, SAZ and qlog captures return a typed unsupported error. Parents inspected with `sensitive_values`, or whose Evidence does not record `sensitive_value_count`, return a typed unsupported error whose reason is the canonical token `sensitive_values_unretained`. The view would write path @@ -291,14 +291,15 @@ Complete evidence has a 32 MiB input, 96 MiB decoder reply and 64-level nesting budget. Exceeding a budget returns no partial success. Oversized selected captures return an input error identifying `capture_path` and the byte budget; owned reply/output failures remain output or provider errors. -HAR and SAZ run in an owned +HAR, SAZ and qlog run in an owned 192 MiB Node old-generation heap with one V8 worker. Mitmproxy decoding applies Linux limits of 768 MiB address space, 30 seconds CPU and 96 MiB file output; both commands have a 30-second wall deadline and independently supervised process cleanup. Owned decoder diagnostics retain at most 1 MiB across stdout and stderr; the collector drops overflow as it arrives, reports the limit failure and verifies cleanup. Heap limits alone do not establish aggregate RSS limits. -When REA runs on Bun, the HAR/SAZ decoder runs on Bun, which ignores these V8 heap +SAZ and qlog refuse decoding when neither a Node heap limit nor a Windows Job Object bound can be enforced. Existing HAR availability is preserved. +When REA runs on Bun, the HAR/SAZ/qlog decoder runs on Bun, which ignores these V8 heap flags; its limitations and resource failures then omit the heap sizes. On Windows, the decoder's owned Job Object instead limits its committed memory to 704 MiB, the 192 MiB budget plus a 512 MiB reserve for Bun itself. The @@ -306,7 +307,7 @@ limitations state that limit, and reaching it returns `resource_constraint` with `process_memory_limit_bytes`, whatever exit status Bun reports. Observed decoder memory exhaustion returns `resource_constraint`, with the -selected capture, applicable limits and retained process diagnostics. HAR/SAZ's +selected capture, applicable limits and retained process diagnostics. HAR/SAZ/qlog's old-generation heap is fixed; inherited `NODE_OPTIONS` cannot raise it. Record selection happens after complete decoding and does not reduce that workload. Use a smaller capture exported by its producer or another decoder @@ -333,3 +334,108 @@ Command cleanup failures retain the original status and both diagnostic streams; the decoder's ownership preparation receives the same cancellation signal. MCP advertises the same strict named input contract used by the application. Historical inspection delegates argument validation to that shared application boundary so accepted explicit-sensitive declarations also cover invalid argument names and other correction details. The SDK receives raw arguments through its Standard Schema interface; malformed inputs still fail before decoder or filesystem effects and return the normal structured REA error. Protocol-envelope validation remains owned by MCP. + +## Retained aioquic qlog + +Select `format: "qlog"` through `inspect_web_network_capture`, or: + +```bash +rea inspect-web-network-capture /absolute/origin.qlog qlog --json +rea inspect-web-network-capture /absolute/origin.qlog qlog --record 95 --record 0 --json +``` + +The initial profile is **aioquic 1.5.0, JSON qlog 0.3**: one JSON object with +`qlog_format: "JSON"`, `qlog_version: "0.3"`, `traces` containing object events +with `name`, finite numeric `time` and object `data`. JSON-SEQ, JSONL, columnar +array events and other versions are unsupported. Duplicate JSON members, +malformed UTF-8, invalid interpreted QUIC integers and excessive nesting fail +without partial success, including in events outside the selected ordinals. +Well-formed event subsets and missing frame metadata remain inspectable. +Unknown extensions are retained; this is not validation of every qlog schema. + +Choosing the format selects the interpretation profile; the file does not +necessarily report or authenticate an aioquic version. The profile's clock +uses Unix-epoch milliseconds, verified in aioquic 1.5.0's `QuicLoggerTrace` +`log_event`/`encode_time`. The `qlog.clock.basis` and Evidence analysis profile +state that assumption explicitly. Numeric lexemes survive independently of +safe JSON-number projections; decimal spelling does not establish physical +clock precision or monotonicity. + +Every event has a global ordinal from trace/event source traversal and its +original `/traces/T/events/E` pointer. This order is not a global time sort. +Trace index and the selected artifact identify a connection observation; +reported ODCID is preserved without claiming global uniqueness. Streams are +scoped by trace, stream ID and sent/received direction. Control/QPACK streams +are retained, so counting streams does not count HTTP requests. + +`qlog` is a typed facet required exactly for this format. Its scope is +`all-recorded-events-before-selection`: selection changes returned event +records while facts cite the original ordinals across the entire recorded +file. The container retains trace metadata with each events array replaced +by null; there is no single events pointer shared by all traces. + +| Fact | Basis and limits | +| ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Original event and loss declaration | Observed endpoint report; `recovery:packet_lost` does not prove actual network loss. Missing packet number/type remains unknown. | +| STREAM ranges and byte totals | Derived per trace and direction from reported offsets and lengths using exact integer arithmetic. `unique_bytes` unions intervals; `frame_length_sum` includes overlaps; `repeated_bytes` is their difference. Overlap alone does not prove retransmission or equal contents. | +| FIN final sizes | Reported offset plus length where FIN is true; multiple conflicting sizes remain visible. A final size does not establish complete coverage or a received body. | +| Incomplete fact records | Ordinals with omitted/redacted interpretable frame metadata; usable fields in other records survive. An empty list does not establish complete logging or capture. | +| HTTP bodies and causality | `body_bytes: "not-reported"`; lengths and headers are metadata, with no reconstructed body, hash, timing causality or live process attribution. | + +QUIC integer fields interpreted here accept exact nonnegative JSON integer +values through `2^62 - 1`, including equivalent exponent notation. Offsets plus +lengths must fit that range. Derived sums are decimal strings and are not +rounded to JavaScript numbers. A missing offset is unknown, never defaulted to +zero. Ranges can start above zero or contain gaps. Packet headers, HTTP events, +raw lengths and unknown extensions remain producer evidence rather than being +normalized into invented HAR requests. + +The adapter excludes authorization/proxy authorization/cookie/set-cookie +values in known HTTP frame events, reset tokens in transport parameters and +NEW_CONNECTION_ID frames, and NEW_TOKEN frame tokens. Known URL header userinfo +and explicit sensitive values follow the shared exclusions. Ordinary extension +fields and token-looking query text remain retained. Explicit declarations also +cover normalized numeric projections, derived totals and trace pointers; facts +that cannot be safely shown are omitted with incomplete-record coordinates. +No keylog, PCAP, live URL or recorded target is opened by the qlog decoder. + +To repeat the real-provider lane with **local caller-supplied evidence** from +the educational aioquic 1.5.0 experiment (one origin trace, known 32,803-byte +stream coverage and one 1,169-byte repeated range in the loss case): + +```bash +npm run verify:web:qlog -- /absolute/control/origin.qlog /absolute/loss/origin.qlog /absolute/new-report.json +``` + +This lane checks built CLI/stdio MCP parity, SDK-converted JSON Schemas, +artifact digests, original event count, selection scope, retained parent lookup, +credential exclusions and the known control/loss facts. The report is created +exclusively and contains no keys or captured credential values. It is a +fixture-specific verification, not a general loss detector. CI uses synthetic +format inputs for malformed cases and limits; those tests do not establish +producer provenance. PCAP parsing and correlation remain outside this stage. + +### Focused qlog views + +Use MCP `inspect_web_network_capture` with `format: "qlog", detail: "summary"` +for identity, profile, scope and retained-fact counts. Then pass its +`normalized_result.parent_evidence_id` to `inspect_analysis_view` as a +`retained-evidence` source. + +```json +{"kind":"qlog","facet":"traces","offset":0,"limit":10} +{"kind":"qlog","facet":"streams","trace_index":0,"offset":0,"limit":10} +{"kind":"qlog","facet":"contributions","trace_index":0,"stream_id":"0","direction":"sent","offset":0,"limit":20} +``` + +Trace pages omit nested lists and give their counts. Stream pages preserve +exact byte totals but omit contributions, ranges and final-size lists. For +those lists select `contributions`, `ranges` or `final-sizes` with all three +stream selectors. `loss-declarations` and `incomplete-records` need only +`trace_index`. Offsets count rows in that retained facet; defaults are 0 and 64. Missing or ambiguous selectors fail rather than choosing a connection. + +These views page already interpreted facts without decoding again or opening +the source, and work after its deletion. Facts still cover all recorded events +before parent record selection. Their exhaustion does not prove logging or +network completeness. Event preview/complete pages remain available through +kind `capture`, with original numeric lexemes and producer ordinals preserved. diff --git a/package.json b/package.json index f803cc00c..801f72ebf 100644 --- a/package.json +++ b/package.json @@ -278,7 +278,8 @@ "artifacts:error-schema": "node scripts/generate-error-schema.mjs", "test:prepare": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts turbo run artifacts:mcp-catalog artifacts:product-catalog artifacts:managed-evidence", "evidence:check:built": "node scripts/generate-completion-ledger.mjs --check", - "docs:prepare": "node scripts/generate-package-metadata.mjs && turbo run artifacts:product-catalog artifacts:managed-evidence artifacts:error-schema" + "docs:prepare": "node scripts/generate-package-metadata.mjs && turbo run artifacts:product-catalog artifacts:managed-evidence artifacts:error-schema", + "verify:web:qlog": "npm run build:cached && node scripts/verify-qlog-captures.mjs" }, "lint-staged": { "*.{cjs,cts,js,jsx,mjs,mts,ts,tsx}": [ diff --git a/scripts/verify-qlog-captures.mjs b/scripts/verify-qlog-captures.mjs new file mode 100644 index 000000000..8d1a16f6c --- /dev/null +++ b/scripts/verify-qlog-captures.mjs @@ -0,0 +1,215 @@ +#!/usr/bin/env node +import assert from "node:assert/strict"; +import { execFile } from "node:child_process"; +import { createHash } from "node:crypto"; +import { readFile, writeFile } from "node:fs/promises"; +import { isAbsolute } from "node:path"; +import { fileURLToPath } from "node:url"; +import { promisify } from "node:util"; +import { Client } from "@modelcontextprotocol/client"; +import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; +import { Ajv2020 } from "ajv/dist/2020.js"; +import { parseEvidence } from "../dist/domain/evidence.js"; + +// Caller-supplied producer evidence stays local; no fixture capture or secrets are bundled. +const [controlPath, lossPath, reportPath] = process.argv.slice(2); +if ( + ![controlPath, lossPath, reportPath].every( + (value) => typeof value === "string" && isAbsolute(value), + ) +) + throw new Error( + "Usage: node scripts/verify-qlog-captures.mjs /absolute/control/origin.qlog /absolute/loss/origin.qlog /absolute/new-report.json", + ); +const entrypoint = fileURLToPath(new URL("./rea.mjs", import.meta.url)); +const client = new Client({ name: "qlog-producer-verifier", version: "1" }); +const environment = { + PATH: process.env.PATH ?? "/usr/bin:/bin", + REA_LOG_LEVEL: "silent", + HOPPER_LAUNCHER_PATH: "/rea-unconfigured-deep-provider/hopper", +}; +const transport = new StdioClientTransport({ + command: process.execPath, + args: [entrypoint, "mcp"], + env: environment, + stderr: "pipe", +}); +let diagnostics = ""; +transport.stderr?.on("data", (bytes) => { + if (diagnostics.length < 65536) diagnostics += bytes.toString("utf8"); +}); +const ajv = new Ajv2020({ strict: false, validateFormats: false }); +const measurements = []; +try { + try { + await client.connect(transport); + } catch (cause) { + throw new Error(`MCP startup failed: ${diagnostics}`, { cause }); + } + const tool = (await client.listTools()).tools.find( + (tool) => tool.name === "inspect_web_network_capture", + ); + assert.ok(tool?.outputSchema); + for (const schema of [tool.inputSchema, tool.outputSchema]) + assert.ok(ajv.validateSchema(schema)); + const validateOutput = ajv.compile(tool.outputSchema); + for (const [lane, path] of [ + ["control", controlPath], + ["loss", lossPath], + ]) { + const source = await readFile(path); + const raw = JSON.parse(source.toString("utf8")); + const sha256 = createHash("sha256").update(source).digest("hex"); + assert.equal(raw.qlog_format, "JSON"); + assert.equal(raw.qlog_version, "0.3"); + const input = { capture_path: path, format: "qlog" }; + assert.ok(ajv.validate(tool.inputSchema, input)); + const response = await client.callTool({ + name: tool.name, + arguments: input, + }); + assert.notEqual(response.isError, true); + assert.ok( + validateOutput(response.structuredContent), + JSON.stringify(validateOutput.errors), + ); + const mcp = parseEvidence(response.structuredContent); + const completed = await promisify(execFile)( + process.execPath, + [entrypoint, "inspect-web-network-capture", path, "qlog", "--json"], + { env: environment, timeout: 40_000, maxBuffer: 96 * 1024 * 1024 }, + ); + const cli = parseEvidence(JSON.parse(completed.stdout)); + assert.equal(cli.evidence_id, mcp.evidence_id); + assert.deepEqual(cli.normalized_result, mcp.normalized_result); + const capture = mcp.normalized_result; + assert.equal(capture.artifact.sha256, sha256); + assert.equal( + capture.total_records, + raw.traces.reduce((sum, trace) => sum + trace.events.length, 0), + ); + for (const [ordinal, original] of raw.traces + .flatMap((trace) => trace.events) + .entries()) { + const record = capture.records[ordinal]; + assert.equal(record.ordinal, ordinal); + assert.equal(record.reported.name, original.name); + assert.equal(record.reported.time, original.time); + assert.deepEqual(record.binary_fields, []); + assert.ok( + record.numeric_literals.some( + (number) => + number.pointer === "/time" && + Number(number.literal) === original.time, + ), + ); + } + assert.equal(capture.qlog.body_bytes, "not-reported"); + assert.equal(mcp.authority, "historical-reference"); + assert.equal( + mcp.analysis_profile.parameters.qlog_profile, + "aioquic-1.5.0-json-0.3", + ); + const trace = capture.qlog.traces[0]; + assert.equal(trace.vantage_point, "server"); + const stream = trace.streams.find( + (stream) => stream.stream_id === "0" && stream.direction === "sent", + ); + assert.ok(stream); + assert.equal(stream.unique_bytes, "32803"); + assert.equal( + stream.frame_length_sum, + lane === "control" ? "32803" : "33972", + ); + assert.equal(stream.repeated_bytes, lane === "control" ? "0" : "1169"); + assert.equal(trace.loss_declarations.length, lane === "control" ? 0 : 1); + if (lane === "loss") + assert.deepEqual(trace.loss_declarations[0].packet_number, { + state: "reported", + value: "5", + }); + // Structurally-known secrets may not survive in Evidence, including sidecars. + const serialized = JSON.stringify(mcp); + for (const original of raw.traces.flatMap((trace) => trace.events)) { + if ( + original.name === "transport:parameters_set" && + typeof original.data.stateless_reset_token === "string" + ) + assert.ok(!serialized.includes(original.data.stateless_reset_token)); + for (const frame of original.data.frames ?? []) { + if ( + frame.frame_type === "new_connection_id" && + typeof frame.reset_token === "string" + ) + assert.ok(!serialized.includes(frame.reset_token)); + if (frame.frame_type === "new_token" && typeof frame.token === "string") + assert.ok(!serialized.includes(frame.token)); + } + } + const retained = await client.callTool({ + name: "inspect_analysis_view", + arguments: { + source: { kind: "retained-evidence", evidence_id: mcp.evidence_id }, + view: { kind: "api-surface" }, + }, + }); + assert.equal(retained.isError, true); + assert.equal( + JSON.parse(retained.content.find((item) => item.type === "text").text) + .error.code, + "unsupported_target", + ); + const selected = await client.callTool({ + name: tool.name, + arguments: { ...input, record_ordinals: [capture.total_records - 1, 0] }, + }); + const subset = parseEvidence(selected.structuredContent).normalized_result; + assert.deepEqual( + subset.records.map((record) => record.ordinal), + [capture.total_records - 1, 0], + ); + assert.deepEqual(subset.qlog, capture.qlog); + measurements.push({ + lane, + path, + sha256, + evidence_id: mcp.evidence_id, + profile_digest: mcp.analysis_profile.digest, + events: capture.total_records, + stream_id: stream.stream_id, + unique_bytes: stream.unique_bytes, + frame_length_sum: stream.frame_length_sum, + repeated_bytes: stream.repeated_bytes, + loss_declarations: trace.loss_declarations.length, + cli_mcp_parity: true, + retained_reference: true, + selected_scope_preserved: true, + auth_exclusions: true, + }); + } + await client.ping(); + const report = { + status: "passed", + profile: "aioquic-1.5.0-json-0.3", + measurements, + limitations: [ + "Caller supplies provenance; format selection does not authenticate the producer.", + "These known loopback cases do not establish WAN behavior, other qlog producers, platforms or general completeness.", + "No PCAP or keylog is read by this verifier; repeated STREAM coverage is not proof of retransmission or network loss.", + ], + }; + await writeFile(reportPath, JSON.stringify(report, null, 2) + "\n", { + flag: "wx", + mode: 0o600, + }); + console.log( + JSON.stringify({ + status: report.status, + cases: measurements.length, + report: reportPath, + }), + ); +} finally { + await client.close(); + await transport.close(); +} diff --git a/src/application/WebNetworkCaptureService.ts b/src/application/WebNetworkCaptureService.ts index a571fa8c1..8e29ebb10 100644 --- a/src/application/WebNetworkCaptureService.ts +++ b/src/application/WebNetworkCaptureService.ts @@ -10,6 +10,7 @@ import { AnalysisInputError, AnalysisOutputError, } from "../domain/analysisErrorCore.js"; +import { createAnalysisProfile } from "../domain/analysisProfile.js"; import { createEvidence, type Evidence } from "../domain/evidence.js"; import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js"; import { jsonObjectSchema } from "../domain/jsonValue.js"; @@ -175,6 +176,15 @@ export class WebNetworkCaptureService { ? [...limitations, subjectUnavailableReason] : limitations, }, + ...(safeValue.format !== "qlog" + ? {} + : { + analysisProfile: createAnalysisProfile(value.decoder, { + qlog_profile: safeValue.qlog.profile, + clock: safeValue.qlog.clock, + scope: safeValue.qlog.scope, + }), + }), confidence: "observed", authority: "historical-reference", limitations, diff --git a/src/browser/history/CaptureFailures.ts b/src/browser/history/CaptureFailures.ts index 51eceb0b5..b525126a5 100644 --- a/src/browser/history/CaptureFailures.ts +++ b/src/browser/history/CaptureFailures.ts @@ -86,7 +86,7 @@ export const historicalCaptureFailure = ( ) return captureMemoryFailure( input, - "The HAR decoder exhausted its fixed V8 heap", + `The ${input.format} decoder exhausted its fixed V8 heap`, cause, ); } diff --git a/src/browser/history/HarJson.ts b/src/browser/history/CaptureJson.ts similarity index 83% rename from src/browser/history/HarJson.ts rename to src/browser/history/CaptureJson.ts index 257630932..59743788a 100644 --- a/src/browser/history/HarJson.ts +++ b/src/browser/history/CaptureJson.ts @@ -2,6 +2,7 @@ import { visit } from "jsonc-parser"; import { LosslessNumber } from "lossless-json"; import { WEB_NETWORK_CAPTURE_LIMITS } from "../../domain/webNetworkCapture.js"; import { CaptureFormatError } from "./CaptureFormatError.js"; +import { TextDecoder } from "node:util"; type Frame = | { @@ -13,7 +14,10 @@ type Frame = | { kind: "array"; pointer: string; items: unknown[] }; /** Materialize upstream JSON events without duplicate folding, prototype setters or numeric rounding. */ -export const parseHarJson = (text: string): unknown => { +export const parseCaptureJson = ( + text: string, + format: "HAR" | "qlog", +): unknown => { const frames: Frame[] = []; let result: unknown; const valuePointer = (): string => { @@ -29,7 +33,7 @@ export const parseHarJson = (text: string): unknown => { if (frames.length > WEB_NETWORK_CAPTURE_LIMITS.depth) throw new CaptureFormatError( "input-limit", - "HAR exceeds the 64-level complete-evidence nesting budget.", + `${format} exceeds the 64-level complete-evidence nesting budget.`, valuePointer(), ); }; @@ -69,7 +73,7 @@ export const parseHarJson = (text: string): unknown => { if (frame.members.has(key)) throw new CaptureFormatError( "format", - "HAR JSON contains duplicate object members.", + `${format} JSON contains duplicate object members.`, ); frame.pendingKey = key; }, @@ -103,7 +107,7 @@ export const parseHarJson = (text: string): unknown => { throw new TypeError("JSON visitor emitted an unsupported literal."); }, onError: () => { - throw new CaptureFormatError("format", "HAR is malformed JSON."); + throw new CaptureFormatError("format", `${format} is malformed JSON.`); }, }, { @@ -114,3 +118,18 @@ export const parseHarJson = (text: string): unknown => { ); return result; }; + +/** UTF-8 JSON decoding skips one permitted BOM and rejects malformed bytes. */ +export const decodeCaptureText = ( + bytes: Uint8Array, + format: "HAR" | "qlog", +): string => { + try { + return new TextDecoder("utf-8", { fatal: true }).decode(bytes); + } catch { + throw new CaptureFormatError( + "format", + `${format} is not valid UTF-8 JSON.`, + ); + } +}; diff --git a/src/browser/history/CaptureRelease.ts b/src/browser/history/CaptureRelease.ts index 512c2fa6e..77f29df51 100644 --- a/src/browser/history/CaptureRelease.ts +++ b/src/browser/history/CaptureRelease.ts @@ -35,3 +35,11 @@ export const SAZ_CAPTURE_PROVIDER_IDENTITY = { name: "REA Fiddler SAZ capture adapter", version: "1;zip.js@2.15.0;xmldom@0.9.12", } as const; + +/** Fixed offline interpretation profile, not an authenticated producer claim. */ +export const QLOG_CAPTURE_PROVIDER_IDENTITY = { + id: "aioquic-qlog-json", + name: "REA aioquic qlog capture adapter", + version: + "aioquic-profile@1.5.0;qlog@0.3;jsonc-parser@3.3.1;lossless-json@4.3.1", +} as const; diff --git a/src/browser/history/HarCapture.test.ts b/src/browser/history/HarCapture.test.ts index 2eb1fe21a..9614384cc 100644 --- a/src/browser/history/HarCapture.test.ts +++ b/src/browser/history/HarCapture.test.ts @@ -1,7 +1,8 @@ import { createHash } from "node:crypto"; import { expect, it } from "vitest"; import { historicalHar } from "../../../tests/fixtures/historicalHar.js"; -import { decodeHarCapture, decodeHarText } from "./HarCapture.js"; +import { decodeCaptureText } from "./CaptureJson.js"; +import { decodeHarCapture } from "./HarCapture.js"; it.each([ { values: [] }, @@ -563,18 +564,21 @@ it("ignores one leading byte-order mark, as HAR 1.2 requires of readers", () => const text = JSON.stringify(historicalHar()); const body = Buffer.from(text, "utf8"); const marked = Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), body]); - expect(decodeHarText(marked)).toBe(text); - expect(decodeHarCapture(decodeHarText(marked), [])).toEqual( - decodeHarCapture(decodeHarText(body), []), + expect(decodeCaptureText(marked, "HAR")).toBe(text); + expect(decodeHarCapture(decodeCaptureText(marked, "HAR"), [])).toEqual( + decodeHarCapture(decodeCaptureText(body, "HAR"), []), ); // A second mark is content, not an encoding signature. expect(() => decodeHarCapture( - decodeHarText(Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), marked])), + decodeCaptureText( + Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), marked]), + "HAR", + ), [], ), ).toThrow("HAR is malformed JSON."); - expect(() => decodeHarText(Buffer.from([0x7b, 0xff, 0x7d]))).toThrow( - "HAR is not valid UTF-8 JSON.", - ); + expect(() => + decodeCaptureText(Buffer.from([0x7b, 0xff, 0x7d]), "HAR"), + ).toThrow("HAR is not valid UTF-8 JSON."); }); diff --git a/src/browser/history/HarCapture.ts b/src/browser/history/HarCapture.ts index f3a83af15..83b442f5c 100644 --- a/src/browser/history/HarCapture.ts +++ b/src/browser/history/HarCapture.ts @@ -1,7 +1,6 @@ import { createHash } from "node:crypto"; -import { TextDecoder } from "node:util"; import { isSafeNumber, LosslessNumber } from "lossless-json"; -import { parseHarJson } from "./HarJson.js"; +import { parseCaptureJson } from "./CaptureJson.js"; import type { JsonValue } from "../../domain/jsonValue.js"; import { WEB_NETWORK_CAPTURE_LIMITS, @@ -26,25 +25,12 @@ type Projection = Pick< "reported" | "numeric_literals" | "binary_fields" | "redactions" >; -/** - * Decode retained HAR bytes as UTF-8. HAR 1.2 lets writers emit a byte-order - * mark and requires readers to ignore it, so one leading BOM is skipped. - */ -export const decodeHarText = (bytes: Uint8Array): string => { - try { - // ignoreBOM: false (the default) is what skips the mark; true keeps it. - return new TextDecoder("utf-8", { fatal: true }).decode(bytes); - } catch { - throw new CaptureFormatError("format", "HAR is not valid UTF-8 JSON."); - } -}; - /** Decode a retained HAR with exact numeric lexemes and unchanged upstream schema validation. */ export const decodeHarCapture = ( text: string, sensitiveValues: readonly string[], ): DecodedCapture => { - const raw = parseHarJson(text); + const raw = parseCaptureJson(text, "HAR"); const validation = projectHar(raw, [], false); createHarValidator()(validation.reported); if ( diff --git a/src/browser/history/HistoricalCaptureDecoder.ts b/src/browser/history/HistoricalCaptureDecoder.ts index f17f45a21..7f6e7b83a 100644 --- a/src/browser/history/HistoricalCaptureDecoder.ts +++ b/src/browser/history/HistoricalCaptureDecoder.ts @@ -198,15 +198,14 @@ export class HistoricalCaptureDecoder { OPERATION, "Capture decoder changed the selected upstream profile identity.", ); - result = ok({ - ...reply.value, + const common = { artifact: { path: input.capture_path, sha256: snapshot.sha256, bytes: snapshot.bytes.length, }, - format: input.format, - runtime_attribution: "unknown", + + runtime_attribution: "unknown" as const, limitations: [ ...(adapter.limitations ?? []), "This is retained producer evidence, without live browser transaction IDs, scenario provenance, execution attribution or deployment authenticity. Recorded URLs are never fetched.", @@ -214,7 +213,22 @@ export class HistoricalCaptureDecoder { "Known authentication headers/cookies and explicitly marked literal values are excluded. Unknown extension names do not establish sensitivity.", `A 32 MiB input, 96 MiB reply and 64-level nesting budget bound complete evidence. ${nodeMemoryLimitation()}; mitmproxy decoding uses a 768 MiB Linux address-space limit. Each owned command has a 30-second deadline with independent cleanup.`, ], - }); + }; + if (input.format === "qlog") { + if (!("qlog" in reply.value)) + throw new AnalysisOutputError( + OPERATION, + "Capture decoder omitted qlog transport facts.", + ); + result = ok({ ...reply.value, ...common, format: "qlog" }); + } else { + if ("qlog" in reply.value) + throw new AnalysisOutputError( + OPERATION, + "Capture decoder supplied inapplicable qlog transport facts.", + ); + result = ok({ ...reply.value, ...common, format: input.format }); + } } catch (cause: unknown) { result = err( redactExplicitFailure( @@ -255,10 +269,10 @@ export class HistoricalCaptureDecoder { /** The Node capture decoder runs on this executable; only Node.js enforces its V8 heap flags. */ const nodeMemoryLimitation = (): string => { if (NODE_CAPTURE_HEAP_ENFORCED) - return `HAR/SAZ use a ${String(NODE_CAPTURE_HEAP_LIMITS.oldGenerationMiB)} MiB Node old-generation heap`; + return `HAR/SAZ/qlog use a ${String(NODE_CAPTURE_HEAP_LIMITS.oldGenerationMiB)} MiB Node old-generation heap`; return NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES === null - ? "HAR/SAZ run on Bun, which does not enforce REA's V8 heap flags" - : `HAR/SAZ run on Bun, which does not enforce REA's V8 heap flags, under a ${String(NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES)}-byte Windows Job Object commit limit`; + ? "HAR/SAZ/qlog run on Bun, which does not enforce REA's V8 heap flags" + : `HAR/SAZ/qlog run on Bun, which does not enforce REA's V8 heap flags, under a ${String(NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES)}-byte Windows Job Object commit limit`; }; const replyFailureReason = (cause: unknown): string => { diff --git a/src/browser/history/NodeCaptureAdapter.test.ts b/src/browser/history/NodeCaptureAdapter.test.ts index c341949db..bf2528f5b 100644 --- a/src/browser/history/NodeCaptureAdapter.test.ts +++ b/src/browser/history/NodeCaptureAdapter.test.ts @@ -8,8 +8,11 @@ vi.mock("./CaptureRelease.js", async (original) => ({ NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES: null, })); -it("refuses a SAZ worker when the runtime has no enforceable memory bound", async () => { - await expect( - new NodeCaptureAdapter("saz").command("/owned/request.json"), - ).rejects.toBeInstanceOf(AnalysisCapabilityUnavailableError); -}); +it.each(["saz", "qlog"] as const)( + "refuses a %s worker when the runtime has no enforceable memory bound", + async (format) => { + await expect( + new NodeCaptureAdapter(format).command("/owned/request.json"), + ).rejects.toBeInstanceOf(AnalysisCapabilityUnavailableError); + }, +); diff --git a/src/browser/history/NodeCaptureAdapter.ts b/src/browser/history/NodeCaptureAdapter.ts index f1e98a07b..8f33430ca 100644 --- a/src/browser/history/NodeCaptureAdapter.ts +++ b/src/browser/history/NodeCaptureAdapter.ts @@ -4,6 +4,7 @@ import { NODE_CAPTURE_HEAP_ENFORCED, HAR_CAPTURE_PROVIDER_IDENTITY, SAZ_CAPTURE_PROVIDER_IDENTITY, + QLOG_CAPTURE_PROVIDER_IDENTITY, NODE_CAPTURE_HEAP_LIMITS, NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES, } from "./CaptureRelease.js"; @@ -13,11 +14,13 @@ import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCo export class NodeCaptureAdapter implements HistoricalCaptureFormatAdapter { readonly identity; readonly limitations; - constructor(readonly format: "har" | "saz") { + constructor(readonly format: "har" | "saz" | "qlog") { this.identity = format === "har" ? HAR_CAPTURE_PROVIDER_IDENTITY - : SAZ_CAPTURE_PROVIDER_IDENTITY; + : format === "saz" + ? SAZ_CAPTURE_PROVIDER_IDENTITY + : QLOG_CAPTURE_PROVIDER_IDENTITY; this.limitations = format === "saz" ? [ @@ -30,14 +33,14 @@ export class NodeCaptureAdapter implements HistoricalCaptureFormatAdapter { } async command(requestPath: string) { if ( - this.format === "saz" && + this.format !== "har" && !NODE_CAPTURE_HEAP_ENFORCED && NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES === null ) throw new AnalysisCapabilityUnavailableError( this.identity.id, "inspect_web_network_capture", - "SAZ decoding requires an enforced decoder memory limit. Run REA with Node.js on this host.", + `${this.format} decoding requires an enforced decoder memory limit. Run REA with Node.js on this host.`, ); return { command: process.execPath, diff --git a/src/browser/history/NodeCaptureProcess.ts b/src/browser/history/NodeCaptureProcess.ts index e94295da4..ff45a869a 100644 --- a/src/browser/history/NodeCaptureProcess.ts +++ b/src/browser/history/NodeCaptureProcess.ts @@ -12,22 +12,25 @@ const requestSchema = z.strictObject({ /** Parse untrusted captures in an owned heap and return one complete bounded reply. */ const main = async (): Promise => { - const format = z.enum(["har", "saz"]).parse(process.argv[3]); + const format = z.enum(["har", "saz", "qlog"]).parse(process.argv[3]); const request = requestSchema.parse( JSON.parse(await readFile(process.argv[2] ?? "", "utf8")), ); let reply: unknown; try { let value; - if (format === "har") { - const { decodeHarCapture, decodeHarText } = - await import("./HarCapture.js"); + if (format !== "saz") { + const { decodeCaptureText } = await import("./CaptureJson.js"); + const decoder = + format === "har" + ? (await import("./HarCapture.js")).decodeHarCapture + : (await import("./QlogCapture.js")).decodeQlogCapture; const snapshot = await readStableArtifact( request.snapshot_path, WEB_NETWORK_CAPTURE_LIMITS.inputBytes, ); - value = decodeHarCapture( - decodeHarText(snapshot.bytes), + value = decoder( + decodeCaptureText(snapshot.bytes, format === "har" ? "HAR" : "qlog"), request.sensitive_values, ); } else { diff --git a/src/browser/history/QlogCapture.test.ts b/src/browser/history/QlogCapture.test.ts new file mode 100644 index 000000000..bd67d4e80 --- /dev/null +++ b/src/browser/history/QlogCapture.test.ts @@ -0,0 +1,259 @@ +import { expect, it } from "vitest"; +import { decodeQlogCapture } from "./QlogCapture.js"; +import { decodeCaptureText } from "./CaptureJson.js"; +import { CaptureFormatError } from "./CaptureFormatError.js"; + +// Synthetic format inputs exercise distinct parser semantics, not producer provenance. +const event = (frames: unknown[], name = "transport:packet_sent") => ({ + name, + time: 1791819895000.25, + data: { header: { packet_type: "1RTT", packet_number: 5 }, frames }, +}); +const frame = (offset: number, length: number, fin = false) => ({ + frame_type: "stream", + stream_id: 0, + offset, + length, + fin, +}); +const capture = (events: unknown[], extra: Record = {}) => + JSON.stringify({ + qlog_format: "JSON", + qlog_version: "0.3", + ...extra, + traces: [ + { + common_fields: { ODCID: "same" }, + vantage_point: { name: "aioquic", type: "server" }, + events, + }, + ], + }); +const facts = (text: string, sensitive: string[] = []) => { + const decoded = decodeQlogCapture(text, sensitive); + if (!("qlog" in decoded)) throw new Error("qlog facet required"); + return { decoded, traces: decoded.qlog.traces }; +}; + +it("derives interval unions per trace/direction while retaining event source order and extensions", () => { + const original = JSON.parse( + capture([ + event([frame(10, 10)]), + event([frame(0, 15), frame(30, 5, true)]), + event([frame(0, 4)], "transport:packet_received"), + { + name: "recovery:packet_lost", + time: 1, + data: { packet_number: 5, type: "1RTT" }, + }, + ]), + ); + original.traces.push({ + ...original.traces[0], + events: [event([frame(0, 3)])], + }); + Object.defineProperty(original, "__proto__", { + enumerable: true, + value: { ordinary: true }, + }); + original.extra = "https://example.test/path?token=ordinary"; + const { decoded, traces } = facts(JSON.stringify(original)); + expect(traces[0]?.streams[0]).toMatchObject({ + stream_id: "0", + direction: "sent", + unique_bytes: "25", + frame_length_sum: "30", + repeated_bytes: "5", + covered_ranges: [ + { start: "0", end: "20" }, + { start: "30", end: "35" }, + ], + reported_final_sizes: ["35"], + }); + expect(traces[0]?.streams[1]).toMatchObject({ + direction: "received", + unique_bytes: "4", + }); + expect(traces[1]?.streams[0]).toMatchObject({ unique_bytes: "3" }); + expect(traces[0]?.loss_declarations[0]).toMatchObject({ + confidence: "observed", + packet_number: { state: "reported", value: "5" }, + packet_number_space: "application", + }); + expect(decoded.records.at(-1)?.location).toEqual({ + kind: "json-pointer", + pointer: "/traces/1/events/0", + }); + expect(decoded.container.reported).toMatchObject({ + extra: original.extra, + ["__proto__"]: { ordinary: true }, + }); + expect(Reflect.get(Object.prototype, "ordinary")).toBeUndefined(); +}); + +it("keeps exact 62-bit integers and numeric lexemes without allocating exponent expansion", () => { + const text = capture([event([frame(0, 0, true)])]) + .replace('"stream_id":0', '"stream_id":4611686018427387903') + .replace('"offset":0', '"offset":9007199254740993') + .replace('"length":0', '"length":1e3'); + const { decoded, traces } = facts(text); + expect(traces[0]?.streams[0]).toMatchObject({ + stream_id: "4611686018427387903", + unique_bytes: "1000", + reported_final_sizes: ["9007199254741993"], + }); + expect(decoded.records[0]?.numeric_literals).toContainEqual({ + pointer: "/data/frames/0/length", + producer_type: "json-number", + literal: "1e3", + }); + expect(decoded.records[0]?.reported).toMatchObject({ + data: { frames: [{ stream_id: null, offset: null, length: 1000 }] }, + }); +}); + +it("keeps incomplete fields unknown and FIN without bytes as metadata", () => { + const { traces } = facts( + capture([ + event([{ frame_type: "stream", stream_id: 0, length: 5 }]), + event([frame(10, 0, true)]), + { name: "recovery:packet_lost", time: 1, data: {} }, + { + name: "extension:custom", + time: 0, + data: { body: "ordinary evidence" }, + }, + ]), + ); + expect(traces[0]?.incomplete_fact_records).toEqual([0]); + expect(traces[0]?.streams[0]).toMatchObject({ + unique_bytes: "0", + covered_ranges: [], + reported_final_sizes: ["10"], + }); + expect(traces[0]?.loss_declarations[0]?.packet_number.state).toBe("unknown"); +}); + +it("excludes structural auth material and explicit normalized values without guessing extension secrecy", () => { + const text = capture([ + { + name: "http:frame_parsed", + time: 1, + data: { + frame: { + frame_type: "headers", + headers: [ + { name: "authorization", value: "Bearer credential" }, + { name: ":path", value: "/?token=ordinary" }, + { name: "location", value: "https://user:pass@example.test/a" }, + ], + }, + }, + }, + { + name: "transport:parameters_set", + time: 2, + data: { stateless_reset_token: "reset-credential" }, + }, + event([ + { frame_type: "new_connection_id", reset_token: "reset-frame" }, + { frame_type: "new_token", token: "new-token" }, + frame(0, 100), + ]), + { + name: "extension:custom", + time: 3, + data: { + secret: "caller-selected ordinary value", + reset_token: "extension ordinary", + }, + }, + ]).replace('"length":100', '"length":1e2'); + const { decoded, traces } = facts(text, ["100"]); + const serialized = JSON.stringify(decoded); + for (const hidden of [ + "Bearer credential", + "reset-credential", + "reset-frame", + "new-token", + "100", + "user:pass", + ]) + expect(serialized).not.toContain(hidden); + expect(serialized).toContain("token=ordinary"); + expect(serialized).toContain("caller-selected ordinary value"); + expect(traces[0]?.streams).toEqual([]); + expect(traces[0]?.incomplete_fact_records).toContain(2); +}); + +it("hides marked ancestor keys and safe derived sums while still validating hidden subtrees", () => { + const text = capture([event([frame(0, 3), frame(3, 4)])]); + expect(facts(text, ["7"]).traces[0]?.streams).toEqual([]); + const { decoded, traces } = facts(text, ["events"]); + expect(decoded.records[0]?.reported).toBe(null); + expect(decoded.records[0]?.location.kind).toBe("unknown"); + expect(traces[0]?.location.kind).toBe("unknown"); + expect(traces[0]?.streams).toEqual([]); + expect(() => + facts(text.replace('"length":3', '"length":-3'), ["events"]), + ).toThrow(CaptureFormatError); +}); + +it.each([ + "-1", + "0.5", + "4611686018427387904", + "1e1000000", + "1e-1000000", + '"3"', + "true", +])("rejects malformed or out-of-range STREAM integer %s", (literal) => { + expect(() => + facts( + capture([event([frame(0, 3)])]).replace( + '"length":3', + `"length":${literal}`, + ), + ), + ).toThrow(CaptureFormatError); +}); + +it.each([ + '{"qlog_format":"JSON","qlog_format":"JSON","qlog_version":"0.3","traces":[]}', + capture([]).slice(0, -1), + capture([]) + " {}", + capture([{ name: "invalid", time: "1", data: {} }]), + capture([[1, "transport", "packet_sent", {}]]), + capture([event([{ ...frame(0, 1), fin: "true" }])]), + capture([event([{ ...frame(0, 1), offset: 4611686018427388000 }])]), +])( + "rejects malformed JSON/profile events with no partial success: %s", + (text) => { + expect(() => facts(text)).toThrow(CaptureFormatError); + }, +); + +it("rejects other qlog containers/versions and excessive nesting, accepts empty profiles and UTF-8 BOM", () => { + for (const extra of [{ qlog_format: "JSON-SEQ" }, { qlog_version: "0.4" }]) + expect(() => facts(capture([], extra))).toThrow("Only the aioquic"); + expect(() => + facts( + capture([], { + extension: JSON.parse("[".repeat(65) + "0" + "]".repeat(65)), + }), + ), + ).toThrow("nesting budget"); + expect( + facts('{"qlog_format":"JSON","qlog_version":"0.3","traces":[]}').traces, + ).toEqual([]); + const raw = Buffer.from(capture([])); + expect( + facts( + decodeCaptureText( + Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), raw]), + "qlog", + ), + ).decoded.total_records, + ).toBe(0); + expect(() => decodeCaptureText(Buffer.from([0xff]), "qlog")).toThrow("UTF-8"); +}); diff --git a/src/browser/history/QlogCapture.ts b/src/browser/history/QlogCapture.ts new file mode 100644 index 000000000..2e8a1f790 --- /dev/null +++ b/src/browser/history/QlogCapture.ts @@ -0,0 +1,503 @@ +import { isSafeNumber, LosslessNumber } from "lossless-json"; +import type { JsonValue } from "../../domain/jsonValue.js"; +import type { QlogCaptureFacts } from "../../domain/qlogCapture.js"; +import { + webNetworkCapturePayloadSchema, + type WebNetworkCaptureRecord, +} from "../../domain/webNetworkCapture.js"; +import type { z } from "zod"; +import { parseCaptureJson } from "./CaptureJson.js"; +import { CaptureFormatError } from "./CaptureFormatError.js"; +import { + CaptureRedaction, + captureObject, + capturePointer, +} from "./CaptureRedaction.js"; +import { QLOG_CAPTURE_PROVIDER_IDENTITY } from "./CaptureRelease.js"; + +const MAX_VARINT = (1n << 62n) - 1n; +const credentials = new Set([ + "authorization", + "proxy-authorization", + "cookie", + "set-cookie", +]); +type Projection = Pick< + WebNetworkCaptureRecord, + "reported" | "numeric_literals" | "binary_fields" | "redactions" +>; +type Trace = QlogCaptureFacts["traces"][number]; +type Stream = Trace["streams"][number]; + +/** Inspect the selected aioquic JSON profile without authenticating its producer or interpreting other qlog layouts. */ +export const decodeQlogCapture = ( + text: string, + sensitiveValues: readonly string[], +): z.output => { + const raw = parseCaptureJson(text, "qlog"); + if (!object(raw) || raw.qlog_format !== "JSON" || raw.qlog_version !== "0.3") + throw new CaptureFormatError( + "unsupported", + "Only the aioquic 1.5.0 JSON qlog 0.3 profile is supported.", + ); + if (!Array.isArray(raw.traces)) + return invalid("qlog must contain a traces array.", "/traces"); + const traces = raw.traces.map(validateTrace); + const container = project( + { ...raw, traces: traces.map((trace) => ({ ...trace, events: null })) }, + sensitiveValues, + ); + const records: WebNetworkCaptureRecord[] = []; + const facts: Trace[] = []; + const excluded = (value: string): boolean => + sensitiveValues.some((literal) => value.includes(literal)); + const ancestorsHidden = excluded("traces") || excluded("events"); + for (const [traceIndex, trace] of traces.entries()) { + const tracePointer = `/traces/${traceIndex}`; + const fact = createTraceFact( + trace, + traceIndex, + sensitiveValues, + ancestorsHidden, + ); + const streams = new Map(); + for (const [eventIndex, event] of trace.events.entries()) { + const ordinal = records.length; + const projection: Projection = ancestorsHidden + ? { + reported: null, + numeric_literals: [], + binary_fields: [], + redactions: [{ pointer: "", reason: "explicit-sensitive-value" }], + } + : project(event, sensitiveValues); + records.push({ + ordinal, + location: ancestorsHidden + ? { kind: "unknown", reason: "explicit-sensitive-value" } + : { + kind: "json-pointer", + pointer: `${tracePointer}/events/${eventIndex}`, + }, + ...projection, + limitations: [ + "Record ordinal is derived from trace/event source traversal, not global time order. Qlog lengths and FIN do not expose original HTTP body bytes or establish network completeness.", + ], + }); + collectTransportFacts(projection, ordinal, { fact, streams, excluded }); + } + for (const stream of streams.values()) { + summarizeStream(stream); + const numericValues = [ + stream.frame_length_sum, + stream.unique_bytes, + stream.repeated_bytes, + ...stream.reported_final_sizes, + ...stream.covered_ranges.flatMap((r) => [r.start, r.end]), + ]; + if (numericValues.some(excluded)) + fact.incomplete_fact_records.push( + ...stream.contributions.map((c) => c.record_ordinal), + ); + else fact.streams.push(stream); + } + fact.incomplete_fact_records = [...new Set(fact.incomplete_fact_records)]; + facts.push(fact); + } + return { + decoder: { ...QLOG_CAPTURE_PROVIDER_IDENTITY }, + container: { + reported: container.reported, + numeric_literals: container.numeric_literals, + redactions: container.redactions, + records_pointer: null, + }, + total_records: records.length, + records, + qlog: { + profile: "aioquic-1.5.0-json-0.3", + scope: "all-recorded-events-before-selection", + body_bytes: "not-reported", + clock: { + unit: "milliseconds", + domain: "unix-epoch", + basis: "selected-aioquic-profile-not-producer-authentication", + }, + traces: facts, + }, + }; +}; + +const createTraceFact = ( + trace: Record & { events: unknown[] }, + traceIndex: number, + sensitiveValues: readonly string[], + ancestorsHidden: boolean, +): Trace => { + const tracePointer = `/traces/${traceIndex}`; + const metadata = project( + { ...trace, events: null }, + sensitiveValues, + ).reported; + const meta = !ancestorsHidden && object(metadata) ? metadata : null; + const common = + meta !== null && object(meta.common_fields) ? meta.common_fields : null; + const vantage = + meta !== null && object(meta.vantage_point) + ? meta.vantage_point.type + : null; + return { + trace_index: traceIndex, + location: ancestorsHidden + ? { kind: "unknown", reason: "explicit-sensitive-value" } + : { kind: "json-pointer", pointer: tracePointer }, + event_count: trace.events.length, + odcid: typeof common?.ODCID === "string" ? common.ODCID : null, + vantage_point: + vantage === "client" || vantage === "server" ? vantage : null, + loss_declarations: [], + streams: [], + incomplete_fact_records: [], + }; +}; + +const validateTrace = ( + trace: unknown, + index: number, +): Record & { events: unknown[] } => { + const pointer = `/traces/${index}`; + if (!object(trace) || !Array.isArray(trace.events)) + return invalid("qlog trace must contain an events array.", pointer); + for (const [eventIndex, event] of trace.events.entries()) { + const eventPointer = `${pointer}/events/${eventIndex}`; + if ( + !object(event) || + typeof event.name !== "string" || + !(event.time instanceof LosslessNumber) || + !Number.isFinite(Number(event.time.value)) || + !object(event.data) + ) + invalid( + "qlog events must be objects with name, finite numeric time and object data.", + eventPointer, + ); + // Validate interpretable fields on ORIGINAL values even when a declaration + // subsequently hides them. Missing fields remain meaningful unknowns. + if (event.name === "recovery:packet_lost") + unsigned(event.data.packet_number, `${eventPointer}/data/packet_number`); + if (packetEvent(event.name) && Array.isArray(event.data.frames)) { + for (const [frameIndex, frame] of event.data.frames.entries()) { + if (object(frame) && frame.frame_type === "stream") { + validateStreamFrame( + frame, + `${eventPointer}/data/frames/${frameIndex}`, + ); + } + } + } + } + return { ...trace, events: trace.events }; +}; + +const validateStreamFrame = ( + frame: Record, + p: string, +): void => { + unsigned(frame.stream_id, `${p}/stream_id`); + const offset = unsigned(frame.offset, `${p}/offset`); + const length = unsigned(frame.length, `${p}/length`); + if (offset !== null && length !== null && offset + length > MAX_VARINT) + invalid("qlog stream final offset exceeds the QUIC varint range.", p); + if ( + frame.fin !== undefined && + frame.fin !== null && + typeof frame.fin !== "boolean" + ) + invalid("qlog STREAM fin must be boolean when reported.", `${p}/fin`); +}; + +const collectTransportFacts = ( + projection: Projection, + ordinal: number, + state: { + fact: Trace; + streams: Map; + excluded: (value: string) => boolean; + }, +): void => { + const { fact, streams, excluded } = state; + if (!object(projection.reported) || !object(projection.reported.data)) { + fact.incomplete_fact_records.push(ordinal); + return; + } + const visible = projection.reported; + const data = visible.data; + if (!object(data)) return; + const literals = new Map( + projection.numeric_literals.map((n) => [n.pointer, n.literal]), + ); + const read = (value: unknown, pointer: string): bigint | null => { + const literal = literals.get(pointer); + const number = unsigned( + literal === undefined ? value : new LosslessNumber(literal), + pointer, + ); + return number !== null && excluded(number.toString()) ? null : number; + }; + if (visible.name === "recovery:packet_lost") { + fact.loss_declarations.push(lossDeclaration(data, ordinal, read)); + } + if (!packetEvent(visible.name)) return; + if (!Array.isArray(data.frames)) { + fact.incomplete_fact_records.push(ordinal); + return; + } + for (const [frameIndex, frame] of data.frames.entries()) { + if ( + !object(frame) || + frame.frame_type === undefined || + frame.frame_type === null + ) { + fact.incomplete_fact_records.push(ordinal); + continue; + } + if (frame.frame_type !== "stream") continue; + const p = `/data/frames/${frameIndex}`; + const id = read(frame.stream_id, `${p}/stream_id`), + offset = read(frame.offset, `${p}/offset`), + length = read(frame.length, `${p}/length`); + if (id === null || offset === null || length === null) { + fact.incomplete_fact_records.push(ordinal); + continue; + } + const direction = + visible.name === "transport:packet_sent" ? "sent" : "received"; + const key = `${direction}/${id}`; + let stream = streams.get(key); + if (stream === undefined) { + stream = { + stream_id: id.toString(), + direction, + confidence: "derived", + scope: "observed-stream-frame-ranges", + contributions: [], + covered_ranges: [], + unique_bytes: "0", + frame_length_sum: "0", + repeated_bytes: "0", + reported_final_sizes: [], + }; + streams.set(key, stream); + } + stream.contributions.push({ + record_ordinal: ordinal, + frame_index: frameIndex, + offset: offset.toString(), + length: length.toString(), + fin: typeof frame.fin === "boolean" ? frame.fin : null, + }); + } +}; + +const lossDeclaration = ( + data: Record, + ordinal: number, + read: (value: unknown, pointer: string) => bigint | null, +): Trace["loss_declarations"][number] => { + const pn = read(data.packet_number, "/data/packet_number"); + const type = typeof data.type === "string" ? data.type : null; + return { + record_ordinal: ordinal, + confidence: "observed", + packet_number: + pn === null + ? { + state: "unknown", + reason: "omitted-null-or-redacted", + } + : { state: "reported", value: pn.toString() }, + packet_type: type, + packet_number_space: + type === "initial" + ? "initial" + : type === "handshake" + ? "handshake" + : type === "0RTT" || type === "1RTT" + ? "application" + : null, + }; +}; + +const summarizeStream = (stream: Stream): void => { + const spans = stream.contributions + .map( + (c) => [BigInt(c.offset), BigInt(c.offset) + BigInt(c.length)] as const, + ) + .sort((a, b) => (a[0] < b[0] ? -1 : a[0] > b[0] ? 1 : 0)); + const merged: { start: bigint; end: bigint }[] = []; + for (const [start, end] of spans) { + if (start === end) continue; + const last = merged.at(-1); + if (last !== undefined && start <= last.end) + last.end = end > last.end ? end : last.end; + else merged.push({ start, end }); + } + const sum = stream.contributions.reduce((s, c) => s + BigInt(c.length), 0n); + const unique = merged.reduce((s, r) => s + r.end - r.start, 0n); + stream.covered_ranges = merged.map((r) => ({ + start: r.start.toString(), + end: r.end.toString(), + })); + stream.frame_length_sum = sum.toString(); + stream.unique_bytes = unique.toString(); + stream.repeated_bytes = (sum - unique).toString(); + stream.reported_final_sizes = [ + ...new Set( + stream.contributions + .filter((c) => c.fin === true) + .map((c) => (BigInt(c.offset) + BigInt(c.length)).toString()), + ), + ]; +}; + +const packetEvent = (name: unknown): boolean => + name === "transport:packet_sent" || name === "transport:packet_received"; +const object = (value: unknown): value is Record => + captureObject(value) && !(value instanceof LosslessNumber); +const invalid = (message: string, pointer: string): never => { + throw new CaptureFormatError("format", message, pointer); +}; + +/** Canonicalize JSON integer notation exactly, checking expansion before allocation. */ +const unsigned = (value: unknown, pointer: string): bigint | null => { + if (value === undefined || value === null) return null; + const literal = + value instanceof LosslessNumber + ? value.value + : typeof value === "number" + ? String(value) + : ""; + const m = /^(-?)([0-9]+)(?:\.([0-9]+))?(?:[eE]([+-]?[0-9]+))?$/u.exec( + literal, + ); + if (m === null) + return invalid("qlog QUIC integer must be a JSON number.", pointer); + const digits = `${m[2]}${m[3] ?? ""}`.replace(/^0+/u, ""); + if (digits === "") return 0n; + const exponent = m[4] === undefined ? 0 : Number(m[4]); + const shift = exponent - (m[3]?.length ?? 0); + const count = digits.length + shift; + if (m[1] === "-" || !Number.isSafeInteger(shift) || count > 19 || count <= 0) + return invalid( + "qlog QUIC integer is negative, fractional or outside the 62-bit range.", + pointer, + ); + if (shift < 0 && /[1-9]/u.test(digits.slice(count))) + return invalid("qlog QUIC integer is fractional.", pointer); + const canonical = + shift < 0 ? digits.slice(0, count) : digits + "0".repeat(shift); + const result = BigInt(canonical); + if (result > MAX_VARINT) + return invalid("qlog QUIC integer exceeds the 62-bit range.", pointer); + return result; +}; + +/** Preserve JSON values/lexemes and structurally exclude only transport credentials and declared values. */ +const project = ( + value: unknown, + sensitiveValues: readonly string[], +): Projection => { + const redactor = new CaptureRedaction(sensitiveValues); + const numeric_literals: Projection["numeric_literals"] = []; + const eventName = object(value) ? value.name : null; + const visit = ( + item: unknown, + pointer: string, + parent?: Record, + ): JsonValue => { + const header = + (eventName === "http:frame_created" || + eventName === "http:frame_parsed") && + /^\/data\/frame\/headers\/\d+\/value$/u.test(pointer) && + typeof parent?.name === "string"; + if ( + (header && credentials.has(String(parent?.name).toLowerCase())) || + qlogToken(eventName, pointer, parent) + ) + return redactor.credential(pointer); + if (item instanceof LosslessNumber) { + if ( + redactor.text(item.value, pointer) === null || + (isSafeNumber(item.value) && + redactor.text(String(Number(item.value)), pointer) === null) + ) + return null; + numeric_literals.push({ + pointer, + producer_type: "json-number", + literal: item.value, + }); + return isSafeNumber(item.value) ? Number(item.value) : null; + } + if (typeof item === "string") + return redactor.text( + item, + pointer, + header && + ["location", "referer", "origin"].includes( + String(parent?.name).toLowerCase(), + ), + ); + if (item === null || typeof item === "boolean") return item; + if (Array.isArray(item)) + return item.map((child, index) => + visit(child, capturePointer(pointer, String(index))), + ); + if (object(item)) + return Object.fromEntries( + Object.entries(item).flatMap(([key, child]) => { + if (sensitiveValues.some((literal) => key.includes(literal))) { + redactor.redactions.push({ + pointer, + reason: "explicit-sensitive-value", + scope: "property-name", + }); + return []; + } + return [[key, visit(child, capturePointer(pointer, key), item)]]; + }), + ); + return invalid("qlog parser returned a non-JSON value.", pointer); + }; + return { + reported: visit(value, ""), + numeric_literals, + binary_fields: [], + redactions: redactor.redactions, + }; +}; + +const qlogToken = ( + eventName: unknown, + pointer: string, + parent?: Record, +): boolean => { + if ( + packetEvent(eventName) && + /^\/data\/frames\/\d+\/reset_token$/u.test(pointer) && + parent?.frame_type === "new_connection_id" + ) + return true; + if ( + packetEvent(eventName) && + /^\/data\/frames\/\d+\/token$/u.test(pointer) && + parent?.frame_type === "new_token" + ) + return true; + if ( + eventName === "transport:parameters_set" && + pointer === "/data/stateless_reset_token" + ) + return true; + return false; +}; diff --git a/src/cli/webNetworkCaptureCommands.ts b/src/cli/webNetworkCaptureCommands.ts index 336c79364..655d34bc1 100644 --- a/src/cli/webNetworkCaptureCommands.ts +++ b/src/cli/webNetworkCaptureCommands.ts @@ -17,7 +17,7 @@ export const registerWebNetworkCaptureCommands = ( const service = createWebNetworkCaptureService(environment); cli.command(CLI_COMMANDS.inspectWebNetworkCapture, { description: - "Inspect retained HAR, native mitmproxy, or Fiddler SAZ records without fetching recorded URLs", + "Inspect retained HAR, native mitmproxy, Fiddler SAZ or aioquic qlog records without fetching recorded URLs", args: z.object({ capturePath: z.string().describe("Absolute local capture path"), format: z diff --git a/src/composition/webNetworkCaptures.ts b/src/composition/webNetworkCaptures.ts index 017182500..9d9e8c6cc 100644 --- a/src/composition/webNetworkCaptures.ts +++ b/src/composition/webNetworkCaptures.ts @@ -12,6 +12,7 @@ export const createWebNetworkCaptureService = ( [ new NodeCaptureAdapter("har"), new NodeCaptureAdapter("saz"), + new NodeCaptureAdapter("qlog"), new MitmproxyCaptureAdapter(environment), ], environment, diff --git a/src/contracts/analysisViewToolContracts.ts b/src/contracts/analysisViewToolContracts.ts index 161d9d7b4..06506c83e 100644 --- a/src/contracts/analysisViewToolContracts.ts +++ b/src/contracts/analysisViewToolContracts.ts @@ -16,7 +16,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [ ...toolContractMetadata("inspect_analysis_view"), kind: "application", description: - 'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_evm_interface, inspect_managed_artifact, inspect_android_package or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. For layout, JavaScript, native, inventory, WASM, EVM, managed artifact and Android package sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Native views select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', + 'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_evm_interface, inspect_managed_artifact, inspect_android_package or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. For layout, JavaScript, native, inventory, WASM, EVM, managed artifact and Android package sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Native views select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', inputSchema: inspectAnalysisViewInputSchema, outputSchema: evidenceResultOf(analysisViewResultSchema), examples: [ diff --git a/src/contracts/webNetworkCaptureToolContracts.ts b/src/contracts/webNetworkCaptureToolContracts.ts index 06420f960..592dd9b54 100644 --- a/src/contracts/webNetworkCaptureToolContracts.ts +++ b/src/contracts/webNetworkCaptureToolContracts.ts @@ -16,7 +16,7 @@ export const WEB_NETWORK_CAPTURE_TOOL_CONTRACTS = [ name: "inspect_web_network_capture", ...toolContractMetadata("inspect_web_network_capture"), description: - "Request detail summary for capture identity and counts; then page retained records with inspect_analysis_view kind capture (preview or complete), avoiding repeated decoding. Default detail complete preserves existing inline delivery. Inspect retained local HAR 1.2, native mitmproxy, or Fiddler SAZ capture records without contacting recorded URLs. Returns original ordinals, pointers/byte ranges/archive member names, preserved producer fields, precise numeric representations and exact exposed binary bytes inline. Excludes known transport credentials and explicitly marked values. Decodes every record before optional ordinal selection; malformed or oversized captures return no partial success. Duplicate HAR members are invalid; prototype-named members such as __proto__ are preserved as ordinary own JSON members without prototype mutation. Uses pinned upstream parsers, owned temporary files and bounded offline processes. SAZ preserves HTTP fields, XML metadata and encoded body/message bytes without replay or HAR conversion; encrypted archives are unsupported. Native mitmproxy requires caller-supplied mitmdump 12.2.3 on Linux. Browser transaction IDs, runtime attribution and original bytes of HAR Unicode text remain unknown.", + "Request detail summary for capture identity and counts; then page retained records with inspect_analysis_view kind capture (preview or complete), avoiding repeated decoding. Default detail complete preserves existing inline delivery. Inspect retained local HAR 1.2, native mitmproxy, Fiddler SAZ or aioquic 1.5.0 JSON qlog 0.3 events without contacting recorded URLs. Returns original ordinals, pointers/byte ranges/archive member names, preserved producer fields, precise numeric representations and exact exposed binary bytes inline. Excludes known transport credentials and explicitly marked values. Decodes every record before optional ordinal selection; malformed or oversized captures return no partial success. Duplicate JSON members are invalid; prototype-named members such as __proto__ are preserved as ordinary own JSON members without prototype mutation. Qlog reports trace-scoped STREAM interval unions and endpoint loss declarations; it does not recover bodies, prove network loss or correlate PCAP. Other qlog layouts are unsupported. Uses pinned upstream parsers, owned temporary files and bounded offline processes. SAZ preserves HTTP fields, XML metadata and encoded body/message bytes without replay or HAR conversion; encrypted archives are unsupported. Native mitmproxy requires caller-supplied mitmdump 12.2.3 on Linux. Browser transaction IDs, runtime attribution and original bytes of HAR Unicode text remain unknown.", kind: "browser-provider", inputSchema: inspectWebNetworkCaptureInputSchema.extend({ detail: analysisDetailSchema, @@ -26,6 +26,14 @@ export const WEB_NETWORK_CAPTURE_TOOL_CONTRACTS = [ webNetworkCaptureSummaryViewResultSchema, ), examples: [ + { + title: "Inspect retained aioquic qlog events", + input: { + capture_path: "/captures/origin.qlog", + format: "qlog", + record_ordinals: [0, 2], + }, + }, { title: "Inspect historical HAR entries", input: { diff --git a/src/domain/analysisView/analysisView.ts b/src/domain/analysisView/analysisView.ts index 6b3b6bef8..05f879024 100644 --- a/src/domain/analysisView/analysisView.ts +++ b/src/domain/analysisView/analysisView.ts @@ -62,6 +62,11 @@ import { captureSummarySchema, } from "./webNetworkCaptureView.js"; import type { WebNetworkApiSurfaceAccumulator } from "./WebNetworkApiSurfaceAccumulator.js"; +import { + qlogViewRequestSchema, + qlogViewPageSchema, + qlogViewContextSchema, +} from "./qlogCaptureView.js"; import { projectAndroidPackageView, androidPackageSummarySchema, @@ -184,6 +189,7 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [ androidViewRequestSchema, apiSurfaceViewSchema, captureViewRequestSchema, + qlogViewRequestSchema, ]); // The application owns admission and authenticated Evidence parsing. SDK and @@ -403,6 +409,14 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ ...viewResultBase, parent_operation: z.literal("inspect_web_network_capture"), }), + z.strictObject({ + kind: z.literal("qlog"), + view: qlogViewRequestSchema, + page: qlogViewPageSchema, + context: qlogViewContextSchema, + ...viewResultBase, + parent_operation: z.literal("inspect_web_network_capture"), + }), z.strictObject({ kind: z.literal("facet"), view: facetViewSchema, @@ -523,6 +537,7 @@ export type UnsignedAnalysisView = > | Omit, "view_digest"> | Omit, "view_digest"> + | Omit, "view_digest"> | Omit< Extract, "view_digest" diff --git a/src/domain/analysisView/qlogCaptureView.ts b/src/domain/analysisView/qlogCaptureView.ts new file mode 100644 index 000000000..a3bdf88c3 --- /dev/null +++ b/src/domain/analysisView/qlogCaptureView.ts @@ -0,0 +1,248 @@ +import { z } from "zod"; +import { + qlogCaptureFactsSchema, + type QlogCaptureFacts, +} from "../qlogCapture.js"; +import { err, ok, type Result } from "../result.js"; +import type { AnalysisError } from "../analysisErrorBase.js"; +import { analysisViewInputError } from "./analysisView.js"; +import { pageViewCoverage } from "./analysisViewCoverage.js"; + +const traceSchema = qlogCaptureFactsSchema.shape.traces.element; +const streamSchema = traceSchema.shape.streams.element; +export const qlogViewContextSchema = qlogCaptureFactsSchema.omit({ + traces: true, +}); +export const qlogSummarySchema = qlogViewContextSchema.extend({ + trace_count: z.number().int().nonnegative(), + retained_stream_count: z.number().int().nonnegative(), + loss_declaration_count: z.number().int().nonnegative(), + incomplete_fact_record_count: z.number().int().nonnegative(), +}); + +/** Select retained trace facts without copying event inventories or every contribution. */ +const pageRequestShape = { + kind: z.literal("qlog"), + offset: z.number().int().nonnegative().default(0), + limit: z.number().int().positive().default(64), +}; +const traceSelectorShape = { trace_index: z.number().int().nonnegative() }; +const streamSelectorShape = { + ...traceSelectorShape, + stream_id: streamSchema.shape.stream_id, + direction: streamSchema.shape.direction, +}; +export const qlogViewRequestSchema = z.discriminatedUnion("facet", [ + z.strictObject({ ...pageRequestShape, facet: z.literal("traces") }), + z.strictObject({ + ...pageRequestShape, + ...traceSelectorShape, + facet: z.literal("streams"), + }), + z.strictObject({ + ...pageRequestShape, + ...traceSelectorShape, + facet: z.literal("loss-declarations"), + }), + z.strictObject({ + ...pageRequestShape, + ...traceSelectorShape, + facet: z.literal("incomplete-records"), + }), + z.strictObject({ + ...pageRequestShape, + ...streamSelectorShape, + facet: z.literal("contributions"), + }), + z.strictObject({ + ...pageRequestShape, + ...streamSelectorShape, + facet: z.literal("ranges"), + }), + z.strictObject({ + ...pageRequestShape, + ...streamSelectorShape, + facet: z.literal("final-sizes"), + }), +]); +export type QlogViewRequest = z.output; + +const traceSummarySchema = traceSchema + .omit({ + streams: true, + loss_declarations: true, + incomplete_fact_records: true, + }) + .extend({ + stream_count: z.number().int().nonnegative(), + loss_declaration_count: z.number().int().nonnegative(), + incomplete_fact_record_count: z.number().int().nonnegative(), + }); +const streamSummarySchema = streamSchema + .omit({ + contributions: true, + covered_ranges: true, + reported_final_sizes: true, + }) + .extend({ + contribution_count: z.number().int().nonnegative(), + covered_range_count: z.number().int().nonnegative(), + reported_final_size_count: z.number().int().nonnegative(), + }); +/** Each page advertises exactly the retained item contract for its facet. */ +export const qlogViewPageSchema = z.discriminatedUnion("facet", [ + z.strictObject({ + facet: z.literal("traces"), + items: z.array(traceSummarySchema), + }), + z.strictObject({ + facet: z.literal("streams"), + items: z.array(streamSummarySchema), + }), + z.strictObject({ + facet: z.literal("loss-declarations"), + items: traceSchema.shape.loss_declarations, + }), + z.strictObject({ + facet: z.literal("incomplete-records"), + items: traceSchema.shape.incomplete_fact_records, + }), + z.strictObject({ + facet: z.literal("contributions"), + items: streamSchema.shape.contributions, + }), + z.strictObject({ + facet: z.literal("ranges"), + items: streamSchema.shape.covered_ranges, + }), + z.strictObject({ + facet: z.literal("final-sizes"), + items: streamSchema.shape.reported_final_sizes, + }), +]); +type QlogViewPage = z.output; + +export const summarizeQlogCapture = (qlog: QlogCaptureFacts) => { + const { traces, ...context } = qlog; + return { + ...context, + trace_count: traces.length, + retained_stream_count: traces.reduce( + (sum, trace) => sum + trace.streams.length, + 0, + ), + loss_declaration_count: traces.reduce( + (sum, trace) => sum + trace.loss_declarations.length, + 0, + ), + incomplete_fact_record_count: traces.reduce( + (sum, trace) => sum + trace.incomplete_fact_records.length, + 0, + ), + }; +}; + +const summarizeTrace = ({ + streams, + loss_declarations, + incomplete_fact_records, + ...trace +}: QlogCaptureFacts["traces"][number]) => ({ + ...trace, + stream_count: streams.length, + loss_declaration_count: loss_declarations.length, + incomplete_fact_record_count: incomplete_fact_records.length, +}); +const summarizeStream = ({ + contributions, + covered_ranges, + reported_final_sizes, + ...stream +}: QlogCaptureFacts["traces"][number]["streams"][number]) => ({ + ...stream, + contribution_count: contributions.length, + covered_range_count: covered_ranges.length, + reported_final_size_count: reported_final_sizes.length, +}); + +/** Array offsets refer to retained facts, whose scope remains all recorded events. */ +export const projectQlogCapturePage = ( + qlog: QlogCaptureFacts, + view: QlogViewRequest, +): Result< + { + context: z.output; + page: QlogViewPage; + coverage: ReturnType; + }, + AnalysisError +> => { + const invalid = (field: string, message: string) => + err( + analysisViewInputError([ + { path: ["view", field], reason: "invalid_value", message }, + ]), + ); + const slice = (items: readonly T[]) => + items.slice(view.offset, view.offset + view.limit); + let page: QlogViewPage; + let total: number; + if (view.facet === "traces") { + total = qlog.traces.length; + page = { + facet: "traces", + items: slice(qlog.traces).map(summarizeTrace), + }; + } else { + const matches = qlog.traces.filter( + (trace) => trace.trace_index === view.trace_index, + ); + if (matches.length !== 1) + return invalid( + "trace_index", + "Select one unambiguous retained trace index.", + ); + const trace = matches[0]!; + if (view.facet === "streams") { + total = trace.streams.length; + page = { + facet: "streams", + items: slice(trace.streams).map(summarizeStream), + }; + } else if (view.facet === "loss-declarations") { + total = trace.loss_declarations.length; + page = { facet: view.facet, items: slice(trace.loss_declarations) }; + } else if (view.facet === "incomplete-records") { + total = trace.incomplete_fact_records.length; + page = { facet: view.facet, items: slice(trace.incomplete_fact_records) }; + } else { + const matches = trace.streams.filter( + (stream) => + stream.stream_id === view.stream_id && + stream.direction === view.direction, + ); + if (matches.length !== 1) + return invalid( + "stream_id", + "Select one unambiguous retained stream ID and direction within the trace.", + ); + const stream = matches[0]!; + if (view.facet === "contributions") { + total = stream.contributions.length; + page = { facet: view.facet, items: slice(stream.contributions) }; + } else if (view.facet === "ranges") { + total = stream.covered_ranges.length; + page = { facet: view.facet, items: slice(stream.covered_ranges) }; + } else { + total = stream.reported_final_sizes.length; + page = { facet: view.facet, items: slice(stream.reported_final_sizes) }; + } + } + } + const { traces: _traces, ...context } = qlog; + return ok({ + context, + page, + coverage: pageViewCoverage(view.offset, page.items.length, total), + }); +}; diff --git a/src/domain/analysisView/webNetworkCaptureView.ts b/src/domain/analysisView/webNetworkCaptureView.ts index 07495466c..b335e9241 100644 --- a/src/domain/analysisView/webNetworkCaptureView.ts +++ b/src/domain/analysisView/webNetworkCaptureView.ts @@ -7,7 +7,11 @@ import { webNetworkCaptureRecordSchema, webNetworkCaptureBinaryDescriptorSchema, webNetworkCaptureSchema, + webNetworkCaptureArtifactSchema, + webNetworkCaptureDecoderSchema, + WEB_NETWORK_CAPTURE_FORMATS, type WebNetworkCaptureRecord, + type WebNetworkCapture, } from "../webNetworkCapture.js"; import { analysisViewInputError, @@ -19,6 +23,11 @@ import { completeWithinViewCoverage, pageViewCoverage, } from "./analysisViewCoverage.js"; +import { + qlogSummarySchema, + summarizeQlogCapture, + projectQlogCapturePage, +} from "./qlogCaptureView.js"; /** Offsets page the parent's retained record order, preserving producer ordinals. */ export const captureViewRequestSchema = z.strictObject({ @@ -37,12 +46,13 @@ export const captureViewItemsSchema = z.array( /** Count/identity overview; bodies and the archive member inventory stay in the parent. */ export const captureSummarySchema = z.strictObject({ - artifact: webNetworkCaptureSchema.shape.artifact, - decoder: webNetworkCaptureSchema.shape.decoder, - format: webNetworkCaptureSchema.shape.format, + artifact: webNetworkCaptureArtifactSchema, + decoder: webNetworkCaptureDecoderSchema, + format: z.enum(WEB_NETWORK_CAPTURE_FORMATS), total_records: z.number().int().nonnegative(), retained_records: z.number().int().nonnegative(), runtime_attribution: z.literal("unknown"), + qlog: qlogSummarySchema.optional(), }); const object = ( @@ -104,22 +114,10 @@ const preview = (record: WebNetworkCaptureRecord) => { }; }; -/** Page immutable retained facts; no capture file, provider or network is consulted. */ -export const projectWebNetworkCaptureView = ( +/** Authenticate the retained report and its subject at one owning boundary. */ +const parseCaptureParent = ( parent: AnalysisViewParent, - view: AnalysisViewRequest, -): Result => { - if (view.kind !== "summary" && view.kind !== "capture") - return err( - analysisViewInputError([ - { - path: ["view"], - reason: "invalid_value", - message: - "Historical captures support summary, capture record pages, and admitted api-surface views.", - }, - ]), - ); +): Result => { const parsed = webNetworkCaptureSchema.safeParse(parent.normalizedResult); if (!parsed.success) return err( @@ -150,6 +148,32 @@ export const projectWebNetworkCaptureView = ( }, ]), ); + return ok(report); +}; + +/** Page immutable retained facts; no capture file, provider or network is consulted. */ +export const projectWebNetworkCaptureView = ( + parent: AnalysisViewParent, + view: AnalysisViewRequest, +): Result => { + if ( + view.kind !== "summary" && + view.kind !== "capture" && + view.kind !== "qlog" + ) + return err( + analysisViewInputError([ + { + path: ["view"], + reason: "invalid_value", + message: + "Historical captures support summary, capture record pages, qlog fact pages and admitted api-surface views.", + }, + ]), + ); + const parsed = parseCaptureParent(parent); + if (!parsed.ok) return parsed; + const report = parsed.value; const shared = { parent_evidence_id: parent.evidenceId, parent_operation: "inspect_web_network_capture" as const, @@ -159,7 +183,7 @@ export const projectWebNetworkCaptureView = ( ...new Set([ ...parent.limitations, ...report.limitations, - "Pages cover the records selected into the parent Evidence, in that selected order; total_records counts the original decoded capture.", + "Capture record pages cover the records selected into the parent Evidence, in that selected order; total_records counts the original decoded capture.", ]), ], unknowns: [ @@ -181,12 +205,40 @@ export const projectWebNetworkCaptureView = ( total_records: report.total_records, retained_records: report.records.length, runtime_attribution: report.runtime_attribution, + ...(report.format === "qlog" + ? { qlog: summarizeQlogCapture(report.qlog) } + : {}), }, coverage: completeWithinViewCoverage( report.records.length, report.records.length, ), }); + if (view.kind === "qlog") { + if (report.format !== "qlog") + return err( + analysisViewInputError([ + { + path: ["view", "kind"], + reason: "invalid_value", + message: "Qlog fact views require retained qlog Evidence.", + }, + ]), + ); + const projected = projectQlogCapturePage(report.qlog, view); + return projected.ok + ? ok({ + kind: "qlog", + view, + ...shared, + ...projected.value, + limitations: [ + ...shared.limitations, + "Qlog fact pages retain all-recorded-events-before-selection scope. Exhausting a page does not establish network completeness, body recovery, retransmission or actual loss.", + ], + }) + : projected; + } const records = report.records.slice(view.offset, view.offset + view.limit); return ok({ kind: "capture", diff --git a/src/domain/qlogCapture.ts b/src/domain/qlogCapture.ts new file mode 100644 index 000000000..575cf2a60 --- /dev/null +++ b/src/domain/qlogCapture.ts @@ -0,0 +1,76 @@ +import { z } from "zod"; + +const decimal = z.string().regex(/^(0|[1-9][0-9]*)$/u); +const unsignedFact = z.discriminatedUnion("state", [ + z.strictObject({ state: z.literal("reported"), value: decimal }), + z.strictObject({ + state: z.literal("unknown"), + reason: z.literal("omitted-null-or-redacted"), + }), +]); +const traceLocation = z.discriminatedUnion("kind", [ + z.strictObject({ kind: z.literal("json-pointer"), pointer: z.string() }), + z.strictObject({ + kind: z.literal("unknown"), + reason: z.literal("explicit-sensitive-value"), + }), +]); + +/** Exact, source-bound transport facts; coverage never establishes network completeness. */ +export const qlogCaptureFactsSchema = z.strictObject({ + profile: z.literal("aioquic-1.5.0-json-0.3"), + scope: z.literal("all-recorded-events-before-selection"), + body_bytes: z.literal("not-reported"), + clock: z.strictObject({ + unit: z.literal("milliseconds"), + domain: z.literal("unix-epoch"), + basis: z.literal("selected-aioquic-profile-not-producer-authentication"), + }), + traces: z.array( + z.strictObject({ + trace_index: z.number().int().nonnegative(), + location: traceLocation, + event_count: z.number().int().nonnegative(), + odcid: z.string().nullable(), + vantage_point: z.enum(["client", "server"]).nullable(), + incomplete_fact_records: z.array(z.number().int().nonnegative()), + loss_declarations: z.array( + z.strictObject({ + record_ordinal: z.number().int().nonnegative(), + confidence: z.literal("observed"), + packet_number: unsignedFact, + packet_type: z.string().nullable(), + packet_number_space: z + .enum(["initial", "handshake", "application"]) + .nullable(), + }), + ), + streams: z.array( + z.strictObject({ + stream_id: decimal, + direction: z.enum(["sent", "received"]), + confidence: z.literal("derived"), + scope: z.literal("observed-stream-frame-ranges"), + contributions: z.array( + z.strictObject({ + record_ordinal: z.number().int().nonnegative(), + frame_index: z.number().int().nonnegative(), + offset: decimal, + length: decimal, + fin: z.boolean().nullable(), + }), + ), + covered_ranges: z.array( + z.strictObject({ start: decimal, end: decimal }), + ), + unique_bytes: decimal, + frame_length_sum: decimal, + repeated_bytes: decimal, + reported_final_sizes: z.array(decimal), + }), + ), + }), + ), +}); + +export type QlogCaptureFacts = z.output; diff --git a/src/domain/webNetworkCapture.ts b/src/domain/webNetworkCapture.ts index ec98ca496..77752f8e5 100644 --- a/src/domain/webNetworkCapture.ts +++ b/src/domain/webNetworkCapture.ts @@ -1,6 +1,7 @@ import { z } from "zod"; import { jsonValueSchema } from "./jsonValue.js"; import { localPathStringSchema } from "./localPath.js"; +import { qlogCaptureFactsSchema } from "./qlogCapture.js"; /** Complete historical evidence budgets; exceeding a budget returns no partial capture. */ export const WEB_NETWORK_CAPTURE_LIMITS = { @@ -10,12 +11,17 @@ export const WEB_NETWORK_CAPTURE_LIMITS = { depth: 64, } as const; -export const WEB_NETWORK_CAPTURE_FORMATS = ["har", "mitmproxy", "saz"] as const; +export const WEB_NETWORK_CAPTURE_FORMATS = [ + "har", + "mitmproxy", + "saz", + "qlog", +] as const; /** Inspect an explicit retained capture without contacting its recorded destinations. */ export const inspectWebNetworkCaptureInputSchema = z.strictObject({ capture_path: localPathStringSchema.describe( - "Absolute local HAR, mitmproxy, or Fiddler SAZ capture path.", + "Absolute local HAR, mitmproxy, Fiddler SAZ or aioquic qlog capture path.", ), format: z.enum(WEB_NETWORK_CAPTURE_FORMATS), record_ordinals: z @@ -123,69 +129,85 @@ export const webNetworkCaptureRecordSchema = z.strictObject({ limitations: z.array(z.string()), }); -/** Decoded records retain unique producer ordinals within their declared coverage. */ -export const webNetworkCapturePayloadSchema = z - .strictObject({ - decoder: z.strictObject({ - id: z.string().min(1), - name: z.string().min(1), - version: z.string().min(1), - }), - container: z.strictObject({ - reported: jsonValueSchema, - numeric_literals: z.array(numberSchema), - redactions: z.array(redactionSchema), - records_pointer: pointerSchema.nullable(), - }), - total_records: z.number().int().nonnegative(), - records: z.array(webNetworkCaptureRecordSchema), - }) - .superRefine((capture, context) => { - if (capture.records.length > capture.total_records) { +/** The selected offline parser profile, independent of authenticated producer identity. */ +export const webNetworkCaptureDecoderSchema = z.strictObject({ + id: z.string().min(1), + name: z.string().min(1), + version: z.string().min(1), +}); + +/** Source coverage is shared by every offline decoder. */ +const payloadShape = { + decoder: webNetworkCaptureDecoderSchema, + container: z.strictObject({ + reported: jsonValueSchema, + numeric_literals: z.array(numberSchema), + redactions: z.array(redactionSchema), + records_pointer: pointerSchema.nullable(), + }), + total_records: z.number().int().nonnegative(), + records: z.array(webNetworkCaptureRecordSchema), +}; +const refineCoverage = ( + capture: { total_records: number; records: { ordinal: number }[] }, + context: z.RefinementCtx, +): void => { + const seen = new Set(); + for (const [index, record] of capture.records.entries()) { + if (record.ordinal >= capture.total_records || seen.has(record.ordinal)) { context.addIssue({ code: "custom", - path: ["records"], - message: "Retained record count exceeds the declared capture total.", + path: ["records", index, "ordinal"], + message: + "Retained ordinals must be unique and within the declared capture total.", }); return; } - const seen = new Set(); - for (const [index, record] of capture.records.entries()) { - if (record.ordinal >= capture.total_records || seen.has(record.ordinal)) { - context.addIssue({ - code: "custom", - path: ["records", index, "ordinal"], - message: - "Retained ordinals must be unique and within the declared capture total.", - }); - return; - } - seen.add(record.ordinal); - } - }); - -/** Historical capture inspection has file identity, producer provenance and explicit unknowns. */ -export const webNetworkCaptureSchema = - webNetworkCapturePayloadSchema.safeExtend({ - artifact: z.strictObject({ - path: z - .string() - .describe( - "Selected artifact path; empty when explicitly excluded from Evidence.", - ), - sha256: z - .string() - .regex(/^[0-9a-f]{64}$/) - .nullable() - .describe( - "Observed artifact SHA-256; null only when explicitly excluded by a sensitive declaration.", - ), - bytes: z.number().int().nonnegative(), + seen.add(record.ordinal); + } +}; +/** Strict alternatives prevent undefined-valued facets from entering JSON Evidence. */ +export const webNetworkCapturePayloadSchema = z + .union([ + z.strictObject({ ...payloadShape, qlog: qlogCaptureFactsSchema }), + z.strictObject(payloadShape), + ]) + .superRefine(refineCoverage); +/** Caller-selected file identity; explicit exclusions remain unknown. */ +export const webNetworkCaptureArtifactSchema = z.strictObject({ + path: z + .string() + .describe( + "Selected artifact path; empty when explicitly excluded from Evidence.", + ), + sha256: z + .string() + .regex(/^[0-9a-f]{64}$/) + .nullable() + .describe("Observed SHA-256; null only when explicitly excluded."), + bytes: z.number().int().nonnegative(), +}); +const artifactShape = { + artifact: webNetworkCaptureArtifactSchema, + runtime_attribution: z.literal("unknown"), + limitations: z.array(z.string()), +}; +/** Qlog transport facts occur exactly on the qlog alternative. */ +export const webNetworkCaptureSchema = z + .discriminatedUnion("format", [ + z.strictObject({ + ...payloadShape, + ...artifactShape, + format: z.enum(["har", "mitmproxy", "saz"]), }), - format: z.enum(WEB_NETWORK_CAPTURE_FORMATS), - runtime_attribution: z.literal("unknown"), - limitations: z.array(z.string()), - }); + z.strictObject({ + ...payloadShape, + ...artifactShape, + format: z.literal("qlog"), + qlog: qlogCaptureFactsSchema, + }), + ]) + .superRefine(refineCoverage); export type InspectWebNetworkCaptureInput = z.output< typeof inspectWebNetworkCaptureInputSchema diff --git a/src/domain/webNetworkCaptureCoordinates.ts b/src/domain/webNetworkCaptureCoordinates.ts index 617dd7607..a975c6b4a 100644 --- a/src/domain/webNetworkCaptureCoordinates.ts +++ b/src/domain/webNetworkCaptureCoordinates.ts @@ -53,8 +53,30 @@ export const excludeCaptureCoordinates = ( numeric_literals: sidecars(value.container.numeric_literals), redactions: sidecars(value.container.redactions), }; + const qlog = + value.format === "qlog" + ? { + ...value.qlog, + traces: value.qlog.traces.map((trace) => { + if ( + trace.location.kind !== "json-pointer" || + !sensitive(trace.location.pointer) + ) + return trace; + excluded = true; + return { + ...trace, + location: { + kind: "unknown" as const, + reason: "explicit-sensitive-value" as const, + }, + }; + }), + } + : undefined; return { ...value, + ...(qlog === undefined ? {} : { qlog }), container, records, limitations: excluded diff --git a/src/server/registerAnalysisViewTool.ts b/src/server/registerAnalysisViewTool.ts index ebd204edd..d427c0cb4 100644 --- a/src/server/registerAnalysisViewTool.ts +++ b/src/server/registerAnalysisViewTool.ts @@ -7,7 +7,10 @@ import { toolContract } from "../contracts/toolContracts.js"; import type { Logger } from "pino"; import { createEvidenceToolHandler } from "./contractToolHandler.js"; import type { WithAdmittedAnalysis } from "./analysisAdmission.js"; -import { toolRegistrationOptions } from "./toolRegistrationOptions.js"; +import { + toolRegistrationOptions, + applicationValidatedInputSchema, +} from "./toolRegistrationOptions.js"; /** Bind selected-view projection to its named contract and session Evidence owner. */ export const registerAnalysisViewTool = ( @@ -18,6 +21,7 @@ export const registerAnalysisViewTool = ( withAdmittedAnalysis?: WithAdmittedAnalysis, ): void => { const contract = toolContract("inspect_analysis_view"); + const registration = toolRegistrationOptions(contract); const handler = createEvidenceToolHandler( server, (input) => Promise.resolve(inspectAnalysisView(input, evidenceById)), @@ -27,7 +31,10 @@ export const registerAnalysisViewTool = ( ); server.registerTool( contract.name, - toolRegistrationOptions(contract), + { + ...registration, + inputSchema: applicationValidatedInputSchema(registration.inputSchema), + }, handler(contract), ); }; diff --git a/src/server/registerWebNetworkCaptureTool.ts b/src/server/registerWebNetworkCaptureTool.ts index 8b1203745..3e9759504 100644 --- a/src/server/registerWebNetworkCaptureTool.ts +++ b/src/server/registerWebNetworkCaptureTool.ts @@ -1,5 +1,4 @@ import type { EvidenceMcpServer } from "./EvidenceMcpServer.js"; -import type { StandardSchemaWithJSON } from "@modelcontextprotocol/server"; import type { EvidenceLookup } from "../application/EvidenceInputResolver.js"; import { requireAnalysisViewRetention, @@ -15,7 +14,10 @@ import { toolContract } from "../contracts/toolContracts.js"; import type { Logger } from "pino"; import type { WithAdmittedAnalysis } from "./analysisAdmission.js"; import { createEvidenceToolHandler } from "./contractToolHandler.js"; -import { toolRegistrationOptions } from "./toolRegistrationOptions.js"; +import { + toolRegistrationOptions, + applicationValidatedInputSchema, +} from "./toolRegistrationOptions.js"; /** Bind historical inspection to its named contract and caller-owned Evidence writer. */ export const registerWebNetworkCaptureTool = ( @@ -31,13 +33,7 @@ export const registerWebNetworkCaptureTool = ( // Keep the exact advertised contract; the shared service validates raw input // before effects so caller-declared sensitivity also covers invalid arguments. // SDK pre-validation otherwise emits unprojected issue paths as plain text. - const inputSchema: StandardSchemaWithJSON = { - "~standard": { - ...registration.inputSchema["~standard"], - vendor: "rea-application", - validate: (value: unknown) => ({ value }), - }, - }; + const inputSchema = applicationValidatedInputSchema(registration.inputSchema); const handler = createEvidenceToolHandler( server, async (rawInput, options) => { diff --git a/src/server/toolRegistrationOptions.ts b/src/server/toolRegistrationOptions.ts index 4615971d4..c943be3f2 100644 --- a/src/server/toolRegistrationOptions.ts +++ b/src/server/toolRegistrationOptions.ts @@ -1,9 +1,21 @@ import type { ToolContract } from "../contracts/toolContractTypes.js"; +import type { StandardSchemaWithJSON } from "@modelcontextprotocol/server"; import { toolInputSchemaWithMetadata, toolOutputSchemaWithMetadata, } from "../contracts/toolSchemaMetadata.js"; +/** Preserve advertised contracts while the application owns structured input failures. */ +export const applicationValidatedInputSchema = ( + schema: StandardSchemaWithJSON, +): StandardSchemaWithJSON => ({ + "~standard": { + ...schema["~standard"], + vendor: "rea-application", + validate: (value: unknown) => ({ value }), + }, +}); + /** Project the canonical Zod contracts directly into SDK registration. */ export const toolRegistrationOptions = ( contract: Contract, diff --git a/tests/acceptance/tools/qlogCapture.test.ts b/tests/acceptance/tools/qlogCapture.test.ts new file mode 100644 index 000000000..be67c3a34 --- /dev/null +++ b/tests/acceptance/tools/qlogCapture.test.ts @@ -0,0 +1,457 @@ +import { execFile } from "node:child_process"; +import { writeFile, truncate, rm } from "node:fs/promises"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { promisify } from "node:util"; +import { expect, it as runtimeTest } from "vitest"; +import { + NODE_CAPTURE_HEAP_ENFORCED, + NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES, +} from "../../../src/browser/history/CaptureRelease.js"; +import { webNetworkCaptureSummaryViewResultSchema } from "../../../src/domain/analysisView/analysisView.js"; +import { parseEvidence } from "../../../src/domain/evidence.js"; +import { + webNetworkCaptureSchema, + WEB_NETWORK_CAPTURE_LIMITS, +} from "../../../src/domain/webNetworkCapture.js"; +import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +const execute = promisify(execFile); +const hasMemoryBound = + NODE_CAPTURE_HEAP_ENFORCED || + NODE_CAPTURE_PROCESS_MEMORY_LIMIT_BYTES !== null; +const it = runtimeTest.skipIf(!hasMemoryBound); +const cli = fileURLToPath(new URL("../../../scripts/rea.mjs", import.meta.url)); +const event = { + name: "transport:packet_sent", + time: 1791819895000.25, + data: { + header: { packet_type: "1RTT", packet_number: 5 }, + frames: [ + { frame_type: "stream", stream_id: 0, offset: 10, length: 5, fin: true }, + ], + }, +}; +const qlog = (events: unknown[]) => + JSON.stringify({ + qlog_format: "JSON", + qlog_version: "0.3", + traces: [ + { + vantage_point: { name: "aioquic", type: "server" }, + common_fields: { ODCID: "trace-local" }, + events, + }, + ], + }); + +const assertAdditionalQlogFacets = async ( + call: Awaited>["call"], + retained: { kind: string; evidence_id: string }, +): Promise => { + for (const [view, expected] of [ + [ + { kind: "qlog", facet: "traces", offset: 1, limit: 1 }, + [{ trace_index: 1, event_count: 1, stream_count: 1 }], + ], + [ + { kind: "qlog", facet: "loss-declarations", trace_index: 0 }, + [{ record_ordinal: 1, packet_number: { state: "reported", value: "5" } }], + ], + [{ kind: "qlog", facet: "incomplete-records", trace_index: 0 }, []], + [ + { + kind: "qlog", + facet: "ranges", + trace_index: 0, + stream_id: "0", + direction: "sent", + }, + [{ start: "10", end: "15" }], + ], + [ + { + kind: "qlog", + facet: "final-sizes", + trace_index: 0, + stream_id: "0", + direction: "sent", + }, + ["15"], + ], + [{ kind: "qlog", facet: "traces", offset: 99 }, []], + ] as const) { + const response = await call("inspect_analysis_view", { + source: retained, + view, + }); + expect(response.isError, JSON.stringify(response)).not.toBe(true); + expect( + parseEvidence(response.structuredContent).normalized_result, + ).toMatchObject({ + page: { items: expected }, + coverage: { exhausted: true }, + }); + } +}; + +it("pages original trace-scoped facts after source deletion without conflating selected events or identical stream IDs", async () => { + const root = await createTestTempDirectory("rea-qlog-pages-"); + const path = join(root, "input.qlog"); + const source = JSON.parse( + qlog([ + event, + { + name: "recovery:packet_lost", + time: 1791819895001, + data: { packet_number: 5, type: "1RTT" }, + }, + ]), + ); + source.traces.push({ + ...source.traces[0], + events: [ + { + ...event, + data: { + frames: [ + { + frame_type: "stream", + stream_id: 0, + offset: 100, + length: 3, + fin: true, + }, + ], + }, + }, + ], + }); + await writeFile(path, JSON.stringify(source)); + const { call } = await connectLocalToolsMcp(); + const response = await call("inspect_web_network_capture", { + capture_path: path, + format: "qlog", + detail: "summary", + record_ordinals: [2], + }); + expect(response.isError, JSON.stringify(response)).not.toBe(true); + const summary = webNetworkCaptureSummaryViewResultSchema.parse( + parseEvidence(response.structuredContent).normalized_result, + ); + expect(summary).toMatchObject({ + summary: { + total_records: 3, + retained_records: 1, + qlog: { + trace_count: 2, + retained_stream_count: 2, + loss_declaration_count: 1, + }, + }, + }); + expect(JSON.stringify(summary)).not.toContain('"contributions"'); + if ( + !("parent_evidence_id" in summary) || + typeof summary.parent_evidence_id !== "string" + ) + throw new Error("Missing retained parent"); + const retained = { + kind: "retained-evidence", + evidence_id: summary.parent_evidence_id, + }; + const parent = parseEvidence( + ( + await call("inspect_web_network_capture", { + capture_path: path, + format: "qlog", + record_ordinals: [2], + }) + ).structuredContent, + ); + expect(parent.evidence_id).toBe(summary.parent_evidence_id); + const portablePath = join(root, "parent.json"); + await rm(path); + const pages = []; + for (const trace_index of [0, 1]) { + const page = await call("inspect_analysis_view", { + source: retained, + view: { kind: "qlog", facet: "streams", trace_index, limit: 1 }, + }); + expect(page.isError, JSON.stringify(page)).not.toBe(true); + pages.push(parseEvidence(page.structuredContent).normalized_result); + } + expect(pages[0]).toMatchObject({ + page: { items: [{ stream_id: "0", unique_bytes: "5" }] }, + context: { scope: "all-recorded-events-before-selection" }, + }); + expect(pages[1]).toMatchObject({ + page: { items: [{ stream_id: "0", unique_bytes: "3" }] }, + }); + await assertAdditionalQlogFacets(call, retained); + const view = { + kind: "qlog", + facet: "contributions", + trace_index: 0, + stream_id: "0", + direction: "sent", + offset: 0, + limit: 1, + }; + const contribution = parseEvidence( + (await call("inspect_analysis_view", { source: retained, view })) + .structuredContent, + ); + expect(contribution.normalized_result).toMatchObject({ + page: { + items: [{ record_ordinal: 0, frame_index: 0, offset: "10", length: "5" }], + }, + coverage: { total: 1, exhausted: true }, + }); + await writeFile( + portablePath, + JSON.stringify({ source: { kind: "inline", evidence: parent }, view }), + ); + const cliResult = await execute(process.execPath, [ + cli, + "inspect-analysis-view", + portablePath, + "--json", + ]); + expect(parseEvidence(JSON.parse(cliResult.stdout)).normalized_result).toEqual( + contribution.normalized_result, + ); + for (const invalid of [ + { kind: "qlog", facet: "streams" }, + { kind: "qlog", facet: "streams", trace_index: 99 }, + { ...view, stream_id: "99" }, + { kind: "qlog", facet: "traces", trace_index: 0 }, + ]) + expect( + parseMcpToolError( + await call("inspect_analysis_view", { + source: retained, + view: invalid, + }), + ), + ).toMatchObject({ error: { code: "invalid_request" } }); +}, 60_000); + +runtimeTest.runIf(!hasMemoryBound)( + "refuses qlog when this runtime cannot enforce a decoder memory bound", + async () => { + const { call } = await connectLocalToolsMcp(); + const root = await createTestTempDirectory("rea-qlog-unbounded-"); + const path = join(root, "input.qlog"); + await writeFile(path, qlog([event])); + expect( + parseMcpToolError( + await call("inspect_web_network_capture", { + capture_path: path, + format: "qlog", + }), + ), + ).toMatchObject({ error: { code: "capability_unavailable" } }); + }, +); + +it("runs built CLI and real stdio SDK with exact evidence parity, selection, retained references and strict schemas", async () => { + const root = await createTestTempDirectory("rea-qlog-public-"); + const path = join(root, "input.qlog"); + await writeFile( + path, + qlog([ + event, + { + name: "recovery:packet_lost", + time: 1791819895001.125, + data: { packet_number: 5, type: "1RTT" }, + }, + ]), + ); + const { call, client } = await connectLocalToolsMcp(); + const input = { capture_path: path, format: "qlog", record_ordinals: [1, 0] }; + const response = await call("inspect_web_network_capture", input); + expect(response.isError, JSON.stringify(response)).not.toBe(true); + const mcp = parseEvidence(response.structuredContent); + const result = await execute(process.execPath, [ + cli, + "inspect-web-network-capture", + path, + "qlog", + "--record", + "1", + "--record", + "0", + "--json", + ]); + const cliEvidence = parseEvidence(JSON.parse(result.stdout)); + expect(cliEvidence.evidence_id).toBe(mcp.evidence_id); + expect(cliEvidence.normalized_result).toEqual(mcp.normalized_result); + expect(mcp.analysis_profile?.parameters).toMatchObject({ + qlog_profile: "aioquic-1.5.0-json-0.3", + }); + expect(mcp.authority).toBe("historical-reference"); + const decoded = webNetworkCaptureSchema.parse(mcp.normalized_result); + expect(decoded.records.map((record) => record.ordinal)).toEqual([1, 0]); + if (decoded.format !== "qlog") throw new Error("qlog format required"); + expect(decoded.qlog.traces[0]?.streams[0]).toMatchObject({ + unique_bytes: "5", + covered_ranges: [{ start: "10", end: "15" }], + }); + const retained = await call("inspect_analysis_view", { + source: { kind: "retained-evidence", evidence_id: mcp.evidence_id }, + view: { kind: "api-surface" }, + }); + // An unsupported view over an existing retained parent proves lookup without pretending qlog is HAR. + expect(parseMcpToolError(retained)).toMatchObject({ + error: { code: "unsupported_target" }, + }); + await client.ping(); + for (const record_ordinals of [[2], [0, 0]]) { + expect( + parseMcpToolError( + await call("inspect_web_network_capture", { + ...input, + record_ordinals, + }), + ), + ).toMatchObject({ error: { code: "invalid_request" } }); + } + const hidden = await call("inspect_web_network_capture", { + capture_path: path, + format: "qlog", + sensitive_values: ["trace-local", "/traces/0", "input.qlog"], + }); + expect(JSON.stringify(hidden)).not.toContain("trace-local"); + expect(JSON.stringify(hidden)).not.toContain("/traces/0"); + expect(JSON.stringify(hidden)).not.toContain("input.qlog"); + const hiddenEvidence = parseEvidence(hidden.structuredContent); + expect( + webNetworkCaptureSchema.parse(hiddenEvidence.normalized_result).artifact + .path, + ).toBe(""); + const invalid = await call("inspect_web_network_capture", { + ...input, + mysecret: true, + sensitive_values: ["mysecret"], + }); + expect(parseMcpToolError(invalid)).toMatchObject({ + error: { code: "invalid_request" }, + }); + expect(JSON.stringify(invalid)).not.toContain("mysecret"); +}, 60_000); + +it("fails whole-capture validation before selection and survives malformed, unsupported and oversized qlog", async () => { + const root = await createTestTempDirectory("rea-qlog-errors-"); + const path = join(root, "input.qlog"); + const { call, client } = await connectLocalToolsMcp(); + for (const [text, code] of [ + [qlog([event, { ...event, time: "invalid" }]), "invalid_request"], + [qlog([event]).slice(0, -1), "invalid_request"], + [ + qlog([event]).replace('"qlog_version":"0.3"', '"qlog_version":"0.4"'), + "capability_unavailable", + ], + [ + qlog([ + { + ...event, + extension: JSON.parse("[".repeat(65) + "0" + "]".repeat(65)), + }, + ]), + "invalid_request", + ], + ] as const) { + await writeFile(path, text); + expect( + parseMcpToolError( + await call("inspect_web_network_capture", { + capture_path: path, + format: "qlog", + record_ordinals: [0], + }), + ), + ).toMatchObject({ error: { code } }); + const failed: unknown = await execute(process.execPath, [ + cli, + "inspect-web-network-capture", + path, + "qlog", + "--json", + ]).catch((cause: unknown) => cause); + expect(failed).toMatchObject({ + code: 1, + stdout: expect.stringContaining(`"code":"${code}"`), + }); + } + await truncate(path, WEB_NETWORK_CAPTURE_LIMITS.inputBytes + 1); + expect( + parseMcpToolError( + await call("inspect_web_network_capture", { + capture_path: path, + format: "qlog", + }), + ), + ).toMatchObject({ error: { code: "invalid_request" } }); + await writeFile(path, qlog([event])); + await client.ping(); + expect( + ( + await call("inspect_web_network_capture", { + capture_path: path, + format: "qlog", + }) + ).isError, + ).not.toBe(true); +}, 60_000); + +runtimeTest.skipIf(process.versions.bun !== undefined || !hasMemoryBound)( + "contains actual qlog decoder heap exhaustion and keeps MCP usable", + async () => { + const root = await createTestTempDirectory("rea-qlog-capacity-"); + const path = join(root, "dense.qlog"); + const dense = qlog(Array(100_000).fill(event)); + expect(Buffer.byteLength(dense)).toBeLessThan( + WEB_NETWORK_CAPTURE_LIMITS.inputBytes, + ); + await writeFile(path, dense); + const { call, client } = await connectLocalToolsMcp(); + const response = await call("inspect_web_network_capture", { + capture_path: path, + format: "qlog", + record_ordinals: [0], + }); + expect(parseMcpToolError(response)).toMatchObject({ + error: { + code: "resource_constraint", + details: { + resource: "memory", + reported_limits: { old_generation_heap_mib: 192 }, + }, + }, + }); + const failed: unknown = await execute( + process.execPath, + [cli, "inspect-web-network-capture", path, "qlog", "--json"], + { timeout: 40_000, maxBuffer: 1024 * 1024 }, + ).catch((cause: unknown) => cause); + expect(failed).toMatchObject({ + code: 1, + stdout: expect.stringContaining('"code":"resource_constraint"'), + }); + await writeFile(path, qlog([event])); + await client.ping(); + expect( + ( + await call("inspect_web_network_capture", { + capture_path: path, + format: "qlog", + }) + ).isError, + ).not.toBe(true); + }, + 60_000, +); From 23de4f77c74dd299d45cab880c36321d802b21fe Mon Sep 17 00:00:00 2001 From: Ysopwnable <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 11:39:30 +0800 Subject: [PATCH 004/159] perf(flutter): add opt-in retained AOT summary and string pages Add explicit MCP Dart AOT summary delivery and exact retained string-pool pages while preserving complete defaults, CLI behavior, ABI selection, original Evidence identity, partial coverage and lower bounds. Real LocalSend SDK replies shrink 92.05% with approximately unchanged SDK time. Two actual Codex pairs preserve all selected facts and reduce total input 48.75% / 42.38%, uncached input 33.63% / 42.52%, and task time 23.95% / 30.30%; all client errors/repeated calls remain counted. Preserve old portable Evidence and the real CLI complete result after rebasing onto SAZ/qlog main. Curated results keep measured and integration runtime pins separate; raw artifacts stay outside Git. Refs #2016. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- docs/flutter-analysis.md | 20 ++ experiments/analysis-efficiency/README.md | 8 + experiments/analysis-efficiency/corpus.json | 15 + .../flutter-aot-summary.md | 160 ++++++++++ .../analysis-efficiency/latest-main-audit.md | 75 +++++ experiments/analysis-efficiency/measure.mjs | 48 +++ .../questions/flutter-aot.txt | 1 + .../analysis-efficiency/review-model.py | 5 +- src/application/flutter/DartAotInspection.ts | 40 +++ src/contracts/analysisViewToolContracts.ts | 2 +- src/contracts/flutter/flutterToolContracts.ts | 18 +- src/domain/analysisView/analysisView.ts | 31 +- src/domain/analysisView/dartAotView.ts | 123 ++++++++ src/domain/flutter/flutterBuildAnalysis.ts | 6 + src/server/createServer.ts | 1 + src/server/registerFlutterTools.ts | 37 ++- tests/acceptance/flutterAnalysis.test.ts | 11 +- tests/boundary/mcp/dartAotSummaryMcp.test.ts | 285 ++++++++++++++++++ 18 files changed, 867 insertions(+), 19 deletions(-) create mode 100644 experiments/analysis-efficiency/flutter-aot-summary.md create mode 100644 experiments/analysis-efficiency/latest-main-audit.md create mode 100644 experiments/analysis-efficiency/questions/flutter-aot.txt create mode 100644 src/application/flutter/DartAotInspection.ts create mode 100644 src/domain/analysisView/dartAotView.ts create mode 100644 tests/boundary/mcp/dartAotSummaryMcp.test.ts diff --git a/docs/flutter-analysis.md b/docs/flutter-analysis.md index 75c23dd66..ebf0ccd84 100644 --- a/docs/flutter-analysis.md +++ b/docs/flutter-analysis.md @@ -25,6 +25,26 @@ The equivalent MCP methods are `identify_flutter_build` (with `path`) and arm64-v8a, then x86_64, then x86, then armeabi-v7a, then the remaining ABIs alphabetically). +## Optional retained AOT delivery + +MCP `inspect_dart_aot` keeps complete output when `detail` is omitted or +`"complete"`. Explicit `detail: "summary"` returns the selected target/ABI/library +identity, all snapshot headers, structural coverage, string-pool coverage and +original lower-bound counts without the string lists. It retains complete source +Evidence and returns a distinct linked `inspect_analysis_view` summary. Analysis +and the caller's ABI choice are unchanged; one-shot CLI output stays complete. + +Use `normalized_result.parent_evidence_id` as the retained source for +`inspect_analysis_view`, with `view: {kind: "dart-aot", collection: "package_uris", +offset: 0, limit: 64}`. Collections are `package_uris`, `dart_uris` and +`class_like_tokens`. Pages preserve the original strings and order, selected ABI, +library identity and source coverage. `returned_counts` describe producer-returned +rows; counts ending in `_at_least` remain lower bounds. Exhausting a page sequence +does not make a partial pool complete or turn byte-pattern tokens into typed +classes/functions. Views do not reopen the APK. Original Evidence remains +exportable under its existing session lifetime. Summary requires a retaining +server; older servers keep the complete request after schema discovery. + ## What the AOT inspection returns `inspect_dart_aot` reads one ABI's `libapp.so` in full and reports: diff --git a/experiments/analysis-efficiency/README.md b/experiments/analysis-efficiency/README.md index 8afef3a66..b028953ce 100644 --- a/experiments/analysis-efficiency/README.md +++ b/experiments/analysis-efficiency/README.md @@ -7,6 +7,14 @@ JSON measurements belong outside the checkout. This document keeps the measured findings, verification scope and revision references; the scripts and `corpus.json` keep the reproduction inputs. Generated `results/` is ignored. +Follow-up: [latest-main capability audit](latest-main-audit.md) records the +Flutter, SQLite, TShark and sigrok priorities and the first shipped Dart AOT +baseline, preserving existing controls before introducing another selector. + +Follow-up: [Dart AOT summary and retained string pools](flutter-aot-summary.md) +records real LocalSend delivery pairs and old Evidence continuation, preserving +ABI selection, snapshot identity, original lower bounds and partial coverage. + Follow-up: [native PE/DOS entry views and real model verification](native-entry-views.md) records the post-merge native continuation, including a small-function case where four selected replies cost more bytes than the complete dossier. diff --git a/experiments/analysis-efficiency/corpus.json b/experiments/analysis-efficiency/corpus.json index 540e88cdd..3b0f5c574 100644 --- a/experiments/analysis-efficiency/corpus.json +++ b/experiments/analysis-efficiency/corpus.json @@ -235,6 +235,21 @@ }, "immutable_scope": "Published Hardhat artifact does not record an immutable-reference map; absence is unknown, not proof of no immutables. Runtime template bytes remain exact." } + }, + { + "id": "localsend-arm64.apk", + "file": "LocalSend-1.18.2-android-arm64v8.apk", + "bytes": 46558706, + "sha256": "82ec3568fba2aa5295b9aae8b76f701d7a4703d86b9f8bad749472038fbaeab3", + "provenance": { + "project": "LocalSend", + "version": "1.18.2", + "upstreamFile": "LocalSend-1.18.2-android-arm64v8.apk", + "url": "https://github.com/localsend/localsend/releases/download/v1.18.2/LocalSend-1.18.2-android-arm64v8.apk", + "transport_sha256": "82ec3568fba2aa5295b9aae8b76f701d7a4703d86b9f8bad749472038fbaeab3", + "transport_bytes": 46558706, + "derivation": "Official arm64 release APK, compared with GitHub release asset digest; byte-only analysis without execution" + } } ] } diff --git a/experiments/analysis-efficiency/flutter-aot-summary.md b/experiments/analysis-efficiency/flutter-aot-summary.md new file mode 100644 index 000000000..8893cd917 --- /dev/null +++ b/experiments/analysis-efficiency/flutter-aot-summary.md @@ -0,0 +1,160 @@ +# Dart AOT summary and retained string pools + +Continues [#2016](https://github.com/morluto/rea/issues/2016) and +[#2455](https://github.com/morluto/rea/pull/2455). The selected analyst question +asks for shipped LocalSend target/ABI/library identity, every nullable snapshot +observation, string-pool lower bounds and returned counts, and original coverage +and limitations. It does not require individual observed strings. + +## Compatibility and ownership + +MCP `inspect_dart_aot` accepts explicit `detail: "summary"`. Omitted and complete +requests preserve the complete result and original provider inputs. The CLI +remains complete. ABI omission and explicit ABI selection retain their original +meaning; the provider never receives the delivery field. Missing retention, +invalid detail and pre-cancellation fail before provider work. + +The existing application workflow retains the complete original Evidence, then +returns a distinct derived summary linked to its parent ID. It preserves target +and libapp digests, every snapshot section and nullable fact, original pool and +overall coverage, lower-bound counts and limitations. `returned_counts` reports +the lengths of the producer-returned arrays; it does not upgrade lower bounds +to a complete snapshot inventory. The analysis and original source retention +are unchanged. No new engine, persistent cache or plugin SDK is introduced. + +Read exact retained strings using `inspect_analysis_view`, the parent ID, and +`view: {kind: "dart-aot", collection: "package_uris", offset: 0, limit: 64}`. +Other collections are `dart_uris` and `class_like_tokens`. Follow the returned +next offset. Pages preserve source order, ABI/library identity and partial pool +coverage without opening the APK again. Page exhaustion describes retained +strings, not the entire snapshot. Strings remain byte-pattern observations, +rather than recovered typed class/function semantics. Missing historical +subjects remain explicitly unknown; conflicting recorded target digests fail. + +## Frozen runtime and real SDK pairs + +The official LocalSend 1.18.2 arm64 APK acquisition, size and SHA-256 are pinned +in `corpus.json`. No APK installation, target execution or version-dataset +lookup occurred. Source runtime: `811034a7283c9751a3ecbff33e8b2f28bec55793`; +compiled dist/bridge file-table SHA-256: +`dabbb63536c5862be839ecdc0f3eaad05b29c2a3816f57041528d30f754311be`. +The previous runtime is `28c84e78cf2fbfde4aceed345b773c59eb889f68`, digest +`5332dda13981f043e146b3601ebec1f380d9f698280ca53d75786b638a912e11`. +Its production sources were verified byte-equal to main f254 at acquisition. +Subsequent test-only changes and main's SAZ addition do not relabel these pins. + +Node is 24.18.0 and the SDK client/server are 2.3.1. Each workflow starts a fresh +server and validates the selected live advertised schema. Artifact bytes, +provider environment/search-path fingerprints, driver and locked dependencies +stay equal. Host page caches are not flushed. Heavy tasks run serially under +the unchanged CPU, memory, task and host guards. + +| Trial | AOT reply bytes | SDK workflow seconds | Payload CPU seconds | Kernel peak bytes | +| -------------- | --------------: | -------------------: | ------------------: | ----------------: | +| sdk-complete-1 | 151,341 | 5.139 | 6.819 | 691,441,664 | +| sdk-summary-1 | 12,037 | 5.219 | 6.608 | 639,430,656 | +| sdk-summary-2 | 12,037 | 5.064 | 6.405 | 640,868,352 | +| sdk-complete-2 | 151,341 | 5.128 | 6.468 | 633,614,336 | + +Both pairs use one successful public tool call and pass settings/fact comparison, +including all snapshot fields, coverage, original limitations and all counts. +Reply bytes fall 92.05%. Whole SDK time stays approximately 5.1 seconds; these +trials do not establish engine speed, CPU or memory savings. Payload CPU includes +driver/catalog work beyond the SDK workflow timer. Catalog JSON grows from +2,506,213 bytes on the previous runtime to 2,514,896 (about 0.35%). + +A fresh old-runtime complete request passes comparison against candidate +complete, including identical 151,341-byte replies. This is scoped compatibility +evidence, not a guarantee for every historical release or third-party client. + +## Real old Evidence continuation + +Fifteen successful public MCP calls import genuine old-runtime AOT Evidence, +project retained summary, reconstruct all 500 package URIs, 216 Dart URIs and +2,000 class-like tokens, compare inline portable summary, and export the original +record. The APK is absent during all these operations. Original string order, +lower bounds, partial coverage, nullable snapshot facts, subject and parent ID +remain intact; the original exported record is deeply equal to its source. + +The first private verification attempt used a nonexistent receipt filename and +failed before import or artifact relocation. That driver error is retained; +the corrected attempt selects the fresh old-runtime record. It is not a REA +defect or a successful migration trial. + +## Actual Codex task outcomes + +Four serial trials use the same frozen candidate in complete/summary/summary/ +complete order. Requested configuration is Codex CLI 0.161.0, `gpt-5.6-terra` +high, one exact enabled tool (`inspect_dart_aot`), and identical optional owned +context capture/discovery guidance. This requested alias does not certify the +server-resolved model. Temporary configuration points to the modified frozen +REA MCP; all copied credential profiles are removed and target bytes remain +unchanged. The source question is fixed; equal treatment instructions choose +detail and explain the summary carrier and parent reference. + +| Trial | Input | Cached | Uncached | Output | Seconds | MCP attempts | Failed attempts | Facts | +| ---------------- | -----: | -----: | -------: | -----: | ------: | -----------: | --------------: | ----- | +| model-complete-1 | 74,641 | 50,176 | 24,465 | 2,765 | 116.898 | 3 | 1 | pass | +| model-summary-1 | 38,253 | 22,016 | 16,237 | 1,449 | 88.904 | 1 | 0 | pass | +| model-summary-2 | 38,363 | 22,016 | 16,347 | 1,515 | 55.190 | 1 | 0 | pass | +| model-complete-2 | 66,585 | 38,144 | 28,441 | 2,217 | 79.180 | 2 | 0 | pass | + +Both pairs pass checks for every selected producer fact, exact nullable +observations, target digest, observed response Evidence IDs, original limitations, +artifact preservation and profile cleanup. The grader compares the reported +facts with producer observations; it is not an independent Dart deserializer. +Total input falls 48.75% / 42.38%, uncached input 33.63% / 42.52%, reported output +47.59% / 31.66%, and whole-task time 23.95% / 30.30%. Usage is the final aggregate +reported by the actual CLI, one usage record per trial; cached tokens are not +subtracted from total input. Reported output includes reasoning output (1,262, +321, 383 and 709 tokens respectively). + +The first complete trial initially supplied `artifact_path` instead of the +advertised `path`, corrected it, then repeated a successful complete call. +The second complete trial made two successful calls. Both summary trials made +one successful call. All attempts, discovery and correction costs remain +included. The paired whole-task improvement therefore includes actual model +decisions and the convenience of explicit returned counts; it does not isolate +serialization as the sole cause or establish engine acceleration. Two pairs on +one app/question/model configuration support this scope, not a universal gain. + +Payload CPU seconds / kernel peak bytes are respectively 12.556 / 876,937,216; +9.213 / 785,272,832; 9.108 / 793,939,968; and 13.164 / 803,209,216. Model payload +observations verify owned cleanup and zero phase limit events, with no guard +aborts. Whole-scope final counters remain unknown, so every strict all-selected +resource grade remains false even though the separate factual grade passes. + +## Resource and verification scope + +SDK and migration payload-exit observations verify owned cleanup and zero +observed memory/pids limit events, with peaks below 692 MB. Missing whole-scope +terminal counters remain unknown; the strict whole-resource grader stays false. +One-CPU quotas are experiment controls rather than CPU-efficiency claims. + +Production-provider fixture tests cover explicit ABI, old complete defaults, +actual SDK-converted schemas, exact pages after APK removal, portable records, +nullable/partial facts, malformed inputs, target digest conflicts and early +refusals. CLI/MCP acceptance covers original report equality, duplicate-member +refusals and unsupported snapshot kinds. Fixture successes are distinguished +from the real LocalSend SDK/migration workflows. Other Flutter apps, architectures, +SDK layouts, typed class recovery and runtime behavior remain unverified here. +Raw replies, binaries, model traces and temporary credentials stay outside Git. + +## Integration with the newer main + +The rebased verification runtime is `63984ce0898bd7997e6d9b138e6d96d0a24f2b09` +on main `a2d8b94dd50dab8d966239dd63ab84718aa8b4b4`, with compiled file-table +SHA-256 `19c8ccdd3e99bb46a5c54f4edd8c6b2d2eb4f3a4870b3c129061d38145382b68`. +Targeted Flutter/SAZ/shared-view integration passes 35 tests; one opposite +runtime-capacity branch is skipped because this host enforces decoder memory. +Documentation checks and two README conformance tests pass. + +Fresh real complete/summary SDK checks on this runtime match the earlier selected +facts and settings, with unchanged 151,341 / 12,037-byte AOT replies. Fifteen +old-Evidence migration calls again preserve every original string and record +with the source APK absent. The real one-shot CLI complete result also matches +the previous-runtime normalized report, subject, parameters and original ID. +An initial CLI-only controller check lacked an owned resource receipt; it is +retained and excluded from resource/performance claims, then repeated in the +bounded owned scope. These checks establish integration, rather than relabeling +the earlier paired model/SDK measurements as the rebased runtime. diff --git a/experiments/analysis-efficiency/latest-main-audit.md b/experiments/analysis-efficiency/latest-main-audit.md new file mode 100644 index 000000000..8f767fb64 --- /dev/null +++ b/experiments/analysis-efficiency/latest-main-audit.md @@ -0,0 +1,75 @@ +# Latest-main efficiency continuation + +Continues [#2016](https://github.com/morluto/rea/issues/2016). The 2026-10-13 +fetch resolved main to `f254f5ac2b804b353a711264bb178f43fdf07867`, the same +revision as the preceding Android conclusion. Newer capabilities below were +added during that earlier cycle; they are additional measurement scopes, not +new commits after that conclusion. + +| Capability | Existing controls | Next evidence needed | +| ----------------------------- | ---------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | +| Flutter / Dart AOT (#1582) | Explicit ABI, bounded string observations, partial coverage | Real shipped snapshot/count question; opt-in delivery only if unrelated strings dominate | +| SQLite (#2271) | Compact inventory, selected table/columns, row limit | Real application database and model workflow before adding another selector | +| TShark (#2318) | Fields, filter, decode-as, pass, default 50 matching rows, paging, explicit null for all | Real public capture and repeated task cost; paging repeats dissection | +| sigrok (#2339) | Decoder/channels/annotation selection, default 50 display lines, paging | Real public capture; display-line count is not annotation count and paging repeats decoding | +| Retained unknown head results | Existing retained paging | Reuse the existing view rather than duplicate its implementation | + +Attached IDA, Frida and other live runtime workflows involve separate authority +and lifecycle questions and are outside this static parsing continuation. +Plugin SDKs and cross-session engine caches remain deferred until an observed +need and benefit justify them. + +## First selected scope: Dart AOT + +The official LocalSend 1.18.2 arm64 APK is pinned in `corpus.json`. The initial +real public SDK baseline inspected actual bytes without executing/installing +the APK. Its complete default AOT reply was 151,341 compact client-result JSON +bytes (70,487 normalized bytes), with 500 package URIs, 216 Dart URIs and 2,000 +class-like tokens. String-pool and overall coverage were partial. A second +explicit arm64 request returned the same normalized facts. + +These are exploratory first/warm calls, not an optimization pair. The frozen +runtime source is `28c84e78cf2fbfde4aceed345b773c59eb889f68`, with production +source bytes verified equal to main f254 at baseline acquisition. Its compiled +file-table digest is +`5332dda13981f043e146b3601ebec1f380d9f698280ca53d75786b638a912e11`; +the receipt is not relabeled as a newer build. + +The candidate adds explicit MCP `detail: "summary"` and retained pool pages. +Omitted/complete requests and the CLI preserve the old complete path. Analysis, +ABI selection and original Evidence stay intact. Summary retains target/libapp +identity, exact nullable snapshot observations, lower bounds, returned counts, +coverage and limitations. Pages preserve original string order and source +coverage without rescanning. No engine cache, provider protocol or third-party +plugin API is introduced. + +Follow-up measurements must compare identical selected facts, portable old +Evidence and actual model tasks. Reply bytes do not establish token or latency +improvements. Whole-scope final counters remain unknown if their cgroup has +disappeared, even when the narrower payload-exit observation is verified. +Raw records, binaries and model traces stay in the private lab outside Git. +Heavy work remains serial under the unchanged one-CPU, 3 GiB early-stop / 4 GiB +hard memory, no-swap and host-availability guards. + +## Main advanced during the experiment + +Main subsequently advanced to `a2d8b94dd50dab8d966239dd63ab84718aa8b4b4`, +adding native Fiddler SAZ parsing. Its web-network capture contract already +offers opt-in summary and retained capture pages, archive-member coordinates +and exact exposed encoded bytes. It decodes all records before ordinal +selection and rejects malformed/oversized inputs without partial success. +Measure real public SAZ artifacts using these controls before adding another +API or changing decoding or credential-redaction semantics. SAZ performance +remains unverified in this Flutter continuation. + +The Flutter cohorts keep their original frozen runtime; subsequent integration +onto this newer main does not relabel earlier measurement receipts. + +Main advanced again to `bb154d106` during CI, adding retained aioquic 1.5.0 +qlog 0.3 JSON parsing and qlog views. The existing capture summary/page path +already applies. Qlog's trace-scoped STREAM interval unions and endpoint loss +declarations have separate semantics: they do not recover bodies, prove network +loss or establish PCAP correlation. Measure genuine supported recordings and +existing retained views before adding delivery or caching. Other qlog layouts +and qlog performance remain unverified in the Flutter experiment. Shared-view +integration is checked after rebasing; earlier runtime receipts remain pinned. diff --git a/experiments/analysis-efficiency/measure.mjs b/experiments/analysis-efficiency/measure.mjs index bfac3adb6..8774ed30b 100644 --- a/experiments/analysis-efficiency/measure.mjs +++ b/experiments/analysis-efficiency/measure.mjs @@ -726,6 +726,54 @@ const workflows = { }, "sharex.dll": inspectManagedTarget, "sharex.helpers.dll": inspectManagedTarget, + "localsend-arm64.apk": async () => { + const input = { + path: values.path, + ...(values.delivery === "summary" ? { detail: "summary" } : {}), + }; + const definition = catalogDefinitions.get("inspect_dart_aot"); + if ( + definition === undefined || + !new Ajv2020({ strict: false, validateFormats: false }).validate( + definition.inputSchema, + input, + ) + ) + throw new Error( + "Selected Dart AOT delivery is not advertised by this runtime", + ); + const inspected = await call("inspect_dart_aot", input); + const result = normalized(inspected); + if ( + values.delivery === "summary" && + (result.kind !== "summary" || + result.parent_operation !== "inspect_dart_aot") + ) + throw new Error("Runtime returned no retained Dart AOT summary"); + const report = values.delivery === "summary" ? result.summary : result; + const { package_uris, dart_uris, class_like_tokens, ...pool } = + report.string_pool; + return { + artifact_sha256: targetSha(inspected), + target: report.target, + abi: report.abi, + libapp: report.libapp, + string_pool: + values.delivery === "summary" + ? pool + : { + ...pool, + returned_counts: { + package_uris: package_uris.length, + dart_uris: dart_uris.length, + class_like_tokens: class_like_tokens.length, + }, + }, + coverage: report.coverage, + limitations: inspected.limitations, + evidence_ids: [inspected.evidence_id], + }; + }, "openwrt.bin": async () => { const info = await call("inspect_firmware_regions", { path: values.path }); return { diff --git a/experiments/analysis-efficiency/questions/flutter-aot.txt b/experiments/analysis-efficiency/questions/flutter-aot.txt new file mode 100644 index 000000000..c4c3c3897 --- /dev/null +++ b/experiments/analysis-efficiency/questions/flutter-aot.txt @@ -0,0 +1 @@ +Statically inspect the shipped Flutter APK at {{artifact_path}} using only the registered REA inspect_dart_aot tool. Use its default ABI selection. Report target identity, chosen ABI, every libapp ELF/snapshot field exactly as reported including null, original string-pool coverage, all three counts ending in _at_least, printable_run_count, and returned_counts for package_uris, dart_uris and class_like_tokens. Compute returned_counts from the returned arrays if needed. Preserve original overall coverage and producer limitations exactly, including order. Counts ending in _at_least are lower bounds and class-like tokens are byte-pattern observations, not recovered typed classes. Do not return individual strings or infer Dart SDK versions, class semantics, runtime behavior or complete inventory. Final answer must be JSON with artifact_sha256, target, abi, libapp, string_pool, coverage, limitations, evidence_ids. Use actual response Evidence IDs. Do not execute the APK, use shell or host file tools, access the network, or call other agents. Stop when these facts are supported. diff --git a/experiments/analysis-efficiency/review-model.py b/experiments/analysis-efficiency/review-model.py index a61bb0984..b8fe16bb3 100644 --- a/experiments/analysis-efficiency/review-model.py +++ b/experiments/analysis-efficiency/review-model.py @@ -69,7 +69,7 @@ def scope_with_verified_counts(value, expected_scope, returned_counts): def main(): parser = argparse.ArgumentParser() - parser.add_argument('--domain', choices=['wasm', 'evm', 'android', 'managed', 'managed-nonempty', 'firmware', 'inventory', 'javascript', 'native', 'browser'], required=True) + parser.add_argument('--domain', choices=['wasm', 'evm', 'android', 'flutter-aot', 'managed', 'managed-nonempty', 'firmware', 'inventory', 'javascript', 'native', 'browser'], required=True) parser.add_argument('--trial', type=Path, required=True) parser.add_argument('--reference', type=Path) parser.add_argument('--resource-log', type=Path, required=True) @@ -133,6 +133,9 @@ def main(): for key, expected in {'artifact_bytes': n['artifact']['bytes'], 'bytecode_sha256': n['bytecode']['sha256'], 'bytecode_bytes': n['bytecode']['bytes'], 'functions': n['functions'], 'evidence_kind': n['evidence_kind'], 'discovery_completeness': n['discovery_completeness'], 'runtime_execution': n['runtime_execution'], 'bytecode_kind': n['bytecode']['kind'], 'hardfork': n['bytecode']['hardfork'], 'deployment_authenticity': n['bytecode']['deployment_authenticity']}.items(): check(key, answer.get(key), expected) check('evm_original_limitations', answer.get('limitations'), n['limitations']) + elif domain == 'flutter-aot': + for key in ['target', 'abi', 'libapp', 'string_pool', 'coverage', 'limitations']: + check(key, answer.get(key), reference[key]) elif domain == 'android': for key in ['package_name', 'version_name', 'version_code', 'matches']: check(key, answer.get(key), reference[key]) diff --git a/src/application/flutter/DartAotInspection.ts b/src/application/flutter/DartAotInspection.ts new file mode 100644 index 000000000..c11bbe1f4 --- /dev/null +++ b/src/application/flutter/DartAotInspection.ts @@ -0,0 +1,40 @@ +import type { ExecutionOptions } from "../AnalysisProvider.js"; +import type { FlutterBuildAnalysisService } from "./FlutterBuildAnalysisService.js"; +import { + requireAnalysisViewRetention, + summarizeRetainedAnalysis, + type AnalysisViewRetention, +} from "../analysisView/AnalysisViewService.js"; +import type { AnalysisError } from "../../domain/analysisErrorBase.js"; +import { AnalysisCancelledError } from "../../domain/analysisErrorCore.js"; +import { dartAotInspectionInputSchema } from "../../domain/flutter/flutterBuildAnalysis.js"; +import type { Evidence } from "../../domain/evidence.js"; +import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js"; +import { err, type Result } from "../../domain/result.js"; + +/** Keep the caller's ABI choice and complete analysis; select only its delivery. */ +export const inspectDartAot = async ( + service: FlutterBuildAnalysisService, + rawInput: unknown, + retention: AnalysisViewRetention, + options?: ExecutionOptions, +): Promise> => { + const operation = "inspect_dart_aot"; + if (options?.signal?.aborted === true) + return err(new AnalysisCancelledError(operation)); + const parsed = dartAotInspectionInputSchema.safeParse(rawInput); + if (!parsed.success) + return err( + analysisInputErrorFromIssues(operation, parsed.error.issues, rawInput, { + cause: parsed.error, + }), + ); + if (parsed.data.detail === "summary") { + const ready = requireAnalysisViewRetention(operation, retention); + if (!ready.ok) return ready; + } + const { detail, ...providerInput } = parsed.data; + const executed = await service.execute(operation, providerInput, options); + if (!executed.ok || detail !== "summary") return executed; + return summarizeRetainedAnalysis(executed.value, retention); +}; diff --git a/src/contracts/analysisViewToolContracts.ts b/src/contracts/analysisViewToolContracts.ts index 06506c83e..6011d544b 100644 --- a/src/contracts/analysisViewToolContracts.ts +++ b/src/contracts/analysisViewToolContracts.ts @@ -16,7 +16,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [ ...toolContractMetadata("inspect_analysis_view"), kind: "application", description: - 'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_evm_interface, inspect_managed_artifact, inspect_android_package or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. For layout, JavaScript, native, inventory, WASM, EVM, managed artifact and Android package sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Native views select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', + 'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_evm_interface, inspect_managed_artifact, inspect_android_package, inspect_dart_aot or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. For layout, JavaScript, native, inventory, WASM, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Dart AOT summary preserves target/ABI/library identity, snapshot headers and pool lower bounds. String pages use kind dart-aot, collection package_uris/dart_uris/class_like_tokens; source coverage remains partial when the producer says partial. Native views select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', inputSchema: inspectAnalysisViewInputSchema, outputSchema: evidenceResultOf(analysisViewResultSchema), examples: [ diff --git a/src/contracts/flutter/flutterToolContracts.ts b/src/contracts/flutter/flutterToolContracts.ts index 9a663d492..9ade1057b 100644 --- a/src/contracts/flutter/flutterToolContracts.ts +++ b/src/contracts/flutter/flutterToolContracts.ts @@ -1,7 +1,10 @@ +import { z } from "zod"; import { flutterInputSchemas, + dartAotInspectionInputSchema, flutterResultSchemas, } from "../../domain/flutter/flutterBuildAnalysis.js"; +import { dartAotSummaryViewResultSchema } from "../../domain/analysisView/analysisView.js"; import type { ToolContract } from "../toolContractTypes.js"; import { evidenceResultOf } from "../toolOutputSchemaPrimitives.js"; import { toolContractMetadata } from "../toolEffects.js"; @@ -27,10 +30,15 @@ export const FLUTTER_TOOL_CONTRACTS = [ name: "inspect_dart_aot", ...toolContractMetadata("inspect_dart_aot"), description: - "Inspect one ABI's Dart AOT snapshot inside an APK: the four snapshot symbols located through the ELF dynamic symbol table with their offsets, sizes, and section headers (magic, kind, length), the snapshot hash, and a bounded string-pool projection from the isolate data section (package: source URIs, dart: SDK URIs, identifier-like tokens) exposing the app's dependency structure. String-pool entries are byte-pattern observations, not deserialized cluster semantics; typed class and function recovery needs an SDK-specific parser. No engine, no execution.", + "Inspect one ABI's Dart AOT snapshot inside an APK: the four snapshot symbols located through the ELF dynamic symbol table with their offsets, sizes, and section headers (magic, kind, length), the snapshot hash, and a bounded string-pool projection from the isolate data section (package: source URIs, dart: SDK URIs, identifier-like tokens) exposing the app's dependency structure. String-pool entries are byte-pattern observations, not deserialized cluster semantics; typed class and function recovery needs an SDK-specific parser. Omitted or complete detail returns the full observation. Select detail: \"summary\" for target/ABI/library identity, all snapshot headers, coverage and original lower-bound counts without string lists. The complete source stays retained; read returned strings via inspect_analysis_view with parent_evidence_id and kind dart-aot, collection package_uris/dart_uris/class_like_tokens, offset and limit. Exhausting pages does not imply complete pool coverage. No engine, no execution.", kind: "flutter-provider", - inputSchema: flutterInputSchemas.inspect_dart_aot, - outputSchema: evidenceResultOf(flutterResultSchemas.inspect_dart_aot), + inputSchema: dartAotInspectionInputSchema, + outputSchema: evidenceResultOf( + z.union([ + flutterResultSchemas.inspect_dart_aot, + dartAotSummaryViewResultSchema, + ]), + ), examples: [ { title: "Inspect the preferred ABI's AOT snapshot", @@ -40,6 +48,10 @@ export const FLUTTER_TOOL_CONTRACTS = [ title: "Inspect one ABI explicitly", input: { path: "/targets/Example.apk", abi: "arm64-v8a" }, }, + { + title: "Inspect snapshot identity with retained string lists", + input: { path: "/targets/Example.apk", detail: "summary" }, + }, ], }, ] as const satisfies readonly ToolContract[]; diff --git a/src/domain/analysisView/analysisView.ts b/src/domain/analysisView/analysisView.ts index 05f879024..48eee73f0 100644 --- a/src/domain/analysisView/analysisView.ts +++ b/src/domain/analysisView/analysisView.ts @@ -73,6 +73,12 @@ import { androidViewRequestSchema, androidManifestViewItemSchema, } from "./androidPackageView.js"; +import { + projectDartAotView, + dartAotSummarySchema, + dartAotViewRequestSchema, + dartAotViewContextSchema, +} from "./dartAotView.js"; /** Public name of the selected-view workflow. */ export const INSPECT_ANALYSIS_VIEW_OPERATION = "inspect_analysis_view" as const; @@ -190,6 +196,7 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [ apiSurfaceViewSchema, captureViewRequestSchema, qlogViewRequestSchema, + dartAotViewRequestSchema, ]); // The application owns admission and authenticated Evidence parsing. SDK and @@ -305,6 +312,7 @@ const viewResultBase = { "inspect_managed_artifact", "inspect_web_network_capture", "inspect_android_package", + "inspect_dart_aot", ]), parent_digest: digestSchema, view_digest: digestSchema, @@ -376,6 +384,13 @@ export const androidPackageSummaryViewResultSchema = z.strictObject({ summary: androidPackageSummarySchema, }); +/** Exact snapshot and pool-coverage carrier for selected AOT delivery. */ +export const dartAotSummaryViewResultSchema = z.strictObject({ + ...summaryViewResultBase, + parent_operation: z.literal("inspect_dart_aot"), + summary: dartAotSummarySchema, +}); + /** Projected facts for one selected view of retained analysis Evidence. */ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ z.strictObject({ @@ -400,6 +415,7 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ managedSummarySchema, captureSummarySchema, androidPackageSummarySchema, + dartAotSummarySchema, ]), }), z.strictObject({ @@ -470,6 +486,14 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ ...viewResultBase, parent_operation: z.literal("inspect_android_package"), }), + z.strictObject({ + kind: z.literal("dart-aot"), + view: dartAotViewRequestSchema, + items: z.array(z.string()), + ...viewResultBase, + ...dartAotViewContextSchema.shape, + parent_operation: z.literal("inspect_dart_aot"), + }), z.strictObject({ kind: z.literal("inventory"), view: inventoryViewSchema, @@ -549,6 +573,10 @@ export type UnsignedAnalysisView = | Omit< Extract, "view_digest" + > + | Omit< + Extract, + "view_digest" >; /** Attach the content digest that must change when projected bytes change. */ @@ -594,7 +622,7 @@ const unsupportedParent = ( new AnalysisUnsupportedTargetError( INSPECT_ANALYSIS_VIEW_OPERATION, parentArtifactPath(parent), - `Parent Evidence operation ${JSON.stringify(parent.operation)} does not support analysis views. Use completed binary layout, JavaScript application, native function, artifact inventory/inspection, WASM, EVM, managed artifact, Android package, or web network capture Evidence.`, + `Parent Evidence operation ${JSON.stringify(parent.operation)} does not support analysis views. Use completed binary layout, JavaScript application, native function, artifact inventory/inspection, WASM, EVM, managed artifact, Android package, Dart AOT, or web network capture Evidence.`, ); const parentArtifactPath = (parent: AnalysisViewParent): string => { @@ -624,6 +652,7 @@ const retainedViewProjectors = new Map([ ["inspect_evm_interface", projectEvmInterfaceView], ["inspect_managed_artifact", projectManagedArtifactView], ["inspect_android_package", projectAndroidPackageView], + ["inspect_dart_aot", projectDartAotView], ["inspect_artifact", projectArtifactInventoryView], ["inventory_artifact", projectArtifactInventoryView], ]); diff --git a/src/domain/analysisView/dartAotView.ts b/src/domain/analysisView/dartAotView.ts new file mode 100644 index 000000000..4d66aa190 --- /dev/null +++ b/src/domain/analysisView/dartAotView.ts @@ -0,0 +1,123 @@ +import { z } from "zod"; +import type { AnalysisError } from "../analysisErrorBase.js"; +import { flutterResultSchemas } from "../flutter/flutterBuildAnalysis.js"; +import { analysisInputErrorFromIssues } from "../inputIssueProjection.js"; +import { err, ok, type Result } from "../result.js"; +import { + analysisViewInputError, + type AnalysisViewParent, + type AnalysisViewRequest, + type UnsignedAnalysisView, +} from "./analysisView.js"; +import { + completeWithinViewCoverage, + pageViewCoverage, +} from "./analysisViewCoverage.js"; + +const aotSchema = flutterResultSchemas.inspect_dart_aot; + +/** Snapshot identity and original coverage, without unrelated string inventories. */ +export const dartAotSummarySchema = aotSchema.extend({ + string_pool: aotSchema.shape.string_pool + .omit({ package_uris: true, dart_uris: true, class_like_tokens: true }) + .extend({ + returned_counts: z.strictObject({ + package_uris: z.number().int().nonnegative(), + dart_uris: z.number().int().nonnegative(), + class_like_tokens: z.number().int().nonnegative(), + }), + }), +}); + +/** Pages of the producer-returned strings, preserving original order and scope. */ +export const dartAotViewRequestSchema = z.strictObject({ + kind: z.literal("dart-aot"), + collection: z.enum(["package_uris", "dart_uris", "class_like_tokens"]), + offset: z.number().int().nonnegative().default(0), + limit: z.number().int().positive().default(64), +}); + +export const dartAotViewContextSchema = z.strictObject({ + abi: aotSchema.shape.abi, + libapp: aotSchema.shape.libapp, + source_coverage: aotSchema.shape.coverage, + string_pool: dartAotSummarySchema.shape.string_pool, +}); + +/** Reuse authenticated AOT Evidence without opening the APK or rescanning its ELF. */ +export const projectDartAotView = ( + parent: AnalysisViewParent, + view: AnalysisViewRequest, +): Result => { + if (view.kind !== "summary" && view.kind !== "dart-aot") + return err( + analysisViewInputError([ + { + path: ["view"], + reason: "invalid_value", + message: "Dart AOT Evidence supports summary and string-pool pages.", + }, + ]), + ); + const parsed = aotSchema.safeParse(parent.normalizedResult); + if (!parsed.success) + return err( + analysisInputErrorFromIssues( + "inspect_analysis_view", + parsed.error.issues.map((issue) => ({ + ...issue, + path: ["source", "normalized_result", ...issue.path], + })), + { source: { normalized_result: parent.normalizedResult } }, + { cause: parsed.error }, + ), + ); + const report = parsed.data; + const { package_uris, dart_uris, class_like_tokens, ...pool } = + report.string_pool; + const string_pool = { + ...pool, + returned_counts: { + package_uris: package_uris.length, + dart_uris: dart_uris.length, + class_like_tokens: class_like_tokens.length, + }, + }; + const shared = { + parent_evidence_id: parent.evidenceId, + parent_operation: "inspect_dart_aot" as const, + parent_digest: parent.evidenceId.slice(3), + artifact: { path: report.target.path, sha256: report.target.sha256 }, + limitations: [...parent.limitations], + unknowns: [ + "String-pool entries are byte-pattern observations, not recovered typed classes or functions.", + "Counts ending in _at_least remain lower bounds; exhausting returned strings does not establish a complete snapshot inventory.", + ...(parent.subject == null + ? [ + "The parent has no recorded subject; its normalized target identity cannot be cross-checked against a subject digest.", + ] + : []), + ], + }; + if (view.kind === "summary") + return ok({ + kind: "summary", + view, + ...shared, + summary: { ...report, string_pool }, + coverage: completeWithinViewCoverage(1, 1), + }); + const collection = report.string_pool[view.collection]; + const items = collection.slice(view.offset, view.offset + view.limit); + return ok({ + kind: "dart-aot", + view, + ...shared, + items, + abi: report.abi, + libapp: report.libapp, + source_coverage: report.coverage, + string_pool, + coverage: pageViewCoverage(view.offset, items.length, collection.length), + }); +}; diff --git a/src/domain/flutter/flutterBuildAnalysis.ts b/src/domain/flutter/flutterBuildAnalysis.ts index 068149f68..10c5b7b00 100644 --- a/src/domain/flutter/flutterBuildAnalysis.ts +++ b/src/domain/flutter/flutterBuildAnalysis.ts @@ -30,6 +30,12 @@ export const flutterInputSchemas = { }), } as const; +/** Retained delivery is selected independently of the provider's ABI request. */ +export const dartAotInspectionInputSchema = + flutterInputSchemas.inspect_dart_aot.extend({ + detail: z.enum(["complete", "summary"]).optional(), + }); + /** Supported Flutter-backed operations. */ export type FlutterOperation = keyof typeof flutterInputSchemas; /** Validated Flutter requests carry their selected operation. */ diff --git a/src/server/createServer.ts b/src/server/createServer.ts index c213efb0b..4929142c4 100644 --- a/src/server/createServer.ts +++ b/src/server/createServer.ts @@ -444,6 +444,7 @@ export const createServer = ( toolLogger, recordEvidence, analysisAdmission, + evidenceById, ); registerApktoolTools( server, diff --git a/src/server/registerFlutterTools.ts b/src/server/registerFlutterTools.ts index 5b601c5f7..cd57e12a9 100644 --- a/src/server/registerFlutterTools.ts +++ b/src/server/registerFlutterTools.ts @@ -1,10 +1,15 @@ import type { EvidenceMcpServer } from "./EvidenceMcpServer.js"; import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js"; import { FlutterBuildAnalysisService } from "../application/flutter/FlutterBuildAnalysisService.js"; +import { inspectDartAot } from "../application/flutter/DartAotInspection.js"; +import type { EvidenceLookup } from "../application/EvidenceInputResolver.js"; import { toolContract } from "../contracts/toolContracts.js"; import type { WithAdmittedAnalysis } from "./analysisAdmission.js"; import type { Logger } from "pino"; -import { createContractToolHandler } from "./contractToolHandler.js"; +import { + createContractToolHandler, + type ContractToolService, +} from "./contractToolHandler.js"; import { toolRegistrationOptions } from "./toolRegistrationOptions.js"; /** Bind Flutter handlers to their exact named contracts through admission. */ @@ -14,6 +19,7 @@ export const registerFlutterTools = ( logger: Logger, recordEvidence?: EvidenceWriter["recordEvidence"], withAdmittedAnalysis?: WithAdmittedAnalysis, + evidenceById?: EvidenceLookup, ): void => { const handler = createContractToolHandler( server, @@ -22,14 +28,23 @@ export const registerFlutterTools = ( recordEvidence, withAdmittedAnalysis, ); - for (const contract of [ - toolContract("identify_flutter_build"), - toolContract("inspect_dart_aot"), - ] as const) { - server.registerTool( - contract.name, - toolRegistrationOptions(contract), - handler(contract), - ); - } + const identify = toolContract("identify_flutter_build"); + server.registerTool( + identify.name, + toolRegistrationOptions(identify), + handler(identify), + ); + const aotService: ContractToolService<"inspect_dart_aot"> = { + execute: (_operation, input, options) => + inspectDartAot(service, input, { recordEvidence, evidenceById }, options), + }; + const aotHandler = createContractToolHandler( + server, + aotService, + logger, + recordEvidence, + withAdmittedAnalysis, + ); + const aot = toolContract("inspect_dart_aot"); + server.registerTool(aot.name, toolRegistrationOptions(aot), aotHandler(aot)); }; diff --git a/tests/acceptance/flutterAnalysis.test.ts b/tests/acceptance/flutterAnalysis.test.ts index 457dce96b..11671785e 100644 --- a/tests/acceptance/flutterAnalysis.test.ts +++ b/tests/acceptance/flutterAnalysis.test.ts @@ -2,6 +2,7 @@ import { createHash } from "node:crypto"; import { expect } from "vitest"; import { toolContract } from "../../src/contracts/toolContracts.js"; import { parseEvidence } from "../../src/domain/evidence.js"; +import { flutterResultSchemas } from "../../src/domain/flutter/flutterBuildAnalysis.js"; import { aotFixtureImage, engineFixtureImage, @@ -59,6 +60,9 @@ cliTest( }); expect(result.exitCode, result.stderr).toBe(0); const evidence = toolContract(name).outputSchema.parse(result.json); + const complete = flutterResultSchemas[name].parse( + evidence.normalized_result, + ); const response = await call(name, { path, ...(abi === undefined ? {} : { abi }), @@ -70,7 +74,7 @@ cliTest( expect(mcp.subject).toMatchObject({ local_path: path, format: "apk", - digest: { sha256: evidence.normalized_result.target.sha256 }, + digest: { sha256: complete.target.sha256 }, }); expect(mcp.evidence_id).toBe(evidence.evidence_id); if (name === "inspect_dart_aot") { @@ -134,6 +138,9 @@ cliTest( const evidence = toolContract("inspect_dart_aot").outputSchema.parse( result.json, ); + const complete = flutterResultSchemas.inspect_dart_aot.parse( + evidence.normalized_result, + ); const { call } = await connectLocalToolsMcp(); const response = await call("inspect_dart_aot", { path }); expect(response.isError, JSON.stringify(response)).not.toBe(true); @@ -146,7 +153,7 @@ cliTest( string_pool: { coverage: "complete" }, }); expect( - evidence.normalized_result.libapp.sections + complete.libapp.sections .filter((section) => section.name.endsWith("Data")) .map((section) => section.kind), ).toEqual([4, 4]); diff --git a/tests/boundary/mcp/dartAotSummaryMcp.test.ts b/tests/boundary/mcp/dartAotSummaryMcp.test.ts new file mode 100644 index 000000000..be6a970d3 --- /dev/null +++ b/tests/boundary/mcp/dartAotSummaryMcp.test.ts @@ -0,0 +1,285 @@ +import { unlink } from "node:fs/promises"; +import { Ajv2020 } from "ajv/dist/2020.js"; +import { expect, it, onTestFinished, vi } from "vitest"; +import { FlutterBuildAnalysisService } from "../../../src/application/flutter/FlutterBuildAnalysisService.js"; +import { inspectDartAot } from "../../../src/application/flutter/DartAotInspection.js"; +import { inspectAnalysisView } from "../../../src/application/analysisView/AnalysisViewService.js"; +import { createFlutterBuildAnalysisProvider } from "../../../src/composition/flutter.js"; +import { + analysisViewResultSchema, + dartAotSummaryViewResultSchema, +} from "../../../src/domain/analysisView/analysisView.js"; +import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js"; +import { flutterResultSchemas } from "../../../src/domain/flutter/flutterBuildAnalysis.js"; +import type { JsonValue } from "../../../src/domain/jsonValue.js"; +import { aotFixtureImage } from "../../../src/flutter/FlutterBuild.fixture.js"; +import { createFlutterApk } from "../../fixtures/flutterApk.js"; +import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; + +it("honors explicit ABI and old full requests, validates SDK schemas and reuses complete Evidence after APK removal", async () => { + const path = await createFlutterApk("views.apk", [ + { path: "lib/arm64-v8a/libapp.so", bytes: aotFixtureImage() }, + { path: "lib/x86_64/libapp.so", bytes: aotFixtureImage("b".repeat(32)) }, + ]); + const provider = createFlutterBuildAnalysisProvider(); + const execute = vi.spyOn(provider, "execute"); + const harness = await createApplicationMcpHarness({ + serverOptions: { flutterAnalysis: provider }, + }); + onTestFinished(harness.close); + const { client, session } = harness; + const catalog = (await client.listTools()).tools; + const definition = catalog.find((tool) => tool.name === "inspect_dart_aot"); + if (definition?.outputSchema === undefined) + throw new Error("Missing AOT output schema"); + const outputSchema: Record = definition.outputSchema; + const ajv = new Ajv2020({ strict: false, validateFormats: false }); + expect(ajv.validateSchema(definition.inputSchema)).toBe(true); + expect(ajv.validateSchema(outputSchema)).toBe(true); + expect( + ajv.validate( + catalog.find((tool) => tool.name === "identify_flutter_build")! + .inputSchema, + { path, detail: "summary" }, + ), + ).toBe(false); + const reply = await client.callTool({ + name: "inspect_dart_aot", + arguments: { path, abi: "x86_64", detail: "summary" }, + }); + expect(reply.isError).not.toBe(true); + expect( + ajv.validate(outputSchema, reply.structuredContent), + JSON.stringify(ajv.errors), + ).toBe(true); + const receipt = parseEvidence(reply.structuredContent); + const view = dartAotSummaryViewResultSchema.parse(receipt.normalized_result); + const parent = session.evidenceById(view.parent_evidence_id); + if (parent === undefined) + throw new Error("Complete AOT source was not retained"); + const report = flutterResultSchemas.inspect_dart_aot.parse( + parent.normalized_result, + ); + expect(report.abi).toBe("x86_64"); + expect(report.libapp.snapshot_hash).toBe("b".repeat(32)); + expect(parent.parameters).toEqual({ path, abi: "x86_64" }); + expect(execute).toHaveBeenLastCalledWith( + { operation: "inspect_dart_aot", input: { path, abi: "x86_64" } }, + expect.anything(), + ); + expect(view.summary.libapp).toEqual(report.libapp); + expect(view.summary.target).toEqual(report.target); + expect(view.summary.coverage).toBe(report.coverage); + expect(receipt.evidence_id).not.toBe(parent.evidence_id); + expect(receipt.evidence_links).toEqual([parent.evidence_id]); + for (const detail of [undefined, "complete"]) { + const complete = parseEvidence( + ( + await client.callTool({ + name: "inspect_dart_aot", + arguments: { + path, + abi: "x86_64", + ...(detail === undefined ? {} : { detail }), + }, + }) + ).structuredContent, + ); + expect(complete.operation).toBe("inspect_dart_aot"); + expect(complete.normalized_result).toEqual(parent.normalized_result); + expect(complete.parameters).toEqual(parent.parameters); + } + const preferred = parseEvidence( + (await client.callTool({ name: "inspect_dart_aot", arguments: { path } })) + .structuredContent, + ); + expect( + flutterResultSchemas.inspect_dart_aot.parse(preferred.normalized_result) + .abi, + ).toBe("arm64-v8a"); + const original = JSON.stringify(parent); + await unlink(path); + const calls = execute.mock.calls.length; + for (const collection of [ + "package_uris", + "dart_uris", + "class_like_tokens", + ] as const) { + const strings: string[] = []; + let offset = 0; + do { + const reply = await client.callTool({ + name: "inspect_analysis_view", + arguments: { + source: { + kind: "retained-evidence", + evidence_id: parent.evidence_id, + }, + view: { kind: "dart-aot", collection, offset, limit: 1 }, + }, + }); + expect(reply.isError).not.toBe(true); + const page = analysisViewResultSchema.parse( + parseEvidence(reply.structuredContent).normalized_result, + ); + if (page.kind !== "dart-aot") throw new Error("Wrong pool page kind"); + strings.push(...page.items); + expect(page.libapp).toEqual(report.libapp); + expect(page.source_coverage).toBe(report.coverage); + expect(page.string_pool.coverage).toBe(report.string_pool.coverage); + offset = + page.coverage.next_offset ?? report.string_pool[collection].length; + } while (offset < report.string_pool[collection].length); + expect(strings).toEqual(report.string_pool[collection]); + } + expect(execute).toHaveBeenCalledTimes(calls); + expect(JSON.stringify(session.evidenceById(parent.evidence_id))).toBe( + original, + ); + expect( + session + .exportEvidenceBundle() + .records.find((record) => record.evidence_id === parent.evidence_id), + ).toEqual(parent); + const portable = inspectAnalysisView({ + source: { kind: "inline", evidence: parent }, + view: { kind: "summary" }, + }); + if (!portable.ok) throw portable.error; + expect(portable.value.normalized_result).toEqual(view); +}, 30000); + +it("preserves null snapshot facts and pool lower bounds after exhaustion, rejecting malformed and conflicting identities", () => { + const report = { + target: { path: "/source.apk", bytes: 10, sha256: "a".repeat(64) }, + abi: "arm64-v8a", + libapp: { + coverage: "partial", + bytes: 1, + sha256: "b".repeat(64), + snapshot_hash: null, + snapshot_hash_candidates: [], + snapshot_hash_sources: 0, + sections: [], + }, + string_pool: { + coverage: "partial", + package_uris: ["package:opaque/a.dart"], + package_uri_count_at_least: 1000, + dart_uris: [], + dart_uri_count_at_least: 0, + class_like_tokens: ["incidental"], + class_like_token_count_at_least: 4000, + printable_run_count: 12, + }, + coverage: "partial", + }; + const evidence = ( + result: JsonValue = report, + sha: string | null = "a".repeat(64), + ) => + createEvidence( + sha === null + ? undefined + : { path: "/source.apk", sha256: sha, format: "apk" }, + { id: "flutter", name: "Flutter", version: null }, + { + operation: "inspect_dart_aot", + parameters: {}, + result, + limitations: ["Original uncertainty"], + }, + ); + for (const parent of [evidence(), evidence(report, null)]) { + const summary = inspectAnalysisView({ + source: { kind: "inline", evidence: parent }, + view: { kind: "summary" }, + }); + if (!summary.ok) throw summary.error; + const result = dartAotSummaryViewResultSchema.parse( + summary.value.normalized_result, + ); + expect(result.summary.libapp.snapshot_hash).toBeNull(); + expect(result.summary.string_pool.package_uri_count_at_least).toBe(1000); + expect(result.summary.string_pool.returned_counts.package_uris).toBe(1); + expect(result.limitations).toEqual(parent.limitations); + const page = inspectAnalysisView({ + source: { kind: "inline", evidence: parent }, + view: { + kind: "dart-aot", + collection: "package_uris", + offset: 0, + limit: 10, + }, + }); + if (!page.ok) throw page.error; + expect(page.value.normalized_result).toMatchObject({ + kind: "dart-aot", + items: report.string_pool.package_uris, + source_coverage: "partial", + string_pool: { coverage: "partial", package_uri_count_at_least: 1000 }, + coverage: { total: 1, exhausted: true }, + }); + } + expect( + inspectAnalysisView({ + source: { kind: "inline", evidence: evidence(report, "c".repeat(64)) }, + view: { kind: "summary" }, + }), + ).toMatchObject({ ok: false, error: { _tag: "EvidenceIntegrityError" } }); + expect( + inspectAnalysisView({ + source: { + kind: "inline", + evidence: evidence({ + ...report, + string_pool: { + ...report.string_pool, + package_uri_count_at_least: -1, + }, + }), + }, + view: { kind: "summary" }, + }), + ).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } }); + expect( + inspectAnalysisView({ + source: { kind: "inline", evidence: evidence() }, + view: { kind: "android", facet: "manifest", offset: 0, limit: 1 }, + }), + ).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } }); +}); + +it("rejects unavailable retention, invalid delivery and cancellation before producer work", async () => { + const provider = createFlutterBuildAnalysisProvider(); + onTestFinished(() => provider.close()); + const execute = vi.spyOn(provider, "execute"); + const service = new FlutterBuildAnalysisService(provider); + const retention = { recordEvidence: undefined, evidenceById: undefined }; + expect( + await inspectDartAot( + service, + { path: "/missing.apk", abi: "x86_64", detail: "summary" }, + retention, + ), + ).toMatchObject({ + ok: false, + error: { _tag: "AnalysisCapabilityUnavailableError" }, + }); + expect( + await inspectDartAot( + service, + { path: "/missing.apk", detail: "other" }, + retention, + ), + ).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } }); + expect( + await inspectDartAot( + service, + { path: "/missing.apk", detail: "summary" }, + retention, + { signal: AbortSignal.abort() }, + ), + ).toMatchObject({ ok: false, error: { _tag: "AnalysisCancelledError" } }); + expect(execute).not.toHaveBeenCalled(); +}); From 172407787f1842dedbcd27b5385177595f786ce4 Mon Sep 17 00:00:00 2001 From: Ysopwnable <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 11:42:54 +0800 Subject: [PATCH 005/159] fix(windows): preserve transient runtime admission failures (#2467) Fixes #2257. Thanks to @chqcjq for the native self-test retry diagnosis in #2266. Preserve retry disposition through analysis workflows. Native warm admission requires exact sharing/lock codes and affirmative pre-allocation evidence; cleanup ownership takes precedence. No automatic retry or new analysis cache. Verified with real Windows 11 CLI, same-connection SDK MCP normal/conflict/recovery, the Windows native verifier, 99 focused tests and static/docs gates. Official npm 11.16.0 launcher sources were the application input. No IDA or Ghidra engine was used for this change. Win32 5 was not reproduced; no resolution claim for #1701/#2196. Performance measurements and limits are recorded in experiments/analysis-efficiency/windows-runtime-retry.md. --- .../windows-runtime-retry.md | 107 ++++++++++++++++++ native/windows/README.md | 11 ++ native/windows/src/addon.cc | 2 + native/windows/src/authority.hpp | 2 + native/windows/src/filesystem.cc | 7 +- scripts/verify-windows-native.mjs | 33 ++++++ src/domain/analysisErrorCore.ts | 8 +- src/domain/analysisErrorPresentation.ts | 7 ++ src/domain/analysisErrorProjection.test.ts | 85 ++++++++++++++ src/domain/analysisErrorProjection.ts | 2 + .../analysis/JavaScriptAnalysisWorker.ts | 8 +- src/javascript/sourceMaps/SourceMapCommand.ts | 8 +- src/process/PrivateRuntimeRoot.ts | 15 +-- .../PrivateRuntimeRootUnavailableError.ts | 14 +++ src/process/WindowsAuthority.ts | 2 + src/wasm/WabtArtifactProvider.test.ts | 78 +++++++------ src/wasm/WabtFailures.ts | 4 +- src/windows/WindowsPrivateRuntime.ts | 26 ++++- .../sourceMapRuntimeAvailability.test.ts | 35 ++++++ .../windowsRuntimeAdmission.test.ts | 99 ++++++++++++++++ 20 files changed, 491 insertions(+), 62 deletions(-) create mode 100644 experiments/analysis-efficiency/windows-runtime-retry.md create mode 100644 src/process/PrivateRuntimeRootUnavailableError.ts create mode 100644 tests/boundary/javascript/sourceMapRuntimeAvailability.test.ts create mode 100644 tests/process-global/windowsRuntimeAdmission.test.ts diff --git a/experiments/analysis-efficiency/windows-runtime-retry.md b/experiments/analysis-efficiency/windows-runtime-retry.md new file mode 100644 index 000000000..2286727e8 --- /dev/null +++ b/experiments/analysis-efficiency/windows-runtime-retry.md @@ -0,0 +1,107 @@ +# Windows runtime admission retry verification + +Scope: [#2257](https://github.com/morluto/rea/issues/2257). Related initial +diagnosis: [#2266](https://github.com/morluto/rea/pull/2266). + +## Revisions and real input + +The baseline is `4f4f6cd7b5d7a400b90772a138d7b9f942c2a556`. +The candidate was developed from +`4773c152bc5f299fa682575bb8d1efea7587c7bc`; relevant runtime, native and +dependency-lock sources are identical between those two revisions. +These measurements precede integration of subsequent main changes. + +The host ran Windows 11 10.0.22631.0, official Windows x64 Node 24.18.0, +and locked production dependencies. The real application input was the official +npm 11.16.0 launcher distribution's `bin` directory. REA inspected that source; +it did not execute the application as an analysis target. Runtime copies, +raw replies and test executables remain outside the checkout. No IDA or Ghidra +engine was involved in this native-runtime verification. + +## Correctness and recovery + +A test-owned directory handle requesting `FILE_ADD_FILE` produced an actual +kernel sharing violation (Win32 32) during native runtime-parent admission. +Releasing that handle restored admission. Before the fix, public CLI analysis +reported a permanent `unsupported_provider` failure with `retryable: false`. +The corrected CLI preserves `capability_unavailable` and the reason, with +`category: unavailable` and `retryable: true`. + +Normal CLI analysis and post-conflict recovery produced the same canonical +normalized result before and after the correction: 535,733 compact JSON bytes, +SHA-256 `a24d3c2d9897cccc5ba70e9ee4dce03da88f033a939fe87457136fc5b3a3e2db`. +The complete successful CLI reply remained 537,124 bytes. + +A single real SDK 2.3.1 stdio MCP connection first completed analysis, then +encountered the same kernel conflict after native authority caching, then +recovered successfully when the handle was released. Its final receipt passed +and found no residual runtime directories. The native producer supplies +`runtimeAllocation: "not-started"` only when parent admission failed before +directory creation. The adapter requires that fact and exact numeric codes +32/33; permission failures, opaque error text and unknown allocation state +remain nonretryable. Incomplete cleanup also prevents retry. + +The native verifier additionally reproduces a real sharing conflict using an +owned runtime's DELETE lease, verifies the original error and allocation fact, +and confirms no child directory was allocated. This case runs unconditionally +in the Windows native verifier, without the optional process-boundary fixture. +The full verifier passed with 20 positive controls. Its optional independently +built process-boundary fixture was not supplied in this run. + +The related focused suite passed 99 tests across seven files, including worker +ownership, cancellation, cleanup, error projection, loader behavior and adapter +boundaries. Source-map and WABT acquisition-failure tests use injected failures; +they establish disposition propagation, not real engine execution. + +Thirty baseline native self-tests passed with no residual directories. +Win32 5 from #1701/#2196 was not reproduced on this host; these results do not +establish its cause or resolution. + +## Delivery and resource measurements + +One final same-connection MCP trial used one CPU per Node process, BelowNormal +priority, a 512 MiB client heap setting and a 768 MiB server heap setting. +Heap settings do not bound total process or family memory. + +| Stage | Observed wall time | Serialized reply bytes | +| ---------------------------------- | -----------------: | ---------------------: | +| Connect | 11,351 ms | Not recorded | +| Complete tool discovery, 187 tools | 744 ms | 2,498,686 | +| Normal analysis | 12,887 ms | 1,126,797 | +| Sharing failure | 265 ms | 878 | +| Recovery on the same connection | 5,578 ms | 1,126,797 | + +The earlier warm failure reply was 1,218 bytes. Successful reply sizes and the +catalogue digest were unchanged within the tested source comparison: +`7bdadddc909f4b2b093f0b3da9f79fafdac879c9d3c147938b8fbce1b52cd726`. +These are delivery bytes, not model tokens. Model-visible discovery handling, +actual model token usage and total model task cost were not measured. + +A targeted process-generation sampler collected 343 observations over +35,630 ms. Its aggregate observed CPU time was a **lower bound** of 26,766 ms: +process exits can hide the final CPU tail. Its peak sampled working-set sum was +727,244,800 bytes (about 694 MiB). This is a sampled sum, can double-count shared +pages and is not an exact physical-memory peak. Only the test-owned Node and +directory-holder processes were included; the observer itself was excluded. + +This is one controlled functional trial, with no paired CPU/memory baseline. +Cold/warm filesystem caches and host noise were not normalized. No speedup, +CPU reduction, memory reduction or model-token savings are claimed. The change +adds no analysis retry loop or cache and performs no additional engine calls. +Existing successful native authority caching remains intact; failed readiness +is retried by a later caller attempt. Retry becomes useful advice only after +the transient condition clears and cleanup permits another attempt. + +## Integration replay + +After rebasing onto main `a2d8b94dd`, runtime candidate +`2c7a09eaf3ffe26667b40081f1cf0fd2ce44a146` again passed the 99 related tests, +static and documentation gates, real Windows CLI and same-connection MCP +normal/conflict/recovery sequence. Both workflows found no residual directories; +the CLI normalized-result digest above still matched. The MCP success and error +reply sizes remained 1,126,797 and 878 bytes respectively. Upstream Android/SAZ +contract changes increased complete discovery to 2,517,904 bytes, digest +`d14588eab7404ad129907eb34a45a0db0d43233c4de829f76babd59723bed199`. +This later catalogue is not evidence of a delivery-size change caused by the +runtime-admission fix. Resource sampling above belongs to the earlier pinned +candidate; aggregate resources were not resampled for this integration replay. diff --git a/native/windows/README.md b/native/windows/README.md index 82dd5526b..c63540140 100644 --- a/native/windows/README.md +++ b/native/windows/README.md @@ -85,6 +85,17 @@ Windows failures retain their constraint, requested coordinate, Win32 code, and system message. Unsupported filesystems and reparse paths are distinct from OS access denial and missing packaged controls. +A failed runtime-parent admission additionally reports +`runtimeAllocation: "not-started"`: no runtime directory was created. The +Windows adapter uses that fact with numeric sharing/lock codes 32/33 to retain +a retryable private-runtime failure, including after successful native +authority caching. Missing allocation evidence, other codes, and opaque error +text do not establish a safe retry. Analysis errors propagate the producer's +retry disposition through JavaScript, source-map and WABT workflows; transient +failures report `category: unavailable` and retain the original reason. +Incomplete cleanup blocks an operation retry until its owned resources settle. +REA adds no automatic retry loop or new analysis cache for this behavior. + Ordinary drive-absolute paths accept backslashes, forward slashes, or mixed separators. File identity retains the original requested spelling. Separator translation does not resolve dot components, repeated separators, or extend diff --git a/native/windows/src/addon.cc b/native/windows/src/addon.cc index 0e39b84c0..bffd4cbd4 100644 --- a/native/windows/src/addon.cc +++ b/native/windows/src/addon.cc @@ -87,6 +87,8 @@ napi_value failureValue(napi_env env, const Failure& failure) { set(env, error, "requestedPath", string(env, failure.path)); set(env, error, "win32Code", number(env, failure.win32)); set(env, error, "win32Message", string(env, detail)); + if (failure.runtimeDirectoryUnallocated) + set(env, error, "runtimeAllocation", string(env, std::string("not-started"))); return error; } diff --git a/native/windows/src/authority.hpp b/native/windows/src/authority.hpp index cc713ce0b..13d7e4d60 100644 --- a/native/windows/src/authority.hpp +++ b/native/windows/src/authority.hpp @@ -33,6 +33,8 @@ struct Failure : std::runtime_error { DWORD win32; std::wstring path; std::string constraint; + // Affirmative proof for a failed runtime-parent admission; omission is unknown. + bool runtimeDirectoryUnallocated = false; Failure(std::string constraint, std::wstring path = L"", DWORD code = GetLastError()) : std::runtime_error(constraint), win32(code), path(std::move(path)), constraint(std::move(constraint)) {} diff --git a/native/windows/src/filesystem.cc b/native/windows/src/filesystem.cc index 6cfbc8e97..c8f66e5c1 100644 --- a/native/windows/src/filesystem.cc +++ b/native/windows/src/filesystem.cc @@ -212,7 +212,12 @@ static void verifyPrivate(HANDLE handle, const std::wstring& path, bool root) { } std::unique_ptr createRuntime(const std::wstring& parentPath, const std::wstring& prefix) { - auto parent = openFile(parentPath, 0, true); + std::unique_ptr parent; + try { parent = openFile(parentPath, 0, true); } + catch (Failure& failure) { + failure.runtimeDirectoryUnallocated = true; + throw; + } require(!prefix.empty() && prefix.find_first_of(L"\\/:\"<>|?*") == std::wstring::npos, "Runtime prefix must be one filename component", parentPath, ERROR_INVALID_PARAMETER); std::array random; diff --git a/scripts/verify-windows-native.mjs b/scripts/verify-windows-native.mjs index c63b6909f..3db2613ad 100644 --- a/scripts/verify-windows-native.mjs +++ b/scripts/verify-windows-native.mjs @@ -11,6 +11,7 @@ import { mkdtemp, readFile, rename, + readdir, rm, symlink, writeFile, @@ -37,6 +38,14 @@ const { WindowsPrivateRuntime, windowsPrivateRuntime } = await import( const { ghidraSessionRoot } = await import( pathToFileURL(join(packageRoot, "dist/ghidra/GhidraSessionRoot.js")) ); +const { PrivateRuntimeRoot } = await import( + pathToFileURL(join(packageRoot, "dist/process/PrivateRuntimeRoot.js")) +); +const { PrivateRuntimeRootUnavailableError } = await import( + pathToFileURL( + join(packageRoot, "dist/process/PrivateRuntimeRootUnavailableError.js"), + ) +); const workspace = await mkdtemp(join(tmpdir(), "rea-native-conformance-")); const environment = Object.entries(process.env) .filter(([, value]) => value !== undefined) @@ -140,6 +149,30 @@ process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVe ]) assert.throws(() => native.call("open", [rejected])); report.controls.ordinaryDriveSeparatorsAndRequestedIdentity = true; + // An owned runtime's DELETE lease conflicts with a fresh parent admission. + // This produces a genuine sharing failure after successful authority caching, + // without another executable, and proves that no child runtime was created. + const sharingParent = WindowsPrivateRuntime.create( + workspace, + "rea-sharing-parent-", + ); + try { + await assert.rejects( + PrivateRuntimeRoot.create({ parent: sharingParent.observation.path }), + (failure) => { + assert.ok(failure instanceof PrivateRuntimeRootUnavailableError); + assert.equal(failure.retryable, true); + assert.ok(failure.cause instanceof Error); + assert.equal(failure.cause.win32Code, 32); + assert.equal(failure.cause.runtimeAllocation, "not-started"); + return true; + }, + ); + assert.deepEqual(await readdir(sharingParent.observation.path), []); + report.controls.transientRuntimeAdmissionBeforeAllocation = true; + } finally { + await sharingParent.close(); + } const driveRoot = parse(workspace).root; const runtimeParent = ghidraSessionRoot({ base: join(driveRoot, ".Tmp", "scratch"), diff --git a/src/domain/analysisErrorCore.ts b/src/domain/analysisErrorCore.ts index 9e08f29a3..50db91ccd 100644 --- a/src/domain/analysisErrorCore.ts +++ b/src/domain/analysisErrorCore.ts @@ -100,18 +100,24 @@ export class AnalysisOutputError extends AnalysisError { export class AnalysisCapabilityUnavailableError extends AnalysisError { readonly _tag = "AnalysisCapabilityUnavailableError"; override readonly userMessage: string | undefined; + /** Incomplete cleanup must settle before the operation can be retried. */ + readonly retryable: boolean; constructor( readonly providerId: string, readonly operation: string, readonly reason: string, - options?: AnalysisErrorOptions & { readonly userMessage?: string }, + options?: AnalysisErrorOptions & { + readonly userMessage?: string; + readonly retryable?: boolean; + }, ) { super( `Provider ${providerId} cannot execute ${operation}: ${reason}`, options, ); this.userMessage = options?.userMessage; + this.retryable = options?.retryable === true && !this.cleanupIncomplete; } } diff --git a/src/domain/analysisErrorPresentation.ts b/src/domain/analysisErrorPresentation.ts index 215ed40fc..19c3a9002 100644 --- a/src/domain/analysisErrorPresentation.ts +++ b/src/domain/analysisErrorPresentation.ts @@ -1,6 +1,7 @@ import { AnalysisAccessDeniedError, AnalysisArtifactChangedError, + AnalysisCapabilityUnavailableError, AnalysisInputError, AnalysisUnsupportedTargetError, AnalysisResourceConstraintError, @@ -33,6 +34,8 @@ import type { AnalysisErrorProjection } from "./analysisErrorProjection.js"; export const analysisErrorRemediationAction = ( error: AnalysisError, ): string => { + if (error instanceof AnalysisCapabilityUnavailableError && error.retryable) + return "Review details.reason, then retry the same analysis after the condition clears."; if (error instanceof EvidenceFileError && error.published !== undefined) return "Use the published file at details.published_path and resolve the reported staging cleanup failure."; if (error instanceof AnalysisSnapshotMismatchError) @@ -112,6 +115,8 @@ export const analysisErrorRemediationAction = ( export const analysisErrorCategory = ( error: AnalysisError, ): AnalysisErrorProjection["category"] => { + if (error instanceof AnalysisCapabilityUnavailableError && error.retryable) + return "unavailable"; if ( error instanceof BinaryTargetError && error.constraint === "load_selection_rejected" @@ -220,6 +225,8 @@ export const analysisErrorUserMessage = (error: AnalysisError): string => { const hopperMessage = hopperErrorUserMessage(error); if (hopperMessage !== undefined) return hopperMessage; if (error.userMessage !== undefined) return error.userMessage; + if (error instanceof AnalysisCapabilityUnavailableError && error.retryable) + return "Required runtime controls are unavailable for this attempt."; const standardMessage = standardErrorMessage(error._tag); if (standardMessage !== undefined) return standardMessage; if (error instanceof ArtifactOperationError) return artifactMessage(error); diff --git a/src/domain/analysisErrorProjection.test.ts b/src/domain/analysisErrorProjection.test.ts index 467704b98..0766ca6e0 100644 --- a/src/domain/analysisErrorProjection.test.ts +++ b/src/domain/analysisErrorProjection.test.ts @@ -25,6 +25,91 @@ import { ProviderSelectionError } from "./providerSelectionError.js"; import { analysisErrorProjectionSchema } from "../contracts/errorSchemas.js"; import { nativeCallPartialObservationSchema } from "./native/nativeCallPartialObservation.js"; +describe("capability failure retry disposition", () => { + it.each([ + { options: {}, retryable: false, category: "unsupported_provider" }, + { + options: { retryable: false }, + retryable: false, + category: "unsupported_provider", + }, + { options: { retryable: true }, retryable: true, category: "unavailable" }, + ] as const)( + "preserves the producer's $retryable disposition", + ({ options, retryable, category }) => { + const reason = "Native self-test failed with a sharing violation"; + const projected = projectAnalysisError( + new AnalysisCapabilityUnavailableError( + "fixture-provider", + "inspect", + reason, + options, + ), + ); + expect(projected).toMatchObject({ + code: "capability_unavailable", + category, + retryable, + details: { reason }, + }); + if (retryable) { + expect(projected.message).toBe( + "Required runtime controls are unavailable for this attempt.", + ); + expect(projected.remediation.action).toContain( + "retry the same analysis", + ); + } + expect(analysisErrorProjectionSchema.safeParse(projected).success).toBe( + true, + ); + }, + ); + + it("preserves a caller-selected message for a retryable failure", () => { + expect( + projectAnalysisError( + new AnalysisCapabilityUnavailableError( + "fixture-provider", + "inspect", + "temporary failure", + { + retryable: true, + userMessage: "Selected runtime diagnostic", + }, + ), + ).message, + ).toBe("Selected runtime diagnostic"); + }); + + it("retains cleanup ownership before allowing a transient operation to retry", () => { + expect( + projectAnalysisError( + new AnalysisCapabilityUnavailableError( + "fixture-provider", + "inspect", + "temporary failure", + { + retryable: true, + cleanup: { + reason: "owned handle is still live", + resources: ["native-owner"], + }, + }, + ), + ), + ).toMatchObject({ + code: "cleanup_incomplete", + retryable: false, + details: { + cleanup: "incomplete", + resources: ["native-owner"], + execution_failure: "capability_unavailable", + }, + }); + }); +}); + it("projects a published file receipt alongside retained cleanup failure", () => { const projected = projectAnalysisError( new EvidenceFileError("write", "io", { diff --git a/src/domain/analysisErrorProjection.ts b/src/domain/analysisErrorProjection.ts index 723200e18..bfdf7eb42 100644 --- a/src/domain/analysisErrorProjection.ts +++ b/src/domain/analysisErrorProjection.ts @@ -83,6 +83,8 @@ export const projectAnalysisError = ( message: analysisErrorUserMessage(error), retryable: RETRYABLE_CODES.has(code) || + (error instanceof AnalysisCapabilityUnavailableError && + error.retryable) || (code === "resource_constraint" && error instanceof AnalysisResourceConstraintError && error.resource === "capacity"), diff --git a/src/javascript/analysis/JavaScriptAnalysisWorker.ts b/src/javascript/analysis/JavaScriptAnalysisWorker.ts index c02ddaa3e..6a587915b 100644 --- a/src/javascript/analysis/JavaScriptAnalysisWorker.ts +++ b/src/javascript/analysis/JavaScriptAnalysisWorker.ts @@ -27,10 +27,8 @@ import { ProviderAdapterError } from "../../domain/providerAdapterError.js"; import { ProviderCleanupError } from "../../domain/providerCleanupError.js"; import { ok, err, type Result } from "../../domain/result.js"; import { javaScriptRuntime } from "../../domain/runtimeVersion.js"; -import { - PrivateRuntimeRoot, - PrivateRuntimeRootUnavailableError, -} from "../../process/PrivateRuntimeRoot.js"; +import { PrivateRuntimeRoot } from "../../process/PrivateRuntimeRoot.js"; +import { PrivateRuntimeRootUnavailableError } from "../../process/PrivateRuntimeRootUnavailableError.js"; import { javaScriptWorkerMemoryLimitBytes } from "../../process/JavaScriptWorkerMemoryLimit.js"; import { ProviderProcessSupervisor, @@ -735,7 +733,7 @@ export class JavaScriptAnalysisWorker { PROVIDER, OPERATION, cause.message, - { cause }, + { cause, retryable: cause.retryable }, ) : new ProviderAdapterError(PROVIDER, OPERATION, { cause, diff --git a/src/javascript/sourceMaps/SourceMapCommand.ts b/src/javascript/sourceMaps/SourceMapCommand.ts index e86954106..9e8837952 100644 --- a/src/javascript/sourceMaps/SourceMapCommand.ts +++ b/src/javascript/sourceMaps/SourceMapCommand.ts @@ -16,10 +16,8 @@ import { ProviderCleanupError } from "../../domain/providerCleanupError.js"; import type { JsonValue } from "../../domain/jsonValue.js"; import { err, ok, type Result } from "../../domain/result.js"; import { WEB_SOURCE_MAP_LIMITS } from "../../domain/webSourceLocation.js"; -import { - PrivateRuntimeRoot, - PrivateRuntimeRootUnavailableError, -} from "../../process/PrivateRuntimeRoot.js"; +import { PrivateRuntimeRoot } from "../../process/PrivateRuntimeRoot.js"; +import { PrivateRuntimeRootUnavailableError } from "../../process/PrivateRuntimeRootUnavailableError.js"; import { ProviderStartupDeadline } from "../../process/ProviderDeadline.js"; import { javaScriptWorkerMemoryLimitBytes } from "../../process/JavaScriptWorkerMemoryLimit.js"; import { cleanupOwnedProcessGroup } from "../../process/ProcessOwnership.js"; @@ -261,7 +259,7 @@ class OwnedCodec implements SourceMapCleanupOwner { PROVIDER, OPERATION, cause.message, - { cause }, + { cause, retryable: cause.retryable }, ) : new ProviderAdapterError(PROVIDER, OPERATION, { cause, diff --git a/src/process/PrivateRuntimeRoot.ts b/src/process/PrivateRuntimeRoot.ts index 7002e2f65..c319df0de 100644 --- a/src/process/PrivateRuntimeRoot.ts +++ b/src/process/PrivateRuntimeRoot.ts @@ -3,6 +3,7 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; import { WindowsPrivateRuntime } from "../windows/WindowsPrivateRuntime.js"; +import { PrivateRuntimeRootUnavailableError } from "./PrivateRuntimeRootUnavailableError.js"; import { windowsNativeCapabilities, @@ -34,16 +35,6 @@ export const privateRuntimeRootCapability = ( ? windowsNativeCapabilities(platform).private_runtime_dacl : { available: true, reason: null, proof: "posix-mkdtemp-owner-only" }; -/** Expected failure when a runtime root's privacy boundary is unavailable. */ -export class PrivateRuntimeRootUnavailableError extends Error { - readonly code = "private-runtime-root-authority-unavailable"; - - constructor(message: string) { - super(message); - this.name = "PrivateRuntimeRootUnavailableError"; - } -} - /** * Owns a private temporary runtime root and removes it idempotently. * @@ -66,7 +57,9 @@ export class PrivateRuntimeRoot { const platform = options.platform ?? process.platform; const capability = privateRuntimeRootCapability(platform); if (!capability.available) - throw new PrivateRuntimeRootUnavailableError(capability.reason); + throw new PrivateRuntimeRootUnavailableError(capability.reason, { + retryable: capability.retryable === true, + }); if (platform === "win32") { const runtime = WindowsPrivateRuntime.create( options.parent ?? tmpdir(), diff --git a/src/process/PrivateRuntimeRootUnavailableError.ts b/src/process/PrivateRuntimeRootUnavailableError.ts new file mode 100644 index 000000000..27e7ae49d --- /dev/null +++ b/src/process/PrivateRuntimeRootUnavailableError.ts @@ -0,0 +1,14 @@ +/** Expected failure when a runtime root's privacy boundary is unavailable. */ +export class PrivateRuntimeRootUnavailableError extends Error { + readonly code = "private-runtime-root-authority-unavailable"; + readonly retryable: boolean; + + constructor( + message: string, + options?: ErrorOptions & { readonly retryable?: boolean }, + ) { + super(message, options); + this.name = "PrivateRuntimeRootUnavailableError"; + this.retryable = options?.retryable ?? false; + } +} diff --git a/src/process/WindowsAuthority.ts b/src/process/WindowsAuthority.ts index ccd041312..b25a22863 100644 --- a/src/process/WindowsAuthority.ts +++ b/src/process/WindowsAuthority.ts @@ -11,6 +11,7 @@ export type WindowsNativeCapability = readonly available: false; readonly reason: string; readonly proof: "not-proven" | "not-applicable"; + readonly retryable?: true; }; /** Security controls required before REA may claim Windows isolation. */ @@ -52,6 +53,7 @@ export const windowsNativeCapabilities = ( available: false, reason: loaded.reason, proof: "not-proven", + ...(loaded.retryable === true ? { retryable: true as const } : {}), } as const); return { job_object_process_ownership: capability, diff --git a/src/wasm/WabtArtifactProvider.test.ts b/src/wasm/WabtArtifactProvider.test.ts index 74d481de3..94c4e0c95 100644 --- a/src/wasm/WabtArtifactProvider.test.ts +++ b/src/wasm/WabtArtifactProvider.test.ts @@ -3,12 +3,11 @@ import { join } from "node:path"; import { expect, it } from "vitest"; import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js"; import { WabtArtifactProvider } from "./WabtArtifactProvider.js"; -import { - PrivateRuntimeRoot, - PrivateRuntimeRootUnavailableError, -} from "../process/PrivateRuntimeRoot.js"; +import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js"; +import { PrivateRuntimeRootUnavailableError } from "../process/PrivateRuntimeRootUnavailableError.js"; import { spawnOwnedProviderProcess } from "../process/ProviderProcess.js"; import { OwnedCommandFailure } from "../process/OwnedCommand.js"; +import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; const failureTags: Readonly> = { invalid: "AnalysisInputError", @@ -40,40 +39,51 @@ it("returns actionable unavailable without creating a workspace when tools are a }); }); -it("reports an unavailable private runtime root without a misleading WABT remediation", async () => { - const toolRoot = await createTestTempDirectory("rea-wabt-tool-root-"); - for (const tool of ["wasm-validate", "wasm-objdump", "wasm2wat"]) - await writeFile( - join(toolRoot, tool + (process.platform === "win32" ? ".exe" : "")), - "source-owned executable seam", - { mode: 0o700 }, - ); - const provider = new WabtArtifactProvider( - { REA_WABT_BIN_DIRECTORY: toolRoot }, - undefined, - () => { - throw new PrivateRuntimeRootUnavailableError( - "REA Windows native controls are unavailable: probe failure", +it.each([false, true])( + "reports an unavailable private runtime root with retryable=%s without a misleading WABT remediation", + async (retryable) => { + const toolRoot = await createTestTempDirectory("rea-wabt-tool-root-"); + for (const tool of ["wasm-validate", "wasm-objdump", "wasm2wat"]) + await writeFile( + join(toolRoot, tool + (process.platform === "win32" ? ".exe" : "")), + "source-owned executable seam", + { mode: 0o700 }, ); - }, - ); - expect( - await provider.inspect({ + const provider = new WabtArtifactProvider( + { REA_WABT_BIN_DIRECTORY: toolRoot }, + undefined, + () => { + throw new PrivateRuntimeRootUnavailableError( + "REA Windows native controls are unavailable: probe failure", + { retryable }, + ); + }, + ); + const result = await provider.inspect({ path: "/module.wasm", glue_paths: [], candidate_paths: [], - }), - ).toMatchObject({ - ok: false, - error: { - _tag: "AnalysisCapabilityUnavailableError", - userMessage: undefined, - message: expect.stringContaining( - "REA Windows native controls are unavailable", - ), - }, - }); -}); + }); + expect(result).toMatchObject({ + ok: false, + error: { + _tag: "AnalysisCapabilityUnavailableError", + retryable, + userMessage: undefined, + message: expect.stringContaining( + "REA Windows native controls are unavailable", + ), + }, + }); + if (result.ok) + throw new Error("Unavailable runtime must refuse inspection"); + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "capability_unavailable", + retryable, + category: retryable ? "unavailable" : "unsupported_provider", + }); + }, +); it.each([ "success", "invalid", diff --git a/src/wasm/WabtFailures.ts b/src/wasm/WabtFailures.ts index ef5692f1b..4f386998f 100644 --- a/src/wasm/WabtFailures.ts +++ b/src/wasm/WabtFailures.ts @@ -11,7 +11,7 @@ import { } from "../domain/analysisErrorCore.js"; import { ProviderCleanupError } from "../domain/providerCleanupError.js"; import { OwnedCommandFailure } from "../process/OwnedCommand.js"; -import { PrivateRuntimeRootUnavailableError } from "../process/PrivateRuntimeRoot.js"; +import { PrivateRuntimeRootUnavailableError } from "../process/PrivateRuntimeRootUnavailableError.js"; import { WABT_LIMITS, WABT_PROVIDER_IDENTITY } from "./WabtRelease.js"; /** Preserve lifecycle failures and diagnostic bytes instead of returning partial success. */ export const wabtFailure = ( @@ -76,7 +76,7 @@ export const wabtFailure = ( WABT_PROVIDER_IDENTITY.id, "inspect_wasm_artifact", reason, - { cause }, + { cause, retryable: cause.retryable }, ); if (phase === "configuration") return new AnalysisCapabilityUnavailableError( diff --git a/src/windows/WindowsPrivateRuntime.ts b/src/windows/WindowsPrivateRuntime.ts index e17544a49..2c361a6d0 100644 --- a/src/windows/WindowsPrivateRuntime.ts +++ b/src/windows/WindowsPrivateRuntime.ts @@ -1,6 +1,7 @@ import { win32 } from "node:path"; import { z } from "zod"; +import { PrivateRuntimeRootUnavailableError } from "../process/PrivateRuntimeRootUnavailableError.js"; import { requireWindowsNativeAuthority, @@ -23,6 +24,13 @@ const fileSchema = windowsFileIdentitySchema.extend({ handle: windowsHandleSchema, }); const runtimes = new Map(); +// Retry only a proven pre-allocation sharing/lock conflict, never opaque text +// or an operation that may already own an unpublished runtime directory. +const transientRuntimeAdmissionSchema = z.object({ + code: z.literal("ERR_REA_WINDOWS_NATIVE"), + win32Code: z.union([z.literal(32), z.literal(33)]), + runtimeAllocation: z.literal("not-started"), +}); /** Retain private DACL and path handles until native recursive cleanup settles. */ export class WindowsPrivateRuntime { @@ -40,9 +48,21 @@ export class WindowsPrivateRuntime { /** Create a new directory with its private DACL installed at creation. */ static create(parent: string, prefix: string): WindowsPrivateRuntime { const authority = requireWindowsNativeAuthority(); - const observation = runtimeSchema.parse( - authority.call("runtime_create", [parent, prefix]), - ); + let observed: unknown; + try { + observed = authority.call("runtime_create", [parent, prefix]); + } catch (cause: unknown) { + if ( + cause instanceof Error && + transientRuntimeAdmissionSchema.safeParse(cause).success + ) + throw new PrivateRuntimeRootUnavailableError(cause.message, { + cause, + retryable: true, + }); + throw cause; + } + const observation = runtimeSchema.parse(observed); const runtime = new WindowsPrivateRuntime(observation, authority); runtimes.set(windowsRuntimeIdentityKey(observation), runtime); return runtime; diff --git a/tests/boundary/javascript/sourceMapRuntimeAvailability.test.ts b/tests/boundary/javascript/sourceMapRuntimeAvailability.test.ts new file mode 100644 index 000000000..753a3bd47 --- /dev/null +++ b/tests/boundary/javascript/sourceMapRuntimeAvailability.test.ts @@ -0,0 +1,35 @@ +import { expect, it } from "vitest"; +import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; +import { runSourceMapCommand } from "../../../src/javascript/sourceMaps/SourceMapCommand.js"; +import { PrivateRuntimeRootUnavailableError } from "../../../src/process/PrivateRuntimeRootUnavailableError.js"; + +it.each([false, true])( + "preserves private-runtime retryable=%s through the source-map operation", + async (retryable) => { + const reason = "Native runtime self-test failed with a sharing violation"; + let launches = 0; + const result = await runSourceMapCommand("{}", "/selected.map", undefined, { + prepareOwnership: async () => {}, + createRuntime: async () => { + throw new PrivateRuntimeRootUnavailableError(reason, { retryable }); + }, + launcher: async () => { + launches += 1; + throw new Error("A refused runtime must not launch a codec"); + }, + }); + expect(launches).toBe(0); + if (result.ok) throw new Error("Unavailable runtime must refuse decoding"); + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "capability_unavailable", + category: retryable ? "unavailable" : "unsupported_provider", + retryable, + details: { + provider_id: "source-map-decoder", + operation: "trace_web_source_location", + reason, + }, + }); + expect(result.error.cleanupIncomplete).toBe(false); + }, +); diff --git a/tests/process-global/windowsRuntimeAdmission.test.ts b/tests/process-global/windowsRuntimeAdmission.test.ts new file mode 100644 index 000000000..a9f328a18 --- /dev/null +++ b/tests/process-global/windowsRuntimeAdmission.test.ts @@ -0,0 +1,99 @@ +import { beforeEach, expect, it, vi } from "vitest"; +import { WindowsPrivateRuntime } from "../../src/windows/WindowsPrivateRuntime.js"; +import { PrivateRuntimeRootUnavailableError } from "../../src/process/PrivateRuntimeRootUnavailableError.js"; + +const { nativeCall } = vi.hoisted(() => ({ nativeCall: vi.fn() })); +vi.mock("../../src/windows/WindowsNativeLoader.js", async (original) => ({ + ...(await original< + typeof import("../../src/windows/WindowsNativeLoader.js") + >()), + requireWindowsNativeAuthority: () => ({ call: nativeCall }), +})); +beforeEach(() => { + nativeCall.mockReset(); +}); + +it.each([ + { + label: "sharing conflict before allocation", + win32Code: 32, + runtimeAllocation: "not-started", + retryable: true, + }, + { + label: "lock conflict before allocation", + win32Code: 33, + runtimeAllocation: "not-started", + retryable: true, + }, + { label: "unknown allocation", win32Code: 32, retryable: false }, + { + label: "possible allocation", + win32Code: 32, + runtimeAllocation: "started", + retryable: false, + }, + { + label: "permission failure", + win32Code: 5, + runtimeAllocation: "not-started", + retryable: false, + }, + { + label: "string code", + win32Code: "32", + runtimeAllocation: "not-started", + retryable: false, + }, + { + label: "inconsistent error code", + win32Code: 32, + code: "EACCES", + runtimeAllocation: "not-started", + retryable: false, + }, +])( + "retains $label without guessing runtime ownership", + ({ retryable, label: _label, ...facts }) => { + const cause = Object.assign(new Error("Selected native diagnostic"), { + code: "ERR_REA_WINDOWS_NATIVE", + ...facts, + }); + nativeCall.mockImplementation(() => { + throw cause; + }); + let observed: unknown; + try { + WindowsPrivateRuntime.create("D:/selected parent", "selected-prefix-"); + } catch (failure: unknown) { + observed = failure; + } + expect(nativeCall).toHaveBeenCalledTimes(1); + expect(nativeCall).toHaveBeenCalledWith("runtime_create", [ + "D:/selected parent", + "selected-prefix-", + ]); + if (retryable) { + expect(observed).toBeInstanceOf(PrivateRuntimeRootUnavailableError); + expect(observed).toMatchObject({ + message: cause.message, + cause, + retryable: true, + }); + } else expect(observed).toBe(cause); + }, +); + +it("does not infer a transient status from a human-readable diagnostic", () => { + const cause = new Error("Open Windows path component failed (Win32 32)"); + nativeCall.mockImplementation(() => { + throw cause; + }); + let observed: unknown; + try { + WindowsPrivateRuntime.create("D:/selected parent", "selected-prefix-"); + } catch (failure: unknown) { + observed = failure; + } + expect(observed).toBe(cause); +}); From a6efa4fcefeffcf5ff5310b8154988b4be30c349 Mon Sep 17 00:00:00 2001 From: Ysopwnable <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:00:04 +0800 Subject: [PATCH 006/159] test(mcp): reuse advertised schema in inventory check (#2469) Reuse the current advertised binary_session definition for real SDK reply validation while preserving all inventory assertions and the 10-second deadline. Avoid initializing validators for unrelated tools in this correctness check. Closes #2453. Assisted-by: Reimu (gensokyo-skills:reimu-incident-triage) --- tests/boundary/mcp/serverWorkflowMcp.test.ts | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/tests/boundary/mcp/serverWorkflowMcp.test.ts b/tests/boundary/mcp/serverWorkflowMcp.test.ts index c72fca5cb..f25027b2a 100644 --- a/tests/boundary/mcp/serverWorkflowMcp.test.ts +++ b/tests/boundary/mcp/serverWorkflowMcp.test.ts @@ -122,11 +122,18 @@ it("advertises the complete currently available inventory with a session", async const listed = await client.listTools(); const names = listed.tools.map((t) => t.name); + const sessionTool = listed.tools.find( + ({ name }) => name === "binary_session", + ); + if (sessionTool === undefined) + throw new Error("Missing advertised binary_session tool"); + // Validate this reply with its current advertised schema. The SDK otherwise + // builds output validators for the entire catalog on the first cached call. const status = structured( - await client.callTool({ - name: "binary_session", - arguments: {}, - }), + await client.callTool( + { name: "binary_session", arguments: {} }, + { toolDefinition: sessionTool }, + ), ); const inventory = expandToolAvailability( toolContract("binary_session").outputSchema.parse(status).result From a86deecac5fa8434a4f50482f86b0c6ab486a936 Mon Sep 17 00:00:00 2001 From: Ysopwnable <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:06:41 +0800 Subject: [PATCH 007/159] fix: align MCP archive contracts and documentation (#2478) Reuse the domain ZIP format list for session schemas, verify full Harmony CLI/MCP lifecycles, and correct native view guidance and audited docs drift. Refresh locked dependency notices and gate inventory/provenance consistency, including notice inputs in the cached documentation gate. Fixes #2454 Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- .../references/native-and-artifacts.md | 5 +- .gitattributes | 3 + SECURITY.md | 9 +- THIRD_PARTY_NOTICES.md | 7 + docs/adb-device-analysis.md | 4 +- docs/client-configuration.md | 5 + docs/ghidra-riscv.md | 20 +- docs/installation.md | 12 +- docs/javascript-runtime-reconciliation.md | 19 +- docs/managed-code-analysis.md | 4 +- docs/raw-binary.md | 2 +- docs/releasing.md | 9 + licenses/LGPL-2.0.txt | 480 +++++ licenses/README.md | 40 +- licenses/frida-17.23.2-COPYING.txt | 50 + licenses/npm-dependencies.txt | 129 +- licenses/npm-runtime-notices.txt | 1589 ++++++++++++++++- licenses/upstream-sources.json | 10 + package.json | 6 +- scripts/generate-third-party-inventory.mjs | 120 ++ scripts/verify/harmony/analysis.mjs | 45 +- src/contracts/analysisViewToolContracts.ts | 2 +- src/contracts/toolOutputSchemaPrimitives.ts | 7 +- src/domain/binaryTargetTypes.ts | 15 +- .../acceptance/harmonyArtifactJourney.test.ts | 135 ++ .../filesystem/thirdPartyInventory.test.ts | 117 ++ tests/boundary/mcp/analysisViewMcp.test.ts | 24 + turbo.json | 1 + 28 files changed, 2798 insertions(+), 71 deletions(-) create mode 100644 licenses/LGPL-2.0.txt create mode 100644 licenses/frida-17.23.2-COPYING.txt create mode 100644 scripts/generate-third-party-inventory.mjs create mode 100644 tests/acceptance/harmonyArtifactJourney.test.ts create mode 100644 tests/boundary/filesystem/thirdPartyInventory.test.ts diff --git a/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md b/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md index e43049e36..58c1671b6 100644 --- a/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md +++ b/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md @@ -194,8 +194,9 @@ every executable in the bundle. It also lists missing or weak loads and earlier `architecture`. System paths stay undetermined because the dyld shared cache provides them. -Use `extract_artifact` when materialized files are needed. It takes no arguments -and materializes all regular files into a fresh temporary directory chosen by +Use `extract_artifact` when materialized files are needed. It extracts the active +target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md). +It materializes all regular files into a fresh temporary directory chosen by REA. Symlinks and encrypted entries are inventory facts, not extractable files. The result reports the materialized directory; callers do not choose the destination. No separate permission grant is needed. diff --git a/.gitattributes b/.gitattributes index 1774c0113..b8247b27f 100644 --- a/.gitattributes +++ b/.gitattributes @@ -3,3 +3,6 @@ # Windows command fixtures retain their native line endings. *.cmd text eol=crlf + +# Preserve whitespace in original upstream licensing texts. +licenses/npm-runtime-notices.txt whitespace=-blank-at-eol diff --git a/SECURITY.md b/SECURITY.md index 56d9ca784..1f3d73f8e 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -24,9 +24,12 @@ provider-dependent behavior to be reproduced safely. ## Security boundary -REA authenticates each local provider bridge session with a random capability -token and a current-user Unix socket. Tokens are transferred through private -session descriptors rather than process arguments or environment variables. +REA authenticates its Hopper and Ghidra bridges with per-session random +capability tokens. POSIX uses current-user Unix sockets; Windows Ghidra uses +authenticated loopback TCP with a private NTFS runtime whose DACL admits only +the current user and SYSTEM. See [Windows Ghidra P0](docs/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle). +Tokens are transferred through private session descriptors rather than process +arguments or environment variables. This is not a sandbox and does not protect against malicious processes already running as the same operating-system user. Opening an untrusted binary delegates parsing and analysis to the selected local provider with that user's diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index 3faad0dba..5fceaaa47 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -28,6 +28,13 @@ entries preserve the declared license and author metadata and include a clearly labeled standard license text. These are documented upstream notice gaps, not recovered copyright notices or a claim that attribution has been fully verified. +The optional `frida@17.23.2` npm archive and tagged Node binding source also +omit a standalone license/copyright document. Its declared +`LGPL-2.0 WITH WxWindows-exception-3.1` license is recorded with unchanged +[Frida licensing text](licenses/frida-17.23.2-COPYING.txt) and the +[GNU Library GPL v2](licenses/LGPL-2.0.txt). These are licensing supplements, +not recovered Node binding or prebuild-component copyright notices. + ## Open-source engine credits and licenses We thank the maintainers and contributors of the projects below. Their diff --git a/docs/adb-device-analysis.md b/docs/adb-device-analysis.md index e5a95c536..12fd77cd2 100644 --- a/docs/adb-device-analysis.md +++ b/docs/adb-device-analysis.md @@ -136,8 +136,8 @@ existing analysis families directly, for example ## Resource and lifecycle limits - Caller values reach adb only as positional arguments in fixed vectors; - no tool accepts an arbitrary shell command line, and UI interaction - (`input`, `am`, `monkey`) is deliberately not exposed. + no tool accepts an arbitrary shell command line, `input`, or `monkey`. + The app/activity lifecycle tools use `am start -W` and `am force-stop`. - Per-invocation budgets: client/devices 10-20 s, shell reads 20-60 s with 64 KiB-8 MiB capture caps (logcat 4 MiB, screen 32 MiB), transfers and bugreports 600 s. Exceeded budgets report `resource_constraint`; stopped diff --git a/docs/client-configuration.md b/docs/client-configuration.md index a6e254488..00e83e179 100644 --- a/docs/client-configuration.md +++ b/docs/client-configuration.md @@ -101,6 +101,11 @@ Setup and doctor follow Qwen Code's tilde and working-directory-relative home overrides. Qwen Code also discovers personal skills in `~/.agents/skills`, where setup installs the bundled workflow. +## Antigravity + +Antigravity 2.0 discovers personal skills in `~/.gemini/config/skills`. +Setup installs its selected skill copy there. + ## Gemini CLI Gemini CLI uses `$GEMINI_CLI_HOME/.gemini/settings.json` when that home override diff --git a/docs/ghidra-riscv.md b/docs/ghidra-riscv.md index e180122d7..5b3407361 100644 --- a/docs/ghidra-riscv.md +++ b/docs/ghidra-riscv.md @@ -22,16 +22,16 @@ header declarations, not proof that every instruction conforms to that ISA. Provider selection returns the specific unsupported constraint rather than guessing a compatible profile: -| Declaration | Outcome | -| ----------------------------------------------- | --------------------------------------------------------------------------- | -| Little-endian ELF64 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:64:default` + `gcc` | -| Little-endian ELF32 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:32:default` + `gcc` | -| Big-endian | refused; Ghidra 12.1.4 ships no big-endian RISC-V language | -| `ET_REL`, `ET_DYN`, other `e_type` | refused with the declared value | -| `EF_RISCV_RVE` | refused; the embedded 15-register ABI needs its own processor specification | -| `EF_RISCV_TSO` | refused; the total-store-order memory model is unverified here | -| Soft, single, or quad float ABI | refused; the pinned generic compiler specification models double-float | -| Reserved or newer `e_flags` bits | refused with the offending bits named | +| Declaration | Outcome | +| ----------------------------------------------- | ---------------------------------------------------------------------------------------- | +| Little-endian ELF64 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:64:default` + `gcc` | +| Little-endian ELF32 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:32:default` + `gcc` | +| Big-endian | refused; Ghidra 12.1.4 ships no big-endian RISC-V language | +| `ET_REL`, `ET_DYN`, other `e_type` | refused with the declared value | +| `EF_RISCV_RVE` | refused; the embedded 16-register profile (x0–x15) needs its own processor specification | +| `EF_RISCV_TSO` | refused; the total-store-order memory model is unverified here | +| Soft, single, or quad float ABI | refused; the pinned generic compiler specification models double-float | +| Reserved or newer `e_flags` bits | refused with the offending bits named | `e_type` is the only load-shape declaration this lane reads. A fixed-address `ET_EXEC` declaration does **not** establish static linkage: `PT_INTERP` and diff --git a/docs/installation.md b/docs/installation.md index 02ae1c0af..b9285a7d4 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -101,13 +101,11 @@ rea setup --yes --all-detected --install-hopper --json ### Skill destinations -The bundled skill is installed where each selected client discovers personal -skills: Claude Code uses `~/.claude/skills` (or -`$CLAUDE_CONFIG_DIR/skills` when configured). Gemini CLI uses `.agents/skills` -under its selected home; Antigravity 2.0 uses `~/.gemini/config/skills`. -Other integrations, including Qwen Code, use `~/.agents/skills`. -A mixed selection plans each distinct destination. Selecting the skill without a client uses the shared -directory. Setup leaves existing skill copies in other locations untouched. +The bundled skill is installed in each selected client's personal skill directory. +See [client configuration](client-configuration.md) for client-specific roots, +profiles, and environment overrides. A mixed selection plans each distinct +destination. Selecting the skill without a client uses `~/.agents/skills`. +Setup leaves existing skill copies in other locations untouched. `rea setup --skill-client claude_code --yes --json` refreshes and verifies the selected skill destination independently of MCP registration selection. The diff --git a/docs/javascript-runtime-reconciliation.md b/docs/javascript-runtime-reconciliation.md index af59e825b..a0a600574 100644 --- a/docs/javascript-runtime-reconciliation.md +++ b/docs/javascript-runtime-reconciliation.md @@ -1,28 +1,31 @@ # JavaScript static/runtime reconciliation REA can combine an existing static JavaScript Application Graph with one or -more passive browser, Electron page, or Node/Electron V8 Inspector captures without collapsing their different -authorities. The `reconcile_javascript_runtime` MCP tool and +more browser, Electron, or Node/Electron V8 Inspector observations while +preserving their different authorities. The `reconcile_javascript_runtime` MCP tool and `rea reconcile-javascript-runtime` CLI command accept Evidence records and return derived Evidence containing a combined JavaScript Application Graph and explicit match, ambiguity, mismatch, and unknown classifications. The operation performs no filesystem, browser, or Electron I/O. It accepts only semantically verified Evidence produced by `analyze_javascript_application`, -`inspect_web_page`, `inspect_electron_page`, or -`observe_javascript_runtime`; matching operation names alone +`inspect_web_page`, `inspect_electron_page`, `observe_javascript_runtime`, or +`capture_electron_scenario`; matching operation names alone is insufficient. Provider identity, predicate type, subject digest, target parameters, and normalized-result schemas are checked before reconciliation. +`capture_electron_scenario` supplies target-only partial observations. It does +not establish loaded-script coverage; see [Electron observation](electron-observation.md). + ## Workflow 1. Run `analyze_javascript_application` against exactly one application layer. Optional cache or assets directories can be analyzed independently and supplied as additional layers. 2. Run `inspect_web_page`, `inspect_electron_page`, or the attach-only - `observe_javascript_runtime`. Capturing script sources - is optional, but a source capture provides the strongest byte - identity. + `observe_javascript_runtime`, or supply existing `capture_electron_scenario` + Evidence with the partial scope above. Capturing script sources is optional, + but a source capture provides the strongest byte identity. 3. Pass the resulting Evidence records to `reconcile_javascript_runtime`. Browser URLs normally need an explicit URL-prefix mapping. Extracted Electron and V8 Inspector file locations are mapped automatically when runtime files remain @@ -104,7 +107,7 @@ nodes. REA evaluates compatible static candidates conservatively: Every successful cross-layer `observed_as` edge has `authority: cross-layer-reconciliation` and `state: inferred`. The underlying -artifact/AST observations and passive-CDP observations remain unchanged and +artifact/AST observations and runtime observations remain unchanged and retain their own Evidence links. ## Loaded, resident, and not observed diff --git a/docs/managed-code-analysis.md b/docs/managed-code-analysis.md index 69da49e9f..060b44bc2 100644 --- a/docs/managed-code-analysis.md +++ b/docs/managed-code-analysis.md @@ -151,7 +151,7 @@ assembly. Older member Evidence without `selection` remains readable. Start by identifying the artifact with `inspect_managed_artifact`, then inspect its types and CIL with `inspect_managed_members`. Use [member comparison](#member-comparison) -for another build, and [managed/native boundary rules](#managednative-boundary-rules) +for another build, and [managed/native boundary rules](#managed-native-boundaries) when matching declared native calls to supplied native Evidence. Static inspection does not load or execute the assembly. Reconstructed structure and unresolved native/runtime questions retain their own authority and coverage. @@ -220,6 +220,8 @@ to native function addresses requires verified provider evidence; names or vtable similarity alone do not prove that a native function implements a specific managed method. +
+ ## Managed/native boundary rules `verify_managed_native_boundaries` compares declared P/Invoke module and entry diff --git a/docs/raw-binary.md b/docs/raw-binary.md index 43fcd54f4..8e277568b 100644 --- a/docs/raw-binary.md +++ b/docs/raw-binary.md @@ -61,7 +61,7 @@ Independent raw verification streams the complete private snapshot digest and extent with bounded memory and supports cancellation during the read. Hopper and IDA refuse raw-binary targets. Windows Ghidra P0 continues to admit -PE applications only. +native PE applications and DLLs only; raw-binary targets are unsupported. ## Limitations diff --git a/docs/releasing.md b/docs/releasing.md index 37a28d532..dd3a98be3 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -218,6 +218,15 @@ release baseline and can propose an already-published version again. After publication, open a PR from the release branch back to main. Preserve main's later implementation changes and regenerate ignored build outputs from the combined contracts with the released package version. +Before merging, check whether GitHub permits merge commits: + +```bash +gh api repos/morluto/rea --jq .allow_merge_commit +``` + +If disabled, a maintainer must arrange an approved ancestry-preserving merge +path before synchronization can finish. Squash and rebase merges do not put the +released tag in main's ancestry. Do not substitute them for this step. Review and test this synchronization PR, then use a merge commit so the release tag remains in main's ancestry. Keep the released tag unchanged. diff --git a/licenses/LGPL-2.0.txt b/licenses/LGPL-2.0.txt new file mode 100644 index 000000000..ede5d5441 --- /dev/null +++ b/licenses/LGPL-2.0.txt @@ -0,0 +1,480 @@ + GNU LIBRARY GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1991 Free Software Foundation, Inc. + + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + +[This is the first released version of the library GPL. It is + numbered 2 because it goes with version 2 of the ordinary GPL.] + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +Licenses are intended to guarantee your freedom to share and change +free software--to make sure the software is free for all its users. + + This license, the Library General Public License, applies to some +specially designated Free Software Foundation software, and to any +other libraries whose authors decide to use it. You can use it for +your libraries, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if +you distribute copies of the library, or if you modify it. + + For example, if you distribute copies of the library, whether gratis +or for a fee, you must give the recipients all the rights that we gave +you. You must make sure that they, too, receive or can get the source +code. If you link a program with the library, you must provide +complete object files to the recipients so that they can relink them +with the library, after making changes to the library and recompiling +it. And you must show them these terms so they know their rights. + + Our method of protecting your rights has two steps: (1) copyright +the library, and (2) offer you this license which gives you legal +permission to copy, distribute and/or modify the library. + + Also, for each distributor's protection, we want to make certain +that everyone understands that there is no warranty for this free +library. If the library is modified by someone else and passed on, we +want its recipients to know that what they have is not the original +version, so that any problems introduced by others will not reflect on +the original authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that companies distributing free +software will individually obtain patent licenses, thus in effect +transforming the program into proprietary software. To prevent this, +we have made it clear that any patent must be licensed for everyone's +free use or not licensed at all. + + Most GNU software, including some libraries, is covered by the ordinary +GNU General Public License, which was designed for utility programs. This +license, the GNU Library General Public License, applies to certain +designated libraries. This license is quite different from the ordinary +one; be sure to read it in full, and don't assume that anything in it is +the same as in the ordinary license. + + The reason we have a separate public license for some libraries is that +they blur the distinction we usually make between modifying or adding to a +program and simply using it. Linking a program with a library, without +changing the library, is in some sense simply using the library, and is +analogous to running a utility program or application program. However, in +a textual and legal sense, the linked executable is a combined work, a +derivative of the original library, and the ordinary General Public License +treats it as such. + + Because of this blurred distinction, using the ordinary General +Public License for libraries did not effectively promote software +sharing, because most developers did not use the libraries. We +concluded that weaker conditions might promote sharing better. + + However, unrestricted linking of non-free programs would deprive the +users of those programs of all benefit from the free status of the +libraries themselves. This Library General Public License is intended to +permit developers of non-free programs to use free libraries, while +preserving your freedom as a user of such programs to change the free +libraries that are incorporated in them. (We have not seen how to achieve +this as regards changes in header files, but we have achieved it as regards +changes in the actual functions of the Library.) The hope is that this +will lead to faster development of free libraries. + + The precise terms and conditions for copying, distribution and +modification follow. Pay close attention to the difference between a +"work based on the library" and a "work that uses the library". The +former contains code derived from the library, while the latter only +works together with the library. + + Note that it is possible for a library to be covered by the ordinary +General Public License rather than by this special one. + + GNU LIBRARY GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License Agreement applies to any software library which +contains a notice placed by the copyright holder or other authorized +party saying it may be distributed under the terms of this Library +General Public License (also called "this License"). Each licensee is +addressed as "you". + + A "library" means a collection of software functions and/or data +prepared so as to be conveniently linked with application programs +(which use some of those functions and data) to form executables. + + The "Library", below, refers to any such software library or work +which has been distributed under these terms. A "work based on the +Library" means either the Library or any derivative work under +copyright law: that is to say, a work containing the Library or a +portion of it, either verbatim or with modifications and/or translated +straightforwardly into another language. (Hereinafter, translation is +included without limitation in the term "modification".) + + "Source code" for a work means the preferred form of the work for +making modifications to it. For a library, complete source code means +all the source code for all modules it contains, plus any associated +interface definition files, plus the scripts used to control compilation +and installation of the library. + + Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running a program using the Library is not restricted, and output from +such a program is covered only if its contents constitute a work based +on the Library (independent of the use of the Library in a tool for +writing it). Whether that is true depends on what the Library does +and what the program that uses the Library does. + + 1. You may copy and distribute verbatim copies of the Library's +complete source code as you receive it, in any medium, provided that +you conspicuously and appropriately publish on each copy an +appropriate copyright notice and disclaimer of warranty; keep intact +all the notices that refer to this License and to the absence of any +warranty; and distribute a copy of this License along with the +Library. + + You may charge a fee for the physical act of transferring a copy, +and you may at your option offer warranty protection in exchange for a +fee. + + 2. You may modify your copy or copies of the Library or any portion +of it, thus forming a work based on the Library, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) The modified work must itself be a software library. + + b) You must cause the files modified to carry prominent notices + stating that you changed the files and the date of any change. + + c) You must cause the whole of the work to be licensed at no + charge to all third parties under the terms of this License. + + d) If a facility in the modified Library refers to a function or a + table of data to be supplied by an application program that uses + the facility, other than as an argument passed when the facility + is invoked, then you must make a good faith effort to ensure that, + in the event an application does not supply such function or + table, the facility still operates, and performs whatever part of + its purpose remains meaningful. + + (For example, a function in a library to compute square roots has + a purpose that is entirely well-defined independent of the + application. Therefore, Subsection 2d requires that any + application-supplied function or table used by this function must + be optional: if the application does not supply it, the square + root function must still compute square roots.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Library, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Library, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote +it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Library. + +In addition, mere aggregation of another work not based on the Library +with the Library (or with a work based on the Library) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may opt to apply the terms of the ordinary GNU General Public +License instead of this License to a given copy of the Library. To do +this, you must alter all the notices that refer to this License, so +that they refer to the ordinary GNU General Public License, version 2, +instead of to this License. (If a newer version than version 2 of the +ordinary GNU General Public License has appeared, then you can specify +that version instead if you wish.) Do not make any other change in +these notices. + + Once this change is made in a given copy, it is irreversible for +that copy, so the ordinary GNU General Public License applies to all +subsequent copies and derivative works made from that copy. + + This option is useful when you wish to copy part of the code of +the Library into a program that is not a library. + + 4. You may copy and distribute the Library (or a portion or +derivative of it, under Section 2) in object code or executable form +under the terms of Sections 1 and 2 above provided that you accompany +it with the complete corresponding machine-readable source code, which +must be distributed under the terms of Sections 1 and 2 above on a +medium customarily used for software interchange. + + If distribution of object code is made by offering access to copy +from a designated place, then offering equivalent access to copy the +source code from the same place satisfies the requirement to +distribute the source code, even though third parties are not +compelled to copy the source along with the object code. + + 5. A program that contains no derivative of any portion of the +Library, but is designed to work with the Library by being compiled or +linked with it, is called a "work that uses the Library". Such a +work, in isolation, is not a derivative work of the Library, and +therefore falls outside the scope of this License. + + However, linking a "work that uses the Library" with the Library +creates an executable that is a derivative of the Library (because it +contains portions of the Library), rather than a "work that uses the +library". The executable is therefore covered by this License. +Section 6 states terms for distribution of such executables. + + When a "work that uses the Library" uses material from a header file +that is part of the Library, the object code for the work may be a +derivative work of the Library even though the source code is not. +Whether this is true is especially significant if the work can be +linked without the Library, or if the work is itself a library. The +threshold for this to be true is not precisely defined by law. + + If such an object file uses only numerical parameters, data +structure layouts and accessors, and small macros and small inline +functions (ten lines or less in length), then the use of the object +file is unrestricted, regardless of whether it is legally a derivative +work. (Executables containing this object code plus portions of the +Library will still fall under Section 6.) + + Otherwise, if the work is a derivative of the Library, you may +distribute the object code for the work under the terms of Section 6. +Any executables containing that work also fall under Section 6, +whether or not they are linked directly with the Library itself. + + 6. As an exception to the Sections above, you may also compile or +link a "work that uses the Library" with the Library to produce a +work containing portions of the Library, and distribute that work +under terms of your choice, provided that the terms permit +modification of the work for the customer's own use and reverse +engineering for debugging such modifications. + + You must give prominent notice with each copy of the work that the +Library is used in it and that the Library and its use are covered by +this License. You must supply a copy of this License. If the work +during execution displays copyright notices, you must include the +copyright notice for the Library among them, as well as a reference +directing the user to the copy of this License. Also, you must do one +of these things: + + a) Accompany the work with the complete corresponding + machine-readable source code for the Library including whatever + changes were used in the work (which must be distributed under + Sections 1 and 2 above); and, if the work is an executable linked + with the Library, with the complete machine-readable "work that + uses the Library", as object code and/or source code, so that the + user can modify the Library and then relink to produce a modified + executable containing the modified Library. (It is understood + that the user who changes the contents of definitions files in the + Library will not necessarily be able to recompile the application + to use the modified definitions.) + + b) Accompany the work with a written offer, valid for at + least three years, to give the same user the materials + specified in Subsection 6a, above, for a charge no more + than the cost of performing this distribution. + + c) If distribution of the work is made by offering access to copy + from a designated place, offer equivalent access to copy the above + specified materials from the same place. + + d) Verify that the user has already received a copy of these + materials or that you have already sent this user a copy. + + For an executable, the required form of the "work that uses the +Library" must include any data and utility programs needed for +reproducing the executable from it. However, as a special exception, +the source code distributed need not include anything that is normally +distributed (in either source or binary form) with the major +components (compiler, kernel, and so on) of the operating system on +which the executable runs, unless that component itself accompanies +the executable. + + It may happen that this requirement contradicts the license +restrictions of other proprietary libraries that do not normally +accompany the operating system. Such a contradiction means you cannot +use both them and the Library together in an executable that you +distribute. + + 7. You may place library facilities that are a work based on the +Library side-by-side in a single library together with other library +facilities not covered by this License, and distribute such a combined +library, provided that the separate distribution of the work based on +the Library and of the other library facilities is otherwise +permitted, and provided that you do these two things: + + a) Accompany the combined library with a copy of the same work + based on the Library, uncombined with any other library + facilities. This must be distributed under the terms of the + Sections above. + + b) Give prominent notice with the combined library of the fact + that part of it is a work based on the Library, and explaining + where to find the accompanying uncombined form of the same work. + + 8. You may not copy, modify, sublicense, link with, or distribute +the Library except as expressly provided under this License. Any +attempt otherwise to copy, modify, sublicense, link with, or +distribute the Library is void, and will automatically terminate your +rights under this License. However, parties who have received copies, +or rights, from you under this License will not have their licenses +terminated so long as such parties remain in full compliance. + + 9. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Library or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Library (or any work based on the +Library), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Library or works based on it. + + 10. Each time you redistribute the Library (or any work based on the +Library), the recipient automatically receives a license from the +original licensor to copy, distribute, link with or modify the Library +subject to these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 11. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Library at all. For example, if a patent +license would not permit royalty-free redistribution of the Library by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Library. + +If any portion of this section is held invalid or unenforceable under any +particular circumstance, the balance of the section is intended to apply, +and the section as a whole is intended to apply in other circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 12. If the distribution and/or use of the Library is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Library under this License may add +an explicit geographical distribution limitation excluding those countries, +so that distribution is permitted only in or among countries not thus +excluded. In such case, this License incorporates the limitation as if +written in the body of this License. + + 13. The Free Software Foundation may publish revised and/or new +versions of the Library General Public License from time to time. +Such new versions will be similar in spirit to the present version, +but may differ in detail to address new problems or concerns. + +Each version is given a distinguishing version number. If the Library +specifies a version number of this License which applies to it and +"any later version", you have the option of following the terms and +conditions either of that version or of any later version published by +the Free Software Foundation. If the Library does not specify a +license version number, you may choose any version ever published by +the Free Software Foundation. + + 14. If you wish to incorporate parts of the Library into other free +programs whose distribution conditions are incompatible with these, +write to the author to ask for permission. For software which is +copyrighted by the Free Software Foundation, write to the Free +Software Foundation; we sometimes make exceptions for this. Our +decision will be guided by the two goals of preserving the free status +of all derivatives of our free software and of promoting the sharing +and reuse of software generally. + + NO WARRANTY + + 15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO +WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW. +EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR +OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY +KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE +LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME +THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN +WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY +AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU +FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR +CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE +LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING +RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A +FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF +SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH +DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Libraries + + If you develop a new library, and you want it to be of the greatest +possible use to the public, we recommend making it free software that +everyone can redistribute and change. You can do so by permitting +redistribution under these terms (or, alternatively, under the terms of the +ordinary General Public License). + + To apply these terms, attach the following notices to the library. It is +safest to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least the +"copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This library is free software; you can redistribute it and/or + modify it under the terms of the GNU Library General Public + License as published by the Free Software Foundation; either + version 2 of the License, or (at your option) any later version. + + This library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Library General Public License for more details. + + You should have received a copy of the GNU Library General Public + License along with this library; if not, see . + +Also add information on how to contact you by electronic and paper mail. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the library, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the + library `Frob' (a library for tweaking knobs) written by James Random Hacker. + + , 1 April 1990 + Moe Ghoul, President of Vice + +That's all there is to it! diff --git a/licenses/README.md b/licenses/README.md index db0c9d05e..c811a4f76 100644 --- a/licenses/README.md +++ b/licenses/README.md @@ -4,19 +4,20 @@ Start with [THIRD_PARTY_NOTICES.md](../THIRD_PARTY_NOTICES.md) for the distincti between distributed material and separately obtained tools. These files retain upstream texts; they do not change [REA's MIT license](../LICENSE). -| Files | Provenance and scope | -| ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `npm-dependencies.txt` | Every package path in the root lockfile, with its exact version, declared license, scope, download URL, and integrity. Includes development and optional platform entries. | -| `npm-runtime-notices.txt` | Original license, copyright, and NOTICE texts collected from integrity-verified locked runtime npm archives across all platforms; also the license of Rolldown's generated runtime helpers. The archive manifest's name and version are checked. | -| `nativeaot-parser-MIT.txt` | Original Washi notice for derived NativeAOT parser code; source revision is recorded in the parser and the root notices. | -| `dotnet-runtime-MIT.txt` | Original .NET Foundation MIT text from the NativeAOT reference revision `a3b2d40328be0be3f8dd43f0e5cc925b8827b044`. | -| `ghidra-12.1.4-{LICENSE,NOTICE,DISCLAIMER}.txt` | Unchanged top-level documents from `Ghidra_12.1.4_build`. Ghidra is not bundled; these reference copies do not replace its component and GPL support-program notices. The US-government-specific disclaimer applies to Ghidra, not to REA. | -| `cfworker-MIT.txt`, `xterm-MIT.txt` | Original notices missing from their published runtime npm archives, recovered from the releases' upstream Git revisions. They are included in the corresponding runtime entries. | -| `Apache-2.0.txt`, `MIT-template.txt` | Standard license texts for explicitly documented upstream notice gaps. The MIT placeholders are not an attribution to an inferred author or year. | -| `MinGW-w64-runtime.txt` | Upstream's combined runtime copyright and license notices, snapshot from v12.0.0, for the statically linked Windows addon. This is a notice snapshot, not a pin of the compiler used in release CI. | -| `GPL-3.0.txt`, `GCC-Runtime-Library-Exception-3.1.txt` | Unchanged GPLv3 and GCC runtime exception texts from GCC 14.2.0. The exception permits eligible compiled combinations under its conditions; the runtime keeps its own license. This snapshot does not assert that every release uses GCC 14.2.0. | -| `Node-22.19.0-LICENSE.txt` | The original Node 22.19.0 license and third-party notices. The Windows addon uses Node-API headers from that release; the complete Node runtime and all components listed in this reference document are not bundled. | -| `upstream-sources.json` | Source URLs and SHA-256 digests for the new unchanged reference and supplemental texts. These digests identify the preserved local texts. | +| Files | Provenance and scope | +| ------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `npm-dependencies.txt` | Every package path in the root lockfile, with its exact version, declared license, scope, download URL, and integrity. Includes development and optional platform entries. | +| `npm-runtime-notices.txt` | Original license, copyright, and NOTICE texts collected from integrity-verified locked runtime npm archives across all platforms; also the license of Rolldown's generated runtime helpers. The archive manifest's name and version are checked. | +| `nativeaot-parser-MIT.txt` | Original Washi notice for derived NativeAOT parser code; source revision is recorded in the parser and the root notices. | +| `dotnet-runtime-MIT.txt` | Original .NET Foundation MIT text from the NativeAOT reference revision `a3b2d40328be0be3f8dd43f0e5cc925b8827b044`. | +| `ghidra-12.1.4-{LICENSE,NOTICE,DISCLAIMER}.txt` | Unchanged top-level documents from `Ghidra_12.1.4_build`. Ghidra is not bundled; these reference copies do not replace its component and GPL support-program notices. The US-government-specific disclaimer applies to Ghidra, not to REA. | +| `cfworker-MIT.txt`, `xterm-MIT.txt` | Original notices missing from their published runtime npm archives, recovered from the releases' upstream Git revisions. They are included in the corresponding runtime entries. | +| `Apache-2.0.txt`, `MIT-template.txt` | Standard license texts for explicitly documented upstream notice gaps. The MIT placeholders are not an attribution to an inferred author or year. | +| `frida-17.23.2-COPYING.txt`, `LGPL-2.0.txt` | Unchanged licensing supplements for the optional Frida dependency: tagged Frida wxWindows notice and GNU Library GPL v2 text. The npm archive and tagged Node binding omit a standalone license/copyright document; these texts do not recover binding or prebuild-component attribution. | +| `MinGW-w64-runtime.txt` | Upstream's combined runtime copyright and license notices, snapshot from v12.0.0, for the statically linked Windows addon. This is a notice snapshot, not a pin of the compiler used in release CI. | +| `GPL-3.0.txt`, `GCC-Runtime-Library-Exception-3.1.txt` | Unchanged GPLv3 and GCC runtime exception texts from GCC 14.2.0. The exception permits eligible compiled combinations under its conditions; the runtime keeps its own license. This snapshot does not assert that every release uses GCC 14.2.0. | +| `Node-22.19.0-LICENSE.txt` | The original Node 22.19.0 license and third-party notices. The Windows addon uses Node-API headers from that release; the complete Node runtime and all components listed in this reference document are not bundled. | +| `upstream-sources.json` | Source URLs and SHA-256 digests for the new unchanged reference and supplemental texts. These digests identify the preserved local texts. | Additional preserved originals live at [evmole/LICENSE](../third_party/evmole/LICENSE), @@ -40,6 +41,10 @@ only license and author metadata: their missing original copyright documents remain explicitly identified in the runtime texts. Do not describe those entries as complete recovered attribution. +`frida@17.23.2` also lacks an original standalone notice in its npm archive and +tagged Node source. Its declared license and the preserved licensing supplements +are recorded explicitly without inferring a copyright holder or year. + These records do not establish the legal accuracy of upstream metadata or prove that every upstream source-file attribution has been supplied in an archive. Review and refresh them when dependencies change; compare reference @@ -52,3 +57,12 @@ runtime/component notices with the retained snapshots and update them if they differ. Do not assume that a compiler change leaves redistribution requirements unchanged. Preserve applicable originals with any additionally bundled engine; top-level reference copies alone may not cover its entire distribution. + +Generate the exact locked inventory with `npm run licenses:generate`. After +refreshing verified runtime notices, update their header fingerprint to the +generator's reported `runtime_dependency_fingerprint`, then run +`npm run licenses:check`; `docs:check` also runs this check. It compares the +inventory, runtime package identities, +license declarations, archive URLs/integrity, runtime dependency fingerprint, +and preserved reference digests. It does not verify legal completeness or +replace inspection of original upstream notices. diff --git a/licenses/frida-17.23.2-COPYING.txt b/licenses/frida-17.23.2-COPYING.txt new file mode 100644 index 000000000..b01d49c67 --- /dev/null +++ b/licenses/frida-17.23.2-COPYING.txt @@ -0,0 +1,50 @@ + wxWindows Library Licence, Version 3.1 + ====================================== + +Copyright (c) 1998-2005 Julian Smart, Robert Roebling et al + +Everyone is permitted to copy and distribute verbatim copies +of this licence document, but changing it is not allowed. + + WXWINDOWS LIBRARY LICENCE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + +This library is free software; you can redistribute it and/or modify it +under the terms of the GNU Library General Public Licence as published by +the Free Software Foundation; either version 2 of the Licence, or (at your +option) any later version. + +This library is distributed in the hope that it will be useful, but WITHOUT +ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +FITNESS FOR A PARTICULAR PURPOSE. See the GNU Library General Public +Licence for more details. + +You should have received a copy of the GNU Library General Public Licence +along with this software, usually in a file named COPYING.LIB. If not, +write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth +Floor, Boston, MA 02110-1301 USA. + +EXCEPTION NOTICE + +1. As a special exception, the copyright holders of this library give +permission for additional uses of the text contained in this release of the +library as licenced under the wxWindows Library Licence, applying either +version 3.1 of the Licence, or (at your option) any later version of the +Licence as published by the copyright holders of version 3.1 of the Licence +document. + +2. The exception is that you may use, copy, link, modify and distribute +under your own terms, binary object code versions of works based on the +Library. + +3. If you copy code from files distributed under the terms of the GNU +General Public Licence or the GNU Library General Public Licence into a +copy of this library, as this licence permits, the exception does not apply +to the code that you add in this way. To avoid misleading anyone as to the +status of such modified files, you must delete this exception notice from +such code and/or adjust the licensing conditions notice accordingly. + +4. If you write modifications of your own for this library, it is your +choice whether to permit this exception to apply to your modifications. If +you do not wish that, you must delete the exception notice from such code +and/or adjust the licensing conditions notice accordingly. diff --git a/licenses/npm-dependencies.txt b/licenses/npm-dependencies.txt index 7ae0aa0ad..ce45f7459 100644 --- a/licenses/npm-dependencies.txt +++ b/licenses/npm-dependencies.txt @@ -2,7 +2,7 @@ REA npm dependency inventory Prepared from the root package-lock.json; refresh this inventory when locked dependencies change. License identifiers are upstream lockfile declarations, not a legal assessment. Runtime includes transitive and optional platform packages; development includes build, test and website tooling. -Locked package entries: 561; runtime: 148. +Locked package entries: 586; runtime: 174. Full runtime copyright/license/NOTICE texts: npm-runtime-notices.txt. @algolia/abtesting@1.25.0 | development | MIT @@ -1375,11 +1375,31 @@ base64-js@1.5.1 | runtime | MIT Package: https://www.npmjs.com/package/base64-js/v/1.5.1 Archive: https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz Integrity: sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA== +bindings@1.5.0 | runtime, optional | MIT + Lockfile path: node_modules/bindings + Package: https://www.npmjs.com/package/bindings/v/1.5.0 + Archive: https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz + Integrity: sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ== birpc@2.9.0 | development | MIT Lockfile path: node_modules/birpc Package: https://www.npmjs.com/package/birpc/v/2.9.0 Archive: https://registry.npmjs.org/birpc/-/birpc-2.9.0.tgz Integrity: sha512-KrayHS5pBi69Xi9JmvoqrIgYGDkD6mcSe/i6YKi3w5kekCLzrX4+nawcXqrj2tIp50Kw/mT/s3p+GVK0A0sKxw== +bl@4.1.0 | runtime, optional | MIT + Lockfile path: node_modules/bl + Package: https://www.npmjs.com/package/bl/v/4.1.0 + Archive: https://registry.npmjs.org/bl/-/bl-4.1.0.tgz + Integrity: sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w== +buffer@5.7.1 | runtime, optional | MIT + Lockfile path: node_modules/bl/node_modules/buffer + Package: https://www.npmjs.com/package/buffer/v/5.7.1 + Archive: https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz + Integrity: sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ== +readable-stream@3.6.2 | runtime, optional | MIT + Lockfile path: node_modules/bl/node_modules/readable-stream + Package: https://www.npmjs.com/package/readable-stream/v/3.6.2 + Archive: https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz + Integrity: sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA== brace-expansion@5.0.12 | runtime | MIT Lockfile path: node_modules/brace-expansion Package: https://www.npmjs.com/package/brace-expansion/v/5.0.12 @@ -1430,6 +1450,11 @@ character-entities-legacy@3.0.0 | development | MIT Package: https://www.npmjs.com/package/character-entities-legacy/v/3.0.0 Archive: https://registry.npmjs.org/character-entities-legacy/-/character-entities-legacy-3.0.0.tgz Integrity: sha512-RpPp0asT/6ufRm//AJVwpViZbGM/MkjQFxJccQRHmISF/22NBtsHqAWmL+/pmkPWoIUJdWyeVleTl1wydHATVQ== +chownr@1.1.4 | runtime, optional | ISC + Lockfile path: node_modules/chownr + Package: https://www.npmjs.com/package/chownr/v/1.1.4 + Archive: https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz + Integrity: sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg== clean-git-ref@2.0.1 | runtime | Apache-2.0 Lockfile path: node_modules/clean-git-ref Package: https://www.npmjs.com/package/clean-git-ref/v/2.0.1 @@ -1485,6 +1510,11 @@ decompress-response@6.0.0 | runtime | MIT Package: https://www.npmjs.com/package/decompress-response/v/6.0.0 Archive: https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz Integrity: sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ== +deep-extend@0.6.0 | runtime, optional | MIT + Lockfile path: node_modules/deep-extend + Package: https://www.npmjs.com/package/deep-extend/v/0.6.0 + Archive: https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz + Integrity: sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA== define-data-property@1.1.4 | runtime | MIT Lockfile path: node_modules/define-data-property Package: https://www.npmjs.com/package/define-data-property/v/1.1.4 @@ -1495,7 +1525,7 @@ dequal@2.0.3 | development | MIT Package: https://www.npmjs.com/package/dequal/v/2.0.3 Archive: https://registry.npmjs.org/dequal/-/dequal-2.0.3.tgz Integrity: sha512-0je+qPKHEMohvfRTCEo3CrPG6cAzAYgmzKyxRiYSSDkS6eGJdyVJm7WaYA5ECaAD9wLB2T4EEeymA5aFVcYXCA== -detect-libc@2.1.2 | development | Apache-2.0 +detect-libc@2.1.2 | runtime | Apache-2.0 Lockfile path: node_modules/detect-libc Package: https://www.npmjs.com/package/detect-libc/v/2.1.2 Archive: https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz @@ -1530,6 +1560,11 @@ emoji-regex-xs@1.0.0 | development | MIT Package: https://www.npmjs.com/package/emoji-regex-xs/v/1.0.0 Archive: https://registry.npmjs.org/emoji-regex-xs/-/emoji-regex-xs-1.0.0.tgz Integrity: sha512-LRlerrMYoIDrT6jgpeZ2YYl/L8EulRTt5hQcYjy5AInh7HWXKimpqx68aknBFpGL2+/IcogTcaydJEgaTmOpDg== +end-of-stream@1.4.5 | runtime, optional | MIT + Lockfile path: node_modules/end-of-stream + Package: https://www.npmjs.com/package/end-of-stream/v/1.4.5 + Archive: https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz + Integrity: sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg== entities@7.0.1 | development | BSD-2-Clause Lockfile path: node_modules/entities Package: https://www.npmjs.com/package/entities/v/7.0.1 @@ -1595,6 +1630,11 @@ evmole@0.9.3 | runtime | MIT Package: https://www.npmjs.com/package/evmole/v/0.9.3 Archive: https://registry.npmjs.org/evmole/-/evmole-0.9.3.tgz Integrity: sha512-UhO5HMAoEPCupSxvBJPlm/L0Qk/KVJ/OJo3r4EyhXADIEHqU5aTkAzsBfzk4lXGjqxVnaYbh1GPSmZtmSHFnQQ== +expand-template@2.0.3 | runtime, optional | (MIT OR WTFPL) + Lockfile path: node_modules/expand-template + Package: https://www.npmjs.com/package/expand-template/v/2.0.3 + Archive: https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz + Integrity: sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg== expect-type@1.4.0 | development | Apache-2.0 Lockfile path: node_modules/expect-type Package: https://www.npmjs.com/package/expect-type/v/1.4.0 @@ -1640,6 +1680,11 @@ fdir@6.5.0 | development | MIT Package: https://www.npmjs.com/package/fdir/v/6.5.0 Archive: https://registry.npmjs.org/fdir/-/fdir-6.5.0.tgz Integrity: sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg== +file-uri-to-path@1.0.0 | runtime, optional | MIT + Lockfile path: node_modules/file-uri-to-path + Package: https://www.npmjs.com/package/file-uri-to-path/v/1.0.0 + Archive: https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz + Integrity: sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw== focus-trap@7.8.0 | development | MIT Lockfile path: node_modules/focus-trap Package: https://www.npmjs.com/package/focus-trap/v/7.8.0 @@ -1655,6 +1700,16 @@ formatly@0.7.1 | development | MIT Package: https://www.npmjs.com/package/formatly/v/0.7.1 Archive: https://registry.npmjs.org/formatly/-/formatly-0.7.1.tgz Integrity: sha512-ESuCEJE0SGgabl9q+zABI8LhJDmF+8nhpuqDzuM48HfJ9QTD5o6PqFJDFZY7w6f8xsh5lixuj5nJsQNvYOWvKQ== +frida@17.23.2 | runtime, optional | LGPL-2.0 WITH WxWindows-exception-3.1 + Lockfile path: node_modules/frida + Package: https://www.npmjs.com/package/frida/v/17.23.2 + Archive: https://registry.npmjs.org/frida/-/frida-17.23.2.tgz + Integrity: sha512-gRCmmTUFdEILyHRN1b66erRoXafwsfWTqXyoBBqkSCFX0Ue7Jhvs/KhA3JO0XrgITNlPd91QAGsFvWzh0mhVgg== +fs-constants@1.0.0 | runtime, optional | MIT + Lockfile path: node_modules/fs-constants + Package: https://www.npmjs.com/package/fs-constants/v/1.0.0 + Archive: https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz + Integrity: sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow== fsevents@2.3.3 | development, optional | MIT Lockfile path: node_modules/fsevents Package: https://www.npmjs.com/package/fsevents/v/2.3.3 @@ -1685,6 +1740,11 @@ get-tsconfig@4.14.3 | development | MIT Package: https://www.npmjs.com/package/get-tsconfig/v/4.14.3 Archive: https://registry.npmjs.org/get-tsconfig/-/get-tsconfig-4.14.3.tgz Integrity: sha512-++QEw4DIY7WGoukz+/+A/8dGYPT9l9yIadnmSgZ8Rjr3YVSVDipQSO9CdnJo9ePqFqUUqh+wk9uIaoiAwsiPkA== +github-from-package@0.0.0 | runtime, optional | MIT + Lockfile path: node_modules/github-from-package + Package: https://www.npmjs.com/package/github-from-package/v/0.0.0 + Archive: https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz + Integrity: sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw== glob@13.0.6 | runtime | BlueOak-1.0.0 Lockfile path: node_modules/glob Package: https://www.npmjs.com/package/glob/v/13.0.6 @@ -1775,6 +1835,11 @@ inherits@2.0.4 | runtime | ISC Package: https://www.npmjs.com/package/inherits/v/2.0.4 Archive: https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz Integrity: sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ== +ini@1.3.8 | runtime, optional | ISC + Lockfile path: node_modules/ini + Package: https://www.npmjs.com/package/ini/v/1.3.8 + Archive: https://registry.npmjs.org/ini/-/ini-1.3.8.tgz + Integrity: sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew== is-callable@1.2.7 | runtime | MIT Lockfile path: node_modules/is-callable Package: https://www.npmjs.com/package/is-callable/v/1.2.7 @@ -2080,11 +2145,26 @@ mitt@3.0.1 | development | MIT Package: https://www.npmjs.com/package/mitt/v/3.0.1 Archive: https://registry.npmjs.org/mitt/-/mitt-3.0.1.tgz Integrity: sha512-vKivATfr97l2/QBCYAkXYDbrIWPM2IIKEl7YPhjCvKlG3kE2gm+uBo6nEXK3M5/Ffh/FLpKExzOQ3JJoJGFKBw== +mkdirp-classic@0.5.3 | runtime, optional | MIT + Lockfile path: node_modules/mkdirp-classic + Package: https://www.npmjs.com/package/mkdirp-classic/v/0.5.3 + Archive: https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz + Integrity: sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A== nanoid@3.3.20 | development | MIT Lockfile path: node_modules/nanoid Package: https://www.npmjs.com/package/nanoid/v/3.3.20 Archive: https://registry.npmjs.org/nanoid/-/nanoid-3.3.20.tgz Integrity: sha512-uKdg2G3GNCKQn9byYOpxbGqrT2fGO5KRt5J/8b3pok8rT6qxGWF6hxMyJiEYtAf+FVyYuD9hRaDqX5uPFYJ4ZQ== +napi-build-utils@2.0.0 | runtime, optional | MIT + Lockfile path: node_modules/napi-build-utils + Package: https://www.npmjs.com/package/napi-build-utils/v/2.0.0 + Archive: https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz + Integrity: sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA== +node-abi@3.98.0 | runtime, optional | MIT + Lockfile path: node_modules/node-abi + Package: https://www.npmjs.com/package/node-abi/v/3.98.0 + Archive: https://registry.npmjs.org/node-abi/-/node-abi-3.98.0.tgz + Integrity: sha512-Llh8noNw7s15m+rmIrZnZQbe4gLyasL9loT74rC92PkmwWKeEedKvc7o6pjp8B3MhrYT/Cl846NOYZbz1wmddw== obug@2.2.1 | development | MIT Lockfile path: node_modules/obug Package: https://www.npmjs.com/package/obug/v/2.2.1 @@ -2230,6 +2310,11 @@ preact@10.29.8 | development | MIT Package: https://www.npmjs.com/package/preact/v/10.29.8 Archive: https://registry.npmjs.org/preact/-/preact-10.29.8.tgz Integrity: sha512-ej2aVZ+vZ8WO7tvlQWRM9N63A0KzF9q4mWJfDUHgYaIofWY9hu74QdnQrjoPMmZi2/nZ5gN0bJCQF49xQqx09Q== +prebuild-install@7.1.3 | runtime, optional | MIT + Lockfile path: node_modules/prebuild-install + Package: https://www.npmjs.com/package/prebuild-install/v/7.1.3 + Archive: https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz + Integrity: sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug== process@0.11.10 | runtime | MIT Lockfile path: node_modules/process Package: https://www.npmjs.com/package/process/v/0.11.10 @@ -2245,6 +2330,11 @@ property-information@7.2.0 | development | MIT Package: https://www.npmjs.com/package/property-information/v/7.2.0 Archive: https://registry.npmjs.org/property-information/-/property-information-7.2.0.tgz Integrity: sha512-IAtzIB6sUiWaJYrX9smp3V46pBGbBeLFRGdh25kg1334VcBlD8HzhPeNIWQH9zhGmo2itIe25EHt9dQP7G5hmg== +pump@3.0.4 | runtime, optional | MIT + Lockfile path: node_modules/pump + Package: https://www.npmjs.com/package/pump/v/3.0.4 + Archive: https://registry.npmjs.org/pump/-/pump-3.0.4.tgz + Integrity: sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA== pure-rand@8.4.2 | development | MIT Lockfile path: node_modules/pure-rand Package: https://www.npmjs.com/package/pure-rand/v/8.4.2 @@ -2255,6 +2345,16 @@ quick-format-unescaped@4.0.4 | runtime | MIT Package: https://www.npmjs.com/package/quick-format-unescaped/v/4.0.4 Archive: https://registry.npmjs.org/quick-format-unescaped/-/quick-format-unescaped-4.0.4.tgz Integrity: sha512-tYC1Q1hgyRuHgloV/YXs2w15unPVh8qfu/qCTfhTYamaw7fyhumKa2yGpdSo87vY32rIclj+4fWYQXUMs9EHvg== +rc@1.2.8 | runtime, optional | (BSD-2-Clause OR MIT OR Apache-2.0) + Lockfile path: node_modules/rc + Package: https://www.npmjs.com/package/rc/v/1.2.8 + Archive: https://registry.npmjs.org/rc/-/rc-1.2.8.tgz + Integrity: sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw== +strip-json-comments@2.0.1 | runtime, optional | MIT + Lockfile path: node_modules/rc/node_modules/strip-json-comments + Package: https://www.npmjs.com/package/strip-json-comments/v/2.0.1 + Archive: https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz + Integrity: sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ== readable-stream@4.7.0 | runtime | MIT Lockfile path: node_modules/readable-stream Package: https://www.npmjs.com/package/readable-stream/v/4.7.0 @@ -2465,6 +2565,21 @@ tabbable@6.5.0 | development | MIT Package: https://www.npmjs.com/package/tabbable/v/6.5.0 Archive: https://registry.npmjs.org/tabbable/-/tabbable-6.5.0.tgz Integrity: sha512-wieBHXygIm7OyQOu5hQlkk62/WyCFYGlWg7L6/ZCUZwx0o398Zkn4pVmMyfYhfMG8kGrj/Krt8eIk6UKC6VzwA== +tar-fs@2.1.5 | runtime, optional | MIT + Lockfile path: node_modules/tar-fs + Package: https://www.npmjs.com/package/tar-fs/v/2.1.5 + Archive: https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.5.tgz + Integrity: sha512-OboTd8mmMhZDNPV+UjQcK9yKAatXu2aJ+r1w4im1Otd4M4fl2hwvdoXUxIYHFTHWK/3y3FarBP70v3vwmGlOxw== +tar-stream@2.2.0 | runtime, optional | MIT + Lockfile path: node_modules/tar-stream + Package: https://www.npmjs.com/package/tar-stream/v/2.2.0 + Archive: https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz + Integrity: sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ== +readable-stream@3.6.2 | runtime, optional | MIT + Lockfile path: node_modules/tar-stream/node_modules/readable-stream + Package: https://www.npmjs.com/package/readable-stream/v/3.6.2 + Archive: https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz + Integrity: sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA== thread-stream@4.2.0 | runtime | MIT Lockfile path: node_modules/thread-stream Package: https://www.npmjs.com/package/thread-stream/v/4.2.0 @@ -2515,6 +2630,11 @@ tslib@2.8.1 | development, optional | 0BSD Package: https://www.npmjs.com/package/tslib/v/2.8.1 Archive: https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz Integrity: sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w== +tunnel-agent@0.6.0 | runtime, optional | Apache-2.0 + Lockfile path: node_modules/tunnel-agent + Package: https://www.npmjs.com/package/tunnel-agent/v/0.6.0 + Archive: https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz + Integrity: sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w== turbo@2.11.7 | development | MIT Lockfile path: node_modules/turbo Package: https://www.npmjs.com/package/turbo/v/2.11.7 @@ -2570,6 +2690,11 @@ unist-util-visit-parents@6.0.2 | development | MIT Package: https://www.npmjs.com/package/unist-util-visit-parents/v/6.0.2 Archive: https://registry.npmjs.org/unist-util-visit-parents/-/unist-util-visit-parents-6.0.2.tgz Integrity: sha512-goh1s1TBrqSqukSc8wrjwWhL0hiJxgA8m4kFxGlQ+8FYQ3C/m11FcTs4YYem7V664AhHVvgoQLk890Ssdsr2IQ== +util-deprecate@1.0.2 | runtime, optional | MIT + Lockfile path: node_modules/util-deprecate + Package: https://www.npmjs.com/package/util-deprecate/v/1.0.2 + Archive: https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz + Integrity: sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw== vfile@6.0.3 | development | MIT Lockfile path: node_modules/vfile Package: https://www.npmjs.com/package/vfile/v/6.0.3 diff --git a/licenses/npm-runtime-notices.txt b/licenses/npm-runtime-notices.txt index f34cd8ee4..d981aa68a 100644 --- a/licenses/npm-runtime-notices.txt +++ b/licenses/npm-runtime-notices.txt @@ -1,7 +1,7 @@ REA third-party npm runtime copyright, license and NOTICE texts Collected from integrity-verified locked npm archives. Release-specific supplements and upstream notice gaps are identified in their entries. External engines are separately obtained; see THIRD_PARTY_NOTICES.md. -Dependency fingerprint: 0b00d9530d00182e3f2b163bf27feea056c24bb6600806cab74061756ffdc36a +Runtime dependency fingerprint (SHA-256): 5de9cf97e7c327da1892efeff30b40507e4c7f8e496b5d305b8016c162bc15e3 ===== @babel/helper-string-parser@8.0.0 ===== Declared license: MIT @@ -6630,3 +6630,1590 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +===== bindings@1.5.0 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz +Integrity: sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE.md --- + +(The MIT License) + +Copyright (c) 2012 Nathan Rajlich <nathan@tootallnate.net> + +Permission is hereby granted, free of charge, to any person obtaining +a copy of this software and associated documentation files (the +'Software'), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice shall be +included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY +CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, +TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + + + +===== bl@4.1.0 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/bl/-/bl-4.1.0.tgz +Integrity: sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE.md --- + +The MIT License (MIT) +===================== + +Copyright (c) 2013-2019 bl contributors +---------------------------------- + +*bl contributors listed at * + +Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + + + +===== buffer@5.7.1 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz +Integrity: sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +The MIT License (MIT) + +Copyright (c) Feross Aboukhadijeh, and other contributors. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. + + + +===== chownr@1.1.4 ===== +Declared license: ISC +Archive: https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz +Integrity: sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +The ISC License + +Copyright (c) Isaac Z. Schlueter and Contributors + +Permission to use, copy, modify, and/or distribute this software for any +purpose with or without fee is hereby granted, provided that the above +copyright notice and this permission notice appear in all copies. + +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR +IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + + + +===== deep-extend@0.6.0 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz +Integrity: sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +The MIT License (MIT) + +Copyright (c) 2013-2018, Viacheslav Lotsmanov + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal in +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of +the Software, and to permit persons to whom the Software is furnished to do so, +subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR +COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER +IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + + + +===== detect-libc@2.1.2 ===== +Declared license: Apache-2.0 +Archive: https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz +Integrity: sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "{}" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright {yyyy} {name of copyright owner} + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + + +===== end-of-stream@1.4.5 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz +Integrity: sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +The MIT License (MIT) + +Copyright (c) 2014 Mathias Buus + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. + + +===== expand-template@2.0.3 ===== +Declared license: (MIT OR WTFPL) +Archive: https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz +Integrity: sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +The MIT License (MIT) + +Copyright (c) 2018 Lars-Magnus Skog + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. + + +===== file-uri-to-path@1.0.0 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz +Integrity: sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +Copyright (c) 2014 Nathan Rajlich + +Permission is hereby granted, free of charge, to any person obtaining +a copy of this software and associated documentation files (the +'Software'), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice shall be +included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY +CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, +TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + + + +===== frida@17.23.2 ===== +Declared license: LGPL-2.0 WITH WxWindows-exception-3.1 +Archive: https://registry.npmjs.org/frida/-/frida-17.23.2.tgz +Integrity: sha512-gRCmmTUFdEILyHRN1b66erRoXafwsfWTqXyoBBqkSCFX0Ue7Jhvs/KhA3JO0XrgITNlPd91QAGsFvWzh0mhVgg== +Scope: npm runtime dependency, including optional platform packages. + +--- Declared metadata and supplemental licensing texts --- + +The integrity-verified npm archive and tagged frida-node source omit a standalone license/copyright document. The package declares LGPL-2.0 WITH WxWindows-exception-3.1. No Frida copyright holder or year is inferred. +The following unchanged Frida 17.23.2 COPYING and GNU Library GPL v2 texts are licensing supplements. The copyright in COPYING belongs to that license document; it is not recovered attribution for the Node binding or every component in its downloaded prebuild. + +Authors as declared by upstream package metadata: ["Frida Developers"] +Frida Node source: https://github.com/frida/frida-node/tree/11d56293686c5494f538247ccf89f08ab57912ef +License text provenance and SHA-256: licenses/upstream-sources.json + +--- licenses/frida-17.23.2-COPYING.txt --- + + wxWindows Library Licence, Version 3.1 + ====================================== + +Copyright (c) 1998-2005 Julian Smart, Robert Roebling et al + +Everyone is permitted to copy and distribute verbatim copies +of this licence document, but changing it is not allowed. + + WXWINDOWS LIBRARY LICENCE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + +This library is free software; you can redistribute it and/or modify it +under the terms of the GNU Library General Public Licence as published by +the Free Software Foundation; either version 2 of the Licence, or (at your +option) any later version. + +This library is distributed in the hope that it will be useful, but WITHOUT +ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +FITNESS FOR A PARTICULAR PURPOSE. See the GNU Library General Public +Licence for more details. + +You should have received a copy of the GNU Library General Public Licence +along with this software, usually in a file named COPYING.LIB. If not, +write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth +Floor, Boston, MA 02110-1301 USA. + +EXCEPTION NOTICE + +1. As a special exception, the copyright holders of this library give +permission for additional uses of the text contained in this release of the +library as licenced under the wxWindows Library Licence, applying either +version 3.1 of the Licence, or (at your option) any later version of the +Licence as published by the copyright holders of version 3.1 of the Licence +document. + +2. The exception is that you may use, copy, link, modify and distribute +under your own terms, binary object code versions of works based on the +Library. + +3. If you copy code from files distributed under the terms of the GNU +General Public Licence or the GNU Library General Public Licence into a +copy of this library, as this licence permits, the exception does not apply +to the code that you add in this way. To avoid misleading anyone as to the +status of such modified files, you must delete this exception notice from +such code and/or adjust the licensing conditions notice accordingly. + +4. If you write modifications of your own for this library, it is your +choice whether to permit this exception to apply to your modifications. If +you do not wish that, you must delete the exception notice from such code +and/or adjust the licensing conditions notice accordingly. + + +--- licenses/LGPL-2.0.txt --- + + GNU LIBRARY GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1991 Free Software Foundation, Inc. + + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + +[This is the first released version of the library GPL. It is + numbered 2 because it goes with version 2 of the ordinary GPL.] + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +Licenses are intended to guarantee your freedom to share and change +free software--to make sure the software is free for all its users. + + This license, the Library General Public License, applies to some +specially designated Free Software Foundation software, and to any +other libraries whose authors decide to use it. You can use it for +your libraries, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if +you distribute copies of the library, or if you modify it. + + For example, if you distribute copies of the library, whether gratis +or for a fee, you must give the recipients all the rights that we gave +you. You must make sure that they, too, receive or can get the source +code. If you link a program with the library, you must provide +complete object files to the recipients so that they can relink them +with the library, after making changes to the library and recompiling +it. And you must show them these terms so they know their rights. + + Our method of protecting your rights has two steps: (1) copyright +the library, and (2) offer you this license which gives you legal +permission to copy, distribute and/or modify the library. + + Also, for each distributor's protection, we want to make certain +that everyone understands that there is no warranty for this free +library. If the library is modified by someone else and passed on, we +want its recipients to know that what they have is not the original +version, so that any problems introduced by others will not reflect on +the original authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that companies distributing free +software will individually obtain patent licenses, thus in effect +transforming the program into proprietary software. To prevent this, +we have made it clear that any patent must be licensed for everyone's +free use or not licensed at all. + + Most GNU software, including some libraries, is covered by the ordinary +GNU General Public License, which was designed for utility programs. This +license, the GNU Library General Public License, applies to certain +designated libraries. This license is quite different from the ordinary +one; be sure to read it in full, and don't assume that anything in it is +the same as in the ordinary license. + + The reason we have a separate public license for some libraries is that +they blur the distinction we usually make between modifying or adding to a +program and simply using it. Linking a program with a library, without +changing the library, is in some sense simply using the library, and is +analogous to running a utility program or application program. However, in +a textual and legal sense, the linked executable is a combined work, a +derivative of the original library, and the ordinary General Public License +treats it as such. + + Because of this blurred distinction, using the ordinary General +Public License for libraries did not effectively promote software +sharing, because most developers did not use the libraries. We +concluded that weaker conditions might promote sharing better. + + However, unrestricted linking of non-free programs would deprive the +users of those programs of all benefit from the free status of the +libraries themselves. This Library General Public License is intended to +permit developers of non-free programs to use free libraries, while +preserving your freedom as a user of such programs to change the free +libraries that are incorporated in them. (We have not seen how to achieve +this as regards changes in header files, but we have achieved it as regards +changes in the actual functions of the Library.) The hope is that this +will lead to faster development of free libraries. + + The precise terms and conditions for copying, distribution and +modification follow. Pay close attention to the difference between a +"work based on the library" and a "work that uses the library". The +former contains code derived from the library, while the latter only +works together with the library. + + Note that it is possible for a library to be covered by the ordinary +General Public License rather than by this special one. + + GNU LIBRARY GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License Agreement applies to any software library which +contains a notice placed by the copyright holder or other authorized +party saying it may be distributed under the terms of this Library +General Public License (also called "this License"). Each licensee is +addressed as "you". + + A "library" means a collection of software functions and/or data +prepared so as to be conveniently linked with application programs +(which use some of those functions and data) to form executables. + + The "Library", below, refers to any such software library or work +which has been distributed under these terms. A "work based on the +Library" means either the Library or any derivative work under +copyright law: that is to say, a work containing the Library or a +portion of it, either verbatim or with modifications and/or translated +straightforwardly into another language. (Hereinafter, translation is +included without limitation in the term "modification".) + + "Source code" for a work means the preferred form of the work for +making modifications to it. For a library, complete source code means +all the source code for all modules it contains, plus any associated +interface definition files, plus the scripts used to control compilation +and installation of the library. + + Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running a program using the Library is not restricted, and output from +such a program is covered only if its contents constitute a work based +on the Library (independent of the use of the Library in a tool for +writing it). Whether that is true depends on what the Library does +and what the program that uses the Library does. + + 1. You may copy and distribute verbatim copies of the Library's +complete source code as you receive it, in any medium, provided that +you conspicuously and appropriately publish on each copy an +appropriate copyright notice and disclaimer of warranty; keep intact +all the notices that refer to this License and to the absence of any +warranty; and distribute a copy of this License along with the +Library. + + You may charge a fee for the physical act of transferring a copy, +and you may at your option offer warranty protection in exchange for a +fee. + + 2. You may modify your copy or copies of the Library or any portion +of it, thus forming a work based on the Library, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) The modified work must itself be a software library. + + b) You must cause the files modified to carry prominent notices + stating that you changed the files and the date of any change. + + c) You must cause the whole of the work to be licensed at no + charge to all third parties under the terms of this License. + + d) If a facility in the modified Library refers to a function or a + table of data to be supplied by an application program that uses + the facility, other than as an argument passed when the facility + is invoked, then you must make a good faith effort to ensure that, + in the event an application does not supply such function or + table, the facility still operates, and performs whatever part of + its purpose remains meaningful. + + (For example, a function in a library to compute square roots has + a purpose that is entirely well-defined independent of the + application. Therefore, Subsection 2d requires that any + application-supplied function or table used by this function must + be optional: if the application does not supply it, the square + root function must still compute square roots.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Library, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Library, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote +it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Library. + +In addition, mere aggregation of another work not based on the Library +with the Library (or with a work based on the Library) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may opt to apply the terms of the ordinary GNU General Public +License instead of this License to a given copy of the Library. To do +this, you must alter all the notices that refer to this License, so +that they refer to the ordinary GNU General Public License, version 2, +instead of to this License. (If a newer version than version 2 of the +ordinary GNU General Public License has appeared, then you can specify +that version instead if you wish.) Do not make any other change in +these notices. + + Once this change is made in a given copy, it is irreversible for +that copy, so the ordinary GNU General Public License applies to all +subsequent copies and derivative works made from that copy. + + This option is useful when you wish to copy part of the code of +the Library into a program that is not a library. + + 4. You may copy and distribute the Library (or a portion or +derivative of it, under Section 2) in object code or executable form +under the terms of Sections 1 and 2 above provided that you accompany +it with the complete corresponding machine-readable source code, which +must be distributed under the terms of Sections 1 and 2 above on a +medium customarily used for software interchange. + + If distribution of object code is made by offering access to copy +from a designated place, then offering equivalent access to copy the +source code from the same place satisfies the requirement to +distribute the source code, even though third parties are not +compelled to copy the source along with the object code. + + 5. A program that contains no derivative of any portion of the +Library, but is designed to work with the Library by being compiled or +linked with it, is called a "work that uses the Library". Such a +work, in isolation, is not a derivative work of the Library, and +therefore falls outside the scope of this License. + + However, linking a "work that uses the Library" with the Library +creates an executable that is a derivative of the Library (because it +contains portions of the Library), rather than a "work that uses the +library". The executable is therefore covered by this License. +Section 6 states terms for distribution of such executables. + + When a "work that uses the Library" uses material from a header file +that is part of the Library, the object code for the work may be a +derivative work of the Library even though the source code is not. +Whether this is true is especially significant if the work can be +linked without the Library, or if the work is itself a library. The +threshold for this to be true is not precisely defined by law. + + If such an object file uses only numerical parameters, data +structure layouts and accessors, and small macros and small inline +functions (ten lines or less in length), then the use of the object +file is unrestricted, regardless of whether it is legally a derivative +work. (Executables containing this object code plus portions of the +Library will still fall under Section 6.) + + Otherwise, if the work is a derivative of the Library, you may +distribute the object code for the work under the terms of Section 6. +Any executables containing that work also fall under Section 6, +whether or not they are linked directly with the Library itself. + + 6. As an exception to the Sections above, you may also compile or +link a "work that uses the Library" with the Library to produce a +work containing portions of the Library, and distribute that work +under terms of your choice, provided that the terms permit +modification of the work for the customer's own use and reverse +engineering for debugging such modifications. + + You must give prominent notice with each copy of the work that the +Library is used in it and that the Library and its use are covered by +this License. You must supply a copy of this License. If the work +during execution displays copyright notices, you must include the +copyright notice for the Library among them, as well as a reference +directing the user to the copy of this License. Also, you must do one +of these things: + + a) Accompany the work with the complete corresponding + machine-readable source code for the Library including whatever + changes were used in the work (which must be distributed under + Sections 1 and 2 above); and, if the work is an executable linked + with the Library, with the complete machine-readable "work that + uses the Library", as object code and/or source code, so that the + user can modify the Library and then relink to produce a modified + executable containing the modified Library. (It is understood + that the user who changes the contents of definitions files in the + Library will not necessarily be able to recompile the application + to use the modified definitions.) + + b) Accompany the work with a written offer, valid for at + least three years, to give the same user the materials + specified in Subsection 6a, above, for a charge no more + than the cost of performing this distribution. + + c) If distribution of the work is made by offering access to copy + from a designated place, offer equivalent access to copy the above + specified materials from the same place. + + d) Verify that the user has already received a copy of these + materials or that you have already sent this user a copy. + + For an executable, the required form of the "work that uses the +Library" must include any data and utility programs needed for +reproducing the executable from it. However, as a special exception, +the source code distributed need not include anything that is normally +distributed (in either source or binary form) with the major +components (compiler, kernel, and so on) of the operating system on +which the executable runs, unless that component itself accompanies +the executable. + + It may happen that this requirement contradicts the license +restrictions of other proprietary libraries that do not normally +accompany the operating system. Such a contradiction means you cannot +use both them and the Library together in an executable that you +distribute. + + 7. You may place library facilities that are a work based on the +Library side-by-side in a single library together with other library +facilities not covered by this License, and distribute such a combined +library, provided that the separate distribution of the work based on +the Library and of the other library facilities is otherwise +permitted, and provided that you do these two things: + + a) Accompany the combined library with a copy of the same work + based on the Library, uncombined with any other library + facilities. This must be distributed under the terms of the + Sections above. + + b) Give prominent notice with the combined library of the fact + that part of it is a work based on the Library, and explaining + where to find the accompanying uncombined form of the same work. + + 8. You may not copy, modify, sublicense, link with, or distribute +the Library except as expressly provided under this License. Any +attempt otherwise to copy, modify, sublicense, link with, or +distribute the Library is void, and will automatically terminate your +rights under this License. However, parties who have received copies, +or rights, from you under this License will not have their licenses +terminated so long as such parties remain in full compliance. + + 9. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Library or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Library (or any work based on the +Library), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Library or works based on it. + + 10. Each time you redistribute the Library (or any work based on the +Library), the recipient automatically receives a license from the +original licensor to copy, distribute, link with or modify the Library +subject to these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 11. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Library at all. For example, if a patent +license would not permit royalty-free redistribution of the Library by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Library. + +If any portion of this section is held invalid or unenforceable under any +particular circumstance, the balance of the section is intended to apply, +and the section as a whole is intended to apply in other circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 12. If the distribution and/or use of the Library is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Library under this License may add +an explicit geographical distribution limitation excluding those countries, +so that distribution is permitted only in or among countries not thus +excluded. In such case, this License incorporates the limitation as if +written in the body of this License. + + 13. The Free Software Foundation may publish revised and/or new +versions of the Library General Public License from time to time. +Such new versions will be similar in spirit to the present version, +but may differ in detail to address new problems or concerns. + +Each version is given a distinguishing version number. If the Library +specifies a version number of this License which applies to it and +"any later version", you have the option of following the terms and +conditions either of that version or of any later version published by +the Free Software Foundation. If the Library does not specify a +license version number, you may choose any version ever published by +the Free Software Foundation. + + 14. If you wish to incorporate parts of the Library into other free +programs whose distribution conditions are incompatible with these, +write to the author to ask for permission. For software which is +copyrighted by the Free Software Foundation, write to the Free +Software Foundation; we sometimes make exceptions for this. Our +decision will be guided by the two goals of preserving the free status +of all derivatives of our free software and of promoting the sharing +and reuse of software generally. + + NO WARRANTY + + 15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO +WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW. +EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR +OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY +KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE +LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME +THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN +WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY +AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU +FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR +CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE +LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING +RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A +FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF +SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH +DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Libraries + + If you develop a new library, and you want it to be of the greatest +possible use to the public, we recommend making it free software that +everyone can redistribute and change. You can do so by permitting +redistribution under these terms (or, alternatively, under the terms of the +ordinary General Public License). + + To apply these terms, attach the following notices to the library. It is +safest to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least the +"copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This library is free software; you can redistribute it and/or + modify it under the terms of the GNU Library General Public + License as published by the Free Software Foundation; either + version 2 of the License, or (at your option) any later version. + + This library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Library General Public License for more details. + + You should have received a copy of the GNU Library General Public + License along with this library; if not, see . + +Also add information on how to contact you by electronic and paper mail. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the library, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the + library `Frob' (a library for tweaking knobs) written by James Random Hacker. + + , 1 April 1990 + Moe Ghoul, President of Vice + +That's all there is to it! + + + +===== fs-constants@1.0.0 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz +Integrity: sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +The MIT License (MIT) + +Copyright (c) 2018 Mathias Buus + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. + + + +===== github-from-package@0.0.0 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz +Integrity: sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +This software is released under the MIT license: + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal in +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of +the Software, and to permit persons to whom the Software is furnished to do so, +subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR +COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER +IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + + + +===== ini@1.3.8 ===== +Declared license: ISC +Archive: https://registry.npmjs.org/ini/-/ini-1.3.8.tgz +Integrity: sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +The ISC License + +Copyright (c) Isaac Z. Schlueter and Contributors + +Permission to use, copy, modify, and/or distribute this software for any +purpose with or without fee is hereby granted, provided that the above +copyright notice and this permission notice appear in all copies. + +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR +IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + + + +===== mkdirp-classic@0.5.3 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz +Integrity: sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +The MIT License (MIT) + +Copyright (c) 2020 James Halliday (mail@substack.net) and Mathias Buus + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. + + + +===== napi-build-utils@2.0.0 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz +Integrity: sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +MIT License + +Copyright (c) 2018 inspiredware + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + + + +===== node-abi@3.98.0 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/node-abi/-/node-abi-3.98.0.tgz +Integrity: sha512-Llh8noNw7s15m+rmIrZnZQbe4gLyasL9loT74rC92PkmwWKeEedKvc7o6pjp8B3MhrYT/Cl846NOYZbz1wmddw== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +MIT License + +Copyright (c) 2016 Lukas Geiger + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + + + +===== prebuild-install@7.1.3 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz +Integrity: sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +The MIT License (MIT) + +Copyright (c) 2015 Mathias Buus + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. + + + +===== pump@3.0.4 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/pump/-/pump-3.0.4.tgz +Integrity: sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +The MIT License (MIT) + +Copyright (c) 2014 Mathias Buus + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. + + +===== rc@1.2.8 ===== +Declared license: (BSD-2-Clause OR MIT OR Apache-2.0) +Archive: https://registry.npmjs.org/rc/-/rc-1.2.8.tgz +Integrity: sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE.APACHE2 --- + +Apache License, Version 2.0 + +Copyright (c) 2011 Dominic Tarr + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + + +--- package/LICENSE.BSD --- + +Copyright (c) 2013, Dominic Tarr +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR +ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +The views and conclusions contained in the software and documentation are those +of the authors and should not be interpreted as representing official policies, +either expressed or implied, of the FreeBSD Project. + + +--- package/LICENSE.MIT --- + +The MIT License + +Copyright (c) 2011 Dominic Tarr + +Permission is hereby granted, free of charge, +to any person obtaining a copy of this software and +associated documentation files (the "Software"), to +deal in the Software without restriction, including +without limitation the rights to use, copy, modify, +merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom +the Software is furnished to do so, +subject to the following conditions: + +The above copyright notice and this permission notice +shall be included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES +OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR +ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, +TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + + + +===== readable-stream@3.6.2 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz +Integrity: sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +Node.js is licensed for use as follows: + +""" +Copyright Node.js contributors. All rights reserved. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to +deal in the Software without restriction, including without limitation the +rights to use, copy, modify, merge, publish, distribute, sublicense, and/or +sell copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +IN THE SOFTWARE. +""" + +This license applies to parts of Node.js originating from the +https://github.com/joyent/node repository: + +""" +Copyright Joyent, Inc. and other Node contributors. All rights reserved. +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to +deal in the Software without restriction, including without limitation the +rights to use, copy, modify, merge, publish, distribute, sublicense, and/or +sell copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +IN THE SOFTWARE. +""" + + + +===== strip-json-comments@2.0.1 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz +Integrity: sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ== +Scope: npm runtime dependency, including optional platform packages. + +--- package/license --- + +The MIT License (MIT) + +Copyright (c) Sindre Sorhus (sindresorhus.com) + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. + + + +===== tar-fs@2.1.5 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.5.tgz +Integrity: sha512-OboTd8mmMhZDNPV+UjQcK9yKAatXu2aJ+r1w4im1Otd4M4fl2hwvdoXUxIYHFTHWK/3y3FarBP70v3vwmGlOxw== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +The MIT License (MIT) + +Copyright (c) 2014 Mathias Buus + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. + + +===== tar-stream@2.2.0 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz +Integrity: sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +The MIT License (MIT) + +Copyright (c) 2014 Mathias Buus + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. + + +===== tunnel-agent@0.6.0 ===== +Declared license: Apache-2.0 +Archive: https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz +Integrity: sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +Apache License + +Version 2.0, January 2004 + +http://www.apache.org/licenses/ + +TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + +1. Definitions. + +"License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. + +"Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. + +"Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. + +"You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. + +"Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. + +"Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. + +"Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). + +"Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. + +"Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." + +"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. + +2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. + +3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. + +4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: + +You must give any other recipients of the Work or Derivative Works a copy of this License; and + +You must cause any modified files to carry prominent notices stating that You changed the files; and + +You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and + +If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. + +5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. + +6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. + +7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. + +8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. + +9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. + +END OF TERMS AND CONDITIONS + + +===== util-deprecate@1.0.2 ===== +Declared license: MIT +Archive: https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz +Integrity: sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +(The MIT License) + +Copyright (c) 2014 Nathan Rajlich + +Permission is hereby granted, free of charge, to any person +obtaining a copy of this software and associated documentation +files (the "Software"), to deal in the Software without +restriction, including without limitation the rights to use, +copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the +Software is furnished to do so, subject to the following +conditions: + +The above copyright notice and this permission notice shall be +included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES +OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT +HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, +WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +OTHER DEALINGS IN THE SOFTWARE. + + diff --git a/licenses/upstream-sources.json b/licenses/upstream-sources.json index 5853460b2..fb1dad975 100644 --- a/licenses/upstream-sources.json +++ b/licenses/upstream-sources.json @@ -58,5 +58,15 @@ "file": "licenses/Node-22.19.0-LICENSE.txt", "source": "https://raw.githubusercontent.com/nodejs/node/v22.19.0/LICENSE", "sha256": "e991d81497a85bb24fc6bffae0a3637a6accd6c6bc5ce1f2c5698bd555cf9d49" + }, + { + "file": "licenses/frida-17.23.2-COPYING.txt", + "source": "https://raw.githubusercontent.com/frida/frida/9adf8ed33ab6f1122b17399ee96dcbe101b941c9/COPYING", + "sha256": "5ea1544b51a28bc823b03159190d4108f9fb4f4ef912389f5137c6d295e175b2" + }, + { + "file": "licenses/LGPL-2.0.txt", + "source": "https://www.gnu.org/licenses/old-licenses/lgpl-2.0.txt", + "sha256": "cc535c21133c895b56b374c8a1dc1eb948d99003ed2b47372069456b62f42b24" } ] diff --git a/package.json b/package.json index 801f72ebf..bc5c0edc9 100644 --- a/package.json +++ b/package.json @@ -236,7 +236,7 @@ "docs:preview": "vitepress preview docs", "docs:generate": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts npm run docs:prepare", "docs:check": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts turbo run docs:check:built && npm run test:local -- tests/conformance/readmeConsistency.test.ts", - "docs:check:built": "node scripts/generate-package-metadata.mjs --check && node scripts/generate-skill-metadata.mjs --check && node scripts/generate-product-catalog.mjs --check && node scripts/generate-completion-ledger.mjs --check && node scripts/check-doc-facts.mjs && node scripts/generate-error-schema.mjs --check", + "docs:check:built": "node scripts/generate-package-metadata.mjs --check && node scripts/generate-skill-metadata.mjs --check && node scripts/generate-product-catalog.mjs --check && node scripts/generate-completion-ledger.mjs --check && node scripts/check-doc-facts.mjs && node scripts/generate-error-schema.mjs --check && npm run licenses:check", "docs:error-schema": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts turbo run artifacts:error-schema", "prepack": "node scripts/prepack.mjs", "prepare": "node scripts/prepare.mjs", @@ -279,7 +279,9 @@ "test:prepare": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts turbo run artifacts:mcp-catalog artifacts:product-catalog artifacts:managed-evidence", "evidence:check:built": "node scripts/generate-completion-ledger.mjs --check", "docs:prepare": "node scripts/generate-package-metadata.mjs && turbo run artifacts:product-catalog artifacts:managed-evidence artifacts:error-schema", - "verify:web:qlog": "npm run build:cached && node scripts/verify-qlog-captures.mjs" + "verify:web:qlog": "npm run build:cached && node scripts/verify-qlog-captures.mjs", + "licenses:generate": "node scripts/generate-third-party-inventory.mjs", + "licenses:check": "node scripts/generate-third-party-inventory.mjs --check" }, "lint-staged": { "*.{cjs,cts,js,jsx,mjs,mts,ts,tsx}": [ diff --git a/scripts/generate-third-party-inventory.mjs b/scripts/generate-third-party-inventory.mjs new file mode 100644 index 000000000..57934ebda --- /dev/null +++ b/scripts/generate-third-party-inventory.mjs @@ -0,0 +1,120 @@ +import { createHash } from "node:crypto"; +import { readFile } from "node:fs/promises"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; + +import { ensureGeneratedFile } from "./lib/generated-file.mjs"; + +const arguments_ = new Set(process.argv.slice(2)); +for (const argument of arguments_) + if (argument !== "--check") + throw new Error(`Unknown inventory option: ${argument}`); +const root = join(dirname(fileURLToPath(import.meta.url)), ".."); +const lock = JSON.parse( + await readFile(join(root, "package-lock.json"), "utf8"), +); +const entries = Object.entries(lock.packages) + .filter(([path]) => path !== "") + .map(([path, record]) => ({ + path, + name: record.name ?? path.slice(path.lastIndexOf("node_modules/") + 13), + version: record.version, + license: record.license ?? "UNDECLARED", + resolved: record.resolved, + integrity: record.integrity, + dev: record.dev === true, + optional: record.optional === true, + })) + .sort((left, right) => + left.path < right.path ? -1 : left.path > right.path ? 1 : 0, + ); +for (const entry of entries) + for (const field of ["name", "version", "resolved", "integrity"]) + if (typeof entry[field] !== "string" || entry[field].length === 0) + throw new Error(`Missing locked ${field}: ${entry.path}`); +const runtime = entries.filter(({ dev }) => !dev); +const runtimeFingerprint = createHash("sha256") + .update(JSON.stringify(runtime)) + .digest("hex"); +const source = [ + "REA npm dependency inventory", + "Prepared from the root package-lock.json; refresh this inventory when locked dependencies change.", + "License identifiers are upstream lockfile declarations, not a legal assessment.", + "Runtime includes transitive and optional platform packages; development includes build, test and website tooling.", + `Locked package entries: ${entries.length}; runtime: ${runtime.length}.`, + "Full runtime copyright/license/NOTICE texts: npm-runtime-notices.txt.", + "", + ...entries.flatMap((entry) => [ + `${entry.name}@${entry.version} | ${entry.dev ? "development" : "runtime"}${entry.optional ? ", optional" : ""} | ${entry.license}`, + ` Lockfile path: ${entry.path}`, + ` Package: https://www.npmjs.com/package/${entry.name}/v/${entry.version}`, + ` Archive: ${entry.resolved}`, + ` Integrity: ${entry.integrity}`, + ]), + "", +].join("\n"); +await ensureGeneratedFile({ + path: join(root, "licenses/npm-dependencies.txt"), + source, + check: arguments_.has("--check"), + generateCommand: "npm run licenses:generate", +}); + +if (arguments_.has("--check")) { + const notices = await readFile( + join(root, "licenses/npm-runtime-notices.txt"), + "utf8", + ); + const headings = [...notices.matchAll(/^===== (.+) =====$/gmu)]; + const sections = new Map(); + for (const [index, heading] of headings.entries()) { + if (sections.has(heading[1])) + throw new Error(`Duplicate runtime notice: ${heading[1]}`); + sections.set( + heading[1], + notices.slice( + heading.index + heading[0].length + 1, + headings[index + 1]?.index, + ), + ); + } + for (const entry of runtime) { + const identity = `${entry.name}@${entry.version}`; + const section = sections.get(identity); + if (section === undefined) + throw new Error(`Missing runtime notice: ${identity}`); + const metadata = [ + `Declared license: ${entry.license}`, + `Archive: ${entry.resolved}`, + `Integrity: ${entry.integrity}`, + "", + ].join("\n"); + if (!section.startsWith(metadata)) + throw new Error(`Runtime notice metadata differs for ${identity}`); + } + if ( + !notices + .slice(0, headings[0]?.index) + .includes( + `Runtime dependency fingerprint (SHA-256): ${runtimeFingerprint}\n`, + ) + ) + throw new Error( + `Runtime notice fingerprint differs; refresh verified notices for ${runtimeFingerprint}`, + ); + const references = JSON.parse( + await readFile(join(root, "licenses/upstream-sources.json"), "utf8"), + ); + for (const reference of references) { + const bytes = await readFile(join(root, reference.file)); + if (createHash("sha256").update(bytes).digest("hex") !== reference.sha256) + throw new Error(`Preserved notice digest differs: ${reference.file}`); + } +} +console.log( + JSON.stringify({ + locked_packages: entries.length, + runtime_packages: runtime.length, + runtime_dependency_fingerprint: runtimeFingerprint, + }), +); diff --git a/scripts/verify/harmony/analysis.mjs b/scripts/verify/harmony/analysis.mjs index 3aa90c9f0..aaa401c75 100644 --- a/scripts/verify/harmony/analysis.mjs +++ b/scripts/verify/harmony/analysis.mjs @@ -153,10 +153,36 @@ try { }); try { await client.connect(transport, { timeout: 30_000 }); + await client.listTools(); + const opened = await client.callTool( + { name: "open_binary", arguments: { path } }, + { timeout: 150_000 }, + ); + assert.notEqual(opened.isError, true, JSON.stringify(opened)); + assert.equal(opened.structuredContent?.result?.format, "hap"); + assert.equal(opened.structuredContent?.result?.sha256, fixture.sha256); + const status = await client.callTool( + { name: "binary_session", arguments: {} }, + { timeout: 150_000 }, + ); + assert.notEqual(status.isError, true, JSON.stringify(status)); + assert.equal(status.structuredContent?.result?.open, true); + assert.equal(status.structuredContent?.result?.format, "hap"); + const inspected = await client.callTool( + { name: "inspect_artifact", arguments: {} }, + { timeout: 150_000 }, + ); + assert.notEqual(inspected.isError, true, JSON.stringify(inspected)); + const mcpInspection = parseEvidence(inspected.structuredContent); + assert.deepEqual( + mcpInspection.normalized_result, + inspection.normalized_result, + ); + console.log("PASS MCP open/status/inspect_artifact parity"); const response = await client.callTool( { name: "project_harmony_application_graph", - arguments: { inventory_evidence: [inspection] }, + arguments: { inventory_evidence: [mcpInspection] }, }, { timeout: 150_000 }, ); @@ -168,6 +194,23 @@ try { projectionEvidence.normalized_result, ); console.log("PASS MCP/CLI parity project_harmony_application_graph"); + const closed = await client.callTool( + { name: "close_binary", arguments: {} }, + { timeout: 150_000 }, + ); + assert.notEqual(closed.isError, true, JSON.stringify(closed)); + const empty = await client.callTool( + { name: "binary_session", arguments: {} }, + { timeout: 150_000 }, + ); + assert.notEqual(empty.isError, true, JSON.stringify(empty)); + assert.equal(empty.structuredContent?.result?.open, false); + assert.equal( + await hashFile(path), + fixture.sha256, + "MCP workflow must preserve source bytes", + ); + console.log("PASS MCP close and unchanged source bytes"); } catch (cause) { process.stderr.write(stderr); throw cause; diff --git a/src/contracts/analysisViewToolContracts.ts b/src/contracts/analysisViewToolContracts.ts index 6011d544b..1a4dbbb5c 100644 --- a/src/contracts/analysisViewToolContracts.ts +++ b/src/contracts/analysisViewToolContracts.ts @@ -16,7 +16,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [ ...toolContractMetadata("inspect_analysis_view"), kind: "application", description: - 'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_evm_interface, inspect_managed_artifact, inspect_android_package, inspect_dart_aot or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. For layout, JavaScript, native, inventory, WASM, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Dart AOT summary preserves target/ABI/library identity, snapshot headers and pool lower bounds. String pages use kind dart-aot, collection package_uris/dart_uris/class_like_tokens; source coverage remains partial when the producer says partial. Native views select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', + 'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_evm_interface, inspect_managed_artifact, inspect_android_package, inspect_dart_aot or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. For layout, JavaScript, inventory, WASM, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Dart AOT summary preserves target/ABI/library identity, snapshot headers and pool lower bounds. String pages use kind dart-aot, collection package_uris/dart_uris/class_like_tokens; source coverage remains partial when the producer says partial. Native function Evidence requires kind native: view: {kind: "native", facet: "procedure"}. Facets select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', inputSchema: inspectAnalysisViewInputSchema, outputSchema: evidenceResultOf(analysisViewResultSchema), examples: [ diff --git a/src/contracts/toolOutputSchemaPrimitives.ts b/src/contracts/toolOutputSchemaPrimitives.ts index 20c5517ee..e63a79ff5 100644 --- a/src/contracts/toolOutputSchemaPrimitives.ts +++ b/src/contracts/toolOutputSchemaPrimitives.ts @@ -24,6 +24,7 @@ import { } from "../domain/providerOperationHealth.js"; import { analysisErrorProjectionSchema } from "./errorSchemas.js"; import { relationshipCoverageSchema } from "../domain/relationshipCoverage.js"; +import { ZIP_PACKAGE_FORMATS } from "../domain/zipPackageFormat.js"; /** Specialize the single normalized result inside a complete Evidence record. */ export const evidenceResultOf = (schema: Schema) => @@ -59,11 +60,7 @@ export const targetFormatSchema = z.enum([ "dos-mz", "dos-com", "raw-binary", - "zip", - "ipa", - "apk", - "msix", - "appx", + ...ZIP_PACKAGE_FORMATS, "asar", "dmg", "pkg", diff --git a/src/domain/binaryTargetTypes.ts b/src/domain/binaryTargetTypes.ts index b4d44c110..ab5046549 100644 --- a/src/domain/binaryTargetTypes.ts +++ b/src/domain/binaryTargetTypes.ts @@ -1,3 +1,5 @@ +import type { ZipPackageFormat } from "./zipPackageFormat.js"; + /** Provider-neutral CPU families detected from supported executable headers. */ export const BINARY_ARCHITECTURES = [ "x86", @@ -167,18 +169,7 @@ export type BinaryTarget = | (BinaryTargetIdentity & NonExecutableMetadata & { readonly kind: "archive"; - readonly format: - | "zip" - | "ipa" - | "apk" - | "msix" - | "appx" - | "hap" - | "hsp" - | "app-pack" - | "asar" - | "dmg" - | "pkg"; + readonly format: ZipPackageFormat | "asar" | "dmg" | "pkg"; }) | (BinaryTargetIdentity & NonExecutableMetadata & { diff --git a/tests/acceptance/harmonyArtifactJourney.test.ts b/tests/acceptance/harmonyArtifactJourney.test.ts new file mode 100644 index 000000000..51178d744 --- /dev/null +++ b/tests/acceptance/harmonyArtifactJourney.test.ts @@ -0,0 +1,135 @@ +import { createHash } from "node:crypto"; +import { readFile, writeFile } from "node:fs/promises"; +import { resolve } from "node:path"; + +import { Client } from "@modelcontextprotocol/client"; +import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; +import { TextReader, Uint8ArrayWriter, ZipWriter } from "@zip.js/zip.js"; +import { expect } from "vitest"; + +import { artifactInspectionResultSchema } from "../../src/domain/artifactInspection.js"; +import { parseEvidence } from "../../src/domain/evidence.js"; +import { harmonyApplicationProjectionResultSchema } from "../../src/domain/harmony/harmonyApplication.js"; +import { workspaceCliTest as test } from "../support/cli/workspaceCliFixture.js"; + +test("preserves Harmony archive formats through the public CLI/MCP lifecycle", async ({ + cli, + workspace, +}) => { + const writer = new ZipWriter(new Uint8ArrayWriter()); + await writer.add( + "module.json", + new TextReader('{"module":{"name":"fixture"}}'), + ); + await writer.add( + "ets/modules.abc", + new TextReader("opaque bytecode fixture"), + ); + const bytes = await writer.close(); + const environment = { + PATH: process.env.PATH ?? "", + HOME: workspace.root, + USERPROFILE: workspace.root, + XDG_CONFIG_HOME: workspace.path("config"), + XDG_CACHE_HOME: workspace.path("cache"), + XDG_STATE_HOME: workspace.path("state"), + NODE_OPTIONS: "--max-old-space-size=512", + REA_LOG_LEVEL: "silent", + }; + const client = new Client({ name: "rea-harmony-acceptance", version: "1" }); + const transport = new StdioClientTransport({ + command: process.execPath, + args: [resolve("scripts/rea.mjs"), "mcp"], + env: environment, + stderr: "pipe", + }); + try { + await client.connect(transport); + await client.listTools(); + for (const { suffix, format } of [ + { suffix: "hap", format: "hap" }, + { suffix: "hsp", format: "hsp" }, + { suffix: "app", format: "app-pack" }, + ]) { + const path = workspace.path(`fixture.${suffix}`); + await writeFile(path, bytes); + const sha256 = createHash("sha256").update(bytes).digest("hex"); + const cliReply = await cli.run({ + arguments: ["inspect-artifact", path, "--format", "json"], + environment, + }); + expect(cliReply.exitCode, cliReply.stderr).toBe(0); + const cliEvidence = parseEvidence(cliReply.json); + const cliInventory = artifactInspectionResultSchema.parse( + cliEvidence.normalized_result, + ); + expect(cliInventory.subject).toMatchObject({ + root_format: format, + root_sha256: sha256, + }); + const opened = await client.callTool({ + name: "open_binary", + arguments: { path }, + }); + expect(opened.isError, JSON.stringify(opened)).not.toBe(true); + expect(opened.structuredContent).toMatchObject({ + result: { kind: "archive", format, sha256 }, + }); + const status = await client.callTool({ + name: "binary_session", + arguments: {}, + }); + expect(status.isError, JSON.stringify(status)).not.toBe(true); + expect(status.structuredContent).toMatchObject({ + result: { open: true, kind: "archive", format, sha256 }, + }); + const inspected = await client.callTool({ + name: "inspect_artifact", + arguments: {}, + }); + expect(inspected.isError, JSON.stringify(inspected)).not.toBe(true); + const inventoryEvidence = parseEvidence(inspected.structuredContent); + expect( + artifactInspectionResultSchema.parse( + inventoryEvidence.normalized_result, + ), + ).toEqual(cliInventory); + + const projected = await client.callTool({ + name: "project_harmony_application_graph", + arguments: { inventory_evidence: [inventoryEvidence] }, + }); + expect(projected.isError, JSON.stringify(projected)).not.toBe(true); + const graph = harmonyApplicationProjectionResultSchema.parse( + parseEvidence(projected.structuredContent).normalized_result, + ); + expect(graph).toMatchObject({ + root_format: format, + root_sha256: sha256, + coverage: { status: "complete-within-inventory" }, + }); + expect( + graph.components.manifests.map(({ path: member }) => member), + ).toEqual(["module.json"]); + expect( + graph.components.bytecode.map(({ path: member }) => member), + ).toEqual(["ets/modules.abc"]); + const closed = await client.callTool({ + name: "close_binary", + arguments: {}, + }); + expect(closed.isError, JSON.stringify(closed)).not.toBe(true); + const empty = await client.callTool({ + name: "binary_session", + arguments: {}, + }); + expect(empty.isError, JSON.stringify(empty)).not.toBe(true); + expect(empty.structuredContent).toMatchObject({ + result: { open: false }, + }); + expect(await readFile(path)).toEqual(Buffer.from(bytes)); + } + } finally { + await client.close(); + } +}, 120_000); diff --git a/tests/boundary/filesystem/thirdPartyInventory.test.ts b/tests/boundary/filesystem/thirdPartyInventory.test.ts new file mode 100644 index 000000000..7e61f883b --- /dev/null +++ b/tests/boundary/filesystem/thirdPartyInventory.test.ts @@ -0,0 +1,117 @@ +import { createHash } from "node:crypto"; +import { copyFile, mkdir, readFile, writeFile } from "node:fs/promises"; +import { resolve } from "node:path"; + +import { expect } from "vitest"; + +import { processTest as test } from "../../support/process/processFixture.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +test("generates locked inventory and rejects stale notice metadata or preserved texts", async ({ + processes, +}) => { + const root = await createTestTempDirectory("rea-notice-records-"); + await mkdir(resolve(root, "scripts/lib"), { recursive: true }); + await mkdir(resolve(root, "licenses")); + for (const file of [ + "scripts/generate-third-party-inventory.mjs", + "scripts/lib/generated-file.mjs", + ]) + await copyFile(resolve(file), resolve(root, file)); + const script = resolve(root, "scripts/generate-third-party-inventory.mjs"); + const runtime = { + version: "1.0.0", + license: "MIT", + resolved: + "https://registry.npmjs.org/fixture-runtime/-/fixture-runtime-1.0.0.tgz", + integrity: "sha512-fixture-integrity", + optional: true, + }; + await writeFile( + resolve(root, "package-lock.json"), + JSON.stringify({ + packages: { + "": { name: "fixture", version: "1.0.0" }, + "node_modules/fixture-runtime": runtime, + "node_modules/fixture-dev": { ...runtime, dev: true, optional: false }, + }, + }), + ); + const generated = await processes.run(process.execPath, [script]); + expect(generated.exitCode, generated.stderr).toBe(0); + const inventory = await readFile( + resolve(root, "licenses/npm-dependencies.txt"), + "utf8", + ); + expect(inventory).toContain("Locked package entries: 2; runtime: 1."); + expect(inventory).toContain( + "fixture-runtime@1.0.0 | runtime, optional | MIT", + ); + expect(inventory).toContain("fixture-dev@1.0.0 | development | MIT"); + const fingerprint = ( + JSON.parse(generated.stdout) as { runtime_dependency_fingerprint: string } + ).runtime_dependency_fingerprint; + const notice = [ + `Runtime dependency fingerprint (SHA-256): ${fingerprint}`, + "", + "===== fixture-runtime@1.0.0 =====", + "Declared license: MIT", + `Archive: ${runtime.resolved}`, + `Integrity: ${runtime.integrity}`, + "", + "Fixture text; this test does not establish real package attribution.", + "", + ].join("\n"); + const noticePath = resolve(root, "licenses/npm-runtime-notices.txt"); + const referencePath = resolve(root, "licenses/reference.txt"); + await writeFile(noticePath, notice); + await writeFile(referencePath, "preserved fixture text\n"); + await writeFile( + resolve(root, "licenses/upstream-sources.json"), + JSON.stringify([ + { + file: "licenses/reference.txt", + sha256: createHash("sha256") + .update("preserved fixture text\n") + .digest("hex"), + }, + ]), + ); + const check = () => processes.run(process.execPath, [script, "--check"]); + const valid = await check(); + expect(valid.exitCode, valid.stderr).toBe(0); + for (const [changed, diagnostic] of [ + [ + notice.replace( + "===== fixture-runtime@1.0.0 =====", + "===== another-package@1.0.0 =====", + ), + "Missing runtime notice", + ], + [ + notice.replace(runtime.integrity, "sha512-different"), + "Runtime notice metadata differs", + ], + [ + notice.replace(fingerprint, "0".repeat(64)), + "Runtime notice fingerprint differs", + ], + [ + notice + notice.slice(notice.indexOf("===== ")), + "Duplicate runtime notice", + ], + ] as const) { + await writeFile(noticePath, changed); + const rejected = await check(); + expect(rejected.exitCode).toBe(1); + expect(rejected.stderr).toContain(diagnostic); + } + await writeFile(noticePath, notice); + await writeFile(referencePath, "changed fixture text\n"); + const tampered = await check(); + expect(tampered.exitCode).toBe(1); + expect(tampered.stderr).toContain("Preserved notice digest differs"); + expect( + await readFile(resolve(root, "licenses/npm-dependencies.txt"), "utf8"), + ).toBe(inventory); +}); diff --git a/tests/boundary/mcp/analysisViewMcp.test.ts b/tests/boundary/mcp/analysisViewMcp.test.ts index f23295829..5b2fe98af 100644 --- a/tests/boundary/mcp/analysisViewMcp.test.ts +++ b/tests/boundary/mcp/analysisViewMcp.test.ts @@ -69,6 +69,12 @@ const connect = async ( it("discovers an oversized native dossier by exact metadata and reads its retained view", async () => { const { client, session } = await connect(); + const advertised = (await client.listTools()).tools.find( + (tool) => tool.name === "inspect_analysis_view", + ); + expect(advertised?.description).toContain( + 'Native function Evidence requires kind native: view: {kind: "native", facet: "procedure"}', + ); const provider = { id: "ghidra", name: "Ghidra", version: "12.1.4" }; const profile = createAnalysisProfile(provider, { language: "fixture" }); const parent = createEvidence( @@ -98,6 +104,24 @@ it("discovers an oversized native dossier by exact metadata and reads its retain const retained = session.recordEvidence(record); if (!retained.ok) throw retained.error; } + const unsupportedSummary = await client.callTool({ + name: "inspect_analysis_view", + arguments: { + source: { kind: "retained-evidence", evidence_id: parent.evidence_id }, + view: { kind: "summary" }, + }, + }); + expect(parseMcpToolError(unsupportedSummary).error).toMatchObject({ + code: "invalid_request", + details: { + issues: [ + { + path: ["view"], + message: "analyze_function Evidence requires a native view.", + }, + ], + }, + }); const broad = await client.callTool({ name: "get_evidence_bundle", arguments: {}, diff --git a/turbo.json b/turbo.json index e8bfbf06e..758f31e32 100644 --- a/turbo.json +++ b/turbo.json @@ -151,6 +151,7 @@ "scripts/fixtures/apple/*.mjs", "src/generated*.ts", "src/contracts/**/*.ts", + "licenses/**", "docs/**", "!docs/public/product-catalog.json", "!docs/verification/managed-conformance-*.json", From 4303dcdcf68af4d71d777ffff75cbd4f17a8877d Mon Sep 17 00:00:00 2001 From: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> Date: Mon, 12 Oct 2026 21:11:53 -0700 Subject: [PATCH 008/159] fix(native): name the retained pwntools process group in cleanup resources (#2281) When a pwntools or pwndbg stage cannot confirm that its owned process group stopped, the cleanup_incomplete result listed only the run ID and runtime directory, although a descendant can still be live. Add the retained supervisor's pid and process-group coordinates, as OwnedCommand reports for a non-transferred cleanup failure. Fixes #2274 Co-authored-by: Claude Opus 5.5 --- src/native/pwntools/PwntoolsDecoder.ts | 7 +++++++ tests/process-global/pwntoolsStageRetirement.test.ts | 8 ++++++++ 2 files changed, 15 insertions(+) diff --git a/src/native/pwntools/PwntoolsDecoder.ts b/src/native/pwntools/PwntoolsDecoder.ts index 3a28edbfa..d8cad3d2e 100644 --- a/src/native/pwntools/PwntoolsDecoder.ts +++ b/src/native/pwntools/PwntoolsDecoder.ts @@ -518,6 +518,13 @@ export class PwntoolsDecoder< [ pending.process.launch.ownership?.runId ?? "owned-decoder", ...(pending.root === undefined ? [] : [pending.root.path]), + // The retained group may still be live; name it as OwnedCommand does. + ...(pending.process.launch.ownership === undefined + ? [] + : [ + `pid:${pending.process.launch.ownership.leaderPid}`, + `process-group:${pending.process.launch.ownership.processGroupId}`, + ]), ], { reason: stopped.reason, diff --git a/tests/process-global/pwntoolsStageRetirement.test.ts b/tests/process-global/pwntoolsStageRetirement.test.ts index bcdaf858f..6ec062535 100644 --- a/tests/process-global/pwntoolsStageRetirement.test.ts +++ b/tests/process-global/pwntoolsStageRetirement.test.ts @@ -261,6 +261,14 @@ it.runIf(supportedHost).each([ if (result.ok) throw new Error("Expected unresolved process ownership"); const projected = projectAnalysisError(result.error); expect(projected.code).toBe("cleanup_incomplete"); + expect(projected).toMatchObject({ + details: { + resources: expect.arrayContaining([ + expect.stringMatching(/^pid:\d+$/u), + expect.stringMatching(/^process-group:\d+$/u), + ]), + }, + }); if (stage === "debugger") expect(projected).toMatchObject({ details: { From 2c7240cf9d8231e16ad01de7678af9dbd01bab46 Mon Sep 17 00:00:00 2001 From: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> Date: Mon, 12 Oct 2026 21:12:05 -0700 Subject: [PATCH 009/159] perf(native): run independent Mach-O slice commands concurrently (#2466) inspect_macho launched otool, nm, two dyld_info reads, dwarfdump and vtool one at a time. Each launch also resolves the tool through `xcrun --find`, so a warm call spent about 350 ms in eight serial supervised launches. The slice commands only read the selected artifact, so launch them together. Every launch still settles before the result, and declared order still fixes provenance and which failure is reported. Co-authored-by: Claude Opus 5.5 --- src/native/NativeMachoInspection.test.ts | 66 +++++++++++++++++++++++- src/native/NativeMachoInspection.ts | 25 ++++----- 2 files changed, 78 insertions(+), 13 deletions(-) diff --git a/src/native/NativeMachoInspection.test.ts b/src/native/NativeMachoInspection.test.ts index 4ed533c2e..6fe24eb90 100644 --- a/src/native/NativeMachoInspection.test.ts +++ b/src/native/NativeMachoInspection.test.ts @@ -1,6 +1,9 @@ import { describe, expect, it } from "vitest"; -import { AnalysisCancelledError } from "../domain/analysisErrorCore.js"; +import { + AnalysisCancelledError, + AnalysisOutputError, +} from "../domain/analysisErrorCore.js"; import { err, ok } from "../domain/result.js"; import type { NativeCommandCapture } from "./CommandRunner.js"; import { inspectNativeMacho } from "./NativeMachoInspection.js"; @@ -186,3 +189,64 @@ describe("Mach-O UUID reconciliation", () => { ); }); }); + +describe("Mach-O slice command scheduling", () => { + const SLICE_TOOLS = ["otool", "nm", "dyld_info", "dyld_info", "dwarfdump"]; + + /** Finish later-launched slice commands first and record overlap. */ + const reorderingRunner = (failing: ReadonlySet = new Set()) => { + const fixtures = new NativeFixtureRunner(); + let inFlight = 0; + let maxInFlight = 0; + let launched = 0; + const run = async (tool: string, args: readonly string[]) => { + if (tool === "file" || tool === "lipo") return fixtures.run(tool, args); + const order = launched++; + inFlight += 1; + maxInFlight = Math.max(maxInFlight, inFlight); + for (let turn = 0; turn < 12 - order * 2; turn += 1) + await new Promise((resolve) => setImmediate(resolve)); + inFlight -= 1; + if (failing.has(tool)) + return err(new AnalysisOutputError("inspect_macho", `${tool} failed`)); + return fixtures.run(tool, args); + }; + return { run, maxInFlight: () => maxInFlight }; + }; + + const inspectWith = (run: ReturnType["run"]) => + inspectNativeMacho({ + target: nativeMachoTarget("/fixture/app"), + run, + invocation: (capture) => ({ + tool: capture.tool, + command: [capture.executable, ...capture.arguments], + tool_version: capture.toolVersion, + version_reason: capture.versionReason, + executable_sha256: capture.executableSha256, + exit: { code: capture.exitCode, signal: capture.signal }, + stdout_bytes: capture.stdoutBytes, + stderr_bytes: capture.stderrBytes, + }), + }); + + it("runs independent slice commands together and keeps provenance in declared order", async () => { + const runner = reorderingRunner(); + const result = await inspectWith(runner.run); + if (!result.ok) throw result.error; + expect(runner.maxInFlight()).toBe(SLICE_TOOLS.length + 1); + expect(result.value.result).toMatchObject({ + provenance: ["file", "lipo", ...SLICE_TOOLS, "vtool"].map((tool) => ({ + tool, + })), + }); + }); + + it("reports the first failing slice command in declared order", async () => { + const runner = reorderingRunner(new Set(["nm", "dwarfdump"])); + const result = await inspectWith(runner.run); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error).toMatchObject({ reason: "nm failed" }); + }); +}); diff --git a/src/native/NativeMachoInspection.ts b/src/native/NativeMachoInspection.ts index 2ec6cd4a9..24ccb2514 100644 --- a/src/native/NativeMachoInspection.ts +++ b/src/native/NativeMachoInspection.ts @@ -79,8 +79,11 @@ export const inspectNativeMacho = async ( ); const selectedArchitecture = selection.status === "selected" ? selection.architecture : null; - for (const [tool, prefix] of REQUIRED_SLICE_COMMANDS) { - const captured = await context.run( + const runSliceCommand = ([tool, prefix]: readonly [ + string, + readonly string[], + ]) => + context.run( tool, [ ...prefix, @@ -89,20 +92,18 @@ export const inspectNativeMacho = async ( ], context.signal, ); + // Slice commands only read the selected artifact, so launch them together. + // Every launch settles before return; declared order fixes provenance and + // which failure is reported. + const [required, optional] = await Promise.all([ + Promise.all(REQUIRED_SLICE_COMMANDS.map(runSliceCommand)), + runSliceCommand(OPTIONAL_VTOOL_COMMAND), + ]); + for (const captured of required) { if (!captured.ok) return err(captured.error); captures.push(captured.value); } const limitations: string[] = []; - const [tool, prefix] = OPTIONAL_VTOOL_COMMAND; - const optional = await context.run( - tool, - [ - ...prefix, - ...selectedArchitectureArguments(tool, selectedArchitecture), - context.target.path, - ], - context.signal, - ); if (optional.ok) captures.push(optional.value); else if (optional.error._tag === "AnalysisCapabilityUnavailableError") limitations.push(VTOOL_UNAVAILABLE_LIMITATION); From bb540826d65316c164d2de8ba4ef10f81a8ad073 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:18:39 +0800 Subject: [PATCH 010/159] fix(artifacts): retry transient DMG verification refusals --- src/artifacts/NativeDmgArtifactReader.ts | 111 +++++++++++---- .../native/nativeDmgArtifactReader.test.ts | 132 ++++++++++++++++++ 2 files changed, 215 insertions(+), 28 deletions(-) diff --git a/src/artifacts/NativeDmgArtifactReader.ts b/src/artifacts/NativeDmgArtifactReader.ts index a3f3fb120..801ee4daf 100644 --- a/src/artifacts/NativeDmgArtifactReader.ts +++ b/src/artifacts/NativeDmgArtifactReader.ts @@ -22,6 +22,7 @@ import { type NativeDmgInfo, } from "./NativeDmgOwnership.js"; const HDIUTIL_COMMAND_TIMEOUT_MS = 120_000; +const VERIFY_SETTLE_DELAYS_MS = [250, 500] as const; // APFS ejects and busy reporting can lag a detach attempt. const DETACH_SETTLE_DELAYS_MS = [250, 500, 1000] as const; const delay = (milliseconds: number): Promise => @@ -44,22 +45,30 @@ export interface NativeDmgHost { class NativeDmgCommandFailure extends ArtifactReaderFailure { readonly processNotStarted: boolean; + readonly verifyDiagnostic: string | undefined; + readonly exitCode: number | null; constructor( - reason: ArtifactReaderFailure["reason"], + failure: { + readonly reason: ArtifactReaderFailure["reason"]; + readonly verifyDiagnostic?: string; + }, message: string, readonly stdout: string, options?: { readonly cause?: unknown; readonly processNotStarted?: boolean; + readonly exitCode?: number; }, ) { super( - reason, + failure.reason, message, options?.cause === undefined ? undefined : { cause: options.cause }, ); this.processNotStarted = options?.processNotStarted ?? false; + this.verifyDiagnostic = failure.verifyDiagnostic; + this.exitCode = options?.exitCode ?? null; } } @@ -84,7 +93,7 @@ const createSystemHost = ( const output = execFileOutputFailure(cause); if (cause instanceof Error && cause.name === "AbortError") throw new NativeDmgCommandFailure( - "cancelled", + { reason: "cancelled" }, "DMG operation cancelled", output?.stdout ?? "", { cause }, @@ -99,7 +108,7 @@ const createSystemHost = ( cause, }; throw new NativeDmgCommandFailure( - commandFailureReason(cause, arguments_[0]), + classifyCommandFailure(cause, arguments_[0]), `hdiutil ${arguments_[0] ?? "operation"} failed: ${describeCommandFailure(cause, arguments_)}`, output?.stdout ?? "", { cause, processNotStarted: processNotStarted(cause) }, @@ -348,8 +357,7 @@ export class NativeDmgArtifactReader implements ArtifactReader { } async #attach(signal?: AbortSignal): Promise { - await runChecked(this.host, ["verify", this.path], signal); - this.#provenance.push(command(["verify", this.path], ["read"])); + await this.#verify(signal); const baseline = await this.#readInfo(); this.#baselineDevices = new Set( baseline.images.flatMap((image) => @@ -431,6 +439,40 @@ export class NativeDmgArtifactReader implements ArtifactReader { } } + async #verify(signal?: AbortSignal): Promise { + const arguments_ = ["verify", this.path]; + const deadline = performance.now() + HDIUTIL_COMMAND_TIMEOUT_MS; + for (let attempt = 0; ; attempt += 1) { + if (signal?.aborted === true) + throw new NativeDmgCommandFailure( + { reason: "cancelled" }, + "DMG operation cancelled", + "", + { cause: signal.reason }, + ); + try { + await runChecked(this.host, arguments_, signal, { + timeoutMs: Math.max(1, Math.ceil(deadline - performance.now())), + }); + this.#provenance.push(command(arguments_, ["read"])); + return; + } catch (cause: unknown) { + if (!(cause instanceof NativeDmgCommandFailure)) throw cause; + this.#provenance.push(command(arguments_, ["read"], cause.exitCode)); + const delayMs = VERIFY_SETTLE_DELAYS_MS[attempt]; + // Retry only the producer's explicit transient diagnostic, before any mount. + if ( + cause.verifyDiagnostic !== "Resource temporarily unavailable" || + delayMs === undefined || + deadline - performance.now() <= delayMs + ) + throw cause; + await (this.host.delay ?? delay)(delayMs); + if (performance.now() >= deadline) throw cause; + } + } + } + async #readInfo(): Promise { const info = await runChecked(this.host, ["info", "-plist"], undefined, { timeoutMs: HDIUTIL_COMMAND_TIMEOUT_MS, @@ -501,14 +543,14 @@ const runChecked = async ( if (cause instanceof ArtifactReaderFailure) throw cause; if (cause instanceof Error && cause.name === "AbortError") throw new NativeDmgCommandFailure( - "cancelled", + { reason: "cancelled" }, "DMG operation cancelled", execFileOutputFailure(cause)?.stdout ?? "", { cause }, ); const output = execFileOutputFailure(cause); throw new NativeDmgCommandFailure( - commandFailureReason(cause, arguments_[0]), + classifyCommandFailure(cause, arguments_[0]), `hdiutil ${arguments_[0] ?? "operation"} failed: ${describeCommandFailure( cause, arguments_, @@ -527,17 +569,17 @@ const runChecked = async ( stdout: result.stdout, ...(result.stderr === undefined ? {} : { stderr: result.stderr }), }; - const reason = commandFailureReason(failure, arguments_[0]); + const classification = classifyCommandFailure(failure, arguments_[0]); const details = describeCommandFailure(failure, arguments_); const unknownVerifyFailure = - arguments_[0] === "verify" && reason === "unavailable"; + arguments_[0] === "verify" && classification.reason === "unavailable"; throw new NativeDmgCommandFailure( - reason, + classification, unknownVerifyFailure ? `hdiutil verify failed; captured diagnostics do not establish the failure cause: ${details}` : `hdiutil ${arguments_[0] ?? "operation"} failed: ${details}`, result.stdout, - { cause: result.cause ?? failure }, + { cause: result.cause ?? failure, exitCode: result.exitCode }, ); } return { stdout: result.stdout, exitCode: 0 }; @@ -587,20 +629,36 @@ const processOutput = (cause: unknown, field: "stdout" | "stderr"): string => { return typeof value === "string" ? value : ""; }; -const commandFailureReason = ( +const classifyCommandFailure = ( cause: unknown, operation: string | undefined, -): ArtifactReaderFailure["reason"] => { - if (typeof cause !== "object" || cause === null) return "unavailable"; +): { + readonly reason: ArtifactReaderFailure["reason"]; + readonly verifyDiagnostic?: string; +} => { + if (typeof cause !== "object" || cause === null) + return { reason: "unavailable" }; const code = Reflect.get(cause, "code"); const exitCode = Reflect.get(cause, "exitCode"); - if (typeof code === "number" || typeof exitCode === "number") - return operation === "verify" ? verifyFailureReason(cause) : "unavailable"; + if (typeof code === "number" || typeof exitCode === "number") { + if (operation !== "verify") return { reason: "unavailable" }; + const diagnostic = verifyFailureDiagnostic( + processOutput(cause, "stdout"), + processOutput(cause, "stderr"), + ); + return { + reason: verifyFailureReason(diagnostic), + ...(diagnostic === undefined ? {} : { verifyDiagnostic: diagnostic }), + }; + } if (code === "ENOENT") { const syscall = Reflect.get(cause, "syscall"); - return typeof syscall === "string" && syscall.startsWith("spawn") - ? "unavailable" - : "io"; + return { + reason: + typeof syscall === "string" && syscall.startsWith("spawn") + ? "unavailable" + : "io", + }; } if ( code === "EACCES" || @@ -613,17 +671,13 @@ const commandFailureReason = ( code === "EMFILE" || code === "ENFILE" ) - return "io"; - return "unavailable"; + return { reason: "io" }; + return { reason: "unavailable" }; }; const verifyFailureReason = ( - cause: unknown, + diagnostic: string | undefined, ): ArtifactReaderFailure["reason"] => { - const diagnostic = verifyFailureDiagnostic( - processOutput(cause, "stdout"), - processOutput(cause, "stderr"), - ); switch (diagnostic) { // hdiutil follows AppleLanguages, so en-GB and en-AU spell it "recognised". case "image not recognized": @@ -656,11 +710,12 @@ const verifyFailureDiagnostic = ( const command = ( arguments_: readonly string[], effects: ArtifactCommand["effects"], + exitCode: number | null = 0, ): ArtifactCommand => ({ tool: "/usr/bin/hdiutil", arguments: [...arguments_], tool_version: null, executable_sha256: null, - exit_code: 0, + exit_code: exitCode, effects, }); diff --git a/tests/boundary/providers/native/nativeDmgArtifactReader.test.ts b/tests/boundary/providers/native/nativeDmgArtifactReader.test.ts index 9e03727c1..5d6af987d 100644 --- a/tests/boundary/providers/native/nativeDmgArtifactReader.test.ts +++ b/tests/boundary/providers/native/nativeDmgArtifactReader.test.ts @@ -1200,6 +1200,138 @@ describe("native DMG command diagnostics", () => { }); }); +describe("native DMG verification resource availability", () => { + const transientFailure = { + stdout: "", + stderr: + 'hdiutil: verify: unable to recognize "/tmp/image.dmg" as a disk image. (Resource temporarily unavailable)\nhdiutil: verify failed - Resource temporarily unavailable\n', + exitCode: 1, + }; + + it("retries the explicit transient refusal before attachment and retains both attempts", async () => { + const { host, state } = createOmittedInventoryHost(); + state.inventoryMode = "listed"; + state.detachAttempts = 4; + const delays = vi.fn(async () => {}); + const timeouts: number[] = []; + let attempts = 0; + const reader = await createAttachedReader( + "/tmp/image.dmg", + process.env, + undefined, + { + ...host, + delay: delays, + async run(arguments_, signal, options) { + if (arguments_[0] === "verify") { + attempts += 1; + if (options === undefined) + throw new Error("missing verify deadline"); + timeouts.push(options.timeoutMs); + if (attempts === 1) return transientFailure; + } + return host.run(arguments_, signal, options); + }, + }, + ); + onTestFinished(() => reader.close()); + expect(attempts).toBe(2); + expect(delays.mock.calls).toEqual([[250]]); + expect(timeouts[0]).toBeLessThanOrEqual(120_000); + expect(timeouts[1]).toBeLessThanOrEqual(timeouts[0]!); + expect(reader.provenance().slice(0, 2)).toMatchObject([ + { + arguments: ["verify", "/tmp/image.dmg"], + exit_code: 1, + effects: ["read"], + }, + { + arguments: ["verify", "/tmp/image.dmg"], + exit_code: 0, + effects: ["read"], + }, + ]); + await reader.close(); + expect(state.attached).toBe(false); + }); + + it("bounds repeated transient refusals and never attaches an unverified image", async () => { + const run = vi.fn(async () => transientFailure); + const delay = vi.fn(async () => {}); + const reader = new NativeDmgArtifactReader("/tmp/image.dmg", process.env, { + run, + delay, + }); + await expect(entryPaths(reader)).rejects.toMatchObject({ + reason: "unavailable", + message: expect.stringContaining("Resource temporarily unavailable"), + }); + await reader.close(); + expect(run.mock.calls).toHaveLength(3); + expect(delay.mock.calls).toEqual([[250], [500]]); + expect(reader.provenance().map(({ exit_code }) => exit_code)).toEqual([ + 1, 1, 1, + ]); + }); + + it("honors cancellation during the wait without starting another command", async () => { + const controller = new AbortController(); + const run = vi.fn(async () => transientFailure); + await expect( + createAttachedReader("/tmp/image.dmg", process.env, controller.signal, { + run, + async delay() { + controller.abort(new Error("cancel verification")); + }, + }), + ).rejects.toMatchObject({ reason: "cancelled" }); + expect(run).toHaveBeenCalledTimes(1); + }); + + it("does not start another verification after the shared deadline", async () => { + const clock = vi.spyOn(performance, "now").mockReturnValue(0); + const run = vi.fn(async () => transientFailure); + try { + await expect( + createAttachedReader("/tmp/image.dmg", process.env, undefined, { + run, + async delay() { + clock.mockReturnValue(120_001); + }, + }), + ).rejects.toMatchObject({ + reason: "unavailable", + message: expect.stringContaining("Resource temporarily unavailable"), + }); + expect(run).toHaveBeenCalledTimes(1); + } finally { + clock.mockRestore(); + } + }); + + it.each([ + ["image data corrupted", "integrity"], + ["image not recognized", "format"], + ["Permission denied", "io"], + ["an unknown failure", "unavailable"], + ])("does not retry %s", async (diagnostic, reason) => { + const run = vi.fn(async () => ({ + stdout: "", + stderr: `hdiutil: verify failed - ${diagnostic}\n`, + exitCode: 1, + })); + const delay = vi.fn(async () => {}); + await expect( + createAttachedReader("/tmp/image.dmg", process.env, undefined, { + run, + delay, + }), + ).rejects.toMatchObject({ reason }); + expect(run).toHaveBeenCalledTimes(1); + expect(delay).not.toHaveBeenCalled(); + }); +}); + describe("native DMG real verification diagnostics", () => { it.skipIf(process.platform !== "darwin")( "classifies a real hdiutil verify refusal as a format failure with captured output", From 45984a4319c8b6dcf2b03e969fbd4330c8432394 Mon Sep 17 00:00:00 2001 From: Ysopwnable <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:38:12 +0800 Subject: [PATCH 011/159] test(perf): verify existing SQLite column selection on QGIS data (#2489) Verify existing SQLite column selection on pinned QGIS application data. Preserve complete requested schema and exact typed sample facts, all failed/repeated trials and mixed timing results; add reproducible corpus/question/grader and an isolated per-tool driver approval entry. Existing controls reduce model input in both corrected pairs; no production/API/default/cache change is justified. Real CLI/MCP parity, independent SQLite binding, current-main SDK comparison and relevant checks pass. Ref #2016. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- experiments/analysis-efficiency/README.md | 4 + .../analysis-efficiency/agent-trial.py | 1 + experiments/analysis-efficiency/corpus.json | 13 ++ experiments/analysis-efficiency/measure.mjs | 58 ++++++ .../questions/sqlite-qgis.txt | 1 + .../analysis-efficiency/review-model.py | 7 +- .../sqlite-existing-controls.md | 181 ++++++++++++++++++ 7 files changed, 264 insertions(+), 1 deletion(-) create mode 100644 experiments/analysis-efficiency/questions/sqlite-qgis.txt create mode 100644 experiments/analysis-efficiency/sqlite-existing-controls.md diff --git a/experiments/analysis-efficiency/README.md b/experiments/analysis-efficiency/README.md index b028953ce..5f656a02c 100644 --- a/experiments/analysis-efficiency/README.md +++ b/experiments/analysis-efficiency/README.md @@ -15,6 +15,10 @@ Follow-up: [Dart AOT summary and retained string pools](flutter-aot-summary.md) records real LocalSend delivery pairs and old Evidence continuation, preserving ABI selection, snapshot identity, original lower bounds and partial coverage. +Follow-up: [SQLite existing-control measurements](sqlite-existing-controls.md) +tests QGIS's shipped GeoPackage through the existing column selection, preserving +full selected-table schema and typed cells before considering another API. + Follow-up: [native PE/DOS entry views and real model verification](native-entry-views.md) records the post-merge native continuation, including a small-function case where four selected replies cost more bytes than the complete dossier. diff --git a/experiments/analysis-efficiency/agent-trial.py b/experiments/analysis-efficiency/agent-trial.py index 682be27ec..6dcc7d7f8 100644 --- a/experiments/analysis-efficiency/agent-trial.py +++ b/experiments/analysis-efficiency/agent-trial.py @@ -178,6 +178,7 @@ def main(): 'inspect_android_package', 'search_android_classes', 'inspect_android_class', 'inspect_android_method', 'inspect_firmware_regions', 'inspect_managed_artifact', + 'inspect_sqlite_database', 'inspect_managed_members', 'inspect_managed_native_boundaries', 'inspect_artifact', 'inspect_wasm_artifact', 'inspect_evm_interface', 'get_evidence_bundle', 'export_evidence_bundle', diff --git a/experiments/analysis-efficiency/corpus.json b/experiments/analysis-efficiency/corpus.json index 3b0f5c574..163b3b49a 100644 --- a/experiments/analysis-efficiency/corpus.json +++ b/experiments/analysis-efficiency/corpus.json @@ -250,6 +250,19 @@ "transport_bytes": 46558706, "derivation": "Official arm64 release APK, compared with GitHub release asset digest; byte-only analysis without execution" } + }, + { + "id": "qgis-world-map.gpkg", + "file": "qgis-world-map.gpkg", + "bytes": 12914688, + "sha256": "01d293159d1b77bd9212b5252587853f61aa9e25b2387135f34afaba8d057190", + "provenance": { + "project": "QGIS bundled world map", + "source_commit": "e1223b8e9e71fc50ee995ed556f711c54ccc101b", + "git_blob_sha1": "c9ff80228dff0de93c9acdef16a97545ba360244", + "url": "https://raw.githubusercontent.com/qgis/QGIS/e1223b8e9e71fc50ee995ed556f711c54ccc101b/resources/data/world_map.gpkg", + "derivation": "unmodified official resource file; streaming Git blob and SHA-256 verification" + } } ] } diff --git a/experiments/analysis-efficiency/measure.mjs b/experiments/analysis-efficiency/measure.mjs index 8774ed30b..160973aa7 100644 --- a/experiments/analysis-efficiency/measure.mjs +++ b/experiments/analysis-efficiency/measure.mjs @@ -28,6 +28,7 @@ const { values } = parseArgs({ "javascript-question": { type: "string", default: "inventory" }, "managed-question": { type: "string", default: "inventory" }, schemas: { type: "string", default: "default" }, + "sqlite-selection": { type: "string", default: "all" }, }, }); if ( @@ -43,6 +44,11 @@ if ( ) || !["inventory", "selected"].includes(values["javascript-question"]) || !["default", "selected"].includes(values.schemas) || + !["all", "columns"].includes(values["sqlite-selection"]) || + (values.artifact !== "qgis-world-map.gpkg" && + values["sqlite-selection"] !== "all") || + (values.artifact === "qgis-world-map.gpkg" && + values.delivery !== "complete") || !["inventory", "identity"].includes(values["managed-question"]) || (!["sharex.dll", "sharex.helpers.dll"].includes(values.artifact) && values["managed-question"] !== "inventory") || @@ -518,6 +524,54 @@ const inspectEvmTarget = async () => { }; const workflows = { + "qgis-world-map.gpkg": async () => { + const columns = ["fid", "NAME", "ISO_A3"]; + const input = { + path: values.path, + table: "countries", + row_limit: 1, + ...(values["sqlite-selection"] === "columns" ? { columns } : {}), + }; + const definition = catalogDefinitions.get("inspect_sqlite_database"); + if ( + definition === undefined || + !new Ajv2020({ strict: false, validateFormats: false }).validate( + definition.inputSchema, + input, + ) + ) + throw new Error( + "Selected SQLite request is not advertised by this runtime", + ); + const inspected = await call("inspect_sqlite_database", input); + const report = normalized(inspected); + if (report.rows === null || report.rows.table !== "countries") + throw new Error("Runtime returned no selected country sample"); + const indexes = columns.map((column) => { + const index = report.rows.columns.indexOf(column); + if (index < 0) + throw new Error(`Missing requested SQLite column ${column}`); + return index; + }); + return { + artifact_sha256: targetSha(inspected), + artifact: report.artifact, + wal: report.wal, + engine: report.engine, + database_encoding: report.database_encoding, + schema: report.schema, + rows: { + ...report.rows, + columns, + values: report.rows.values.map((row) => + indexes.map((index) => row[index]), + ), + }, + producer_limitations: report.limitations, + limitations: inspected.limitations, + evidence_ids: [inspected.evidence_id], + }; + }, "obsidian.asar": async () => { if (values["javascript-question"] === "selected") { const inspected = await call("analyze_javascript_application", { @@ -902,6 +956,10 @@ const report = { ) ? values["managed-question"] : null, + sqlite_selection: + values.artifact === "qgis-world-map.gpkg" + ? values["sqlite-selection"] + : null, provider_settings_sha256: sha(JSON.stringify(selectedProviders)), provider_search_path_sha256: sha( selectedProviders.PATH ?? process.env.PATH ?? "", diff --git a/experiments/analysis-efficiency/questions/sqlite-qgis.txt b/experiments/analysis-efficiency/questions/sqlite-qgis.txt new file mode 100644 index 000000000..636c7a685 --- /dev/null +++ b/experiments/analysis-efficiency/questions/sqlite-qgis.txt @@ -0,0 +1 @@ +Statically inspect the supplied QGIS world-map GeoPackage at {{artifact_path}} using only the registered REA inspect_sqlite_database tool. Select the ordinary countries table with row_limit 1 and its default selected_table schema scope. Report the complete returned schema exactly as reported, including SQL, every declared column, indexes, views, triggers, completeness and explicit nulls. For the one returned sample, report only fid, NAME and ISO_A3 in that order, preserving the original tagged SQLite cells exactly: decimal integer strings, text values with bytes_hex and encoding, and SQL null. Preserve row_limit, returned_rows, truncated and order; row order is unspecified and this is not a complete country inventory. Preserve original artifact identity, wal including null, engine, database_encoding, both normalized-result producer limitations and outer Evidence limitations exactly including order. Do not return geometry blobs or unrequested sample cells, interpret boundaries/political labels as current facts, or infer missing values. Final JSON fields: artifact_sha256, artifact, wal, engine, database_encoding, schema, rows, producer_limitations, limitations, evidence_ids. Include real response Evidence IDs. Do not use shell or host file tools, network, other agents or execute QGIS. Stop once these facts are supported. diff --git a/experiments/analysis-efficiency/review-model.py b/experiments/analysis-efficiency/review-model.py index b8fe16bb3..0d6e1e8d7 100644 --- a/experiments/analysis-efficiency/review-model.py +++ b/experiments/analysis-efficiency/review-model.py @@ -69,7 +69,7 @@ def scope_with_verified_counts(value, expected_scope, returned_counts): def main(): parser = argparse.ArgumentParser() - parser.add_argument('--domain', choices=['wasm', 'evm', 'android', 'flutter-aot', 'managed', 'managed-nonempty', 'firmware', 'inventory', 'javascript', 'native', 'browser'], required=True) + parser.add_argument('--domain', choices=['wasm', 'evm', 'android', 'flutter-aot', 'sqlite', 'managed', 'managed-nonempty', 'firmware', 'inventory', 'javascript', 'native', 'browser'], required=True) parser.add_argument('--trial', type=Path, required=True) parser.add_argument('--reference', type=Path) parser.add_argument('--resource-log', type=Path, required=True) @@ -133,6 +133,11 @@ def main(): for key, expected in {'artifact_bytes': n['artifact']['bytes'], 'bytecode_sha256': n['bytecode']['sha256'], 'bytecode_bytes': n['bytecode']['bytes'], 'functions': n['functions'], 'evidence_kind': n['evidence_kind'], 'discovery_completeness': n['discovery_completeness'], 'runtime_execution': n['runtime_execution'], 'bytecode_kind': n['bytecode']['kind'], 'hardfork': n['bytecode']['hardfork'], 'deployment_authenticity': n['bytecode']['deployment_authenticity']}.items(): check(key, answer.get(key), expected) check('evm_original_limitations', answer.get('limitations'), n['limitations']) + elif domain == 'sqlite': + for key in ['artifact', 'wal', 'engine', 'database_encoding', 'schema', 'rows', 'producer_limitations', 'limitations']: + # Missing is not SQL/JSON null; booleans are not numeric counters. + check(key, (key in answer, json.dumps(answer.get(key), sort_keys=True)), + (True, json.dumps(reference[key], sort_keys=True))) elif domain == 'flutter-aot': for key in ['target', 'abi', 'libapp', 'string_pool', 'coverage', 'limitations']: check(key, answer.get(key), reference[key]) diff --git a/experiments/analysis-efficiency/sqlite-existing-controls.md b/experiments/analysis-efficiency/sqlite-existing-controls.md new file mode 100644 index 000000000..9bac9f064 --- /dev/null +++ b/experiments/analysis-efficiency/sqlite-existing-controls.md @@ -0,0 +1,181 @@ +# SQLite existing-control measurements + +Continues [#2016](https://github.com/morluto/rea/issues/2016) after the Flutter +lane. The measured main runtime is pinned to +`172407787f1842dedbcd27b5385177595f786ce4`. Its only post-Flutter +commit repairs Windows transient runtime admission (#2467); this Linux round +does not establish Windows performance. No production/API change is proposed +merely because another summary or cache could be added. + +Before PR integration, main advanced to +`bb540826d65316c164d2de8ba4ef10f81a8ad073`. Its five intervening commits cover +MCP archive contracts/documentation, an advertised-schema inventory test, +pwntools cleanup identity, concurrent Mach-O slice inspection, and transient +DMG verification retries. None changes the SQLite selector. Rebase and current +main verification do not relabel the frozen measurements below. + +The committed post-format driver passes a real SDK smoke against the frozen +runtime. After rebase, a fresh build, real all/selected-column SDK pair and docs +check pass on the updated main. Selected facts match each other and the frozen +runtime; artifact bytes remain unchanged. These are integration checks, not +additional model performance pairs. + +## Real application data and question + +Use QGIS's bundled `world_map.gpkg`, unmodified from official immutable source +commit `e1223b8e9e71fc50ee995ed556f711c54ccc101b`. `corpus.json` pins its URL, +12,914,688-byte size, Git blob `c9ff80228dff0de93c9acdef16a97545ba360244`, and +SHA-256 `01d293159d1b77bd9212b5252587853f61aa9e25b2387135f34afaba8d057190`. +Streaming acquisition matches the upstream blob. The upstream resource install +list includes `data/*`; [its README](https://github.com/qgis/QGIS/blob/e1223b8e9e71fc50ee995ed556f711c54ccc101b/resources/data/README_world_map.md) +records data provenance. This is shipped application data, not a generated REA +fixture. QGIS is neither installed nor executed. + +The source question asks for the complete returned `countries` table schema +and one sample's `fid`, `NAME`, `ISO_A3` tagged cells, in that order. It preserves +database/WAL identity, engine/encoding, exact original SQL and columns, integer +strings, text bytes, nulls, row limit/count, unspecified order, truncation and +both producer/outer limitations. It does not require geometry blobs or a full +country inventory. Neither political labels nor the sampled ordering are +inferred to be current or globally exhaustive. + +SQLite already defaults to compact schema inventory without a table, detailed +selected-table schema with a table, and exposes exact `columns`. Selecting +columns avoids constructing unrelated cells without changing the requested +table schema. Thus compare existing all-visible columns against exact columns +on the same main runtime, rather than changing an old user's default or adding +a new delivery mechanism. Inventory scope cannot substitute for the detailed +schema requested by this question. + +## Public SDK observations + +Frozen runtime: main `172407787f1842dedbcd27b5385177595f786ce4`, compiled +dist/bridge file-table SHA-256 +`72a4ac6c6db491c455f3e4a798f8f52ed5850c08831e7cdc9f25c1525a5a1d11`. +Node 24.18.0, SDK client/server 2.3.1, observed SQLite 3.53.1, WAL null. Driver, +provider settings/search-path fingerprints, artifact and locked dependencies +remain equal. Each workflow starts a fresh server and validates the selected +actual advertised schema. Host page caches are not flushed. + +The initial default inventory returns 26 table entries, including explicit +virtual/shadow classifications, no rows and `not_requested` column details: +27,185 compact client-result JSON bytes. Selecting `countries` reveals eight +columns, including a geometry blob. Its one-row full reply is 485,277 bytes. +The selected request retains every requested declared-schema field and exact +tagged sample values while omitting unrelated sample cells. + +| Trial | Reply bytes | SDK workflow seconds | Payload CPU seconds | Kernel peak bytes | +| ------------- | ----------: | -------------------: | ------------------: | ----------------: | +| sdk-all-1 | 485,277 | 11.820 | 12.755 | 562,884,608 | +| sdk-columns-1 | 18,509 | 14.017 | 13.715 | 557,510,656 | +| sdk-columns-2 | 18,509 | 15.826 | 14.162 | 548,343,808 | +| sdk-all-2 | 485,277 | 5.679 | 7.310 | 561,704,960 | + +Both pairs make one successful call, pass settings/selected-fact comparison, +retain complete original schema and all requested typed cells, and preserve +source bytes. Reply bytes fall about 96.2%. These observations **do not show SDK +speed or CPU improvement**: selected workflows are slower in both pairs and +timing varies substantially. No cause is inferred from these data. Payload CPU +includes runner/catalog work outside the SDK workflow timer. Memory changes +are small and do not establish a general reduction. Raw replies and the exact +pre-format driver source are archived privately. + +Reproduce with `measure.mjs --artifact qgis-world-map.gpkg` and +`--sqlite-selection all|columns`, using the same `--runtime-root`, `--revision`, +artifact path and `--schemas selected`. `--delivery` stays `complete`; this is +an existing request selection, not a new summary API. Original caller arguments +and schemas remain in the private receipts. The source question is +`questions/sqlite-qgis.txt` and the selected factual grader is +`review-model.py --domain sqlite`. + +## Decision and scope + +The initial four actual Codex attempts never executed SQLite: the client rejected +the tool because its private snapshot writes require approval and the isolated +driver lacked this exact tool's approval override. Every factual grade fails; +these are failed client attempts, not completed-task performance comparisons. +All profiles are removed. The existing experiment-only per-tool approval list +now includes `inspect_sqlite_database`; each corrected trial exposes only that +exact enabled tool. REA annotations, target authority and global Codex configuration +are unchanged. The [official configuration reference](https://learn.chatgpt.com/docs/config-file/config-reference) +documents the per-tool override. Corrected cohorts use fresh output directories +and retain the original failures separately. + +Actual repeated Codex comparisons determine the task token/time benefit before +crediting smaller replies. An existing selector that answers the question is +evidence against adding another production mechanism. Keep negative outcomes +and unavailable coverage, including failure/retry costs and profile cleanup. + +Four corrected serial trials use Codex CLI 0.161.0, requested `gpt-5.6-terra` +high, one enabled tool, identical owned context/discovery capture, and the same +frozen main runtime. The requested alias is not a verified server-resolved model. +Equal treatment instructions select the existing all-visible or exact-column +request; the source question and requested schema/sample facts stay fixed. + +| Corrected trial | Input | Cached | Uncached | Output | Seconds | MCP calls | Facts | +| ------------------------ | -----: | -----: | -------: | -----: | ------: | --------: | ----- | +| approved-model-all-1 | 88,131 | 52,480 | 35,651 | 3,369 | 84.007 | 2 | pass | +| approved-model-columns-1 | 37,340 | 20,992 | 16,348 | 3,006 | 87.514 | 1 | pass | +| approved-model-columns-2 | 56,148 | 36,096 | 20,052 | 3,298 | 97.466 | 2 | pass | +| approved-model-all-2 | 88,340 | 52,480 | 35,860 | 3,084 | 113.847 | 2 | pass | + +Both pairs pass every selected producer fact, exact SQL/schema/null/tagged +cells, observed response Evidence IDs, target-byte preservation and profile +cleanup. The grader checks producer observations, not independent geometry +semantics. Actual events contain only the chosen MCP tool and agent messages; +no shell or other agent execution appears. Usage is the final aggregate actually +reported by the CLI, one usage record per trial; cached input remains included +in total input. Output includes 853 / 357 / 523 / 664 reasoning tokens. + +Total input falls **57.63% / 36.44%**, uncached input **54.14% / 44.08%**. +Time is mixed: columns are **4.17% slower** in pair 1 and **14.39% faster** in +pair 2. Output is 10.77% lower / 6.94% higher. Repeated calls remain counted; +pair 1 has two all-column calls versus one selected call, while pair 2 has two +calls on both sides. This supports scoped token savings from using the existing +selector, not a dependable speedup or a new API requirement. Client context and +usage are observed separately from raw SDK reply bytes. + +Model payload CPU seconds / kernel peaks are 7.596 / 763,838,464; +7.772 / 761,143,296; 8.409 / 762,597,376; and 11.759 / 774,025,216. CPU is mixed +and memory differences are small. Payload cleanup/limit observations pass; +missing terminal counters keep every strict whole-resource grade false. + +The original blocked attempts remain separate and never provide comparison +denominators. They each have one failed call, no SQLite result and failed facts: + +| Blocked trial | Input | Cached | Uncached | Output | Seconds | +| --------------- | -----: | -----: | -------: | -----: | ------: | +| model-all-1 | 35,290 | 29,952 | 5,338 | 798 | 59.881 | +| model-columns-1 | 35,356 | 22,016 | 13,340 | 670 | 44.764 | +| model-columns-2 | 35,034 | 20,992 | 14,042 | 639 | 30.007 | +| model-all-2 | 35,104 | 20,992 | 14,112 | 673 | 25.132 | + +**Decision: no production/API change.** Existing column selection materially +reduces unnecessary delivery and actual model input while retaining the complete +requested schema and exact cells. Existing documentation already recommends +table/column selection. No additional summary, schema-scope default change, +automatic field omission, plugin layer or cross-session cache is justified by +these results. Preserve the negative/mixed time, output and CPU findings. + +Real one-shot CLI inventory, all-column sample and exact-column sample match +the corresponding full MCP normalized results, subjects, parameters and IDs; +the source digest remains unchanged. An independent Python binding using SQLite +3.40.1 opens this immutable artifact read-only and agrees with the exact stored +table SQL, eight declared columns and selected tagged sample cells. This is a +second engine/binding observation of this file, not a general row-order or +geometry-semantics guarantee. The six existing context/driver regression tests +pass; Python compilation and JavaScript syntax checks pass. No new product test +or production implementation is added for this measurement-only finding. + +Manual review tightens the SQLite grader to distinguish missing fields from +explicit null and preserve JSON scalar types. Regrading all four stored model +answers still passes every selected fact; deliberate omission of `wal` and a +boolean substituted for numeric `row_limit` both fail. Original transcripts, +usage and previous review receipts stay intact; no model trial is rerun. + +Heavy tasks remain serial under one CPU, unchanged 3 GiB early stop / 4 GiB hard +memory, no swap, owned task scopes and host guards. SDK payload observations +verify cleanup and zero phase limit events; final whole-scope counters remain +unknown. This does not establish live/WAL atomicity, all SQLite applications, +other runtimes/platforms or geospatial geometry semantics. Other packet/signal +and capture lanes remain incremental follow-ups using their existing controls. From fa4b8288655188426222773ead2ccbdaf76334fd Mon Sep 17 00:00:00 2001 From: Ysopwnable <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:43:52 +0800 Subject: [PATCH 012/159] fix(cli): normalize empty Windows profile before imports (#2482) Follow-up to #2345; thanks to @gefrezim for the import-time home lookup diagnosis. Normalize only an empty Windows USERPROFILE before runtime dependency imports, preserving all nonempty caller selections. Reuse the existing public setup dry-run regression in the Windows curated CI lane. No additional filesystem query, engine call, cache or retry. Verified real Windows Node 24.18.0 CLI unset/empty/selected-profile controls, setup dry-run home selection and unchanged configuration, and baseline/candidate SDK stdio MCP catalogue parity. Nine related tests and static/docs checks passed. No IDA or Ghidra engine was used. Scoped resource observations and measurement limits are recorded in experiments/analysis-efficiency/windows-empty-profile.md. Fixed the review finding and original CI package-scope ownership failure. Current-head CI is green; the unrelated MCP identity timeout passed on a failed-job rerun without changing its code or timeout. Current-head review completed with no further findings. --- .github/workflows/ci.yml | 1 + .../windows-empty-profile.md | 69 +++++++++++++++++++ scripts/ci/scopes.mjs | 1 + scripts/rea.mjs | 6 ++ 4 files changed, 77 insertions(+) create mode 100644 experiments/analysis-efficiency/windows-empty-profile.md diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 86f1ba2e4..6e6521b41 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -509,6 +509,7 @@ jobs: tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts tests/boundary/filesystem/setupUnavailableProvider.test.ts tests/boundary/filesystem/windowsNpxRegistration.test.ts + tests/acceptance/setup/homeDirectoryFallback.test.ts tests/boundary/filesystem/webScriptManifestPaths.test.ts tests/boundary/process/ProcessEvidence.host.test.ts tests/acceptance/webModuleCli.test.ts diff --git a/experiments/analysis-efficiency/windows-empty-profile.md b/experiments/analysis-efficiency/windows-empty-profile.md new file mode 100644 index 000000000..f8e5558e0 --- /dev/null +++ b/experiments/analysis-efficiency/windows-empty-profile.md @@ -0,0 +1,69 @@ +# Windows empty-profile startup verification + +Follow-up to [#2345](https://github.com/morluto/rea/pull/2345), including +@gefrezim's diagnosis of the import-time Windows home lookup. + +## Source and boundary + +Baseline: main `172407787f1842dedbcd27b5385177595f786ce4`. Verification used +the real Windows 11 10.0.22631.0 host, official Windows x64 Node 24.18.0, +locked production dependencies and REA's compiled public CLI / SDK 2.3.1 MCP +server. No IDA or Ghidra engine was used. Runtime copies and compact receipts +live outside the checkout; no account profile path or ambient environment is +recorded here. + +The tested dispatcher SHA-256 is +`526105418604cc6e5a202a6275b700814e27450bfaffd45e7510530ba669de8d`. +Only the dispatcher changed in the deployed runtime. It treats an empty Windows +`USERPROFILE` as unset before dependency imports. Nonempty selections continue +to reach the existing home resolver; no additional lookup or fallback owner is +introduced. + +## Real Windows observations + +With `HOME` empty in each case, the baseline produced: + +| USERPROFILE selection | `--version` | `--help` | MCP discovery | +| --------------------------- | ----------------------- | ----------------------- | ------------- | +| Unset | Exit 0 | Exit 0 | Healthy | +| Empty | Exit 1, `uv_os_homedir` | Exit 1, `uv_os_homedir` | Healthy | +| Nonempty test-owned profile | Exit 0 | Exit 0 | Healthy | + +The corrected CLI passed all six invocations with no stderr. Version output +remained 6 bytes and help output 15,497 bytes. Real setup dry runs with empty +and nonempty profile selections both exited 0, planned their configuration +under the expected home, preserved existing configuration bytes and created no +configuration under the working directory. No installation was performed. + +MCP was already healthy in the baseline; this change does not claim to repair +an MCP startup failure. Complete baseline discovery was 187 tools and 2,545,790 +serialized bytes, SHA-256 +`1e6bbca7f62852134eb703ea098f5a302d1fe333b8433475429f6fc1b1aba18f`. +All three corrected MCP controls also passed with no stderr and the same +catalogue bytes and digest. + +The existing `homeDirectoryFallback` acceptance test exercises the import-time +failure through the public setup dry run. Adding it to the Windows curated lane +makes that regression execute on the affected platform. No duplicate startup +test was retained. Nine related tests passed, including the existing MCP doctor +and curated-lane path checks; Linux passing alone does not prove Windows behavior. +Review caught a missing package-scope registration for the newly listed test; +the original CI shard failed that ownership invariant. The path is now also +registered in `scripts/ci/scopes.mjs`, and all 84 scope checks plus the two +curated-path checks pass. Runtime source and its tested digest are unchanged. + +## Efficiency and resource limits + +Heavy host probes ran serially at BelowNormal priority, with one CPU and a +512 MiB CLI/server heap setting (256 MiB for the probe parent). Heap settings do +not bound total process memory. An exit-time `process.resourceUsage()` observer +recorded individual corrected CLI invocations: CPU time ranged from 7.968 to +9.874 seconds and peak working set from 299,956 to 301,852 KiB. These figures +exclude the probe parent and any observer overhead outside the CLI process. +The six observed wall times ranged from 8.656 to 10.961 seconds. + +The baseline did not use that resource observer, and cold/warm caches were not +normalized. These observations establish functional behavior and bounded probe +scope; no CPU, memory or latency improvement is claimed. The fix adds one +constant-time environment check, no filesystem lookup, engine call, retry or +cache. Output bytes are not model tokens; actual model usage was not measured. diff --git a/scripts/ci/scopes.mjs b/scripts/ci/scopes.mjs index 497dfeed0..4493fe512 100644 --- a/scripts/ci/scopes.mjs +++ b/scripts/ci/scopes.mjs @@ -55,6 +55,7 @@ export const SCOPES = { "tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts", "tests/boundary/filesystem/setupUnavailableProvider.test.ts", "tests/boundary/filesystem/windowsNpxRegistration.test.ts", + "tests/acceptance/setup/homeDirectoryFallback.test.ts", "tests/boundary/filesystem/webScriptManifestPaths.test.ts", "tests/boundary/process/ProcessEvidence.host.test.ts", "tests/acceptance/webModuleCli.test.ts", diff --git a/scripts/rea.mjs b/scripts/rea.mjs index 78ffbdd21..75ee552c5 100755 --- a/scripts/rea.mjs +++ b/scripts/rea.mjs @@ -16,6 +16,12 @@ const isMcpMode = const isMcpDoctorMode = args[0] === "mcp" && args[1] === "doctor"; const packageRoot = resolve(dirname(fileURLToPath(import.meta.url)), ".."); +// Windows home lookup throws for an empty USERPROFILE. Dependencies resolve +// home during import, before REA's own resolver can apply its account fallback. +if (process.platform === "win32" && process.env.USERPROFILE === "") { + delete process.env.USERPROFILE; +} + // Playwright's registry assumes one of its three desktop host platforms when // it chooses the browser cache directory. Termux reports `android`, although // its Node runtime and filesystem are POSIX-compatible. Keep the cache path From b045df100e1489e08a36d6ff04cea7feecf1332c Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:46:05 +0800 Subject: [PATCH 013/159] fix(update): share parsed client activation with maintenance --- src/application/ClientRegistrationStatus.ts | 106 +++++++++++------- src/application/UpdateMaintenance.ts | 51 ++------- .../process-global/updateMaintenance.test.ts | 35 ++++++ 3 files changed, 112 insertions(+), 80 deletions(-) diff --git a/src/application/ClientRegistrationStatus.ts b/src/application/ClientRegistrationStatus.ts index e2696e088..caeba4fb6 100644 --- a/src/application/ClientRegistrationStatus.ts +++ b/src/application/ClientRegistrationStatus.ts @@ -7,6 +7,7 @@ import { clientServerForcedEnabled, clientServerListedDisabled, effectiveClientServer, + type ClientConfigurationDocument, } from "./ClientConfigurationDocument.js"; import { access } from "node:fs/promises"; import { resolve } from "node:path"; @@ -74,24 +75,48 @@ const registrationSchema = z }) .passthrough(); -/** Inspect supported client registrations without reading their environment. */ +interface ClientRegistrationReadOptions { + readonly platform?: NodeJS.Platform; + readonly environment?: NodeJS.ProcessEnv; + readonly resources?: ArtifactResourceScope; +} + +/** Registration facts shared by diagnostics and update maintenance. */ +export interface ClientRegistrationInspection { + readonly status: ClientRegistrationStatus; + readonly activation: "enabled" | "disabled" | "unknown"; +} + +/** Project diagnostics from the same inspection used to select maintenance. */ export const readClientRegistrationStatuses = async ( home: string, currentCommandPath: string = resolve(process.argv[1] ?? "unknown"), - options: { - readonly platform?: NodeJS.Platform; - readonly environment?: NodeJS.ProcessEnv; - readonly resources?: ArtifactResourceScope; - } = {}, -): Promise => { - const statuses: ClientRegistrationStatus[] = []; + options: ClientRegistrationReadOptions = {}, +): Promise => + ( + await readClientRegistrationInspections(home, currentCommandPath, options) + ).map(({ status }) => status); + +/** Parse effective registrations and their client-specific activation once. */ +export const readClientRegistrationInspections = async ( + home: string, + currentCommandPath: string = resolve(process.argv[1] ?? "unknown"), + options: ClientRegistrationReadOptions = {}, +): Promise => { + const inspections: ClientRegistrationInspection[] = []; + const record = ( + status: ClientRegistrationStatus, + activation: ClientRegistrationInspection["activation"] = "unknown", + ): void => { + inspections.push({ status, activation }); + }; for (const client of supportedClients( home, options.platform, options.environment, )) { if (client.configPathError !== undefined) { - statuses.push({ + record({ client: client.name, config_path: client.configPath, command: [], @@ -104,7 +129,7 @@ export const readClientRegistrationStatuses = async ( const manualRemediation = manualRegistrationRemediation(client.name); if (client.format === "unsupported") { if (manualRemediation !== undefined) - statuses.push({ + record({ client: client.name, config_path: client.markerPath ?? client.configPath, command: [], @@ -119,7 +144,7 @@ export const readClientRegistrationStatuses = async ( options.resources, ); if (!files.ok) { - statuses.push({ + record({ client: client.name, config_path: client.configPath, command: [], @@ -130,9 +155,7 @@ export const readClientRegistrationStatuses = async ( } const parsed = effectiveClientConfiguration(files.value); if (parsed === undefined) { - statuses.push( - unavailableStatus(client.name, client.configPath, "missing"), - ); + record(unavailableStatus(client.name, client.configPath, "missing")); continue; } const policy = await readClientPolicyBlock( @@ -142,7 +165,7 @@ export const readClientRegistrationStatuses = async ( options.resources, ); if (!policy.ok) { - statuses.push({ + record({ client: client.name, config_path: client.configPath, command: [], @@ -159,7 +182,7 @@ export const readClientRegistrationStatuses = async ( ? dshDocument(parsed.document).editabilityError : undefined; if (editabilityError !== undefined) { - statuses.push({ + record({ client: client.name, config_path: client.configPath, command: [], @@ -168,9 +191,7 @@ export const readClientRegistrationStatuses = async ( }); continue; } - statuses.push( - unavailableStatus(client.name, client.configPath, "missing"), - ); + record(unavailableStatus(client.name, client.configPath, "missing")); continue; } const registration = parseRegistration(raw, client); @@ -178,21 +199,18 @@ export const readClientRegistrationStatuses = async ( registration.command, ...registration.args, ]; - statuses.push( + const activation = registrationActivation(registration, client, parsed); + record( configuredStatus( client.name, client.configPath, command, - registrationAligned( - registration, - client, - currentCommandPath, - options.platform ?? process.platform, - clientServerForcedEnabled(parsed, PRODUCT_IDENTITY.mcpServerKey), - ) && - !clientServerListedDisabled( - parsed, - PRODUCT_IDENTITY.mcpServerKey, + activation === "enabled" && + registrationAligned( + registration, + client, + currentCommandPath, + options.platform ?? process.platform, ) && (parsed.dialect !== "dsh" || dshDocument(parsed.document).ready) && policyBlock === undefined @@ -200,9 +218,10 @@ export const readClientRegistrationStatuses = async ( : "stale", policyBlock, ), + activation, ); } catch (cause: unknown) { - statuses.push( + record( unavailableStatus( client.name, client.configPath, @@ -211,26 +230,31 @@ export const readClientRegistrationStatuses = async ( ); } } - return statuses.sort((left, right) => - compareUnicodeCodePoints(left.client, right.client), + return inspections.sort((left, right) => + compareUnicodeCodePoints(left.status.client, right.status.client), ); }; +const registrationActivation = ( + registration: z.output, + client: SetupClient, + parsed: ClientConfigurationDocument, +): "enabled" | "disabled" => + (client.format !== "grok" && registration.disabled === true) || + (registration.enabled === false && + !clientServerForcedEnabled(parsed, PRODUCT_IDENTITY.mcpServerKey)) || + (client.format === "zcode" && registration.enable === false) || + clientServerListedDisabled(parsed, PRODUCT_IDENTITY.mcpServerKey) + ? "disabled" + : "enabled"; + const registrationAligned = ( registration: z.output, client: SetupClient, currentCommandPath: string, platform: NodeJS.Platform, - forcedEnabled = false, ): boolean => { const command = [registration.command, ...registration.args]; - if ( - (client.format !== "grok" && registration.disabled === true) || - (registration.enabled === false && !forcedEnabled) || - // ZCode's native flag is `enable`; other dialects leave that field opaque. - (client.format === "zcode" && registration.enable === false) - ) - return false; if (!isOwnedClientRegistrationCommand(command, currentCommandPath)) return false; if ( diff --git a/src/application/UpdateMaintenance.ts b/src/application/UpdateMaintenance.ts index 06ac1b512..fd55c8f23 100644 --- a/src/application/UpdateMaintenance.ts +++ b/src/application/UpdateMaintenance.ts @@ -1,19 +1,10 @@ import { z } from "zod"; -import { PRODUCT_IDENTITY } from "../identity.js"; import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js"; -import { readClientRegistrationStatuses } from "./ClientRegistrationStatus.js"; -import { - clientServerForcedEnabled, - clientServerListedDisabled, - effectiveClientServer, - parseClientConfiguration, -} from "./ClientConfigurationDocument.js"; -import { supportedClients } from "./SupportedClients.js"; +import { readClientRegistrationInspections } from "./ClientRegistrationStatus.js"; import { existingSkillDestinations } from "./SetupSkill.js"; import type { SetupAction } from "./SetupTypes.js"; import type { Result } from "../domain/result.js"; -import { readRegularFileText } from "./RegularFileRead.js"; import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; /** Existing REA integrations selected for maintenance, without discovering new targets. */ @@ -61,41 +52,23 @@ export const existingMaintenanceScope = async ( environment: Readonly, resources?: ArtifactResourceScope, ): Promise => { - const registrations = await readClientRegistrationStatuses(home, entryPoint, { - environment, - ...(resources === undefined ? {} : { resources }), - }); - const supported = supportedClients(home, process.platform, environment); + const registrations = await readClientRegistrationInspections( + home, + entryPoint, + { + environment, + ...(resources === undefined ? {} : { resources }), + }, + ); const clients: string[] = []; - for (const registration of registrations) { + for (const { status: registration, activation } of registrations) { if ( + activation !== "enabled" || (registration.state !== "aligned" && registration.state !== "stale") || !isOwnedClientRegistrationCommand(registration.command, entryPoint) ) continue; - const client = supported.find(({ name }) => name === registration.client); - if (client === undefined) continue; - const parsed = parseClientConfiguration( - await readRegularFileText( - client.configPath, - resources === undefined ? {} : { resources }, - ), - client.format, - ); - const enabled = z - .object({ - enabled: z.boolean().optional(), - disabled: z.boolean().optional(), - }) - .parse(effectiveClientServer(parsed, PRODUCT_IDENTITY.mcpServerKey)); - // Root disable lists override enabled entries and OMP's force-on allowlist. - if ( - (enabled.enabled !== false || - clientServerForcedEnabled(parsed, PRODUCT_IDENTITY.mcpServerKey)) && - enabled.disabled !== true && - !clientServerListedDisabled(parsed, PRODUCT_IDENTITY.mcpServerKey) - ) - clients.push(client.name); + clients.push(registration.client); } const skillDestinations = await existingSkillDestinations( home, diff --git a/tests/process-global/updateMaintenance.test.ts b/tests/process-global/updateMaintenance.test.ts index a60a2e213..ae057bab2 100644 --- a/tests/process-global/updateMaintenance.test.ts +++ b/tests/process-global/updateMaintenance.test.ts @@ -126,6 +126,41 @@ describe("OMP integration maintenance", () => { }); }); +describe("maintenance activation and layered configuration", () => { + it("leaves a natively disabled ZCode registration outside maintenance", async () => { + const configPath = await writeClient( + "zcode", + JSON.stringify({ + mcp: { + servers: { + rea: { command: entryPoint, args: ["mcp"], enable: false }, + }, + }, + }), + ); + const original = await readFile(configPath, "utf8"); + await expect( + existingMaintenanceScope(home, entryPoint, process.env), + ).resolves.toMatchObject({ clients: [] }); + expect(await readFile(configPath, "utf8")).toBe(original); + }); + + it("selects an enabled registration inherited through layered OpenCode files", async () => { + const directory = join(home, ".config", "opencode"); + await mkdir(directory, { recursive: true }); + await writeFile( + join(directory, "opencode.json"), + JSON.stringify({ + mcp: { rea: { type: "local", command: [entryPoint, "mcp"] } }, + }), + ); + await writeClient("opencode", JSON.stringify({ theme: "user-selected" })); + await expect( + existingMaintenanceScope(home, entryPoint, process.env), + ).resolves.toMatchObject({ clients: ["opencode"] }); + }); +}); + describe("existing REA integration maintenance", () => { it("selects owned registrations while preserving unconfigured, foreign, and disabled choices", async () => { await writeClient("codex", staleCodex); From 042dcaa1ee66f1b8668f08b193f38ab01bceaacb Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:46:06 +0800 Subject: [PATCH 014/159] perf(evidence): reuse authenticated unknown mutation snapshots --- .../EvidenceLedger.residualUnknown.test.ts | 45 +++++++++++++++++++ .../investigation/EvidenceLedger.ts | 14 +++--- tests/boundary/mcp/serverUnknownsMcp.test.ts | 15 +++---- 3 files changed, 57 insertions(+), 17 deletions(-) diff --git a/src/application/investigation/EvidenceLedger.residualUnknown.test.ts b/src/application/investigation/EvidenceLedger.residualUnknown.test.ts index bb81f97ce..2885c1906 100644 --- a/src/application/investigation/EvidenceLedger.residualUnknown.test.ts +++ b/src/application/investigation/EvidenceLedger.residualUnknown.test.ts @@ -78,6 +78,51 @@ const update = ( }); describe("residual unknown registry reads and identity", () => { + it("seals mutation Evidence without freezing caller-owned inputs", () => { + const store = ledger(); + const recordMutation = mutation("mutable-record-mutation"); + const recorded = store.recordUnknown( + input("Does record mutation evidence stay stable?"), + recordMutation, + ); + if (!recorded.ok) throw recorded.error; + + const updateMutation = mutation("mutable-update-mutation"); + const updated = store.updateUnknown(update(recorded.value), updateMutation); + if (!updated.ok) throw updated.error; + const atomicEvidence = evidence("mutable-atomic-evidence"); + const atomicMutation = mutation("mutable-atomic-mutation"); + const atomicallyRecorded = store.recordWithUnknown( + atomicEvidence, + input("Does atomic evidence stay stable?"), + atomicMutation, + ); + if (!atomicallyRecorded.ok) throw atomicallyRecorded.error; + + for (const callerEvidence of [ + recordMutation, + updateMutation, + atomicEvidence, + atomicMutation, + ]) { + expect(Object.isFrozen(callerEvidence)).toBe(false); + const retained = store.forAnalysis(callerEvidence.evidence_id); + if (retained === undefined) throw new Error("Missing retained Evidence"); + const originalResult = structuredClone(retained.normalized_result); + expect(Object.isFrozen(retained)).toBe(true); + Reflect.set(callerEvidence, "operation", "caller mutation"); + expect( + Reflect.set( + callerEvidence.normalized_result as object, + "label", + "caller nested mutation", + ), + ).toBe(true); + expect(retained.operation).not.toBe("caller mutation"); + expect(retained.normalized_result).toEqual(originalResult); + } + }); + it("merges record-only bundles without changing retained unknown history", () => { const store = ledger(); const created = store.recordUnknown( diff --git a/src/application/investigation/EvidenceLedger.ts b/src/application/investigation/EvidenceLedger.ts index ce806ad38..0cb1c5e29 100644 --- a/src/application/investigation/EvidenceLedger.ts +++ b/src/application/investigation/EvidenceLedger.ts @@ -11,11 +11,7 @@ import { import { AnalysisInputError } from "../../domain/analysisErrorCore.js"; import { EvidenceIntegrityError } from "../../domain/evidenceErrors.js"; import { UnknownRegistryError } from "../../domain/unknownRegistryError.js"; -import { - immutableEvidence, - parseEvidence, - type Evidence, -} from "../../domain/evidence.js"; +import { immutableEvidence, type Evidence } from "../../domain/evidence.js"; import { createResidualUnknown, updateResidualUnknown, @@ -220,7 +216,7 @@ export class EvidenceLedger { ): Result { let parsedMutation: Evidence; try { - parsedMutation = parseEvidence(mutationEvidence); + parsedMutation = immutableEvidence(mutationEvidence); } catch (cause: unknown) { return err( new EvidenceIntegrityError("Unknown mutation evidence is invalid", { @@ -255,8 +251,8 @@ export class EvidenceLedger { let evidence: Evidence; let mutation: Evidence; try { - evidence = parseEvidence(evidenceInput); - mutation = parseEvidence(mutationInput); + evidence = immutableEvidence(evidenceInput); + mutation = immutableEvidence(mutationInput); } catch (cause: unknown) { return err( new EvidenceIntegrityError("Atomic Evidence validation failed", { @@ -394,7 +390,7 @@ export class EvidenceLedger { ): Result { let mutationEvidence: Evidence; try { - mutationEvidence = parseEvidence(mutationEvidenceInput); + mutationEvidence = immutableEvidence(mutationEvidenceInput); } catch (cause: unknown) { return err( new EvidenceIntegrityError("Unknown mutation evidence is invalid", { diff --git a/tests/boundary/mcp/serverUnknownsMcp.test.ts b/tests/boundary/mcp/serverUnknownsMcp.test.ts index 0a27324ec..e39ce2aec 100644 --- a/tests/boundary/mcp/serverUnknownsMcp.test.ts +++ b/tests/boundary/mcp/serverUnknownsMcp.test.ts @@ -281,7 +281,7 @@ it("pages filtered unknown heads through the public MCP contract", async () => { supporting_evidence_ids: [], contradicting_evidence_ids: [], }); - expect(result.ok).toBe(true); + if (!result.ok) throw result.error; } const defaultPage = structured( @@ -372,12 +372,11 @@ it("pages filtered unknown heads through the public MCP contract", async () => { .result as typeof defaultPage; expect(first).toMatchObject({ total: 3, offset: 0, next_offset: 2 }); expect(first.items).toHaveLength(2); - const second = structured( - await client.callTool({ - ...request, - arguments: { ...request.arguments, offset: first.next_offset ?? 0 }, - }), - ).result as typeof defaultPage; + const secondResponse = await client.callTool({ + ...request, + arguments: { ...request.arguments, offset: first.next_offset ?? 0 }, + }); + const second = structured(secondResponse).result as typeof defaultPage; expect(second).toMatchObject({ total: 3, offset: 2, next_offset: null }); expect(second.items).toHaveLength(1); expect( @@ -389,7 +388,7 @@ it("pages filtered unknown heads through the public MCP contract", async () => { ); expect( toolContract("list_unknowns").outputSchema.safeParse( - (await client.callTool(request)).structuredContent, + secondResponse.structuredContent, ).success, ).toBe(true); From 9581e067ad306aab842024e47f2dd18ac8056859 Mon Sep 17 00:00:00 2001 From: Quentin Limo <85309073+3QL@users.noreply.github.com> Date: Tue, 13 Oct 2026 06:53:03 +0200 Subject: [PATCH 015/159] feat(pcap): add bounded capture summaries through CLI and MCP Summarize saved Wireshark4.x captures with enforced native resource limits, serialized admission, stable snapshot verification and retained fact/text pages. Preserve complete inventories within resource budgets and source uncertainty; report cleanup failure with partial Evidence and retained ownership. Verified six recorded corpora /13026 packets with independent Scapy/Python oracles, real CLI/MCP/schema parity,60 SNI names,132 DNS addresses and1000 URIs. Real TShark CI now exercises summary/paging. Actual Windows/Bun Wireshark remains unverified; macOS capture requires a separate enforceable-bound decision. Co-authored-by: 3QL <85309073+3QL@users.noreply.github.com> Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com> --- .../skills/reverse-engineer-anything/SKILL.md | 6 + .github/workflows/real-packet-capture.yml | 8 + README.md | 1 + README_ar.md | 1 + README_de.md | 1 + README_es.md | 1 + README_fa.md | 1 + README_fr.md | 1 + README_id.md | 1 + README_it.md | 1 + README_ja.md | 1 + README_ko.md | 1 + README_pl.md | 1 + README_pt-BR.md | 1 + README_ru.md | 1 + README_th.md | 1 + README_tr.md | 1 + README_uk.md | 1 + README_vi.md | 1 + README_zh-TW.md | 1 + README_zh.md | 1 + docs/pcap-analysis.md | 118 ++++ docs/testing.md | 8 + package.json | 1 + scripts/lib/catalog-builders.mjs | 9 + scripts/lib/catalog-core.mjs | 4 +- scripts/verify-packet-capture.mjs | 46 ++ scripts/verify-pcap.mjs | 131 +++++ src/application/CapabilityInventory.ts | 2 +- src/application/pcap/PcapAnalysisPort.ts | 23 + src/application/pcap/PcapAnalysisService.ts | 36 ++ src/cli.ts | 2 + src/cli/pcapCommands.ts | 69 +++ src/cliCommandNames.ts | 2 + src/composition/pcap.ts | 7 + src/contracts/analysisViewToolContracts.ts | 2 +- src/contracts/pcap/pcapToolContracts.ts | 46 ++ src/contracts/toolContractTypes.ts | 1 + src/contracts/toolContracts.ts | 2 + src/contracts/toolEffects.test.ts | 2 + src/contracts/toolEffects.ts | 10 + src/domain/analysisView/analysisView.ts | 28 + src/domain/analysisView/pcapCaptureView.ts | 327 +++++++++++ src/domain/pcap/pcapAnalysis.ts | 110 ++++ src/domain/pcap/pcapEvidence.ts | 34 ++ src/pcap/PcapCommand.ts | 207 +++++++ src/pcap/PcapConfiguration.test.ts | 170 ++++++ src/pcap/PcapConfiguration.ts | 153 ++++++ src/pcap/PcapNativeBound.ts | 68 +++ src/pcap/PcapOutput.test.ts | 149 +++++ src/pcap/PcapOutput.ts | 315 +++++++++++ src/pcap/PcapProvider.ts | 493 +++++++++++++++++ src/pcap/PcapSnapshot.ts | 269 ++++++++++ src/process/DarwinProcessRunTokenReader.ts | 6 +- src/server/createServer.ts | 15 + src/server/registerPcapTools.ts | 83 +++ tests/boundary/cli/pcapCli.test.ts | 23 + tests/boundary/mcp/pcapMcp.test.ts | 118 ++++ tests/boundary/pcap/pcapCommand.test.ts | 53 ++ tests/boundary/pcap/pcapProvider.test.ts | 508 ++++++++++++++++++ .../pcap/pcapProviderCrossPlatform.test.ts | 144 +++++ tests/boundary/pcap/pcapRetainedView.test.ts | 196 +++++++ .../darwinProcessRunTokenReader.test.ts | 27 +- tests/fixtures/localToolsMcp.ts | 5 +- tests/fixtures/pcap/capinfos-stub.cmd | 19 + tests/fixtures/pcap/capinfos-stub.sh | 11 + tests/fixtures/pcap/tshark-stub.cmd | 17 + tests/fixtures/pcap/tshark-stub.sh | 6 + vitest.config.ts | 2 +- 69 files changed, 4082 insertions(+), 28 deletions(-) create mode 100644 docs/pcap-analysis.md create mode 100644 scripts/verify-pcap.mjs create mode 100644 src/application/pcap/PcapAnalysisPort.ts create mode 100644 src/application/pcap/PcapAnalysisService.ts create mode 100644 src/cli/pcapCommands.ts create mode 100644 src/composition/pcap.ts create mode 100644 src/contracts/pcap/pcapToolContracts.ts create mode 100644 src/domain/analysisView/pcapCaptureView.ts create mode 100644 src/domain/pcap/pcapAnalysis.ts create mode 100644 src/domain/pcap/pcapEvidence.ts create mode 100644 src/pcap/PcapCommand.ts create mode 100644 src/pcap/PcapConfiguration.test.ts create mode 100644 src/pcap/PcapConfiguration.ts create mode 100644 src/pcap/PcapNativeBound.ts create mode 100644 src/pcap/PcapOutput.test.ts create mode 100644 src/pcap/PcapOutput.ts create mode 100644 src/pcap/PcapProvider.ts create mode 100644 src/pcap/PcapSnapshot.ts create mode 100644 src/server/registerPcapTools.ts create mode 100644 tests/boundary/cli/pcapCli.test.ts create mode 100644 tests/boundary/mcp/pcapMcp.test.ts create mode 100644 tests/boundary/pcap/pcapCommand.test.ts create mode 100644 tests/boundary/pcap/pcapProvider.test.ts create mode 100644 tests/boundary/pcap/pcapProviderCrossPlatform.test.ts create mode 100644 tests/boundary/pcap/pcapRetainedView.test.ts create mode 100644 tests/fixtures/pcap/capinfos-stub.cmd create mode 100755 tests/fixtures/pcap/capinfos-stub.sh create mode 100644 tests/fixtures/pcap/tshark-stub.cmd create mode 100755 tests/fixtures/pcap/tshark-stub.sh diff --git a/.agents/skills/reverse-engineer-anything/SKILL.md b/.agents/skills/reverse-engineer-anything/SKILL.md index 3b508af1c..f8ec45268 100644 --- a/.agents/skills/reverse-engineer-anything/SKILL.md +++ b/.agents/skills/reverse-engineer-anything/SKILL.md @@ -83,6 +83,12 @@ explicit path or endpoint and do not need it. frames; use `match_offset` / `match_limit` for later pages, or explicit null limit for complete matching output. Preserve the source digest across pages. This does not perform live capture. + For address/DNS/SNI/protocol summaries use `inspect_pcap_capture` with MCP + detail summary, then retained kind pcap pages. Requires Wireshark 4.x tshark + and capinfos, with Linux prlimit or Windows Job Objects; see + [packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md). + Select an exact DNS question for address pages; page exhaustion does not prove + network completeness. This reads recordings and does not acquire traffic. - Recorded logic/signal traces: `decode_signal_recording` with explicit decoder, channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through `REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md). diff --git a/.github/workflows/real-packet-capture.yml b/.github/workflows/real-packet-capture.yml index 8faf9ba99..f481ed00d 100644 --- a/.github/workflows/real-packet-capture.yml +++ b/.github/workflows/real-packet-capture.yml @@ -7,6 +7,13 @@ on: - "tests/boundary/cli/captureCleanup.test.ts" - "tests/boundary/mcp/packetCaptureMcp.test.ts" - "src/captures/**" + - "src/pcap/**" + - "src/domain/pcap/**" + - "src/domain/analysisView/pcapCaptureView.ts" + - "src/application/pcap/**" + - "src/contracts/pcap/**" + - "src/server/registerPcapTools.ts" + - "tests/boundary/pcap/**" - "src/domain/captures/**" - "src/application/captures/**" - "src/contracts/captures/**" @@ -48,3 +55,4 @@ jobs: - run: node scripts/verify-packet-capture.mjs env: REA_TSHARK_COMMAND: /usr/bin/tshark + REA_CAPINFOS_COMMAND: /usr/bin/capinfos diff --git a/README.md b/README.md index 49173ab4b..6b78543ef 100644 --- a/README.md +++ b/README.md @@ -170,6 +170,7 @@ Additional tools and host support depend on the target: | Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) | | Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) | | Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) | +| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) | | Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) | | Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) | | Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) | diff --git a/README_ar.md b/README_ar.md index 0693fe67f..4c33ac544 100644 --- a/README_ar.md +++ b/README_ar.md @@ -139,6 +139,7 @@ npx rea-agents@latest setup | JavaScript / Electron | الوحدات، وعمليات الاستيراد، وخرائط المصدر، والمسارات، وIPC، وعلاقات إضافات الشيفرة الأصلية | Node.js وnpm؛ [تحليل التطبيقات](https://rea.tools/ar/guides/javascript/) | | المواقع | بنية الصفحة، والسكربتات، وملاحظات الشبكة، ولقطات الشاشة المطلوبة | متصفح من عائلة Chrome؛ [تحليل المتصفح](https://rea.tools/ar/guides/browser/) | | تسجيلات الشبكة المحفوظة | الطلبات، والاستجابات، ومحتويات الحمولة المتاحة، ومواضعها في المصدر | HAR؛ وmitmdump على Linux لتسجيلات mitmproxy بصيغتها الأصلية؛ [دليل تسجيلات الشبكة](docs/web-network-captures.md) | +| التقاط الحزم | نقاط النهاية وDNS وTLS SNI وتوزيع البروتوكولات وطلبات HTTP | tshark وcapinfos يوفّرهما المُستدعي؛ [دليل pcap](docs/pcap-analysis.md) | | تجميعات .NET | البيانات الوصفية، وتعليمات CIL، واعتماديات الشيفرة الأصلية المعلنة، ومقارنات البناء | فحص ثابت؛ [دليل الشيفرة المُدارة](docs/managed-code-analysis.md) | | حزم Android APK | تصريحات ملف manifest، والأصناف، والدوال الناتجة عن فك الترجمة، والمراجع | JADX دون واجهة رسومية وJDK كامل على Linux/macOS/Windows x64؛ [دليل Android](docs/android-analysis.md) | | أجهزة Android | الأجهزة والحزم والعمليات والسجلات والدامبات ولقطات الشاشة ونقل APK/الملفات | adb يوفّره المُستدعي (محاكي أو جهاز)؛ [دليل ADB](docs/adb-device-analysis.md) | diff --git a/README_de.md b/README_de.md index 1e96cd46b..bf456b12d 100644 --- a/README_de.md +++ b/README_de.md @@ -139,6 +139,7 @@ REA benötigt Node.js 22.x (>=22.19), 24.x (>=24.11) oder 26+ sowie npm. Zusätz | JavaScript / Electron | Module, Imports, Source Maps, Routen, IPC und Beziehungen zu nativen Add-ons | Node.js und npm; [Anwendungsanalyse](https://rea.tools/de/guides/javascript/) | | Websites | Seitenstruktur, Skripte, Netzwerkbeobachtungen und angeforderte Screenshots | Ein Browser aus der Chrome-Familie; [Browseranalyse](https://rea.tools/de/guides/browser/) | | Gespeicherte Netzwerkaufzeichnungen | Anfragen, Antworten, zugängliche Nutzdaten und Quellpositionen | HAR; mitmdump unter Linux für Aufzeichnungen im nativen mitmproxy-Format; [Leitfaden zu Netzwerkaufzeichnungen](docs/web-network-captures.md) | +| Paketerfassungen | Endpunkte, DNS, TLS SNI, Protokollverteilung und HTTP-Anfragen | Vom Aufrufer bereitgestellte tshark- und capinfos-Binaries; [pcap-Leitfaden](docs/pcap-analysis.md) | | .NET-Assemblies | Metadaten, CIL-Anweisungen, deklarierte native Abhängigkeiten und Build-Vergleiche | Statische Untersuchung; [Leitfaden für verwalteten Code](docs/managed-code-analysis.md) | | Android-APKs | Manifest-Deklarationen, Klassen, dekompilierte Methoden und Verweise | JADX ohne grafische Oberfläche und ein vollständiges JDK unter Linux/macOS/Windows x64; [Android-Leitfaden](docs/android-analysis.md) | | Android-Geräte | Geräte, Pakete, Prozesse, Logs, Dumps, Bildschirmaufnahmen sowie APK- und Dateiübertragungen | Vom Aufrufer bereitgestelltes adb (Emulator oder Gerät); [ADB-Leitfaden](docs/adb-device-analysis.md) | diff --git a/README_es.md b/README_es.md index 9a59a74d2..db20cd717 100644 --- a/README_es.md +++ b/README_es.md @@ -139,6 +139,7 @@ REA requiere Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, además de npm. Las h | JavaScript / Electron | Módulos, importaciones, mapas de código fuente, rutas, IPC y relaciones con complementos nativos | Node.js y npm; [análisis de aplicaciones](https://rea.tools/es/guides/javascript/) | | Sitios web | Estructura de la página, scripts, observaciones de red y capturas de pantalla solicitadas | Un navegador de la familia Chrome; [análisis del navegador](https://rea.tools/es/guides/browser/) | | Capturas de red guardadas | Solicitudes, respuestas, contenido accesible de las cargas útiles y ubicaciones de origen | HAR; mitmdump en Linux para capturas en formato nativo de mitmproxy; [guía de capturas](docs/web-network-captures.md) | +| Capturas de paquetes | Endpoints, DNS, TLS SNI, distribución de protocolos y solicitudes HTTP | tshark y capinfos suministrados por el autor de la llamada; [guía pcap](docs/pcap-analysis.md) | | Ensamblados .NET | Metadatos, instrucciones CIL, dependencias nativas declaradas y comparaciones de compilaciones | Inspección estática; [guía de código administrado](docs/managed-code-analysis.md) | | APK de Android | Declaraciones del manifiesto, clases, métodos descompilados y referencias | JADX sin interfaz gráfica y un JDK completo en Linux/macOS/Windows x64; [guía de Android](docs/android-analysis.md) | | Dispositivos Android | Dispositivos, paquetes, procesos, registros, volcados, capturas de pantalla y transferencias de APK/archivos | adb suministrado por el autor de la llamada (emulador o dispositivo); [guía de ADB](docs/adb-device-analysis.md) | diff --git a/README_fa.md b/README_fa.md index e34788e7a..f2c358c63 100644 --- a/README_fa.md +++ b/README_fa.md @@ -139,6 +139,7 @@ REA به Node.js 22.x (>=22.19)، 24.x (>=24.11) یا 26+، به‌همراه np | برنامه‌های فلاتر | هش اسنپ‌شات دارت، هش کتابخانه‌ها به تفکیک ABI و اصالت بیلد موتور | هیچ (تحلیل ایستا APK)؛ [راهنمای فلاتر](docs/flutter-analysis.md) | | Firmware | ناحیه‌ها، نتایج استخراج و تحویل به ابزارهای تحلیل بومی | Binwalk / Unblob در لینوکس؛ [راهنمای Firmware](docs/firmware-analysis.md) | | بسته‌های ضبط‌شده | فیلدهای منتخب، مقادیر تکراری، فیلترها و قواعد رمزگشایی | Linux: TShark 4.x + prlimit; [راهنمای ضبط بسته](docs/packet-captures.md) | +| ضبط بسته‌ها | نقاط پایانی، DNS، TLS SNI، توزیع پروتکل و درخواست‌های HTTP | tshark و capinfos تأمین‌شده توسط فراخوان‌کننده؛ [راهنمای pcap](docs/pcap-analysis.md) | | سیگنال‌های منطقی ضبط‌شده | توضیحات رمزگشا، بازه‌های نمونه و اطلاعات عیب‌یابی | Linux: sigrok-cli 0.7.2 + prlimit; [راهنمای سیگنال‌ها](docs/signal-recordings.md) | | بسته‌ها و منابع | فهرست فایل‌ها، digestها، فایل‌های plist، ساختار bundle اپل و منابع استخراج‌شده | [راهنمای آرتیفکت و JavaScript](docs/javascript-artifact-reconstruction.md)، [برنامه‌های اپل](docs/apple-application-analysis.md) | | رفتار فرایند | خروجی ترمینال، تعاملات، وضعیت خروج، مشاهدات سیستم فایل و مقایسهٔ اجراها | Linux/macOS با PTY بومی؛ [ضبط فرایند](docs/process-capture.md) | diff --git a/README_fr.md b/README_fr.md index f28e53f1a..41281ce5c 100644 --- a/README_fr.md +++ b/README_fr.md @@ -143,6 +143,7 @@ REA nécessite Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, ainsi que npm. Les | JavaScript / Electron | Modules, imports, source maps, routes, IPC et relations avec les modules natifs | Node.js et npm ; [analyse d'applications](https://rea.tools/fr/guides/javascript/) | | Sites web | Structure des pages, scripts, observations réseau et captures d'écran demandées | Un navigateur de la famille Chrome ; [analyse du navigateur](https://rea.tools/fr/guides/browser/) | | Captures réseau enregistrées | Requêtes, réponses, payloads exposés et emplacements dans le code source | HAR ; mitmdump sous Linux pour les captures mitmproxy natives ; [guide des captures](docs/web-network-captures.md) | +| Captures de paquets | Points de terminaison, DNS, TLS SNI, distribution des protocoles et requêtes HTTP | tshark et capinfos fournis par l'appelant ; [guide pcap](docs/pcap-analysis.md) | | Assemblys .NET | Métadonnées, instructions CIL, dépendances natives déclarées et comparaisons de builds | Inspection statique ; [guide du code managé](docs/managed-code-analysis.md) | | APK Android | Déclarations du manifeste, classes, méthodes décompilées et références | JADX headless et un JDK complet sous Linux/macOS/Windows x64 ; [guide Android](docs/android-analysis.md) | | Appareils Android | Appareils, paquets, processus, journaux, dumps, captures d'écran et transferts d'APK/fichiers | adb fourni par l'appelant (émulateur ou appareil) ; [guide ADB](docs/adb-device-analysis.md) | diff --git a/README_id.md b/README_id.md index 440a8f81b..e5e3dd1a4 100644 --- a/README_id.md +++ b/README_id.md @@ -146,6 +146,7 @@ Alat tambahan dan dukungan sistem host bergantung pada target yang dianalisis: | JavaScript / Electron | Modul, impor, source map, rute, IPC, dan hubungan add-on native | Node.js dan npm; [analisis aplikasi](https://rea.tools/id/guides/javascript/) | | Situs web | Struktur halaman, skrip, pengamatan jaringan, dan tangkapan layar yang diminta | Peramban berbasis Chrome; [analisis peramban](https://rea.tools/id/guides/browser/) | | Rekaman jaringan tersimpan | Permintaan, respons, payload yang terekspos, dan lokasi sumber | HAR; mitmdump di Linux untuk rekaman native mitmproxy; [panduan rekaman](docs/web-network-captures.md) | +| Capturan paket | Endpoint, DNS, TLS SNI, distribusi protokol, dan permintaan HTTP | tshark dan capinfos buatan pemanggil; [panduan pcap](docs/pcap-analysis.md) | | Assembly .NET | Metadata, instruksi CIL, dependensi native yang dideklarasikan, dan perbandingan build | Tidak ada (hanya analisis statis); [panduan kode terkelola](docs/managed-code-analysis.md) | | APK Android | Deklarasi manifest, kelas, metode hasil dekompilasi, dan referensi | JADX mode headless dan JDK lengkap di Linux/macOS/Windows x64; [panduan Android](docs/android-analysis.md) | | Perangkat Android | Perangkat, paket, proses, log, dump, tangkapan layar, serta transfer APK/berkas | adb buatan pemanggil (emulator atau perangkat); [panduan ADB](docs/adb-device-analysis.md) | diff --git a/README_it.md b/README_it.md index 24c1d35bc..409799e32 100644 --- a/README_it.md +++ b/README_it.md @@ -143,6 +143,7 @@ REA richiede Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, oltre a npm. Strument | JavaScript / Electron | Moduli, import, source map, route, IPC e relazioni con add-on nativi | Node.js e npm; [analisi delle applicazioni](https://rea.tools/it/guides/javascript/) | | Siti web | Struttura della pagina, script, osservazioni di rete e screenshot richiesti | Un browser della famiglia Chrome; [analisi browser](https://rea.tools/it/guides/browser/) | | Acquisizioni di rete salvate | Richieste, risposte, payload esposti e posizioni sorgente | HAR; mitmdump su Linux per acquisizioni mitmproxy native; [guida alle acquisizioni](docs/web-network-captures.md) | +| Captured di pacchetti | Endpoint, DNS, TLS SNI, distribuzione dei protocolli e richieste HTTP | tshark e capinfos forniti dal chiamante; [guida pcap](docs/pcap-analysis.md) | | Assembly .NET | Metadati, istruzioni CIL, dipendenze native dichiarate e confronti di build | Nessuno (solo statico); [guida managed-code](docs/managed-code-analysis.md) | | APK Android | Dichiarazioni manifest, classi, metodi decompilati e riferimenti | JADX headless e un JDK completo su Linux/macOS/Windows x64; [guida Android](docs/android-analysis.md) | | Dispositivi Android | Dispositivi, pacchetti, processi, log, dump, screenshot e trasferimenti APK/file | adb fornito dal chiamante (emulatore o hardware); [guida ADB](docs/adb-device-analysis.md) | diff --git a/README_ja.md b/README_ja.md index 8b32338aa..ee0875bd4 100644 --- a/README_ja.md +++ b/README_ja.md @@ -139,6 +139,7 @@ REA には Node.js 22.x（>=22.19）、24.x（>=24.11）、または 26+ と npm | JavaScript / Electron | モジュール、インポート、ソースマップ、ルート、IPC、ネイティブアドオンの関係 | Node.js と npm；[アプリケーション分析](https://rea.tools/ja/guides/javascript/) | | ウェブサイト | ページ構造、スクリプト、ネットワークの観測結果、依頼されたスクリーンショット | Chrome 系ブラウザー；[ブラウザー分析](https://rea.tools/ja/guides/browser/) | | 保存済みのネットワークキャプチャ | リクエスト、レスポンス、取得可能なペイロード、ソース内の位置 | HAR；mitmproxy ネイティブ形式のキャプチャには Linux 上の mitmdump；[キャプチャガイド](docs/web-network-captures.md) | +| パケットキャプチャ | エンドポイント、DNS、TLS SNI、プロトコル分布、HTTPリクエスト | 呼び出し側が用意した tshark と capinfos；[pcapガイド](docs/pcap-analysis.md) | | .NET アセンブリ | メタデータ、CIL 命令、宣言されたネイティブ依存関係、ビルドの比較 | 静的な検査；[マネージドコードのガイド](docs/managed-code-analysis.md) | | Android APK | マニフェストの宣言、クラス、逆コンパイルされたメソッド、参照 | Linux/macOS/Windows x64 上のヘッドレス JADX と完全な JDK；[Android ガイド](docs/android-analysis.md) | | Android デバイス | デバイス、パッケージ、プロセス、ログ、ダンプ、スクリーンキャプチャ、APK/ファイル転送 | 呼び出し側が用意した adb（エミュレーターまたは実機）；[ADB ガイド](docs/adb-device-analysis.md) | diff --git a/README_ko.md b/README_ko.md index 656753d7b..39814030c 100644 --- a/README_ko.md +++ b/README_ko.md @@ -139,6 +139,7 @@ REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니 | JavaScript / Electron | 모듈, 임포트, 소스 맵, 라우트, IPC, 네이티브 애드온 관계 | Node.js와 npm; [애플리케이션 분석](https://rea.tools/ko/guides/javascript/) | | 웹사이트 | 페이지 구조, 스크립트, 네트워크 관찰 결과, 요청한 스크린샷 | Chrome 계열 브라우저; [브라우저 분석](https://rea.tools/ko/guides/browser/) | | 저장된 네트워크 캡처 | 요청, 응답, 접근 가능한 페이로드, 소스 위치 | HAR; mitmproxy 네이티브 캡처에는 Linux의 mitmdump; [캡처 가이드](docs/web-network-captures.md) | +| 패킷 캡처 | 엔드포인트, DNS, TLS SNI, 프로토콜 분포, HTTP 요청 | 호출자 제공 tshark 및 capinfos; [pcap 가이드](docs/pcap-analysis.md) | | .NET 어셈블리 | 메타데이터, CIL 명령, 선언된 네이티브 의존성, 빌드 비교 | 정적 검사; [관리 코드 가이드](docs/managed-code-analysis.md) | | Android APK | 매니페스트 선언, 클래스, 디컴파일된 메서드, 참조 | Linux/macOS/Windows x64의 헤드리스 JADX와 전체 JDK; [Android 가이드](docs/android-analysis.md) | | Android 기기 | 기기, 패키지, 프로세스, 로그, 덤프, 화면 캡처, APK/파일 전송 | 호출자 제공 adb(에뮬레이터 또는 실제 기기); [ADB 가이드](docs/adb-device-analysis.md) | diff --git a/README_pl.md b/README_pl.md index f29d5d2fe..31ed33869 100644 --- a/README_pl.md +++ b/README_pl.md @@ -139,6 +139,7 @@ REA wymaga Node.js 22.x (>=22.19), 24.x (>=24.11) lub 26+ oraz npm. Dodatkowe na | JavaScript / Electron | Moduły, importy, mapy źródeł, trasy, IPC i powiązania z natywnymi dodatkami | Node.js i npm; [analiza aplikacji](https://rea.tools/pl/guides/javascript/) | | Strony internetowe | Struktura strony, skrypty, obserwacje sieciowe i zamówione zrzuty ekranu | Przeglądarka z rodziny Chrome; [analiza w przeglądarce](https://rea.tools/pl/guides/browser/) | | Zapisane przechwycenia ruchu sieciowego | Żądania, odpowiedzi, dostępne dane przesyłane i lokalizacje źródłowe | HAR; mitmdump w systemie Linux dla natywnych przechwyceń mitmproxy; [przewodnik przechwytywania](docs/web-network-captures.md) | +| Przechwytywanie pakietów | Punkty końcowe, DNS, TLS SNI, dystrybucja protokołów i żądania HTTP | Dostarczone przez wywołującego tshark i capinfos; [przewodnik pcap](docs/pcap-analysis.md) | | Zestawy .NET | Metadane, instrukcje CIL, zadeklarowane zależności natywne i porównania kompilacji | Analiza statyczna; [przewodnik kodu zarządzanego](docs/managed-code-analysis.md) | | Pakiety APK dla Androida | Deklaracje manifestu, klasy, zdekompilowane metody i odwołania | JADX bez interfejsu graficznego i pełny JDK w systemach Linux/macOS/Windows x64; [przewodnik Androida](docs/android-analysis.md) | | Urządzenia z Androidem | Urządzenia, pakiety, procesy, logi, zrzuty, zrzuty ekranu oraz transfery APK/plików | adb dostarczony przez wywołującego (emulator lub urządzenie); [przewodnik ADB](docs/adb-device-analysis.md) | diff --git a/README_pt-BR.md b/README_pt-BR.md index ec18da8ff..cba8b9c3d 100644 --- a/README_pt-BR.md +++ b/README_pt-BR.md @@ -139,6 +139,7 @@ O REA requer Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, além do npm. As fer | JavaScript / Electron | Módulos, importações, source maps, rotas, IPC e relações com complementos nativos | Node.js e npm; [análise de aplicativos](https://rea.tools/pt-BR/guides/javascript/) | | Sites | Estrutura da página, scripts, observações de rede e capturas de tela solicitadas | Um navegador da família Chrome; [análise do navegador](https://rea.tools/pt-BR/guides/browser/) | | Capturas de rede salvas | Requisições, respostas, payloads acessíveis e localizações de origem | HAR; mitmdump no Linux para capturas no formato nativo do mitmproxy; [guia de capturas](docs/web-network-captures.md) | +| Capturas de pacotes | Endpoints, DNS, TLS SNI, distribuição de protocolos e requisições HTTP | tshark e capinfos fornecidos pelo chamador; [guia pcap](docs/pcap-analysis.md) | | Assemblies .NET | Metadados, instruções CIL, dependências nativas declaradas e comparações de builds | Inspeção estática; [guia de código gerenciado](docs/managed-code-analysis.md) | | APKs Android | Declarações do manifesto, classes, métodos descompilados e referências | JADX sem interface gráfica e um JDK completo no Linux/macOS/Windows x64; [guia de Android](docs/android-analysis.md) | | Dispositivos Android | Dispositivos, pacotes, processos, logs, despejos, capturas de tela e transferências de APK/arquivos | adb fornecido pelo chamador (emulador ou aparelho); [guia de ADB](docs/adb-device-analysis.md) | diff --git a/README_ru.md b/README_ru.md index 2d75af75e..320574bf6 100644 --- a/README_ru.md +++ b/README_ru.md @@ -147,6 +147,7 @@ REA требует Node.js 22.x (>=22.19), 24.x (>=24.11) или 26+, а так | Приложения Flutter | Хеш снапшота Dart, дайджесты библиотек по ABI и происхождение сборки движка | Не требуется (статический разбор APK); [руководство по Flutter](docs/flutter-analysis.md) | | Прошивки | Области, результаты извлечения и передача в нативный анализ | Binwalk / Unblob в Linux; [руководство по прошивкам](docs/firmware-analysis.md) | | Записанные пакеты | Выбранные поля, повторяющиеся значения, фильтры и правила декодирования | Linux: TShark 4.x + prlimit; [руководство по захватам](docs/packet-captures.md) | +| Захват пакетов | Конечные точки, DNS, TLS SNI, распределение протоколов и HTTP-запросы | Предоставленные вызывающей стороной tshark и capinfos; [руководство по pcap](docs/pcap-analysis.md) | | Записанные логические сигналы | Аннотации декодера, диапазоны отсчётов и диагностика | Linux: sigrok-cli 0.7.2 + prlimit; [руководство по сигналам](docs/signal-recordings.md) | | Пакеты и ресурсы | Списки файлов, хеши, plist, структура пакетов Apple и извлечённые ресурсы | [Руководство по артефактам и JavaScript](docs/javascript-artifact-reconstruction.md), [приложения Apple](docs/apple-application-analysis.md) | | Поведение процессов | Вывод терминала, взаимодействия, наблюдения за завершением и файловой системой, сравнения запусков | Linux/macOS с нативным PTY; [захват процессов](docs/process-capture.md) | diff --git a/README_th.md b/README_th.md index 9e0c6831a..f67b04e89 100644 --- a/README_th.md +++ b/README_th.md @@ -139,6 +139,7 @@ REA ต้องใช้ Node.js 22.x (>=22.19), 24.x (>=24.11) หรือ 26 | JavaScript / Electron | โมดูล การนำเข้า ซอร์สแมป เส้นทาง IPC และความสัมพันธ์กับส่วนเสริมเนทีฟ | Node.js และ npm; [การวิเคราะห์แอปพลิเคชัน](https://rea.tools/th/guides/javascript/) | | เว็บไซต์ | โครงสร้างหน้า สคริปต์ ข้อมูลเครือข่ายที่สังเกตได้ และภาพหน้าจอที่ร้องขอ | เบราว์เซอร์ในตระกูล Chrome; [การวิเคราะห์ผ่านเบราว์เซอร์](https://rea.tools/th/guides/browser/) | | ข้อมูลเครือข่ายที่บันทึกไว้ | คำขอ คำตอบ เพย์โหลดที่ปรากฏ และตำแหน่งต้นทาง | HAR; mitmdump บน Linux สำหรับข้อมูลที่บันทึกในรูปแบบเนทีฟของ mitmproxy; [คู่มือการบันทึกข้อมูล](docs/web-network-captures.md) | +| การจับแพ็กเก็ต | จุดเชื่อมต่อ, DNS, TLS SNI, การกระจายโปรโตคอล และคำขอ HTTP | tshark และ capinfos ที่ผู้เรียกจัดเตรียม; [คู่มือ pcap](docs/pcap-analysis.md) | | แอสเซมบลี .NET | เมทาดาทา คำสั่ง CIL การพึ่งพาเนทีฟที่ประกาศไว้ และการเปรียบเทียบผลการบิลด์ | การตรวจสอบแบบสถิต; [คู่มือโค้ดแบบจัดการ](docs/managed-code-analysis.md) | | APK ของ Android | การประกาศใน manifest คลาส เมธอดที่ดีคอมไพล์ และการอ้างอิง | JADX แบบไม่มี GUI และ JDK แบบเต็มบน Linux/macOS/Windows x64; [คู่มือ Android](docs/android-analysis.md) | | อุปกรณ์ Android | อุปกรณ์ แพ็กเกจ โพรเซส บันทึก ดัมพ์ ภาพหน้าจอ และการถ่ายโอน APK/ไฟล์ | adb ที่ผู้เรียกจัดเตรียม (โปรแกรมจำลองหรืออุปกรณ์จริง); [คู่มือ ADB](docs/adb-device-analysis.md) | diff --git a/README_tr.md b/README_tr.md index 3c6e3f3b6..ccf7db7ce 100644 --- a/README_tr.md +++ b/README_tr.md @@ -139,6 +139,7 @@ REA; Node.js 22.x (>=22.19), 24.x (>=24.11) veya 26+ ve npm gerektirir. Ek araç | JavaScript / Electron | Modüller, içe aktarmalar, kaynak haritaları, rotalar, IPC ve yerel eklenti ilişkileri | Node.js ve npm; [uygulama analizi](https://rea.tools/tr/guides/javascript/) | | Web siteleri | Sayfa yapısı, betikler, ağ gözlemleri ve istenen ekran görüntüleri | Chrome ailesinden bir tarayıcı; [tarayıcı analizi](https://rea.tools/tr/guides/browser/) | | Kaydedilmiş ağ yakalamaları | İstekler, yanıtlar, erişilebilir yük içerikleri ve kaynak konumları | HAR; mitmproxy'nin kendi biçimindeki yakalamalar için Linux üzerinde mitmdump; [ağ yakalama rehberi](docs/web-network-captures.md) | +| Paket yakalamaları | Uç noktalar, DNS, TLS SNI, protokol dağılımı ve HTTP istekleri | Çağıranın sağladığı tshark ve capinfos; [pcap rehberi](docs/pcap-analysis.md) | | .NET derlemeleri | Meta veriler, CIL talimatları, bildirilmiş yerel bağımlılıklar ve derleme karşılaştırmaları | Statik inceleme; [yönetilen kod rehberi](docs/managed-code-analysis.md) | | Android APK'leri | Manifest bildirimleri, sınıflar, tersine derlenmiş metotlar ve referanslar | Linux/macOS/Windows x64 üzerinde arayüzsüz JADX ve tam bir JDK; [Android rehberi](docs/android-analysis.md) | | Android cihazları | Cihazlar, paketler, süreçler, günlükler, dökümler, ekran görüntüleri ve APK/dosya aktarımları | Çağıranın sağladığı adb (emülatör veya cihaz); [ADB rehberi](docs/adb-device-analysis.md) | diff --git a/README_uk.md b/README_uk.md index aea416718..78f1af9f4 100644 --- a/README_uk.md +++ b/README_uk.md @@ -139,6 +139,7 @@ REA потребує Node.js 22.x (>=22.19), 24.x (>=24.11) або 26+, а та | JavaScript / Electron | Модулі, імпорти, карти вихідного коду, маршрути, IPC і зв'язки з нативними доповненнями | Node.js і npm; [аналіз застосунків](https://rea.tools/uk/guides/javascript/) | | Сайти | Структура сторінки, скрипти, мережеві спостереження й запитані знімки екрана | Браузер сімейства Chrome; [аналіз браузера](https://rea.tools/uk/guides/browser/) | | Збережені мережеві захоплення | Запити, відповіді, доступні корисні дані та місця в джерелі | HAR; mitmdump у Linux для захоплень у власному форматі mitmproxy; [посібник мережевих захоплень](docs/web-network-captures.md) | +| Захоплення пакетів | Кінцеві точки, DNS, TLS SNI, розподіл протоколів та HTTP-запити | Надані викликаючою стороною tshark та capinfos; [підручник pcap](docs/pcap-analysis.md) | | Збірки .NET | Метадані, інструкції CIL, оголошені нативні залежності й порівняння збірок | Статична перевірка; [посібник керованого коду](docs/managed-code-analysis.md) | | Android APK | Оголошення маніфесту, класи, декомпільовані методи й посилання | JADX без графічного інтерфейсу й повний JDK у Linux/macOS/Windows x64; [посібник Android](docs/android-analysis.md) | | Пристрої Android | Пристрої, пакети, процеси, журнали, дампи, знімки екрана та передавання APK/файлів | adb, наданий викликаючою стороною (емулятор або пристрій); [підручник ADB](docs/adb-device-analysis.md) | diff --git a/README_vi.md b/README_vi.md index ab3fd493d..55aa4dd18 100644 --- a/README_vi.md +++ b/README_vi.md @@ -139,6 +139,7 @@ REA yêu cầu Node.js 22.x (>=22.19), 24.x (>=24.11) hoặc 26+, cùng với np | JavaScript / Electron | Mô-đun, các khai báo import, source map, tuyến, IPC và quan hệ với tiện ích mở rộng mã máy | Node.js và npm; [phân tích ứng dụng](https://rea.tools/vi/guides/javascript/) | | Trang web | Cấu trúc trang, script, quan sát mạng và ảnh chụp màn hình được yêu cầu | Trình duyệt thuộc họ Chrome; [phân tích trình duyệt](https://rea.tools/vi/guides/browser/) | | Bản ghi lưu lượng mạng đã lưu | Yêu cầu, phản hồi, payload có thể truy cập và vị trí nguồn | HAR; mitmdump trên Linux cho bản ghi định dạng gốc của mitmproxy; [hướng dẫn bản ghi mạng](docs/web-network-captures.md) | +| Bắt gói tin | Điểm cuối, DNS, TLS SNI, phân bố giao thức và yêu cầu HTTP | tshark và capinfos do nơi gọi cung cấp; [hướng dẫn pcap](docs/pcap-analysis.md) | | Assembly .NET | Siêu dữ liệu, lệnh CIL, các phụ thuộc mã máy được khai báo và so sánh bản dựng | Kiểm tra tĩnh; [hướng dẫn mã được quản lý](docs/managed-code-analysis.md) | | APK Android | Khai báo manifest, lớp, phương thức được dịch ngược và tham chiếu | JADX không giao diện đồ họa và JDK đầy đủ trên Linux/macOS/Windows x64; [hướng dẫn Android](docs/android-analysis.md) | | Thiết bị Android | Thiết bị, gói, tiến trình, nhật ký, dump, ảnh chụp màn hình và truyền APK/tệp | adb do nơi gọi cung cấp (trình giả lập hoặc thiết bị); [hướng dẫn ADB](docs/adb-device-analysis.md) | diff --git a/README_zh-TW.md b/README_zh-TW.md index 81371e91d..061cc9a09 100644 --- a/README_zh-TW.md +++ b/README_zh-TW.md @@ -139,6 +139,7 @@ REA 需要 Node.js 22.x（>=22.19）、24.x（>=24.11）或 26+，以及 npm。 | JavaScript / Electron | 模組、匯入、source map、路由、IPC 和原生附加元件關係 | Node.js 和 npm；[應用分析](https://rea.tools/zh-TW/guides/javascript/) | | 網站 | 頁面結構、腳本、網路觀察結果和按請求獲取的螢幕擷取畫面 | Chrome 系瀏覽器；[瀏覽器分析](https://rea.tools/zh-TW/guides/browser/) | | 已保存的網路擷取 | 請求、回應、可訪問的酬載和來源位置 | HAR；原生 mitmproxy 擷取需要 Linux 上的 mitmdump；[擷取指南](docs/web-network-captures.md) | +| 封包擷取 | 端點、DNS、TLS SNI、通訊協定分佈和 HTTP 請求 | 呼叫端提供的 tshark 和 capinfos；[pcap 指南](docs/pcap-analysis.md) | | .NET 程式集 | 中繼資料、CIL 指令、聲明的原生依賴和建置對比 | 靜態檢查；[托管程式碼指南](docs/managed-code-analysis.md) | | Android APK | 清單聲明、類別、反編譯的方法和引用 | Linux/macOS/Windows x64 上的無界面 JADX 和完整 JDK；[Android 指南](docs/android-analysis.md) | | Android 裝置 | 裝置、套件、程序、日誌、傾印、螢幕擷取以及 APK/檔案傳輸 | 呼叫端提供的 adb（模擬器或實體裝置）；[ADB 指南](docs/adb-device-analysis.md) | diff --git a/README_zh.md b/README_zh.md index d802cad50..be60efeec 100644 --- a/README_zh.md +++ b/README_zh.md @@ -139,6 +139,7 @@ REA 需要 Node.js 22.x（>=22.19）、24.x（>=24.11）或 26+，以及 npm。 | JavaScript / Electron | 模块、导入、source map、路由、IPC 和原生扩展关系 | Node.js 和 npm；[应用分析](https://rea.tools/zh-CN/guides/javascript/) | | 网站 | 页面结构、脚本、网络观察结果和按请求获取的截图 | Chrome 系浏览器；[浏览器分析](https://rea.tools/zh-CN/guides/browser/) | | 已保存的网络捕获 | 请求、响应、可访问的载荷和来源位置 | HAR；原生 mitmproxy 捕获需要 Linux 上的 mitmdump；[捕获指南](docs/web-network-captures.md) | +| 数据包捕获 | 端点、DNS、TLS SNI、协议分布和 HTTP 请求 | 调用方提供的 tshark 和 capinfos；[pcap 指南](docs/pcap-analysis.md) | | .NET 程序集 | 元数据、CIL 指令、声明的原生依赖和构建对比 | 静态检查；[托管代码指南](docs/managed-code-analysis.md) | | Android APK | 清单声明、类、反编译的方法和引用 | Linux/macOS/Windows x64 上的无界面 JADX 和完整 JDK；[Android 指南](docs/android-analysis.md) | | Android 设备 | 设备、软件包、进程、日志、转储、屏幕截图以及 APK/文件传输 | 调用方提供的 adb（模拟器或实体设备）；[ADB 指南](docs/adb-device-analysis.md) | diff --git a/docs/pcap-analysis.md b/docs/pcap-analysis.md new file mode 100644 index 000000000..1df3e5ccd --- /dev/null +++ b/docs/pcap-analysis.md @@ -0,0 +1,118 @@ +# Packet capture summaries + +REA summarizes already saved pcap/pcapng files through caller-selected Wireshark +4.x `tshark` and `capinfos`. Use `inspect_packet_capture` for selected fields, +filters and decode-as rules; use this family for address, DNS, SNI, protocol-stack +and URI inventories. It does not acquire traffic or fetch recorded targets. + +## Supply tools + +```sh +export REA_TSHARK_COMMAND=/absolute/path/tshark +export REA_CAPINFOS_COMMAND=/absolute/path/capinfos +# Linux only; default /usr/bin/prlimit +export REA_TSHARK_PRLIMIT_COMMAND=/absolute/path/prlimit +rea inspect-pcap-client +rea inspect-pcap-capture /captures/session.pcapng --json +``` + +Without explicit tool commands, this family resolves them through PATH and +records that selection source. Client inspection reads no capture file. Capture +inspection requires Linux with util-linux prlimit or Windows with owned Job +Objects. macOS capture inspection returns a capability gap because the new +native memory boundary is unverified; its version probes remain available. +Tools and installed libraries/plugins are caller-trusted. An empty private +personal Wireshark configuration is used; this is not an OS sandbox. + +## Inspect only the needed facts + +MCP `inspect_pcap_capture` accepts `path` and optional `detail: "summary"`. +Complete delivery remains the default. Summary commits and retains the same +complete Evidence, returning identity, capture metadata, observed counts, +retained counts, source coverage and diagnostic lengths without large lists. +Then use its `normalized_result.parent_evidence_id` with +`inspect_analysis_view` source kind `retained-evidence`: + +```json +{"kind":"pcap","facet":"endpoints","offset":0,"limit":20} +{"kind":"pcap","facet":"dns-queries","offset":0,"limit":20} +{"kind":"pcap","facet":"dns-addresses","query":"example.com","offset":0,"limit":20} +{"kind":"pcap","facet":"tls-sni","offset":0,"limit":20} +{"kind":"pcap","facet":"diagnostics","stream":"tshark","offset":0,"limit":4096} +``` + +Other row facets are `protocols`, `http-uris` and `unattributed-addresses`. +Defaults are offset 0 and limit 64. DNS question pages give address counts; +select one exact retained question to page its addresses. Diagnostic windows +count UTF-16 code units; other windows count rows. Missing/ambiguous selectors +fail. Unknown legacy diagnostics remain unknown, not an observed empty stream. +Portable CLI views use the existing `inspect-analysis-view` workflow with inline +complete Evidence. Views never reread the source or launch Wireshark and work +after the original file is deleted. Page exhaustion covers the retained facet; +it does not establish network completeness or recover missing traffic. + +## Interpret provenance and coverage + +`target.sha256` binds observations to an admitted stable capture snapshot. Both +tools read that copy, which is hashed again after analysis. Capture metadata and +literal field values are producer observations; address/protocol counts and DNS +associations are derivations. Capture Evidence has historical-reference authority. + +Addresses count once per packet, including loopback and nested IP occurrences. +DNS addresses are associated only with one question in the same response packet; +additional records may be included, so this does not prove final resolution. +Ambiguous addresses remain in a separate inventory. Protocol distribution counts +exact `frame.protocols` stack strings. TLS SNI is recorded handshake metadata, +not process identity or proof of the destination's authenticity. + +All examined inventories are retained within the resource budget; no fixed +200/100/50-item truncation is applied. HTTP transport credentials are excluded +without rewriting other URI bytes; ambiguous credential spans cause omission. +Counts preserve distinct observed source URIs before omission or coalescing. +`coverage: "partial"` records unresolved DNS attribution or URI item loss; +component credential exclusion is also counted explicitly. Diagnostics retain +both tool stderr streams. Recorded fields do not establish decrypted payloads, +causality, process identity, absence of unrecorded traffic or current deployments. + +## Resources and lifecycle + +One inspection is admitted at a time per provider. Concurrent calls return a +capacity error; close drains active work before releasing resources. Captures +are streamed into a read-only owned snapshot with a 2 GiB disk budget. Native +capture commands enforce Linux address-space ceilings of at most 1 GiB, +300 CPU seconds, 64 MiB per file and no core dumps, preserving stricter inherited +limits. Windows uses a 1 GiB committed-memory limit for each owned tool tree. +These are not aggregate machine or filesystem quotas. + +TShark output is bounded at 8 MiB; its known five-level selected-field shape and +128 MiB graph-expansion estimate are checked before allocation. Retained facts +have a separate 16 MiB estimate. Capinfos output is limited to 64 KiB. Command +wall deadlines are 300 s for dissection, 60 s for metadata and 20 s for probes. +Native ownership preparation has its own bounded lifecycle. + +Malformed/oversized output returns no partial success. Unvalidated partial JSON +is not returned as facts. Cancellation reaches probes, acquisition, commands and +projection. Cleanup failure returns a typed error, retains completed Evidence +when available, and retains owned resources for retry on close. + +## Verification scope + +Current follow-up verification uses Linux/Node 24.18.0 and Wireshark 4.0.17. +Private real recorded corpora are checked against independent raw-frame/IP +parsing and Python aggregation, public CLI/stdio MCP parity, SDK schemas, source +hashes and retained views. Synthetic native tools separately test malformed +output, cancellation, source/snapshot mutation, capacity and cleanup; they do +not establish real Wireshark or other-platform behavior. + +The original proposal reported Linux 4.6.6, macOS/Windows 4.6.9 verification on +its earlier implementation. Those reports do not verify the new final memory +boundary. Actual Windows/Bun Wireshark execution is unverified here; macOS +capture support needs a separate enforceable-bound decision. + +```sh +npm run verify:pcap -- --pcap /absolute/saved-capture.pcapng +``` + +This verifier exercises built CLI/stdio MCP, schemas, artifact identity and +cleanup. Without `--pcap`, capture inspection is reported as unverified. Private +captures, keys and provider runtime files are not committed. diff --git a/docs/testing.md b/docs/testing.md index 6f0b80131..a36eff6ec 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -849,6 +849,14 @@ establish passing Android coverage; inspect its execution receipt. The `termux-browser-diagnostics` artifact retains the Termux command log, exit status, and Android logcat on success or failure. The emulator is disposable. +## Packet capture analysis + +`npm run verify:pcap -- --pcap PATH` exercises caller-supplied tshark and +capinfos against a saved capture. Linux verification uses Wireshark 4.6.6 +and captures containing DNS, TLS and HTTP traffic. Native Windows and macOS +engine behavior remains unverified; process fixtures do not establish it. +See [Packet capture analysis](pcap-analysis.md) for contracts and budgets. + ## Apple native metadata and UI `npm run verify:apple-dispatch` compiles the Objective-C fixture (classes, diff --git a/package.json b/package.json index bc5c0edc9..6fdb0d9de 100644 --- a/package.json +++ b/package.json @@ -167,6 +167,7 @@ "verify:legacy-simulator": "npm run build:cached && node scripts/verify/apple/legacy-simulator.mjs", "verify:architecture-guards": "node scripts/verify/architecture-guards.mjs", "verify:apktool": "npm run build:cached && node scripts/verify-apktool.mjs", + "verify:pcap": "npm run build:cached && node scripts/verify-pcap.mjs", "verify:flutter": "npm run build:cached && node scripts/verify-flutter.mjs", "verify:adb": "npm run build:cached && node scripts/verify-adb.mjs", "verify:ghidra": "npm run build:cached && node scripts/verify-real-ghidra.mjs", diff --git a/scripts/lib/catalog-builders.mjs b/scripts/lib/catalog-builders.mjs index f97f5ed5c..32ec51ee3 100644 --- a/scripts/lib/catalog-builders.mjs +++ b/scripts/lib/catalog-builders.mjs @@ -52,6 +52,11 @@ export const toolFamilyCatalog = (sources) => { surface: "android-provider", contracts: sources.androidContracts.ANDROID_TOOL_CONTRACTS, }, + { + id: "pcap", + surface: "pcap-provider", + contracts: sources.pcapContracts.PCAP_TOOL_CONTRACTS, + }, { id: "flutter", surface: "flutter-provider", @@ -251,6 +256,10 @@ export const providerCatalog = (sources) => { identity: sources.androidProvider.JADX_PROVIDER_IDENTITY, contracts: sources.androidContracts.ANDROID_TOOL_CONTRACTS, }, + { + identity: sources.pcapProvider.PCAP_PROVIDER_IDENTITY, + contracts: sources.pcapContracts.PCAP_TOOL_CONTRACTS, + }, { identity: sources.flutterProvider.FLUTTER_PROVIDER_IDENTITY, contracts: sources.flutterContracts.FLUTTER_TOOL_CONTRACTS, diff --git a/scripts/lib/catalog-core.mjs b/scripts/lib/catalog-core.mjs index 5a3b0e909..f5488a9e6 100644 --- a/scripts/lib/catalog-core.mjs +++ b/scripts/lib/catalog-core.mjs @@ -68,8 +68,8 @@ const SOURCE_PATHS = { flutterProvider: "dist/flutter/FlutterBuildProvider.js", androidProvider: "dist/android/JadxRelease.js", apktoolContracts: "dist/contracts/apktool/apktoolToolContracts.js", - flutterContracts: "dist/contracts/flutter/flutterToolContracts.js", - flutterProvider: "dist/flutter/FlutterBuildProvider.js", + pcapContracts: "dist/contracts/pcap/pcapToolContracts.js", + pcapProvider: "dist/pcap/PcapProvider.js", apktoolProvider: "dist/apktool/ApktoolProvider.js", jebContracts: "dist/contracts/jeb/jebToolContracts.js", jebProvider: "dist/jeb/JebProvider.js", diff --git a/scripts/verify-packet-capture.mjs b/scripts/verify-packet-capture.mjs index 371c72367..2b959e602 100644 --- a/scripts/verify-packet-capture.mjs +++ b/scripts/verify-packet-capture.mjs @@ -236,6 +236,52 @@ try { const hci = await inspect({ path: bluetooth, fields: ["bthci_cmd.opcode"] }); assert.deepEqual(hci.packets[0]["bthci_cmd.opcode"], ["0x0c03"]); assert.deepEqual(await readFile(path), source); + // Exercise the incremental summary workflow against the same owned bytes + // using real installed Wireshark tools; these inputs are format fixtures. + const pcapTool = (await client.listTools()).tools.find( + (t) => t.name === "inspect_pcap_capture", + ); + assert.ok(pcapTool?.outputSchema); + const pcapAjv = new Ajv2020({ strict: false, validateFormats: false }); + assert.ok(pcapAjv.validateSchema(pcapTool.inputSchema)); + assert.ok(pcapAjv.validateSchema(pcapTool.outputSchema)); + for (const capturePath of [path, bluetooth]) { + const response = await client.callTool({ + name: pcapTool.name, + arguments: { path: capturePath, detail: "summary" }, + }); + assert.notEqual(response.isError, true, JSON.stringify(response)); + assert.ok( + pcapAjv.validate(pcapTool.outputSchema, response.structuredContent), + pcapAjv.errorsText(), + ); + const summary = parseEvidence(response.structuredContent).normalized_result; + assert.equal( + summary.summary.target.sha256, + createHash("sha256") + .update(await readFile(capturePath)) + .digest("hex"), + ); + const source = { + kind: "retained-evidence", + evidence_id: summary.parent_evidence_id, + }; + const page = await client.callTool({ + name: "inspect_analysis_view", + arguments: { + source, + view: { kind: "pcap", facet: "protocols", limit: 10 }, + }, + }); + assert.notEqual(page.isError, true, JSON.stringify(page)); + assert.equal( + parseEvidence(page.structuredContent).normalized_result + .parent_evidence_id, + source.evidence_id, + ); + cases++; + } + assert.deepEqual(await readFile(bluetooth), bluetoothSource); for (const input of [ { path, fields: ["frame.number"], display_filter: "this is not a filter" }, diff --git a/scripts/verify-pcap.mjs b/scripts/verify-pcap.mjs new file mode 100644 index 000000000..3adf14531 --- /dev/null +++ b/scripts/verify-pcap.mjs @@ -0,0 +1,131 @@ +import assert from "node:assert/strict"; +import { execFile } from "node:child_process"; +import { mkdtemp, readdir, rmdir } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { promisify } from "node:util"; +import { Client } from "@modelcontextprotocol/client"; +import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; +import { toolContract } from "../dist/contracts/toolContracts.js"; +import { hashStableFile } from "../dist/filesystem/StableFileHash.js"; +import { mcpTextValue } from "./lib/mcp-verifier-results.mjs"; + +const args = process.argv.slice(2); +if (args.includes("--help") || args.includes("-h")) { + console.log( + "Usage: npm run verify:pcap -- [--pcap PATH]\nRequires caller-supplied tshark/capinfos. Without --pcap, only client readiness is verified.", + ); + process.exit(0); +} +if (!(args.length === 0 || (args.length === 2 && args[0] === "--pcap"))) + throw new Error("Expected --pcap PATH"); +const path = args[1] === undefined ? null : resolve(args[1]); +// This parent is shared by independent CLI/MCP owners. A leased Windows +// runtime root would prevent their native admission of its directory handle. +const root = await mkdtemp(join(tmpdir(), "rea-pcap-verifier-")); +const environment = Object.fromEntries( + Object.entries(process.env).filter(([, value]) => typeof value === "string"), +); +Object.assign(environment, { + TMPDIR: root, + TMP: root, + TEMP: root, +}); +const entrypoint = resolve("scripts/rea.mjs"); +const execute = promisify(execFile); +const client = new Client({ name: "pcap-real-verifier", version: "1" }); +const transport = new StdioClientTransport({ + command: process.execPath, + args: [entrypoint, "mcp"], + env: environment, + stderr: "pipe", +}); +transport.stderr?.resume(); +try { + await client.connect(transport); + const readiness = await run("inspect_pcap_client", {}); + assert(readiness.client.tshark_version && readiness.client.capinfos_version); + if (path === null) + console.log("[unverified] Capture inspection: no --pcap was supplied"); + else { + const identity = await hashStableFile(path); + const result = await run("inspect_pcap_capture", { path }); + assert.deepEqual(result.target, { path, ...identity }); + assert(result.packet_count >= 0); + assert( + result.endpoints.every( + (entry) => entry.packet_count <= result.packet_count, + ), + ); + assert( + result.protocol_distribution.every( + (entry) => entry.packet_count <= result.packet_count, + ), + ); + const partial = + result.endpoint_count > result.endpoints.length || + result.dns_query_count > result.dns_queries.length || + result.tls_sni_count > result.tls_sni_values.length || + result.protocol_count > result.protocol_distribution.length || + result.http_request_uri_count > result.http_request_uris.length || + result.unattributed_dns_packet_count > 0 || + result.dns_queries.some( + (entry) => + entry.response_address_count > entry.response_addresses.length, + ); + assert.equal(result.coverage, partial ? "partial" : "complete"); + assert.deepEqual( + await hashStableFile(path), + identity, + "Input capture must remain unchanged", + ); + console.log( + `[ok] ${result.packet_count} packets; ${result.endpoint_count} addresses; coverage=${result.coverage}`, + ); + } +} finally { + try { + await client.close(); + } finally { + await transport.close(); + } + assert.deepEqual( + await readdir(root), + [], + `Owned capture resources remain under ${root}`, + ); + await rmdir(root); +} +console.log("All asserted pcap CLI/MCP checks passed"); + +async function run(name, input) { + const contract = toolContract(name); + const { stdout } = await execute( + process.execPath, + [ + entrypoint, + name.replaceAll("_", "-"), + ...(input.path === undefined ? [] : [input.path]), + "--json", + ], + { env: environment, timeout: 360_000, maxBuffer: 8 * 1024 * 1024 }, + ); + const cli = contract.outputSchema.parse(JSON.parse(stdout)); + const response = await client.callTool( + { name, arguments: input }, + { timeout: 360_000 }, + ); + assert.notEqual(response.isError, true, mcpTextValue(response)); + const mcp = contract.outputSchema.parse(JSON.parse(mcpTextValue(response))); + assert.deepEqual(mcp.normalized_result, cli.normalized_result); + assert.deepEqual(mcp.subject, cli.subject); + if (input.path !== undefined) { + assert.equal(cli.subject.local_path, input.path); + assert.equal( + cli.subject.digest.sha256, + cli.normalized_result.target.sha256, + ); + } + console.log(`[ok] ${name}: named output schema and CLI/MCP Evidence parity`); + return cli.normalized_result; +} diff --git a/src/application/CapabilityInventory.ts b/src/application/CapabilityInventory.ts index c0f341506..34b418029 100644 --- a/src/application/CapabilityInventory.ts +++ b/src/application/CapabilityInventory.ts @@ -481,7 +481,7 @@ const workflowAvailabilityFor = ({ remediation: "Run process capture on a supported Linux or macOS host.", }; if (kind === "application") return { reason: "available", remediation: null }; - if (kind === "flutter-provider") + if (kind === "pcap-provider" || kind === "flutter-provider") return { reason: "available", remediation: null }; const electron = electronProviderAvailability(name, kind, policy); if (electron !== null) return electron; diff --git a/src/application/pcap/PcapAnalysisPort.ts b/src/application/pcap/PcapAnalysisPort.ts new file mode 100644 index 000000000..c5acecd38 --- /dev/null +++ b/src/application/pcap/PcapAnalysisPort.ts @@ -0,0 +1,23 @@ +import type { + AnalysisExecution, + ExecutionOptions, + ProviderAvailability, +} from "../AnalysisProvider.js"; +import type { PcapRequest } from "../../domain/pcap/pcapAnalysis.js"; +import type { AnalysisError } from "../../domain/analysisErrorBase.js"; +import type { Result } from "../../domain/result.js"; + +/** + * Packet-capture analysis boundary over caller-supplied tshark/capinfos. + * Implementation details belong to the provider adapter. + */ +export interface PcapAnalysisPort { + /** Probe both tools without reading any capture file. */ + inspectAvailability(signal?: AbortSignal): Promise; + /** Retry cleanup of retained owned commands and capture snapshots. */ + close(): Promise; + execute( + request: PcapRequest, + options?: ExecutionOptions, + ): Promise>; +} diff --git a/src/application/pcap/PcapAnalysisService.ts b/src/application/pcap/PcapAnalysisService.ts new file mode 100644 index 000000000..23b968e58 --- /dev/null +++ b/src/application/pcap/PcapAnalysisService.ts @@ -0,0 +1,36 @@ +import type { PcapAnalysisPort } from "./PcapAnalysisPort.js"; +import type { ExecutionOptions } from "../AnalysisProvider.js"; +import { + pcapRequestSchema, + type PcapOperation, +} from "../../domain/pcap/pcapAnalysis.js"; +import { + AnalysisCancelledError, + AnalysisInputError, +} from "../../domain/analysisErrorCore.js"; +import type { AnalysisError } from "../../domain/analysisErrorBase.js"; +import type { Evidence } from "../../domain/evidence.js"; +import { createPcapEvidence } from "../../domain/pcap/pcapEvidence.js"; +import { err, ok, type Result } from "../../domain/result.js"; + +/** Shared CLI/MCP admission, execution, and Evidence composition for pcap. */ +export class PcapAnalysisService { + constructor(readonly provider: PcapAnalysisPort) {} + + /** Run one caller-selected pcap request and retain capture identity inline. */ + async execute( + operation: PcapOperation, + input: unknown, + options?: ExecutionOptions, + ): Promise> { + if (options?.signal?.aborted === true) + return err(new AnalysisCancelledError(operation)); + const request = pcapRequestSchema.safeParse({ operation, input }); + if (!request.success) + return err(new AnalysisInputError(operation, { cause: request.error })); + const executed = await this.provider.execute(request.data, options); + if (!executed.ok) return executed; + const execution = executed.value; + return ok(createPcapEvidence(request.data, execution)); + } +} diff --git a/src/cli.ts b/src/cli.ts index 0b545c0ba..d1bdf2912 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -24,6 +24,7 @@ import { registerManagedCommands } from "./cli/managedCommands.js"; import { registerFirmwareCommands } from "./cli/firmwareCommands.js"; import { registerAndroidCommands } from "./cli/androidCommands.js"; import { registerApktoolCommands } from "./cli/apktoolCommands.js"; +import { registerPcapCommands } from "./cli/pcapCommands.js"; import { registerFlutterCommands } from "./cli/flutterCommands.js"; import { registerJebCommands } from "./cli/jebCommands.js"; import { registerAdbCommands } from "./cli/adbCommands.js"; @@ -105,6 +106,7 @@ export const createCli = ( registerManagedCommands(cli, logger, analysis.runProviderAnalysis); registerAndroidCommands(cli, logger, environment); registerApktoolCommands(cli, logger, environment); + registerPcapCommands(cli, logger, environment); registerFlutterCommands(cli, logger); registerJebCommands(cli, logger, environment); registerAdbCommands(cli, logger, environment); diff --git a/src/cli/pcapCommands.ts b/src/cli/pcapCommands.ts new file mode 100644 index 000000000..1b53e47cc --- /dev/null +++ b/src/cli/pcapCommands.ts @@ -0,0 +1,69 @@ +import { z } from "incur"; +import { AnalysisError } from "../domain/analysisErrorBase.js"; +import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js"; +import { PcapAnalysisService } from "../application/pcap/PcapAnalysisService.js"; +import type { PcapAnalysisPort } from "../application/pcap/PcapAnalysisPort.js"; +import { createPcapAnalysisProvider } from "../composition/pcap.js"; +import { CLI_COMMANDS } from "../cliCommandNames.js"; +import { logCliCommand } from "../cliLogging.js"; +import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; +import type { PcapOperation } from "../domain/pcap/pcapAnalysis.js"; +import type { Logger } from "pino"; +import type { CliInstance } from "./types.js"; +import { withCommandCancellation } from "./commandCancellation.js"; + +/** Expose the same explicit pcap requests through one-shot CLI commands. */ +export const registerPcapCommands = ( + cli: CliInstance, + logger: Logger, + environment: Readonly>, + provider: PcapAnalysisPort = createPcapAnalysisProvider(environment), +): void => { + const service = new PcapAnalysisService(provider); + const execute = (name: string, operation: PcapOperation, input: unknown) => + withCommandCancellation((signal) => + logCliCommand(logger, name, async () => { + const result = await service.execute(operation, input, { signal }); + try { + await provider.close(); + } catch (cause) { + if (cause instanceof AnalysisError) { + if (result.ok) cause.retainPartialObservation(result.value); + return projectAnalysisError( + result.ok + ? cause + : analysisErrorWithCleanupFailure( + result.error, + cause, + operation, + ), + ); + } + throw cause; + } + return result.ok ? result.value : projectAnalysisError(result.error); + }), + ); + + cli.command(CLI_COMMANDS.inspectPcapClient, { + description: + "Inspect the caller-selected tshark and capinfos binaries without reading a capture", + args: z.object({}), + run: () => + execute(CLI_COMMANDS.inspectPcapClient, "inspect_pcap_client", {}), + }); + cli.command(CLI_COMMANDS.inspectPcapCapture, { + description: + "Inspect one pcap/pcapng capture: endpoints, DNS, TLS SNI, and protocol distribution", + args: z.object({ + path: z + .string() + .min(1) + .describe("Local pcap or pcapng capture file to inspect"), + }), + run: ({ args }) => + execute(CLI_COMMANDS.inspectPcapCapture, "inspect_pcap_capture", { + path: args.path, + }), + }); +}; diff --git a/src/cliCommandNames.ts b/src/cliCommandNames.ts index 1278cbea8..e7c9bc922 100644 --- a/src/cliCommandNames.ts +++ b/src/cliCommandNames.ts @@ -55,6 +55,8 @@ export const CLI_COMMANDS = Object.freeze({ inspectFirmwareRegions: "inspect-firmware-regions", extractFirmware: "extract-firmware", inspectAndroidPackage: "inspect-android-package", + inspectPcapClient: "inspect-pcap-client", + inspectPcapCapture: "inspect-pcap-capture", identifyFlutterBuild: "identify-flutter-build", inspectDartAot: "inspect-dart-aot", inspectApktoolClient: "inspect-apktool-client", diff --git a/src/composition/pcap.ts b/src/composition/pcap.ts new file mode 100644 index 000000000..f3aaa3915 --- /dev/null +++ b/src/composition/pcap.ts @@ -0,0 +1,7 @@ +import type { PcapAnalysisPort } from "../application/pcap/PcapAnalysisPort.js"; +import { PcapProvider } from "../pcap/PcapProvider.js"; + +/** Construct one pcap provider bound to the caller-selected tools. */ +export const createPcapAnalysisProvider = ( + environment: Readonly> = process.env, +): PcapAnalysisPort => new PcapProvider({ environment }); diff --git a/src/contracts/analysisViewToolContracts.ts b/src/contracts/analysisViewToolContracts.ts index 1a4dbbb5c..aba5b9a93 100644 --- a/src/contracts/analysisViewToolContracts.ts +++ b/src/contracts/analysisViewToolContracts.ts @@ -16,7 +16,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [ ...toolContractMetadata("inspect_analysis_view"), kind: "application", description: - 'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_evm_interface, inspect_managed_artifact, inspect_android_package, inspect_dart_aot or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. For layout, JavaScript, inventory, WASM, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Dart AOT summary preserves target/ABI/library identity, snapshot headers and pool lower bounds. String pages use kind dart-aot, collection package_uris/dart_uris/class_like_tokens; source coverage remains partial when the producer says partial. Native function Evidence requires kind native: view: {kind: "native", facet: "procedure"}. Facets select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', + 'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_evm_interface, inspect_managed_artifact, inspect_android_package, inspect_dart_aot or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. For layout, JavaScript, inventory, WASM, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Dart AOT summary preserves target/ABI/library identity, snapshot headers and pool lower bounds. String pages use kind dart-aot, collection package_uris/dart_uris/class_like_tokens; source coverage remains partial when the producer says partial. Native function Evidence requires kind native: view: {kind: "native", facet: "procedure"}. Facets select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', inputSchema: inspectAnalysisViewInputSchema, outputSchema: evidenceResultOf(analysisViewResultSchema), examples: [ diff --git a/src/contracts/pcap/pcapToolContracts.ts b/src/contracts/pcap/pcapToolContracts.ts new file mode 100644 index 000000000..66f281b4e --- /dev/null +++ b/src/contracts/pcap/pcapToolContracts.ts @@ -0,0 +1,46 @@ +import { + pcapInputSchemas, + pcapResultSchemas, +} from "../../domain/pcap/pcapAnalysis.js"; +import type { ToolContract } from "../toolContractTypes.js"; +import { evidenceResultOf } from "../toolOutputSchemaPrimitives.js"; +import { toolContractMetadata } from "../toolEffects.js"; +import { + analysisDetailSchema, + analysisDetailOutputSchemaOf, +} from "../analysisDetail.js"; +import { pcapCaptureSummaryViewResultSchema } from "../../domain/analysisView/analysisView.js"; + +/** Packet-capture analysis served by caller-supplied tshark and capinfos. */ +export const PCAP_TOOL_CONTRACTS = [ + { + name: "inspect_pcap_client", + ...toolContractMetadata("inspect_pcap_client"), + description: + "Inspect the caller-selected tshark and capinfos binaries: selected commands, selection sources, and versions. Both tools must be usable. Reads no capture file. REA never installs the Wireshark suite; select binaries with REA_TSHARK_COMMAND and REA_CAPINFOS_COMMAND (absolute) or from PATH.", + kind: "pcap-provider", + inputSchema: pcapInputSchemas.inspect_pcap_client, + outputSchema: evidenceResultOf(pcapResultSchemas.inspect_pcap_client), + examples: [{ title: "Check the analysis tools", input: {} }], + }, + { + name: "inspect_pcap_capture", + ...toolContractMetadata("inspect_pcap_capture"), + description: + "Request detail summary for capture identity/counts, then page retained facts with inspect_analysis_view kind pcap. Default detail complete preserves inline delivery. Inspect one saved pcap or pcapng capture through an owned snapshot: metadata, exact packet count, SHA-256, IPv4/IPv6 address counts, DNS questions and address values from the same single-question response, TLS SNI, protocol distribution and HTTP URIs. All derived inventories are retained within declared resource budgets; unknown DNS attribution and excluded credentials remain explicit. Linux requires util-linux prlimit; Windows uses owned Job Objects. Requires caller-supplied capinfos and tshark; REA never captures traffic.", + kind: "pcap-provider", + inputSchema: pcapInputSchemas.inspect_pcap_capture.extend({ + detail: analysisDetailSchema, + }), + outputSchema: analysisDetailOutputSchemaOf( + pcapResultSchemas.inspect_pcap_capture, + pcapCaptureSummaryViewResultSchema, + ), + examples: [ + { + title: "Inspect a capture file", + input: { path: "/captures/session.pcap" }, + }, + ], + }, +] as const satisfies readonly ToolContract[]; diff --git a/src/contracts/toolContractTypes.ts b/src/contracts/toolContractTypes.ts index d7e19d37e..de5f53411 100644 --- a/src/contracts/toolContractTypes.ts +++ b/src/contracts/toolContractTypes.ts @@ -33,6 +33,7 @@ export const TOOL_KINDS = [ "browser-provider", "electron-provider", "runtime-provider", + "pcap-provider", "application", "session", ] as const; diff --git a/src/contracts/toolContracts.ts b/src/contracts/toolContracts.ts index 101cca17e..2fdac1fbe 100644 --- a/src/contracts/toolContracts.ts +++ b/src/contracts/toolContracts.ts @@ -12,6 +12,7 @@ import { MANAGED_TOOL_CONTRACTS } from "./managed/managedToolContracts.js"; import { FIRMWARE_TOOL_CONTRACTS } from "./firmware/firmwareToolContracts.js"; import { ANDROID_TOOL_CONTRACTS } from "./android/androidToolContracts.js"; import { APKTOOL_TOOL_CONTRACTS } from "./apktool/apktoolToolContracts.js"; +import { PCAP_TOOL_CONTRACTS } from "./pcap/pcapToolContracts.js"; import { FLUTTER_TOOL_CONTRACTS } from "./flutter/flutterToolContracts.js"; import { JEB_TOOL_CONTRACTS } from "./jeb/jebToolContracts.js"; import { ADB_TOOL_CONTRACTS } from "./adb/adbToolContracts.js"; @@ -46,6 +47,7 @@ export const TOOL_CONTRACTS = [ ...MANAGED_TOOL_CONTRACTS, ...ANDROID_TOOL_CONTRACTS, ...APKTOOL_TOOL_CONTRACTS, + ...PCAP_TOOL_CONTRACTS, ...FLUTTER_TOOL_CONTRACTS, ...JEB_TOOL_CONTRACTS, ...ADB_TOOL_CONTRACTS, diff --git a/src/contracts/toolEffects.test.ts b/src/contracts/toolEffects.test.ts index 836d190b1..fe18c96b3 100644 --- a/src/contracts/toolEffects.test.ts +++ b/src/contracts/toolEffects.test.ts @@ -44,6 +44,8 @@ const EXECUTION_WORDING_EXCEPTIONS: Readonly> = { search_android_classes: "states it does not execute the APK", inspect_apktool_client: "inspects the operator-selected apktool launcher; never installs it", + inspect_pcap_client: + "probes the operator-selected tshark and capinfos; never installs them", inspect_jeb_client: "talks to a caller-started JEB; never launches it", open_jeb_project: "the engine analyzes the file; REA does not execute it", inspect_adb_client: diff --git a/src/contracts/toolEffects.ts b/src/contracts/toolEffects.ts index 4b0d0b745..7ce3e4ded 100644 --- a/src/contracts/toolEffects.ts +++ b/src/contracts/toolEffects.ts @@ -182,6 +182,16 @@ export const TOOL_EFFECTS: Readonly> = { launchesProcess: true, writesFilesystem: true, }), + inspect_pcap_client: effects({ + mutatesSession: true, + launchesProcess: true, + writesFilesystem: true, + }), + inspect_pcap_capture: effects({ + mutatesSession: true, + launchesProcess: true, + writesFilesystem: true, + }), identify_flutter_build: effects({ mutatesSession: true, writesFilesystem: true, diff --git a/src/domain/analysisView/analysisView.ts b/src/domain/analysisView/analysisView.ts index 48eee73f0..6cc5ae8a3 100644 --- a/src/domain/analysisView/analysisView.ts +++ b/src/domain/analysisView/analysisView.ts @@ -62,6 +62,12 @@ import { captureSummarySchema, } from "./webNetworkCaptureView.js"; import type { WebNetworkApiSurfaceAccumulator } from "./WebNetworkApiSurfaceAccumulator.js"; +import { + pcapViewRequestSchema, + pcapViewPageSchema, + pcapSummarySchema, + projectPcapCaptureView, +} from "./pcapCaptureView.js"; import { qlogViewRequestSchema, qlogViewPageSchema, @@ -196,6 +202,7 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [ apiSurfaceViewSchema, captureViewRequestSchema, qlogViewRequestSchema, + pcapViewRequestSchema, dartAotViewRequestSchema, ]); @@ -312,6 +319,7 @@ const viewResultBase = { "inspect_managed_artifact", "inspect_web_network_capture", "inspect_android_package", + "inspect_pcap_capture", "inspect_dart_aot", ]), parent_digest: digestSchema, @@ -384,6 +392,12 @@ export const androidPackageSummaryViewResultSchema = z.strictObject({ summary: androidPackageSummarySchema, }); +export const pcapCaptureSummaryViewResultSchema = z.strictObject({ + ...summaryViewResultBase, + parent_operation: z.literal("inspect_pcap_capture"), + summary: pcapSummarySchema, +}); + /** Exact snapshot and pool-coverage carrier for selected AOT delivery. */ export const dartAotSummaryViewResultSchema = z.strictObject({ ...summaryViewResultBase, @@ -415,6 +429,7 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ managedSummarySchema, captureSummarySchema, androidPackageSummarySchema, + pcapSummarySchema, dartAotSummarySchema, ]), }), @@ -433,6 +448,14 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ ...viewResultBase, parent_operation: z.literal("inspect_web_network_capture"), }), + z.strictObject({ + kind: z.literal("pcap"), + view: pcapViewRequestSchema, + page: pcapViewPageSchema, + source_coverage: z.enum(["complete", "partial"]), + ...viewResultBase, + parent_operation: z.literal("inspect_pcap_capture"), + }), z.strictObject({ kind: z.literal("facet"), view: facetViewSchema, @@ -562,6 +585,7 @@ export type UnsignedAnalysisView = | Omit, "view_digest"> | Omit, "view_digest"> | Omit, "view_digest"> + | Omit, "view_digest"> | Omit< Extract, "view_digest" @@ -686,6 +710,10 @@ export const projectAnalysisView = ( const projected = projectWebNetworkCaptureView(parent, view); return projected.ok ? ok(sealAnalysisView(projected.value)) : projected; } + if (parent.operation === "inspect_pcap_capture") { + const projected = projectPcapCaptureView(parent, view); + return projected.ok ? ok(sealAnalysisView(projected.value)) : projected; + } if (parent.operation === "inspect_binary_layout") { const layout = binaryLayoutSchema.safeParse(parent.normalizedResult); if (!layout.success) diff --git a/src/domain/analysisView/pcapCaptureView.ts b/src/domain/analysisView/pcapCaptureView.ts new file mode 100644 index 000000000..a28398d44 --- /dev/null +++ b/src/domain/analysisView/pcapCaptureView.ts @@ -0,0 +1,327 @@ +import { z } from "zod"; +import { pcapResultSchemas } from "../pcap/pcapAnalysis.js"; +import { analysisInputErrorFromIssues } from "../inputIssueProjection.js"; +import type { AnalysisError } from "../analysisErrorBase.js"; +import { err, ok, type Result } from "../result.js"; +import { + analysisViewInputError, + type AnalysisViewParent, + type AnalysisViewRequest, + type UnsignedAnalysisView, +} from "./analysisView.js"; +import { + completeWithinViewCoverage, + pageViewCoverage, +} from "./analysisViewCoverage.js"; + +const captureSchema = pcapResultSchemas.inspect_pcap_capture; +const fields = captureSchema.shape; +const pageShape = { + kind: z.literal("pcap"), + offset: z.number().int().nonnegative().default(0), + limit: z.number().int().positive().default(64), +}; +export const pcapViewRequestSchema = z.discriminatedUnion("facet", [ + z.strictObject({ + ...pageShape, + facet: z.enum([ + "endpoints", + "dns-queries", + "tls-sni", + "protocols", + "http-uris", + "unattributed-addresses", + ]), + }), + z.strictObject({ + ...pageShape, + facet: z.literal("dns-addresses"), + query: z.string().min(1), + }), + z.strictObject({ + ...pageShape, + facet: z.literal("diagnostics"), + stream: z.enum(["tshark", "capinfos"]), + }), +]); +export const pcapViewPageSchema = z.discriminatedUnion("facet", [ + z.strictObject({ facet: z.literal("endpoints"), items: fields.endpoints }), + z.strictObject({ + facet: z.literal("dns-queries"), + items: z.array( + fields.dns_queries.element.omit({ response_addresses: true }).extend({ + retained_response_address_count: z.number().int().nonnegative(), + }), + ), + }), + z.strictObject({ facet: z.literal("tls-sni"), items: fields.tls_sni_values }), + z.strictObject({ + facet: z.literal("protocols"), + items: fields.protocol_distribution, + }), + z.strictObject({ + facet: z.literal("http-uris"), + items: fields.http_request_uris, + }), + z.strictObject({ + facet: z.literal("unattributed-addresses"), + items: fields.unattributed_dns_addresses, + }), + z.strictObject({ + facet: z.literal("dns-addresses"), + items: fields.unattributed_dns_addresses, + }), + z.strictObject({ + facet: z.literal("diagnostics"), + text: z.string(), + status: z.enum(["retained", "unknown"]), + }), +]); +export const pcapSummarySchema = captureSchema + .omit({ + endpoints: true, + dns_queries: true, + tls_sni_values: true, + protocol_distribution: true, + http_request_uris: true, + unattributed_dns_addresses: true, + diagnostics: true, + }) + .extend({ + retained_counts: z.strictObject({ + endpoints: z.number().int().nonnegative(), + dns_queries: z.number().int().nonnegative(), + tls_sni_values: z.number().int().nonnegative(), + protocol_distribution: z.number().int().nonnegative(), + http_request_uris: z.number().int().nonnegative(), + unattributed_dns_addresses: z.number().int().nonnegative(), + }), + diagnostics_code_units: z + .strictObject({ + tshark: z.number().int().nonnegative(), + capinfos: z.number().int().nonnegative(), + }) + .nullable(), + }); +type Capture = z.output; +type PcapView = z.output; +type Page = z.output; + +const parseParent = ( + parent: AnalysisViewParent, +): Result => { + const parsed = captureSchema.safeParse(parent.normalizedResult); + if (!parsed.success) + return err( + analysisInputErrorFromIssues( + "inspect_analysis_view", + parsed.error.issues, + parent.normalizedResult, + { cause: parsed.error }, + ), + ); + if ( + parent.subject?.local_path !== parsed.data.target.path || + parent.subject.digest.sha256 !== parsed.data.target.sha256 + ) + return err( + analysisViewInputError([ + { + path: ["source", "subject"], + reason: "invalid_value", + message: + "Packet parent subject must identify the report's exact path and SHA-256.", + }, + ]), + ); + return ok(parsed.data); +}; +const summarize = ({ + endpoints, + dns_queries, + tls_sni_values, + protocol_distribution, + http_request_uris, + unattributed_dns_addresses, + diagnostics, + ...metadata +}: Capture) => ({ + ...metadata, + retained_counts: { + endpoints: endpoints.length, + dns_queries: dns_queries.length, + tls_sni_values: tls_sni_values.length, + protocol_distribution: protocol_distribution.length, + http_request_uris: http_request_uris.length, + unattributed_dns_addresses: unattributed_dns_addresses.length, + }, + diagnostics_code_units: + diagnostics === undefined + ? null + : { + tshark: diagnostics.tshark_stderr.length, + capinfos: diagnostics.capinfos_stderr.length, + }, +}); + +const selectRows = ( + capture: Capture, + view: Exclude, +): Result< + { page: Exclude; total: number }, + AnalysisError +> => { + const slice = (items: readonly T[]) => + items.slice(view.offset, view.offset + view.limit); + switch (view.facet) { + case "endpoints": + return ok({ + page: { facet: view.facet, items: slice(capture.endpoints) }, + total: capture.endpoints.length, + }); + case "dns-queries": + return ok({ + page: { + facet: view.facet, + items: slice(capture.dns_queries).map( + ({ response_addresses, ...query }) => ({ + ...query, + retained_response_address_count: response_addresses.length, + }), + ), + }, + total: capture.dns_queries.length, + }); + case "tls-sni": + return ok({ + page: { facet: view.facet, items: slice(capture.tls_sni_values) }, + total: capture.tls_sni_values.length, + }); + case "protocols": + return ok({ + page: { + facet: view.facet, + items: slice(capture.protocol_distribution), + }, + total: capture.protocol_distribution.length, + }); + case "http-uris": + return ok({ + page: { facet: view.facet, items: slice(capture.http_request_uris) }, + total: capture.http_request_uris.length, + }); + case "unattributed-addresses": + return ok({ + page: { + facet: view.facet, + items: slice(capture.unattributed_dns_addresses), + }, + total: capture.unattributed_dns_addresses.length, + }); + case "dns-addresses": { + const matches = capture.dns_queries.filter( + (query) => query.query === view.query, + ); + if (matches.length !== 1) + return err( + analysisViewInputError([ + { + path: ["view", "query"], + reason: "invalid_value", + message: "Select one unambiguous exact retained DNS question.", + }, + ]), + ); + return ok({ + page: { + facet: view.facet, + items: slice(matches[0]!.response_addresses), + }, + total: matches[0]!.response_addresses.length, + }); + } + } +}; + +/** Historical packet facts are paged without reopening the source or rerunning Wireshark. */ +export const projectPcapCaptureView = ( + parent: AnalysisViewParent, + view: AnalysisViewRequest, +): Result => { + if (view.kind !== "summary" && view.kind !== "pcap") + return err( + analysisViewInputError([ + { + path: ["view"], + reason: "invalid_value", + message: + "Packet summary parents support summary and pcap fact pages.", + }, + ]), + ); + const parsed = parseParent(parent); + if (!parsed.ok) return parsed; + const capture = parsed.value; + const shared = { + parent_evidence_id: parent.evidenceId, + parent_operation: "inspect_pcap_capture" as const, + parent_digest: parent.evidenceId.slice(3), + artifact: { path: capture.target.path, sha256: capture.target.sha256 }, + limitations: [ + ...parent.limitations, + "Page totals count retained rows or UTF-16 diagnostic code units. Exhaustion does not prove whole-network completeness; source coverage and examined counts remain separate.", + ], + unknowns: [ + "Saved packet observations do not establish process identity, causality, encrypted payload contents or traffic missing from the recording.", + ], + }; + if (view.kind === "summary") + return ok({ + kind: "summary", + view, + ...shared, + summary: summarize(capture), + coverage: completeWithinViewCoverage(1, 1), + }); + if (view.facet === "diagnostics") { + const value = + view.stream === "tshark" + ? capture.diagnostics?.tshark_stderr + : capture.diagnostics?.capinfos_stderr; + const text = value?.slice(view.offset, view.offset + view.limit) ?? ""; + return ok({ + kind: "pcap", + view, + ...shared, + source_coverage: capture.coverage, + page: { + facet: "diagnostics", + text, + status: value === undefined ? "unknown" : "retained", + }, + coverage: pageViewCoverage(view.offset, text.length, value?.length ?? 0), + unknowns: + value === undefined + ? [ + ...shared.unknowns, + "The parent did not retain diagnostics; an empty text page does not establish their absence.", + ] + : shared.unknowns, + }); + } + const selected = selectRows(capture, view); + return selected.ok + ? ok({ + kind: "pcap", + view, + ...shared, + source_coverage: capture.coverage, + page: selected.value.page, + coverage: pageViewCoverage( + view.offset, + selected.value.page.items.length, + selected.value.total, + ), + }) + : selected; +}; diff --git a/src/domain/pcap/pcapAnalysis.ts b/src/domain/pcap/pcapAnalysis.ts new file mode 100644 index 000000000..21c64f365 --- /dev/null +++ b/src/domain/pcap/pcapAnalysis.ts @@ -0,0 +1,110 @@ +import { z } from "zod"; +import { localPathStringSchema } from "../localPath.js"; +import { digestSchema } from "../digests.js"; + +/** + * Caller intent for packet-capture analysis through caller-supplied tshark + * and capinfos. REA never installs Wireshark or its tools. + */ +export const pcapInputSchemas = { + inspect_pcap_client: z.strictObject({}), + inspect_pcap_capture: z.strictObject({ + path: localPathStringSchema.describe( + "Local pcap or pcapng capture file to inspect", + ), + }), +} as const; + +/** Supported pcap-backed operations. */ +export type PcapOperation = keyof typeof pcapInputSchemas; +/** Validated pcap requests carry their selected operation. */ +export type PcapRequest = { + [Name in PcapOperation]: { + operation: Name; + input: z.infer<(typeof pcapInputSchemas)[Name]>; + }; +}[PcapOperation]; + +/** Validate operation and input together so their types remain correlated. */ +export const pcapRequestSchema = z.discriminatedUnion("operation", [ + z.strictObject({ + operation: z.literal("inspect_pcap_client"), + input: pcapInputSchemas.inspect_pcap_client, + }), + z.strictObject({ + operation: z.literal("inspect_pcap_capture"), + input: pcapInputSchemas.inspect_pcap_capture, + }), +]); + +const client = z.strictObject({ + tshark_command: z.string().min(1), + tshark_source: z.enum(["environment", "path"]), + tshark_version: z.string().min(1).nullable(), + capinfos_command: z.string().min(1), + capinfos_source: z.enum(["environment", "path"]), + capinfos_version: z.string().min(1).nullable(), +}); + +const endpoint = z.strictObject({ + address: z.string().min(1), + port: z.number().int().positive().nullable(), + packet_count: z.number().int().positive(), +}); + +const dnsQuery = z.strictObject({ + query: z.string().min(1), + response_addresses: z.array(z.string().min(1)), + response_address_count: z.number().int().nonnegative(), +}); + +const protocolCount = z.strictObject({ + protocol: z.string().min(1), + packet_count: z.number().int().positive(), +}); + +/** Portable pcap observations with explicit coverage limitations. */ +export const pcapResultSchemas = { + inspect_pcap_client: z.strictObject({ + client, + }), + inspect_pcap_capture: z.strictObject({ + client, + target: z.strictObject({ + path: localPathStringSchema, + bytes: z.number().int().nonnegative(), + sha256: digestSchema, + }), + /** File format as capinfos reports it (pcap vs pcapng). */ + file_type: z.string().min(1).nullable(), + encapsulation: z.string().min(1).nullable(), + packet_count: z.number().int().nonnegative().nullable(), + capture_start: z.string().nullable(), + capture_end: z.string().nullable(), + capture_duration_seconds: z.number().nonnegative().nullable(), + snapshot_length: z.number().int().positive().nullable(), + /** Distinct endpoints with packet counts, bounded. */ + endpoints: z.array(endpoint), + endpoint_count: z.number().int().nonnegative(), + /** DNS queries with resolved addresses, bounded. */ + dns_queries: z.array(dnsQuery), + dns_query_count: z.number().int().nonnegative(), + /** TLS Server Name Indication values observed in handshakes. */ + tls_sni_values: z.array(z.string().min(1)), + tls_sni_count: z.number().int().nonnegative(), + /** Protocol distribution (frame.protocols column counts). */ + protocol_distribution: z.array(protocolCount), + protocol_count: z.number().int().nonnegative(), + /** HTTP request URIs seen on plain-text ports, bounded. */ + http_request_uris: z.array(z.string().min(1)), + http_request_uri_count: z.number().int().nonnegative(), + http_uri_redaction_count: z.number().int().nonnegative(), + unattributed_dns_addresses: z.array(z.string().min(1)), + unattributed_dns_address_count: z.number().int().nonnegative(), + unattributed_dns_packet_count: z.number().int().nonnegative(), + coverage: z.enum(["complete", "partial"]), + diagnostics: z + .strictObject({ capinfos_stderr: z.string(), tshark_stderr: z.string() }) + .optional(), + }), +} as const; diff --git a/src/domain/pcap/pcapEvidence.ts b/src/domain/pcap/pcapEvidence.ts new file mode 100644 index 000000000..848350fe3 --- /dev/null +++ b/src/domain/pcap/pcapEvidence.ts @@ -0,0 +1,34 @@ +import { createEvidence } from "../evidence.js"; +import { jsonObjectSchema, type JsonValue } from "../jsonValue.js"; +import type { PcapRequest } from "./pcapAnalysis.js"; + +/** Domain facts needed by both successful and retained partial observations. */ +interface PcapObservation { + readonly subject: Parameters[0] | null; + readonly provider: Parameters[1]; + readonly result: JsonValue; + readonly rawResult: JsonValue | null; + readonly limitations: readonly string[]; + readonly locations: NonNullable< + Parameters[2]["locations"] + >; +} + +/** One owner commits packet aggregation provenance and caller intent into Evidence. */ +export const createPcapEvidence = ( + request: PcapRequest, + observation: PcapObservation, +) => + createEvidence(observation.subject ?? undefined, observation.provider, { + operation: request.operation, + parameters: jsonObjectSchema.parse(request.input), + result: observation.result, + rawResult: observation.rawResult, + limitations: observation.limitations, + locations: observation.locations, + confidence: + request.operation === "inspect_pcap_capture" ? "derived" : "observed", + ...(request.operation === "inspect_pcap_capture" + ? { authority: "historical-reference" as const } + : {}), + }); diff --git a/src/pcap/PcapCommand.ts b/src/pcap/PcapCommand.ts new file mode 100644 index 000000000..a0833f599 --- /dev/null +++ b/src/pcap/PcapCommand.ts @@ -0,0 +1,207 @@ +import { randomUUID } from "node:crypto"; +import { AnalysisError } from "../domain/analysisErrorBase.js"; +import { + AnalysisCancelledError, + AnalysisCapabilityUnavailableError, + AnalysisProtocolError, + AnalysisResourceConstraintError, + AnalysisTimeoutError, + AnalysisUnsupportedTargetError, +} from "../domain/analysisErrorCore.js"; +import { + OwnedCommandFailure, + runOwnedCommand, +} from "../process/OwnedCommand.js"; +import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js"; +import { prepareProcessOwnershipInspection } from "../process/ProcessOwnershipObservation.js"; +import { + isWindowsBatchFile, + windowsBatchCommand, + WindowsBatchCommandError, +} from "../process/WindowsBatchCommand.js"; + +export interface PcapCommandLimits { + readonly timeoutMs: number; + readonly maxOutputBytes: number; +} + +/** Owned process execution shared by version probes and capture reads. */ +export const runPcapCommand = async (options: { + readonly operation: string; + readonly command: string; + readonly arguments: readonly string[]; + readonly limits: PcapCommandLimits; + readonly environment: Readonly>; + readonly signal: AbortSignal | undefined; + readonly inputPath?: string; + readonly nativeBound?: { + readonly limiter?: { command: string; arguments: readonly string[] }; + readonly memoryLimitBytes?: number; + }; + readonly retain: ( + owner: ProviderProcessSupervisor, + resources: readonly string[], + ) => void; +}): Promise<{ stdout: string; stderr: string }> => { + const { operation, command, limits, signal } = options; + if (signal?.aborted) + throw new AnalysisCancelledError(operation, { cause: signal.reason }); + const hostEnvironment = { + ...process.env, + ...options.environment, + LC_ALL: "C", + LANG: "C", + }; + try { + const batch = process.platform === "win32" && isWindowsBatchFile(command); + const invocation = batch + ? windowsBatchCommand({ + environment: hostEnvironment, + batchPath: command, + arguments: options.arguments, + }) + : { command, arguments: options.arguments }; + const boundedInvocation = + options.nativeBound?.limiter === undefined + ? invocation + : { + command: options.nativeBound.limiter.command, + arguments: [ + ...options.nativeBound.limiter.arguments, + invocation.command, + ...invocation.arguments, + ], + }; + // Native ownership preparation has its own bounded compiler lifecycle; + // do not spend the tool's execution deadline rebuilding a cold SDK cache. + await prepareProcessOwnershipInspection(signal); + const output = await runOwnedCommand( + { + ...boundedInvocation, + runId: randomUUID(), + hostEnvironment, + windowsVerbatimArguments: batch, + expectedCommand: null, + ...(options.nativeBound?.memoryLimitBytes === undefined + ? {} + : { memoryLimitBytes: options.nativeBound.memoryLimitBytes }), + }, + { timeoutMs: limits.timeoutMs, diagnosticBytes: limits.maxOutputBytes }, + signal === undefined ? {} : { signal }, + ); + return { stdout: output.stdout.text, stderr: output.stderr.text }; + } catch (cause) { + if (cause instanceof AnalysisError) throw cause; + if ( + cause instanceof OwnedCommandFailure && + cause.cleanupOwner !== undefined + ) + options.retain(cause.cleanupOwner, cause.resources); + const details = { + cause, + ...(cause instanceof OwnedCommandFailure && cause.snapshot !== null + ? { + capturedOutput: { + stdout: "", + stderr: cause.snapshot.stderr.text, + truncated: + cause.snapshot.stdout.observedBytes > 0 || + (cause.snapshot.diagnosticTruncated ?? false), + exit_code: cause.snapshot.exitCode ?? null, + signal: cause.snapshot.signal ?? null, + }, + } + : {}), + ...(cause instanceof OwnedCommandFailure && cause.cleanupFailure !== null + ? { + cleanup: { + reason: cause.cleanupFailure, + resources: cause.resources, + }, + } + : {}), + }; + if ( + signal?.aborted || + (cause instanceof OwnedCommandFailure && cause.reason === "cancelled") + ) + throw new AnalysisCancelledError(operation, details); + if ( + cause instanceof OwnedCommandFailure && + cause.snapshot?.memoryLimitReached === true + ) + throw new AnalysisResourceConstraintError( + operation, + "memory", + "The owned Windows tool tree reached its enforced committed-memory limit.", + { + process_memory_limit_bytes: + options.nativeBound?.memoryLimitBytes ?? null, + }, + details, + ); + if ( + cause instanceof OwnedCommandFailure && + (cause.snapshot?.signal === "SIGXCPU" || + cause.snapshot?.signal === "SIGXFSZ") + ) + throw new AnalysisResourceConstraintError( + operation, + cause.snapshot.signal === "SIGXCPU" ? "cpu" : "file-size", + `Producer terminated with ${cause.snapshot.signal}; its exact cause is not independently established.`, + null, + details, + ); + if (cause instanceof OwnedCommandFailure && cause.reason === "timeout") + throw new AnalysisTimeoutError(operation, limits.timeoutMs, details); + if (cause instanceof OwnedCommandFailure && cause.reason === "output-limit") + throw new AnalysisResourceConstraintError( + operation, + "memory", + "Tool output exceeded its complete-output budget", + { max_output_bytes: limits.maxOutputBytes }, + details, + ); + if ( + cause instanceof WindowsBatchCommandError || + (cause instanceof Error && + "code" in cause && + (cause.code === "ENOENT" || cause.code === "EACCES")) + ) + throw new AnalysisCapabilityUnavailableError( + "pcap", + operation, + `Cannot run selected tool ${command}: ${cause.message}`, + details, + ); + const diagnostic = + cause instanceof OwnedCommandFailure + ? cause.snapshot?.stderr.text.trim() || cause.message + : cause instanceof Error + ? cause.message + : String(cause); + if (options.inputPath === undefined) + throw new AnalysisCapabilityUnavailableError( + "pcap", + operation, + `The selected tool ${command} cannot report its version: ${diagnostic}`, + details, + ); + if ( + options.inputPath !== undefined && + /not a capture file|file.*isn't a capture file|file format.*not supported/iu.test( + diagnostic, + ) + ) + throw new AnalysisUnsupportedTargetError( + operation, + options.inputPath, + diagnostic, + details, + ); + throw new AnalysisProtocolError( + `Packet capture tool ${command} failed: ${diagnostic}`, + details, + ); + } +}; diff --git a/src/pcap/PcapConfiguration.test.ts b/src/pcap/PcapConfiguration.test.ts new file mode 100644 index 000000000..edbd223b2 --- /dev/null +++ b/src/pcap/PcapConfiguration.test.ts @@ -0,0 +1,170 @@ +import { describe, expect, it } from "vitest"; + +import { + CAPINFOS_COMMAND_ENV, + TSHARK_COMMAND_ENV, + resolvePcapTools, +} from "./PcapConfiguration.js"; +import { parseCapinfosOutput, parseTsharkJsonOutput } from "./PcapOutput.js"; + +describe("resolvePcapTools", () => { + it("prefers absolute environment paths for both tools", () => { + expect( + resolvePcapTools({ + [TSHARK_COMMAND_ENV]: "/usr/bin/tshark", + [CAPINFOS_COMMAND_ENV]: "/usr/bin/capinfos", + }), + ).toEqual({ + tshark: { command: "/usr/bin/tshark", source: "environment" }, + capinfos: { command: "/usr/bin/capinfos", source: "environment" }, + }); + }); + + it("falls back to PATH resolution without the environment keys", () => { + expect(resolvePcapTools({})).toEqual({ + tshark: { command: "tshark", source: "path" }, + capinfos: { command: "capinfos", source: "path" }, + }); + }); + + it("rejects a relative tshark path explicitly", () => { + expect(() => + resolvePcapTools({ [TSHARK_COMMAND_ENV]: "bin/tshark" }), + ).toThrow(/must be an absolute path/u); + }); + + it("rejects a relative capinfos path explicitly", () => { + expect(() => + resolvePcapTools({ [CAPINFOS_COMMAND_ENV]: "capinfos" }), + ).toThrow(/must be an absolute path/u); + }); +}); + +const REAL_CAPINFOS = [ + "File name: /work/rich-capture.pcap", + "File type: Wireshark/tcpdump/... - pcap", + "File encapsulation: Linux cooked-mode capture v2", + "Packet size limit: file hdr: 262144 bytes", + "Number of packets: 50", + "File size: 23279 bytes", + "Capture duration: 0.373473 seconds", + "Earliest packet time: 2026-10-11 14:13:40.608505", + "Latest packet time: 2026-10-11 14:13:40.981978", + "SHA256: c94f94742b3288b9a979711509ed5eab3028576301f8938319de3c22a6f5c40b", +].join("\n"); + +describe("parseCapinfosOutput", () => { + it("projects metadata from real capinfos output", () => { + expect(parseCapinfosOutput(REAL_CAPINFOS)).toEqual({ + fileType: "Wireshark/tcpdump/... - pcap", + encapsulation: "Linux cooked-mode capture v2", + packetCount: 50, + captureStart: "2026-10-11 14:13:40.608505", + captureEnd: "2026-10-11 14:13:40.981978", + captureDurationSeconds: 0.373473, + snapshotLength: 262144, + sha256: + "c94f94742b3288b9a979711509ed5eab3028576301f8938319de3c22a6f5c40b", + }); + }); + + it("rejects empty metadata rather than reporting complete unknown facts", () => { + expect(() => parseCapinfosOutput("")).toThrow(/machine-readable/u); + }); +}); + +const REAL_TSHARK_JSON = JSON.stringify([ + { + _source: { + layers: { + "frame.number": ["1"], + "frame.protocols": ["sll:ethertype:ip:udp:dns"], + "ip.src": ["192.168.1.1"], + "ip.dst": ["8.8.8.8"], + "dns.qry.name": ["example.com"], + "dns.flags.response": ["1"], + "dns.a": ["93.184.216.34"], + }, + }, + }, + { + _source: { + layers: { + "frame.number": ["2"], + "frame.protocols": ["sll:ethertype:ip:tcp:tls"], + "ip.src": ["192.168.1.1"], + "ip.dst": ["93.184.216.34"], + "tls.handshake.extensions_server_name": ["example.com"], + }, + }, + }, + { + _source: { + layers: { + "frame.number": ["3"], + "frame.protocols": ["sll:ethertype:ip:tcp:http"], + "ip.src": ["192.168.1.1"], + "ip.dst": ["93.184.216.34"], + "http.request.full_uri": ["http://example.com/page"], + }, + }, + }, +]); + +describe("parseTsharkJsonOutput", () => { + it("projects endpoints, DNS, TLS SNI, protocols, and HTTP URIs", async () => { + const result = await parseTsharkJsonOutput(REAL_TSHARK_JSON); + expect(result.endpoints).toEqual( + expect.arrayContaining([ + { address: "192.168.1.1", port: null, packet_count: 3 }, + { address: "8.8.8.8", port: null, packet_count: 1 }, + { address: "93.184.216.34", port: null, packet_count: 2 }, + ]), + ); + expect(result.dnsQueries).toEqual([ + { + query: "example.com", + response_addresses: ["93.184.216.34"], + response_address_count: 1, + }, + ]); + expect(result.tlsSniValues).toEqual(["example.com"]); + expect(result.httpRequestUris).toEqual(["http://example.com/page"]); + expect(result.protocolDistribution).toEqual( + expect.arrayContaining([ + { protocol: "sll:ethertype:ip:udp:dns", packet_count: 1 }, + { protocol: "sll:ethertype:ip:tcp:tls", packet_count: 1 }, + { protocol: "sll:ethertype:ip:tcp:http", packet_count: 1 }, + ]), + ); + }); + + it("retains every endpoint within the aggregate resource budget", async () => { + const packets = Array.from({ length: 10 }, (_, index) => ({ + _source: { + layers: { + "frame.number": [String(index + 1)], + "frame.protocols": ["ip:tcp"], + "ip.src": [`10.0.0.${index}`], + "ip.dst": ["10.0.1.1"], + }, + }, + })); + const result = await parseTsharkJsonOutput(JSON.stringify(packets)); + expect(result.endpoints).toHaveLength(11); + expect(result.endpointCount).toBe(11); // 10 src + 1 dst + }); + + it.each([ + "", + "not json", + "{}", + "null", + "[null]", + '[{"_source":{"layers":{"ip.src":[3]}}}]', + ])("rejects malformed producer output %s", async (text) => { + await expect(parseTsharkJsonOutput(text)).rejects.toMatchObject({ + _tag: "AnalysisProtocolError", + }); + }); +}); diff --git a/src/pcap/PcapConfiguration.ts b/src/pcap/PcapConfiguration.ts new file mode 100644 index 000000000..8f75b4602 --- /dev/null +++ b/src/pcap/PcapConfiguration.ts @@ -0,0 +1,153 @@ +import { isAbsolute } from "node:path"; + +import type { ProviderAvailability } from "../application/AnalysisProvider.js"; +import type { ProviderRejectionCode } from "../contracts/providerSelection.js"; +import type { JsonValue } from "../domain/jsonValue.js"; + +/** Environment keys selecting the caller's tshark and capinfos binaries. */ +export const TSHARK_COMMAND_ENV = "REA_TSHARK_COMMAND"; +export const CAPINFOS_COMMAND_ENV = "REA_CAPINFOS_COMMAND"; + +/** How each tool was selected for this provider instance. */ +export interface PcapToolSelection { + readonly tshark: { command: string; source: "environment" | "path" }; + readonly capinfos: { command: string; source: "environment" | "path" }; +} + +/** + * Resolve the caller-supplied tshark and capinfos binaries. Each accepts an + * absolute environment path or falls back to PATH resolution, mirroring the + * mitmdump pattern. REA never installs the Wireshark suite. + */ +export const resolvePcapTools = ( + environment: Readonly>, +): PcapToolSelection => { + const tsharkEnv = environment[TSHARK_COMMAND_ENV]; + const capinfosEnv = environment[CAPINFOS_COMMAND_ENV]; + if ( + tsharkEnv !== undefined && + tsharkEnv.trim() !== "" && + !isAbsolute(tsharkEnv) + ) + throw new PcapConfigurationFailure( + "tshark_not_absolute", + `${TSHARK_COMMAND_ENV} must be an absolute path to the tshark binary`, + { configured: tsharkEnv }, + ); + if ( + capinfosEnv !== undefined && + capinfosEnv.trim() !== "" && + !isAbsolute(capinfosEnv) + ) + throw new PcapConfigurationFailure( + "capinfos_not_absolute", + `${CAPINFOS_COMMAND_ENV} must be an absolute path to the capinfos binary`, + { configured: capinfosEnv }, + ); + return { + tshark: { + command: + tsharkEnv === undefined || tsharkEnv.trim() === "" + ? "tshark" + : tsharkEnv, + source: + tsharkEnv === undefined || tsharkEnv.trim() === "" + ? "path" + : "environment", + }, + capinfos: { + command: + capinfosEnv === undefined || capinfosEnv.trim() === "" + ? "capinfos" + : capinfosEnv, + source: + capinfosEnv === undefined || capinfosEnv.trim() === "" + ? "path" + : "environment", + }, + }; +}; + +export type PcapConfigurationFailureCode = + | "tshark_not_absolute" + | "capinfos_not_absolute" + | "tshark_missing" + | "capinfos_missing" + | "tshark_version_unresolved" + | "capinfos_version_unresolved" + | "tshark_version_unsupported" + | "capinfos_version_unsupported"; + +export class PcapConfigurationFailure extends Error { + constructor( + readonly code: PcapConfigurationFailureCode, + message: string, + readonly diagnostics: Readonly> = {}, + options?: ErrorOptions, + ) { + super(message, options); + } +} + +/** Recognize the selected Wireshark tool's version banner. */ +const parseVersion = ( + stdout: string, + tool: "tshark" | "capinfos", +): string | null => { + const line = stdout + .split(/\r?\n/u) + .map((entry) => entry.trim()) + .find((entry) => entry !== ""); + const parsed = + line === undefined + ? null + : new RegExp(`^${tool} \\(Wireshark\\) (\\d+)\\.\\d+\\.\\d+`, "iu").exec( + line, + ); + if (parsed === null || line === undefined) return null; + if (parsed[1] !== "4") + throw new PcapConfigurationFailure( + `${tool}_version_unsupported`, + "Packet summary inspection currently requires Wireshark 4.x tools; select an installation with the admitted output profile.", + { version: line }, + ); + return line; +}; + +export interface PcapClientIdentity { + readonly tsharkVersion: string | null; + readonly capinfosVersion: string | null; +} + +/** Probe both tools without reading any capture file. */ +export const inspectPcapClient = async ( + selection: PcapToolSelection, + probe: (command: string) => Promise, +): Promise => ({ + tsharkVersion: parseVersion(await probe(selection.tshark.command), "tshark"), + capinfosVersion: parseVersion( + await probe(selection.capinfos.command), + "capinfos", + ), +}); + +/** Availability in the shape shared by every provider. */ +const rejectionCodeOf = ( + code: PcapConfigurationFailureCode, +): ProviderRejectionCode => + code.includes("missing") + ? "executable_missing" + : code.includes("unsupported") + ? "unsupported_version" + : code.includes("not_absolute") + ? "not_configured" + : "version_unresolved"; + +export const pcapConfigurationAvailability = ( + cause: PcapConfigurationFailure, +): ProviderAvailability => ({ + status: "unavailable", + code: rejectionCodeOf(cause.code), + reason: cause.message, + diagnostics: { ...cause.diagnostics }, +}); diff --git a/src/pcap/PcapNativeBound.ts b/src/pcap/PcapNativeBound.ts new file mode 100644 index 000000000..172eaba91 --- /dev/null +++ b/src/pcap/PcapNativeBound.ts @@ -0,0 +1,68 @@ +import { access, realpath } from "node:fs/promises"; +import { constants } from "node:fs"; +import { isAbsolute } from "node:path"; +import { AnalysisError } from "../domain/analysisErrorBase.js"; +import { AnalysisCapabilityUnavailableError } from "../domain/analysisErrorCore.js"; +import type { ArtifactAnalysisResources } from "../process/ArtifactAnalysisResources.js"; +import type { runPcapCommand } from "./PcapCommand.js"; + +/** Own native capture admission and the exact enforced resource profile. */ +export const preparePcapNativeBound = async ( + environment: Readonly>, + resources: ArtifactAnalysisResources, +): Promise<{ + nativeBound: NonNullable[0]["nativeBound"]>; + limitations: readonly string[]; +}> => { + if (process.platform === "win32") { + return { + nativeBound: { memoryLimitBytes: 1024 ** 3 }, + limitations: [ + "Each owned Windows tool tree has a 1 GiB Job Object committed-memory limit; this is not a machine-wide quota.", + ], + }; + } + if (process.platform !== "linux") + throw new AnalysisCapabilityUnavailableError( + "pcap", + "inspect_pcap_capture", + "Packet summaries require an enforceable native memory bound: Linux with util-linux prlimit, or Windows with owned Job Objects. This host's capture boundary remains unsupported.", + ); + const configured = + environment.REA_TSHARK_PRLIMIT_COMMAND ?? "/usr/bin/prlimit"; + try { + if (!isAbsolute(configured)) + throw new Error("REA_TSHARK_PRLIMIT_COMMAND must be absolute."); + const limiter = await realpath(configured); + await access(limiter, constants.X_OK); + const limits = await resources.readLimits({ + address_space_bytes: 1024 ** 3, + file_bytes: 64 * 1024 ** 2, + cpu_seconds: 300, + }); + const nativeBound = { + limiter: { + command: limiter, + arguments: [ + `--as=${limits.address_space_bytes}:${limits.address_space_bytes}`, + `--fsize=${limits.file_bytes}:${limits.file_bytes}`, + `--cpu=${limits.cpu_seconds}:${limits.cpu_seconds}`, + "--core=0:0", + "--", + ], + }, + }; + const limitations = [ + `Linux prlimit preserves stricter inherited ceilings: ${limits.address_space_bytes} bytes of per-process address space, ${limits.file_bytes} bytes per file and ${limits.cpu_seconds} CPU seconds; core dumps are disabled. These are not aggregate machine quotas.`, + ]; + return { nativeBound, limitations }; + } catch (cause) { + if (cause instanceof AnalysisError) throw cause; + throw new AnalysisCapabilityUnavailableError( + "pcap", + "inspect_pcap_capture", + "Cannot establish the selected Linux resource limiter before processing capture bytes.", + { cause }, + ); + } +}; diff --git a/src/pcap/PcapOutput.test.ts b/src/pcap/PcapOutput.test.ts new file mode 100644 index 000000000..6cfa6b0d3 --- /dev/null +++ b/src/pcap/PcapOutput.test.ts @@ -0,0 +1,149 @@ +import { expect, it } from "vitest"; +import { parseCapinfosOutput, parseTsharkJsonOutput } from "./PcapOutput.js"; + +const capture = (layers: ReadonlyArray>) => + JSON.stringify( + layers.map((fields, index) => ({ + _source: { + layers: { + "frame.number": [String(index + 1)], + "frame.protocols": ["ip:udp:dns"], + ...fields, + }, + }, + })), + ); + +it("binds interleaved DNS answers to the question in their response packet", async () => { + const result = await parseTsharkJsonOutput( + capture([ + { "dns.qry.name": ["alpha.example"], "dns.flags.response": ["False"] }, + { "dns.qry.name": ["beta.example"], "dns.flags.response": ["False"] }, + { + "dns.qry.name": ["alpha.example"], + "dns.flags.response": ["True"], + "dns.a": ["192.0.2.10"], + }, + { + "dns.qry.name": ["beta.example"], + "dns.flags.response": ["True"], + "dns.a": ["192.0.2.20"], + }, + ]), + ); + expect(result.dnsQueries).toEqual([ + { + query: "alpha.example", + response_addresses: ["192.0.2.10"], + response_address_count: 1, + }, + { + query: "beta.example", + response_addresses: ["192.0.2.20"], + response_address_count: 1, + }, + ]); + expect(result.partial).toBe(false); +}); + +it("retains unassignable DNS addresses without inventing a query relationship", async () => { + const result = await parseTsharkJsonOutput( + capture([ + { + "dns.qry.name": ["alpha.example", "beta.example"], + "dns.flags.response": ["True"], + "dns.a": ["192.0.2.9"], + }, + { "dns.flags.response": ["True"], "dns.aaaa": ["2001:db8::9"] }, + ]), + ); + expect( + result.dnsQueries.every((query) => query.response_addresses.length === 0), + ).toBe(true); + expect(result.unattributedDnsAddresses).toEqual(["192.0.2.9", "2001:db8::9"]); + expect(result.unattributedDnsPackets).toBe(2); + expect(result.partial).toBe(true); +}); + +it("counts IPv6 endpoints once per packet and preserves every examined fact", async () => { + const packets = Array.from({ length: 51 }, (_, index) => ({ + "ipv6.src": ["2001:db8::1"], + "ipv6.dst": ["2001:db8::1"], + "tls.handshake.extensions_server_name": [`host${String(index)}.example`], + "frame.protocols": [`ipv6:protocol${String(index)}`], + })); + const result = await parseTsharkJsonOutput(capture(packets)); + expect(result.endpoints).toEqual([ + { address: "2001:db8::1", port: null, packet_count: 51 }, + ]); + expect(result.tlsSniCount).toBe(51); + expect(result.tlsSniValues).toHaveLength(51); + expect(result.protocolCount).toBe(51); + expect(result.protocolDistribution).toHaveLength(51); + expect(result.partial).toBe(false); +}); + +it("preserves all DNS addresses and their exact examined count", async () => { + const result = await parseTsharkJsonOutput( + capture( + Array.from({ length: 101 }, (_, index) => ({ + "dns.qry.name": ["rotating.example"], + "dns.flags.response": ["True"], + "dns.a": [`192.0.2.${String(index)}`], + })), + ), + ); + expect(result.dnsQueries[0]?.response_addresses).toHaveLength(101); + expect(result.dnsQueries[0]?.response_address_count).toBe(101); + expect(result.partial).toBe(false); +}); + +it("accepts exact machine packet counts and rejects scaled human counts", () => { + const prefix = + "File type: pcap\nFile encapsulation: Ethernet\nNumber of packets: "; + expect(parseCapinfosOutput(`${prefix}20010`).packetCount).toBe(20010); + expect(() => parseCapinfosOutput(`${prefix}20 k`)).toThrow( + /machine-readable/u, + ); +}); + +it("excludes transport credentials without normalizing observed URL bytes", async () => { + const result = await parseTsharkJsonOutput( + capture([ + { + "http.request.full_uri": [ + "http://user:password@Example.com/a/../b?token=local-evidence#keep", + ], + }, + { + "http.request.full_uri": [ + "http://other:secret@Example.com/a/../b?token=local-evidence#keep", + ], + }, + ]), + ); + expect(result.httpRequestUris).toEqual([ + "http://Example.com/a/../b?token=local-evidence#keep", + ]); + expect(result.httpRequestUriCount).toBe(2); + expect(result.httpUriRedactionCount).toBe(2); + expect(result.partial).toBe(true); +}); + +it("refuses dense JSON before its expanded graph exceeds the decoding budget", async () => { + await expect( + parseTsharkJsonOutput(`[${"{},".repeat(600_000)}{}]`), + ).rejects.toMatchObject({ + _tag: "AnalysisResourceConstraintError", + }); +}); + +it("permits cancellation while validating and aggregating producer output", async () => { + const controller = new AbortController(); + const result = parseTsharkJsonOutput( + capture([{ "ip.src": ["192.0.2.1"] }]), + controller.signal, + ); + controller.abort(new Error("stopped")); + await expect(result).rejects.toThrow("stopped"); +}); diff --git a/src/pcap/PcapOutput.ts b/src/pcap/PcapOutput.ts new file mode 100644 index 000000000..95cb81926 --- /dev/null +++ b/src/pcap/PcapOutput.ts @@ -0,0 +1,315 @@ +import { setImmediate as yieldToEventLoop } from "node:timers/promises"; +import { + AnalysisProtocolError, + AnalysisResourceConstraintError, +} from "../domain/analysisErrorCore.js"; +import { parseRecordedTransportUrl } from "../domain/recordedTransportUrl.js"; + +const OPERATION = "inspect_pcap_capture"; +/** Bounds JSON decoding plus its expanded object graph before JSON.parse. */ +export const TSHARK_JSON_MAX_BYTES = 8 * 1024 * 1024; +const MAX_RETAINED_BYTES = 16 * 1024 * 1024; +const MAX_DECODED_BYTES = 128 * 1024 * 1024; + +const protocolFailure = (reason: string): never => { + throw new AnalysisProtocolError(`Invalid packet-capture output: ${reason}`); +}; + +export interface CapinfosMetadata { + readonly fileType: string; + readonly encapsulation: string; + readonly packetCount: number; + readonly captureStart: string | null; + readonly captureEnd: string | null; + readonly captureDurationSeconds: number | null; + readonly snapshotLength: number | null; + readonly sha256: string | null; +} + +/** capinfos -M with a C locale provides exact numbers instead of "20 k". */ +export const parseCapinfosOutput = (stdout: string): CapinfosMetadata => { + const lines = new Map(); + for (const line of stdout.split(/\r?\n/u)) { + const at = line.indexOf(":"); + if (at >= 0 && !/^\s/u.test(line)) + lines.set(line.slice(0, at), line.slice(at + 1).trim()); + } + const fileType = lines.get("File type"); + const encapsulation = lines.get("File encapsulation"); + const countText = lines.get("Number of packets"); + if ( + !fileType || + !encapsulation || + countText === undefined || + !/^\d+$/u.test(countText) + ) + return protocolFailure( + "capinfos did not provide machine-readable type, encapsulation and packet count", + ); + const packetCount = Number(countText); + if (!Number.isSafeInteger(packetCount)) + return protocolFailure("packet count is not representable"); + const durationText = lines.get("Capture duration"); + const duration = + durationText === undefined + ? null + : /^(\d+(?:\.\d+)?)(?:\s+seconds?)?$/u.exec(durationText); + const snapshot = /^file hdr:\s*(\d+) bytes$/u.exec( + lines.get("Packet size limit") ?? "", + ); + const snapshotLength = snapshot === null ? null : Number(snapshot[1]); + const sha256 = lines.get("SHA256") ?? null; + if (sha256 !== null && !/^[0-9a-f]{64}$/u.test(sha256)) + return protocolFailure("capinfos SHA256 is malformed"); + return { + fileType, + encapsulation, + packetCount, + captureStart: lines.get("Earliest packet time") ?? null, + captureEnd: lines.get("Latest packet time") ?? null, + captureDurationSeconds: duration === null ? null : Number(duration[1]), + snapshotLength: + snapshotLength !== null && + Number.isSafeInteger(snapshotLength) && + snapshotLength > 0 + ? snapshotLength + : null, + sha256, + }; +}; + +export interface TsharkFields { + readonly endpoints: { address: string; port: null; packet_count: number }[]; + readonly endpointCount: number; + readonly dnsQueries: { + query: string; + response_addresses: string[]; + response_address_count: number; + }[]; + readonly dnsQueryCount: number; + readonly tlsSniValues: string[]; + readonly tlsSniCount: number; + readonly protocolDistribution: { protocol: string; packet_count: number }[]; + readonly protocolCount: number; + readonly httpRequestUris: string[]; + readonly httpRequestUriCount: number; + readonly httpUriRedactionCount: number; + readonly unattributedDnsAddresses: string[]; + readonly unattributedDnsAddressCount: number; + readonly unattributedDnsPackets: number; + readonly packetCount: number; + readonly partial: boolean; +} + +const object = (value: unknown): value is Record => + typeof value === "object" && value !== null && !Array.isArray(value); + +/** Validate the fixed JSON field representation before retaining aggregate facts. */ +export const parseTsharkJsonOutput = async ( + stdout: string, + signal?: AbortSignal, +): Promise => { + signal?.throwIfAborted(); + if (Buffer.byteLength(stdout) > TSHARK_JSON_MAX_BYTES) + throw new AnalysisResourceConstraintError( + OPERATION, + "memory", + "tshark JSON exceeds the bounded decoding budget", + { max_json_bytes: TSHARK_JSON_MAX_BYTES }, + ); + // The selected-field protocol has five container levels. Reject deeper + // graphs before materializing them; quoted brackets are string data. + let depth = 0, + quoted = false, + escaped = false; + let decodedBytes = stdout.length * 2; + for (let index = 0; index < stdout.length; index += 1) { + if (index % 65536 === 0) { + await yieldToEventLoop(); + signal?.throwIfAborted(); + } + const char = stdout[index]; + if (quoted) { + if (escaped) escaped = false; + else if (char === "\\") escaped = true; + else if (char === '"') quoted = false; + } else if (char === '"') { + quoted = true; + decodedBytes += 64; + } else if (char === "{" || char === "[") { + decodedBytes += 128; + if (++depth > 5) protocolFailure("unexpected JSON nesting"); + } else if (char === "}" || char === "]") depth -= 1; + else if (char === ":" || char === ",") decodedBytes += 128; + if (decodedBytes > MAX_DECODED_BYTES) + throw new AnalysisResourceConstraintError( + OPERATION, + "memory", + "tshark JSON expansion exceeds the decoding budget", + { max_decoded_bytes: MAX_DECODED_BYTES }, + ); + } + let packets: unknown; + try { + packets = JSON.parse(stdout) as unknown; + } catch (cause) { + throw new AnalysisProtocolError("tshark emitted malformed JSON", { cause }); + } + if (!Array.isArray(packets)) + return protocolFailure("tshark packet root must be an array"); + const endpoints = new Map(); + const dns = new Map>(); + const sni = new Set(); + const protocols = new Map(); + const uris = new Set(); + const observedUris = new Set(); + let uriRedactions = 0; + const unattributed = new Set(); + let unattributedPackets = 0; + let retainedBytes = 0; + const charge = (value: string): void => { + retainedBytes += 192 + 2 * Buffer.byteLength(value); + if (retainedBytes > MAX_RETAINED_BYTES) + throw new AnalysisResourceConstraintError( + OPERATION, + "memory", + "Packet inventory exceeds the retained-fact budget", + { max_retained_bytes: MAX_RETAINED_BYTES }, + ); + }; + const add = (set: Set, value: string): void => { + if (!set.has(value)) { + charge(value); + set.add(value); + } + }; + for (let index = 0; index < packets.length; index += 1) { + if (index % 256 === 0) { + await yieldToEventLoop(); + signal?.throwIfAborted(); + } + const packet: unknown = packets[index]; + if ( + !object(packet) || + !object(packet._source) || + !object(packet._source.layers) + ) + return protocolFailure( + `packet ${String(index + 1)} lacks _source.layers`, + ); + const layers = packet._source.layers; + for (const [key, value] of Object.entries(layers)) + if ( + !Array.isArray(value) || + !value.every((item) => typeof item === "string") + ) + return protocolFailure( + `packet ${String(index + 1)} field ${key} must be an array of strings`, + ); + const field = (name: string): readonly string[] => + (layers[name] as string[] | undefined) ?? []; + const frame = field("frame.number"); + if ( + frame.length !== 1 || + !/^\d+$/u.test(frame[0]!) || + Number(frame[0]) !== index + 1 + ) + return protocolFailure( + "frame inventory is missing, duplicated or out of order", + ); + if ( + field("frame.protocols").length !== 1 || + field("frame.protocols")[0] === "" + ) + return protocolFailure( + "packet protocol inventory is missing or ambiguous", + ); + // Count one occurrence per packet even for loopback or nested IP layers. + for (const address of new Set([ + ...field("ip.src"), + ...field("ip.dst"), + ...field("ipv6.src"), + ...field("ipv6.dst"), + ])) { + if (address === "") continue; + if (!endpoints.has(address)) charge(address); + endpoints.set(address, (endpoints.get(address) ?? 0) + 1); + } + const questions = new Set( + field("dns.qry.name").filter((value) => value !== ""), + ); + for (const question of questions) + if (!dns.has(question)) { + charge(question); + dns.set(question, new Set()); + } + const addresses = [...field("dns.a"), ...field("dns.aaaa")].filter( + (value) => value !== "", + ); + const response = field("dns.flags.response"); + if (addresses.length > 0) { + if ( + questions.size === 1 && + response.length > 0 && + response.every((value) => value === "True" || value === "1") + ) { + const values = dns.get(questions.values().next().value!)!; + for (const address of addresses) add(values, address); + } else { + unattributedPackets += 1; + for (const address of addresses) add(unattributed, address); + } + } + for (const value of field("tls.handshake.extensions_server_name")) + if (value !== "") add(sni, value); + for (const value of new Set(field("frame.protocols"))) { + if (value === "") continue; + if (!protocols.has(value)) charge(value); + protocols.set(value, (protocols.get(value) ?? 0) + 1); + } + for (const value of field("http.request.full_uri")) { + if (value === "" || observedUris.has(value)) continue; + add(observedUris, value); + const { credentials } = parseRecordedTransportUrl(value); + if (credentials !== null) uriRedactions += 1; + if (credentials === "ambiguous") continue; + add( + uris, + credentials === null + ? value + : value.slice(0, credentials.start) + value.slice(credentials.end), + ); + } + } + signal?.throwIfAborted(); + return { + endpoints: [...endpoints] + .sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0])) + .map(([address, packet_count]) => ({ + address, + port: null, + packet_count, + })), + endpointCount: endpoints.size, + dnsQueries: [...dns].map(([query, addresses]) => ({ + query, + response_addresses: [...addresses], + response_address_count: addresses.size, + })), + dnsQueryCount: dns.size, + tlsSniValues: [...sni], + tlsSniCount: sni.size, + protocolDistribution: [...protocols] + .sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0])) + .map(([protocol, packet_count]) => ({ protocol, packet_count })), + protocolCount: protocols.size, + httpRequestUris: [...uris], + httpRequestUriCount: observedUris.size, + httpUriRedactionCount: uriRedactions, + unattributedDnsAddresses: [...unattributed], + unattributedDnsAddressCount: unattributed.size, + unattributedDnsPackets: unattributedPackets, + packetCount: packets.length, + partial: observedUris.size > uris.size || unattributedPackets > 0, + }; +}; diff --git a/src/pcap/PcapProvider.ts b/src/pcap/PcapProvider.ts new file mode 100644 index 000000000..7228d694f --- /dev/null +++ b/src/pcap/PcapProvider.ts @@ -0,0 +1,493 @@ +import type { + ExecutionOptions, + ProviderAvailability, +} from "../application/AnalysisProvider.js"; +import { + createAnalysisExecution, + type AnalysisExecution, +} from "../application/AnalysisProvider.js"; +import { + pcapResultSchemas, + type PcapRequest, + type PcapOperation, +} from "../domain/pcap/pcapAnalysis.js"; +import { + AnalysisCancelledError, + AnalysisCapabilityUnavailableError, + AnalysisProtocolError, +} from "../domain/analysisErrorCore.js"; +import { AnalysisError } from "../domain/analysisErrorBase.js"; +import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js"; +import { err, ok, type Result } from "../domain/result.js"; +import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js"; +import { + PcapConfigurationFailure, + inspectPcapClient, + resolvePcapTools, + pcapConfigurationAvailability, + type PcapToolSelection, +} from "./PcapConfiguration.js"; +import { + parseCapinfosOutput, + parseTsharkJsonOutput, + TSHARK_JSON_MAX_BYTES, +} from "./PcapOutput.js"; +import { runPcapCommand, type PcapCommandLimits } from "./PcapCommand.js"; +import { ProviderCleanupError } from "../domain/providerCleanupError.js"; +import { createPcapEvidence } from "../domain/pcap/pcapEvidence.js"; +import { preparePcapNativeBound } from "./PcapNativeBound.js"; +import { PcapSnapshot } from "./PcapSnapshot.js"; +import { readStableArtifact } from "../artifacts/readStableArtifact.js"; +import { ArtifactAnalysisResources } from "../process/ArtifactAnalysisResources.js"; +import { readLinuxResourceLimits } from "../process/LinuxResourceLimits.js"; + +export const PCAP_PROVIDER_IDENTITY = { + id: "pcap", + name: "Packet capture analysis", + version: null, +} as const; +const FIELDS = [ + "frame.number", + "frame.protocols", + "ip.src", + "ip.dst", + "ipv6.src", + "ipv6.dst", + "dns.flags.response", + "dns.qry.name", + "dns.a", + "dns.aaaa", + "tls.handshake.extensions_server_name", + "http.request.full_uri", +]; + +export interface PcapProviderOptions { + readonly environment?: Readonly>; +} + +/** Inspect saved captures through bounded, owned tool processes and one stable input copy. */ +export class PcapProvider { + readonly #resources = new ArtifactAnalysisResources( + "pcap", + "inspect_pcap_capture", + readStableArtifact, + readLinuxResourceLimits, + { + message: "A packet summary is already active; retry after it completes.", + details: { max_concurrent_inspections: 1 }, + }, + ); + private readonly environment: Readonly>; + private readonly snapshots = new Set(); + private readonly commands = new Map< + ProviderProcessSupervisor, + { snapshot: PcapSnapshot | undefined; resources: readonly string[] } + >(); + + constructor(options: PcapProviderOptions = {}) { + this.environment = options.environment ?? process.env; + } + + private run( + operation: PcapOperation, + command: string, + args: readonly string[], + limits: PcapCommandLimits, + signal: AbortSignal | undefined, + snapshot?: PcapSnapshot, + inputPath?: string, + ) { + return runPcapCommand({ + operation, + command, + arguments: args, + limits, + signal, + environment: + snapshot === undefined + ? this.environment + : { + ...this.environment, + WIRESHARK_CONFIG_DIR: snapshot.configurationPath, + }, + ...(inputPath === undefined ? {} : { inputPath }), + ...(inputPath === undefined || this.nativeBound === undefined + ? {} + : { nativeBound: this.nativeBound }), + retain: (owner, resources) => { + this.commands.set(owner, { snapshot, resources }); + }, + }); + } + + private nativeBound: Parameters[0]["nativeBound"]; + private nativeLimitations: readonly string[] = []; + + private async prepareCaptureBound(): Promise { + const prepared = await preparePcapNativeBound( + this.environment, + this.#resources, + ); + this.nativeBound = prepared.nativeBound; + this.nativeLimitations = prepared.limitations; + } + + private async client(operation: PcapOperation, signal?: AbortSignal) { + const selection = resolvePcapTools(this.environment); + const identity = await inspectPcapClient( + selection, + async (command) => + ( + await this.run( + operation, + command, + ["--version"], + { timeoutMs: 20_000, maxOutputBytes: 64 * 1024 }, + signal, + ) + ).stdout, + ); + for (const [name, version] of [ + ["tshark", identity.tsharkVersion], + ["capinfos", identity.capinfosVersion], + ] as const) { + if (version === null) + throw new PcapConfigurationFailure( + `${name}_version_unresolved`, + `The selected ${name} command did not report a version. Set REA_${name.toUpperCase()}_COMMAND to its absolute executable path.`, + ); + } + return { + selection, + client: { + tshark_command: selection.tshark.command, + tshark_source: selection.tshark.source, + tshark_version: identity.tsharkVersion, + capinfos_command: selection.capinfos.command, + capinfos_source: selection.capinfos.source, + capinfos_version: identity.capinfosVersion, + }, + }; + } + + async inspectAvailability( + signal?: AbortSignal, + ): Promise { + const result = await this.#resources.run( + async () => ok(await this.inspectAvailabilityAdmitted(signal)), + signal, + ); + if (!result.ok) throw result.error; + return result.value; + } + + private async inspectAvailabilityAdmitted( + signal?: AbortSignal, + ): Promise { + try { + const { client } = await this.client("inspect_pcap_client", signal); + if (process.platform !== "linux" && process.platform !== "win32") + return { + status: "unavailable", + code: "unsupported_host", + reason: + "No enforceable native memory boundary is available for capture summaries on this host.", + diagnostics: client, + }; + await this.prepareCaptureBound(); + return { + status: "available", + code: null, + reason: null, + diagnostics: client, + }; + } catch (cause) { + if (signal?.aborted || cause instanceof AnalysisCancelledError) + throw new AnalysisCancelledError("inspect_pcap_client", { cause }); + if (cause instanceof PcapConfigurationFailure) + return pcapConfigurationAvailability(cause); + return { + status: "unavailable", + code: + cause instanceof AnalysisCapabilityUnavailableError + ? "executable_missing" + : "version_unresolved", + reason: cause instanceof Error ? cause.message : String(cause), + diagnostics: {}, + }; + } + } + + private async discard( + snapshot: PcapSnapshot, + ): Promise { + try { + if ( + [...this.commands.values()].some( + (command) => command.snapshot === snapshot, + ) + ) + throw new Error( + "An unreleased command still owns the capture snapshot", + ); + await snapshot.close(); + this.snapshots.delete(snapshot); + return undefined; + } catch (cause) { + return new ProviderCleanupError( + "pcap", + [snapshot.path], + { reason: String(cause) }, + { operation: "inspect_pcap_capture", cause }, + ); + } + } + + async close(): Promise { + const failures: unknown[] = []; + try { + await this.#resources.close(); + } catch (cause) { + failures.push(cause); + } + for (const owner of this.commands.keys()) { + try { + const result = await owner.stop(); + if (result.status === "incomplete") failures.push(result.reason); + else if (result.status === "not-owned") + failures.push("Command ownership remains unresolved"); + else this.commands.delete(owner); + } catch (cause) { + failures.push(cause); + } + } + for (const snapshot of this.snapshots) { + const failure = await this.discard(snapshot); + if (failure !== undefined) failures.push(failure); + } + if (failures.length > 0) + throw new ProviderCleanupError( + "pcap", + [...this.snapshots] + .map((snapshot) => snapshot.path) + .concat( + [...this.commands.values()].flatMap((command) => command.resources), + ), + { reason: "Owned process or snapshot cleanup failed" }, + { operation: "pcap.close", cause: new AggregateError(failures) }, + ); + } + + async execute( + request: PcapRequest, + options?: ExecutionOptions, + ): Promise> { + return this.#resources.run( + () => this.executeAdmitted(request, options), + options?.signal, + ); + } + + private async executeAdmitted( + request: PcapRequest, + options?: ExecutionOptions, + ): Promise> { + const signal = options?.signal; + let snapshot: PcapSnapshot | undefined; + let result: Result; + try { + signal?.throwIfAborted(); + const { selection, client } = await this.client( + request.operation, + signal, + ); + const provider = { + ...PCAP_PROVIDER_IDENTITY, + version: client.tshark_version, + }; + if (request.operation === "inspect_pcap_client") + result = ok( + createAnalysisExecution( + pcapResultSchemas.inspect_pcap_client.parse({ client }), + provider, + ), + ); + else { + await this.prepareCaptureBound(); + snapshot = await PcapSnapshot.create(); + this.snapshots.add(snapshot); + const identity = await snapshot.capture(request.input.path, signal); + const captured = await this.inspectCapture( + selection, + snapshot, + request.input.path, + identity, + signal, + ); + const data = pcapResultSchemas.inspect_pcap_capture.parse({ + client, + ...captured.result, + }); + result = ok( + createAnalysisExecution(data, provider, { + subject: { + path: request.input.path, + format: "file", + sha256: identity.sha256, + }, + limitations: captured.limitations, + }), + ); + } + } catch (cause) { + result = err( + cause instanceof AnalysisError + ? cause + : signal?.aborted + ? new AnalysisCancelledError(request.operation, { cause }) + : cause instanceof PcapConfigurationFailure + ? new AnalysisCapabilityUnavailableError( + "pcap", + request.operation, + cause.message, + { cause }, + ) + : new AnalysisProtocolError( + `Packet-capture analysis failed: ${cause instanceof Error ? cause.message : String(cause)}`, + { cause }, + ), + ); + } + if (signal?.aborted && result.ok) + result = err( + new AnalysisCancelledError(request.operation, { + cause: signal.reason, + partialObservation: createPcapEvidence(request, result.value), + }), + ); + if (snapshot !== undefined) { + const cleanup = await this.discard(snapshot); + if (signal?.aborted && result.ok) + result = err( + new AnalysisCancelledError(request.operation, { + cause: signal.reason, + partialObservation: createPcapEvidence(request, result.value), + }), + ); + if (cleanup !== undefined) + result = result.ok + ? err( + cleanup.retainPartialObservation( + createPcapEvidence(request, result.value), + ), + ) + : err( + analysisErrorWithCleanupFailure( + result.error, + cleanup, + request.operation, + ), + ); + } + return result; + } + + private async inspectCapture( + selection: PcapToolSelection, + snapshot: PcapSnapshot, + path: string, + identity: { bytes: number; sha256: string }, + signal: AbortSignal | undefined, + ) { + const info = await this.run( + "inspect_pcap_capture", + selection.capinfos.command, + ["-M", "--", snapshot.path], + { timeoutMs: 60_000, maxOutputBytes: 64 * 1024 }, + signal, + snapshot, + path, + ); + const metadata = parseCapinfosOutput(info.stdout); + if (metadata.sha256 !== null && metadata.sha256 !== identity.sha256) + throw new AnalysisProtocolError( + "capinfos SHA256 disagrees with the owned capture snapshot", + ); + const dissected = await this.run( + "inspect_pcap_capture", + selection.tshark.command, + [ + "-n", + "-r", + snapshot.path, + "-T", + "json", + "--no-duplicate-keys", + `--temp-dir=${snapshot.configurationPath}`, + ...FIELDS.flatMap((field) => ["-e", field]), + ], + { timeoutMs: 300_000, maxOutputBytes: TSHARK_JSON_MAX_BYTES }, + signal, + snapshot, + path, + ); + const fields = await parseTsharkJsonOutput(dissected.stdout, signal); + await snapshot.verify(identity, path, signal); + if (fields.packetCount !== metadata.packetCount) + throw new AnalysisProtocolError( + `capinfos counted ${String(metadata.packetCount)} packets but tshark returned ${String(fields.packetCount)}`, + ); + return { + result: { + target: { path, ...identity }, + file_type: metadata.fileType, + encapsulation: metadata.encapsulation, + packet_count: metadata.packetCount, + capture_start: metadata.captureStart, + capture_end: metadata.captureEnd, + capture_duration_seconds: metadata.captureDurationSeconds, + diagnostics: { + capinfos_stderr: info.stderr, + tshark_stderr: dissected.stderr, + }, + snapshot_length: metadata.snapshotLength, + endpoints: fields.endpoints, + endpoint_count: fields.endpointCount, + dns_queries: fields.dnsQueries, + dns_query_count: fields.dnsQueryCount, + tls_sni_values: fields.tlsSniValues, + tls_sni_count: fields.tlsSniCount, + protocol_distribution: fields.protocolDistribution, + protocol_count: fields.protocolCount, + http_request_uris: fields.httpRequestUris, + http_request_uri_count: fields.httpRequestUriCount, + http_uri_redaction_count: fields.httpUriRedactionCount, + unattributed_dns_addresses: fields.unattributedDnsAddresses, + unattributed_dns_address_count: fields.unattributedDnsAddressCount, + unattributed_dns_packet_count: fields.unattributedDnsPackets, + coverage: fields.partial ? "partial" : "complete", + }, + limitations: [ + ...this.nativeLimitations, + "Both tools read one owned snapshot; target identity comes from its admitted source descriptor", + "DNS address values are associated only with a single question in the same response packet; values may include additional DNS records, not only final resolution answers", + "Only bytes present in the saved capture are examined; encryption and capture truncation can hide higher-level fields", + ...(fields.partial + ? [ + "One or more projected categories were capped, credential-bearing URIs were omitted or coalesced, or DNS address attribution remains unresolved; counts retain the examined totals", + ] + : []), + ...(fields.httpUriRedactionCount > 0 + ? [ + "HTTP transport credentials were excluded; ambiguous credential spans cause the URI to be omitted", + ] + : []), + ...(fields.unattributedDnsPackets > 0 + ? [ + `DNS address attribution was unresolved in ${String(fields.unattributedDnsPackets)} packet(s); unattributed addresses are retained separately`, + ] + : []), + ], + }; + } +} diff --git a/src/pcap/PcapSnapshot.ts b/src/pcap/PcapSnapshot.ts new file mode 100644 index 000000000..08f1ea552 --- /dev/null +++ b/src/pcap/PcapSnapshot.ts @@ -0,0 +1,269 @@ +import { createHash } from "node:crypto"; +import { chmod, open, stat, mkdir } from "node:fs/promises"; +import { join } from "node:path"; +import { AnalysisError } from "../domain/analysisErrorBase.js"; +import { + AnalysisAccessDeniedError, + AnalysisArtifactChangedError, + AnalysisCancelledError, + AnalysisCapabilityUnavailableError, + AnalysisInputError, + AnalysisResourceConstraintError, +} from "../domain/analysisErrorCore.js"; +import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js"; +import { + openRegularFile, + RegularFileAdmissionFailure, + sameRegularFileState, +} from "../filesystem/RegularFile.js"; +import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js"; +import { PrivateRuntimeRootUnavailableError } from "../process/PrivateRuntimeRootUnavailableError.js"; + +const OPERATION = "inspect_pcap_capture"; +const MAX_CAPTURE_BYTES = 2 * 1024 ** 3; + +/** An owned capture copy and every descriptor needed to release it safely. */ +export class PcapSnapshot { + private readonly handles = new Set(); + readonly path: string; + readonly configurationPath: string; + private constructor(private readonly root: PrivateRuntimeRoot) { + this.path = join(root.path, "capture.pcap"); + this.configurationPath = join(root.path, "configuration"); + } + + static async create(): Promise { + try { + return new PcapSnapshot( + await PrivateRuntimeRoot.create({ prefix: "rea-pcap-" }), + ); + } catch (cause) { + if (cause instanceof PrivateRuntimeRootUnavailableError) + throw new AnalysisCapabilityUnavailableError( + "pcap", + OPERATION, + cause.message, + { cause }, + ); + throw new AnalysisResourceConstraintError( + OPERATION, + "file-size", + "Cannot allocate the owned capture workspace", + null, + { cause }, + ); + } + } + + private own(handle: OwnedFileHandle): OwnedFileHandle { + this.handles.add(handle); + return handle; + } + private async release(handle: OwnedFileHandle): Promise { + await handle.close(); + this.handles.delete(handle); + } + + async capture( + path: string, + signal?: AbortSignal, + ): Promise<{ bytes: number; sha256: string }> { + const sourceRead = async (read: () => Promise): Promise => { + try { + return await read(); + } catch (cause) { + if (signal?.aborted) + throw new AnalysisCancelledError(OPERATION, { cause }); + if (cause instanceof AnalysisError) throw cause; + if ( + cause instanceof Error && + "code" in cause && + (cause.code === "EACCES" || cause.code === "EPERM") + ) + throw new AnalysisAccessDeniedError(OPERATION, path, cause.code, { + cause, + }); + throw new AnalysisInputError(OPERATION, { cause }, [ + { + path: ["path"], + reason: "invalid_value", + message: `Cannot read selected capture ${path}: ${cause instanceof Error ? cause.message : String(cause)}`, + }, + ]); + } + }; + try { + signal?.throwIfAborted(); + await mkdir(this.configurationPath, { mode: 0o700 }); + const source = await sourceRead(async () => { + try { + return this.own( + new OwnedFileHandle( + await openRegularFile(path, { symlinks: "follow", signal }), + ), + ); + } catch (cause) { + if (cause instanceof RegularFileAdmissionFailure) { + this.own(cause.owner); + throw cause.cause; + } + throw cause; + } + }); + const before = await sourceRead(() => + source.handle.stat({ bigint: true }), + ); + if (before.size > BigInt(MAX_CAPTURE_BYTES)) + throw new AnalysisResourceConstraintError( + OPERATION, + "file-size", + "Capture exceeds the owned-snapshot disk budget", + { max_capture_bytes: MAX_CAPTURE_BYTES }, + ); + const destination = this.own( + new OwnedFileHandle(await open(this.path, "wx", 0o600)), + ); + const buffer = Buffer.alloc(64 * 1024); + const hash = createHash("sha256"); + let copied = 0; + while (copied <= Number(before.size)) { + signal?.throwIfAborted(); + const { bytesRead } = await sourceRead(() => + source.handle.read( + buffer, + 0, + Math.min(buffer.length, Number(before.size) - copied + 1), + copied, + ), + ); + signal?.throwIfAborted(); + if (bytesRead === 0) break; + copied += bytesRead; + if (BigInt(copied) > before.size) + throw new AnalysisArtifactChangedError( + OPERATION, + path, + "Capture grew during acquisition", + ); + const chunk = buffer.subarray(0, bytesRead); + await destination.handle.writeFile(chunk); + hash.update(chunk); + } + const after = await sourceRead(() => + source.handle.stat({ bigint: true }), + ); + const selected = await stat(path, { bigint: true }).catch( + (cause: unknown) => { + if ( + cause instanceof Error && + "code" in cause && + (cause.code === "EACCES" || cause.code === "EPERM") + ) + throw new AnalysisAccessDeniedError(OPERATION, path, cause.code, { + cause, + }); + throw new AnalysisArtifactChangedError( + OPERATION, + path, + "Selected path became unavailable during acquisition", + { cause }, + ); + }, + ); + if ( + BigInt(copied) !== before.size || + !sameRegularFileState(before, after) || + !sameRegularFileState(before, selected) + ) + throw new AnalysisArtifactChangedError( + OPERATION, + path, + "Capture identity or contents changed during acquisition", + ); + await this.release(source); + await this.release(destination); + await chmod(this.path, 0o400); + signal?.throwIfAborted(); + return { bytes: copied, sha256: hash.digest("hex") }; + } catch (cause) { + if (signal?.aborted) + throw new AnalysisCancelledError(OPERATION, { cause }); + if (cause instanceof AnalysisError) throw cause; + throw new AnalysisResourceConstraintError( + OPERATION, + "file-size", + "Owned capture snapshot I/O failed", + null, + { cause }, + ); + } + } + + /** Recheck byte identity after both tools; a read-only mode alone is not proof. */ + async verify( + expected: { bytes: number; sha256: string }, + selectedPath: string, + signal?: AbortSignal, + ): Promise { + try { + const file = this.own( + new OwnedFileHandle( + await openRegularFile(this.path, { symlinks: "reject", signal }), + ), + ); + const before = await file.handle.stat({ bigint: true }); + const buffer = Buffer.alloc(64 * 1024); + const hash = createHash("sha256"); + let bytes = 0; + while (bytes <= expected.bytes) { + signal?.throwIfAborted(); + const read = await file.handle.read( + buffer, + 0, + Math.min(buffer.length, expected.bytes - bytes + 1), + bytes, + ); + if (read.bytesRead === 0) break; + bytes += read.bytesRead; + hash.update(buffer.subarray(0, read.bytesRead)); + } + const after = await file.handle.stat({ bigint: true }); + const selected = await stat(this.path, { bigint: true }); + if ( + bytes !== expected.bytes || + hash.digest("hex") !== expected.sha256 || + !sameRegularFileState(before, after) || + !sameRegularFileState(before, selected) + ) + throw new AnalysisArtifactChangedError( + OPERATION, + selectedPath, + "Owned packet snapshot changed during tool analysis.", + ); + await this.release(file); + } catch (cause) { + if (signal?.aborted) + throw new AnalysisCancelledError(OPERATION, { cause }); + if (cause instanceof RegularFileAdmissionFailure) this.own(cause.owner); + if (cause instanceof AnalysisError) throw cause; + throw new AnalysisArtifactChangedError( + OPERATION, + selectedPath, + "Cannot establish the owned packet snapshot's post-analysis identity.", + { cause }, + ); + } + } + + async close(): Promise { + const results = await Promise.allSettled( + [...this.handles].map((handle) => this.release(handle)), + ); + const failures = results.flatMap((result) => + result.status === "rejected" ? [result.reason as unknown] : [], + ); + if (failures.length > 0) + throw new AggregateError(failures, "Capture descriptor cleanup failed"); + await this.root.close(); + } +} diff --git a/src/process/DarwinProcessRunTokenReader.ts b/src/process/DarwinProcessRunTokenReader.ts index 5aafcd787..335e72fbe 100644 --- a/src/process/DarwinProcessRunTokenReader.ts +++ b/src/process/DarwinProcessRunTokenReader.ts @@ -74,7 +74,9 @@ const sourceFiles = [ export const SWIFTC_INTERRUPT = "SIGINT"; /** Deadline for the one-shot helper compilation. */ -const COMPILER_TIMEOUT_MS = 60_000; +// A fresh Swift SDK cache took 88 seconds on the supported Intel macOS VM. +// Keep compilation bounded independently of short provider version probes. +const COMPILER_TIMEOUT_MS = 120_000; /** Bound on compiler diagnostics embedded in the inspection failure text. */ const STDERR_EXCERPT_LIMIT = 512; @@ -253,7 +255,7 @@ export type DarwinProcessReaderCompiler = export const createDarwinProcessRunTokenReader = ( options: { readonly compiler?: DarwinProcessReaderCompiler; - /** Deadline for the helper compilation; the default keeps verification fast. */ + /** Deadline for helper compilation, independent of provider execution. */ readonly compileTimeoutMs?: number; /** * Caller-owned directory holding the compiler's module cache. Unset keeps diff --git a/src/server/createServer.ts b/src/server/createServer.ts index 4929142c4..294984339 100644 --- a/src/server/createServer.ts +++ b/src/server/createServer.ts @@ -84,6 +84,7 @@ import type { FirmwareAnalysisPort } from "../application/firmware/FirmwareAnaly import { createFirmwareAnalysisProvider } from "../composition/firmware.js"; import { registerAndroidTools } from "./registerAndroidTools.js"; import { registerApktoolTools } from "./registerApktoolTools.js"; +import { registerPcapTools } from "./registerPcapTools.js"; import { registerFlutterTools } from "./registerFlutterTools.js"; import { registerJebTools } from "./registerJebTools.js"; import { registerAdbTools } from "./registerAdbTools.js"; @@ -91,11 +92,14 @@ import { AndroidAnalysisService } from "../application/android/AndroidAnalysisSe import { JebAnalysisService } from "../application/jeb/JebAnalysisService.js"; import type { AndroidAnalysisPort } from "../application/android/AndroidAnalysisPort.js"; import type { ApktoolResourceAnalysisPort } from "../application/apktool/ApktoolResourceAnalysisPort.js"; +import type { PcapAnalysisPort } from "../application/pcap/PcapAnalysisPort.js"; +import { PcapAnalysisService } from "../application/pcap/PcapAnalysisService.js"; import { ApktoolResourceAnalysisService } from "../application/apktool/ApktoolResourceAnalysisService.js"; import type { FlutterBuildAnalysisPort } from "../application/flutter/FlutterBuildAnalysisPort.js"; import { FlutterBuildAnalysisService } from "../application/flutter/FlutterBuildAnalysisService.js"; import { createAndroidAnalysisProvider } from "../composition/android.js"; import { createApktoolResourceAnalysisProvider } from "../composition/apktool.js"; +import { createPcapAnalysisProvider } from "../composition/pcap.js"; import { createFlutterBuildAnalysisProvider } from "../composition/flutter.js"; import type { JebAnalysisPort } from "../application/jeb/JebAnalysisPort.js"; import { createJebAnalysisProvider } from "../composition/jeb.js"; @@ -146,6 +150,7 @@ export interface CreateServerOptions { readonly webNetworkCapture?: WebNetworkCaptureService; readonly androidAnalysis?: AndroidAnalysisPort; readonly apktoolAnalysis?: ApktoolResourceAnalysisPort; + readonly pcapAnalysis?: PcapAnalysisPort; readonly flutterAnalysis?: FlutterBuildAnalysisPort; readonly jebAnalysis?: JebAnalysisPort; readonly adbDeviceAnalysis?: AdbDeviceAnalysisPort; @@ -303,6 +308,7 @@ export const createServer = ( ); const android = options.androidAnalysis ?? createAndroidAnalysisProvider(environment); + const pcap = options.pcapAnalysis ?? createPcapAnalysisProvider(environment); const flutter = options.flutterAnalysis ?? createFlutterBuildAnalysisProvider(); const fridaInstrumentation = @@ -333,6 +339,7 @@ export const createServer = ( jeb.close(), adbDevice.close(), flutter.close(), + pcap.close(), evmInterface.close(), sqliteDatabase.close(), binaryLayout.close(), @@ -438,6 +445,14 @@ export const createServer = ( recordEvidence, analysisAdmission, ); + registerPcapTools( + server, + new PcapAnalysisService(pcap), + toolLogger, + recordEvidence, + analysisAdmission, + evidenceById, + ); registerFlutterTools( server, new FlutterBuildAnalysisService(flutter), diff --git a/src/server/registerPcapTools.ts b/src/server/registerPcapTools.ts new file mode 100644 index 000000000..b2ebd239a --- /dev/null +++ b/src/server/registerPcapTools.ts @@ -0,0 +1,83 @@ +import type { EvidenceMcpServer } from "./EvidenceMcpServer.js"; +import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js"; +import { PcapAnalysisService } from "../application/pcap/PcapAnalysisService.js"; +import { toolContract } from "../contracts/toolContracts.js"; +import type { WithAdmittedAnalysis } from "./analysisAdmission.js"; +import type { Logger } from "pino"; +import { + createContractToolHandler, + createEvidenceToolHandler, +} from "./contractToolHandler.js"; +import { + toolRegistrationOptions, + applicationValidatedInputSchema, +} from "./toolRegistrationOptions.js"; +import type { EvidenceLookup } from "../application/EvidenceInputResolver.js"; +import { + requireAnalysisViewRetention, + summarizeRetainedAnalysis, +} from "../application/analysisView/AnalysisViewService.js"; +import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js"; +import { err } from "../domain/result.js"; + +/** Bind pcap handlers to their exact named contracts through admission. */ +export const registerPcapTools = ( + server: EvidenceMcpServer, + service: PcapAnalysisService, + logger: Logger, + recordEvidence?: EvidenceWriter["recordEvidence"], + withAdmittedAnalysis?: WithAdmittedAnalysis, + evidenceById?: EvidenceLookup, +): void => { + const handler = createContractToolHandler( + server, + service, + logger, + recordEvidence, + withAdmittedAnalysis, + ); + const clientContract = toolContract("inspect_pcap_client"); + server.registerTool( + clientContract.name, + toolRegistrationOptions(clientContract), + handler(clientContract), + ); + const contract = toolContract("inspect_pcap_capture"); + const registration = toolRegistrationOptions(contract); + const capture = createEvidenceToolHandler( + server, + async (rawInput, options) => { + const parsed = contract.inputSchema.safeParse(rawInput); + if (!parsed.success) + return err( + analysisInputErrorFromIssues( + contract.name, + parsed.error.issues, + rawInput, + { cause: parsed.error }, + ), + ); + const { detail, ...input } = parsed.data; + const retention = { recordEvidence, evidenceById }; + if (detail === "summary") { + const ready = requireAnalysisViewRetention(contract.name, retention); + if (!ready.ok) return ready; + } + const result = await service.execute(contract.name, input, options); + return detail === "summary" && result.ok + ? summarizeRetainedAnalysis(result.value, retention) + : result; + }, + logger, + recordEvidence, + withAdmittedAnalysis, + ); + server.registerTool( + contract.name, + { + ...registration, + inputSchema: applicationValidatedInputSchema(registration.inputSchema), + }, + capture(contract), + ); +}; diff --git a/tests/boundary/cli/pcapCli.test.ts b/tests/boundary/cli/pcapCli.test.ts new file mode 100644 index 000000000..26e253455 --- /dev/null +++ b/tests/boundary/cli/pcapCli.test.ts @@ -0,0 +1,23 @@ +import { expect, it } from "vitest"; + +import { serveCli } from "./cliServeHarness.js"; + +it.each(["inspect-pcap-client", "inspect-pcap-capture"])( + "%s advertises a schema through the public CLI", + async (command) => { + const result = await serveCli([command, "--schema", "--json"]); + expect(result.exitCode).toBe(0); + expect(JSON.parse(result.stdout)).toBeTruthy(); + }, +); + +it("reports missing tools with a capability refusal", async () => { + const result = await serveCli( + ["inspect-pcap-capture", "/nonexistent/capture.pcap", "--json"], + { + REA_TSHARK_COMMAND: "/nonexistent/tshark", + REA_CAPINFOS_COMMAND: "/nonexistent/capinfos", + }, + ); + expect(result.stdout).toContain("capability_unavailable"); +}); diff --git a/tests/boundary/mcp/pcapMcp.test.ts b/tests/boundary/mcp/pcapMcp.test.ts new file mode 100644 index 000000000..d3a6d203d --- /dev/null +++ b/tests/boundary/mcp/pcapMcp.test.ts @@ -0,0 +1,118 @@ +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import { expect, it, onTestFinished } from "vitest"; + +import type { PcapAnalysisPort } from "../../../src/application/pcap/PcapAnalysisPort.js"; +import { createAnalysisExecution } from "../../../src/application/AnalysisProvider.js"; +import { assertAdvertisedMcpContracts } from "./mcpContractHarness.js"; +import { parseEvidence } from "../../../src/domain/evidence.js"; +import { ok } from "../../../src/domain/result.js"; +import { createServer } from "../../../src/server/createServer.js"; +import { createTestBinarySession } from "../../fixtures/binarySession.js"; + +const identity = { id: "pcap", name: "Packet capture analysis", version: null }; + +const clientIdentity = { + tshark_command: "/usr/bin/tshark", + tshark_source: "path", + tshark_version: "TShark (Wireshark) 4.6.6", + capinfos_command: "/usr/bin/capinfos", + capinfos_source: "path", + capinfos_version: "Capinfos (Wireshark) 4.6.6", +}; + +it("advertises exact valid pcap schemas and records inline evidence through MCP", async () => { + const provider: PcapAnalysisPort = { + async inspectAvailability() { + return { status: "available", code: null, reason: null, diagnostics: {} }; + }, + async close() {}, + async execute(request) { + if (request.operation === "inspect_pcap_client") + return ok( + createAnalysisExecution({ client: clientIdentity }, identity), + ); + return ok( + createAnalysisExecution( + { + client: clientIdentity, + target: { + path: "/captures/session.pcap", + bytes: 23279, + sha256: "a".repeat(64), + }, + file_type: "Wireshark/tcpdump/... - pcap", + encapsulation: "Linux cooked-mode capture v2", + packet_count: 50, + capture_start: "2026-10-11 14:13:40.608505", + capture_end: "2026-10-11 14:13:40.981978", + capture_duration_seconds: 0.373473, + snapshot_length: 262144, + endpoints: [ + { address: "192.168.1.100", port: null, packet_count: 30 }, + { address: "8.8.8.8", port: 53, packet_count: 5 }, + ], + endpoint_count: 2, + dns_queries: [ + { + query: "example.com", + response_addresses: ["93.184.216.34"], + response_address_count: 1, + }, + ], + dns_query_count: 1, + tls_sni_values: ["example.com"], + tls_sni_count: 1, + protocol_distribution: [ + { protocol: "ip:tcp:tls", packet_count: 25 }, + { protocol: "ip:udp:dns", packet_count: 5 }, + ], + protocol_count: 2, + http_request_uris: ["http://example.com/api"], + http_request_uri_count: 1, + http_uri_redaction_count: 0, + unattributed_dns_addresses: [], + unattributed_dns_address_count: 0, + unattributed_dns_packet_count: 0, + coverage: "complete", + }, + identity, + ), + ); + }, + }; + const session = createTestBinarySession(() => { + throw new Error("deep provider must not start"); + }); + const server = createServer( + { kind: "session", session }, + { pcapAnalysis: provider }, + ); + onTestFinished(() => server.close()); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + await server.connect(serverTransport); + const client = new Client({ name: "pcap-boundary", version: "0.0.0" }); + onTestFinished(() => client.close()); + await client.connect(clientTransport); + + const { tools } = await client.listTools(); + assertAdvertisedMcpContracts( + ["inspect_pcap_client", "inspect_pcap_capture"], + tools, + ); + + const inspected = await client.callTool({ + name: "inspect_pcap_capture", + arguments: { path: "/captures/session.pcap" }, + }); + expect(inspected.isError).not.toBe(true); + const text = (inspected.content as { readonly text?: string }[]) + .map((part) => part.text ?? "") + .join(""); + const evidence = parseEvidence(JSON.parse(text)); + expect(evidence.normalized_result).toMatchObject({ + packet_count: 50, + tls_sni_values: ["example.com"], + coverage: "complete", + }); +}); diff --git a/tests/boundary/pcap/pcapCommand.test.ts b/tests/boundary/pcap/pcapCommand.test.ts new file mode 100644 index 000000000..cda8dbf77 --- /dev/null +++ b/tests/boundary/pcap/pcapCommand.test.ts @@ -0,0 +1,53 @@ +import { setTimeout as delay } from "node:timers/promises"; +import { expect, it, onTestFinished, vi } from "vitest"; +import { runPcapCommand } from "../../../src/pcap/PcapCommand.js"; +import * as ownership from "../../../src/process/ProcessOwnershipObservation.js"; + +it("does not spend the tool execution deadline preparing native ownership", async () => { + const original = ownership.prepareProcessOwnershipInspection; + let entered!: () => void; + let release!: () => void; + const preparing = new Promise((resolve) => { + entered = resolve; + }); + const gate = new Promise((resolve) => { + release = resolve; + }); + let first = true; + const prepare = vi + .spyOn(ownership, "prepareProcessOwnershipInspection") + .mockImplementation(async (signal) => { + await original(signal); + if (first) { + first = false; + entered(); + await gate; + } + }); + onTestFinished(() => { + prepare.mockRestore(); + release(); + }); + const pending = runPcapCommand({ + operation: "inspect_pcap_client", + command: process.execPath, + arguments: ["--version"], + limits: { timeoutMs: 1000, maxOutputBytes: 1024 }, + environment: {}, + signal: undefined, + retain: (owner) => { + onTestFinished(async () => { + expect((await owner.stop()).status).not.toBe("incomplete"); + }); + }, + }).then( + (value) => ({ ok: true as const, value }), + (error: unknown) => ({ ok: false as const, error }), + ); + await preparing; + await delay(1100); + release(); + const result = await pending; + if (!result.ok) throw result.error; + expect(result.value.stdout.trim()).toBe(process.version); +}); diff --git a/tests/boundary/pcap/pcapProvider.test.ts b/tests/boundary/pcap/pcapProvider.test.ts new file mode 100644 index 000000000..50204a479 --- /dev/null +++ b/tests/boundary/pcap/pcapProvider.test.ts @@ -0,0 +1,508 @@ +import { createHash } from "node:crypto"; +import { watch } from "node:fs"; +import { chmod, readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { expect, onTestFinished, test, vi } from "vitest"; + +import { PcapProvider } from "../../../src/pcap/PcapProvider.js"; +import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; +import { + pcapResultSchemas, + type PcapRequest, +} from "../../../src/domain/pcap/pcapAnalysis.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { holdFirstFileRead } from "../../support/holdFirstFileRead.js"; +import { PrivateRuntimeRoot } from "../../../src/process/PrivateRuntimeRoot.js"; + +const it = test.skipIf(process.platform !== "linux"); + +const CAPINFOS_OUTPUT = [ + "File name: /work/capture.pcap", + "File type: Wireshark/tcpdump/... - pcap", + "File encapsulation: Ethernet", + "Packet size limit: file hdr: 262144 bytes", + "Number of packets: 3", + "Capture duration: 0.123 seconds", +].join("\n"); + +const TSHARK_JSON = JSON.stringify([ + { + _source: { + layers: { + "frame.number": ["1"], + "frame.protocols": ["eth:ethertype:ip:udp:dns"], + "ip.src": ["192.168.1.100"], + "ip.dst": ["8.8.8.8"], + "dns.qry.name": ["api.example.com"], + "dns.flags.response": ["1"], + "dns.a": ["93.184.216.34"], + }, + }, + }, + { + _source: { + layers: { + "frame.number": ["2"], + "frame.protocols": ["eth:ethertype:ip:tcp:tls"], + "ip.src": ["192.168.1.100"], + "ip.dst": ["93.184.216.34"], + "tls.handshake.extensions_server_name": ["api.example.com"], + }, + }, + }, + { + _source: { + layers: { + "frame.number": ["3"], + "frame.protocols": ["eth:ethertype:ip:tcp:http"], + "ip.src": ["192.168.1.100"], + "ip.dst": ["93.184.216.34"], + "http.request.full_uri": ["http://api.example.com/v1/data"], + }, + }, + }, +]); + +const writeStubTools = async ( + root: string, + options: { readonly failCapinfos?: boolean } = {}, +): Promise<{ tshark: string; capinfos: string }> => { + const tshark = join(root, "tshark"); + const capinfos = join(root, "capinfos"); + await writeFile( + tshark, + `#!/bin/sh +if [ "$1" = "--version" ]; then printf 'TShark (Wireshark) 4.6.6-stub\\n'; exit 0; fi +[ "$1" = "-n" ] && [ "$2" = "-r" ] && [ "$4" = "-T" ] && [ "$5" = "json" ] || exit 2 +printf '%s' '${TSHARK_JSON.replace(/'/gu, "'\\''")}' +exit 0 +`, + { mode: 0o755 }, + ); + await writeFile( + capinfos, + `#!/bin/sh +if [ "$1" = "--version" ]; then printf 'Capinfos (Wireshark) 4.6.6-stub\\n'; exit 0; fi +[ "$1" = "-M" ] && [ "$2" = "--" ] && [ "$#" = "3" ] || exit 2 +if [ "${String(options.failCapinfos === true)}" = "true" ]; then + printf 'capinfos: The file is not a capture file' >&2; exit 1 +fi +printf '%s\\n' '${CAPINFOS_OUTPUT.replace(/'/gu, "'\\''")}' +exit 0 +`, + { mode: 0o755 }, + ); + await chmod(tshark, 0o755); + await chmod(capinfos, 0o755); + return { tshark, capinfos }; +}; + +const providerWith = async (options: { failCapinfos?: boolean } = {}) => { + const root = await createTestTempDirectory("rea-pcap-"); + const { tshark, capinfos } = await writeStubTools(root, options); + const provider = new PcapProvider({ + environment: { REA_TSHARK_COMMAND: tshark, REA_CAPINFOS_COMMAND: capinfos }, + }); + onTestFinished(() => provider.close()); + return { + provider, + root, + }; +}; + +const evidenceResult = async ( + provider: PcapProvider, + request: PcapRequest, +): Promise> => { + const result = await provider.execute(request); + if (!result.ok) throw new Error(result.error.message); + return pcapResultSchemas[request.operation].parse(result.value.result); +}; + +const captureFixture = async (root: string): Promise => { + const path = join(root, "capture.pcap"); + await writeFile(path, Buffer.from("pcap-fixture-bytes")); + return path; +}; + +it("inspects both tools without reading a capture", async () => { + const { provider } = await providerWith(); + const result = await evidenceResult(provider, { + operation: "inspect_pcap_client", + input: {}, + }); + expect(result.client).toMatchObject({ + tshark_source: "environment", + tshark_version: "TShark (Wireshark) 4.6.6-stub", + capinfos_source: "environment", + capinfos_version: "Capinfos (Wireshark) 4.6.6-stub", + }); +}); + +it("inspects a capture with endpoints, DNS, TLS SNI, and HTTP", async () => { + const { provider, root } = await providerWith(); + const path = await captureFixture(root); + const result = await evidenceResult(provider, { + operation: "inspect_pcap_capture", + input: { path }, + }); + expect(result.target).toMatchObject({ + path, + bytes: 18, + sha256: createHash("sha256").update("pcap-fixture-bytes").digest("hex"), + }); + expect(result.file_type).toBe("Wireshark/tcpdump/... - pcap"); + expect(result.packet_count).toBe(3); + expect(result.endpoints).toEqual( + expect.arrayContaining([ + { address: "192.168.1.100", port: null, packet_count: 3 }, + { address: "8.8.8.8", port: null, packet_count: 1 }, + ]), + ); + expect(result.dns_queries).toEqual([ + { + query: "api.example.com", + response_addresses: ["93.184.216.34"], + response_address_count: 1, + }, + ]); + expect(result.tls_sni_values).toEqual(["api.example.com"]); + expect(result.http_request_uris).toEqual(["http://api.example.com/v1/data"]); + expect(result.coverage).toBe("complete"); +}); + +it("reports a non-capture file as unsupported", async () => { + const { provider, root } = await providerWith({ failCapinfos: true }); + const failPath = join(root, "fail.pcap"); + await writeFile(failPath, "not a capture file"); + const result = await provider.execute({ + operation: "inspect_pcap_capture", + input: { path: failPath }, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "unsupported_target", + }); +}); + +it("reports a missing tshark binary as a capability gap", async () => { + const provider = new PcapProvider({ + environment: { REA_TSHARK_COMMAND: "/nonexistent/tshark" }, + }); + expect(await provider.inspectAvailability()).toMatchObject({ + status: "unavailable", + code: "executable_missing", + }); +}); + +it("rejects a capture that is not a readable file", async () => { + const { provider } = await providerWith(); + const result = await provider.execute({ + operation: "inspect_pcap_capture", + input: { path: "/nonexistent/capture.pcap" }, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "invalid_request", + }); +}); + +it("cancels before spawning anything", async () => { + const { provider, root } = await providerWith(); + await writeFile(join(root, "spawned"), ""); + const controller = new AbortController(); + controller.abort(); + const result = await provider.execute( + { + operation: "inspect_pcap_capture", + input: { path: "/some/capture.pcap" }, + }, + { signal: controller.signal }, + ); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "cancelled", + }); +}); + +it("reports relative configuration and missing capinfos as capability gaps", async () => { + const root = await createTestTempDirectory("rea-pcap-config-"); + const { tshark } = await writeStubTools(root); + for (const environment of [ + { REA_TSHARK_COMMAND: "relative/tshark" }, + { + REA_TSHARK_COMMAND: tshark, + REA_CAPINFOS_COMMAND: join(root, "missing-capinfos"), + }, + ]) { + const provider = new PcapProvider({ environment }); + onTestFinished(() => provider.close()); + expect((await provider.inspectAvailability()).status).toBe("unavailable"); + const result = await provider.execute({ + operation: "inspect_pcap_client", + input: {}, + }); + if (result.ok) throw new Error("expected unavailable tools"); + expect(projectAnalysisError(result.error).code).toBe( + "capability_unavailable", + ); + } +}); + +const nodeTool = async ( + root: string, + name: string, + body: string, +): Promise => { + const path = join(root, name); + await writeFile( + path, + `#!/usr/bin/env -S ${process.execPath} --jitless --max-old-space-size=64\nconst fs = require('node:fs'); const crypto = require('node:crypto');\n${body}\n`, + { mode: 0o755 }, + ); + return path; +}; +const versionProbe = + "if (process.argv[2] === '--version') { console.log((process.argv[1].includes('capinfos') ? 'Capinfos' : 'TShark') + ' (Wireshark) 4.6.6-test'); process.exit(0); }"; + +it("refuses observations whose owned snapshot changed during dissection", async () => { + const root = await createTestTempDirectory("rea-pcap-mutated-"); + const { capinfos } = await writeStubTools(root); + const path = await captureFixture(root); + const before = await readFile(path); + const tshark = await nodeTool( + root, + "tshark.cjs", + `${versionProbe} +const selected = process.argv[process.argv.indexOf('-r') + 1]; +fs.chmodSync(selected, 0o600); fs.writeFileSync(selected, 'changed private snapshot'); +console.log(${JSON.stringify(TSHARK_JSON)});`, + ); + const provider = new PcapProvider({ + environment: { REA_TSHARK_COMMAND: tshark, REA_CAPINFOS_COMMAND: capinfos }, + }); + onTestFinished(() => provider.close()); + const result = await provider.execute({ + operation: "inspect_pcap_capture", + input: { path }, + }); + expect(result.ok).toBe(false); + if (!result.ok) + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "artifact_changed", + }); + expect(await readFile(path)).toEqual(before); +}); + +it("binds both tools to the snapshot when the original capture changes", async () => { + const root = await createTestTempDirectory("rea-pcap-identity-"); + const path = join(root, "caller capture.pcap"); + await writeFile(path, "192.0.2.1"); + const capinfos = await nodeTool( + root, + "capinfos.cjs", + `${versionProbe} +const selected = process.argv.at(-1); const bytes = fs.readFileSync(selected); +console.log('File type: pcap\\nFile encapsulation: Raw IP\\nNumber of packets: 1\\nSHA256: ' + crypto.createHash('sha256').update(bytes).digest('hex')); +fs.writeFileSync(${JSON.stringify(path)}, '192.0.2.2');`, + ); + const tshark = await nodeTool( + root, + "tshark.cjs", + `${versionProbe} +const selected = process.argv[process.argv.indexOf('-r') + 1]; +console.log(JSON.stringify([{_source:{layers:{'frame.number':['1'],'frame.protocols':['ip'],'ip.src':[fs.readFileSync(selected,'utf8')]}}}]));`, + ); + const provider = new PcapProvider({ + environment: { REA_TSHARK_COMMAND: tshark, REA_CAPINFOS_COMMAND: capinfos }, + }); + onTestFinished(() => provider.close()); + const result = await provider.execute({ + operation: "inspect_pcap_capture", + input: { path }, + }); + if (!result.ok) throw result.error; + const value = pcapResultSchemas.inspect_pcap_capture.parse( + result.value.result, + ); + expect(value.target.sha256).toBe( + createHash("sha256").update("192.0.2.1").digest("hex"), + ); + expect(value.endpoints).toEqual([ + { address: "192.0.2.1", port: null, packet_count: 1 }, + ]); + expect(await readFile(path, "utf8")).toBe("192.0.2.2"); +}); + +it.each(["probe", "capinfos", "tshark"] as const)( + "preserves in-flight %s cancellation and cleans up", + async (phase) => { + const root = await createTestTempDirectory("rea-pcap-cancel-"); + const marker = join(root, "ready"); + const slow = await nodeTool( + root, + `slow-${phase}.cjs`, + `${phase !== "probe" ? versionProbe : ""} +fs.writeFileSync(${JSON.stringify(marker)}, 'ready'); setInterval(() => {}, 1000);`, + ); + const { capinfos, tshark } = await writeStubTools(root); + const path = await captureFixture(root); + const provider = new PcapProvider({ + environment: { + REA_TSHARK_COMMAND: phase === "capinfos" ? tshark : slow, + REA_CAPINFOS_COMMAND: phase === "capinfos" ? slow : capinfos, + }, + }); + onTestFinished(() => provider.close()); + const controller = new AbortController(); + let watcher: ReturnType | undefined; + let timer: ReturnType | undefined; + const ready = new Promise((resolve, reject) => { + watcher = watch(root, (_event, name) => { + if (name === "ready") resolve(); + }); + timer = setTimeout(() => { + controller.abort(); + reject(new Error("tool did not start")); + }, 5000); + }); + try { + const pending = provider.execute( + { operation: "inspect_pcap_capture", input: { path } }, + { signal: controller.signal }, + ); + await ready; + controller.abort(new Error("caller cancellation")); + const result = await pending; + if (result.ok) throw new Error("expected cancellation"); + expect(projectAnalysisError(result.error).code).toBe("cancelled"); + await provider.close(); + } finally { + watcher?.close(); + clearTimeout(timer); + } + }, +); + +it.each(["change", "cancel"] as const)( + "preserves %s during snapshot acquisition", + async (action) => { + const { provider, root } = await providerWith(); + const path = await captureFixture(root); + const controller = new AbortController(); + const held = await holdFirstFileRead(path); + onTestFinished(() => { + vi.restoreAllMocks(); + }); + const pending = provider.execute( + { operation: "inspect_pcap_capture", input: { path } }, + { signal: controller.signal }, + ); + try { + await held.firstRead; + if (action === "cancel") controller.abort(); + else await writeFile(path, "changed source bytes"); + } finally { + held.release(); + } + const result = await pending; + if (result.ok) throw new Error("expected interrupted acquisition"); + expect(result.error._tag).toBe( + action === "cancel" + ? "AnalysisCancelledError" + : "AnalysisArtifactChangedError", + ); + }, +); + +it("admits one capture and drains it before close releases resources", async () => { + const { provider, root } = await providerWith(); + const path = await captureFixture(root); + const held = await holdFirstFileRead(path); + onTestFinished(() => { + vi.restoreAllMocks(); + }); + const pending = provider.execute({ + operation: "inspect_pcap_capture", + input: { path }, + }); + let closing: Promise | undefined; + try { + await held.firstRead; + const concurrent = await provider.execute({ + operation: "inspect_pcap_capture", + input: { path }, + }); + expect(concurrent.ok).toBe(false); + if (!concurrent.ok) + expect(projectAnalysisError(concurrent.error)).toMatchObject({ + code: "resource_constraint", + details: { resource: "capacity" }, + }); + let closed = false; + closing = provider.close().then(() => { + closed = true; + }); + await Promise.resolve(); + expect(closed).toBe(false); + } finally { + held.release(); + } + expect((await pending).ok).toBe(true); + await closing; + const afterClose = await provider.execute({ + operation: "inspect_pcap_capture", + input: { path }, + }); + expect(afterClose.ok).toBe(false); +}); + +it.each([false, true])( + "retains failed snapshot cleanup after an analysis failure=%s", + async (failCapinfos) => { + const { provider, root } = await providerWith({ failCapinfos }); + const path = await captureFixture(root); + let retained: PrivateRuntimeRoot | undefined; + const spy = vi + .spyOn(PrivateRuntimeRoot.prototype, "close") + .mockImplementation(function (this: PrivateRuntimeRoot) { + retained = this; + return Promise.reject(new Error("controlled cleanup failure")); + }); + const result = await provider.execute({ + operation: "inspect_pcap_capture", + input: { path }, + }); + try { + expect(result.ok).toBe(false); + if (result.ok) throw new Error("Cleanup failure cannot return success"); + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "cleanup_incomplete", + }); + if (failCapinfos) + expect(projectAnalysisError(result.error)).toMatchObject({ + details: { + diagnostics: { primary_error: { code: "unsupported_target" } }, + }, + }); + else + expect(result.error.partialObservation).toMatchObject({ + operation: "inspect_pcap_capture", + normalized_result: { packet_count: 3 }, + }); + expect(retained).toBeDefined(); + await expect(provider.close()).rejects.toMatchObject({ + cleanupIncomplete: true, + }); + } finally { + spy.mockRestore(); + } + await provider.close(); + await expect( + readFile(join(retained!.path, "capture.pcap")), + ).rejects.toMatchObject({ code: "ENOENT" }); + }, +); diff --git a/tests/boundary/pcap/pcapProviderCrossPlatform.test.ts b/tests/boundary/pcap/pcapProviderCrossPlatform.test.ts new file mode 100644 index 000000000..3d7f28b54 --- /dev/null +++ b/tests/boundary/pcap/pcapProviderCrossPlatform.test.ts @@ -0,0 +1,144 @@ +/** + * Cross-platform pcap provider tests. + * + * POSIX hosts run the #!/bin/sh fixtures; + * Windows runs the .cmd stubs from tests/fixtures/pcap/. Both validate + * the same provider contract surface: tool probing, capture inspection, + * refusal paths, and cancellation before spawn. + */ +import { createHash } from "node:crypto"; +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { expect, it, onTestFinished } from "vitest"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +import { PcapProvider } from "../../../src/pcap/PcapProvider.js"; +import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; +import type { PcapRequest } from "../../../src/domain/pcap/pcapAnalysis.js"; + +const isWindows = process.platform === "win32"; +const hasNativeBound = process.platform === "linux" || isWindows; +const stubDir = join(import.meta.dirname, "../../fixtures/pcap"); +const tsharkStub = isWindows + ? join(stubDir, "tshark-stub.cmd") + : join(stubDir, "tshark-stub.sh"); +const capinfosStub = isWindows + ? join(stubDir, "capinfos-stub.cmd") + : join(stubDir, "capinfos-stub.sh"); + +const providerOver = (): PcapProvider => { + const provider = new PcapProvider({ + environment: { + REA_TSHARK_COMMAND: tsharkStub, + REA_CAPINFOS_COMMAND: capinfosStub, + }, + }); + onTestFinished(() => provider.close()); + return provider; +}; + +const evidenceResult = async ( + provider: PcapProvider, + request: PcapRequest, +): Promise> => { + const result = await provider.execute(request); + if (!result.ok) throw new Error(result.error.message); + return result.value.result as Record; +}; + +it("inspects both tools through cross-platform stubs", async () => { + const result = await evidenceResult(providerOver(), { + operation: "inspect_pcap_client", + input: {}, + }); + expect(result.client).toMatchObject({ + tshark_source: "environment", + tshark_version: expect.stringContaining("4.6.6") as unknown, + capinfos_source: "environment", + capinfos_version: expect.stringContaining("4.6.6") as unknown, + }); +}); + +it.runIf(hasNativeBound)( + "inspects a capture with metadata, endpoints, DNS, and protocol facts", + async () => { + const root = await createTestTempDirectory("rea-pcap-x-"); + const capturePath = join(root, "capture.pcap"); + const captureBytes = "cross-platform-pcap-fixture"; + await writeFile(capturePath, captureBytes); + + const result = await evidenceResult(providerOver(), { + operation: "inspect_pcap_capture", + input: { path: capturePath }, + }); + + expect(result.target).toMatchObject({ + path: capturePath, + bytes: captureBytes.length, + sha256: createHash("sha256").update(captureBytes).digest("hex"), + }); + expect(result.file_type).toContain("pcap"); + expect(result.packet_count).toBe(3); + expect((result.endpoints as unknown[]).length).toBeGreaterThan(0); + expect((result.protocol_distribution as unknown[]).length).toBeGreaterThan( + 0, + ); + expect(result.coverage).toBe("complete"); + }, +); + +it("rejects a path that is not a readable file", async () => { + const result = await providerOver().execute({ + operation: "inspect_pcap_capture", + input: { path: "/nonexistent/definitely-missing.pcap" }, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "invalid_request", + }); +}); + +it("reports a missing binary as a capability gap", async () => { + const provider = new PcapProvider({ + environment: { REA_TSHARK_COMMAND: "/nonexistent/tshark" }, + }); + expect(await provider.inspectAvailability()).toMatchObject({ + status: "unavailable", + code: "executable_missing", + }); +}); + +it("cancels before spawning anything", async () => { + const controller = new AbortController(); + controller.abort(); + const result = await providerOver().execute( + { operation: "inspect_pcap_capture", input: { path: "/any/capture.pcap" } }, + { signal: controller.signal }, + ); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "cancelled", + }); +}); + +it.runIf(!hasNativeBound)( + "reports an unsupported native memory boundary before reading the capture", + async () => { + const provider = providerOver(); + expect(await provider.inspectAvailability()).toMatchObject({ + status: "unavailable", + code: "unsupported_host", + }); + const result = await provider.execute({ + operation: "inspect_pcap_capture", + input: { path: "/not-opened/capture.pcap" }, + }); + expect(result.ok).toBe(false); + if (!result.ok) + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "capability_unavailable", + }); + }, +); diff --git a/tests/boundary/pcap/pcapRetainedView.test.ts b/tests/boundary/pcap/pcapRetainedView.test.ts new file mode 100644 index 000000000..a3f890a37 --- /dev/null +++ b/tests/boundary/pcap/pcapRetainedView.test.ts @@ -0,0 +1,196 @@ +import { createHash } from "node:crypto"; +import { readFile, rm, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { execFile } from "node:child_process"; +import { promisify } from "node:util"; +import { expect, it } from "vitest"; +import { parseEvidence } from "../../../src/domain/evidence.js"; +import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; + +const quote = (value: string) => `'${value.replaceAll("'", "'\\''")}'`; + +const createPcapToolFixture = async (root: string) => { + const path = join(root, "caller capture.pcap"); + const bytes = Buffer.from("recording fixture bytes"); + const sha = createHash("sha256").update(bytes).digest("hex"); + await writeFile(path, bytes); + const tshark = join(root, "tshark"); + const capinfos = join(root, "capinfos"); + const marker = join(root, "reads"); + const frames = Array.from({ length: 132 }, (_, index) => ({ + _source: { + layers: { + "frame.number": [String(index + 1)], + "frame.protocols": ["ip:udp:dns"], + "ip.src": ["192.0.2.1"], + "ip.dst": ["192.0.2.2"], + "dns.qry.name": ["many.example"], + "dns.flags.response": ["True"], + "dns.a": [`198.51.100.${index}`], + "tls.handshake.extensions_server_name": [`host${index}.example`], + "http.request.full_uri": [ + "http://user:password@example.invalid/p?q=keep", + ], + }, + }, + })); + await writeFile( + tshark, + `#!/bin/sh\nif [ "$1" = "--version" ]; then printf 'TShark (Wireshark) 4.6.6-emulated\\n'; exit 0; fi\nprintf 'read\\n' >> ${quote(marker)}\nprintf '%s' ${quote(JSON.stringify(frames))}\nprintf 'retained warning' >&2\n`, + { mode: 0o755 }, + ); + await writeFile( + capinfos, + `#!/bin/sh\nif [ "$1" = "--version" ]; then printf 'Capinfos (Wireshark) 4.6.6-emulated\\n'; exit 0; fi\nprintf '%s\\n' ${quote(`File type: pcap\nFile encapsulation: Ethernet\nNumber of packets: 132\nSHA256: ${sha}`)}\n`, + { mode: 0o755 }, + ); + const environment = { + REA_TSHARK_COMMAND: tshark, + REA_CAPINFOS_COMMAND: capinfos, + }; + return { path, marker, environment }; +}; + +// Native tools are emulated here; this proves the production CLI/MCP boundary, +// retention and paging, not Wireshark's real dissector semantics. +it.runIf(process.platform === "linux")( + "retains every fact and pages late DNS/SNI rows after source deletion without another tool launch", + async () => { + const root = await createTestTempDirectory("rea-pcap-view-"); + const { path, marker, environment } = await createPcapToolFixture(root); + const complete = await promisify(execFile)( + process.execPath, + [ + join(import.meta.dirname, "../../../scripts/rea.mjs"), + "inspect-pcap-capture", + path, + "--json", + ], + { env: { ...process.env, ...environment } }, + ); + const parent = parseEvidence(JSON.parse(complete.stdout)); + expect(parent.normalized_result).toMatchObject({ + tls_sni_count: 132, + tls_sni_values: expect.arrayContaining(["host131.example"]), + dns_queries: [{ query: "many.example", response_address_count: 132 }], + }); + expect(JSON.stringify(parent)).not.toContain("user:password"); + const { call } = await connectLocalToolsMcp(environment); + const response = await call("inspect_pcap_capture", { + path, + detail: "summary", + }); + expect(response.isError, JSON.stringify(response)).not.toBe(true); + const summary = parseEvidence(response.structuredContent).normalized_result; + expect(summary).toMatchObject({ + parent_evidence_id: parent.evidence_id, + summary: { + tls_sni_count: 132, + dns_query_count: 1, + retained_counts: { tls_sni_values: 132 }, + diagnostics_code_units: { tshark: 16, capinfos: 0 }, + }, + }); + expect(JSON.stringify(summary)).not.toContain("host131.example"); + await rm(path); + const source = { + kind: "retained-evidence", + evidence_id: parent.evidence_id, + }; + const before = await readFile(marker, "utf8"); + const cases = [ + [ + { kind: "pcap", facet: "endpoints", limit: 1 }, + { items: [{ address: "192.0.2.1", packet_count: 132 }] }, + ], + [ + { kind: "pcap", facet: "dns-queries", limit: 1 }, + { + items: [ + { query: "many.example", retained_response_address_count: 132 }, + ], + }, + ], + [ + { + kind: "pcap", + facet: "dns-addresses", + query: "many.example", + offset: 130, + limit: 2, + }, + { items: ["198.51.100.130", "198.51.100.131"] }, + ], + [ + { kind: "pcap", facet: "tls-sni", offset: 130, limit: 2 }, + { items: ["host130.example", "host131.example"] }, + ], + [ + { kind: "pcap", facet: "protocols" }, + { items: [{ protocol: "ip:udp:dns", packet_count: 132 }] }, + ], + [ + { kind: "pcap", facet: "http-uris" }, + { items: ["http://example.invalid/p?q=keep"] }, + ], + [{ kind: "pcap", facet: "unattributed-addresses" }, { items: [] }], + [ + { + kind: "pcap", + facet: "diagnostics", + stream: "tshark", + offset: 9, + limit: 7, + }, + { text: "warning", status: "retained" }, + ], + ] as const; + for (const [view, expected] of cases) { + const response = await call("inspect_analysis_view", { source, view }); + expect(response.isError, JSON.stringify(response)).not.toBe(true); + expect( + parseEvidence(response.structuredContent).normalized_result, + ).toMatchObject({ page: expected, source_coverage: "complete" }); + } + const workflow = join(root, "portable.json"); + const view = { + kind: "pcap", + facet: "dns-addresses", + query: "many.example", + offset: 130, + limit: 2, + }; + await writeFile( + workflow, + JSON.stringify({ source: { kind: "inline", evidence: parent }, view }), + ); + const portable = await promisify(execFile)(process.execPath, [ + join(import.meta.dirname, "../../../scripts/rea.mjs"), + "inspect-analysis-view", + workflow, + "--json", + ]); + const retained = parseEvidence( + (await call("inspect_analysis_view", { source, view })).structuredContent, + ); + expect( + parseEvidence(JSON.parse(portable.stdout)).normalized_result, + ).toEqual(retained.normalized_result); + expect(await readFile(marker, "utf8")).toBe(before); + expect( + parseMcpToolError( + await call("inspect_analysis_view", { + source, + view: { + kind: "pcap", + facet: "dns-addresses", + query: "missing.example", + }, + }), + ), + ).toMatchObject({ error: { code: "invalid_request" } }); + }, + 60_000, +); diff --git a/tests/boundary/process/darwinProcessRunTokenReader.test.ts b/tests/boundary/process/darwinProcessRunTokenReader.test.ts index bfdb047f2..9f377440c 100644 --- a/tests/boundary/process/darwinProcessRunTokenReader.test.ts +++ b/tests/boundary/process/darwinProcessRunTokenReader.test.ts @@ -26,6 +26,7 @@ import { import { createDarwinProcessRunTokenReader, DarwinProcessOwnershipInspectionError, + SWIFTC_INTERRUPT, } from "../../../src/process/DarwinProcessRunTokenReader.js"; import { execFileOutput } from "../../../src/process/ExecFileOutput.js"; import { @@ -1110,16 +1111,8 @@ it.skipIf(!onDarwin)( try { await execFileOutput( "/usr/bin/xcrun", - [ - "swiftc", - "-module-cache-path", - join(root, "modules"), - core, - fixture, - "-o", - executable, - ], - { timeout: 60_000 }, + ["swiftc", core, fixture, "-o", executable], + { timeout: 120_000, stopSignal: SWIFTC_INTERRUPT }, ); const { stdout } = await execFileAsync(executable, [], { encoding: "utf8", @@ -1179,6 +1172,7 @@ it.skipIf(!onDarwin)( await rm(root, { recursive: true, force: true }); } }, + 150_000, ); it.skipIf(!onDarwin)( @@ -1203,16 +1197,8 @@ it.skipIf(!onDarwin)( try { await execFileOutput( "/usr/bin/xcrun", - [ - "swiftc", - "-module-cache-path", - join(root, "modules"), - core, - fixture, - "-o", - executable, - ], - { timeout: 60_000 }, + ["swiftc", core, fixture, "-o", executable], + { timeout: 120_000, stopSignal: SWIFTC_INTERRUPT }, ); const { stdout } = await execFileAsync(executable, [], { encoding: "utf8", @@ -1226,6 +1212,7 @@ it.skipIf(!onDarwin)( await rm(root, { recursive: true, force: true }); } }, + 150_000, ); it.skipIf(!onDarwin)( diff --git a/tests/fixtures/localToolsMcp.ts b/tests/fixtures/localToolsMcp.ts index 1e1e91909..bf9c7617a 100644 --- a/tests/fixtures/localToolsMcp.ts +++ b/tests/fixtures/localToolsMcp.ts @@ -34,7 +34,9 @@ const compileValidator = (schema: unknown): ValidateFunction => { }; /** Start the built production server without configuring deep binary providers. */ -export async function connectLocalToolsMcp() { +export async function connectLocalToolsMcp( + environment: Readonly> = {}, +) { const transport = new StdioClientTransport({ command: process.execPath, args: [ @@ -45,6 +47,7 @@ export async function connectLocalToolsMcp() { PATH: process.env.PATH ?? "/usr/bin:/bin", REA_LOG_LEVEL: "silent", HOPPER_LAUNCHER_PATH: "/rea-unconfigured-deep-provider/hopper", + ...environment, }, stderr: "pipe", }); diff --git a/tests/fixtures/pcap/capinfos-stub.cmd b/tests/fixtures/pcap/capinfos-stub.cmd new file mode 100644 index 000000000..6f783bcf5 --- /dev/null +++ b/tests/fixtures/pcap/capinfos-stub.cmd @@ -0,0 +1,19 @@ +@echo off +rem capinfos stub for Windows: emulates capinfos --version and metadata output. + +if "%~1"=="--version" ( + echo Capinfos ^(Wireshark^) 4.6.6-windows-stub + exit /b 0 +) + +if not "%~1"=="-M" exit /b 2 +if not "%~2"=="--" exit /b 2 +if "%~3"=="" exit /b 2 + +echo File name: capture.pcap +echo File type: Wireshark/tcpdump/... - pcap +echo File encapsulation: Ethernet +echo Packet size limit: file hdr: 262144 bytes +echo Number of packets: 3 +echo Capture duration: 0.123 seconds +exit /b 0 diff --git a/tests/fixtures/pcap/capinfos-stub.sh b/tests/fixtures/pcap/capinfos-stub.sh new file mode 100755 index 000000000..47d61b1b1 --- /dev/null +++ b/tests/fixtures/pcap/capinfos-stub.sh @@ -0,0 +1,11 @@ +#!/bin/sh +# POSIX capinfos stub (mirrors capinfos-stub.cmd for Windows parity). +if [ "$1" = "--version" ]; then printf 'Capinfos (Wireshark) 4.6.6-posix-stub\n'; exit 0; fi +[ "$1" = "-M" ] && [ "$2" = "--" ] && [ "$#" = "3" ] || exit 2 +printf '%s\n' 'File name: capture.pcap' +printf '%s\n' 'File type: Wireshark/tcpdump/... - pcap' +printf '%s\n' 'File encapsulation: Ethernet' +printf '%s\n' 'Packet size limit: file hdr: 262144 bytes' +printf '%s\n' 'Number of packets: 3' +printf '%s\n' 'Capture duration: 0.123 seconds' +exit 0 diff --git a/tests/fixtures/pcap/tshark-stub.cmd b/tests/fixtures/pcap/tshark-stub.cmd new file mode 100644 index 000000000..d157ed721 --- /dev/null +++ b/tests/fixtures/pcap/tshark-stub.cmd @@ -0,0 +1,17 @@ +@echo off +rem Pcap tool stub for Windows: emulates tshark --version and field output. +rem %1 is the first argument; the capture path arrives as %2 for -r/-c. + +if "%~1"=="--version" ( + echo TShark ^(Wireshark^) 4.6.6-windows-stub + exit /b 0 +) + +if not "%~1"=="-n" exit /b 2 +if not "%~2"=="-r" exit /b 2 +if not "%~4"=="-T" exit /b 2 +if not "%~5"=="json" exit /b 2 + +rem Emit the same JSON as the POSIX stub +echo [{"_source":{"layers":{"frame.number":["1"],"frame.protocols":["eth:ethertype:ip:udp:dns"],"ip.src":["192.168.1.100"],"ip.dst":["8.8.8.8"],"dns.qry.name":["api.example.com"],"dns.flags.response":["1"],"dns.a":["93.184.216.34"]}}},{"_source":{"layers":{"frame.number":["2"],"frame.protocols":["eth:ethertype:ip:tcp:tls"],"ip.src":["192.168.1.100"],"ip.dst":["93.184.216.34"],"tls.handshake.extensions_server_name":["api.example.com"]}}},{"_source":{"layers":{"frame.number":["3"],"frame.protocols":["eth:ethertype:ip:tcp:http"],"ip.src":["192.168.1.100"],"ip.dst":["93.184.216.34"],"http.request.full_uri":["http://api.example.com/v1/data"]}}}] +exit /b 0 diff --git a/tests/fixtures/pcap/tshark-stub.sh b/tests/fixtures/pcap/tshark-stub.sh new file mode 100755 index 000000000..8633dd7c0 --- /dev/null +++ b/tests/fixtures/pcap/tshark-stub.sh @@ -0,0 +1,6 @@ +#!/bin/sh +# POSIX pcap tool stub (mirrors tshark-stub.cmd for Windows parity). +if [ "$1" = "--version" ]; then printf 'TShark (Wireshark) 4.6.6-posix-stub\n'; exit 0; fi +[ "$1" = "-n" ] && [ "$2" = "-r" ] && [ "$4" = "-T" ] && [ "$5" = "json" ] || exit 2 +printf '%s' '[{"_source":{"layers":{"frame.number":["1"],"frame.protocols":["eth:ethertype:ip:udp:dns"],"ip.src":["192.168.1.100"],"ip.dst":["8.8.8.8"],"dns.qry.name":["api.example.com"],"dns.flags.response":["1"],"dns.a":["93.184.216.34"]}}},{"_source":{"layers":{"frame.number":["2"],"frame.protocols":["eth:ethertype:ip:tcp:tls"],"ip.src":["192.168.1.100"],"ip.dst":["93.184.216.34"],"tls.handshake.extensions_server_name":["api.example.com"]}}},{"_source":{"layers":{"frame.number":["3"],"frame.protocols":["eth:ethertype:ip:tcp:http"],"ip.src":["192.168.1.100"],"ip.dst":["93.184.216.34"],"http.request.full_uri":["http://api.example.com/v1/data"]}}}]' +exit 0 diff --git a/vitest.config.ts b/vitest.config.ts index 74dbf5305..bfef8e139 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -51,7 +51,7 @@ const TEST_PROJECTS = [ name: "adapters", include: [ "src/*.test.ts", - "src/{apktool,jeb,adb,artifacts,browser,dotnet,evm,firmware,flutter,ghidra,go,hopper,ida,inspector,javascript,native,process,reference,server,wasm,windows}/**/*.test.ts", + "src/{apktool,jeb,adb,artifacts,browser,dotnet,evm,firmware,flutter,pcap,ghidra,go,hopper,ida,inspector,javascript,native,process,reference,server,wasm,windows}/**/*.test.ts", ], pool: "forks" as const, maxWorkers: MAX_TEST_WORKERS, From a73a55f53cc2b2e5b7618effe6ecb5cee6e0cee2 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:54:22 +0800 Subject: [PATCH 016/159] fix(artifacts): preserve decode coverage and cleanup ownership --- src/firmware/FirmwareFiles.ts | 22 ++-- src/native/AppleDispatchCoverage.ts | 73 ++++++++++-- src/native/AppleDispatchMetadata.test.ts | 142 ++++++++++++++++++++++- src/native/AppleMachoFixups.test.ts | 67 ++++++++++- src/native/AppleMachoFixups.ts | 39 +++++-- src/native/AppleObjcCategories.ts | 4 +- src/native/AppleObjcDispatchFacets.ts | 1 - src/native/AppleSwiftVtables.test.ts | 2 + 8 files changed, 305 insertions(+), 45 deletions(-) diff --git a/src/firmware/FirmwareFiles.ts b/src/firmware/FirmwareFiles.ts index 10201905e..d20226e76 100644 --- a/src/firmware/FirmwareFiles.ts +++ b/src/firmware/FirmwareFiles.ts @@ -213,19 +213,15 @@ export const inventoryFirmwareOutput = async ( }); if (handle === undefined) continue; const names: string[] = []; - try { - for await (const entry of handle) { - throwIfFirmwareInventoryCancelled(signal, operation); - count++; - if (count > maxEntries) - throw new AnalysisOutputError( - operation, - `Staging entry budget exceeded (${maxEntries})`, - ); - names.push(entry.name); - } - } finally { - await handle.close().catch(() => undefined); + for await (const entry of handle) { + throwIfFirmwareInventoryCancelled(signal, operation); + count++; + if (count > maxEntries) + throw new AnalysisOutputError( + operation, + `Staging entry budget exceeded (${maxEntries})`, + ); + names.push(entry.name); } for (const name of names.sort()) { const path = join(directory, name); diff --git a/src/native/AppleDispatchCoverage.ts b/src/native/AppleDispatchCoverage.ts index c49e35acc..a6c68ca51 100644 --- a/src/native/AppleDispatchCoverage.ts +++ b/src/native/AppleDispatchCoverage.ts @@ -42,14 +42,62 @@ export const pushDispatchCoverage = (input: { vtables, swift, } = input; + const classOwnedMethods = result.objc_dispatch_implementations.filter( + ({ category }) => category === undefined, + ); + const classRecords = [ + ...result.objc_classes.flatMap((record) => + record.decode?.status === "decoded" + ? [] + : [ + `class ${record.name}: ${record.decode?.reason ?? record.decode?.status ?? "decode_status_missing"}`, + ], + ), + ...result.objc_protocol_records.flatMap((record) => + record.decode.status === "decoded" + ? [] + : [ + `protocol ${record.name}: ${record.decode.reason ?? record.decode.status}`, + ], + ), + ...classOwnedMethods.flatMap((record) => + record.decode.status === "decoded" + ? [] + : [ + `method ${record.class_name} ${record.method_type} ${record.selector}: ${record.decode.reason ?? record.decode.status}`, + ], + ), + ...result.objc_ivars.flatMap((record) => + record.decode.status === "decoded" + ? [] + : [ + `ivar ${record.class_name}.${record.name}: ${record.decode.reason ?? record.decode.status}`, + ], + ), + ]; + const classReasons = [ + ...failures, + ...classRecords, + ...(truncated ? ["max_records_reached"] : []), + ]; + const categoryMethods = result.objc_dispatch_implementations.filter( + ({ category }) => category !== undefined, + ); + const categoryMethodReasons = categoryMethods.flatMap((record) => + record.decode.status === "decoded" + ? [] + : [ + `category method ${record.category} ${record.class_name} ${record.method_type} ${record.selector}: ${record.decode.reason ?? record.decode.status}`, + ], + ); result.coverage.push({ facet: "objc_class_method_ivar_metadata", - status: failures.length > 0 || truncated ? "partial" : "complete", - reason: - [...failures, ...(truncated ? ["max_records_reached"] : [])].join("; ") || - null, + status: classReasons.length > 0 ? "partial" : "complete", + reason: classReasons.join("; ") || null, examined, - decoded: result.objc_classes.length, + decoded: result.objc_classes.filter( + ({ decode }) => decode?.status === "decoded", + ).length, }); result.coverage.push({ facet: "binary_relative_pointers", @@ -79,13 +127,18 @@ export const pushDispatchCoverage = (input: { status: truncated || categoryFailures.length > 0 || - result.objc_categories.some(({ decode }) => decode.status !== "decoded") + result.objc_categories.some( + ({ decode }) => decode.status !== "decoded", + ) || + categoryMethodReasons.length > 0 ? "partial" : "complete", reason: - [...categoryFailures, ...(truncated ? ["max_records_reached"] : [])].join( - "; ", - ) || null, + [ + ...categoryFailures, + ...categoryMethodReasons, + ...(truncated ? ["max_records_reached"] : []), + ].join("; ") || null, examined: categoriesExamined, decoded: result.objc_categories.filter( ({ decode }) => decode.status === "decoded", @@ -110,7 +163,7 @@ export const pushDispatchCoverage = (input: { : "complete", reason: fixupReason.length > 0 ? fixupReason : null, examined: fixups.examined, - decoded: fixups.issues.length > 0 ? 0 : fixups.examined, + decoded: fixups.decoded, }); result.coverage.push({ facet: "swift_generic_resilient_witnesses_overrides_async_coroutines", diff --git a/src/native/AppleDispatchMetadata.test.ts b/src/native/AppleDispatchMetadata.test.ts index 6791ba1b0..244a35aeb 100644 --- a/src/native/AppleDispatchMetadata.test.ts +++ b/src/native/AppleDispatchMetadata.test.ts @@ -2,7 +2,10 @@ import { describe, expect, it } from "vitest"; import { decodeAppleDispatchMetadata } from "./AppleDispatchMetadata.js"; const base = 0x100000000n; -const fixture = (relative = false) => { +const fixture = ( + relative = false, + category?: "partial-method" | "unmapped-pointer", +) => { const bytes = Buffer.alloc(4096); const u32 = (offset: number, value: number) => bytes.writeUInt32LE(value, offset); @@ -11,19 +14,26 @@ const fixture = (relative = false) => { u32(0, 0xfeedfacf); u32(4, 0x0100000c); u32(16, 1); - u32(20, 152); + u32(20, category === undefined ? 152 : 232); u32(32, 0x19); - u32(36, 152); + u32(36, category === undefined ? 152 : 232); bytes.write("__DATA", 40); ptr(56, 0); bytes.writeBigUInt64LE(4096n, 64); bytes.writeBigUInt64LE(4096n, 80); u32(92, 5); - u32(96, 1); + u32(96, category === undefined ? 1 : 2); bytes.write("__objc_classlist", 104); bytes.write("__DATA", 120); ptr(136, 0x180); bytes.writeBigUInt64LE(8n, 144); + if (category !== undefined) { + bytes.write("__objc_catlist", 184); + bytes.write("__DATA", 200); + ptr(216, 0xb00); + bytes.writeBigUInt64LE(8n, 224); + u32(232, 0xb00); + } u32(152, 0x180); ptr(0x180, 0x200); ptr(0x200, 0x280); @@ -86,10 +96,58 @@ const fixture = (relative = false) => { bytes.write("FixtureProtocol\0", 0x900); bytes.write("requiredAction:\0", 0x920); bytes.write("optionalAction:\0", 0x940); + if (category !== undefined) { + if (category === "unmapped-pointer") { + bytes.writeBigUInt64LE(0xffffffffffffffffn, 0xb00); + } else { + ptr(0xb00, 0xb40); + ptr(0xb40, 0xbc0); + ptr(0xb48, 0x200); + ptr(0xb50, 0xb80); + u32(0xb80, 24); + u32(0xb84, 1); + ptr(0xb88, 0x460); + ptr(0xb90, 0x5c0); + bytes.writeBigUInt64LE(0xffffffffffffffffn, 0xb98); + bytes.write("FixtureCategory\0", 0xbc0); + } + } return bytes; }; const provenance = { path: "/fixture/app", sha256: "a".repeat(64) }; -describe("chained Apple dispatch binary metadata", () => { +describe("Apple dispatch pointer fixup metadata", () => { + it("keeps decoded bindings and their coverage when an independent stream fails", () => { + const bytes = fixture(); + bytes.writeUInt32LE(2, 16); + bytes.writeUInt32LE(200, 20); + bytes.writeUInt32LE(0x80000022, 184); + bytes.writeUInt32LE(48, 188); + const bindings = Buffer.from([ + 0x40, + ...Buffer.from("_OBJC_CLASS_$_NSObject\0"), + 0x70, + 0x88, + 0x04, + 0x90, + 0x00, + ]); + bindings.copy(bytes, 0xc00); + bytes[0xc80] = 0xd0; + bytes.writeUInt32LE(0xc00, 200); + bytes.writeUInt32LE(bindings.length, 204); + bytes.writeUInt32LE(0xc80, 208); + bytes.writeUInt32LE(1, 212); + + const result = decodeAppleDispatchMetadata(bytes, 100, provenance); + expect(result.objc_classes[0]).toMatchObject({ + name: "Fixture", + super_class: "NSObject", + }); + expect( + result.coverage.find(({ facet }) => facet === "pointer_fixups"), + ).toMatchObject({ status: "partial", examined: 2, decoded: 1 }); + }); + it("keeps absolute metadata pointers between declared chained fixups", () => { const bytes = fixture(); bytes.writeUInt32LE(2, 16); @@ -120,7 +178,7 @@ describe("chained Apple dispatch binary metadata", () => { }); expect( result.coverage.find(({ facet }) => facet === "pointer_fixups"), - ).toMatchObject({ status: "complete" }); + ).toMatchObject({ status: "complete", examined: 1, decoded: 1 }); }); }); describe("Apple dispatch binary metadata", () => { @@ -221,6 +279,9 @@ describe("Apple dispatch binary metadata", () => { }); }, ); +}); + +describe("Apple dispatch coverage ownership", () => { it("keeps unsupported pointers and record truncation explicit", () => { const bytes = fixture(); bytes.writeBigUInt64LE(0xffffffffffffffffn, 0x498); @@ -229,6 +290,14 @@ describe("Apple dispatch binary metadata", () => { implementation_address: null, decode: { status: "partial" }, }); + expect( + result.coverage.find( + ({ facet }) => facet === "objc_class_method_ivar_metadata", + ), + ).toMatchObject({ + status: "partial", + reason: expect.stringContaining("method Fixture instance performAction:"), + }); const truncated = decodeAppleDispatchMetadata(fixture(), 1, provenance); expect(truncated.coverage[0]?.reason).toContain("max_records_reached"); expect( @@ -237,6 +306,67 @@ describe("Apple dispatch binary metadata", () => { ), ).toMatchObject({ status: "partial", reason: "max_records_reached" }); }); + it("keeps partial category methods in category coverage only", () => { + const baseline = decodeAppleDispatchMetadata(fixture(), 100, provenance); + const result = decodeAppleDispatchMetadata( + fixture(false, "partial-method"), + 100, + provenance, + ); + + expect(result.objc_dispatch_implementations).toContainEqual( + expect.objectContaining({ + category: "FixtureCategory", + decode: { + status: "partial", + reason: "implementation_pointer_unmapped_or_encoded", + }, + }), + ); + const baselineClassCoverage = baseline.coverage.find( + ({ facet }) => facet === "objc_class_method_ivar_metadata", + ); + const classCoverage = result.coverage.find( + ({ facet }) => facet === "objc_class_method_ivar_metadata", + ); + expect(baselineClassCoverage).toBeDefined(); + expect(classCoverage).toEqual(baselineClassCoverage); + expect( + result.coverage.find( + ({ facet }) => facet === "objc_properties_categories", + ), + ).toMatchObject({ + status: "partial", + reason: expect.stringContaining("category method FixtureCategory"), + }); + }); + it("keeps malformed category pointers out of class failure coverage", () => { + const baseline = decodeAppleDispatchMetadata(fixture(), 100, provenance); + const result = decodeAppleDispatchMetadata( + fixture(false, "unmapped-pointer"), + 100, + provenance, + ); + const baselineClassCoverage = baseline.coverage.find( + ({ facet }) => facet === "objc_class_method_ivar_metadata", + ); + const classCoverage = result.coverage.find( + ({ facet }) => facet === "objc_class_method_ivar_metadata", + ); + expect(baselineClassCoverage).toBeDefined(); + expect(classCoverage).toEqual(baselineClassCoverage); + expect( + result.coverage.find( + ({ facet }) => facet === "objc_properties_categories", + ), + ).toMatchObject({ + status: "partial", + reason: expect.stringContaining("category_decode_failed"), + }); + }); +}); + +describe("malformed Apple dispatch metadata", () => { it("rejects malformed command and section boundaries", () => { const bytes = fixture(); bytes.writeUInt32LE(7, 36); diff --git a/src/native/AppleMachoFixups.test.ts b/src/native/AppleMachoFixups.test.ts index 40c50c59b..bf4ae348f 100644 --- a/src/native/AppleMachoFixups.test.ts +++ b/src/native/AppleMachoFixups.test.ts @@ -149,6 +149,64 @@ const SEGMENTS: Segment[] = [ { address: BASE + 0x4000n, size: 0x4000n, offset: 0x4000, executable: false }, ]; +describe("independent chained segment coverage", () => { + it("counts successfully decoded chained segments alongside an unsupported segment", () => { + const starts = 28; + const startsSize = 4 + 2 * 4; + const firstInfo = starts + startsSize; + const secondInfo = firstInfo + 24; + const imports = secondInfo + 24; + const data = Buffer.alloc(0x3000); + data.writeUInt32LE(starts, 4); + data.writeUInt32LE(imports, 8); + data.writeUInt32LE(imports, 12); + data.writeUInt32LE(1, 20); + data.writeUInt32LE(2, starts); + data.writeUInt32LE(firstInfo - starts, starts + 4); + data.writeUInt32LE(secondInfo - starts, starts + 8); + for (const [info, format, segmentOffset] of [ + [firstInfo, 6, 0], + [secondInfo, 99, 0x1000], + ] as const) { + data.writeUInt32LE(24, info); + data.writeUInt16LE(0x1000, info + 4); + data.writeUInt16LE(format, info + 6); + data.writeBigUInt64LE(BigInt(segmentOffset), info + 8); + data.writeUInt16LE(1, info + 20); + data.writeUInt16LE(8, info + 22); + } + const segments: Segment[] = [ + { address: BASE, size: 0x1000n, offset: 0x1000, executable: false }, + { + address: BASE + 0x1000n, + size: 0x1000n, + offset: 0x2000, + executable: false, + }, + ]; + const fixups = parsePointerFixups( + data, + { + chained: { offset: 0, size: imports }, + binds: [], + rebases: [], + dylibs: [], + }, + segments, + BASE, + ); + + expect(fixups).toMatchObject({ examined: 2, decoded: 1 }); + expect(fixups.issues).toMatchObject([ + { code: "unsupported_pointer_format" }, + ]); + expect(fixups.decode(BASE + 8n, 0n)).toEqual({ + kind: "rebase", + target: BASE, + }); + }); +}); + describe("pointer fixup tables", () => { it("reports unsupported pointer formats even when no pointer is visited", () => { const data = chainedData(1); @@ -196,20 +254,27 @@ describe("pointer fixup tables", () => { { chained: { offset: 0, size: 128 }, binds: [], - rebases: [], + rebases: [{ offset: 8, size: 12 }], dylibs: [], }, SEGMENTS, BASE, ); + expect(fixups).toMatchObject({ + examined: 1, + decoded: 1, + unreadRebaseBytes: 12, + }); expect(fixups.decode(BASE + 0x4008n, field(1n, 63))).toMatchObject({ kind: "bind", symbol: "_OBJC_CLASS_$_NSObject", }); expect(fixups.issues).toEqual([]); + expect(fixups.decoded).toBe(1); expect(fixups.decode(BASE + 0x4008n, 1n | field(1n, 63))).toMatchObject({ kind: "unsupported", }); + expect(fixups.decoded).toBe(0); expect(fixups.issues).toMatchObject([ { code: "chained_pointer_decode_failed" }, ]); diff --git a/src/native/AppleMachoFixups.ts b/src/native/AppleMachoFixups.ts index 2ec711cc8..09080ec1d 100644 --- a/src/native/AppleMachoFixups.ts +++ b/src/native/AppleMachoFixups.ts @@ -42,6 +42,8 @@ export interface PointerFixups { readonly issues: readonly DecodeIssue[]; /** Fixup segments or bind streams this reader walked. */ readonly examined: number; + /** Fully decoded fixup segments or bind streams, in the same units as `examined`. */ + readonly decoded: number; /** Rebase opcode bytes present in the load command and not decoded. */ readonly unreadRebaseBytes: number; /** Decode the pointer stored at `address` whose raw 64-bit value is `raw`. */ @@ -260,6 +262,7 @@ const readChainedPage = ( const chainedFixups = ( image: FixupImage, range: { readonly offset: number; readonly size: number }, + unreadRebaseBytes: number, ): PointerFixups => { const { bytes, segments, dylibs, baseAddress } = image; const base = range.offset; @@ -400,19 +403,25 @@ const chainedFixups = ( ] : [], ); - const examinedViews = new Set(); - for (const chain of ranges) { + const examinedViews = new Map(); + const decodedRanges = new Set(); + for (const [chainIndex, chain] of ranges.entries()) { // Unknown formats retain their compact starts table; only a queried page // needs its NONE/presence value. Do not expand unsupported declarations. if (POINTER_FORMATS[chain.format] === undefined) continue; - const view = `${chain.pageStarts}:${chain.segment.offset}:${chain.segment.size}:${chain.segment.address}`; - if (examinedViews.has(view)) continue; - examinedViews.add(view); + const view = `${chain.pageStarts}:${chain.segment.offset}:${chain.segment.size}:${chain.segment.address}:${chain.format}:${chain.pageSize}:${chain.pageCount}`; + const prior = examinedViews.get(view); + if (prior !== undefined) { + if (prior) decodedRanges.add(chainIndex); + continue; + } + let complete = true; for (let page = 0; page < chain.pageCount; page++) { const start = bytes.readUInt16LE(chain.pageStarts + page * 2); if (start === 0xffff) continue; const facts = pageFacts(chain, page, start); if (facts.failure !== undefined) { + complete = false; issues.push( issue( "pointer_fixups", @@ -427,6 +436,8 @@ const chainedFixups = ( break; } } + examinedViews.set(view, complete); + if (complete) decodedRanges.add(chainIndex); } const failures = issues.map(({ message }) => message); return { @@ -435,13 +446,17 @@ const chainedFixups = ( failures, issues, examined: Math.max(ranges.length, 1), - unreadRebaseBytes: 0, + get decoded() { + return ranges.length === 0 ? 1 : decodedRanges.size; + }, + unreadRebaseBytes, decode(address, raw) { - const chain = ranges.find( + const chainIndex = ranges.findIndex( ({ segment }) => address >= segment.address && address < segment.address + segment.size, ); + const chain = ranges[chainIndex]; // Pointers outside fixup segments, such as in __TEXT, are stored plainly. if (chain === undefined) return { kind: "rebase", target: raw }; const within = Number(address - chain.segment.address); @@ -486,6 +501,7 @@ const chainedFixups = ( ); failures.push(reason); } + decodedRanges.delete(chainIndex); return { kind: "unsupported", reason }; } }, @@ -684,10 +700,7 @@ export const parsePointerFixups = ( 0, ); if (commands.chained !== null) { - const chained = chainedFixups(image, commands.chained); - return unreadRebaseBytes === 0 - ? chained - : { ...chained, unreadRebaseBytes }; + return chainedFixups(image, commands.chained, unreadRebaseBytes); } if (commands.binds.length === 0) return { @@ -696,14 +709,17 @@ export const parsePointerFixups = ( failures: [], issues: [], examined: 0, + decoded: 0, unreadRebaseBytes, decode: (_address, raw) => ({ kind: "rebase", target: raw }), }; const binds: BindRun[] = []; const failures: string[] = []; + let decoded = 0; for (const stream of commands.binds) try { readBindStream(image, stream, binds); + decoded++; } catch (cause: unknown) { failures.push( `${stream.stream} bind opcodes: ${cause instanceof Error ? cause.message : String(cause)}`, @@ -717,6 +733,7 @@ export const parsePointerFixups = ( issue("pointer_fixups", "bind_stream_decode_failed", message), ), examined: commands.binds.length, + decoded, unreadRebaseBytes, decode(address, raw) { let match: BindRun | undefined; diff --git a/src/native/AppleObjcCategories.ts b/src/native/AppleObjcCategories.ts index bd936e0ce..7beb7fc03 100644 --- a/src/native/AppleObjcCategories.ts +++ b/src/native/AppleObjcCategories.ts @@ -23,7 +23,6 @@ export const decodeObjcCategories = (input: { readonly sections: readonly Section[]; readonly read: ObjcDispatchReaders; readonly result: ObjcSwiftMetadata; - readonly failures: string[]; readonly issues: DecodeIssue[]; readonly admit: () => boolean; /** Decode a method list into implementations attributed to the category. */ @@ -37,7 +36,7 @@ export const decodeObjcCategories = (input: { /** Name of a class defined in this image, from its class_ro_t. */ readonly localClassName: (address: bigint) => string; }): number => { - const { read, result, failures } = input; + const { read, result } = input; let examined = 0; for (const section of input.sections.filter( ({ name }) => name === "__objc_catlist", @@ -93,7 +92,6 @@ export const decodeObjcCategories = (input: { }); } catch (cause: unknown) { const message = cause instanceof Error ? cause.message : String(cause); - failures.push(`Category at 0x${field.toString(16)}: ${message}`); input.issues.push( issue( "objc_properties_categories", diff --git a/src/native/AppleObjcDispatchFacets.ts b/src/native/AppleObjcDispatchFacets.ts index 14f0b42b8..b52545840 100644 --- a/src/native/AppleObjcDispatchFacets.ts +++ b/src/native/AppleObjcDispatchFacets.ts @@ -335,7 +335,6 @@ export const decodeObjcDispatchFacets = (input: { sections, read, result, - failures, issues: categoryIssues, admit: () => budget.admit(), methods, diff --git a/src/native/AppleSwiftVtables.test.ts b/src/native/AppleSwiftVtables.test.ts index dbb1c12b5..fb1615820 100644 --- a/src/native/AppleSwiftVtables.test.ts +++ b/src/native/AppleSwiftVtables.test.ts @@ -67,6 +67,7 @@ describe("Swift vtable coverage facts: unresolved Swift dispatch slots", () => { issues: [], decode: (_address, raw) => ({ kind: "rebase", target: raw }), examined: 0, + decoded: 0, unreadRebaseBytes: 0, }, swift: { @@ -181,6 +182,7 @@ describe("Swift vtable coverage facts: unsupported pointer fixup coverage", () = reason: "Unsupported chained pointer format 99", }), examined: 0, + decoded: 0, unreadRebaseBytes: 0, }, swift: { From c105391db53e2e2ab8b8624d5f3870c14aae3637 Mon Sep 17 00:00:00 2001 From: Ysopwnable <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:59:22 +0800 Subject: [PATCH 017/159] fix(hopper): stop Linux demo children when the REA owner exits (#2499) Bind the owned demo helper to REA's private stdin stream and unwind child cleanup on EOF during startup or analysis. Verify graceful and SIGKILL owner exit through real CLI/MCP workflows and preserve unrelated processes. Assisted-by: Reimu (gensokyo-skills:reimu-incident-triage) --- docs/installation.md | 19 ++- scripts/hopper-demo-x11.py | 56 +++++++- scripts/lib/real-hopper-owner-exit.mjs | 130 ++++++++++++++++++ scripts/verify-real-hopper.mjs | 9 ++ src/hopper/BridgeLauncher.ts | 3 + .../providers/hopper/bridgeLauncher.test.ts | 58 ++++++++ .../hopper/linuxPrivateDisplay.test.ts | 20 +++ tests/fixtures/hopperLifetimeLaunch.py | 16 +++ tests/fixtures/hopperOwnerLifetime.py | 110 +++++++++++++++ 9 files changed, 412 insertions(+), 9 deletions(-) create mode 100644 scripts/lib/real-hopper-owner-exit.mjs create mode 100644 tests/fixtures/hopperLifetimeLaunch.py create mode 100644 tests/fixtures/hopperOwnerLifetime.py diff --git a/docs/installation.md b/docs/installation.md index b9285a7d4..0a8041898 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -317,8 +317,8 @@ The Hopper launcher action used by REA creates a document from an executable; its supported command-line interface does not attach to an already-open document. While another live REA session owns the same target and loader profile, a second session reports the owning run ID instead of opening a -duplicate document. Closing the owning REA session releases this guard; Hopper -keeps its document open. A later REA session can therefore open another +duplicate document. On macOS, closing the owning REA session releases this guard; +Hopper keeps its document open. A later REA session can therefore open another document for the same target. REA does not currently identify, focus, or reuse that existing GUI document, and the supported launcher exposes no attach or reuse action for it. @@ -341,8 +341,14 @@ Hopper application per user across cooperating REA processes. A competing CLI or MCP session receives the owning session ID before Hopper is launched, for both the same target and a different target. Switch targets through that MCP session, or close it before starting another session. Closing the owner -releases the reservation. This does not attach to or claim ownership of -manually opened Hopper applications. +releases the reservation and stops REA's owned demo application and private +display; the headless Linux demo does not retain documents after MCP shutdown. +Its helper watches a private lifetime pipe, so an owner crash or SIGKILL also +stops those children, without relying on a reusable PID or a process-name scan. +A later session recovers the dead owner's stale lease after child cleanup; +private runtime files can remain when REA itself was killed. This does not attach to +or claim ownership of manually opened Hopper applications. An application left +running by an older REA version must still be closed before starting a new session. Stale lease recovery is serialized. If an interrupted recovery leaves its reservation directory, the diagnostic identifies that exact path. Stop REA @@ -435,8 +441,9 @@ For cursor navigation, string decoding, symbol-name limits, and regex matching, see [Hopper analysis behavior](native-investigation.md#hopper-analysis-behavior). Closing or switching a target closes its bound Hopper document, shuts down REA's -bridge and removes its temporary socket directory while preserving the Hopper -application and unrelated documents. A `cleanup_incomplete` +bridge and removes its temporary socket directory. The macOS Hopper application +and unrelated documents remain open; the owned Linux demo application stops. +A `cleanup_incomplete` result identifies resources whose cleanup could not be verified. REA retains unresolved cleanup ownership and any confirmed shutdown phases. After addressing the reported failure, retry `close_binary` on the same diff --git a/scripts/hopper-demo-x11.py b/scripts/hopper-demo-x11.py index 470c9fd28..da2fe884f 100644 --- a/scripts/hopper-demo-x11.py +++ b/scripts/hopper-demo-x11.py @@ -10,6 +10,7 @@ import json import os from pathlib import Path import secrets +import select import stat import subprocess import tempfile @@ -118,6 +119,7 @@ class XWindowAttributes(ctypes.Structure): def parse_args() -> argparse.Namespace: parser = argparse.ArgumentParser() parser.add_argument("--probe", action="store_true") + parser.add_argument("--owner-stdin", action="store_true") parser.add_argument("--hopper") parser.add_argument("--socket") parser.add_argument( @@ -133,6 +135,43 @@ def parse_args() -> argparse.Namespace: return parser.parse_args() +class OwnerLifetime: + """The launcher's private pipe is a lifetime capability, independent of PIDs.""" + + def __init__(self, enabled: bool) -> None: + self._poll = select.poll() if enabled else None + if self._poll is not None: + # Node/libuv can implement piped stdio with a Unix socket pair. + mode = os.fstat(0).st_mode + if not (stat.S_ISFIFO(mode) or stat.S_ISSOCK(mode)): + raise AdapterFailure( + INVALID_LAUNCH_COMMAND, + "invalid_launch_command", + "owner lifetime requires a private stdin pipe", + ) + self._poll.register(0, select.POLLIN | select.POLLHUP | select.POLLERR) + + def alive(self) -> bool: + if self._poll is None: + return True + for _, events in self._poll.poll(0): + if events & (select.POLLHUP | select.POLLERR | select.POLLNVAL): + return False + if events & select.POLLIN and not os.read(0, 4096): + return False + return True + + +def wait_for_hopper(child: subprocess.Popen[bytes], owner: OwnerLifetime) -> int: + while owner.alive(): + try: + return child.wait(timeout=0.1) + except subprocess.TimeoutExpired: + continue + # run_launch's finally owns cleanup of both direct, unreaped children. + return 0 + + def digest(path: Path) -> str: value = hashlib.sha256() with path.open("rb") as file: @@ -572,6 +611,7 @@ def start_xvfb( "-auth", str(authority), ], + stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE, ) @@ -627,6 +667,9 @@ def run_launch(args: argparse.Namespace) -> int: "invalid_launch_command", "missing Hopper launch coordinates", ) + owner = OwnerLifetime(args.owner_stdin) + if not owner.alive(): + return 0 hopper = Path(args.hopper) socket = Path(args.socket) try: @@ -654,6 +697,8 @@ def run_launch(args: argparse.Namespace) -> int: child: subprocess.Popen[bytes] | None = None display: int | None = None try: + if not owner.alive(): + return 0 hopper_environment = dict(os.environ) for name in ("WAYLAND_DISPLAY", "WAYLAND_SOCKET"): hopper_environment.pop(name, None) @@ -666,7 +711,10 @@ def run_launch(args: argparse.Namespace) -> int: "XDG_SESSION_TYPE": "x11", } ) - child = subprocess.Popen(command, env=hopper_environment) + # Children must not consume or retain REA's lifetime pipe. + child = subprocess.Popen( + command, env=hopper_environment, stdin=subprocess.DEVNULL + ) leader = os.getpid() os.environ["XAUTHORITY"] = str(authority) @@ -702,10 +750,12 @@ def run_launch(args: argparse.Namespace) -> int: observed: list[tuple[int, int, int, int, int]] = [] dialog: tuple[int, int, int, int, int] | None = None while time.monotonic() < deadline: + if not owner.alive(): + return 0 if socket.exists(): x11.XCloseDisplay(display) display = None - return child.wait() + return wait_for_hopper(child, owner) if not hopper_is_owned(leader, hopper): if child.poll() is None: time.sleep(0.1) @@ -749,7 +799,7 @@ def run_launch(args: argparse.Namespace) -> int: x11.XFlush(display) x11.XCloseDisplay(display) display = None - return child.wait() + return wait_for_hopper(child, owner) except AdapterFailure as error: raise error.with_stderr(capture) finally: diff --git a/scripts/lib/real-hopper-owner-exit.mjs b/scripts/lib/real-hopper-owner-exit.mjs new file mode 100644 index 000000000..300b35ada --- /dev/null +++ b/scripts/lib/real-hopper-owner-exit.mjs @@ -0,0 +1,130 @@ +import assert from "node:assert/strict"; +import { execFile } from "node:child_process"; +import { setTimeout as delay } from "node:timers/promises"; +import { promisify } from "node:util"; + +import { Client } from "@modelcontextprotocol/client"; +import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; + +import { verifyNoTokenOwnedProcesses } from "../../dist/process/ProcessOwnership.js"; +import { + observeProcessStartIdentity, + signalProcessWithStartIdentity, +} from "../../dist/process/ProcessOwnershipObservation.js"; +import { startUnrelatedHopperSentinel } from "./unrelated-hopper-sentinel.mjs"; +import { + requireMcpEvidenceResult, + requireMcpLifecycleResult, +} from "./mcp-verifier-results.mjs"; + +const execFileAsync = promisify(execFile); + +/** Prove actual MCP owner exit releases the demo singleton for subsequent CLI/MCP sessions. */ +export async function verifyLinuxHopperOwnerExit({ + primary, + secondary, + serverCommand, + serverArgs, + environment, +}) { + const sentinel = await startUnrelatedHopperSentinel(); + const outcomes = []; + try { + for (const mode of ["graceful", "SIGKILL"]) { + const transport = new StdioClientTransport({ + command: serverCommand, + args: serverArgs, + env: environment, + stderr: "pipe", + }); + // Drain launcher diagnostics even on the deliberately interrupted owner. + transport.stderr?.resume(); + const client = new Client({ name: "hopper-owner-exit", version: "1" }); + const options = { timeout: 180_000 }; + try { + await client.connect(transport); + const ownerPid = transport.pid; + assert.ok(ownerPid !== null); + const owner = await observeProcessStartIdentity(ownerPid); + assert.equal(owner?.state, "readable"); + requireMcpLifecycleResult( + await client.callTool( + { + name: "open_binary", + arguments: { path: primary }, + }, + options, + ), + "open_binary", + ); + const procedures = requireMcpEvidenceResult( + await client.callTool( + { name: "list_procedures", arguments: {} }, + options, + ), + "list_procedures", + ); + assert.ok(procedures.length > 0); + const status = requireMcpLifecycleResult( + await client.callTool( + { name: "binary_session", arguments: {} }, + options, + ), + "binary_session", + ); + const runId = status.analysis_run?.run_id; + assert.equal(typeof runId, "string"); + assert.equal(status.analysis_provider_binding?.provider.id, "hopper"); + assert.equal((await verifyNoTokenOwnedProcesses(runId)).cleaned, false); + if (mode === "graceful") await client.close(); + else { + assert.equal( + await signalProcessWithStartIdentity( + ownerPid, + owner.identity, + "SIGKILL", + ), + "signaled", + ); + } + const deadline = Date.now() + 10_000; + let cleanup; + do { + cleanup = await verifyNoTokenOwnedProcesses(runId); + if (cleanup.cleaned) break; + await delay(100); + } while (Date.now() < deadline); + assert.equal(cleanup.cleaned, true, cleanup.reason); + // The SIGKILL case must recover the stale lease for both target identities. + for (const target of [primary, secondary]) { + const result = await execFileAsync( + process.execPath, + [ + "scripts/rea.mjs", + "inspect", + target, + "--provider", + "hopper", + "--format", + "json", + ], + { env: environment, timeout: 180_000, maxBuffer: 16 * 1024 * 1024 }, + ); + assert.ok(JSON.parse(result.stdout).normalized_result); + } + outcomes.push({ + mode, + ownedProcessesStopped: true, + sameTargetReopened: true, + differentTargetReopened: true, + }); + } finally { + await client.close(); + await transport.close(); + } + } + } finally { + assert.equal(await sentinel.verifyAndClose(), true); + } + return { outcomes, unrelatedHopperSurvived: true }; +} diff --git a/scripts/verify-real-hopper.mjs b/scripts/verify-real-hopper.mjs index 12400cb94..8cad9a256 100644 --- a/scripts/verify-real-hopper.mjs +++ b/scripts/verify-real-hopper.mjs @@ -47,6 +47,7 @@ import { } from "./lib/real-hopper-boundaries.mjs"; import { startUnrelatedHopperSentinel } from "./lib/unrelated-hopper-sentinel.mjs"; import { snapshotHopperRuntime } from "./lib/real-hopper-cleanup.mjs"; +import { verifyLinuxHopperOwnerExit } from "./lib/real-hopper-owner-exit.mjs"; const execFileAsync = promisify(execFile); const verifierRun = createVerifierRun(); const timeout = 180_000; @@ -577,6 +578,14 @@ if (sessionsAfter.length > 0) { } if (summary === undefined) throw new Error("Real-Hopper verification did not produce a summary"); +if (process.platform === "linux") + summary.ownerExit = await verifyLinuxHopperOwnerExit({ + primary: targetA, + secondary: targetB, + serverCommand, + serverArgs, + environment: serverEnvironment, + }); const completedVerifierRun = await completeVerifierRun(verifierRun); if ( completedVerifierRun.process_lineage.status !== "verified" || diff --git a/src/hopper/BridgeLauncher.ts b/src/hopper/BridgeLauncher.ts index 8201ee2df..043d394a4 100644 --- a/src/hopper/BridgeLauncher.ts +++ b/src/hopper/BridgeLauncher.ts @@ -413,6 +413,8 @@ const launchHopperProcess = async (input: { runId: input.session.runId, expectedCommand: ownershipCommand, hostEnvironment: input.options.environment, + // The helper observes EOF when this REA process exits, including SIGKILL. + stdin: "pipe", ...(input.signal === undefined ? {} : { signal: input.signal }), }); return spawn(input.argumentsForTarget); @@ -434,6 +436,7 @@ export const linuxDemoLaunch = ( return undefined; const helperArguments = [ options.demoHelperPath, + "--owner-stdin", "--strategy", strategy ?? "direct", ...(strategy === "user-mount-namespace" ? ["--mount-private-x11"] : []), diff --git a/tests/boundary/providers/hopper/bridgeLauncher.test.ts b/tests/boundary/providers/hopper/bridgeLauncher.test.ts index 722a25bed..542858a0d 100644 --- a/tests/boundary/providers/hopper/bridgeLauncher.test.ts +++ b/tests/boundary/providers/hopper/bridgeLauncher.test.ts @@ -84,6 +84,7 @@ describe("Hopper bridge launcher selection", () => { "private", "/usr/bin/python3", "/rea/hopper-demo-x11.py", + "--owner-stdin", "--strategy", "user-mount-namespace", "--mount-private-x11", @@ -117,6 +118,7 @@ describe("Hopper bridge launcher selection", () => { command: "/usr/bin/python3", args: [ "/rea/hopper-demo-x11.py", + "--owner-stdin", "--strategy", "direct", "--hopper", @@ -202,3 +204,59 @@ it("launches Hopper helpers with the caller-selected environment without restori await launched.value.releaseLease?.(); } }); + +it.skipIf(process.platform !== "linux")( + "gives the demo helper a live owner stream through Node's actual stdio transport", + async () => { + const directory = await createTestTempDirectory("rea-hopper-owner-stdio-"); + const launcher = new HopperApplicationLauncher( + { + ...options(process.execPath), + launchMode: "verified_linux_demo", + demoHelperPath: fileURLToPath( + new URL("../../../fixtures/hopperLifetimeLaunch.py", import.meta.url), + ), + }, + { + selectPrivateDisplay: async () => ({ + ok: true, + strategy: "direct", + diagnostic: { + component: "hopper_private_display", + operation: "probe", + status: "ready", + failure_code: null, + reason: "ready", + socket_directory: "/tmp/.X11-unix", + socket_directory_mode: "1777", + mount_read_only: false, + effective_socket_directory_mode: "1777", + effective_mount_read_only: false, + wsl: false, + strategy: "direct", + fallback_reason: null, + xvfb_stderr_bytes: 0, + }, + }), + }, + ); + const launched = await launcher.launch({ + directory, + socketPath: join(directory, "bridge.sock"), + token: "fixture-token", + runId: randomUUID(), + }); + if (!launched.ok) throw launched.error; + const supervisor = new ProviderProcessSupervisor(launched.value); + try { + expect(await supervisor.waitForOutputClose(5_000)).toBe(true); + expect(supervisor.snapshot().exitCode).toBe(0); + expect(JSON.parse(supervisor.snapshot().stdout.text)).toEqual({ + owner_alive: true, + }); + } finally { + await supervisor.stop(); + await launched.value.releaseLease?.(); + } + }, +); diff --git a/tests/boundary/providers/hopper/linuxPrivateDisplay.test.ts b/tests/boundary/providers/hopper/linuxPrivateDisplay.test.ts index f2c99980f..c5fd45c96 100644 --- a/tests/boundary/providers/hopper/linuxPrivateDisplay.test.ts +++ b/tests/boundary/providers/hopper/linuxPrivateDisplay.test.ts @@ -31,6 +31,9 @@ const dialogDetectionFixturePath = fileURLToPath( const cleanupTimeoutFixturePath = fileURLToPath( new URL("../../../fixtures/hopperCleanupTimeout.py", import.meta.url), ); +const ownerLifetimeFixturePath = fileURLToPath( + new URL("../../../fixtures/hopperOwnerLifetime.py", import.meta.url), +); type DiagnosticOverrides = Partial< Omit @@ -107,6 +110,23 @@ describe("Hopper demo dialog detection", () => { }); describe("Hopper process cleanup", () => { + it.skipIf(process.platform !== "linux").each(["startup", "ready"])( + "reaps only its own children when the owner pipe closes during %s", + (phase) => { + const result = spawnSync( + "/usr/bin/python3", + [ownerLifetimeFixturePath, helperPath, phase], + { encoding: "utf8", timeout: 10_000 }, + ); + expect(result.status, result.stderr).toBe(0); + expect(JSON.parse(result.stdout)).toEqual({ + phase, + children_cleaned: true, + sentinel_alive: true, + }); + }, + ); + it.skipIf(process.platform !== "linux")( "kills a child that ignores SIGTERM after the bounded timeout", () => { diff --git a/tests/fixtures/hopperLifetimeLaunch.py b/tests/fixtures/hopperLifetimeLaunch.py new file mode 100644 index 000000000..68d65c9ff --- /dev/null +++ b/tests/fixtures/hopperLifetimeLaunch.py @@ -0,0 +1,16 @@ +#!/usr/bin/env python3 +"""Inspect the real Node launcher stdio with the production lifetime parser.""" + +import importlib.util +import json +from pathlib import Path +import sys + +spec = importlib.util.spec_from_file_location( + "hopper_demo_x11", Path(__file__).resolve().parents[2] / "scripts/hopper-demo-x11.py" +) +assert spec is not None and spec.loader is not None +helper = importlib.util.module_from_spec(spec) +spec.loader.exec_module(helper) +assert "--owner-stdin" in sys.argv +print(json.dumps({"owner_alive": helper.OwnerLifetime(True).alive()})) diff --git a/tests/fixtures/hopperOwnerLifetime.py b/tests/fixtures/hopperOwnerLifetime.py new file mode 100644 index 000000000..741bffbd9 --- /dev/null +++ b/tests/fixtures/hopperOwnerLifetime.py @@ -0,0 +1,110 @@ +#!/usr/bin/env python3 +"""Exercise the production runner with real children and a simulated X11 surface.""" + +import argparse +import importlib.util +import json +from pathlib import Path +import signal +import subprocess +import sys +import tempfile +import time + + +spec = importlib.util.spec_from_file_location("hopper_demo_x11", sys.argv[1]) +assert spec is not None and spec.loader is not None +helper = importlib.util.module_from_spec(spec) +spec.loader.exec_module(helper) + + +def child_command(receipt): + return [ + sys.executable, + "-c", + "import os,pathlib,time;" + f"receipt=pathlib.Path({str(receipt)!r});" + "pending=receipt.with_suffix('.pending');" + "pending.write_text(str(os.getpid()));pending.replace(receipt);" + "time.sleep(100)", + ] + + +if len(sys.argv) > 3: + phase, directory = sys.argv[2], Path(sys.argv[3]) + + def terminate_worker(*_): + raise SystemExit(143) + + # Failed assertions in the parent must still unwind run_launch's finally. + signal.signal(signal.SIGTERM, terminate_worker) + + class X11: + def XOpenDisplay(self, *_): + return 1 + + def XDefaultRootWindow(self, *_): + return 1 + + def XDisplayWidth(self, *_): + return 1280 + + def XDisplayHeight(self, *_): + return 1024 + + def XCloseDisplay(self, *_): + return 1 + + def start_xvfb(*_): + child = subprocess.Popen( + child_command(directory / "xvfb.pid"), + stdin=subprocess.DEVNULL, + stdout=subprocess.DEVNULL, + stderr=subprocess.PIPE, + ) + return child, ":123", directory / "Xauthority", helper.CapturedStderr(child.stderr) + + helper.digest = lambda _: next(iter(helper.SUPPORTED_HOPPER_SHA256)) + helper.start_xvfb = start_xvfb + helper.ctypes.CDLL = lambda _: X11() + helper.configure_x11 = lambda *_: None + helper.windows = lambda *_: [] + socket = directory / "bridge.sock" + if phase == "ready": + socket.touch() + raise SystemExit(helper.run_launch(argparse.Namespace( + owner_stdin=True, + hopper=sys.executable, + socket=str(socket), + command=child_command(directory / "hopper.pid"), + ))) + + +with tempfile.TemporaryDirectory(prefix="rea-owner-lifetime-") as root: + directory = Path(root) + sentinel = subprocess.Popen([sys.executable, "-c", "import time;time.sleep(100)"]) + worker = subprocess.Popen( + [sys.executable, __file__, sys.argv[1], sys.argv[2], root], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + ) + try: + deadline = time.monotonic() + 5 + receipts = [directory / "hopper.pid", directory / "xvfb.pid"] + while not all(receipt.exists() for receipt in receipts): + assert worker.poll() is None, worker.communicate() + assert time.monotonic() < deadline, "owned children did not start" + time.sleep(0.01) + pids = [int(receipt.read_text()) for receipt in receipts] + assert worker.poll() is None, "live owner did not retain the helper" + assert worker.stdin is not None + worker.stdin.close() + assert worker.wait(timeout=5) == 0, worker.stderr.read() + for pid in pids: + assert not Path(f"/proc/{pid}").exists(), f"owned child {pid} survived EOF" + assert sentinel.poll() is None, "unrelated process was stopped" + print(json.dumps({"phase": sys.argv[2], "children_cleaned": True, "sentinel_alive": True})) + finally: + helper.stop_process(worker) + helper.stop_process(sentinel) From acf75e6700ace28bb1db25e54e56673ac15e0ed3 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:59:17 +0800 Subject: [PATCH 018/159] fix(evidence): reject unsupported metadata before authentication Close persisted Evidence, bundle, profile and unknown record shapes so parsing cannot silently remove caller metadata before checking semantic identity. Keep constructor provider projection explicit and JSON payload fields open. Exercise actual MCP export/import with independent record and manifest mutations, unchanged state after rejection, and a valid round trip. Prune duplicate source assertions. Validation: focused domain/CLI/MCP/filesystem checks, converted JSON Schemas, docs and typecheck/lint. --- docs/mcp-contracts.md | 6 + src/domain/analysisProfile.test.ts | 7 + src/domain/analysisProfile.ts | 10 +- src/domain/evidence.test.ts | 24 ++ src/domain/evidence.ts | 21 +- src/domain/evidenceBundle.ts | 19 +- src/domain/residualUnknown.ts | 12 +- .../boundary/mcp/evidenceBundleInline.test.ts | 258 ++++++++++++++++++ 8 files changed, 331 insertions(+), 26 deletions(-) diff --git a/docs/mcp-contracts.md b/docs/mcp-contracts.md index b6990b3c9..c4169dd73 100644 --- a/docs/mcp-contracts.md +++ b/docs/mcp-contracts.md @@ -21,6 +21,12 @@ directly to `compare_functions`, and `inspect_artifact` Evidence to `compare_artifacts`. Use `get_evidence_bundle` when the task needs broader retained session history or an explicit bundle for transfer. +Evidence and bundle imports reject unsupported fields in their envelopes, +identities, manifests and other fixed metadata before authentication. Imports +leave retained session state unchanged when validation fails. Keep application +metadata in the supported JSON payloads (`parameters`, `raw_result`, +`normalized_result`, or analysis-profile parameters), where keys are preserved. + Tool execution failures return `isError: true` and the complete canonical `{ "error": ... }` diagnostic as JSON in text `content`. Read that diagnostic for the error code, target details, partial observations, and remediation. diff --git a/src/domain/analysisProfile.test.ts b/src/domain/analysisProfile.test.ts index 1fb18f0ae..a39fb389d 100644 --- a/src/domain/analysisProfile.test.ts +++ b/src/domain/analysisProfile.test.ts @@ -65,4 +65,11 @@ describe("analysis profile commitments", () => { analysisProfileSchema.parse({ ...profile, digest: "0".repeat(64) }), ).toThrow(/digest/u); }); + + it("projects known provider identity fields at construction", () => { + const liveProvider = { ...PROVIDER, transient_status: "starting" }; + const profile = createAnalysisProfile(liveProvider, { loader: "default" }); + expect(profile.provider).toEqual(PROVIDER); + expect(analysisProfileSchema.parse(profile)).toEqual(profile); + }); }); diff --git a/src/domain/analysisProfile.ts b/src/domain/analysisProfile.ts index 6719e2ccd..f6e6e0004 100644 --- a/src/domain/analysisProfile.ts +++ b/src/domain/analysisProfile.ts @@ -7,13 +7,13 @@ import { jsonObjectSchema, type JsonValue } from "./jsonValue.js"; import { digestSchema } from "./../domain/digests.js"; /** Concrete provider identity committed by an analysis profile. */ -export const committedProviderSchema = z.object({ +export const committedProviderSchema = z.strictObject({ id: z.string().min(1), name: z.string().min(1), version: z.string().min(1), }); -const unsignedAnalysisProfileSchema = z.object({ +const unsignedAnalysisProfileSchema = z.strictObject({ provider: committedProviderSchema, parameters: jsonObjectSchema, }); @@ -40,7 +40,11 @@ export const createAnalysisProfile = ( parameters: Readonly>, ): AnalysisProfileCommitment => { const unsigned = unsignedAnalysisProfileSchema.parse({ - provider, + provider: { + id: provider.id, + name: provider.name, + version: provider.version, + }, parameters, }); return analysisProfileSchema.parse({ diff --git a/src/domain/evidence.test.ts b/src/domain/evidence.test.ts index c5898aecc..5cdf30a27 100644 --- a/src/domain/evidence.test.ts +++ b/src/domain/evidence.test.ts @@ -52,6 +52,30 @@ it("preserves omitted relationship diagnostics and authenticates reported diagno ).toThrow("semantic identifier"); }); +it("projects known provider fields and preserves open JSON payloads", () => { + const liveProvider = { ...PROVIDER, transient_status: "starting" }; + const evidence = createEvidence(TARGET, liveProvider, { + operation: "inspect", + parameters: { caller_metadata: { note: "kept" } }, + result: { producer_extension: { value: "kept" } }, + analysisProfile: PROFILE, + environment: { + id: "fixture", + platform: "linux", + architecture: "x86_64", + isolation: "process", + }, + locations: [{ kind: "address", address: "0x1000" }], + }); + + expect(evidence.provider).toEqual(PROVIDER); + expect(evidence.parameters).toEqual({ caller_metadata: { note: "kept" } }); + expect(evidence.normalized_result).toEqual({ + producer_extension: { value: "kept" }, + }); + expect(parseEvidence(evidence)).toEqual(evidence); +}); + it("preserves identity and bytes when reusing an owned immutable result", () => { const value = { nested: { values: ["observed", 1] } }; const mutable = createEvidence(TARGET, PROVIDER, { diff --git a/src/domain/evidence.ts b/src/domain/evidence.ts index 8e21686e5..8a450f478 100644 --- a/src/domain/evidence.ts +++ b/src/domain/evidence.ts @@ -33,14 +33,14 @@ import { } from "./relationshipCoverage.js"; /** Provider identity schema shared by evidence-bearing persistence formats. */ -export const providerSchema = z.object({ +export const providerSchema = z.strictObject({ id: z.string().min(1), name: z.string().min(1), version: z.string().nullable(), }); -const subjectSchema = z.object({ +const subjectSchema = z.strictObject({ name: z.string().min(1), - digest: z.object({ sha256: digestSchema }), + digest: z.strictObject({ sha256: digestSchema }), format: z.enum([ "hopper", "analysis-database", @@ -77,15 +77,18 @@ const subjectSchema = z.object({ /** Source location attached to an evidence observation. */ /** Source-location schema shared by evidence-bearing persistence formats. */ export const evidenceLocationSchema = z.discriminatedUnion("kind", [ - z.object({ kind: z.literal("address"), address: z.string().min(1) }), - z.object({ + z.strictObject({ kind: z.literal("address"), address: z.string().min(1) }), + z.strictObject({ kind: z.literal("address-range"), start: z.string().min(1), end: z.string().min(1), }), - z.object({ kind: z.literal("artifact-path"), path: z.string().min(1) }), - z.object({ kind: z.literal("file-offset"), offset: z.number().int().min(0) }), - z.object({ + z.strictObject({ kind: z.literal("artifact-path"), path: z.string().min(1) }), + z.strictObject({ + kind: z.literal("file-offset"), + offset: z.number().int().min(0), + }), + z.strictObject({ kind: z.literal("file-offset-range"), start: z.number().int().min(0), end: z.number().int().min(0), @@ -100,7 +103,7 @@ const evidenceAuthoritySchema = z.enum([ "analyst-inference", ]); -const executionEnvironmentSchema = z.object({ +const executionEnvironmentSchema = z.strictObject({ id: z.string().min(1), platform: z.string().min(1), architecture: z.string().min(1), diff --git a/src/domain/evidenceBundle.ts b/src/domain/evidenceBundle.ts index c1fa3ce1c..7e6fd2b3b 100644 --- a/src/domain/evidenceBundle.ts +++ b/src/domain/evidenceBundle.ts @@ -16,24 +16,27 @@ import { } from "./residualUnknown.js"; import { freezeJsonSnapshot } from "./immutableJson.js"; -const artifactManifestSchema = evidenceSchema.shape.subject.unwrap().pick({ - digest: true, - format: true, - architecture: true, -}); +const artifactManifestSchema = evidenceSchema.shape.subject + .unwrap() + .pick({ + digest: true, + format: true, + architecture: true, + }) + .strict(); const providerManifestSchema = providerSchema; const environmentManifestSchema = evidenceSchema.shape.environment.unwrap(); -const scenarioManifestSchema = z.object({ +const scenarioManifestSchema = z.strictObject({ evidence_id: evidenceSchema.shape.evidence_id, operation: z.string().min(1), authority: z.literal("controlled-replay"), }); -const captureManifestSchema = z.object({ +const captureManifestSchema = z.strictObject({ evidence_id: evidenceSchema.shape.evidence_id, predicate_type: z.string().min(1), }); -export const evidenceBundleSchema = z.object({ +export const evidenceBundleSchema = z.strictObject({ artifacts: z.array(artifactManifestSchema), providers: z.array(providerManifestSchema), environments: z.array(environmentManifestSchema), diff --git a/src/domain/residualUnknown.ts b/src/domain/residualUnknown.ts index 0819a3964..e5db1cc54 100644 --- a/src/domain/residualUnknown.ts +++ b/src/domain/residualUnknown.ts @@ -12,7 +12,7 @@ import { functionComparisonDimensionNameSchema } from "./functionComparisonSchem const evidenceIdSchema = prefixedDigestSchema("ev"); const unknownIdSchema = prefixedDigestSchema("unk"); const text = z.string().trim().min(1); -const environmentRequirementSchema = z.object({ +const environmentRequirementSchema = z.strictObject({ id: z.string().min(1).nullable(), platform: z.string().min(1).nullable(), architecture: z.string().min(1).nullable(), @@ -39,7 +39,7 @@ const stableUnknownId = ( ) .digest("hex")}`; -const probeSchema = z.object({ +const probeSchema = z.strictObject({ operation: z.string().min(1), rationale: text, }); @@ -60,17 +60,17 @@ const comparisonConclusionScopeSchema = z message: "Comparison dimensions must be sorted and unique", }); }); -const relationshipSchema = z.object({ +const relationshipSchema = z.strictObject({ type: z.enum(["depends-on", "related-to", "contradicts"]), unknown_id: unknownIdSchema, }); const resolutionSchema = z.discriminatedUnion("disposition", [ - z.object({ + z.strictObject({ disposition: z.literal("verified"), rationale: text, evidence_ids: z.array(evidenceIdSchema).min(1), }), - z.object({ + z.strictObject({ disposition: z.enum(["withdrawn", "out-of-scope"]), rationale: text, evidence_ids: z.array(evidenceIdSchema), @@ -78,7 +78,7 @@ const resolutionSchema = z.discriminatedUnion("disposition", [ ]); /** Strict durable record for one unresolved or evidence-qualified question. */ -const residualUnknownObjectSchema = z.object({ +const residualUnknownObjectSchema = z.strictObject({ unknown_id: unknownIdSchema, revision: z.number().int().min(1), previous_revision_digest: z diff --git a/tests/boundary/mcp/evidenceBundleInline.test.ts b/tests/boundary/mcp/evidenceBundleInline.test.ts index adb95ef8f..ae07a1be5 100644 --- a/tests/boundary/mcp/evidenceBundleInline.test.ts +++ b/tests/boundary/mcp/evidenceBundleInline.test.ts @@ -1,10 +1,20 @@ +import { readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; + import { ok as resultOk } from "../../../src/domain/result.js"; import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; import { expect, it } from "vitest"; import { createTestBinarySession } from "../../fixtures/binarySession.js"; import { observed } from "../../fixtures/analysisExecution.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js"; import { createEvidence } from "../../../src/domain/evidence.js"; +import { ANALYSIS_SNAPSHOT_TARGET } from "../../../src/domain/analysisSnapshot.fixture.js"; +import { jsonValueSchema } from "../../../src/domain/jsonValue.js"; +import { EMPTY_PROCESS_CAPTURE_EXAMPLE } from "../../../src/domain/process/processCaptureExample.js"; +import { PROCESS_PROVIDER } from "../../../src/domain/process/processEvidenceProvider.js"; import { createServer } from "../../../src/server/createServer.js"; it("returns the complete session Evidence bundle in the tool result", async () => { @@ -44,3 +54,251 @@ it("returns the complete session Evidence bundle in the tool result", async () = await Promise.allSettled([client.close(), server.close(), session.close()]); } }); + +it("rejects bundle field stripping at MCP import without changing retained evidence", async () => { + const root = await createTestTempDirectory("rea-evidence-bundle-mcp-"); + const sourceSession = createBundleSourceSession(); + const targetSession = createTestBinarySession(() => ({ + execute: () => Promise.resolve(observed(null)), + close: () => Promise.resolve(resultOk(null)), + })); + const source = await connectEvidenceMcp("bundle-source", sourceSession); + const target = await connectEvidenceMcp("bundle-target", targetSession); + const { client: sourceClient } = source; + const { client: targetClient } = target; + const bundlePath = join(root, "exported.json"); + const targetExportPath = join(root, "round-trip.json"); + + try { + const { sourceJson, records, evidenceIndex } = await exportFixtureBundle( + sourceClient, + bundlePath, + ); + const initial = structured( + await targetClient.callTool({ + name: "get_evidence_bundle", + arguments: {}, + }), + ).result; + for (const [label, path] of unsupportedBundleFields(evidenceIndex)) { + const candidate = structuredClone(sourceJson); + setPath(candidate, path, "unsupported-metadata"); + const malformedPath = join(root, `${label.replaceAll(" ", "-")}.json`); + await writeFile(malformedPath, JSON.stringify(candidate)); + const rejected = await targetClient.callTool({ + name: "import_evidence_bundle", + arguments: { path: malformedPath }, + }); + expect(parseMcpToolError(rejected), label).toMatchObject({ + error: { code: "evidence_integrity_mismatch" }, + }); + expect( + structured( + await targetClient.callTool({ + name: "get_evidence_bundle", + arguments: {}, + }), + ).result, + label, + ).toEqual(initial); + } + + const imported = await targetClient.callTool({ + name: "import_evidence_bundle", + arguments: { path: bundlePath }, + }); + if (imported.isError === true) + throw new Error(JSON.stringify(parseMcpToolError(imported))); + await targetClient.callTool({ + name: "export_evidence_bundle", + arguments: { path: targetExportPath }, + }); + expect(JSON.parse(await readFile(targetExportPath, "utf8"))).toEqual( + sourceJson, + ); + const observedRecord = records[evidenceIndex]!; + expect(observedRecord.parameters).toEqual({ caller: { note: "preserve" } }); + expect(observedRecord.normalized_result).toEqual({ + producer: { extension: "preserve" }, + }); + expect(object(observedRecord.analysis_profile).parameters).toEqual({ + loader: { mode: "full" }, + }); + } finally { + await Promise.allSettled([ + sourceClient.close(), + source.server.close(), + sourceSession.close(), + targetClient.close(), + target.server.close(), + targetSession.close(), + ]); + } +}); + +const createBundleSourceSession = () => { + const provider = { id: "fixture", name: "Fixture", version: "1" }; + const evidence = createEvidence(ANALYSIS_SNAPSHOT_TARGET, provider, { + operation: "observe", + parameters: { caller: { note: "preserve" } }, + result: { producer: { extension: "preserve" } }, + analysisProfile: createAnalysisProfile(provider, { + loader: { mode: "full" }, + }), + environment: { + id: "fixture-linux", + platform: "linux", + architecture: "x86_64", + isolation: "process", + }, + locations: [{ kind: "address", address: "0x1000" }], + }); + const session = createTestBinarySession(() => ({ + execute: () => Promise.resolve(observed(null)), + close: () => Promise.resolve(resultOk(null)), + })); + expect(session.recordEvidence(evidence).ok).toBe(true); + for (const extra of [ + createEvidence(ANALYSIS_SNAPSHOT_TARGET, provider, { + operation: "controlled_replay", + parameters: {}, + result: { completed: true }, + authority: "controlled-replay", + }), + createEvidence(undefined, PROCESS_PROVIDER, { + predicateType: "rea.process-capture", + operation: "capture_process_scenario", + parameters: {}, + result: jsonValueSchema.parse(EMPTY_PROCESS_CAPTURE_EXAMPLE), + confidence: "observed", + authority: "controlled-replay", + }), + ]) + expect(session.recordEvidence(extra).ok).toBe(true); + return session; +}; + +const exportFixtureBundle = async ( + client: Client, + path: string, +): Promise<{ + readonly sourceJson: unknown; + readonly records: Record[]; + readonly evidenceIndex: number; +}> => { + const unknown = await client.callTool({ + name: "record_unknown", + arguments: { + question: "Does this remain unknown?", + severity: "medium", + domain: "bundle-format", + required_authority: null, + required_confidence: "derived", + required_environment: null, + recommended_probes: [{ operation: "observe", rationale: "Inspect it." }], + relationships: [], + }, + }); + expect(unknown.isError).not.toBe(true); + const exported = await client.callTool({ + name: "export_evidence_bundle", + arguments: { path }, + }); + expect(exported.isError).not.toBe(true); + const sourceJson = JSON.parse(await readFile(path, "utf8")) as unknown; + const records = array(object(sourceJson).records).map(object); + const evidenceIndex = records.findIndex( + (record) => record.operation === "observe", + ); + expect(evidenceIndex).toBeGreaterThanOrEqual(0); + return { sourceJson, records, evidenceIndex }; +}; + +const unsupportedBundleFields = ( + evidenceIndex: number, +): readonly [string, readonly (string | number)[]][] => [ + ["bundle envelope", ["schema_version"]], + ["record provider", ["records", evidenceIndex, "provider", "future_field"]], + ["provider manifest", ["providers", 0, "future_field"]], + ["record subject", ["records", evidenceIndex, "subject", "future_field"]], + [ + "subject digest", + ["records", evidenceIndex, "subject", "digest", "future_field"], + ], + ["artifact manifest", ["artifacts", 0, "future_field"]], + [ + "record environment", + ["records", evidenceIndex, "environment", "future_field"], + ], + ["environment manifest", ["environments", 0, "future_field"]], + [ + "source location", + ["records", evidenceIndex, "locations", 0, "future_field"], + ], + ["scenario manifest", ["scenarios", 0, "future_field"]], + ["capture manifest", ["captures", 0, "future_field"]], + [ + "analysis profile", + ["records", evidenceIndex, "analysis_profile", "future_field"], + ], + [ + "profile provider", + ["records", evidenceIndex, "analysis_profile", "provider", "future_field"], + ], + ["unknown revision", ["unknowns", 0, "future_field"]], + ["unknown probe", ["unknowns", 0, "recommended_probes", 0, "future_field"]], +]; + +const connectEvidenceMcp = async ( + name: string, + session: ReturnType, +) => { + const server = createServer({ kind: "session", session }); + const client = new Client({ name, version: "1" }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + await server.connect(serverTransport); + await client.connect(clientTransport); + return { client, server }; +}; + +const object = (value: unknown): Record => { + if (typeof value !== "object" || value === null || Array.isArray(value)) + throw new Error("Expected object in exported Evidence bundle"); + return value as Record; +}; + +const array = (value: unknown): unknown[] => { + if (!Array.isArray(value)) + throw new Error("Expected array in Evidence bundle"); + return value; +}; + +const setPath = ( + root: unknown, + path: readonly (string | number)[], + value: unknown, +): void => { + const [last, ...parents] = [...path].reverse(); + if (last === undefined) throw new Error("Expected a nonempty bundle path"); + const target = parents + .reverse() + .reduce( + (current, segment) => + typeof segment === "number" + ? array(current)[segment] + : object(current)[segment], + root, + ); + object(target)[String(last)] = value; +}; + +const structured = (result: Awaited>) => { + if (result.isError === true) throw parseMcpToolError(result); + if ( + typeof result.structuredContent !== "object" || + result.structuredContent === null + ) + throw new Error("Missing structured MCP result"); + return result.structuredContent as Record; +}; From 10935b7b1136accab2a43b308ab380749da0f7ae Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:59:44 +0800 Subject: [PATCH 019/159] fix(process): reject deep trace cycles without recursive expansion Use iterative cycle detection, memoized reachability and compact unordered-group membership. Preserve declaration diagnostics on cyclic input while skipping checks requiring a valid partial order. Report duplicate group declarations once instead of repeating pair errors. Replace redundant source cycle/missing-pair assertions with public CLI/MCP coverage including 6000 events, dense not-before constraints and a large unordered group. Validate acyclic branching and mixed relation semantics. Document bounded Flameox CPU/latency gains and higher observed peak memory; agent/model costs remain unmeasured. --- .../process-trace-validation.md | 59 +++++++ .../process/processTraceSpecification.test.ts | 99 +++++++----- .../process/processTraceSpecification.ts | 148 ++++++++++++------ .../processComparisonDiagnosticsMcp.test.ts | 50 ++++++ 4 files changed, 265 insertions(+), 91 deletions(-) create mode 100644 experiments/analysis-efficiency/process-trace-validation.md diff --git a/experiments/analysis-efficiency/process-trace-validation.md b/experiments/analysis-efficiency/process-trace-validation.md new file mode 100644 index 000000000..0a4f9b0f6 --- /dev/null +++ b/experiments/analysis-efficiency/process-trace-validation.md @@ -0,0 +1,59 @@ +# Process trace validation + +A 6,000-event directed cycle previously overflowed the JavaScript stack inside +the schema refinement. Cycle detection now uses iterative topological traversal. +Reachability is computed once per queried origin; cyclic inputs retain +independent declaration diagnostics without expanding all event pairs. Unordered +declarations retain group membership, with one diagnostic per overlapping group, +instead of allocating a pair set and repeating the same error for every pair. +Valid partial orders still receive complete pair validation. + +## Bounded comparison + +Flameox ran three randomized paired blocks (seed 917), each limited to 20 seconds +and 512 MiB. Node 24.18.0 used a 384-MiB old-space limit on macOS arm64. Each +process validated the same 2,500-event acyclic chain four times, then checked the +known 6,000-event cycle failure with per-event not-before constraints and one +unordered group containing every event. All six executions passed the embedded answer +digest and expected rejection checks. + +The baseline source is from `bb540826d`; the candidate includes this trace fix. +Both modules had TypeScript annotations removed with Node's identical built-in +transform and used the same copied runtime dependencies. This isolates schema +validation rather than comparing complete packaged CLI/MCP processes. + +| Median measurement | Baseline | Candidate | +| -------------------------------------------- | -------: | --------: | +| First validation elapsed milliseconds | 335.80 | 177.25 | +| Warm validation elapsed milliseconds | 313.36 | 165.23 | +| First validation CPU milliseconds | 366.85 | 218.39 | +| Warm validation CPU milliseconds | 335.32 | 194.08 | +| Whole-process elapsed milliseconds | 1,344.13 | 787.00 | +| Process CPU milliseconds before final report | 1,457.67 | 941.05 | +| Peak process-family MiB | 328.36 | 395.52 | + +The input is identical: 233,095 JSON bytes, estimated at 77,010 tokens by tokenx +1.3.0. The valid parsed result digest is +`28bd99ce83f0da8e45600ef20f114cb52bb26b5f5e6bfc1601ca3e712eaf3147`. +Actual agent tokens, catalog injection, transport, model reasoning and provider +costs were not measured. This is a component measurement, not an end-to-end +investigation speed claim. + +First validation excludes module import and fixture preparation; warm runs reuse +the process but create fresh per-validation graph state. Whole-process elapsed +includes startup, imports, all four validations, the cyclic-input check, token +estimation and reporting. CPU includes startup through metrics publication; +peak RSS covers the complete single-process workload, with no children. Host +caches were not flushed. Peak memory increased in this cohort. The remaining +valid-order closure and pair validation can still require quadratic resources. +Three pairs describe this fixture only. + +Preserved Flameox evidence: +`d6211639446e8b6ec2e82636913eaf471def96673e28f03e9c7f98bec07f8e13`; +analysis `e663b6bd8f5693975542d5b5e2b18437b3c167a0be939bb7386eb68549936231`. +An earlier cohort lost modules during another checkout build; its three failed +executions and timing results are excluded. The successful cohort uses an +isolated runtime copy. Source-module SHA-256 values are +`a7729faa2183b0e493b9ac4fc55f6feaee5b2e5294be13d1b29b57a44b899788` +(baseline) and `725409ce84902125941251037a8a214ccc3c1eec577e484c164c361ca580d693` +(candidate). diff --git a/src/domain/process/processTraceSpecification.test.ts b/src/domain/process/processTraceSpecification.test.ts index aeedf4d31..9484e9075 100644 --- a/src/domain/process/processTraceSpecification.test.ts +++ b/src/domain/process/processTraceSpecification.test.ts @@ -39,27 +39,61 @@ const specification = (): ProcessTraceSpecification => ({ }); describe("process trace specification", () => { - it("summarizes an incomplete language with its exact missing pair count", () => { - const events = Array.from({ length: 200 }, (_, index) => ({ - id: `e${String(index)}`, - source: "lifecycle" as const, - exact: { fixture: index }, - cardinality: { kind: "optional" as const }, - })); + it("reports independent declaration errors alongside a cycle", () => { + const base = specification(); const result = processTraceSpecificationSchema.safeParse({ - events, - language: { kind: "partial_order" }, + ...base, + language: { + ...base.language, + happens_before: [ + { before: "ready", after: "worker" }, + { before: "worker", after: "ready" }, + ], + not_before: [{ event: "missing", anchor: "ready" }], + unordered_groups: [{ events: ["missing", "missing"] }], + prefix: ["missing"], + suffix: ["missing"], + }, }); - expect(result.success).toBe(false); - if (result.success) - throw new Error("Expected incomplete pair declarations"); - expect(result.error.issues).toEqual([ - expect.objectContaining({ - path: ["language"], - message: - "19900 event pairs must be explicitly ordered or unordered; first undeclared pair: e0/e1", - }), - ]); + if (result.success) throw new Error("Expected invalid trace declarations"); + expect(result.error.issues).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + message: "happens-before graph must be acyclic", + }), + expect.objectContaining({ + path: ["language", "not_before", 0, "event"], + }), + expect.objectContaining({ + path: ["language", "unordered_groups", 0, "events"], + }), + expect.objectContaining({ path: ["language", "prefix", 0] }), + expect.objectContaining({ path: ["language", "suffix", 0] }), + ]), + ); + }); + + it("accepts an acyclic order with shared successors and explicit unordered peers", () => { + const result = processTraceSpecificationSchema.parse({ + events: ["a", "b", "c", "d"].map((id) => ({ + id, + source: "lifecycle", + exact: { marker: id }, + })), + language: { + kind: "partial_order", + happens_before: [ + { before: "a", after: "b" }, + { before: "a", after: "c" }, + { before: "b", after: "d" }, + { before: "c", after: "d" }, + ], + unordered_groups: [{ events: ["b", "c"] }], + prefix: ["a"], + suffix: ["d"], + }, + }); + expect(result.language.kind).toBe("partial_order"); }); it("counts only undeclared pairs after transitive, not-before and unordered relations", () => { @@ -91,7 +125,7 @@ describe("process trace specification", () => { ]); }); - it("retains duplicate unordered-pair diagnostics for reversed declarations", () => { + it("reports one duplicate-pair diagnostic per overlapping group", () => { const result = processTraceSpecificationSchema.safeParse({ events: ["z", "a", "b"].map((id) => ({ id, @@ -101,7 +135,10 @@ describe("process trace specification", () => { })), language: { kind: "partial_order", - unordered_groups: [{ events: ["z", "a", "b"] }, { events: ["a", "z"] }], + unordered_groups: [ + { events: ["z", "a", "b"] }, + { events: ["b", "a", "z"] }, + ], }, }); expect(result.success).toBe(false); @@ -114,26 +151,8 @@ describe("process trace specification", () => { ]); }); - it("rejects cycles, implicit ordering gaps, invalid ignores, and unsatisfiable variants", () => { + it("rejects invalid ignores and unsatisfiable variants", () => { const base = specification(); - expect( - processTraceSpecificationSchema.safeParse({ - ...base, - language: { - kind: "partial_order", - happens_before: [ - { before: "ready", after: "worker" }, - { before: "worker", after: "ready" }, - ], - }, - }).success, - ).toBe(false); - expect( - processTraceSpecificationSchema.safeParse({ - ...base, - language: { kind: "partial_order" }, - }).success, - ).toBe(false); expect( processTraceSpecificationSchema.safeParse({ events: [ diff --git a/src/domain/process/processTraceSpecification.ts b/src/domain/process/processTraceSpecification.ts index 61ace7ca1..8e4e6fca6 100644 --- a/src/domain/process/processTraceSpecification.ts +++ b/src/domain/process/processTraceSpecification.ts @@ -105,6 +105,7 @@ export type ProcessTraceSpecification = z.infer< typeof specificationShapeSchema >; type PartialOrderLanguage = z.infer; +type Reachability = (origin: string) => ReadonlySet; export const canonicalTraceJson = (value: unknown): string => { const serialized = canonicalize(value); @@ -170,7 +171,6 @@ const validateNotBefore = ( language: PartialOrderLanguage, context: z.RefinementCtx, known: ReadonlySet, - reachable: ReadonlyMap>, ): ReadonlySet => { const pairs = new Set(); for (const [index, constraint] of language.not_before.entries()) { @@ -191,63 +191,81 @@ const validateNotBefore = ( path, }); pairs.add(key); - if (reachable.get(constraint.event)?.has(constraint.anchor) === true) + } + return pairs; +}; + +const validateNotBeforeOrder = ( + language: PartialOrderLanguage, + context: z.RefinementCtx, + reachable: Reachability, +): void => { + for (const [index, constraint] of language.not_before.entries()) + if (reachable(constraint.event).has(constraint.anchor)) context.addIssue({ code: "custom", message: "not-before constraint conflicts with happens-before order", - path, + path: ["language", "not_before", index], }); - } - return pairs; }; const graphHasCycle = ( ids: readonly string[], adjacency: ReadonlyMap>, ): boolean => { - const active = new Set(); - const complete = new Set(); - const visit = (current: string): boolean => { - if (active.has(current)) return true; - if (complete.has(current)) return false; - active.add(current); - const cyclic = [...(adjacency.get(current) ?? [])].some(visit); - active.delete(current); - complete.add(current); - return cyclic; - }; - return ids.some(visit); + const incoming = new Map(ids.map((id) => [id, 0])); + for (const successors of adjacency.values()) + for (const successor of successors) { + const count = incoming.get(successor); + if (count !== undefined) incoming.set(successor, count + 1); + } + const ready = [...incoming].flatMap(([id, count]) => + count === 0 ? [id] : [], + ); + let examined = 0; + while (ready.length > 0) { + const current = ready.pop(); + if (current === undefined) break; + examined++; + for (const successor of adjacency.get(current) ?? []) { + const count = incoming.get(successor); + if (count === undefined) continue; + incoming.set(successor, count - 1); + if (count === 1) ready.push(successor); + } + } + return examined !== incoming.size; }; -const computeReachability = ( - ids: readonly string[], +const createReachability = ( adjacency: ReadonlyMap>, -): ReadonlyMap> => { +): Reachability => { const reachable = new Map>(); - for (const origin of ids) { + return (origin) => { + const cached = reachable.get(origin); + if (cached !== undefined) return cached; const seen = new Set(); const pending = [...(adjacency.get(origin) ?? [])]; while (pending.length > 0) { const current = pending.pop(); if (current === undefined || seen.has(current)) continue; seen.add(current); - pending.push(...(adjacency.get(current) ?? [])); + for (const successor of adjacency.get(current) ?? []) + pending.push(successor); } reachable.set(origin, seen); - } - return reachable; + return seen; + }; }; const isOrdered = ( - reachable: ReadonlyMap>, + reachable: Reachability, left: string, right: string, -): boolean => - reachable.get(left)?.has(right) === true || - reachable.get(right)?.has(left) === true; +): boolean => reachable(left).has(right) || reachable(right).has(left); type OrderingRelations = { - readonly reachable: ReadonlyMap>; + readonly reachable: Reachability; readonly notBeforePairs: ReadonlySet; }; @@ -255,12 +273,12 @@ const validateUnorderedGroups = ( language: PartialOrderLanguage, context: z.RefinementCtx, known: ReadonlySet, - relations: OrderingRelations, -): ReadonlySet => { - const pairs = new Set(); +): ((left: string, right: string) => boolean) => { + const memberships = new Map>(); for (const [groupIndex, group] of language.unordered_groups.entries()) { const path = ["language", "unordered_groups", groupIndex]; - if (new Set(group.events).size !== group.events.length) + const uniqueIds = new Set(group.events); + if (uniqueIds.size !== group.events.length) context.addIssue({ code: "custom", message: "unordered groups must not repeat event ids", @@ -268,6 +286,35 @@ const validateUnorderedGroups = ( }); for (const [eventIndex, id] of group.events.entries()) addReferenceIssue(context, known, id, [...path, "events", eventIndex]); + const overlaps = new Map(); + for (const id of uniqueIds) { + const priorGroups = memberships.get(id) ?? new Set(); + for (const prior of priorGroups) + overlaps.set(prior, (overlaps.get(prior) ?? 0) + 1); + priorGroups.add(groupIndex); + memberships.set(id, priorGroups); + } + if ([...overlaps.values()].some((count) => count > 1)) + context.addIssue({ + code: "custom", + message: "unordered event pairs must be declared once", + path, + }); + } + return (left, right) => { + const rightGroups = memberships.get(right); + for (const group of memberships.get(left) ?? []) + if (rightGroups?.has(group) === true) return true; + return false; + }; +}; + +const validateUnorderedOrder = ( + language: PartialOrderLanguage, + context: z.RefinementCtx, + relations: OrderingRelations, +): void => { + for (const [groupIndex, group] of language.unordered_groups.entries()) for (const [leftIndex, leftId] of group.events.entries()) { for ( let rightIndex = leftIndex + 1; @@ -277,13 +324,6 @@ const validateUnorderedGroups = ( const rightId = group.events[rightIndex]; if (rightId === undefined) continue; const key = pairKey(leftId, rightId); - if (pairs.has(key)) - context.addIssue({ - code: "custom", - message: "unordered event pairs must be declared once", - path, - }); - pairs.add(key); if ( isOrdered(relations.reachable, leftId, rightId) || relations.notBeforePairs.has(key) @@ -291,18 +331,16 @@ const validateUnorderedGroups = ( context.addIssue({ code: "custom", message: "ordered events cannot also be declared unordered", - path, + path: ["language", "unordered_groups", groupIndex], }); } } - } - return pairs; }; const validateCompletePairDeclarations = ( ids: readonly string[], relations: OrderingRelations, - unorderedPairs: ReadonlySet, + unorderedPair: (left: string, right: string) => boolean, context: z.RefinementCtx, ): void => { let missingPairCount = 0; @@ -317,7 +355,10 @@ const validateCompletePairDeclarations = ( if (rightId === undefined) continue; if (isOrdered(relations.reachable, leftId, rightId)) continue; const key = pairKey(leftId, rightId); - if (!relations.notBeforePairs.has(key) && !unorderedPairs.has(key)) { + if ( + !relations.notBeforePairs.has(key) && + !unorderedPair(leftId, rightId) + ) { if (missingPairCount === 0) firstMissingPair = `${leftId}/${rightId}`; missingPairCount++; } @@ -342,27 +383,32 @@ const validatePartialOrder = ( const ids = specification.events.map(({ id }) => id); const known = new Set(ids); const adjacency = buildAdjacency(specification.language, context, known); - if (graphHasCycle(ids, adjacency)) + const cyclic = graphHasCycle(ids, adjacency); + if (cyclic) context.addIssue({ code: "custom", message: "happens-before graph must be acyclic", path: ["language"], }); - const reachable = computeReachability(ids, adjacency); + const reachable = createReachability(adjacency); const notBeforePairs = validateNotBefore( specification.language, context, known, - reachable, ); const relations = { reachable, notBeforePairs }; - const unorderedPairs = validateUnorderedGroups( + const unorderedPair = validateUnorderedGroups( specification.language, context, known, - relations, ); - validateCompletePairDeclarations(ids, relations, unorderedPairs, context); + // Preserve independent declaration diagnostics on cyclic input. Computing + // all pair relations cannot establish a valid partial order in that case. + if (!cyclic) { + validateNotBeforeOrder(specification.language, context, reachable); + validateUnorderedOrder(specification.language, context, relations); + validateCompletePairDeclarations(ids, relations, unorderedPair, context); + } for (const [kind, values] of [ ["prefix", specification.language.prefix], ["suffix", specification.language.suffix], diff --git a/tests/boundary/mcp/processComparisonDiagnosticsMcp.test.ts b/tests/boundary/mcp/processComparisonDiagnosticsMcp.test.ts index 69fdc3944..bda4672de 100644 --- a/tests/boundary/mcp/processComparisonDiagnosticsMcp.test.ts +++ b/tests/boundary/mcp/processComparisonDiagnosticsMcp.test.ts @@ -252,6 +252,56 @@ it("reports CLI trace specification issues like the MCP trace_spec input", async }); }); +it("rejects a deep trace cycle as an input error through CLI and MCP", async () => { + const events = Array.from({ length: 6000 }, (_, index) => ({ + id: `e${String(index)}`, + source: "lifecycle", + exact: { marker: index }, + })); + const trace = { + events, + language: { + kind: "partial_order", + happens_before: events.map((event, index) => ({ + before: event.id, + after: events[(index + 1) % events.length]!.id, + })), + not_before: events.map((event, index) => ({ + event: event.id, + anchor: events[(index + 1) % events.length]!.id, + })), + unordered_groups: [{ events: events.map(({ id }) => id) }], + }, + }; + const issue = { + path: ["trace_spec", "language"], + message: "happens-before graph must be acyclic", + }; + const capture = captureEvidence("left"); + const root = await createTestTempDirectory("rea-process-trace-cycle-"); + const capturePath = join(root, "capture.json"); + const specPath = join(root, "spec.json"); + await writeFile(capturePath, JSON.stringify(capture)); + await writeFile(specPath, JSON.stringify(trace)); + expect( + await compareProcessEvidenceFiles(capturePath, capturePath, specPath), + ).toMatchObject({ + code: "invalid_request", + details: { issues: [issue] }, + }); + + const client = await connect(); + const response = await client.callTool({ + name: "compare_process_captures", + arguments: { left: capture, right: capture, trace_spec: trace }, + }); + expect(response.isError).toBe(true); + expect(response.content).toEqual([ + { type: "text", text: expect.stringContaining(issue.message) }, + ]); + await client.ping(); +}); + it("names the side and constraint when Evidence is not a usable capture", async () => { const client = await connect(); const capture = captureEvidence("right"); From 8b181716aca2cb467be499cba226abfb5ae19141 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:59:52 +0800 Subject: [PATCH 020/159] fix(install): honor redirected setup diagnostics Require terminal stderr before launching the automatic setup wizard, matching the CLI's interactive admission rule. Print the manual setup command when diagnostics are redirected. Exercise a real PTY with stderr redirected to a fixture file and verify the installed command is not launched for setup. Installer lane: 33 tests passed; targeted regression, typecheck/lint and npm pack dry-run passed. Full package verifier was not run under the requested focused-check resource policy. --- install.sh | 2 +- .../boundary/filesystem/installScript.test.ts | 68 ++++++++++++++++++- 2 files changed, 68 insertions(+), 2 deletions(-) diff --git a/install.sh b/install.sh index 46260fe71..c18726611 100755 --- a/install.sh +++ b/install.sh @@ -167,7 +167,7 @@ main() { [ "$installed_version" = "$version" ] || fail "installed version $installed_version does not match $version. Reinstall the requested version, then retry." printf 'REA %s is installed.\n' "$version" - if [ "$start_setup" = true ] && [ -r /dev/tty ] && [ -w /dev/tty ] && [ -t 1 ]; then + if [ "$start_setup" = true ] && [ -r /dev/tty ] && [ -w /dev/tty ] && [ -t 1 ] && [ -t 2 ]; then printf 'Starting REA setup...\n' "$install_bin" setup /dev/tty else diff --git a/tests/boundary/filesystem/installScript.test.ts b/tests/boundary/filesystem/installScript.test.ts index 3fbcf7a3c..0f2e14730 100644 --- a/tests/boundary/filesystem/installScript.test.ts +++ b/tests/boundary/filesystem/installScript.test.ts @@ -11,6 +11,7 @@ import { import { join } from "node:path"; import { promisify } from "node:util"; +import { spawn as spawnPty } from "@lydell/node-pty"; import { describe, expect, it } from "vitest"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; @@ -185,6 +186,69 @@ describe("curl installer scenarios", { timeout: 20_000 }, () => { }); }); +describe("installer TTY fallback", { timeout: 20_000 }, () => { + it("skips automatic setup when stderr is redirected", async () => { + const fixture = await createFixture(); + const script = join(process.cwd(), "install.sh"); + const wrapper = ` + const { spawnSync } = require("node:child_process"); + const { openSync, closeSync } = require("node:fs"); + const stderr = openSync(process.argv[2], "w"); + const child = spawnSync("/bin/bash", [process.argv[1], ...process.argv.slice(3)], { + stdio: ["inherit", "inherit", stderr], + env: process.env, + }); + closeSync(stderr); + process.exit(child.status ?? 1); + `; + let output = ""; + const terminal = spawnPty( + process.execPath, + ["-e", wrapper, script, fixture.installerStderrLog, "--version", "0.3.0"], + { + cwd: process.cwd(), + env: { + ...process.env, + HOME: fixture.home, + TMPDIR: fixture.temporary, + PATH: `${fixture.bin}:/usr/bin:/bin`, + FAKE_NPM_LOG: fixture.npmLog, + FAKE_REA_LOG: fixture.reaLog, + FAKE_REA_SOURCE: join(fixture.bin, "rea-source"), + FAKE_NPM_PREFIX: join(fixture.home, ".npm-global"), + TERM: "xterm-256color", + }, + name: "xterm-256color", + cols: 120, + rows: 40, + }, + ); + let timedOut = false; + const exitCode = await new Promise((resolve, reject) => { + const timeout = setTimeout(() => { + timedOut = true; + terminal.kill(); + }, 10_000); + terminal.onData((data) => { + output += data; + }); + terminal.onExit(({ exitCode: code }) => { + clearTimeout(timeout); + if (timedOut) + reject(new Error(`Installer PTY timed out after output: ${output}`)); + else resolve(code); + }); + }); + + expect(exitCode).toBe(0); + expect(output).toContain( + `Run ${join(fixture.home, ".local/bin/rea")} setup to configure Hopper and agents.`, + ); + expect(await readFile(fixture.reaLog, "utf8")).toBe("--version\n"); + expect(await readFile(fixture.installerStderrLog, "utf8")).toBe(""); + }); +}); + describe("truncated installer download", { timeout: 20_000 }, () => { it("runs nothing from a download truncated after the npm install line", async () => { const fixture = await createFixture(); @@ -433,6 +497,7 @@ interface InstallerFixture { readonly temporary: string; readonly npmLog: string; readonly reaLog: string; + readonly installerStderrLog: string; } const getProcessOutput = ( @@ -456,6 +521,7 @@ const createFixture = async (): Promise => { const temporary = join(root, "tmp"); const npmLog = join(root, "npm.log"); const reaLog = join(root, "rea.log"); + const installerStderrLog = join(root, "installer.stderr.log"); await Promise.all([mkdir(home), mkdir(bin), mkdir(temporary)]); await Promise.all([ executable( @@ -507,7 +573,7 @@ exit 0 symlink(`/usr/bin/${command}`, join(bin, command)), ), ]); - return { home, bin, temporary, npmLog, reaLog }; + return { home, bin, temporary, npmLog, reaLog, installerStderrLog }; }; const executable = async (path: string, source: string): Promise => { From 7a2cd1ea0a63e8c44166860b7ee9c7a1aa43492b Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:12:06 +0800 Subject: [PATCH 021/159] fix(domain): preserve derived byte counts and uncertain identity --- .../ManagedReconstructionService.test.ts | 20 +++++++++++++ src/domain/functionComparison.test.ts | 28 +++++++++++++++++++ src/domain/functionComparisonNormalization.ts | 21 ++++++-------- src/domain/managed/managedReconstruction.ts | 3 +- 4 files changed, 59 insertions(+), 13 deletions(-) diff --git a/src/application/managed/ManagedReconstructionService.test.ts b/src/application/managed/ManagedReconstructionService.test.ts index 56b9d9442..793951ae9 100644 --- a/src/application/managed/ManagedReconstructionService.test.ts +++ b/src/application/managed/ManagedReconstructionService.test.ts @@ -22,6 +22,26 @@ const exampleMethod = () => { }; describe("managed decompiler reconstruction import", () => { + it("reports UTF-8 byte totals while preserving UTF-16 text lengths", () => { + const text = 'return "é"; // 😀'; + const method = exampleMethod(); + const result = importManagedReconstructionEvidence({ + ...MANAGED_RECONSTRUCTION_IMPORT_EXAMPLE, + methods: [ + { + ...method, + reconstruction: { ...method.reconstruction, text }, + }, + ], + }); + if (!result.ok) throw result.error; + + expect(result.value.normalized_result).toMatchObject({ + summary: { total_text_bytes: 20 }, + methods: [{ reconstruction: { text, text_length: 17 } }], + }); + }); + it("rejects reconstruction text hash drift", () => { expect(() => importManagedReconstruction({ diff --git a/src/domain/functionComparison.test.ts b/src/domain/functionComparison.test.ts index acb505dac..e03a27748 100644 --- a/src/domain/functionComparison.test.ts +++ b/src/domain/functionComparison.test.ts @@ -119,6 +119,34 @@ const observe = ( ); describe("function comparison normalized identity", () => { + it("keeps differing generated names ambiguous and stable names mismatched", () => { + const named = (base: "0x1000" | "0x2000", name: string) => + functionDossierSchema.parse({ + ...dossier("return 0;", base), + procedure: { + ...procedure(base, name), + signature: "int helper(void)", + locals: [], + }, + }); + const generated = compareFunctions( + observe("1", named("0x1000", "sub_1000")), + observe("2", named("0x2000", "sub_2000")), + ); + expect(generated.function_match.status).toBe("ambiguous"); + expect(generated.status).toBe("unknown"); + expect( + generated.dimensions.find(({ dimension }) => dimension === "identity"), + ).toMatchObject({ status: "unknown" }); + + const stable = compareFunctions( + observe("3", named("0x1000", "firstFunction")), + observe("4", named("0x2000", "secondFunction")), + ); + expect(stable.function_match.status).toBe("mismatched"); + expect(stable.status).toBe("changed"); + }); + it.each(["sub_deallocate", "sub_deadbeef_handler", "fcn.dispatch"])( "recognizes the explicit symbol %s without a generated-name prefix false positive", (name) => { diff --git a/src/domain/functionComparisonNormalization.ts b/src/domain/functionComparisonNormalization.ts index a676e1e49..5282272e1 100644 --- a/src/domain/functionComparisonNormalization.ts +++ b/src/domain/functionComparisonNormalization.ts @@ -16,24 +16,21 @@ export const functionMatch = ( left: FunctionSnapshot, right: FunctionSnapshot, ): FunctionComparisonResult["function_match"] => { - if ( - !isAutoName(left.dossier.procedure.name) && - left.dossier.procedure.name === right.dossier.procedure.name - ) + const leftName = left.dossier.procedure.name; + const rightName = right.dossier.procedure.name; + const stableNames = !isAutoName(leftName) && !isAutoName(rightName); + if (stableNames && leftName === rightName) return { status: "matched", method: "symbol", - left_name: left.dossier.procedure.name, - right_name: right.dossier.procedure.name, + left_name: leftName, + right_name: rightName, }; return { - status: - left.dossier.procedure.name === right.dossier.procedure.name - ? "ambiguous" - : "mismatched", + status: stableNames ? "mismatched" : "ambiguous", method: "explicit", - left_name: left.dossier.procedure.name, - right_name: right.dossier.procedure.name, + left_name: leftName, + right_name: rightName, }; }; diff --git a/src/domain/managed/managedReconstruction.ts b/src/domain/managed/managedReconstruction.ts index e10be0ffd..9600166ae 100644 --- a/src/domain/managed/managedReconstruction.ts +++ b/src/domain/managed/managedReconstruction.ts @@ -204,7 +204,8 @@ const buildReconstructionSummary = ( ({ reconstruction }) => reconstruction.kind === "semantic-pseudocode", ).length, total_text_bytes: methods.reduce( - (total, { reconstruction }) => total + reconstruction.text_length, + (total, { reconstruction }) => + total + Buffer.byteLength(reconstruction.text, "utf8"), 0, ), }); From 1447fd2f990c5d9820337dfd4b8a170f8cad17cd Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:12:12 +0800 Subject: [PATCH 022/159] fix(evaluation): share completed transcript facts across graders --- src/evaluation/CodexAgentEval.ts | 66 +++----- src/evaluation/KnownAnswerEvaluation.ts | 148 +++++++++++------ .../model-evals/codexAgentEval.test.ts | 154 +++++++++++++++--- 3 files changed, 257 insertions(+), 111 deletions(-) diff --git a/src/evaluation/CodexAgentEval.ts b/src/evaluation/CodexAgentEval.ts index bac8d8191..55ec8333e 100644 --- a/src/evaluation/CodexAgentEval.ts +++ b/src/evaluation/CodexAgentEval.ts @@ -1,10 +1,11 @@ import { - deliveredCodexEvidence, evaluateKnownAnswers, + parseCodexToolCall, type FixtureClaimExpectation, type KnownAnswerAssessment, + type ParsedCodexToolCall, } from "./KnownAnswerEvaluation.js"; -import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js"; +import { canonicalJson } from "../domain/comparisonSemantics.js"; import { TOOL_CONTRACTS } from "../contracts/toolContracts.js"; /** One REA MCP invocation observed in a Codex JSONL transcript. */ @@ -57,17 +58,6 @@ const textValue = (value: unknown): string | undefined => const numberValue = (value: unknown): number => typeof value === "number" && Number.isFinite(value) ? value : 0; -const canonicalValue = (value: unknown): unknown => { - if (Array.isArray(value)) return value.map(canonicalValue); - const object = record(value); - if (object === undefined) return value; - return Object.fromEntries( - Object.entries(object) - .sort(([left], [right]) => compareUnicodeCodePoints(left, right)) - .map(([key, child]) => [key, canonicalValue(child)]), - ); -}; - const toolResultError = ( item: Record, ): Record | undefined => { @@ -113,29 +103,18 @@ const toolResultFailed = ( ); }; -const callFromEvent = ( - event: Record, +const callFromParsed = ( + parsed: ParsedCodexToolCall | undefined, ): CodexMcpCall | undefined => { - const item = record( - event.type === "item.completed" - ? event.item - : event.type === "mcp_tool_call" - ? event - : undefined, - ); - if (item?.type !== "mcp_tool_call") return undefined; - const tool = textValue(item.tool) ?? textValue(item.name); - if (tool === undefined) return undefined; - const error = toolResultError(item); + if (parsed === undefined) return undefined; + const error = toolResultError(parsed.item); return { - id: textValue(item.id) ?? null, - server: textValue(item.server) ?? textValue(item.server_name) ?? null, - tool, - arguments: item.arguments ?? item.input ?? {}, - evidenceIds: deliveredCodexEvidence(event).map( - ({ evidence_id }) => evidence_id, - ), - error: toolResultFailed(item, error), + id: parsed.id, + server: parsed.server, + tool: parsed.tool, + arguments: parsed.arguments, + evidenceIds: parsed.evidence.map(({ evidence_id }) => evidence_id), + error: toolResultFailed(parsed.item, error), errorCode: textValue(error?.code) ?? null, }; }; @@ -163,16 +142,20 @@ interface EvaluationState { outputTokens: number; } -const consumeEvent = (state: EvaluationState, value: unknown): void => { +const consumeEvent = ( + state: EvaluationState, + value: unknown, + parsedCall: ParsedCodexToolCall | undefined, +): void => { const event = record(value); if (event === undefined) return; const completedItem = event.type === "item.completed" ? event.item : undefined; - const call = callFromEvent(event); + const call = callFromParsed(parsedCall); if (call !== undefined) { const identity = call.id ?? - `${call.server ?? ""}:${call.tool}:${JSON.stringify(canonicalValue(call.arguments))}`; + `${call.server ?? ""}:${call.tool}:${canonicalJson(call.arguments)}`; if (!state.seenCallIds.has(identity)) { state.seenCallIds.add(identity); state.calls.push(call); @@ -216,7 +199,10 @@ export const evaluateCodexEvents = ( cachedInputTokens: 0, outputTokens: 0, }; - for (const event of events) consumeEvent(state, event); + const parsedCalls = events.map(parseCodexToolCall); + events.forEach((event, index) => + consumeEvent(state, event, parsedCalls[index]), + ); const reaCalls = state.calls.filter( ({ server }) => server === null || server.toLowerCase() === "rea", @@ -228,7 +214,7 @@ export const evaluateCodexEvents = ( // useful verification, not a duplicate analysis or unchanged failed retry. const context = call.tool === "binary_session" ? `${String(targetRevision)}:` : ""; - const signature = `${context}${call.tool}:${JSON.stringify(canonicalValue(call.arguments))}`; + const signature = `${context}${call.tool}:${canonicalJson(call.arguments)}`; signatures.set(signature, (signatures.get(signature) ?? 0) + 1); if ( !call.error && @@ -271,7 +257,7 @@ export const evaluateCodexEvents = ( finalMessage, ); const factualAssessment = evaluateKnownAnswers( - events, + parsedCalls, finalMessage, options.fixtureClaims, ); diff --git a/src/evaluation/KnownAnswerEvaluation.ts b/src/evaluation/KnownAnswerEvaluation.ts index 807308b59..c6e4af72b 100644 --- a/src/evaluation/KnownAnswerEvaluation.ts +++ b/src/evaluation/KnownAnswerEvaluation.ts @@ -3,7 +3,7 @@ import { z } from "zod"; import { parseEvidence, type Evidence } from "../domain/evidence.js"; import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js"; -import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js"; +import { canonicalJson } from "../domain/comparisonSemantics.js"; /** One closed fixture assertion and the producer facts that must support it. */ export interface FixtureClaimExpectation { @@ -104,23 +104,13 @@ const isRecord = (value: unknown): value is Record => typeof value === "object" && value !== null && !Array.isArray(value); const record = (value: unknown): Record | undefined => isRecord(value) ? value : undefined; -const canonicalValue = (value: JsonValue): JsonValue => { - if (Array.isArray(value)) return value.map(canonicalValue); - if (value === null || typeof value !== "object") return value; - return Object.fromEntries( - Object.entries(value) - .sort(([left], [right]) => compareUnicodeCodePoints(left, right)) - .map(([key, child]) => [key, canonicalValue(child)]), - ); -}; const sameJson = (left: unknown, right: unknown): boolean => { const parsedLeft = jsonValueSchema.safeParse(left); const parsedRight = jsonValueSchema.safeParse(right); return ( parsedLeft.success && parsedRight.success && - JSON.stringify(canonicalValue(parsedLeft.data)) === - JSON.stringify(canonicalValue(parsedRight.data)) + canonicalJson(parsedLeft.data) === canonicalJson(parsedRight.data) ); }; @@ -168,6 +158,16 @@ interface DeliveredEvidence { readonly conflicts: Set; } +/** One completed tool call, with aliases normalized and successful Evidence authenticated. */ +export interface ParsedCodexToolCall { + readonly item: Record; + readonly id: string | null; + readonly server: string | null; + readonly tool: string; + readonly arguments: JsonValue; + readonly evidence: readonly Evidence[]; +} + const consistentAliases = ( value: Record, left: string, @@ -177,30 +177,14 @@ const consistentAliases = ( value[right] === undefined || value[left] === value[right]; -const completedCall = (event: unknown): Record | undefined => { - const value = record(event); - const item = - value?.type === "item.completed" - ? record(value.item) - : value?.type === "mcp_tool_call" && value.status === "completed" - ? value - : undefined; - if (item?.type !== "mcp_tool_call") return undefined; - const server = item.server ?? item.server_name; - if ( - typeof server !== "string" || - server.toLowerCase() !== "rea" || - (item.status !== undefined && item.status !== "completed") || - (item.error !== undefined && item.error !== null) - ) - return undefined; - if ( - !consistentAliases(item, "server", "server_name") || - !consistentAliases(item, "tool", "name") - ) - return undefined; - return item; -}; +const consistentJsonFields = ( + value: Record, + left: string, + right: string, +): boolean => + value[left] === undefined || + value[right] === undefined || + sameJson(value[left], value[right]); const consistentJsonAliases = ( value: Record, @@ -215,6 +199,7 @@ const consistentJsonAliases = ( const evidenceFromCall = ( item: Record, + tool: string, ): Evidence | undefined => { if (!consistentJsonAliases(item, "result", "output")) return undefined; const result = record(item.result ?? item.output); @@ -254,22 +239,42 @@ const evidenceFromCall = ( // Transcript output is untrusted; invalid Evidence cannot support a claim. return undefined; } - if ((item.tool ?? item.name) !== evidence.operation) return undefined; + if (tool !== evidence.operation) return undefined; return evidence; }; -/** Authenticate completed REA deliveries for both citations and fixture grading. */ -export const deliveredCodexEvidence = (event: unknown): readonly Evidence[] => { - const item = completedCall(event); - if (item === undefined) return []; - const evidence = evidenceFromCall(item); - if (evidence === undefined) return []; +const completedToolItem = ( + event: unknown, +): Record | undefined => { + const value = record(event); + const wrapped = value?.type === "item.completed"; + const item = wrapped + ? record(value.item) + : value?.type === "mcp_tool_call" + ? value + : undefined; + if (item?.type !== "mcp_tool_call") return undefined; + const terminalStatus = + item.status === "completed" || item.status === "failed"; + if ( + (wrapped + ? item.status !== undefined && !terminalStatus + : !terminalStatus) || + !consistentAliases(item, "server", "server_name") || + !consistentAliases(item, "tool", "name") || + !consistentJsonFields(item, "arguments", "input") + ) + return undefined; + return item; +}; + +const artifactDelivery = (evidence: Evidence): readonly Evidence[] => { const records = [evidence]; - // Artifact inspection delivers its inventory Evidence inline. Bare links and - // arbitrary IDs in normalized facts are references, not delivered records. if (evidence.operation !== "inspect_artifact") return records; const substeps = record(evidence.normalized_result)?.substeps; if (!Array.isArray(substeps)) return records; + // Artifact inspection delivers linked inventory Evidence inline; bare IDs in + // normalized facts are references, not authenticated deliveries. for (const value of substeps) { const substep = record(value); if ( @@ -293,13 +298,56 @@ export const deliveredCodexEvidence = (event: unknown): readonly Evidence[] => { return records; }; +/** Parse one completed transcript call and authenticate any successful REA Evidence. */ +export const parseCodexToolCall = ( + event: unknown, +): ParsedCodexToolCall | undefined => { + const item = completedToolItem(event); + if (item === undefined) return undefined; + const status = item.status; + const server = item.server ?? item.server_name; + const tool = item.tool ?? item.name; + if ( + ((item.server !== undefined || item.server_name !== undefined) && + typeof server !== "string") || + typeof tool !== "string" || + tool.length === 0 + ) + return undefined; + const argumentsValue = + item.arguments !== undefined + ? item.arguments + : item.input !== undefined + ? item.input + : {}; + const parsedArguments = jsonValueSchema.safeParse(argumentsValue); + if (!parsedArguments.success) return undefined; + const parsed: ParsedCodexToolCall = { + item, + id: typeof item.id === "string" && item.id.length > 0 ? item.id : null, + server: typeof server === "string" ? server : null, + tool, + arguments: parsedArguments.data, + evidence: [], + }; + if ( + parsed.server?.toLowerCase() !== "rea" || + status === "failed" || + (item.error !== undefined && item.error !== null) + ) + return parsed; + const evidence = evidenceFromCall(item, tool); + if (evidence === undefined) return parsed; + return { ...parsed, evidence: artifactDelivery(evidence) }; +}; + const collectDeliveredEvidence = ( - events: readonly unknown[], + calls: readonly (ParsedCodexToolCall | undefined)[], ): DeliveredEvidence => { const records = new Map(); const conflicts = new Set(); - for (const event of events) { - for (const evidence of deliveredCodexEvidence(event)) { + for (const call of calls) { + for (const evidence of call?.evidence ?? []) { const previous = records.get(evidence.evidence_id); if (previous !== undefined && !sameJson(previous, evidence)) conflicts.add(evidence.evidence_id); @@ -401,7 +449,7 @@ const assessClaim = ( /** Grade closed answers only against configured fixture truth and delivered authenticated Evidence. */ export const evaluateKnownAnswers = ( - events: readonly unknown[], + calls: readonly (ParsedCodexToolCall | undefined)[], finalMessage: string, expectations: readonly FixtureClaimExpectation[] | undefined, ): KnownAnswerAssessment => { @@ -441,7 +489,7 @@ export const evaluateKnownAnswers = ( manifestFailures.push(`unknown_claim:${claim.id}`); submitted.set(claim.id, claim); } - const delivered = collectDeliveredEvidence(events); + const delivered = collectDeliveredEvidence(calls); const claims = expectations.map((expected) => assessClaim(expected, submitted.get(expected.id), delivered), ); diff --git a/tests/evaluation/model-evals/codexAgentEval.test.ts b/tests/evaluation/model-evals/codexAgentEval.test.ts index d96265202..6b46d9b1d 100644 --- a/tests/evaluation/model-evals/codexAgentEval.test.ts +++ b/tests/evaluation/model-evals/codexAgentEval.test.ts @@ -134,6 +134,114 @@ describe("Codex agent release evaluation", () => { }); }); +describe("Codex completed tool call parsing", () => { + it("does not let an unfinished direct event reserve a completed call identity", () => { + const evidence = observation(); + const metrics = evaluateCodexEvents( + [ + { + type: "mcp_tool_call", + id: "same-call", + status: "in_progress", + server: "rea", + tool: evidence.operation, + arguments: {}, + }, + { + type: "item.completed", + item: { + id: "same-call", + type: "mcp_tool_call", + status: "completed", + server: "rea", + tool: evidence.operation, + arguments: {}, + result: { structuredContent: evidence }, + }, + }, + { + type: "item.completed", + item: { + type: "agent_message", + text: `Observed Evidence ${evidence.evidence_id} supports the artifact claim; runtime behavior remains unknown without runtime authority.`, + }, + }, + ], + evidence.operation, + { requireEvidence: true }, + ); + + expect(metrics).toMatchObject({ + naturalUse: true, + correctFirstTool: true, + reaCalls: [{ id: "same-call" }], + evidenceIds: [evidence.evidence_id], + finalCitesEvidence: true, + answerHeuristicsMet: true, + }); + }); + + it.each([ + { server: "rea", server_name: "other" }, + { tool: "inspect_managed_artifact", name: "other_tool" }, + { server: 42 }, + { arguments: { value: 1 }, input: { value: 2 } }, + ])( + "rejects completed calls with invalid or conflicting fields: %j", + (aliases) => { + const evidence = observation(); + const metrics = evaluateCodexEvents( + [ + { + type: "item.completed", + item: { + type: "mcp_tool_call", + status: "completed", + server: "rea", + tool: evidence.operation, + arguments: {}, + result: { structuredContent: evidence }, + ...aliases, + }, + }, + ], + evidence.operation, + { requiredToolSubsequence: [evidence.operation] }, + ); + + expect(metrics).toMatchObject({ + naturalUse: false, + correctFirstTool: false, + requiredToolSubsequenceMet: false, + evidenceIds: [], + }); + }, + ); + + it("preserves legacy completed calls with no server field", () => { + const metrics = evaluateCodexEvents( + [ + { + type: "item.completed", + item: { + type: "mcp_tool_call", + status: "completed", + tool: "inspect_managed_artifact", + arguments: {}, + }, + }, + ], + "inspect_managed_artifact", + ); + + expect(metrics).toMatchObject({ + naturalUse: true, + correctFirstTool: true, + reaCalls: [{ server: null }], + }); + }); +}); + describe("Codex Evidence delivery", () => { it("rejects diagnostic IDs and failed results as delivered Evidence", () => { const evidenceId = `ev_${"a".repeat(64)}`; @@ -524,28 +632,32 @@ describe("Codex agent completion evaluation", () => { }); }); - it("recognizes Codex MCP failed status when the error field is null", () => { - const metrics = evaluateCodexEvents( - [ - { - type: "item.completed", - item: { - id: "item-failed", - type: "mcp_tool_call", - server: "rea", - tool: "analyze_javascript_application", - arguments: { input_path: "/tmp/app.asar" }, - result: { structured_content: { error: { code: "denied" } } }, - error: null, - status: "failed", - }, - }, - ], - "analyze_javascript_application", - ); + it.each(["item.completed", "mcp_tool_call"] as const)( + "counts a failed completed invocation from %s without delivering Evidence", + (type) => { + const item = { + id: "item-failed", + type: "mcp_tool_call", + server: "rea", + tool: "analyze_javascript_application", + arguments: { input_path: "/tmp/app.asar" }, + result: { structured_content: { error: { code: "denied" } } }, + error: null, + status: "failed", + }; + const metrics = evaluateCodexEvents( + [type === "item.completed" ? { type, item } : item], + "analyze_javascript_application", + ); - expect(metrics.reaCalls[0]?.error).toBe(true); - }); + expect(metrics).toMatchObject({ + naturalUse: true, + correctFirstTool: true, + evidenceIds: [], + reaCalls: [{ error: true }], + }); + }, + ); }); describe("Codex agent workflow evaluation", () => { From 16f5ded4468d7100d3640f874f33ddbaa953a787 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:12:17 +0800 Subject: [PATCH 023/159] test(cli): consolidate development runner workflow coverage --- tests/boundary/cli/developmentTests.test.ts | 243 ++++++++++++------ .../filesystem/developmentTests.test.ts | 110 -------- 2 files changed, 159 insertions(+), 194 deletions(-) delete mode 100644 tests/boundary/filesystem/developmentTests.test.ts diff --git a/tests/boundary/cli/developmentTests.test.ts b/tests/boundary/cli/developmentTests.test.ts index 21d8c5127..2987206c8 100644 --- a/tests/boundary/cli/developmentTests.test.ts +++ b/tests/boundary/cli/developmentTests.test.ts @@ -1,100 +1,175 @@ -import { execFile } from "node:child_process"; -import { mkdir, symlink, writeFile } from "node:fs/promises"; -import { join, resolve } from "node:path"; -import { promisify } from "node:util"; +import { + copyFile, + mkdir, + readFile, + symlink, + writeFile, +} from "node:fs/promises"; +import { join } from "node:path"; import { describe, expect, it } from "vitest"; +import { execFileOutput } from "../../../src/process/ExecFileOutput.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; -const execute = promisify(execFile); -const runner = resolve("scripts/run-development-tests.mjs"); - -describe("development test runner", () => { - it("rejects nonexistent explicit paths instead of returning a zero-test success", async () => { - await expect( - execute(process.execPath, [ - runner, - "focused", - "src/missing.test.ts", - "--dry-run", - ]), - ).rejects.toMatchObject({ code: 1 }); +const run = (root: string, command: string, arguments_: readonly string[]) => + execFileOutput(command, arguments_, { + cwd: root, + stopSignal: "SIGTERM", + timeout: 30_000, + maxBuffer: 1024 * 1024, }); - it("reports actionable remediation when the comparison base is unavailable", async () => { - await expect( - execute(process.execPath, [ - runner, - "changed", - "--base", - "missing-test-base", - "--dry-run", +const git = (root: string, ...arguments_: string[]) => + run(root, "git", arguments_); + +const commit = (root: string, message: string) => + git( + root, + "-c", + "user.name=REA fixture", + "-c", + "user.email=fixture@example.test", + "-c", + "commit.gpgSign=false", + "commit", + "--quiet", + "-m", + message, + ); + +const fixture = async () => { + const root = await createTestTempDirectory("rea-development-tests-"); + await mkdir(join(root, "scripts", "lib"), { recursive: true }); + await mkdir(join(root, "src")); + for (const path of [ + "scripts/run-development-tests.mjs", + "scripts/lib/development-tests.mjs", + ]) + await copyFile(join(process.cwd(), path), join(root, path)); + await symlink( + join(process.cwd(), "node_modules"), + join(root, "node_modules"), + process.platform === "win32" ? "junction" : undefined, + ); + await writeFile(join(root, "package.json"), '{"type":"module"}\n'); + await writeFile(join(root, ".gitignore"), "node_modules\nselected.txt\n"); + await writeFile( + join(root, "vitest.config.mjs"), + 'export default { test: { projects: [{ test: { name: "domain", include: ["src/**/*.test.ts"] } }] } };\n', + ); + await writeFile(join(root, "src/value.ts"), "export const value = 1;\n"); + await writeFile( + join(root, "src/value.test.ts"), + 'import { expect, it } from "vitest"; import { writeFileSync } from "node:fs"; import { value } from "./value.js"; it("checks the selected source", () => { expect(value).toBe(1); writeFileSync("selected.txt", String(value)); });\n', + ); + await git(root, "init", "--quiet", "-b", "main"); + await git(root, "add", "."); + await commit(root, "fixture"); + return root; +}; + +const testCommand = (root: string, ...arguments_: string[]) => + run(root, process.execPath, [ + "scripts/run-development-tests.mjs", + ...arguments_, + ]); + +describe("development test runner public CLI", () => { + it("runs an explicitly selected source test on a clean tree without build artifacts", async () => { + const root = await fixture(); + const { stdout: status } = await git(root, "status", "--porcelain"); + expect(status).toBe(""); + await testCommand(root, "local", "src/value.test.ts"); + expect(await readFile(join(root, "selected.txt"), "utf8")).toBe("1"); + }); + + it("plans an exact local test without building", async () => { + const root = await fixture(); + const { stdout } = await testCommand( + root, + "local", + "src/value.test.ts", + "--dry-run", + ); + expect(JSON.parse(stdout)).toMatchObject({ + needsBuild: false, + artifactTasks: [], + vitestArguments: expect.arrayContaining([ + "run", + "--project", + "domain", + "--project", + "services", + "--project", + "adapters", + "--project", + "composition", + "--project", + "conformance", + "--project", + "evaluation", + "src/value.test.ts", ]), + }); + }); + + it("uses committed dependency changes to select source tests and reports failures", async () => { + const root = await fixture(); + const { stdout: base } = await git(root, "rev-parse", "HEAD"); + await writeFile(join(root, "src/value.ts"), "export const value = 2;\n"); + await git(root, "add", "src/value.ts"); + await commit(root, "change dependency"); + await expect( + testCommand(root, "changed", "--base", base.trim()), + ).rejects.toMatchObject({ + code: 1, + stdout: expect.stringContaining("src/value.test.ts"), + stderr: expect.stringContaining("expected 2 to be 1"), + }); + }); + + it("runs changed and focused tests from a committed branch regression", async () => { + const root = await fixture(); + const { stdout: baseline } = await git(root, "rev-parse", "HEAD"); + await git(root, "update-ref", "refs/remotes/origin/main", baseline.trim()); + await git(root, "switch", "--quiet", "-c", "regression"); + await writeFile(join(root, "src/value.ts"), "export const value = 2;\n"); + await writeFile( + join(root, "src/value.test.ts"), + 'import { expect, it } from "vitest"; import { value } from "./value.js"; it("checks current source", () => expect(value).toBe(2));\n', + ); + await git(root, "add", "src"); + await commit(root, "regression fix"); + expect((await git(root, "status", "--porcelain")).stdout).toBe(""); + + const changed = await testCommand(root, "changed"); + expect(changed.stdout).toContain("1 passed"); + const focused = await testCommand(root, "focused", "src/value.test.ts"); + expect(focused.stdout).toContain("1 passed"); + }, 60_000); + + it("reports a missing comparison base and rejects missing or compiled-boundary paths", async () => { + const root = await fixture(); + await expect( + testCommand(root, "changed", "--base", "missing-test-base", "--dry-run"), ).rejects.toMatchObject({ code: 1, stderr: expect.stringContaining( "Fetch origin/main or pass --base REVISION", ), }); - }); - - it("runs a committed branch regression through real Git and Vitest without dist", async () => { - const root = await createTestTempDirectory("rea-development-tests-"); - await mkdir(join(root, "src")); - await symlink( - resolve("node_modules"), - join(root, "node_modules"), - "junction", - ); - await writeFile( - join(root, "package.json"), - JSON.stringify({ type: "module" }), - ); - await writeFile(join(root, ".gitignore"), "node_modules\n"); - await writeFile( - join(root, "vitest.config.ts"), - `export default { test: { projects: [{ test: { name: "domain", include: ["src/**/*.test.ts"] } }] } };`, - ); - await writeFile(join(root, "src/value.ts"), "export const value = 1;\n"); - await writeFile( - join(root, "src/value.test.ts"), - `import { it, expect } from "vitest"; import { value } from "./value"; it("checks current source", () => expect(value).toBe(1));\n`, - ); - const git = (arguments_: string[]) => - execute("git", arguments_, { cwd: root }); - await git(["init", "-b", "main"]); - await git(["add", ".gitignore", "package.json", "vitest.config.ts", "src"]); - const commit = [ - "-c", - "user.name=REA fixture", - "-c", - "user.email=fixture@example.test", - "commit", - "-m", - "fixture", - ]; - await git(commit); - const { stdout: baseline } = await git(["rev-parse", "HEAD"]); - await git(["update-ref", "refs/remotes/origin/main", baseline.trim()]); - await git(["switch", "-c", "regression"]); - await writeFile(join(root, "src/value.ts"), "export const value = 2;\n"); - await writeFile( - join(root, "src/value.test.ts"), - `import { it, expect } from "vitest"; import { value } from "./value"; it("checks current source", () => expect(value).toBe(2));\n`, - ); - await git(["add", "src"]); - await git(commit); - expect((await git(["status", "--porcelain"])).stdout).toBe(""); - const { stdout } = await execute(process.execPath, [runner, "changed"], { - cwd: root, + await expect( + testCommand(root, "focused", "src/missing.test.ts"), + ).rejects.toMatchObject({ + code: 1, + stderr: expect.stringContaining("ENOENT"), }); - expect(stdout).toContain("1 passed"); - const explicit = await execute( - process.execPath, - [runner, "focused", "src/value.test.ts"], - { cwd: root }, - ); - expect(explicit.stdout).toContain("1 passed"); - }, 30_000); + await expect( + testCommand(root, "local", "tests/acceptance/runtime.test.ts"), + ).rejects.toMatchObject({ + code: 1, + stderr: expect.stringContaining("Use test:focused"), + }); + }); }); diff --git a/tests/boundary/filesystem/developmentTests.test.ts b/tests/boundary/filesystem/developmentTests.test.ts deleted file mode 100644 index 1b52bb475..000000000 --- a/tests/boundary/filesystem/developmentTests.test.ts +++ /dev/null @@ -1,110 +0,0 @@ -import { - copyFile, - mkdir, - readFile, - symlink, - writeFile, -} from "node:fs/promises"; -import { join } from "node:path"; - -import { expect, it } from "vitest"; - -import { execFileOutput } from "../../../src/process/ExecFileOutput.js"; -import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; - -const run = (root: string, command: string, arguments_: readonly string[]) => - execFileOutput(command, arguments_, { - cwd: root, - stopSignal: "SIGTERM", - timeout: 20_000, - }); - -const commit = (root: string, message: string) => - run(root, "git", [ - "-c", - "user.name=REA fixture", - "-c", - "user.email=fixture@example.invalid", - "-c", - "commit.gpgSign=false", - "commit", - "--quiet", - "-m", - message, - ]); - -const fixture = async () => { - const root = await createTestTempDirectory("rea-development-tests-"); - await mkdir(join(root, "scripts", "lib"), { recursive: true }); - await mkdir(join(root, "src")); - for (const path of [ - "scripts/run-development-tests.mjs", - "scripts/lib/development-tests.mjs", - ]) - await copyFile(join(process.cwd(), path), join(root, path)); - await symlink( - join(process.cwd(), "node_modules"), - join(root, "node_modules"), - process.platform === "win32" ? "junction" : undefined, - ); - await writeFile(join(root, "package.json"), '{"type":"module"}\n'); - await writeFile(join(root, ".gitignore"), "node_modules\nselected.txt\n"); - await writeFile( - join(root, "vitest.config.mjs"), - 'export default { test: { projects: [{ test: { name: "domain", include: ["src/**/*.test.ts"] } }] } };\n', - ); - await writeFile(join(root, "src", "value.ts"), "export const value = 1;\n"); - await writeFile( - join(root, "src", "value.test.ts"), - 'import { expect, it } from "vitest"; import { writeFileSync } from "node:fs"; import { value } from "./value.js"; it("checks the selected source", () => { expect(value).toBe(1); writeFileSync("selected.txt", String(value)); });\n', - ); - await run(root, "git", ["init", "--quiet"]); - await run(root, "git", ["add", "."]); - await commit(root, "fixture"); - return root; -}; - -const testCommand = (root: string, ...arguments_: string[]) => - run(root, process.execPath, [ - "scripts/run-development-tests.mjs", - ...arguments_, - ]); - -it("executes an explicitly selected source test on a clean tree without build artifacts", async () => { - const root = await fixture(); - const { stdout: status } = await run(root, "git", ["status", "--porcelain"]); - expect(status).toBe(""); - await testCommand(root, "local", "src/value.test.ts"); - expect(await readFile(join(root, "selected.txt"), "utf8")).toBe("1"); -}); - -it("uses committed dependency changes since the requested Git base to select source tests", async () => { - const root = await fixture(); - const { stdout: base } = await run(root, "git", ["rev-parse", "HEAD"]); - await writeFile(join(root, "src", "value.ts"), "export const value = 2;\n"); - await run(root, "git", ["add", "src/value.ts"]); - await commit(root, "change dependency"); - await expect( - testCommand(root, "changed", "--base", base.trim()), - ).rejects.toMatchObject({ - code: 1, - stdout: expect.stringContaining("src/value.test.ts"), - stderr: expect.stringContaining("expected 2 to be 1"), - }); -}); - -it("rejects a compiled boundary in the source-only command and a missing exact test", async () => { - const root = await fixture(); - await expect( - testCommand(root, "local", "tests/acceptance/runtime.test.ts"), - ).rejects.toMatchObject({ - code: 1, - stderr: expect.stringContaining("Use test:focused"), - }); - await expect( - testCommand(root, "local", "src/missing.test.ts"), - ).rejects.toMatchObject({ - code: 1, - stderr: expect.stringContaining("ENOENT"), - }); -}); From 63e796d50ea323ae8dcfef80f86c8154bb106790 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:17:44 +0800 Subject: [PATCH 024/159] fix(process): bound retained trace ordering state Prove total orders with Kahn traversal and use ranks; stream forward and reverse reachability for general partial orders instead of retaining every closure. Aggregate unordered conflicts by declaration while preserving missing-pair counts and independent malformed-input diagnostics. Validation: 33 focused trace domain and CLI/MCP boundary tests passed. Six bounded Flameox executions preserved answer digests and cycle rejection; combined median peak RSS fell from 391.13 to 127.72 MiB. General partial-order traversal costs more CPU; the experiment records that tradeoff. Full suite and real-provider lanes were not run. --- .../process-trace-validation.md | 72 ++++- .../process/processTraceSpecification.test.ts | 123 ++++++- .../process/processTraceSpecification.ts | 304 ++++++++++++------ 3 files changed, 394 insertions(+), 105 deletions(-) diff --git a/experiments/analysis-efficiency/process-trace-validation.md b/experiments/analysis-efficiency/process-trace-validation.md index 0a4f9b0f6..a32848596 100644 --- a/experiments/analysis-efficiency/process-trace-validation.md +++ b/experiments/analysis-efficiency/process-trace-validation.md @@ -2,13 +2,19 @@ A 6,000-event directed cycle previously overflowed the JavaScript stack inside the schema refinement. Cycle detection now uses iterative topological traversal. -Reachability is computed once per queried origin; cyclic inputs retain -independent declaration diagnostics without expanding all event pairs. Unordered -declarations retain group membership, with one diagnostic per overlapping group, -instead of allocating a pair set and repeating the same error for every pair. -Valid partial orders still receive complete pair validation. +Cyclic inputs retain independent declaration diagnostics without expanding all +event pairs. Unordered declarations retain group membership, with one diagnostic +per overlapping or conflicting group, instead of allocating a pair set and +repeating the same error for every pair. -## Bounded comparison +A unique topological order proves that every event pair is ordered, allowing +rank comparisons without retaining transitive closure. Other acyclic graphs +retain only one origin's forward and backward reachability sets at a time. +Complete pair validation preserves the missing count and first declaration-order +missing pair. General partial orders still require quadratic pair examination +and repeated graph traversal; they trade CPU time for bounded retained state. + +## Initial cycle fix comparison Flameox ran three randomized paired blocks (seed 917), each limited to 20 seconds and 512 MiB. Node 24.18.0 used a 384-MiB old-space limit on macOS arm64. Each @@ -17,7 +23,8 @@ known 6,000-event cycle failure with per-event not-before constraints and one unordered group containing every event. All six executions passed the embedded answer digest and expected rejection checks. -The baseline source is from `bb540826d`; the candidate includes this trace fix. +The baseline source is from `bb540826d`; the candidate is the initial cycle fix +in `10935b7b1`, which still memoized reachability sets for every queried origin. Both modules had TypeScript annotations removed with Node's identical built-in transform and used the same copied runtime dependencies. This isolates schema validation rather than comparing complete packaged CLI/MCP processes. @@ -44,8 +51,8 @@ the process but create fresh per-validation graph state. Whole-process elapsed includes startup, imports, all four validations, the cyclic-input check, token estimation and reporting. CPU includes startup through metrics publication; peak RSS covers the complete single-process workload, with no children. Host -caches were not flushed. Peak memory increased in this cohort. The remaining -valid-order closure and pair validation can still require quadratic resources. +caches were not flushed. Peak memory increased in this cohort. Its retained +valid-order closure still required quadratic memory, prompting the follow-up. Three pairs describe this fixture only. Preserved Flameox evidence: @@ -57,3 +64,50 @@ isolated runtime copy. Source-module SHA-256 values are `a7729faa2183b0e493b9ac4fc55f6feaee5b2e5294be13d1b29b57a44b899788` (baseline) and `725409ce84902125941251037a8a214ccc3c1eec577e484c164c361ca580d693` (candidate). + +## Bounded retention follow-up + +Flameox compared `10935b7b1` with the rank/streaming implementation in three +randomized paired blocks (seed 624), under the same host, Node, heap, memory and +20-second execution bounds. Each execution validated the chain above four times, +then a 1,200-event partial order with 600 unordered two-event layers and all four +edges between consecutive layers four times, then the same cyclic failure. +All six executions passed both parsed-answer digests and the cycle diagnostic. +Runtime dependencies were copied and isolated from concurrent checkout builds. + +| Median measurement | Memoized closure | Rank/streaming | +| ---------------------------------------- | ---------------: | -------------: | +| Chain first elapsed milliseconds | 166.11 | 11.37 | +| Chain warm elapsed milliseconds | 154.25 | 5.32 | +| Chain first CPU milliseconds | 210.13 | 24.38 | +| Chain warm CPU milliseconds | 184.76 | 16.84 | +| Partial order first elapsed milliseconds | 49.44 | 116.54 | +| Partial order warm elapsed milliseconds | 50.67 | 110.29 | +| Partial order first CPU milliseconds | 72.47 | 137.69 | +| Partial order warm CPU milliseconds | 70.65 | 121.77 | +| Capture process elapsed milliseconds | 901.24 | 557.30 | +| Process CPU milliseconds before report | 1,101.12 | 665.03 | +| Peak process-family MiB | 391.13 | 127.72 | + +First samples exclude module import and fixture preparation. Partial-order first +samples follow four chain validations, so they are first use of that fixture +within an already running process. Warm medians pool the three subsequent samples +from each of three processes. CPU and peak RSS cover the combined workload; +capture elapsed additionally includes Flameox capture startup and overhead. +Host caches were not flushed. The partial-order fixture is slower after this +change; these results establish a retention tradeoff, not universal acceleration. + +The partial-order input is 165,590 JSON bytes, estimated at 52,783 tokens by tokenx +1.3.0; its parsed-answer SHA-256 is +`41d06e593813e497221c36793bf00d5fece7055081895b5f6c123ba6ad9a8227`. +Chain bytes, token estimate and answer digest remain unchanged. Actual agent +tokens, transport, catalog injection, model reasoning and provider costs remain +unmeasured. Both cohorts are component measurements, not complete investigations. + +Preserved evidence: +`e29086af99afd8e67cbbd92e43d960ea40f73b4d318ea0cbca60f493575462b8`; +analysis `086133516f2acf031032d357a3ddb1ac611ce4f12556a24065a2becd567c5f7b`. +The baseline module SHA-256 is +`725409ce84902125941251037a8a214ccc3c1eec577e484c164c361ca580d693`; +the candidate module SHA-256 is +`2109f1096406e9ecac8bbf4f7ae8abb5113404790a9382487285d2831fcddf26`. diff --git a/src/domain/process/processTraceSpecification.test.ts b/src/domain/process/processTraceSpecification.test.ts index 9484e9075..ab649d2aa 100644 --- a/src/domain/process/processTraceSpecification.test.ts +++ b/src/domain/process/processTraceSpecification.test.ts @@ -95,8 +95,54 @@ describe("process trace specification", () => { }); expect(result.language.kind).toBe("partial_order"); }); +}); - it("counts only undeclared pairs after transitive, not-before and unordered relations", () => { +describe("process trace ordering validation", () => { + it("uses a unique topological order for transitive declarations and conflicts", () => { + const events = ["a", "b", "c"].map((id) => ({ + id, + source: "lifecycle" as const, + exact: { marker: id }, + cardinality: { kind: "optional" as const }, + })); + const accepted = processTraceSpecificationSchema.safeParse({ + events, + language: { + kind: "partial_order", + happens_before: [ + { before: "a", after: "b" }, + { before: "b", after: "c" }, + ], + }, + }); + expect(accepted.success).toBe(true); + + const rejected = processTraceSpecificationSchema.safeParse({ + events, + language: { + kind: "partial_order", + happens_before: [ + { before: "a", after: "b" }, + { before: "b", after: "c" }, + ], + not_before: [{ event: "a", anchor: "c" }], + unordered_groups: [{ events: ["a", "b", "c"] }], + }, + }); + if (rejected.success) throw new Error("Expected ordered pair conflicts"); + expect(rejected.error.issues).toEqual([ + expect.objectContaining({ + path: ["language", "not_before", 0], + message: "not-before constraint conflicts with happens-before order", + }), + expect.objectContaining({ + path: ["language", "unordered_groups", 0], + message: "ordered events cannot also be declared unordered", + }), + ]); + }); + + it("counts only undeclared pairs across transitive and disconnected relations", () => { const events = ["a", "b", "c", "d"].map((id) => ({ id, source: "lifecycle" as const, @@ -150,7 +196,82 @@ describe("process trace specification", () => { }), ]); }); +}); +describe("malformed process trace declarations", () => { + it("preserves independent diagnostics when event ids are duplicated", () => { + const result = processTraceSpecificationSchema.safeParse({ + events: [ + { + id: "a", + source: "lifecycle", + exact: { fixture: "first" }, + cardinality: { kind: "optional" }, + }, + { + id: "a", + source: "lifecycle", + exact: { fixture: "second" }, + cardinality: { kind: "optional" }, + }, + ], + language: { + kind: "partial_order", + unordered_groups: [{ events: ["a", "b"] }], + }, + }); + expect(result.success).toBe(false); + if (result.success) throw new Error("Expected a duplicate event id"); + expect(result.error.issues).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + path: ["events", 1, "id"], + message: "event ids must be unique", + }), + expect.objectContaining({ + path: ["language", "unordered_groups", 0, "events", 1], + message: "unknown event id: b", + }), + ]), + ); + }); + + it("keeps reachability diagnostics for unknown referenced successors", () => { + const result = processTraceSpecificationSchema.safeParse({ + events: ["a", "b"].map((id) => ({ + id, + source: "lifecycle", + exact: { fixture: id }, + cardinality: { kind: "optional" }, + })), + language: { + kind: "partial_order", + happens_before: [{ before: "a", after: "missing" }], + not_before: [{ event: "a", anchor: "missing" }], + }, + }); + expect(result.success).toBe(false); + if (result.success) throw new Error("Expected an unknown event reference"); + expect(result.error.issues).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + path: ["language", "happens_before", 0, "after"], + message: "unknown event id: missing", + }), + expect.objectContaining({ + path: ["language", "not_before", 0, "anchor"], + message: "unknown event id: missing", + }), + expect.objectContaining({ + path: ["language", "not_before", 0], + message: "not-before constraint conflicts with happens-before order", + }), + ]), + ); + }); +}); + +describe("finite trace specification validation", () => { it("rejects invalid ignores and unsatisfiable variants", () => { const base = specification(); expect( diff --git a/src/domain/process/processTraceSpecification.ts b/src/domain/process/processTraceSpecification.ts index 8e4e6fca6..09c14cf10 100644 --- a/src/domain/process/processTraceSpecification.ts +++ b/src/domain/process/processTraceSpecification.ts @@ -105,7 +105,17 @@ export type ProcessTraceSpecification = z.infer< typeof specificationShapeSchema >; type PartialOrderLanguage = z.infer; -type Reachability = (origin: string) => ReadonlySet; + +type AcyclicGraphFacts = + | { + readonly kind: "total-order"; + readonly rank: ReadonlyMap; + } + | { + readonly kind: "partial-order"; + readonly reverseAdjacency: ReadonlyMap>; + }; +type GraphFacts = { readonly kind: "cyclic" } | AcyclicGraphFacts; export const canonicalTraceJson = (value: unknown): string => { const serialized = canonicalize(value); @@ -195,38 +205,28 @@ const validateNotBefore = ( return pairs; }; -const validateNotBeforeOrder = ( - language: PartialOrderLanguage, - context: z.RefinementCtx, - reachable: Reachability, -): void => { - for (const [index, constraint] of language.not_before.entries()) - if (reachable(constraint.event).has(constraint.anchor)) - context.addIssue({ - code: "custom", - message: "not-before constraint conflicts with happens-before order", - path: ["language", "not_before", index], - }); -}; - -const graphHasCycle = ( +const classifyGraph = ( ids: readonly string[], adjacency: ReadonlyMap>, -): boolean => { +): GraphFacts => { const incoming = new Map(ids.map((id) => [id, 0])); + let hasUnknownSuccessor = false; for (const successors of adjacency.values()) for (const successor of successors) { const count = incoming.get(successor); - if (count !== undefined) incoming.set(successor, count + 1); + if (count === undefined) hasUnknownSuccessor = true; + else incoming.set(successor, count + 1); } const ready = [...incoming].flatMap(([id, count]) => count === 0 ? [id] : [], ); - let examined = 0; + const order: string[] = []; + let unique = true; while (ready.length > 0) { + if (ready.length !== 1) unique = false; const current = ready.pop(); - if (current === undefined) break; - examined++; + if (current === undefined) continue; + order.push(current); for (const successor of adjacency.get(current) ?? []) { const count = incoming.get(successor); if (count === undefined) continue; @@ -234,50 +234,67 @@ const graphHasCycle = ( if (count === 1) ready.push(successor); } } - return examined !== incoming.size; + if (order.length !== incoming.size) return { kind: "cyclic" }; + if (unique && order.length === ids.length && !hasUnknownSuccessor) { + return { + kind: "total-order", + rank: new Map(order.map((id, index) => [id, index])), + }; + } + const reverseAdjacency = new Map( + [...incoming.keys()].map((id) => [id, new Set()]), + ); + for (const [before, successors] of adjacency) + for (const after of successors) reverseAdjacency.get(after)?.add(before); + return { kind: "partial-order", reverseAdjacency }; }; -const createReachability = ( +const collectReachable = ( + origin: string, adjacency: ReadonlyMap>, -): Reachability => { - const reachable = new Map>(); - return (origin) => { - const cached = reachable.get(origin); - if (cached !== undefined) return cached; - const seen = new Set(); - const pending = [...(adjacency.get(origin) ?? [])]; - while (pending.length > 0) { - const current = pending.pop(); - if (current === undefined || seen.has(current)) continue; - seen.add(current); - for (const successor of adjacency.get(current) ?? []) - pending.push(successor); - } - reachable.set(origin, seen); - return seen; - }; -}; - -const isOrdered = ( - reachable: Reachability, - left: string, - right: string, -): boolean => reachable(left).has(right) || reachable(right).has(left); - -type OrderingRelations = { - readonly reachable: Reachability; - readonly notBeforePairs: ReadonlySet; +): ReadonlySet => { + const seen = new Set(); + const pending = [...(adjacency.get(origin) ?? [])]; + while (pending.length > 0) { + const current = pending.pop(); + if (current === undefined || seen.has(current)) continue; + seen.add(current); + for (const successor of adjacency.get(current) ?? []) + pending.push(successor); + } + return seen; }; const validateUnorderedGroups = ( language: PartialOrderLanguage, context: z.RefinementCtx, known: ReadonlySet, -): ((left: string, right: string) => boolean) => { +): { + readonly groups: readonly ReadonlySet[]; + hasPair(left: string, right: string): boolean; + markDeclaredPairConflicts( + pairs: ReadonlySet, + conflicts: boolean[], + ): void; + markSharedGroups(left: string, right: string, conflicts: boolean[]): void; +} => { const memberships = new Map>(); + const groups: Set[] = []; + const repeatedMemberships = new Map>(); for (const [groupIndex, group] of language.unordered_groups.entries()) { const path = ["language", "unordered_groups", groupIndex]; - const uniqueIds = new Set(group.events); + const uniqueIds = new Set(); + const repeatedIds = new Set(); + for (const id of group.events) { + if (uniqueIds.has(id)) repeatedIds.add(id); + else uniqueIds.add(id); + } + groups.push(uniqueIds); + for (const id of repeatedIds) { + const groupIndexes = repeatedMemberships.get(id) ?? new Set(); + groupIndexes.add(groupIndex); + repeatedMemberships.set(id, groupIndexes); + } if (uniqueIds.size !== group.events.length) context.addIssue({ code: "custom", @@ -301,51 +318,121 @@ const validateUnorderedGroups = ( path, }); } - return (left, right) => { + const markSharedGroups = ( + left: string, + right: string, + conflicts: boolean[], + ): void => { + if (left === right) { + for (const groupIndex of repeatedMemberships.get(left) ?? []) + conflicts[groupIndex] = true; + return; + } const rightGroups = memberships.get(right); for (const group of memberships.get(left) ?? []) - if (rightGroups?.has(group) === true) return true; - return false; + if (rightGroups?.has(group) === true) conflicts[group] = true; + }; + return { + groups, + hasPair(left, right) { + if (left === right) return memberships.has(left); + const rightGroups = memberships.get(right); + for (const group of memberships.get(left) ?? []) + if (rightGroups?.has(group) === true) return true; + return false; + }, + markDeclaredPairConflicts(pairs, conflicts) { + for (const key of pairs) { + const separator = key.indexOf("\0"); + const left = key.slice(0, separator); + const right = key.slice(separator + 1); + markSharedGroups(left, right, conflicts); + } + }, + markSharedGroups, }; }; -const validateUnorderedOrder = ( +const validateTotalOrderRelations = ( language: PartialOrderLanguage, + graph: Extract, + unorderedGroups: ReturnType, + notBeforePairs: ReadonlySet, context: z.RefinementCtx, - relations: OrderingRelations, ): void => { - for (const [groupIndex, group] of language.unordered_groups.entries()) - for (const [leftIndex, leftId] of group.events.entries()) { - for ( - let rightIndex = leftIndex + 1; - rightIndex < group.events.length; - rightIndex++ - ) { - const rightId = group.events[rightIndex]; - if (rightId === undefined) continue; - const key = pairKey(leftId, rightId); - if ( - isOrdered(relations.reachable, leftId, rightId) || - relations.notBeforePairs.has(key) - ) - context.addIssue({ - code: "custom", - message: "ordered events cannot also be declared unordered", - path: ["language", "unordered_groups", groupIndex], - }); + for (const [index, constraint] of language.not_before.entries()) { + const before = graph.rank.get(constraint.event); + const after = graph.rank.get(constraint.anchor); + if (before !== undefined && after !== undefined && before < after) + context.addIssue({ + code: "custom", + message: "not-before constraint conflicts with happens-before order", + path: ["language", "not_before", index], + }); + } + + const conflicts = new Array(language.unordered_groups.length).fill( + false, + ); + for (const [groupIndex, group] of unorderedGroups.groups.entries()) { + let comparableEvents = 0; + for (const id of group) + if (graph.rank.has(id) && ++comparableEvents === 2) { + conflicts[groupIndex] = true; + break; } - } + } + unorderedGroups.markDeclaredPairConflicts(notBeforePairs, conflicts); + addUnorderedConflictIssues(conflicts, context); }; -const validateCompletePairDeclarations = ( +const addUnorderedConflictIssues = ( + conflicts: readonly boolean[], + context: z.RefinementCtx, +): void => { + for (const [groupIndex, conflict] of conflicts.entries()) + if (conflict) + context.addIssue({ + code: "custom", + message: "ordered events cannot also be declared unordered", + path: ["language", "unordered_groups", groupIndex], + }); +}; + +const validatePartialOrderRelations = ( ids: readonly string[], - relations: OrderingRelations, - unorderedPair: (left: string, right: string) => boolean, + language: PartialOrderLanguage, + adjacency: ReadonlyMap>, + graph: Extract, + unorderedGroups: ReturnType, + notBeforePairs: ReadonlySet, context: z.RefinementCtx, ): void => { let missingPairCount = 0; let firstMissingPair = ""; + const groupConflicts = new Array( + language.unordered_groups.length, + ).fill(false); + unorderedGroups.markDeclaredPairConflicts(notBeforePairs, groupConflicts); + const notBeforeConflictFlags = new Uint8Array(language.not_before.length); + const constraintsBySource = new Map(); + for (const [index, constraint] of language.not_before.entries()) { + const indexes = constraintsBySource.get(constraint.event) ?? []; + indexes.push(index); + constraintsBySource.set(constraint.event, indexes); + } + + // Keep only one origin's forward and backward closure live at a time. Pair + // iteration stays in declaration order so the count and first missing pair + // retain the established diagnostic semantics. for (const [leftIndex, leftId] of ids.entries()) { + const forward = collectReachable(leftId, adjacency); + const backward = collectReachable(leftId, graph.reverseAdjacency); + for (const constraintIndex of constraintsBySource.get(leftId) ?? []) { + const constraint = language.not_before[constraintIndex]; + if (constraint !== undefined && forward.has(constraint.anchor)) + notBeforeConflictFlags[constraintIndex] = 1; + } for ( let rightIndex = leftIndex + 1; rightIndex < ids.length; @@ -353,17 +440,32 @@ const validateCompletePairDeclarations = ( ) { const rightId = ids[rightIndex]; if (rightId === undefined) continue; - if (isOrdered(relations.reachable, leftId, rightId)) continue; + const sameEvent = leftId === rightId; + const ordered = + !sameEvent && (forward.has(rightId) || backward.has(rightId)); const key = pairKey(leftId, rightId); - if ( - !relations.notBeforePairs.has(key) && - !unorderedPair(leftId, rightId) - ) { + const notBefore = notBeforePairs.has(key); + const unordered = unorderedGroups.hasPair(leftId, rightId); + if (ordered || notBefore) { + if (!sameEvent) + unorderedGroups.markSharedGroups(leftId, rightId, groupConflicts); + } + if (!ordered && !notBefore && !unordered) { if (missingPairCount === 0) firstMissingPair = `${leftId}/${rightId}`; missingPairCount++; } } } + + for (const [index, conflict] of notBeforeConflictFlags.entries()) + if (conflict === 1) + context.addIssue({ + code: "custom", + message: "not-before constraint conflicts with happens-before order", + path: ["language", "not_before", index], + }); + addUnorderedConflictIssues(groupConflicts, context); + if (missingPairCount > 0) context.addIssue({ code: "custom", @@ -383,31 +485,43 @@ const validatePartialOrder = ( const ids = specification.events.map(({ id }) => id); const known = new Set(ids); const adjacency = buildAdjacency(specification.language, context, known); - const cyclic = graphHasCycle(ids, adjacency); - if (cyclic) + const graph = classifyGraph(ids, adjacency); + if (graph.kind === "cyclic") context.addIssue({ code: "custom", message: "happens-before graph must be acyclic", path: ["language"], }); - const reachable = createReachability(adjacency); const notBeforePairs = validateNotBefore( specification.language, context, known, ); - const relations = { reachable, notBeforePairs }; - const unorderedPair = validateUnorderedGroups( + const unorderedGroups = validateUnorderedGroups( specification.language, context, known, ); // Preserve independent declaration diagnostics on cyclic input. Computing // all pair relations cannot establish a valid partial order in that case. - if (!cyclic) { - validateNotBeforeOrder(specification.language, context, reachable); - validateUnorderedOrder(specification.language, context, relations); - validateCompletePairDeclarations(ids, relations, unorderedPair, context); + if (graph.kind === "total-order") { + validateTotalOrderRelations( + specification.language, + graph, + unorderedGroups, + notBeforePairs, + context, + ); + } else if (graph.kind === "partial-order") { + validatePartialOrderRelations( + ids, + specification.language, + adjacency, + graph, + unorderedGroups, + notBeforePairs, + context, + ); } for (const [kind, values] of [ ["prefix", specification.language.prefix], From 3c57e1a8b21b379fca55032d177078e1d8426e64 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:23:03 +0800 Subject: [PATCH 025/159] fix(artifacts): authenticate file subjects across provider and view boundaries Narrow WASM and EVM execution ports to file subjects and preserve supplied subject types at the shared execution factory. Authenticate format with path and digest before recording or projecting Evidence, using the existing WASM identity owner in both consumers. Validation: focused WASM/EVM service and public MCP workflows passed, including malformed provider subjects and self-contained retained Evidence with contradictory format. Global typecheck/lint and documentation checks passed. Real WABT/Evmole execution and the full suite were not run. --- src/application/AnalysisProvider.ts | 74 +++++++++++-------- src/application/evm/EvmInterfacePort.ts | 8 +- .../evm/EvmInterfaceService.test.ts | 4 +- src/application/evm/EvmInterfaceService.ts | 1 + src/application/wasm/WasmArtifactPort.ts | 8 +- src/domain/analysisView/evmInterfaceView.ts | 1 + src/domain/analysisView/wasmArtifactView.ts | 1 + src/domain/wasm/wasmArtifact.ts | 18 ++++- src/evm/EvmoleInterfaceProvider.ts | 12 +-- src/wasm/WabtArtifactProvider.ts | 14 ++-- tests/boundary/mcp/analysisViewMcp.test.ts | 61 +++++++++++++++ tests/boundary/mcp/evmInterfaceMcp.test.ts | 24 ++++++ tests/boundary/mcp/wasmArtifactMcp.test.ts | 25 +++++++ tests/fixtures/evm/interface.ts | 8 +- tests/fixtures/wasm/artifact.ts | 8 +- 15 files changed, 214 insertions(+), 53 deletions(-) diff --git a/src/application/AnalysisProvider.ts b/src/application/AnalysisProvider.ts index cc17cac82..f3b51257a 100644 --- a/src/application/AnalysisProvider.ts +++ b/src/application/AnalysisProvider.ts @@ -54,6 +54,15 @@ export interface AnalysisExecution { readonly subject: EvidenceSubjectTarget | null; } +interface CreateAnalysisExecutionOptions { + readonly rawResult?: unknown; + readonly analysisProfile?: AnalysisProfileCommitment; + readonly limitations?: readonly string[]; + readonly relationshipCoverage?: RelationshipCoverage; + readonly locations?: readonly EvidenceLocation[]; + readonly subject?: EvidenceSubjectTarget; +} + /** * Build a validated atomic provider observation at an adapter boundary. * @@ -61,38 +70,43 @@ export interface AnalysisExecution { * `result`. Omitting it records `null` rather than a second complete copy of * the normalized result in every Evidence record and tool response. */ -export const createAnalysisExecution = ( +export function createAnalysisExecution( result: unknown, provider: ProviderIdentity, - options: { - readonly rawResult?: unknown; - readonly analysisProfile?: AnalysisProfileCommitment; - readonly limitations?: readonly string[]; - readonly relationshipCoverage?: RelationshipCoverage; - readonly locations?: readonly EvidenceLocation[]; - readonly subject?: EvidenceSubjectTarget; - } = {}, -): AnalysisExecution => ({ - result: jsonValueSchema.parse(result), - rawResult: - options.rawResult === undefined - ? null - : jsonValueSchema.parse(options.rawResult), - provider, - ...(options.analysisProfile === undefined - ? {} - : { analysisProfile: structuredClone(options.analysisProfile) }), - limitations: [...(options.limitations ?? [])], - ...(options.relationshipCoverage === undefined - ? {} - : { - relationshipCoverage: relationshipCoverageSchema.parse( - options.relationshipCoverage, - ), - }), - locations: [...(options.locations ?? [])], - subject: options.subject ?? null, -}); + options: CreateAnalysisExecutionOptions & { readonly subject: Subject }, +): Omit & { readonly subject: Subject }; +export function createAnalysisExecution( + result: unknown, + provider: ProviderIdentity, + options?: CreateAnalysisExecutionOptions, +): AnalysisExecution; +export function createAnalysisExecution( + result: unknown, + provider: ProviderIdentity, + options: CreateAnalysisExecutionOptions = {}, +): AnalysisExecution { + return { + result: jsonValueSchema.parse(result), + rawResult: + options.rawResult === undefined + ? null + : jsonValueSchema.parse(options.rawResult), + provider, + ...(options.analysisProfile === undefined + ? {} + : { analysisProfile: structuredClone(options.analysisProfile) }), + limitations: [...(options.limitations ?? [])], + ...(options.relationshipCoverage === undefined + ? {} + : { + relationshipCoverage: relationshipCoverageSchema.parse( + options.relationshipCoverage, + ), + }), + locations: [...(options.locations ?? [])], + subject: options.subject ?? null, + }; +} /** Provider-neutral application capability for one analysis operation. */ export interface AnalysisOperationPort { diff --git a/src/application/evm/EvmInterfacePort.ts b/src/application/evm/EvmInterfacePort.ts index e9e29b756..a297e092c 100644 --- a/src/application/evm/EvmInterfacePort.ts +++ b/src/application/evm/EvmInterfacePort.ts @@ -4,13 +4,19 @@ import type { } from "../AnalysisProvider.js"; import type { AnalysisError } from "../../domain/analysisErrorBase.js"; import type { InspectEvmInterfaceInput } from "../../domain/evm/evmInterface.js"; +import type { EvidenceSubjectTarget } from "../../domain/evidence.js"; import type { Result } from "../../domain/result.js"; +/** Successful EVM interface recovery must identify the selected local file. */ +export type EvmInterfaceExecution = Omit & { + readonly subject: EvidenceSubjectTarget & { readonly format: "file" }; +}; + /** Replaceable offline interface-recovery engine, independent of wallets and chain clients. */ export interface EvmInterfacePort { close?(): Promise; inspect( input: InspectEvmInterfaceInput, options?: ExecutionOptions, - ): Promise>; + ): Promise>; } diff --git a/src/application/evm/EvmInterfaceService.test.ts b/src/application/evm/EvmInterfaceService.test.ts index 08cf15487..03f501742 100644 --- a/src/application/evm/EvmInterfaceService.test.ts +++ b/src/application/evm/EvmInterfaceService.test.ts @@ -1,5 +1,6 @@ import { expect, it } from "vitest"; import { EvmInterfaceService } from "./EvmInterfaceService.js"; +import type { EvmInterfaceExecution } from "./EvmInterfacePort.js"; import { evmInterfaceExecution, evmInterfaceFixture, @@ -43,7 +44,8 @@ it.each(["carrier", "encoding", "digest", "no-subject"])( }, }; const service = new EvmInterfaceService({ - inspect: () => Promise.resolve(ok(execution)), + inspect: () => + Promise.resolve(ok(execution as unknown as EvmInterfaceExecution)), }); expect( await service.inspect({ diff --git a/src/application/evm/EvmInterfaceService.ts b/src/application/evm/EvmInterfaceService.ts index eac1986d3..86ac77865 100644 --- a/src/application/evm/EvmInterfaceService.ts +++ b/src/application/evm/EvmInterfaceService.ts @@ -67,6 +67,7 @@ export class EvmInterfaceService { !report.success || report.data.artifact.path !== input.data.path || report.data.artifact.encoding !== input.data.encoding || + execution.subject?.format !== "file" || execution.subject?.path !== input.data.path || execution.subject.sha256 !== report.data.artifact.sha256 ) diff --git a/src/application/wasm/WasmArtifactPort.ts b/src/application/wasm/WasmArtifactPort.ts index 6cf6de352..90daab4b1 100644 --- a/src/application/wasm/WasmArtifactPort.ts +++ b/src/application/wasm/WasmArtifactPort.ts @@ -4,8 +4,14 @@ import type { } from "../AnalysisProvider.js"; import type { AnalysisError } from "../../domain/analysisErrorBase.js"; import type { InspectWasmArtifactInput } from "../../domain/wasm/wasmArtifact.js"; +import type { EvidenceSubjectTarget } from "../../domain/evidence.js"; import type { Result } from "../../domain/result.js"; +/** Successful WASM inspection must identify the selected local file. */ +export type WasmArtifactExecution = Omit & { + readonly subject: EvidenceSubjectTarget & { readonly format: "file" }; +}; + /** Replaceable offline WASM inspection engine, independent of target execution and network clients. */ export interface WasmArtifactPort { /** Retry removal of retained workspaces; failures preserve ownership for another attempt. */ @@ -13,5 +19,5 @@ export interface WasmArtifactPort { inspect( input: InspectWasmArtifactInput, options?: ExecutionOptions, - ): Promise>; + ): Promise>; } diff --git a/src/domain/analysisView/evmInterfaceView.ts b/src/domain/analysisView/evmInterfaceView.ts index a7b343291..d9f3994d4 100644 --- a/src/domain/analysisView/evmInterfaceView.ts +++ b/src/domain/analysisView/evmInterfaceView.ts @@ -76,6 +76,7 @@ export const projectEvmInterfaceView = ( const report = parsed.data; if ( parent.subject == null || + parent.subject.format !== "file" || parent.subject.local_path !== report.artifact.path || parent.subject.digest.sha256 !== report.artifact.sha256 ) diff --git a/src/domain/analysisView/wasmArtifactView.ts b/src/domain/analysisView/wasmArtifactView.ts index 111078600..dc3036b26 100644 --- a/src/domain/analysisView/wasmArtifactView.ts +++ b/src/domain/analysisView/wasmArtifactView.ts @@ -103,6 +103,7 @@ export const projectWasmArtifactView = ( : { path: parent.subject.local_path, sha256: parent.subject.digest.sha256, + format: parent.subject.format, }, ) ) diff --git a/src/domain/wasm/wasmArtifact.ts b/src/domain/wasm/wasmArtifact.ts index 254a55a3a..d8c89ddf3 100644 --- a/src/domain/wasm/wasmArtifact.ts +++ b/src/domain/wasm/wasmArtifact.ts @@ -1,5 +1,6 @@ import { z } from "zod"; import { createHash } from "node:crypto"; +import type { EvidenceSubjectTarget } from "../evidence.js"; import { localPathStringSchema } from "../localPath.js"; @@ -137,11 +138,20 @@ export type InspectWasmArtifactInput = z.infer< >; export type WasmArtifact = z.infer; -/** A WASM observation must identify the exact selected path and byte digest. */ +/** A WASM observation must identify the selected file path and byte digest. */ +export type WasmArtifactSubject = Pick< + EvidenceSubjectTarget, + "path" | "sha256" | "format" +>; +type WasmArtifactFileSubject = WasmArtifactSubject & { + readonly format: "file"; +}; + export const wasmArtifactMatchesSubject = ( artifact: WasmArtifact["artifact"], - subject: Pick | null | undefined, -): subject is Pick => + subject: WasmArtifactSubject | null | undefined, +): subject is WasmArtifactFileSubject => subject != null && subject.path === artifact.path && - subject.sha256 === artifact.sha256; + subject.sha256 === artifact.sha256 && + subject.format === "file"; diff --git a/src/evm/EvmoleInterfaceProvider.ts b/src/evm/EvmoleInterfaceProvider.ts index 03731a92a..7437edc23 100644 --- a/src/evm/EvmoleInterfaceProvider.ts +++ b/src/evm/EvmoleInterfaceProvider.ts @@ -6,10 +6,12 @@ import { fileURLToPath } from "node:url"; import { z } from "zod"; import { createAnalysisExecution, - type AnalysisExecution, type ExecutionOptions, } from "../application/AnalysisProvider.js"; -import type { EvmInterfacePort } from "../application/evm/EvmInterfacePort.js"; +import type { + EvmInterfaceExecution, + EvmInterfacePort, +} from "../application/evm/EvmInterfacePort.js"; import { readStableArtifact } from "../artifacts/readStableArtifact.js"; import { OwnedCommandFailure } from "../process/OwnedCommand.js"; import type { @@ -98,7 +100,7 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort { inspect( input: InspectEvmInterfaceInput, options?: ExecutionOptions, - ): Promise> { + ): Promise> { if (this.#closing) return Promise.resolve(err(this.#closedError())); const operation = this.#tail.then(() => this.#closing ? err(this.#closedError()) : this.#inspect(input, options), @@ -113,7 +115,7 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort { async #inspect( input: InspectEvmInterfaceInput, options?: ExecutionOptions, - ): Promise> { + ): Promise> { const pending = await this.#retire(); if (pending !== undefined) return err(pending); let root: Pick | undefined; @@ -123,7 +125,7 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort { let workerLimits: EvmWorkerLimits | undefined; const limiter = this.environment.REA_EVM_PRLIMIT_COMMAND ?? "/usr/bin/prlimit"; - let result: Result; + let result: Result; try { if (options?.signal?.aborted) throw new AnalysisCancelledError(OPERATION); if (process.platform !== "linux" || process.arch !== "x64") diff --git a/src/wasm/WabtArtifactProvider.ts b/src/wasm/WabtArtifactProvider.ts index ffed3871b..fe79914ca 100644 --- a/src/wasm/WabtArtifactProvider.ts +++ b/src/wasm/WabtArtifactProvider.ts @@ -2,12 +2,12 @@ import { randomUUID, createHash } from "node:crypto"; import { access, realpath, writeFile } from "node:fs/promises"; import { constants } from "node:fs"; import { isAbsolute, join } from "node:path"; -import type { - AnalysisExecution, - ExecutionOptions, -} from "../application/AnalysisProvider.js"; +import type { ExecutionOptions } from "../application/AnalysisProvider.js"; import { createAnalysisExecution } from "../application/AnalysisProvider.js"; -import type { WasmArtifactPort } from "../application/wasm/WasmArtifactPort.js"; +import type { + WasmArtifactExecution, + WasmArtifactPort, +} from "../application/wasm/WasmArtifactPort.js"; import { readStableArtifact } from "../artifacts/readStableArtifact.js"; import type { AnalysisError, @@ -76,10 +76,10 @@ export class WabtArtifactProvider implements WasmArtifactPort { async inspect( input: InspectWasmArtifactInput, options?: ExecutionOptions, - ): Promise> { + ): Promise> { let root: Pick | undefined; let phase: "configuration" | "artifact" | "producer" = "configuration"; - let result: Result; + let result: Result; let capturedOutput: AnalysisCapturedOutput | undefined; try { if (options?.signal?.aborted) throw new AnalysisCancelledError(OPERATION); diff --git a/tests/boundary/mcp/analysisViewMcp.test.ts b/tests/boundary/mcp/analysisViewMcp.test.ts index 5b2fe98af..b95aebe57 100644 --- a/tests/boundary/mcp/analysisViewMcp.test.ts +++ b/tests/boundary/mcp/analysisViewMcp.test.ts @@ -39,6 +39,8 @@ import { } from "../../fixtures/analysisView.js"; import { createTestBinarySession } from "../../fixtures/binarySession.js"; import { BINARY_LAYOUT_TEST_PROVIDER } from "../../fixtures/binaryDiagnostics/layout.js"; +import { wasmArtifactFixture } from "../../fixtures/wasm/artifact.js"; +import { evmInterfaceFixture } from "../../fixtures/evm/interface.js"; const connect = async ( binaryLayout?: BinaryLayoutService, @@ -310,6 +312,65 @@ it("reports a tampered inline Evidence ID at its full MCP source path", async () }); }); +it("rejects self-contained WASM and EVM Evidence views when the subject format differs", async () => { + const { client } = await connect(); + const wasm = wasmArtifactFixture(); + const evm = evmInterfaceFixture(); + const cases = [ + { + evidence: createEvidence( + { + path: wasm.artifact.path, + format: "elf", + sha256: wasm.artifact.sha256, + }, + { id: "fixture-wabt", name: "WABT", version: "1" }, + { + operation: "inspect_wasm_artifact", + parameters: { path: wasm.artifact.path }, + result: wasm, + }, + ), + }, + { + evidence: createEvidence( + { + path: evm.artifact.path, + format: "elf", + sha256: evm.artifact.sha256, + }, + { id: "fixture-evm", name: "EVMole", version: "1" }, + { + operation: "inspect_evm_interface", + parameters: { path: evm.artifact.path, encoding: "hex" }, + result: evm, + }, + ), + }, + ] as const; + + for (const { evidence } of cases) { + const response = await client.callTool({ + name: "inspect_analysis_view", + arguments: { + source: { kind: "inline", evidence }, + view: { kind: "summary" }, + }, + }); + expect(parseMcpToolError(response).error).toMatchObject({ + code: "invalid_request", + details: { + issues: [ + expect.objectContaining({ + path: ["source", "subject"], + reason: "invalid_value", + }), + ], + }, + }); + } +}); + it("admits portable captures before SDK Evidence parsing and preserves capture caveats", async () => { const { client } = await connect(); const dense = analysisViewCaptureEvidence(analysisViewObjectDenseCapture()); diff --git a/tests/boundary/mcp/evmInterfaceMcp.test.ts b/tests/boundary/mcp/evmInterfaceMcp.test.ts index f3594dd89..823a4d3cf 100644 --- a/tests/boundary/mcp/evmInterfaceMcp.test.ts +++ b/tests/boundary/mcp/evmInterfaceMcp.test.ts @@ -107,6 +107,30 @@ it("advertises exact valid schemas and records inline EVM interface evidence wit ).toBe(true); }); +it("rejects an EVM provider response with a non-file subject before recording Evidence", async () => { + const value = evmInterfaceFixture(); + const execution = evmInterfaceExecution(value); + const malformed = { + ...execution, + subject: { ...execution.subject, format: "elf" }, + } as unknown as typeof execution; + const harness = await createApplicationMcpHarness({ + serverOptions: { + evmInterface: new EvmInterfaceService({ + inspect: () => Promise.resolve(ok(malformed)), + }), + }, + }); + onTestFinished(harness.close); + const before = harness.session.exportEvidenceBundle().records.length; + const response = await harness.client.callTool({ + name: "inspect_evm_interface", + arguments: { path: value.artifact.path, encoding: "hex" }, + }); + expect(parseMcpToolError(response).error.code).toBe("unreadable_output"); + expect(harness.session.exportEvidenceBundle().records).toHaveLength(before); +}); + it.each(["unavailable", "incompatible-output"])( "preserves a healthy WASM peer and explicit EVM failure: %s", async (scenario) => { diff --git a/tests/boundary/mcp/wasmArtifactMcp.test.ts b/tests/boundary/mcp/wasmArtifactMcp.test.ts index 49c7ad762..003a37cba 100644 --- a/tests/boundary/mcp/wasmArtifactMcp.test.ts +++ b/tests/boundary/mcp/wasmArtifactMcp.test.ts @@ -213,6 +213,31 @@ it("advertises exact valid schemas and records inline WASM interface evidence wi expect(close).toHaveBeenCalled(); }); +it("rejects a WASM provider response with a non-file subject before recording Evidence", async () => { + const value = wasmArtifactFixture(); + const execution = wasmArtifactExecution(value); + const malformed = { + ...execution, + subject: { ...execution.subject, format: "elf" }, + } as unknown as typeof execution; + const harness = await createApplicationMcpHarness({ + serverOptions: { + wasmArtifact: new WasmArtifactService({ + close: () => Promise.resolve(), + inspect: () => Promise.resolve(ok(malformed)), + }), + }, + }); + onTestFinished(harness.close); + const before = harness.session.exportEvidenceBundle().records.length; + const response = await harness.client.callTool({ + name: "inspect_wasm_artifact", + arguments: { path: value.artifact.path }, + }); + expect(parseMcpToolError(response).error.code).toBe("unreadable_output"); + expect(harness.session.exportEvidenceBundle().records).toHaveLength(before); +}); + it("reads an oversized completed WASM result through its reported retained reference", async () => { const text = "(module)\n" + ";;😀\n" + " ".repeat(6 * 1024 * 1024); const value = { diff --git a/tests/fixtures/evm/interface.ts b/tests/fixtures/evm/interface.ts index 23bfcfa02..f0f1960ba 100644 --- a/tests/fixtures/evm/interface.ts +++ b/tests/fixtures/evm/interface.ts @@ -1,5 +1,6 @@ import { createHash } from "node:crypto"; import { createAnalysisExecution } from "../../../src/application/AnalysisProvider.js"; +import type { EvmInterfaceExecution } from "../../../src/application/evm/EvmInterfacePort.js"; import type { EvmInterface } from "../../../src/domain/evm/evmInterface.js"; /** Source-owned transport seam, independent of real engine verification. */ @@ -27,8 +28,10 @@ export const evmInterfaceFixture = ( }); /** Bind a fixture to its explicit producing seam and carrier identity. */ -export const evmInterfaceExecution = (value = evmInterfaceFixture()) => - createAnalysisExecution( +export const evmInterfaceExecution = ( + value = evmInterfaceFixture(), +): EvmInterfaceExecution => { + return createAnalysisExecution( value, { id: "fixture-evm", name: "Source-owned EVM seam", version: "1" }, { @@ -42,3 +45,4 @@ export const evmInterfaceExecution = (value = evmInterfaceFixture()) => locations: [{ kind: "artifact-path", path: value.artifact.path }], }, ); +}; diff --git a/tests/fixtures/wasm/artifact.ts b/tests/fixtures/wasm/artifact.ts index d4cc803b4..3b615ca06 100644 --- a/tests/fixtures/wasm/artifact.ts +++ b/tests/fixtures/wasm/artifact.ts @@ -1,5 +1,6 @@ import { createHash } from "node:crypto"; import { createAnalysisExecution } from "../../../src/application/AnalysisProvider.js"; +import type { WasmArtifactExecution } from "../../../src/application/wasm/WasmArtifactPort.js"; import type { WasmArtifact } from "../../../src/domain/wasm/wasmArtifact.js"; /** Source-owned transport seam; upstream binaries are verified in a separate lane. */ export const wasmArtifactFixture = ( @@ -42,8 +43,10 @@ export const wasmArtifactFixture = ( network_fetch: "not-performed" as const, limitations: ["Source-owned transport seam."], }); -export const wasmArtifactExecution = (value = wasmArtifactFixture()) => - createAnalysisExecution( +export const wasmArtifactExecution = ( + value = wasmArtifactFixture(), +): WasmArtifactExecution => { + return createAnalysisExecution( value, { id: "fixture-wabt", name: "Source-owned WABT seam", version: "1" }, { @@ -55,3 +58,4 @@ export const wasmArtifactExecution = (value = wasmArtifactFixture()) => limitations: value.limitations, }, ); +}; From 1f7f808c4f3bb5eca6b13bcc71d5cceb2ace6614 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:23:38 +0800 Subject: [PATCH 026/159] fix(reconstruction): resolve latest verifier observations without input-order bias Group observations by exact normalized instant, retaining arbitrary fractional precision and offset equivalence. Coalesce equivalent assessments and all supporting Evidence; report conflicting latest assessments as unknown with verifier-result-ambiguous. Use the same instant comparison for freshness and future checks. Replace first-input-wins coverage with permutation, provenance, offset, sub-millisecond ordering, and future-clock regressions, including the public MCP workflow. Validation: focused reconstruction/offset tests and MCP workflow passed; converted JSON Schema validation, typecheck/lint, and docs checks passed. Full suite and real-provider lanes were not run. --- src/contracts/applicationToolContracts.ts | 2 +- src/domain/offsetDateTime.ts | 83 ++++++++ src/domain/reconstructionCoverage.test.ts | 182 ++++++++++++++++-- src/domain/reconstructionCoverage.ts | 9 +- .../reconstructionCoverageEvaluation.ts | 68 ++++--- .../reconstructionCoverageVerifierResults.ts | 57 ++++++ .../mcp/reconstructionCoverageMcp.test.ts | 41 +++- 7 files changed, 402 insertions(+), 40 deletions(-) create mode 100644 src/domain/reconstructionCoverageVerifierResults.ts diff --git a/src/contracts/applicationToolContracts.ts b/src/contracts/applicationToolContracts.ts index 9a592b536..ade29e960 100644 --- a/src/contracts/applicationToolContracts.ts +++ b/src/contracts/applicationToolContracts.ts @@ -218,7 +218,7 @@ export const APPLICATION_TOOL_CONTRACTS = [ name: "evaluate_reconstruction_coverage", ...toolContractMetadata("evaluate_reconstruction_coverage"), description: - "Evaluate inline evidence-backed reconstruction coverage against one named boundary. Missing ownership or inventory is partial; stale, weak, truncated, skipped, or unresolved proof is unknown; contradictions, failed proof, missing owners, and authority routing fail closed.", + "Evaluate inline evidence-backed reconstruction coverage against one named boundary. Missing ownership or inventory is partial; stale, weak, truncated, skipped, unresolved, or conflicting tied verifier proof is unknown; contradictions, failed proof, missing owners, and authority routing fail closed.", kind: "application", inputSchema: reconstructionCoverageEvaluationInputSchema, outputSchema: reconstructionClosureResultSchema, diff --git a/src/domain/offsetDateTime.ts b/src/domain/offsetDateTime.ts index bccda7494..b7d38550f 100644 --- a/src/domain/offsetDateTime.ts +++ b/src/domain/offsetDateTime.ts @@ -1,5 +1,11 @@ import { z } from "zod"; +export interface OffsetDateTimeInstant { + readonly epochSecond: number; + /** Exact fractional second with insignificant trailing zeroes removed. */ + readonly fraction: string; +} + /** * ISO-8601 date-time that requires an explicit UTC offset. * @@ -19,3 +25,80 @@ export const offsetDateTimeSchema = z /^(?:(?:\d\d[2468][048]|\d\d[13579][26]|\d\d0[48]|[02468][048]00|[13579][26]00)-02-29|\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\d|30)|(?:02)-(?:0[1-9]|1\d|2[0-8])))T(?:(?:[01]\d|2[0-3]):[0-5]\d(?::[0-5]\d(?:\.\d+)?)?(?:Z|(?:[+\x2d](?:[01]\d|2[0-3]):[0-5]\d)))$/u, "Use an ISO-8601 date-time with an explicit UTC offset", ); + +/** Parse an already validated offset timestamp without losing sub-millisecond precision. */ +export const parseOffsetDateTimeInstant = ( + value: string, +): OffsetDateTimeInstant => { + const match = /^(.*?)(?:\.(\d+))?(Z|[+\x2d]\d{2}:\d{2})$/u.exec(value); + if (match === null) + throw new TypeError("Expected a validated offset date-time."); + const [, dateTime, fractionalDigits, offset] = match; + if (dateTime === undefined || offset === undefined) + throw new TypeError("Expected a validated offset date-time."); + const wholeSecondMilliseconds = Date.parse(`${dateTime}${offset}`); + if ( + !Number.isSafeInteger(wholeSecondMilliseconds) || + wholeSecondMilliseconds % 1000 !== 0 + ) + throw new TypeError( + "Offset date-time is outside the supported time range.", + ); + return { + epochSecond: wholeSecondMilliseconds / 1000, + fraction: fractionalDigits?.replace(/0+$/u, "") ?? "", + }; +}; + +/** Convert a Date.now-style integer into the same exact comparison shape. */ +export const offsetDateTimeInstantFromEpochMilliseconds = ( + epochMilliseconds: number, +): OffsetDateTimeInstant => { + if (!Number.isFinite(epochMilliseconds)) + throw new TypeError("Expected finite epoch milliseconds."); + if (Math.abs(epochMilliseconds) > Number.MAX_SAFE_INTEGER) + return { epochSecond: epochMilliseconds / 1000, fraction: "" }; + const epochSecond = Math.floor(epochMilliseconds / 1000); + const millisecondsWithinSecond = epochMilliseconds - epochSecond * 1000; + return { + epochSecond, + fraction: fractionalSecondFromMilliseconds(millisecondsWithinSecond), + }; +}; + +const fractionalSecondFromMilliseconds = (milliseconds: number): string => { + if (milliseconds === 0) return ""; + const [coefficient = "", exponentText] = String(milliseconds) + .toLowerCase() + .split("e"); + const exponent = exponentText === undefined ? 0 : Number(exponentText); + const point = coefficient.indexOf("."); + const decimalPosition = + (point < 0 ? coefficient.length : point) + exponent - 3; + const digits = coefficient.replace(".", ""); + const fraction = + decimalPosition <= 0 + ? `0.${"0".repeat(-decimalPosition)}${digits}` + : decimalPosition < digits.length + ? `${digits.slice(0, decimalPosition)}.${digits.slice(decimalPosition)}` + : `${digits}${"0".repeat(decimalPosition - digits.length)}`; + return ( + fraction.includes(".") ? fraction.slice(fraction.indexOf(".") + 1) : "" + ).replace(/0+$/u, ""); +}; + +/** Compare instants exactly while treating absent fraction digits as zeroes. */ +export const compareOffsetDateTimeInstants = ( + left: OffsetDateTimeInstant, + right: OffsetDateTimeInstant, +): number => { + if (left.epochSecond !== right.epochSecond) + return left.epochSecond < right.epochSecond ? -1 : 1; + const length = Math.max(left.fraction.length, right.fraction.length); + for (let index = 0; index < length; index += 1) { + const leftDigit = left.fraction.charCodeAt(index) || 48; + const rightDigit = right.fraction.charCodeAt(index) || 48; + if (leftDigit !== rightDigit) return leftDigit < rightDigit ? -1 : 1; + } + return 0; +}; diff --git a/src/domain/reconstructionCoverage.test.ts b/src/domain/reconstructionCoverage.test.ts index 4c0b41831..ac7552571 100644 --- a/src/domain/reconstructionCoverage.test.ts +++ b/src/domain/reconstructionCoverage.test.ts @@ -286,14 +286,18 @@ describe("reconstruction coverage verifier selection", () => { }); }); -describe("reconstruction coverage verifier result ordering", () => { - it("preserves the first input result when verifier timestamps tie", () => { +describe("reconstruction coverage conflicting latest verifier observations", () => { + it("keeps conflicting tied assessments unknown regardless of input order", () => { const workspace = completeReconstructionCoverageData(); const result = workspace.verifier_results[0]; const boundary = workspace.boundaries[0]; if (result === undefined || boundary === undefined) throw new Error("Expected complete reconstruction coverage fixture"); - const failed = { ...result, status: "fail" as const }; + const failed = { + ...result, + status: "fail" as const, + evidence_ids: [reconstructionCoverageEvidenceId("1")], + }; const firstPass = createReconstructionCoverageData({ ...workspace, verifier_results: [result, failed], @@ -303,23 +307,109 @@ describe("reconstruction coverage verifier result ordering", () => { verifier_results: [failed, result], }); + const firstPassResult = evaluateReconstructionClosure( + firstPass, + boundary.boundary_id, + RECONSTRUCTION_COVERAGE_NOW, + ); + const firstFailResult = evaluateReconstructionClosure( + firstFail, + boundary.boundary_id, + RECONSTRUCTION_COVERAGE_NOW, + ); + expect(firstPassResult).toEqual(firstFailResult); + expect(firstPassResult).toMatchObject({ + status: "unknown", + reasons: [ + expect.objectContaining({ + code: "verifier-result-ambiguous", + subject_id: "claim.cli.help", + }), + ], + }); + expect(firstPassResult.evidence_ids).toEqual( + expect.arrayContaining([ + reconstructionCoverageEvidenceId("3"), + reconstructionCoverageEvidenceId("1"), + ]), + ); + }); +}); + +describe("reconstruction coverage equivalent latest verifier observations", () => { + it("coalesces equivalent equal-instant observations and retains all evidence", () => { + const workspace = completeReconstructionCoverageData(); + const result = workspace.verifier_results[0]; + const boundary = workspace.boundaries[0]; + if (result === undefined || boundary === undefined) + throw new Error("Expected complete reconstruction coverage fixture"); + const first = { + ...result, + observed_at: "2026-07-16T11:00:00.123456789012345678900Z", + }; + const equivalent = { + ...first, + observed_at: "2026-07-16T06:00:00.1234567890123456789-05:00", + covered_claim_ids: [...result.covered_claim_ids].reverse(), + covered_dimensions: [...result.covered_dimensions].reverse(), + artifact_sha256s: [...result.artifact_sha256s].reverse(), + owner_sha256s: [...result.owner_sha256s].reverse(), + evidence_ids: [reconstructionCoverageEvidenceId("1")], + }; + const forward = createReconstructionCoverageData({ + ...workspace, + verifier_results: [first, equivalent], + }); + const reversed = createReconstructionCoverageData({ + ...workspace, + verifier_results: [equivalent, first], + }); + const forwardResult = evaluateReconstructionClosure( + forward, + boundary.boundary_id, + RECONSTRUCTION_COVERAGE_NOW, + ); + const reversedResult = evaluateReconstructionClosure( + reversed, + boundary.boundary_id, + RECONSTRUCTION_COVERAGE_NOW, + ); + + expect(forwardResult).toEqual(reversedResult); + expect(forwardResult.status).toBe("ready"); + expect(forwardResult.evidence_ids).toEqual( + expect.arrayContaining([ + reconstructionCoverageEvidenceId("3"), + reconstructionCoverageEvidenceId("1"), + ]), + ); + }); +}); + +describe("reconstruction coverage verifier timestamp selection", () => { + it("lets a later unambiguous observation supersede an older tie", () => { + const workspace = completeReconstructionCoverageData(); + const result = workspace.verifier_results[0]; + const boundary = workspace.boundaries[0]; + if (result === undefined || boundary === undefined) + throw new Error("Expected complete reconstruction coverage fixture"); + const failed = { ...result, status: "fail" as const }; + const latestPass = { + ...result, + observed_at: "2026-07-16T11:30:00.000Z", + }; + const changed = createReconstructionCoverageData({ + ...workspace, + verifier_results: [result, failed, latestPass], + }); + expect( evaluateReconstructionClosure( - firstPass, - boundary.boundary_id, - RECONSTRUCTION_COVERAGE_NOW, - ).status, - ).toBe("ready"); - expect( - evaluateReconstructionClosure( - firstFail, + changed, boundary.boundary_id, RECONSTRUCTION_COVERAGE_NOW, ), - ).toMatchObject({ - status: "failed", - reasons: [expect.objectContaining({ code: "verifier-failed" })], - }); + ).toMatchObject({ status: "ready", reasons: [] }); }); it("orders offset-bearing verifier timestamps chronologically", () => { @@ -352,6 +442,68 @@ describe("reconstruction coverage verifier result ordering", () => { }); }); + it("selects the later result when timestamps differ below millisecond precision", () => { + const workspace = completeReconstructionCoverageData(); + const result = workspace.verifier_results[0]; + if (result === undefined) + throw new Error("Expected fixture verifier result"); + const changed = createReconstructionCoverageData({ + ...workspace, + verifier_results: [ + { ...result, observed_at: "2026-07-16T11:00:00.1231Z" }, + { + ...result, + observed_at: "2026-07-16T11:00:00.1239Z", + status: "fail", + }, + ], + }); + + expect( + evaluateReconstructionClosure( + changed, + "replacement.cli", + RECONSTRUCTION_COVERAGE_NOW, + ), + ).toMatchObject({ + status: "failed", + reasons: [expect.objectContaining({ code: "verifier-failed" })], + }); + }); +}); + +describe("reconstruction coverage verifier future timestamps", () => { + it("treats an observation beyond millisecond-precision now as future", () => { + const workspace = completeReconstructionCoverageData(); + const result = workspace.verifier_results[0]; + if (result === undefined) + throw new Error("Expected fixture verifier result"); + const changed = createReconstructionCoverageData({ + ...workspace, + verifier_results: [ + { ...result, observed_at: "2026-07-16T12:00:00.1239Z" }, + ], + }); + + expect( + evaluateReconstructionClosure( + changed, + "replacement.cli", + Date.parse("2026-07-16T12:00:00.123Z") + 0.5, + ), + ).toMatchObject({ + status: "unknown", + reasons: [expect.objectContaining({ code: "verifier-result-stale" })], + }); + }); + + it("rejects a non-finite evaluation clock", () => { + const workspace = completeReconstructionCoverageData(); + expect(() => + evaluateReconstructionClosure(workspace, "replacement.cli", Number.NaN), + ).toThrow("Expected finite epoch milliseconds."); + }); + it("rejects verifier observations from the future", () => { const workspace = completeReconstructionCoverageData(); const result = workspace.verifier_results[0]; diff --git a/src/domain/reconstructionCoverage.ts b/src/domain/reconstructionCoverage.ts index 485b66a5a..04913d4ae 100644 --- a/src/domain/reconstructionCoverage.ts +++ b/src/domain/reconstructionCoverage.ts @@ -19,7 +19,10 @@ import { } from "./reconstructionCoverageEvaluation.js"; import { digestSchema } from "./../domain/digests.js"; import { prefixedDigestSchema } from "./../domain/digests.js"; -import { offsetDateTimeSchema } from "./offsetDateTime.js"; +import { + offsetDateTimeInstantFromEpochMilliseconds, + offsetDateTimeSchema, +} from "./offsetDateTime.js"; const evidenceIdSchema = prefixedDigestSchema("ev"); const stableIdSchema = z @@ -213,6 +216,7 @@ const closureReasonSchema = z.strictObject({ "verifier-result-missing", "verifier-result-stale", "verifier-result-incompatible", + "verifier-result-ambiguous", "verifier-failed", "verifier-unknown", "active-unknown", @@ -274,10 +278,12 @@ export const evaluateReconstructionClosure = ( ); if (boundary === undefined) throw new TypeError(`Unknown reconstruction boundary: ${boundaryId}`); + const nowInstant = offsetDateTimeInstantFromEpochMilliseconds(nowEpochMs); const context: ReconstructionEvaluationContext = { coverage, boundary, nowEpochMs, + nowInstant, reasons: [], evidenceIds: new Set(), }; @@ -394,6 +400,7 @@ const closureStatus = ( [ "verifier-result-stale", "verifier-result-incompatible", + "verifier-result-ambiguous", "verifier-unknown", "active-unknown", "owner-path-unknown", diff --git a/src/domain/reconstructionCoverageEvaluation.ts b/src/domain/reconstructionCoverageEvaluation.ts index ea82e5636..f8a56b261 100644 --- a/src/domain/reconstructionCoverageEvaluation.ts +++ b/src/domain/reconstructionCoverageEvaluation.ts @@ -3,6 +3,16 @@ import type { ReconstructionCoverageData, ReconstructionVerifierContract, } from "./reconstructionCoverage.js"; +import { + compareOffsetDateTimeInstants, + offsetDateTimeInstantFromEpochMilliseconds, + type OffsetDateTimeInstant, +} from "./offsetDateTime.js"; +import { + latestVerifierObservationsByVerifier, + sameVerifierAssessment, + type VerifierObservation, +} from "./reconstructionCoverageVerifierResults.js"; export const recommendedReconstructionProbes = ( reasons: readonly ClosureReason[], @@ -29,6 +39,7 @@ export interface ReconstructionEvaluationContext { readonly coverage: ReconstructionCoverageData; readonly boundary: Boundary; readonly nowEpochMs: number; + readonly nowInstant: OffsetDateTimeInstant; readonly reasons: ClosureReason[]; readonly evidenceIds: Set; } @@ -184,7 +195,7 @@ interface VerifierIndexes { >; readonly latestResultsByVerifier: ReadonlyMap< string, - ReconstructionCoverageData["verifier_results"][number] + readonly VerifierObservation[] >; readonly artifactDigests: ReadonlySet; readonly ownerDigests: ReadonlySet; @@ -213,20 +224,6 @@ const createVerifierIndexes = ( contractsByClaim.set(claimId, contracts); } - const latestResultsByVerifier = new Map< - string, - ReconstructionCoverageData["verifier_results"][number] - >(); - for (const result of coverage.verifier_results) { - const latest = latestResultsByVerifier.get(result.verifier_id); - // Stable descending sort used to choose the first input record on ties. - if ( - latest === undefined || - Date.parse(result.observed_at) > Date.parse(latest.observed_at) - ) - latestResultsByVerifier.set(result.verifier_id, result); - } - const ownersBySurface = new Map< string, ReconstructionCoverageData["owners"][number][] @@ -239,7 +236,9 @@ const createVerifierIndexes = ( return { contractsByClaim, - latestResultsByVerifier, + latestResultsByVerifier: latestVerifierObservationsByVerifier( + coverage.verifier_results, + ), artifactDigests: new Set( coverage.artifacts.map(({ artifact_sha256: value }) => value), ), @@ -260,8 +259,10 @@ const evaluateVerifierResult = ( contract: ReconstructionVerifierContract, indexes: VerifierIndexes, ): void => { - const result = indexes.latestResultsByVerifier.get(contract.verifier_id); - if (result === undefined) { + const latestResults = indexes.latestResultsByVerifier.get( + contract.verifier_id, + ); + if (latestResults === undefined) { addReason( context, "verifier-result-missing", @@ -270,7 +271,26 @@ const evaluateVerifierResult = ( ); return; } - addEvidence(context, result.evidence_ids); + addEvidence( + context, + latestResults.flatMap(({ result }) => result.evidence_ids), + ); + const latest = latestResults[0]; + if (latest === undefined) return; + const { result } = latest; + if ( + !latestResults.every((candidate) => + sameVerifierAssessment(result, candidate.result), + ) + ) { + addReason( + context, + "verifier-result-ambiguous", + claim.claim_id, + "Latest verifier observations have conflicting assessment content.", + ); + return; + } if (!verifierResultIsCompatible(claim, contract, result, indexes)) { addReason( context, @@ -280,10 +300,14 @@ const evaluateVerifierResult = ( ); return; } - const observedAt = Date.parse(result.observed_at); + const cutoffEpochMs = context.nowEpochMs - contract.max_age_ms; if ( - observedAt > context.nowEpochMs || - context.nowEpochMs - observedAt > contract.max_age_ms + compareOffsetDateTimeInstants(latest.instant, context.nowInstant) > 0 || + (cutoffEpochMs >= -Number.MAX_SAFE_INTEGER && + compareOffsetDateTimeInstants( + latest.instant, + offsetDateTimeInstantFromEpochMilliseconds(cutoffEpochMs), + ) < 0) ) { addReason( context, diff --git a/src/domain/reconstructionCoverageVerifierResults.ts b/src/domain/reconstructionCoverageVerifierResults.ts new file mode 100644 index 000000000..de7f55874 --- /dev/null +++ b/src/domain/reconstructionCoverageVerifierResults.ts @@ -0,0 +1,57 @@ +import type { ReconstructionCoverageData } from "./reconstructionCoverage.js"; +import { uniqueSorted } from "./canonicalOrdering.js"; +import { + compareOffsetDateTimeInstants, + parseOffsetDateTimeInstant, + type OffsetDateTimeInstant, +} from "./offsetDateTime.js"; + +type VerifierResult = ReconstructionCoverageData["verifier_results"][number]; + +export interface VerifierObservation { + readonly result: VerifierResult; + readonly instant: OffsetDateTimeInstant; +} + +/** Index each verifier's latest timestamp group without relying on array order. */ +export const latestVerifierObservationsByVerifier = ( + results: readonly VerifierResult[], +): ReadonlyMap => { + const latestByVerifier = new Map(); + for (const result of results) { + const observation = { + result, + instant: parseOffsetDateTimeInstant(result.observed_at), + }; + const latest = latestByVerifier.get(result.verifier_id); + const latestInstant = latest?.[0]?.instant; + const comparison = + latestInstant === undefined + ? 1 + : compareOffsetDateTimeInstants(observation.instant, latestInstant); + if (latest === undefined || comparison > 0) + latestByVerifier.set(result.verifier_id, [observation]); + else if (comparison === 0) latest.push(observation); + } + return latestByVerifier; +}; + +/** Compare assessment content while excluding timestamp spelling and provenance. */ +export const sameVerifierAssessment = ( + left: VerifierResult, + right: VerifierResult, +): boolean => + verifierAssessmentIdentity(left) === verifierAssessmentIdentity(right); + +const verifierAssessmentIdentity = (result: VerifierResult): string => + JSON.stringify({ + verifier_id: result.verifier_id, + contract_sha256: result.contract_sha256, + status: result.status, + covered_claim_ids: uniqueSorted(result.covered_claim_ids), + covered_dimensions: uniqueSorted(result.covered_dimensions), + artifact_sha256s: uniqueSorted(result.artifact_sha256s), + owner_sha256s: uniqueSorted(result.owner_sha256s), + normalization_sha256: result.normalization_sha256, + repeats: result.repeats, + }); diff --git a/tests/boundary/mcp/reconstructionCoverageMcp.test.ts b/tests/boundary/mcp/reconstructionCoverageMcp.test.ts index cb3605d14..1e167b8b0 100644 --- a/tests/boundary/mcp/reconstructionCoverageMcp.test.ts +++ b/tests/boundary/mcp/reconstructionCoverageMcp.test.ts @@ -3,7 +3,10 @@ import { ok as resultOk } from "../../../src/domain/result.js"; import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; import { describe, expect, it } from "vitest"; -import { completeReconstructionCoverageData } from "../../../src/domain/reconstructionCoverage.fixture.js"; +import { + completeReconstructionCoverageData, + reconstructionCoverageEvidenceId, +} from "../../../src/domain/reconstructionCoverage.fixture.js"; import { createReconstructionCoverageData } from "../../../src/domain/reconstructionCoverage.js"; import { createTestBinarySession } from "../../fixtures/binarySession.js"; import { createServer } from "../../../src/server/createServer.js"; @@ -36,6 +39,42 @@ describe("reconstruction coverage MCP", () => { boundary_id: boundaryId, summary: { reasons: 0 }, }); + const coverage = currentFixtureCoverage(boundaryId); + const latest = coverage.verifier_results[0]; + if (latest === undefined) + throw new Error("Expected fixture verifier result"); + const ambiguous = await client.callTool({ + name: "evaluate_reconstruction_coverage", + arguments: { + coverage: createReconstructionCoverageData({ + ...coverage, + verifier_results: [ + latest, + { + ...latest, + status: "fail", + evidence_ids: [reconstructionCoverageEvidenceId("1")], + }, + ], + }), + boundary_id: boundaryId, + }, + }); + expect(ambiguous.isError).not.toBe(true); + expect(ambiguous.structuredContent).toMatchObject({ + status: "unknown", + boundary_id: boundaryId, + reasons: [ + expect.objectContaining({ + code: "verifier-result-ambiguous", + subject_id: "claim.cli.help", + }), + ], + evidence_ids: expect.arrayContaining([ + reconstructionCoverageEvidenceId("3"), + reconstructionCoverageEvidenceId("1"), + ]), + }); const unknown = await client.callTool({ name: "evaluate_reconstruction_coverage", arguments: { From 1991542e59f64d46269b3ff942be77a5ff06e88f Mon Sep 17 00:00:00 2001 From: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> Date: Mon, 12 Oct 2026 22:26:48 -0700 Subject: [PATCH 027/159] test(pcap): run the missing-path refusal only where captures are admitted (#2510) On hosts without an enforceable native bound (macOS), the provider refuses inspect_pcap_capture with capability_unavailable before it touches the path, which the unsupported-boundary test in the same file pins. The missing-path test expected invalid_request on every host, so it always failed on macOS while Linux and Windows CI stayed green. Co-authored-by: Claude Opus 5.5 --- .../pcap/pcapProviderCrossPlatform.test.ts | 27 +++++++++++-------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/tests/boundary/pcap/pcapProviderCrossPlatform.test.ts b/tests/boundary/pcap/pcapProviderCrossPlatform.test.ts index 3d7f28b54..59ff76c57 100644 --- a/tests/boundary/pcap/pcapProviderCrossPlatform.test.ts +++ b/tests/boundary/pcap/pcapProviderCrossPlatform.test.ts @@ -87,17 +87,22 @@ it.runIf(hasNativeBound)( }, ); -it("rejects a path that is not a readable file", async () => { - const result = await providerOver().execute({ - operation: "inspect_pcap_capture", - input: { path: "/nonexistent/definitely-missing.pcap" }, - }); - expect(result.ok).toBe(false); - if (result.ok) return; - expect(projectAnalysisError(result.error)).toMatchObject({ - code: "invalid_request", - }); -}); +// Hosts without a native bound refuse before touching the path; that order +// is pinned by the unsupported-boundary test below. +it.runIf(hasNativeBound)( + "rejects a path that is not a readable file", + async () => { + const result = await providerOver().execute({ + operation: "inspect_pcap_capture", + input: { path: "/nonexistent/definitely-missing.pcap" }, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "invalid_request", + }); + }, +); it("reports a missing binary as a capability gap", async () => { const provider = new PcapProvider({ From bb96afdbe32f5318d57fcb00099191c5a7712f53 Mon Sep 17 00:00:00 2001 From: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> Date: Mon, 12 Oct 2026 22:29:09 -0700 Subject: [PATCH 028/159] fix(pcap): name the host tool to configure in capability refusals (#2507) * fix(pcap): name the host tool to configure in capability refusals Missing or misconfigured tshark/capinfos, the unsupported macOS capture boundary and an unusable Linux prlimit were projected with the generic "unavailable for the current target; choose another target" advice, even for inspect_pcap_client, which has no target. Route tool refusals through one pcapToolUnavailable owner that names REA_TSHARK_COMMAND and REA_CAPINFOS_COMMAND, give host-boundary refusals their own reason, and keep the private workspace refusal's retryable flag like WABT and source maps do. Co-Authored-By: Claude Opus 5.5 * test(pcap): cover the Linux prlimit refusal guidance Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 --- src/pcap/PcapCommand.ts | 8 ++--- src/pcap/PcapConfiguration.ts | 19 +++++++++++ src/pcap/PcapNativeBound.ts | 24 ++++++++++---- src/pcap/PcapProvider.ts | 10 +++--- src/pcap/PcapSnapshot.ts | 2 +- tests/boundary/cli/pcapCli.test.ts | 32 +++++++++++++------ tests/boundary/pcap/pcapProvider.test.ts | 29 +++++++++++++++-- .../pcap/pcapProviderCrossPlatform.test.ts | 12 ++++--- 8 files changed, 102 insertions(+), 34 deletions(-) diff --git a/src/pcap/PcapCommand.ts b/src/pcap/PcapCommand.ts index a0833f599..0d28a3d75 100644 --- a/src/pcap/PcapCommand.ts +++ b/src/pcap/PcapCommand.ts @@ -2,7 +2,6 @@ import { randomUUID } from "node:crypto"; import { AnalysisError } from "../domain/analysisErrorBase.js"; import { AnalysisCancelledError, - AnalysisCapabilityUnavailableError, AnalysisProtocolError, AnalysisResourceConstraintError, AnalysisTimeoutError, @@ -19,6 +18,7 @@ import { windowsBatchCommand, WindowsBatchCommandError, } from "../process/WindowsBatchCommand.js"; +import { pcapToolUnavailable } from "./PcapConfiguration.js"; export interface PcapCommandLimits { readonly timeoutMs: number; @@ -168,8 +168,7 @@ export const runPcapCommand = async (options: { "code" in cause && (cause.code === "ENOENT" || cause.code === "EACCES")) ) - throw new AnalysisCapabilityUnavailableError( - "pcap", + throw pcapToolUnavailable( operation, `Cannot run selected tool ${command}: ${cause.message}`, details, @@ -181,8 +180,7 @@ export const runPcapCommand = async (options: { ? cause.message : String(cause); if (options.inputPath === undefined) - throw new AnalysisCapabilityUnavailableError( - "pcap", + throw pcapToolUnavailable( operation, `The selected tool ${command} cannot report its version: ${diagnostic}`, details, diff --git a/src/pcap/PcapConfiguration.ts b/src/pcap/PcapConfiguration.ts index 8f75b4602..b2a6212d1 100644 --- a/src/pcap/PcapConfiguration.ts +++ b/src/pcap/PcapConfiguration.ts @@ -2,12 +2,31 @@ import { isAbsolute } from "node:path"; import type { ProviderAvailability } from "../application/AnalysisProvider.js"; import type { ProviderRejectionCode } from "../contracts/providerSelection.js"; +import type { AnalysisErrorOptions } from "../domain/analysisErrorBase.js"; +import { AnalysisCapabilityUnavailableError } from "../domain/analysisErrorCore.js"; import type { JsonValue } from "../domain/jsonValue.js"; /** Environment keys selecting the caller's tshark and capinfos binaries. */ export const TSHARK_COMMAND_ENV = "REA_TSHARK_COMMAND"; export const CAPINFOS_COMMAND_ENV = "REA_CAPINFOS_COMMAND"; +const TOOL_REMEDIATION = `Supply Wireshark 4.x tshark and capinfos on PATH or through absolute ${TSHARK_COMMAND_ENV} and ${CAPINFOS_COMMAND_ENV}; REA does not install them.`; + +/** + * Host-tool refusals concern the caller's Wireshark installation, not the + * target, so they carry their own guidance instead of the generic advice to + * choose another target. + */ +export const pcapToolUnavailable = ( + operation: string, + reason: string, + options?: AnalysisErrorOptions, +): AnalysisCapabilityUnavailableError => + new AnalysisCapabilityUnavailableError("pcap", operation, reason, { + ...options, + userMessage: `${/[.!?]$/u.test(reason) ? reason : `${reason}.`} ${TOOL_REMEDIATION}`, + }); + /** How each tool was selected for this provider instance. */ export interface PcapToolSelection { readonly tshark: { command: string; source: "environment" | "path" }; diff --git a/src/pcap/PcapNativeBound.ts b/src/pcap/PcapNativeBound.ts index 172eaba91..dc98bfca0 100644 --- a/src/pcap/PcapNativeBound.ts +++ b/src/pcap/PcapNativeBound.ts @@ -6,6 +6,8 @@ import { AnalysisCapabilityUnavailableError } from "../domain/analysisErrorCore. import type { ArtifactAnalysisResources } from "../process/ArtifactAnalysisResources.js"; import type { runPcapCommand } from "./PcapCommand.js"; +const PRLIMIT_COMMAND_ENV = "REA_TSHARK_PRLIMIT_COMMAND"; + /** Own native capture admission and the exact enforced resource profile. */ export const preparePcapNativeBound = async ( environment: Readonly>, @@ -22,17 +24,20 @@ export const preparePcapNativeBound = async ( ], }; } - if (process.platform !== "linux") + if (process.platform !== "linux") { + const reason = + "Packet summaries require an enforceable native memory bound: Linux with util-linux prlimit, or Windows with owned Job Objects. This host's capture boundary remains unsupported."; throw new AnalysisCapabilityUnavailableError( "pcap", "inspect_pcap_capture", - "Packet summaries require an enforceable native memory bound: Linux with util-linux prlimit, or Windows with owned Job Objects. This host's capture boundary remains unsupported.", + reason, + { userMessage: reason }, ); - const configured = - environment.REA_TSHARK_PRLIMIT_COMMAND ?? "/usr/bin/prlimit"; + } + const configured = environment[PRLIMIT_COMMAND_ENV] ?? "/usr/bin/prlimit"; try { if (!isAbsolute(configured)) - throw new Error("REA_TSHARK_PRLIMIT_COMMAND must be absolute."); + throw new Error(`${PRLIMIT_COMMAND_ENV} must be absolute.`); const limiter = await realpath(configured); await access(limiter, constants.X_OK); const limits = await resources.readLimits({ @@ -58,11 +63,16 @@ export const preparePcapNativeBound = async ( return { nativeBound, limitations }; } catch (cause) { if (cause instanceof AnalysisError) throw cause; + const reason = + "Cannot establish the selected Linux resource limiter before processing capture bytes."; throw new AnalysisCapabilityUnavailableError( "pcap", "inspect_pcap_capture", - "Cannot establish the selected Linux resource limiter before processing capture bytes.", - { cause }, + reason, + { + cause, + userMessage: `${reason} (${cause instanceof Error ? cause.message : String(cause)}) Check that ${PRLIMIT_COMMAND_ENV} (default /usr/bin/prlimit) names an absolute, executable util-linux prlimit.`, + }, ); } }; diff --git a/src/pcap/PcapProvider.ts b/src/pcap/PcapProvider.ts index 7228d694f..be2c12ed2 100644 --- a/src/pcap/PcapProvider.ts +++ b/src/pcap/PcapProvider.ts @@ -25,6 +25,7 @@ import { inspectPcapClient, resolvePcapTools, pcapConfigurationAvailability, + pcapToolUnavailable, type PcapToolSelection, } from "./PcapConfiguration.js"; import { @@ -346,12 +347,9 @@ export class PcapProvider { : signal?.aborted ? new AnalysisCancelledError(request.operation, { cause }) : cause instanceof PcapConfigurationFailure - ? new AnalysisCapabilityUnavailableError( - "pcap", - request.operation, - cause.message, - { cause }, - ) + ? pcapToolUnavailable(request.operation, cause.message, { + cause, + }) : new AnalysisProtocolError( `Packet-capture analysis failed: ${cause instanceof Error ? cause.message : String(cause)}`, { cause }, diff --git a/src/pcap/PcapSnapshot.ts b/src/pcap/PcapSnapshot.ts index 08f1ea552..2e9082ccd 100644 --- a/src/pcap/PcapSnapshot.ts +++ b/src/pcap/PcapSnapshot.ts @@ -43,7 +43,7 @@ export class PcapSnapshot { "pcap", OPERATION, cause.message, - { cause }, + { cause, retryable: cause.retryable }, ); throw new AnalysisResourceConstraintError( OPERATION, diff --git a/tests/boundary/cli/pcapCli.test.ts b/tests/boundary/cli/pcapCli.test.ts index 26e253455..1eb61cdcf 100644 --- a/tests/boundary/cli/pcapCli.test.ts +++ b/tests/boundary/cli/pcapCli.test.ts @@ -1,6 +1,6 @@ import { expect, it } from "vitest"; -import { serveCli } from "./cliServeHarness.js"; +import { expectConfigurationRefusal, serveCli } from "./cliServeHarness.js"; it.each(["inspect-pcap-client", "inspect-pcap-capture"])( "%s advertises a schema through the public CLI", @@ -11,13 +11,27 @@ it.each(["inspect-pcap-client", "inspect-pcap-capture"])( }, ); -it("reports missing tools with a capability refusal", async () => { - const result = await serveCli( - ["inspect-pcap-capture", "/nonexistent/capture.pcap", "--json"], - { - REA_TSHARK_COMMAND: "/nonexistent/tshark", - REA_CAPINFOS_COMMAND: "/nonexistent/capinfos", - }, +it.each([ + ["inspect-pcap-client"], + ["inspect-pcap-capture", "/nonexistent/capture.pcap"], +])("%s names the missing Wireshark tool to configure", async (...argv) => { + const result = await serveCli([...argv, "--json"], { + REA_TSHARK_COMMAND: "/nonexistent/tshark", + REA_CAPINFOS_COMMAND: "/nonexistent/capinfos", + }); + expectConfigurationRefusal( + result, + "Cannot run selected tool /nonexistent/tshark", + ); + expectConfigurationRefusal(result, "through absolute REA_TSHARK_COMMAND"); +}); + +it("names a relative tool selection as configuration to fix", async () => { + const result = await serveCli(["inspect-pcap-client", "--json"], { + REA_TSHARK_COMMAND: "tshark", + }); + expectConfigurationRefusal( + result, + "REA_TSHARK_COMMAND must be an absolute path to the tshark binary", ); - expect(result.stdout).toContain("capability_unavailable"); }); diff --git a/tests/boundary/pcap/pcapProvider.test.ts b/tests/boundary/pcap/pcapProvider.test.ts index 50204a479..fcb299ade 100644 --- a/tests/boundary/pcap/pcapProvider.test.ts +++ b/tests/boundary/pcap/pcapProvider.test.ts @@ -246,12 +246,37 @@ it("reports relative configuration and missing capinfos as capability gaps", asy input: {}, }); if (result.ok) throw new Error("expected unavailable tools"); - expect(projectAnalysisError(result.error).code).toBe( - "capability_unavailable", + const projected = projectAnalysisError(result.error); + expect(projected.code).toBe("capability_unavailable"); + expect(projected.message).toContain( + "Supply Wireshark 4.x tshark and capinfos", ); } }); +it("names the prlimit selection when the Linux limiter is unusable", async () => { + const root = await createTestTempDirectory("rea-pcap-prlimit-"); + const tools = await writeStubTools(root); + const provider = new PcapProvider({ + environment: { + REA_TSHARK_COMMAND: tools.tshark, + REA_CAPINFOS_COMMAND: tools.capinfos, + REA_TSHARK_PRLIMIT_COMMAND: "relative/prlimit", + }, + }); + onTestFinished(() => provider.close()); + const result = await provider.execute({ + operation: "inspect_pcap_capture", + input: { path: "/not-opened/capture.pcap" }, + }); + if (result.ok) throw new Error("expected an unusable limiter"); + const projected = projectAnalysisError(result.error); + expect(projected.code).toBe("capability_unavailable"); + expect(projected.message).toContain( + "(REA_TSHARK_PRLIMIT_COMMAND must be absolute.) Check that REA_TSHARK_PRLIMIT_COMMAND", + ); +}); + const nodeTool = async ( root: string, name: string, diff --git a/tests/boundary/pcap/pcapProviderCrossPlatform.test.ts b/tests/boundary/pcap/pcapProviderCrossPlatform.test.ts index 59ff76c57..c19e682fe 100644 --- a/tests/boundary/pcap/pcapProviderCrossPlatform.test.ts +++ b/tests/boundary/pcap/pcapProviderCrossPlatform.test.ts @@ -141,9 +141,13 @@ it.runIf(!hasNativeBound)( input: { path: "/not-opened/capture.pcap" }, }); expect(result.ok).toBe(false); - if (!result.ok) - expect(projectAnalysisError(result.error)).toMatchObject({ - code: "capability_unavailable", - }); + if (result.ok) return; + const projected = projectAnalysisError(result.error); + expect(projected.code).toBe("capability_unavailable"); + // The refusal concerns this host, so it must not suggest another target. + expect(projected.message).toContain( + "Linux with util-linux prlimit, or Windows with owned Job Objects", + ); + expect(projected.remediation.action).toBe(projected.message); }, ); From 2187e16f149a7a7b512a4cb45d5f468cc04091fe Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:39:48 +0800 Subject: [PATCH 029/159] fix(ghidra): derive installation admission from Java property semantics --- src/ghidra/GhidraAnalysisSeeds.test.ts | 2 +- src/ghidra/GhidraDoctor.test.ts | 12 +- src/ghidra/GhidraInstallation.test.ts | 184 +++++++++++++++- src/ghidra/GhidraInstallation.ts | 20 +- src/ghidra/GhidraInstallationPolicy.ts | 232 ++++++++++++++++++--- src/ghidra/GhidraLanguageOverride.test.ts | 2 +- src/ghidra/GhidraMipsProfile.test.ts | 2 +- src/ghidra/GhidraProvider.behavior.test.ts | 2 +- src/ghidra/GhidraRiscvProfile.test.ts | 2 +- src/ghidra/GhidraSnapshotCache.test.ts | 2 +- 10 files changed, 408 insertions(+), 52 deletions(-) diff --git a/src/ghidra/GhidraAnalysisSeeds.test.ts b/src/ghidra/GhidraAnalysisSeeds.test.ts index 8af76a4da..37437ebf7 100644 --- a/src/ghidra/GhidraAnalysisSeeds.test.ts +++ b/src/ghidra/GhidraAnalysisSeeds.test.ts @@ -196,7 +196,7 @@ describe("Ghidra seed profile", () => { const host: GhidraInstallationHost = { platform: "linux", architecture: "x64", - readText: () => "application.version=12.1.4\n", + readText: () => "application.version=12.1.4\napplication.java.min=21\n", executable: () => true, probeJava: () => ({ version: "21.0.11", diff --git a/src/ghidra/GhidraDoctor.test.ts b/src/ghidra/GhidraDoctor.test.ts index e1ea7fe37..5335e4f60 100644 --- a/src/ghidra/GhidraDoctor.test.ts +++ b/src/ghidra/GhidraDoctor.test.ts @@ -15,7 +15,7 @@ const INSTALL = "/opt/ghidra_12.1.4_PUBLIC"; const installationHost = ( overrides: Partial = {}, ): GhidraInstallationHost => ({ - readText: () => "application.version=12.1.4\n", + readText: () => "application.version=12.1.4\napplication.java.min=21\n", executable: () => true, probeJava: () => ({ version: "21.0.11", @@ -111,7 +111,7 @@ describe("Ghidra doctor integration", () => { "wrong version", inspection( { installDir: INSTALL }, - { readText: () => "application.version=11.4\n" }, + { readText: () => "application.version=11.4\napplication.java.min=21\n" }, ), "ghidra-version", ], @@ -136,6 +136,14 @@ describe("Ghidra doctor integration", () => { ), "ghidra-java", ], + [ + "missing Java minimum", + inspection( + { installDir: INSTALL }, + { readText: () => "application.version=12.1.4\n" }, + ), + "ghidra-java", + ], [ "unsupported architecture", inspection({ installDir: INSTALL, architecture: "arm" }), diff --git a/src/ghidra/GhidraInstallation.test.ts b/src/ghidra/GhidraInstallation.test.ts index 21c4195ad..776c10308 100644 --- a/src/ghidra/GhidraInstallation.test.ts +++ b/src/ghidra/GhidraInstallation.test.ts @@ -12,6 +12,8 @@ import { import { SUPPORTED_GHIDRA_JAVA_MAJOR, SUPPORTED_GHIDRA_VERSION, + ghidraApplicationProperties, + ghidraJavaRequirement, } from "./GhidraInstallationPolicy.js"; const INSTALL = "/opt/ghidra"; @@ -42,7 +44,7 @@ const host = ( ): GhidraInstallationHost => ({ readText: (path) => path === PROPERTIES - ? `application.version=${SUPPORTED_GHIDRA_VERSION}\n` + ? `application.version=${SUPPORTED_GHIDRA_VERSION}\napplication.java.min=${String(SUPPORTED_GHIDRA_JAVA_MAJOR)}\n` : undefined, executable: (path) => path === HEADLESS || path === MAC_DECOMPILER || path === LINUX_DECOMPILER, @@ -190,7 +192,7 @@ describe("Ghidra installation inspection", () => { architecture: "x64", readText: (path) => path === WINDOWS_PROPERTIES - ? `application.version=${SUPPORTED_GHIDRA_VERSION}\n` + ? `application.version=${SUPPORTED_GHIDRA_VERSION}\napplication.java.min=21\n` : undefined, executable: (path) => path === WINDOWS_HEADLESS || path === WINDOWS_DECOMPILER, @@ -275,7 +277,9 @@ describe("Ghidra installation rejection diagnostics", () => { platform: "linux" as const, architecture: "x64" as const, }, - override: { readText: () => "application.version=11.4\n" }, + override: { + readText: () => "application.version=11.4\napplication.java.min=21\n", + }, failed: "version", code: "unsupported_version", }, @@ -350,6 +354,167 @@ describe("Ghidra installation rejection diagnostics", () => { }); describe("Ghidra release line and JDK bounds", () => { + it("uses Java Properties last-key, delimiter, escape, and continuation rules", () => { + const properties = ghidraApplicationProperties( + [ + "application.version=11.4", + "# comment ending with a slash " + "\\", + "application.\\u0076ersion : 12.\\", + " 1.\\u0034", + "! another comment " + "\\", + "application.java.\\u006din\t21", + "application.java.max=", + ].join("\n"), + ); + expect(properties).toEqual({ + providerVersion: "12.1.4", + javaMinimum: { status: "major", major: 21 }, + javaMaximum: { status: "absent" }, + parseError: null, + }); + expect( + ghidraApplicationProperties("application.version=12.1.4").javaMinimum, + ).toEqual({ + status: "absent", + }); + expect( + ghidraApplicationProperties( + "application.version=12.1.4\napplication.java.min=\n", + ).javaMinimum, + ).toEqual({ status: "unresolved", raw: "" }); + expect( + ghidraApplicationProperties("application.version=12.1.4\\") + .providerVersion, + ).toBe("12.1.4"); + expect( + ghidraApplicationProperties("application.version=12.\\\n #.1") + .providerVersion, + ).toBe("12.#.1"); + expect( + ghidraApplicationProperties( + "application.version=12.\\\n\napplication.version=12.1.4", + ).providerVersion, + ).toBe("12.1.4"); + }); + + it("matches Ghidra integer Java bounds and zero maximum semantics", () => { + const properties = ghidraApplicationProperties( + "application.java.min=21.0\napplication.java.max=0\n", + ); + expect(properties.javaMinimum).toEqual({ + status: "unresolved", + raw: "21.0", + }); + expect(properties.javaMaximum).toEqual({ status: "absent" }); + expect( + ghidraApplicationProperties("application.java.max=+0").javaMaximum, + ).toEqual({ + status: "absent", + }); + expect( + ghidraApplicationProperties("application.java.max=+０").javaMaximum, + ).toEqual({ status: "absent" }); + expect( + ghidraApplicationProperties("application.java.min=+21").javaMinimum, + ).toEqual({ + status: "major", + major: 21, + }); + expect( + ghidraApplicationProperties("application.java.min=٢١").javaMinimum, + ).toEqual({ + status: "major", + major: 21, + }); + expect( + ghidraApplicationProperties("application.java.min=２１").javaMinimum, + ).toEqual({ + status: "major", + major: 21, + }); + expect( + ghidraApplicationProperties("application.java.max=＋０").javaMaximum, + ).toEqual({ + status: "unresolved", + raw: "＋０", + }); + expect( + ghidraApplicationProperties("application.java.min=②").javaMinimum, + ).toEqual({ + status: "unresolved", + raw: "②", + }); + expect( + ghidraApplicationProperties("application.java.min=2147483648") + .javaMinimum, + ).toEqual({ status: "unresolved", raw: "2147483648" }); + }); + + it("keeps an unavailable properties file unresolved for Java admission", () => { + expect(ghidraJavaRequirement(undefined).unresolved).toBe( + "application.properties is unavailable", + ); + }); + + it("preserves property value trailing whitespace instead of normalizing it", () => { + const properties = ghidraApplicationProperties( + "application.version=12.1.4 \napplication.java.min=21 \n", + ); + expect(properties.providerVersion).toBe("12.1.4 "); + expect(properties.javaMinimum).toEqual({ + status: "unresolved", + raw: "21 ", + }); + expect(properties.parseError).toBeNull(); + }); +}); + +describe("Ghidra malformed property admission", () => { + it("fails installation and doctor admission closed on malformed Unicode escapes", () => { + const result = inspectGhidraInstallation( + { + environment: {}, + installDir: INSTALL, + platform: "linux", + architecture: "x64", + }, + host({ + readText: () => + "application.version=12.1.4\nbroken=\\u12G4\napplication.java.min=17\n", + }), + ); + expect(result.status).toBe("unavailable"); + expect(result.providerVersion).toBeNull(); + expect(result.checks).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + name: "version", + status: "failed", + code: "version_unresolved", + detail: expect.stringContaining("Malformed Unicode escape"), + }), + expect.objectContaining({ + name: "java", + status: "failed", + code: "version_unresolved", + remediation: expect.stringContaining( + "application.properties parse failed", + ), + }), + ]), + ); + expect(projectGhidraDoctorInspection(result)).toMatchObject({ + available: false, + registrationEnvironment: {}, + checks: expect.arrayContaining([ + expect.objectContaining({ name: "version", ok: false }), + expect.objectContaining({ name: "java", ok: false }), + ]), + }); + }); +}); + +describe("Ghidra installation version admission", () => { it.each([ { name: "JDK above the installation maximum", @@ -367,16 +532,23 @@ describe("Ghidra release line and JDK bounds", () => { failed: "java", code: "version_unresolved", }, + { + name: "missing Java minimum", + readText: () => "application.version=12.1.4\n", + probeJava: () => JAVA, + failed: "java", + code: "version_unresolved", + }, { name: "unparseable Ghidra version", - readText: () => "application.version=latest\n", + readText: () => "application.version=latest\napplication.java.min=21\n", probeJava: () => JAVA, failed: "version", code: "version_unresolved", }, { name: "different Ghidra line", - readText: () => "application.version=12.2.0\n", + readText: () => "application.version=12.2.0\napplication.java.min=21\n", probeJava: () => JAVA, failed: "version", code: "unsupported_version", @@ -422,7 +594,7 @@ describe("Ghidra release line and JDK bounds", () => { }); }); - it("honors an installation that declares a lower Java minimum", () => { + it("accepts a lower Java minimum declared by the installation", () => { expect( inspectGhidraInstallation( { diff --git a/src/ghidra/GhidraInstallation.ts b/src/ghidra/GhidraInstallation.ts index 713fe820f..c6ca5212d 100644 --- a/src/ghidra/GhidraInstallation.ts +++ b/src/ghidra/GhidraInstallation.ts @@ -641,14 +641,20 @@ const versionCheck = ( properties: GhidraApplicationProperties | undefined, ): GhidraInstallationCheck => { const raw = properties?.providerVersion; + const parseError = properties?.parseError; return installationCheck({ name: "version", - passed: ghidraReleaseAccepted(raw), + passed: parseError == null && ghidraReleaseAccepted(raw), code: - ghidraReleaseCompatibility(raw).status === "unsupported" - ? "unsupported_version" - : "version_unresolved", - detail: ghidraVersionDetail(raw), + parseError != null + ? "version_unresolved" + : ghidraReleaseCompatibility(raw).status === "unsupported" + ? "unsupported_version" + : "version_unresolved", + detail: + parseError == null + ? ghidraVersionDetail(raw) + : `application.properties could not be parsed: ${parseError}`, remediation: `Install a Ghidra ${ghidraReleaseLine()} release (verified with ${SUPPORTED_GHIDRA_VERSION}), or update REA for another Ghidra line.`, }); }; @@ -701,7 +707,9 @@ const javaRemediation = ( const systemGhidraInstallationHost = (): GhidraInstallationHost => ({ readText(path) { try { - return readFileSync(path, "utf8"); + // Ghidra loads this file with Properties.load(InputStream), which maps + // bytes as ISO-8859-1 rather than decoding UTF-8. + return readFileSync(path).toString("latin1"); } catch (cause: unknown) { // best-effort cleanup: optional install probing; unreadable means unknown. void cause; diff --git a/src/ghidra/GhidraInstallationPolicy.ts b/src/ghidra/GhidraInstallationPolicy.ts index bd19dc1d1..ead513cd8 100644 --- a/src/ghidra/GhidraInstallationPolicy.ts +++ b/src/ghidra/GhidraInstallationPolicy.ts @@ -1,10 +1,8 @@ +import { err, ok, type Result } from "../domain/result.js"; + /** Ghidra build whose bridge behavior this REA release verifies. */ export const SUPPORTED_GHIDRA_VERSION = "12.1.4"; -/** - * JDK major used when an installation omits `application.java.min`. - * Admission uses that installation's `application.java.min` and - * `application.java.max`. An empty maximum means the release sets no upper bound. - */ +/** Verified JDK major; admission uses the installation's declared bounds. */ export const SUPPORTED_GHIDRA_JAVA_MAJOR = 21; /** Numeric Ghidra release read from `application.version`. */ @@ -40,30 +38,49 @@ type GhidraJavaBound = | { readonly status: "unresolved"; readonly raw: string }; /** JDK majors this installation admits, after reading its own properties. */ -export interface GhidraJavaRequirement { - readonly minimumMajor: number; - readonly maximumMajor: number | null; - readonly unresolved: string | null; -} +export type GhidraJavaRequirement = + | { + readonly minimumMajor: number; + readonly maximumMajor: number | null; + readonly unresolved: null; + } + | { readonly unresolved: string }; /** Version and Java bounds read from one `application.properties` file. */ export interface GhidraApplicationProperties { readonly providerVersion: string | null; readonly javaMinimum: GhidraJavaBound; readonly javaMaximum: GhidraJavaBound; + /** A malformed Java properties escape invalidates the entire file. */ + readonly parseError: string | null; } const GHIDRA_RELEASE_PATTERN = /^(\d+)\.(\d+)(?:\.(\d+))?([-+].*)?$/u; -const JAVA_MAJOR_PATTERN = /^(\d+)(?:\.\d+)*$/u; /** Read the version and Java bounds Ghidra publishes for one installation. */ export const ghidraApplicationProperties = ( content: string, -): GhidraApplicationProperties => ({ - providerVersion: propertyValue(content, "application.version"), - javaMinimum: parseJavaBound(propertyValue(content, "application.java.min")), - javaMaximum: parseJavaBound(propertyValue(content, "application.java.max")), -}); +): GhidraApplicationProperties => { + const parsed = parseJavaProperties(content); + if (!parsed.ok) + return { + providerVersion: null, + javaMinimum: { status: "unresolved", raw: parsed.error }, + javaMaximum: { status: "unresolved", raw: parsed.error }, + parseError: parsed.error, + }; + const providerVersion = parsed.value.get("application.version"); + const minimum = parsed.value.get("application.java.min"); + const maximum = parsed.value.get("application.java.max"); + return { + providerVersion: providerVersion ?? null, + javaMinimum: + minimum === undefined ? { status: "absent" } : parseJavaBound(minimum), + // Ghidra uses an explicitly empty maximum to mean no upper bound. + javaMaximum: parseJavaMaximum(maximum), + parseError: null, + }; +}; /** Classify `application.version` against the verified release line. */ export const ghidraReleaseCompatibility = ( @@ -130,14 +147,21 @@ export const ghidraReleaseAccepted = ( export const ghidraJavaRequirement = ( properties: GhidraApplicationProperties | undefined, ): GhidraJavaRequirement => { - const minimum = properties?.javaMinimum ?? { status: "absent" }; - const maximum = properties?.javaMaximum ?? { status: "absent" }; + if (properties?.parseError !== null && properties?.parseError !== undefined) + return unresolvedJavaRequirement( + `application.properties parse failed: ${properties.parseError}`, + ); + if (properties === undefined) + return unresolvedJavaRequirement("application.properties is unavailable"); + const minimum = properties.javaMinimum; + const maximum = properties.javaMaximum; if (minimum.status === "unresolved") return unresolvedJavaRequirement(`application.java.min=${minimum.raw}`); + if (minimum.status === "absent") + return unresolvedJavaRequirement("application.java.min is missing"); if (maximum.status === "unresolved") return unresolvedJavaRequirement(`application.java.max=${maximum.raw}`); - const minimumMajor = - minimum.status === "major" ? minimum.major : SUPPORTED_GHIDRA_JAVA_MAJOR; + const minimumMajor = minimum.major; const maximumMajor = maximum.status === "major" ? maximum.major : null; if (maximumMajor !== null && maximumMajor < minimumMajor) return unresolvedJavaRequirement( @@ -187,14 +211,56 @@ const parseGhidraRelease = (raw: string): GhidraReleaseVersion | undefined => { return patch === undefined ? undefined : { major, minor, patch }; }; -const parseJavaBound = (raw: string | null): GhidraJavaBound => { - if (raw === null) return { status: "absent" }; - const major = integerGroup(JAVA_MAJOR_PATTERN.exec(raw)?.[1]); +const parseJavaBound = (raw: string): GhidraJavaBound => { + const major = parseJavaInteger(raw); return major === undefined ? { status: "unresolved", raw } : { status: "major", major }; }; +const parseJavaMaximum = (raw: string | undefined): GhidraJavaBound => { + if (raw === undefined || raw === "") return { status: "absent" }; + const bound = parseJavaBound(raw); + return bound.status === "major" && bound.major === 0 + ? { status: "absent" } + : bound; +}; + +const parseJavaInteger = (raw: string): number | undefined => { + let index = 0; + let sign = 1; + if (raw[index] === "+" || raw[index] === "-") { + if (raw[index] === "-") sign = -1; + index += 1; + } + if (index === raw.length) return undefined; + const limit = sign < 0 ? 2_147_483_648 : 2_147_483_647; + let value = 0; + for (; index < raw.length; index += 1) { + const character = raw[index]; + if (character === undefined) return undefined; + const digit = javaDecimalDigit(character); + if (digit === undefined) return undefined; + if (value > Math.floor((limit - digit) / 10)) return undefined; + value = value * 10 + digit; + } + return sign * value; +}; + +/** Match Integer.parseInt(charAt) for BMP Decimal_Number characters. */ +const javaDecimalDigit = (character: string): number | undefined => { + const code = character.charCodeAt(0); + if (code >= 48 && code <= 57) return code - 48; + if (!/^\p{Decimal_Number}$/u.test(character)) return undefined; + let blockStart = code; + while ( + blockStart > 0 && + /^\p{Decimal_Number}$/u.test(String.fromCharCode(blockStart - 1)) + ) + blockStart -= 1; + return (code - blockStart) % 10; +}; + const integerGroup = (value: string | undefined): number | undefined => { if (value === undefined) return undefined; const parsed = Number(value); @@ -204,17 +270,119 @@ const integerGroup = (value: string | undefined): number | undefined => { const unresolvedJavaRequirement = ( unresolved: string, ): GhidraJavaRequirement => ({ - minimumMajor: SUPPORTED_GHIDRA_JAVA_MAJOR, - maximumMajor: null, unresolved, }); -const propertyValue = (content: string, name: string): string | null => { - for (const line of content.split(/\r?\n/u)) { - const separator = line.indexOf("="); - if (separator < 0 || line.slice(0, separator).trim() !== name) continue; - const value = line.slice(separator + 1).trim(); - return value.length === 0 ? null : value; +/** Parse Java Properties.load(InputStream) syntax from its ISO-8859-1 text. */ +const parseJavaProperties = ( + content: string, +): Result, string> => { + const values = new Map(); + for (const line of logicalPropertyLines(content)) { + let start = 0; + while (isPropertyWhitespace(line[start])) start += 1; + if (start === line.length || line[start] === "#" || line[start] === "!") + continue; + let end = start; + let escaped = false; + while (end < line.length) { + const character = line[end]; + if ( + !escaped && + (character === "=" || + character === ":" || + isPropertyWhitespace(character)) + ) + break; + if (character === "\\") escaped = !escaped; + else escaped = false; + end += 1; + } + let valueStart = end; + while (isPropertyWhitespace(line[valueStart])) valueStart += 1; + if (line[valueStart] === "=" || line[valueStart] === ":") valueStart += 1; + while (isPropertyWhitespace(line[valueStart])) valueStart += 1; + const key = decodePropertyEscapes(line.slice(start, end)); + if (!key.ok) return err(key.error); + const value = decodePropertyEscapes(line.slice(valueStart)); + if (!value.ok) return err(value.error); + values.set(key.value, value.value); } - return null; + return ok(values); +}; + +const isPropertyWhitespace = (value: string | undefined): boolean => + value === " " || value === "\t" || value === "\f"; + +/** Join odd-backslash continuations while stripping leading continuation space. */ +function* logicalPropertyLines(content: string): Generator { + let physicalStart = 0; + let logical = ""; + let continuing = false; + while (physicalStart <= content.length) { + let physicalEnd = physicalStart; + while ( + physicalEnd < content.length && + content[physicalEnd] !== "\n" && + content[physicalEnd] !== "\r" + ) + physicalEnd += 1; + let line = content.slice(physicalStart, physicalEnd); + let leading = 0; + while (isPropertyWhitespace(line[leading])) leading += 1; + const comment = + !continuing && (line[leading] === "#" || line[leading] === "!"); + if (continuing) line = line.slice(leading); + let continued = false; + if (!comment) { + let slashes = 0; + for ( + let index = line.length - 1; + index >= 0 && line[index] === "\\"; + index -= 1 + ) + slashes += 1; + continued = slashes % 2 === 1; + logical += continued ? line.slice(0, -1) : line; + } + if (!continued) { + yield logical; + logical = ""; + } + continuing = continued; + if (physicalEnd === content.length) break; + physicalStart = + content[physicalEnd] === "\r" && content[physicalEnd + 1] === "\n" + ? physicalEnd + 2 + : physicalEnd + 1; + } + // Properties.load still consumes the accumulated line when EOF follows a + // continuation marker; the marker itself was already removed above. + if (continuing) yield logical; +} + +const decodePropertyEscapes = (value: string): Result => { + let result = ""; + for (let index = 0; index < value.length; index += 1) { + const character = value[index]; + if (character !== "\\") { + result += character; + continue; + } + const escaped = value[++index]; + if (escaped === undefined) + return err("Trailing escape in Java properties value"); + if (escaped === "t") result += "\t"; + else if (escaped === "n") result += "\n"; + else if (escaped === "r") result += "\r"; + else if (escaped === "f") result += "\f"; + else if (escaped === "u") { + const digits = value.slice(index + 1, index + 5); + if (digits.length !== 4 || !/^[0-9a-f]{4}$/iu.test(digits)) + return err("Malformed Unicode escape in Java properties file"); + result += String.fromCharCode(Number.parseInt(digits, 16)); + index += 4; + } else result += escaped; + } + return ok(result); }; diff --git a/src/ghidra/GhidraLanguageOverride.test.ts b/src/ghidra/GhidraLanguageOverride.test.ts index a355b39fa..5f90a1454 100644 --- a/src/ghidra/GhidraLanguageOverride.test.ts +++ b/src/ghidra/GhidraLanguageOverride.test.ts @@ -42,7 +42,7 @@ const X86_LDEFS = ` const host: GhidraInstallationHost = { platform: "linux", architecture: "x64", - readText: () => "application.version=12.1.4\n", + readText: () => "application.version=12.1.4\napplication.java.min=21\n", executable: () => true, probeJava: () => ({ version: "21.0.11", diff --git a/src/ghidra/GhidraMipsProfile.test.ts b/src/ghidra/GhidraMipsProfile.test.ts index 31a121cd1..ae72a3069 100644 --- a/src/ghidra/GhidraMipsProfile.test.ts +++ b/src/ghidra/GhidraMipsProfile.test.ts @@ -19,7 +19,7 @@ import { ghidraProfileLanguageOverride } from "./GhidraAnalysisProfile.js"; const host: GhidraInstallationHost = { platform: "linux", architecture: "x64", - readText: () => "application.version=12.1.4\n", + readText: () => "application.version=12.1.4\napplication.java.min=21\n", executable: () => true, probeJava: () => ({ version: "21.0.11", diff --git a/src/ghidra/GhidraProvider.behavior.test.ts b/src/ghidra/GhidraProvider.behavior.test.ts index 8b9c25461..3979222d9 100644 --- a/src/ghidra/GhidraProvider.behavior.test.ts +++ b/src/ghidra/GhidraProvider.behavior.test.ts @@ -38,7 +38,7 @@ const INSTALL = "/opt/ghidra_12.1.4_PUBLIC"; const installationHost = (): GhidraInstallationHost => ({ platform: "linux", architecture: "x64", - readText: () => "application.version=12.1.4\n", + readText: () => "application.version=12.1.4\napplication.java.min=21\n", executable: () => true, probeJava: () => ({ version: "21.0.11", diff --git a/src/ghidra/GhidraRiscvProfile.test.ts b/src/ghidra/GhidraRiscvProfile.test.ts index 9bbb65cf3..881ba0078 100644 --- a/src/ghidra/GhidraRiscvProfile.test.ts +++ b/src/ghidra/GhidraRiscvProfile.test.ts @@ -53,7 +53,7 @@ const RISCV_LDEFS = ` const host: GhidraInstallationHost = { platform: "linux", architecture: "x64", - readText: () => "application.version=12.1.4\n", + readText: () => "application.version=12.1.4\napplication.java.min=21\n", executable: () => true, probeJava: () => ({ version: "21.0.11", diff --git a/src/ghidra/GhidraSnapshotCache.test.ts b/src/ghidra/GhidraSnapshotCache.test.ts index 703bb4906..5185054e9 100644 --- a/src/ghidra/GhidraSnapshotCache.test.ts +++ b/src/ghidra/GhidraSnapshotCache.test.ts @@ -39,7 +39,7 @@ const fixture = (version = "12.1.4") => { { platform: "linux", architecture: "x64", - readText: () => `application.version=${version}\n`, + readText: () => `application.version=${version}\napplication.java.min=21\n`, executable: () => true, probeJava: () => ({ version: "21.0.11", From ec0b081371f118e6e5e042fbef3479359c672ddb Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:39:57 +0800 Subject: [PATCH 030/159] fix(filesystem): enforce symlink and decoded-text admission --- src/apktool/ApktoolProvider.test.ts | 48 +++++++++++++++- src/apktool/ApktoolProvider.ts | 21 ++++--- src/filesystem/RegularFile.ts | 12 +++- .../regularFileNoFollowFallback.test.ts | 55 +++++++++++++++++++ 4 files changed, 127 insertions(+), 9 deletions(-) create mode 100644 tests/process-global/regularFileNoFollowFallback.test.ts diff --git a/src/apktool/ApktoolProvider.test.ts b/src/apktool/ApktoolProvider.test.ts index 5b2a32114..b6ee249e5 100644 --- a/src/apktool/ApktoolProvider.test.ts +++ b/src/apktool/ApktoolProvider.test.ts @@ -96,6 +96,7 @@ const decodeStub = ( readonly withStrings?: boolean; readonly withGerman?: boolean; readonly failDecode?: boolean; + readonly ymlFile?: string; readonly manifestFile?: string; readonly stringsFile?: string; readonly stringsXml?: string; @@ -104,6 +105,10 @@ const decodeStub = ( const withStrings = options.withStrings ?? true; const withGerman = options.withGerman ?? true; const stringsXml = options.stringsXml ?? REAL_STRINGS; + const ymlWrite = + options.ymlFile === undefined + ? `printf '%s\\n' '${REAL_YML.replace(/'/gu, "'\\''")}' > "$5/apktool.yml"` + : `cp '${options.ymlFile}' "$5/apktool.yml"`; const manifestWrite = options.manifestFile === undefined ? `printf '%s\\n' '${REAL_MANIFEST.replace(/'/gu, "'\\''")}' > "$5/AndroidManifest.xml"` @@ -117,7 +122,7 @@ if [ "$1" = "d" ]; then ${ options.failDecode === true ? `printf 'Exception in thread "main" brut.androlib.AndrolibException: Could not decode the APK\\n' >&2; exit 1;` - : `printf '%s\\n' '${REAL_YML.replace(/'/gu, "'\\''")}' > "$5/apktool.yml" + : `${ymlWrite} ${manifestWrite} mkdir -p "$5/res/values"${withGerman ? ' "$5/res/values-de"' : ""} ${withStrings ? stringsWrite : ""} @@ -192,6 +197,47 @@ it("decodes resources with target identity, metadata, and strings", async () => expect(result.coverage).toBe("complete"); }); +it.each([ + { + name: "malformed UTF-8", + bytes: Buffer.concat([ + Buffer.from("versionInfo:\n versionName: com."), + Buffer.from([0xff]), + Buffer.from(".app\n"), + ]), + valid: false, + }, + { + name: "a literal replacement character", + bytes: Buffer.from("versionInfo:\n versionName: com.�.app\n", "utf8"), + valid: true, + }, +])("handles $name in producer metadata", async ({ bytes, valid }) => { + const { command, root } = await writeApktoolStub((root) => + decodeStub({ ymlFile: join(root, "malformed-apktool.yml") })(root), + ); + await writeFile(join(root, "malformed-apktool.yml"), bytes); + const apk = await apkFixture(root); + if (valid) { + const result = await evidenceResult(providerOver(command), { + operation: "decode_android_resources", + input: { path: apk.path, include_strings: false }, + }); + expect(result.metadata).toMatchObject({ version_name: "com.�.app" }); + expect(result.coverage).toBe("complete"); + return; + } + const result = await providerOver(command).execute({ + operation: "decode_android_resources", + input: { path: apk.path, include_strings: false }, + }); + if (result.ok) throw new Error("expected invalid producer output"); + const projected = projectAnalysisError(result.error); + expect(projected).toMatchObject({ code: "unreadable_output" }); + expect(result.error.message).toContain("invalid UTF-8"); + expect(result.error.message).toContain("apktool.yml"); +}); + it("preserves empty resource strings and reports markup omissions as partial", async () => { const stringsXml = ' A & B Hello world'; diff --git a/src/apktool/ApktoolProvider.ts b/src/apktool/ApktoolProvider.ts index 682e2f2ea..1e1f187c0 100644 --- a/src/apktool/ApktoolProvider.ts +++ b/src/apktool/ApktoolProvider.ts @@ -117,13 +117,20 @@ const readBoundedText = async ( throwIfCancelled(operation, signal); const truncated = length > budget; // A prefix may end within a UTF-8 character; omit those trailing bytes. - const decoder = new TextDecoder("utf-8"); - return { - text: decoder.decode(bytes.subarray(0, Math.min(length, budget)), { - stream: truncated, - }), - truncated, - }; + try { + const decoder = new TextDecoder("utf-8", { fatal: true }); + return { + text: decoder.decode(bytes.subarray(0, Math.min(length, budget)), { + stream: truncated, + }), + truncated, + }; + } catch (cause: unknown) { + throw new AnalysisProtocolError( + `apktool produced invalid UTF-8 in ${path}`, + { cause }, + ); + } } finally { await file.close(); } diff --git a/src/filesystem/RegularFile.ts b/src/filesystem/RegularFile.ts index a3037c728..4942e2e46 100644 --- a/src/filesystem/RegularFile.ts +++ b/src/filesystem/RegularFile.ts @@ -1,6 +1,6 @@ import { constants } from "node:fs"; import type { BigIntStats, Stats } from "node:fs"; -import { open, type FileHandle } from "node:fs/promises"; +import { lstat, open, type FileHandle } from "node:fs/promises"; import { OwnedFileHandle } from "./OwnedFileHandle.js"; @@ -74,6 +74,16 @@ export const openRegularFile = async ( options.signal?.throwIfAborted(); if (!metadata.isFile()) throw new NonRegularFileReadError(path, metadata.isDirectory()); + // O_NOFOLLOW is unavailable on Windows. Recheck the name only on hosts + // where open could have followed a symlink, while the descriptor is owned. + if (options.symlinks === "reject" && (constants.O_NOFOLLOW ?? 0) === 0) { + const named = await lstat(path); + options.signal?.throwIfAborted(); + if (named.isSymbolicLink()) + throw new NonRegularFileReadError(path, false); + if (!sameRegularFileState(metadata, named)) + throw new RegularFileChangedError(path); + } return handle; } catch (cause: unknown) { try { diff --git a/tests/process-global/regularFileNoFollowFallback.test.ts b/tests/process-global/regularFileNoFollowFallback.test.ts new file mode 100644 index 000000000..44ce66c7c --- /dev/null +++ b/tests/process-global/regularFileNoFollowFallback.test.ts @@ -0,0 +1,55 @@ +import { readFile, symlink } from "node:fs/promises"; +import { join } from "node:path"; + +import { expect, it, vi } from "vitest"; + +vi.mock("node:fs", async (importOriginal) => { + const original = await importOriginal(); + return { + ...original, + constants: { ...original.constants, O_NOFOLLOW: 0 }, + }; +}); + +import { openRegularFile } from "../../src/filesystem/RegularFile.js"; +import { ZipArtifactReader } from "../../src/artifacts/ZipArtifactReader.js"; +import { writeOrderedZip } from "../fixtures/artifactEntryOrder.js"; +import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js"; + +it("rejects a symlink when O_NOFOLLOW is unavailable and still follows when requested", async () => { + const root = await createTestTempDirectory("rea-no-nofollow-"); + const target = join(root, "target.zip"); + const link = join(root, "selected.zip"); + await writeOrderedZip(target, ["member.bin"]); + await symlink(target, link); + + const reader = new ZipArtifactReader(target, "zip"); + try { + const entries = []; + for await (const entry of reader.entries()) entries.push(entry); + expect(entries.map(({ path }) => path)).toEqual(["member.bin"]); + } finally { + await reader.close(); + } + + const linkedReader = new ZipArtifactReader(link, "zip"); + try { + await expect(async () => { + for await (const _entry of linkedReader.entries()) { + // Consume through the public reader boundary. + } + }).rejects.toMatchObject({ + name: "ArtifactReaderFailure", + reason: "format", + }); + } finally { + await linkedReader.close(); + } + + const followed = await openRegularFile(link, { symlinks: "follow" }); + try { + await expect(readFile(followed)).resolves.toEqual(await readFile(target)); + } finally { + await followed.close(); + } +}); From 3ae826ceba6f25b08bce50bcb51001209bc16b57 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:48:24 +0800 Subject: [PATCH 031/159] fix(javascript): share source-map resolution and preserve path uncertainty --- docs/javascript-application-workflows.md | 9 +- ...aScriptArtifactAnalysis.sourceMaps.test.ts | 71 ++++++ .../javascript/JavaScriptArtifactAnalysis.ts | 18 +- src/browser/WebSourceMapFetcher.ts | 8 +- src/domain/analysisView/analysisView.test.ts | 6 +- .../javascriptApplicationGraph.fixture.ts | 15 ++ .../javascriptApplicationGraph.test.ts | 7 +- .../javascript/javascriptSourceMapPaths.ts | 67 ++++-- .../sourceToBundleComparison.test.ts | 203 +++++++++++++++++- .../javascript/sourceToBundleComparison.ts | 6 +- .../sourceToBundleComparisonSchemas.ts | 2 + .../javascript/sourceToBundleSignals.ts | 39 +++- src/javascript/sourceMaps/DecodedSourceMap.ts | 25 --- .../filesystem/sourceToBundlePaths.test.ts | 4 +- .../mcp/applicationEvidenceMcp.test.ts | 134 ++++++++++++ .../process/sourceMapUncHosts.test.ts | 4 +- 16 files changed, 551 insertions(+), 67 deletions(-) create mode 100644 src/application/javascript/JavaScriptArtifactAnalysis.sourceMaps.test.ts diff --git a/docs/javascript-application-workflows.md b/docs/javascript-application-workflows.md index af0b6ba94..2d2d9e64a 100644 --- a/docs/javascript-application-workflows.md +++ b/docs/javascript-application-workflows.md @@ -156,9 +156,12 @@ to the comparison Evidence in a live session. `compare_source_to_bundle` compares one cryptographically committed `HistoricalSourceGraph` with authenticated application-graph Evidence. The stable scoring model reports every admitted signal and weight: exact source -digest, source-map original path, exact current path, path suffix, basename, and -language extension. Exact digest wins over location inference; weak basename -or extension evidence never forces a mapping. +digest, source-map original path, exact current path, path suffix, Unicode- +equivalent path, basename, and language extension. Exact digest wins over +location inference; weak basename, extension, and Unicode-equivalence evidence +never forces a mapping. Unicode-equivalent spellings are candidate evidence +only because distinct filesystem paths can preserve those spellings as separate +files; multiple canonical-equivalent candidates remain unknown. Each historical source file is classified as `unchanged`, `modified`, `removed`, `split`, `merged`, `duplicated`, or `unknown`. Multiple equally diff --git a/src/application/javascript/JavaScriptArtifactAnalysis.sourceMaps.test.ts b/src/application/javascript/JavaScriptArtifactAnalysis.sourceMaps.test.ts new file mode 100644 index 000000000..cbd9c745c --- /dev/null +++ b/src/application/javascript/JavaScriptArtifactAnalysis.sourceMaps.test.ts @@ -0,0 +1,71 @@ +import { expect, it } from "vitest"; +import { TraceMap } from "@jridgewell/trace-mapping"; + +import { analyzeJavaScriptArtifactFiles } from "./JavaScriptArtifactAnalysis.js"; + +const HASH = "a".repeat(64); + +it("retains batched TraceMap-resolved paths and null source positions", () => { + const sourceMapText = JSON.stringify({ + version: 3, + file: "main.js", + sourceRoot: "../src/", + sources: ["cafe\u0301.ts", null, "other.ts"], + sourcesContent: ["export const value = 1;", null, "export {};"], + names: [], + mappings: "", + }); + const analysis = analyzeJavaScriptArtifactFiles({ + files: [ + { + path: "dist/main.js.map", + container_sha256: HASH, + sha256: HASH, + bytes: Buffer.byteLength(sourceMapText), + inventory_artifact_id: "artifact_test", + kind: "source-map", + unpacked: false, + text: { included: true, value: sourceMapText }, + }, + ], + containers: [], + text_bytes_read: Buffer.byteLength(sourceMapText), + invalid_utf8_files: 0, + }); + const originals = analysis.source_maps[0]?.sources; + const original = originals?.[0]; + const producerPath = new TraceMap( + { + version: 3, + file: "main.js", + sourceRoot: "../src/", + sources: ["cafe\u0301.ts", "", "other.ts"], + names: [], + mappings: "", + }, + "file:///artifact/dist/main.js.map", + ).resolvedSources[0]; + + if (producerPath === undefined) + throw new Error("TraceMap should resolve its single source declaration"); + expect(originals?.[1]?.reference).toBeNull(); + expect(originals?.[2]?.reference?.resolution).toEqual({ + kind: "artifact-relative", + path: "src/other.ts", + }); + expect(original?.reference).toMatchObject({ + source_name: "cafe\u0301.ts", + source_root: "../src/", + map_path: "dist/main.js.map", + resolution: { + kind: "artifact-relative", + path: "src/cafe\u0301.ts", + }, + }); + expect(original?.reference?.resolution).toEqual({ + kind: "artifact-relative", + path: decodeURIComponent( + new URL(producerPath).pathname.slice("/artifact/".length), + ), + }); +}); diff --git a/src/application/javascript/JavaScriptArtifactAnalysis.ts b/src/application/javascript/JavaScriptArtifactAnalysis.ts index e52ba04dd..dd1a50488 100644 --- a/src/application/javascript/JavaScriptArtifactAnalysis.ts +++ b/src/application/javascript/JavaScriptArtifactAnalysis.ts @@ -3,7 +3,7 @@ import { createHash } from "node:crypto"; import { moduleKindForJavaScriptReference } from "./JavaScriptArtifactPathResolution.js"; import { setImmediate } from "node:timers/promises"; -import { resolveJavaScriptSourceMapReference } from "../../domain/javascript/javascriptSourceMapPaths.js"; +import { resolveJavaScriptSourceMapReferences } from "../../domain/javascript/javascriptSourceMapPaths.js"; import { analyzeParsedJavaScriptStaticSourceSteps } from "../../domain/javascript/javascriptStaticAnalysis.js"; import { analyzeParsedJavaScriptSemanticsSteps, @@ -561,21 +561,27 @@ const collectSourceMapOriginals = ( "Source map sourceRoot must be a string or null.", ); const root = typeof map.sourceRoot === "string" ? map.sourceRoot : null; - for (const [index, raw] of names.entries()) { + const sourceNames: (string | null)[] = []; + for (const raw of names) { if (raw !== null && typeof raw !== "string") return invalidSourceMap( file, "Source map contains a non-string source name.", ); + sourceNames.push(raw); + } + const references = resolveJavaScriptSourceMapReferences( + sourceNames, + root, + file.path, + ); + for (const [index, reference] of references.entries()) { const rawContent = Array.isArray(contents) ? contents[index] : undefined; const content = typeof rawContent === "string" ? rawContent : null; sources.push({ source_index: index, declaration_index: declarationIndex++, - reference: - typeof raw === "string" - ? resolveJavaScriptSourceMapReference(raw, root, file.path) - : null, + reference, content, content_sha256: content === null ? null : sha256(content), }); diff --git a/src/browser/WebSourceMapFetcher.ts b/src/browser/WebSourceMapFetcher.ts index 0b17df0cb..b950926ee 100644 --- a/src/browser/WebSourceMapFetcher.ts +++ b/src/browser/WebSourceMapFetcher.ts @@ -2,7 +2,6 @@ import * as t from "@babel/types"; import { decodeValidatedSourceMapLeaves, isBeforeSourceMapLeafStop, - resolveSourceMapSource, } from "../javascript/sourceMaps/DecodedSourceMap.js"; import { sanitizeBrowserUrl } from "../domain/browserObservation.js"; @@ -21,6 +20,7 @@ import { webSourceMapsSchema } from "../domain/webBundleAnalysis.js"; import { createWebTextArtifact } from "../domain/webContentArtifact.js"; import { safeParseJson } from "../domain/safeJson.js"; import { sourceMapJsonText } from "../domain/sourceMapEnvelope.js"; +import { resolveJavaScriptSourceMapUrl } from "../domain/javascript/javascriptSourceMapPaths.js"; import { jsonParts } from "../domain/jsonSerialization.js"; import { inspectSourceMapValue, @@ -867,7 +867,11 @@ const preflightResolvedSourceBytes = ( const sourceKey = source; let resolved = sourcesForRoot.get(sourceKey); if (resolved === undefined) - resolved = resolveSourceMapSource(source, map.sourceRoot, mapUrl); + resolved = resolveJavaScriptSourceMapUrl( + source, + map.sourceRoot, + mapUrl, + ); const bytes = Buffer.byteLength(resolved, "utf8"); if (bytes > SOURCE_MAP_RESPONSE_BYTES - resolvedBytes) throw resolvedSourceLimitFailure(); diff --git a/src/domain/analysisView/analysisView.test.ts b/src/domain/analysisView/analysisView.test.ts index 631f9d506..f1aa3e829 100644 --- a/src/domain/analysisView/analysisView.test.ts +++ b/src/domain/analysisView/analysisView.test.ts @@ -1,6 +1,5 @@ import { expect, it } from "vitest"; -import { resolveJavaScriptSourceMapReference } from "../javascript/javascriptSourceMapPaths.js"; import { functionDossierSchema } from "../hopperValues.js"; import { ghidraFunctionDossier } from "../ghidraValues.fixture.js"; import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js"; @@ -24,13 +23,14 @@ import { completeWithinViewCoverage, pageViewCoverage, } from "./analysisViewCoverage.js"; +import { sourceMapReference } from "../javascript/javascriptApplicationGraph.fixture.js"; import { javascriptModulePath } from "./javascriptApplicationView.js"; - import { inspectAnalysisViewInputSchema, projectAnalysisView, type AnalysisViewParent, } from "./analysisView.js"; + const layoutParent = (): AnalysisViewParent => { const evidence = analysisViewLayoutEvidence(); return { @@ -532,7 +532,7 @@ it("pages actual modules, including unknown and empty paths, without unrelated r observations: original.observations.map((observation) => ({ ...(kind === "source-module" ? { - source_map_reference: resolveJavaScriptSourceMapReference( + source_map_reference: sourceMapReference( "", null, "renderer.js", diff --git a/src/domain/javascript/javascriptApplicationGraph.fixture.ts b/src/domain/javascript/javascriptApplicationGraph.fixture.ts index 3002b822f..0e78e5115 100644 --- a/src/domain/javascript/javascriptApplicationGraph.fixture.ts +++ b/src/domain/javascript/javascriptApplicationGraph.fixture.ts @@ -6,6 +6,7 @@ import { } from "./javascriptApplicationGraph.js"; import type { ApplicationNode } from "./javascriptApplicationGraphSchemas.js"; import type { ApplicationGraphEvidence } from "./javascriptApplicationEvidenceSchemas.js"; +import { resolveJavaScriptSourceMapReferences } from "./javascriptSourceMapPaths.js"; export const APPLICATION_GRAPH_DIGESTS = { package: "1".repeat(64), @@ -22,6 +23,20 @@ export const completeCoverage = { limits: [], } satisfies ApplicationGraphEvidence["coverage"]; +export const sourceMapReference = ( + sourceName: string, + sourceRoot: string | null, + mapPath: string, +) => { + const reference = resolveJavaScriptSourceMapReferences( + [sourceName], + sourceRoot, + mapPath, + )[0]; + if (reference == null) throw new TypeError("Expected a source-map reference"); + return reference; +}; + const extractor = (operation: string) => ({ name: "rea-test-fixture", version: "1.0.0", diff --git a/src/domain/javascript/javascriptApplicationGraph.test.ts b/src/domain/javascript/javascriptApplicationGraph.test.ts index 5e396a545..5bf917480 100644 --- a/src/domain/javascript/javascriptApplicationGraph.test.ts +++ b/src/domain/javascript/javascriptApplicationGraph.test.ts @@ -20,10 +20,9 @@ import { inferredArtifactEvidence, nodeByLabel, runtimeEvidence, + sourceMapReference, unknownEvidence, } from "./javascriptApplicationGraph.fixture.js"; - -import { resolveJavaScriptSourceMapReference } from "./javascriptSourceMapPaths.js"; const createLargeNativeExportGraph = (): JavaScriptApplicationGraph => { const properties = Object.fromEntries( Array.from({ length: 1_001 }, (_, index) => [ @@ -438,7 +437,7 @@ describe("JavaScript Application Graph: source maps and observations", () => { observations: [ { label: "src/editor.ts", - source_map_reference: resolveJavaScriptSourceMapReference( + source_map_reference: sourceMapReference( "webpack:///src/editor.ts", null, "dist/app.js.map", @@ -535,7 +534,7 @@ describe("JavaScript Application Graph: source-map digest consistency", () => { observations: [ { label: null, - source_map_reference: resolveJavaScriptSourceMapReference( + source_map_reference: sourceMapReference( "src/editor.ts", null, "dist/app.js.map", diff --git a/src/domain/javascript/javascriptSourceMapPaths.ts b/src/domain/javascript/javascriptSourceMapPaths.ts index d897bdd26..06b5f8265 100644 --- a/src/domain/javascript/javascriptSourceMapPaths.ts +++ b/src/domain/javascript/javascriptSourceMapPaths.ts @@ -1,5 +1,6 @@ import { posix } from "node:path"; import { domainToASCII, fileURLToPath, pathToFileURL } from "node:url"; +import { TraceMap } from "@jridgewell/trace-mapping"; import { z } from "zod"; @@ -25,22 +26,64 @@ export type JavaScriptSourceMapReference = z.infer< typeof javascriptSourceMapReferenceSchema >; -/** Interpret a declaration once at the source-map ingestion boundary. */ -export const resolveJavaScriptSourceMapReference = ( - sourceName: string, +/** Interpret declarations together once at their source-map ingestion boundary. */ +export const resolveJavaScriptSourceMapReferences = ( + sourceNames: readonly (string | null)[], sourceRoot: string | null, mapPath: string, -): JavaScriptSourceMapReference => { - // TraceMap's producer prefixes sourceRoot before resolving the URL reference. - const reference = `${sourceRoot ? `${sourceRoot}/` : ""}${sourceName}`; - return { - source_name: sourceName, - source_root: sourceRoot, - map_path: mapPath, - resolution: resolveJavaScriptSourceMapPath(reference, mapPath), - }; +): (JavaScriptSourceMapReference | null)[] => { + if (sourceNames.length === 0) return []; + const resolvedSources = resolveSourceMapUrls( + sourceNames.map((sourceName) => sourceName ?? ""), + sourceRoot, + "", + ); + return sourceNames.map((sourceName, index) => { + if (sourceName === null) return null; + const reference = resolvedSources[index]; + if (reference === undefined) + throw new TypeError( + `TraceMap did not resolve source declaration ${String(index)}`, + ); + return { + source_name: sourceName, + source_root: sourceRoot, + map_path: mapPath, + resolution: resolveJavaScriptSourceMapPath(reference, mapPath), + }; + }); }; +/** Resolve a source declaration using the same pinned format implementation as map consumers. */ +export const resolveJavaScriptSourceMapUrl = ( + source: string | null, + sourceRoot: string | null | undefined, + mapUrl: string, +): string => { + const resolved = resolveSourceMapUrls([source ?? ""], sourceRoot, mapUrl)[0]; + if (resolved === undefined) + throw new TypeError( + "TraceMap did not resolve its single source declaration", + ); + return resolved; +}; + +const resolveSourceMapUrls = ( + sources: string[], + sourceRoot: string | null | undefined, + mapUrl: string, +): readonly string[] => + new TraceMap( + { + version: 3, + names: [], + sources, + ...(sourceRoot == null ? {} : { sourceRoot }), + mappings: "", + }, + mapUrl, + ).resolvedSources; + /** Resolve a source-map URL reference without changing its recorded identity. */ const resolveJavaScriptSourceMapPath = ( source: string, diff --git a/src/domain/javascript/sourceToBundleComparison.test.ts b/src/domain/javascript/sourceToBundleComparison.test.ts index f488ee598..d231cb5ec 100644 --- a/src/domain/javascript/sourceToBundleComparison.test.ts +++ b/src/domain/javascript/sourceToBundleComparison.test.ts @@ -1,10 +1,11 @@ import { describe, expect, it } from "vitest"; +import { TraceMap } from "@jridgewell/trace-mapping"; -import { resolveJavaScriptSourceMapReference } from "./javascriptSourceMapPaths.js"; import { createJavaScriptApplicationGraph, createJavaScriptApplicationNode, } from "./javascriptApplicationGraph.js"; +import type { ApplicationNode } from "./javascriptApplicationGraphSchemas.js"; import { createHistoricalSourceGraph } from "../referenceSourceGraph.js"; import { compareUnicodeCodePoints } from "../unicodeCodePointOrder.js"; import { compareSourceToBundle } from "./sourceToBundleComparison.js"; @@ -12,7 +13,10 @@ import { SOURCE_TO_BUNDLE_SIGNAL_WEIGHTS, sourceToBundleComparisonResultSchema, } from "./sourceToBundleComparisonSchemas.js"; -import { artifactEvidence } from "./javascriptApplicationGraph.fixture.js"; +import { + artifactEvidence, + sourceMapReference, +} from "./javascriptApplicationGraph.fixture.js"; const HASH = { artifact: "1".repeat(64), @@ -42,9 +46,6 @@ describe("historical source to bundle comparison", () => { const second = compareSourceToBundle(input); expect(first).toEqual(second); - expect(() => - sourceToBundleComparisonResultSchema.parse(first), - ).not.toThrow(); expect(first.summary).toEqual({ unchanged: 1, modified: 0, @@ -153,7 +154,7 @@ describe("source to bundle digest changes", () => { observations: [ { label: "display label", - source_map_reference: resolveJavaScriptSourceMapReference( + source_map_reference: sourceMapReference( "webpack://fixture/./src/modified.ts", null, "dist/main.js.map", @@ -202,6 +203,90 @@ describe("source to bundle digest changes", () => { }); }); +describe("source to bundle Unicode path collisions", () => { + it("keeps colliding canonical-equivalent current paths ambiguous", () => { + const historicalPath = "src/a\u0301\u0323.ts"; + const currentPaths = ["src/a\u0323\u0301.ts", "src/\u1ea1\u0301.ts"]; + const nodes = currentPaths.map((path, index) => + moduleNode(`unicode-${index}`, { path }), + ); + const result = comparePathToNodes(historicalPath, nodes); + const mapping = item(result, historicalPath); + + expect(mapping).toMatchObject({ + status: "unknown", + confidence: "unknown", + current_node_ids: [], + candidates: [ + expect.objectContaining({ + signals: expect.arrayContaining([ + expect.objectContaining({ kind: "unicode-equivalent-path" }), + ]), + }), + expect.objectContaining({ + signals: expect.arrayContaining([ + expect.objectContaining({ kind: "unicode-equivalent-path" }), + ]), + }), + ], + limitations: [ + expect.stringContaining( + "filesystem paths with distinct Unicode spellings", + ), + ], + }); + }); + + it("prefers a raw source-map path over canonically equivalent alternatives", () => { + const historicalPath = "src/cafe\u0301.ts"; + const exact = sourceMapNode( + "exact", + "cafe\u0301.ts", + "../src/", + HASH.modified, + ); + const equivalent = sourceMapNode( + "equivalent", + "caf\u00e9.ts", + "../src/", + HASH.removed, + ); + const result = comparePathToNodes(historicalPath, [exact, equivalent]); + + expect(item(result, historicalPath)).toMatchObject({ + status: "modified", + current_node_ids: [exact.node_id], + }); + }); + + it("finds Unicode-equivalent suffixes of absolute source-map URLs", () => { + const historicalPath = "src/caf\u00e9.ts"; + const node = sourceMapNode( + "remote", + "cafe\u0301.ts", + "https://cdn.example/src/", + HASH.modified, + ); + const result = comparePathToNodes(historicalPath, [node]); + + expect(item(result, historicalPath)).toMatchObject({ + status: "unknown", + current_node_ids: [], + candidates: [ + expect.objectContaining({ + current_node_id: node.node_id, + signals: expect.arrayContaining([ + expect.objectContaining({ + kind: "unicode-equivalent-path", + current_values: ["cdn.example/src/cafe\u0301.ts"], + }), + ]), + }), + ], + }); + }); +}); + describe("source to bundle path syntax", () => { it.each([ "src/modified#part.ts", @@ -282,9 +367,60 @@ describe("source to bundle path syntax", () => { }), ], }); + expect( + item(result, "src/modified.ts").candidates.flatMap( + ({ signals }) => signals, + ), + ).not.toContainEqual( + expect.objectContaining({ kind: "unicode-equivalent-path" }), + ); }); }); +describe("source-map resolver equivalence", () => { + it.each([ + { + sourceRoot: "https://cdn.example/src/", + expectedSource: "https://cdn.example/src/a.js", + expectedResolution: { kind: "suffix", path: "cdn.example/src/a.js" }, + }, + { + sourceRoot: "https://cdn.example/src//", + expectedSource: "https://cdn.example/src/a.js", + expectedResolution: { kind: "suffix", path: "cdn.example/src/a.js" }, + }, + { + sourceRoot: "webpack://", + expectedSource: "webpack:///a.js", + expectedResolution: { kind: "suffix", path: "a.js" }, + }, + ])( + "matches TraceMap for sourceRoot $sourceRoot", + ({ sourceRoot, expectedSource, expectedResolution }) => { + const producerSource = new TraceMap( + { + version: 3, + file: "main.js", + sourceRoot, + sources: ["a.js"], + names: [], + mappings: "", + }, + "file:///artifact/dist/main.js.map", + ).resolvedSources[0]; + const reference = sourceMapReference( + "a.js", + sourceRoot, + "dist/main.js.map", + ); + + expect(producerSource).toBe(expectedSource); + expect(reference.source_root).toBe(sourceRoot); + expect(reference.resolution).toEqual(expectedResolution); + }, + ); +}); + describe("source to bundle complete metadata", () => { it("accepts long source metadata and all reported limitations", () => { const result = compareSourceToBundle({ @@ -328,6 +464,61 @@ describe("source to bundle complete metadata", () => { }); }); +const sourceMapNode = ( + key: string, + sourceName: string, + sourceRoot: string, + digest: string, +) => + createJavaScriptApplicationNode({ + kind: "source-module", + identity: { + strategy: "source-map-original", + source_root: sourceRoot, + stability: "source-map-exact", + source_map_sha256: HASH.artifact, + original_source: sourceName, + source_sha256: digest, + }, + observations: [ + { + label: `${key} source-map spelling`, + source_map_reference: sourceMapReference( + sourceName, + sourceRoot, + `dist/${key}.map`, + ), + properties: { content_available: true, content_sha256: digest }, + evidence: artifactEvidence(HASH.artifact, `dist/${key}.map`), + }, + ], + }); + +const comparePathToNodes = ( + sourcePath: string, + nodes: readonly ApplicationNode[], +) => + compareSourceToBundle({ + reference: historicalGraph("complete", [sourcePath]), + application: { + evidenceId: EVIDENCE_ID, + rootArtifactSha256: HASH.artifact, + graph: createJavaScriptApplicationGraph({ + schema: "JavaScriptApplicationGraph", + root_node_ids: nodes.map(({ node_id: nodeId }) => nodeId), + nodes, + edges: [], + coverage: { + status: "complete", + truncated: false, + omitted_count: 0, + limits: [], + }, + limitations: [], + }), + }, + }); + const historicalGraph = ( inventoryState: "complete" | "partial", paths = [ diff --git a/src/domain/javascript/sourceToBundleComparison.ts b/src/domain/javascript/sourceToBundleComparison.ts index ac0007cb4..5a70c57f3 100644 --- a/src/domain/javascript/sourceToBundleComparison.ts +++ b/src/domain/javascript/sourceToBundleComparison.ts @@ -258,7 +258,11 @@ const classifyMapping = ( confidence: "unknown", currentNodeIds: [], limitations: [ - "Only weak basename or extension signals were available; no mapping was forced.", + candidates.some((candidate) => + hasSignal(candidate, "unicode-equivalent-path"), + ) + ? "Unicode-equivalent paths are inferred candidates only; filesystem paths with distinct Unicode spellings may identify different files, so no mapping was forced." + : "Only weak basename or extension signals were available; no mapping was forced.", ], }; return absenceComplete diff --git a/src/domain/javascript/sourceToBundleComparisonSchemas.ts b/src/domain/javascript/sourceToBundleComparisonSchemas.ts index 2e27cadb3..9d6bd6b2f 100644 --- a/src/domain/javascript/sourceToBundleComparisonSchemas.ts +++ b/src/domain/javascript/sourceToBundleComparisonSchemas.ts @@ -15,6 +15,7 @@ export const SOURCE_TO_BUNDLE_SIGNAL_WEIGHTS = [ ["source-map-original-path", 70], ["current-path-exact", 60], ["current-path-suffix", 40], + ["unicode-equivalent-path", 20], ["basename-match", 20], ["language-extension", 5], ] as const; @@ -24,6 +25,7 @@ const sourceToBundleSignalKindSchema = z.enum([ "source-map-original-path", "current-path-exact", "current-path-suffix", + "unicode-equivalent-path", "basename-match", "language-extension", ]); diff --git a/src/domain/javascript/sourceToBundleSignals.ts b/src/domain/javascript/sourceToBundleSignals.ts index 2190988fe..b3546aecc 100644 --- a/src/domain/javascript/sourceToBundleSignals.ts +++ b/src/domain/javascript/sourceToBundleSignals.ts @@ -102,8 +102,12 @@ export const buildSourceToBundleCandidateIndex = ( for (const path of projection.paths) { for (const suffix of path.allowSuffix ? pathSuffixes(path.value) - : [path.value]) + : [path.value]) { addIndexValue(byPathSuffix, suffix, projection.node.node_id); + const canonical = suffix.normalize("NFC"); + if (canonical !== suffix) + addIndexValue(byPathSuffix, canonical, projection.node.node_id); + } addIndexValue( byBasename, posix.basename(path.value), @@ -126,9 +130,13 @@ export const candidateIdsForSource = ( source: SourceFile, index: SourceToBundleCandidateIndex, ): string[] => { + const canonicalPath = source.path.normalize("NFC"); const groups = [ source.sha256 === null ? undefined : index.byDigest.get(source.sha256), index.byPathSuffix.get(source.path), + canonicalPath === source.path + ? undefined + : index.byPathSuffix.get(canonicalPath), index.byBasename.get(posix.basename(source.path)), ]; const seen = new Set(); @@ -220,6 +228,14 @@ const candidateSignals = ( signals.push( signal("source-map-original-path", source.path, sourceMapPaths), ); + const unicodeEquivalentPaths = matchingUnicodePaths( + source.path, + current.paths, + ); + if (unicodeEquivalentPaths.length > 0) + signals.push( + signal("unicode-equivalent-path", source.path, unicodeEquivalentPaths), + ); const otherPaths = current.paths.filter( ({ kind }) => kind !== "source-map-original", ); @@ -286,6 +302,27 @@ const matchingPaths = ( ) .map(({ value }) => value); +const matchingUnicodePaths = ( + sourcePath: string, + currentPaths: readonly CurrentPath[], +): string[] => { + const canonicalSource = sourcePath.normalize("NFC"); + return currentPaths + .filter(({ value, allowSuffix }) => { + if ( + value === sourcePath || + (allowSuffix && value.endsWith(`/${sourcePath}`)) + ) + return false; + const canonicalValue = value.normalize("NFC"); + return ( + canonicalValue === canonicalSource || + (allowSuffix && canonicalValue.endsWith(`/${canonicalSource}`)) + ); + }) + .map(({ value }) => value); +}; + const addJsonPath = (paths: CurrentPath[], value: unknown): void => { if (typeof value === "string") addPath(paths, "observation-path", value); }; diff --git a/src/javascript/sourceMaps/DecodedSourceMap.ts b/src/javascript/sourceMaps/DecodedSourceMap.ts index 60bfcfc0e..e9ed304e1 100644 --- a/src/javascript/sourceMaps/DecodedSourceMap.ts +++ b/src/javascript/sourceMaps/DecodedSourceMap.ts @@ -19,31 +19,6 @@ export const isBeforeSourceMapLeafStop = ( ); }; -/** Resolve one declaration with the same pinned upstream implementation used by full maps. */ -export const resolveSourceMapSource = ( - source: string | null, - sourceRoot: string | null | undefined, - mapUrl: string, -): string => { - const map = new TraceMap( - { - version: 3, - names: [], - sources: [source ?? ""], - ...(sourceRoot == null ? {} : { sourceRoot }), - mappings: "", - }, - mapUrl, - ); - const resolved = map.resolvedSources[0]; - if (resolved === undefined) - throw new SourceMapFormatFailure( - "format", - "Source-map resolver did not return the requested source identity.", - ); - return resolved; -}; - /** Decode leaf mappings and validate producer indices before any consumer publishes them. */ export const decodeValidatedSourceMapLeaves = ( leaves: readonly SourceMapLeaf[], diff --git a/tests/boundary/filesystem/sourceToBundlePaths.test.ts b/tests/boundary/filesystem/sourceToBundlePaths.test.ts index 19aa07a1a..8bc9caa27 100644 --- a/tests/boundary/filesystem/sourceToBundlePaths.test.ts +++ b/tests/boundary/filesystem/sourceToBundlePaths.test.ts @@ -6,7 +6,7 @@ import { analyzeJavaScriptApplication } from "../../support/javascriptApplicatio import { parseApplicationGraphEvidence } from "../../../src/application/javascript/JavaScriptApplicationEvidenceGraph.js"; import { findApplicationFeatureSeeds } from "../../../src/domain/javascript/javascriptFeatureSeed.js"; import { compareSourceToBundle } from "../../../src/domain/javascript/sourceToBundleComparison.js"; -import { resolveSourceMapSource } from "../../../src/javascript/sourceMaps/DecodedSourceMap.js"; +import { resolveJavaScriptSourceMapUrl } from "../../../src/domain/javascript/javascriptSourceMapPaths.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { importReferenceSource } from "../../support/referenceSourceResourceScope.js"; @@ -180,7 +180,7 @@ it.each([ decoy: decoyPath = "src/a.js", }) => { if (sourceRoot !== undefined) { - const resolved = resolveSourceMapSource( + const resolved = resolveJavaScriptSourceMapUrl( source, sourceRoot, `https://example.test/${mapPath}`, diff --git a/tests/boundary/mcp/applicationEvidenceMcp.test.ts b/tests/boundary/mcp/applicationEvidenceMcp.test.ts index 16dfc142c..ba71b7921 100644 --- a/tests/boundary/mcp/applicationEvidenceMcp.test.ts +++ b/tests/boundary/mcp/applicationEvidenceMcp.test.ts @@ -2,6 +2,7 @@ import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; import type { Client } from "@modelcontextprotocol/client"; import { describe, expect, it } from "vitest"; import { Ajv2020 } from "ajv/dist/2020.js"; +import type { AnySchema } from "ajv"; import type { BinarySession } from "../../../src/application/binary/BinarySession.js"; import { @@ -10,12 +11,143 @@ import { SOURCE_TO_BUNDLE_COMPARISON_EXAMPLE, } from "../../../src/contracts/javascript/javascriptApplicationWorkflowExamples.js"; import { createEvidence } from "../../../src/domain/evidence.js"; +import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js"; +import { + createJavaScriptApplicationGraph, + createJavaScriptApplicationNode, +} from "../../../src/domain/javascript/javascriptApplicationGraph.js"; +import { createJavaScriptSemanticGraph } from "../../../src/domain/javascript/javascriptSemanticGraph.js"; +import { + artifactEvidence, + sourceMapReference, +} from "../../../src/domain/javascript/javascriptApplicationGraph.fixture.js"; import { createApplicationMcpHarness, type ApplicationMcpHarness, } from "../../fixtures/applicationMcpHarness.js"; import { createHistoricalSourceGraph } from "../../../src/domain/referenceSourceGraph.js"; +async function runUnicodeSourceMapComparison( + harness: ApplicationMcpHarness, +): Promise { + const baseApplication = SOURCE_TO_BUNDLE_COMPARISON_EXAMPLE.application; + const subject = baseApplication.subject; + if (subject === null) throw new Error("Application Evidence has no subject"); + const baseResult = javascriptApplicationAnalysisResultSchema.parse( + baseApplication.normalized_result, + ); + const baseGraph = baseResult.graph; + const decomposedSource = "cafe\u0301.ts"; + const unicodeSourceNode = createJavaScriptApplicationNode({ + kind: "source-module", + identity: { + strategy: "source-map-original", + source_root: "../src/", + stability: "source-map-exact", + source_map_sha256: "1".repeat(64), + original_source: decomposedSource, + source_sha256: "3".repeat(64), + }, + observations: [ + { + label: "decomposed source-map spelling", + source_map_reference: sourceMapReference( + decomposedSource, + "../src/", + "dist/main.js.map", + ), + properties: { content_available: true, content_sha256: "3".repeat(64) }, + evidence: artifactEvidence("1".repeat(64), "dist/main.js.map"), + }, + ], + }); + const applicationGraph = createJavaScriptApplicationGraph({ + schema: "JavaScriptApplicationGraph", + root_node_ids: [...baseGraph.root_node_ids, unicodeSourceNode.node_id], + nodes: [...baseGraph.nodes, unicodeSourceNode], + edges: baseGraph.edges, + coverage: baseGraph.coverage, + limitations: baseGraph.limitations, + }); + const semantic = baseResult.semantic_graph; + const { graph_id: _semanticGraphId, ...semanticInput } = semantic; + const semanticGraph = createJavaScriptSemanticGraph({ + ...semanticInput, + application_graph_id: applicationGraph.graph_id, + }); + const unicodeApplication = createEvidence( + { + path: baseResult.input_path, + sha256: subject.digest.sha256, + format: subject.format, + }, + baseApplication.provider, + { + predicateType: baseApplication.predicate_type, + operation: baseApplication.operation, + parameters: baseApplication.parameters, + result: { + ...baseResult, + graph: applicationGraph, + semantic_graph: semanticGraph, + }, + confidence: baseApplication.confidence, + authority: baseApplication.authority, + limitations: baseApplication.limitations, + evidenceLinks: baseApplication.evidence_links, + }, + ); + const historicalUnicodePath = "src/caf\u00e9.ts"; + const unicodeReference = createHistoricalSourceGraph({ + ...SOURCE_TO_BUNDLE_COMPARISON_EXAMPLE.reference, + root_sha256: undefined, + entries: SOURCE_TO_BUNDLE_COMPARISON_EXAMPLE.reference.entries.map( + (entry) => + entry.path === "src/main.ts" + ? { ...entry, path: historicalUnicodePath } + : entry, + ), + }); + const reply = await harness.client.callTool({ + name: "compare_source_to_bundle", + arguments: { reference: unicodeReference, application: unicodeApplication }, + }); + expect(reply.isError).not.toBe(true); + expect(reply.structuredContent).toMatchObject({ + normalized_result: { + items: [ + { + source_path: historicalUnicodePath, + status: "unknown", + current_node_ids: [], + candidates: [ + expect.objectContaining({ + confidence: "low", + signals: expect.arrayContaining([ + expect.objectContaining({ + kind: "unicode-equivalent-path", + source_value: historicalUnicodePath, + current_values: ["src/cafe\u0301.ts"], + }), + ]), + }), + ], + }, + ], + }, + }); + const advertised = (await harness.client.listTools()).tools.find( + ({ name }) => name === "compare_source_to_bundle", + ); + if (advertised?.outputSchema === undefined) + throw new Error("Comparison tool output schema is missing"); + const ajv = new Ajv2020({ strict: false, validateFormats: false }); + const outputSchema = advertised.outputSchema as AnySchema; + expect(ajv.validateSchema(outputSchema)).toBe(true); + const validateOutput = ajv.compile(outputSchema); + expect(validateOutput(reply.structuredContent)).toBe(true); +} + async function runInlineWorkflowScenarios( harness: ApplicationMcpHarness, ): Promise { @@ -92,6 +224,8 @@ async function runInlineWorkflowScenarios( }, }); + await runUnicodeSourceMapComparison(harness); + const referenceWithExclusion = createHistoricalSourceGraph({ ...SOURCE_TO_BUNDLE_COMPARISON_EXAMPLE.reference, root_sha256: undefined, diff --git a/tests/boundary/process/sourceMapUncHosts.test.ts b/tests/boundary/process/sourceMapUncHosts.test.ts index 293a3f71a..99953509f 100644 --- a/tests/boundary/process/sourceMapUncHosts.test.ts +++ b/tests/boundary/process/sourceMapUncHosts.test.ts @@ -73,9 +73,9 @@ processTest.for(cases)( [ "--input-type=module", "--eval", - `import { resolveJavaScriptSourceMapReference } from "./dist/domain/javascript/javascriptSourceMapPaths.js"; + `import { resolveJavaScriptSourceMapReferences } from "./dist/domain/javascript/javascriptSourceMapPaths.js"; const [source, root] = JSON.parse(process.argv[1]); - console.log(JSON.stringify(resolveJavaScriptSourceMapReference(source, root, "maps/main.js.map")));`, + console.log(JSON.stringify(resolveJavaScriptSourceMapReferences([source], root, "maps/main.js.map")[0]));`, JSON.stringify([scenario.source, sourceRoot]), ], { timeoutMs: 10_000 }, From 91886982fccc50a9f7f31d4f320b25ffbcff35af Mon Sep 17 00:00:00 2001 From: Ysopwnable <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:58:56 +0800 Subject: [PATCH 032/159] =?UTF-8?q?docs:=20celebrate=20170,000=20GitHub=20?= =?UTF-8?q?stars=20=F0=9F=8E=89=20(#2462)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 2 +- README_ar.md | 2 +- README_de.md | 2 +- README_es.md | 2 +- README_fa.md | 2 +- README_fr.md | 2 +- README_id.md | 2 +- README_it.md | 2 +- README_ja.md | 2 +- README_ko.md | 2 +- README_pl.md | 2 +- README_pt-BR.md | 2 +- README_ru.md | 2 +- README_th.md | 2 +- README_tr.md | 2 +- README_uk.md | 2 +- README_vi.md | 2 +- README_zh-TW.md | 2 +- README_zh.md | 2 +- 19 files changed, 19 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index 6b78543ef..c4a89bcee 100644 --- a/README.md +++ b/README.md @@ -322,7 +322,7 @@ Report vulnerabilities through [SECURITY.md](SECURITY.md). ## Star history -🎉 **110,000 GitHub stars — thank you!** +🎉 **170,000 GitHub stars — thank you!** Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes. diff --git a/README_ar.md b/README_ar.md index 4c33ac544..1c0bc367e 100644 --- a/README_ar.md +++ b/README_ar.md @@ -264,7 +264,7 @@ npx rea-agents@latest setup ## سجل النجوم -🎉 **110,000 نجمة على GitHub — شكرًا لكم!** +🎉 **170,000 نجمة على GitHub — شكرًا لكم!** شكرًا لكل من يستخدم REA، ويبلّغ عن الأخطاء، ويقترح ميزات، ويختبر إصدارات البناء، ويساهم بالإصلاحات. diff --git a/README_de.md b/README_de.md index bf456b12d..2c6d20290 100644 --- a/README_de.md +++ b/README_de.md @@ -262,7 +262,7 @@ Melde Sicherheitslücken gemäß [SECURITY.md](SECURITY.md). ## Sternverlauf -🎉 **110.000 GitHub-Sterne – vielen Dank!** +🎉 **170.000 GitHub-Sterne – vielen Dank!** Danke an alle, die REA nutzen, Fehler melden, Funktionswünsche äußern, Builds testen und Fehlerbehebungen beitragen. diff --git a/README_es.md b/README_es.md index db20cd717..f955b5891 100644 --- a/README_es.md +++ b/README_es.md @@ -262,7 +262,7 @@ Informa de vulnerabilidades siguiendo [SECURITY.md](SECURITY.md). ## Historial de estrellas -🎉 **110.000 estrellas en GitHub: ¡gracias!** +🎉 **170.000 estrellas en GitHub: ¡gracias!** Gracias a quienes usan REA, informan de errores, solicitan funciones, prueban compilaciones y contribuyen con correcciones. diff --git a/README_fa.md b/README_fa.md index f2c358c63..54af87c57 100644 --- a/README_fa.md +++ b/README_fa.md @@ -247,7 +247,7 @@ npx rea-agents@latest setup ## تاریخچهٔ ستاره‌ها -🎉 **۱۱۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!** +🎉 **۱۷۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!** از همهٔ کسانی که از REA استفاده می‌کنند، خطاها را گزارش می‌دهند، قابلیت‌های جدید پیشنهاد می‌کنند، buildها را آزمایش می‌کنند و در بهبود پروژه مشارکت دارند سپاسگزاریم. diff --git a/README_fr.md b/README_fr.md index 41281ce5c..346210525 100644 --- a/README_fr.md +++ b/README_fr.md @@ -266,7 +266,7 @@ Signalez les vulnérabilités via [SECURITY.md](SECURITY.md). ## Historique des étoiles -🎉 **110 000 étoiles GitHub — merci !** +🎉 **170 000 étoiles GitHub — merci !** Merci à toutes les personnes qui utilisent REA, signalent des bugs, proposent des fonctionnalités, testent les builds et contribuent des correctifs. diff --git a/README_id.md b/README_id.md index e5e3dd1a4..7a564c434 100644 --- a/README_id.md +++ b/README_id.md @@ -287,7 +287,7 @@ Laporkan kerentanan melalui [SECURITY.md](SECURITY.md). ## Riwayat Bintang -🎉 **110.000 bintang GitHub, terima kasih!** +🎉 **170.000 bintang GitHub, terima kasih!** Terima kasih kepada semua orang yang menggunakan REA, melaporkan bug, meminta fitur, menguji build, dan berkontribusi dalam perbaikan. diff --git a/README_it.md b/README_it.md index 409799e32..8b1c55afe 100644 --- a/README_it.md +++ b/README_it.md @@ -266,7 +266,7 @@ Segnala vulnerabilità tramite [SECURITY.md](SECURITY.md). ## Cronologia delle stelle -🎉 **110.000 stelle su GitHub — grazie!** +🎉 **170.000 stelle su GitHub — grazie!** Grazie a tutti coloro che usano REA, segnalano bug, propongono funzionalità, testano le build e contribuiscono con correzioni. diff --git a/README_ja.md b/README_ja.md index ee0875bd4..5fb42d096 100644 --- a/README_ja.md +++ b/README_ja.md @@ -264,7 +264,7 @@ npx rea-agents@latest setup ## スター履歴 -🎉 **GitHub スター 110,000 件、ありがとうございます！** +🎉 **GitHub スター 170,000 件、ありがとうございます！** REA の利用、バグ報告、機能の要望、ビルドのテスト、修正への貢献に感謝します。 diff --git a/README_ko.md b/README_ko.md index 39814030c..5a9e7cd5d 100644 --- a/README_ko.md +++ b/README_ko.md @@ -264,7 +264,7 @@ npx rea-agents@latest setup ## 스타 기록 -🎉 **GitHub 스타 110,000개, 감사합니다!** +🎉 **GitHub 스타 170,000개, 감사합니다!** REA를 사용하고, 버그를 보고하고, 기능을 요청하고, 빌드를 테스트하고, 수정에 기여해 주신 모든 분께 감사드립니다. diff --git a/README_pl.md b/README_pl.md index 31ed33869..c54af50fe 100644 --- a/README_pl.md +++ b/README_pl.md @@ -262,7 +262,7 @@ Zgłaszaj podatności zgodnie z [SECURITY.md](SECURITY.md). ## Historia gwiazdek -🎉 **110 000 gwiazdek na GitHubie — dziękujemy!** +🎉 **170 000 gwiazdek na GitHubie — dziękujemy!** Dziękujemy wszystkim, którzy używają REA, zgłaszają błędy, proponują nowe funkcje, testują kompilacje i przesyłają poprawki. diff --git a/README_pt-BR.md b/README_pt-BR.md index cba8b9c3d..dd138f7ff 100644 --- a/README_pt-BR.md +++ b/README_pt-BR.md @@ -262,7 +262,7 @@ Relate vulnerabilidades conforme [SECURITY.md](SECURITY.md). ## Histórico de estrelas -🎉 **110.000 estrelas no GitHub — muito obrigado!** +🎉 **170.000 estrelas no GitHub — muito obrigado!** Obrigado a todos que usam o REA, relatam bugs, sugerem funcionalidades, testam builds e contribuem com correções. diff --git a/README_ru.md b/README_ru.md index 320574bf6..34e7d3af2 100644 --- a/README_ru.md +++ b/README_ru.md @@ -262,7 +262,7 @@ npx rea-agents@latest setup ## История звёзд -🎉 **110 000 звёзд на GitHub — спасибо!** +🎉 **170 000 звёзд на GitHub — спасибо!** Спасибо всем, кто использует REA, сообщает об ошибках, предлагает новые функции, тестирует сборки и помогает с исправлениями. diff --git a/README_th.md b/README_th.md index f67b04e89..d07bbb6a3 100644 --- a/README_th.md +++ b/README_th.md @@ -262,7 +262,7 @@ npx rea-agents@latest setup ## ประวัติดาว -🎉 **110,000 ดาวบน GitHub — ขอบคุณทุกคน!** +🎉 **170,000 ดาวบน GitHub — ขอบคุณทุกคน!** ขอบคุณทุกคนที่ใช้ REA รายงานบั๊ก เสนอฟีเจอร์ ทดสอบบิลด์ และช่วยแก้ไข diff --git a/README_tr.md b/README_tr.md index ccf7db7ce..1daaaf9d0 100644 --- a/README_tr.md +++ b/README_tr.md @@ -262,7 +262,7 @@ Güvenlik açıklarını [SECURITY.md](SECURITY.md) üzerinden bildirin. ## Yıldız geçmişi -🎉 **GitHub'da 110.000 yıldız — teşekkürler!** +🎉 **GitHub'da 170.000 yıldız — teşekkürler!** REA'yı kullanan, hata bildiren, özellik talep eden, derlemeleri test eden ve düzeltmelere katkıda bulunan herkese teşekkürler. diff --git a/README_uk.md b/README_uk.md index 78f1af9f4..14c3d4984 100644 --- a/README_uk.md +++ b/README_uk.md @@ -262,7 +262,7 @@ npx rea-agents@latest setup ## Історія зірок -🎉 **110 000 зірок на GitHub — дякуємо!** +🎉 **170 000 зірок на GitHub — дякуємо!** Дякуємо всім, хто використовує REA, повідомляє про помилки, пропонує нові функції, тестує збірки й допомагає з виправленнями. diff --git a/README_vi.md b/README_vi.md index 55aa4dd18..86544a535 100644 --- a/README_vi.md +++ b/README_vi.md @@ -262,7 +262,7 @@ Báo cáo lỗ hổng theo [SECURITY.md](SECURITY.md). ## Lịch sử sao -🎉 **110.000 sao trên GitHub — xin cảm ơn!** +🎉 **170.000 sao trên GitHub — xin cảm ơn!** Cảm ơn mọi người đã sử dụng REA, báo lỗi, đề xuất tính năng, kiểm thử bản dựng và đóng góp bản sửa lỗi. diff --git a/README_zh-TW.md b/README_zh-TW.md index 061cc9a09..62c9ab2f3 100644 --- a/README_zh-TW.md +++ b/README_zh-TW.md @@ -264,7 +264,7 @@ npx rea-agents@latest setup ## Star 歷史 -🎉 **GitHub Star 達到 110,000 個，感謝大家！** +🎉 **GitHub Star 達到 170,000 個，感謝大家！** 感謝每一位使用 REA、報告錯誤、提出功能需求、測試建置和貢獻修復的朋友。 diff --git a/README_zh.md b/README_zh.md index be60efeec..c8d66ce08 100644 --- a/README_zh.md +++ b/README_zh.md @@ -264,7 +264,7 @@ npx rea-agents@latest setup ## Star 历史 -🎉 **GitHub Star 达到 110,000 个，感谢大家！** +🎉 **GitHub Star 达到 170,000 个，感谢大家！** 感谢每一位使用 REA、报告错误、提出功能需求、测试构建和贡献修复的朋友。 From 18d56f5321c3c5d5be40c19bcb0a0f2b25c2d459 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:05:32 +0800 Subject: [PATCH 033/159] test(ghidra): restore complete installation fixtures across CI boundaries Declare application.java.min=21 at the four synthetic installation producers so profile, PSP, lifecycle and MCP workflows exercise their intended paths under fail-closed Java admission. Keep missing-bound refusal coverage intact and format the two Ghidra tests rejected by the CI static lane. Validation: all 82 tests across the 11 failing CI files passed; 66 Ghidra admission, doctor and snapshot tests passed, including missing Java bounds. No production admission policy changed. Full suite and real Ghidra lanes were not run. --- src/ghidra/GhidraDoctor.test.ts | 4 +++- src/ghidra/GhidraSnapshotCache.test.ts | 3 ++- tests/boundary/filesystem/ghidraPspExtension.test.ts | 2 +- tests/boundary/mcp/ghidraMcpHarness.ts | 2 +- tests/boundary/mcp/ghidraReopen.test.ts | 2 +- tests/boundary/providers/ghidra/ghidraJavaLaunch.test.ts | 3 ++- 6 files changed, 10 insertions(+), 6 deletions(-) diff --git a/src/ghidra/GhidraDoctor.test.ts b/src/ghidra/GhidraDoctor.test.ts index 5335e4f60..e5d20f4c8 100644 --- a/src/ghidra/GhidraDoctor.test.ts +++ b/src/ghidra/GhidraDoctor.test.ts @@ -111,7 +111,9 @@ describe("Ghidra doctor integration", () => { "wrong version", inspection( { installDir: INSTALL }, - { readText: () => "application.version=11.4\napplication.java.min=21\n" }, + { + readText: () => "application.version=11.4\napplication.java.min=21\n", + }, ), "ghidra-version", ], diff --git a/src/ghidra/GhidraSnapshotCache.test.ts b/src/ghidra/GhidraSnapshotCache.test.ts index 5185054e9..9cf5c24ad 100644 --- a/src/ghidra/GhidraSnapshotCache.test.ts +++ b/src/ghidra/GhidraSnapshotCache.test.ts @@ -39,7 +39,8 @@ const fixture = (version = "12.1.4") => { { platform: "linux", architecture: "x64", - readText: () => `application.version=${version}\napplication.java.min=21\n`, + readText: () => + `application.version=${version}\napplication.java.min=21\n`, executable: () => true, probeJava: () => ({ version: "21.0.11", diff --git a/tests/boundary/filesystem/ghidraPspExtension.test.ts b/tests/boundary/filesystem/ghidraPspExtension.test.ts index 88e25939a..0e07ad629 100644 --- a/tests/boundary/filesystem/ghidraPspExtension.test.ts +++ b/tests/boundary/filesystem/ghidraPspExtension.test.ts @@ -52,7 +52,7 @@ const target = { const host: GhidraInstallationHost = { platform: "linux", architecture: "x64", - readText: () => "application.version=12.1.3\n", + readText: () => "application.version=12.1.3\napplication.java.min=21\n", executable: () => true, probeJava: () => ({ version: "21.0.12", diff --git a/tests/boundary/mcp/ghidraMcpHarness.ts b/tests/boundary/mcp/ghidraMcpHarness.ts index 7e2c3ae73..ad3565992 100644 --- a/tests/boundary/mcp/ghidraMcpHarness.ts +++ b/tests/boundary/mcp/ghidraMcpHarness.ts @@ -95,7 +95,7 @@ export const sessionEvidence = (session: BinarySession, value: unknown) => { const installationHost = (): GhidraInstallationHost => ({ platform: "linux", architecture: "x64", - readText: () => "application.version=12.1.4\n", + readText: () => "application.version=12.1.4\napplication.java.min=21\n", executable: () => true, probeJava: () => ({ version: "21.0.11", diff --git a/tests/boundary/mcp/ghidraReopen.test.ts b/tests/boundary/mcp/ghidraReopen.test.ts index bdac57233..6a90f0118 100644 --- a/tests/boundary/mcp/ghidraReopen.test.ts +++ b/tests/boundary/mcp/ghidraReopen.test.ts @@ -99,7 +99,7 @@ class FixtureLauncher implements GhidraLauncher { const installationHost: GhidraInstallationHost = { platform: "linux", architecture: "x64", - readText: () => "application.version=12.1.4\n", + readText: () => "application.version=12.1.4\napplication.java.min=21\n", executable: () => true, probeJava: () => ({ version: "21.0.11", diff --git a/tests/boundary/providers/ghidra/ghidraJavaLaunch.test.ts b/tests/boundary/providers/ghidra/ghidraJavaLaunch.test.ts index dc4be5ef2..56fec2151 100644 --- a/tests/boundary/providers/ghidra/ghidraJavaLaunch.test.ts +++ b/tests/boundary/providers/ghidra/ghidraJavaLaunch.test.ts @@ -286,7 +286,8 @@ describe.skipIf(process.platform === "win32")( { platform: process.platform, architecture: process.arch, - readText: () => "application.version=12.1.4\n", + readText: () => + "application.version=12.1.4\napplication.java.min=21\n", executable: () => true, probeJava: () => ({ version: "21.0.12", From 18831491cfb5cb98abbffce0cb04dae12a90fe28 Mon Sep 17 00:00:00 2001 From: Ysopwnable <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:32:22 +0800 Subject: [PATCH 034/159] fix(ci): ignore generated website translation staging (#2511) Ignore generator-owned website translation staging output while preserving authored lint errors and complete diagnostics. Verified with a real checker regression, a production translation build, complete coverage, developer/CI regression checks and native lanes. Fixes #2508 --- .gitignore | 2 ++ .../filesystem/staticCheckReports.test.ts | 32 ++++++++++++++++++- 2 files changed, 33 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index 90d431ec0..14f9ac533 100644 --- a/.gitignore +++ b/.gitignore @@ -72,3 +72,5 @@ Thumbs.db # Complete static website output, including generated language editions. website/_site/ +# Temporary translation output prepared before replacing the complete site. +website/.i18n-*/ diff --git a/tests/boundary/filesystem/staticCheckReports.test.ts b/tests/boundary/filesystem/staticCheckReports.test.ts index c2708e1db..7d022da72 100644 --- a/tests/boundary/filesystem/staticCheckReports.test.ts +++ b/tests/boundary/filesystem/staticCheckReports.test.ts @@ -1,6 +1,6 @@ import { spawnSync } from "node:child_process"; import { access, mkdir, readFile, writeFile } from "node:fs/promises"; -import { join, sep } from "node:path"; +import { dirname, join, sep } from "node:path"; import { fileURLToPath } from "node:url"; import { expect, it } from "vitest"; import { z } from "zod"; @@ -138,6 +138,36 @@ it("keeps lint errors failing and visible alongside the retained warnings", asyn ); }); +it("excludes translation staging while retaining authored website errors", async () => { + const root = await fixture(); + for (const name of [".gitignore", ".oxlintrc.json"]) + await writeFile(join(root, name), await readFile(join(repository, name))); + const authored = "website/public/assets/authored.js"; + const staged = + "website/.i18n-regression/site/examples/notes-electron/main.js"; + for (const name of [authored, staged]) { + const path = join(root, name); + await mkdir(dirname(path), { recursive: true }); + await writeFile( + path, + 'const source = require("node:fs");\nconsole.log(source);\n', + ); + } + + const result = invoke(root, "lint"); + expect(result.error).toBeUndefined(); + expect(result.status).toBe(1); + expect(result.stdout).toContain("1 errors, 0 warnings"); + expect(result.stderr).toContain(authored); + expect(await lintDiagnostics(root)).toEqual([ + expect.objectContaining({ + severity: "error", + code: "typescript(no-var-requires)", + filename: authored, + }), + ]); +}); + it("keeps malformed source failing with its parser diagnostic", async () => { const root = await fixture(); await writeFile(join(root, "src/malformed.js"), "const value = ;\n"); From e1cd292aa6e8f5f13441d408c66126782c8dbb0d Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:38:39 +0800 Subject: [PATCH 035/159] fix(adb): admit canonical device states and preserve malformed output --- src/adb/AdbDeviceOutput.test.ts | 22 ++++++++++++-- src/adb/AdbDeviceOutput.ts | 39 +++++++++--------------- src/adb/AdbProvider.test.ts | 46 +++++++++++++++++++++++++++++ src/domain/adb/adbDeviceAnalysis.ts | 21 ++++++++++++- 4 files changed, 100 insertions(+), 28 deletions(-) diff --git a/src/adb/AdbDeviceOutput.test.ts b/src/adb/AdbDeviceOutput.test.ts index fe11c2a26..12b82ab84 100644 --- a/src/adb/AdbDeviceOutput.test.ts +++ b/src/adb/AdbDeviceOutput.test.ts @@ -84,6 +84,24 @@ describe("parseAdbDevicesOutput", () => { ]); }); + it("accepts complete multiword states and rejects state prefixes", () => { + const output = [ + "serial-no-permission no permissions usb:1-1", + "serial-bad-device deviceXYZ ", + "serial-bad-offline offline-ish", + "serial-bad-auth unauthorizedXYZ", + ].join("\n"); + const { devices, unparsedLines } = parseAdbDevicesOutput(output); + expect(devices.map(({ serial, state }) => [serial, state])).toEqual([ + ["serial-no-permission", "no permissions"], + ]); + expect(unparsedLines).toEqual([ + "serial-bad-device deviceXYZ ", + "serial-bad-offline offline-ish", + "serial-bad-auth unauthorizedXYZ", + ]); + }); + it("returns unrecognized lines instead of dropping them", () => { const output = [ "List of devices attached", @@ -194,10 +212,10 @@ describe("parsePackageListOutput", () => { it("keeps unrecognized lines for partial coverage reporting", () => { const { packages, unparsedLines } = parsePackageListOutput( - "package:/data/app/x/base.apk=com.ok\ngarbage line\n", + "package:/data/app/x/base.apk=com.ok\n garbage line \n", ); expect(packages).toHaveLength(1); - expect(unparsedLines).toEqual(["garbage line"]); + expect(unparsedLines).toEqual([" garbage line "]); }); }); diff --git a/src/adb/AdbDeviceOutput.ts b/src/adb/AdbDeviceOutput.ts index 3380de608..8d3c74f5f 100644 --- a/src/adb/AdbDeviceOutput.ts +++ b/src/adb/AdbDeviceOutput.ts @@ -10,11 +10,15 @@ import { adbHostFileName, type AdbHostFileNameReason, } from "./AdbHostFileName.js"; +import { + ADB_DEVICE_STATES, + type AdbDeviceState, +} from "../domain/adb/adbDeviceAnalysis.js"; /** One device exactly as `adb devices -l` reports it. */ export interface AdbDeviceEntry { readonly serial: string; - readonly state: string; + readonly state: AdbDeviceState; readonly transport: "usb" | "tcp" | "unknown"; readonly product: string | null; readonly model: string | null; @@ -24,25 +28,6 @@ export interface AdbDeviceEntry { readonly kind_basis: "emulator_serial_prefix" | "usb_transport" | "none"; } -/** - * Device states adb prints after the serial, longest first so two-word - * states match before their single-word prefixes. A line whose remainder - * starts with none of them is not a device line. - */ -const DEVICE_STATES = [ - "no permissions", - "unauthorized", - "authorizing", - "offline", - "recovery", - "sideload", - "connecting", - "bootloader", - "device", - "unknown", - "host", -] as const; - const DEVICE_LINE = /^(\S+)\s+(.*)$/u; const metadataValue = (remainder: string, key: string): string | null => { @@ -73,11 +58,15 @@ export const parseAdbDevicesOutput = ( continue; } const match = DEVICE_LINE.exec(line); - const state = DEVICE_STATES.find((candidate) => - match?.[2]?.startsWith(candidate), - ); + const state = ADB_DEVICE_STATES.find((candidate) => { + const remainder = match?.[2]; + if (remainder === undefined || !remainder.startsWith(candidate)) + return false; + const next = remainder[candidate.length]; + return next === undefined || /\s/u.test(next); + }); if (match === null || state === undefined) { - if (line.trim() !== "") unparsedLines.push(line); + if (rawLine.trim() !== "") unparsedLines.push(rawLine); continue; } const serial = match[1]!; @@ -192,7 +181,7 @@ export const parsePackageListOutput = ( if (line === "") continue; const match = PACKAGE_LINE.exec(line); if (match === null) { - unparsedLines.push(line); + unparsedLines.push(rawLine); continue; } packages.push({ diff --git a/src/adb/AdbProvider.test.ts b/src/adb/AdbProvider.test.ts index d1a153848..b89ab9dfd 100644 --- a/src/adb/AdbProvider.test.ts +++ b/src/adb/AdbProvider.test.ts @@ -121,6 +121,52 @@ fi`, ]); }); +it("refuses device listings with an invalid state token", async () => { + const malformedLine = "serial deviceXYZ "; + const { binary } = await writeAdbStub( + () => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi +if [ "$1" = "devices" ]; then + printf 'List of devices attached\\n${malformedLine}\\n' + exit 0 +fi`, + ); + const result = await providerOver(binary).execute({ + operation: "list_adb_devices", + input: {}, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error._tag).toBe("AnalysisProtocolError"); + expect(result.error.message).toContain(malformedLine); +}); + +it("lists rescue and detached transports alongside online devices", async () => { + const { binary } = await writeAdbStub( + () => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi +if [ "$1" = "devices" ]; then + printf 'List of devices attached\\n' + printf 'serial-online device transport_id:1\\n' + printf 'serial-rescue rescue transport_id:2\\n' + printf 'serial-detached detached transport_id:3\\n' + exit 0 +fi`, + ); + const result = await evidenceResult(providerOver(binary), { + operation: "list_adb_devices", + input: {}, + }); + expect(result.devices).toEqual( + expect.arrayContaining([ + expect.objectContaining({ serial: "serial-online", state: "device" }), + expect.objectContaining({ serial: "serial-rescue", state: "rescue" }), + expect.objectContaining({ + serial: "serial-detached", + state: "detached", + }), + ]), + ); +}); + it("inspects a device through the getprop whitelist", async () => { const { binary } = await writeAdbStub( () => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi diff --git a/src/domain/adb/adbDeviceAnalysis.ts b/src/domain/adb/adbDeviceAnalysis.ts index 17cdc223e..c524ebd02 100644 --- a/src/domain/adb/adbDeviceAnalysis.ts +++ b/src/domain/adb/adbDeviceAnalysis.ts @@ -30,6 +30,25 @@ const packageNameSchema = z "Package name must be dot-separated Java identifiers", ); +/** Text states emitted by adb's ConnectionState formatter. */ +export const ADB_DEVICE_STATES = [ + "no permissions", + "unauthorized", + "authorizing", + "offline", + "recovery", + "sideload", + "connecting", + "bootloader", + "device", + "unknown", + "host", + "rescue", + "detached", +] as const; +export const adbDeviceStateSchema = z.enum(ADB_DEVICE_STATES); +export type AdbDeviceState = (typeof ADB_DEVICE_STATES)[number]; + const keyStringSchema = z.string().min(1).max(256); const keyPattern = /^[A-Za-z0-9._\x2d]+$/u; @@ -412,7 +431,7 @@ const client = z.strictObject({ const device = z.strictObject({ serial: z.string().min(1), - state: z.string().min(1), + state: adbDeviceStateSchema, transport: z.enum(["usb", "tcp", "unknown"]), product: z.string().nullable(), model: z.string().nullable(), From 1b953021eadf4cac09e20be388d79536903633b3 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:39:06 +0800 Subject: [PATCH 036/159] fix(javascript): centralize module syntax and separate erased type dependencies --- docs/javascript-application-workflows.md | 8 + .../ReferenceSourceImportEntries.ts | 6 +- src/application/ReferenceSourceImportTypes.ts | 3 - .../JavaScriptArtifactPathResolution.ts | 22 +- .../JavaScriptModuleRelationships.ts | 14 +- .../javascript/javascriptModuleDeclaration.ts | 60 +++++ .../javascript/javascriptModuleSpecifier.ts | 6 + ...riptSemanticAnalysis.require_links.test.ts | 62 +++++ .../javascript/javascriptSemanticAnalysis.ts | 6 + .../javascriptSemanticProjection.ts | 47 +++- .../javascript/javascriptSemanticReturns.ts | 1 + .../javascript/javascriptStaticAnalysis.ts | 20 +- src/domain/referenceSourceImportParsing.ts | 14 +- src/domain/webModuleImports.test.ts | 11 +- src/domain/webModuleImports.ts | 37 ++- .../JavaScriptModuleLookupCli.test.ts | 238 +++++++++++------- .../javascriptModuleRelationships.test.ts | 54 ++++ .../filesystem/referenceSourceImport.test.ts | 38 +++ 18 files changed, 513 insertions(+), 134 deletions(-) create mode 100644 src/domain/javascript/javascriptModuleDeclaration.ts create mode 100644 src/domain/javascript/javascriptModuleSpecifier.ts diff --git a/docs/javascript-application-workflows.md b/docs/javascript-application-workflows.md index 2d2d9e64a..1e9e7fe19 100644 --- a/docs/javascript-application-workflows.md +++ b/docs/javascript-application-workflows.md @@ -23,6 +23,14 @@ artifact digest and structural graph ID. Semantic tracing requires the semantic relation graph and reports when it is unavailable rather than treating missing data as an empty graph. +Explicit TypeScript `import type` and `export type` declarations remain source +facts and do not create runtime import or export relationships. Type export names +are reported separately from value export names. Inline `type` specifiers can +leave an empty import or re-export with side effects under +[`verbatimModuleSyntax`](https://www.typescriptlang.org/tsconfig/verbatimModuleSyntax.html); +REA retains those dependency candidates and reports the unavailable compiler +options rather than assuming the dependency is erased. + The semantic graph wire format stores shared non-location provenance once in its required `evidence_contexts` table. Each node, relation, fingerprint, and unknown retains its exact `evidence.location` and names the owning context with diff --git a/src/application/ReferenceSourceImportEntries.ts b/src/application/ReferenceSourceImportEntries.ts index 3254d7cfb..a6e962118 100644 --- a/src/application/ReferenceSourceImportEntries.ts +++ b/src/application/ReferenceSourceImportEntries.ts @@ -12,12 +12,14 @@ import { detectReferenceSourceLanguage, } from "../domain/referenceSourceClassification.js"; import type { HistoricalSourceGraphInput } from "../domain/referenceSourceGraph.js"; -import { parseReferenceSourceImports } from "../domain/referenceSourceImportParsing.js"; +import { + parseReferenceSourceImports, + PARSEABLE_REFERENCE_SOURCE_LANGUAGES, +} from "../domain/referenceSourceImportParsing.js"; import type { ReferenceSourceEntry, ReferenceSourceRead, } from "../reference/ReferenceSourceReaderTypes.js"; -import { PARSEABLE_REFERENCE_SOURCE_LANGUAGES } from "./ReferenceSourceImportTypes.js"; import { typescriptEmittedSourceCandidates } from "./TypeScriptEmittedSourceCandidates.js"; export interface ParsedReferenceSourceEntries { diff --git a/src/application/ReferenceSourceImportTypes.ts b/src/application/ReferenceSourceImportTypes.ts index 9395dc5c0..f8474315a 100644 --- a/src/application/ReferenceSourceImportTypes.ts +++ b/src/application/ReferenceSourceImportTypes.ts @@ -87,6 +87,3 @@ export const DEFAULT_REFERENCE_SOURCE_IGNORE_PATTERNS = [ ".coverage", "*.log", ] as const; - -export const PARSEABLE_REFERENCE_SOURCE_LANGUAGES: ReadonlySet = - new Set(["JavaScript", "TypeScript", "JSX", "TSX"]); diff --git a/src/application/javascript/JavaScriptArtifactPathResolution.ts b/src/application/javascript/JavaScriptArtifactPathResolution.ts index b3d769af6..3ea23ab11 100644 --- a/src/application/javascript/JavaScriptArtifactPathResolution.ts +++ b/src/application/javascript/JavaScriptArtifactPathResolution.ts @@ -20,6 +20,7 @@ import { lookupExactPackageImport, } from "../../domain/javascript/javascriptPackageImportAliases.js"; import { typescriptEmittedSourceCandidates } from "../TypeScriptEmittedSourceCandidates.js"; +import { isRelativeJavaScriptModuleSpecifier } from "../../domain/javascript/javascriptModuleSpecifier.js"; type ArtifactPathResolutionContext = | "package-entrypoint" @@ -166,7 +167,7 @@ const resolvePath = (input: ResolveArtifactPathInput): CandidateResolution => { ); const isModuleUrlPath = input.context === "module-specifier" && - (input.declaredPath.startsWith(".") || + (isRelativeJavaScriptModuleSpecifier(input.declaredPath) || input.declaredPath.startsWith("/") || fileUrlPath(input.declaredPath) !== undefined); const emittedSourceCandidates = @@ -229,7 +230,10 @@ const contextualCandidate = ( return unresolvedCandidate("external", [ "URL and Node builtin schemes are outside static artifact module resolution.", ]); - if (!declared.startsWith(".") && !declared.startsWith("/")) + if ( + !isRelativeJavaScriptModuleSpecifier(declared) && + !declared.startsWith("/") + ) return bareModuleCandidate(input); if (input.moduleKind !== "require") { const decoded = decodeModuleUrlPath(stripQueryAndFragment(declared)); @@ -275,7 +279,10 @@ const contextualCandidate = ( "URL", ); } - return relativeCandidate(input.sourcePath, declared); + const candidate = relativeCandidate(input.sourcePath, declared); + return context === "module-specifier" && input.moduleKind === "import" + ? exactUrlPathCandidate(declared, candidate, "module URL") + : candidate; }; const decodeModuleUrlPath = ( @@ -310,9 +317,12 @@ const bareModuleCandidate = ( return unresolvedCandidate("external", [ "The bare specifier is a Node builtin or invalid package name.", ]); - if (input.moduleKind === "import" && request.packageName.includes("%")) + if ( + input.moduleKind === "import" && + (request.packageName.startsWith(".") || request.packageName.includes("%")) + ) return unresolvedCandidate("rejected", [ - "Percent signs are not admitted in ES module package names.", + "ES module package names cannot start with a dot or contain percent signs.", ]); const self = packageSelfCandidate(input, request); if (self !== null) return self; @@ -735,7 +745,7 @@ const htmlCandidate = ( const exactUrlPathCandidate = ( declared: string, candidate: string, - kind: "HTML" | "URL", + kind: "HTML" | "URL" | "module URL", ): string | CandidateResolution => { if (!urlPathIsDirectory(declared)) return candidate; const confined = confineCandidate(candidate); diff --git a/src/application/javascript/JavaScriptModuleRelationships.ts b/src/application/javascript/JavaScriptModuleRelationships.ts index 874377810..514a9bb7c 100644 --- a/src/application/javascript/JavaScriptModuleRelationships.ts +++ b/src/application/javascript/JavaScriptModuleRelationships.ts @@ -81,8 +81,13 @@ export const addJavaScriptSourceModules = ( if (asset === undefined || semantic === null || program === undefined) continue; const exports = semantic.ir.moduleLinks - .flatMap(({ exportedName }) => - exportedName === null ? [] : [exportedName], + .flatMap(({ exportedName, kind }) => + exportedName === null || kind === "type-import" ? [] : [exportedName], + ) + .sort(compareUnicodeCodePoints); + const typeExports = semantic.ir.moduleLinks + .flatMap(({ exportedName, kind }) => + exportedName === null || kind !== "type-import" ? [] : [exportedName], ) .sort(compareUnicodeCodePoints); const retainedExports = [...new Set(exports)]; @@ -105,8 +110,12 @@ export const addJavaScriptSourceModules = ( import_relationships: semantic.ir.moduleLinks.filter( ({ kind }) => kind === "import" || kind === "require", ).length, + type_relationships: semantic.ir.moduleLinks.filter( + ({ kind }) => kind === "type-import", + ).length, export_relationships: exports.length, export_names: retainedExports, + type_export_names: [...new Set(typeExports)], omitted_export_names: 0, }, evidence: astObservationEvidence({ @@ -152,6 +161,7 @@ export function* addJavaScriptModuleRelationshipsSteps( for (const link of semantic.ir.moduleLinks) { // One module can export thousands of bindings, each with return shapes. yield; + if (link.kind === "type-import") continue; const input = { context, file, semantic, source, link }; if (isExportLink(link)) addExportRelationship(input, omissions); else if (link.specifier !== null) diff --git a/src/domain/javascript/javascriptModuleDeclaration.ts b/src/domain/javascript/javascriptModuleDeclaration.ts new file mode 100644 index 000000000..d7a8d6754 --- /dev/null +++ b/src/domain/javascript/javascriptModuleDeclaration.ts @@ -0,0 +1,60 @@ +import * as t from "@babel/types"; + +/** Whether TypeScript explicitly erases this entire import declaration. */ +export const isTypeOnlyImportDeclaration = ( + node: t.ImportDeclaration, +): boolean => node.importKind === "type" || node.importKind === "typeof"; + +/** Whether TypeScript explicitly erases this entire export declaration. */ +export const isTypeOnlyExportDeclaration = ( + node: t.ExportNamedDeclaration | t.ExportAllDeclaration, +): boolean => + node.exportKind === "type" || + (t.isExportNamedDeclaration(node) && + node.declaration !== null && + (t.isTSInterfaceDeclaration(node.declaration) || + t.isTSTypeAliasDeclaration(node.declaration) || + t.isTSDeclareFunction(node.declaration))); + +/** Whether this import binding is explicitly type-only. */ +export const isTypeOnlyImportSpecifier = (node: t.ImportSpecifier): boolean => + node.importKind === "type" || node.importKind === "typeof"; + +/** Whether this exported binding is explicitly type-only. */ +export const isTypeOnlyExportSpecifier = (node: t.ExportSpecifier): boolean => + node.exportKind === "type"; + +/** A regular import whose bindings are all erased but may retain side effects. */ +export const hasOnlyInlineTypeImportSpecifiers = ( + node: t.ImportDeclaration, +): boolean => + !isTypeOnlyImportDeclaration(node) && + node.specifiers.length > 0 && + node.specifiers.every( + (specifier) => + t.isImportSpecifier(specifier) && isTypeOnlyImportSpecifier(specifier), + ); + +/** A regular re-export whose bindings are all erased but may retain side effects. */ +export const hasOnlyInlineTypeExportSpecifiers = ( + node: t.ExportNamedDeclaration, +): boolean => + !isTypeOnlyExportDeclaration(node) && + node.source != null && + node.specifiers.length > 0 && + node.specifiers.every( + (specifier) => + t.isExportSpecifier(specifier) && isTypeOnlyExportSpecifier(specifier), + ); + +/** Whether compiler options leave an inline type-only declaration's side effect unknown. */ +export const hasUncertainTypeOnlyModuleSideEffect = ( + program: t.Program, +): boolean => + program.body.some( + (node) => + (t.isImportDeclaration(node) && + hasOnlyInlineTypeImportSpecifiers(node)) || + (t.isExportNamedDeclaration(node) && + hasOnlyInlineTypeExportSpecifiers(node)), + ); diff --git a/src/domain/javascript/javascriptModuleSpecifier.ts b/src/domain/javascript/javascriptModuleSpecifier.ts new file mode 100644 index 000000000..f52f081dc --- /dev/null +++ b/src/domain/javascript/javascriptModuleSpecifier.ts @@ -0,0 +1,6 @@ +/** Relative module syntax differs from an arbitrary dot-prefixed filename. */ +export const isRelativeJavaScriptModuleSpecifier = (value: string): boolean => + value === "." || + value === ".." || + value.startsWith("./") || + value.startsWith("../"); diff --git a/src/domain/javascript/javascriptSemanticAnalysis.require_links.test.ts b/src/domain/javascript/javascriptSemanticAnalysis.require_links.test.ts index 333ad4f9c..a0d93c3c3 100644 --- a/src/domain/javascript/javascriptSemanticAnalysis.require_links.test.ts +++ b/src/domain/javascript/javascriptSemanticAnalysis.require_links.test.ts @@ -22,6 +22,11 @@ describe("TypeScript import-equals links", () => { localName: "dependency", exportedName: null, }), + expect.objectContaining({ + kind: "type-import", + specifier: "./types", + localName: "types", + }), ]); }); @@ -51,6 +56,57 @@ describe("TypeScript import-equals links", () => { }); }); +describe("TypeScript type-only module links", () => { + it("retains declarations without runtime bindings or callable claims", () => { + const ir = analyzeJavaScriptSemantics(` + import type { Whole } from "./whole"; + import { type Inline } from "./inline"; + import { type Erased, value } from "./mixed"; + export type { Reexported } from "./reexported"; + export { type InlineExport } from "./inline-export"; + export type LocalType = string; + export interface LocalInterface { value: string } + function hiddenFunction() { return 1; } + export type { hiddenFunction }; + `); + + const externalLinks = ir.moduleLinks + .filter(({ specifier }) => specifier !== null) + .map( + ({ kind, specifier, importedName, exportedName }) => + [kind, specifier, importedName, exportedName] as const, + ); + expect(externalLinks).toEqual([ + ["type-import", "./whole", "Whole", null], + ["type-import", "./inline", "Inline", null], + ["type-import", "./mixed", "Erased", null], + ["import", "./mixed", "value", null], + ["type-import", "./reexported", "Reexported", "Reexported"], + ["type-import", "./inline-export", "InlineExport", "InlineExport"], + ]); + + const typeLinks = ir.moduleLinks + .filter(({ kind }) => kind === "type-import") + .map( + ({ specifier, localName, exportedName, callableId }) => + [specifier, localName, exportedName, callableId] as const, + ); + expect(typeLinks).toEqual([ + ["./whole", "Whole", null, null], + ["./inline", "Inline", null, null], + ["./mixed", "Erased", null, null], + ["./reexported", null, "Reexported", null], + ["./inline-export", null, "InlineExport", null], + [null, "LocalType", "LocalType", null], + [null, "LocalInterface", "LocalInterface", null], + [null, "hiddenFunction", "hiddenFunction", null], + ]); + expect(ir.limitations).toContain( + "Compiler options are unavailable; inline type-only imports or re-exports may preserve a module side-effect dependency.", + ); + }); +}); + describe("CommonJS dependency links", () => { it("retains an import-equals binding and its module origin", () => { const ir = analyzeJavaScriptSemantics(` @@ -76,6 +132,12 @@ describe("CommonJS dependency links", () => { importedName: null, localName: "typed", }), + expect.objectContaining({ + kind: "type-import", + specifier: "type-only", + importedName: null, + localName: "TypeOnly", + }), ]); expect(ir.references.filter(({ name }) => name === "typed")).toHaveLength( 1, diff --git a/src/domain/javascript/javascriptSemanticAnalysis.ts b/src/domain/javascript/javascriptSemanticAnalysis.ts index b56e1a112..35a8b7dc9 100644 --- a/src/domain/javascript/javascriptSemanticAnalysis.ts +++ b/src/domain/javascript/javascriptSemanticAnalysis.ts @@ -43,6 +43,7 @@ import { import { semanticRequireOrigin } from "./javascriptSemanticRequireOrigin.js"; import { collectSemanticMemberMutationsSteps } from "./javascriptSemanticMemberMutations.js"; import { indexJavaScriptAmbientWritesSteps } from "./javascriptSemanticAmbientWrites.js"; +import { hasUncertainTypeOnlyModuleSideEffect } from "./javascriptModuleDeclaration.js"; import { collectSemanticReturns, resolveSemanticModuleCallables, @@ -200,6 +201,11 @@ export function* analyzeParsedJavaScriptSemanticsSteps( "The parser recovered from syntax errors; affected bindings are partial.", ] : []), + ...(hasUncertainTypeOnlyModuleSideEffect(file.program) + ? [ + "Compiler options are unavailable; inline type-only imports or re-exports may preserve a module side-effect dependency.", + ] + : []), "Values and aliases were recovered from inert syntax only; no JavaScript was executed.", "Return sites include only direct callable returns; nested callable returns remain separate.", "Local call, argument, return, and closure relations are static candidates and do not prove runtime invocation.", diff --git a/src/domain/javascript/javascriptSemanticProjection.ts b/src/domain/javascript/javascriptSemanticProjection.ts index 7e4663efe..9d252a0d8 100644 --- a/src/domain/javascript/javascriptSemanticProjection.ts +++ b/src/domain/javascript/javascriptSemanticProjection.ts @@ -28,6 +28,12 @@ import { propertyName, semanticStaticPropertyKey, } from "./javascriptAstValues.js"; +import { + isTypeOnlyExportDeclaration, + isTypeOnlyExportSpecifier, + isTypeOnlyImportDeclaration, + isTypeOnlyImportSpecifier, +} from "./javascriptModuleDeclaration.js"; interface CollectCallableInput { readonly node: t.Node; @@ -75,7 +81,7 @@ export const collectSemanticModuleLink = ( addModuleLink( state, { - kind: "re-export", + kind: isTypeOnlyExportDeclaration(node) ? "type-import" : "re-export", specifier: node.source.value, importedName: "*", localName: null, @@ -108,12 +114,19 @@ const collectImportEquals = ( node: t.TSImportEqualsDeclaration, state: JavaScriptSemanticAnalysisState, ): void => { - const specifier = semanticImportEqualsSpecifier(node); + const moduleReference = node.moduleReference; + const specifier = + semanticImportEqualsSpecifier(node) ?? + (node.importKind === "type" && + t.isTSExternalModuleReference(moduleReference) && + t.isStringLiteral(moduleReference.expression) + ? moduleReference.expression.value + : null); if (specifier === null) return; addModuleLink( state, { - kind: "require", + kind: node.importKind === "type" ? "type-import" : "require", specifier, importedName: null, localName: node.id.name, @@ -287,7 +300,7 @@ const collectImports = ( addModuleLink( state, { - kind: "import", + kind: isTypeOnlyImportDeclaration(node) ? "type-import" : "import", specifier: node.source.value, importedName: null, localName: null, @@ -300,7 +313,12 @@ const collectImports = ( addModuleLink( state, { - kind: "import", + kind: + isTypeOnlyImportDeclaration(node) || + (t.isImportSpecifier(specifier) && + isTypeOnlyImportSpecifier(specifier)) + ? "type-import" + : "import", specifier: node.source.value, importedName: t.isImportDefaultSpecifier(specifier) ? "default" @@ -319,12 +337,13 @@ const collectNamedExports = ( node: t.ExportNamedDeclaration, state: JavaScriptSemanticAnalysisState, ): void => { + const declarationTypeOnly = isTypeOnlyExportDeclaration(node); if (node.declaration !== null && node.declaration !== undefined) for (const localName of declarationNames(node.declaration)) addModuleLink( state, { - kind: "export", + kind: declarationTypeOnly ? "type-import" : "export", specifier: null, importedName: null, localName, @@ -342,7 +361,14 @@ const collectNamedExports = ( addModuleLink( state, { - kind: node.source == null ? "export" : "re-export", + kind: + declarationTypeOnly || + (t.isExportSpecifier(specifier) && + isTypeOnlyExportSpecifier(specifier)) + ? "type-import" + : node.source == null + ? "export" + : "re-export", specifier: node.source?.value ?? null, importedName: t.isExportNamespaceSpecifier(specifier) ? "*" @@ -745,6 +771,13 @@ const declarationNames = (declaration: t.Declaration): string[] => { t.isIdentifier(declaration.id) ) return [declaration.id.name]; + if ( + (t.isTSInterfaceDeclaration(declaration) || + t.isTSTypeAliasDeclaration(declaration) || + t.isTSDeclareFunction(declaration)) && + t.isIdentifier(declaration.id) + ) + return [declaration.id.name]; if ( t.isTSImportEqualsDeclaration(declaration) && semanticImportEqualsSpecifier(declaration) !== null diff --git a/src/domain/javascript/javascriptSemanticReturns.ts b/src/domain/javascript/javascriptSemanticReturns.ts index 189a1385e..a3ca92ead 100644 --- a/src/domain/javascript/javascriptSemanticReturns.ts +++ b/src/domain/javascript/javascriptSemanticReturns.ts @@ -101,6 +101,7 @@ export const resolveSemanticModuleCallables = ( bindings.map(({ bindingId, provenance }) => [bindingId, provenance]), ); return state.moduleLinks.map((link) => { + if (link.kind === "type-import") return { ...link, callableId: null }; // Named exports follow their lexical binding, including every assignment. // A declaration's ID or a callable's display name cannot prove that link. if (link.localName === null) diff --git a/src/domain/javascript/javascriptStaticAnalysis.ts b/src/domain/javascript/javascriptStaticAnalysis.ts index 4d10638bc..c34462498 100644 --- a/src/domain/javascript/javascriptStaticAnalysis.ts +++ b/src/domain/javascript/javascriptStaticAnalysis.ts @@ -53,6 +53,11 @@ import { import { createJavaScriptRpcScan } from "./javascriptRpcEndpoints.js"; import { semanticImportEqualsSpecifier } from "./javascriptSemanticProjection.js"; +import { + hasUncertainTypeOnlyModuleSideEffect, + isTypeOnlyExportDeclaration, + isTypeOnlyImportDeclaration, +} from "./javascriptModuleDeclaration.js"; /** Parse one JavaScript artifact and recover static structure only. */ export const analyzeJavaScriptStaticSource = ( @@ -154,6 +159,11 @@ const finalizeStaticAnalysis = ( : [ "One or more static keys, expressions, or Electron boundary values were dynamic and remain unknown.", ]), + ...(hasUncertainTypeOnlyModuleSideEffect(file.program) + ? [ + "Compiler options are unavailable; inline type-only imports or re-exports may preserve a module side-effect dependency.", + ] + : []), ...(accumulator.references.some( ({ value }) => value.specifier !== null && ELECTRON_MODULE.test(value.specifier), @@ -246,14 +256,20 @@ const inspectNode = ( addReference(findings, { node, kind: "require", specifier }); } else if ( (t.isImportDeclaration(node) || t.isExportAllDeclaration(node)) && - node.source !== undefined + node.source !== undefined && + !(t.isImportDeclaration(node) && isTypeOnlyImportDeclaration(node)) && + !(t.isExportAllDeclaration(node) && isTypeOnlyExportDeclaration(node)) ) addReference(findings, { node, kind: "static-import", specifier: node.source.value, }); - else if (t.isExportNamedDeclaration(node) && node.source != null) + else if ( + t.isExportNamedDeclaration(node) && + node.source != null && + !isTypeOnlyExportDeclaration(node) + ) addReference(findings, { node, kind: "static-import", diff --git a/src/domain/referenceSourceImportParsing.ts b/src/domain/referenceSourceImportParsing.ts index 998e368c4..18f76a358 100644 --- a/src/domain/referenceSourceImportParsing.ts +++ b/src/domain/referenceSourceImportParsing.ts @@ -11,6 +11,7 @@ import { parserPluginsForPath, } from "./javascript/javascriptSourceParser.js"; import { unshadowedReferenceSourceRequireCalls } from "./referenceSourceRequireBindings.js"; +import { isRelativeJavaScriptModuleSpecifier } from "./javascript/javascriptModuleSpecifier.js"; import { isCallExpression, isExportAllDeclaration, @@ -60,7 +61,8 @@ export interface ReferenceSourceImportParseResult { readonly limitations: readonly string[]; } -const codeSourceLanguages = new Set(["JavaScript", "TypeScript", "JSX", "TSX"]); +export const PARSEABLE_REFERENCE_SOURCE_LANGUAGES: ReadonlySet = + new Set(["JavaScript", "TypeScript", "JSX", "TSX"]); const safeModuleName = (node: Node | null | undefined): string | undefined => { if (isStringLiteral(node)) return node.value; @@ -70,9 +72,11 @@ const safeModuleName = (node: Node | null | undefined): string | undefined => { const resolveSpecifier = ( specifier: string, ): ReferenceSourceImportResolution => { - if (specifier.startsWith(".") || specifier.startsWith("/")) return "internal"; - if (specifier.startsWith("node:") || specifier.startsWith("data:")) - return "external"; + if ( + isRelativeJavaScriptModuleSpecifier(specifier) || + specifier.startsWith("/") + ) + return "internal"; return "external"; }; @@ -314,7 +318,7 @@ export const parseReferenceSourceImports = ( bytes: Uint8Array, language: string | null, ): ReferenceSourceImportParseResult => { - if (language !== null && !codeSourceLanguages.has(language)) + if (language !== null && !PARSEABLE_REFERENCE_SOURCE_LANGUAGES.has(language)) return { relationships: [], parse_failures: [], limitations: [] }; let source: string; diff --git a/src/domain/webModuleImports.test.ts b/src/domain/webModuleImports.test.ts index 26db977a1..c0bcabdbf 100644 --- a/src/domain/webModuleImports.test.ts +++ b/src/domain/webModuleImports.test.ts @@ -25,11 +25,18 @@ describe("native module syntax observations", () => { item.expression, ); }); - it("excludes erased TypeScript imports/exports and bundler inference", () => { + it("excludes whole type declarations and retains inline type dependency syntax", () => { const result = collectWebModuleImports( 'import type {T} from "types"; import {type X} from "also-types"; export type {T} from "export-types"; export {type Y} from "only-types"; import {type T, value} from "mixed"; __webpack_require__.e(4); __vitePreload(() => 1, ["chunk.js"]);', ); - expect(result.imports.map((item) => item.specifier)).toEqual(["mixed"]); + expect(result.imports.map((item) => item.specifier)).toEqual([ + "also-types", + "only-types", + "mixed", + ]); + expect(result.diagnostics).toContain( + "Compiler options are unavailable; inline type-only imports or re-exports may preserve a module side-effect dependency.", + ); }); it("uses UTF-16 source locations after astral characters", () => { const source = 'const emoji = "😀";\nimport "./x.js";'; diff --git a/src/domain/webModuleImports.ts b/src/domain/webModuleImports.ts index 026868955..0f9846871 100644 --- a/src/domain/webModuleImports.ts +++ b/src/domain/webModuleImports.ts @@ -4,8 +4,13 @@ import { traverseJavaScriptAst } from "./javascript/javascriptSemanticTraversal. import { parseJavaScriptSource } from "./javascript/javascriptSourceParser.js"; import type { WebModuleImport } from "./webModuleTrace.js"; import type { ExportedWebScript } from "./webScriptExport.js"; +import { + hasUncertainTypeOnlyModuleSideEffect, + isTypeOnlyExportDeclaration, + isTypeOnlyImportDeclaration, +} from "./javascript/javascriptModuleDeclaration.js"; -/** Collect native import syntax, excluding erased type-only references. */ +/** Collect module declarations, omitting whole-statement type-only forms. */ export const collectWebModuleImports = ( source: string, ): { @@ -55,35 +60,18 @@ export const collectWebModuleImports = ( traverseJavaScriptAst(ast, { enter: (node) => { if (node.type === "ImportDeclaration") { - if (node.importKind === "type" || node.importKind === "typeof") return; - if ( - node.specifiers.length > 0 && - node.specifiers.every( - (s) => - s.type === "ImportSpecifier" && - (s.importKind === "type" || s.importKind === "typeof"), - ) - ) - return; + if (isTypeOnlyImportDeclaration(node)) return; add("static-import", node.source); } else if ( node.type === "ExportAllDeclaration" || node.type === "ExportNamedDeclaration" ) { if ( - node.exportKind === "type" || + isTypeOnlyExportDeclaration(node) || node.source === null || node.source === undefined ) return; - if ( - node.type === "ExportNamedDeclaration" && - node.specifiers.length > 0 && - node.specifiers.every( - (s) => s.type === "ExportSpecifier" && s.exportKind === "type", - ) - ) - return; add("re-export", node.source); } else if (node.type === "ImportExpression") add("dynamic-import", node.source); @@ -99,7 +87,14 @@ export const collectWebModuleImports = ( return { imports, state: "parsed", - diagnostics: ast.errors?.map((error) => error.message) ?? [], + diagnostics: [ + ...(ast.errors?.map((error) => error.message) ?? []), + ...(hasUncertainTypeOnlyModuleSideEffect(ast.program) + ? [ + "Compiler options are unavailable; inline type-only imports or re-exports may preserve a module side-effect dependency.", + ] + : []), + ], }; }; diff --git a/tests/acceptance/applications/JavaScriptModuleLookupCli.test.ts b/tests/acceptance/applications/JavaScriptModuleLookupCli.test.ts index cd045a00a..3a7fad35f 100644 --- a/tests/acceptance/applications/JavaScriptModuleLookupCli.test.ts +++ b/tests/acceptance/applications/JavaScriptModuleLookupCli.test.ts @@ -29,6 +29,14 @@ const createModuleLookupFixture = async () => { 'import "ancestor-empty";', 'import "ancestor-blocked";', 'import "ancestor-sub/parser";', + 'import "./.hidden.js";', + 'import "./..pkg.js";', + 'import "./exact.js/.";', + 'import "./exact.js/child/..";', + 'import ".hidden.js";', + 'import "..pkg.js";', + 'import ".";', + 'import "..";', "void fallback;", ].join("\n"), "esm/extensionless.js": "module.exports = 1;", @@ -36,6 +44,10 @@ const createModuleLookupFixture = async () => { "esm/explicit.js.js": "module.exports = 3;", "esm/exact.js": "module.exports = 4;", "esm/actual.cjs": "module.exports = 5;", + "esm/.hidden.js": "module.exports = 'local hidden';", + "esm/..pkg.js": "module.exports = 'local dot package';", + "node_modules/.hidden.js/index.js": "module.exports = 'hidden package';", + "node_modules/..pkg.js/index.js": "module.exports = 'dot package';", "esm/typed.mts": 'import value from "./typed-dep.js"; void value;', "esm/typed-dep.ts": "export default 42;", "cjs/entry.cjs": [ @@ -54,8 +66,15 @@ const createModuleLookupFixture = async () => { 'require("ancestor-empty");', 'require("ancestor-blocked");', 'require("ancestor-sub/parser");', + 'require(".hidden.js");', + 'require("./.hidden.js");', + 'require(".");', + 'require("..");', ].join("\n"), "cjs/directory/index.cjs": "module.exports = 6;", + "cjs/index.js": "module.exports = 'current directory';", + "index.js": "module.exports = 'parent directory';", + "cjs/.hidden.js": "module.exports = 'local hidden';", "cjs/extensionless.cjs": "module.exports = 1;", "cjs/feature.cjs": "module.exports = 2;", "cjs/module-only.mjs": "export default 42;", @@ -79,7 +98,7 @@ const createModuleLookupFixture = async () => { "node_modules/ancestor-blocked/index.js": "module.exports = 9;", "node_modules/ancestor-sub/parser.js": "module.exports = 10;", "esm/runtime-control.mjs": [ - 'const specifiers = ["./extensionless", "./directory", "./explicit.js", "./exact.js", "./actual.cjs?cache=1", "empty-main", "nested-main", "file-only", "ancestor-empty", "ancestor-blocked", "ancestor-sub/parser"];', + 'const specifiers = ["./extensionless", "./directory", "./explicit.js", "./exact.js", "./actual.cjs?cache=1", "empty-main", "nested-main", "file-only", "ancestor-empty", "ancestor-blocked", "ancestor-sub/parser", "./.hidden.js", "./..pkg.js", "./exact.js/.", "./exact.js/child/..", ".hidden.js", "..pkg.js", ".", ".."];', "const results = [];", "for (const specifier of specifiers) {", " try { const value = (await import(specifier)).default; results.push({ specifier, ok: true, value }); }", @@ -88,7 +107,7 @@ const createModuleLookupFixture = async () => { "console.log(JSON.stringify(results));", ].join("\n"), "cjs/runtime-control.cjs": [ - 'const specifiers = ["./extensionless", "./directory", "./feature", "./feature.cjs", "./module-only", "./valid-directory", "./default-js", "./default-json", "empty-main", "nested-main", "file-only", "ancestor-empty", "ancestor-blocked", "ancestor-sub/parser"];', + 'const specifiers = ["./extensionless", "./directory", "./feature", "./feature.cjs", "./module-only", "./valid-directory", "./default-js", "./default-json", "empty-main", "nested-main", "file-only", "ancestor-empty", "ancestor-blocked", "ancestor-sub/parser", ".hidden.js", "./.hidden.js", ".", ".."];', "const results = [];", "for (const specifier of specifiers) {", " try { const value = require(specifier); results.push({ specifier, ok: true, value }); }", @@ -109,6 +128,110 @@ const createModuleLookupFixture = async () => { return root; }; +type RuntimeControlResult = { + readonly specifier: string; + readonly ok: boolean; + readonly code?: string; + readonly value?: unknown; +}; + +const expectNodeRuntimeControls = ( + esmOutput: string, + cjsOutput: string, +): void => { + const esmResults = JSON.parse(esmOutput) as RuntimeControlResult[]; + const cjsResults = JSON.parse(cjsOutput) as RuntimeControlResult[]; + expect(esmResults).toEqual([ + { + specifier: "./extensionless", + ok: false, + code: "ERR_MODULE_NOT_FOUND", + }, + { + specifier: "./directory", + ok: false, + code: "ERR_UNSUPPORTED_DIR_IMPORT", + }, + { + specifier: "./explicit.js", + ok: false, + code: "ERR_MODULE_NOT_FOUND", + }, + { specifier: "./exact.js", ok: true, value: 4 }, + { specifier: "./actual.cjs?cache=1", ok: true, value: 5 }, + { specifier: "empty-main", ok: true, value: 42 }, + { specifier: "nested-main", ok: false, code: "ERR_MODULE_NOT_FOUND" }, + { specifier: "file-only", ok: false, code: "ERR_MODULE_NOT_FOUND" }, + { specifier: "ancestor-empty", ok: false, code: "ERR_MODULE_NOT_FOUND" }, + { + specifier: "ancestor-blocked", + ok: false, + code: "ERR_MODULE_NOT_FOUND", + }, + { + specifier: "ancestor-sub/parser", + ok: false, + code: "ERR_MODULE_NOT_FOUND", + }, + { specifier: "./.hidden.js", ok: true, value: "local hidden" }, + { specifier: "./..pkg.js", ok: true, value: "local dot package" }, + { + specifier: "./exact.js/.", + ok: false, + code: "ERR_UNSUPPORTED_DIR_IMPORT", + }, + { + specifier: "./exact.js/child/..", + ok: false, + code: "ERR_UNSUPPORTED_DIR_IMPORT", + }, + { + specifier: ".hidden.js", + ok: false, + code: "ERR_INVALID_MODULE_SPECIFIER", + }, + { + specifier: "..pkg.js", + ok: false, + code: "ERR_INVALID_MODULE_SPECIFIER", + }, + { specifier: ".", ok: false, code: "ERR_UNSUPPORTED_DIR_IMPORT" }, + { specifier: "..", ok: false, code: "ERR_UNSUPPORTED_DIR_IMPORT" }, + ]); + expect(cjsResults).toEqual([ + { + specifier: "./extensionless", + ok: false, + code: "MODULE_NOT_FOUND", + }, + { + specifier: "./directory", + ok: false, + code: "MODULE_NOT_FOUND", + }, + { + specifier: "./feature", + ok: false, + code: "MODULE_NOT_FOUND", + }, + { specifier: "./feature.cjs", ok: true, value: 2 }, + { specifier: "./module-only", ok: false, code: "MODULE_NOT_FOUND" }, + { specifier: "./valid-directory", ok: true, value: 42 }, + { specifier: "./default-js", ok: true, value: 3 }, + { specifier: "./default-json", ok: true, value: { value: 4 } }, + { specifier: "empty-main", ok: true, value: 42 }, + { specifier: "nested-main", ok: false, code: "MODULE_NOT_FOUND" }, + { specifier: "file-only", ok: true, value: 7 }, + { specifier: "ancestor-empty", ok: true, value: 8 }, + { specifier: "ancestor-blocked", ok: false, code: "MODULE_NOT_FOUND" }, + { specifier: "ancestor-sub/parser", ok: true, value: 10 }, + { specifier: ".hidden.js", ok: true, value: "hidden package" }, + { specifier: "./.hidden.js", ok: true, value: "local hidden" }, + { specifier: ".", ok: true, value: "current directory" }, + { specifier: "..", ok: true, value: "parent directory" }, + ]); +}; + cliTest.for(["CLI", "stdio MCP"] as const)( "matches Node module lookup for ESM and CommonJS through %s", async (surface, { cli }) => { @@ -123,79 +246,7 @@ cliTest.for(["CLI", "stdio MCP"] as const)( timeout: 5_000, }), ]); - const esmResults = JSON.parse(esmControl.stdout) as Array<{ - readonly specifier: string; - readonly ok: boolean; - readonly code?: string; - readonly value?: unknown; - }>; - const cjsResults = JSON.parse(cjsControl.stdout) as Array<{ - readonly specifier: string; - readonly ok: boolean; - readonly code?: string; - readonly value?: unknown; - }>; - expect(esmResults).toEqual([ - { - specifier: "./extensionless", - ok: false, - code: "ERR_MODULE_NOT_FOUND", - }, - { - specifier: "./directory", - ok: false, - code: "ERR_UNSUPPORTED_DIR_IMPORT", - }, - { - specifier: "./explicit.js", - ok: false, - code: "ERR_MODULE_NOT_FOUND", - }, - { specifier: "./exact.js", ok: true, value: 4 }, - { specifier: "./actual.cjs?cache=1", ok: true, value: 5 }, - { specifier: "empty-main", ok: true, value: 42 }, - { specifier: "nested-main", ok: false, code: "ERR_MODULE_NOT_FOUND" }, - { specifier: "file-only", ok: false, code: "ERR_MODULE_NOT_FOUND" }, - { specifier: "ancestor-empty", ok: false, code: "ERR_MODULE_NOT_FOUND" }, - { - specifier: "ancestor-blocked", - ok: false, - code: "ERR_MODULE_NOT_FOUND", - }, - { - specifier: "ancestor-sub/parser", - ok: false, - code: "ERR_MODULE_NOT_FOUND", - }, - ]); - expect(cjsResults).toEqual([ - { - specifier: "./extensionless", - ok: false, - code: "MODULE_NOT_FOUND", - }, - { - specifier: "./directory", - ok: false, - code: "MODULE_NOT_FOUND", - }, - { - specifier: "./feature", - ok: false, - code: "MODULE_NOT_FOUND", - }, - { specifier: "./feature.cjs", ok: true, value: 2 }, - { specifier: "./module-only", ok: false, code: "MODULE_NOT_FOUND" }, - { specifier: "./valid-directory", ok: true, value: 42 }, - { specifier: "./default-js", ok: true, value: 3 }, - { specifier: "./default-json", ok: true, value: { value: 4 } }, - { specifier: "empty-main", ok: true, value: 42 }, - { specifier: "nested-main", ok: false, code: "MODULE_NOT_FOUND" }, - { specifier: "file-only", ok: true, value: 7 }, - { specifier: "ancestor-empty", ok: true, value: 8 }, - { specifier: "ancestor-blocked", ok: false, code: "MODULE_NOT_FOUND" }, - { specifier: "ancestor-sub/parser", ok: true, value: 10 }, - ]); + expectNodeRuntimeControls(esmControl.stdout, cjsControl.stdout); const { graph } = await analyzeJavaScriptDirectory(surface, cli, root); const expectations = [ @@ -211,6 +262,14 @@ cliTest.for(["CLI", "stdio MCP"] as const)( ["import", "ancestor-empty", null, "not-found"], ["import", "ancestor-blocked", null, "not-found"], ["import", "ancestor-sub/parser", null, "not-found"], + ["import", "./.hidden.js", "esm/.hidden.js", "resolved"], + ["import", "./..pkg.js", "esm/..pkg.js", "resolved"], + ["import", ".hidden.js", null, "rejected"], + ["import", "..pkg.js", null, "rejected"], + ["import", ".", null, "not-found"], + ["import", "..", null, "not-found"], + ["import", "./exact.js/.", null, "not-found"], + ["import", "./exact.js/child/..", null, "not-found"], ["require", "./extensionless", null, "not-found"], ["require", "./directory", null, "not-found"], ["require", "./feature", null, "not-found"], @@ -242,19 +301,30 @@ cliTest.for(["CLI", "stdio MCP"] as const)( "node_modules/ancestor-sub/parser.js", "resolved", ], + ["require", ".hidden.js", "node_modules/.hidden.js/index.js", "resolved"], + ["require", "./.hidden.js", "cjs/.hidden.js", "resolved"], + ["require", ".", "cjs/index.js", "resolved"], + ["require", "..", "index.js", "resolved"], ] as const; - for (const [kind, specifier, resolvedPath, status] of expectations) - expect(graph.edges).toContainEqual( + for (const [kind, specifier, resolvedPath, status] of expectations) { + const matchingProperties = graph.edges + .filter((edge) => edge.relation === "imports") + .map((edge) => edge.properties) + .filter( + (properties) => + "module_link_kind" in properties && + properties.module_link_kind === kind && + properties.specifier === specifier, + ); + expect(matchingProperties).toContainEqual( expect.objectContaining({ - relation: "imports", - properties: expect.objectContaining({ - module_link_kind: kind, - specifier, - resolved_path: resolvedPath, - resolution_status: status, - }), + module_link_kind: kind, + specifier, + resolved_path: resolvedPath, + resolution_status: status, }), ); + } }, ); diff --git a/tests/boundary/filesystem/javascript-applications/javascriptModuleRelationships.test.ts b/tests/boundary/filesystem/javascript-applications/javascriptModuleRelationships.test.ts index be51b9bb4..9b4df9992 100644 --- a/tests/boundary/filesystem/javascript-applications/javascriptModuleRelationships.test.ts +++ b/tests/boundary/filesystem/javascript-applications/javascriptModuleRelationships.test.ts @@ -15,6 +15,60 @@ import { } from "../../../support/javascriptApplicationFixture.js"; describe("TypeScript source module loader semantics", () => { + it("keeps whole type-only declarations out of runtime module relationships", async () => { + const root = await createTestTempDirectory("rea-type-module-links-"); + await writeFixtureFiles(root, { + "main.ts": ` + import type { Whole } from "./whole"; + export type { Remote } from "./remote"; + import { type Inline } from "./inline"; + export type LocalType = string; + export interface LocalInterface { value: string } + `, + "whole.ts": "export interface Whole { value: string }", + "remote.ts": "export interface Remote { value: string }", + "inline.ts": "export interface Inline { value: string }", + }); + + const { graph } = await reconstructJavaScriptFixture(root); + const source = findSourceModule(graph, "main.ts"); + const sourceProperties = source?.observations.find( + ({ properties }) => properties.semantic_role === "source-module", + )?.properties; + + expect(sourceProperties).toMatchObject({ + import_relationships: 0, + type_relationships: 5, + export_names: [], + type_export_names: ["LocalInterface", "LocalType", "Remote"], + }); + expect( + graph.edges.filter( + ({ relation, properties }) => + relation === "imports" && + ["./whole", "./remote"].includes(String(properties.specifier)), + ), + ).toEqual([]); + expect( + graph.edges.find( + ({ source_node_id, relation, properties }) => + source_node_id === source?.node_id && + relation === "imports" && + properties.kind === "static-import" && + properties.specifier === "./inline", + ), + ).toBeDefined(); + expect( + source?.observations + .flatMap(({ evidence }) => evidence.limitations) + .join(" "), + ).toContain( + "inline type-only imports or re-exports may preserve a module side-effect dependency", + ); + expect(findExportNode(graph, "main.ts", "LocalType")).toBeUndefined(); + expect(findExportNode(graph, "main.ts", "LocalInterface")).toBeUndefined(); + }); + it("uses emitted TypeScript loader semantics without changing dynamic imports", async () => { const root = await createTestTempDirectory("rea-cts-module-loader-"); await writeFixtureFiles(root, { diff --git a/tests/boundary/filesystem/referenceSourceImport.test.ts b/tests/boundary/filesystem/referenceSourceImport.test.ts index 75eab7acb..8443dfbdd 100644 --- a/tests/boundary/filesystem/referenceSourceImport.test.ts +++ b/tests/boundary/filesystem/referenceSourceImport.test.ts @@ -378,6 +378,44 @@ describe("reference source path selection", () => { }); describe("reference source rooted module specifiers", () => { + it("keeps dot-prefixed bare specifiers external while resolving explicit relative paths", async () => { + const root = await createTestTempDirectory("rea-reference-dot-prefix-"); + await Promise.all([ + writeFile( + join(root, "main.js"), + 'import ".hidden.js"; import "..pkg.js"; import "./.hidden.js"; require(".hidden.js");\n', + ), + writeFile(join(root, ".hidden.js"), "export {};\n"), + writeFile(join(root, "..pkg.js"), "export {};\n"), + ]); + const result = await importTree(root); + if (!result.ok) throw result.error; + expect(result.value.relationships).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + to: ".hidden.js", + kind: "imports", + resolution: "external", + }), + expect.objectContaining({ + to: "..pkg.js", + kind: "imports", + resolution: "external", + }), + expect.objectContaining({ + to: ".hidden.js", + kind: "imports", + resolution: "internal", + }), + expect.objectContaining({ + to: ".hidden.js", + kind: "requires", + resolution: "external", + }), + ]), + ); + }); + it("does not rebase an absolute import onto a coincidentally matching source member", async () => { const root = await createTestTempDirectory("rea-reference-rooted-"); await mkdir(join(root, "src", "outside"), { recursive: true }); From 141d2095f5c6faaf2d3c9b81b0f0f992cacec79c Mon Sep 17 00:00:00 2001 From: Ysopwnable <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:39:18 +0800 Subject: [PATCH 037/159] test(hopper): repair Linux demo setup and native verification (#2638) * fix(verification): align Linux Hopper setup lane with current contracts * fix(verification): retain orphan checks on clean Linux runners * fix(verification): detect Hopper through alternate launcher paths * fix(verification): report identified Hopper leaks before proc errors * fix(verification): distinguish Hopper string recognition from byte fidelity * fix(verification): report full literal decoding coverage explicitly --------- Co-authored-by: Luis Taffelli <86675342+LuisTaffelli@users.noreply.github.com> --- docs/client-verification.md | 33 +++- package.json | 2 +- scripts/lib/clean-linux-hopper-runtime.d.mts | 3 + scripts/lib/clean-linux-hopper-runtime.mjs | 64 +++++++ scripts/lib/real-hopper-search.d.mts | 30 ++++ scripts/lib/real-hopper-search.mjs | 160 +++++++++++++++--- scripts/verify-linux-setup-e2e.mjs | 54 +++--- scripts/verify-real-hopper-linux.mjs | 5 +- tests/conformance/c/fixture.c | 19 ++- .../providers/hopper/realHopperSearch.test.ts | 97 +++++++++++ .../cleanLinuxHopperRuntime.test.ts | 113 +++++++++++++ 11 files changed, 520 insertions(+), 60 deletions(-) create mode 100644 scripts/lib/clean-linux-hopper-runtime.d.mts create mode 100644 scripts/lib/clean-linux-hopper-runtime.mjs create mode 100644 scripts/lib/real-hopper-search.d.mts create mode 100644 tests/conformance/providers/hopper/realHopperSearch.test.ts create mode 100644 tests/process-global/cleanLinuxHopperRuntime.test.ts diff --git a/docs/client-verification.md b/docs/client-verification.md index a02c67edf..42e059382 100644 --- a/docs/client-verification.md +++ b/docs/client-verification.md @@ -298,13 +298,42 @@ subsequent wire recovery and clean shutdown. Caller timeout does not interrupt Hopper's synchronous native operation. Socket boundary tests deterministically cover active and queued expiry, late replies and timer cleanup. -The Linux demo lane remains a separate `verify:hopper:linux` command. Its -lifecycle checks attempt competing CLI launches for the active target and a +The Linux demo lane remains a separate `verify:hopper:linux` command. Hopper +6.4.2's ELF loader did not recognize the original 4,119-byte literal or three +non-ASCII byte strings in the measured fixtures. The lane retains complete raw +byte checks for those literals through source-owned symbols. Recognition of the +original full-length literal is bound to its exact source address; its decoded +object coverage remains explicitly unverified on Linux even when recognized. +Absent non-ASCII recognition also reports decoded-object coverage as unverified. +Recognized non-ASCII and control-text objects must still match their source +bytes, encoding, filters and searches. +A separate 548-byte ASCII literal requires native fragmentation, shortened +provider display, complete decoded fragments and dossier provenance on Linux; +the macOS lane retains the original full-length recognition requirement. These +are observed fixture/build capabilities, not universal Hopper length limits. +Its lifecycle checks attempt competing CLI launches for the active target and a different target, requiring rejection before either can change the owning MCP session's documents or procedures. Lease boundary tests also preserve unresponsive or malformed live endpoints while allowing confirmed stale sockets to recover. +`verify:setup:linux` exercises installation on a clean supported Linux runner as +root. Use a disposable VM: this lane installs Hopper and its Linux runtime +dependencies through the system package manager. Set `REA_VERIFY_SERVER_COMMAND` +to the CLI entrypoint being verified. The command builds and checks source-owned +ELF fixtures before testing the read-only Hopper setup plan, approved +installation, scoped doctor, real CLI/MCP analysis, rejection of a modified +launcher, idempotent setup without requesting a reinstall, and owned child +cleanup. Before installation and after teardown, a clean-runner process scan +also rejects surviving Hopper demo helpers or Xvfb groups, including orphans +whose launcher exited. This scan observes processes without signaling them. +Hopper detection compares executable file identity through `/proc`, covering +alternate or configured launcher paths even after an invoked symlink is removed. +The dedicated runner must have no unrelated Hopper/Xvfb sessions. It requires +the repository's pinned development toolchain and an +existing host C compiler, `file`, `nm`, and `strings`. Use `verify:hopper:linux` +instead when Hopper is already installed. + `verify:hopper:fat` is a separate macOS lane requiring installed Hopper and the existing Xcode clang/lipo toolchain. It compiles arm64/x86-64 thin executables and one- and two-slice FAT32 containers, verifies exact/interior address mappings diff --git a/package.json b/package.json index 6fdb0d9de..cbb173b4f 100644 --- a/package.json +++ b/package.json @@ -207,7 +207,7 @@ "fixtures:harmony": "node scripts/fixtures/harmony/fetch.mjs", "verify:agent": "npm run build:cached && node scripts/verify-agent-experience.mjs", "verify:hopper:linux": "npm run verify:fixtures && npm run build:cached && node scripts/verify-real-hopper-linux.mjs", - "verify:setup:linux": "npm run build:cached && node scripts/verify-linux-setup-e2e.mjs", + "verify:setup:linux": "npm run verify:fixtures && npm run build:cached && node scripts/verify-linux-setup-e2e.mjs", "build:fixtures": "node scripts/build-conformance-fixtures.mjs", "build:ghidra-windows-fixture": "node scripts/create-ghidra-windows-fixture.mjs", "verify:fixtures": "npm run build:fixtures && node scripts/verify-conformance-fixtures.mjs", diff --git a/scripts/lib/clean-linux-hopper-runtime.d.mts b/scripts/lib/clean-linux-hopper-runtime.d.mts new file mode 100644 index 000000000..dd1010aa1 --- /dev/null +++ b/scripts/lib/clean-linux-hopper-runtime.d.mts @@ -0,0 +1,3 @@ +export function assertCleanLinuxHopperRuntime( + launcherPaths?: readonly string[], +): Promise; diff --git a/scripts/lib/clean-linux-hopper-runtime.mjs b/scripts/lib/clean-linux-hopper-runtime.mjs new file mode 100644 index 000000000..27a306fce --- /dev/null +++ b/scripts/lib/clean-linux-hopper-runtime.mjs @@ -0,0 +1,64 @@ +import { execFile } from "node:child_process"; +import { promisify } from "node:util"; +import { readdir, stat } from "node:fs/promises"; + +const execFileAsync = promisify(execFile); + +/** + * Observe leftover demo processes on a dedicated clean Linux runner, including + * groups reparented after their launcher exits. This is not ownership proof and + * never authorizes signaling; unrelated runtime processes also block this lane. + */ +export const assertCleanLinuxHopperRuntime = async ( + launcherPaths = [ + "/opt/hopper/bin/Hopper", + ...(process.env.HOPPER_LAUNCHER_PATH === undefined + ? [] + : [process.env.HOPPER_LAUNCHER_PATH]), + ], +) => { + const { stdout } = await execFileAsync("ps", ["-ax", "-o", "command="]); + if (/\/opt\/hopper\/bin\/Hopper|\bXvfb\b|hopper-demo-x11\.py/u.test(stdout)) + throw new Error("Clean Linux runner retained a Hopper demo process"); + const launchers = []; + for (const path of launcherPaths) { + try { + launchers.push(await stat(path, { bigint: true })); + } catch (error) { + if (error?.code !== "ENOENT") throw error; + } + } + if (launchers.length === 0) return; + // /proc exposes the executed file even when argv[0] names a removed symlink. + // Compare file identity, not a display path or a process name. + let inspectionError; + for (const entry of await readdir("/proc", { withFileTypes: true })) { + if (!entry.isDirectory() || !/^\d+$/u.test(entry.name)) continue; + let executable; + try { + executable = await stat(`/proc/${entry.name}/exe`, { bigint: true }); + } catch (error) { + if (error?.code === "ENOENT" || error?.code === "ESRCH") continue; + // Other users' executables may be unreadable. The setup lane launches + // providers as the caller; unreadable caller-owned processes fail closed. + if (error?.code === "EACCES") { + const owner = await stat(`/proc/${entry.name}`).catch((cause) => { + if (cause?.code === "ENOENT") return undefined; + throw cause; + }); + if (owner === undefined || owner.uid !== process.getuid()) continue; + } + // Finish the inventory so an unrelated restricted process cannot mask a + // positively identified leak. Unknown caller processes still fail closed. + inspectionError ??= error; + continue; + } + if ( + launchers.some( + ({ dev, ino }) => dev === executable.dev && ino === executable.ino, + ) + ) + throw new Error("Clean Linux runner retained a Hopper demo process"); + } + if (inspectionError !== undefined) throw inspectionError; +}; diff --git a/scripts/lib/real-hopper-search.d.mts b/scripts/lib/real-hopper-search.d.mts new file mode 100644 index 000000000..e64031652 --- /dev/null +++ b/scripts/lib/real-hopper-search.d.mts @@ -0,0 +1,30 @@ +type VerifierCall = ( + name: string, + arguments_?: Record, +) => Promise; + +interface StringExpectation { + readonly value: string; + readonly encoding: string; + readonly symbol?: string; + readonly recognitionOptional?: boolean; +} + +/** Verify source bytes, observed decoding, address filters and literal searches. */ +export function verifyHopperStringObjects( + call: VerifierCall, + expectations: readonly StringExpectation[], + inventory?: readonly z.output[] | null, +): Promise<{ + readonly sourceBytesVerified: true; + readonly recognition: readonly { + readonly value: string; + readonly address: string; + readonly state: "observed" | "not_observed"; + readonly decodedObject: "verified" | "unverified"; + }[]; + readonly displayEscapesExcludedFromMatching: true; +}>; +import type { z } from "zod"; + +import type { analysisStringSchema } from "../../src/domain/hopperValues.js"; diff --git a/scripts/lib/real-hopper-search.mjs b/scripts/lib/real-hopper-search.mjs index 0d796f3d2..3456bc6b2 100644 --- a/scripts/lib/real-hopper-search.mjs +++ b/scripts/lib/real-hopper-search.mjs @@ -1,13 +1,16 @@ import assert from "node:assert/strict"; import { mcpTextValue } from "./mcp-verifier-results.mjs"; +const fixtureSymbolPrefix = process.platform === "darwin" ? "_" : ""; + /** Exercise literal/regex search, Unicode identities and cache invalidation in Hopper. */ export async function verifyHopperSearch(call, invalid, procedure) { const address = procedure.address; const original = await call("address_name", { address }); + let stringCoverage; try { await verifySymbolNames(call, invalid, address); - await verifySearchModes(call, address); + stringCoverage = await verifySearchModes(call, address); } finally { await call("set_address_name", { address, name: original }); } @@ -26,7 +29,7 @@ export async function verifyHopperSearch(call, invalid, procedure) { symbolUtf16LimitValidatedBeforeMutation: true, regexAndCaseModes: true, annotationCacheInvalidation: true, - escapedUtf8AndByteStrings: true, + stringCoverage, }; } @@ -103,11 +106,26 @@ async function verifySearchModes(call, address) { [{ address, value: "REA_Search_Upper" }], ); const strings = await call("list_strings"); - const long = "REA_LONG_LITERAL_" + "x".repeat(4096) + "needle"; - const longItem = strings.find((item) => - item.value.startsWith("REA_LONG_LITERAL_"), + const linux = process.platform === "linux"; + const fullLiteral = "REA_LONG_LITERAL_" + "x".repeat(4096) + "needle"; + // Recognition is a provider/loader capability, not an adapter guarantee. + // Always retain the original size in raw-byte coverage; Linux additionally + // requires an extent observed to produce native fragmented string objects. + const fullBytes = Buffer.from(fullLiteral + "\0"); + const raw = await call("read_bytes", { + address: `${fixtureSymbolPrefix}rea_long_literal`, + length: fullBytes.length, + }); + assert.equal(raw.complete, true); + assert.equal(raw.bytes_hex, fullBytes.toString("hex")); + const long = linux + ? "REA_FRAGMENTED_LITERAL_" + "x".repeat(512) + "fragment_tail" + : fullLiteral; + const longItem = strings.find( + (item) => long.startsWith(item.value) && item.value.startsWith("REA_"), ); assert.ok(longItem, "fixture omitted its real long literal"); + if (!linux) assert.equal(longItem.address, raw.address); assert.ok(long.startsWith(longItem.value)); assert.ok(longItem.provider_value.endsWith("…")); assert.equal(longItem.string.termination, "missing"); @@ -158,7 +176,7 @@ async function verifySearchModes(call, address) { } assert.deepEqual( await call("search_strings", { - pattern: "needle", + pattern: linux ? "fragment_tail" : "needle", mode: "literal", case_sensitive: true, }), @@ -176,36 +194,122 @@ async function verifySearchModes(call, address) { }), expected, ); - await verifyHopperStringObjects(call, [ - { value: "REA_UTF8_é_😀", encoding: "utf-8" }, - { value: 'REA_ESCAPED_"\\line\nend\t\r', encoding: "utf-8" }, - { value: "REA_LITERAL_BACKSLASH_\\n", encoding: "utf-8" }, - { value: "REA_LITERAL_…", encoding: "utf-8" }, - { value: "REA_LATIN1_ÿ", encoding: "latin-1" }, - ]); + const objects = await verifyHopperStringObjects( + call, + [ + { + value: "REA_UTF8_é_😀", + encoding: "utf-8", + symbol: `${fixtureSymbolPrefix}rea_utf8_literal`, + recognitionOptional: linux, + }, + { value: 'REA_ESCAPED_"\\line\nend\t\r', encoding: "utf-8" }, + { value: "REA_LITERAL_BACKSLASH_\\n", encoding: "utf-8" }, + { + value: "REA_LITERAL_…", + encoding: "utf-8", + symbol: `${fixtureSymbolPrefix}rea_ellipsis_literal`, + recognitionOptional: linux, + }, + { + value: "REA_LATIN1_ÿ", + encoding: "latin-1", + symbol: `${fixtureSymbolPrefix}rea_latin1_literal`, + recognitionOptional: linux, + }, + ], + strings, + ); + return { + fullLiteralSourceBytes: fullBytes.length, + fragmentedLiteralSourceBytes: Buffer.byteLength(long), + fullLiteralRecognition: { + address: raw.address, + state: strings.some((item) => item.address === raw.address) + ? "observed" + : "not_observed", + // Linux validates the original through raw bytes; its required decoded + // fragment/dossier checks exercise the separate supported-size literal. + decodedObject: linux ? "unverified" : "verified", + }, + ...objects, + }; } /** Compare native string identities and values with the source fixture's bytes. */ -export async function verifyHopperStringObjects(call, expectations) { - const strings = await call("list_strings"); - for (const { value, encoding } of expectations) { - const item = strings.find((candidate) => candidate.value === value); - assert.ok( - item, - `Hopper omitted the decoded string ${JSON.stringify(value)}`, - ); - assert.equal(item.string.encoding, encoding); - assert.equal(item.string.encoding_status, "inferred"); - assert.equal(item.string.termination, "present_or_not_required"); +export async function verifyHopperStringObjects( + call, + expectations, + inventory = null, +) { + // Reuse only the caller's inventory for this unchanged document, never live + // state from an earlier session. Address-filter and search checks remain real. + const strings = inventory ?? (await call("list_strings")); + const recognition = []; + for (const { + value, + encoding, + symbol, + recognitionOptional = false, + } of expectations) { const expectedBytes = Buffer.from( value + "\0", encoding.replaceAll("-", ""), ); - assert.equal(item.string.byte_length, expectedBytes.length); - const bytes = await call("read_bytes", { + let sourceBytes; + if (symbol !== undefined) { + const raw = await call("read_bytes", { + address: symbol, + length: expectedBytes.length, + }); + sourceBytes = raw; + assert.equal(raw.complete, true); + assert.equal(raw.bytes_hex, expectedBytes.toString("hex")); + } + const item = strings.find((candidate) => + sourceBytes === undefined + ? candidate.value === value + : candidate.address === sourceBytes.address, + ); + if (item === undefined && recognitionOptional) { + assert.ok( + symbol, + "optional recognition requires independent source-byte coverage", + ); + recognition.push({ + value, + address: sourceBytes.address, + state: "not_observed", + decodedObject: "unverified", + }); + continue; + } + assert.ok( + item, + `Hopper omitted the decoded string ${JSON.stringify(value)}`, + ); + assert.equal( + item.value, + value, + "native string decoding changed the source value", + ); + recognition.push({ + value, address: item.address, - length: expectedBytes.length, + state: "observed", + decodedObject: "verified", }); + assert.equal(item.string.encoding, encoding); + assert.equal(item.string.encoding_status, "inferred"); + assert.equal(item.string.termination, "present_or_not_required"); + assert.equal(item.string.byte_length, expectedBytes.length); + const bytes = + sourceBytes ?? + (await call("read_bytes", { + address: item.address, + length: expectedBytes.length, + })); + assert.equal(bytes.complete, true); assert.equal(bytes.bytes_hex, expectedBytes.toString("hex")); assert.deepEqual(await call("list_strings", { address: item.address }), [ item, @@ -225,7 +329,7 @@ export async function verifyHopperStringObjects(call, expectations) { ); return { sourceBytesVerified: true, - inferredEncodingReported: true, + recognition, displayEscapesExcludedFromMatching: true, }; } diff --git a/scripts/verify-linux-setup-e2e.mjs b/scripts/verify-linux-setup-e2e.mjs index 8b57fbacc..e08e8b160 100644 --- a/scripts/verify-linux-setup-e2e.mjs +++ b/scripts/verify-linux-setup-e2e.mjs @@ -12,6 +12,7 @@ import { tmpdir } from "node:os"; import { promisify } from "node:util"; import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs"; +import { assertCleanLinuxHopperRuntime } from "./lib/clean-linux-hopper-runtime.mjs"; const execFileAsync = promisify(execFile); const verifierRun = createVerifierRun(); @@ -19,12 +20,12 @@ const command = process.env.REA_VERIFY_SERVER_COMMAND; if (process.platform !== "linux" || process.getuid?.() !== 0) throw new Error("Linux setup verification requires a root Linux runner"); if (!command) throw new Error("REA_VERIFY_SERVER_COMMAND is required"); +await assertCleanLinuxHopperRuntime(); try { await access("/opt/hopper/bin/Hopper"); throw new Error("Linux setup verification requires a clean runner"); } catch (error) { - if (error instanceof Error && error.message.includes("clean runner")) - throw error; + if (error?.code !== "ENOENT") throw error; } const runJson = async (args) => { @@ -35,14 +36,25 @@ const runJson = async (args) => { return JSON.parse(stdout); }; -const plan = await runJson(["setup", "--json"]); -if (plan.status !== "needs_confirmation") - throw new Error("setup did not produce a read-only confirmation plan"); +const plan = await runJson([ + "setup", + "--install-hopper", + "--dry-run", + "--json", +]); +if ( + plan.status !== "planned" || + !plan.plannedActions.some(({ kind }) => kind === "install_hopper") || + plan.appliedActions.length !== 0 +) + throw new Error("setup did not produce a read-only Hopper installation plan"); await access("/opt/hopper/bin/Hopper").then( () => { throw new Error("setup plan mutated the clean runner"); }, - () => undefined, + (error) => { + if (error?.code !== "ENOENT") throw error; + }, ); const installed = await runJson([ @@ -58,7 +70,7 @@ if ( throw new Error( `approved setup did not become ready: ${JSON.stringify(installed)}`, ); -const doctor = await runJson(["doctor", "--json"]); +const doctor = await runJson(["doctor", "--provider", "hopper", "--json"]); if (doctor.healthy !== true) throw new Error(`doctor is unhealthy: ${JSON.stringify(doctor)}`); @@ -67,8 +79,6 @@ await execFileAsync(command, ["analyze", "/usr/bin/true"], { timeout: 180_000, maxBuffer: 10 * 1024 * 1024, }); -process.env.HOPPER_TARGET_PATH = "/usr/bin/true"; -process.env.HOPPER_SECOND_TARGET_PATH = "/usr/bin/false"; process.env.REA_VERIFY_SERVER_ARGS_JSON = '["mcp"]'; await import("./verify-real-hopper-linux.mjs"); @@ -95,28 +105,24 @@ try { await rm(modifiedDirectory, { recursive: true, force: true }); } -const repeated = await runJson([ - "setup", - "--yes", - "--install-hopper", - "--json", -]); +const repeated = await runJson(["setup", "--yes", "--json"]); if ( repeated.status !== "ready" || - repeated.appliedActions.includes("installed_hopper") + repeated.plannedActions.length !== 0 || + repeated.appliedActions.length !== 0 ) throw new Error("a second setup was not idempotent"); -const { stdout: processes } = await execFileAsync("ps", [ - "-ax", - "-o", - "command=", -]); -if (/\/opt\/hopper\/bin\/Hopper|\bXvfb\b|hopper-demo-x11\.py/u.test(processes)) - throw new Error("Linux setup verification leaked a Hopper demo session"); +await assertCleanLinuxHopperRuntime(); +const completedVerifierRun = await completeVerifierRun(verifierRun); +if ( + completedVerifierRun.process_lineage.status !== "verified" || + completedVerifierRun.process_lineage.descendants.length !== 0 +) + throw new Error("Linux setup verification did not prove owned child cleanup"); console.log( JSON.stringify({ - verifier_run: await completeVerifierRun(verifierRun), + verifier_run: completedVerifierRun, status: "verified", setup: true, cli: true, diff --git a/scripts/verify-real-hopper-linux.mjs b/scripts/verify-real-hopper-linux.mjs index 854a58ce0..951b71078 100644 --- a/scripts/verify-real-hopper-linux.mjs +++ b/scripts/verify-real-hopper-linux.mjs @@ -11,7 +11,8 @@ createVerifierRun(); if (process.platform !== "linux") throw new Error("Linux real-Hopper verification must run on Linux"); -const launcher = process.env.HOPPER_LAUNCHER_PATH ?? "/opt/hopper/bin/Hopper"; +const configuredLauncher = process.env.HOPPER_LAUNCHER_PATH; +const launcher = configuredLauncher ?? "/opt/hopper/bin/Hopper"; await access(launcher); await access("/usr/bin/Xvfb"); await access("/usr/bin/xauth"); @@ -28,5 +29,7 @@ try { process.env.HOPPER_LAUNCHER_PATH = alternateLauncher; await import("./verify-real-hopper.mjs"); } finally { + if (configuredLauncher === undefined) delete process.env.HOPPER_LAUNCHER_PATH; + else process.env.HOPPER_LAUNCHER_PATH = configuredLauncher; await rm(alternateDirectory, { recursive: true, force: true }); } diff --git a/tests/conformance/c/fixture.c b/tests/conformance/c/fixture.c index 6708ac9dd..e1dff9a9c 100644 --- a/tests/conformance/c/fixture.c +++ b/tests/conformance/c/fixture.c @@ -5,13 +5,24 @@ volatile int rea_c_global = 7; #define REA_LONG_CHUNK "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" #define REA_LONG_256 REA_LONG_CHUNK REA_LONG_CHUNK REA_LONG_CHUNK REA_LONG_CHUNK #define REA_LONG_1024 REA_LONG_256 REA_LONG_256 REA_LONG_256 REA_LONG_256 +/* Keep the full-size literal even when a loader does not type it as a string. */ +__attribute__((used)) const char rea_long_literal[] = + "REA_LONG_LITERAL_" REA_LONG_1024 REA_LONG_1024 REA_LONG_1024 REA_LONG_1024 "needle"; +/* Linux Hopper 6.4.2 recognizes this extent and splits it into native objects. */ +__attribute__((used)) const char rea_fragmented_literal[] = + "REA_FRAGMENTED_LITERAL_" REA_LONG_256 REA_LONG_256 "fragment_tail"; +__attribute__((used)) const char rea_utf8_literal[] = "REA_UTF8_é_😀"; +__attribute__((used)) const char rea_ellipsis_literal[] = "REA_LITERAL_…"; +__attribute__((used)) const char rea_latin1_literal[] = "REA_LATIN1_\xff"; + __attribute__((noinline, used)) void rea_long_literal_procedure(void) { - puts("REA_LONG_LITERAL_" REA_LONG_1024 REA_LONG_1024 REA_LONG_1024 REA_LONG_1024 "needle"); - puts("REA_UTF8_é_😀"); + puts(rea_long_literal); + puts(rea_fragmented_literal); + puts(rea_utf8_literal); puts("REA_ESCAPED_\"\\line\nend\t\r"); puts("REA_LITERAL_BACKSLASH_\\n"); - puts("REA_LITERAL_…"); - puts("REA_LATIN1_\xff"); + puts(rea_ellipsis_literal); + puts(rea_latin1_literal); } __attribute__((noinline, used)) int rea_leaf(int value) { diff --git a/tests/conformance/providers/hopper/realHopperSearch.test.ts b/tests/conformance/providers/hopper/realHopperSearch.test.ts new file mode 100644 index 000000000..10aedb433 --- /dev/null +++ b/tests/conformance/providers/hopper/realHopperSearch.test.ts @@ -0,0 +1,97 @@ +import { expect, it } from "vitest"; + +import { verifyHopperStringObjects } from "../../../../scripts/lib/real-hopper-search.mjs"; + +const value = "REA_UTF8_é_😀"; +const address = "0x402020"; +const bytes = Buffer.from(`${value}\0`); +const item = { + address, + value, + string: { + encoding: "utf-8", + encoding_status: "inferred" as const, + termination: "present_or_not_required" as const, + byte_length: bytes.length, + }, +}; +const expectation = { + value, + encoding: "utf-8", + symbol: "rea_utf8_literal", + recognitionOptional: true, +}; + +it("keeps source identity and unknown decoding when the same text exists elsewhere", async () => { + const report = await verifyHopperStringObjects( + fixtureCall(), + [expectation], + [{ ...item, address: "0x403000" }], + ); + expect(report.recognition).toEqual([ + { value, address, state: "not_observed", decodedObject: "unverified" }, + ]); +}); + +it("reuses source bytes and inventory while checking real address filters and searches", async () => { + const calls: string[] = []; + const report = await verifyHopperStringObjects( + fixtureCall(calls), + [expectation], + [item], + ); + expect(report.recognition).toEqual([ + { value, address, state: "observed", decodedObject: "verified" }, + ]); + expect(calls).toEqual([ + "read_bytes:rea_utf8_literal", + `list_strings:${address}`, + `search_strings:${value}`, + "search_strings:\\xC3\\xA9", + ]); +}); + +it.each([ + ["changed source bytes", { bytes_hex: "00" }, item], + ["incomplete source read", { complete: false }, item], + ["wrong decoded value", {}, { ...item, value: "wrong" }], + [ + "wrong decoded extent", + {}, + { ...item, string: { ...item.string, byte_length: 1 } }, + ], +])( + "rejects %s even when native recognition is optional", + async (_label, override, decoded) => { + await expect( + verifyHopperStringObjects( + fixtureCall([], override), + [expectation], + [decoded], + ), + ).rejects.toThrow(); + }, +); + +function fixtureCall( + calls: string[] = [], + override: Record = {}, +) { + return async (name: string, arguments_: Record = {}) => { + calls.push( + `${name}:${String(arguments_.address ?? arguments_.pattern ?? "")}`, + ); + if (name === "read_bytes") + return { + address, + bytes_hex: bytes.toString("hex"), + complete: true, + ...override, + }; + if (name === "list_strings" && arguments_.address === address) + return [item]; + if (name === "search_strings") + return arguments_.pattern === value ? [item] : []; + throw new Error(`Unmodeled verifier call: ${name}`); + }; +} diff --git a/tests/process-global/cleanLinuxHopperRuntime.test.ts b/tests/process-global/cleanLinuxHopperRuntime.test.ts new file mode 100644 index 000000000..d356f5abf --- /dev/null +++ b/tests/process-global/cleanLinuxHopperRuntime.test.ts @@ -0,0 +1,113 @@ +import { execFile } from "node:child_process"; +import { randomUUID } from "node:crypto"; +import { chmod, copyFile, readFile, symlink, unlink } from "node:fs/promises"; +import { join } from "node:path"; +import { promisify } from "node:util"; + +import { expect, it } from "vitest"; + +import { assertCleanLinuxHopperRuntime } from "../../scripts/lib/clean-linux-hopper-runtime.mjs"; +import { completeVerifierRun } from "../../scripts/lib/verifier-run.mjs"; +import { cleanupOwnedProcessGroup } from "../../src/process/ProcessOwnership.js"; +import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js"; + +const execFileAsync = promisify(execFile); + +it + .runIf(process.platform === "linux") + .each(["demo helper", "alternate launcher"])( + "rejects a reparented %s without signaling it", + async (mode) => { + const runId = randomUUID(); + const childCode = + "process.title = 'Xvfb'; setInterval(() => {}, 1000); process.send('ready');"; + const directory = await createTestTempDirectory("rea-clean-runtime-test-"); + const launcher = join(directory, "Hopper"); + const alternate = join(directory, "Hopper-alternate"); + const alternateMode = mode === "alternate launcher"; + if (alternateMode) { + await copyFile("/usr/bin/sleep", launcher); + await chmod(launcher, 0o700); + await symlink(launcher, alternate); + } + const command = alternateMode ? alternate : process.execPath; + const arguments_ = alternateMode ? ["30"] : ["-e", childCode]; + const parentCode = ` + const { spawn } = require('node:child_process'); + const child = spawn(${JSON.stringify(command)}, ${JSON.stringify(arguments_)}, { + detached: true, stdio: ${JSON.stringify(alternateMode ? "ignore" : ["ignore", "ignore", "ignore", "ipc"])} + }); + child.once('error', () => process.exit(1)); + child.once(${JSON.stringify(alternateMode ? "spawn" : "message")}, () => { + process.stdout.write(JSON.stringify({ pid: child.pid, parentPid: process.pid })); + if (child.connected) child.disconnect(); + child.unref(); + }); + `; + const parent = execFileAsync(process.execPath, ["-e", parentCode], { + env: { ...process.env, REA_PROCESS_RUN_ID: runId }, + timeout: 10_000, + }); + const parentPid = parent.child.pid; + if (parentPid === undefined) + throw new Error("Fixture parent did not spawn"); + let orphanPid: number | undefined; + try { + const { stdout } = await parent; + const { pid } = JSON.parse(stdout) as { pid: number }; + orphanPid = pid; + const { stdout: coordinates } = await execFileAsync("ps", [ + "-p", + String(pid), + "-o", + "ppid=,pgid=", + ]); + const [currentParent, group] = coordinates + .trim() + .split(/\s+/u) + .map(Number); + expect(currentParent).not.toBe(parentPid); + expect(group).toBe(pid); + + const lineage = await completeVerifierRun({ + run_id: runId, + verifier_pid: process.pid, + parent_pid: process.ppid, + }); + expect(lineage.process_lineage.status).toBe("verified"); + if (lineage.process_lineage.status === "verified") + expect( + lineage.process_lineage.descendants.some((p) => p.pid === pid), + ).toBe(false); + + if (alternateMode) { + const commandLine = await readFile( + `/proc/${String(pid)}/cmdline`, + "utf8", + ); + expect(commandLine.split("\0")[0]).toBe(alternate); + await unlink(alternate); + } + await expect( + assertCleanLinuxHopperRuntime(alternateMode ? [launcher] : []), + ).rejects.toThrow("Clean Linux runner retained a Hopper demo process"); + // Observation must not terminate the fixture, even though it resembles Xvfb. + const { stdout: alive } = await execFileAsync("ps", [ + "-p", + String(pid), + "-o", + "stat=", + ]); + expect(alive.trim()).not.toMatch(/^Z/u); + } finally { + if (orphanPid !== undefined) { + const cleanup = await cleanupOwnedProcessGroup({ + runId, + leaderPid: orphanPid, + processGroupId: orphanPid, + }); + expect(cleanup.cleaned, JSON.stringify(cleanup)).toBe(true); + } + } + }, +); From 7017c4d68cab49aed998283e09069d9d791e1b0b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=90=B4=E4=BD=B3=E6=B5=A9Alben?= <35291372+XiaoBinGan@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:39:29 +0800 Subject: [PATCH 038/159] fix(artifacts): read every ZIP package family extraction is asked to open (#2493) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit extract_artifact carried its own copy of the ZIP package format list with five of the eight members the domain owner declares, and used it both as the guard that refuses a format without an extraction reader and as the selector for the reader itself. A .hap, .hsp or App Pack target was therefore refused with "Artifact format has no extraction reader" even though the next branch builds a ZipArtifactReader for exactly those formats, while inventory_artifact classifies the same path through the owner and reads it. Import isZipPackageFormat from the domain owner and delete the duplicate list and its predicate, removing the second representation instead of reconciling it. The reader branch now narrows to ZipPackageFormat, which is what the ZipArtifactReader constructor already requires. Verified with 41 boundary extraction, ZIP integrity and detection tests, check:fast and verify:test-discovery on macOS arm64. Reverting only the source change fails exactly the three added families and leaves the zip control green. Co-authored-by: 吴佳浩 --- .../extraction/ArtifactExtraction.ts | 12 ++------ .../artifactProvider.extractionFormat.test.ts | 29 +++++++++++++++++++ 2 files changed, 32 insertions(+), 9 deletions(-) diff --git a/src/artifacts/extraction/ArtifactExtraction.ts b/src/artifacts/extraction/ArtifactExtraction.ts index 6306a08bf..ee92954eb 100644 --- a/src/artifacts/extraction/ArtifactExtraction.ts +++ b/src/artifacts/extraction/ArtifactExtraction.ts @@ -17,6 +17,7 @@ import { DirectoryArtifactReader } from "../DirectoryArtifactReader.js"; import { SafeOutputTree } from "../SafeOutputTree.js"; import { SafeOutputTreeCreationFailure } from "../SafeOutputTreeCreationFailure.js"; import { ZipArtifactReader } from "../ZipArtifactReader.js"; +import { isZipPackageFormat } from "../../domain/zipPackageFormat.js"; import { MachOSliceArtifactReader } from "../MachOSliceArtifactReader.js"; import { artifactExtractionResultSchema, @@ -400,13 +401,6 @@ interface ExtractionInventory { readonly integrityContradictions: readonly IntegrityContradiction[]; } -const ZIP_FORMATS = ["ipa", "apk", "msix", "appx", "zip"] as const; - -const isZipFormat = ( - format: BinaryTarget["format"], -): format is (typeof ZIP_FORMATS)[number] => - (ZIP_FORMATS as readonly string[]).includes(format); - /** * Refuse a format without an extraction reader before inventory work starts. * The target kind, not the host, is unsupported by extraction. The error keeps @@ -422,7 +416,7 @@ const requireExtractableFormat = async ( !directory && format !== "asar" && !isMachOFormat(format) && - !isZipFormat(format) + !isZipPackageFormat(format) ) throw new AnalysisUnsupportedTargetError( "extract_artifact", @@ -443,7 +437,7 @@ const createReader = async ( if (await requireExtractableFormat(path, input)) return new DirectoryArtifactReader(path); if (format === "asar") return new AsarArtifactReader(path); - if (isZipFormat(format)) return new ZipArtifactReader(path, format); + if (isZipPackageFormat(format)) return new ZipArtifactReader(path, format); return new MachOSliceArtifactReader(path, input.environment); }; diff --git a/tests/boundary/providers/native/artifactProvider.extractionFormat.test.ts b/tests/boundary/providers/native/artifactProvider.extractionFormat.test.ts index 81b3cce52..e4accd098 100644 --- a/tests/boundary/providers/native/artifactProvider.extractionFormat.test.ts +++ b/tests/boundary/providers/native/artifactProvider.extractionFormat.test.ts @@ -3,10 +3,12 @@ import { join } from "node:path"; import { describe, expect, it } from "vitest"; +import { writeOrderedZip } from "../../../fixtures/artifactEntryOrder.js"; import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js"; import { ArtifactProvider } from "../../../../src/artifacts/ArtifactProvider.js"; import { artifactExtractionExecutionSchema } from "../../../../src/contracts/artifactToolContracts.js"; +import { artifactExtractionResultSchema } from "../../../../src/domain/artifactGraph.js"; import type { BinaryTarget } from "../../../../src/domain/binaryTargetTypes.js"; describe("artifact extraction format support", () => { @@ -42,6 +44,33 @@ describe("artifact extraction format support", () => { }); await expect(access(output)).rejects.toThrow(); }); + + it.each(["zip", "hap", "hsp", "app-pack"] as const)( + "extracts the ZIP-backed %s package family that inventory already supports", + async (format) => { + const root = await createTestTempDirectory("rea-extract-zip-family-"); + const archive = join( + root, + `Payload.${format === "app-pack" ? "app" : format}`, + ); + await writeOrderedZip(archive, ["payload.txt"]); + const result = await new ArtifactProvider(process.env) + .createClient(archiveTarget(archive, format)) + .execute( + "extract_artifact", + artifactExtractionExecutionSchema.parse({ + output_root: join(root, "output"), + }), + ); + if (!result.ok) throw result.error; + const extracted = artifactExtractionResultSchema.parse( + result.value.result, + ); + expect( + extracted.artifacts.map(({ relative_path }) => relative_path), + ).toEqual(["payload.txt"]); + }, + ); }); const archiveTarget = ( From 26bffb94bf80355c35e22a98a8fc629c6210307e Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:39:49 +0800 Subject: [PATCH 039/159] perf(evidence): reuse authenticated comparisons through aggregation --- src/domain/changedBehavior.test.ts | 42 +++--- src/domain/changedBehavior.ts | 133 ++++++++++++------ src/server/registerInvestigationTools.ts | 30 ++-- .../mcp/processEvidenceIdentityMcp.test.ts | 17 ++- 4 files changed, 146 insertions(+), 76 deletions(-) diff --git a/src/domain/changedBehavior.test.ts b/src/domain/changedBehavior.test.ts index cc19c4c50..6d99e828f 100644 --- a/src/domain/changedBehavior.test.ts +++ b/src/domain/changedBehavior.test.ts @@ -22,6 +22,9 @@ const source = (digit: string): Evidence => const left = source("1"); const right = source("2"); +const aggregate = (comparisons: readonly unknown[]) => + findChangedBehavior({ comparisons }); + const comparison = ( operation: "compare_process_captures" | "compare_artifacts", result: JsonValue, @@ -136,9 +139,9 @@ const expectInvalidSummary = (result: ChangedBehaviorResult): void => { describe("changed behavior", () => { it("accepts process comparison Evidence", () => { - const result = findChangedBehavior([ + const result = aggregate([ comparison("compare_process_captures", processResult()), - ]); + ]).result; expect(result.behavior_status).toBe("observed_unchanged"); }); @@ -152,7 +155,7 @@ describe("changed behavior", () => { }), ); - expect(findChangedBehavior([evidence])).toMatchObject({ + expect(aggregate([evidence]).result).toMatchObject({ behavior_status: "observed_changed", summary: { observed_changes: 2 }, findings: { @@ -172,7 +175,7 @@ describe("changed behavior", () => { "compare_process_captures", processResult({ status: "changed", terminal: "changed" }), ); - const result = findChangedBehavior([evidence]); + const result = aggregate([evidence]).result; expect(changedBehaviorResultSchema.parse(result)).toMatchObject({ behavior_status: "observed_changed", summary: { observed_changes: 1, static_candidates: 0 }, @@ -193,7 +196,7 @@ describe("changed behavior", () => { it("keeps static differences as candidates, never runtime observations", () => { const evidence = comparison("compare_artifacts", artifactResult()); - const result = findChangedBehavior([evidence]); + const result = aggregate([evidence]).result; expect(result).toMatchObject({ behavior_status: "unknown", summary: { observed_changes: 0, static_candidates: 1 }, @@ -217,17 +220,17 @@ describe("changed behavior", () => { "compare_process_captures", processResult({ status: "unknown", exit: "unknown" }), ); - const result = findChangedBehavior([changed, unknown]); + const result = aggregate([changed, unknown]).result; expect(result).toMatchObject({ behavior_status: "unknown", findings: { items: expect.any(Array) }, }); - expect(findChangedBehavior([changed, unknown])).toEqual(result); + expect(aggregate([changed, unknown]).result).toEqual(result); }); it("uses complete inline artifact deltas as static candidates", () => { const complete = comparison("compare_artifacts", artifactResult()); - expect(findChangedBehavior([complete]).findings.items).toEqual( + expect(aggregate([complete]).result.findings.items).toEqual( expect.arrayContaining([ expect.objectContaining({ dimension: "artifact:main.js" }), ]), @@ -236,7 +239,7 @@ describe("changed behavior", () => { it("rejects bundles, duplicate comparisons, and malformed exact results", () => { const valid = comparison("compare_process_captures", processResult()); - expect(() => findChangedBehavior([valid, valid])).toThrow( + expect(() => aggregate([valid, valid])).toThrow( /duplicate comparison Evidence/u, ); const bundle = createEvidence( @@ -252,25 +255,25 @@ describe("changed behavior", () => { evidenceLinks: [left.evidence_id, right.evidence_id], }, ); - expect(() => findChangedBehavior([bundle])).toThrow( + expect(() => aggregate([bundle])).toThrow( /requires process, artifact, or function/u, ); const malformed = comparison("compare_process_captures", { status: "unchanged", }); - expect(() => findChangedBehavior([malformed])).toThrow(); + expect(() => aggregate([malformed])).toThrow(); const contradictory = comparison( "compare_process_captures", processResult({ status: "unchanged", terminal: "changed" }), ); - expect(() => findChangedBehavior([contradictory])).toThrow( + expect(() => aggregate([contradictory])).toThrow( /contradicts its dimensions/u, ); const danglingNested = comparison( "compare_artifacts", artifactResult([`ev_${"f".repeat(64)}`, right.evidence_id]), ); - expect(() => findChangedBehavior([danglingNested])).toThrow( + expect(() => aggregate([danglingNested])).toThrow( /outside its top-level closure/u, ); }); @@ -299,10 +302,15 @@ describe("changed behavior aggregation", () => { ), ); - const result = findChangedBehavior(comparisons); + const result = aggregate(comparisons); - expect(result.summary.observed_changes).toBe(comparisons.length); - expect(result.evidence_links).toHaveLength(comparisons.length * 201 + 2); - expect(result.findings.items[0]?.evidence_links).toHaveLength(203); + expect(result.result.summary.observed_changes).toBe(comparisons.length); + expect(result.result.evidence_links).toHaveLength( + comparisons.length * 201 + 2, + ); + expect(result.result.findings.items[0]?.evidence_links).toHaveLength(203); + expect(result.comparisonEvidenceIds).toEqual( + comparisons.map(({ evidence_id }) => evidence_id), + ); }); }); diff --git a/src/domain/changedBehavior.ts b/src/domain/changedBehavior.ts index b785cdd23..56e9b6ad9 100644 --- a/src/domain/changedBehavior.ts +++ b/src/domain/changedBehavior.ts @@ -91,6 +91,22 @@ export const changedBehaviorResultSchema = z export type ChangedBehaviorResult = z.infer; type Finding = z.infer; type RuntimeStatus = z.infer["status"]; +type ParsedComparison = + | { + readonly operation: "compare_process_captures"; + readonly evidence: Evidence; + readonly result: z.infer; + } + | { + readonly operation: "compare_functions"; + readonly evidence: Evidence; + readonly result: z.infer; + } + | { + readonly operation: "compare_artifacts"; + readonly evidence: Evidence; + readonly result: z.infer; + }; const EXPECTED_PROVIDERS = { compare_process_captures: { @@ -117,24 +133,37 @@ const acceptsPredicate = ( ): boolean => expected.some((predicate) => predicate === actual); /** Find observed runtime changes and separately report static change candidates. */ -export const findChangedBehavior = ( - comparisonsInput: unknown, -): ChangedBehaviorResult => { - const parsedInput = changedBehaviorInputSchema.parse({ - comparisons: comparisonsInput, - }); - const comparisons = parsedInput.comparisons; +interface FoundChangedBehavior { + readonly result: ChangedBehaviorResult; + /** Authenticated IDs retained from the comparisons used to compute result. */ + readonly comparisonEvidenceIds: readonly string[]; +} + +export const findChangedBehavior = (input: unknown): FoundChangedBehavior => { + // The public input schema remains available to clients; this internal + // admission only checks the list shape because parseEvidence below owns + // validation and authentication of each retained Evidence item. + const { comparisons } = z + .strictObject({ comparisons: z.array(z.unknown()).min(1) }) + .parse(input); const evidence = comparisons.map(parseComparisonEvidence); assertUnique( - evidence.map(({ evidence_id: id }) => id), + evidence.map(({ evidence: { evidence_id: id } }) => id), "comparison Evidence", ); const findings = evidence.flatMap(findingsFor).sort(compareFindings); - const runtimeStatuses = evidence.flatMap(runtimeStatus); - const links = uniqueSorted( - evidence.flatMap((item) => [item.evidence_id, ...item.evidence_links]), + const runtimeStatuses = evidence.flatMap((comparison) => + comparison.operation === "compare_process_captures" + ? [comparison.result.status] + : [], ); - return changedBehaviorResultSchema.parse({ + const links = uniqueSorted( + evidence.flatMap(({ evidence: item }) => [ + item.evidence_id, + ...item.evidence_links, + ]), + ); + const result = changedBehaviorResultSchema.parse({ behavior_status: behaviorStatus(runtimeStatuses), summary: { observed_changes: count(findings, "observed_change"), @@ -147,7 +176,7 @@ export const findChangedBehavior = ( }, evidence_links: links, limitations: uniqueSorted([ - ...evidence.flatMap(({ limitations }) => limitations), + ...evidence.flatMap(({ evidence: item }) => item.limitations), ...(evidence.some( ({ operation }) => operation !== "compare_process_captures", ) @@ -160,9 +189,15 @@ export const findChangedBehavior = ( : []), ]), }); + return { + result, + comparisonEvidenceIds: evidence.map( + ({ evidence: { evidence_id } }) => evidence_id, + ), + }; }; -const parseComparisonEvidence = (input: unknown): Evidence => { +const parseComparisonEvidence = (input: unknown): ParsedComparison => { const evidence = parseEvidence(input); if (!(evidence.operation in EXPECTED_PROVIDERS)) throw new TypeError( @@ -198,12 +233,17 @@ const parseComparisonEvidence = (input: unknown): Evidence => { throw new TypeError( "Comparison Evidence must cite both selected source observations", ); - parseResult(evidence); - return evidence; + return parseResult(evidence, operation); }; -const parseResult = (evidence: Evidence): void => { - if (evidence.operation === "compare_process_captures") { +const parseResult = ( + evidence: Evidence, + operation: + | "compare_process_captures" + | "compare_artifacts" + | "compare_functions", +): ParsedComparison => { + if (operation === "compare_process_captures") { const result = processCaptureComparisonSchema.parse( evidence.normalized_result, ); @@ -214,9 +254,9 @@ const parseResult = (evidence: Evidence): void => { throw new TypeError( "Process comparison status contradicts its dimensions", ); - return; + return { operation, evidence, result }; } - if (evidence.operation === "compare_functions") { + if (operation === "compare_functions") { const result = functionComparisonResultSchema.parse( evidence.normalized_result, ); @@ -224,7 +264,7 @@ const parseResult = (evidence: Evidence): void => { evidence, result.dimensions.flatMap(({ evidence_links: links }) => links), ); - return; + return { operation, evidence, result }; } const result = artifactComparisonResultSchema.parse( evidence.normalized_result, @@ -233,6 +273,7 @@ const parseResult = (evidence: Evidence): void => { evidence, result.changes.flatMap(({ evidence_links: links }) => links), ); + return { operation, evidence, result }; }; const assertNestedLinks = ( @@ -246,18 +287,21 @@ const assertNestedLinks = ( ); }; -const findingsFor = (evidence: Evidence): Finding[] => { - if (evidence.operation === "compare_process_captures") - return processFindings(evidence); - if (evidence.operation === "compare_functions") - return functionFindings(evidence); - return artifactFindings(evidence); +const findingsFor = (comparison: ParsedComparison): Finding[] => { + if (comparison.operation === "compare_process_captures") + return processFindings(comparison); + if (comparison.operation === "compare_functions") + return functionFindings(comparison); + return artifactFindings(comparison); }; -const processFindings = (evidence: Evidence): Finding[] => { - const result = processCaptureComparisonSchema.parse( - evidence.normalized_result, - ); +const processFindings = ( + comparison: Extract< + ParsedComparison, + { readonly operation: "compare_process_captures" } + >, +): Finding[] => { + const { evidence, result } = comparison; const dimensions = [ ["terminal", result.terminal, "runtime"], ["interaction", result.interaction, "runtime"], @@ -283,10 +327,13 @@ const processFindings = (evidence: Evidence): Finding[] => { ); }; -const functionFindings = (evidence: Evidence): Finding[] => { - const result = functionComparisonResultSchema.parse( - evidence.normalized_result, - ); +const functionFindings = ( + comparison: Extract< + ParsedComparison, + { readonly operation: "compare_functions" } + >, +): Finding[] => { + const { evidence, result } = comparison; return result.dimensions.flatMap((dimension) => dimension.status === "unchanged" ? [] @@ -307,10 +354,13 @@ const functionFindings = (evidence: Evidence): Finding[] => { ); }; -const artifactFindings = (evidence: Evidence): Finding[] => { - const result = artifactComparisonResultSchema.parse( - evidence.normalized_result, - ); +const artifactFindings = ( + comparison: Extract< + ParsedComparison, + { readonly operation: "compare_artifacts" } + >, +): Finding[] => { + const { evidence, result } = comparison; const changes = result.changes.map((change) => makeFinding({ evidence, @@ -367,11 +417,6 @@ const makeFinding = (input: { limitations: uniqueSorted(input.limitations), }); -const runtimeStatus = (evidence: Evidence): RuntimeStatus[] => - evidence.operation === "compare_process_captures" - ? [processCaptureComparisonSchema.parse(evidence.normalized_result).status] - : []; - const behaviorStatus = ( statuses: readonly RuntimeStatus[], ): ChangedBehaviorResult["behavior_status"] => { diff --git a/src/server/registerInvestigationTools.ts b/src/server/registerInvestigationTools.ts index d5a2a4706..02e1c52d0 100644 --- a/src/server/registerInvestigationTools.ts +++ b/src/server/registerInvestigationTools.ts @@ -21,7 +21,10 @@ import { RECONSTRUCTION_PROVIDER, STATIC_RUNTIME_PROVIDER, } from "../application/InvestigationProviders.js"; -import { toolRegistrationOptions } from "./toolRegistrationOptions.js"; +import { + applicationValidatedInputSchema, + toolRegistrationOptions, +} from "./toolRegistrationOptions.js"; import { verifyReconstruction } from "../domain/reconstructionVerification.js"; /** Register Evidence-composed differential investigation workflows. */ @@ -52,30 +55,29 @@ const registerChangedBehavior = ( session: BinarySessionPort, contract: ReturnType>, ): void => { + const registration = toolRegistrationOptions(contract); server.registerTool( contract.name, - toolRegistrationOptions(contract), + { + ...registration, + inputSchema: applicationValidatedInputSchema(registration.inputSchema), + }, async (input, context) => { - const closure = evidenceClosure( - session, - comparisonClosure(input.comparisons), - ); - if (!closure.ok) - return server.delivery.toCallToolResult(closure, contract); - const links = closure.value; const computed = await runDerivedOperation(context, contract.name, () => - findChangedBehavior(input.comparisons), + findChangedBehavior(input), ); if (!computed.ok) return server.delivery.toCallToolResult(computed, contract); - const result = computed.value; + const { result, comparisonEvidenceIds } = computed.value; + const closure = evidenceClosure(session, result.evidence_links); + if (!closure.ok) + return server.delivery.toCallToolResult(closure, contract); + const links = closure.value; const evidence = createEvidence(undefined, CHANGED_BEHAVIOR_PROVIDER, { predicateType: "rea.changed-behavior", operation: contract.name, parameters: { - comparison_evidence_ids: input.comparisons.map( - ({ evidence_id: id }) => id, - ), + comparison_evidence_ids: [...comparisonEvidenceIds], }, result: jsonValueSchema.parse(result), confidence: "derived", diff --git a/tests/boundary/mcp/processEvidenceIdentityMcp.test.ts b/tests/boundary/mcp/processEvidenceIdentityMcp.test.ts index fb20dd79b..993f1df69 100644 --- a/tests/boundary/mcp/processEvidenceIdentityMcp.test.ts +++ b/tests/boundary/mcp/processEvidenceIdentityMcp.test.ts @@ -309,7 +309,9 @@ it("separately rejects a tampered digest before admission to any consumer or ses }; const digestMismatch = /semantic identifier does not match/u; expect(() => parseEvidence(tampered)).toThrow(digestMismatch); - expect(() => findChangedBehavior([tampered])).toThrow(digestMismatch); + expect(() => findChangedBehavior({ comparisons: [tampered] })).toThrow( + digestMismatch, + ); expect(() => correlateStaticAndRuntime(correlationInput(tampered))).toThrow( digestMismatch, ); @@ -336,6 +338,19 @@ it("separately rejects a tampered digest before admission to any consumer or ses } }); +it("maps malformed changed-behavior Evidence to an invalid request before closure lookup", async () => { + const { client, comparison } = await connectedComparison(); + const malformed = { ...comparison, evidence_links: undefined } as unknown; + const response = await client.callTool({ + name: "find_changed_behavior", + arguments: { comparisons: [malformed] }, + }); + expect(response.isError).toBe(true); + expect(parseMcpToolError(response)).toMatchObject({ + error: { code: "invalid_request" }, + }); +}); + it("keeps withheld ownership unknown through CLI and MCP comparison", async () => { const { client, right } = await connectedComparison(); const left = captureEvidence("left", { From 650237efb01b3c4a154799dbbedb2af1084f2312 Mon Sep 17 00:00:00 2001 From: The one -Niu <912906590@qq.com> Date: Tue, 13 Oct 2026 14:40:04 +0800 Subject: [PATCH 040/159] fix(setup): pin Hopper's macOS publisher identity before install (#2481) The macOS installer accepted any bundle that passed `codesign --verify` and whose `Info.plist` said `com.cryptic-apps.hopper-web-4`. An ad-hoc signed bundle with that identifier satisfies both, so a substituted DMG could be copied to ~/Applications and launched. Verify a pinned code requirement instead, covering the sealed bundle identifier and the signing team together, and report a distinct `publisher_verification_failed` code so a vendor key change is not confused with a corrupt package. --- src/application/MacHopper.test.ts | 27 ++++- src/application/MacHopper.ts | 74 ++++++++---- src/application/SetupInstallFailure.test.ts | 23 ++++ src/application/SetupInstallFailure.ts | 10 ++ .../macosHopperPublisherVerification.test.ts | 108 ++++++++++++++++++ 5 files changed, 216 insertions(+), 26 deletions(-) create mode 100644 src/application/SetupInstallFailure.test.ts create mode 100644 tests/boundary/setup/macosHopperPublisherVerification.test.ts diff --git a/src/application/MacHopper.test.ts b/src/application/MacHopper.test.ts index d75959fd5..7896208c5 100644 --- a/src/application/MacHopper.test.ts +++ b/src/application/MacHopper.test.ts @@ -2,12 +2,16 @@ import { createHash } from "node:crypto"; import { describe, expect, it } from "vitest"; -import { installMacHopper, type MacHopperInstallHost } from "./MacHopper.js"; +import { + installMacHopper, + type MacHopperBundleVerification, + type MacHopperInstallHost, +} from "./MacHopper.js"; class FakeMacHopperHost implements MacHopperInstallHost { readonly archive = new TextEncoder().encode("hopper-dmg"); archiveOk = true; - valid = true; + verification: MacHopperBundleVerification = "verified"; existing = false; mountSucceeds = true; copySucceeds = true; @@ -15,6 +19,7 @@ class FakeMacHopperHost implements MacHopperInstallHost { badHash = false; unmounts = 0; cleanups = 0; + installs = 0; opens = 0; replaces = false; @@ -47,7 +52,7 @@ class FakeMacHopperHost implements MacHopperInstallHost { return Promise.resolve(); }; appBundle = () => Promise.resolve("/mounted/Hopper Disassembler.app"); - validBundle = () => Promise.resolve(this.valid); + validBundle = () => Promise.resolve(this.verification); destinationExists = () => Promise.resolve(this.existing); installBundle = ( _source: string, @@ -55,6 +60,7 @@ class FakeMacHopperHost implements MacHopperInstallHost { _stage: string, replaceExisting: boolean, ) => { + this.installs += 1; this.replaces = replaceExisting; return Promise.resolve(this.copySucceeds); }; @@ -115,7 +121,7 @@ describe("macOS Hopper installation", () => { it("cleans and detaches after bundle validation fails", async () => { const host = new FakeMacHopperHost(); - host.valid = false; + host.verification = "invalid_signature"; await expect(installMacHopper({}, host)).resolves.toEqual({ status: "failed", reason: "bundle", @@ -123,4 +129,17 @@ describe("macOS Hopper installation", () => { expect(host.unmounts).toBe(1); expect(host.cleanups).toBe(1); }); + + it("reports a publisher mismatch without copying or launching the bundle", async () => { + const host = new FakeMacHopperHost(); + host.verification = "unverified_publisher"; + await expect(installMacHopper({}, host)).resolves.toEqual({ + status: "failed", + reason: "publisher_verification_failed", + }); + expect(host.installs).toBe(0); + expect(host.opens).toBe(0); + expect(host.unmounts).toBe(1); + expect(host.cleanups).toBe(1); + }); }); diff --git a/src/application/MacHopper.ts b/src/application/MacHopper.ts index 31ca522f4..a211e9f2d 100644 --- a/src/application/MacHopper.ts +++ b/src/application/MacHopper.ts @@ -24,7 +24,35 @@ const execFileAsync = promisify(execFile); const RELEASES_URL = "https://www.hopperapp.com/include/files-api.php?request=releases&public=true"; const DOWNLOAD_PREFIX = "https://www.hopperapp.com:443/downloader/public/"; -const BUNDLE_IDENTIFIER = "com.cryptic-apps.hopper-web-4"; + +/** + * Apple publisher identity pinned for the official Hopper macOS bundle. The + * team identifier was captured from `Hopper-6.6.0-demo.dmg` at + * `https://www.hopperapp.com:443/downloader/public/Hopper-6.6.0-demo.dmg` + * (34846271 bytes, SHA-1 0be530f038249742590992199c7b042ea20ca955) with + * `codesign -dvvv`, which reported + * `Authority=Developer ID Application: Cryptic Apps (2AMA2753NF)`. + */ +export const HOPPER_PUBLISHER = Object.freeze({ + bundleIdentifier: "com.cryptic-apps.hopper-web-4", + teamIdentifier: "2AMA2753NF", +}); + +/** + * Code requirement enforcing the bundle identifier and signing team together. + * The identifier is read from the sealed CodeDirectory, so it cannot be changed + * by editing `Info.plist` the way the previous string comparison could. + */ +const HOPPER_PUBLISHER_REQUIREMENT = + `= identifier "${HOPPER_PUBLISHER.bundleIdentifier}"` + + ` and anchor apple generic` + + ` and certificate leaf[subject.OU] = "${HOPPER_PUBLISHER.teamIdentifier}"`; + +/** Outcome of reading a mounted bundle's signature and publisher identity. */ +export type MacHopperBundleVerification = + | "verified" + | "invalid_signature" + | "unverified_publisher"; /** Stable network and command effects disclosed before macOS Hopper setup. */ export const macHopperInstallDisclosure = { @@ -32,6 +60,7 @@ export const macHopperInstallDisclosure = { commands: [ "hdiutil attach -readonly -nobrowse -mountpoint ", "codesign --verify --deep --strict ", + `codesign --verify --strict '-R${HOPPER_PUBLISHER_REQUIREMENT}' `, "ditto ", "open ", ] as const, @@ -52,6 +81,7 @@ export type MacHopperInstallResult = | "integrity" | "mount" | "bundle" + | "publisher_verification_failed" | "destination_exists" | "copy" | "launcher_missing"; @@ -69,7 +99,7 @@ export interface MacHopperInstallHost { mount(packagePath: string, mountPath: string): Promise; unmount(mountPath: string): Promise; appBundle(mountPath: string): Promise; - validBundle(appPath: string): Promise; + validBundle(appPath: string): Promise; destinationExists(path: string): Promise; installBundle( source: string, @@ -124,8 +154,12 @@ export const installMacHopper = async ( return { status: "failed", reason: "mount" }; mounted = true; const source = await host.appBundle(mountPath); - if (source === undefined || !(await host.validBundle(source))) + if (source === undefined) return { status: "failed", reason: "bundle" }; + const verification = await host.validBundle(source); + if (verification === "invalid_signature") return { status: "failed", reason: "bundle" }; + if (verification === "unverified_publisher") + return { status: "failed", reason: "publisher_verification_failed" }; const stage = `${destination}.rea-stage`; if ( !(await host.installBundle( @@ -222,25 +256,21 @@ export const systemMacHopperInstallHost = ( environment, )) ) - return false; - try { - const identifier = ( - await execFileAsync( - "/usr/libexec/PlistBuddy", - [ - "-c", - "Print :CFBundleIdentifier", - join(appPath, "Contents/Info.plist"), - ], - { env: environment }, - ) - ).stdout.trim(); - return identifier === BUNDLE_IDENTIFIER; - } catch (cause: unknown) { - // best-effort cleanup: optional bundle probing; unreadable means invalid. - void cause; - return false; - } + return "invalid_signature"; + if ( + !(await commandSucceeds( + "codesign", + [ + "--verify", + "--strict", + `-R${HOPPER_PUBLISHER_REQUIREMENT}`, + appPath, + ], + environment, + )) + ) + return "unverified_publisher"; + return "verified"; }, async destinationExists(path) { try { diff --git a/src/application/SetupInstallFailure.test.ts b/src/application/SetupInstallFailure.test.ts new file mode 100644 index 000000000..980e22151 --- /dev/null +++ b/src/application/SetupInstallFailure.test.ts @@ -0,0 +1,23 @@ +import { describe, expect, it } from "vitest"; + +import { HOPPER_PUBLISHER } from "./MacHopper.js"; +import { setupInstallFailure } from "./SetupInstallFailure.js"; + +describe("macOS Hopper setup failure projection", () => { + it("projects a publisher mismatch onto its own code and recovery", () => { + expect(setupInstallFailure("publisher_verification_failed")).toEqual({ + status: "failed", + code: "publisher_verification_failed", + remediation: `The mounted Hopper application is not signed by the publisher REA pins (${HOPPER_PUBLISHER.bundleIdentifier}, Apple team ${HOPPER_PUBLISHER.teamIdentifier}); nothing was copied or launched. Install Hopper yourself and set HOPPER_LAUNCHER_PATH to its launcher, then rerun rea setup.`, + }); + }); + + it("keeps a broken bundle distinct from a publisher mismatch", () => { + expect(setupInstallFailure("bundle")).toEqual({ + status: "failed", + code: "bundle_invalid", + remediation: + "The mounted package did not contain the expected Hopper application bundle. Retry setup; if it repeats, update REA before installing.", + }); + }); +}); diff --git a/src/application/SetupInstallFailure.ts b/src/application/SetupInstallFailure.ts index fe2a2739e..0b332fa25 100644 --- a/src/application/SetupInstallFailure.ts +++ b/src/application/SetupInstallFailure.ts @@ -1,3 +1,5 @@ +import { HOPPER_PUBLISHER } from "./MacHopper.js"; + export type SetupFailureCode = | "unsupported_host" | "download_failed" @@ -10,6 +12,7 @@ export type SetupFailureCode = | "destination_exists" | "mount_failed" | "bundle_invalid" + | "publisher_verification_failed" | "copy_failed" | "skill_install_failed" | "cleanup_failed"; @@ -39,6 +42,7 @@ export type HopperInstallFailureReason = | "destination_exists" | "mount" | "bundle" + | "publisher_verification_failed" | "copy"; /** Map provider installer failures to stable setup codes and safe recovery. */ @@ -115,6 +119,12 @@ export const setupInstallFailure = ( remediation: "The mounted package did not contain the expected Hopper application bundle. Retry setup; if it repeats, update REA before installing.", }; + case "publisher_verification_failed": + return { + status: "failed", + code: "publisher_verification_failed", + remediation: `The mounted Hopper application is not signed by the publisher REA pins (${HOPPER_PUBLISHER.bundleIdentifier}, Apple team ${HOPPER_PUBLISHER.teamIdentifier}); nothing was copied or launched. Install Hopper yourself and set HOPPER_LAUNCHER_PATH to its launcher, then rerun rea setup.`, + }; case "copy": return { status: "failed", diff --git a/tests/boundary/setup/macosHopperPublisherVerification.test.ts b/tests/boundary/setup/macosHopperPublisherVerification.test.ts new file mode 100644 index 000000000..29b696e85 --- /dev/null +++ b/tests/boundary/setup/macosHopperPublisherVerification.test.ts @@ -0,0 +1,108 @@ +import { execFile } from "node:child_process"; +import { appendFile, copyFile, mkdir, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { promisify } from "node:util"; + +import { describe, expect, it } from "vitest"; + +import { + HOPPER_PUBLISHER, + macHopperInstallDisclosure, + systemMacHopperInstallHost, +} from "../../../src/application/MacHopper.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +const run = promisify(execFile); +const itDarwin = it.skipIf(process.platform !== "darwin"); +const executableSegments = ["Contents", "MacOS", "hopper"]; + +const infoPlist = (identifier: string): string => + ` + + + + CFBundleIdentifier + ${identifier} + CFBundleExecutable + hopper + CFBundlePackageType + APPL + + +`; + +/** Build a minimal `.app` whose launcher is a real signed Mach-O copy. */ +const buildBundle = async ( + root: string, + identifier: string, + options: { readonly sign?: boolean } = {}, +): Promise => { + const app = join(root, "Hopper Disassembler.app"); + const executable = join(app, ...executableSegments); + await mkdir(join(app, "Contents", "MacOS"), { recursive: true }); + await copyFile("/usr/bin/true", executable); + await writeFile(join(app, "Contents", "Info.plist"), infoPlist(identifier)); + if (options.sign !== false) + await run("/usr/bin/codesign", [ + "--sign", + "-", + "--force", + "--deep", + "--identifier", + identifier, + app, + ]); + return app; +}; + +const host = () => + systemMacHopperInstallHost("/home/rea-test", { ...process.env }); + +describe("macOS Hopper publisher verification", () => { + itDarwin( + "rejects an ad-hoc signed bundle carrying the pinned bundle identifier", + async () => { + const root = await createTestTempDirectory("rea-hopper-publisher-"); + const app = await buildBundle(root, HOPPER_PUBLISHER.bundleIdentifier); + // The pre-pinned checks accept this bundle: `codesign --verify` succeeds + // for ad-hoc signatures and the identifier clause alone also matches. + await run("/usr/bin/codesign", ["--verify", "--deep", "--strict", app]); + await expect(host().validBundle(app)).resolves.toBe( + "unverified_publisher", + ); + }, + ); + + itDarwin("rejects an unsigned bundle before any copy", async () => { + const root = await createTestTempDirectory("rea-hopper-unsigned-"); + const app = await buildBundle(root, HOPPER_PUBLISHER.bundleIdentifier, { + sign: false, + }); + await expect(host().validBundle(app)).resolves.toBe("invalid_signature"); + }); + + itDarwin("rejects a bundle modified after it was signed", async () => { + const root = await createTestTempDirectory("rea-hopper-tampered-"); + const app = await buildBundle(root, HOPPER_PUBLISHER.bundleIdentifier); + await appendFile(join(app, ...executableSegments), "appended\n"); + await expect(host().validBundle(app)).resolves.toBe("invalid_signature"); + }); + + it("discloses the exact code requirement the installer enforces", () => { + expect(macHopperInstallDisclosure.commands).toContain( + `codesign --verify --strict '-R= identifier "${HOPPER_PUBLISHER.bundleIdentifier}" and anchor apple generic and certificate leaf[subject.OU] = "${HOPPER_PUBLISHER.teamIdentifier}"' `, + ); + }); + + // Opt-in positive control: point this at a mounted official Hopper bundle, + // for example `hdiutil attach -readonly -nobrowse Hopper-6.6.0-demo.dmg`. + // It proves the pinned requirement is satisfiable by the real publisher, so + // a quoting or argument mistake cannot hide behind the rejection arms. + const realApp = process.env.REA_HOPPER_MACOS_APP; + it.skipIf(realApp === undefined)( + "accepts the officially signed Hopper application", + async () => { + await expect(host().validBundle(realApp ?? "")).resolves.toBe("verified"); + }, + ); +}); From 60f771a11e9889ce69cce6ad631b6b186ac742dd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Vin=C3=ADcius=20Andrade?= <95101505+viniciusdsandrade@users.noreply.github.com> Date: Tue, 13 Oct 2026 03:40:43 -0300 Subject: [PATCH 041/159] fix(apple): limit already-loaded reuse to rpath dependencies (#2292) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(apple): limit already-loaded reuse to rpath dependencies dyld reuses an already-loaded image by install name only for @rpath/ dependencies (Loader.cpp:802-816) and evaluates other dependency forms against their expanded path for each loader. When multiple images in different directories depend on @loader_path/ libraries sharing an install name, checking loaded images before candidate expansion caused subsequent loaders to bind to the first loaded image's copy or appear resolved when their own copy was absent. Limit byInstallName registration and lookup to install names that start with @rpath/, leaving caller-relative and literal dependencies to expand and evaluate their own candidate paths. Signed-off-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com> * test(apple): verify loader_path dependencies across isolated directories Assert that multiple images depending on @loader_path/ libraries with the same install name do not reuse already-loaded images across loaders. When the second loader's helper is absent, its candidate path is absent and dyld reports required-load-unresolved; when both helpers exist, each loader resolves to its own directory. Signed-off-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com> --------- Signed-off-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com> --- docs/apple-application-analysis.md | 2 +- src/domain/apple/dylibResolution.test.ts | 130 ++++++++++++++++++++ src/domain/apple/dylibResolution.ts | 10 +- src/domain/apple/dylibResolutionFindings.ts | 2 +- 4 files changed, 140 insertions(+), 4 deletions(-) diff --git a/docs/apple-application-analysis.md b/docs/apple-application-analysis.md index e00454c3e..666717fc5 100644 --- a/docs/apple-application-analysis.md +++ b/docs/apple-application-analysis.md @@ -174,7 +174,7 @@ Resolution follows dyld: - `@rpath` tries each `LC_RPATH` of the loading image first, then those of the image that loaded it, up to the process root. - Dependencies load dependents-first in load-command order, and each image - loads once per process. A later request whose install name matches an + loads once per process. A later @rpath request whose install name matches an already loaded image reuses it. - Symlinks such as `Versions/Current` are followed inside the analyzed root. An escape through `..` or an absolute link is `escapes-target`. diff --git a/src/domain/apple/dylibResolution.test.ts b/src/domain/apple/dylibResolution.test.ts index 2d63ce393..13bb7c4f6 100644 --- a/src/domain/apple/dylibResolution.test.ts +++ b/src/domain/apple/dylibResolution.test.ts @@ -597,6 +597,136 @@ describe("dyld load order", () => { }); }); +describe("loader-relative dylib resolution", () => { + it("does not reuse already loaded images by install name for @loader_path dependencies", async () => { + const pluginA = "Contents/PlugIns/A/pluginA.dylib"; + const helperA = "Contents/PlugIns/A/libhelper.dylib"; + const pluginB = "Contents/PlugIns/B/pluginB.dylib"; + const trace = await traceDylibLoading( + memoryView({ + [MAIN]: executable({ + dependencies: [ + dependency("@executable_path/../PlugIns/A/pluginA.dylib"), + dependency("@executable_path/../PlugIns/B/pluginB.dylib"), + ], + }), + [pluginA]: parsed( + slice({ + dependencies: [dependency("@loader_path/libhelper.dylib")], + }), + ), + [helperA]: parsed( + slice({ + install_name: "@loader_path/libhelper.dylib", + }), + ), + [pluginB]: parsed( + slice({ + dependencies: [dependency("@loader_path/libhelper.dylib")], + }), + ), + }), + { roots: [MAIN] }, + ); + + const edgeA = edgeFor(trace, pluginA, "@loader_path/libhelper.dylib"); + expect(edgeA.resolution).toEqual({ + status: "resolved", + image: helperA, + }); + expect(edgeA.candidates).toEqual([ + expect.objectContaining({ + source: "loader_path", + outcome: "resolved", + resolved_path: helperA, + }), + ]); + + const edgeB = edgeFor(trace, pluginB, "@loader_path/libhelper.dylib"); + expect(edgeB.resolution).toEqual({ + status: "unresolved", + image: null, + }); + expect(edgeB.candidates).toEqual([ + expect.objectContaining({ + path: "Contents/PlugIns/B/libhelper.dylib", + source: "loader_path", + outcome: "absent", + resolved_path: null, + }), + ]); + expect(trace.findings).toContainEqual( + expect.objectContaining({ + kind: "required-load-unresolved", + image: pluginB, + }), + ); + }); + + it("resolves @loader_path dependencies to the loader's own directory when multiple helpers exist", async () => { + const pluginA = "Contents/PlugIns/A/pluginA.dylib"; + const helperA = "Contents/PlugIns/A/libhelper.dylib"; + const pluginB = "Contents/PlugIns/B/pluginB.dylib"; + const helperB = "Contents/PlugIns/B/libhelper.dylib"; + const trace = await traceDylibLoading( + memoryView({ + [MAIN]: executable({ + dependencies: [ + dependency("@executable_path/../PlugIns/A/pluginA.dylib"), + dependency("@executable_path/../PlugIns/B/pluginB.dylib"), + ], + }), + [pluginA]: parsed( + slice({ + dependencies: [dependency("@loader_path/libhelper.dylib")], + }), + ), + [helperA]: parsed( + slice({ + install_name: "@loader_path/libhelper.dylib", + }), + ), + [pluginB]: parsed( + slice({ + dependencies: [dependency("@loader_path/libhelper.dylib")], + }), + ), + [helperB]: parsed( + slice({ + install_name: "@loader_path/libhelper.dylib", + }), + ), + }), + { roots: [MAIN] }, + ); + + expect( + edgeFor(trace, pluginA, "@loader_path/libhelper.dylib"), + ).toMatchObject({ + resolution: { status: "resolved", image: helperA }, + candidates: [ + expect.objectContaining({ + source: "loader_path", + outcome: "resolved", + resolved_path: helperA, + }), + ], + }); + expect( + edgeFor(trace, pluginB, "@loader_path/libhelper.dylib"), + ).toMatchObject({ + resolution: { status: "resolved", image: helperB }, + candidates: [ + expect.objectContaining({ + source: "loader_path", + outcome: "resolved", + resolved_path: helperB, + }), + ], + }); + }); +}); + describe("dyld slice compatibility and coverage", () => { it("lets an x86_64h process load generic x86_64 but not arm64e load arm64", async () => { const files = { diff --git a/src/domain/apple/dylibResolution.ts b/src/domain/apple/dylibResolution.ts index 00077efa1..799befdb1 100644 --- a/src/domain/apple/dylibResolution.ts +++ b/src/domain/apple/dylibResolution.ts @@ -427,7 +427,9 @@ const resolveDependency = async ( ): Promise => { const current = context.loaded.get(loader); const chain = current?.chain ?? [loader]; - const loaded = context.byInstallName.get(dependency.install_name); + const loaded = dependency.install_name.startsWith("@rpath/") + ? context.byInstallName.get(dependency.install_name) + : undefined; const candidates: Candidate[] = []; if (loaded !== undefined) candidates.push({ @@ -506,7 +508,10 @@ const traceProcess = async ( chain: [context.root], conditional: false, }); - if (rootSlice.install_name !== null) + if ( + rootSlice.install_name !== null && + rootSlice.install_name.startsWith("@rpath/") + ) context.byInstallName.set(rootSlice.install_name, context.root); const pending = [context.root]; while (pending.length > 0) { @@ -537,6 +542,7 @@ const traceProcess = async ( }); if ( slice.install_name !== null && + slice.install_name.startsWith("@rpath/") && !context.byInstallName.has(slice.install_name) ) context.byInstallName.set(slice.install_name, image); diff --git a/src/domain/apple/dylibResolutionFindings.ts b/src/domain/apple/dylibResolutionFindings.ts index e4cc1ddf2..ca506c17b 100644 --- a/src/domain/apple/dylibResolutionFindings.ts +++ b/src/domain/apple/dylibResolutionFindings.ts @@ -91,7 +91,7 @@ export const DYLIB_RESOLUTION_LIMITATIONS = [ "Slices are matched by dyld's graded architectures (an x86_64h process also loads x86_64). arm64e processes that disable pointer authentication can also load arm64 slices; that fallback is not modeled.", "Absolute install names and rpaths are outside the analyzed root and are not evaluated, including /System and /usr/lib libraries that the dyld shared cache usually provides.", "Leaf and relative install names depend on dyld fallback paths, DYLD_* variables, and the working directory; they are undetermined.", - "Load order follows dyld's dependents-first traversal in load-command order. An image reached through several chains is resolved once per process, with the rpath stack and conditionality of the first chain; a later request whose install name matches an already loaded image reuses it.", + "Load order follows dyld's dependents-first traversal in load-command order. An image reached through several chains is resolved once per process, with the rpath stack and conditionality of the first chain; a later @rpath request whose install name matches an already loaded image reuses it.", "@loader_path uses each image's symlink-resolved path within the analyzed root.", "Code-signing checks that can reject a found image, such as library validation and the hardened runtime, are not evaluated.", "Dylib compatibility versions are not compared. Apple's linker documentation describes a load-time rejection when an image's compatibility version is older than the loader requires, but dyld on macOS 26 was observed to load such an image.", From d73ed6fce24f6d1372c80319cd41d29d66666159 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=90=B4=E4=BD=B3=E6=B5=A9Alben?= <35291372+XiaoBinGan@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:43:10 +0800 Subject: [PATCH 042/159] fix(domain): preserve retryable capability failures through redaction (#2494) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit redactExplicitFailure rebuilds each typed failure field by field so explicit literals are removed while the projected code stays the same. The rebuild for AnalysisCapabilityUnavailableError carried providerId, operation, reason and userMessage but not retryable, and the constructor defaults that field to false. Redacting a transient capability failure therefore made it permanent and changed the remediation shown to the caller. Pass the field through with the same conditional-spread shape used for userMessage. The constructor recomputes retryable from cleanupIncomplete, and cleanup is already part of the reconstructed context, so a value that was true before redaction stays true and a false value stays absent. Verified with 66 error projection, access-denied, redaction and capture-failure tests, check:fast and verify:test-discovery on macOS arm64. The regression test failed with "expected false to be true" before the change. Co-authored-by: 吴佳浩 --- src/domain/explicitSensitiveFailure.test.ts | 18 ++++++++++++++++++ src/domain/explicitSensitiveFailure.ts | 1 + 2 files changed, 19 insertions(+) diff --git a/src/domain/explicitSensitiveFailure.test.ts b/src/domain/explicitSensitiveFailure.test.ts index 68aaf9f8f..757e9887a 100644 --- a/src/domain/explicitSensitiveFailure.test.ts +++ b/src/domain/explicitSensitiveFailure.test.ts @@ -116,6 +116,24 @@ it("keeps unsupported profile reasons distinct from malformed input", () => { expect(JSON.stringify(projectAnalysisError(result))).not.toContain("private"); }); +it("preserves a retryable capability failure through explicit-value redaction", () => { + // Retryability is a semantic field of the error, not presentation: dropping + // it turns a transient admission failure into a permanent one and changes + // the remediation the caller is shown. + const error = new AnalysisCapabilityUnavailableError( + "mitmproxy", + "inspect_web_network_capture", + "Runtime admission for /targets/private/capture failed transiently", + { retryable: true }, + ); + expect(projectAnalysisError(error).retryable).toBe(true); + const result = redactExplicitFailure(error, ["private"]); + expect(result).toBeInstanceOf(AnalysisCapabilityUnavailableError); + expect((result as AnalysisCapabilityUnavailableError).retryable).toBe(true); + expect(projectAnalysisError(result).retryable).toBe(true); + expect(JSON.stringify(projectAnalysisError(result))).not.toContain("private"); +}); + it("coarsens format issue pointers without creating replacement coordinates", () => { const error = new AnalysisInputError( "inspect_web_network_capture", diff --git a/src/domain/explicitSensitiveFailure.ts b/src/domain/explicitSensitiveFailure.ts index 05a4f2f88..8a0fac095 100644 --- a/src/domain/explicitSensitiveFailure.ts +++ b/src/domain/explicitSensitiveFailure.ts @@ -154,6 +154,7 @@ export const redactExplicitFailure = ( ...(error.userMessage === undefined ? {} : { userMessage: text(error.userMessage) }), + ...(error.retryable ? { retryable: true } : {}), }, ); if (error instanceof AnalysisResourceConstraintError) From f6556ad2bc102d80223f5b434136f546dbd2bbab Mon Sep 17 00:00:00 2001 From: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> Date: Mon, 12 Oct 2026 23:44:09 -0700 Subject: [PATCH 043/159] fix(pcap): order evidence ties by code point instead of host locale (#2509) The pcap summary sorted equal-count endpoints and protocol stacks with String#localeCompare, and Flutter build inspection sorted APK ABI directory names the same way. localeCompare follows the host's ICU locale, so the same capture produced differently ordered Evidence on different hosts: under da_DK "aarp" sorts after "zz", under lt_LT "syslog" sorts before "ssdp", and every locale puts "fe80::1" before "fe80:1::1". Use the repository's compareUnicodeCodePoints, as the other Evidence producers already do. Co-authored-by: Claude Opus 5.5 --- src/flutter/FlutterBuildProvider.ts | 5 +++-- src/pcap/PcapOutput.test.ts | 18 ++++++++++++++++++ src/pcap/PcapOutput.ts | 5 +++-- 3 files changed, 24 insertions(+), 4 deletions(-) diff --git a/src/flutter/FlutterBuildProvider.ts b/src/flutter/FlutterBuildProvider.ts index 2c088b2c9..9a6e6fe50 100644 --- a/src/flutter/FlutterBuildProvider.ts +++ b/src/flutter/FlutterBuildProvider.ts @@ -23,6 +23,7 @@ import { AnalysisUnsupportedTargetError, } from "../domain/analysisErrorCore.js"; import { AnalysisProtocolError } from "../domain/analysisErrorCore.js"; +import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js"; import { AnalysisError } from "../domain/analysisErrorBase.js"; import { err, ok, type Result } from "../domain/result.js"; import { @@ -759,7 +760,7 @@ export class FlutterBuildProvider { toolchain_capped: value.libflutter.toolchainCapped, }, })) - .sort((left, right) => left.abi.localeCompare(right.abi)); + .sort((left, right) => compareUnicodeCodePoints(left.abi, right.abi)); const flutterDetected = abis.some((entry) => entry.libapp.present); const partial = abis.some( @@ -871,7 +872,7 @@ export class FlutterBuildProvider { (left, right) => (preference.indexOf(left.abi) + 1 || preference.length + 1) - (preference.indexOf(right.abi) + 1 || preference.length + 1) || - left.abi.localeCompare(right.abi), + compareUnicodeCodePoints(left.abi, right.abi), )[0]! : candidates.find((candidate) => candidate.abi === abi); if (selected === undefined) diff --git a/src/pcap/PcapOutput.test.ts b/src/pcap/PcapOutput.test.ts index 6cfa6b0d3..c0e158438 100644 --- a/src/pcap/PcapOutput.test.ts +++ b/src/pcap/PcapOutput.test.ts @@ -83,6 +83,24 @@ it("counts IPv6 endpoints once per packet and preserves every examined fact", as expect(result.partial).toBe(false); }); +it("orders equal-count endpoints and protocols by code point, not host locale", async () => { + // Locale collation puts "fe80::1" first and, under da-DK, "aarp" after "zz". + const result = await parseTsharkJsonOutput( + capture([ + { "ipv6.src": ["fe80::1"], "frame.protocols": ["eth:llc:zz"] }, + { "ipv6.src": ["fe80:1::1"], "frame.protocols": ["eth:llc:aarp"] }, + ]), + ); + expect(result.endpoints.map(({ address }) => address)).toEqual([ + "fe80:1::1", + "fe80::1", + ]); + expect(result.protocolDistribution.map(({ protocol }) => protocol)).toEqual([ + "eth:llc:aarp", + "eth:llc:zz", + ]); +}); + it("preserves all DNS addresses and their exact examined count", async () => { const result = await parseTsharkJsonOutput( capture( diff --git a/src/pcap/PcapOutput.ts b/src/pcap/PcapOutput.ts index 95cb81926..b14d7ba3c 100644 --- a/src/pcap/PcapOutput.ts +++ b/src/pcap/PcapOutput.ts @@ -4,6 +4,7 @@ import { AnalysisResourceConstraintError, } from "../domain/analysisErrorCore.js"; import { parseRecordedTransportUrl } from "../domain/recordedTransportUrl.js"; +import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js"; const OPERATION = "inspect_pcap_capture"; /** Bounds JSON decoding plus its expanded object graph before JSON.parse. */ @@ -284,7 +285,7 @@ export const parseTsharkJsonOutput = async ( signal?.throwIfAborted(); return { endpoints: [...endpoints] - .sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0])) + .sort((a, b) => b[1] - a[1] || compareUnicodeCodePoints(a[0], b[0])) .map(([address, packet_count]) => ({ address, port: null, @@ -300,7 +301,7 @@ export const parseTsharkJsonOutput = async ( tlsSniValues: [...sni], tlsSniCount: sni.size, protocolDistribution: [...protocols] - .sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0])) + .sort((a, b) => b[1] - a[1] || compareUnicodeCodePoints(a[0], b[0])) .map(([protocol, packet_count]) => ({ protocol, packet_count })), protocolCount: protocols.size, httpRequestUris: [...uris], From 12303812a8d70f198c99c7aeb0df3aaa8166543c Mon Sep 17 00:00:00 2001 From: Rohit Ghumare <48523873+rohitg00@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:15:03 +0530 Subject: [PATCH 044/159] fix(evidence): admit bundle imports across session transitions (#2303) --- src/server/registerSessionRecordTools.ts | 78 +++++---- .../mcp/evidenceImportLifecycle.test.ts | 155 ++++++++++++++++++ 2 files changed, 202 insertions(+), 31 deletions(-) create mode 100644 tests/boundary/mcp/evidenceImportLifecycle.test.ts diff --git a/src/server/registerSessionRecordTools.ts b/src/server/registerSessionRecordTools.ts index a41d60cac..68daab23b 100644 --- a/src/server/registerSessionRecordTools.ts +++ b/src/server/registerSessionRecordTools.ts @@ -12,6 +12,8 @@ import { err, ok } from "../domain/result.js"; import { AnalysisCancelledError } from "../domain/analysisErrorCore.js"; import { inspectEvidenceBundle } from "../application/investigation/InspectEvidenceBundle.js"; import { toolRegistrationOptions } from "./toolRegistrationOptions.js"; +import { withAdmittedAnalysis } from "./analysisAdmission.js"; +import { runAdmittedToolOperation } from "./admittedToolOperation.js"; interface EvidenceToolRegistration { readonly server: EvidenceMcpServer; @@ -92,40 +94,54 @@ const registerImportEvidenceTool = ({ resources, importContract, }: EvidenceToolRegistration): void => { + const admission = withAdmittedAnalysis({ kind: "session", session }); server.registerTool( importContract.name, toolRegistrationOptions(importContract), - async (input, context) => { - const path = input.path; - const signal = context.mcpReq.signal; - const loaded = await readImmutableEvidenceBundle(path, signal, resources); - if (!loaded.ok) - return server.delivery.toCallToolResult(loaded, importContract); - const retainedUnknownRevisions = new Set( - bundleForSerialization(session).unknowns.map((unknown) => - unknownRevisionKey(unknown), - ), - ); - if (signal.aborted) - return server.delivery.toCallToolResult( - err(new AnalysisCancelledError("import_evidence_bundle")), - importContract, - ); - const imported = session.importEvidenceBundle(loaded.value); - return imported.ok - ? server.delivery.toCallToolResult( - ok({ - imported: imported.value, - unknowns_added: loaded.value.unknowns.filter( - (unknown) => - !retainedUnknownRevisions.has(unknownRevisionKey(unknown)), - ).length, - total: bundleForSerialization(session).records.length, - }), - importContract, - ) - : server.delivery.toCallToolResult(imported, importContract); - }, + async (input, context) => + runAdmittedToolOperation( + server, + admission, + importContract.name, + context.mcpReq.signal, + async () => { + const path = input.path; + const signal = context.mcpReq.signal; + const loaded = await readImmutableEvidenceBundle( + path, + signal, + resources, + ); + if (!loaded.ok) + return server.delivery.toCallToolResult(loaded, importContract); + const retainedUnknownRevisions = new Set( + bundleForSerialization(session).unknowns.map((unknown) => + unknownRevisionKey(unknown), + ), + ); + if (signal.aborted) + return server.delivery.toCallToolResult( + err(new AnalysisCancelledError("import_evidence_bundle")), + importContract, + ); + const imported = session.importEvidenceBundle(loaded.value); + return imported.ok + ? server.delivery.toCallToolResult( + ok({ + imported: imported.value, + unknowns_added: loaded.value.unknowns.filter( + (unknown) => + !retainedUnknownRevisions.has( + unknownRevisionKey(unknown), + ), + ).length, + total: bundleForSerialization(session).records.length, + }), + importContract, + ) + : server.delivery.toCallToolResult(imported, importContract); + }, + ), ); }; diff --git a/tests/boundary/mcp/evidenceImportLifecycle.test.ts b/tests/boundary/mcp/evidenceImportLifecycle.test.ts new file mode 100644 index 000000000..f0371ef89 --- /dev/null +++ b/tests/boundary/mcp/evidenceImportLifecycle.test.ts @@ -0,0 +1,155 @@ +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { expect, it, onTestFinished, vi } from "vitest"; + +import { createEvidence } from "../../../src/domain/evidence.js"; +import { createEvidenceBundle } from "../../../src/domain/evidenceBundle.js"; +import { OwnedFileHandle } from "../../../src/filesystem/OwnedFileHandle.js"; +import { createServer } from "../../../src/server/createServer.js"; +import { + createDeferred, + createTestBinarySession, +} from "../../fixtures/binarySession.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +const connectImportSession = async () => { + const root = await createTestTempDirectory("rea-mcp-import-lifecycle-"); + const session = createTestBinarySession(() => { + throw new Error("Evidence import must not start a binary provider"); + }); + const server = createServer({ kind: "session", session }); + const client = new Client({ + name: "evidence-import-lifecycle", + version: "1", + }); + const readCompleted = createDeferred(); + const releaseRead = createDeferred(); + const originalFileClose = OwnedFileHandle.prototype.close; + vi.spyOn(OwnedFileHandle.prototype, "close").mockImplementationOnce( + async function (this: OwnedFileHandle) { + await originalFileClose.call(this); + readCompleted.resolve(); + await releaseRead.promise; + }, + ); + const closeStarted = createDeferred(); + const originalSessionClose = session.close.bind(session); + const events: string[] = []; + vi.spyOn(session, "close").mockImplementation((...args) => { + const closing = originalSessionClose(...args); + closeStarted.resolve(); + return closing.then((result) => { + events.push("close"); + return result; + }); + }); + const originalImport = session.importEvidenceBundle.bind(session); + vi.spyOn(session, "importEvidenceBundle").mockImplementation((bundle) => { + const imported = originalImport(bundle); + events.push("import"); + return imported; + }); + onTestFinished(async () => { + releaseRead.resolve(); + vi.restoreAllMocks(); + await client.close(); + await server.close(); + await session.close(); + }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + const laterImportDispatched = createDeferred(); + let importsDispatched = 0; + const send = clientTransport.send.bind(clientTransport); + vi.spyOn(clientTransport, "send").mockImplementation( + async (message, ...args) => { + await send(message, ...args); + if ( + "method" in message && + message.method === "tools/call" && + message.params?.name === "import_evidence_bundle" + ) { + importsDispatched += 1; + if (importsDispatched === 2) laterImportDispatched.resolve(); + } + }, + ); + await server.connect(serverTransport); + await client.connect(clientTransport); + const importBundle = async (operation: string) => { + const record = createEvidence( + undefined, + { id: "fixture", name: "Fixture", version: "1" }, + { operation, parameters: {}, result: { retained: operation } }, + ); + const path = join(root, `${operation}.json`); + await writeFile(path, JSON.stringify(createEvidenceBundle([record]))); + return client.callTool({ + name: "import_evidence_bundle", + arguments: { path }, + }); + }; + return { + client, + session, + events, + importBundle, + readCompleted: readCompleted.promise, + closeStarted: closeStarted.promise, + laterImportDispatched: laterImportDispatched.promise, + releaseRead: () => releaseRead.resolve(), + }; +}; + +it("drains a file import before close_binary clears the session", async () => { + const fixture = await connectImportSession(); + const importing = fixture.importBundle("before_close"); + await fixture.readCompleted; + const closing = fixture.client.callTool({ + name: "close_binary", + arguments: {}, + }); + await fixture.closeStarted; + await fixture.client.ping(); + fixture.releaseRead(); + const [imported, closed] = await Promise.all([importing, closing]); + + expect(imported.isError, JSON.stringify(imported)).not.toBe(true); + expect(imported.structuredContent).toMatchObject({ + result: { imported: 1, unknowns_added: 0, total: 1 }, + }); + expect(closed.isError, JSON.stringify(closed)).not.toBe(true); + expect(fixture.events).toEqual(["import", "close"]); + expect(fixture.session.exportEvidenceBundle().records).toEqual([]); + await fixture.client.ping(); +}); + +it("admits a later import after close_binary without retaining the earlier bundle", async () => { + const fixture = await connectImportSession(); + const earlier = fixture.importBundle("before_close"); + await fixture.readCompleted; + const closing = fixture.client.callTool({ + name: "close_binary", + arguments: {}, + }); + await fixture.closeStarted; + const later = fixture.importBundle("after_close"); + await fixture.laterImportDispatched; + await fixture.client.ping(); + fixture.releaseRead(); + const results = await Promise.all([earlier, closing, later]); + + for (const result of results) + expect(result.isError, JSON.stringify(result)).not.toBe(true); + expect(results[2]?.structuredContent).toMatchObject({ + result: { imported: 1, unknowns_added: 0, total: 1 }, + }); + expect(fixture.events).toEqual(["import", "close", "import"]); + expect( + fixture.session + .exportEvidenceBundle() + .records.map((record) => record.operation), + ).toEqual(["after_close"]); + await fixture.client.ping(); +}); From 1a1b67c8885e9ff33f7a06ed2e027ebfdb83e084 Mon Sep 17 00:00:00 2001 From: Aurantium <3550271410@qq.com> Date: Mon, 12 Oct 2026 23:46:28 -0700 Subject: [PATCH 045/159] fix(javascript): preserve POSIX runtime path case on Windows (#2422) Use POSIX path semantics for recorded POSIX roots when reconciling runtime Evidence on Windows. Preserve Windows mapping behavior and cover implicit and explicit file roots through the reconciliation service. Assisted-by: OpenAI GPT-6 Co-authored-by: Aurantium-47 <280805072+Aurantium-47@users.noreply.github.com> --- .../javascriptRuntimeStaticCandidates.ts | 4 +- .../javascriptApplicationWorkflows.test.ts | 68 +++++++++++++++++++ 2 files changed, 70 insertions(+), 2 deletions(-) diff --git a/src/domain/javascript/javascriptRuntimeStaticCandidates.ts b/src/domain/javascript/javascriptRuntimeStaticCandidates.ts index 1f02c820d..810ab7f5e 100644 --- a/src/domain/javascript/javascriptRuntimeStaticCandidates.ts +++ b/src/domain/javascript/javascriptRuntimeStaticCandidates.ts @@ -1,4 +1,4 @@ -import { posix, relative, win32 } from "node:path"; +import { posix, win32 } from "node:path"; import type { ApplicationNode } from "./javascriptApplicationGraphSchemas.js"; import type { JavaScriptApplicationGraph } from "./javascriptApplicationGraph.js"; @@ -216,7 +216,7 @@ const pathBelowRoot = (root: string, value: string): string | null => { const remainder = rootStyle === "windows" ? win32.relative(root, value) - : relative(root, value); + : posix.relative(root, value); const portableRemainder = remainder.replaceAll("\\", "/"); if ( remainder === "" || diff --git a/tests/boundary/filesystem/javascriptApplicationWorkflows.test.ts b/tests/boundary/filesystem/javascriptApplicationWorkflows.test.ts index 17c3f0048..b79407821 100644 --- a/tests/boundary/filesystem/javascriptApplicationWorkflows.test.ts +++ b/tests/boundary/filesystem/javascriptApplicationWorkflows.test.ts @@ -21,6 +21,7 @@ import { type ApplicationVersionComparisonResult, } from "../../../src/domain/javascript/javascriptApplicationVersionComparisonSchemas.js"; import { javascriptRuntimeReconciliationResultSchema } from "../../../src/domain/javascript/javascriptRuntimeReconciliationSchemas.js"; +import { electronPageInspectionSchema } from "../../../src/domain/javascript/electronObservation.js"; import { compareJavaScriptApplicationVersions } from "../../../src/domain/javascript/javascriptApplicationVersionComparison.js"; import { createJavaScriptApplicationGraph } from "../../../src/domain/javascript/javascriptApplicationGraph.js"; import { traceApplicationFeature } from "../../../src/domain/javascript/javascriptFeatureTrace.js"; @@ -444,6 +445,73 @@ describe("JavaScript application workflow error diagnostics", () => { }); describe("JavaScript runtime reconciliation local paths", () => { + it.each([ + { + name: "case-sensitive POSIX input root", + root: null, + path: "/applications/Example.app/Contents/Resources/app/renderer.js", + status: "unknown", + }, + { + name: "case-sensitive POSIX mapping root", + root: "/runtime/App", + path: "/runtime/app/renderer.js", + status: "unknown", + }, + { + name: "matching POSIX mapping root", + root: "/runtime/App", + path: "/runtime/App/renderer.js", + status: "matched", + }, + { + name: "case-insensitive Windows mapping root", + root: String.raw`C:\runtime\App`, + path: String.raw`c:\runtime\app\renderer.js`, + status: "matched", + }, + ])("uses $name on every host", ({ root, path, status }) => { + const base = JAVASCRIPT_RUNTIME_RECONCILIATION_EXAMPLE; + const original = base.runtime_observations[0]!; + const inspection = electronPageInspectionSchema.parse( + original.normalized_result, + ); + const runtime = createEvidence(undefined, original.provider, { + predicateType: original.predicate_type, + operation: original.operation, + parameters: { ...original.parameters, include_script_sources: false }, + result: { + ...inspection, + scripts: { + ...inspection.scripts, + items: inspection.scripts.items.map((script) => ({ + ...script, + file_path: path, + source: { included: false, reason: "not-requested" }, + })), + }, + }, + confidence: original.confidence, + authority: original.authority, + }); + const reconciled = reconcileJavaScriptRuntimeEvidence({ + static_layers: base.static_layers.map((layer) => ({ + ...layer, + runtime_mappings: root === null ? [] : [{ kind: "file-root", root }], + })), + runtime_observations: [runtime], + }); + if (!reconciled.ok) throw reconciled.error; + const result = javascriptRuntimeReconciliationResultSchema.parse( + reconciled.value.normalized_result, + ); + expect( + result.reconciliations.find( + ({ entity_kind }) => entity_kind === "script", + ), + ).toMatchObject({ status }); + }); + it("resolves relative file mapping roots before deriving result Evidence", () => { const absoluteRoot = resolve(process.cwd(), "runtime-cache-fixture"); const relativeRoot = relative(process.cwd(), absoluteRoot); From c6acd19b6d842699ab807d9526dc762c743686f4 Mon Sep 17 00:00:00 2001 From: Mykola Dementiev Date: Tue, 13 Oct 2026 08:48:17 +0200 Subject: [PATCH 046/159] fix(ida): stop the private input of a universal Mach-O at the end of the selected slice (#2291) * test(ida): expect the private input of a universal Mach-O to end at the selected slice Cover a universal file larger than one read chunk with another slice after the selected one, for both a middle slice and the first slice. * fix(ida): stop the private input of a universal Mach-O at the end of the selected slice A read chunk that starts after the slice produced a negative end offset, and Buffer.subarray counts a negative bound from the end, so the head of the following slice was written to the private input and its digest. --- src/ida/IdaWorkspace.ts | 6 +- src/ida/IdaWorkspaceThinning.test.ts | 93 +++++++++++++++++++++++++++- 2 files changed, 96 insertions(+), 3 deletions(-) diff --git a/src/ida/IdaWorkspace.ts b/src/ida/IdaWorkspace.ts index 0b3c36231..c7d9e642c 100644 --- a/src/ida/IdaWorkspace.ts +++ b/src/ida/IdaWorkspace.ts @@ -85,8 +85,10 @@ export class IdaWorkspace { slice === undefined ? position + bytes.length : Math.min(position + bytes.length, slice.offset + slice.size); - const admitted = bytes.subarray(start - position, end - position); - if (admitted.length > 0) { + // A chunk after the slice has end < position, and a negative subarray + // bound would count from the chunk's end instead of being empty. + if (end > start) { + const admitted = bytes.subarray(start - position, end - position); inputHash.update(admitted); await input.write(admitted); } diff --git a/src/ida/IdaWorkspaceThinning.test.ts b/src/ida/IdaWorkspaceThinning.test.ts index 7feca3ddc..7d83f089b 100644 --- a/src/ida/IdaWorkspaceThinning.test.ts +++ b/src/ida/IdaWorkspaceThinning.test.ts @@ -5,7 +5,7 @@ import { afterEach, describe, expect, it } from "vitest"; import { parseBinaryTarget } from "../application/BinaryTargetResolver.js"; import { parseExecutableHeader } from "../domain/binaryTarget.js"; -import { fat } from "../domain/binaryTarget.fixture.js"; +import { fat, thinMach } from "../domain/binaryTarget.fixture.js"; import { createIdaTarget, RecordingIdaMcp, @@ -35,6 +35,45 @@ const universalTarget = async (hostArchitecture: "arm64" | "x64") => { return { bytes, target: resolved.value }; }; +/** One slice of a synthetic universal file: CPU type, offset and size. */ +interface Layout { + readonly cpu: number; + readonly offset: number; + readonly size: number; +} + +/** + * Write a universal file whose slices span more than one 64 KiB read, each + * filled with its own byte so bytes outside the selected slice are visible. + */ +const largeUniversalTarget = async ( + layout: readonly Layout[], + hostArchitecture: "arm64" | "x64", +) => { + const directory = await createTestTempDirectory("rea-ida-thin-large-"); + roots.push(directory); + const path = join(directory, "universal"); + const total = Math.max(...layout.map(({ offset, size }) => offset + size)); + const bytes = Buffer.alloc(total); + bytes.writeUInt32BE(0xcafebabe, 0); + bytes.writeUInt32BE(layout.length, 4); + layout.forEach(({ cpu, offset, size }, index) => { + const entry = 8 + index * 20; + bytes.writeUInt32BE(cpu, entry); + bytes.writeUInt32BE(offset, entry + 8); + bytes.writeUInt32BE(size, entry + 12); + bytes.writeUInt32BE(14, entry + 16); + bytes.fill(0x11 * (index + 1), offset, offset + size); + thinMach(0xfeedfacf, cpu).copy(bytes, offset); + }); + await writeFile(path, bytes); + const resolved = await parseBinaryTarget(path, { hostArchitecture }); + if (!resolved.ok) throw resolved.error; + if (resolved.value.format !== "mach-o-universal") + throw new Error("the universal fixture did not resolve as a FAT container"); + return { bytes, target: resolved.value }; +}; + describe("IDA workspace input thinning", () => { it.skipIf(process.platform === "win32")( "thins the selected arm64 slice into the private headless input", @@ -78,6 +117,58 @@ describe("IDA workspace input thinning", () => { }, ); + it.skipIf(process.platform === "win32")( + "copies exactly the selected slice when later slices follow it across read boundaries", + async () => { + const { bytes, target } = await largeUniversalTarget( + [ + { cpu: 0x01000007, offset: 4096, size: 8480 }, + { cpu: 0x0100000c, offset: 16384, size: 33464 }, + { cpu: 0x00000007, offset: 65536, size: 33464 }, + ], + "arm64", + ); + const workspace = await IdaWorkspace.allocate(target); + roots.push(workspace.directory); + await workspace.copyInput(); + + const input = await readFile(workspace.inputPath); + const { offset, size } = target.universalSlice; + expect(size, "the arm64 slice is selected").toBe(33464); + expect(input.length, "no byte after the slice is copied").toBe(size); + expect(input.equals(bytes.subarray(offset, offset + size))).toBe(true); + expect(workspace.inputSha256).toBe( + createHash("sha256") + .update(bytes.subarray(offset, offset + size)) + .digest("hex"), + ); + await workspace.remove(); + }, + ); + + it.skipIf(process.platform === "win32")( + "copies exactly the first slice when it is selected and others follow", + async () => { + const { bytes, target } = await largeUniversalTarget( + [ + { cpu: 0x01000007, offset: 16384, size: 37456 }, + { cpu: 0x0100000c, offset: 65536, size: 33464 }, + ], + "x64", + ); + const workspace = await IdaWorkspace.allocate(target); + roots.push(workspace.directory); + await workspace.copyInput(); + + const input = await readFile(workspace.inputPath); + const { offset, size } = target.universalSlice; + expect(size, "the x86_64 slice is selected").toBe(37456); + expect(input.length, "no byte after the slice is copied").toBe(size); + expect(input.equals(bytes.subarray(offset, offset + size))).toBe(true); + await workspace.remove(); + }, + ); + it("keeps a whole-file copy for thin targets", async () => { const { root, target } = await createIdaTarget("arm64"); roots.push(root); From 2c6a4fac4ff3477ac14f3d780894c5b390880607 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=90=B4=E4=BD=B3=E6=B5=A9Alben?= <35291372+XiaoBinGan@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:48:26 +0800 Subject: [PATCH 047/159] fix(process): accept a trace identifier the comparison result can represent (#2488) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The trace specification owns the declared identifier vocabulary and the comparison result echoes those identifiers verbatim into raw_trace[].event_id, matched_variant and diagnostic event_ids. The result module restated the shared IDENTIFIER_PATTERN by hand with a 64-character cap, so a specification with a longer identifier validated and then its own comparison result failed to parse: compareProcessTraces threw a raw ZodError at the caller instead of returning a comparison. Export the identifier schema from the specification as the single owner and reuse it in the evaluation module, removing the capped copy. IDENTIFIER_PATTERN has no length bound and every other consumer uses it unbounded, so the duplicate was the outlier. Removing the bound rather than adding one keeps the advertised trace_spec input contract unchanged; identifiers of 64 characters or fewer behave exactly as before. Verified with 48 process tests, 24 boundary CLI/MCP trace tests, check:fast and verify:test-discovery on macOS arm64. The regression test failed with the ZodError before the change and passes after it. Co-authored-by: 吴佳浩 --- .../process/processTraceComparison.test.ts | 28 +++++++++++++++ src/domain/process/processTraceEvaluation.ts | 9 +++-- .../process/processTraceSpecification.ts | 35 ++++++++++++------- 3 files changed, 55 insertions(+), 17 deletions(-) diff --git a/src/domain/process/processTraceComparison.test.ts b/src/domain/process/processTraceComparison.test.ts index caeaf7999..2207352e3 100644 --- a/src/domain/process/processTraceComparison.test.ts +++ b/src/domain/process/processTraceComparison.test.ts @@ -1,6 +1,7 @@ import { describe, expect, it } from "vitest"; import { parseProcessCapture } from "./processCaptureParsing.js"; import type { ProcessTraceSpecification } from "./processTraceSpecification.js"; +import { processTraceSpecificationSchema as specificationSchema } from "./processTraceSpecification.js"; import { processTraceComparisonResultSchema } from "./processTraceEvaluation.js"; import { capture, @@ -100,6 +101,33 @@ describe("generic capture trace comparison", () => { right: { matched_variant: "two" }, }); }); + + it("compares a declared identifier the comparison result can represent", () => { + // The specification owns the trace identifier vocabulary and the result + // echoes those identifiers verbatim, so both must accept the same values + // rather than handing the caller a raw schema error for its own input. + const eventId = `e${"0".repeat(64)}`; + const specification: ProcessTraceSpecification = { + events: [ + { + id: eventId, + source: "terminal_raw", + exact: terminal, + cardinality: { kind: "required" }, + }, + ], + language: { + kind: "finite_traces", + variants: [{ id: "only", trace: [eventId] }], + }, + }; + expect(specificationSchema.safeParse(specification).success).toBe(true); + const left = capture(values(["terminal", "process"])); + expect(compareProcessTraces(left, left, specification)).toMatchObject({ + verdict: "equivalent", + left: { matched_variant: "only", raw_trace: [{ event_id: eventId }] }, + }); + }); }); describe("process trace constraints and evidence sufficiency", () => { diff --git a/src/domain/process/processTraceEvaluation.ts b/src/domain/process/processTraceEvaluation.ts index 51a2a15f8..7c232a480 100644 --- a/src/domain/process/processTraceEvaluation.ts +++ b/src/domain/process/processTraceEvaluation.ts @@ -13,13 +13,12 @@ import { import { canonicalTraceJson, processTraceCardinalityBounds, + processTraceIdentifierSchema, type ProcessTraceSpecification, } from "./processTraceSpecification.js"; -const identifierSchema = z.string().regex(/^[A-Za-z][A-Za-z0-9._\x2d]{0,63}$/u); - const matchedEventSchema = z.strictObject({ - event_id: identifierSchema, + event_id: processTraceIdentifierSchema, location: processObservationLocationSchema, }); @@ -32,7 +31,7 @@ const sideResultSchema = z.union([ z.strictObject({ ...sideResultShape, status: z.literal("pass"), - matched_variant: identifierSchema.nullable(), + matched_variant: processTraceIdentifierSchema.nullable(), }), z.strictObject({ ...sideResultShape, @@ -53,7 +52,7 @@ const diagnosticSchema = z.strictObject({ ]), side: z.enum(["left", "right"]), message: z.string(), - event_ids: z.array(identifierSchema), + event_ids: z.array(processTraceIdentifierSchema), locations: z.array(processObservationLocationSchema), }); diff --git a/src/domain/process/processTraceSpecification.ts b/src/domain/process/processTraceSpecification.ts index 09c14cf10..4e28774e0 100644 --- a/src/domain/process/processTraceSpecification.ts +++ b/src/domain/process/processTraceSpecification.ts @@ -9,7 +9,16 @@ import { processObservationSourceSchema, } from "./processObservation.js"; -const identifierSchema = z.string().regex(IDENTIFIER_PATTERN); +/** + * A declared trace, event or variant identifier. The specification owns this + * vocabulary and the comparison result echoes those identifiers verbatim, so + * the result module must reuse this schema rather than restate the pattern: + * two bounds on the same identifier would reject the result of an accepted + * specification. + */ +export const processTraceIdentifierSchema = z + .string() + .regex(IDENTIFIER_PATTERN); const cardinalitySchema = z.discriminatedUnion("kind", [ z.strictObject({ kind: z.literal("required") }), @@ -31,7 +40,7 @@ const cardinalitySchema = z.discriminatedUnion("kind", [ ]); const eventDeclarationSchema = z.strictObject({ - id: identifierSchema, + id: processTraceIdentifierSchema, source: processObservationSourceSchema, exact: jsonValueSchema, ignore_fields: z @@ -56,29 +65,31 @@ const partialOrderLanguageSchema = z.strictObject({ happens_before: z .array( z.strictObject({ - before: identifierSchema, - after: identifierSchema, + before: processTraceIdentifierSchema, + after: processTraceIdentifierSchema, }), ) .default([]), not_before: z .array( z.strictObject({ - event: identifierSchema, - anchor: identifierSchema, + event: processTraceIdentifierSchema, + anchor: processTraceIdentifierSchema, }), ) .default([]), unordered_groups: z - .array(z.strictObject({ events: z.array(identifierSchema).min(2) })) + .array( + z.strictObject({ events: z.array(processTraceIdentifierSchema).min(2) }), + ) .default([]), - prefix: z.array(identifierSchema).default([]), - suffix: z.array(identifierSchema).default([]), + prefix: z.array(processTraceIdentifierSchema).default([]), + suffix: z.array(processTraceIdentifierSchema).default([]), }); const finiteTraceTokenSchema = z.union([ - identifierSchema, - z.strictObject({ unordered: z.array(identifierSchema).min(2) }), + processTraceIdentifierSchema, + z.strictObject({ unordered: z.array(processTraceIdentifierSchema).min(2) }), ]); const finiteTraceLanguageSchema = z.strictObject({ @@ -86,7 +97,7 @@ const finiteTraceLanguageSchema = z.strictObject({ variants: z .array( z.strictObject({ - id: identifierSchema, + id: processTraceIdentifierSchema, trace: z.array(finiteTraceTokenSchema), }), ) From ab62ee9dcf98e4717d0b3445330f7cfdfe50c9f4 Mon Sep 17 00:00:00 2001 From: Stefano Rizzello Date: Tue, 13 Oct 2026 08:49:34 +0200 Subject: [PATCH 048/159] fix(windows): preserve caller affinity only within one processor group (#2202) * fix(windows): preserve caller affinity only within one processor group `spawnProcess` read the caller's affinity mask and applied it to the suspended child unconditionally. A mask names processors relative to one processor group, and Windows assigns each new process to a group of its own choosing, so on hosts with more than 64 logical processors the child is often placed in another group. `SetProcessAffinityMask` then rejects the mask with ERROR_INVALID_PARAMETER (87) and the spawn fails, making every owned provider process start intermittently on such hosts. Affinity is a nicety, not an authority or isolation control, so it is now applied only when the caller and the suspended child both occupy the same single processor group. A child placed in another group keeps that group's default affinity and the spawn proceeds. Job Object membership, kill-on-owner-close policy, owner normalization and priority preservation are unchanged, and the mask failure remains fatal within a shared group, where it is not expected. Single-group hosts, including every host with 64 or fewer logical processors, are unaffected: `GetProcessGroupAffinity` reports one group for both processes and the same mask is applied as before. The optional process-boundary fixture gains an `affinity` observation, and the native verification lane asserts that an owned spawn succeeds and that the child's mask matches the caller's whenever both share a single group. Fixes #2181 Co-Authored-By: Claude Opus 5 * fix(windows): refuse to widen a pinned caller's affinity across groups Addresses review feedback on the first commit. An unconstrained caller loses nothing when Windows places an owned child in another processor group, so that spawn proceeds with the group's default. A caller deliberately pinned to fewer processors than its group holds is a different case: silently accepting the default would hand the provider tree the whole group, which `docs/installation.md` tells operators to rely on affinity to prevent. Such a spawn now fails naming the constraint instead. `GetProcessAffinityMask` already reports the group's active set, which distinguishes the two without another query. Selecting the caller's group at creation through PROC_THREAD_ATTRIBUTE_GROUP_AFFINITY was tried first and rejected: `CreateProcessW` fails with ERROR_INVALID_PARAMETER on a host with a single processor group, both alongside PROC_THREAD_ATTRIBUTE_JOB_LIST and on its own, so it cannot carry the ordinary path. The boundary fixture's affinity observation now reports a process id's own mask rather than its own, because a separate probe process is assigned its own group and cannot stand in for the process that spawns the child. Masks and the group's active set cross as fixed-width hexadecimal strings: a mask can set bits above the safe-integer range, where `JSON.parse` would round two different masks to one value. The verification lane asserts the mask is inherited exactly when caller and child share a group, and otherwise that the caller was unconstrained. Co-Authored-By: Claude Opus 5 * docs(windows): record what owned-process affinity does not preserve A mask covering its group's whole active set does not distinguish a caller the system placed in that group from one that selected the group deliberately, so an owned child can still be placed elsewhere with its processor count bounded and its group placement lost. Treating that case as constrained instead is not available: on a multi-group host an ordinary process is assigned a group by the system and its mask is that group's active set, so refusing it would fail every owned spawn there. Co-Authored-By: Claude Opus 5 --------- Co-authored-by: Claude Opus 5 --- docs/windows-ghidra-p0.md | 8 +++-- native/windows/README.md | 20 ++++++++++--- native/windows/src/process.cc | 30 ++++++++++++++++++- scripts/verify-windows-native.mjs | 36 +++++++++++++++++++++++ tests/fixtures/windows/processBoundary.cc | 36 +++++++++++++++++++++++ 5 files changed, 123 insertions(+), 7 deletions(-) diff --git a/docs/windows-ghidra-p0.md b/docs/windows-ghidra-p0.md index d14add07c..9cd3bb6d8 100644 --- a/docs/windows-ghidra-p0.md +++ b/docs/windows-ghidra-p0.md @@ -153,8 +153,12 @@ the private runtime, isolating Ghidra preferences, logs, and caches. Process creation assigns a suspended child atomically to an owned Job Object. Membership and kill-on-owner-close policy are verified before resume. Breakaway is disabled. Only the child's default file owner is normalized; -the caller token stays unchanged. Caller CPU affinity and priority are -preserved. Close, cancellation, timeout, startup failure, and provider failure +the caller token stays unchanged. Caller priority is preserved, and caller CPU +affinity whenever the caller and the child share a processor group. A child that +Windows places in another group keeps that group's default when the caller asked +for no bound of its own; a caller pinned to fewer processors than its group +holds is refused there rather than widened. +Close, cancellation, timeout, startup failure, and provider failure terminate the job and verify all members have settled before runtime cleanup. Cleanup removes admitted objects by handle and unlinks reparse entries without following their targets. Abrupt owner death kills the job but can leave diff --git a/native/windows/README.md b/native/windows/README.md index c63540140..e2543260f 100644 --- a/native/windows/README.md +++ b/native/windows/README.md @@ -69,10 +69,22 @@ without this lane reports Windows native controls as unavailable. limits only for its JavaScript helpers when running on Bun, which ignores their V8 heap flags; Node.js launches are unchanged. - Only the suspended child's default file owner is normalized. Caller token - privileges and ownership remain unchanged. Caller CPU affinity and process - priority are preserved in the child. Environment names are sorted and - deduplicated using Windows ordinal comparison, preserving their native Unicode - semantics and caller-selected values. + privileges and ownership remain unchanged. Caller process priority is + preserved in the child, and caller CPU affinity whenever both occupy the same + processor group. Windows chooses a new process's group itself and rejects a + mask belonging to another group, so a child placed elsewhere keeps that + group's default when the caller asked for no bound of its own. A caller + deliberately pinned to fewer processors than its group holds is a constraint + such a child cannot express, and is refused rather than widened to that + group's default. A mask covering its group's whole active set does not + distinguish a caller Windows placed in that group from one that selected the + group deliberately, so such a child can still be placed elsewhere: its + processor count stays bounded, its group placement is not preserved, and + neither that case nor the refusal above has a verification lane, because both + need a host whose processors span more than one group. + Environment names are sorted and deduplicated using Windows + ordinal comparison, preserving their native Unicode semantics and + caller-selected values. - Cleanup walks locked directory objects, removes files by handle, and unlinks reparse entries without traversing their targets. Failed removal retains the root and ancestor leases for retry; a partially removed runtime accepts only diff --git a/native/windows/src/process.cc b/native/windows/src/process.cc index ede2b5622..80a4d0ac5 100644 --- a/native/windows/src/process.cc +++ b/native/windows/src/process.cc @@ -68,6 +68,18 @@ static ULONGLONG jobMemoryLimit(napi_env env, napi_value value) { return limit; } +// An affinity mask is meaningful only inside one processor group, so only a +// process occupying exactly one group has a group a child can inherit. A +// process whose threads span groups, or whose membership cannot be read, has no +// single group. +static bool singleProcessorGroup(HANDLE process, USHORT& group) { + USHORT count = 1; + std::array groups{}; + if (!GetProcessGroupAffinity(process, &count, groups.data()) || count != 1) return false; + group = groups[0]; + return true; +} + static void pipe(Handle& read, Handle& write, bool parentReads = true) { SECURITY_ATTRIBUTES attributes{sizeof(SECURITY_ATTRIBUTES), nullptr, TRUE}; HANDLE rawRead, rawWrite; @@ -168,11 +180,27 @@ std::unique_ptr spawnProcess(const std::wstring& command, const std::ws TOKEN_OWNER owner{reinterpret_cast(tokenUser.data())->User.Sid}; require(SetTokenInformation(token.get(), TokenOwner, &owner, sizeof(owner)), "Set suspended process file owner failed", command); + // A mask names processors inside one processor group, and Windows assigns a + // new process to a group of its own choosing, so a mask belonging to another + // group is rejected (ERROR_INVALID_PARAMETER). An unconstrained caller, whose + // mask is its group's whole active set, asks for no bound the child's own + // group cannot give, so such a child keeps that group's default and the spawn + // proceeds. A narrower mask is a deliberate CPU constraint that a child in + // another group cannot express, and silently widening it would hand the + // provider tree the whole group, so that spawn fails naming the constraint. DWORD_PTR affinity = 0, systemAffinity = 0; require(GetProcessAffinityMask(GetCurrentProcess(), &affinity, &systemAffinity), "Read caller process affinity failed", command); - if (affinity != 0) + USHORT callerGroup = 0, childGroup = 0; + const bool sharedGroup = singleProcessorGroup(GetCurrentProcess(), callerGroup) && + singleProcessorGroup(result->process.get(), childGroup) && + callerGroup == childGroup; + if (affinity != 0 && sharedGroup) require(SetProcessAffinityMask(result->process.get(), affinity), "Preserve caller process affinity failed", command); + else + require(affinity == 0 || affinity == systemAffinity, + "Caller CPU affinity cannot bound an owned process in another processor group", command, + ERROR_INVALID_PARAMETER); const DWORD priority = GetPriorityClass(GetCurrentProcess()); require(priority != 0 && SetPriorityClass(result->process.get(), priority), "Preserve caller process priority failed", command); diff --git a/scripts/verify-windows-native.mjs b/scripts/verify-windows-native.mjs index 3db2613ad..8623081f9 100644 --- a/scripts/verify-windows-native.mjs +++ b/scripts/verify-windows-native.mjs @@ -662,6 +662,42 @@ setInterval(()=>{},1000);\n`, unicodeNamesPreserved: true, }); report.processes.windowsEnvironmentSemantics = true; + // Observed through this process's own id: a separate probe process would + // get its own processor group, so it cannot stand in for the process that + // spawns the child. + const callerAffinity = JSON.parse( + execFileSync(boundaryFixture, ["affinity", String(process.pid)], { + encoding: "utf8", + }), + ); + const affinityChild = native.call("process_spawn", [ + boundaryFixture, + `${quote(boundaryFixture)} affinity`, + workspace, + environment, + ]); + const observedAffinity = await wait(affinityChild); + native.call("process_close", [affinityChild.handle]); + assert.equal(observedAffinity.exitCode, 0); + const childAffinity = JSON.parse(observedAffinity.output); + // Masks cross as hexadecimal strings because a mask can exceed the + // safe-integer range. Sharing one processor group makes the caller's mask + // applicable, so it must be inherited exactly. Windows placing the child in + // another group is only permitted when the caller asked for no bound of its + // own, which its group's whole active set establishes; a narrower mask has + // to fail the spawn rather than widen to that group's default. + const sharedGroup = + callerAffinity.singleProcessorGroup && + childAffinity.singleProcessorGroup && + callerAffinity.processorGroup === childAffinity.processorGroup; + if (sharedGroup) + assert.equal(childAffinity.affinityMask, callerAffinity.affinityMask); + else assert.equal(callerAffinity.affinityMask, callerAffinity.systemMask); + report.processes.callerAffinityBoundsOwnedProcessOrRefusesToWiden = true; + report.callerAffinity = { + ...callerAffinity, + sharedGroupWithChild: sharedGroup, + }; } const ownerFixture = join(workspace, "owner.mjs"); const ownerPidFile = join(workspace, "crash-child.pid"); diff --git a/tests/fixtures/windows/processBoundary.cc b/tests/fixtures/windows/processBoundary.cc index 18dcb2dbd..05ee218e8 100644 --- a/tests/fixtures/windows/processBoundary.cc +++ b/tests/fixtures/windows/processBoundary.cc @@ -4,6 +4,7 @@ #include #include #include +#include #include #include @@ -33,6 +34,41 @@ int wmain(int count, wchar_t** args) { value(L"REA_\u00df") == expectedSharp && value(L"REA_SS") == L"double" ? "true" : "false"); return 0; } + if ((count == 2 || count == 3) && std::wstring(args[1]) == L"affinity") { + // Without a process id this observes itself, which is what an owned child + // reports. With one it observes that process, so a caller's own affinity is + // read from the process that spawns the child rather than from a separate + // probe Windows may place in another processor group. + HANDLE opened = nullptr; + if (count == 3) { + opened = OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, FALSE, + static_cast(std::wcstoul(args[2], nullptr, 10))); + if (!opened) return 9; + } + const HANDLE target = opened ? opened : GetCurrentProcess(); + DWORD_PTR affinity = 0, systemAffinity = 0; + USHORT groupCount = 1; + USHORT groups[1]{}; + const bool read = GetProcessAffinityMask(target, &affinity, &systemAffinity) != 0; + const bool single = read && GetProcessGroupAffinity(target, &groupCount, groups) && groupCount == 1; + if (opened) CloseHandle(opened); + if (!read) return 8; + // A mask can set bits above JavaScript's safe-integer range, so masks cross + // as fixed-width hexadecimal strings and are compared without rounding. The + // group's own active set distinguishes an unconstrained process from one + // deliberately pinned to fewer processors. + auto mask = [](DWORD_PTR value, char* text) { + std::snprintf(text, 19, "0x%08lX%08lX", static_cast(value >> 32), + static_cast(value & 0xFFFFFFFFu)); + return text; + }; + char affinityText[19]{}, systemText[19]{}; + std::printf("{\"affinityMask\":\"%s\",\"systemMask\":\"%s\"," + "\"singleProcessorGroup\":%s,\"processorGroup\":%u}\n", + mask(affinity, affinityText), mask(systemAffinity, systemText), + single ? "true" : "false", single ? static_cast(groups[0]) : 0u); + return 0; + } if (count == 4 && std::wstring(args[1]) == L"reparse-directory") { // Only the conformance lane's disposable, nonempty source directory is used. const auto substitute = L"\\??\\" + std::wstring(args[3]); From 68ecf3df8c666fdbe1ced8089c64340af614d1c3 Mon Sep 17 00:00:00 2001 From: chAngE <2815456271@qq.com> Date: Tue, 13 Oct 2026 14:49:47 +0800 Subject: [PATCH 049/159] fix(browser): exclude sensitive binary lengths from historical captures (#2477) Co-authored-by: chAngE <172472630+chAng-L19@users.noreply.github.com> --- bridge/mitmproxy/capture.py | 2 +- .../fixtures/generate-mitmproxy-capture.py | 14 +++ src/browser/history/HarCapture.ts | 3 +- .../browser/historicalWebCapture.test.ts | 88 +++++++++++++++++++ 4 files changed, 105 insertions(+), 2 deletions(-) diff --git a/bridge/mitmproxy/capture.py b/bridge/mitmproxy/capture.py index fa0535c02..dcd90e745 100644 --- a/bridge/mitmproxy/capture.py +++ b/bridge/mitmproxy/capture.py @@ -93,7 +93,7 @@ def project_record(state, secrets): safe_bytes, credential_url = transport_value(value, path, transport_url) content_base64 = base64.b64encode(value).decode("ascii") digest = hashlib.sha256(value).hexdigest() - hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest for secret in secrets) + hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest or secret in str(len(value)) for secret in secrets) safe_text = None try: safe_text = safe_bytes.decode("utf-8", errors="strict") diff --git a/scripts/fixtures/generate-mitmproxy-capture.py b/scripts/fixtures/generate-mitmproxy-capture.py index 4be3dd4a7..880c5e690 100644 --- a/scripts/fixtures/generate-mitmproxy-capture.py +++ b/scripts/fixtures/generate-mitmproxy-capture.py @@ -45,6 +45,20 @@ class Generator: assert "ows-secret" not in json.dumps(projected), projected if isinstance(value, bytes): assert projected["binary_fields"][-1]["state"] == "redacted", projected + body = b"A" * 12345 + state = {"response": {"content": body}} + control = adapter.project_record(state, []) + field = control["binary_fields"][0] + assert field["state"] == "retained" and field["bytes"] == len(body), control + for literal in ["12345", "2345"]: + # Only the generated byte count matches this exclusion. + assert literal not in control["reported"]["response"]["content"] + assert literal not in field["content_base64"] and literal not in field["sha256"] + projected = adapter.project_record(state, [literal]) + assert projected["reported"]["response"]["content"] is None, projected + assert projected["binary_fields"] == [{"pointer": "/response/content", "representation": "producer-bytes", "state": "redacted", "content_base64": None, "bytes": None, "sha256": None}], projected + assert {"pointer": "/response/content", "reason": "explicit-sensitive-value"} in projected["redactions"], projected + assert literal not in json.dumps(projected), projected (root / "native-error-classifications.json").write_text(json.dumps({"passed": len(classifications) + 2})) first = http.HTTPFlow(connection.Client(peername=("127.0.0.1", 1), sockname=("127.0.0.1", 2)), connection.Server(address=("example.test", 80))) first.id = "original-producer-id" diff --git a/src/browser/history/HarCapture.ts b/src/browser/history/HarCapture.ts index 83b442f5c..81c5ffb15 100644 --- a/src/browser/history/HarCapture.ts +++ b/src/browser/history/HarCapture.ts @@ -144,7 +144,8 @@ const projectHar = ( (literal) => item.includes(literal) || contentBase64.includes(literal) || - sha256.includes(literal), + sha256.includes(literal) || + String(bytes.length).includes(literal), ); binary_fields.push( hidden diff --git a/tests/boundary/browser/historicalWebCapture.test.ts b/tests/boundary/browser/historicalWebCapture.test.ts index a0a9f10e2..781172cd3 100644 --- a/tests/boundary/browser/historicalWebCapture.test.ts +++ b/tests/boundary/browser/historicalWebCapture.test.ts @@ -7,6 +7,9 @@ import { execFile } from "node:child_process"; import { fileURLToPath } from "node:url"; import { promisify } from "node:util"; import { parseEvidence } from "../../../src/domain/evidence.js"; +import { webNetworkCaptureSchema } from "../../../src/domain/webNetworkCapture.js"; +import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; const entrypoint = fileURLToPath( new URL("../../../scripts/rea.mjs", import.meta.url), ); @@ -41,6 +44,91 @@ const cli = async ( }; import { historicalHar } from "../../fixtures/historicalHar.js"; +it("excludes declared binary lengths through CLI, MCP and retained capture pages", async () => { + const root = await createTestTempDirectory("rea-har-byte-length-"); + const path = join(root, "capture.har"); + const fixture = historicalHar(); + const entry = fixture.log.entries[0]; + if (entry === undefined) throw new Error("fixture missing"); + const body = Buffer.alloc(12_345, "A"); + const content = body.toString("base64"); + const digest = createHash("sha256").update(body).digest("hex"); + await writeFile( + path, + JSON.stringify({ + log: { + ...fixture.log, + entries: [ + { + ...entry, + response: { + ...entry.response, + content: { + size: 99, + mimeType: "text/plain", + encoding: "base64", + text: content, + }, + }, + }, + ], + }, + }), + ); + const { call } = await connectLocalToolsMcp(); + const args = { capture_path: path, format: "har" }; + const control = await call("inspect_web_network_capture", args); + expect(control.isError).not.toBe(true); + const retained = parseEvidence(control.structuredContent); + const controlCapture = webNetworkCaptureSchema.parse( + retained.normalized_result, + ); + expect(controlCapture.records[0]?.binary_fields[0]).toMatchObject({ + state: "retained", + bytes: body.length, + content_base64: content, + sha256: digest, + }); + for (const literal of ["12345", "2345"]) { + // Only the derived count matches, so this cannot pass via another exclusion. + expect(content).not.toContain(literal); + expect(digest).not.toContain(literal); + const fromCli = await cli(path, "har", ["--sensitive-value", literal]); + expect(fromCli.ok, fromCli.stdout).toBe(true); + const evidence = parseEvidence(JSON.parse(fromCli.stdout)); + const response = await call("inspect_web_network_capture", { + ...args, + sensitive_values: [literal], + }); + expect(response.isError).not.toBe(true); + const fromMcp = parseEvidence(response.structuredContent); + expect(fromMcp.normalized_result).toEqual(evidence.normalized_result); + const capture = webNetworkCaptureSchema.parse(evidence.normalized_result); + expect(capture.records[0]?.reported).toMatchObject({ + response: { content: { size: 99, text: null } }, + }); + expect(capture.records[0]?.binary_fields[0]).toEqual({ + pointer: "/response/content/text", + representation: "har-base64-content", + state: "redacted", + bytes: null, + content_base64: null, + sha256: null, + }); + for (const detail of ["preview", "complete"]) { + const page = await call("inspect_analysis_view", { + source: { kind: "retained-evidence", evidence_id: fromMcp.evidence_id }, + view: { kind: "capture", detail }, + }); + expect(page.isError).not.toBe(true); + const result = parseEvidence(page.structuredContent); + expect(result.normalized_result).toMatchObject({ + items: [{ binary_fields: [{ state: "redacted", bytes: null }] }], + }); + } + } +}, 60_000); + it("reads actual upstream HAR packages in an owned process and retains selected producer order", async () => { const root = await mkdtemp(join(tmpdir(), "rea-history-har-")); onTestFinished(() => rm(root, { recursive: true, force: true })); From e74e4e951bf59c42b25779b45d1c7b8dde6b618f Mon Sep 17 00:00:00 2001 From: Rudy Mizrahi Celekli <47457359+rudycelekli@users.noreply.github.com> Date: Tue, 13 Oct 2026 02:50:44 -0400 Subject: [PATCH 050/159] fix(screenshot): reject invalid PNG header dimensions (#2472) --- src/domain/pngHeader.test.ts | 21 +++++++++++++ src/domain/pngHeader.ts | 8 ++++- src/native/NativeUiObservation.test.ts | 31 +++++++++++++++++++ .../cdpScreenshotDocumentIdentity.test.ts | 31 +++++++++++++++++++ 4 files changed, 90 insertions(+), 1 deletion(-) diff --git a/src/domain/pngHeader.test.ts b/src/domain/pngHeader.test.ts index e8df531bc..676a832e0 100644 --- a/src/domain/pngHeader.test.ts +++ b/src/domain/pngHeader.test.ts @@ -33,3 +33,24 @@ it("rejects incomplete headers, invalid signatures and zero dimensions", () => { expect(readPngHeaderDimensions(zero)).toBeUndefined(); } }); + +it.each([0, 12, 14, 0xffff_ffff])("rejects IHDR data length %s", (length) => { + const bytes = Buffer.from(png); + bytes.writeUInt32BE(length, 8); + expect(readPngHeaderDimensions(bytes)).toBeUndefined(); +}); + +it.each([16, 20])("rejects out-of-range PNG dimension at %s", (offset) => { + const bytes = Buffer.from(png); + bytes.writeUInt32BE(0x8000_0000, offset); + expect(readPngHeaderDimensions(bytes)).toBeUndefined(); +}); + +it("accepts the largest dimension representable by the PNG specification", () => { + const bytes = Buffer.from(png); + bytes.writeUInt32BE(0x7fff_ffff, 16); + expect(readPngHeaderDimensions(bytes)).toEqual({ + width: 0x7fff_ffff, + height: 1, + }); +}); diff --git a/src/domain/pngHeader.ts b/src/domain/pngHeader.ts index 58cdd7d3f..5ab8c7c3d 100644 --- a/src/domain/pngHeader.ts +++ b/src/domain/pngHeader.ts @@ -8,10 +8,16 @@ export const readPngHeaderDimensions = ( if ( bytes.byteLength < 24 || !bytes.subarray(0, 8).equals(PNG_SIGNATURE) || + bytes.readUInt32BE(8) !== 13 || !bytes.subarray(12, 16).equals(IHDR_TYPE) ) return undefined; const width = bytes.readUInt32BE(16); const height = bytes.readUInt32BE(20); - return width === 0 || height === 0 ? undefined : { width, height }; + return width === 0 || + height === 0 || + width > 0x7fff_ffff || + height > 0x7fff_ffff + ? undefined + : { width, height }; }; diff --git a/src/native/NativeUiObservation.test.ts b/src/native/NativeUiObservation.test.ts index d2238bd01..527afd8db 100644 --- a/src/native/NativeUiObservation.test.ts +++ b/src/native/NativeUiObservation.test.ts @@ -35,6 +35,37 @@ const validPng = Buffer.from( "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+jv4sAAAAASUVORK5CYII=", "base64", ); +it.each([ + { offset: 8, value: 12 }, + { offset: 16, value: 0x8000_0000 }, + { offset: 20, value: 0x8000_0000 }, +])( + "rejects invalid IHDR metadata with a matching digest", + async ({ offset, value }) => { + const bytes = Buffer.from(validPng); + bytes.writeUInt32BE(value, offset); + const screenshot = { + mime_type: "image/png", + base64: bytes.toString("base64"), + sha256: createHash("sha256").update(bytes).digest("hex"), + width: bytes.readUInt32BE(16), + height: bytes.readUInt32BE(20), + }; + const result = await observeNativeUi( + target, + "observe_native_ui", + { ...scope, screenshot: true }, + { + environment: {}, + invoke: async () => ({ + ok: true, + result: { ...snapshot, screenshot }, + }), + }, + ); + expect(result.ok).toBe(false); + }, +); describe("native UI screenshot validation", () => { it("rejects malformed or mismatched screenshot bytes from the helper", async () => { const screenshot = { diff --git a/tests/boundary/browser/cdpScreenshotDocumentIdentity.test.ts b/tests/boundary/browser/cdpScreenshotDocumentIdentity.test.ts index 012ad9663..b3fcbc592 100644 --- a/tests/boundary/browser/cdpScreenshotDocumentIdentity.test.ts +++ b/tests/boundary/browser/cdpScreenshotDocumentIdentity.test.ts @@ -3,6 +3,37 @@ import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js"; import { captureWebScreenshotInputSchema } from "../../../src/domain/webScreenshot.js"; import { startFakeCdpBrowser } from "../../fixtures/fakeCdpBrowser.js"; describe("screenshot committed document identity", () => { + it.each([ + { offset: 8, value: 14 }, + { offset: 16, value: 0x8000_0000 }, + { offset: 20, value: 0x8000_0000 }, + ])("rejects invalid screenshot IHDR metadata", async ({ offset, value }) => { + const bytes = Buffer.from( + "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+jv4sAAAAASUVORK5CYII=", + "base64", + ); + bytes.writeUInt32BE(value, offset); + const browser = await startFakeCdpBrowser({ + commandResult: ({ method }) => + method === "Page.captureScreenshot" + ? { data: bytes.toString("base64") } + : undefined, + }); + onTestFinished(async () => { + await browser.close(); + }); + const result = await new CdpBrowserProvider().captureScreenshot( + captureWebScreenshotInputSchema.parse({ + cdp_endpoint: browser.endpoint, + allowed_origins: [browser.allowedOrigin], + target_id: "allowed-page", + }), + ); + expect(result).toMatchObject({ + ok: false, + error: { reason: "protocol_error" }, + }); + }); it.each(["loader-main", "loader-reloaded"])( "distinguishes %s for the same URL", async (screenshotDocumentLoader) => { From 6a04f922e16ad4d171ba0c07e32238efdaf43e2d Mon Sep 17 00:00:00 2001 From: Rudy Mizrahi Celekli <47457359+rudycelekli@users.noreply.github.com> Date: Tue, 13 Oct 2026 02:51:23 -0400 Subject: [PATCH 051/159] fix(javascript): parse explicit CommonJS wrapper syntax (#2475) --- .../javascript/javascriptSourceParser.ts | 59 +++++++++++++------ ...eferenceSourceTypeScriptExtensions.test.ts | 25 ++++++++ .../process/javascriptSourceModes.test.ts | 39 ++++++++++++ 3 files changed, 106 insertions(+), 17 deletions(-) create mode 100644 tests/boundary/process/javascriptSourceModes.test.ts diff --git a/src/domain/javascript/javascriptSourceParser.ts b/src/domain/javascript/javascriptSourceParser.ts index ce5619ead..630d5e9bf 100644 --- a/src/domain/javascript/javascriptSourceParser.ts +++ b/src/domain/javascript/javascriptSourceParser.ts @@ -37,10 +37,19 @@ export const parserPluginsForPath = (sourcePath?: string): ParserPlugin[] => { : ["decorators-legacy", typescript]; }; -/** `.mjs` and `.mts` are modules by extension; other paths leave the mode to syntax. */ +/** `.mjs` and `.mts` explicitly select module mode. */ export const isExplicitModuleSourcePath = (path?: string): boolean => /\.m[jt]s$/iu.test(path ?? ""); +const runtimeSourceTypeForPath = ( + path?: string, +): "commonjs" | "module" | "unambiguous" => + /\.cjs$/iu.test(path ?? "") + ? "commonjs" + : isExplicitModuleSourcePath(path) + ? "module" + : "unambiguous"; + export interface RecoveredJavaScriptParse { readonly file: ParsedJavaScriptSource; /** Errors that apply only if a source of unknown mode is an ES module. */ @@ -58,25 +67,25 @@ export const parseRecoveredJavaScript = ( sourcePath: string | undefined, options: Omit, ): RecoveredJavaScriptParse => { - // `.cjs` is CommonJS by extension, so module-only errors cannot apply. - if (/\.cjs$/iu.test(sourcePath ?? "")) + const sourceType = runtimeSourceTypeForPath(sourcePath); + if (sourceType !== "unambiguous") { + const normalizedOptions = { ...options }; + if (sourceType === "commonjs") { + // Babel owns these wrapper semantics in CommonJS mode and rejects + // explicit script overrides, including the reference-import options. + delete normalizedOptions.allowAwaitOutsideFunction; + delete normalizedOptions.allowReturnOutsideFunction; + delete normalizedOptions.allowNewTargetOutsideFunction; + } return { file: parse(source, { - ...options, - sourceType: "script", - errorRecovery: true, - }), - moduleOnlyErrors: [], - }; - if (isExplicitModuleSourcePath(sourcePath)) - return { - file: parse(source, { - ...options, - sourceType: "module", + ...normalizedOptions, + sourceType, errorRecovery: true, }), moduleOnlyErrors: [], }; + } const file = parse(source, { ...options, sourceType: "unambiguous", @@ -98,18 +107,34 @@ export const parseRecoveredJavaScript = ( }; }; -/** Parse JavaScript or TypeScript once without attaching comments to AST nodes. */ +/** Parse inert JavaScript or TypeScript facts without attaching comments to AST nodes. */ export const parseJavaScriptSource = ( source: string, sourcePath?: string, ): ParsedJavaScriptSource | null => { try { - return parse(source, { - sourceType: "unambiguous", + const runtimeSourceType = runtimeSourceTypeForPath(sourcePath); + const commonjs = runtimeSourceType === "commonjs"; + const options = { errorRecovery: true, attachComment: false, plugins: parserPluginsForPath(sourcePath), + }; + const file = parse(source, { + ...options, + // Inert artifact analysis retains module declarations even in .cjs assets. + // Infer their syntax here; reference imports keep filename-mode diagnostics. + sourceType: commonjs ? "unambiguous" : runtimeSourceType, + allowReturnOutsideFunction: commonjs, + allowNewTargetOutsideFunction: commonjs, }); + // Unambiguous mode retains module-only diagnostics after choosing a script. + // The .cjs extension establishes wrapper mode, so obtain its own diagnostics. + return commonjs && + file.program.sourceType === "script" && + file.errors.length > 0 + ? parse(source, { ...options, sourceType: "commonjs" }) + : file; } catch (cause: unknown) { // Unparseable source is represented by the null return. void cause; diff --git a/tests/boundary/filesystem/referenceSourceTypeScriptExtensions.test.ts b/tests/boundary/filesystem/referenceSourceTypeScriptExtensions.test.ts index d537a1dca..7b3d8189c 100644 --- a/tests/boundary/filesystem/referenceSourceTypeScriptExtensions.test.ts +++ b/tests/boundary/filesystem/referenceSourceTypeScriptExtensions.test.ts @@ -6,6 +6,31 @@ import { expect, it } from "vitest"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { importReferenceSource } from "../../support/referenceSourceResourceScope.js"; +it("imports valid CommonJS wrapper syntax without inventing a parse failure", async () => { + const root = await createTestTempDirectory("rea-reference-commonjs-mode-"); + await Promise.all([ + writeFile( + join(root, "main.cjs"), + 'void new.target; require("./dep.cjs"); return;\n', + ), + writeFile(join(root, "dep.cjs"), "module.exports = 1;\n"), + ]); + const imported = await importReferenceSource({ + root, + caller: "native-commonjs-source", + policy: { secretPatterns: [] }, + }); + if (!imported.ok) throw imported.error; + expect(imported.value.parse_failures).toEqual([]); + expect(imported.value.relationships).toContainEqual({ + from_path: "main.cjs", + to: "dep.cjs", + kind: "requires", + resolution: "internal", + parse_state: "parsed", + }); +}); + it.each([ ["main.ts", "dep.js", "dep.ts"], ["main.mts", "dep.mjs", "dep.mts"], diff --git a/tests/boundary/process/javascriptSourceModes.test.ts b/tests/boundary/process/javascriptSourceModes.test.ts new file mode 100644 index 000000000..64c15a2d8 --- /dev/null +++ b/tests/boundary/process/javascriptSourceModes.test.ts @@ -0,0 +1,39 @@ +import { expect } from "vitest"; + +import { parseJavaScriptSource } from "../../../src/domain/javascript/javascriptSourceParser.js"; +import { processTest } from "../../support/process/processFixture.js"; + +for (const source of [ + "return;", + "void new.target;", + "var await = 1; return;", + "with ({ value: 1 }) { void value; } void new.target;", +]) { + processTest( + `accepts Node CommonJS syntax while retaining module errors: ${source}`, + async ({ processes }) => { + const commonjs = await processes.run( + process.execPath, + ["--check", "--input-type=commonjs"], + { input: source, timeoutMs: 5_000 }, + ); + expect(commonjs.exitCode).toBe(0); + const module = await processes.run( + process.execPath, + ["--check", "--input-type=module"], + { input: source, timeoutMs: 5_000 }, + ); + expect(module.exitCode).not.toBe(0); + + const parsed = parseJavaScriptSource(source, "main.cjs"); + expect(parsed).not.toBeNull(); + expect(parsed?.errors).toEqual([]); + expect( + parseJavaScriptSource(source, "main.mjs")?.errors.length, + ).toBeGreaterThan(0); + expect( + parseJavaScriptSource(source, "main.mts")?.errors.length, + ).toBeGreaterThan(0); + }, + ); +} From a07ceaec36346a7c73e66efe35c8af4654029c84 Mon Sep 17 00:00:00 2001 From: Rudy Mizrahi Celekli <47457359+rudycelekli@users.noreply.github.com> Date: Tue, 13 Oct 2026 02:51:29 -0400 Subject: [PATCH 052/159] fix(config): reject NUL bytes in Hopper loader arguments (#2474) --- src/config.test.ts | 12 ++++++++++++ src/config/parsers.ts | 13 +++++-------- 2 files changed, 17 insertions(+), 8 deletions(-) diff --git a/src/config.test.ts b/src/config.test.ts index eb784802f..67db7e111 100644 --- a/src/config.test.ts +++ b/src/config.test.ts @@ -3,6 +3,18 @@ import { describe, expect, it } from "vitest"; import { parseConfig } from "./config/parseConfig.js"; import { projectAnalysisError } from "./domain/analysisErrorProjection.js"; +it("reports a configuration failure for a NUL in Hopper loader arguments", () => { + const result = parseConfig({ + HOPPER_LOADER_ARGS_JSON: JSON.stringify(["--option", "value\0suffix"]), + }); + if (result.ok) + throw new Error("NUL argv cannot be passed to a child process"); + expect(result.error).toMatchObject({ + settings: [{ setting: "HOPPER_LOADER_ARGS_JSON" }], + }); + expect(result.error.message).toContain("NUL"); +}); + describe("runtime configuration", () => { it("allows target-free startup and applies runtime defaults", () => { const empty = parseConfig({}); diff --git a/src/config/parsers.ts b/src/config/parsers.ts index fc842f884..52feae622 100644 --- a/src/config/parsers.ts +++ b/src/config/parsers.ts @@ -1,6 +1,7 @@ import { z } from "zod"; import { ConfigurationError } from "../domain/configurationErrors.js"; +import { nulFreeStringSchema } from "../domain/localPath.js"; import { err, ok, type Result } from "../domain/result.js"; import { safeParseJson } from "../domain/safeJson.js"; @@ -20,7 +21,8 @@ export const parseLoaderArgs = ( return parseConfiguredStringArray( encoded, "HOPPER_LOADER_ARGS_JSON", - z.string(), + nulFreeStringSchema, + "must encode an array of strings without NUL bytes", ); }; @@ -28,6 +30,7 @@ const parseConfiguredStringArray = ( encoded: string, setting: string, itemSchema: z.ZodString, + requirement = "must encode an array of strings", ): Result => { const decoded = safeParseJson(encoded); if (!decoded.ok) @@ -37,13 +40,7 @@ const parseConfiguredStringArray = ( const parsed = z.array(itemSchema).safeParse(decoded.value); return parsed.success ? ok(parsed.data) - : err( - configuredArrayError( - setting, - "must encode an array of strings", - parsed.error, - ), - ); + : err(configuredArrayError(setting, requirement, parsed.error)); }; const configuredArrayError = ( From f6775a36694a6a9c620f4248638e8056a6cbcb66 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:51:55 +0800 Subject: [PATCH 053/159] fix(setup): preserve current recovery snapshots and decoding failures --- docs/installation.md | 4 + src/application/RegularFileRead.ts | 65 ++-- src/application/Setup.recovery.test.ts | 34 ++ src/application/SetupClientConfiguration.ts | 175 +++++++++- src/application/SetupClients.ts | 18 +- src/application/SetupHost.ts | 8 +- src/application/SetupPlan.ts | 2 +- src/application/SetupTypes.ts | 10 + src/application/Uninstall.ts | 12 +- .../clientConfigurationDiagnosticsCli.test.ts | 17 +- .../clientConfigurationTransactions.test.ts | 80 ++++- .../setupClientConfigCleanupOwnership.test.ts | 227 +++++++----- .../setupConfigurationRollback.test.ts | 325 ++++++++++++++++++ 13 files changed, 835 insertions(+), 142 deletions(-) create mode 100644 tests/process-global/setupConfigurationRollback.test.ts diff --git a/docs/installation.md b/docs/installation.md index 0a8041898..3bd4e0928 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -57,6 +57,10 @@ Hopper installation or any file write. Declining or pressing Ctrl-C makes no changes. Agent configuration writes preserve unrelated entries and comments, create backups, use atomic replacement, and verify their result. Setup and uninstall retain an existing `.rea.backup` rather than replacing the first snapshot. +If that snapshot differs from the current configuration, setup previews and +preserves a verified `.rea.backup..recovery` copy of the current bytes. +Both snapshots remain available; a failed rollback reports the current recovery +copy's path and the restore failure. Setup also retains valid environment overrides in REA's existing server entry, such as `REA_MCP_INPUT_SCHEMA_PROFILE` and `REA_MCP_MAX_RESPONSE_BYTES`. Detected provider settings replace only matching environment keys. diff --git a/src/application/RegularFileRead.ts b/src/application/RegularFileRead.ts index f09f4c192..ee4d6170a 100644 --- a/src/application/RegularFileRead.ts +++ b/src/application/RegularFileRead.ts @@ -19,6 +19,17 @@ export interface RegularFileReadOptions { readonly resources?: ArtifactResourceScope; } +export class InvalidUtf8FileError extends TypeError { + readonly _tag = "InvalidUtf8FileError"; + constructor( + readonly path: string, + cause: unknown, + ) { + super(`File is not valid UTF-8: ${path}`, { cause }); + this.name = this._tag; + } +} + export type RegularFileReadOutcome = | { readonly kind: "completed"; readonly value: Value } | { readonly kind: "failed"; readonly cause: unknown }; @@ -142,7 +153,34 @@ const withRegularFileOwned = async ( export const readRegularFile = ( path: string, options: RegularFileReadOptions = {}, -): Promise => +): Promise => readRegularFileContent(path, (bytes) => bytes, options); + +/** Read valid UTF-8 without replacing source bytes or stripping an initial BOM. */ +export const readRegularFileText = ( + path: string, + options: RegularFileReadOptions = {}, +): Promise => + readRegularFileContent( + path, + (bytes) => { + try { + return new TextDecoder("utf-8", { + fatal: true, + ignoreBOM: true, + }).decode(bytes); + } catch (cause: unknown) { + if (!(cause instanceof TypeError)) throw cause; + throw new InvalidUtf8FileError(path, cause); + } + }, + options, + ); + +const readRegularFileContent = ( + path: string, + decode: (bytes: Buffer) => Value, + options: RegularFileReadOptions, +): Promise => withRegularFile( path, async (handle, stats) => { @@ -153,30 +191,7 @@ export const readRegularFile = ( ); if (bytes === undefined || bytes.length !== stats.size) throw new RegularFileChangedError(path); - return bytes; + return decode(bytes); }, options, ); - -/** Read one admitted regular file as UTF-8 without waiting on a pipe. */ -export const readRegularFileText = async ( - path: string, - options: RegularFileReadOptions = {}, -): Promise => { - try { - return (await readRegularFile(path, options)).toString("utf8"); - } catch (cause: unknown) { - if ( - cause instanceof RegularFileCleanupFailure && - cause.outcome.kind === "completed" && - Buffer.isBuffer(cause.outcome.value) - ) - throw new RegularFileCleanupFailure( - cause.path, - cause.owner, - { kind: "completed", value: cause.outcome.value.toString("utf8") }, - cause.cleanupCause, - ); - throw cause; - } -}; diff --git a/src/application/Setup.recovery.test.ts b/src/application/Setup.recovery.test.ts index 951ef3a54..f29c9d375 100644 --- a/src/application/Setup.recovery.test.ts +++ b/src/application/Setup.recovery.test.ts @@ -54,6 +54,40 @@ describe("setup workflow", () => { expect(result.remediation).toContain("could not be updated"); }); + it("reports a failed config rollback with its destination and recovery backup", async () => { + const host = new FakeSetupHost(); + host.hopper = "/Applications/Hopper"; + host.clients = [{ name: "codex", configPath: "/fixture/codex.json" }]; + host.clientResults.set("codex", { + status: "failed", + reason: "rollback", + backupPath: "/fixture/codex.json.rea.backup", + failure: { + kind: "restore", + path: "/fixture/codex.json", + detail: "permission denied", + }, + }); + + const result = await runSetup( + { ...options(true), clientIds: ["codex"] }, + host, + ); + + expect(result.clients.codex).toEqual({ + status: "failed", + reason: "rollback", + backupPath: "/fixture/codex.json.rea.backup", + failure: { + kind: "restore", + path: "/fixture/codex.json", + detail: "permission denied", + }, + }); + expect(result.remediation).toContain("permission denied"); + expect(result.remediation).toContain("/fixture/codex.json.rea.backup"); + }); + it("explains skill installation recovery", async () => { const host = new FakeSetupHost(); host.hopper = "/Applications/Hopper"; diff --git a/src/application/SetupClientConfiguration.ts b/src/application/SetupClientConfiguration.ts index bfab80cbf..981b829a4 100644 --- a/src/application/SetupClientConfiguration.ts +++ b/src/application/SetupClientConfiguration.ts @@ -19,6 +19,7 @@ import { } from "./ClientConfigurationDocument.js"; import { dshDocument, DshCordisDocument } from "./DshCordisPatch.js"; import { constants as fsConstants } from "node:fs"; +import { createHash } from "node:crypto"; import { copyFile, mkdir, rm, writeFile } from "node:fs/promises"; import { dirname } from "node:path"; import writeFileAtomic from "write-file-atomic"; @@ -29,6 +30,7 @@ import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js"; import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js"; import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js"; import { + readRegularFile, readRegularFileText, RegularFileCleanupFailure, } from "./RegularFileRead.js"; @@ -40,6 +42,26 @@ import type { } from "./SetupTypes.js"; import type { SetupClient } from "./SupportedClients.js"; +export class ClientConfigurationCleanupRollbackFailure extends RegularFileCleanupFailure { + constructor( + failure: RegularFileCleanupFailure, + readonly configurationFailure: Extract< + ClientConfigurationResult, + { readonly status: "failed"; readonly reason: "rollback" } + >, + ) { + super( + failure.path, + failure.owner, + failure.outcome, + new AggregateError( + [failure.cleanupCause, new Error(configurationFailure.failure.detail)], + `Configuration readback cleanup and rollback failed: ${configurationFailure.failure.path}`, + ), + ); + } +} + /** Configure one supported client's native stdio MCP registration shape. */ export const configureClientConfiguration = ( client: SetupClient, @@ -167,11 +189,23 @@ const persistClientConfiguration = async ({ }: ClientConfigurationUpdate): Promise => { const backupPath = original === undefined ? undefined : `${client.configPath}.rea.backup`; - if ( - backupPath !== undefined && - !(await preserveConfigBackup(transactionPath, backupPath)) - ) - return { status: "failed", reason: "backup" }; + let recoveryBackupPath: string | undefined; + if (backupPath !== undefined && original !== undefined) { + const selection = await selectCurrentRecoveryBackup(backupPath, original); + if (!selection.ok) return { status: "failed", reason: "backup" }; + if ( + !(await preserveConfigBackup(transactionPath, selection.value.backupPath)) + ) + return { status: "failed", reason: "backup" }; + const verified = await selectCurrentRecoveryBackup(backupPath, original); + if ( + !verified.ok || + verified.value.backupPath !== selection.value.backupPath || + !verified.value.present + ) + return { status: "failed", reason: "backup" }; + recoveryBackupPath = verified.value.backupPath; + } // A native OpenCode V2 entry replaces REA's legacy V1 entry, which would // otherwise conflict with it. const legacyPath = legacyClientServerPath( @@ -239,19 +273,39 @@ const persistClientConfiguration = async ({ !clientConfigurationValuesEqual(readback.servers[name], entry), ) ) { - await restoreConfig(transactionPath, original); - return { status: "failed", reason: "readback" }; + return rollbackOrReadbackFailure( + transactionPath, + original, + recoveryBackupPath, + ); } } catch (cause: unknown) { - if (cause instanceof RegularFileCleanupFailure) throw cause; + if (cause instanceof RegularFileCleanupFailure) { + const rollback = await restoreConfig(transactionPath, original); + if (!rollback.ok) + throw new ClientConfigurationCleanupRollbackFailure(cause, { + status: "failed", + reason: "rollback", + ...(recoveryBackupPath === undefined + ? {} + : { backupPath: recoveryBackupPath }), + failure: rollback.error, + }); + throw cause; + } // Readback failure restores the transaction before reporting. void cause; - await restoreConfig(transactionPath, original); - return { status: "failed", reason: "readback" }; + return rollbackOrReadbackFailure( + transactionPath, + original, + recoveryBackupPath, + ); } return { status: "configured", - ...(backupPath === undefined ? {} : { backupPath }), + ...(recoveryBackupPath === undefined + ? {} + : { backupPath: recoveryBackupPath }), }; }; @@ -406,9 +460,18 @@ export const inspectClientConfiguration = async ( "The existing configuration is malformed. Repair it before setup applies any other changes.", }; } + const backupSelection = await selectCurrentRecoveryBackup( + `${client.configPath}.rea.backup`, + original, + ); + if (!backupSelection.ok) + return { + status: "invalid", + remediation: backupSelection.error.detail, + }; return { status: "update", - backupPath: `${client.configPath}.rea.backup`, + backupPath: backupSelection.value.backupPath, }; }; @@ -473,19 +536,101 @@ const preserveConfigBackup = async ( } }; +interface RecoveryBackupSelection { + readonly backupPath: string; + readonly present: boolean; +} + +interface RecoveryBackupSelectionFailure { + readonly kind: "unreadable" | "occupied"; + readonly path: string; + readonly detail: string; +} + +const selectCurrentRecoveryBackup = async ( + firstBackupPath: string, + original: string, +): Promise> => { + const expected = Buffer.from(original, "utf8"); + let firstBackup: Buffer | undefined; + try { + firstBackup = await readRegularFile(firstBackupPath, { + symlinks: "reject", + }); + } catch (cause: unknown) { + if (cause instanceof RegularFileCleanupFailure) throw cause; + if (!isMissing(cause)) + return err({ + kind: "unreadable", + path: firstBackupPath, + detail: `The recovery backup could not be checked: ${errorDetail(cause)}`, + }); + } + if (firstBackup === undefined) + return ok({ backupPath: firstBackupPath, present: false }); + if (firstBackup.equals(expected)) + return ok({ backupPath: firstBackupPath, present: true }); + + const digest = createHash("sha256").update(original, "utf8").digest("hex"); + const recoveryPath = `${firstBackupPath}.${digest}.recovery`; + let recovery: Buffer | undefined; + try { + recovery = await readRegularFile(recoveryPath, { symlinks: "reject" }); + } catch (cause: unknown) { + if (cause instanceof RegularFileCleanupFailure) throw cause; + if (!isMissing(cause)) + return err({ + kind: "unreadable", + path: recoveryPath, + detail: `The recovery backup could not be checked: ${errorDetail(cause)}`, + }); + } + if (recovery !== undefined && !recovery.equals(expected)) + return err({ + kind: "occupied", + path: recoveryPath, + detail: `The recovery path already contains different data: ${recoveryPath}`, + }); + return ok({ backupPath: recoveryPath, present: recovery !== undefined }); +}; + const restoreConfig = async ( path: string, original: string | undefined, -): Promise => { +): Promise< + Result< + void, + { readonly kind: "restore"; readonly path: string; readonly detail: string } + > +> => { try { if (original === undefined) await rm(path, { force: true }); else await writeFile(path, original, { encoding: "utf8", mode: 0o600 }); + return ok(undefined); } catch (cause: unknown) { - // The backup remains available for the remediation reported by setup. - void cause; + return err({ kind: "restore", path, detail: errorDetail(cause) }); } }; +const rollbackOrReadbackFailure = async ( + targetPath: string, + original: string | undefined, + backupPath: string | undefined, +): Promise => { + const rollback = await restoreConfig(targetPath, original); + return rollback.ok + ? { status: "failed", reason: "readback" } + : { + status: "failed", + reason: "rollback", + ...(backupPath === undefined ? {} : { backupPath }), + failure: rollback.error, + }; +}; + +const errorDetail = (cause: unknown): string => + cause instanceof Error ? cause.message : String(cause); + /** Whether REA's entry matches, no legacy entry remains, and no client disable list suppresses it. */ const registrationCurrent = ( parsed: ClientConfigurationDocument, diff --git a/src/application/SetupClients.ts b/src/application/SetupClients.ts index bbeeedf3f..d86d04b9e 100644 --- a/src/application/SetupClients.ts +++ b/src/application/SetupClients.ts @@ -6,15 +6,19 @@ import type { } from "./SetupTypes.js"; import type { SetupClient } from "./SupportedClients.js"; -const failedConfigurationMessage = ( - reason: "path" | "backup" | "write" | "readback", +export const failedConfigurationMessage = ( + result: Extract, ): string => { - if (reason === "path") + if (result.reason === "path") return "Agent configuration path could not be safely verified. Check its permissions and, if it is a symbolic link, verify that the link resolves to a regular file owned by the current user, then rerun setup."; - if (reason === "backup") + if (result.reason === "backup") return "Agent configuration could not be backed up, so no change was made. Check file permissions, then rerun setup."; - if (reason === "write") + if (result.reason === "write") return "Agent configuration could not be updated. Check file permissions, then rerun setup."; + if (result.reason === "rollback") + return result.backupPath === undefined + ? `Agent configuration rollback failed at ${result.failure.path}: ${result.failure.detail}; the failed first-run file may remain. Remove or repair it before rerunning setup.` + : `Agent configuration rollback failed at ${result.failure.path}: ${result.failure.detail}. Inspect the preserved backup at ${result.backupPath} and restore the intended configuration, then rerun setup.`; return "Agent configuration could not be verified after writing. Repair the configuration file or restore its `.rea.backup`, then rerun setup."; }; @@ -40,12 +44,12 @@ export const configureDetectedClients = async (options: { ); options.clients[client.name] = result; if (result.status === "failed") { - failure ??= failedConfigurationMessage(result.reason); + failure ??= failedConfigurationMessage(result); options.onProgress?.({ actionId, label, state: "failed", - detail: failedConfigurationMessage(result.reason), + detail: failedConfigurationMessage(result), }); } else if (result.status === "configured") { options.appliedActions.push(`configured_${client.name}`); diff --git a/src/application/SetupHost.ts b/src/application/SetupHost.ts index 9e2ae95f1..7ac6828fd 100644 --- a/src/application/SetupHost.ts +++ b/src/application/SetupHost.ts @@ -33,9 +33,11 @@ import { } from "./SetupSkill.js"; import { clientConfigurationAligned, + ClientConfigurationCleanupRollbackFailure, configureClientConfiguration, inspectClientConfiguration, } from "./SetupClientConfiguration.js"; +import { failedConfigurationMessage } from "./SetupClients.js"; import { setupInstallFailure } from "./SetupInstallFailure.js"; import { RegularFileCleanupFailure, @@ -275,8 +277,12 @@ const runClientConfigurationOperation = async ( cleanup.outcome.kind === "failed" ? cleanup.outcome.cause : cause.cleanupCause; + const rollbackMessage = + cause instanceof ClientConfigurationCleanupRollbackFailure + ? ` ${failedConfigurationMessage(cause.configurationFailure)}` + : ""; throw configurationCleanupError( - `Could not safely access client configuration at ${cause.path}: ${errorMessage(primary)}`, + `Could not safely access client configuration at ${cause.path}: ${errorMessage(primary)}.${rollbackMessage}`, cause, cleanup.cleanup, ); diff --git a/src/application/SetupPlan.ts b/src/application/SetupPlan.ts index 83402d1a4..a78697c80 100644 --- a/src/application/SetupPlan.ts +++ b/src/application/SetupPlan.ts @@ -320,5 +320,5 @@ const clientConfigurationDetail = ( ? " If this plan installs Hopper, REA will add HOPPER_LAUNCHER_PATH using the launcher path reported by that installation." : " The HOPPER_LAUNCHER_PATH value shown above is the current fallback; if this plan installs Hopper, REA will replace it with the launcher path reported by that installation." : ""; - return `Add the REA MCP registration for ${client}; preserve unrelated configuration. Command: ${command.map((part) => JSON.stringify(part)).join(" ")}.${environmentDetail}${hopperDetail}`; + return `Add the REA MCP registration for ${client}; preserve unrelated configuration. The backup path reflects the configuration inspected for this plan; if that file changes before execution, setup will select a recovery path for its current contents. Command: ${command.map((part) => JSON.stringify(part)).join(" ")}.${environmentDetail}${hopperDetail}`; }; diff --git a/src/application/SetupTypes.ts b/src/application/SetupTypes.ts index da58abbff..241e0f6dd 100644 --- a/src/application/SetupTypes.ts +++ b/src/application/SetupTypes.ts @@ -26,6 +26,16 @@ export type ClientConfigurationResult = | { readonly status: "failed"; readonly reason: "path" | "backup" | "write" | "readback"; + } + | { + readonly status: "failed"; + readonly reason: "rollback"; + readonly backupPath?: string; + readonly failure: { + readonly kind: "restore"; + readonly path: string; + readonly detail: string; + }; }; /** Read-only state used to block unsafe setup plans before any mutation. */ diff --git a/src/application/Uninstall.ts b/src/application/Uninstall.ts index c410280bc..a7f051a49 100644 --- a/src/application/Uninstall.ts +++ b/src/application/Uninstall.ts @@ -25,7 +25,11 @@ import { import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js"; import { dshDocument } from "./DshCordisPatch.js"; import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js"; -import { readRegularFileText } from "./RegularFileRead.js"; +import { + InvalidUtf8FileError, + RegularFileCleanupFailure, + readRegularFileText, +} from "./RegularFileRead.js"; import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; import { manualRegistrationRemediation, @@ -269,13 +273,17 @@ const readClientConfiguration = async ( try { original = await fileSystem.readText(transactionPath); } catch (cause: unknown) { + const readFailure = + cause instanceof RegularFileCleanupFailure ? cause.cause : cause; return itemRead( isMissing(cause) ? item(client.name, "skipped", "Configuration does not exist.") : item( client.name, "failed", - "Configuration could not be read. Check file permissions, then rerun uninstall.", + readFailure instanceof InvalidUtf8FileError + ? "Configuration is not valid UTF-8 and was not changed. Repair it, then rerun uninstall." + : "Configuration could not be read. Check file permissions, then rerun uninstall.", ), ); } diff --git a/tests/acceptance/setup/clientConfigurationDiagnosticsCli.test.ts b/tests/acceptance/setup/clientConfigurationDiagnosticsCli.test.ts index d9363988f..425e0e610 100644 --- a/tests/acceptance/setup/clientConfigurationDiagnosticsCli.test.ts +++ b/tests/acceptance/setup/clientConfigurationDiagnosticsCli.test.ts @@ -6,6 +6,17 @@ import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { cliTest } from "../../support/cli/cliFixture.js"; const cases = [ + { + client: "cursor", + path: ".cursor/mcp.json", + text: "malformed UTF-8", + bytes: Buffer.concat([ + Buffer.from('{"label":"'), + Buffer.from([0xff]), + Buffer.from('"}'), + ]), + diagnostic: "not valid UTF-8", + }, { client: "cursor", path: ".cursor/mcp.json", @@ -92,7 +103,9 @@ for (const scenario of cases) ); const config = join(home, scenario.path); await mkdir(dirname(config), { recursive: true }); - await writeFile(config, scenario.text); + const original = + "bytes" in scenario ? scenario.bytes : Buffer.from(scenario.text); + await writeFile(config, original); const environment = { HOME: home, USERPROFILE: home, @@ -146,7 +159,7 @@ for (const scenario of cases) }); if (!("syntax" in scenario)) expect(uninstall.stdout).not.toContain("not valid JSON"); - expect(await readFile(config, "utf8")).toBe(scenario.text); + expect(await readFile(config)).toEqual(original); expect(await readdir(dirname(config))).toEqual([ scenario.path.split("/").at(-1), ]); diff --git a/tests/boundary/filesystem/clientConfigurationTransactions.test.ts b/tests/boundary/filesystem/clientConfigurationTransactions.test.ts index 477085381..507b7137e 100644 --- a/tests/boundary/filesystem/clientConfigurationTransactions.test.ts +++ b/tests/boundary/filesystem/clientConfigurationTransactions.test.ts @@ -1,13 +1,91 @@ import { chmod, readFile, writeFile } from "node:fs/promises"; +import { createHash } from "node:crypto"; import { join } from "node:path"; import { describe, expect, it } from "vitest"; import { parse as parseToml } from "smol-toml"; -import { configureClientConfiguration } from "../../../src/application/SetupClientConfiguration.js"; +import { + configureClientConfiguration, + inspectClientConfiguration, +} from "../../../src/application/SetupClientConfiguration.js"; import { systemUninstallHost } from "../../../src/application/Uninstall.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +it("accepts a literal UTF-8 replacement character without changing its value", async () => { + const root = await createTestTempDirectory("rea-client-config-utf8-"); + const configPath = join(root, "config.json"); + const original = Buffer.concat([ + Buffer.from('{"label":"'), + Buffer.from("\uFFFD"), + Buffer.from('"}'), + ]); + await writeFile(configPath, original); + const result = await configureClientConfiguration( + { name: "cursor", format: "json", configPath }, + {}, + ["rea", "mcp"], + ); + expect(result.status).toBe("configured"); + expect(await readFile(`${configPath}.rea.backup`)).toEqual(original); + expect(JSON.parse(await readFile(configPath, "utf8")).label).toBe("\uFFFD"); +}); + +it("previews and creates a current recovery backup while preserving the first backup", async () => { + const root = await createTestTempDirectory("rea-client-current-recovery-"); + const configPath = join(root, "config.json"); + const firstOriginal = '{"keep":"first"}\n'; + const currentOriginal = '{"keep":"current"}\n'; + const client = { name: "cursor", format: "json", configPath } as const; + await writeFile(configPath, currentOriginal); + await writeFile(`${configPath}.rea.backup`, firstOriginal); + + const recoveryPath = `${configPath}.rea.backup.${createHash("sha256") + .update(currentOriginal, "utf8") + .digest("hex")}.recovery`; + expect(await inspectClientConfiguration(client, {}, ["rea", "mcp"])).toEqual({ + status: "update", + backupPath: recoveryPath, + }); + + expect( + await configureClientConfiguration(client, {}, ["rea", "mcp"]), + ).toEqual({ status: "configured", backupPath: recoveryPath }); + expect(await readFile(`${configPath}.rea.backup`, "utf8")).toBe( + firstOriginal, + ); + expect(await readFile(recoveryPath, "utf8")).toBe(currentOriginal); +}); + +it("blocks configuration when the planned content-addressed backup is occupied", async () => { + const root = await createTestTempDirectory("rea-client-recovery-conflict-"); + const configPath = join(root, "config.json"); + const firstOriginal = '{"keep":"first"}\n'; + const currentOriginal = '{"keep":"current"}\n'; + const client = { name: "cursor", format: "json", configPath } as const; + await writeFile(configPath, currentOriginal); + await writeFile(`${configPath}.rea.backup`, firstOriginal); + const recoveryPath = `${configPath}.rea.backup.${createHash("sha256") + .update(currentOriginal, "utf8") + .digest("hex")}.recovery`; + await writeFile(recoveryPath, "occupied by other data"); + + expect( + await inspectClientConfiguration(client, {}, ["rea", "mcp"]), + ).toMatchObject({ + status: "invalid", + remediation: expect.stringContaining(recoveryPath), + }); + expect( + await configureClientConfiguration(client, {}, ["rea", "mcp"]), + ).toEqual({ status: "failed", reason: "backup" }); + expect(await readFile(configPath, "utf8")).toBe(currentOriginal); + expect(await readFile(`${configPath}.rea.backup`, "utf8")).toBe( + firstOriginal, + ); + expect(await readFile(recoveryPath, "utf8")).toBe("occupied by other data"); +}); + describe("client configuration write failures", () => { it.skipIf(process.platform === "win32" || process.getuid?.() === 0).each([ ["json", configureClientConfiguration], diff --git a/tests/boundary/filesystem/setupClientConfigCleanupOwnership.test.ts b/tests/boundary/filesystem/setupClientConfigCleanupOwnership.test.ts index 3bf33c312..ab2b5e5a7 100644 --- a/tests/boundary/filesystem/setupClientConfigCleanupOwnership.test.ts +++ b/tests/boundary/filesystem/setupClientConfigCleanupOwnership.test.ts @@ -11,6 +11,7 @@ import { } from "../../../src/application/SetupHost.js"; import { RegularFileCleanupFailure } from "../../../src/application/RegularFileRead.js"; import { configureDetectedClients } from "../../../src/application/SetupClients.js"; +import { systemUninstallHost } from "../../../src/application/Uninstall.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; vi.mock("node:fs/promises", async (importOriginal) => { @@ -31,6 +32,61 @@ beforeEach(async () => { afterEach(() => openMock.mockReset()); +const holdReadHandle = (path: string, reason: string) => { + const closeAttempts: object[] = []; + let released = false; + openMock.mockImplementation(async (...args) => { + const handle = await actualFs.open(...args); + if (String(args[0]) !== path) return handle; + const nativeClose = handle.close.bind(handle); + vi.spyOn(handle, "close").mockImplementation(async () => { + closeAttempts.push(handle); + if (!released) throw new Error(reason); + await nativeClose(); + }); + return handle; + }); + return { + closeAttempts, + release: () => { + released = true; + }, + }; +}; + +it("retains invalid UTF-8 diagnostics and the uninstall read owner when close fails", async () => { + const home = await createTestTempDirectory("rea-uninstall-utf8-close-"); + const path = join(home, "client.json"); + const original = Buffer.from([0xff]); + await writeFile(path, original); + const { closeAttempts, release } = holdReadHandle( + path, + "held uninstall handle", + ); + const host = systemUninstallHost(home, undefined, { HOME: home }, "linux"); + try { + expect( + await host.removeClient({ + name: "cursor", + format: "json", + configPath: path, + }), + ).toMatchObject({ + status: "failed", + detail: expect.stringContaining("not valid UTF-8"), + }); + expect(await readFile(path)).toEqual(original); + expect(closeAttempts).toHaveLength(2); + release(); + expect(await host.close?.()).toBeUndefined(); + expect(closeAttempts).toHaveLength(3); + expect(closeAttempts[2]).toBe(closeAttempts[0]); + } finally { + release(); + await host.close?.(); + } +}); + describe("ZCode fallback cleanup ownership", () => { it.each(["preflight", "configure"])( "retains a failed ZCode fallback close during %s until host cleanup", @@ -49,19 +105,10 @@ describe("ZCode fallback cleanup ownership", () => { await mkdir(dirname(fallback), { recursive: true }); const original = '{"mcpServers":{"other":{"command":"other"}}}\n'; await writeFile(fallback, original); - const closeAttempts: object[] = []; - let allowClose = false; - openMock.mockImplementation(async (...args) => { - const handle = await actualFs.open(...args); - if (String(args[0]) !== fallback) return handle; - const nativeClose = handle.close.bind(handle); - vi.spyOn(handle, "close").mockImplementation(async () => { - closeAttempts.push(handle); - if (!allowClose) throw new Error("held fallback handle"); - await nativeClose(); - }); - return handle; - }); + const { closeAttempts, release } = holdReadHandle( + fallback, + "held fallback handle", + ); try { const action = operation === "preflight" @@ -79,12 +126,12 @@ describe("ZCode fallback cleanup ownership", () => { await expect(readFile(client.configPath)).rejects.toMatchObject({ code: "ENOENT", }); - allowClose = true; + release(); expect(await host.close?.()).toBeUndefined(); expect(closeAttempts).toHaveLength(3); expect(closeAttempts[2]).toBe(closeAttempts[0]); } finally { - allowClose = true; + release(); await host.close?.(); } }, @@ -92,83 +139,87 @@ describe("ZCode fallback cleanup ownership", () => { }); describe("setup client configuration cleanup ownership", () => { - it("stops client configuration and retries the same alignment handle from host close", async () => { - const home = await createTestTempDirectory("rea-setup-config-close-"); - const environment = { - HOME: home, - XDG_CONFIG_HOME: join(home, "xdg"), - }; - const host = systemSetupHost( - { platform: "linux", homeDirectory: home } as DoctorHost, - environment, - ); - const client = (await host.supportedClients()).find( - ({ name }) => name === "vscode", - ); - if (client === undefined) throw new Error("VS Code setup client missing"); - await mkdir(dirname(client.configPath), { recursive: true }); - const original = Buffer.from('{"servers": {"other": {}}}\n'); - await writeFile(client.configPath, original); + it.each([ + ["valid UTF-8", Buffer.from('{"servers": {"other": {}}}\n')], + ["malformed UTF-8", Buffer.from([0xff])], + ] as const)( + "stops configuration and retains the %s read owner until host close", + async (encoding, original) => { + const home = await createTestTempDirectory("rea-setup-config-close-"); + const environment = { + HOME: home, + XDG_CONFIG_HOME: join(home, "xdg"), + }; + const host = systemSetupHost( + { platform: "linux", homeDirectory: home } as DoctorHost, + environment, + ); + const client = (await host.supportedClients()).find( + ({ name }) => name === "vscode", + ); + if (client === undefined) throw new Error("VS Code setup client missing"); + await mkdir(dirname(client.configPath), { recursive: true }); + await writeFile(client.configPath, original); - const closeAttempts: object[] = []; - let allowClose = false; - openMock.mockImplementation(async (...args) => { - const handle = await actualFs.open(...args); - if (String(args[0]) !== client.configPath) return handle; - const nativeClose = handle.close.bind(handle); - vi.spyOn(handle, "close").mockImplementation(async () => { - closeAttempts.push(handle); - if (!allowClose) throw new Error("test holds the config handle open"); - await nativeClose(); - }); - return handle; - }); - const configure = vi.spyOn(host, "configureClient"); + const { closeAttempts, release } = holdReadHandle( + client.configPath, + "test holds the config handle open", + ); + const configure = vi.spyOn(host, "configureClient"); - try { - const failure = await filterClientsNeedingConfigure(host, [client], {}, [ - "rea", - "mcp", - ]) - .then((detectedClients) => - configureDetectedClients({ - host, - detectedClients, - providerEnvironment: {}, - command: ["rea", "mcp"], - clients: {}, - appliedActions: [], - }), + try { + const failure = await filterClientsNeedingConfigure( + host, + [client], + {}, + ["rea", "mcp"], ) - .then( - () => undefined, - (cause: unknown) => cause, - ); - expect(failure).toBeInstanceOf(ConfigurationError); - expect(failure).toMatchObject({ - _tag: "ConfigurationError", - cause: expect.any(RegularFileCleanupFailure), - cleanup: { - resources: [client.configPath], - reason: expect.stringContaining("test holds the config handle open"), - }, - }); - expect(configure).not.toHaveBeenCalled(); - expect(closeAttempts).toHaveLength(2); - expect(closeAttempts[0]).toBe(closeAttempts[1]); - expect(await readFile(client.configPath)).toEqual(original); + .then((detectedClients) => + configureDetectedClients({ + host, + detectedClients, + providerEnvironment: {}, + command: ["rea", "mcp"], + clients: {}, + appliedActions: [], + }), + ) + .then( + () => undefined, + (cause: unknown) => cause, + ); + expect(failure).toBeInstanceOf(ConfigurationError); + if (encoding === "malformed UTF-8") + expect(failure).toMatchObject({ + cause: { + outcome: { kind: "failed", cause: expect.any(TypeError) }, + }, + }); + expect(failure).toMatchObject({ + _tag: "ConfigurationError", + cause: expect.any(RegularFileCleanupFailure), + cleanup: { + resources: [client.configPath], + reason: expect.stringContaining( + "test holds the config handle open", + ), + }, + }); + expect(configure).not.toHaveBeenCalled(); + expect(closeAttempts).toHaveLength(2); + expect(closeAttempts[0]).toBe(closeAttempts[1]); + expect(await readFile(client.configPath)).toEqual(original); - allowClose = true; - const close = host.close; - if (close === undefined) throw new Error("Setup host close is missing"); - expect(await close()).toBeUndefined(); - expect(closeAttempts).toHaveLength(3); - expect(closeAttempts[2]).toBe(closeAttempts[0]); - } finally { - allowClose = true; - await host.close?.(); - } - }); + release(); + expect(await host.close?.()).toBeUndefined(); + expect(closeAttempts).toHaveLength(3); + expect(closeAttempts[2]).toBe(closeAttempts[0]); + } finally { + release(); + await host.close?.(); + } + }, + ); it("waits for an admitted configuration read and rejects later admissions", async () => { const home = await createTestTempDirectory("rea-setup-config-inflight-"); diff --git a/tests/process-global/setupConfigurationRollback.test.ts b/tests/process-global/setupConfigurationRollback.test.ts new file mode 100644 index 000000000..cf6504c32 --- /dev/null +++ b/tests/process-global/setupConfigurationRollback.test.ts @@ -0,0 +1,325 @@ +import { open, readFile } from "node:fs/promises"; +import { createHash } from "node:crypto"; +import { join } from "node:path"; + +import { afterEach, expect, it, onTestFinished, vi } from "vitest"; + +const state = vi.hoisted(() => ({ + readbackFailed: false, + rejectRestore: false, + rejectRemove: false, + removeCopiedRecovery: false, + initialCleanupFailure: undefined as unknown, + readbackCleanupFailure: undefined as unknown, +})); + +afterEach(() => { + state.readbackFailed = false; + state.rejectRestore = false; + state.rejectRemove = false; + state.removeCopiedRecovery = false; + state.initialCleanupFailure = undefined; + state.readbackCleanupFailure = undefined; +}); + +vi.mock("node:fs/promises", async (importOriginal) => { + const original = await importOriginal(); + return { + ...original, + rm: ( + ...arguments_: Parameters + ): ReturnType => { + if (state.rejectRemove && state.readbackFailed) + return Promise.reject(new Error("injected restore remove failure")); + return original.rm(...arguments_); + }, + writeFile: ( + ...arguments_: Parameters + ): ReturnType => { + if (state.rejectRestore && state.readbackFailed) + return Promise.reject(new Error("injected restore write failure")); + return original.writeFile(...arguments_); + }, + copyFile: async ( + ...arguments_: Parameters + ): Promise => { + await original.copyFile(...arguments_); + const destination = arguments_[1]; + if ( + state.removeCopiedRecovery && + typeof destination === "string" && + destination.endsWith(".recovery") + ) + await original.rm(destination); + }, + }; +}); + +vi.mock("write-file-atomic", async (importOriginal) => { + const original = await importOriginal<{ + default: typeof import("write-file-atomic"); + }>(); + return { + ...original, + default: async (...arguments_: Parameters) => { + if (state.initialCleanupFailure !== undefined) + throw new Error("unexpected config write before initial read"); + const result = await original.default(...arguments_); + state.readbackFailed = true; + return result; + }, + }; +}); + +vi.mock("../../src/application/RegularFileRead.js", async (importOriginal) => { + const original = + await importOriginal< + typeof import("../../src/application/RegularFileRead.js") + >(); + return { + ...original, + readRegularFileText: ( + ...arguments_: Parameters + ) => + state.initialCleanupFailure !== undefined + ? Promise.reject(state.initialCleanupFailure) + : state.readbackFailed && state.readbackCleanupFailure !== undefined + ? Promise.reject(state.readbackCleanupFailure) + : state.readbackFailed + ? Promise.reject(new Error("injected readback failure")) + : original.readRegularFileText(...arguments_), + }; +}); + +import { configureClientConfiguration } from "../../src/application/SetupClientConfiguration.js"; +import { RegularFileCleanupFailure } from "../../src/application/RegularFileRead.js"; +import { OwnedFileHandle } from "../../src/filesystem/OwnedFileHandle.js"; +import type { DoctorHost } from "../../src/application/Doctor.js"; +import { systemSetupHost } from "../../src/application/SetupHost.js"; +import { ConfigurationError } from "../../src/domain/configurationErrors.js"; +import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js"; + +const ownReadHandle = async (path: string) => { + const owner = new OwnedFileHandle(await open(path, "r")); + onTestFinished(() => owner.close()); + return owner; +}; + +it("reports a failed config rollback and retains its backup path", async () => { + state.readbackFailed = false; + state.rejectRestore = false; + state.rejectRemove = false; + state.initialCleanupFailure = undefined; + state.readbackCleanupFailure = undefined; + const root = await createTestTempDirectory("rea-config-rollback-"); + const path = join(root, "client.json"); + const original = '{"keep":true}\n'; + await import("node:fs/promises").then(({ writeFile }) => + writeFile(path, original), + ); + state.rejectRestore = true; + + const result = await configureClientConfiguration( + { name: "cursor", configPath: path, format: "json" }, + {}, + ["rea", "mcp"], + ); + + expect(result).toEqual({ + status: "failed", + reason: "rollback", + backupPath: `${path}.rea.backup`, + failure: { + kind: "restore", + path, + detail: "injected restore write failure", + }, + }); + expect(await readFile(`${path}.rea.backup`, "utf8")).toBe(original); + expect(JSON.parse(await readFile(path, "utf8"))).toMatchObject({ + mcpServers: { rea: { command: "rea", args: ["mcp"] } }, + }); +}); + +it("preserves the first backup and reports a verified backup of the current original", async () => { + state.readbackFailed = false; + state.rejectRestore = false; + state.rejectRemove = false; + state.initialCleanupFailure = undefined; + state.readbackCleanupFailure = undefined; + const root = await createTestTempDirectory("rea-config-stale-backup-"); + const path = join(root, "client.json"); + const firstOriginal = '{"keep":"first"}\n'; + const currentOriginal = '{"keep":"current"}\n'; + await import("node:fs/promises").then(async ({ writeFile }) => { + await writeFile(path, currentOriginal); + await writeFile(`${path}.rea.backup`, firstOriginal); + }); + state.rejectRestore = true; + + const result = await configureClientConfiguration( + { name: "cursor", configPath: path, format: "json" }, + {}, + ["rea", "mcp"], + ); + state.rejectRestore = false; + + const recoveryPath = `${path}.rea.backup.${createHash("sha256") + .update(currentOriginal, "utf8") + .digest("hex")}.recovery`; + expect(result).toMatchObject({ + status: "failed", + reason: "rollback", + backupPath: recoveryPath, + failure: { + kind: "restore", + path, + detail: "injected restore write failure", + }, + }); + expect(await readFile(`${path}.rea.backup`, "utf8")).toBe(firstOriginal); + expect(await readFile(recoveryPath, "utf8")).toBe(currentOriginal); +}); + +it("does not write config when a recovery copy disappears before verification", async () => { + state.readbackFailed = false; + state.rejectRestore = false; + state.rejectRemove = false; + state.removeCopiedRecovery = false; + state.initialCleanupFailure = undefined; + state.readbackCleanupFailure = undefined; + const root = await createTestTempDirectory("rea-config-vanished-recovery-"); + const path = join(root, "client.json"); + const firstOriginal = '{"keep":"first"}\n'; + const currentOriginal = '{"keep":"current"}\n'; + await import("node:fs/promises").then(async ({ writeFile }) => { + await writeFile(path, currentOriginal); + await writeFile(`${path}.rea.backup`, firstOriginal); + }); + state.removeCopiedRecovery = true; + + const result = await configureClientConfiguration( + { name: "cursor", configPath: path, format: "json" }, + {}, + ["rea", "mcp"], + ); + state.removeCopiedRecovery = false; + + expect(result).toEqual({ status: "failed", reason: "backup" }); + expect(await readFile(path, "utf8")).toBe(currentOriginal); + expect(await readFile(`${path}.rea.backup`, "utf8")).toBe(firstOriginal); +}); + +it("does not roll back or remove config when its initial read cleanup fails", async () => { + state.readbackFailed = false; + state.rejectRestore = false; + state.rejectRemove = false; + state.readbackCleanupFailure = undefined; + const root = await createTestTempDirectory("rea-config-initial-cleanup-"); + const path = join(root, "client.json"); + const original = '{"keep":true}\n'; + await import("node:fs/promises").then(({ writeFile }) => + writeFile(path, original), + ); + const failure = new RegularFileCleanupFailure( + path, + await ownReadHandle(path), + { kind: "completed", value: original }, + new Error("injected initial close failure"), + ); + state.initialCleanupFailure = failure; + + await expect( + configureClientConfiguration( + { name: "cursor", configPath: path, format: "json" }, + {}, + ["rea", "mcp"], + ), + ).rejects.toBe(failure); + expect(await readFile(path, "utf8")).toBe(original); +}); + +it("reports first-run rollback removal failure without a backup", async () => { + state.readbackFailed = false; + state.rejectRestore = false; + state.rejectRemove = false; + state.initialCleanupFailure = undefined; + state.readbackCleanupFailure = undefined; + const root = await createTestTempDirectory("rea-config-first-run-rollback-"); + const path = join(root, "client.json"); + state.rejectRemove = true; + + const result = await configureClientConfiguration( + { name: "cursor", configPath: path, format: "json" }, + {}, + ["rea", "mcp"], + ); + state.rejectRemove = false; + + expect(result).toMatchObject({ + status: "failed", + reason: "rollback", + failure: { + kind: "restore", + path, + detail: "injected restore remove failure", + }, + }); + expect(result).not.toHaveProperty("backupPath"); + expect(JSON.parse(await readFile(path, "utf8"))).toMatchObject({ + mcpServers: { rea: { command: "rea", args: ["mcp"] } }, + }); +}); + +it("retains readback cleanup ownership when rollback also fails", async () => { + state.readbackFailed = false; + state.rejectRestore = false; + state.rejectRemove = false; + state.initialCleanupFailure = undefined; + const root = await createTestTempDirectory("rea-config-readback-cleanup-"); + const path = join(root, "client.json"); + const original = '{"keep":true}\n'; + await import("node:fs/promises").then(({ writeFile }) => + writeFile(path, original), + ); + const owner = await ownReadHandle(path); + const cleanupFailure = new RegularFileCleanupFailure( + path, + owner, + { kind: "completed", value: "" }, + new Error("injected readback close failure"), + ); + state.readbackCleanupFailure = cleanupFailure; + state.rejectRestore = true; + + const host = systemSetupHost( + { platform: "linux", homeDirectory: root } as DoctorHost, + { HOME: root }, + ); + let caught: unknown; + try { + await host.configureClient( + { name: "cursor", configPath: path, format: "json" }, + {}, + ["rea", "mcp"], + ); + } catch (cause: unknown) { + caught = cause; + } finally { + await host.close?.(); + } + + if (!(caught instanceof ConfigurationError)) throw caught; + expect(caught.message).toContain("injected restore write failure"); + expect(caught.message).toContain(`${path}.rea.backup`); + const cleanupOwnerError = caught.cause; + if (!(cleanupOwnerError instanceof RegularFileCleanupFailure)) + throw new Error("Configuration failure lost its read owner"); + expect(cleanupOwnerError).toMatchObject({ + path, + outcome: { kind: "completed", value: "" }, + cleanupCause: expect.any(AggregateError), + }); + expect(cleanupOwnerError.owner).toBe(owner); + expect(await readFile(`${path}.rea.backup`, "utf8")).toBe(original); +}); From 15463bdbe605b9dd11da5dbba031f99cd1ec9190 Mon Sep 17 00:00:00 2001 From: Jin Date: Tue, 13 Oct 2026 02:52:48 -0400 Subject: [PATCH 054/159] fix(adb): count requested permissions that the device marks restricted (#2359) Android 10 and 11 print restricted entries of the requested-permissions list as "NAME: restricted=true". The parser took the first such line as the end of the list, so the count stopped there while coverage still read complete. Co-authored-by: Claude Opus 5.5 --- src/adb/AdbProvider.test.ts | 28 ++++++++++++++++++++++++++++ src/adb/AdbShellOutput.ts | 6 +++++- 2 files changed, 33 insertions(+), 1 deletion(-) diff --git a/src/adb/AdbProvider.test.ts b/src/adb/AdbProvider.test.ts index b89ab9dfd..ae81353c7 100644 --- a/src/adb/AdbProvider.test.ts +++ b/src/adb/AdbProvider.test.ts @@ -626,6 +626,34 @@ it("projects package details from dumpsys package output", async () => { }); }); +// Android 10 and 11 append ": restricted=true" to hard- and soft-restricted +// entries of the requested-permissions list (Settings.dumpPackageLPr). +it("counts requested permissions that the device marks as restricted", async () => { + const { binary } = await writeAdbStub( + shellStub({ + "dumpsys:package": `Packages: + Package [com.example] (abcdef): + userId=10234 + versionName=1.2.3 + requested permissions: + android.permission.INTERNET + android.permission.WRITE_EXTERNAL_STORAGE: restricted=true + android.permission.CAMERA + install permissions: + android.permission.INTERNET: granted=true +`, + }), + ); + const result = await evidenceResult(providerOver(binary), { + operation: "inspect_adb_package", + input: { serial: "emulator-5554", package: "com.example" }, + }); + expect(result).toMatchObject({ + requested_permissions_count: 3, + coverage: "complete", + }); +}); + it("reports an uninstalled package from dumpsys refusal output", async () => { const { binary } = await writeAdbStub( shellStub({ "dumpsys:package": "Unable to find package: com.absent\n" }), diff --git a/src/adb/AdbShellOutput.ts b/src/adb/AdbShellOutput.ts index f3323e7b2..726e6164d 100644 --- a/src/adb/AdbShellOutput.ts +++ b/src/adb/AdbShellOutput.ts @@ -236,7 +236,11 @@ export const parseDumpsysPackageOutput = ( inPermissions = true; continue; } - if (inPermissions && /^\s{2,}[A-Za-z0-9._]+$/.test(line)) { + // Android 10 and 11 mark restricted entries as "NAME: restricted=true". + if ( + inPermissions && + /^\s{2,}[A-Za-z0-9._]+(?:: restricted=true)?$/u.test(line) + ) { permissions += 1; continue; } From b9ee921560dcf4590c2b13a1cc92c49c93e10675 Mon Sep 17 00:00:00 2001 From: Rudy Mizrahi Celekli <47457359+rudycelekli@users.noreply.github.com> Date: Tue, 13 Oct 2026 02:54:18 -0400 Subject: [PATCH 055/159] fix(process): preserve nested path identities during normalization (#2468) --- src/domain/process/processScenario.ts | 3 + .../capture/ProcessNormalization.test.ts | 44 ++++++++++++++ src/process/capture/ProcessNormalization.ts | 48 +++++++++++---- .../processCaptureNormalization.test.ts | 60 ++++++++++++++++++- 4 files changed, 141 insertions(+), 14 deletions(-) diff --git a/src/domain/process/processScenario.ts b/src/domain/process/processScenario.ts index a87ffd86e..e98d7a537 100644 --- a/src/domain/process/processScenario.ts +++ b/src/domain/process/processScenario.ts @@ -128,6 +128,9 @@ export const processComparisonContract = ( : {}), limits: scenario.limits, normalization: scenario.normalization, + ...(scenario.normalization.paths + ? { path_normalization_version: "longest-original-paths-v1" } + : {}), ...(scenario.normalization.ports ? { port_normalization_version: "contextual-endpoints-v2" } : {}), diff --git a/src/process/capture/ProcessNormalization.test.ts b/src/process/capture/ProcessNormalization.test.ts index 9c9ba1951..e5ef61062 100644 --- a/src/process/capture/ProcessNormalization.test.ts +++ b/src/process/capture/ProcessNormalization.test.ts @@ -266,3 +266,47 @@ it("normalizes representative process identities and preserves null, zero, and u "/program child 42020 parent 41010 group 43030 session 44040 listen: selected", ); }); + +it("normalizes nested paths once using the most specific selected identity", () => { + const declared = parseProcessScenario({ + executable: "/workspace/project/bin/program", + working_directory: "/workspace/project", + filesystem_observation_paths: [ + "/workspace/project/data", + "/workspace/project/data/cache", + ], + normalization: { paths: true, pids: false, ports: false, patterns: [] }, + }); + expect( + normalizeProcessText( + "/workspace/project/bin/program /workspace/project/data/cache/item /workspace/project/data/result /workspace/project/log /workspace/tmp", + declared, + "/workspace", + rootPid, + ), + ).toBe( + " /item /result /log /tmp", + ); +}); + +it("treats path metacharacters literally while preferring nested observed roots", () => { + const declared = parseProcessScenario({ + executable: "/selected/[tool]+", + working_directory: "/selected", + filesystem_observation_paths: [ + "/selected/(data)", + "/selected/(data)/cache", + ], + normalization: { paths: true, pids: false, ports: false, patterns: [] }, + }); + expect( + normalizeProcessText( + "/selected/[tool]+ /selected/(data)/cache/item /selected/(data)/file /selected/log /selected/toolll", + declared, + "/tmp", + rootPid, + ), + ).toBe( + " /item /file /log /toolll", + ); +}); diff --git a/src/process/capture/ProcessNormalization.ts b/src/process/capture/ProcessNormalization.ts index 9364ec95b..74db31b1c 100644 --- a/src/process/capture/ProcessNormalization.ts +++ b/src/process/capture/ProcessNormalization.ts @@ -17,19 +17,8 @@ export const normalizeProcessText = ( pid: number, ): string => { let normalized = value; - if (scenario.normalization.paths) { - normalized = normalized.replaceAll(temporaryRoot, ""); - normalized = normalized.replaceAll( - scenario.working_directory, - "", - ); - normalized = normalized.replaceAll(scenario.executable, ""); - for (const [index, root] of scenario.filesystem_observation_paths.entries()) - normalized = normalized.replaceAll( - root, - ``, - ); - } + if (scenario.normalization.paths) + normalized = normalizePathTokens(normalized, scenario, temporaryRoot); if (scenario.normalization.ports) normalized = normalizePortTokens(normalized); if (scenario.normalization.pids) @@ -44,6 +33,39 @@ export const normalizeProcessText = ( return normalized; }; +/** Match original paths once, preferring the most specific selected identity. */ +const normalizePathTokens = ( + value: string, + scenario: ProcessScenario, + temporaryRoot: string, +): string => { + const aliases = new Map(); + const selected: readonly (readonly [string, string])[] = [ + [temporaryRoot, ""], + [scenario.working_directory, ""], + [scenario.executable, ""], + ...scenario.filesystem_observation_paths.map( + (root, index): readonly [string, string] => [ + root, + ``, + ], + ), + ]; + for (const [path, alias] of selected) + if (path !== "" && !aliases.has(path)) aliases.set(path, alias); + const paths = [...aliases.keys()].sort( + (left, right) => right.length - left.length, + ); + if (paths.length === 0) return value; + const pattern = paths + .map((path) => path.replace(/[.*+?^${}()|[\]\\]/gu, "\\$&")) + .join("|"); + return value.replace( + new RegExp(pattern, "gu"), + (path) => aliases.get(path) ?? path, + ); +}; + // A port may end a sentence, but a period followed by a letter, digit, or // underscore in any script continues a decimal, version, or host name and is // not a port boundary. diff --git a/tests/boundary/process/processCaptureNormalization.test.ts b/tests/boundary/process/processCaptureNormalization.test.ts index 965dc2af5..2837ccf40 100644 --- a/tests/boundary/process/processCaptureNormalization.test.ts +++ b/tests/boundary/process/processCaptureNormalization.test.ts @@ -1,4 +1,4 @@ -import { readFile } from "node:fs/promises"; +import { mkdir, readFile, symlink } from "node:fs/promises"; import { join } from "node:path"; import { fileURLToPath } from "node:url"; import { expect, it } from "vitest"; @@ -202,3 +202,61 @@ itWithLinuxCaptureCapability.each([false, true])( }, 20_000, ); + +itWithCaptureCapability( + "preserves selected nested path identities in actual PTY Evidence", + async () => { + const root = await createTestTempDirectory("rea-process-nested-paths-"); + const executable = join(root, "selected-node"); + const observed = join(root, "data"); + await symlink(process.execPath, executable); + await mkdir(observed); + const original = `${executable}\n${observed}/result\n${root}/log`; + const scenario = parseProcessScenario({ + executable, + working_directory: root, + filesystem_observation_paths: [observed], + arguments: ["-e", `process.stdout.write(${JSON.stringify(original)})`], + normalization: { paths: true, pids: false, ports: false, patterns: [] }, + settle_ms: 100, + }); + const result = await captureProcessScenario(scenario); + if (!result.ok) throw result.error; + const capture = parseProcessCapture( + createProcessCaptureEvidence(scenario, result.value).normalized_result, + ); + expect(capture.frames.map(({ data }) => data).join("")).toBe( + "\r\n/result\r\n/log", + ); + expect( + capture.frames.map(({ raw_data, data }) => raw_data ?? data).join(""), + ).toBe(original.replaceAll("\n", "\r\n")); + expect(capture.manifest.comparison_contract).toHaveProperty( + "path_normalization_version", + "longest-original-paths-v1", + ); + const legacyContract = { ...capture.manifest.comparison_contract }; + delete legacyContract.path_normalization_version; + const legacy = parseProcessCapture({ + ...capture, + manifest: { + ...capture.manifest, + comparison_contract: legacyContract, + comparison_contract_sha256: digestProcessCommitment(legacyContract), + }, + }); + expect(() => compareProcessCaptures(legacy, capture)).toThrow( + expect.objectContaining({ + issues: [ + expect.objectContaining({ expected: ["path_normalization_version"] }), + ], + }), + ); + expect(capture.exit).toMatchObject({ code: 0, reason: "exited" }); + expect(capture.cleanup).toMatchObject({ + owned_process_group: "verified", + temporary_root: "removed", + }); + }, + 20_000, +); From 9c34c091b3b8c47f8bd18d6f0321efde2dedb92a Mon Sep 17 00:00:00 2001 From: Olorin <59321404+Olorinm@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:55:25 +0800 Subject: [PATCH 056/159] feat(captures): discover installed protocol and field definitions (#2496) * feat(captures): discover installed protocol and field definitions * test(captures): isolate the direct glossary oracle profile * test(captures): accept padded kernel limit reports * fix(captures): detach every retained glossary column * fix(captures): describe every field discovery CLI option * fix(captures): retain discovery operation across cleanup retries --------- Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com> --- .../skills/reverse-engineer-anything/SKILL.md | 3 + .github/workflows/real-packet-capture.yml | 2 +- docs/packet-captures.md | 46 +++ scripts/verify-packet-capture.mjs | 153 +++++++- src/application/CapabilityInventory.ts | 2 +- .../captures/PacketCaptureService.ts | 70 +++- src/captures/TsharkCaptureProvider.ts | 342 +++++++++++------- src/captures/parseFieldDiscovery.ts | 92 +++++ src/cli/captureCommands.ts | 93 +++-- src/cliCommandNames.ts | 1 + .../captures/captureToolContracts.ts | 20 + src/contracts/toolEffects.ts | 5 + src/domain/captures/packetFields.ts | 100 +++++ src/process/ArtifactAnalysisResources.ts | 45 ++- src/server/registerCaptureTools.ts | 13 + .../boundary/captures/captureProvider.test.ts | 164 ++++++++- .../boundary/captures/fieldDiscovery.test.ts | 67 ++++ tests/boundary/cli/captureCleanup.test.ts | 9 + tests/boundary/mcp/packetCaptureMcp.test.ts | 87 +++-- 19 files changed, 1095 insertions(+), 219 deletions(-) create mode 100644 src/captures/parseFieldDiscovery.ts create mode 100644 src/domain/captures/packetFields.ts create mode 100644 tests/boundary/captures/fieldDiscovery.test.ts diff --git a/.agents/skills/reverse-engineer-anything/SKILL.md b/.agents/skills/reverse-engineer-anything/SKILL.md index f8ec45268..48bc8b7df 100644 --- a/.agents/skills/reverse-engineer-anything/SKILL.md +++ b/.agents/skills/reverse-engineer-anything/SKILL.md @@ -76,6 +76,9 @@ explicit path or endpoint and do not need it. Facts retain all-recorded-events-before-selection scope. Other qlog layouts and PCAP correlation are unsupported. - User-owned Electron runtime already open: `list_electron_targets`. +- Packet-field discovery: `inspect_packet_fields` searches installed protocols + and field names/types before choosing fields for `inspect_packet_capture`. + Query/page the relevant records; registrations do not establish observed traffic. - Recorded packet captures: `inspect_packet_capture` with selected TShark fields, display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see [capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md). diff --git a/.github/workflows/real-packet-capture.yml b/.github/workflows/real-packet-capture.yml index f481ed00d..b5e9f9201 100644 --- a/.github/workflows/real-packet-capture.yml +++ b/.github/workflows/real-packet-capture.yml @@ -51,7 +51,7 @@ jobs: - run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3 - run: npm ci - run: npm run build:cached - - run: npm run test:focused -- src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts + - run: npm run test:focused -- tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts - run: node scripts/verify-packet-capture.mjs env: REA_TSHARK_COMMAND: /usr/bin/tshark diff --git a/docs/packet-captures.md b/docs/packet-captures.md index e8ce3ae92..32cedb11b 100644 --- a/docs/packet-captures.md +++ b/docs/packet-captures.md @@ -128,3 +128,49 @@ Each provider admits one inspection at a time, including acquisition and cleanup An occupied provider returns a retryable capacity error before acquiring another file or worker. Closing the provider stops new admission and drains accepted work; failed cleanup remains owned and can be retried. + +## Discover protocols and field names + +`inspect_packet_fields` / `inspect-packet-fields` queries the installed TShark +registration glossary before selecting fields for capture inspection: + +```sh +rea inspect-packet-fields --query usb +rea inspect-packet-fields --kind fields --protocol usb --query idVendor +rea inspect-packet-fields --kind fields --protocol udp --all +``` + +`kind` defaults to `protocols`; `fields` selects field definitions. `protocol` +selects an exact protocol abbreviation (the protocol itself or its child fields). +`query` is a case-insensitive literal substring in name, abbreviation or field +description. `offset` skips matching records; `limit` defaults to 50. MCP +`limit: null` or CLI `--all` requests all remaining matches within the resource +budget. `--all` and `--limit` conflict. The window reports examined records of +the selected kind before filters, matching records, returned records and +`has_more`; it never counts capture packets. + +Returned records preserve field type, parent protocol, display base, bitmask, +description and original glossary line number. Empty producer strings remain +empty; protocol records have null field-only properties. Repeated abbreviations +remain separate registrations. A listed field is an installation capability, not +proof that the field appears in a selected recording. + +The adapter uses the documented `tshark -G fields` grammar under its existing +private empty profile and Linux process limits. TShark 4.6.9's full glossary was +measured at 22,303,197 bytes; discovery admits at most **32 MiB** combined command +output (capture inspection remains at **4 MiB**). Parsing walks lines once and +constructs objects only for the selected page. Returned strings are copied so +retained pages do not retain the complete glossary string. Before constructing a selected +record it charges six times the source UTF-8 bytes plus 512 bytes against an +**8 MiB** representation budget, covering JSON escaping and record bookkeeping. +If that budget would be exceeded, narrow the query or page size; results are not +silently truncated. This is a conservative admission budget, not measured heap +usage. Parent pipe buffers and producer per-process limits remain distinct. + +Tool/limiter identity, version, stderr and selected arguments are retained in +Evidence. No capture, interface or device is opened. Discovery shares capture +analysis's admission slot and cleanup owner. Each call rereads the glossary; +installed plugins, libraries and global configuration remain trusted and are not +fully fingerprinted. There is no cache or snapshot-consistent pagination claim. +The real Linux lane checks public CLI/MCP queries against the direct glossary, +including USB vendor-ID typing, negative queries and paging. diff --git a/scripts/verify-packet-capture.mjs b/scripts/verify-packet-capture.mjs index 2b959e602..9993117c2 100644 --- a/scripts/verify-packet-capture.mjs +++ b/scripts/verify-packet-capture.mjs @@ -1,8 +1,9 @@ #!/usr/bin/env node import assert from "node:assert/strict"; +import { estimateTokenCount } from "tokenx"; import { execFile } from "node:child_process"; import { createHash } from "node:crypto"; -import { readFile, writeFile, readdir } from "node:fs/promises"; +import { readFile, writeFile, readdir, mkdir, rm } from "node:fs/promises"; import { isAbsolute, join } from "node:path"; import { fileURLToPath } from "node:url"; import { promisify } from "node:util"; @@ -91,6 +92,20 @@ const source = pcap(1, [ // LINKTYPE_BLUETOOTH_HCI_H4 = 187; H4 command packet, HCI Reset opcode 0x0c03, no parameters. const bluetoothSource = pcap(187, [Buffer.from("01030c00", "hex")]); let cases = 0; +const discoveryMeasurements = []; +const inheritedAddressSpace = (await readFile("/proc/self/limits", "utf8")) + .split("\n") + .find((line) => line.startsWith("Max address space ")) + ?.trim() + .match(/^Max address space\s+(\S+)\s+(\S+)\s+bytes$/u); +assert.ok(inheritedAddressSpace); +const expectedAddressSpace = Math.min( + 1073741824, + ...inheritedAddressSpace + .slice(1) + .map((value) => (value === "unlimited" ? Infinity : Number(value))), +); + try { await writeFile(path, source); await writeFile(bluetooth, bluetoothSource); @@ -121,14 +136,16 @@ try { .digest("hex"), ); assert.equal(report.hardware_acquisition, "not-performed"); - assert.equal(report.limits.address_space_bytes, 1073741824); + assert.equal(report.limits.address_space_bytes, expectedAddressSpace); assert.equal(report.tool.configured_path, process.env.REA_TSHARK_COMMAND); assert.equal( report.resource_limiter.configured_path, process.env.REA_TSHARK_PRLIMIT_COMMAND ?? "/usr/bin/prlimit", ); assert.ok( - report.resource_limiter.arguments.includes("--as=1073741824:1073741824"), + report.resource_limiter.arguments.includes( + `--as=${expectedAddressSpace}:${expectedAddressSpace}`, + ), ); assert.equal(report.configuration, "private-empty-personal-profile"); const flags = Object.entries(input) @@ -313,6 +330,133 @@ try { true, ); cases++; + const discovery = (await client.listTools()).tools.find( + ({ name }) => name === "inspect_packet_fields", + ); + assert.ok(discovery?.outputSchema); + assert.ok(ajv.validateSchema(discovery.inputSchema)); + assert.ok(ajv.validateSchema(discovery.outputSchema)); + const nativeProfile = join(root.path, "native-profile"); + await mkdir(nativeProfile); + const native = ( + await execute(process.env.REA_TSHARK_COMMAND, ["-G", "fields"], { + env: { ...environment, WIRESHARK_CONFIG_DIR: nativeProfile }, + timeout: 30000, + maxBuffer: 32 * 1024 * 1024, + }) + ).stdout; + await rm(nativeProfile, { recursive: true }); + const nativeLines = native.split(/\r?\n/); + if (nativeLines.at(-1) === "") nativeLines.pop(); + for (const input of [ + { query: "usb" }, + { limit: 1 }, + { offset: 1, limit: 2 }, + { kind: "fields", protocol: "usb", query: "idVendor" }, + { kind: "fields", protocol: "udp", limit: null }, + { kind: "fields", query: "rea_no_such_field" }, + ]) { + const started = performance.now(); + const response = await client.callTool({ + name: discovery.name, + arguments: input, + }); + assert.notEqual(response.isError, true, JSON.stringify(response)); + assert.ok( + ajv.validate(discovery.outputSchema, response.structuredContent), + JSON.stringify(ajv.errors), + ); + const evidence = parseEvidence(response.structuredContent); + const report = evidence.normalized_result; + const kind = input.kind ?? "protocols"; + const query = (input.query ?? "").toLowerCase(); + const expectedMatches = nativeLines.flatMap((line, index) => { + const columns = line.split("\t"); + if ((kind === "protocols") !== (columns[0] === "P")) return []; + if ( + input.protocol !== undefined && + (columns[0] === "P" ? columns[2] : columns[4]) !== input.protocol + ) + return []; + if ( + ![ + columns[1], + columns[2], + ...(columns[0] === "F" ? [columns[7]] : []), + ].some((value) => value.toLowerCase().includes(query)) + ) + return []; + return [{ sourceLine: index + 1, columns }]; + }); + const offset = input.offset ?? 0; + const limit = input.limit === null ? undefined : (input.limit ?? 50); + const expectedPage = expectedMatches.slice( + offset, + limit === undefined ? undefined : offset + limit, + ); + assert.deepEqual( + report.records.map((record) => record.source_line), + expectedPage.map((record) => record.sourceLine), + ); + assert.deepEqual(report.output_window, { + examined: nativeLines.filter((line) => + line.startsWith(kind === "protocols" ? "P\t" : "F\t"), + ).length, + matching: expectedMatches.length, + returned: expectedPage.length, + has_more: + Math.max(0, expectedMatches.length - offset) > expectedPage.length, + }); + discoveryMeasurements.push({ + input, + elapsed_ms: +(performance.now() - started).toFixed(2), + response_bytes: Buffer.byteLength(JSON.stringify(response)), + estimated_tokens_one_tool_request_response: estimateTokenCount( + JSON.stringify(discovery) + + JSON.stringify(input) + + JSON.stringify(response), + ), + }); + for (const record of report.records) { + const columns = nativeLines[record.source_line - 1].split("\t"); + assert.equal(record.name, columns[1]); + assert.equal(record.abbreviation, columns[2]); + assert.equal(record.field_type, columns[0] === "F" ? columns[3] : null); + assert.equal( + record.parent_protocol, + columns[0] === "F" ? columns[4] : null, + ); + assert.equal(record.description, columns[0] === "F" ? columns[7] : null); + assert.equal(record.display_base, columns[0] === "F" ? columns[5] : null); + assert.equal(record.bitmask, columns[0] === "F" ? columns[6] : null); + } + if (input.query === "idVendor") + assert.ok( + report.records.some( + (row) => + row.abbreviation === "usb.idVendor" && + row.field_type === "FT_UINT16", + ), + ); + if (input.query === "usb") + assert.ok(report.records.some((row) => row.abbreviation === "usb")); + if (input.query === "rea_no_such_field") + assert.equal(report.output_window.matching, 0); + const flags = Object.entries(input).flatMap(([key, value]) => + value === null ? ["--all"] : [`--${key}`, String(value)], + ); + const cli = JSON.parse( + ( + await execute( + process.execPath, + [entrypoint, "inspect-packet-fields", ...flags, "--format", "json"], + { env: environment, timeout: 60000, maxBuffer: 16 * 1024 * 1024 }, + ) + ).stdout, + ); + assert.deepEqual(cli.normalized_result, report); + cases++; + } await client.close(); assert.deepEqual((await readdir(root.path)).sort(), [ "bluetooth.pcap", @@ -329,6 +473,9 @@ console.log( { status: "passed", cases, + discovery_measurements: discoveryMeasurements, + measurement_scope: + "Warm MCP server, fresh TShark processes; tokenx estimates include one advertised tool/request/response, not billed usage. Model-driven selection, CPU and peak process-family memory are unmeasured; no cache/speedup claim.", host: `${process.platform}/${process.arch}`, scope: "real TShark CLI/MCP; synthetic Ethernet/IPv4/UDP/DNS and Bluetooth HCI PCAP; no hardware acquisition", diff --git a/src/application/CapabilityInventory.ts b/src/application/CapabilityInventory.ts index 34b418029..94518db85 100644 --- a/src/application/CapabilityInventory.ts +++ b/src/application/CapabilityInventory.ts @@ -397,7 +397,7 @@ const workflowAvailabilityFor = ({ remediation: "On Linux x64, set absolute REA_PWNTOOLS_PYTHON to caller-supplied Python with pwntools 4.15.0. No active binary target is required.", }; - if (name === "inspect_packet_capture") + if (name === "inspect_packet_capture" || name === "inspect_packet_fields") return policy.packetCaptureEnabled === true ? { reason: "available", remediation: null } : { diff --git a/src/application/captures/PacketCaptureService.ts b/src/application/captures/PacketCaptureService.ts index f2098b49c..e2877fa3f 100644 --- a/src/application/captures/PacketCaptureService.ts +++ b/src/application/captures/PacketCaptureService.ts @@ -1,3 +1,9 @@ +import { + inspectPacketFieldsInputSchema, + packetFieldsCatalogSchema, + type InspectPacketFieldsInput, +} from "../../domain/captures/packetFields.js"; +import type { JsonValue } from "../../domain/jsonValue.js"; import { isAbsolute } from "node:path"; import type { AnalysisExecution, @@ -24,6 +30,10 @@ export interface PacketCapturePort { input: InspectPacketCaptureInput, options?: ExecutionOptions, ): Promise>; + inspectFields( + input: InspectPacketFieldsInput, + options?: ExecutionOptions, + ): Promise>; close(): Promise; } @@ -74,14 +84,58 @@ export class PacketCaptureService { "Capture analyzer returned malformed data or changed the selected artifact identity/options.", ), ); - const evidence = createEvidence(execution.subject, execution.provider, { - operation, - parameters: input.data, - result: report.data, - confidence: "observed", - locations: execution.locations, - limitations: execution.limitations, - }); + return this.#evidence(operation, input.data, execution, options); + } + async inspectFields( + raw: unknown, + options?: ExecutionOptions, + ): Promise> { + const operation = "inspect_packet_fields"; + if (options?.signal?.aborted) + return err(new AnalysisCancelledError(operation)); + const input = inspectPacketFieldsInputSchema.safeParse(raw); + if (!input.success) + return err( + analysisInputErrorFromIssues(operation, input.error.issues, raw, { + cause: input.error, + }), + ); + const decoded = await this.provider.inspectFields(input.data, options); + if (!decoded.ok) return decoded; + const execution = decoded.value; + const report = packetFieldsCatalogSchema.safeParse(execution.result); + if ( + !report.success || + JSON.stringify(report.data.selection) !== JSON.stringify(input.data) || + execution.subject?.path !== report.data.tool.path || + execution.subject.sha256 !== report.data.tool.sha256 + ) + return err( + new AnalysisOutputError( + operation, + "Field discovery returned malformed data or changed the selected installation/options.", + ), + ); + return this.#evidence(operation, input.data, execution, options); + } + #evidence( + operation: string, + parameters: Readonly>, + execution: AnalysisExecution, + options?: ExecutionOptions, + ): Result { + const evidence = createEvidence( + execution.subject ?? undefined, + execution.provider, + { + operation, + parameters, + result: execution.result, + confidence: "observed", + locations: execution.locations, + limitations: execution.limitations, + }, + ); return options?.signal?.aborted ? err( new AnalysisCancelledError(operation, { diff --git a/src/captures/TsharkCaptureProvider.ts b/src/captures/TsharkCaptureProvider.ts index 13cadd774..d22515929 100644 --- a/src/captures/TsharkCaptureProvider.ts +++ b/src/captures/TsharkCaptureProvider.ts @@ -1,3 +1,12 @@ +import { + type InspectPacketFieldsInput, + packetFieldsCatalogSchema, +} from "../domain/captures/packetFields.js"; +import { + parseFieldDiscovery, + FieldDiscoveryBudgetError, + FIELD_RECORD_BUDGET, +} from "./parseFieldDiscovery.js"; import { readLinuxResourceLimits, type LinuxResourceLimits, @@ -91,17 +100,31 @@ export class TsharkCaptureProvider implements PacketCapturePort { options?: ExecutionOptions, ): Promise> { return this.#resources.run( - () => this.#inspect(input, options), + () => this.#executeRequest(input, options), options?.signal, ); } + inspectFields( + input: InspectPacketFieldsInput, + options?: ExecutionOptions, + ): Promise> { + return this.#resources.run( + () => this.#executeRequest(input, options), + options?.signal, + "inspect_packet_fields", + ); + } close(): Promise { return this.#resources.close(); } - async #inspect( - input: InspectPacketCaptureInput, + async #executeRequest( + input: InspectPacketCaptureInput | InspectPacketFieldsInput, options?: ExecutionOptions, ): Promise> { + const operation = "path" in input ? OPERATION : "inspect_packet_fields"; + // TShark 4.6.9 emits a 22.3 MB glossary; keep capture output capped at 4 MiB. + const limits = + "path" in input ? LIMITS : { ...LIMITS, output_bytes: 32 * 1024 * 1024 }; let phase: | "configuration" | "artifact" @@ -109,12 +132,12 @@ export class TsharkCaptureProvider implements PacketCapturePort { | "producer" | "integrity" = "configuration"; let capturedOutput: AnalysisCapturedOutput | undefined; - let resourcePath = input.path; + let resourcePath = "path" in input ? input.path : "TShark field glossary"; let effectiveLimits: LinuxResourceLimits | undefined; let root: ArtifactAnalysisRuntime | undefined; let result: Result; try { - if (options?.signal?.aborted) throw new AnalysisCancelledError(OPERATION); + if (options?.signal?.aborted) throw new AnalysisCancelledError(operation); await this.#resources.release(); if (this.platform !== "linux") throw new Error( @@ -141,7 +164,7 @@ export class TsharkCaptureProvider implements PacketCapturePort { 64 * 1024 * 1024, options?.signal, ); - effectiveLimits = await this.#resources.readLimits(LIMITS); + effectiveLimits = await this.#resources.readLimits(limits); const limitArguments = [ `--as=${effectiveLimits.address_space_bytes}:${effectiveLimits.address_space_bytes}`, `--fsize=${effectiveLimits.file_bytes}:${effectiveLimits.file_bytes}`, @@ -152,7 +175,7 @@ export class TsharkCaptureProvider implements PacketCapturePort { ]; phase = "workspace"; root = await this.createRuntime(); - this.#resources.retain(root, undefined); + this.#resources.retain(root, undefined, operation); const snapshotPath = join(root.path, "capture"); const configurationPath = join(root.path, "configuration"); await mkdir(configurationPath, { mode: 0o700 }); @@ -170,8 +193,8 @@ export class TsharkCaptureProvider implements PacketCapturePort { }, }, { - timeoutMs: LIMITS.command_timeout_ms, - diagnosticBytes: LIMITS.output_bytes, + timeoutMs: limits.command_timeout_ms, + diagnosticBytes: limits.output_bytes, acceptNonZeroExit: true, }, { @@ -200,24 +223,24 @@ export class TsharkCaptureProvider implements PacketCapturePort { ); if (currentLimiter.sha256 !== limiterIdentity.sha256) throw new AnalysisArtifactChangedError( - OPERATION, + operation, limiter, "Configured resource limiter changed during analysis.", { capturedOutput }, ); if (current.sha256 !== tool.sha256) throw new AnalysisArtifactChangedError( - OPERATION, + operation, command, "Configured TShark executable changed during analysis.", { capturedOutput }, ); phase = previousPhase; - resourcePath = input.path; + resourcePath = "path" in input ? input.path : command; if (output.exitCode !== 0) throw new ProviderAdapterError( TSHARK_PROVIDER_IDENTITY.id, - OPERATION, + operation, { diagnostics: { exit_code: output.exitCode ?? null, @@ -238,86 +261,15 @@ export class TsharkCaptureProvider implements PacketCapturePort { throw new Error( "Expected TShark 4.x; inspect the selected installation.", ); - phase = "artifact"; - const selected = await this.#resources.read( - input.path, - LIMITS.input_bytes, - options?.signal, - ); - phase = "workspace"; - await writeFile(snapshotPath, selected.bytes, { - flag: "wx", - mode: 0o400, - }); - phase = "producer"; - const args = [ - "-n", - "-r", - "capture", - "-T", - "json", - "--no-duplicate-keys", - "--temp-dir=.", - ...(input.two_pass ? ["-2"] : []), - ...(input.packet_limit === null - ? [] - : ["-c", String(input.packet_limit)]), - ...(input.display_filter === null ? [] : ["-Y", input.display_filter]), - ...input.decode_as.flatMap((value) => ["-d", value]), - ...[...new Set(["frame.number", ...input.fields])].flatMap((field) => [ - "-e", - field, - ]), - ]; - const output = await execute(args); - phase = "integrity"; - resourcePath = snapshotPath; - if ( - (await this.#resources.read(snapshotPath, LIMITS.input_bytes)) - .sha256 !== selected.sha256 - ) - throw new AnalysisArtifactChangedError( - OPERATION, - snapshotPath, - "TShark changed the private capture copy.", - capturedOutput === undefined ? {} : { capturedOutput }, - ); - phase = "producer"; - resourcePath = input.path; - const matchingPackets = parsePacketOutput( - output.stdout.text, - input.fields, - ); - const start = Math.min(input.match_offset, matchingPackets.length); - const end = - input.match_limit === null - ? matchingPackets.length - : start + Math.min(input.match_limit, matchingPackets.length - start); - const packets = matchingPackets.slice(start, end); - const report = packetCaptureSchema.parse({ - artifact: { - path: input.path, - sha256: selected.sha256, - bytes: selected.bytes.length, - }, - selection: input, - packets, - output_window: { - matching_packets: matchingPackets.length, - returned_packets: packets.length, - has_more: end < matchingPackets.length, - }, - read_scope: - input.packet_limit === null ? "to-eof" : "up-to-input-packet-limit", - configuration: "private-empty-personal-profile", - diagnostics: output.stderr.text, + const metadata = (args: readonly string[], diagnostics: string) => ({ + diagnostics, tool: { configured_path: configured, path: command, sha256: tool.sha256, version_banner: version.stdout.text, version_diagnostics: version.stderr.text, - arguments: args, + arguments: [...args], }, resource_limiter: { configured_path: configuredLimiter, @@ -325,33 +277,136 @@ export class TsharkCaptureProvider implements PacketCapturePort { sha256: limiterIdentity.sha256, arguments: limitArguments, }, - limits: { ...LIMITS, ...effectiveLimits }, - hardware_acquisition: "not-performed", + limits: { ...limits, ...effectiveLimits }, + hardware_acquisition: "not-performed" as const, }); - result = ok( - createAnalysisExecution(report, TSHARK_PROVIDER_IDENTITY, { - subject: { + if (!("path" in input)) { + phase = "producer"; + const args = ["-G", "fields"]; + const output = await execute(args); + const report = packetFieldsCatalogSchema.parse({ + selection: input, + ...parseFieldDiscovery(output.stdout.text, input), + ...metadata(args, output.stderr.text), + limits: { + ...limits, + ...effectiveLimits, + record_budget_bytes: FIELD_RECORD_BUDGET, + }, + configuration: "private-empty-personal-profile", + }); + result = ok( + createAnalysisExecution(report, TSHARK_PROVIDER_IDENTITY, { + subject: { path: command, sha256: tool.sha256, format: "file" }, + limitations: [ + "Protocol and field registrations describe the selected installation, not traffic observed in a capture. Duplicate abbreviations remain distinct producer records with source line numbers.", + "Each request probes again without caching. The private empty personal profile does not fingerprint or isolate installed libraries, global configuration or plugins; they remain trusted.", + "No capture file, interface or device is opened. Resource ceilings do not constitute an OS sandbox. Glossary output is capped at 32 MiB and selected record representation at 8 MiB before expansion.", + ], + }), + ); + } else { + phase = "artifact"; + const selected = await this.#resources.read( + input.path, + LIMITS.input_bytes, + options?.signal, + ); + phase = "workspace"; + await writeFile(snapshotPath, selected.bytes, { + flag: "wx", + mode: 0o400, + }); + phase = "producer"; + const args = [ + "-n", + "-r", + "capture", + "-T", + "json", + "--no-duplicate-keys", + "--temp-dir=.", + ...(input.two_pass ? ["-2"] : []), + ...(input.packet_limit === null + ? [] + : ["-c", String(input.packet_limit)]), + ...(input.display_filter === null + ? [] + : ["-Y", input.display_filter]), + ...input.decode_as.flatMap((value) => ["-d", value]), + ...[...new Set(["frame.number", ...input.fields])].flatMap( + (field) => ["-e", field], + ), + ]; + const output = await execute(args); + phase = "integrity"; + resourcePath = snapshotPath; + if ( + (await this.#resources.read(snapshotPath, LIMITS.input_bytes)) + .sha256 !== selected.sha256 + ) + throw new AnalysisArtifactChangedError( + operation, + snapshotPath, + "TShark changed the private capture copy.", + capturedOutput === undefined ? {} : { capturedOutput }, + ); + phase = "producer"; + resourcePath = "path" in input ? input.path : command; + const matchingPackets = parsePacketOutput( + output.stdout.text, + input.fields, + ); + const start = Math.min(input.match_offset, matchingPackets.length); + const end = + input.match_limit === null + ? matchingPackets.length + : start + + Math.min(input.match_limit, matchingPackets.length - start); + const packets = matchingPackets.slice(start, end); + const report = packetCaptureSchema.parse({ + artifact: { path: input.path, sha256: selected.sha256, - format: "file", + bytes: selected.bytes.length, }, - locations: [{ kind: "artifact-path", path: input.path }], - limitations: [ - "Fields are TShark dissector observations under the selected filter/decode-as/pass settings, not independently verified protocol semantics. Missing fields are absent from the producer output, not known zero or empty values.", - "Packet limits count input packets before display filtering. Filtered results do not establish whole-capture or device traffic absence. Two-pass reassembly cannot recover bytes missing from the recording or beyond the read limit.", - "Repeated field values remain ordered string arrays; raw packet bytes are included only when explicitly selected as a supported field. Export index/score metadata are omitted. All stderr, including warnings on exit zero, is retained.", - "A private empty personal configuration is used. Installed libraries, global configuration and dissector plugins are trusted and not fully fingerprinted. File-only analysis and disabled name resolution do not constitute an OS sandbox.", - "Linux prlimit enforces soft and hard ceilings before execution, preserving stricter inherited limits: at most 1 GiB address space per process, 64 MiB per file, 30 CPU seconds and no core dumps. Effective limits are reported. Resident input is capped at 32 MiB, each command at 30 wall-clock seconds and combined output at 4 MiB. Output overflow fails. These are per-process/per-file limits, not aggregate memory or filesystem quotas; caller-supplied tools and the limiter are trusted.", - ], - }), - ); + selection: input, + packets, + output_window: { + matching_packets: matchingPackets.length, + returned_packets: packets.length, + has_more: end < matchingPackets.length, + }, + read_scope: + input.packet_limit === null ? "to-eof" : "up-to-input-packet-limit", + configuration: "private-empty-personal-profile", + ...metadata(args, output.stderr.text), + }); + result = ok( + createAnalysisExecution(report, TSHARK_PROVIDER_IDENTITY, { + subject: { + path: input.path, + sha256: selected.sha256, + format: "file", + }, + locations: [{ kind: "artifact-path", path: input.path }], + limitations: [ + "Fields are TShark dissector observations under the selected filter/decode-as/pass settings, not independently verified protocol semantics. Missing fields are absent from the producer output, not known zero or empty values.", + "Packet limits count input packets before display filtering. Filtered results do not establish whole-capture or device traffic absence. Two-pass reassembly cannot recover bytes missing from the recording or beyond the read limit.", + "Repeated field values remain ordered string arrays; raw packet bytes are included only when explicitly selected as a supported field. Export index/score metadata are omitted. All stderr, including warnings on exit zero, is retained.", + "A private empty personal configuration is used. Installed libraries, global configuration and dissector plugins are trusted and not fully fingerprinted. File-only analysis and disabled name resolution do not constitute an OS sandbox.", + "Linux prlimit enforces soft and hard ceilings before execution, preserving stricter inherited limits: at most 1 GiB address space per process, 64 MiB per file, 30 CPU seconds and no core dumps. Effective limits are reported. Resident input is capped at 32 MiB, each command at 30 wall-clock seconds and combined output at 4 MiB. Output overflow fails. These are per-process/per-file limits, not aggregate memory or filesystem quotas; caller-supplied tools and the limiter are trusted.", + ], + }), + ); + } } catch (cause) { if ( root !== undefined && cause instanceof OwnedCommandFailure && cause.cleanupOwner !== undefined ) - this.#resources.retain(root, cause.cleanupOwner); + this.#resources.retain(root, cause.cleanupOwner, operation); result = err( this.#failure( cause, @@ -360,18 +415,23 @@ export class TsharkCaptureProvider implements PacketCapturePort { root?.path ?? null, effectiveLimits, capturedOutput, + operation, + limits, ), ); } - return this.#resources.finish(result, (execution) => - createEvidence(execution.subject ?? undefined, execution.provider, { - operation: OPERATION, - parameters: input, - result: execution.result, - confidence: "observed", - locations: execution.locations, - limitations: execution.limitations, - }), + return this.#resources.finish( + result, + (execution) => + createEvidence(execution.subject ?? undefined, execution.provider, { + operation, + parameters: input, + result: execution.result, + confidence: "observed", + locations: execution.locations, + limitations: execution.limitations, + }), + operation, ); } #failure( @@ -386,6 +446,11 @@ export class TsharkCaptureProvider implements PacketCapturePort { workspacePath: string | null, effectiveLimits: LinuxResourceLimits | undefined, observedOutput: AnalysisCapturedOutput | undefined, + operation = OPERATION, + limits: { + readonly output_bytes: number; + readonly command_timeout_ms: number; + } = LIMITS, ): AnalysisError { if (cause instanceof AnalysisError) return cause; if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined) @@ -393,15 +458,34 @@ export class TsharkCaptureProvider implements PacketCapturePort { cause.cleanup, cause, observedOutput, + operation, ); if ( (cause instanceof Error && cause.name === "AbortError") || (cause instanceof ArtifactReaderFailure && cause.reason === "cancelled") ) - return new AnalysisCancelledError(OPERATION, { cause }); + return new AnalysisCancelledError(operation, { cause }); + if (cause instanceof FieldDiscoveryBudgetError) + return new AnalysisResourceConstraintError( + operation, + "memory", + cause.message, + { record_budget_bytes: FIELD_RECORD_BUDGET }, + { + cause, + ...(observedOutput === undefined + ? {} + : { capturedOutput: observedOutput }), + }, + ); const reason = cause instanceof Error ? cause.message : String(cause); if (cause instanceof OwnedCommandFailure) - return this.#resources.commandFailure(cause, LIMITS, effectiveLimits); + return this.#resources.commandFailure( + cause, + limits, + effectiveLimits, + operation, + ); if (phase === "workspace") { const code = cause instanceof Error && @@ -411,13 +495,13 @@ export class TsharkCaptureProvider implements PacketCapturePort { : null; if (code === "ENOSPC" || code === "EDQUOT" || code === "EFBIG") return new AnalysisResourceConstraintError( - OPERATION, + operation, "file-size", `Private analysis workspace failed (${code}): ${reason}`, { workspace_path: workspacePath, system_code: code }, { cause }, ); - return new ProviderAdapterError(TSHARK_PROVIDER_IDENTITY.id, OPERATION, { + return new ProviderAdapterError(TSHARK_PROVIDER_IDENTITY.id, operation, { cause, userMessage: `Cannot prepare the private analysis workspace: ${reason}`, diagnostics: { @@ -440,12 +524,12 @@ export class TsharkCaptureProvider implements PacketCapturePort { ["limit", "integrity", "path"].includes(cause.reason) ) return new AnalysisArtifactChangedError( - OPERATION, + operation, path, `Post-run identity verification failed for ${path}: ${reason}`, detail, ); - return new ProviderAdapterError(TSHARK_PROVIDER_IDENTITY.id, OPERATION, { + return new ProviderAdapterError(TSHARK_PROVIDER_IDENTITY.id, operation, { ...detail, diagnostics: { stage: "integrity", resource_path: path, reason }, }); @@ -453,10 +537,13 @@ export class TsharkCaptureProvider implements PacketCapturePort { if (phase === "configuration") return new AnalysisCapabilityUnavailableError( TSHARK_PROVIDER_IDENTITY.id, - OPERATION, + operation, reason, { cause, + ...(observedOutput === undefined + ? {} + : { capturedOutput: observedOutput }), userMessage: `${reason} On Linux, configure TShark 4.x through REA_TSHARK_COMMAND and util-linux prlimit through REA_TSHARK_PRLIMIT_COMMAND (default /usr/bin/prlimit).`, }, ); @@ -466,14 +553,14 @@ export class TsharkCaptureProvider implements PacketCapturePort { cause.reason === "limit" ) return new AnalysisResourceConstraintError( - OPERATION, + operation, "file-size", reason, { input_bytes: LIMITS.input_bytes }, { cause }, ); if (cause instanceof ArtifactReaderFailure && cause.reason === "integrity") - return new AnalysisArtifactChangedError(OPERATION, path, reason, { + return new AnalysisArtifactChangedError(operation, path, reason, { cause, }); if ( @@ -482,13 +569,18 @@ export class TsharkCaptureProvider implements PacketCapturePort { "code" in cause && (cause.code === "EACCES" || cause.code === "EPERM") ) - return new AnalysisAccessDeniedError(OPERATION, path, cause.code, { + return new AnalysisAccessDeniedError(operation, path, cause.code, { cause, }); if (phase === "artifact") - return new AnalysisInputError(OPERATION, { cause }, [ + return new AnalysisInputError(operation, { cause }, [ { path: ["path"], reason: "invalid_value", message: reason }, ]); - return new AnalysisOutputError(OPERATION, reason, { cause }); + return new AnalysisOutputError(operation, reason, { + cause, + ...(observedOutput === undefined + ? {} + : { capturedOutput: observedOutput }), + }); } } diff --git a/src/captures/parseFieldDiscovery.ts b/src/captures/parseFieldDiscovery.ts new file mode 100644 index 000000000..6498e56dc --- /dev/null +++ b/src/captures/parseFieldDiscovery.ts @@ -0,0 +1,92 @@ +import type { + InspectPacketFieldsInput, + PacketFieldRecord, +} from "../domain/captures/packetFields.js"; + +export class FieldDiscoveryBudgetError extends Error {} +export const FIELD_RECORD_BUDGET = 8 * 1024 * 1024; + +/** TShark -G fields: P/name/abbreviation or F/name/abbreviation/type/parent/base/mask/blurb. */ +export const parseFieldDiscovery = ( + text: string, + input: InspectPacketFieldsInput, +) => { + const copy = (value: string) => + Buffer.from(value, "utf16le").toString("utf16le"); + const query = input.query.toLowerCase(); + const records: PacketFieldRecord[] = []; + let examined = 0, + matching = 0, + sourceLine = 0, + budget = 0; + // Walk the bounded text once; allocate record objects only for the requested page. + for (let start = 0; start < text.length;) { + const end = text.indexOf("\n", start); + let line = text.slice(start, end < 0 ? text.length : end); + start = end < 0 ? text.length : end + 1; + sourceLine++; + if (line.endsWith("\r")) line = line.slice(0, -1); + const columns = line.split("\t", 9); + const [tag, name, abbreviation] = columns; + if ( + !( + (tag === "P" && columns.length === 3) || + (tag === "F" && columns.length === 8) + ) || + name === undefined || + abbreviation === undefined || + abbreviation === "" + ) + throw new Error( + `Malformed TShark field glossary record at line ${sourceLine}`, + ); + if ((input.kind === "protocols") !== (tag === "P")) continue; + examined++; + if ( + input.protocol !== null && + (tag === "P" ? abbreviation : columns[4]) !== input.protocol + ) + continue; + if ( + ![name, abbreviation, ...(tag === "F" ? [columns[7] ?? ""] : [])].some( + (value) => value.toLowerCase().includes(query), + ) + ) + continue; + matching++; + if ( + matching <= input.offset || + records.length >= (input.limit ?? Number.MAX_SAFE_INTEGER) + ) + continue; + // Six bytes per source byte covers worst-case JSON escaping, plus record bookkeeping. + budget += Buffer.byteLength(line, "utf8") * 6 + 512; + if (budget > FIELD_RECORD_BUDGET) + throw new FieldDiscoveryBudgetError( + "Selected glossary records exceed the 8 MiB representation budget; narrow the query or page size", + ); + // Detach retained fields from the much larger producer string. + records.push({ + source_line: sourceLine, + kind: tag === "P" ? "protocol" : "field", + name: copy(name), + abbreviation: copy(abbreviation), + field_type: tag === "F" ? copy(columns[3] ?? "") : null, + parent_protocol: tag === "F" ? copy(columns[4] ?? "") : null, + display_base: tag === "F" ? copy(columns[5] ?? "") : null, + bitmask: tag === "F" ? copy(columns[6] ?? "") : null, + description: tag === "F" ? copy(columns[7] ?? "") : null, + }); + } + if (sourceLine === 0) + throw new Error("TShark returned an empty field glossary"); + return { + records, + output_window: { + examined, + matching, + returned: records.length, + has_more: Math.max(0, matching - input.offset) > records.length, + }, + }; +}; diff --git a/src/cli/captureCommands.ts b/src/cli/captureCommands.ts index 281be7457..2a3566703 100644 --- a/src/cli/captureCommands.ts +++ b/src/cli/captureCommands.ts @@ -1,3 +1,4 @@ +import { inspectPacketFieldsInputSchema } from "../domain/captures/packetFields.js"; import { z } from "incur"; import type { PacketCaptureService } from "../application/captures/PacketCaptureService.js"; import { AnalysisError } from "../domain/analysisErrorBase.js"; @@ -22,9 +23,73 @@ export const registerCaptureCommands = ( environment: Readonly, service: Pick< PacketCaptureService, - "inspect" | "close" + "inspect" | "inspectFields" | "close" > = createPacketCaptureService(environment), ): void => { + const finish = async ( + result: Awaited>, + operation: string, + ) => { + // A one-shot command must retry retained cleanup before discarding its owner. + try { + await service.close(); + } catch (cause) { + const cleanup = + cause instanceof AnalysisError + ? cause + : new ProviderCleanupError( + "tshark-capture", + [], + { reason: String(cause) }, + { cause, operation: operation }, + ); + if (result.ok) cleanup.retainPartialObservation(result.value); + return projectAnalysisError( + result.ok + ? cleanup + : analysisErrorWithCleanupFailure(result.error, cleanup, operation), + ); + } + return result.ok ? result.value : projectAnalysisError(result.error); + }; + cli.command(CLI_COMMANDS.inspectPacketFields, { + description: "Search installed TShark protocol and field definitions", + options: z + .object({ + ...inspectPacketFieldsInputSchema.shape, + limit: z + .number() + .int() + .positive() + .max(Number.MAX_SAFE_INTEGER) + .optional() + .describe("Maximum matching records to return; defaults to 50"), + all: z + .boolean() + .default(false) + .describe("Return all remaining matches within resource limits"), + }) + .superRefine((options, context) => { + if (options.all && options.limit !== undefined) + context.addIssue({ + code: "custom", + path: ["all"], + message: "all and limit are mutually exclusive", + }); + }), + run: ({ options: { all, ...options } }) => + withCommandCancellation((signal) => + logCliCommand(logger, CLI_COMMANDS.inspectPacketFields, async () => + finish( + await service.inspectFields( + { ...options, ...(all ? { limit: null } : {}) }, + { signal }, + ), + "inspect_packet_fields", + ), + ), + ), + }); cli.command(CLI_COMMANDS.inspectPacketCapture, { description: "Inspect an existing packet capture with caller-supplied TShark", @@ -68,31 +133,7 @@ export const registerCaptureCommands = ( }, { signal }, ); - // A one-shot command must retry retained cleanup before discarding its owner. - try { - await service.close(); - } catch (cause) { - const cleanup = - cause instanceof AnalysisError - ? cause - : new ProviderCleanupError( - "tshark-capture", - [], - { reason: String(cause) }, - { cause, operation: "inspect_packet_capture" }, - ); - if (result.ok) cleanup.retainPartialObservation(result.value); - return projectAnalysisError( - result.ok - ? cleanup - : analysisErrorWithCleanupFailure( - result.error, - cleanup, - "inspect_packet_capture", - ), - ); - } - return result.ok ? result.value : projectAnalysisError(result.error); + return finish(result, "inspect_packet_capture"); }), ), }); diff --git a/src/cliCommandNames.ts b/src/cliCommandNames.ts index e7c9bc922..c3d373e4e 100644 --- a/src/cliCommandNames.ts +++ b/src/cliCommandNames.ts @@ -1,5 +1,6 @@ /** Canonical names used by every one-shot CLI command registration. */ export const CLI_COMMANDS = Object.freeze({ + inspectPacketFields: "inspect-packet-fields", inspectPacketCapture: "inspect-packet-capture", decodeSignalRecording: "decode-signal-recording", inspectWasmArtifact: "inspect-wasm-artifact", diff --git a/src/contracts/captures/captureToolContracts.ts b/src/contracts/captures/captureToolContracts.ts index 3e67b3b19..dca211511 100644 --- a/src/contracts/captures/captureToolContracts.ts +++ b/src/contracts/captures/captureToolContracts.ts @@ -1,3 +1,7 @@ +import { + inspectPacketFieldsInputSchema, + packetFieldsCatalogSchema, +} from "../../domain/captures/packetFields.js"; import { inspectPacketCaptureInputSchema, packetCaptureSchema, @@ -6,6 +10,22 @@ import type { ToolContract } from "../toolContractTypes.js"; import { toolContractMetadata } from "../toolEffects.js"; import { evidenceResultOf } from "../toolOutputSchemaPrimitives.js"; export const CAPTURE_TOOL_CONTRACTS = [ + { + name: "inspect_packet_fields", + ...toolContractMetadata("inspect_packet_fields"), + kind: "artifact-provider", + description: + "Discover protocols or field names/types registered in the installed TShark 4.x. kind defaults to protocols; use fields and optional exact protocol abbreviation to select field definitions. query searches name/abbreviation/description by case-insensitive literal substring. Return 50 matches by default with offset/limit paging; null requests all remaining within resource limits. Preserve type/base/bitmask/description, duplicate registrations, source line numbers, coverage, tool identity and diagnostics. Linux, REA_TSHARK_COMMAND and prlimit required. No capture file, interface or binary target needed. Registrations describe tool capabilities, not observed traffic. Each call rereads the glossary; use selected fields with inspect_packet_capture.", + inputSchema: inspectPacketFieldsInputSchema, + outputSchema: evidenceResultOf(packetFieldsCatalogSchema), + examples: [ + { title: "Find USB protocol registrations", input: { query: "usb" } }, + { + title: "Find the installed USB vendor-ID field", + input: { kind: "fields", protocol: "usb", query: "idVendor" }, + }, + ], + }, { name: "inspect_packet_capture", ...toolContractMetadata("inspect_packet_capture"), diff --git a/src/contracts/toolEffects.ts b/src/contracts/toolEffects.ts index 7ce3e4ded..8ea7f23e0 100644 --- a/src/contracts/toolEffects.ts +++ b/src/contracts/toolEffects.ts @@ -59,6 +59,11 @@ export const TOOL_EFFECTS: Readonly> = { writesFilesystem: true, launchesProcess: true, }), + inspect_packet_fields: effects({ + mutatesSession: true, + launchesProcess: true, + writesFilesystem: true, + }), inspect_packet_capture: effects({ mutatesSession: true, launchesProcess: true, diff --git a/src/domain/captures/packetFields.ts b/src/domain/captures/packetFields.ts new file mode 100644 index 000000000..23b25cfcd --- /dev/null +++ b/src/domain/captures/packetFields.ts @@ -0,0 +1,100 @@ +import { z } from "zod"; +import { packetCaptureSchema } from "./packetCapture.js"; + +export const inspectPacketFieldsInputSchema = z.strictObject({ + kind: z + .enum(["protocols", "fields"]) + .default("protocols") + .describe( + "Return protocol registrations or field definitions; defaults to protocols", + ), + protocol: z + .string() + .min(1) + .max(8192) + .nullable() + .default(null) + .describe( + "Exact protocol abbreviation; selects that protocol or its registered fields", + ), + query: z + .string() + .max(8192) + .default("") + .describe( + "Case-insensitive literal substring in name, abbreviation or field description", + ), + offset: z + .number() + .int() + .nonnegative() + .max(Number.MAX_SAFE_INTEGER) + .default(0) + .describe("Number of matching records to skip; defaults to 0"), + limit: z + .number() + .int() + .positive() + .max(Number.MAX_SAFE_INTEGER) + .nullable() + .default(50) + .describe( + "Maximum matches returned; null requests all remaining within the record budget", + ), +}); +export type InspectPacketFieldsInput = z.infer< + typeof inspectPacketFieldsInputSchema +>; +export const packetFieldRecordSchema = z.strictObject({ + source_line: z.number().int().positive(), + kind: z.enum(["protocol", "field"]), + name: z.string(), + abbreviation: z.string(), + field_type: z.string().nullable(), + parent_protocol: z.string().nullable(), + display_base: z.string().nullable(), + bitmask: z.string().nullable(), + description: z.string().nullable(), +}); +export type PacketFieldRecord = z.infer; +export const packetFieldsCatalogSchema = z + .strictObject({ + selection: inspectPacketFieldsInputSchema, + records: z.array(packetFieldRecordSchema), + output_window: z.strictObject({ + examined: z.number().int().nonnegative(), + matching: z.number().int().nonnegative(), + returned: z.number().int().nonnegative(), + has_more: z.boolean(), + }), + diagnostics: z.string(), + tool: packetCaptureSchema.shape.tool, + resource_limiter: packetCaptureSchema.shape.resource_limiter, + limits: packetCaptureSchema.shape.limits.extend({ + record_budget_bytes: z.literal(8388608), + }), + configuration: z.literal("private-empty-personal-profile"), + hardware_acquisition: z.literal("not-performed"), + }) + .superRefine((report, context) => { + const w = report.output_window; + const remaining = Math.max(0, w.matching - report.selection.offset); + const expected = Math.min(remaining, report.selection.limit ?? remaining); + if ( + w.matching > w.examined || + w.returned !== expected || + report.records.length !== expected || + w.has_more !== remaining > expected || + report.records.some( + (row) => + row.kind !== + (report.selection.kind === "protocols" ? "protocol" : "field"), + ) + ) + context.addIssue({ + code: "custom", + path: ["output_window"], + message: + "Field discovery coverage must match the selected page and record kind", + }); + }); diff --git a/src/process/ArtifactAnalysisResources.ts b/src/process/ArtifactAnalysisResources.ts index 5eb90b8e2..b0083e05b 100644 --- a/src/process/ArtifactAnalysisResources.ts +++ b/src/process/ArtifactAnalysisResources.ts @@ -36,7 +36,10 @@ export class ArtifactAnalysisResources { readonly #artifacts = new ArtifactResourceScope(); readonly #pending = new Map< ArtifactAnalysisRuntime, - ProviderProcessSupervisor | undefined + { + readonly owner: ProviderProcessSupervisor | undefined; + readonly operation: string; + } >(); #tail: Promise = Promise.resolve(); #busy = false; @@ -56,14 +59,15 @@ export class ArtifactAnalysisResources { run( work: () => Promise>, signal?: AbortSignal, + operation = this.operation, ): Promise> { if (this.#closed || signal?.aborted) - return Promise.resolve(err(new AnalysisCancelledError(this.operation))); + return Promise.resolve(err(new AnalysisCancelledError(operation))); if (this.#busy) return Promise.resolve( err( new AnalysisResourceConstraintError( - this.operation, + operation, "capacity", this.capacity.message, this.capacity.details, @@ -73,7 +77,7 @@ export class ArtifactAnalysisResources { this.#busy = true; const running = this.#tail .then(() => this.#artifacts.run(work)) - .catch((cause: unknown) => err(this.#artifactFailure(cause))) + .catch((cause: unknown) => err(this.#artifactFailure(cause, operation))) .finally(() => { this.#busy = false; }); @@ -101,8 +105,9 @@ export class ArtifactAnalysisResources { retain( root: ArtifactAnalysisRuntime, owner?: ProviderProcessSupervisor, + operation = this.operation, ): void { - this.#pending.set(root, owner); + this.#pending.set(root, { owner, operation }); } get pendingPaths(): readonly string[] { @@ -110,12 +115,13 @@ export class ArtifactAnalysisResources { } async release(): Promise { - for (const [root, owner] of this.#pending) { + for (const [root, retained] of this.#pending) { try { + const { owner } = retained; if (owner !== undefined) { const stopped = await owner.stop(); if (stopped.status === "incomplete") throw new Error(stopped.reason); - this.#pending.set(root, undefined); + this.#pending.set(root, { ...retained, owner: undefined }); } await root.close(); this.#pending.delete(root); @@ -124,7 +130,7 @@ export class ArtifactAnalysisResources { this.providerId, [root.path], { reason: String(cause) }, - { operation: this.operation, cause }, + { operation: retained.operation, cause }, ); } } @@ -134,6 +140,7 @@ export class ArtifactAnalysisResources { observation: NonNullable, cause: unknown, capturedOutput?: AnalysisCapturedOutput, + operation = this.operation, ): ProviderCleanupError { return new ProviderCleanupError( this.providerId, @@ -144,7 +151,7 @@ export class ArtifactAnalysisResources { ? {} : { captured_output: { ...capturedOutput } }), }, - { operation: this.operation, cause }, + { operation, cause }, ); } @@ -155,6 +162,7 @@ export class ArtifactAnalysisResources { readonly output_bytes: number; }, effectiveLimits: LinuxResourceLimits | undefined, + operation = this.operation, ): AnalysisError { const capturedOutput = cause.snapshot === null @@ -171,16 +179,16 @@ export class ArtifactAnalysisResources { ...(capturedOutput === undefined ? {} : { capturedOutput }), }; if (cause.reason === "cancelled") - return new AnalysisCancelledError(this.operation, detail); + return new AnalysisCancelledError(operation, detail); if (cause.reason === "timeout") return new AnalysisTimeoutError( - this.operation, + operation, limits.command_timeout_ms, detail, ); if (cause.reason === "output-limit") return new AnalysisResourceConstraintError( - this.operation, + operation, "memory", cause.message, { output_bytes: limits.output_bytes }, @@ -192,18 +200,19 @@ export class ArtifactAnalysisResources { cause.snapshot?.signal === "SIGXCPU") ) return new AnalysisResourceConstraintError( - this.operation, + operation, cause.snapshot.signal === "SIGXFSZ" ? "file-size" : "cpu", `Producer terminated with ${cause.snapshot.signal}; the signal's exact cause is not independently established.`, effectiveLimits === undefined ? null : { ...effectiveLimits }, detail, ); - return new AnalysisOutputError(this.operation, cause.message, detail); + return new AnalysisOutputError(operation, cause.message, detail); } async finish( result: Result, observation: (value: T) => AnalysisPartialObservation, + operation = this.operation, ): Promise> { try { await this.release(); @@ -223,7 +232,7 @@ export class ArtifactAnalysisResources { : projectAnalysisError(result.error), }, { - operation: this.operation, + operation, cause, ...(partialObservation === undefined ? {} : { partialObservation }), }, @@ -260,11 +269,11 @@ export class ArtifactAnalysisResources { if (descriptorFailure !== undefined) throw descriptorFailure; } - #artifactFailure(cause: unknown): AnalysisError { + #artifactFailure(cause: unknown, operation = this.operation): AnalysisError { if (cause instanceof AnalysisError) return cause; if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined) - return this.cleanupFailure(cause.cleanup, cause); - return new ProviderAdapterError(this.providerId, this.operation, { + return this.cleanupFailure(cause.cleanup, cause, undefined, operation); + return new ProviderAdapterError(this.providerId, operation, { cause, diagnostics: { stage: "artifact-resources", reason: String(cause) }, }); diff --git a/src/server/registerCaptureTools.ts b/src/server/registerCaptureTools.ts index c097f89be..296faba5c 100644 --- a/src/server/registerCaptureTools.ts +++ b/src/server/registerCaptureTools.ts @@ -28,4 +28,17 @@ export const registerCaptureTools = ( toolRegistrationOptions(contract), handler(contract), ); + const discovery = toolContract("inspect_packet_fields"); + const discover = createEvidenceToolHandler( + server, + (input, options) => service.inspectFields(input, options), + logger, + recordEvidence, + withAdmittedAnalysis, + ); + server.registerTool( + discovery.name, + toolRegistrationOptions(discovery), + discover(discovery), + ); }; diff --git a/tests/boundary/captures/captureProvider.test.ts b/tests/boundary/captures/captureProvider.test.ts index 9f5fb4afc..2428b39d4 100644 --- a/tests/boundary/captures/captureProvider.test.ts +++ b/tests/boundary/captures/captureProvider.test.ts @@ -1,3 +1,4 @@ +import { packetFieldsCatalogSchema } from "../../../src/domain/captures/packetFields.js"; import { OwnedCommandFailure } from "../../../src/process/OwnedCommand.js"; import { it, expect, onTestFinished } from "vitest"; import { writeFile, readdir, chmod, symlink } from "node:fs/promises"; @@ -20,14 +21,15 @@ const fixture = async (mode = "ok", failCleanup = false) => { await writeFile( script, `import {readFileSync, writeFileSync, chmodSync, truncateSync} from 'node:fs'; -if (process.argv.includes('--version')) { console.log('TShark (Wireshark) 4.6.9 (fixture)'); console.error('version warning'); } +if (process.argv.includes('--version')) { console.log(${JSON.stringify(mode)} === 'bad-version' ? 'TShark (Wireshark) 3.0.0' : 'TShark (Wireshark) 4.6.9 (fixture)'); console.error('version warning'); } else if (${JSON.stringify(mode)} === 'nonzero') { console.error('invalid decoder'); process.exitCode = 1; } else if (${JSON.stringify(mode)} === 'malformed-copy') { chmodSync('capture', 0o600); writeFileSync('capture', 'changed'); } else if (${JSON.stringify(mode)} === 'oversized-copy') { chmodSync('capture', 0o600); truncateSync('capture', 32 * 1024 * 1024 + 1); console.log('[]'); } -else if (${JSON.stringify(mode)} === 'overflow') process.stdout.write('x'.repeat(5 * 1024 * 1024)); +else if (${JSON.stringify(mode)} === 'overflow') process.stdout.write('x'.repeat((process.argv.includes('-G') ? 33 : 5) * 1024 * 1024)); else if (${JSON.stringify(mode)} === 'wait') { console.log('ready'); setInterval(() => {}, 1000); } else if (${JSON.stringify(mode)} === 'invalid-json') console.log('not json'); else if (${JSON.stringify(mode)} === 'many') console.log(JSON.stringify(Array.from({length:90},(_,i)=>({_source:{layers:{"frame.number":[String(i+1)],"ip.addr":["192.0.2.1"]}}})))); +else if (process.argv.includes('-G')) { console.log('P\\tUniversal Serial Bus\\tusb\\nF\\tVendor ID\\tusb.idVendor\\tFT_UINT16\\tusb\\tBASE_HEX\\t0x0\\t\\nF\\tOther Vendor\\tusb.idVendor\\tFT_UINT32\\tusb\\tBASE_HEX\\t0x0\\tSecond registration\\nP\\tInternet Protocol\\tip\\nF\\tAddress\\tip.addr\\tFT_IPv4\\tip\\t\\t0x0\\tSource or destination'); console.error('glossary warning'); } else { if(readFileSync('capture', 'utf8') !== 'recording bytes') throw Error('wrong copy'); console.log(JSON.stringify([{_source:{layers:{"frame.number":["1"],"ip.addr":["192.0.2.1","192.0.2.2"]}}}])); console.error('producer warning'); } `, ); @@ -301,6 +303,9 @@ it("retains a completed observation when cancellation arrives at the service bou const execution = await f.service.provider.inspect(input); if (!execution.ok) throw execution.error; const service = new PacketCaptureService({ + inspectFields: async () => { + throw new Error("Unexpected field discovery call"); + }, inspect: async () => { f.controller.abort(); return execution; @@ -465,3 +470,158 @@ it("rejects oversized selector lists and UTF-8 argv expansion before provider wo } expect(f.calls()).toBe(0); }); + +it("discovers installed protocols and field definitions without a capture", async () => { + const f = await fixture(); + for (const [input, expected] of [ + [ + { query: "USB" }, + { + records: [{ abbreviation: "usb", kind: "protocol", field_type: null }], + output_window: { examined: 2, matching: 1, returned: 1 }, + }, + ], + [ + { kind: "fields", protocol: "usb", limit: 1 }, + { + records: [ + { + abbreviation: "usb.idVendor", + field_type: "FT_UINT16", + source_line: 2, + }, + ], + output_window: { + examined: 3, + matching: 2, + returned: 1, + has_more: true, + }, + }, + ], + [ + { kind: "fields", protocol: "usb", offset: 1, limit: null }, + { + records: [ + { + abbreviation: "usb.idVendor", + field_type: "FT_UINT32", + source_line: 3, + }, + ], + output_window: { matching: 2, returned: 1, has_more: false }, + }, + ], + [ + { kind: "fields", query: "destination" }, + { + records: [ + { + abbreviation: "ip.addr", + description: "Source or destination", + display_base: "", + }, + ], + }, + ], + [ + { query: "no-such-protocol" }, + { records: [], output_window: { matching: 0, returned: 0 } }, + ], + ] as const) { + const result = await f.service.inspectFields(input); + if (!result.ok) throw result.error; + const report = packetFieldsCatalogSchema.parse( + result.value.normalized_result, + ); + expect(report).toMatchObject(expected); + expect(report.tool.arguments).toEqual(["-G", "fields"]); + expect(report.tool.version_diagnostics).toBe("version warning\n"); + expect(report.diagnostics).toBe("glossary warning\n"); + expect(report.limits.output_bytes).toBe(32 * 1024 * 1024); + } + expect( + (await readdir(f.directory)).filter((name) => name.startsWith("decoder-")), + ).toEqual([]); +}); + +it("shares discovery admission with inspection and refuses work after shutdown", async () => { + const f = await fixture("gated"); + const active = f.service.inspect({ path: f.path, fields: ["ip.addr"] }); + await f.acquired; + const busy = await f.service.inspectFields({}); + f.release(); + if (!(await active).ok) throw new Error("Inspection failed"); + expect(busy).toMatchObject({ + ok: false, + error: { operation: "inspect_packet_fields", resource: "capacity" }, + }); + await f.service.close(); + expect(await f.service.inspectFields({})).toMatchObject({ + ok: false, + error: { + operation: "inspect_packet_fields", + _tag: "AnalysisCancelledError", + }, + }); +}); + +it.each([ + "bad-version", + "nonzero", + "overflow", + "wait", + "oversized-tool", + "invalid-json", +])("retains field discovery failure facts: %s", async (mode) => { + const f = await fixture(mode); + const result = await f.service.inspectFields( + {}, + { signal: f.controller.signal }, + ); + if (result.ok) throw new Error("Expected failure"); + expect(result.error).toMatchObject({ operation: "inspect_packet_fields" }); + if (mode === "bad-version") + expect(result.error.capturedOutput).toMatchObject({ + stdout: "TShark (Wireshark) 3.0.0\n", + stderr: "version warning\n", + }); + if (mode === "invalid-json") + expect(result.error.capturedOutput?.stdout).toBe("not json\n"); + if (mode === "overflow") + expect(result.error).toMatchObject({ + resource: "memory", + reportedLimits: { output_bytes: 32 * 1024 * 1024 }, + }); +}); + +it("retains completed discovery on late cancellation and cleanup failure", async () => { + for (const mode of ["post-run-abort", "cleanup"]) { + const f = await fixture(mode, mode === "cleanup"); + const result = await f.service.inspectFields( + { query: "usb" }, + { signal: f.controller.signal }, + ); + expect(result).toMatchObject({ + ok: false, + error: { + partialObservation: { + operation: "inspect_packet_fields", + normalized_result: { records: [{ abbreviation: "usb" }] }, + }, + }, + }); + if (mode === "cleanup") { + await f.service.inspectFields({}); + await f.service.inspect({ path: f.path, fields: ["ip.addr"] }); + expect(f.calls()).toBe(2); + for (let attempt = 0; attempt < 2; attempt++) + await expect(f.service.close()).rejects.toMatchObject({ + operation: "inspect_packet_fields", + cleanupIncomplete: true, + }); + } + f.allowCleanup(); + await f.service.close(); + } +}); diff --git a/tests/boundary/captures/fieldDiscovery.test.ts b/tests/boundary/captures/fieldDiscovery.test.ts new file mode 100644 index 000000000..79a27f7bf --- /dev/null +++ b/tests/boundary/captures/fieldDiscovery.test.ts @@ -0,0 +1,67 @@ +import { expect, it } from "vitest"; +import { inspectPacketFieldsInputSchema } from "../../../src/domain/captures/packetFields.js"; +import { parseFieldDiscovery } from "../../../src/captures/parseFieldDiscovery.js"; + +it("keeps duplicate registrations and source ordinals instead of overwriting abbreviations", () => { + const text = + "P\tUSB\tusb\r\nF\tVendor\tusb.idVendor\tFT_UINT16\tusb\tBASE_HEX\t0x0\t\r\nF\tVendor\tusb.idVendor\tFT_UINT32\tusb\tBASE_HEX\t0x0\t"; + const result = parseFieldDiscovery( + text, + inspectPacketFieldsInputSchema.parse({ + kind: "fields", + query: "idVendor", + offset: 1, + }), + ); + expect(result.records).toMatchObject([ + { + source_line: 3, + abbreviation: "usb.idVendor", + field_type: "FT_UINT32", + description: "", + }, + ]); + expect(result.output_window).toEqual({ + examined: 2, + matching: 2, + returned: 1, + has_more: false, + }); +}); + +it("rejects damaged records even outside the selected page", () => { + for (const text of [ + "", + "P\tmissing", + "F\tname\tabbr", + "P\tname\t", + "P\tUSB\tusb\nBAD", + ]) + expect(() => + parseFieldDiscovery( + text, + inspectPacketFieldsInputSchema.parse({ limit: 1 }), + ), + ).toThrow(); +}); + +it("bounds explicit full-result expansion while allowing a page from the same large glossary", () => { + const text = "P\t" + "Long name ".repeat(30) + "\tabbr\n"; + const glossary = text.repeat(6000); + expect(() => + parseFieldDiscovery( + glossary, + inspectPacketFieldsInputSchema.parse({ limit: null }), + ), + ).toThrow("representation budget"); + const result = parseFieldDiscovery( + glossary, + inspectPacketFieldsInputSchema.parse({ offset: 5999, limit: 1 }), + ); + expect(result.output_window).toEqual({ + examined: 6000, + matching: 6000, + returned: 1, + has_more: false, + }); +}); diff --git a/tests/boundary/cli/captureCleanup.test.ts b/tests/boundary/cli/captureCleanup.test.ts index 1ec6adedd..8f34b6839 100644 --- a/tests/boundary/cli/captureCleanup.test.ts +++ b/tests/boundary/cli/captureCleanup.test.ts @@ -32,6 +32,9 @@ it.each([false, true])( silentLogger, {}, { + inspectFields: async () => { + throw new Error("Unexpected field discovery call"); + }, inspect: async () => err(original), close: async () => { await Promise.resolve(); @@ -87,6 +90,9 @@ it("rejects an oversized selector budget before the CLI invokes its service", as silentLogger, {}, { + inspectFields: async () => { + throw new Error("Unexpected field discovery call"); + }, inspect: async () => { called = true; throw new Error("Must not inspect"); @@ -125,6 +131,9 @@ it("rejects conflicting output options before the CLI invokes its service", asyn silentLogger, {}, { + inspectFields: async () => { + throw new Error("Unexpected field discovery call"); + }, inspect: async () => { called = true; throw new Error("Must not inspect"); diff --git a/tests/boundary/mcp/packetCaptureMcp.test.ts b/tests/boundary/mcp/packetCaptureMcp.test.ts index e11b60d3e..a5acf1925 100644 --- a/tests/boundary/mcp/packetCaptureMcp.test.ts +++ b/tests/boundary/mcp/packetCaptureMcp.test.ts @@ -8,42 +8,53 @@ import { Ajv2020 } from "ajv/dist/2020.js"; import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; -it("advertises valid target-free packet capture schemas and actionable missing-tool diagnostics", async () => { - const harness = await createApplicationMcpHarness({ - serverOptions: { environment: {} }, - }); - onTestFinished(harness.close); - const tool = (await harness.client.listTools()).tools.find( - ({ name }) => name === "inspect_packet_capture", - ); - if (tool?.outputSchema === undefined) throw new Error("Capture tool missing"); - const ajv = new Ajv2020({ strict: false, validateFormats: false }); - const inputSchema: Record = tool.inputSchema; - const outputSchema: Record = tool.outputSchema; - expect(ajv.validateSchema(inputSchema)).toBe(true); - expect(ajv.validateSchema(outputSchema)).toBe(true); - const input = { - path: - process.platform === "win32" ? "C:\\recording.pcap" : "/recording.pcap", - fields: ["ip.addr"], - }; - expect(ajv.validate(inputSchema, input)).toBe(true); - expect(ajv.validate(inputSchema, { ...input, interface: "lo0" })).toBe(false); - const result = await harness.client.callTool({ - name: tool.name, - arguments: input, - }); - expect(parseMcpToolError(result)).toMatchObject({ - error: { - code: "capability_unavailable", - details: { - operation: tool.name, - reason: expect.stringContaining("REA_TSHARK_COMMAND"), +it.each(["inspect_packet_capture", "inspect_packet_fields"])( + "advertises valid target-free schemas and missing-tool diagnostics: %s", + async (name) => { + const harness = await createApplicationMcpHarness({ + serverOptions: { environment: {} }, + }); + onTestFinished(harness.close); + const tool = (await harness.client.listTools()).tools.find( + (tool) => tool.name === name, + ); + if (tool?.outputSchema === undefined) + throw new Error("Capture tool missing"); + const ajv = new Ajv2020({ strict: false, validateFormats: false }); + const inputSchema: Record = tool.inputSchema; + const outputSchema: Record = tool.outputSchema; + expect(ajv.validateSchema(inputSchema)).toBe(true); + expect(ajv.validateSchema(outputSchema)).toBe(true); + const input = + name === "inspect_packet_fields" + ? { query: "usb" } + : { + path: + process.platform === "win32" + ? "C:\\recording.pcap" + : "/recording.pcap", + fields: ["ip.addr"], + }; + expect(ajv.validate(inputSchema, input)).toBe(true); + expect(ajv.validate(inputSchema, { ...input, interface: "lo0" })).toBe( + false, + ); + const result = await harness.client.callTool({ + name: tool.name, + arguments: input, + }); + expect(parseMcpToolError(result)).toMatchObject({ + error: { + code: "capability_unavailable", + details: { + operation: tool.name, + reason: expect.stringContaining("REA_TSHARK_COMMAND"), + }, }, - }, - }); - await harness.client.ping(); -}); + }); + await harness.client.ping(); + }, +); it.each(["cancelled", "cleanup"])( "retains completed Evidence after %s failure", @@ -78,6 +89,9 @@ it.each(["cancelled", "cleanup"])( }, ); const packetCapture = new PacketCaptureService({ + inspectFields: async () => { + throw new Error("Unexpected field discovery call"); + }, inspect: async () => err(failure), close: async () => {}, }); @@ -120,6 +134,9 @@ it.each(["cancelled", "cleanup"])( it("rejects oversized selector expansion through the MCP boundary before invoking the provider", async () => { let calls = 0; const packetCapture = new PacketCaptureService({ + inspectFields: async () => { + throw new Error("Unexpected field discovery call"); + }, inspect: async () => { calls++; return err(new AnalysisCancelledError("inspect_packet_capture")); From fe8d0bdd36fe12bc0836468d1a702a34fbdc219b Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:56:04 +0800 Subject: [PATCH 057/159] fix(android): reject malformed UTF-8 at the JADX protocol boundary --- src/android/JadxMcpTransport.ts | 8 +- .../boundary/android/jadxMcpTransport.test.ts | 94 +++++++++++-------- 2 files changed, 61 insertions(+), 41 deletions(-) diff --git a/src/android/JadxMcpTransport.ts b/src/android/JadxMcpTransport.ts index c648696fc..a40d517f5 100644 --- a/src/android/JadxMcpTransport.ts +++ b/src/android/JadxMcpTransport.ts @@ -199,9 +199,11 @@ export class JadxMcpTransport implements Transport { throw new Error(`JADX MCP frame exceeds ${MAX_FRAME_BYTES} bytes`); this.#appendFrame(fragment, frameBytes); if (newline === -1) return; - const line = this.#frameBuffer - .subarray(0, this.#frameLength) - .toString("utf8"); + const line = new TextDecoder("utf-8", { + fatal: true, + // Preserve a leading BOM so the JSON-RPC parser can reject it. + ignoreBOM: true, + }).decode(this.#frameBuffer.subarray(0, this.#frameLength)); this.#frameBuffer = Buffer.alloc(0); this.#frameLength = 0; this.onmessage?.(deserializeMessage(line)); diff --git a/tests/boundary/android/jadxMcpTransport.test.ts b/tests/boundary/android/jadxMcpTransport.test.ts index afe79fc09..fe05f14b4 100644 --- a/tests/boundary/android/jadxMcpTransport.test.ts +++ b/tests/boundary/android/jadxMcpTransport.test.ts @@ -26,6 +26,26 @@ class FixtureProcess extends EventEmitter implements ProviderProcessHandle { } } +const fixtureLauncher = + (process: FixtureProcess): JadxLauncher => + async ({ runId }) => ({ + process, + ownership: { + runId, + leaderPid: process.pid, + processGroupId: process.pid, + expectedParentPid: process.pid, + }, + cleanup: async () => { + process.emit("exit", 0, null); + process.emit("close", 0, null); + process.stdout.end(); + process.stderr.end(); + process.stdin.end(); + return { cleaned: true, signaled: false }; + }, + }); + it("joins pending input errors before detaching the owned protocol stream", async () => { let completeWrite: ((cause: Error) => void) | undefined; const failure = new Error("owned pipe closed during cancellation"); @@ -70,26 +90,9 @@ it("joins pending input errors before detaching the owned protocol stream", asyn it("decodes split UTF-8 and newline bytes, then routes multiple complete frames", async () => { const process = new FixtureProcess(); - const launcher: JadxLauncher = async ({ runId }) => ({ - process, - ownership: { - runId, - leaderPid: process.pid, - processGroupId: process.pid, - expectedParentPid: process.pid, - }, - cleanup: async () => { - process.emit("exit", 0, null); - process.emit("close", 0, null); - process.stdout.end(); - process.stderr.end(); - process.stdin.end(); - return { cleaned: true, signaled: false }; - }, - }); const transport = new JadxMcpTransport( { command: "fixture", arguments: [] }, - launcher, + fixtureLauncher(process), ); const received: JSONRPCMessage[] = []; transport.onmessage = (message) => received.push(message); @@ -98,7 +101,9 @@ it("decodes split UTF-8 and newline bytes, then routes multiple complete frames" const first = { jsonrpc: "2.0", id: 1, - result: { content: [{ type: "text", text: "snow 雪 🦊" }] }, + result: { + content: [{ type: "text", text: "snow 雪 🦊 and literal �" }], + }, } satisfies JSONRPCMessage; const second = { jsonrpc: "2.0", @@ -120,28 +125,41 @@ it("decodes split UTF-8 and newline bytes, then routes multiple complete frames" await transport.close(); }); -it("accepts a complete JSON-RPC frame at the 8 MiB boundary", async () => { +it.each([ + ["invalid UTF-8 byte", Buffer.from([0xff])], + ["truncated UTF-8 sequence", Buffer.from([0xe2, 0x82])], +])("rejects %s in a complete JSON-RPC frame", async (_label, invalid) => { const process = new FixtureProcess(); - const launcher: JadxLauncher = async ({ runId }) => ({ - process, - ownership: { - runId, - leaderPid: process.pid, - processGroupId: process.pid, - expectedParentPid: process.pid, - }, - cleanup: async () => { - process.emit("exit", 0, null); - process.emit("close", 0, null); - process.stdout.end(); - process.stderr.end(); - process.stdin.end(); - return { cleaned: true, signaled: false }; - }, - }); const transport = new JadxMcpTransport( { command: "fixture", arguments: [] }, - launcher, + fixtureLauncher(process), + ); + const received: JSONRPCMessage[] = []; + const errors: Error[] = []; + transport.onmessage = (message) => received.push(message); + transport.onerror = (cause) => errors.push(cause); + + await transport.start(); + process.stdout.write( + Buffer.concat([ + Buffer.from('{"jsonrpc":"2.0","id":1,"result":{"text":"'), + invalid, + Buffer.from('"}}\n'), + ]), + ); + await new Promise((resolve) => setImmediate(resolve)); + await transport.close(); + + expect(received).toEqual([]); + expect(errors).toHaveLength(1); + expect(errors[0]?.message).toMatch(/encoded data was not valid/i); +}); + +it("accepts a complete JSON-RPC frame at the 8 MiB boundary", async () => { + const process = new FixtureProcess(); + const transport = new JadxMcpTransport( + { command: "fixture", arguments: [] }, + fixtureLauncher(process), ); const received: JSONRPCMessage[] = []; transport.onmessage = (message) => received.push(message); From 4276aaf18bb607cdd0182280f76cf50fcd110c3c Mon Sep 17 00:00:00 2001 From: Olorin <59321404+Olorinm@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:58:19 +0800 Subject: [PATCH 058/159] feat(signals): discover installed decoders and option help (#2487) * feat(signals): discover installed decoders and selected option help * fix(signals): bound retained discovery data and preserve resource failures * fix(signals): complete CLI help and portable decoder patterns * test(signals): verify case-insensitive decoder help queries * fix(signals): retain discovery operation across cleanup retries --- .../skills/reverse-engineer-anything/SKILL.md | 3 + .github/workflows/real-signal-recording.yml | 2 +- docs/signal-recordings.md | 34 +++ scripts/verify-signal-recording.mjs | 106 ++++++- src/application/CapabilityInventory.ts | 2 +- .../signals/SignalRecordingService.ts | 70 ++++- src/cli/signalCommands.ts | 95 ++++-- src/cliCommandNames.ts | 1 + src/contracts/signals/signalToolContracts.ts | 20 ++ src/contracts/toolEffects.ts | 5 + src/domain/signals/signalDecoders.ts | 86 ++++++ src/server/registerSignalTools.ts | 13 + src/signals/SigrokRecordingProvider.ts | 278 +++++++++++------- src/signals/parseDecoderDiscovery.ts | 79 +++++ tests/boundary/cli/signalCleanup.test.ts | 6 + tests/boundary/mcp/signalRecordingMcp.test.ts | 81 ++--- .../boundary/signals/decoderDiscovery.test.ts | 48 +++ .../signals/recordingProvider.test.ts | 189 +++++++++++- 18 files changed, 945 insertions(+), 173 deletions(-) create mode 100644 src/domain/signals/signalDecoders.ts create mode 100644 src/signals/parseDecoderDiscovery.ts create mode 100644 tests/boundary/signals/decoderDiscovery.test.ts diff --git a/.agents/skills/reverse-engineer-anything/SKILL.md b/.agents/skills/reverse-engineer-anything/SKILL.md index 48bc8b7df..de9f6b7c1 100644 --- a/.agents/skills/reverse-engineer-anything/SKILL.md +++ b/.agents/skills/reverse-engineer-anything/SKILL.md @@ -92,6 +92,9 @@ explicit path or endpoint and do not need it. [packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md). Select an exact DNS question for address pages; page exhaustion does not prove network completeness. This reads recordings and does not acquire traffic. +- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs + or selected decoder help, including channels and options. Query and page the + relevant details before choosing settings; this does not scan hardware. - Recorded logic/signal traces: `decode_signal_recording` with explicit decoder, channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through `REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md). diff --git a/.github/workflows/real-signal-recording.yml b/.github/workflows/real-signal-recording.yml index 5ec413e72..cc902037e 100644 --- a/.github/workflows/real-signal-recording.yml +++ b/.github/workflows/real-signal-recording.yml @@ -44,7 +44,7 @@ jobs: - run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3 - run: npm ci - run: npm run build:cached - - run: npm run test:focused -- src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts + - run: npm run test:focused -- tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts - run: node scripts/verify-signal-recording.mjs env: REA_SIGROK_COMMAND: /usr/bin/sigrok-cli diff --git a/docs/signal-recordings.md b/docs/signal-recordings.md index 520d6e882..69b14d193 100644 --- a/docs/signal-recordings.md +++ b/docs/signal-recordings.md @@ -106,3 +106,37 @@ One decode is admitted at a time, including acquisition and cleanup. Excess requests receive a retryable capacity error before acquiring another file or worker. Shutdown closes admission and drains accepted work; failed cleanup remains owned and retryable. + +## Discover decoders before choosing settings + +`inspect_signal_decoders` / `inspect-signal-decoders` queries the **installed** +sigrok catalog without opening a recording or scanning devices: + +```sh +rea inspect-signal-decoders --query uart +rea inspect-signal-decoders --decoder uart --query baud +rea inspect-signal-decoders --decoder i2c --offset 0 --limit 50 +``` + +Omit `decoder` to search decoder IDs and names. Supply one exact decoder ID to +read its literal help, including required/optional channels, options, annotation +classes and documentation. `query` is a case-insensitive literal substring; +matching help retains original one-based line numbers. Empty query selects all. +`offset` skips matches; `limit` defaults to 50. MCP `limit: null` or CLI `--all` +returns all remaining matches; CLI `--all` and `--limit` conflict. +The window reports examined, matching and returned records and `has_more`. +A filtered empty page is not evidence that the installation has no decoders. + +Discovery refuses more than 65,535 line separators before constructing record +objects (65,536-line representation budget), in addition to the 4 MiB output cap. +Exceeding that budget is a resource failure with captured output. Returned text +is copied so a retained page does not keep the complete producer output alive. + +Discovery shares decoding's owned process, admission slot, resource limits, +cleanup retries and Evidence delivery. It records the executable and limiter +identities, version banner and warnings. Failed version checks and malformed +producer output preserve diagnostics. Help is producer text, not an independently +validated parameter schema. Each request probes again; it does not cache or claim +stable pages across installation changes. Libraries/decoder modules remain +trusted and are not fully fingerprinted. Catalog output includes a SHA-256 of +its executable, not a digest of the complete installation. diff --git a/scripts/verify-signal-recording.mjs b/scripts/verify-signal-recording.mjs index a095b97ff..f5baaf0b9 100644 --- a/scripts/verify-signal-recording.mjs +++ b/scripts/verify-signal-recording.mjs @@ -1,5 +1,6 @@ #!/usr/bin/env node import assert from "node:assert/strict"; +import { estimateTokenCount } from "tokenx"; import { execFile } from "node:child_process"; import { createHash } from "node:crypto"; import { readFile, writeFile, readdir } from "node:fs/promises"; @@ -68,6 +69,20 @@ for (const byte of Buffer.from("REA")) { lines.push(`#${tick + 1000}`, "1!"); const source = lines.join("\n") + "\n"; let cases = 0; +const discoveryMeasurements = []; +const inheritedAddressSpace = (await readFile("/proc/self/limits", "utf8")) + .split("\n") + .find((line) => line.startsWith("Max address space ")) + ?.trim() + .match(/^Max address space\s+(\S+)\s+(\S+)\s+bytes$/u); +assert.ok(inheritedAddressSpace); +const expectedAddressSpace = Math.min( + 1073741824, + ...inheritedAddressSpace + .slice(1) + .map((value) => (value === "unlimited" ? Infinity : Number(value))), +); + try { await writeFile(path, source); await client.connect(transport); @@ -104,14 +119,16 @@ try { createHash("sha256").update(source).digest("hex"), ); assert.equal(report.hardware_acquisition, "not-performed"); - assert.equal(report.limits.address_space_bytes, 1073741824); + assert.equal(report.limits.address_space_bytes, expectedAddressSpace); assert.equal(report.tool.configured_path, process.env.REA_SIGROK_COMMAND); assert.equal( report.resource_limiter.configured_path, process.env.REA_SIGROK_PRLIMIT_COMMAND ?? "/usr/bin/prlimit", ); assert.ok( - report.resource_limiter.arguments.includes("--as=1073741824:1073741824"), + report.resource_limiter.arguments.includes( + `--as=${expectedAddressSpace}:${expectedAddressSpace}`, + ), ); assert.equal(report.diagnostics, ""); const flags = Object.entries(input) @@ -245,6 +262,88 @@ try { }); assert.equal(invalid.isError, true); cases++; + const discoveryTool = (await client.listTools()).tools.find( + ({ name }) => name === "inspect_signal_decoders", + ); + assert.ok(discoveryTool?.outputSchema); + assert.ok(ajv.validateSchema(discoveryTool.inputSchema)); + assert.ok(ajv.validateSchema(discoveryTool.outputSchema)); + const nativeHelp = ( + await execute( + process.env.REA_SIGROK_COMMAND, + ["--dont-scan", "--protocol-decoders=uart", "--show"], + { env: environment, timeout: 30000, maxBuffer: 4 * 1024 * 1024 }, + ) + ).stdout.split(/\r?\n/); + if (nativeHelp.at(-1) === "") nativeHelp.pop(); + for (const input of [ + { query: "uart" }, + { limit: 1 }, + { offset: 1, limit: 2 }, + { decoder: "uart", query: "BaUd" }, + { decoder: "uart", offset: 50, limit: null }, + { query: "rea_no_such_decoder" }, + ]) { + const started = performance.now(); + const response = await client.callTool({ + name: discoveryTool.name, + arguments: input, + }); + assert.notEqual(response.isError, true, JSON.stringify(response)); + assert.ok( + ajv.validate(discoveryTool.outputSchema, response.structuredContent), + JSON.stringify(ajv.errors), + ); + const evidence = parseEvidence(response.structuredContent); + const report = evidence.normalized_result; + discoveryMeasurements.push({ + input, + elapsed_ms: +(performance.now() - started).toFixed(2), + response_bytes: Buffer.byteLength(JSON.stringify(response)), + estimated_tokens_one_tool_request_response: estimateTokenCount( + JSON.stringify(discoveryTool) + + JSON.stringify(input) + + JSON.stringify(response), + ), + }); + if (input.decoder) { + const matched = nativeHelp + .map((text, i) => ({ line_number: i + 1, text })) + .filter(({ text }) => + text.toLowerCase().includes((input.query ?? "").toLowerCase()), + ); + assert.deepEqual( + report.help_lines, + matched.slice( + input.offset ?? 0, + input.limit === null + ? undefined + : (input.offset ?? 0) + (input.limit ?? 50), + ), + ); + } else if (input.query === "uart") + assert.ok(report.decoders.some(({ id }) => id === "uart")); + const flags = Object.entries(input).flatMap(([key, value]) => + value === null ? ["--all"] : [`--${key}`, String(value)], + ); + const cli = JSON.parse( + ( + await execute( + process.execPath, + [entrypoint, "inspect-signal-decoders", ...flags, "--format", "json"], + { env: environment, timeout: 60000, maxBuffer: 8 * 1024 * 1024 }, + ) + ).stdout, + ); + assert.deepEqual(cli.normalized_result, report); + cases++; + } + const unavailable = await client.callTool({ + name: discoveryTool.name, + arguments: { decoder: "rea_no_such_decoder" }, + }); + assert.equal(unavailable.isError, true); + cases++; await client.close(); assert.deepEqual((await readdir(root.path)).sort(), [ "paged.vcd", @@ -260,6 +359,9 @@ console.log( { status: "passed", cases, + discovery_measurements: discoveryMeasurements, + measurement_scope: + "Warm MCP server, fresh sigrok processes; tokenx estimates include one advertised tool/request/response, not billed model usage. No model-driven selection benchmark, CPU or peak process-family memory measurement; no cache or speedup claim.", host: `${process.platform}/${process.arch}`, scope: "real sigrok UART VCD through compiled CLI/MCP; no physical acquisition", diff --git a/src/application/CapabilityInventory.ts b/src/application/CapabilityInventory.ts index 94518db85..6fefb8e20 100644 --- a/src/application/CapabilityInventory.ts +++ b/src/application/CapabilityInventory.ts @@ -405,7 +405,7 @@ const workflowAvailabilityFor = ({ remediation: "On Linux, supply TShark 4.x via absolute REA_TSHARK_COMMAND and util-linux prlimit (REA_TSHARK_PRLIMIT_COMMAND, default /usr/bin/prlimit). No active binary target or hardware is required; the tool is checked at execution.", }; - if (name === "decode_signal_recording") + if (name === "decode_signal_recording" || name === "inspect_signal_decoders") return policy.signalRecordingEnabled === true ? { reason: "available", remediation: null } : { diff --git a/src/application/signals/SignalRecordingService.ts b/src/application/signals/SignalRecordingService.ts index 921ac19c7..e5b99f1b7 100644 --- a/src/application/signals/SignalRecordingService.ts +++ b/src/application/signals/SignalRecordingService.ts @@ -1,3 +1,9 @@ +import { + inspectSignalDecodersInputSchema, + signalDecodersSchema, + type InspectSignalDecodersInput, +} from "../../domain/signals/signalDecoders.js"; +import type { JsonValue } from "../../domain/jsonValue.js"; import { isAbsolute } from "node:path"; import type { AnalysisExecution, @@ -24,6 +30,10 @@ export interface SignalRecordingPort { input: DecodeSignalRecordingInput, options?: ExecutionOptions, ): Promise>; + inspectDecoders( + input: InspectSignalDecodersInput, + options?: ExecutionOptions, + ): Promise>; close(): Promise; } @@ -74,14 +84,58 @@ export class SignalRecordingService { "Decoder returned malformed data or changed the selected recording identity/options.", ), ); - const evidence = createEvidence(execution.subject, execution.provider, { - operation, - parameters: input.data, - result: report.data, - confidence: "observed", - locations: execution.locations, - limitations: execution.limitations, - }); + return this.#evidence(operation, input.data, execution, options); + } + async inspectDecoders( + raw: unknown, + options?: ExecutionOptions, + ): Promise> { + const operation = "inspect_signal_decoders"; + if (options?.signal?.aborted) + return err(new AnalysisCancelledError(operation)); + const input = inspectSignalDecodersInputSchema.safeParse(raw); + if (!input.success) + return err( + analysisInputErrorFromIssues(operation, input.error.issues, raw, { + cause: input.error, + }), + ); + const decoded = await this.provider.inspectDecoders(input.data, options); + if (!decoded.ok) return decoded; + const execution = decoded.value; + const report = signalDecodersSchema.safeParse(execution.result); + if ( + !report.success || + JSON.stringify(report.data.selection) !== JSON.stringify(input.data) || + execution.subject?.path !== report.data.tool.path || + execution.subject.sha256 !== report.data.tool.sha256 + ) + return err( + new AnalysisOutputError( + operation, + "Decoder discovery returned malformed data or changed the selected installation/options.", + ), + ); + return this.#evidence(operation, input.data, execution, options); + } + #evidence( + operation: string, + parameters: Readonly>, + execution: AnalysisExecution, + options?: ExecutionOptions, + ): Result { + const evidence = createEvidence( + execution.subject ?? undefined, + execution.provider, + { + operation, + parameters, + result: execution.result, + confidence: "observed", + locations: execution.locations, + limitations: execution.limitations, + }, + ); return options?.signal?.aborted ? err( new AnalysisCancelledError(operation, { diff --git a/src/cli/signalCommands.ts b/src/cli/signalCommands.ts index 02e5555f5..e7d254507 100644 --- a/src/cli/signalCommands.ts +++ b/src/cli/signalCommands.ts @@ -1,3 +1,4 @@ +import { inspectSignalDecodersInputSchema } from "../domain/signals/signalDecoders.js"; import { z } from "incur"; import type { SignalRecordingService } from "../application/signals/SignalRecordingService.js"; import { AnalysisError } from "../domain/analysisErrorBase.js"; @@ -22,9 +23,75 @@ export const registerSignalCommands = ( environment: Readonly, service: Pick< SignalRecordingService, - "decode" | "close" + "decode" | "inspectDecoders" | "close" > = createSignalRecordingService(environment), ): void => { + const finish = async ( + result: Awaited>, + operation: string, + ) => { + // A one-shot command must retry retained cleanup before discarding its owner. + try { + await service.close(); + } catch (cause) { + const cleanup = + cause instanceof AnalysisError + ? cause + : new ProviderCleanupError( + "sigrok-recording", + [], + { reason: String(cause) }, + { cause, operation: operation }, + ); + if (result.ok) cleanup.retainPartialObservation(result.value); + return projectAnalysisError( + result.ok + ? cleanup + : analysisErrorWithCleanupFailure(result.error, cleanup, operation), + ); + } + return result.ok ? result.value : projectAnalysisError(result.error); + }; + cli.command(CLI_COMMANDS.inspectSignalDecoders, { + description: "Search installed sigrok decoders or selected decoder help", + options: z + .object({ + ...inspectSignalDecodersInputSchema.shape, + limit: z + .number() + .int() + .positive() + .max(Number.MAX_SAFE_INTEGER) + .optional() + .describe( + "Maximum matching decoders or help lines to return; defaults to 50", + ), + all: z + .boolean() + .default(false) + .describe("Return all remaining matches"), + }) + .superRefine((options, context) => { + if (options.all && options.limit !== undefined) + context.addIssue({ + code: "custom", + path: ["all"], + message: "all and limit are mutually exclusive", + }); + }), + run: ({ options: { all, ...options } }) => + withCommandCancellation((signal) => + logCliCommand(logger, CLI_COMMANDS.inspectSignalDecoders, async () => + finish( + await service.inspectDecoders( + { ...options, ...(all ? { limit: null } : {}) }, + { signal }, + ), + "inspect_signal_decoders", + ), + ), + ), + }); cli.command(CLI_COMMANDS.decodeSignalRecording, { description: "Decode an existing signal recording with caller-supplied sigrok", @@ -68,31 +135,7 @@ export const registerSignalCommands = ( }, { signal }, ); - // A one-shot command must retry retained cleanup before discarding its owner. - try { - await service.close(); - } catch (cause) { - const cleanup = - cause instanceof AnalysisError - ? cause - : new ProviderCleanupError( - "sigrok-recording", - [], - { reason: String(cause) }, - { cause, operation: "decode_signal_recording" }, - ); - if (result.ok) cleanup.retainPartialObservation(result.value); - return projectAnalysisError( - result.ok - ? cleanup - : analysisErrorWithCleanupFailure( - result.error, - cleanup, - "decode_signal_recording", - ), - ); - } - return result.ok ? result.value : projectAnalysisError(result.error); + return finish(result, "decode_signal_recording"); }), ), }); diff --git a/src/cliCommandNames.ts b/src/cliCommandNames.ts index c3d373e4e..d63a9a2bc 100644 --- a/src/cliCommandNames.ts +++ b/src/cliCommandNames.ts @@ -2,6 +2,7 @@ export const CLI_COMMANDS = Object.freeze({ inspectPacketFields: "inspect-packet-fields", inspectPacketCapture: "inspect-packet-capture", + inspectSignalDecoders: "inspect-signal-decoders", decodeSignalRecording: "decode-signal-recording", inspectWasmArtifact: "inspect-wasm-artifact", inspectSqliteDatabase: "inspect-sqlite-database", diff --git a/src/contracts/signals/signalToolContracts.ts b/src/contracts/signals/signalToolContracts.ts index 4a055f12e..f545c452c 100644 --- a/src/contracts/signals/signalToolContracts.ts +++ b/src/contracts/signals/signalToolContracts.ts @@ -1,3 +1,7 @@ +import { + inspectSignalDecodersInputSchema, + signalDecodersSchema, +} from "../../domain/signals/signalDecoders.js"; import { decodeSignalRecordingInputSchema, signalRecordingSchema, @@ -6,6 +10,22 @@ import type { ToolContract } from "../toolContractTypes.js"; import { toolContractMetadata } from "../toolEffects.js"; import { evidenceResultOf } from "../toolOutputSchemaPrimitives.js"; export const SIGNAL_TOOL_CONTRACTS = [ + { + name: "inspect_signal_decoders", + ...toolContractMetadata("inspect_signal_decoders"), + kind: "artifact-provider", + description: + "Discover installed sigrok protocol decoders or inspect one decoder's channel/option help on Linux. Omit decoder to list ID/name pairs; supply its exact ID for literal help lines with source line numbers. query is a case-insensitive literal substring. Page matches with offset/limit (default 50; null returns all remaining). Reports tool identity, version, diagnostics and examined/matching coverage. Uses configured REA_SIGROK_COMMAND and prlimit; no recording, binary target, hardware scan or acquisition. Each call probes again; decoder modules are trusted, not fully fingerprinted. Use returned IDs/options with decode_signal_recording.", + inputSchema: inspectSignalDecodersInputSchema, + outputSchema: evidenceResultOf(signalDecodersSchema), + examples: [ + { title: "Find installed UART decoders", input: { query: "uart" } }, + { + title: "Read the installed UART baud-rate option", + input: { decoder: "uart", query: "baud" }, + }, + ], + }, { name: "decode_signal_recording", ...toolContractMetadata("decode_signal_recording"), diff --git a/src/contracts/toolEffects.ts b/src/contracts/toolEffects.ts index 8ea7f23e0..70a40c6a1 100644 --- a/src/contracts/toolEffects.ts +++ b/src/contracts/toolEffects.ts @@ -69,6 +69,11 @@ export const TOOL_EFFECTS: Readonly> = { launchesProcess: true, writesFilesystem: true, }), + inspect_signal_decoders: effects({ + mutatesSession: true, + launchesProcess: true, + writesFilesystem: true, + }), decode_signal_recording: effects({ mutatesSession: true, launchesProcess: true, diff --git a/src/domain/signals/signalDecoders.ts b/src/domain/signals/signalDecoders.ts new file mode 100644 index 000000000..ae6754cca --- /dev/null +++ b/src/domain/signals/signalDecoders.ts @@ -0,0 +1,86 @@ +import { z } from "zod"; +import { signalRecordingSchema } from "./signalRecording.js"; + +export const inspectSignalDecodersInputSchema = z.strictObject({ + decoder: z + .string() + .regex(/^[a-zA-Z0-9_+\x2d]+$/u) + .max(256) + .nullable() + .default(null) + .describe( + "Exact installed decoder ID for help; null searches the decoder catalog", + ), + query: z + .string() + .max(8192) + .default("") + .describe( + "Case-insensitive literal substring in decoder ID/name or help lines; empty selects all", + ), + offset: z + .number() + .int() + .nonnegative() + .max(Number.MAX_SAFE_INTEGER) + .default(0) + .describe( + "Number of matching decoders or help lines to skip; defaults to 0", + ), + limit: z + .number() + .int() + .positive() + .max(Number.MAX_SAFE_INTEGER) + .nullable() + .default(50) + .describe( + "Maximum matching decoders or help lines returned; null requests all remaining", + ), +}); +export type InspectSignalDecodersInput = z.infer< + typeof inspectSignalDecodersInputSchema +>; +export const signalDecodersSchema = z + .strictObject({ + selection: inspectSignalDecodersInputSchema, + decoders: z.array(z.strictObject({ id: z.string(), name: z.string() })), + help_lines: z.array( + z.strictObject({ + line_number: z.number().int().positive(), + text: z.string(), + }), + ), + output_window: z.strictObject({ + unit: z.enum(["decoders", "help-lines"]), + examined: z.number().int().nonnegative(), + matching: z.number().int().nonnegative(), + returned: z.number().int().nonnegative(), + has_more: z.boolean(), + }), + diagnostics: z.string(), + tool: signalRecordingSchema.shape.tool, + resource_limiter: signalRecordingSchema.shape.resource_limiter, + limits: signalRecordingSchema.shape.limits, + hardware_acquisition: z.literal("not-performed"), + }) + .superRefine((report, context) => { + const w = report.output_window; + const catalog = report.selection.decoder === null; + const remaining = Math.max(0, w.matching - report.selection.offset); + const expected = Math.min(remaining, report.selection.limit ?? remaining); + if ( + w.unit !== (catalog ? "decoders" : "help-lines") || + w.matching > w.examined || + w.returned !== expected || + (catalog ? report.decoders : report.help_lines).length !== expected || + (catalog ? report.help_lines : report.decoders).length !== 0 || + w.has_more !== remaining > expected + ) + context.addIssue({ + code: "custom", + path: ["output_window"], + message: + "Discovery coverage must match the selected page and result kind", + }); + }); diff --git a/src/server/registerSignalTools.ts b/src/server/registerSignalTools.ts index 87a67361f..f833ffbd5 100644 --- a/src/server/registerSignalTools.ts +++ b/src/server/registerSignalTools.ts @@ -28,4 +28,17 @@ export const registerSignalTools = ( toolRegistrationOptions(contract), handler(contract), ); + const discovery = toolContract("inspect_signal_decoders"); + const discover = createEvidenceToolHandler( + server, + (input, options) => service.inspectDecoders(input, options), + logger, + recordEvidence, + withAdmittedAnalysis, + ); + server.registerTool( + discovery.name, + toolRegistrationOptions(discovery), + discover(discovery), + ); }; diff --git a/src/signals/SigrokRecordingProvider.ts b/src/signals/SigrokRecordingProvider.ts index b0eb5c4ba..2e903d4c6 100644 --- a/src/signals/SigrokRecordingProvider.ts +++ b/src/signals/SigrokRecordingProvider.ts @@ -1,3 +1,11 @@ +import { + type InspectSignalDecodersInput, + signalDecodersSchema, +} from "../domain/signals/signalDecoders.js"; +import { + DecoderDiscoveryBudgetError, + parseDecoderDiscovery, +} from "./parseDecoderDiscovery.js"; import { readLinuxResourceLimits, type LinuxResourceLimits, @@ -81,7 +89,7 @@ export class SigrokRecordingProvider implements SignalRecordingPort { readLimits, { message: - "A signal decoding operation is already active; retry after it completes.", + "A sigrok operation is already active; retry after it completes.", details: { max_concurrent_decodes: 1 }, }, ); @@ -91,17 +99,28 @@ export class SigrokRecordingProvider implements SignalRecordingPort { options?: ExecutionOptions, ): Promise> { return this.#resources.run( - () => this.#decode(input, options), + () => this.#executeRequest(input, options), options?.signal, ); } + inspectDecoders( + input: InspectSignalDecodersInput, + options?: ExecutionOptions, + ): Promise> { + return this.#resources.run( + () => this.#executeRequest(input, options), + options?.signal, + "inspect_signal_decoders", + ); + } close(): Promise { return this.#resources.close(); } - async #decode( - input: DecodeSignalRecordingInput, + async #executeRequest( + input: DecodeSignalRecordingInput | InspectSignalDecodersInput, options?: ExecutionOptions, ): Promise> { + const operation = "path" in input ? OPERATION : "inspect_signal_decoders"; let phase: | "configuration" | "artifact" @@ -109,12 +128,12 @@ export class SigrokRecordingProvider implements SignalRecordingPort { | "producer" | "integrity" = "configuration"; let capturedOutput: AnalysisCapturedOutput | undefined; - let resourcePath = input.path; + let resourcePath = "path" in input ? input.path : "sigrok decoder catalog"; let effectiveLimits: LinuxResourceLimits | undefined; let root: ArtifactAnalysisRuntime | undefined; let result: Result; try { - if (options?.signal?.aborted) throw new AnalysisCancelledError(OPERATION); + if (options?.signal?.aborted) throw new AnalysisCancelledError(operation); await this.#resources.release(); if (this.platform !== "linux") throw new Error( @@ -154,7 +173,7 @@ export class SigrokRecordingProvider implements SignalRecordingPort { ]; phase = "workspace"; root = await this.createRuntime(); - this.#resources.retain(root, undefined); + this.#resources.retain(root, undefined, operation); const snapshotPath = join(root.path, "recording"); const execute = async (args: readonly string[]) => { const output = await runOwnedCommand( @@ -197,24 +216,24 @@ export class SigrokRecordingProvider implements SignalRecordingPort { ); if (currentLimiter.sha256 !== limiterIdentity.sha256) throw new AnalysisArtifactChangedError( - OPERATION, + operation, limiter, "Configured resource limiter changed during analysis.", { capturedOutput }, ); if (current.sha256 !== tool.sha256) throw new AnalysisArtifactChangedError( - OPERATION, + operation, command, "Configured decoder executable changed during analysis.", { capturedOutput }, ); phase = previousPhase; - resourcePath = input.path; + resourcePath = "path" in input ? input.path : command; if (output.exitCode !== 0) throw new ProviderAdapterError( SIGROK_PROVIDER_IDENTITY.id, - OPERATION, + operation, { diagnostics: { exit_code: output.exitCode ?? null, @@ -234,65 +253,15 @@ export class SigrokRecordingProvider implements SignalRecordingPort { throw new Error( "Expected sigrok-cli 0.7.2 with libsigrokdecode; inspect the selected installation.", ); - phase = "artifact"; - const selected = await this.#resources.read( - input.path, - LIMITS.input_bytes, - options?.signal, - ); - phase = "workspace"; - await writeFile(snapshotPath, selected.bytes, { - flag: "wx", - mode: 0o400, - }); - phase = "producer"; - const args = [ - "--dont-scan", - "--input-file=recording", - `--protocol-decoders=${input.decoders}`, - "--protocol-decoder-samplenum", - ...(input.input_format === null - ? [] - : [`--input-format=${input.input_format}`]), - ...(input.annotations === null - ? [] - : [`--protocol-decoder-annotations=${input.annotations}`]), - ]; - const output = await execute(args); - phase = "integrity"; - resourcePath = snapshotPath; - if ( - (await this.#resources.read(snapshotPath, LIMITS.input_bytes)) - .sha256 !== selected.sha256 - ) - throw new AnalysisArtifactChangedError( - OPERATION, - snapshotPath, - "Decoder changed the private recording copy.", - capturedOutput === undefined ? {} : { capturedOutput }, - ); - phase = "producer"; - resourcePath = input.path; - const report = signalRecordingSchema.parse({ - artifact: { - path: input.path, - sha256: selected.sha256, - bytes: selected.bytes.length, - }, - selection: input, - ...windowAnnotationText( - output.stdout.text, - input.line_offset, - input.line_limit, - ), - diagnostics: output.stderr.text, + const metadata = (args: readonly string[], diagnostics: string) => ({ + diagnostics, tool: { configured_path: configured, path: command, sha256: tool.sha256, version_banner: version.stdout.text, version_diagnostics: version.stderr.text, - arguments: args, + arguments: [...args], }, resource_limiter: { configured_path: configuredLimiter, @@ -301,31 +270,108 @@ export class SigrokRecordingProvider implements SignalRecordingPort { arguments: limitArguments, }, limits: { ...LIMITS, ...effectiveLimits }, - hardware_acquisition: "not-performed", + hardware_acquisition: "not-performed" as const, }); - result = ok( - createAnalysisExecution(report, SIGROK_PROVIDER_IDENTITY, { - subject: { + if (!("path" in input)) { + phase = "producer"; + const args = + input.decoder === null + ? ["--dont-scan", "--list-supported"] + : ["--dont-scan", `--protocol-decoders=${input.decoder}`, "--show"]; + const output = await execute(args); + const report = signalDecodersSchema.parse({ + selection: input, + ...parseDecoderDiscovery(output.stdout.text, input), + ...metadata(args, output.stderr.text), + }); + result = ok( + createAnalysisExecution(report, SIGROK_PROVIDER_IDENTITY, { + subject: { path: command, sha256: tool.sha256, format: "file" }, + limitations: [ + "Catalog and help are observed from the selected sigrok installation. Help lines preserve producer descriptions, not a separately validated parameter schema.", + "Each request probes the installation again; pages do not share a frozen catalog. Decoder modules and libraries are trusted and not fully fingerprinted.", + "Hardware discovery, recording reads and acquisition are not requested. Linux per-process limits and bounded output apply; this is not an OS sandbox.", + ], + }), + ); + } else { + phase = "artifact"; + const selected = await this.#resources.read( + input.path, + LIMITS.input_bytes, + options?.signal, + ); + phase = "workspace"; + await writeFile(snapshotPath, selected.bytes, { + flag: "wx", + mode: 0o400, + }); + phase = "producer"; + const args = [ + "--dont-scan", + "--input-file=recording", + `--protocol-decoders=${input.decoders}`, + "--protocol-decoder-samplenum", + ...(input.input_format === null + ? [] + : [`--input-format=${input.input_format}`]), + ...(input.annotations === null + ? [] + : [`--protocol-decoder-annotations=${input.annotations}`]), + ]; + const output = await execute(args); + phase = "integrity"; + resourcePath = snapshotPath; + if ( + (await this.#resources.read(snapshotPath, LIMITS.input_bytes)) + .sha256 !== selected.sha256 + ) + throw new AnalysisArtifactChangedError( + operation, + snapshotPath, + "Decoder changed the private recording copy.", + capturedOutput === undefined ? {} : { capturedOutput }, + ); + phase = "producer"; + resourcePath = "path" in input ? input.path : command; + const report = signalRecordingSchema.parse({ + artifact: { path: input.path, sha256: selected.sha256, - format: "file", + bytes: selected.bytes.length, }, - locations: [{ kind: "artifact-path", path: input.path }], - limitations: [ - "Annotations are the selected decoder's interpretation of the recorded samples and caller-selected settings, not independently verified protocol semantics. Empty output does not establish absence of traffic.", - "Sample-number ranges and display text are preserved verbatim, not converted to timestamps or parsed as a lossless byte stream. All producer stderr is retained, including warnings on exit zero.", - "Only a private copy of the selected file is decoded. Hardware discovery/acquisition and target execution are not requested. Caller-supplied tools, plugins and decoder modules are trusted; this is not an OS sandbox.", - "Linux prlimit enforces soft and hard ceilings before execution, preserving stricter inherited limits: at most 1 GiB address space per process, 64 MiB per file, 30 CPU seconds and no core dumps. Effective limits are reported. Resident input is capped at 32 MiB, each command at 30 wall-clock seconds and combined output at 4 MiB. Output overflow fails. These are per-process/per-file limits, not aggregate memory or filesystem quotas; caller-supplied tools and the limiter are trusted.", - ], - }), - ); + selection: input, + ...windowAnnotationText( + output.stdout.text, + input.line_offset, + input.line_limit, + ), + ...metadata(args, output.stderr.text), + }); + result = ok( + createAnalysisExecution(report, SIGROK_PROVIDER_IDENTITY, { + subject: { + path: input.path, + sha256: selected.sha256, + format: "file", + }, + locations: [{ kind: "artifact-path", path: input.path }], + limitations: [ + "Annotations are the selected decoder's interpretation of the recorded samples and caller-selected settings, not independently verified protocol semantics. Empty output does not establish absence of traffic.", + "Sample-number ranges and display text are preserved verbatim, not converted to timestamps or parsed as a lossless byte stream. All producer stderr is retained, including warnings on exit zero.", + "Only a private copy of the selected file is decoded. Hardware discovery/acquisition and target execution are not requested. Caller-supplied tools, plugins and decoder modules are trusted; this is not an OS sandbox.", + "Linux prlimit enforces soft and hard ceilings before execution, preserving stricter inherited limits: at most 1 GiB address space per process, 64 MiB per file, 30 CPU seconds and no core dumps. Effective limits are reported. Resident input is capped at 32 MiB, each command at 30 wall-clock seconds and combined output at 4 MiB. Output overflow fails. These are per-process/per-file limits, not aggregate memory or filesystem quotas; caller-supplied tools and the limiter are trusted.", + ], + }), + ); + } } catch (cause) { if ( root !== undefined && cause instanceof OwnedCommandFailure && cause.cleanupOwner !== undefined ) - this.#resources.retain(root, cause.cleanupOwner); + this.#resources.retain(root, cause.cleanupOwner, operation); result = err( this.#failure( cause, @@ -334,18 +380,22 @@ export class SigrokRecordingProvider implements SignalRecordingPort { root?.path ?? null, effectiveLimits, capturedOutput, + operation, ), ); } - return this.#resources.finish(result, (execution) => - createEvidence(execution.subject ?? undefined, execution.provider, { - operation: OPERATION, - parameters: input, - result: execution.result, - confidence: "observed", - locations: execution.locations, - limitations: execution.limitations, - }), + return this.#resources.finish( + result, + (execution) => + createEvidence(execution.subject ?? undefined, execution.provider, { + operation, + parameters: input, + result: execution.result, + confidence: "observed", + locations: execution.locations, + limitations: execution.limitations, + }), + operation, ); } #failure( @@ -360,6 +410,7 @@ export class SigrokRecordingProvider implements SignalRecordingPort { workspacePath: string | null, effectiveLimits: LinuxResourceLimits | undefined, observedOutput: AnalysisCapturedOutput | undefined, + operation = OPERATION, ): AnalysisError { if (cause instanceof AnalysisError) return cause; if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined) @@ -367,15 +418,34 @@ export class SigrokRecordingProvider implements SignalRecordingPort { cause.cleanup, cause, observedOutput, + operation, ); if ( (cause instanceof Error && cause.name === "AbortError") || (cause instanceof ArtifactReaderFailure && cause.reason === "cancelled") ) - return new AnalysisCancelledError(OPERATION, { cause }); + return new AnalysisCancelledError(operation, { cause }); + if (cause instanceof DecoderDiscoveryBudgetError) + return new AnalysisResourceConstraintError( + operation, + "memory", + cause.message, + { line_budget: 65536 }, + { + cause, + ...(observedOutput === undefined + ? {} + : { capturedOutput: observedOutput }), + }, + ); const reason = cause instanceof Error ? cause.message : String(cause); if (cause instanceof OwnedCommandFailure) - return this.#resources.commandFailure(cause, LIMITS, effectiveLimits); + return this.#resources.commandFailure( + cause, + LIMITS, + effectiveLimits, + operation, + ); if (phase === "workspace") { const code = cause instanceof Error && @@ -385,13 +455,13 @@ export class SigrokRecordingProvider implements SignalRecordingPort { : null; if (code === "ENOSPC" || code === "EDQUOT" || code === "EFBIG") return new AnalysisResourceConstraintError( - OPERATION, + operation, "file-size", `Private analysis workspace failed (${code}): ${reason}`, { workspace_path: workspacePath, system_code: code }, { cause }, ); - return new ProviderAdapterError(SIGROK_PROVIDER_IDENTITY.id, OPERATION, { + return new ProviderAdapterError(SIGROK_PROVIDER_IDENTITY.id, operation, { cause, userMessage: `Cannot prepare the private analysis workspace: ${reason}`, diagnostics: { @@ -414,12 +484,12 @@ export class SigrokRecordingProvider implements SignalRecordingPort { ["limit", "integrity", "path"].includes(cause.reason) ) return new AnalysisArtifactChangedError( - OPERATION, + operation, path, `Post-run identity verification failed for ${path}: ${reason}`, detail, ); - return new ProviderAdapterError(SIGROK_PROVIDER_IDENTITY.id, OPERATION, { + return new ProviderAdapterError(SIGROK_PROVIDER_IDENTITY.id, operation, { ...detail, diagnostics: { stage: "integrity", resource_path: path, reason }, }); @@ -427,10 +497,13 @@ export class SigrokRecordingProvider implements SignalRecordingPort { if (phase === "configuration") return new AnalysisCapabilityUnavailableError( SIGROK_PROVIDER_IDENTITY.id, - OPERATION, + operation, reason, { cause, + ...(observedOutput === undefined + ? {} + : { capturedOutput: observedOutput }), userMessage: `${reason} On Linux, configure sigrok-cli 0.7.2 through REA_SIGROK_COMMAND and util-linux prlimit through REA_SIGROK_PRLIMIT_COMMAND (default /usr/bin/prlimit).`, }, ); @@ -440,14 +513,14 @@ export class SigrokRecordingProvider implements SignalRecordingPort { cause.reason === "limit" ) return new AnalysisResourceConstraintError( - OPERATION, + operation, "file-size", reason, { input_bytes: LIMITS.input_bytes }, { cause }, ); if (cause instanceof ArtifactReaderFailure && cause.reason === "integrity") - return new AnalysisArtifactChangedError(OPERATION, path, reason, { + return new AnalysisArtifactChangedError(operation, path, reason, { cause, }); if ( @@ -456,13 +529,18 @@ export class SigrokRecordingProvider implements SignalRecordingPort { "code" in cause && (cause.code === "EACCES" || cause.code === "EPERM") ) - return new AnalysisAccessDeniedError(OPERATION, path, cause.code, { + return new AnalysisAccessDeniedError(operation, path, cause.code, { cause, }); if (phase === "artifact") - return new AnalysisInputError(OPERATION, { cause }, [ + return new AnalysisInputError(operation, { cause }, [ { path: ["path"], reason: "invalid_value", message: reason }, ]); - return new AnalysisOutputError(OPERATION, reason, { cause }); + return new AnalysisOutputError(operation, reason, { + cause, + ...(observedOutput === undefined + ? {} + : { capturedOutput: observedOutput }), + }); } } diff --git a/src/signals/parseDecoderDiscovery.ts b/src/signals/parseDecoderDiscovery.ts new file mode 100644 index 000000000..230abe2b0 --- /dev/null +++ b/src/signals/parseDecoderDiscovery.ts @@ -0,0 +1,79 @@ +import type { InspectSignalDecodersInput } from "../domain/signals/signalDecoders.js"; + +export class DecoderDiscoveryBudgetError extends Error {} + +const copyText = (value: string): string => + Buffer.from(value, "utf16le").toString("utf16le"); + +/** Parse only the 0.7.2 catalog grammar; help remains literal producer text. */ +export const parseDecoderDiscovery = ( + text: string, + input: InspectSignalDecodersInput, +) => { + // Bound record-object expansion before splitting the already byte-bounded output. + // 65,536 records leaves an 8 MiB budget at 128 bytes of bookkeeping per record. + let separators = 0; + for (const character of text) + if (character === "\n" && ++separators >= 65536) + throw new DecoderDiscoveryBudgetError( + "Decoder discovery exceeds the 65,536-line representation budget", + ); + const lines = text.split(/\r?\n/u); + if (lines.at(-1) === "") lines.pop(); + const query = input.query.toLowerCase(); + const decoders: { id: string; name: string }[] = []; + const help: { line_number: number; text: string }[] = []; + if (input.decoder === null) { + const start = lines.indexOf("Supported protocol decoders:"); + if (start < 0) throw new Error("sigrok decoder catalog heading is missing"); + const seen = new Set(); + for (const line of lines.slice(start + 1)) { + if (line === "") continue; + const match = /^ {2}([a-zA-Z0-9_+-]+) +(.+)$/u.exec(line); + if ( + match?.[1] === undefined || + match[2] === undefined || + seen.has(match[1]) + ) + throw new Error("Malformed or duplicate sigrok decoder catalog record"); + seen.add(match[1]); + decoders.push({ id: match[1], name: match[2] }); + } + if (decoders.length === 0) + throw new Error("sigrok returned an empty decoder catalog"); + } else { + if (lines[0] !== `ID: ${input.decoder}`) + throw new Error("Decoder help does not identify the selected decoder"); + lines.forEach((line, i) => help.push({ line_number: i + 1, text: line })); + } + const catalog = input.decoder === null; + const matching = catalog + ? decoders.filter(({ id, name }) => + `${id} ${name}`.toLowerCase().includes(query), + ) + : help.filter(({ text: line }) => line.toLowerCase().includes(query)); + const start = Math.min(input.offset, matching.length); + const count = Math.min( + input.limit ?? matching.length, + matching.length - start, + ); + // Keep retained pages independent of the complete producer-output allocation. + const page = matching + .slice(start, start + count) + .map((record) => + "id" in record + ? { id: copyText(record.id), name: copyText(record.name) } + : { line_number: record.line_number, text: copyText(record.text) }, + ); + return { + decoders: catalog ? page : [], + help_lines: catalog ? [] : page, + output_window: { + unit: catalog ? "decoders" : "help-lines", + examined: catalog ? decoders.length : help.length, + matching: matching.length, + returned: page.length, + has_more: start + count < matching.length, + }, + }; +}; diff --git a/tests/boundary/cli/signalCleanup.test.ts b/tests/boundary/cli/signalCleanup.test.ts index 594a46cc0..dc706c17c 100644 --- a/tests/boundary/cli/signalCleanup.test.ts +++ b/tests/boundary/cli/signalCleanup.test.ts @@ -30,6 +30,9 @@ it.each([ silentLogger, {}, { + inspectDecoders: async () => { + throw new Error("Unexpected discovery call"); + }, decode: async () => { called = true; throw new Error("Must not decode"); @@ -84,6 +87,9 @@ it.each([false, true])( silentLogger, {}, { + inspectDecoders: async () => { + throw new Error("Unexpected discovery call"); + }, decode: async () => err(original), close: async () => { await Promise.resolve(); diff --git a/tests/boundary/mcp/signalRecordingMcp.test.ts b/tests/boundary/mcp/signalRecordingMcp.test.ts index ecabfd8e5..edd74a2bf 100644 --- a/tests/boundary/mcp/signalRecordingMcp.test.ts +++ b/tests/boundary/mcp/signalRecordingMcp.test.ts @@ -8,41 +8,51 @@ import { Ajv2020 } from "ajv/dist/2020.js"; import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; -it("advertises valid target-free signal decoding schemas and actionable missing-tool diagnostics", async () => { - const harness = await createApplicationMcpHarness({ - serverOptions: { environment: {} }, - }); - onTestFinished(harness.close); - const tool = (await harness.client.listTools()).tools.find( - ({ name }) => name === "decode_signal_recording", - ); - if (tool?.outputSchema === undefined) throw new Error("Signal tool missing"); - const ajv = new Ajv2020({ strict: false, validateFormats: false }); - const inputSchema: Record = tool.inputSchema; - const outputSchema: Record = tool.outputSchema; - expect(ajv.validateSchema(inputSchema)).toBe(true); - expect(ajv.validateSchema(outputSchema)).toBe(true); - const input = { - path: process.platform === "win32" ? "C:\\recording.vcd" : "/recording.vcd", - decoders: "uart", - }; - expect(ajv.validate(inputSchema, input)).toBe(true); - expect(ajv.validate(inputSchema, { ...input, driver: "demo" })).toBe(false); - const result = await harness.client.callTool({ - name: tool.name, - arguments: input, - }); - expect(parseMcpToolError(result)).toMatchObject({ - error: { - code: "capability_unavailable", - details: { - operation: tool.name, - reason: expect.stringContaining("REA_SIGROK_COMMAND"), +it.each(["decode_signal_recording", "inspect_signal_decoders"])( + "advertises valid target-free schemas and missing-tool diagnostics: %s", + async (name) => { + const harness = await createApplicationMcpHarness({ + serverOptions: { environment: {} }, + }); + onTestFinished(harness.close); + const tool = (await harness.client.listTools()).tools.find( + (tool) => tool.name === name, + ); + if (tool?.outputSchema === undefined) + throw new Error("Signal tool missing"); + const ajv = new Ajv2020({ strict: false, validateFormats: false }); + const inputSchema: Record = tool.inputSchema; + const outputSchema: Record = tool.outputSchema; + expect(ajv.validateSchema(inputSchema)).toBe(true); + expect(ajv.validateSchema(outputSchema)).toBe(true); + const input = + name === "inspect_signal_decoders" + ? { query: "uart" } + : { + path: + process.platform === "win32" + ? "C:\\recording.vcd" + : "/recording.vcd", + decoders: "uart", + }; + expect(ajv.validate(inputSchema, input)).toBe(true); + expect(ajv.validate(inputSchema, { ...input, driver: "demo" })).toBe(false); + const result = await harness.client.callTool({ + name: tool.name, + arguments: input, + }); + expect(parseMcpToolError(result)).toMatchObject({ + error: { + code: "capability_unavailable", + details: { + operation: tool.name, + reason: expect.stringContaining("REA_SIGROK_COMMAND"), + }, }, - }, - }); - await harness.client.ping(); -}); + }); + await harness.client.ping(); + }, +); it.each(["cancelled", "cleanup"])( "retains completed Evidence after %s failure", @@ -77,6 +87,9 @@ it.each(["cancelled", "cleanup"])( }, ); const signalRecording = new SignalRecordingService({ + inspectDecoders: async () => { + throw new Error("Unexpected discovery call"); + }, decode: async () => err(failure), close: async () => {}, }); diff --git a/tests/boundary/signals/decoderDiscovery.test.ts b/tests/boundary/signals/decoderDiscovery.test.ts new file mode 100644 index 000000000..6b69d679f --- /dev/null +++ b/tests/boundary/signals/decoderDiscovery.test.ts @@ -0,0 +1,48 @@ +import { expect, it } from "vitest"; +import { inspectSignalDecodersInputSchema } from "../../../src/domain/signals/signalDecoders.js"; +import { parseDecoderDiscovery } from "../../../src/signals/parseDecoderDiscovery.js"; + +it("rejects malformed catalogs and the wrong decoder identity", () => { + for (const text of [ + "", + "Supported protocol decoders:\n", + "Supported protocol decoders:\n uart UART\n uart Duplicate", + "Supported protocol decoders:\n malformed", + ]) { + expect(() => + parseDecoderDiscovery(text, inspectSignalDecodersInputSchema.parse({})), + ).toThrow(); + } + expect(() => + parseDecoderDiscovery( + "ID: spi\nOptions:", + inspectSignalDecodersInputSchema.parse({ decoder: "uart" }), + ), + ).toThrow(); +}); + +it("keeps help line ordinals, blank lines and literal special characters through selection", () => { + const text = "ID: uart\r\nOptions:\r\n\r\n- mode: A[0] (default 'x')\r\n"; + const result = parseDecoderDiscovery( + text, + inspectSignalDecodersInputSchema.parse({ + decoder: "uart", + query: "[0]", + offset: 0, + limit: 1, + }), + ); + expect(result.help_lines).toEqual([ + { line_number: 4, text: "- mode: A[0] (default 'x')" }, + ]); + expect(result.output_window.examined).toBe(4); +}); + +it("bounds line expansion before constructing help records", () => { + expect(() => + parseDecoderDiscovery( + "ID: uart\n" + "\n".repeat(65536), + inspectSignalDecodersInputSchema.parse({ decoder: "uart" }), + ), + ).toThrow("representation budget"); +}); diff --git a/tests/boundary/signals/recordingProvider.test.ts b/tests/boundary/signals/recordingProvider.test.ts index 9ddf2c21a..27bc6ff09 100644 --- a/tests/boundary/signals/recordingProvider.test.ts +++ b/tests/boundary/signals/recordingProvider.test.ts @@ -1,3 +1,4 @@ +import { signalDecodersSchema } from "../../../src/domain/signals/signalDecoders.js"; import { OwnedCommandFailure } from "../../../src/process/OwnedCommand.js"; import { it, expect, onTestFinished } from "vitest"; import { writeFile, readdir, chmod, symlink } from "node:fs/promises"; @@ -23,13 +24,17 @@ const fixture = async (mode = "ok", failCleanup = false) => { await writeFile( script, `import {readFileSync, writeFileSync, chmodSync, truncateSync} from 'node:fs'; -if (process.argv.includes('--version')) { console.log('sigrok-cli 0.7.2\\nlibsigrokdecode 0.5.3'); console.error('version warning'); } +if (process.argv.includes('--version')) { console.log(${JSON.stringify(mode)} === 'bad-version' ? 'sigrok-cli 0.7.1' : 'sigrok-cli 0.7.2\\nlibsigrokdecode 0.5.3'); console.error('version warning'); } +else if (${JSON.stringify(mode)} === 'bad-version') console.log('unexpected version fallback'); else if (${JSON.stringify(mode)} === 'nonzero') { console.error('invalid decoder'); process.exitCode = 1; } else if (${JSON.stringify(mode)} === 'malformed-copy') { chmodSync('recording', 0o600); writeFileSync('recording', 'changed'); } else if (${JSON.stringify(mode)} === 'oversized-copy') { chmodSync('recording', 0o600); truncateSync('recording', 32 * 1024 * 1024 + 1); } +else if (${JSON.stringify(mode)} === 'line-budget') process.stdout.write('ID: uart' + '\\n'.repeat(65536)); else if (${JSON.stringify(mode)} === 'overflow') process.stdout.write('x'.repeat(5 * 1024 * 1024)); else if (${JSON.stringify(mode)} === 'wait') { console.log('ready'); setInterval(() => {}, 1000); } else if (${JSON.stringify(mode)} === 'many') process.stdout.write(Array.from({length:90},(_,i)=>'line '+(i+1)+'\\r\\n').join('')); +else if (process.argv.includes('--list-supported')) { console.log('Supported hardware drivers:\\n ignored Device\\nSupported protocol decoders:\\n uart Universal Asynchronous Receiver/Transmitter\\n i2c Inter-Integrated Circuit\\n'); console.error('catalog warning'); } +else if (process.argv.includes('--show')) console.log('ID: uart\\nOptions:\\n- baudrate: Baud rate (default 115200)\\n- parity: Parity'); else { if(readFileSync('recording', 'utf8') !== 'recording bytes') throw Error('wrong copy'); console.log('10-20 uart-1: 42'); console.error('producer warning'); } `, ); @@ -285,6 +290,9 @@ it("retains a completed observation when cancellation arrives at the service bou const execution = await f.service.provider.decode(input); if (!execution.ok) throw execution.error; const service = new SignalRecordingService({ + inspectDecoders: async () => { + throw new Error("Unexpected discovery call"); + }, decode: async () => { f.controller.abort(); return execution; @@ -419,3 +427,182 @@ it("preserves termination signal and effective file ceiling as a resource failur capturedOutput: { signal: "SIGXFSZ", stderr: "before file limit\n" }, }); }); + +it("discovers decoders without reading a recording and retains installation provenance", async () => { + const f = await fixture(); + for (const [input, expected] of [ + [ + { query: "UART" }, + { + decoders: [ + { id: "uart", name: "Universal Asynchronous Receiver/Transmitter" }, + ], + output_window: { + examined: 2, + matching: 1, + returned: 1, + has_more: false, + }, + }, + ], + [ + { limit: 1 }, + { + decoders: [ + { id: "uart", name: "Universal Asynchronous Receiver/Transmitter" }, + ], + output_window: { matching: 2, returned: 1, has_more: true }, + }, + ], + [ + { offset: 1, limit: null }, + { + decoders: [{ id: "i2c", name: "Inter-Integrated Circuit" }], + output_window: { returned: 1, has_more: false }, + }, + ], + [ + { decoder: "uart", query: "baud" }, + { + help_lines: [ + { line_number: 3, text: "- baudrate: Baud rate (default 115200)" }, + ], + output_window: { unit: "help-lines", examined: 4, matching: 1 }, + }, + ], + [ + { query: "no-such-decoder" }, + { + decoders: [], + output_window: { examined: 2, matching: 0, returned: 0 }, + }, + ], + ] as const) { + const result = await f.service.inspectDecoders(input); + if (!result.ok) throw result.error; + const report = signalDecodersSchema.parse(result.value.normalized_result); + expect(report).toMatchObject(expected); + expect(result.value.operation).toBe("inspect_signal_decoders"); + expect(report.tool.version_diagnostics).toBe("version warning\n"); + expect(report.tool.arguments).toContain("--dont-scan"); + expect( + report.tool.arguments.some((arg) => arg.startsWith("--input-file")), + ).toBe(false); + } + expect( + (await readdir(f.directory)).filter((name) => name.startsWith("decoder-")), + ).toEqual([]); +}); + +it("attributes discovery admission failures to discovery while sharing the decoding slot", async () => { + const f = await fixture("gated"); + const active = f.service.decode({ path: f.path, decoders: "uart" }); + await f.acquired; + const busy = await f.service.inspectDecoders({}); + f.release(); + if (!(await active).ok) throw new Error("Decode failed"); + expect(busy).toMatchObject({ + ok: false, + error: { operation: "inspect_signal_decoders", resource: "capacity" }, + }); + await f.service.close(); + expect(await f.service.inspectDecoders({})).toMatchObject({ + ok: false, + error: { + operation: "inspect_signal_decoders", + _tag: "AnalysisCancelledError", + }, + }); +}); + +it.each([ + "bad-version", + "nonzero", + "overflow", + "line-budget", + "wait", + "oversized-tool", +])("preserves discovery failure diagnostics: %s", async (mode) => { + const f = await fixture(mode); + const result = await f.service.inspectDecoders( + {}, + { signal: f.controller.signal }, + ); + expect(result.ok).toBe(false); + if (result.ok) throw new Error("Expected failure"); + expect(result.error).toMatchObject({ + operation: "inspect_signal_decoders", + }); + if (mode === "line-budget") { + expect(result.error).toMatchObject({ + _tag: "AnalysisResourceConstraintError", + }); + expect(result.error.capturedOutput?.stdout).toBe( + "ID: uart" + "\n".repeat(65536), + ); + } + if (mode === "bad-version") + expect(result.error.capturedOutput).toMatchObject({ + stdout: "sigrok-cli 0.7.1\n", + stderr: "version warning\n", + }); +}); + +it("retains completed discovery when cleanup fails, then retries before accepting work", async () => { + const f = await fixture("ok", true); + const first = await f.service.inspectDecoders({ query: "uart" }); + expect(first).toMatchObject({ + ok: false, + error: { + operation: "inspect_signal_decoders", + partialObservation: { operation: "inspect_signal_decoders" }, + }, + }); + await f.service.inspectDecoders({}); + await f.service.decode({ path: f.path, decoders: "uart" }); + expect(f.calls()).toBe(2); + for (let attempt = 0; attempt < 2; attempt++) + await expect(f.service.close()).rejects.toMatchObject({ + operation: "inspect_signal_decoders", + cleanupIncomplete: true, + }); + f.allowCleanup(); + await f.service.close(); +}); + +it("rejects ambiguous decoder selectors before launch and retains malformed help output", async () => { + const f = await fixture(); + for (const decoder of ["uart:baudrate=1", "uart,spi", "uart\0"]) { + expect(await f.service.inspectDecoders({ decoder })).toMatchObject({ + ok: false, + error: { _tag: "AnalysisInputError" }, + }); + } + expect(f.calls()).toBe(0); + expect(await f.service.inspectDecoders({ decoder: "spi" })).toMatchObject({ + ok: false, + error: { + _tag: "AnalysisOutputError", + capturedOutput: { stdout: expect.stringContaining("ID: uart") }, + }, + }); +}); + +it("retains completed discovery on cancellation during identity verification", async () => { + const f = await fixture("post-run-abort"); + expect( + await f.service.inspectDecoders( + { query: "uart" }, + { signal: f.controller.signal }, + ), + ).toMatchObject({ + ok: false, + error: { + _tag: "AnalysisCancelledError", + partialObservation: { + operation: "inspect_signal_decoders", + normalized_result: { decoders: [{ id: "uart" }] }, + }, + }, + }); +}); From b953f96c0aae27e1876c317a8f7db1d6f292b8ff Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:00:14 +0800 Subject: [PATCH 059/159] fix(ghidra): share Java properties semantics with PSP extension admission --- src/ghidra/GhidraInstallationPolicy.ts | 116 +----------------- src/ghidra/GhidraJavaProperties.ts | 115 +++++++++++++++++ src/ghidra/GhidraPspExtension.ts | 23 ++-- .../filesystem/ghidraPspExtension.test.ts | 37 +++++- 4 files changed, 161 insertions(+), 130 deletions(-) create mode 100644 src/ghidra/GhidraJavaProperties.ts diff --git a/src/ghidra/GhidraInstallationPolicy.ts b/src/ghidra/GhidraInstallationPolicy.ts index ead513cd8..8a35c66c3 100644 --- a/src/ghidra/GhidraInstallationPolicy.ts +++ b/src/ghidra/GhidraInstallationPolicy.ts @@ -1,4 +1,4 @@ -import { err, ok, type Result } from "../domain/result.js"; +import { parseGhidraJavaProperties } from "./GhidraJavaProperties.js"; /** Ghidra build whose bridge behavior this REA release verifies. */ export const SUPPORTED_GHIDRA_VERSION = "12.1.4"; @@ -61,7 +61,7 @@ const GHIDRA_RELEASE_PATTERN = /^(\d+)\.(\d+)(?:\.(\d+))?([-+].*)?$/u; export const ghidraApplicationProperties = ( content: string, ): GhidraApplicationProperties => { - const parsed = parseJavaProperties(content); + const parsed = parseGhidraJavaProperties(content); if (!parsed.ok) return { providerVersion: null, @@ -274,115 +274,3 @@ const unresolvedJavaRequirement = ( }); /** Parse Java Properties.load(InputStream) syntax from its ISO-8859-1 text. */ -const parseJavaProperties = ( - content: string, -): Result, string> => { - const values = new Map(); - for (const line of logicalPropertyLines(content)) { - let start = 0; - while (isPropertyWhitespace(line[start])) start += 1; - if (start === line.length || line[start] === "#" || line[start] === "!") - continue; - let end = start; - let escaped = false; - while (end < line.length) { - const character = line[end]; - if ( - !escaped && - (character === "=" || - character === ":" || - isPropertyWhitespace(character)) - ) - break; - if (character === "\\") escaped = !escaped; - else escaped = false; - end += 1; - } - let valueStart = end; - while (isPropertyWhitespace(line[valueStart])) valueStart += 1; - if (line[valueStart] === "=" || line[valueStart] === ":") valueStart += 1; - while (isPropertyWhitespace(line[valueStart])) valueStart += 1; - const key = decodePropertyEscapes(line.slice(start, end)); - if (!key.ok) return err(key.error); - const value = decodePropertyEscapes(line.slice(valueStart)); - if (!value.ok) return err(value.error); - values.set(key.value, value.value); - } - return ok(values); -}; - -const isPropertyWhitespace = (value: string | undefined): boolean => - value === " " || value === "\t" || value === "\f"; - -/** Join odd-backslash continuations while stripping leading continuation space. */ -function* logicalPropertyLines(content: string): Generator { - let physicalStart = 0; - let logical = ""; - let continuing = false; - while (physicalStart <= content.length) { - let physicalEnd = physicalStart; - while ( - physicalEnd < content.length && - content[physicalEnd] !== "\n" && - content[physicalEnd] !== "\r" - ) - physicalEnd += 1; - let line = content.slice(physicalStart, physicalEnd); - let leading = 0; - while (isPropertyWhitespace(line[leading])) leading += 1; - const comment = - !continuing && (line[leading] === "#" || line[leading] === "!"); - if (continuing) line = line.slice(leading); - let continued = false; - if (!comment) { - let slashes = 0; - for ( - let index = line.length - 1; - index >= 0 && line[index] === "\\"; - index -= 1 - ) - slashes += 1; - continued = slashes % 2 === 1; - logical += continued ? line.slice(0, -1) : line; - } - if (!continued) { - yield logical; - logical = ""; - } - continuing = continued; - if (physicalEnd === content.length) break; - physicalStart = - content[physicalEnd] === "\r" && content[physicalEnd + 1] === "\n" - ? physicalEnd + 2 - : physicalEnd + 1; - } - // Properties.load still consumes the accumulated line when EOF follows a - // continuation marker; the marker itself was already removed above. - if (continuing) yield logical; -} - -const decodePropertyEscapes = (value: string): Result => { - let result = ""; - for (let index = 0; index < value.length; index += 1) { - const character = value[index]; - if (character !== "\\") { - result += character; - continue; - } - const escaped = value[++index]; - if (escaped === undefined) - return err("Trailing escape in Java properties value"); - if (escaped === "t") result += "\t"; - else if (escaped === "n") result += "\n"; - else if (escaped === "r") result += "\r"; - else if (escaped === "f") result += "\f"; - else if (escaped === "u") { - const digits = value.slice(index + 1, index + 5); - if (digits.length !== 4 || !/^[0-9a-f]{4}$/iu.test(digits)) - return err("Malformed Unicode escape in Java properties file"); - result += String.fromCharCode(Number.parseInt(digits, 16)); - index += 4; - } else result += escaped; - } - return ok(result); -}; diff --git a/src/ghidra/GhidraJavaProperties.ts b/src/ghidra/GhidraJavaProperties.ts new file mode 100644 index 000000000..ab3b7bead --- /dev/null +++ b/src/ghidra/GhidraJavaProperties.ts @@ -0,0 +1,115 @@ +import { err, ok, type Result } from "../domain/result.js"; + +/** Parse Java Properties.load(InputStream) syntax from its ISO-8859-1 text. */ +export const parseGhidraJavaProperties = ( + content: string, +): Result, string> => { + const values = new Map(); + for (const line of logicalPropertyLines(content)) { + let start = 0; + while (isPropertyWhitespace(line[start])) start += 1; + if (start === line.length || line[start] === "#" || line[start] === "!") + continue; + let end = start; + let escaped = false; + while (end < line.length) { + const character = line[end]; + if ( + !escaped && + (character === "=" || + character === ":" || + isPropertyWhitespace(character)) + ) + break; + if (character === "\\") escaped = !escaped; + else escaped = false; + end += 1; + } + let valueStart = end; + while (isPropertyWhitespace(line[valueStart])) valueStart += 1; + if (line[valueStart] === "=" || line[valueStart] === ":") valueStart += 1; + while (isPropertyWhitespace(line[valueStart])) valueStart += 1; + const key = decodePropertyEscapes(line.slice(start, end)); + if (!key.ok) return err(key.error); + const value = decodePropertyEscapes(line.slice(valueStart)); + if (!value.ok) return err(value.error); + values.set(key.value, value.value); + } + return ok(values); +}; + +const isPropertyWhitespace = (value: string | undefined): boolean => + value === " " || value === "\t" || value === "\f"; + +/** Join odd-backslash continuations while stripping leading continuation space. */ +function* logicalPropertyLines(content: string): Generator { + let physicalStart = 0; + let logical = ""; + let continuing = false; + while (physicalStart <= content.length) { + let physicalEnd = physicalStart; + while ( + physicalEnd < content.length && + content[physicalEnd] !== "\n" && + content[physicalEnd] !== "\r" + ) + physicalEnd += 1; + let line = content.slice(physicalStart, physicalEnd); + let leading = 0; + while (isPropertyWhitespace(line[leading])) leading += 1; + const comment = + !continuing && (line[leading] === "#" || line[leading] === "!"); + if (continuing) line = line.slice(leading); + let continued = false; + if (!comment) { + let slashes = 0; + for ( + let index = line.length - 1; + index >= 0 && line[index] === "\\"; + index -= 1 + ) + slashes += 1; + continued = slashes % 2 === 1; + logical += continued ? line.slice(0, -1) : line; + } + if (!continued) { + yield logical; + logical = ""; + } + continuing = continued; + if (physicalEnd === content.length) break; + physicalStart = + content[physicalEnd] === "\r" && content[physicalEnd + 1] === "\n" + ? physicalEnd + 2 + : physicalEnd + 1; + } + // Properties.load still consumes the accumulated line when EOF follows a + // continuation marker; the marker itself was already removed above. + if (continuing) yield logical; +} + +const decodePropertyEscapes = (value: string): Result => { + let result = ""; + for (let index = 0; index < value.length; index += 1) { + const character = value[index]; + if (character !== "\\") { + result += character; + continue; + } + const escaped = value[++index]; + if (escaped === undefined) + return err("Trailing escape in Java properties value"); + if (escaped === "t") result += "\t"; + else if (escaped === "n") result += "\n"; + else if (escaped === "r") result += "\r"; + else if (escaped === "f") result += "\f"; + else if (escaped === "u") { + const digits = value.slice(index + 1, index + 5); + if (digits.length !== 4 || !/^[0-9a-f]{4}$/iu.test(digits)) + return err("Malformed Unicode escape in Java properties file"); + result += String.fromCharCode(Number.parseInt(digits, 16)); + index += 4; + } else result += escaped; + } + return ok(result); +}; diff --git a/src/ghidra/GhidraPspExtension.ts b/src/ghidra/GhidraPspExtension.ts index 1a3769378..9c017f367 100644 --- a/src/ghidra/GhidraPspExtension.ts +++ b/src/ghidra/GhidraPspExtension.ts @@ -13,6 +13,7 @@ import { import type { AnalysisError } from "../domain/analysisErrorBase.js"; import { err, ok, type Result } from "../domain/result.js"; import type { GhidraInstallationInspection } from "./GhidraInstallation.js"; +import { parseGhidraJavaProperties } from "./GhidraJavaProperties.js"; /** Installed processor code and data, distinct from REA's analysis-extension ABI. */ export const ghidraPspExtensionSchema = z.strictObject({ @@ -88,7 +89,7 @@ export const inspectGhidraPspExtension = async ( throw new Error("The installed Allegrex extension has no loader JAR."); const hash = createHash("sha256"); let bytesRead = 0; - let properties = ""; + let properties = Buffer.alloc(0); for (const relative of files) { signal?.throwIfAborted(); const path = join(root, relative); @@ -135,22 +136,22 @@ export const inspectGhidraPspExtension = async ( ); bytesRead += before.size; if (relative === "extension.properties") - properties = Buffer.concat(chunks).toString("utf8"); + properties = Buffer.concat(chunks); } finally { await handle.close(); } } // Ghidra's extension.properties version is the compatible Ghidra version, // not the extension release (e.g. v21.4); content identity supplies the latter. - const property = (name: string): string => { - const values = properties.split(/\r?\n/u).flatMap((line) => { - const match = /^\s*([A-Za-z][A-Za-z0-9]*)\s*=\s*(.*?)\s*$/u.exec(line); - return match?.[1] === name ? [match[2] ?? ""] : []; - }); - if (values.length !== 1) - throw new Error(`Allegrex extension must declare exactly one ${name}.`); - return values[0] ?? ""; - }; + const parsedProperties = parseGhidraJavaProperties( + properties.toString("latin1"), + ); + if (!parsedProperties.ok) + throw new Error( + `Invalid Allegrex extension properties: ${parsedProperties.error}`, + ); + const property = (name: string): string | undefined => + parsedProperties.value.get(name); if ( property("name") !== "ghidra-allegrex" || property("version") !== ghidraVersion diff --git a/tests/boundary/filesystem/ghidraPspExtension.test.ts b/tests/boundary/filesystem/ghidraPspExtension.test.ts index 0e07ad629..0b90bd194 100644 --- a/tests/boundary/filesystem/ghidraPspExtension.test.ts +++ b/tests/boundary/filesystem/ghidraPspExtension.test.ts @@ -188,15 +188,42 @@ describe.skipIf(process.platform === "win32")( expect(missing.error.message).toContain("PSP_EXTENSION_UNAVAILABLE"); }); + it.each([ + { + form: "escaped key and value with colon separator", + properties: "n\\u0061me : ghidra-allegrex\nver\\u0073ion=12.1.3\n", + }, + { + form: "continued key and value lines", + properties: "name=ghidra-\\\n allegrex\nversion=12.1.\\\n 3\n", + }, + { + form: "duplicate keys use the last value", + properties: + "name=wrong\nname=ghidra-allegrex\nversion=12.1.4\nversion=12.1.3\n", + }, + ])( + "accepts Java Properties $form and hashes raw bytes", + async ({ properties }) => { + const { root } = await installation(); + const canonical = await inspectGhidraPspExtension(root, "12.1.3"); + const latin1Properties = Buffer.from( + `${properties}author=caf\u00e9\n`, + "latin1", + ); + await writeFile(join(root, "extension.properties"), latin1Properties); + const extension = await inspectGhidraPspExtension(root, "12.1.3"); + expect(extension.id).toBe("ghidra-allegrex"); + expect(extension.sha256).not.toBe(canonical.sha256); + }, + ); + it.each([ ["name=other\nversion=12.1.3\n", "does not match"], ["name=ghidra-allegrex\nversion=12.1.4\n", "does not match"], - [ - "name=ghidra-allegrex\nversion=12.1.3\nversion=12.1.3\n", - "exactly one version", - ], + ["name=ghidra-allegrex\nversion=12.1.\\uZZZZ\n", "Unicode escape"], ])( - "rejects incompatible or ambiguous extension metadata %s", + "rejects mismatched or malformed extension metadata %s", async (properties, reason) => { const { root } = await installation(); await writeFile(join(root, "extension.properties"), properties); From d9e9fb2273543caab4eb21d896c33d462ada6971 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:01:59 +0800 Subject: [PATCH 060/159] fix(process): use monotonic time for provider execution budgets --- src/firmware/FirmwareCommand.ts | 4 +-- src/javascript/recovery/RecoveryFiles.ts | 2 +- src/javascript/recovery/WakaruCommand.test.ts | 8 ++--- src/javascript/recovery/WakaruProvider.ts | 2 +- src/process/ProviderDeadline.test.ts | 35 +++++++++++++++++++ src/process/ProviderDeadline.ts | 8 ++--- 6 files changed, 47 insertions(+), 12 deletions(-) create mode 100644 src/process/ProviderDeadline.test.ts diff --git a/src/firmware/FirmwareCommand.ts b/src/firmware/FirmwareCommand.ts index b5db919f5..1604de901 100644 --- a/src/firmware/FirmwareCommand.ts +++ b/src/firmware/FirmwareCommand.ts @@ -176,11 +176,11 @@ export const runFirmwareCommand = async (context: { }; let failure: unknown; try { - const deadline = Date.now() + FIRMWARE_LIMITS.timeoutMs; + const deadline = performance.now() + FIRMWARE_LIMITS.timeoutMs; while (!(await supervisor.waitForOutputClose(100))) { checkDiagnostics(); if (signal.aborted) throw new AnalysisCancelledError(operation); - if (Date.now() >= deadline) + if (performance.now() >= deadline) throw new AnalysisTimeoutError(operation, FIRMWARE_LIMITS.timeoutMs); if (context.outputBudget !== undefined) await inventoryFirmwareOutput( diff --git a/src/javascript/recovery/RecoveryFiles.ts b/src/javascript/recovery/RecoveryFiles.ts index 0503a83e1..c64f011b4 100644 --- a/src/javascript/recovery/RecoveryFiles.ts +++ b/src/javascript/recovery/RecoveryFiles.ts @@ -279,6 +279,6 @@ export const checkRecoveryDeadline = ( signal?: AbortSignal, ): void => { checkRecoveryCancellation(signal); - if (Date.now() >= deadline) + if (performance.now() >= deadline) throw new AnalysisTimeoutError(OPERATION, RECOVERY_LIMITS.timeoutMs); }; diff --git a/src/javascript/recovery/WakaruCommand.test.ts b/src/javascript/recovery/WakaruCommand.test.ts index 08e43e44a..8dcb74be4 100644 --- a/src/javascript/recovery/WakaruCommand.test.ts +++ b/src/javascript/recovery/WakaruCommand.test.ts @@ -33,7 +33,7 @@ it("rejects an exhausted shared execution budget before launching the next comma arguments: ["--version"], cwd: "/tmp/unused", environment: {}, - deadline: Date.now() - 1, + deadline: performance.now() - 1, launcher: async () => { throw new Error("An expired budget must not launch"); }, @@ -52,7 +52,7 @@ it("preserves caller cancellation before acquiring an expired execution budget", arguments: ["--version"], cwd: "/tmp/unused", environment: {}, - deadline: Date.now() - 1, + deadline: performance.now() - 1, signal: AbortSignal.abort(), launcher: async () => { throw new Error("A cancelled request must not launch"); @@ -86,7 +86,7 @@ const commandFixture = ( arguments: ["--version"], cwd: "/tmp/unused", environment: {}, - deadline: Date.now() + 5_000, + deadline: performance.now() + 5_000, launcher, }, }; @@ -140,7 +140,7 @@ it("enforces the shared deadline while exited output streams remain open", async }); try { await expect( - runWakaruCommand({ ...context, deadline: Date.now() + 50 }), + runWakaruCommand({ ...context, deadline: performance.now() + 50 }), ).rejects.toMatchObject({ _tag: "AnalysisTimeoutError", operation: "recover_javascript_sources", diff --git a/src/javascript/recovery/WakaruProvider.ts b/src/javascript/recovery/WakaruProvider.ts index 7a51c8bd5..b1260bcd1 100644 --- a/src/javascript/recovery/WakaruProvider.ts +++ b/src/javascript/recovery/WakaruProvider.ts @@ -100,7 +100,7 @@ export class WakaruProvider implements JavaScriptRecoveryPort { let tree: SafeOutputTree | undefined; let failed: { readonly cause: unknown } | undefined; let execution: AnalysisExecution | undefined; - const deadline = Date.now() + RECOVERY_LIMITS.timeoutMs; + const deadline = performance.now() + RECOVERY_LIMITS.timeoutMs; try { checkRecoveryCancellation(options?.signal); const engine = await resolveWakaruCommand(this.environment); diff --git a/src/process/ProviderDeadline.test.ts b/src/process/ProviderDeadline.test.ts new file mode 100644 index 000000000..b43ce6654 --- /dev/null +++ b/src/process/ProviderDeadline.test.ts @@ -0,0 +1,35 @@ +import { afterEach, expect, it, vi } from "vitest"; +import { ProviderStartupDeadline } from "./ProviderDeadline.js"; + +afterEach(() => { + vi.restoreAllMocks(); +}); + +it("keeps remaining time and cancellation classification stable across wall-clock jumps", () => { + const dateNow = vi.spyOn(Date, "now").mockReturnValue(10_000); + const caller = new AbortController(); + const deadline = new ProviderStartupDeadline(10_000, caller.signal); + try { + dateNow.mockReturnValue(9_000_000_000_000); + const remaining = deadline.remainingMs(); + expect(remaining).toBeGreaterThan(0); + expect(remaining).toBeLessThanOrEqual(10_000); + expect(Number.isInteger(remaining)).toBe(true); + expect(deadline.interruption).toBeUndefined(); + caller.abort(new Error("caller stopped startup")); + expect(deadline.interruption).toBe("cancelled"); + } finally { + deadline.dispose(); + } +}); + +it("aborts a bounded wait when its real monotonic deadline expires", async () => { + const deadline = new ProviderStartupDeadline(40); + try { + await expect(deadline.wait(500)).resolves.toBe("aborted"); + expect(deadline.interruption).toBe("timeout"); + expect(deadline.remainingMs()).toBe(0); + } finally { + deadline.dispose(); + } +}); diff --git a/src/process/ProviderDeadline.ts b/src/process/ProviderDeadline.ts index 3081f469f..33cb3a8d0 100644 --- a/src/process/ProviderDeadline.ts +++ b/src/process/ProviderDeadline.ts @@ -42,11 +42,11 @@ export class ProviderStartupDeadline { ) { if (!Number.isFinite(timeoutMs) || timeoutMs <= 0) throw new RangeError("Provider startup timeout must be positive"); - this.#deadlineAt = Date.now() + timeoutMs; + this.#deadlineAt = performance.now() + timeoutMs; this.#externalSignal = signal; this.#onExternalAbort = () => { if (this.#controller.signal.aborted) return; - if (Date.now() >= this.#deadlineAt) { + if (performance.now() >= this.#deadlineAt) { this.#interruption = "timeout"; this.#abort( new DOMException("Provider startup deadline elapsed", "TimeoutError"), @@ -79,13 +79,13 @@ export class ProviderStartupDeadline { get interruption(): "cancelled" | "timeout" | undefined { return ( this.#interruption ?? - (Date.now() >= this.#deadlineAt ? "timeout" : undefined) + (performance.now() >= this.#deadlineAt ? "timeout" : undefined) ); } /** Milliseconds still available to the next startup phase. */ remainingMs(): number { - return Math.max(0, this.#deadlineAt - Date.now()); + return Math.max(0, Math.ceil(this.#deadlineAt - performance.now())); } /** Wait for at most one polling interval under the shared deadline signal. */ From 6bd2b28161411f70ed5dc72e42c2736a1aab758b Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:02:54 +0800 Subject: [PATCH 061/159] fix(browser): keep elapsed capture timing independent of wall-clock changes --- .../BrowserScenarioPartialObservation.ts | 2 +- .../PlaywrightBrowserScenarioProvider.test.ts | 28 +++++++++++++++++++ .../PlaywrightBrowserScenarioProvider.ts | 8 +++--- .../PlaywrightElectronActiveActions.ts | 14 +++++++--- .../PlaywrightElectronActiveProvider.ts | 8 +++--- .../PlaywrightExecutionBoundary.test.ts | 21 +++++++++++++- src/browser/PlaywrightExecutionBoundary.ts | 4 +-- src/browser/PlaywrightScenarioSession.ts | 2 +- src/browser/WebSourceMapFetcher.ts | 6 ++-- 9 files changed, 73 insertions(+), 20 deletions(-) diff --git a/src/browser/BrowserScenarioPartialObservation.ts b/src/browser/BrowserScenarioPartialObservation.ts index ca0df4273..a573f600f 100644 --- a/src/browser/BrowserScenarioPartialObservation.ts +++ b/src/browser/BrowserScenarioPartialObservation.ts @@ -91,7 +91,7 @@ export const browserScenarioCaptureData = (input: { network_content: input.scenario.capture.network, event_families: [...new Set(input.scenario.capture.events)].sort(), }, - duration_ms: Date.now() - input.startedAt, + duration_ms: Math.max(0, Math.round(performance.now() - input.startedAt)), steps: [...input.steps], events, limitations: [ diff --git a/src/browser/PlaywrightBrowserScenarioProvider.test.ts b/src/browser/PlaywrightBrowserScenarioProvider.test.ts index aacf766ad..883b0c3b4 100644 --- a/src/browser/PlaywrightBrowserScenarioProvider.test.ts +++ b/src/browser/PlaywrightBrowserScenarioProvider.test.ts @@ -303,6 +303,34 @@ describe("PlaywrightBrowserScenarioProvider", () => { }); expect(result.value.limitations).toContain(session.eventGaps[0]); }); + + it("keeps capture elapsed values valid across a backward wall-clock jump", async () => { + let wallClock = 10_000; + vi.spyOn(Date, "now").mockImplementation(() => wallClock); + const session = new FakeSession("launch"); + session.onPerform = () => { + wallClock = 1; + }; + const provider = new PlaywrightBrowserScenarioProvider(() => + Promise.resolve(session), + ); + + try { + const result = await provider.captureScenario(scenario()); + if (!result.ok) throw result.error; + expect(wallClock).toBe(1); + expect(Number.isInteger(result.value.duration_ms)).toBe(true); + expect(result.value.duration_ms).toBeGreaterThanOrEqual(0); + expect( + result.value.steps.every( + ({ elapsed_ms }) => Number.isInteger(elapsed_ms) && elapsed_ms >= 0, + ), + ).toBe(true); + } finally { + vi.restoreAllMocks(); + } + }); + it("returns an initial state and every step beyond the former action ceiling", async () => { const session = new FakeSession("launch"); const provider = new PlaywrightBrowserScenarioProvider(() => diff --git a/src/browser/PlaywrightBrowserScenarioProvider.ts b/src/browser/PlaywrightBrowserScenarioProvider.ts index a3c6d6a62..b0833e89c 100644 --- a/src/browser/PlaywrightBrowserScenarioProvider.ts +++ b/src/browser/PlaywrightBrowserScenarioProvider.ts @@ -172,7 +172,7 @@ const executeStep = async (input: { ), }); - const actionStartedAt = Date.now(); + const actionStartedAt = performance.now(); session.setStep(stepIndex); const requestCancelled = (): boolean => signal?.aborted === true; let outcome: BrowserScenarioStepOutcome = { @@ -200,7 +200,7 @@ const executeStep = async (input: { stepId: action.step_id, action: action.action, ...outcome, - elapsedMs: Date.now() - actionStartedAt, + elapsedMs: Math.max(0, Math.round(performance.now() - actionStartedAt)), beforeUrl, afterUrl: session.sanitizeUrl(session.currentUrl()), eventStart, @@ -233,7 +233,7 @@ const runScenario = async ( ): Promise => { if (options.signal?.aborted === true) throw new BrowserObservationError(OPERATION, "cancelled"); - const startedAt = Date.now(); + const startedAt = performance.now(); const session = await openSession(scenario, { ...(options.signal === undefined ? {} : { signal: options.signal }), retainCleanup, @@ -250,7 +250,7 @@ const runScenario = async ( await initialStep({ session, scenario, - elapsedMs: Date.now() - startedAt, + elapsedMs: Math.max(0, Math.round(performance.now() - startedAt)), signal: options.signal, }), ); diff --git a/src/browser/PlaywrightElectronActiveActions.ts b/src/browser/PlaywrightElectronActiveActions.ts index d59afb676..7bb9ec83b 100644 --- a/src/browser/PlaywrightElectronActiveActions.ts +++ b/src/browser/PlaywrightElectronActiveActions.ts @@ -72,7 +72,7 @@ export const runElectronActions = async ( ): Promise => { const actions: ElectronActions = []; for (const action of input.actions) { - const actionStartedAt = Date.now(); + const actionStartedAt = performance.now(); const windowIndex = actionWindowIndex(action); let selectedWindow: Page | undefined; try { @@ -107,7 +107,10 @@ export const runElectronActions = async ( target: selectedWindow === undefined ? null : `window:${String(windowIndex)}`, status: "completed", - elapsed_ms: Date.now() - actionStartedAt, + elapsed_ms: Math.max( + 0, + Math.round(performance.now() - actionStartedAt), + ), error: null, }); } catch (cause: unknown) { @@ -118,7 +121,10 @@ export const runElectronActions = async ( target: selectedWindow === undefined ? null : `window:${String(windowIndex)}`, status: options.signal?.aborted ? "cancelled" : "failed", - elapsed_ms: Date.now() - actionStartedAt, + elapsed_ms: Math.max( + 0, + Math.round(performance.now() - actionStartedAt), + ), error: safeActionErrorMessage(cause), }); break; @@ -366,7 +372,7 @@ export const runWithExecutionLimits = async ( new Promise((_, reject) => { timer = setTimeout( () => reject(new BrowserObservationError(OPERATION, "timeout")), - Math.max(1, deadline - Date.now()), + Math.max(1, deadline - performance.now()), ); }), ); diff --git a/src/browser/PlaywrightElectronActiveProvider.ts b/src/browser/PlaywrightElectronActiveProvider.ts index 4b6e6bb14..23379524d 100644 --- a/src/browser/PlaywrightElectronActiveProvider.ts +++ b/src/browser/PlaywrightElectronActiveProvider.ts @@ -86,7 +86,7 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati if (options.signal?.aborted === true) throw new BrowserObservationError(OPERATION, "cancelled"); await prepareProcessOwnershipInspection(options.signal); - const startupDeadline = Date.now() + STARTUP_TIMEOUT_MS; + const startupDeadline = performance.now() + STARTUP_TIMEOUT_MS; paths = { executable: input.executable_path, application: input.application_path, @@ -101,7 +101,7 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati cwd: paths.root, env: safeElectronEnvironment(this.environment, runId), args: ["-r", hookPath, paths.application, ...input.args], - timeout: Math.max(1, startupDeadline - Date.now()), + timeout: Math.max(1, startupDeadline - performance.now()), }); const leaderPid = application.process().pid; if ( @@ -128,7 +128,7 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati const state = await runWithExecutionLimits( readApplicationState(application), undefined, - options.signal?.aborted ? Date.now() + 5_000 : undefined, + options.signal?.aborted ? performance.now() + 5_000 : undefined, ); capture = projectElectronActiveCapture(paths, actions, state); outcome = options.signal?.aborted @@ -187,7 +187,7 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati await runWithExecutionLimits( application.close(), undefined, - Date.now() + 5_000, + performance.now() + 5_000, ); } catch (cause: unknown) { closeError = cause; diff --git a/src/browser/PlaywrightExecutionBoundary.test.ts b/src/browser/PlaywrightExecutionBoundary.test.ts index 323b4506c..51df81701 100644 --- a/src/browser/PlaywrightExecutionBoundary.test.ts +++ b/src/browser/PlaywrightExecutionBoundary.test.ts @@ -7,7 +7,10 @@ import { } from "../domain/analysisErrorCore.js"; describe("withPlaywrightExecutionBoundary", () => { - afterEach(() => vi.useRealTimers()); + afterEach(() => { + vi.useRealTimers(); + vi.restoreAllMocks(); + }); it("rejects work that exceeds its absolute remaining duration", async () => { vi.useFakeTimers(); @@ -31,4 +34,20 @@ describe("withPlaywrightExecutionBoundary", () => { controller.abort(); await expect(result).rejects.toBeInstanceOf(AnalysisCancelledError); }); + + it("keeps its timeout stable across a wall-clock jump", async () => { + vi.spyOn(Date, "now") + .mockReturnValueOnce(1_000) + .mockReturnValue(10_000_000); + + await expect( + withPlaywrightExecutionBoundary( + () => + new Promise((resolve) => + setTimeout(() => resolve("done"), 5), + ), + 1_000, + ), + ).resolves.toBe("done"); + }); }); diff --git a/src/browser/PlaywrightExecutionBoundary.ts b/src/browser/PlaywrightExecutionBoundary.ts index 9fd21fcf3..759fcd484 100644 --- a/src/browser/PlaywrightExecutionBoundary.ts +++ b/src/browser/PlaywrightExecutionBoundary.ts @@ -19,10 +19,10 @@ export const withPlaywrightExecutionBoundary = ( let settled = false; let timer: ReturnType | undefined; const deadline = - timeoutMs === undefined ? undefined : Date.now() + timeoutMs; + timeoutMs === undefined ? undefined : performance.now() + timeoutMs; const scheduleTimeout = (): void => { if (deadline === undefined) return; - const remaining = deadline - Date.now(); + const remaining = deadline - performance.now(); if (remaining <= 0) { fail(new AnalysisTimeoutError(OPERATION, timeoutMs ?? 0)); return; diff --git a/src/browser/PlaywrightScenarioSession.ts b/src/browser/PlaywrightScenarioSession.ts index 486f555ea..949193b7c 100644 --- a/src/browser/PlaywrightScenarioSession.ts +++ b/src/browser/PlaywrightScenarioSession.ts @@ -133,7 +133,7 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort { const secrets = BrowserScenarioSecrets.resolve(scenario, environment); if (secrets === undefined) throw new BrowserObservationError(OPERATION, "secret_unavailable"); - const startedAt = Date.now(); + const startedAt = performance.now(); const opening = openPlaywrightScenarioBrowser(scenario, environment, { signal: options.signal, ...(options.retainCleanup === undefined diff --git a/src/browser/WebSourceMapFetcher.ts b/src/browser/WebSourceMapFetcher.ts index b950926ee..c4bae782a 100644 --- a/src/browser/WebSourceMapFetcher.ts +++ b/src/browser/WebSourceMapFetcher.ts @@ -86,7 +86,7 @@ export const fetchWebSourceMaps = async ( signal?.addEventListener("abort", abortFromCaller, { once: true }); const operationSignal = operationController.signal; const deadlineAt = - Date.now() + (host.timeoutMs ?? SOURCE_MAP_FETCH_TIMEOUT_MS); + performance.now() + (host.timeoutMs ?? SOURCE_MAP_FETCH_TIMEOUT_MS); const budget = { retainedBytes: 0, deadlineAt, @@ -110,7 +110,7 @@ export const fetchWebSourceMaps = async ( } const expandedBytesBeforeItem = budget.decodedRecords.expandedBytes; budget.decodedRecords.expandedBytes += reservation; - if (operationSignal.aborted || Date.now() >= deadlineAt) { + if (operationSignal.aborted || performance.now() >= deadlineAt) { items.push( emptySourceMapItem( request, @@ -789,7 +789,7 @@ const checkOperation = ( ): void => { if (callerSignal?.aborted === true) throw callerSignal.reason; if (operationSignal?.aborted === true) throw operationSignal.reason; - if (Date.now() >= deadlineAt) throw new SourceMapDeadlineError(); + if (performance.now() >= deadlineAt) throw new SourceMapDeadlineError(); }; const approvedUrl = ( From 20f51d7286b1759e42f99c0512c73230052dda86 Mon Sep 17 00:00:00 2001 From: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> Date: Tue, 13 Oct 2026 00:10:58 -0700 Subject: [PATCH 062/159] fix(native): name the ownership helper prerequisite in command failures (#2506) * fix(native): name the ownership helper prerequisite in command failures Every supervised native command launch needs the Darwin process-ownership helper. When that helper cannot be compiled (relative REA_DARWIN_SWIFTC_PATH, missing SDKROOT, a missing compiler, or an SDK/compiler mismatch), the runner reduced the typed DarwinProcessOwnershipInspectionError to a bare `io` failure. inspect_macho then reported `execution_failure` with no hint, and inspect_artifact on any universal Mach-O said only "lipo could not enumerate universal Mach-O slices". The runner now keeps the inspection message as the failure's `detail` at the launch boundary and carries it through xcrun resolution. The native provider adds it to its diagnostics and the slice reader appends it to its message, matching what the SQLite and capture providers already report. Co-Authored-By: Claude Opus 5.5 * test(native): cover the provider's ownership helper diagnostics Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 --- src/artifacts/MachOSliceArtifactReader.ts | 4 ++- src/native/CommandRunner.ts | 19 +++++++++++++- src/native/NativeMacOSProvider.ts | 1 + .../providers/native/machoSliceReader.test.ts | 22 ++++++++++++++++ .../native/nativeCommandTermination.test.ts | 21 ++++++++++++++++ .../providers/native/nativeProvider.test.ts | 25 +++++++++++++++++++ 6 files changed, 90 insertions(+), 2 deletions(-) diff --git a/src/artifacts/MachOSliceArtifactReader.ts b/src/artifacts/MachOSliceArtifactReader.ts index 9420bef26..5928956ee 100644 --- a/src/artifacts/MachOSliceArtifactReader.ts +++ b/src/artifacts/MachOSliceArtifactReader.ts @@ -81,7 +81,9 @@ export class MachOSliceArtifactReader implements ArtifactReader { if (!captured.ok) throw new ArtifactReaderFailure( captured.error.reason === "cancelled" ? "cancelled" : "unavailable", - "lipo could not enumerate universal Mach-O slices", + captured.error.detail === null + ? "lipo could not enumerate universal Mach-O slices" + : `lipo could not enumerate universal Mach-O slices: ${captured.error.detail}`, { cause: captured.error, ...(captured.error.cleanupFailure === null && diff --git a/src/native/CommandRunner.ts b/src/native/CommandRunner.ts index e9775d1b7..ccd49c570 100644 --- a/src/native/CommandRunner.ts +++ b/src/native/CommandRunner.ts @@ -4,6 +4,7 @@ import { createReadStream } from "node:fs"; import { realpath } from "node:fs/promises"; import { err, ok, type Result } from "../domain/result.js"; +import { DarwinProcessOwnershipInspectionError } from "../process/DarwinProcessRunTokenReader.js"; import { OwnedCommandFailure, runOwnedCommand, @@ -39,6 +40,7 @@ export interface NativeCommandCapture { export class NativeCommandFailure extends Error { readonly cleanupOwner: ProviderProcessSupervisor | undefined; readonly cleanupFailures: readonly NativeCommandCleanupFailure[]; + readonly detail: string | null; constructor( readonly tool: string, @@ -58,6 +60,7 @@ export class NativeCommandFailure extends Error { super(`Native command ${tool} failed: ${reason}`, options); this.cleanupOwner = options?.cleanupOwner; this.cleanupFailures = options?.cleanupFailures ?? []; + this.detail = options?.detail ?? null; } } @@ -67,6 +70,8 @@ export interface NativeCommandFailureOptions extends ErrorOptions { readonly cleanupOwner?: ProviderProcessSupervisor; /** All owners still incomplete after a preflight retry blocked this command. */ readonly cleanupFailures?: readonly NativeCommandCleanupFailure[]; + /** Host-side reason the command could not be supervised, such as a missing ownership helper compiler. */ + readonly detail?: string; } /** One failed cleanup attempt; the runner retains `owner` until it is verified stopped. */ @@ -310,6 +315,9 @@ export const resolveXcrunTool = async ( ...(found.error.cleanupFailures.length === 0 ? {} : { cleanupFailures: found.error.cleanupFailures }), + ...(found.error.detail === null + ? {} + : { detail: found.error.detail }), }, found.error.capture, found.error.cleanupFailure, @@ -449,7 +457,16 @@ const captureProcess = async ( ), ); } - return err(new NativeCommandFailure(tool, "io", null, { cause })); + return err( + new NativeCommandFailure(tool, "io", null, { + cause, + // Process supervision needs the Darwin ownership helper; its typed + // message names the compiler or SDK prerequisite to fix. + ...(cause instanceof DarwinProcessOwnershipInspectionError + ? { detail: cause.message } + : {}), + }), + ); } }; diff --git a/src/native/NativeMacOSProvider.ts b/src/native/NativeMacOSProvider.ts index 1d1c2a36c..a59ec90a7 100644 --- a/src/native/NativeMacOSProvider.ts +++ b/src/native/NativeMacOSProvider.ts @@ -966,6 +966,7 @@ const translateCommandFailure = ( signal: failure.capture?.signal ?? null, stdout_bytes: failure.capture?.stdoutBytes ?? 0, stderr_bytes: failure.capture?.stderrBytes ?? 0, + ...(failure.detail === null ? {} : { detail: failure.detail }), }, }); }; diff --git a/tests/boundary/providers/native/machoSliceReader.test.ts b/tests/boundary/providers/native/machoSliceReader.test.ts index f67b48388..5891eaf53 100644 --- a/tests/boundary/providers/native/machoSliceReader.test.ts +++ b/tests/boundary/providers/native/machoSliceReader.test.ts @@ -3,6 +3,8 @@ import { readFile } from "node:fs/promises"; import { describe, expect, it, onTestFinished } from "vitest"; import { MachOSliceArtifactReader } from "../../../../src/artifacts/MachOSliceArtifactReader.js"; +import { XcrunCommandRunner } from "../../../../src/native/CommandRunner.js"; +import { DarwinProcessOwnershipInspectionError } from "../../../../src/process/DarwinProcessRunTokenReader.js"; /** * Read a 32-bit FAT table directly. The host's /bin/ls changes between macOS @@ -20,6 +22,26 @@ const fatSlices = async ( }; describe("Mach-O slice reader with system lipo", () => { + it("names the process ownership helper prerequisite when lipo cannot be supervised", async () => { + const prerequisite = + "Direct Darwin helper compilation requires absolute REA_DARWIN_SWIFTC_PATH and SDKROOT paths."; + const reader = new MachOSliceArtifactReader( + process.execPath, + {}, + new XcrunCommandRunner({}, undefined, () => + Promise.reject(new DarwinProcessOwnershipInspectionError(prerequisite)), + ), + ); + onTestFinished(() => reader.close()); + const enumerate = async () => { + for await (const _entry of reader.entries()) continue; + }; + await expect(enumerate()).rejects.toMatchObject({ + reason: "unavailable", + message: `lipo could not enumerate universal Mach-O slices: ${prerequisite}`, + }); + }); + it.skipIf(process.platform !== "darwin")( "checks real /bin/ls slice ranges against its FAT table", async () => { diff --git a/tests/boundary/providers/native/nativeCommandTermination.test.ts b/tests/boundary/providers/native/nativeCommandTermination.test.ts index 233ebe85e..5d23e6b22 100644 --- a/tests/boundary/providers/native/nativeCommandTermination.test.ts +++ b/tests/boundary/providers/native/nativeCommandTermination.test.ts @@ -8,6 +8,7 @@ import { type NativeCommandLauncher, } from "../../../../src/native/CommandRunner.js"; import { ok } from "../../../../src/domain/result.js"; +import { DarwinProcessOwnershipInspectionError } from "../../../../src/process/DarwinProcessRunTokenReader.js"; afterEach(() => vi.unstubAllEnvs()); @@ -210,3 +211,23 @@ it.skipIf(process.platform === "win32")( if (!killed.ok) expect(killed.error.reason).toBe("nonzero-exit"); }, ); + +it("keeps the ownership helper prerequisite when xcrun cannot be supervised", async () => { + const inspection = new DarwinProcessOwnershipInspectionError( + "macOS process ownership inspection requires the Apple Swift compiler via xcrun", + ); + const runner = new XcrunCommandRunner({}, undefined, () => + Promise.reject(inspection), + ); + + const result = await runner.run("file", [], {}); + + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error).toMatchObject({ + tool: "file", + reason: "io", + detail: inspection.message, + cause: { tool: "xcrun", detail: inspection.message, cause: inspection }, + }); +}); diff --git a/tests/boundary/providers/native/nativeProvider.test.ts b/tests/boundary/providers/native/nativeProvider.test.ts index b4a713204..3dd2a9b2c 100644 --- a/tests/boundary/providers/native/nativeProvider.test.ts +++ b/tests/boundary/providers/native/nativeProvider.test.ts @@ -1,4 +1,5 @@ import { createHash } from "node:crypto"; +import { DarwinProcessOwnershipInspectionError } from "../../../../src/process/DarwinProcessRunTokenReader.js"; import { EventEmitter } from "node:events"; import { chmod, mkdir, readFile, rm, writeFile } from "node:fs/promises"; import { join } from "node:path"; @@ -280,6 +281,30 @@ describe("native command failure capture and limits", () => { stderr_bytes: expect.any(Number), }, }); + expect(projectAnalysisError(result.error).details).not.toHaveProperty( + "diagnostics.detail", + ); + }); + + it("projects the ownership helper prerequisite when a command cannot be supervised", async () => { + const inspection = new DarwinProcessOwnershipInspectionError( + "Direct Darwin helper compilation requires absolute REA_DARWIN_SWIFTC_PATH and SDKROOT paths.", + ); + const runner = new XcrunCommandRunner( + {}, + () => + Promise.resolve(ok({ path: process.execPath, sha256: "a".repeat(64) })), + () => Promise.reject(inspection), + ); + const result = await new NativeMacOSProvider({}, runner, "darwin") + .createClient(machoTarget("/private/fixture")) + .execute("demangle_swift", { symbols: ["fixture-symbol"] }); + + expect(result.ok).toBe(false); + if (result.ok) return; + expect(projectAnalysisError(result.error).details).toMatchObject({ + diagnostics: { reason: "io", detail: inspection.message }, + }); }); it("retains output written before cancellation and reaps the stalled command", async () => { From 6a05248d464387c0ce4ec1577b4c1b24d3d355c7 Mon Sep 17 00:00:00 2001 From: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> Date: Tue, 13 Oct 2026 00:11:38 -0700 Subject: [PATCH 063/159] perf(frida): evict captured messages without copying the whole queue (#2628) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Once a Frida session reached its 16 MiB capture budget, every new message evicted the oldest one with Array#shift, re-serialized it to recover its byte charge, and searched and spliced its per-script array. Each of those is linear in the retained messages, so with about 188k small messages retained the cost per message rose from 0.14 µs to 32 µs and capped the main thread at roughly 31k messages per second, which a hooked hot function can exceed. Keep both queues as head-offset FIFOs that compact once half their slots are evicted, and record each message's charge when it is admitted. Arrival order already makes the globally oldest message the oldest one of its script, so per-script eviction becomes a head check. Eviction is now amortized O(1) at about 0.14 µs per message; retained messages, their order and the truncation flag are unchanged. Co-authored-by: Claude Opus 5.5 --- .../frida/FridaInstrumentationManager.test.ts | 36 ++++++++ src/frida/FridaInstrumentationManager.ts | 85 ++++++++++++++----- 2 files changed, 98 insertions(+), 23 deletions(-) diff --git a/src/application/frida/FridaInstrumentationManager.test.ts b/src/application/frida/FridaInstrumentationManager.test.ts index 29ba97965..7a527a807 100644 --- a/src/application/frida/FridaInstrumentationManager.test.ts +++ b/src/application/frida/FridaInstrumentationManager.test.ts @@ -809,6 +809,42 @@ it("keeps an active script message bucket after its last message is evicted", as await manager.closeAll(); }); +it("retains the newest contiguous messages across repeated budget evictions", async () => { + const { bindings, manager, started } = await startLocalAttachedSession(); + expect(started.ok).toBe(true); + if (!started.ok) return; + bindings.device.session.nextPayload = { index: -1 }; + const loaded = await manager.loadScript(started.value.sessionId, { + sourceKind: "inline", + source: "send({index: -1});", + }); + expect(loaded.ok).toBe(true); + const script = bindings.device.session.scripts[0]; + expect(script).toBeDefined(); + if (script === undefined) return; + + // About 500 of these fit the 16 MiB budget, so 2,000 sends evict and + // compact the queue several times. + const filler = "x".repeat(32_000); + for (let index = 0; index < 2_000; index += 1) + script.message.emit({ type: "send", payload: { index, filler } }, null); + + const status = manager.status(started.value.sessionId); + expect(status?.messagesTruncated).toBe(true); + const indexes = (status?.messages ?? []).map( + (message) => (message as { payload: { index: number } }).payload.index, + ); + expect(indexes.length).toBeGreaterThan(400); + expect(indexes.at(-1)).toBe(1_999); + expect(indexes).toEqual( + Array.from( + { length: indexes.length }, + (_, offset) => 2_000 - indexes.length + offset, + ), + ); + await manager.closeAll(); +}); + it("serializes a final remote release against a new same-endpoint session", async () => { const bindings = new FakeDeviceManager(); let enteredRemoval: (() => void) | undefined; diff --git a/src/frida/FridaInstrumentationManager.ts b/src/frida/FridaInstrumentationManager.ts index 406258e5a..22b38c442 100644 --- a/src/frida/FridaInstrumentationManager.ts +++ b/src/frida/FridaInstrumentationManager.ts @@ -83,8 +83,8 @@ interface ManagedSession extends ManagedRemoteConnection { readonly pid: number; readonly mode: "attach" | "spawn"; readonly scripts: Map; - readonly messages: JsonValue[]; - readonly messagesByScript: Map; + readonly messages: MessageFifo; + readonly messagesByScript: Map>; state: "paused" | "running" | "detached"; targetResumeObserved: boolean; resumeAfterDetachPending: boolean; @@ -277,7 +277,7 @@ export class FridaInstrumentationManager implements FridaInstrumentationPort { pid, mode: input.mode, scripts: new Map(), - messages: [], + messages: new MessageFifo(), messagesByScript: new Map(), state: input.mode === "spawn" ? "paused" : "running", targetResumeObserved: false, @@ -373,7 +373,7 @@ export class FridaInstrumentationManager implements FridaInstrumentationPort { .digest("hex"); const scriptId = randomUUID(); const scriptName = `rea-${scriptId}`; - managed.messagesByScript.set(scriptId, []); + managed.messagesByScript.set(scriptId, new MessageFifo()); let nativeScript: Script | undefined; const onMessage = (message: FridaMessage, data: Buffer | null): void => { this.#recordMessage(managed, scriptId, message, data); @@ -397,7 +397,7 @@ export class FridaInstrumentationManager implements FridaInstrumentationPort { sourceKind: source.sourceKind, sourcePath: source.sourceKind === "file" ? source.path : null, sourceSha256, - messages: [...(managed.messagesByScript.get(scriptId) ?? [])], + messages: managed.messagesByScript.get(scriptId)?.toArray() ?? [], messagesTruncated: managed.messagesTruncated, }); } catch (cause: unknown) { @@ -517,7 +517,9 @@ export class FridaInstrumentationManager implements FridaInstrumentationPort { scriptId: id, name, })), - messages: [...managed.messages], + messages: managed.messages + .toArray() + .map(({ observation }) => observation), messagesTruncated: managed.messagesTruncated, }; } @@ -851,30 +853,21 @@ export class FridaInstrumentationManager implements FridaInstrumentationPort { return; } while ( - managed.messages.length > 0 && + managed.messages.size > 0 && managed.capturedBytes + charge > MAX_SESSION_CAPTURE_BYTES ) { const removed = managed.messages.shift(); if (removed !== undefined) { - managed.capturedBytes -= Buffer.byteLength(JSON.stringify(removed)); - if ( - typeof removed === "object" && - removed !== null && - !Array.isArray(removed) && - typeof removed.script_id === "string" - ) { - const scriptMessages = managed.messagesByScript.get( - removed.script_id, - ); - const index = scriptMessages?.indexOf(removed) ?? -1; - if (scriptMessages !== undefined && index >= 0) { - scriptMessages.splice(index, 1); - } - } + managed.capturedBytes -= removed.charge; + // Both queues are in arrival order, so the globally oldest message is + // also the oldest one retained for its script. + const scriptMessages = managed.messagesByScript.get(removed.scriptId); + if (scriptMessages?.peek() === removed.observation) + scriptMessages.shift(); } managed.messagesTruncated = true; } - managed.messages.push(observation); + managed.messages.push({ observation, scriptId, charge }); managed.messagesByScript.get(scriptId)?.push(observation); managed.capturedBytes += charge; } @@ -1240,3 +1233,49 @@ const providerError = ( const errorMessage = (cause: unknown): string => cause instanceof Error ? cause.message : String(cause); + +interface CapturedMessage { + readonly observation: JsonValue; + readonly scriptId: string; + /** Serialized bytes charged against the session capture budget. */ + readonly charge: number; +} + +/** + * Arrival-ordered queue whose eviction advances a head offset. A full capture + * budget evicts on every message, and `Array#shift` would copy every retained + * message each time. + */ +class MessageFifo { + #items: (T | undefined)[] = []; + #head = 0; + + get size(): number { + return this.#items.length - this.#head; + } + + push(item: T): void { + this.#items.push(item); + } + + peek(): T | undefined { + return this.#items[this.#head]; + } + + shift(): T | undefined { + if (this.#head >= this.#items.length) return undefined; + const item = this.#items[this.#head]; + this.#items[this.#head] = undefined; + this.#head += 1; + // Compacting once half the slots are evicted keeps eviction amortized O(1). + if (this.#head * 2 >= this.#items.length) { + this.#items = this.#items.slice(this.#head); + this.#head = 0; + } + return item; + } + + toArray(): T[] { + return this.#items.slice(this.#head) as T[]; + } +} From 7fc0b5ec9c2532bbf789d7783b7a0cab6fc054d8 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:12:08 +0800 Subject: [PATCH 064/159] fix(mcp): join owned doctor shutdown without signaling saved PIDs --- src/mcpDoctor.ts | 146 +++++++++++++++++++++++------------------------ 1 file changed, 73 insertions(+), 73 deletions(-) diff --git a/src/mcpDoctor.ts b/src/mcpDoctor.ts index 62a588280..d7bf1e366 100644 --- a/src/mcpDoctor.ts +++ b/src/mcpDoctor.ts @@ -1,6 +1,7 @@ import { Client } from "@modelcontextprotocol/client"; import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; import { Formatter, Help, z } from "incur"; +import { performance } from "node:perf_hooks"; import { resolve } from "node:path"; import { CATALOG_IDENTITY } from "./catalogIdentity.js"; @@ -74,14 +75,15 @@ interface McpDoctorCheck { /** Diagnose the actual production stdio adapter with one startup-bounded SDK session. */ export const runProductionMcpDoctor = async (options: McpDoctorOptions) => { const deadline = - Date.now() + (options.deadlineMs ?? MCP_STARTUP_POLICY.doctorDeadlineMs); + performance.now() + + (options.deadlineMs ?? MCP_STARTUP_POLICY.doctorDeadlineMs); const controller = new AbortController(); const timer = setTimeout( () => controller.abort(new Error("Production MCP doctor deadline expired")), - Math.max(1, deadline - Date.now()), + Math.max(1, Math.ceil(deadline - performance.now())), ); const stderr = captureStderr(); - const transport = new TrackedStdioClientTransport({ + const transport = new OwnedStdioClientTransport({ command: options.command, args: [...options.args], cwd: options.cwd, @@ -90,21 +92,20 @@ export const runProductionMcpDoctor = async (options: McpDoctorOptions) => { }); transport.stderr?.on("data", stderr.append); const client = new Client({ name: "rea-mcp-doctor", version: "1" }); + let result: ProductionMcpDoctorResult; + let cleanupFailure: { readonly cause: unknown } | undefined; try { await client.connect( transport, requestOptions(deadline, controller.signal), ); - return { - ...(await inspectProductionMcpSession( - client, - deadline, - controller.signal, - )), - diagnostics: stderr.value(), - }; + result = await inspectProductionMcpSession( + client, + deadline, + controller.signal, + ); } catch (cause: unknown) { - return { + result = { healthy: false, adapter: "production-stdio" as const, server: null, @@ -118,14 +119,71 @@ export const runProductionMcpDoctor = async (options: McpDoctorOptions) => { detail: errorMessage(cause), }, ], - diagnostics: stderr.value(), }; } finally { clearTimeout(timer); - await closeMcpChild(client, transport); + try { + await client.close(); + } catch (cause: unknown) { + cleanupFailure = { cause }; + } + try { + // An interrupted connect can clear the client's transport reference + // before its startup child is closed. The transport retains that child. + await transport.close(); + } catch (cause: unknown) { + cleanupFailure = { + cause: + cleanupFailure === undefined || cleanupFailure.cause === cause + ? cause + : new AggregateError( + [cleanupFailure.cause, cause], + `MCP client and transport cleanup failed: ${errorMessage(cleanupFailure.cause)}; ${errorMessage(cause)}`, + ), + }; + } } + if (cleanupFailure !== undefined) + result = { + ...result, + healthy: false, + checks: [ + ...result.checks, + { + name: "cleanup", + ok: false, + detail: `MCP stdio cleanup failed: ${errorMessage(cleanupFailure.cause)}`, + }, + ], + }; + return { ...result, diagnostics: stderr.value() }; }; +type ProductionMcpDoctorResult = + | Awaited> + | { + readonly healthy: false; + readonly adapter: "production-stdio"; + readonly server: null; + readonly inventory: null; + readonly identity: null; + readonly request_flow: { + readonly tool: "binary_session"; + readonly ok: false; + }; + readonly checks: readonly McpDoctorCheck[]; + }; + +/** Join SDK-triggered and caller-triggered close calls for the same child. */ +class OwnedStdioClientTransport extends StdioClientTransport { + #closePromise: Promise | undefined; + + override close(): Promise { + this.#closePromise ??= super.close(); + return this.#closePromise; + } +} + const inspectProductionMcpSession = async ( client: Client, deadline: number, @@ -385,7 +443,7 @@ const isOutputFormat = (value: string | undefined): value is OutputFormat => const requestOptions = (deadline: number, signal: AbortSignal) => ({ signal, - timeout: Math.max(1, deadline - Date.now()), + timeout: Math.max(1, Math.ceil(deadline - performance.now())), }); const targetFreeEnvironment = ( @@ -428,61 +486,3 @@ const isRecord = (value: unknown): value is Readonly> => const errorMessage = (cause: unknown): string => cause instanceof Error ? cause.message : String(cause); - -class TrackedStdioClientTransport extends StdioClientTransport { - spawnedPid: number | null = null; - - override async start(): Promise { - await super.start(); - this.spawnedPid = this.pid; - } -} - -const closeMcpChild = async ( - client: Client, - transport: TrackedStdioClientTransport, -): Promise => { - const pid = transport.spawnedPid ?? transport.pid; - await Promise.allSettled([client.close()]); - await Promise.allSettled([transport.close()]); - if (pid === null || !(await processExists(pid))) return; - await terminateProcess(pid, "SIGTERM"); - if (await waitForProcessExit(pid, 1_000)) return; - await terminateProcess(pid, "SIGKILL"); - await waitForProcessExit(pid, 1_000); -}; - -const terminateProcess = async ( - pid: number, - signal: NodeJS.Signals, -): Promise => { - try { - process.kill(pid, signal); - } catch (cause: unknown) { - if (!(cause instanceof Error && "code" in cause && cause.code === "ESRCH")) - throw cause; - } -}; - -const waitForProcessExit = async ( - pid: number, - timeoutMs: number, -): Promise => { - const deadline = Date.now() + timeoutMs; - do { - if (!(await processExists(pid))) return true; - await new Promise((resolve) => setTimeout(resolve, 20)); - } while (Date.now() < deadline); - return !(await processExists(pid)); -}; - -const processExists = async (pid: number): Promise => { - try { - process.kill(pid, 0); - return true; - } catch (cause: unknown) { - if (cause instanceof Error && "code" in cause && cause.code === "ESRCH") - return false; - throw cause; - } -}; From 9049dbaf0c3df32357419f9899965df2b611f493 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:12:08 +0800 Subject: [PATCH 065/159] fix(process): preserve monotonic capture timing and observed wall dates --- .../PlaywrightBrowserScenarioProvider.test.ts | 56 ++++++------ .../process/processCaptureValidation.ts | 4 - src/process/capture/ProcessCaptureClock.ts | 3 + src/process/capture/ProcessCaptureJournal.ts | 6 +- .../capture/ProcessCaptureLifecycle.test.ts | 5 +- .../capture/ProcessCaptureLifecycle.ts | 74 ++++++++++----- src/process/capture/ProcessCaptureProgress.ts | 3 +- src/process/capture/ProcessHarness.ts | 7 +- src/process/capture/ProcessSampling.ts | 3 +- .../filesystem/ghidraPspExtension.test.ts | 65 +++++++------- .../process/processCaptureLifecycle.test.ts | 89 ++++++++++++++++++- 11 files changed, 224 insertions(+), 91 deletions(-) create mode 100644 src/process/capture/ProcessCaptureClock.ts diff --git a/src/browser/PlaywrightBrowserScenarioProvider.test.ts b/src/browser/PlaywrightBrowserScenarioProvider.test.ts index 883b0c3b4..734e32426 100644 --- a/src/browser/PlaywrightBrowserScenarioProvider.test.ts +++ b/src/browser/PlaywrightBrowserScenarioProvider.test.ts @@ -304,33 +304,6 @@ describe("PlaywrightBrowserScenarioProvider", () => { expect(result.value.limitations).toContain(session.eventGaps[0]); }); - it("keeps capture elapsed values valid across a backward wall-clock jump", async () => { - let wallClock = 10_000; - vi.spyOn(Date, "now").mockImplementation(() => wallClock); - const session = new FakeSession("launch"); - session.onPerform = () => { - wallClock = 1; - }; - const provider = new PlaywrightBrowserScenarioProvider(() => - Promise.resolve(session), - ); - - try { - const result = await provider.captureScenario(scenario()); - if (!result.ok) throw result.error; - expect(wallClock).toBe(1); - expect(Number.isInteger(result.value.duration_ms)).toBe(true); - expect(result.value.duration_ms).toBeGreaterThanOrEqual(0); - expect( - result.value.steps.every( - ({ elapsed_ms }) => Number.isInteger(elapsed_ms) && elapsed_ms >= 0, - ), - ).toBe(true); - } finally { - vi.restoreAllMocks(); - } - }); - it("returns an initial state and every step beyond the former action ceiling", async () => { const session = new FakeSession("launch"); const provider = new PlaywrightBrowserScenarioProvider(() => @@ -404,6 +377,35 @@ describe("PlaywrightBrowserScenarioProvider", () => { }); }); +describe("PlaywrightBrowserScenarioProvider timing", () => { + it("keeps capture elapsed values valid across a backward wall-clock jump", async () => { + let wallClock = 10_000; + vi.spyOn(Date, "now").mockImplementation(() => wallClock); + const session = new FakeSession("launch"); + session.onPerform = () => { + wallClock = 1; + }; + const provider = new PlaywrightBrowserScenarioProvider(() => + Promise.resolve(session), + ); + + try { + const result = await provider.captureScenario(scenario()); + if (!result.ok) throw result.error; + expect(wallClock).toBe(1); + expect(Number.isInteger(result.value.duration_ms)).toBe(true); + expect(result.value.duration_ms).toBeGreaterThanOrEqual(0); + expect( + result.value.steps.every( + ({ elapsed_ms }) => Number.isInteger(elapsed_ms) && elapsed_ms >= 0, + ), + ).toBe(true); + } finally { + vi.restoreAllMocks(); + } + }); +}); + describe("PlaywrightBrowserScenarioProvider comparison coverage", () => { it("keeps quiet captures with different event families from comparing unchanged", async () => { const capture = async (events: readonly BrowserScenarioEventFamily[]) => { diff --git a/src/domain/process/processCaptureValidation.ts b/src/domain/process/processCaptureValidation.ts index 10d6b831b..33c6d5483 100644 --- a/src/domain/process/processCaptureValidation.ts +++ b/src/domain/process/processCaptureValidation.ts @@ -47,10 +47,6 @@ const validateCommitments = ( digestProcessCommitment( value, ), `manifest.${field}`, "commitment does not match its canonical value"); - require(Date.parse(manifest.started_at) <= - Date.parse( - manifest.completed_at, - ), "manifest.completed_at", "completion precedes start"); }; const validateOrdering = ( diff --git a/src/process/capture/ProcessCaptureClock.ts b/src/process/capture/ProcessCaptureClock.ts new file mode 100644 index 000000000..1fe9fda0a --- /dev/null +++ b/src/process/capture/ProcessCaptureClock.ts @@ -0,0 +1,3 @@ +/** Keep wall-clock timestamps separate from capture-relative elapsed time. */ +export const monotonicProcessCaptureClock = (): number => + Math.floor(performance.now()); diff --git a/src/process/capture/ProcessCaptureJournal.ts b/src/process/capture/ProcessCaptureJournal.ts index b8a0874d9..07c1f0351 100644 --- a/src/process/capture/ProcessCaptureJournal.ts +++ b/src/process/capture/ProcessCaptureJournal.ts @@ -13,6 +13,7 @@ import type { ProcessCaptureEventJournalEntry, RecordProcessCaptureEvent, } from "../../domain/process/processCapture.js"; +import { monotonicProcessCaptureClock } from "./ProcessCaptureClock.js"; /** Mutable observation-order ledger whose writer is shared by capture producers. */ export interface ProcessCaptureJournal { readonly entries: readonly ProcessCaptureEventJournalEntry[]; @@ -55,7 +56,10 @@ export const scheduleScenarioInteractions = ( const timer = scheduleProcessDelay(event.at_ms, () => { options.dispatchedEventIndexes.add(eventIndex); const terminal = getTerminal(); - const dispatchedAt = Math.max(0, Date.now() - started); + const dispatchedAt = Math.max( + 0, + monotonicProcessCaptureClock() - started, + ); let outcome: InteractionEvent["outcome"] = "dispatched"; if (terminal === undefined) outcome = "target_exited"; else { diff --git a/src/process/capture/ProcessCaptureLifecycle.test.ts b/src/process/capture/ProcessCaptureLifecycle.test.ts index 765635ccc..abad38ac2 100644 --- a/src/process/capture/ProcessCaptureLifecycle.test.ts +++ b/src/process/capture/ProcessCaptureLifecycle.test.ts @@ -5,6 +5,7 @@ import { parseProcessScenario } from "../../domain/process/processScenario.js"; import type { ProcessCaptureFinalization } from "../../domain/process/processCapture.js"; import { awaitTerminalExit } from "./ProcessCaptureLifecycle.js"; import type { ProcessTimer } from "./ProcessTimer.js"; +import { monotonicProcessCaptureClock } from "./ProcessCaptureClock.js"; type Delivery = ProcessCaptureFinalization["signals"][number]["delivery"]; @@ -79,8 +80,8 @@ const start = ( idle_timeout_ms: 10_000, ...scenario, }), - started: Date.now(), - lastOutput: () => Date.now(), + started: monotonicProcessCaptureClock(), + lastOutput: monotonicProcessCaptureClock, signal, signalTarget: fake.signalTarget, recordFinalization: (snapshot) => { diff --git a/src/process/capture/ProcessCaptureLifecycle.ts b/src/process/capture/ProcessCaptureLifecycle.ts index 2d2fcec00..ab9271dda 100644 --- a/src/process/capture/ProcessCaptureLifecycle.ts +++ b/src/process/capture/ProcessCaptureLifecycle.ts @@ -48,6 +48,7 @@ import { type ProcessCaptureTruncationDetails, } from "../../domain/process/processCaptureCoverage.js"; import { scheduleProcessInterval, type ProcessTimer } from "./ProcessTimer.js"; +import { monotonicProcessCaptureClock } from "./ProcessCaptureClock.js"; import type { FilesystemCheckpoint, @@ -266,10 +267,10 @@ export const settleProcessCaptureJournal = async ( quietMs = 25, maxWaitMs = 500, ): Promise => { - const deadline = Date.now() + maxWaitMs; + const deadline = monotonicProcessCaptureClock() + maxWaitMs; let observed = journal.length; - let lastChangeAt = Date.now(); - while (Date.now() < deadline) { + let lastChangeAt = monotonicProcessCaptureClock(); + while (monotonicProcessCaptureClock() < deadline) { await new Promise((resolve) => setTimeout(resolve, 4)); // A stalled event loop can leave timer and poll callbacks that were already // due queued behind this wake-up. Drain them before judging silence so a @@ -277,8 +278,8 @@ export const settleProcessCaptureJournal = async ( await setImmediate(); if (journal.length !== observed) { observed = journal.length; - lastChangeAt = Date.now(); - } else if (Date.now() - lastChangeAt >= quietMs) return; + lastChangeAt = monotonicProcessCaptureClock(); + } else if (monotonicProcessCaptureClock() - lastChangeAt >= quietMs) return; } }; @@ -337,6 +338,12 @@ export const buildCaptureResult = ( truncation_details: options.truncationDetails, limitations: [ "The executable digest is a prelaunch file sample; matching path metadata immediately after spawn does not prove an atomic operating-system image binding.", + ...(Date.parse(options.manifest.completed_at) < + Date.parse(options.manifest.started_at) + ? [ + "Wall-clock completion time precedes start time; elapsed capture timing is recorded separately and remains monotonic.", + ] + : []), "Process trees are sampled and may omit short-lived descendants.", ...(options.samplingPartial ? ["Process-tree sampling ended with an incomplete observation."] @@ -660,7 +667,7 @@ export const observeSettlement = async ( recordEvent("lifecycle", 1); return { state: "unverifiable", elapsed_ms: 0 }; } - const started = Date.now(); + const started = monotonicProcessCaptureClock(); let consecutiveEmpty = 0; let sawLive = false; let deadlineReached = false; @@ -678,19 +685,25 @@ export const observeSettlement = async ( [...observations.values()].some(({ state }) => state === "unverifiable") ) { recordEvent("lifecycle", 1); - return { state: "unverifiable", elapsed_ms: Date.now() - started }; + return { + state: "unverifiable", + elapsed_ms: monotonicProcessCaptureClock() - started, + }; } if ([...observations.values()].every(({ state }) => state === "empty")) { consecutiveEmpty += 1; if (consecutiveEmpty >= 2) { recordEvent("lifecycle", 1); - return { state: "quiesced", elapsed_ms: Date.now() - started }; + return { + state: "quiesced", + elapsed_ms: monotonicProcessCaptureClock() - started, + }; } } else { consecutiveEmpty = 0; sawLive = true; } - deadlineReached = Date.now() - started >= settleMs; + deadlineReached = monotonicProcessCaptureClock() - started >= settleMs; if (deadlineReached) break; await delay(50, undefined, { signal }); } @@ -698,8 +711,14 @@ export const observeSettlement = async ( // A lone empty observation proves neither liveness nor quiescence; only an // observed live member justifies alive_at_deadline. return sawLive - ? { state: "alive_at_deadline", elapsed_ms: Date.now() - started } - : { state: "unverifiable", elapsed_ms: Date.now() - started }; + ? { + state: "alive_at_deadline", + elapsed_ms: monotonicProcessCaptureClock() - started, + } + : { + state: "unverifiable", + elapsed_ms: monotonicProcessCaptureClock() - started, + }; }; /** Name why an attempt is unverified; a blank message never reaches the capture schema. */ @@ -786,7 +805,10 @@ export const awaitTerminalExit = async ({ if (finalizationStartedAt === undefined) return; const record: (typeof finalizationSignals)[number] = { signal: finalizationSignal, - sent_at_ms: Math.round(performance.now() - finalizationStartedAt), + sent_at_ms: Math.max( + 0, + monotonicProcessCaptureClock() - finalizationStartedAt, + ), delivery: "unverified" as FinalizationDelivery, reason: "delivery had not settled when the record was published", }; @@ -856,7 +878,7 @@ export const awaitTerminalExit = async ({ terminal.kill("SIGKILL"); return; } - finalizationStartedAt = performance.now(); + finalizationStartedAt = monotonicProcessCaptureClock(); attempt("SIGTERM"); }; terminal.onExit((exit) => { @@ -867,7 +889,7 @@ export const awaitTerminalExit = async ({ const finalizationElapsedMs = finalizationStartedAt === undefined ? undefined - : Math.round(performance.now() - finalizationStartedAt); + : Math.max(0, monotonicProcessCaptureClock() - finalizationStartedAt); recordEvent("lifecycle", 0); for (const [eventIndex, event] of scenario.events.entries()) { if (dispatchedEventIndexes.has(eventIndex)) continue; @@ -875,7 +897,10 @@ export const awaitTerminalExit = async ({ interactions.push({ sequence, scheduled_at_ms: event.at_ms, - dispatched_at_ms: Math.max(0, Date.now() - started), + dispatched_at_ms: Math.max( + 0, + monotonicProcessCaptureClock() - started, + ), type: event.type, data: event.type === "input" @@ -935,14 +960,20 @@ export const awaitTerminalExit = async ({ forceKill(); } else if (finalizationStartedAt !== undefined) { if ( - performance.now() - finalizationStartedAt >= + monotonicProcessCaptureClock() - finalizationStartedAt >= scenario.finalization_ms ) forceKill(); - } else if (Date.now() - started >= scenario.timeout_ms) { + } else if ( + monotonicProcessCaptureClock() - started >= + scenario.timeout_ms + ) { reason = "timeout"; terminateAtDeadline(); - } else if (Date.now() - lastOutput() >= scenario.idle_timeout_ms) { + } else if ( + monotonicProcessCaptureClock() - lastOutput() >= + scenario.idle_timeout_ms + ) { reason = "idle_timeout"; terminateAtDeadline(); } @@ -971,10 +1002,11 @@ const PROCESS_CLEANUP_VERIFICATION_INTERVAL_MS = 25; const verifyTokenOwnedProcessesUntilSettled = async ( verify: () => Promise, ): Promise => { - const deadline = Date.now() + PROCESS_CLEANUP_VERIFICATION_GRACE_MS; + const deadline = + monotonicProcessCaptureClock() + PROCESS_CLEANUP_VERIFICATION_GRACE_MS; let result = await verify(); while (!result.cleaned) { - const remaining = deadline - Date.now(); + const remaining = deadline - monotonicProcessCaptureClock(); if (remaining <= 0) return result; await new Promise((resolve) => setTimeout( @@ -1288,7 +1320,7 @@ export const captureTerminalFrames = (options: { } outputBytes += bytes; const atMs = normalizeProcessElapsedTime( - Date.now() - options.started, + monotonicProcessCaptureClock() - options.started, options.scenario.normalization.time_bucket_ms, ); const normalized = normalizeProcessText( diff --git a/src/process/capture/ProcessCaptureProgress.ts b/src/process/capture/ProcessCaptureProgress.ts index 83d5a51f5..205334db7 100644 --- a/src/process/capture/ProcessCaptureProgress.ts +++ b/src/process/capture/ProcessCaptureProgress.ts @@ -1,4 +1,5 @@ import type { ProcessCaptureCleanupReport } from "../../domain/process/processCapture.js"; +import { monotonicProcessCaptureClock } from "./ProcessCaptureClock.js"; /** * Progress port accepted by process capture. @@ -61,7 +62,7 @@ const formatProgressMessage = ( /** Serialize observed capture status through one rate-limited reporter. */ export const createProcessCaptureProgressTracker = ( progress: ProcessCaptureProgress | undefined, - now: () => number = Date.now, + now: () => number = monotonicProcessCaptureClock, ) => { const started = now(); let completed = 0; diff --git a/src/process/capture/ProcessHarness.ts b/src/process/capture/ProcessHarness.ts index 1033e68cd..847e512f8 100644 --- a/src/process/capture/ProcessHarness.ts +++ b/src/process/capture/ProcessHarness.ts @@ -70,6 +70,7 @@ import { observeProcessStartIdentity, signalProcessWithStartIdentity, } from "../ProcessOwnershipObservation.js"; +import { monotonicProcessCaptureClock } from "./ProcessCaptureClock.js"; interface StartedCaptureRuntime { readonly renderer: TerminalRenderer; @@ -268,7 +269,7 @@ const startCaptureRuntime = async ( options.signal, ); const startedAt = new Date(); - const started = Date.now(); + const started = monotonicProcessCaptureClock(); let lastOutput = started; renderer = createTerminalRenderer( scenario, @@ -296,7 +297,7 @@ const startCaptureRuntime = async ( ...options, terminal, started, - onOutput: () => (lastOutput = Date.now()), + onOutput: () => (lastOutput = monotonicProcessCaptureClock()), renderer, }); scheduleScenarioInteractions({ @@ -541,7 +542,7 @@ const completeCapture = async (options: { }, { name: "after_settlement", - at_ms: Math.max(0, Date.now() - runtime.started), + at_ms: Math.max(0, monotonicProcessCaptureClock() - runtime.started), files: after.files, effects: classifyFilesystemEffects(options.before, after), truncated: after.truncated, diff --git a/src/process/capture/ProcessSampling.ts b/src/process/capture/ProcessSampling.ts index 69e7dd343..59b67f577 100644 --- a/src/process/capture/ProcessSampling.ts +++ b/src/process/capture/ProcessSampling.ts @@ -6,6 +6,7 @@ import type { } from "../../domain/process/processCapture.js"; import { execFileOutput } from "../ExecFileOutput.js"; import { readProcessRunId } from "../ProcessOwnership.js"; +import { monotonicProcessCaptureClock } from "./ProcessCaptureClock.js"; const PROC_READ_CONCURRENCY = 64; interface ProcessRow { @@ -491,7 +492,7 @@ export const startProcessSampler = (options: { } return sampleProcesses({ rootPid, - elapsedMs: Date.now() - started, + elapsedMs: monotonicProcessCaptureClock() - started, limit: limit + 1, identities, signal: controller.signal, diff --git a/tests/boundary/filesystem/ghidraPspExtension.test.ts b/tests/boundary/filesystem/ghidraPspExtension.test.ts index 0b90bd194..65a4d1512 100644 --- a/tests/boundary/filesystem/ghidraPspExtension.test.ts +++ b/tests/boundary/filesystem/ghidraPspExtension.test.ts @@ -188,36 +188,6 @@ describe.skipIf(process.platform === "win32")( expect(missing.error.message).toContain("PSP_EXTENSION_UNAVAILABLE"); }); - it.each([ - { - form: "escaped key and value with colon separator", - properties: "n\\u0061me : ghidra-allegrex\nver\\u0073ion=12.1.3\n", - }, - { - form: "continued key and value lines", - properties: "name=ghidra-\\\n allegrex\nversion=12.1.\\\n 3\n", - }, - { - form: "duplicate keys use the last value", - properties: - "name=wrong\nname=ghidra-allegrex\nversion=12.1.4\nversion=12.1.3\n", - }, - ])( - "accepts Java Properties $form and hashes raw bytes", - async ({ properties }) => { - const { root } = await installation(); - const canonical = await inspectGhidraPspExtension(root, "12.1.3"); - const latin1Properties = Buffer.from( - `${properties}author=caf\u00e9\n`, - "latin1", - ); - await writeFile(join(root, "extension.properties"), latin1Properties); - const extension = await inspectGhidraPspExtension(root, "12.1.3"); - expect(extension.id).toBe("ghidra-allegrex"); - expect(extension.sha256).not.toBe(canonical.sha256); - }, - ); - it.each([ ["name=other\nversion=12.1.3\n", "does not match"], ["name=ghidra-allegrex\nversion=12.1.4\n", "does not match"], @@ -531,3 +501,38 @@ describe.skipIf(process.platform === "win32")( }); }, ); + +describe.skipIf(process.platform === "win32")( + "PSP extension property syntax", + () => { + it.each([ + { + form: "escaped key and value with colon separator", + properties: "n\\u0061me : ghidra-allegrex\nver\\u0073ion=12.1.3\n", + }, + { + form: "continued key and value lines", + properties: "name=ghidra-\\\n allegrex\nversion=12.1.\\\n 3\n", + }, + { + form: "duplicate keys use the last value", + properties: + "name=wrong\nname=ghidra-allegrex\nversion=12.1.4\nversion=12.1.3\n", + }, + ])( + "accepts Java Properties $form and hashes raw bytes", + async ({ properties }) => { + const { root } = await installation(); + const canonical = await inspectGhidraPspExtension(root, "12.1.3"); + const latin1Properties = Buffer.from( + `${properties}author=caf\u00e9\n`, + "latin1", + ); + await writeFile(join(root, "extension.properties"), latin1Properties); + const extension = await inspectGhidraPspExtension(root, "12.1.3"); + expect(extension.id).toBe("ghidra-allegrex"); + expect(extension.sha256).not.toBe(canonical.sha256); + }, + ); + }, +); diff --git a/tests/boundary/process/processCaptureLifecycle.test.ts b/tests/boundary/process/processCaptureLifecycle.test.ts index dcba0510f..c3dc21542 100644 --- a/tests/boundary/process/processCaptureLifecycle.test.ts +++ b/tests/boundary/process/processCaptureLifecycle.test.ts @@ -4,7 +4,7 @@ import { access, readFile, rm, symlink, writeFile } from "node:fs/promises"; import { dirname, join } from "node:path"; import { fileURLToPath } from "node:url"; import { promisify } from "node:util"; -import { expect } from "vitest"; +import { expect, vi } from "vitest"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { itWithCaptureCapability } from "./processCaptureCapability.js"; @@ -123,6 +123,93 @@ const terminateFixtureProcess = async (pid: number): Promise => { await waitForProcessExit(pid); }; +itWithCaptureCapability( + "keeps the process deadline bounded when the wall clock moves backward", + async () => { + const wallStart = Date.now(); + let wallReads = 0; + const now = vi.spyOn(Date, "now").mockImplementation(() => { + wallReads += 1; + return wallReads === 1 ? wallStart : wallStart - 60_000 + wallReads; + }); + const controller = new AbortController(); + const fallback = setTimeout(() => controller.abort(), 1_500); + try { + const result = await captureProcessScenario( + parseProcessScenario({ + executable: process.execPath, + arguments: [processFixture, "hang"], + working_directory: dirname(processFixture), + timeout_ms: 100, + idle_timeout_ms: 5_000, + }), + controller.signal, + ); + expect(result.ok).toBe(true); + if (!result.ok) throw result.error; + expect(result.value.exit.reason).toBe("timeout"); + } finally { + clearTimeout(fallback); + now.mockRestore(); + } + }, + 10_000, +); + +itWithCaptureCapability( + "preserves reversed wall-clock observations and monotonic frames across capture", + async () => { + vi.useFakeTimers({ toFake: ["Date"] }); + vi.setSystemTime(new Date("2026-01-02T00:00:00.000Z")); + try { + const result = await captureProcessScenario( + parseProcessScenario({ + executable: process.execPath, + arguments: [processFixture, "partial"], + working_directory: dirname(processFixture), + settle_ms: 0, + timeout_ms: 5_000, + idle_timeout_ms: 5_000, + }), + undefined, + process.platform, + process.env, + undefined, + undefined, + { + async report(update) { + if (update.phase === "running") + vi.setSystemTime(new Date("2026-01-01T00:00:00.000Z")); + }, + }, + ); + expect(result.ok).toBe(true); + if (!result.ok) throw result.error; + expect(result.value.frames.length).toBeGreaterThan(0); + expect(Date.parse(result.value.manifest.completed_at)).toBeLessThan( + Date.parse(result.value.manifest.started_at), + ); + expect(result.value.limitations).toContain( + "Wall-clock completion time precedes start time; elapsed capture timing is recorded separately and remains monotonic.", + ); + expect( + result.value.frames.map(({ at_ms }) => at_ms), + "capture-relative frame times stay ordered and nonnegative", + ).toEqual( + [...result.value.frames.map(({ at_ms }) => at_ms)].sort( + (left, right) => left - right, + ), + ); + expect(result.value.frames.every(({ at_ms }) => at_ms >= 0)).toBe(true); + expect(result.value.exit.reason).toBe("exited"); + expect(result.value.settlement.elapsed_ms).toBeGreaterThan(0); + } finally { + vi.useRealTimers(); + } + }, + 10_000, +); + itWithCaptureCapability( "cleans detached run-token children omitted by sampling and preserves an unowned neighbor", async () => { From e7882b34f4ac4a26f903c5f49264acc2c19bb562 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:12:17 +0800 Subject: [PATCH 066/159] fix(artifacts): honor cancellation at the output publication boundary --- src/artifacts/SafeOutputTree.ts | 33 ++++++++++------ .../extraction/ArtifactExtraction.ts | 2 +- src/browser/assets/PublishWebScripts.ts | 2 +- src/firmware/FirmwarePublication.ts | 2 +- src/javascript/recovery/WakaruProvider.ts | 2 +- .../safeOutputTreeCloseFailures.test.ts | 39 +++++++++++++++++++ ...actProvider.extractionCompleteness.test.ts | 39 ++++++++++++++++++- 7 files changed, 103 insertions(+), 16 deletions(-) diff --git a/src/artifacts/SafeOutputTree.ts b/src/artifacts/SafeOutputTree.ts index 4bb4cfe1a..8fa40876d 100644 --- a/src/artifacts/SafeOutputTree.ts +++ b/src/artifacts/SafeOutputTree.ts @@ -280,9 +280,11 @@ export class SafeOutputTree { } /** Sync the owned output tree and prevent further writes through this instance. */ - async commit(): Promise { + async commit(signal?: AbortSignal): Promise { this.#assertWritable(); + abortIfNeeded(signal); await this.#assertOwnedDirectories(); + abortIfNeeded(signal); // Windows has no directory fsync; file contents are already synced in write(). if (this.platform === "win32") { this.#published = true; @@ -315,6 +317,7 @@ export class SafeOutputTree { ); await output.handle.sync(); await parent.handle.sync(); + abortIfNeeded(signal); this.#published = true; } catch (cause: unknown) { sync = { kind: "failed", cause }; @@ -336,23 +339,26 @@ export class SafeOutputTree { closeFailure, this.#outputRoot, ); - if (sync.kind === "failed") - throw ArtifactReaderFailure.withCleanup( - new ArtifactReaderFailure( - "path", - "Could not durably sync extraction output", - { cause: sync.cause }, - ), - cleanup, - ); + if (sync.kind === "failed") { + const primary = isCancellationFailure(sync.cause) + ? sync.cause + : new ArtifactReaderFailure( + "path", + "Could not durably sync extraction output", + { cause: sync.cause }, + ); + throw ArtifactReaderFailure.withCleanup(primary, cleanup); + } throw ArtifactReaderFailure.withCleanup(closeFailure, cleanup); } - if (sync.kind === "failed") + if (sync.kind === "failed") { + if (isCancellationFailure(sync.cause)) throw sync.cause; throw new ArtifactReaderFailure( "path", "Could not durably sync extraction output", { cause: sync.cause }, ); + } } /** Remove only this operation's unsealed tree and verify absence. */ @@ -590,6 +596,11 @@ const abortIfNeeded = (signal?: AbortSignal): void => { ); }; +const isCancellationFailure = ( + cause: unknown, +): cause is ArtifactReaderFailure => + cause instanceof ArtifactReaderFailure && cause.reason === "cancelled"; + const isAlreadyExists = (cause: unknown): boolean => cause instanceof Error && "code" in cause && cause.code === "EEXIST"; diff --git a/src/artifacts/extraction/ArtifactExtraction.ts b/src/artifacts/extraction/ArtifactExtraction.ts index ee92954eb..8cad3543e 100644 --- a/src/artifacts/extraction/ArtifactExtraction.ts +++ b/src/artifacts/extraction/ArtifactExtraction.ts @@ -249,7 +249,7 @@ const materializeSelection = async ({ compareUnicodeCodePoints(left.relative_path, right.relative_path), ); completed = createExtractionResult(input, inventory, selected, extracted); - await output.commit(); + await output.commit(signal); return completed; } catch (cause: unknown) { const cleanupFailures = await releaseExtractionOwners( diff --git a/src/browser/assets/PublishWebScripts.ts b/src/browser/assets/PublishWebScripts.ts index 2db5d7dcf..ba886f00e 100644 --- a/src/browser/assets/PublishWebScripts.ts +++ b/src/browser/assets/PublishWebScripts.ts @@ -113,7 +113,7 @@ export const publishWebScripts = async ( options.signal, ); options.signal?.throwIfAborted(); - await tree.commit(); + await tree.commit(options.signal); return result; } catch (cause: unknown) { if (cause instanceof SafeOutputTreeCreationFailure) tree = cause.tree; diff --git a/src/firmware/FirmwarePublication.ts b/src/firmware/FirmwarePublication.ts index bc53084b1..b2bc4ddd5 100644 --- a/src/firmware/FirmwarePublication.ts +++ b/src/firmware/FirmwarePublication.ts @@ -96,7 +96,7 @@ export const publishFirmwareExtraction = async (context: { published_bytes: files.reduce((sum, file) => sum + file.size, 0), }); context.signal.throwIfAborted(); - await tree.commit(); + await tree.commit(context.signal); return result; } catch (cause: unknown) { if (cause instanceof SafeOutputTreeCreationFailure) tree = cause.tree; diff --git a/src/javascript/recovery/WakaruProvider.ts b/src/javascript/recovery/WakaruProvider.ts index b1260bcd1..88a51a74b 100644 --- a/src/javascript/recovery/WakaruProvider.ts +++ b/src/javascript/recovery/WakaruProvider.ts @@ -139,7 +139,7 @@ export class WakaruProvider implements JavaScriptRecoveryPort { await rm(root, { recursive: true, force: true }); root = undefined; checkRecoveryDeadline(deadline, options?.signal); - await tree.commit(); + await tree.commit(options?.signal); } catch (cause: unknown) { failed = { cause }; } diff --git a/tests/boundary/filesystem/safeOutputTreeCloseFailures.test.ts b/tests/boundary/filesystem/safeOutputTreeCloseFailures.test.ts index ee285dbd5..d835b1e46 100644 --- a/tests/boundary/filesystem/safeOutputTreeCloseFailures.test.ts +++ b/tests/boundary/filesystem/safeOutputTreeCloseFailures.test.ts @@ -10,6 +10,8 @@ const injectedFilesystemFailures = vi.hoisted(() => ({ closeNext: false, blockCloseRetries: false, chmodNext: false, + syncAbortAfter: undefined as AbortController | undefined, + syncCount: 0, rootIdentityPath: undefined as string | undefined, })); @@ -44,6 +46,15 @@ vi.mock("node:fs/promises", async (importOriginal) => { await close(); }; } + if (injectedFilesystemFailures.syncAbortAfter !== undefined) { + const sync = handle.sync.bind(handle); + handle.sync = async () => { + await sync(); + injectedFilesystemFailures.syncCount += 1; + if (injectedFilesystemFailures.syncCount === 2) + injectedFilesystemFailures.syncAbortAfter?.abort(); + }; + } return handle; }, }; @@ -82,6 +93,34 @@ describe("safe artifact output close failures", () => { }, ); + it.skipIf(process.platform === "win32")( + "keeps the tree unpublished when cancellation arrives during commit durability", + async () => { + const parent = await createTestTempDirectory("rea-safe-output-cancel-"); + const output = join(parent, "unpublished"); + const tree = await SafeOutputTree.create(output); + const bytes = Buffer.from("pending output"); + await tree.write("file.txt", Readable.from(bytes), { + sha256: createHash("sha256").update(bytes).digest("hex"), + bytes: bytes.byteLength, + }); + const controller = new AbortController(); + injectedFilesystemFailures.syncCount = 0; + injectedFilesystemFailures.syncAbortAfter = controller; + try { + await expect(tree.commit(controller.signal)).rejects.toMatchObject({ + reason: "cancelled", + }); + expect(tree.published).toBe(false); + } finally { + injectedFilesystemFailures.syncAbortAfter = undefined; + injectedFilesystemFailures.syncCount = 0; + } + expect(await tree.rollback()).toMatchObject({ status: "complete" }); + await expect(access(output)).rejects.toMatchObject({ code: "ENOENT" }); + }, + ); + it.skipIf(process.platform === "win32")( "preserves setup failure when closing the staging handle also fails", async () => { diff --git a/tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts b/tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts index d42762601..0bcf12bb2 100644 --- a/tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts +++ b/tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts @@ -3,7 +3,7 @@ import { access, mkdir, readFile, rm, writeFile } from "node:fs/promises"; import { join } from "node:path"; import { createPackage, createPackageWithOptions } from "@electron/asar"; -import { expect, it } from "vitest"; +import { expect, it, vi } from "vitest"; import { ArtifactProvider } from "../../../../src/artifacts/ArtifactProvider.js"; import { @@ -13,6 +13,7 @@ import { import { artifactExtractionResultSchema } from "../../../../src/domain/artifactGraph.js"; import { projectAnalysisError } from "../../../../src/domain/analysisErrorProjection.js"; import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js"; +import { DirectoryArtifactReader } from "../../../../src/artifacts/DirectoryArtifactReader.js"; it.each(["file", "directory"] as const)( "rolls back when an inventoried %s disappears before extraction", @@ -61,6 +62,42 @@ it("still refuses a regular file added after inventory and rolls back", async () await expect(access(output)).rejects.toThrow(); }); +it("rolls back when cancellation arrives while closing the materialization reader", async () => { + const root = await createTestTempDirectory("rea-extract-cancel-close-"); + const source = join(root, "source"); + await mkdir(source); + await writeFile(join(source, "payload.txt"), "complete payload\n"); + const snapshot = await scanArtifactInventory(source); + const output = join(root, "output"); + const controller = new AbortController(); + const close = DirectoryArtifactReader.prototype.close; + const closeSpy = vi + .spyOn(DirectoryArtifactReader.prototype, "close") + .mockImplementation(async function (this: DirectoryArtifactReader) { + controller.abort(); + await close.call(this); + }); + try { + await expect( + materializeArtifactInventory( + { + inputPath: source, + inputFormat: "asar", + outputRoot: output, + environment: process.env, + integrityPolicy: "fail", + }, + source, + snapshot, + controller.signal, + ), + ).rejects.toMatchObject({ reason: "cancelled" }); + await expect(access(output)).rejects.toThrow(); + } finally { + closeSpy.mockRestore(); + } +}); + it("preserves every equal-content occurrence and copies a nested ASAR as its containing file", async () => { const root = await createTestTempDirectory("rea-extract-nested-"); const source = join(root, "source"); From bf34d0d2fc3cc22076f49d682e74a0b71aa1db18 Mon Sep 17 00:00:00 2001 From: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> Date: Tue, 13 Oct 2026 00:13:12 -0700 Subject: [PATCH 067/159] perf(native): add opt-in Mach-O summary delivery and retained pages (#2634) * perf(native): add opt-in Mach-O summary delivery and retained pages inspect_macho returns every import and export inline. For VS Code's Electron Framework one MCP call carries 2.5 MB of text (about 828k o200k tokens), and the same again as structuredContent; the MCP response budget admits it. Agents asking for slice identity, UUID or dependency counts paid that cost on every first call. Add the established opt-in detail selection: omitted or complete keeps the existing complete result and Evidence identity, while summary retains the complete Evidence and returns slice identity with per-collection returned/total/exhaustive counts (956 tokens). inspect_analysis_view kind macho pages the original rows of any collection or the command provenance, repeating the producer's own total and exhaustive flag so page exhaustion is never read as absence. Delivery detail never reaches the native tools or the source Evidence ID: registerEvidenceTools gains a parametersForEvidence hook, so the summary's parent is the same Evidence as a complete call. Co-Authored-By: Claude Opus 5.5 * fix(mcp): refuse unretainable summary delivery before analysis A server without Evidence retention (a fixed analysis source) ran the complete inspect_macho or inspect_artifact analysis for detail summary and only then refused it in summarizeRetainedAnalysis, paying the very cost summary delivery exists to avoid. registerEvidenceTools gains a preflight hook; both registrations check retention there, as the WASM, EVM, managed, pcap and capture handlers already do before dispatch. Co-Authored-By: Claude Opus 5.5 * fix(native): keep whole rows in Mach-O architecture and segment pages The page item union listed plain-object row arrays, and zod returns the first union member that parses while dropping unknown keys. Entrypoint rows declare only file_offset, so every architecture and segment row (which also carry a numeric file_offset) matched it and lost its name, CPU fields, permissions and sections. Rows now use exact key sets, and imports/exports share one member. The MCP test pages every facet of a real /bin/ls and compares it with the unparsed complete rows. Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 --- .../references/native-and-artifacts.md | 8 + docs/mcp-contracts.md | 17 +- src/contracts/analysisViewToolContracts.ts | 4 +- src/contracts/native/nativeToolContracts.ts | 12 +- src/contracts/toolOutputSchemaGroups.ts | 5 +- src/domain/analysisView/analysisView.ts | 29 +- src/domain/analysisView/machoView.test.ts | 246 +++++++++++++++++ src/domain/analysisView/machoView.ts | 247 ++++++++++++++++++ src/server/createServer.ts | 5 +- src/server/registerArtifactTools.ts | 18 +- src/server/registerEvidenceTools.ts | 20 +- src/server/registerNativeTools.ts | 54 ++++ .../mcp/nativeMachoSummaryMcp.test.ts | 216 +++++++++++++++ 13 files changed, 864 insertions(+), 17 deletions(-) create mode 100644 src/domain/analysisView/machoView.test.ts create mode 100644 src/domain/analysisView/machoView.ts create mode 100644 src/server/registerNativeTools.ts create mode 100644 tests/boundary/mcp/nativeMachoSummaryMcp.test.ts diff --git a/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md b/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md index 58c1671b6..71fe3e950 100644 --- a/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md +++ b/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md @@ -83,6 +83,14 @@ names, the native API record, and value-flow summaries. Pseudocode offsets and l code units; use the returned `next_offset` for continuation. Provider limitations and unavailable facts remain explicit; the complete parent Evidence is retained. +`inspect_macho` on a large framework returns megabytes of imports and exports. +Select `detail: "summary"` for slice identity, UUID and per-collection counts, +then page rows with `view: {kind: "macho", facet: "exports", offset: 0, limit: 64}` +and `normalized_result.parent_evidence_id`. Facets are entrypoints, +architectures, build_metadata, load_commands, dependencies, imports, exports, +segments and provenance. `source_exhaustive: false` means unreturned symbols +may exist. + To see which functions or Objective-C methods a Mach-O actually calls in one run, use `observe_native_calls` with explicit breakpoints and a bounded `duration_ms`/`max_events`. It launches a new process under LLDB, so the target diff --git a/docs/mcp-contracts.md b/docs/mcp-contracts.md index c4169dd73..f3f7892f6 100644 --- a/docs/mcp-contracts.md +++ b/docs/mcp-contracts.md @@ -147,8 +147,8 @@ after target content was read. ## Retained application Evidence inputs -`analyze_javascript_application`, `inspect_binary_layout`, `inspect_artifact`, and -`inspect_wasm_artifact` accept `detail`: +`analyze_javascript_application`, `inspect_binary_layout`, `inspect_artifact`, +`inspect_wasm_artifact`, and `inspect_macho` accept `detail`: `complete` (default) returns the complete analysis Evidence; `summary` records that complete Evidence in the session and returns the derived `inspect_analysis_view` summary Evidence instead. Its @@ -218,6 +218,19 @@ attempting oversized delivery, and returns the same summary with an exact `parent_evidence_id` for later export or facet reads. Delivery detail does not reach WABT or change the original source Evidence ID. +Completed `inspect_macho` Evidence supports a summary and +`view: {kind: "macho", facet: "exports", offset: 0, limit: 64}`. Facets are +`entrypoints`, `architectures`, `build_metadata`, `load_commands`, +`dependencies`, `imports`, `exports`, `segments`, and `provenance`; pages return +the original rows. The summary keeps the selected slice, header fields and UUID, +and replaces each collection with its returned count, reported total and +`exhaustive` flag. Each page repeats the paged collection's `source_total` and +`source_exhaustive`; exhausting a non-exhaustive collection does not establish +that unreturned symbols are absent. Large frameworks return megabytes of +symbols (Electron Framework: 2.5 MB of imports and exports, 3.3 KB as a +summary), so select `detail: "summary"` on the first call. Delivery detail +never reaches the native tools or changes the source Evidence ID. + `trace_application_feature`, `trace_javascript_semantics`, `compare_application_versions`, `compare_source_to_bundle`, and `compare_javascript_export_shapes` accept complete inline application Evidence diff --git a/src/contracts/analysisViewToolContracts.ts b/src/contracts/analysisViewToolContracts.ts index aba5b9a93..7d023752d 100644 --- a/src/contracts/analysisViewToolContracts.ts +++ b/src/contracts/analysisViewToolContracts.ts @@ -9,14 +9,14 @@ import { evidenceResultOf } from "./toolOutputSchemaPrimitives.js"; const RETAINED_EXAMPLE_ID = `ev_${"a".repeat(64)}`; -/** Selected views of completed layout, JavaScript, native, inventory, WASM and web network Evidence. */ +/** Selected views of completed layout, JavaScript, native, inventory, WASM, Mach-O and web network Evidence. */ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [ { name: "inspect_analysis_view", ...toolContractMetadata("inspect_analysis_view"), kind: "application", description: - 'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_evm_interface, inspect_managed_artifact, inspect_android_package, inspect_dart_aot or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. For layout, JavaScript, inventory, WASM, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Dart AOT summary preserves target/ABI/library identity, snapshot headers and pool lower bounds. String pages use kind dart-aot, collection package_uris/dart_uris/class_like_tokens; source coverage remains partial when the producer says partial. Native function Evidence requires kind native: view: {kind: "native", facet: "procedure"}. Facets select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', + 'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_macho, inspect_evm_interface, inspect_managed_artifact, inspect_android_package, inspect_dart_aot or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: ""}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: }. For layout, JavaScript, inventory, WASM, Mach-O, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. Mach-O views use kind macho and facet entrypoints/architectures/build_metadata/load_commands/dependencies/imports/exports/segments/provenance; source_exhaustive false means unreturned rows may exist. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Dart AOT summary preserves target/ABI/library identity, snapshot headers and pool lower bounds. String pages use kind dart-aot, collection package_uris/dart_uris/class_like_tokens; source coverage remains partial when the producer says partial. Native function Evidence requires kind native: view: {kind: "native", facet: "procedure"}. Facets select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.', inputSchema: inspectAnalysisViewInputSchema, outputSchema: evidenceResultOf(analysisViewResultSchema), examples: [ diff --git a/src/contracts/native/nativeToolContracts.ts b/src/contracts/native/nativeToolContracts.ts index 093add61e..16101368b 100644 --- a/src/contracts/native/nativeToolContracts.ts +++ b/src/contracts/native/nativeToolContracts.ts @@ -11,6 +11,7 @@ import { jsonValueSchema } from "../../domain/jsonValue.js"; import { localPathStringSchema } from "../../domain/localPath.js"; import { toolContractMetadata } from "../toolEffects.js"; import { requireOutputSchema } from "../toolOutputSchemaPrimitives.js"; +import { analysisDetailSchema } from "../analysisDetail.js"; const examples: Readonly>>> = { inspect_macho: {}, @@ -94,8 +95,15 @@ export const NATIVE_TOOL_CONTRACTS = [ ), native( "inspect_macho", - "Inspect Mach-O slices, load commands, imports, exports, dependencies, build metadata, segments, sections, permissions, and exact command provenance without launching Hopper.", - z.object({}), + "Inspect Mach-O slices, load commands, imports, exports, dependencies, build metadata, segments, sections, permissions, and exact command provenance without launching Hopper. Large frameworks return megabytes of symbols; detail summary retains the complete Evidence and returns slice identity with per-collection counts, then page rows with inspect_analysis_view kind macho.", + z.object({ + detail: analysisDetailSchema + .unwrap() + .optional() + .describe( + "Omitted or complete returns the complete inspection Evidence. summary retains that complete Evidence in this session and returns its inspect_analysis_view summary; use normalized_result.parent_evidence_id for macho pages.", + ), + }), ), native( "inspect_signature", diff --git a/src/contracts/toolOutputSchemaGroups.ts b/src/contracts/toolOutputSchemaGroups.ts index e828486ba..174339bdf 100644 --- a/src/contracts/toolOutputSchemaGroups.ts +++ b/src/contracts/toolOutputSchemaGroups.ts @@ -1,6 +1,7 @@ import { z } from "zod"; import { artifactInventorySummaryViewResultSchema, + machoSummaryViewResultSchema, managedArtifactSummaryViewResultSchema, } from "../domain/analysisView/analysisView.js"; import { evidenceBundleSummarySchema } from "../domain/evidenceRecordSummary.js"; @@ -381,7 +382,9 @@ export const nativeOutputSchemas: Readonly> = { observe_native_ui: evidenceResultOf(nativeUiResultSchema), capture_native_ui_scenario: evidenceResultOf(nativeUiResultSchema), observe_native_calls: evidenceResultOf(nativeCallObservationResultSchema), - inspect_macho: evidenceResultOf(inspectMachoSchema), + inspect_macho: evidenceResultOf( + z.union([inspectMachoSchema, machoSummaryViewResultSchema]), + ), inspect_signature: evidenceResultOf(inspectSignatureSchema), inspect_plist: evidenceResultOf(inspectPlistSchema), list_architectures: evidenceResultOf(listArchitecturesSchema), diff --git a/src/domain/analysisView/analysisView.ts b/src/domain/analysisView/analysisView.ts index 6cc5ae8a3..7f985f349 100644 --- a/src/domain/analysisView/analysisView.ts +++ b/src/domain/analysisView/analysisView.ts @@ -55,6 +55,13 @@ import { wasmViewItemSchema, wasmSummarySchema, } from "./wasmArtifactView.js"; +import { + machoSummarySchema, + machoViewContextSchema, + machoViewItemSchema, + machoViewRequestSchema, + projectMachoView, +} from "./machoView.js"; import { projectWebNetworkCaptureView, captureViewRequestSchema, @@ -196,6 +203,7 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [ nativeViewSchema, inventoryViewSchema, wasmViewRequestSchema, + machoViewRequestSchema, evmViewRequestSchema, managedViewRequestSchema, androidViewRequestSchema, @@ -315,6 +323,7 @@ const viewResultBase = { "inspect_artifact", "inventory_artifact", "inspect_wasm_artifact", + "inspect_macho", "inspect_evm_interface", "inspect_managed_artifact", "inspect_web_network_capture", @@ -364,6 +373,13 @@ export const wasmArtifactSummaryViewResultSchema = z.strictObject({ summary: wasmSummarySchema, }); +/** Exact slice identity and collection counts for Mach-O's selected delivery. */ +export const machoSummaryViewResultSchema = z.strictObject({ + ...summaryViewResultBase, + parent_operation: z.literal("inspect_macho"), + summary: machoSummarySchema, +}); + /** Exact inferred-interface summary for EVM's caller-selected delivery. */ export const evmInterfaceSummaryViewResultSchema = z.strictObject({ ...summaryViewResultBase, @@ -425,6 +441,7 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ javascriptSummarySchema, inventorySummarySchema, wasmSummarySchema, + machoSummarySchema, evmSummarySchema, managedSummarySchema, captureSummarySchema, @@ -487,6 +504,14 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [ item: wasmViewItemSchema, ...viewResultBase, }), + z.strictObject({ + kind: z.literal("macho"), + view: machoViewRequestSchema, + item: machoViewItemSchema, + ...viewResultBase, + ...machoViewContextSchema.shape, + parent_operation: z.literal("inspect_macho"), + }), z.strictObject({ kind: z.literal("evm"), view: evmViewRequestSchema, @@ -583,6 +608,7 @@ export type UnsignedAnalysisView = "view_digest" > | Omit, "view_digest"> + | Omit, "view_digest"> | Omit, "view_digest"> | Omit, "view_digest"> | Omit, "view_digest"> @@ -646,7 +672,7 @@ const unsupportedParent = ( new AnalysisUnsupportedTargetError( INSPECT_ANALYSIS_VIEW_OPERATION, parentArtifactPath(parent), - `Parent Evidence operation ${JSON.stringify(parent.operation)} does not support analysis views. Use completed binary layout, JavaScript application, native function, artifact inventory/inspection, WASM, EVM, managed artifact, Android package, Dart AOT, or web network capture Evidence.`, + `Parent Evidence operation ${JSON.stringify(parent.operation)} does not support analysis views. Use completed binary layout, JavaScript application, native function, artifact inventory/inspection, WASM, Mach-O, EVM, managed artifact, Android package, Dart AOT, or web network capture Evidence.`, ); const parentArtifactPath = (parent: AnalysisViewParent): string => { @@ -673,6 +699,7 @@ const parentArtifactPath = (parent: AnalysisViewParent): string => { /** Named source owners for views that need no extra decoder or accumulator. */ const retainedViewProjectors = new Map([ ["inspect_wasm_artifact", projectWasmArtifactView], + ["inspect_macho", projectMachoView], ["inspect_evm_interface", projectEvmInterfaceView], ["inspect_managed_artifact", projectManagedArtifactView], ["inspect_android_package", projectAndroidPackageView], diff --git a/src/domain/analysisView/machoView.test.ts b/src/domain/analysisView/machoView.test.ts new file mode 100644 index 000000000..2018438be --- /dev/null +++ b/src/domain/analysisView/machoView.test.ts @@ -0,0 +1,246 @@ +import { describe, expect, it } from "vitest"; + +import { + projectAnalysisView, + type AnalysisViewParent, +} from "./analysisView.js"; + +const symbol = (name: string) => ({ + name, + address: null, + weak: null, + reexport: null, + source: "libSystem", +}); +const coverage = (items: T[], exhaustive = true) => ({ + items, + total: exhaustive ? items.length : null, + exhaustive, + limitations: exhaustive ? [] : ["nm output may omit stripped symbols."], +}); +const report = { + format: "mach-o", + selected_architecture: { name: "arm64", container_file_offset: 0 }, + endian: null, + word_size: 64, + file_type: "2", + flags: ["0x00200085"], + uuid: "4C4C44E9-5555-3144-A1E9-D858E24A1CCE", + entrypoints: coverage([{ file_offset: 16384 }]), + architectures: coverage([ + { + name: "arm64", + cpu_type: "16777228", + cpu_subtype: "0", + file_offset: 0, + size: 4096, + alignment: 14, + }, + ]), + build_metadata: coverage([ + { platform: "1", minimum_os: "14.0", sdk: "15.0", tools: [] }, + ]), + load_commands: coverage([{ index: 0, kind: "LC_MAIN", fields: {} }]), + dependencies: coverage([ + { + path: "/usr/lib/libSystem.B.dylib", + kind: "LC_LOAD_DYLIB", + current_version: "1351.0.0", + compatibility_version: "1.0.0", + }, + ]), + imports: coverage( + ["_malloc", "_free", "_open"].map((name) => symbol(name)), + false, + ), + exports: coverage([symbol("__mh_execute_header")]), + segments: coverage([ + { + name: "__TEXT", + vm_address: "0x100000000", + vm_size: 16384, + file_offset: 0, + file_size: 16384, + maximum_permissions: { + read: true, + write: false, + execute: true, + raw: "5", + }, + initial_permissions: { + read: true, + write: false, + execute: true, + raw: "5", + }, + sections: coverage([]), + }, + ]), + provenance: [ + { + tool: "nm", + command: ["/usr/bin/nm", "$ARTIFACT"], + tool_version: null, + version_reason: "fixture", + executable_sha256: "a".repeat(64), + exit: { code: 0, signal: null }, + stdout_bytes: 10, + stderr_bytes: 0, + }, + ], + limitations: ["Imports and exports combine dyld_info and nm."], +}; +const parent = ( + normalizedResult: unknown = report, + subject: AnalysisViewParent["subject"] = { + name: "tool", + digest: { sha256: "b".repeat(64) }, + format: "mach-o", + architecture: "arm64", + local_path: "/bin/tool", + }, +): AnalysisViewParent => ({ + evidenceId: `ev_${"c".repeat(64)}`, + operation: "inspect_macho", + normalizedResult, + subject, + limitations: ["Parent limitation."], +}); + +describe("Mach-O analysis views", () => { + it("summarizes slice identity and collection counts without rows", () => { + const projected = projectAnalysisView(parent(), { kind: "summary" }); + if (!projected.ok) throw projected.error; + expect(projected.value).toMatchObject({ + kind: "summary", + parent_operation: "inspect_macho", + artifact: { path: "/bin/tool", sha256: "b".repeat(64) }, + summary: { + selected_architecture: { name: "arm64", container_file_offset: 0 }, + uuid: report.uuid, + collections: { + imports: { returned: 3, total: null, exhaustive: false }, + exports: { returned: 1, total: 1, exhaustive: true }, + segments: { returned: 1, total: 1, exhaustive: true }, + }, + provenance_count: 1, + limitation_count: 3, + }, + coverage: { status: "complete-within-view", examined: 10, total: 10 }, + limitations: [ + "Imports and exports combine dyld_info and nm.", + "Parent limitation.", + "nm output may omit stripped symbols.", + ], + unknowns: [expect.stringContaining("exhaustive imports;")], + }); + expect(JSON.stringify(projected.value)).not.toContain("_malloc"); + }); + + it("pages original rows with the producer's own coverage", () => { + const first = projectAnalysisView(parent(), { + kind: "macho", + facet: "imports", + offset: 1, + limit: 1, + }); + if (!first.ok) throw first.error; + expect(first.value).toMatchObject({ + kind: "macho", + item: [symbol("_free")], + source_total: null, + source_exhaustive: false, + coverage: { status: "page", examined: 1, total: 3, next_offset: 2 }, + }); + expect(first.value.limitations).toContain( + "nm output may omit stripped symbols.", + ); + // Rows whose keys include an earlier row shape's keys stay whole. + for (const facet of ["architectures", "segments"] as const) { + const page = projectAnalysisView(parent(), { + kind: "macho", + facet, + offset: 0, + limit: 64, + }); + if (!page.ok) throw page.error; + expect(page.value.kind === "macho" && page.value.item).toEqual( + report[facet].items, + ); + } + const provenance = projectAnalysisView(parent(), { + kind: "macho", + facet: "provenance", + offset: 0, + limit: 64, + }); + if (!provenance.ok) throw provenance.error; + expect(provenance.value).toMatchObject({ + item: report.provenance, + source_total: 1, + source_exhaustive: true, + coverage: { exhausted: true, next_offset: null }, + }); + const past = projectAnalysisView(parent(), { + kind: "macho", + facet: "exports", + offset: 5, + limit: 64, + }); + if (!past.ok) throw past.error; + expect(past.value.coverage).toMatchObject({ + status: "empty", + exhausted: true, + }); + }); + + it("keeps a missing subject unknown instead of inventing identity", () => { + const projected = projectAnalysisView(parent(report, null), { + kind: "summary", + }); + if (!projected.ok) throw projected.error; + expect(projected.value.artifact).toEqual({ path: "", sha256: null }); + expect(projected.value.unknowns).toContain( + "The parent has no recorded subject, so the inspected artifact path and SHA-256 are unknown.", + ); + }); + + it("rejects other view kinds and malformed retained results", () => { + const wrongKind = projectAnalysisView(parent(), { + kind: "wasm", + facet: "exports", + offset: 0, + limit: 64, + }); + if (wrongKind.ok) throw new Error("expected a view-kind refusal"); + expect(wrongKind.error).toMatchObject({ + _tag: "AnalysisInputError", + issues: [ + { + path: ["view"], + message: "Mach-O inspection supports summary and macho views.", + }, + ], + }); + const contradictory = projectAnalysisView( + parent(report, { + name: "tool", + digest: { sha256: "b".repeat(64) }, + format: "elf", + architecture: null, + local_path: "/bin/tool", + }), + { kind: "summary" }, + ); + if (contradictory.ok) throw new Error("expected a subject refusal"); + expect(contradictory.error).toMatchObject({ + issues: [{ path: ["source", "subject"] }], + }); + const malformed = projectAnalysisView( + parent({ ...report, imports: { items: "x" } }), + { kind: "summary" }, + ); + expect(malformed.ok).toBe(false); + if (!malformed.ok) expect(malformed.error._tag).toBe("AnalysisInputError"); + }); +}); diff --git a/src/domain/analysisView/machoView.ts b/src/domain/analysisView/machoView.ts new file mode 100644 index 000000000..1a64d23aa --- /dev/null +++ b/src/domain/analysisView/machoView.ts @@ -0,0 +1,247 @@ +import { z } from "zod"; +import type { AnalysisError } from "../analysisErrorBase.js"; +import { uniqueSorted } from "../canonicalOrdering.js"; +import { isMachOFormat } from "../binaryTargetTypes.js"; +import { analysisInputErrorFromIssues } from "../inputIssueProjection.js"; +import { inspectMachoSchema } from "../native/nativeInspection.js"; +import { err, ok, type Result } from "../result.js"; +import { + analysisViewInputError, + type AnalysisViewParent, + type AnalysisViewRequest, + type UnsignedAnalysisView, +} from "./analysisView.js"; +import { + completeWithinViewCoverage, + pageViewCoverage, +} from "./analysisViewCoverage.js"; + +const MACHO_COLLECTIONS = [ + "entrypoints", + "architectures", + "build_metadata", + "load_commands", + "dependencies", + "imports", + "exports", + "segments", +] as const; +type MachoCollection = (typeof MACHO_COLLECTIONS)[number]; + +/** Original inspect_macho rows; offsets count rows in the retained collection. */ +export const machoViewRequestSchema = z.strictObject({ + kind: z.literal("macho"), + facet: z.enum([...MACHO_COLLECTIONS, "provenance"]), + offset: z.number().int().nonnegative().default(0), + limit: z.number().int().positive().default(64), +}); + +const { shape } = inspectMachoSchema; +/** + * Union members are tried in order and plain objects drop unknown keys, so a + * segment or architecture row would otherwise match the entrypoint row and + * lose every field but file_offset. Exact key sets keep each page's rows whole. + */ +const exactRows = ( + rows: z.ZodArray>, +) => z.array(z.strictObject(rows.element.shape)); +export const machoViewItemSchema = z.union([ + exactRows(shape.entrypoints.shape.items), + exactRows(shape.architectures.shape.items), + exactRows(shape.build_metadata.shape.items), + exactRows(shape.load_commands.shape.items), + exactRows(shape.dependencies.shape.items), + // Imports and exports share one row shape. + exactRows(shape.imports.shape.items), + exactRows(shape.segments.shape.items), + exactRows(shape.provenance), +]); + +/** The producer's own coverage of the paged collection, not of this page. */ +export const machoViewContextSchema = z.strictObject({ + source_total: z.number().int().nonnegative().nullable(), + source_exhaustive: z.boolean(), +}); + +const collectionCountSchema = z.strictObject({ + returned: z.number().int().nonnegative(), + total: z.number().int().nonnegative().nullable(), + exhaustive: z.boolean(), +}); + +/** Slice identity and per-collection counts without the collections themselves. */ +export const machoSummarySchema = z.strictObject({ + format: shape.format, + selected_architecture: shape.selected_architecture, + endian: shape.endian, + word_size: shape.word_size, + file_type: shape.file_type, + flags: shape.flags, + uuid: shape.uuid, + collections: z.strictObject( + Object.fromEntries( + MACHO_COLLECTIONS.map((name) => [name, collectionCountSchema]), + ) as Record, + ), + provenance_count: z.number().int().nonnegative(), + limitation_count: z.number().int().nonnegative(), +}); + +type MachoReport = z.output; + +/** Admit a well-formed report whose recorded subject, if any, is a Mach-O. */ +const parseMachoReport = ( + parent: AnalysisViewParent, +): Result => { + const parsed = inspectMachoSchema.safeParse(parent.normalizedResult); + if (!parsed.success) + return err( + analysisInputErrorFromIssues( + "inspect_analysis_view", + parsed.error.issues.map((issue) => ({ + ...issue, + path: ["source", "normalized_result", ...issue.path], + })), + { source: { normalized_result: parent.normalizedResult } }, + { cause: parsed.error }, + ), + ); + if (parent.subject != null && !isMachOFormat(parent.subject.format)) + return err( + analysisViewInputError([ + { + path: ["source", "subject"], + reason: "invalid_value", + message: + "Mach-O parent Evidence subject must identify a thin or universal Mach-O.", + }, + ]), + ); + return ok(parsed.data); +}; + +/** Counts replace rows; every collection caveat stays in the limitations. */ +const summarizeMacho = ( + report: MachoReport, + parentLimitations: readonly string[], +) => { + const limitations = uniqueSorted([ + ...parentLimitations, + ...report.limitations, + ...MACHO_COLLECTIONS.flatMap((name) => report[name].limitations), + ]); + const collections = Object.fromEntries( + MACHO_COLLECTIONS.map((name) => [ + name, + { + returned: report[name].items.length, + total: report[name].total, + exhaustive: report[name].exhaustive, + }, + ]), + ) as Record>; + return { + limitations, + returned: MACHO_COLLECTIONS.reduce( + (sum, name) => sum + report[name].items.length, + 0, + ), + summary: { + format: report.format, + selected_architecture: report.selected_architecture, + endian: report.endian, + word_size: report.word_size, + file_type: report.file_type, + flags: report.flags, + uuid: report.uuid, + collections, + provenance_count: report.provenance.length, + limitation_count: limitations.length, + }, + }; +}; + +/** Project retained inspect_macho Evidence without rerunning native tools. */ +export const projectMachoView = ( + parent: AnalysisViewParent, + view: AnalysisViewRequest, +): Result => { + if (view.kind !== "summary" && view.kind !== "macho") + return err( + analysisViewInputError([ + { + path: ["view"], + reason: "invalid_value", + message: "Mach-O inspection supports summary and macho views.", + }, + ]), + ); + const parsed = parseMachoReport(parent); + if (!parsed.ok) return parsed; + const report = parsed.value; + const incomplete = MACHO_COLLECTIONS.filter( + (name) => !report[name].exhaustive, + ); + const shared = { + parent_evidence_id: parent.evidenceId, + parent_operation: "inspect_macho" as const, + parent_digest: parent.evidenceId.slice(3), + artifact: + parent.subject == null + ? { path: "", sha256: null } + : { + path: parent.subject.local_path, + sha256: parent.subject.digest.sha256, + }, + unknowns: [ + ...(incomplete.length === 0 + ? [] + : [ + `Native tools did not report exhaustive ${incomplete.join(", ")}; page exhaustion does not establish absence of unreturned rows.`, + ]), + ...(parent.subject == null + ? [ + "The parent has no recorded subject, so the inspected artifact path and SHA-256 are unknown.", + ] + : []), + ], + }; + if (view.kind === "summary") { + const { summary, limitations, returned } = summarizeMacho( + report, + parent.limitations, + ); + return ok({ + kind: "summary", + view, + ...shared, + limitations, + summary, + coverage: completeWithinViewCoverage(returned, returned), + }); + } + const source = + view.facet === "provenance" + ? { + items: report.provenance, + total: report.provenance.length, + exhaustive: true, + limitations: [], + } + : report[view.facet]; + const item = source.items.slice(view.offset, view.offset + view.limit); + return ok({ + kind: "macho", + view, + item, + ...shared, + limitations: uniqueSorted([ + ...parent.limitations, + ...report.limitations, + ...source.limitations, + ]), + source_total: source.total, + source_exhaustive: source.exhaustive, + coverage: pageViewCoverage(view.offset, item.length, source.items.length), + }); +}; diff --git a/src/server/createServer.ts b/src/server/createServer.ts index 294984339..777028cf8 100644 --- a/src/server/createServer.ts +++ b/src/server/createServer.ts @@ -54,6 +54,7 @@ import { silentLogger } from "../logger.js"; import type { Logger } from "pino"; import { registerApplicationTools } from "./registerApplicationTools.js"; import { registerArtifactTools } from "./registerArtifactTools.js"; +import { registerNativeTools } from "./registerNativeTools.js"; import { registerBrowserScenarioTool } from "./registerBrowserScenarioTool.js"; import { registerBrowserTools } from "./registerBrowserTools.js"; import { registerWebScriptTool } from "./registerWebScriptTool.js"; @@ -107,8 +108,6 @@ import type { AdbDeviceAnalysisPort } from "../application/adb/AdbDeviceAnalysis import { AdbDeviceAnalysisService } from "../application/adb/AdbDeviceAnalysisService.js"; import { createAdbDeviceAnalysisProvider } from "../composition/adb.js"; import { registerManagedWorkflowTools } from "./registerManagedWorkflowTools.js"; -import { NATIVE_TOOL_CONTRACTS } from "../contracts/native/nativeToolContracts.js"; -import { registerEvidenceTools } from "./registerEvidenceTools.js"; import { registerOfficialTools } from "./registerOfficialTools.js"; import { registerGuidedPrompts } from "./registerPrompts.js"; import { registerSessionTools } from "./registerSessionTools.js"; @@ -725,7 +724,7 @@ const registerBinaryAnalysisTools = ({ recordEvidence, withAdmittedAnalysis, }; - registerEvidenceTools(server, NATIVE_TOOL_CONTRACTS, evidenceOptions); + registerNativeTools(server, { ...evidenceOptions, evidenceById }); registerArtifactTools(server, { ...evidenceOptions, evidenceById }); registerManagedTools(server, { ...evidenceOptions, evidenceById }); if (session !== undefined) diff --git a/src/server/registerArtifactTools.ts b/src/server/registerArtifactTools.ts index cf95fea41..1217664e6 100644 --- a/src/server/registerArtifactTools.ts +++ b/src/server/registerArtifactTools.ts @@ -6,7 +6,10 @@ import { artifactInspectionInputSchema, } from "../contracts/artifactToolContracts.js"; import type { EvidenceLookup } from "../application/EvidenceInputResolver.js"; -import { summarizeRetainedAnalysis } from "../application/analysisView/AnalysisViewService.js"; +import { + requireAnalysisViewRetention, + summarizeRetainedAnalysis, +} from "../application/analysisView/AnalysisViewService.js"; import { createArtifactExtractionDestination } from "../application/artifacts/ArtifactExtractionDestination.js"; import { ok } from "../domain/result.js"; import type { BinaryTarget } from "../domain/binaryTargetTypes.js"; @@ -25,6 +28,10 @@ export const registerArtifactTools = ( readonly withAdmittedAnalysis: import("./analysisAdmission.js").WithAdmittedAnalysis; }, ): void => { + const retention = { + recordEvidence: options.recordEvidence, + evidenceById: options.evidenceById, + }; registerEvidenceTools(server, ARTIFACT_TOOL_CONTRACTS, { ...options, parametersForExecution: (operation, parameters) => { @@ -37,12 +44,13 @@ export const registerArtifactTools = ( ? { ...parameters, output_root: createArtifactExtractionDestination() } : parameters; }, + preflight: (operation, parameters) => + operation === "inspect_artifact" && parameters.detail === "summary" + ? requireAnalysisViewRetention(operation, retention) + : ok(null), projectEvidence: (operation, parameters, evidence) => operation === "inspect_artifact" && parameters.detail === "summary" - ? summarizeRetainedAnalysis(evidence, { - recordEvidence: options.recordEvidence, - evidenceById: options.evidenceById, - }) + ? summarizeRetainedAnalysis(evidence, retention) : ok(evidence), sourceEvidence: (operation, result) => operation === "inspect_artifact" diff --git a/src/server/registerEvidenceTools.ts b/src/server/registerEvidenceTools.ts index a8efac6de..b3b2358d1 100644 --- a/src/server/registerEvidenceTools.ts +++ b/src/server/registerEvidenceTools.ts @@ -33,6 +33,16 @@ interface EvidenceToolRegistration { operation: Exclude, parameters: Readonly>, ) => Readonly>; + /** Exclude adapter-only selections, such as delivery detail, from Evidence identity. */ + readonly parametersForEvidence?: ( + operation: Exclude, + parameters: Readonly>, + ) => Readonly>; + /** Refuse an unsatisfiable delivery selection before any analysis runs. */ + readonly preflight?: ( + operation: Exclude, + parameters: Readonly>, + ) => Result; readonly projectEvidence?: ( operation: Exclude, parameters: Readonly>, @@ -51,6 +61,12 @@ export const registerEvidenceTools = ( contract.name, toolRegistrationOptions(contract), async (input, context) => { + const ready = options.preflight?.( + contract.name, + jsonObjectSchema.parse(input), + ); + if (ready !== undefined && !ready.ok) + return server.delivery.toCallToolResult(ready, contract); const admitted = await options.withAdmittedAnalysis( contract.name, context.mcpReq.signal, @@ -95,7 +111,9 @@ export const registerEvidenceTools = ( execution.value.provider, { operation: contract.name, - parameters, + parameters: + options.parametersForEvidence?.(contract.name, parameters) ?? + parameters, result: execution.value.result, ...(execution.value.analysisProfile === undefined ? {} diff --git a/src/server/registerNativeTools.ts b/src/server/registerNativeTools.ts new file mode 100644 index 000000000..036389d16 --- /dev/null +++ b/src/server/registerNativeTools.ts @@ -0,0 +1,54 @@ +import type { EvidenceMcpServer } from "./EvidenceMcpServer.js"; +import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js"; + +import { NATIVE_TOOL_CONTRACTS } from "../contracts/native/nativeToolContracts.js"; +import type { EvidenceLookup } from "../application/EvidenceInputResolver.js"; +import { + requireAnalysisViewRetention, + summarizeRetainedAnalysis, +} from "../application/analysisView/AnalysisViewService.js"; +import type { JsonValue } from "../domain/jsonValue.js"; +import { ok } from "../domain/result.js"; +import type { BinaryTarget } from "../domain/binaryTargetTypes.js"; +import type { Logger } from "pino"; +import { registerEvidenceTools } from "./registerEvidenceTools.js"; + +/** Delivery detail selects a response shape; it never reaches the tools or Evidence identity. */ +const withoutDelivery = ( + operation: string, + parameters: Readonly>, +): Readonly> => { + if (operation !== "inspect_macho") return parameters; + const { detail: _detail, ...selected } = parameters; + return selected; +}; + +/** Register macOS native contracts; Mach-O inspection can deliver a retained summary. */ +export const registerNativeTools = ( + server: EvidenceMcpServer, + options: { + readonly logger: Logger; + readonly activeTarget: (() => BinaryTarget | undefined) | undefined; + readonly recordEvidence: EvidenceWriter["recordEvidence"] | undefined; + readonly evidenceById?: EvidenceLookup | undefined; + readonly withAdmittedAnalysis: import("./analysisAdmission.js").WithAdmittedAnalysis; + }, +): void => { + const retention = { + recordEvidence: options.recordEvidence, + evidenceById: options.evidenceById, + }; + registerEvidenceTools(server, NATIVE_TOOL_CONTRACTS, { + ...options, + parametersForExecution: withoutDelivery, + parametersForEvidence: withoutDelivery, + preflight: (operation, parameters) => + operation === "inspect_macho" && parameters.detail === "summary" + ? requireAnalysisViewRetention(operation, retention) + : ok(null), + projectEvidence: (operation, parameters, evidence) => + operation === "inspect_macho" && parameters.detail === "summary" + ? summarizeRetainedAnalysis(evidence, retention) + : ok(evidence), + }); +}; diff --git a/tests/boundary/mcp/nativeMachoSummaryMcp.test.ts b/tests/boundary/mcp/nativeMachoSummaryMcp.test.ts new file mode 100644 index 000000000..e492263fa --- /dev/null +++ b/tests/boundary/mcp/nativeMachoSummaryMcp.test.ts @@ -0,0 +1,216 @@ +import { silentLogger } from "../../../src/logger.js"; +import { copyFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { describe, expect } from "vitest"; +import { z } from "zod"; + +import { createBinarySession } from "../../../src/composition/binary.js"; +import { parseConfig } from "../../../src/config/parseConfig.js"; +import { inspectMachoSchema } from "../../../src/domain/native/nativeInspection.js"; +import { createServer } from "../../../src/server/createServer.js"; +import { thinMach } from "../../../src/domain/binaryTarget.fixture.js"; +import { createTestBinarySession } from "../../fixtures/binarySession.js"; +import { observed } from "../../fixtures/analysisExecution.js"; +import { ok } from "../../../src/domain/result.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { mcpTest } from "../../support/mcp/mcpFixture.js"; + +const evidenceOf = (schema: Schema) => + z.object({ + evidence_id: z.string(), + parameters: z.record(z.string(), z.unknown()), + normalized_result: schema, + }); +const countSchema = z.object({ + returned: z.number(), + total: z.number().nullable(), + exhaustive: z.boolean(), +}); +const summarySchema = z.object({ + kind: z.literal("summary"), + parent_evidence_id: z.string(), + summary: z.object({ + uuid: z.string().nullable(), + collections: z.record(z.string(), countSchema), + provenance_count: z.number(), + }), +}); +const pageSchema = z.object({ + kind: z.literal("macho"), + item: z.array(z.unknown()), + coverage: z.object({ next_offset: z.number().nullable() }), +}); + +describe.skipIf(process.platform !== "darwin")( + "Mach-O summary delivery through MCP", + () => { + mcpTest( + "retains complete inspect_macho Evidence and pages its exact rows", + async ({ mcp, onTestFinished }) => { + const target = join( + await createTestTempDirectory("rea-macho-summary-mcp-"), + "ls", + ); + await copyFile("/bin/ls", target); + const configured = parseConfig({ REA_ANALYSIS_PROVIDER: "auto" }); + if (!configured.ok) throw configured.error; + const session = createBinarySession( + configured.value, + silentLogger, + process.env, + ); + onTestFinished(async () => { + await session.close(); + }); + const client = await mcp.connect( + createServer({ kind: "session", session }), + ); + const opened = await client.callTool({ + name: "open_binary", + arguments: { path: target }, + }); + expect(opened.isError, JSON.stringify(opened.content)).not.toBe(true); + + const complete = await client.callTool({ + name: "inspect_macho", + arguments: {}, + }); + expect(complete.isError, JSON.stringify(complete.content)).not.toBe( + true, + ); + const full = evidenceOf(inspectMachoSchema).parse( + complete.structuredContent, + ); + // Omitted detail keeps the default Evidence parameters unchanged. + expect(full.parameters).toEqual({}); + + const summarized = await client.callTool({ + name: "inspect_macho", + arguments: { detail: "summary" }, + }); + expect(summarized.isError, JSON.stringify(summarized.content)).not.toBe( + true, + ); + const summary = evidenceOf(summarySchema).parse( + summarized.structuredContent, + ).normalized_result; + // Delivery detail never reaches the tools or the source Evidence ID. + expect(summary.parent_evidence_id).toBe(full.evidence_id); + const report = full.normalized_result; + expect(summary.summary.uuid).toBe(report.uuid); + expect(summary.summary.provenance_count).toBe(report.provenance.length); + for (const name of ["imports", "exports", "dependencies"] as const) + expect(summary.summary.collections[name]).toEqual({ + returned: report[name].items.length, + total: report[name].total, + exhaustive: report[name].exhaustive, + }); + expect(JSON.stringify(summarized.structuredContent)).not.toContain( + report.imports.items[0]?.name ?? "_", + ); + + // Every facet must reassemble the unparsed complete rows exactly. + const raw = z + .object({ normalized_result: z.record(z.string(), z.unknown()) }) + .parse(complete.structuredContent).normalized_result; + const facets = [ + "entrypoints", + "architectures", + "build_metadata", + "load_commands", + "dependencies", + "imports", + "exports", + "segments", + "provenance", + ] as const; + for (const facet of facets) { + const rows: unknown[] = []; + let offset: number | null = 0; + while (offset !== null) { + const paged = await client.callTool({ + name: "inspect_analysis_view", + arguments: { + source: { + kind: "retained-evidence", + evidence_id: summary.parent_evidence_id, + }, + view: { kind: "macho", facet, offset, limit: 16 }, + }, + }); + expect(paged.isError, JSON.stringify(paged.content)).not.toBe(true); + const page = evidenceOf(pageSchema).parse( + paged.structuredContent, + ).normalized_result; + rows.push(...page.item); + offset = page.coverage.next_offset; + } + expect(rows, facet).toEqual( + facet === "provenance" + ? raw.provenance + : z.object({ items: z.array(z.unknown()) }).parse(raw[facet]) + .items, + ); + } + + const wrongKind = await client.callTool({ + name: "inspect_analysis_view", + arguments: { + source: { + kind: "retained-evidence", + evidence_id: summary.parent_evidence_id, + }, + view: { kind: "wasm", facet: "exports" }, + }, + }); + expect(wrongKind.isError).toBe(true); + expect(JSON.stringify(wrongKind.content)).toContain( + "Mach-O inspection supports summary and macho views.", + ); + }, + ); + }, +); + +mcpTest( + "refuses summary delivery before analysis when the server retains no Evidence", + async ({ mcp, onTestFinished }) => { + const target = join( + await createTestTempDirectory("rea-summary-preflight-"), + "tool", + ); + await writeFile(target, thinMach(0xfeedfacf, 0x0100000c)); + const executed: string[] = []; + const session = createTestBinarySession(() => ({ + execute: (operation) => { + executed.push(operation); + return Promise.resolve(observed(null)); + }, + close: () => Promise.resolve(ok(null)), + })); + onTestFinished(async () => { + await session.close(); + }); + const opened = await session.open(target); + if (!opened.ok) throw opened.error; + const client = await mcp.connect( + createServer({ kind: "fixed", analysis: session }), + ); + const refused = []; + for (const name of ["inspect_macho", "inspect_artifact"]) + refused.push( + await client.callTool({ name, arguments: { detail: "summary" } }), + ); + // Neither analysis ran only to be discarded by the delivery refusal. + expect(executed.filter((operation) => operation !== "health")).toEqual([]); + for (const result of refused) + expect(parseMcpToolError(result)).toMatchObject({ + error: { + code: "capability_unavailable", + message: expect.stringContaining("does not retain Evidence"), + }, + }); + }, +); From eaa961658578522cecfbba7c7be1acf1c64da65a Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:19:18 +0800 Subject: [PATCH 068/159] fix(javascript): preserve cancellation through recovery publication cleanup --- src/javascript/recovery/WakaruProvider.ts | 12 +- .../wakaruPublicationCancellation.test.ts | 128 ++++++++++++++++++ 2 files changed, 138 insertions(+), 2 deletions(-) create mode 100644 tests/boundary/javascript-recovery/wakaruPublicationCancellation.test.ts diff --git a/src/javascript/recovery/WakaruProvider.ts b/src/javascript/recovery/WakaruProvider.ts index 88a51a74b..e25812663 100644 --- a/src/javascript/recovery/WakaruProvider.ts +++ b/src/javascript/recovery/WakaruProvider.ts @@ -9,7 +9,11 @@ import type { import { SafeOutputTree } from "../../artifacts/SafeOutputTree.js"; import { SafeOutputTreeCreationFailure } from "../../artifacts/SafeOutputTreeCreationFailure.js"; import { AnalysisError } from "../../domain/analysisErrorBase.js"; -import { AnalysisOutputError } from "../../domain/analysisErrorCore.js"; +import { + AnalysisCancelledError, + AnalysisOutputError, +} from "../../domain/analysisErrorCore.js"; +import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js"; import { javascriptRecoveryResultSchema, type JavaScriptRecoveryInput, @@ -144,7 +148,11 @@ export class WakaruProvider implements JavaScriptRecoveryPort { failed = { cause }; } if (failed !== undefined) { - const failure = failed.cause; + const failure = + failed.cause instanceof ArtifactReaderFailure && + failed.cause.reason === "cancelled" + ? new AnalysisCancelledError(OPERATION, { cause: failed.cause }) + : failed.cause; const residuals: string[] = []; const cleanupFailures: { resource: string; reason: string }[] = []; const cleanupOwner = diff --git a/tests/boundary/javascript-recovery/wakaruPublicationCancellation.test.ts b/tests/boundary/javascript-recovery/wakaruPublicationCancellation.test.ts new file mode 100644 index 000000000..250e75623 --- /dev/null +++ b/tests/boundary/javascript-recovery/wakaruPublicationCancellation.test.ts @@ -0,0 +1,128 @@ +import { createHash } from "node:crypto"; +import { access, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { Readable } from "node:stream"; +import { expect, it, vi } from "vitest"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +const publicationCancellation = vi.hoisted(() => ({ + controller: undefined as AbortController | undefined, + directorySyncs: 0, +})); + +vi.mock("node:fs/promises", async (importOriginal) => { + const filesystem = await importOriginal(); + return { + ...filesystem, + open: async (...args: Parameters) => { + const handle = await filesystem.open(...args); + if (publicationCancellation.controller !== undefined) { + const stats = await handle.stat(); + if (stats.isDirectory()) { + const sync = handle.sync.bind(handle); + handle.sync = async () => { + await sync(); + publicationCancellation.directorySyncs += 1; + if (publicationCancellation.directorySyncs === 2) + publicationCancellation.controller?.abort(); + }; + } + } + return handle; + }, + }; +}); + +vi.mock( + "../../../src/javascript/recovery/WakaruCommand.js", + async (importOriginal) => { + const actual = + await importOriginal< + typeof import("../../../src/javascript/recovery/WakaruCommand.js") + >(); + return { + ...actual, + resolveWakaruCommand: async () => ({ + command: "/unused/wakaru", + limiter: "/unused/prlimit", + sha256: "a".repeat(64), + bytes: 1, + }), + }; + }, +); + +vi.mock( + "../../../src/javascript/recovery/WakaruExecution.js", + async (importOriginal) => { + const actual = + await importOriginal< + typeof import("../../../src/javascript/recovery/WakaruExecution.js") + >(); + const { createAnalysisExecution } = + await import("../../../src/application/AnalysisProvider.js"); + return { + ...actual, + prepareWakaruExecution: async ( + context: Parameters[0], + ) => { + const source = context.sources[0]; + if (source === undefined) throw new Error("Missing recovery source"); + const bytes = Buffer.from("module.exports = 42;"); + await context.tree.write("module.js", Readable.from([bytes]), { + sha256: createHash("sha256").update(bytes).digest("hex"), + bytes: bytes.byteLength, + }); + return createAnalysisExecution( + { status: "complete" }, + { id: "wakaru", name: "Wakaru", version: "1.14.0" }, + { + subject: { + path: source.path, + sha256: source.sha256, + format: "javascript", + }, + }, + ); + }, + }; + }, +); + +import { AnalysisCancelledError } from "../../../src/domain/analysisErrorCore.js"; +import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; +import { WakaruProvider } from "../../../src/javascript/recovery/WakaruProvider.js"; + +it.skipIf(process.platform === "win32")( + "keeps mid-commit output-tree cancellation typed through the provider", + async () => { + const root = await createTestTempDirectory("rea-wakaru-cancel-"); + const sourcePath = join(root, "source.js"); + const output = join(root, "output"); + await writeFile(sourcePath, "module.exports = 42;"); + const controller = new AbortController(); + publicationCancellation.controller = controller; + publicationCancellation.directorySyncs = 0; + const provider = new WakaruProvider({}); + try { + const result = await provider.recover( + { + path: sourcePath, + output_directory: output, + extraction_mode: "structural", + rewrite_level: "standard", + }, + { signal: controller.signal }, + ); + if (result.ok) throw new Error("Expected publication cancellation"); + expect(result.error).toBeInstanceOf(AnalysisCancelledError); + expect(projectAnalysisError(result.error).code).toBe("cancelled"); + expect(publicationCancellation.directorySyncs).toBe(2); + await expect(access(output)).rejects.toMatchObject({ code: "ENOENT" }); + } finally { + publicationCancellation.controller = undefined; + publicationCancellation.directorySyncs = 0; + await provider.close(); + } + }, +); From ed881c6d2ef6a8adba0c2a241ebc9ab4603b7868 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:23:56 +0800 Subject: [PATCH 069/159] fix(progress): measure delivery intervals with a monotonic clock --- src/application/ProgressReporter.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/application/ProgressReporter.ts b/src/application/ProgressReporter.ts index cf5ed7b7f..bb5a98f0d 100644 --- a/src/application/ProgressReporter.ts +++ b/src/application/ProgressReporter.ts @@ -23,7 +23,7 @@ export const createProgressReporter = ( } = {}, ): ProgressReporter => { const minimumIntervalMs = options.minimumIntervalMs ?? 100; - const now = options.now ?? Date.now; + const now = options.now ?? (() => performance.now()); let lastCompleted = -1; let lastSentAt = Number.NEGATIVE_INFINITY; let sequence = 0; From c923fb0dc9462e302b84a15770131ff899f89455 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:58:25 +0800 Subject: [PATCH 070/159] fix(javascript): retain compiler-dependent module resolution facts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Compare import and require outcomes for static TypeScript references when compiler options are unknown. Preserve branch facts through worker IR and both graph projections; retain declaration fallbacks as candidates without changing explicitly selected paths. Validation: 210 focused resolver and semantic tests; 21 production-boundary and source-extension tests. A bounded 2,000-reference probe adds about 16–17 ms wall time for dual resolution; full task token costs and real compiled application behavior remain unmeasured. --- docs/javascript-application-graph.md | 9 + .../ReferenceSourceImportEntries.ts | 6 +- .../TypeScriptEmittedSourceCandidates.ts | 12 +- .../javascript/JavaScriptArtifactAnalysis.ts | 7 +- .../JavaScriptArtifactGraphContext.ts | 6 +- .../JavaScriptArtifactGraphFindings.ts | 108 +++++++-- .../JavaScriptArtifactPathResolution.test.ts | 23 +- .../JavaScriptArtifactPathResolution.ts | 209 ++++++++++++++---- .../JavaScriptModuleRelationships.ts | 29 ++- .../javascriptAnalysisWorkerProcess.ts | 4 +- .../javascriptModuleResolutionMode.ts | 23 ++ .../javascript/javascriptSemanticAnalysis.ts | 14 +- src/domain/javascript/javascriptSemanticIr.ts | 5 +- .../javascriptSemanticProjection.ts | 15 +- .../javascript/javascriptSemanticReturns.ts | 4 +- .../javascript/javascriptSemanticState.ts | 3 +- .../JavaScriptAnalysisWorkerSchemas.ts | 8 +- .../javascriptModuleRelationships.test.ts | 118 +++++++++- 18 files changed, 503 insertions(+), 100 deletions(-) create mode 100644 src/domain/javascript/javascriptModuleResolutionMode.ts diff --git a/docs/javascript-application-graph.md b/docs/javascript-application-graph.md index 2290491a5..48e17bf44 100644 --- a/docs/javascript-application-graph.md +++ b/docs/javascript-application-graph.md @@ -104,6 +104,15 @@ static inferences. Dynamic or ambiguous IPC is retained without a pairing edge. Runtime entities retain passive-CDP authority, while a successful static/runtime correspondence is a separately identified cross-layer inference. +Source module links retain their resolution mode. Static imports in `.ts` and +`.tsx` are `compiler-dependent` when compiler options have not been retained. +REA compares Node import and require resolution: a shared outcome remains +available with a limitation; divergent outcomes remain unavailable with both +conditional branches and their limitations. This models source relationships, +not verified emitted code or execution. A declaration file found only through +an emitted-extension fallback is retained as a candidate, not a runtime target; +an explicitly selected path keeps its original identity. + ## Evidence and authority Every observation and edge carries all of the following: diff --git a/src/application/ReferenceSourceImportEntries.ts b/src/application/ReferenceSourceImportEntries.ts index a6e962118..36dae9949 100644 --- a/src/application/ReferenceSourceImportEntries.ts +++ b/src/application/ReferenceSourceImportEntries.ts @@ -104,7 +104,11 @@ const resolveInternalSpecifier = ( ]); // Retain present runtime files first; substitute only a missing counterpart. // Counterparts cover explicit filenames, not directory/index resolution. - candidates.push(...typescriptEmittedSourceCandidates(fromPath, normalized)); + candidates.push( + ...typescriptEmittedSourceCandidates(fromPath, normalized).map( + ({ path }) => path, + ), + ); const match = lookupPath.resolveCandidates(candidates); return match === undefined ? { to: normalized, resolution: "unresolved" } diff --git a/src/application/TypeScriptEmittedSourceCandidates.ts b/src/application/TypeScriptEmittedSourceCandidates.ts index 663c7dada..0e7adf41c 100644 --- a/src/application/TypeScriptEmittedSourceCandidates.ts +++ b/src/application/TypeScriptEmittedSourceCandidates.ts @@ -8,6 +8,11 @@ const EMITTED_SUFFIXES: ReadonlyArray< [".jsx", [".tsx", ".d.ts"]], ]; +export interface TypeScriptEmittedSourceCandidate { + readonly path: string; + readonly kind: "implementation" | "declaration"; +} + /** * Return TypeScript source counterparts for an explicit emitted filename. * Callers retain exact runtime-file precedence and must not use these for @@ -16,7 +21,7 @@ const EMITTED_SUFFIXES: ReadonlyArray< export const typescriptEmittedSourceCandidates = ( sourcePath: string, candidate: string, -): readonly string[] => { +): readonly TypeScriptEmittedSourceCandidate[] => { if (!/\.(?:ts|tsx|mts|cts)$/u.test(sourcePath)) return []; const emitted = EMITTED_SUFFIXES.find(([suffix]) => candidate.endsWith(suffix), @@ -25,5 +30,8 @@ export const typescriptEmittedSourceCandidates = ( // Strip the matched suffix by length. posix.extname would miss dotfile // names such as ".js", whose whole basename is the emitted suffix itself. const stem = candidate.slice(0, candidate.length - emitted[0].length); - return emitted[1].map((suffix) => `${stem}${suffix}`); + return emitted[1].map((suffix) => ({ + path: `${stem}${suffix}`, + kind: suffix.startsWith(".d.") ? "declaration" : "implementation", + })); }; diff --git a/src/application/javascript/JavaScriptArtifactAnalysis.ts b/src/application/javascript/JavaScriptArtifactAnalysis.ts index dd1a50488..e61d3ae94 100644 --- a/src/application/javascript/JavaScriptArtifactAnalysis.ts +++ b/src/application/javascript/JavaScriptArtifactAnalysis.ts @@ -1,6 +1,6 @@ import { createHash } from "node:crypto"; -import { moduleKindForJavaScriptReference } from "./JavaScriptArtifactPathResolution.js"; +import { moduleResolutionModeForJavaScriptReference } from "../../domain/javascript/javascriptModuleResolutionMode.js"; import { setImmediate } from "node:timers/promises"; import { resolveJavaScriptSourceMapReferences } from "../../domain/javascript/javascriptSourceMapPaths.js"; @@ -384,7 +384,10 @@ function* analyzeArtifactFileSteps< analysis.parse_status === "complete" || analysis.parse_status === "partial" ? yield* analyzeParsedJavaScriptSemanticsSteps( parsed, - moduleKindForJavaScriptReference(file.path, "static-import"), + moduleResolutionModeForJavaScriptReference( + file.path, + "static-import", + ), ) : null; const projected = diff --git a/src/application/javascript/JavaScriptArtifactGraphContext.ts b/src/application/javascript/JavaScriptArtifactGraphContext.ts index 8b0fed5f1..1779bdaae 100644 --- a/src/application/javascript/JavaScriptArtifactGraphContext.ts +++ b/src/application/javascript/JavaScriptArtifactGraphContext.ts @@ -66,6 +66,7 @@ export interface AstEdgeInput { /** Inferred static relationship edge values. */ export interface InferenceEdgeInput extends AstEdgeInput { + readonly limitations?: readonly string[]; readonly relation: | "imports" | "loads" @@ -162,6 +163,9 @@ export const addStaticInferenceEdge = ( range: input.range, operation: "map-static-relationship", coverage: input.coverage, + ...(input.limitations === undefined + ? {} + : { limitations: input.limitations }), }), }); }; @@ -347,7 +351,7 @@ export const resolveArtifactPath = ( specifier: string, sourcePath: string, files: ReadonlyMap, - moduleKind?: "import" | "require", + moduleKind?: "import" | "require" | "compiler-dependent", ): string | null => resolveArtifactPathByContext({ declaredPath: specifier, diff --git a/src/application/javascript/JavaScriptArtifactGraphFindings.ts b/src/application/javascript/JavaScriptArtifactGraphFindings.ts index 3e92ff1a4..0020ef577 100644 --- a/src/application/javascript/JavaScriptArtifactGraphFindings.ts +++ b/src/application/javascript/JavaScriptArtifactGraphFindings.ts @@ -12,7 +12,6 @@ import { javascriptAnalysisCoverage, linkElectronRoleToAsset, moduleLookupKey, - resolveArtifactPath, selfReferenceOmissions, sourceNodeFor, type JavaScriptArtifactGraphContext, @@ -23,9 +22,11 @@ import { staticInferenceEvidence, } from "./JavaScriptArtifactGraphEvidence.js"; import { - moduleKindForJavaScriptReference, + artifactPathResolutionProperties, resolveArtifactPathByContext, + type ArtifactPathResolution, } from "./JavaScriptArtifactPathResolution.js"; +import { moduleResolutionModeForJavaScriptReference } from "../../domain/javascript/javascriptModuleResolutionMode.js"; type StaticReference = JavaScriptStaticAnalysis["references"][number]; type StaticEndpoint = JavaScriptStaticAnalysis["endpoints"][number]; @@ -41,9 +42,12 @@ interface FindingInput { } interface ResolvedReference { - readonly path: string; - readonly file: JavaScriptArtifactFile; + readonly path: string | null; + readonly file: JavaScriptArtifactFile | null; readonly node: ApplicationNode; + readonly limitations: readonly string[]; + readonly resolutionStatus?: string; + readonly resolution?: ArtifactPathResolution; } /** Project imports, workers, roles, endpoints, storage, and source-map links. */ @@ -90,7 +94,7 @@ const addReference = ( const resolved = input.value.specifier === null ? null - : resolveReference(context, input.file.path, input.value); + : resolveReference(context, input.file.path, input.value, input.coverage); if (input.value.kind === "worker" || input.value.kind === "service-worker") { const worker = context.accumulator.addNode({ kind: input.value.kind === "worker" ? "worker" : "service-worker", @@ -126,7 +130,7 @@ const addReference = ( relation: "loads", properties: { kind: input.value.kind }, }); - if (resolved !== null) + if (resolved !== null && resolved.path !== null) addStaticInferenceEdge(context, { source: worker, target: resolved.node, @@ -135,6 +139,7 @@ const addReference = ( coverage: input.coverage, relation: "maps_to", properties: { resolved_path: resolved.path }, + limitations: resolved.limitations, }); return; } @@ -154,13 +159,25 @@ const addReference = ( file: input.file, range: input.value.location, coverage: input.coverage, - relation: resolved?.file.kind === "native-addon" ? "loads" : "imports", + relation: resolved?.file?.kind === "native-addon" ? "loads" : "imports", properties: { kind: input.value.kind, specifier: input.value.specifier, expression: input.value.expression, + resolution_mode: + input.value.kind === "static-import" || + input.value.kind === "dynamic-import" || + input.value.kind === "require" + ? moduleResolutionModeForJavaScriptReference( + input.file.path, + input.value.kind, + ) + : null, resolved_path: resolved?.path ?? null, + resolution_status: resolved?.resolutionStatus ?? null, + ...artifactPathResolutionProperties(resolved?.resolution), }, + ...(resolved === null ? {} : { limitations: resolved.limitations }), }); }; @@ -373,6 +390,7 @@ const resolveReference = ( context: JavaScriptArtifactGraphContext, sourcePath: string, reference: StaticReference, + coverage: JavaScriptArtifactGraphCoverage, ): ResolvedReference | null => { const specifier = reference.specifier; if (specifier === null) return null; @@ -392,30 +410,70 @@ const resolveReference = ( const file = context.filesByPath.get(sourcePath); return file === undefined ? null - : { path: `${sourcePath}#module:${specifier}`, file, node: module }; + : { + path: `${sourcePath}#module:${specifier}`, + file, + node: module, + limitations: [], + }; } // Worker scripts are URLs relative to the script, not module specifiers. - const path = isWorker + const resolution = isWorker ? resolveArtifactPathByContext({ declaredPath: specifier, sourcePath, context: "url-reference", files: context.filesByPath, - }).resolved_path - : resolveArtifactPath( - specifier, + }) + : resolveArtifactPathByContext({ + declaredPath: specifier, sourcePath, - context.filesByPath, - reference.kind === "require" || - reference.kind === "static-import" || - reference.kind === "dynamic-import" - ? moduleKindForJavaScriptReference(sourcePath, reference.kind) - : undefined, - ); - if (path === null) return null; + context: "module-specifier", + files: context.filesByPath, + ...("require" === reference.kind || + "static-import" === reference.kind || + "dynamic-import" === reference.kind + ? { + moduleKind: moduleResolutionModeForJavaScriptReference( + sourcePath, + reference.kind, + ), + } + : {}), + }); + const path = resolution.resolved_path; + if (path === null) { + if (isWorker) return null; + const source = context.filesByPath.get(sourcePath); + return source === undefined + ? null + : { + path: null, + file: null, + node: unresolvedReferenceNode( + context, + source, + reference, + coverage, + resolution, + ), + limitations: resolution.limitations, + resolutionStatus: resolution.resolution_status, + resolution, + }; + } const file = context.filesByPath.get(path); const node = context.fileNodes.get(path); - return file === undefined || node === undefined ? null : { path, file, node }; + return file === undefined || node === undefined + ? null + : { + path, + file, + node, + limitations: resolution.limitations, + resolutionStatus: resolution.resolution_status, + resolution, + }; }; const resolveBundlerChunkReference = ( @@ -453,7 +511,7 @@ const resolveBundlerChunkReference = ( const node = unique[0]; return node === undefined ? null - : { path: `${sourcePath}#chunk:${key}`, file, node }; + : { path: `${sourcePath}#chunk:${key}`, file, node, limitations: [] }; }; const unresolvedReferenceNode = ( @@ -461,6 +519,7 @@ const unresolvedReferenceNode = ( file: JavaScriptArtifactFile, reference: StaticReference, coverage: JavaScriptArtifactGraphCoverage, + resolution?: ArtifactPathResolution, ): ApplicationNode => context.accumulator.addNode({ kind: "javascript-module", @@ -477,6 +536,10 @@ const unresolvedReferenceNode = ( specifier: reference.specifier, expression: reference.expression, resolution: "unresolved", + ...(resolution === undefined + ? {} + : { resolution_status: resolution.resolution_status }), + ...artifactPathResolutionProperties(resolution), }, evidence: staticInferenceEvidence({ sha256: file.sha256, @@ -487,6 +550,7 @@ const unresolvedReferenceNode = ( confidence: "low", limitations: [ "The static reference could not be resolved within the inventoried artifact.", + ...(resolution?.limitations ?? []), ], }), }, diff --git a/src/application/javascript/JavaScriptArtifactPathResolution.test.ts b/src/application/javascript/JavaScriptArtifactPathResolution.test.ts index 1a8713929..b746d7e97 100644 --- a/src/application/javascript/JavaScriptArtifactPathResolution.test.ts +++ b/src/application/javascript/JavaScriptArtifactPathResolution.test.ts @@ -1890,21 +1890,40 @@ describe("TypeScript NodeNext emitted-extension resolution", () => { ).toMatchObject({ resolution_status: "resolved", resolved_path: target }); }, ); +}); + +describe("TypeScript declaration fallback ownership", () => { + const container = "container"; it.each([ ["./dep.js", "src/dep.d.ts"], ["./cfg.mjs", "src/cfg.d.mts"], ["./legacy.cjs", "src/legacy.d.cts"], ] as const)( - "resolves %s to a declaration counterpart", + "retains %s as compile-time evidence without inventing a runtime target", (specifier, target) => { const files = fileMap([ file("src/main.ts", container), file(target, container, "export const value = 1;"), ]); + for (const moduleKind of ["import", "require"] as const) { + expect( + resolve({ + declaredPath: specifier, + sourcePath: "src/main.ts", + context: "module-specifier", + moduleKind, + files, + }), + ).toMatchObject({ + resolution_status: "unavailable", + resolved_path: null, + candidate_path: target, + }); + } expect( resolve({ - declaredPath: specifier, + declaredPath: "./" + target.slice(4), sourcePath: "src/main.ts", context: "module-specifier", moduleKind: "import", diff --git a/src/application/javascript/JavaScriptArtifactPathResolution.ts b/src/application/javascript/JavaScriptArtifactPathResolution.ts index 3ea23ab11..cb29ab2a9 100644 --- a/src/application/javascript/JavaScriptArtifactPathResolution.ts +++ b/src/application/javascript/JavaScriptArtifactPathResolution.ts @@ -19,8 +19,13 @@ import { classifyPackageImportTargetString, lookupExactPackageImport, } from "../../domain/javascript/javascriptPackageImportAliases.js"; -import { typescriptEmittedSourceCandidates } from "../TypeScriptEmittedSourceCandidates.js"; +import { + typescriptEmittedSourceCandidates, + type TypeScriptEmittedSourceCandidate, +} from "../TypeScriptEmittedSourceCandidates.js"; import { isRelativeJavaScriptModuleSpecifier } from "../../domain/javascript/javascriptModuleSpecifier.js"; +import type { JavaScriptModuleResolutionMode } from "../../domain/javascript/javascriptModuleResolutionMode.js"; +import type { JsonValue } from "../../domain/jsonValue.js"; type ArtifactPathResolutionContext = | "package-entrypoint" @@ -39,6 +44,16 @@ interface ArtifactPathResolutionBase { readonly declared_path: string; readonly resolution_context: ArtifactPathResolutionContext; readonly limitations: readonly string[]; + readonly conditional_resolutions?: readonly ConditionalModuleResolution[]; + readonly candidate_path?: string; +} + +export interface ConditionalModuleResolution { + readonly mode: "import" | "require"; + readonly resolution_status: UnresolvedArtifactPathStatus | "resolved"; + readonly resolved_path: string | null; + readonly candidate_path?: string; + readonly limitations: readonly string[]; } /** Explicit outcome of artifact-confined path resolution. */ @@ -62,26 +77,13 @@ export interface ResolveArtifactPathInput { readonly files: ReadonlyMap; readonly htmlBaseHref?: string | null; /** Import/require semantics; runtime resource paths omit this provenance. */ - readonly moduleKind?: "import" | "require"; + readonly moduleKind?: JavaScriptModuleResolutionMode; } -/** Loader selected by a source-level JavaScript module reference. */ -export const moduleKindForJavaScriptReference = ( - sourcePath: string, - referenceKind: "static-import" | "dynamic-import" | "require", - resolutionMode?: "import" | "require", -): "import" | "require" => - resolutionMode ?? - (referenceKind === "require" || - (referenceKind === "static-import" && - posix.extname(sourcePath).toLowerCase() === ".cts") - ? "require" - : "import"); - /** Preserve an explicitly retained loader mode on graph relationships. */ export const moduleResolutionModeProperties = ( - mode: "import" | "require" | undefined, -): Readonly> => + mode: JavaScriptModuleResolutionMode | undefined, +): Readonly> => mode === undefined ? {} : { resolution_mode: mode }; const EXTENSIONS = [ @@ -110,43 +112,116 @@ interface PackageFileTarget { readonly url: URL; } -type CandidateResolution = - | { - readonly resolvedPath: string; - readonly status: "resolved"; - readonly limitations: readonly string[]; - } +type CandidateResolution = { + readonly limitations: readonly string[]; + readonly candidatePath?: string; + readonly conditionalResolutions?: readonly ConditionalModuleResolution[]; +} & ( + | { readonly resolvedPath: string; readonly status: "resolved" } | { readonly resolvedPath: null; readonly status: UnresolvedArtifactPathStatus; - readonly limitations: readonly string[]; /** A truthy package main failed, so Node must not try ancestor packages. */ readonly stopsAncestorLookup?: true; - }; + } +); /** Resolve one declaration while preserving its original input identity. */ export const resolveArtifactPathByContext = ( input: ResolveArtifactPathInput, ): ArtifactPathResolution => { const resolution = resolvePath(input); + const facts = { + declared_path: input.declaredPath, + resolution_context: input.context, + limitations: resolution.limitations, + ...(resolution.candidatePath === undefined + ? {} + : { candidate_path: resolution.candidatePath }), + ...(resolution.conditionalResolutions === undefined + ? {} + : { conditional_resolutions: resolution.conditionalResolutions }), + }; return resolution.status === "resolved" ? { - declared_path: input.declaredPath, - resolution_context: input.context, + ...facts, resolved_path: resolution.resolvedPath, resolution_status: resolution.status, - limitations: resolution.limitations, } - : { - declared_path: input.declaredPath, - resolution_context: input.context, - resolved_path: null, - resolution_status: resolution.status, - limitations: resolution.limitations, - }; + : { ...facts, resolved_path: null, resolution_status: resolution.status }; }; +/** Project retained conditional identities without flattening them into display text. */ +export const artifactPathResolutionProperties = ( + resolution: ArtifactPathResolution | undefined, +): Readonly> => ({ + ...(resolution?.candidate_path === undefined + ? {} + : { candidate_path: resolution.candidate_path }), + ...(resolution?.conditional_resolutions === undefined + ? {} + : { + conditional_resolutions: resolution.conditional_resolutions.map( + (branch) => ({ + mode: branch.mode, + resolution_status: branch.resolution_status, + resolved_path: branch.resolved_path, + limitations: [...branch.limitations], + ...(branch.candidate_path === undefined + ? {} + : { candidate_path: branch.candidate_path }), + }), + ), + }), +}); + +const conditionalResolution = ( + mode: "import" | "require", + result: CandidateResolution, +): ConditionalModuleResolution => ({ + mode, + resolution_status: result.status, + resolved_path: result.resolvedPath, + limitations: result.limitations, + ...(result.candidatePath === undefined + ? {} + : { candidate_path: result.candidatePath }), +}); + const resolvePath = (input: ResolveArtifactPathInput): CandidateResolution => { + if (input.moduleKind === "compiler-dependent") { + const imported = resolvePath({ ...input, moduleKind: "import" }); + const required = resolvePath({ ...input, moduleKind: "require" }); + const sameOutcome = + imported.status === required.status && + imported.resolvedPath === required.resolvedPath && + imported.candidatePath === required.candidatePath; + const limitation = + "Static TypeScript loader mode is compiler-dependent without retained compiler options. " + + (sameOutcome + ? "Node import and require resolution agree on this outcome." + : "Node import and require resolution disagree; neither branch is an established target."); + return sameOutcome + ? { + ...imported, + limitations: [ + ...new Set([ + ...imported.limitations, + ...required.limitations, + limitation, + ]), + ], + } + : { + resolvedPath: null, + status: "unavailable", + limitations: [limitation], + conditionalResolutions: [ + conditionalResolution("import", imported), + conditionalResolution("require", required), + ], + }; + } // Package import keys are literal lookup identities, not artifact paths. // The selected target is admitted separately by resolveExportTarget. if ( @@ -176,12 +251,47 @@ const resolvePath = (input: ResolveArtifactPathInput): CandidateResolution => { : []; if (isModuleUrlPath && input.moduleKind === "import") return ( - resolveFileCandidates(input, [confined, ...emittedSourceCandidates]) ?? - notFoundCandidate() + resolveRuntimeSourceCandidates( + input, + confined, + emittedSourceCandidates, + ) ?? notFoundCandidate() ); return resolveCandidate(input, confined, new Set(), emittedSourceCandidates); }; +const resolveRuntimeSourceCandidates = ( + input: ResolveArtifactPathInput, + exactPath: string, + sourceCandidates: readonly TypeScriptEmittedSourceCandidate[], +): CandidateResolution | null => { + const source = input.files.get(input.sourcePath); + const exactCandidate: TypeScriptEmittedSourceCandidate = { + path: exactPath, + kind: "implementation", + }; + for (const candidate of [exactCandidate, ...sourceCandidates]) { + const target = input.files.get(candidate.path); + if ( + target === undefined || + (source !== undefined && + target.container_sha256 !== source.container_sha256) + ) + continue; + return candidate.kind === "declaration" + ? { + resolvedPath: null, + status: "unavailable", + candidatePath: candidate.path, + limitations: [ + "Only a TypeScript declaration fallback was found; it does not establish the requested runtime module.", + ], + } + : { resolvedPath: candidate.path, status: "resolved", limitations: [] }; + } + return null; +}; + const rejectDeclaration = (declared: string): CandidateResolution | null => { if (declared.length === 0) return unresolvedCandidate("rejected", ["The declared path is empty."]); @@ -849,12 +959,10 @@ const resolveCandidate = ( input: ResolveArtifactPathInput, candidate: string, packageChain: ReadonlySet, - emittedSourceCandidates: readonly string[] = [], + emittedSourceCandidates: readonly TypeScriptEmittedSourceCandidate[] = [], ): CandidateResolution => - resolveFileCandidates(input, [ - ...fileCandidates(candidate, input.moduleKind), - ...emittedSourceCandidates, - ]) ?? + resolveFileCandidates(input, fileCandidates(candidate, input.moduleKind)) ?? + resolveRuntimeSourceCandidates(input, candidate, emittedSourceCandidates) ?? resolveDirectory( input, candidate, @@ -940,11 +1048,26 @@ const resolveFileCandidates = ( (source === undefined || target.container_sha256 === source.container_sha256) ) - return { resolvedPath: path, status: "resolved", limitations: [] }; + return isRuntimeDeclarationPath(input, path) + ? { + resolvedPath: null, + status: "unavailable", + candidatePath: path, + limitations: [ + `The TypeScript declaration ${path} is compile-time evidence and cannot identify a runtime module implementation.`, + ], + } + : { resolvedPath: path, status: "resolved", limitations: [] }; } return null; }; +const isRuntimeDeclarationPath = ( + input: ResolveArtifactPathInput, + path: string, +): boolean => + input.moduleKind !== undefined && /\.d\.(?:ts|mts|cts)$/iu.test(path); + const NODE_MODULE_EXTENSIONS = [".js", ".json", ".node"] as const; const fileCandidates = ( diff --git a/src/application/javascript/JavaScriptModuleRelationships.ts b/src/application/javascript/JavaScriptModuleRelationships.ts index 514a9bb7c..3b46e95ae 100644 --- a/src/application/javascript/JavaScriptModuleRelationships.ts +++ b/src/application/javascript/JavaScriptModuleRelationships.ts @@ -18,6 +18,7 @@ import type { JavaScriptModuleOrigin, JavaScriptSemanticModuleLink, } from "../../domain/javascript/javascriptSemanticIr.js"; +import { moduleResolutionModeForJavaScriptReference } from "../../domain/javascript/javascriptModuleResolutionMode.js"; import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js"; import { addAstContainsEdge, @@ -30,7 +31,7 @@ import { staticInferenceEvidence, } from "./JavaScriptArtifactGraphEvidence.js"; import { - moduleKindForJavaScriptReference, + artifactPathResolutionProperties, moduleResolutionModeProperties, resolveArtifactPathByContext, type ArtifactPathResolution, @@ -54,6 +55,7 @@ interface RelationshipInput { type ResolvedModuleTarget = { readonly node: ApplicationNode; readonly limitations: readonly string[]; + readonly resolution?: ArtifactPathResolution; } & ( | { readonly path: string; @@ -287,7 +289,7 @@ const resolveModuleTarget = ( sourcePath: input.file.path, context: "module-specifier", files: input.context.filesByPath, - moduleKind: moduleKindForJavaScriptReference( + moduleKind: moduleResolutionModeForJavaScriptReference( input.file.path, input.link.kind === "require" || input.link.kind === "commonjs-export" ? "require" @@ -297,15 +299,12 @@ const resolveModuleTarget = ( }); if (resolution.resolution_status !== "resolved") return { - node: unresolvedModuleNode( - input, - specifier, - resolution.resolution_status, - ), + node: unresolvedModuleNode(input, specifier, resolution), path: null, file: null, status: resolution.resolution_status, limitations: resolution.limitations, + resolution, }; const path = resolution.resolved_path; const file = input.context.filesByPath.get(path); @@ -322,20 +321,24 @@ const resolveModuleTarget = ( file, status: "resolved", limitations: resolution.limitations, + resolution, }; }; const unresolvedModuleNode = ( input: RelationshipInput, specifier: string, - status: Exclude, + resolution: Extract< + ArtifactPathResolution, + { resolution_status: Exclude } + >, ): ApplicationNode => input.context.accumulator.addNode({ kind: "javascript-module", identity: artifactLocalIdentity( input.file.sha256, "unresolved-semantic-module", - `${status}:${specifier}`, + `${resolution.resolution_status}:${specifier}`, ), observations: [ { @@ -343,7 +346,8 @@ const unresolvedModuleNode = ( properties: { semantic_role: "module-reference", declared_specifier: specifier, - resolution_status: status, + resolution_status: resolution.resolution_status, + ...artifactPathResolutionProperties(resolution), }, evidence: staticInferenceEvidence({ sha256: input.file.sha256, @@ -364,6 +368,9 @@ const moduleOriginForExport = ( ): JavaScriptModuleOrigin | null => { if (link.specifier !== null) return { + ...(link.resolutionMode === undefined + ? {} + : { resolutionMode: link.resolutionMode }), specifier: link.specifier, importedPath: link.importedName === null ? [] : [link.importedName], }; @@ -393,6 +400,7 @@ const relationshipProperties = ( exported_name: link.exportedName, resolved_path: target?.path ?? null, resolution_status: target?.status ?? null, + ...artifactPathResolutionProperties(target?.resolution), target_file_kind: target?.file?.kind ?? null, target_json_status: target?.file?.kind === "json" || target?.file?.kind === "package-json" @@ -441,6 +449,7 @@ const moduleFormat = ( const extension = posix.extname(path).toLowerCase(); if (extension === ".mjs" || extension === ".mts") return "esm"; if (extension === ".cjs" || extension === ".cts") return "commonjs"; + if (extension === ".ts" || extension === ".tsx") return "unknown"; const esm = ir.moduleLinks.some(({ kind }) => ["import", "export", "re-export"].includes(kind), ); diff --git a/src/composition/javascriptAnalysisWorkerProcess.ts b/src/composition/javascriptAnalysisWorkerProcess.ts index 09a9c9bc3..d1c3c134e 100644 --- a/src/composition/javascriptAnalysisWorkerProcess.ts +++ b/src/composition/javascriptAnalysisWorkerProcess.ts @@ -1,4 +1,4 @@ -import { moduleKindForJavaScriptReference } from "../application/javascript/JavaScriptArtifactPathResolution.js"; +import { moduleResolutionModeForJavaScriptReference } from "../domain/javascript/javascriptModuleResolutionMode.js"; import { parseJavaScriptSource } from "../domain/javascript/javascriptSourceParser.js"; import { analyzeParsedJavaScriptStaticSourceSteps } from "../domain/javascript/javascriptStaticAnalysis.js"; import { @@ -56,7 +56,7 @@ const analyze: JavaScriptSourceWorkerAnalyzer = async ( const ir = await completeJavaScriptAnalysisSteps( analyzeParsedJavaScriptSemanticsSteps( parsed, - moduleKindForJavaScriptReference(file.path, "static-import"), + moduleResolutionModeForJavaScriptReference(file.path, "static-import"), ), ); parsed = null; diff --git a/src/domain/javascript/javascriptModuleResolutionMode.ts b/src/domain/javascript/javascriptModuleResolutionMode.ts new file mode 100644 index 000000000..d14b91bd9 --- /dev/null +++ b/src/domain/javascript/javascriptModuleResolutionMode.ts @@ -0,0 +1,23 @@ +import { posix } from "node:path"; + +/** Loader evidence retained for a source-level module reference. */ +export type JavaScriptModuleResolutionMode = + | "import" + | "require" + | "compiler-dependent"; + +/** Derive source mode only when syntax or an explicit source format owns it. */ +export const moduleResolutionModeForJavaScriptReference = ( + sourcePath: string, + referenceKind: "static-import" | "dynamic-import" | "require", + explicitMode?: JavaScriptModuleResolutionMode, +): JavaScriptModuleResolutionMode => { + if (explicitMode !== undefined) return explicitMode; + if (referenceKind === "require") return "require"; + if (referenceKind === "dynamic-import") return "import"; + const extension = posix.extname(sourcePath).toLowerCase(); + if (extension === ".cts") return "require"; + if (extension === ".mts") return "import"; + if (extension === ".ts" || extension === ".tsx") return "compiler-dependent"; + return "import"; +}; diff --git a/src/domain/javascript/javascriptSemanticAnalysis.ts b/src/domain/javascript/javascriptSemanticAnalysis.ts index 35a8b7dc9..f6c6db25d 100644 --- a/src/domain/javascript/javascriptSemanticAnalysis.ts +++ b/src/domain/javascript/javascriptSemanticAnalysis.ts @@ -68,6 +68,7 @@ import { import { isJavaScriptRpcClientFactory } from "./javascriptRpcEndpoints.js"; import { createSemanticModuleInterop } from "./javascriptModuleInterop.js"; import { SemanticPropertyPathCoverage } from "./javascriptSemanticPropertyPaths.js"; +import type { JavaScriptModuleResolutionMode } from "./javascriptModuleResolutionMode.js"; interface BindPatternInput { readonly pattern: t.Node; @@ -138,7 +139,7 @@ export function* indexJavaScriptBindingsSteps( /** Recover semantics from an already parsed JavaScript artifact. */ export const analyzeParsedJavaScriptSemantics = ( file: ParsedJavaScriptSource, - staticImportResolutionMode: "import" | "require" = "import", + staticImportResolutionMode: JavaScriptModuleResolutionMode = "import", ): JavaScriptSemanticIr => completeSemanticSteps( analyzeParsedJavaScriptSemanticsSteps(file, staticImportResolutionMode), @@ -152,7 +153,7 @@ export const analyzeParsedJavaScriptSemantics = ( */ export function* analyzeParsedJavaScriptSemanticsSteps( file: ParsedJavaScriptSource, - staticImportResolutionMode: "import" | "require" = "import", + staticImportResolutionMode: JavaScriptModuleResolutionMode = "import", ): Generator { const state = createState(file.program, staticImportResolutionMode); yield* collectDefinitionsSteps(file.program, state); @@ -539,7 +540,7 @@ const bindingOrigin = ( const createState = ( program: t.Program, - staticImportResolutionMode: "import" | "require" = "import", + staticImportResolutionMode: JavaScriptModuleResolutionMode = "import", ): JavaScriptSemanticAnalysisState => { const root: JavaScriptSemanticScopeState = { scopeId: semanticScopeId("program", program), @@ -1216,8 +1217,11 @@ const addBinding = (input: AddBindingInput): void => { if (directOrigin !== undefined) binding.directOrigins.push( directOrigin.resolutionMode === undefined && - state.staticImportResolutionMode === "require" - ? { ...directOrigin, resolutionMode: "require" } + state.staticImportResolutionMode !== "import" + ? { + ...directOrigin, + resolutionMode: state.staticImportResolutionMode, + } : directOrigin, ); binding.runtimeImport ||= runtimeImport === true; diff --git a/src/domain/javascript/javascriptSemanticIr.ts b/src/domain/javascript/javascriptSemanticIr.ts index 18f1eb485..005471775 100644 --- a/src/domain/javascript/javascriptSemanticIr.ts +++ b/src/domain/javascript/javascriptSemanticIr.ts @@ -4,10 +4,11 @@ import type { JavaScriptSemanticCoverage, JavaScriptSemanticReturnCoverage, } from "./javascriptSemanticCoverage.js"; +import type { JavaScriptModuleResolutionMode } from "./javascriptModuleResolutionMode.js"; /** One exact module origin followed through imports, requires, or aliases. */ export interface JavaScriptModuleOrigin { - readonly resolutionMode?: "import" | "require"; + readonly resolutionMode?: JavaScriptModuleResolutionMode; readonly specifier: string; readonly importedPath: readonly string[]; } @@ -393,7 +394,7 @@ export interface JavaScriptSemanticFrontier { /** Static import/export relationship retained for cross-file composition. */ export interface JavaScriptSemanticModuleLink { - readonly resolutionMode?: "import" | "require"; + readonly resolutionMode?: JavaScriptModuleResolutionMode; readonly kind: | "import" | "type-import" diff --git a/src/domain/javascript/javascriptSemanticProjection.ts b/src/domain/javascript/javascriptSemanticProjection.ts index 9d252a0d8..b477bb5af 100644 --- a/src/domain/javascript/javascriptSemanticProjection.ts +++ b/src/domain/javascript/javascriptSemanticProjection.ts @@ -533,7 +533,20 @@ const addModuleLink = ( }, referenceNode: t.Node, ): void => { - const collected = { ...link, callableId: link.callableId ?? null }; + const resolutionMode = + link.resolutionMode ?? + (link.kind === "import" || + link.kind === "re-export" || + link.kind === "type-import" + ? state.staticImportResolutionMode + : link.kind === "require" + ? "require" + : undefined); + const collected = { + ...link, + ...(resolutionMode === undefined ? {} : { resolutionMode }), + callableId: link.callableId ?? null, + }; state.moduleLinks.push(collected); if (link.localName !== null) { const binding = resolveSemanticBindingState( diff --git a/src/domain/javascript/javascriptSemanticReturns.ts b/src/domain/javascript/javascriptSemanticReturns.ts index a3ca92ead..27bb030d0 100644 --- a/src/domain/javascript/javascriptSemanticReturns.ts +++ b/src/domain/javascript/javascriptSemanticReturns.ts @@ -123,9 +123,7 @@ export const resolveSemanticModuleCallables = ( : undefined; return { ...link, - ...(resolutionMode === "import" || resolutionMode === "require" - ? { resolutionMode } - : {}), + ...(resolutionMode !== undefined ? { resolutionMode } : {}), callableId: resolved.length === 1 ? (resolved[0] ?? null) : null, }; }); diff --git a/src/domain/javascript/javascriptSemanticState.ts b/src/domain/javascript/javascriptSemanticState.ts index b82abc62d..d01a4be52 100644 --- a/src/domain/javascript/javascriptSemanticState.ts +++ b/src/domain/javascript/javascriptSemanticState.ts @@ -12,6 +12,7 @@ import type { JavaScriptSemanticCallable, JavaScriptSemanticScope, } from "./javascriptSemanticIr.js"; +import type { JavaScriptModuleResolutionMode } from "./javascriptModuleResolutionMode.js"; /** Internal initializer plus a destructuring/member projection. */ interface JavaScriptSemanticInitializer { @@ -63,7 +64,7 @@ export interface JavaScriptSemanticScopeState { /** Shared state for semantic collection and evaluation. */ export interface JavaScriptSemanticAnalysisState { /** Effective loader for static imports in this source artifact. */ - readonly staticImportResolutionMode: "import" | "require"; + readonly staticImportResolutionMode: JavaScriptModuleResolutionMode; readonly parentsByNode: WeakMap; readonly scopes: JavaScriptSemanticScopeState[]; readonly scopesById: Map; diff --git a/src/javascript/analysis/JavaScriptAnalysisWorkerSchemas.ts b/src/javascript/analysis/JavaScriptAnalysisWorkerSchemas.ts index b666cfa9b..84a63d79b 100644 --- a/src/javascript/analysis/JavaScriptAnalysisWorkerSchemas.ts +++ b/src/javascript/analysis/JavaScriptAnalysisWorkerSchemas.ts @@ -475,7 +475,9 @@ const origin = z .strictObject({ specifier: z.string(), importedPath: strings, - resolutionMode: z.enum(["import", "require"]).optional(), + resolutionMode: z + .enum(["import", "require", "compiler-dependent"]) + .optional(), }) .transform(({ resolutionMode, ...rest }) => ({ ...rest, @@ -530,7 +532,9 @@ const coverage = z }); const moduleLink = z .strictObject({ - resolutionMode: z.enum(["import", "require"]).optional(), + resolutionMode: z + .enum(["import", "require", "compiler-dependent"]) + .optional(), kind: z.enum([ "import", "type-import", diff --git a/tests/boundary/filesystem/javascript-applications/javascriptModuleRelationships.test.ts b/tests/boundary/filesystem/javascript-applications/javascriptModuleRelationships.test.ts index 9b4df9992..cdd95bc97 100644 --- a/tests/boundary/filesystem/javascript-applications/javascriptModuleRelationships.test.ts +++ b/tests/boundary/filesystem/javascript-applications/javascriptModuleRelationships.test.ts @@ -152,7 +152,7 @@ describe("TypeScript source module loader semantics", () => { }); }); -describe("CommonJS and ESM module relationships", () => { +describe("JavaScript module relationships", () => { it("composes bindings, re-exports, dynamic imports, and JSON modules", async () => { const root = await moduleFixture(); const first = await reconstructJavaScriptFixture(root); @@ -288,6 +288,122 @@ describe("CommonJS and ESM module relationships", () => { }); }); +describe("Compiler-dependent TypeScript resolution", () => { + it("retains only compiler-independent targets for .ts static imports", async () => { + const root = await createTestTempDirectory("rea-ts-conditional-loader-"); + await writeFixtureFiles(root, { + "package.json": '{"type":"commonjs"}', + "main.ts": `import { value as different } from "different"; +import { value as same } from "same"; +void different; void same;`, + "node_modules/different/package.json": JSON.stringify({ + exports: { + ".": { + import: "./import.mjs", + require: "./require.cjs", + }, + }, + }), + "node_modules/different/import.mjs": "export const value = 'import';", + "node_modules/different/require.cjs": "exports.value = 'require';", + "node_modules/same/package.json": JSON.stringify({ + exports: { ".": { import: "./shared.js", require: "./shared.js" } }, + }), + "node_modules/same/shared.js": "export const value = 'shared';", + }); + + const { graph } = await reconstructJavaScriptFixture(root); + const different = findRelationshipEdge(graph, { + kind: "import", + specifier: "different", + localName: "different", + }); + const same = findRelationshipEdge(graph, { + kind: "import", + specifier: "same", + localName: "same", + resolvedPath: "node_modules/same/shared.js", + }); + const staticFinding = (specifier: string) => + graph.edges.find( + ({ relation, properties }) => + relation === "imports" && + properties.kind === "static-import" && + properties.specifier === specifier, + ); + + expect( + findSourceModule(graph, "main.ts")?.observations[0]?.properties, + ).toMatchObject({ module_format: "unknown" }); + expect(different).toMatchObject({ + properties: { + resolution_mode: "compiler-dependent", + resolution_status: "unavailable", + resolved_path: null, + conditional_resolutions: [ + { + mode: "import", + resolution_status: "resolved", + resolved_path: "node_modules/different/import.mjs", + }, + { + mode: "require", + resolution_status: "resolved", + resolved_path: "node_modules/different/require.cjs", + }, + ], + }, + }); + expect(same).toMatchObject({ + properties: { + resolution_mode: "compiler-dependent", + resolved_path: "node_modules/same/shared.js", + resolution_status: "resolved", + }, + evidence: { + limitations: expect.arrayContaining([ + expect.stringContaining( + "compiler-dependent without retained compiler options", + ), + ]), + }, + }); + expect(staticFinding("different")).toMatchObject({ + properties: { + resolution_mode: "compiler-dependent", + resolution_status: "unavailable", + conditional_resolutions: [ + { + mode: "import", + resolved_path: "node_modules/different/import.mjs", + }, + { + mode: "require", + resolved_path: "node_modules/different/require.cjs", + }, + ], + }, + evidence: { + limitations: expect.arrayContaining([ + expect.stringContaining( + "compiler-dependent without retained compiler options", + ), + ]), + }, + }); + expect(staticFinding("same")).toMatchObject({ + properties: { resolved_path: "node_modules/same/shared.js" }, + evidence: { + limitations: expect.arrayContaining([ + expect.stringContaining( + "compiler-dependent without retained compiler options", + ), + ]), + }, + }); + }); +}); + it("does not assign one module origin to an export with colliding path encodings", async () => { const root = await createTestTempDirectory("rea-ambiguous-module-origin-"); await writeFixtureFiles(root, { From 9cf49c7a784ace9b79e4b4c1022c9fbf2b3f77e1 Mon Sep 17 00:00:00 2001 From: jiangjiang248 Date: Tue, 13 Oct 2026 16:04:44 +0800 Subject: [PATCH 071/159] fix(snapshot): classify non-object roots as invalid content (#2685) Co-authored-by: jiangjiang248 <237556902+jiangjiang248@users.noreply.github.com> --- src/application/JsonInputFile.ts | 8 +- .../filesystem/analysisSnapshotInput.test.ts | 88 +++++++++++++++++++ 2 files changed, 94 insertions(+), 2 deletions(-) create mode 100644 tests/boundary/filesystem/analysisSnapshotInput.test.ts diff --git a/src/application/JsonInputFile.ts b/src/application/JsonInputFile.ts index a226cb0cc..25e5fcec0 100644 --- a/src/application/JsonInputFile.ts +++ b/src/application/JsonInputFile.ts @@ -78,7 +78,9 @@ export const readJsonObjectFile = ( value === null || Array.isArray(value) ) - throw new TypeError("JSON document root must be an object"); + return err( + invalidJson(operation, "JSON document root must be an object"), + ); for (const members of scans) for (const [key, member] of members) { if (!Object.hasOwn(value, key)) continue; @@ -177,7 +179,9 @@ export const readJsonObjectFile = ( signal === undefined ? {} : { signal }, ); if (!rootObject) - throw new TypeError("JSON document root must be an object"); + return err( + invalidJson(operation, "JSON document root must be an object"), + ); const digest = hash.digest("hex"); if (previousDigest !== undefined && digest !== previousDigest) throw new RegularFileChangedError(path); diff --git a/tests/boundary/filesystem/analysisSnapshotInput.test.ts b/tests/boundary/filesystem/analysisSnapshotInput.test.ts new file mode 100644 index 000000000..85bd7b7d1 --- /dev/null +++ b/tests/boundary/filesystem/analysisSnapshotInput.test.ts @@ -0,0 +1,88 @@ +import { readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { describe, expect, it } from "vitest"; + +import { readAnalysisSnapshot } from "../../../src/application/binary/AnalysisSnapshotFiles.js"; +import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; +import { + snapshotBinding, + snapshotTarget, +} from "../../../src/domain/analysisSnapshot.js"; +import { + ANALYSIS_SNAPSHOT_PROFILE, + ANALYSIS_SNAPSHOT_TARGET, +} from "../../../src/domain/analysisSnapshot.fixture.js"; +import { createEvidenceBundle } from "../../../src/domain/evidenceBundle.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +describe.each([ + { mode: "native", paddingBytes: 0 }, + { mode: "streamed", paddingBytes: 9 * 1024 * 1024 }, +])("analysis snapshot input ($mode)", ({ paddingBytes }) => { + it.each(["[]", "null", "5", '"x"', "true"])( + "classifies a readable %s root as invalid content, not an I/O failure", + async (document) => { + const root = await createTestTempDirectory("rea-snapshot-root-"); + const path = join(root, "snapshot.json"); + const encoded = `${" ".repeat(paddingBytes)}${document}`; + await writeFile(path, encoded); + + const loaded = await readAnalysisSnapshot(path); + if (loaded.ok) throw new Error("Expected a non-object root rejection"); + const projected = projectAnalysisError(loaded.error); + expect(projected).toMatchObject({ + code: "evidence_integrity_mismatch", + details: { reason: "invalid-json" }, + }); + expect(projected.message).toContain("Repair or recreate"); + expect(loaded.error.cause).toMatchObject({ + message: "JSON document root must be an object", + }); + expect(await readFile(path, "utf8")).toBe(encoded); + }, + ); + + it("retains syntax and snapshot-member validation and loads valid objects", async () => { + const root = await createTestTempDirectory("rea-snapshot-validation-"); + const path = join(root, "snapshot.json"); + const prefix = " ".repeat(paddingBytes); + await writeFile(path, `${prefix}[`); + expect(await readAnalysisSnapshot(path)).toMatchObject({ + ok: false, + error: { _tag: "EvidenceFileError", reason: "invalid-json" }, + }); + await writeFile(path, `${prefix}{}`); + expect(await readAnalysisSnapshot(path)).toMatchObject({ + ok: false, + error: { _tag: "EvidenceIntegrityError" }, + }); + const snapshot = { + target: snapshotTarget(ANALYSIS_SNAPSHOT_TARGET), + binding: snapshotBinding(ANALYSIS_SNAPSHOT_PROFILE), + entries: [], + workflow_entries: [], + evidence_bundle: createEvidenceBundle([]), + }; + await writeFile(path, `${prefix}${JSON.stringify(snapshot)}`); + expect(await readAnalysisSnapshot(path)).toEqual({ + ok: true, + value: snapshot, + }); + }); +}); + +it("keeps file-selection failures distinct from invalid snapshot contents", async () => { + const root = await createTestTempDirectory("rea-snapshot-selection-"); + for (const [path, reason] of [ + [join(root, "missing.json"), "missing"], + [root, "not-file"], + ] as const) { + const loaded = await readAnalysisSnapshot(path); + if (loaded.ok) throw new Error("Expected a file-selection failure"); + expect(projectAnalysisError(loaded.error)).toMatchObject({ + code: "invalid_request", + details: { reason }, + }); + } +}); From b955b825fc002a4bb408f2910b78768c566c2cd1 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 16:08:41 +0800 Subject: [PATCH 072/159] fix(artifacts): retain read cleanup ownership through CLI and MCP shutdown Keep failed stable-read handles and WABT runtime roots under retryable resource scopes. Preserve cancellation causes and primary errors, close the same service instances bound to MCP tools, and consolidate CLI operation/cleanup projection. Validation: fast typecheck and lint, cached runtime build, docs checks, SDK schema validation, focused service and real-file cleanup tests, CLI cleanup regressions, and MCP lifecycle/shutdown workflows. Provider process fixtures simulate engine output; real WABT and Linux EVM execution are not established by this pass. --- src/application/WebModulePorts.ts | 1 + src/application/WebModuleTraceService.ts | 23 +- src/application/WebNetworkCapturePort.ts | 2 + src/application/WebNetworkCaptureService.ts | 5 + src/application/WebScriptArtifactPort.ts | 1 + src/application/WebSourceLocationPorts.ts | 1 + src/application/WebSourceLocationService.ts | 23 +- .../binaryDiagnostics/PeResourcesPort.ts | 1 + .../binaryDiagnostics/PeResourcesService.ts | 4 + src/application/go/GoBinaryPort.ts | 1 + src/application/go/GoBinaryService.ts | 4 + src/artifacts/ArtifactResourceScope.ts | 4 + src/artifacts/readStableArtifact.ts | 12 +- src/browser/assets/LocalWebScriptArtifacts.ts | 32 ++- src/browser/assets/WebArtifactReads.ts | 36 ++- .../assets/WebSourceLocationArtifacts.ts | 61 ++++- .../history/HistoricalCaptureDecoder.ts | 94 ++++++- src/browser/modules/WebModuleArtifacts.ts | 60 ++++- src/cli/binaryDiagnosticsCommands.ts | 39 +-- src/cli/captureCommands.ts | 65 ++--- src/cli/finishAnalysisCommand.ts | 69 +++++ src/cli/goCommands.ts | 26 +- src/cli/signalCommands.ts | 65 ++--- src/cli/webNetworkCaptureCommands.ts | 32 ++- src/evm/EvmoleFailures.ts | 57 +++-- src/evm/EvmoleInterfaceProvider.ts | 116 +++++++-- src/go/GoBinaryProvider.ts | 53 +++- src/native/pe/PeResourcesProvider.ts | 61 ++++- src/native/pwndbg/PwndbgCoreContext.ts | 32 ++- src/native/pwntools/PwntoolsDecoder.ts | 125 +++++++-- .../pwntools/PwntoolsRecordedCrashProvider.ts | 2 + src/native/pwntools/PwntoolsResourceLimits.ts | 8 + src/server/createServer.ts | 23 +- src/wasm/WabtArtifactProvider.test.ts | 241 ++++++++++++++---- src/wasm/WabtArtifactProvider.ts | 94 +++++-- src/wasm/WabtFailures.ts | 27 +- .../browser/historicalCaptureDecoder.test.ts | 63 ++++- tests/boundary/cli/captureCleanup.test.ts | 106 ++++++++ .../filesystem/stableArtifact.test.ts | 9 +- .../webArtifactReadOwnership.test.ts | 96 +++++++ tests/boundary/mcp/serverLifecycleMcp.test.ts | 65 +++++ .../boundary/native/PwndbgCoreContext.test.ts | 2 + .../native/pwntoolsArtifactCleanup.test.ts | 147 +++++++++++ .../artifactReadCleanupRetry.test.ts | 100 ++++++++ 44 files changed, 1758 insertions(+), 330 deletions(-) create mode 100644 src/cli/finishAnalysisCommand.ts create mode 100644 tests/boundary/filesystem/webArtifactReadOwnership.test.ts create mode 100644 tests/boundary/native/pwntoolsArtifactCleanup.test.ts create mode 100644 tests/boundary/providers/artifactReadCleanupRetry.test.ts diff --git a/src/application/WebModulePorts.ts b/src/application/WebModulePorts.ts index 81c94a161..2f03efce0 100644 --- a/src/application/WebModulePorts.ts +++ b/src/application/WebModulePorts.ts @@ -22,6 +22,7 @@ export interface WebModuleArtifacts extends SelectedWebScriptArtifacts { /** Read selected local artifacts without refetching website assets. */ export interface WebModuleArtifactPort { + close?(): Promise; load( input: WebModuleTraceInput, options?: ExecutionOptions, diff --git a/src/application/WebModuleTraceService.ts b/src/application/WebModuleTraceService.ts index f8aa0f8b9..7913158f6 100644 --- a/src/application/WebModuleTraceService.ts +++ b/src/application/WebModuleTraceService.ts @@ -70,7 +70,13 @@ export class WebModuleTraceService { ) {} close(): Promise { - return this.resolver.close?.() ?? Promise.resolve(); + return closeOwnedWebResources( + [ + this.artifacts.close?.bind(this.artifacts), + this.resolver.close?.bind(this.resolver), + ], + "Web module trace cleanup failed", + ); } async trace( @@ -265,6 +271,21 @@ export class WebModuleTraceService { } } +const closeOwnedWebResources = async ( + close: readonly ((() => Promise) | undefined)[], + message: string, +): Promise => { + const results = await Promise.allSettled( + close.flatMap((owner) => + owner === undefined ? [] : [Promise.resolve().then(owner)], + ), + ); + const failures = results.flatMap((result) => + result.status === "rejected" ? [result.reason] : [], + ); + if (failures.length > 0) throw new AggregateError(failures, message); +}; + const admitContext = ( value: string, path: readonly string[], diff --git a/src/application/WebNetworkCapturePort.ts b/src/application/WebNetworkCapturePort.ts index 5f9cd5545..2bd0f1968 100644 --- a/src/application/WebNetworkCapturePort.ts +++ b/src/application/WebNetworkCapturePort.ts @@ -12,4 +12,6 @@ export interface WebNetworkCapturePort { input: InspectWebNetworkCaptureInput, options?: ExecutionOptions, ): Promise>; + /** Retry retained reader/runtime cleanup when the owning service shuts down. */ + close?(): Promise; } diff --git a/src/application/WebNetworkCaptureService.ts b/src/application/WebNetworkCaptureService.ts index 8e29ebb10..5a3d5cfea 100644 --- a/src/application/WebNetworkCaptureService.ts +++ b/src/application/WebNetworkCaptureService.ts @@ -27,6 +27,11 @@ const OPERATION = "inspect_web_network_capture"; export class WebNetworkCaptureService { constructor(readonly port: WebNetworkCapturePort) {} + /** Release resources retained by the selected capture adapter. */ + close(): Promise { + return this.port.close?.() ?? Promise.resolve(); + } + /** Return selected complete records inline, without inventing live browser attribution. */ async inspect( rawInput: unknown, diff --git a/src/application/WebScriptArtifactPort.ts b/src/application/WebScriptArtifactPort.ts index 02214d006..77dcd9cfc 100644 --- a/src/application/WebScriptArtifactPort.ts +++ b/src/application/WebScriptArtifactPort.ts @@ -8,6 +8,7 @@ import type { ExecutionOptions } from "./AnalysisProvider.js"; /** Read the caller-selected capture export and verify the selected source identity. */ export interface WebScriptArtifactPort { + close?(): Promise; load( input: SelectedWebScriptInput, options?: ExecutionOptions, diff --git a/src/application/WebSourceLocationPorts.ts b/src/application/WebSourceLocationPorts.ts index 04fe1742c..1d21783cc 100644 --- a/src/application/WebSourceLocationPorts.ts +++ b/src/application/WebSourceLocationPorts.ts @@ -20,6 +20,7 @@ export interface WebSourceLocationArtifacts extends SelectedWebScriptArtifacts { /** Replaceable artifact acquisition boundary for a caller-selected pairing. */ export interface WebSourceLocationArtifactPort { + close?(): Promise; load( input: WebSourceLocationInput, options?: ExecutionOptions, diff --git a/src/application/WebSourceLocationService.ts b/src/application/WebSourceLocationService.ts index 7be901311..79b7d45fc 100644 --- a/src/application/WebSourceLocationService.ts +++ b/src/application/WebSourceLocationService.ts @@ -71,7 +71,13 @@ export class WebSourceLocationService { ) {} close(): Promise { - return this.decoder.close?.() ?? Promise.resolve(); + return closeOwnedWebResources( + [ + this.artifacts.close?.bind(this.artifacts), + this.decoder.close?.bind(this.decoder), + ], + "Web source-location cleanup failed", + ); } /** Shared CLI/MCP workflow preserves raw declarations and every equal-position match. */ @@ -174,6 +180,21 @@ export class WebSourceLocationService { ); } } + +const closeOwnedWebResources = async ( + close: readonly ((() => Promise) | undefined)[], + message: string, +): Promise => { + const results = await Promise.allSettled( + close.flatMap((owner) => + owner === undefined ? [] : [Promise.resolve().then(owner)], + ), + ); + const failures = results.flatMap((result) => + result.status === "rejected" ? [result.reason] : [], + ); + if (failures.length > 0) throw new AggregateError(failures, message); +}; const invalidOutput = (message: string): Result => err(new AnalysisOutputError(OPERATION, message)); const invalid = ( diff --git a/src/application/binaryDiagnostics/PeResourcesPort.ts b/src/application/binaryDiagnostics/PeResourcesPort.ts index b1a7b53de..ee36d34f3 100644 --- a/src/application/binaryDiagnostics/PeResourcesPort.ts +++ b/src/application/binaryDiagnostics/PeResourcesPort.ts @@ -12,6 +12,7 @@ import type { Result } from "../../domain/result.js"; /** Stable read and PE decoding boundary, independent of an active disassembler session. */ export interface PeResourcesPort { readonly identity: ProviderIdentity; + close?(): Promise; inspect( input: InspectPeResourcesInput, options?: ExecutionOptions, diff --git a/src/application/binaryDiagnostics/PeResourcesService.ts b/src/application/binaryDiagnostics/PeResourcesService.ts index 3c8a53704..87de771a4 100644 --- a/src/application/binaryDiagnostics/PeResourcesService.ts +++ b/src/application/binaryDiagnostics/PeResourcesService.ts @@ -21,6 +21,10 @@ import { err, ok, type Result } from "../../domain/result.js"; export class PeResourcesService { constructor(readonly provider: PeResourcesPort) {} + close(): Promise { + return this.provider.close?.() ?? Promise.resolve(); + } + async inspect( rawInput: unknown, options?: ExecutionOptions, diff --git a/src/application/go/GoBinaryPort.ts b/src/application/go/GoBinaryPort.ts index 2094e9109..cef925921 100644 --- a/src/application/go/GoBinaryPort.ts +++ b/src/application/go/GoBinaryPort.ts @@ -12,6 +12,7 @@ import type { Result } from "../../domain/result.js"; /** Replaceable offline build-information reader with no target-execution authority. */ export interface GoBinaryPort { readonly identity: ProviderIdentity; + close?(): Promise; inspect( input: InspectGoBinaryInput, options?: ExecutionOptions, diff --git a/src/application/go/GoBinaryService.ts b/src/application/go/GoBinaryService.ts index f5642c96c..2b9928f82 100644 --- a/src/application/go/GoBinaryService.ts +++ b/src/application/go/GoBinaryService.ts @@ -34,6 +34,10 @@ const architectureForSubject = (architecture: string) => { export class GoBinaryService { constructor(readonly provider: GoBinaryPort) {} + close(): Promise { + return this.provider.close?.() ?? Promise.resolve(); + } + /** Inspect one explicitly selected file without acquiring a binary session. */ async inspect( rawInput: unknown, diff --git a/src/artifacts/ArtifactResourceScope.ts b/src/artifacts/ArtifactResourceScope.ts index 7c0bcf2c2..872899631 100644 --- a/src/artifacts/ArtifactResourceScope.ts +++ b/src/artifacts/ArtifactResourceScope.ts @@ -57,6 +57,10 @@ export class ArtifactResourceScope { #draining: Promise | undefined; #closed = false; + get hasPending(): boolean { + return this.#pending.size > 0; + } + /** Admit a resource operation while the scope remains open. */ async run(operation: () => Promise): Promise { while (true) { diff --git a/src/artifacts/readStableArtifact.ts b/src/artifacts/readStableArtifact.ts index e35a20783..808133d89 100644 --- a/src/artifacts/readStableArtifact.ts +++ b/src/artifacts/readStableArtifact.ts @@ -38,6 +38,13 @@ const afterValidation = async ( } }; +const assertReadActive = (signal: AbortSignal | undefined): void => { + if (signal?.aborted) + throw new ArtifactReaderFailure("cancelled", "Artifact read cancelled", { + cause: signal.reason, + }); +}; + /** Read one bounded regular file and reject replacement or in-place changes. */ export const readStableArtifact = async ( path: string, @@ -46,7 +53,7 @@ export const readStableArtifact = async ( fileSystem: StableArtifactFileSystem = FILE_SYSTEM, resourceScope?: ArtifactResourceScope, ): Promise<{ readonly bytes: Buffer; readonly sha256: string }> => { - signal?.throwIfAborted(); + assertReadActive(signal); const before = await fileSystem.lstat(path); if (!before.isFile() || before.isSymbolicLink()) throw new ArtifactReaderFailure( @@ -84,7 +91,7 @@ export const readStableArtifact = async ( autoClose: false, highWaterMark: 65536, })) { - signal?.throwIfAborted(); + assertReadActive(signal); const value: unknown = chunk; if (!Buffer.isBuffer(value)) throw new ArtifactReaderFailure( @@ -120,6 +127,7 @@ export const readStableArtifact = async ( "integrity", `Artifact changed during read: ${path}`, ); + assertReadActive(signal); const bytes = Buffer.concat(chunks, count); return { bytes, sha256: createHash("sha256").update(bytes).digest("hex") }; } catch (cause: unknown) { diff --git a/src/browser/assets/LocalWebScriptArtifacts.ts b/src/browser/assets/LocalWebScriptArtifacts.ts index c5d6ea8ed..51fc14c45 100644 --- a/src/browser/assets/LocalWebScriptArtifacts.ts +++ b/src/browser/assets/LocalWebScriptArtifacts.ts @@ -1,6 +1,7 @@ import { lstat, realpath } from "node:fs/promises"; import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js"; +import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js"; import { readStableArtifact } from "../../artifacts/readStableArtifact.js"; import { normalizeArtifactPath } from "../../artifacts/ArtifactPaths.js"; import type { WebScriptArtifactPort } from "../../application/WebScriptArtifactPort.js"; @@ -19,10 +20,36 @@ import { /** Read one digest-verified captured source under the manifest's current location. */ export class LocalWebScriptArtifacts implements WebScriptArtifactPort { - constructor(readonly operation: WebArtifactReadContext["operation"]) {} + constructor( + readonly operation: WebArtifactReadContext["operation"], + readonly resources = new ArtifactResourceScope(), + ) {} + + close(): Promise { + return this.resources.close(); + } /** Read the selected source; relocated exports retain their reported original root. */ async load(input: SelectedWebScriptInput, options?: ExecutionOptions) { + try { + return await this.resources.run(() => this.#load(input, options)); + } catch (cause: unknown) { + return err( + webArtifactReadError( + cause, + { + operation: this.operation, + field: ["manifest_path"], + targetPath: input.manifest_path, + callerSelected: true, + }, + options?.signal, + ), + ); + } + } + + async #load(input: SelectedWebScriptInput, options?: ExecutionOptions) { let field: readonly (string | number)[] = ["manifest_path"]; let targetPath = input.manifest_path; let callerSelected = true; @@ -31,6 +58,7 @@ export class LocalWebScriptArtifacts implements WebScriptArtifactPort { input.manifest_path, 32 * 1024 * 1024, options?.signal, + this.resources, ); const manifest = webScriptExportManifestSchema.parse(manifestBytes.value); field = ["script_index"]; @@ -65,6 +93,8 @@ export class LocalWebScriptArtifacts implements WebScriptArtifactPort { sourcePath, 16 * 1024 * 1024, options?.signal, + undefined, + this.resources, ); await assertContained(root, sourcePath); if ( diff --git a/src/browser/assets/WebArtifactReads.ts b/src/browser/assets/WebArtifactReads.ts index cd142f62f..e045e1575 100644 --- a/src/browser/assets/WebArtifactReads.ts +++ b/src/browser/assets/WebArtifactReads.ts @@ -1,5 +1,6 @@ import { z } from "zod"; import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js"; +import type { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js"; import { readStableArtifact } from "../../artifacts/readStableArtifact.js"; import type { AnalysisError } from "../../domain/analysisErrorBase.js"; import { @@ -9,6 +10,8 @@ import { } from "../../domain/analysisErrorCore.js"; import { ArtifactOperationError } from "../../domain/artifactOperationError.js"; import { ProviderAdapterError } from "../../domain/providerAdapterError.js"; +import { ProviderCleanupError } from "../../domain/providerCleanupError.js"; +import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js"; import { safeParseJson } from "../../domain/safeJson.js"; /** The selected artifact and input field owning a reader failure. */ @@ -42,8 +45,15 @@ export const readWebArtifactJson = async ( path: string, maximumBytes: number, signal?: AbortSignal, + resourceScope?: ArtifactResourceScope, ) => { - const data = await readStableArtifact(path, maximumBytes, signal); + const data = await readStableArtifact( + path, + maximumBytes, + signal, + undefined, + resourceScope, + ); const value = safeParseJson(decodeWebArtifact(data.bytes)); if (!value.ok) throw new WebArtifactFormatFailure(value.error); return { ...data, value: value.value }; @@ -55,8 +65,30 @@ export const webArtifactReadError = ( context: WebArtifactReadContext, signal?: AbortSignal, ): AnalysisError => { + if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined) { + const primary = + signal?.aborted === true + ? new AnalysisCancelledError(context.operation, { cause }) + : new ArtifactOperationError( + context.operation, + cause.reason, + cause.details, + cause.message, + { cause }, + ); + return analysisErrorWithCleanupFailure( + primary, + new ProviderCleanupError( + "captured-web-artifacts", + cause.cleanup.resources, + { reason: cause.cleanup.reason }, + { cause, operation: context.operation }, + ), + context.operation, + ); + } if (signal?.aborted === true) - return new AnalysisCancelledError(context.operation); + return new AnalysisCancelledError(context.operation, { cause }); const selection = context.callerSelected ? selectedFileError(cause, context) : undefined; diff --git a/src/browser/assets/WebSourceLocationArtifacts.ts b/src/browser/assets/WebSourceLocationArtifacts.ts index 21c483bd8..733069469 100644 --- a/src/browser/assets/WebSourceLocationArtifacts.ts +++ b/src/browser/assets/WebSourceLocationArtifacts.ts @@ -1,6 +1,7 @@ import type { WebSourceLocationArtifactPort } from "../../application/WebSourceLocationPorts.js"; import type { WebScriptArtifactPort } from "../../application/WebScriptArtifactPort.js"; import type { ExecutionOptions } from "../../application/AnalysisProvider.js"; +import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js"; import { readStableArtifact } from "../../artifacts/readStableArtifact.js"; import { WEB_SOURCE_MAP_LIMITS, @@ -12,14 +13,64 @@ import { decodeWebArtifact, webArtifactReadError } from "./WebArtifactReads.js"; /** Read one verified capture source and the caller-selected map, without fetching. */ export class LocalWebSourceLocationArtifacts implements WebSourceLocationArtifactPort { + readonly scripts: WebScriptArtifactPort; + readonly #resources: ArtifactResourceScope; + constructor( - readonly scripts: WebScriptArtifactPort = new LocalWebScriptArtifacts( - "trace_web_source_location", - ), - ) {} + scripts?: WebScriptArtifactPort, + resources = scripts instanceof LocalWebScriptArtifacts + ? scripts.resources + : new ArtifactResourceScope(), + ) { + this.#resources = resources; + this.scripts = + scripts ?? + new LocalWebScriptArtifacts("trace_web_source_location", resources); + } + + async close(): Promise { + const closeScript = + this.scripts instanceof LocalWebScriptArtifacts && + this.scripts.resources === this.#resources + ? undefined + : this.scripts.close?.bind(this.scripts); + const results = await Promise.allSettled([ + Promise.resolve().then(() => this.#resources.close()), + ...(closeScript === undefined + ? [] + : [Promise.resolve().then(closeScript)]), + ]); + const failures = results.flatMap((result) => + result.status === "rejected" ? [result.reason] : [], + ); + if (failures.length > 0) + throw new AggregateError( + failures, + "Web source-location artifact cleanup failed", + ); + } /** Preserve actual map identity separately from the selected deployment URL. */ async load(input: WebSourceLocationInput, options?: ExecutionOptions) { + try { + return await this.#resources.run(() => this.#load(input, options)); + } catch (cause: unknown) { + return err( + webArtifactReadError( + cause, + { + operation: "trace_web_source_location", + field: ["source_map", "path"], + targetPath: input.source_map.path, + callerSelected: true, + }, + options?.signal, + ), + ); + } + } + + async #load(input: WebSourceLocationInput, options?: ExecutionOptions) { const loaded = await this.scripts.load(input, options); if (!loaded.ok) return loaded; try { @@ -27,6 +78,8 @@ export class LocalWebSourceLocationArtifacts implements WebSourceLocationArtifac input.source_map.path, WEB_SOURCE_MAP_LIMITS.mapBytes, options?.signal, + undefined, + this.#resources, ); return ok({ ...loaded.value, diff --git a/src/browser/history/HistoricalCaptureDecoder.ts b/src/browser/history/HistoricalCaptureDecoder.ts index 7f6e7b83a..85a91ef76 100644 --- a/src/browser/history/HistoricalCaptureDecoder.ts +++ b/src/browser/history/HistoricalCaptureDecoder.ts @@ -17,6 +17,7 @@ import { z } from "zod"; import type { ExecutionOptions } from "../../application/AnalysisProvider.js"; import { readStableArtifact } from "../../artifacts/readStableArtifact.js"; import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js"; +import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js"; import type { AnalysisError } from "../../domain/analysisErrorBase.js"; import { AnalysisCapabilityUnavailableError, @@ -26,6 +27,7 @@ import { } from "../../domain/analysisErrorCore.js"; import { ProviderAdapterError } from "../../domain/providerAdapterError.js"; import { ProviderCleanupError } from "../../domain/providerCleanupError.js"; +import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js"; import { err, ok, type Result } from "../../domain/result.js"; import { WEB_NETWORK_CAPTURE_LIMITS, @@ -60,18 +62,52 @@ const replySchema = z.discriminatedUnion("ok", [ /** One owned snapshot/process/root lifecycle for replaceable historical format decoders. */ export class HistoricalCaptureDecoder { + readonly #resources: ArtifactResourceScope; + constructor( readonly adapters: readonly HistoricalCaptureFormatAdapter[], readonly environment: Readonly, readonly createRuntime: () => Promise< Pick > = () => PrivateRuntimeRoot.create({ prefix: "rea-web-capture-" }), - ) {} + resources = new ArtifactResourceScope(), + ) { + this.#resources = resources; + } + + /** Retry retained readers and private roots at service shutdown. */ + close(): Promise { + return this.#resources.close(); + } /** Snapshot the selected artifact, decode it offline, and release private credentials and processes. */ async inspect( input: InspectWebNetworkCaptureInput, options?: ExecutionOptions, + ): Promise> { + try { + return await this.#resources.run(() => + this.inspectAdmitted(input, options), + ); + } catch (cause: unknown) { + const error = historicalCaptureFailure( + input, + cause, + "capture-read", + options, + ); + return err( + redactExplicitFailure( + this.withReaderCleanup(error, cause), + input.sensitive_values, + ), + ); + } + } + + private async inspectAdmitted( + input: InspectWebNetworkCaptureInput, + options?: ExecutionOptions, ): Promise> { let runtime: Pick | undefined; let result: Result; @@ -90,6 +126,8 @@ export class HistoricalCaptureDecoder { input.capture_path, WEB_NETWORK_CAPTURE_LIMITS.inputBytes, options?.signal, + undefined, + this.#resources, ); phase = "decoder"; runtime = await this.createRuntime(); @@ -139,9 +177,16 @@ export class HistoricalCaptureDecoder { replyPath, WEB_NETWORK_CAPTURE_LIMITS.outputBytes, options?.signal, + undefined, + this.#resources, ); reply = replySchema.parse(JSON.parse(replyFile.bytes.toString("utf8"))); } catch (cause: unknown) { + if ( + cause instanceof ArtifactReaderFailure && + cause.cleanup !== undefined + ) + throw cause; if (options?.signal?.aborted) throw new AnalysisCancelledError(OPERATION); throw new AnalysisOutputError( @@ -230,17 +275,21 @@ export class HistoricalCaptureDecoder { result = ok({ ...reply.value, ...common, format: input.format }); } } catch (cause: unknown) { + const error = historicalCaptureFailure(input, cause, phase, options); result = err( redactExplicitFailure( - historicalCaptureFailure(input, cause, phase, options), + this.withReaderCleanup(error, cause), input.sensitive_values, ), ); } if (runtime !== undefined) { - try { - await runtime.close(); - } catch (cause: unknown) { + const cleanup = await this.#resources.release({ + kind: "private-runtime-root", + resource: runtime.path, + root: runtime, + }); + if (cleanup.kind === "failed") { return err( redactExplicitFailure( new ProviderCleanupError( @@ -251,9 +300,12 @@ export class HistoricalCaptureDecoder { previous_error: result.ok ? null : projectAnalysisError(result.error), - reason: cause instanceof Error ? cause.message : String(cause), + reason: + cleanup.cause instanceof Error + ? cleanup.cause.message + : String(cleanup.cause), }, - { operation: OPERATION }, + { operation: OPERATION, cause: cleanup.cause }, ), input.sensitive_values, ), @@ -264,6 +316,34 @@ export class HistoricalCaptureDecoder { ? err(new AnalysisCancelledError(OPERATION)) : result; } + + private withReaderCleanup( + primary: AnalysisError, + cause: unknown, + ): AnalysisError { + if ( + !(cause instanceof ArtifactReaderFailure) || + cause.cleanup === undefined + ) + return primary; + if (cause.reason === "unavailable") + return new ProviderCleanupError( + "web-network-capture", + cause.cleanup.resources, + { reason: cause.cleanup.reason }, + { operation: OPERATION, cause }, + ); + const cleanup = new ProviderCleanupError( + "web-network-capture", + cause.cleanup.resources, + { + reason: cause.cleanup.reason, + previous_error: projectAnalysisError(primary), + }, + { operation: OPERATION, cause }, + ); + return analysisErrorWithCleanupFailure(primary, cleanup, OPERATION); + } } /** The Node capture decoder runs on this executable; only Node.js enforces its V8 heap flags. */ diff --git a/src/browser/modules/WebModuleArtifacts.ts b/src/browser/modules/WebModuleArtifacts.ts index 287e6e8de..928025f09 100644 --- a/src/browser/modules/WebModuleArtifacts.ts +++ b/src/browser/modules/WebModuleArtifacts.ts @@ -4,6 +4,7 @@ import type { } from "../../application/WebModulePorts.js"; import type { WebScriptArtifactPort } from "../../application/WebScriptArtifactPort.js"; import type { ExecutionOptions } from "../../application/AnalysisProvider.js"; +import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js"; import { jsonObjectSchema } from "../../domain/jsonValue.js"; import { err, ok } from "../../domain/result.js"; import type { WebModuleTraceInput } from "../../domain/webModuleTrace.js"; @@ -15,14 +16,64 @@ import { /** Add an explicitly selected import map to a verified captured-script source. */ export class LocalWebModuleArtifacts implements WebModuleArtifactPort { + readonly scripts: WebScriptArtifactPort; + readonly #resources: ArtifactResourceScope; + constructor( - readonly scripts: WebScriptArtifactPort = new LocalWebScriptArtifacts( - "trace_web_module_imports", - ), - ) {} + scripts?: WebScriptArtifactPort, + resources = scripts instanceof LocalWebScriptArtifacts + ? scripts.resources + : new ArtifactResourceScope(), + ) { + this.#resources = resources; + this.scripts = + scripts ?? + new LocalWebScriptArtifacts("trace_web_module_imports", resources); + } + + async close(): Promise { + const closeScript = + this.scripts instanceof LocalWebScriptArtifacts && + this.scripts.resources === this.#resources + ? undefined + : this.scripts.close?.bind(this.scripts); + const results = await Promise.allSettled([ + Promise.resolve().then(() => this.#resources.close()), + ...(closeScript === undefined + ? [] + : [Promise.resolve().then(closeScript)]), + ]); + const failures = results.flatMap((result) => + result.status === "rejected" ? [result.reason] : [], + ); + if (failures.length > 0) + throw new AggregateError(failures, "Web module artifact cleanup failed"); + } /** Read selected source and optional map without website requests. */ async load(input: WebModuleTraceInput, options?: ExecutionOptions) { + try { + return await this.#resources.run(() => this.#load(input, options)); + } catch (cause: unknown) { + return err( + webArtifactReadError( + cause, + { + operation: "trace_web_module_imports", + field: + input.import_map === undefined + ? ["manifest_path"] + : ["import_map", "path"], + targetPath: input.import_map?.path ?? input.manifest_path, + callerSelected: input.import_map !== undefined, + }, + options?.signal, + ), + ); + } + } + + async #load(input: WebModuleTraceInput, options?: ExecutionOptions) { const loaded = await this.scripts.load(input, options); if (!loaded.ok) return loaded; if (input.import_map === undefined) @@ -33,6 +84,7 @@ export class LocalWebModuleArtifacts implements WebModuleArtifactPort { selected.path, 4 * 1024 * 1024, options?.signal, + this.#resources, ); const importMap: WebModuleArtifacts["importMap"] = { file: { diff --git a/src/cli/binaryDiagnosticsCommands.ts b/src/cli/binaryDiagnosticsCommands.ts index bddd16bed..5cb9577bd 100644 --- a/src/cli/binaryDiagnosticsCommands.ts +++ b/src/cli/binaryDiagnosticsCommands.ts @@ -7,11 +7,12 @@ import { } from "../composition/binaryDiagnostics.js"; import { CLI_COMMANDS } from "../cliCommandNames.js"; import { logCliCommand } from "../cliLogging.js"; -import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; import type { Logger } from "pino"; import type { CliInstance } from "./types.js"; import { withCommandCancellation } from "./commandCancellation.js"; import { inspectPeResourcesInputSchema } from "../domain/native/peResources.js"; +import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; +import { finishAnalysisCommand } from "./finishAnalysisCommand.js"; /** Inspect operator-selected files through the same shared workflow as MCP. */ export const registerBinaryDiagnosticsCommands = ( @@ -19,7 +20,6 @@ export const registerBinaryDiagnosticsCommands = ( logger: Logger, environment: Readonly, ): void => { - const pe = createPeResourcesService(); cli.command(CLI_COMMANDS.inspectPeResources, { description: "Inspect PE resource identities, payload ranges, hashes and icon-group relationships", @@ -29,19 +29,28 @@ export const registerBinaryDiagnosticsCommands = ( maxEntries: inspectPeResourcesInputSchema.shape.max_entries, }), run: ({ args, options }) => - withCommandCancellation((signal) => - logCliCommand(logger, CLI_COMMANDS.inspectPeResources, async () => { - const result = await pe.inspect( - { - path: resolve(args.path), - max_file_bytes: options.maxFileBytes, - max_entries: options.maxEntries, - }, - { signal }, - ); - return result.ok ? result.value : projectAnalysisError(result.error); - }), - ), + withCommandCancellation(async (signal) => { + const pe = createPeResourcesService(); + return await logCliCommand( + logger, + CLI_COMMANDS.inspectPeResources, + async () => + finishAnalysisCommand( + () => + pe.inspect( + { + path: resolve(args.path), + max_file_bytes: options.maxFileBytes, + max_entries: options.maxEntries, + }, + { signal }, + ), + pe, + "rea-pe-resources", + "inspect_pe_resources", + ), + ); + }), }); const service = createBinaryLayoutService(environment); cli.command(CLI_COMMANDS.inspectBinaryLayout, { diff --git a/src/cli/captureCommands.ts b/src/cli/captureCommands.ts index 2a3566703..542361e90 100644 --- a/src/cli/captureCommands.ts +++ b/src/cli/captureCommands.ts @@ -1,9 +1,6 @@ import { inspectPacketFieldsInputSchema } from "../domain/captures/packetFields.js"; import { z } from "incur"; import type { PacketCaptureService } from "../application/captures/PacketCaptureService.js"; -import { AnalysisError } from "../domain/analysisErrorBase.js"; -import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js"; -import { ProviderCleanupError } from "../domain/providerCleanupError.js"; import { resolve } from "node:path"; import { createPacketCaptureService } from "../composition/captures.js"; import { @@ -13,10 +10,10 @@ import { } from "../domain/captures/packetCapture.js"; import { CLI_COMMANDS } from "../cliCommandNames.js"; import { logCliCommand } from "../cliLogging.js"; -import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; import type { Logger } from "pino"; import type { CliInstance } from "./types.js"; import { withCommandCancellation } from "./commandCancellation.js"; +import { finishAnalysisCommand } from "./finishAnalysisCommand.js"; export const registerCaptureCommands = ( cli: CliInstance, logger: Logger, @@ -26,32 +23,6 @@ export const registerCaptureCommands = ( "inspect" | "inspectFields" | "close" > = createPacketCaptureService(environment), ): void => { - const finish = async ( - result: Awaited>, - operation: string, - ) => { - // A one-shot command must retry retained cleanup before discarding its owner. - try { - await service.close(); - } catch (cause) { - const cleanup = - cause instanceof AnalysisError - ? cause - : new ProviderCleanupError( - "tshark-capture", - [], - { reason: String(cause) }, - { cause, operation: operation }, - ); - if (result.ok) cleanup.retainPartialObservation(result.value); - return projectAnalysisError( - result.ok - ? cleanup - : analysisErrorWithCleanupFailure(result.error, cleanup, operation), - ); - } - return result.ok ? result.value : projectAnalysisError(result.error); - }; cli.command(CLI_COMMANDS.inspectPacketFields, { description: "Search installed TShark protocol and field definitions", options: z @@ -80,11 +51,14 @@ export const registerCaptureCommands = ( run: ({ options: { all, ...options } }) => withCommandCancellation((signal) => logCliCommand(logger, CLI_COMMANDS.inspectPacketFields, async () => - finish( - await service.inspectFields( - { ...options, ...(all ? { limit: null } : {}) }, - { signal }, - ), + finishAnalysisCommand( + () => + service.inspectFields( + { ...options, ...(all ? { limit: null } : {}) }, + { signal }, + ), + service, + "tshark-capture", "inspect_packet_fields", ), ), @@ -125,15 +99,20 @@ export const registerCaptureCommands = ( run: ({ args, options: { all_matches, ...options } }) => withCommandCancellation((signal) => logCliCommand(logger, CLI_COMMANDS.inspectPacketCapture, async () => { - const result = await service.inspect( - { - ...options, - ...(all_matches ? { match_limit: null } : {}), - path: resolve(args.path), - }, - { signal }, + return finishAnalysisCommand( + () => + service.inspect( + { + ...options, + ...(all_matches ? { match_limit: null } : {}), + path: resolve(args.path), + }, + { signal }, + ), + service, + "tshark-capture", + "inspect_packet_capture", ); - return finish(result, "inspect_packet_capture"); }), ), }); diff --git a/src/cli/finishAnalysisCommand.ts b/src/cli/finishAnalysisCommand.ts new file mode 100644 index 000000000..9394ba828 --- /dev/null +++ b/src/cli/finishAnalysisCommand.ts @@ -0,0 +1,69 @@ +import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js"; +import { + AnalysisError, + type AnalysisPartialObservation, +} from "../domain/analysisErrorBase.js"; +import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js"; +import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; +import { ProviderAdapterError } from "../domain/providerAdapterError.js"; +import { ProviderCleanupError } from "../domain/providerCleanupError.js"; +import type { Result } from "../domain/result.js"; + +/** Finish a one-shot CLI operation without discarding its primary result on close failure. */ +export const finishAnalysisCommand = async < + Value extends AnalysisPartialObservation, +>( + run: () => Promise>, + service: { close(): Promise }, + providerId: string, + canonicalOperation: string, + fallbackResource?: string, +): Promise> => { + let result: Result; + try { + result = await run(); + } catch (cause) { + result = { + ok: false, + error: + cause instanceof AnalysisError + ? cause + : new ProviderAdapterError(providerId, canonicalOperation, { cause }), + }; + } + try { + await service.close(); + } catch (cause) { + let cleanup: AnalysisError; + if (cause instanceof AnalysisError) cleanup = cause; + else { + const observation = + cause instanceof ArtifactReaderFailure + ? ArtifactReaderFailure.cleanupObservation( + cause, + fallbackResource ?? `${canonicalOperation} resources`, + ) + : { + reason: cause instanceof Error ? cause.message : String(cause), + resources: [], + }; + cleanup = new ProviderCleanupError( + providerId, + observation.resources, + { reason: observation.reason }, + { cause, operation: canonicalOperation }, + ); + } + if (result.ok) cleanup.retainPartialObservation(result.value); + return projectAnalysisError( + result.ok + ? cleanup + : analysisErrorWithCleanupFailure( + result.error, + cleanup, + canonicalOperation, + ), + ); + } + return result.ok ? result.value : projectAnalysisError(result.error); +}; diff --git a/src/cli/goCommands.ts b/src/cli/goCommands.ts index d67a344e0..12b2b992a 100644 --- a/src/cli/goCommands.ts +++ b/src/cli/goCommands.ts @@ -3,14 +3,13 @@ import { resolve } from "node:path"; import { createGoBinaryService } from "../composition/go.js"; import { CLI_COMMANDS } from "../cliCommandNames.js"; import { logCliCommand } from "../cliLogging.js"; -import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; import type { Logger } from "pino"; import type { CliInstance } from "./types.js"; import { withCommandCancellation } from "./commandCancellation.js"; +import { finishAnalysisCommand } from "./finishAnalysisCommand.js"; /** Resolve caller paths and inspect static build metadata through the shared workflow. */ export const registerGoCommands = (cli: CliInstance, logger: Logger): void => { - const service = createGoBinaryService(); cli.command(CLI_COMMANDS.inspectGoBinary, { description: "Inspect embedded Go compiler, modules and build settings without target execution", @@ -18,14 +17,19 @@ export const registerGoCommands = (cli: CliInstance, logger: Logger): void => { path: z.string().describe("Local ELF, PE or thin Mach-O image"), }), run: ({ args }) => - withCommandCancellation((signal) => - logCliCommand(logger, CLI_COMMANDS.inspectGoBinary, async () => { - const result = await service.inspect( - { path: resolve(args.path) }, - { signal }, - ); - return result.ok ? result.value : projectAnalysisError(result.error); - }), - ), + withCommandCancellation(async (signal) => { + const service = createGoBinaryService(); + return await logCliCommand( + logger, + CLI_COMMANDS.inspectGoBinary, + async () => + finishAnalysisCommand( + () => service.inspect({ path: resolve(args.path) }, { signal }), + service, + "go-build-info", + "inspect_go_binary", + ), + ); + }), }); }; diff --git a/src/cli/signalCommands.ts b/src/cli/signalCommands.ts index e7d254507..63b7d9edf 100644 --- a/src/cli/signalCommands.ts +++ b/src/cli/signalCommands.ts @@ -1,9 +1,6 @@ import { inspectSignalDecodersInputSchema } from "../domain/signals/signalDecoders.js"; import { z } from "incur"; import type { SignalRecordingService } from "../application/signals/SignalRecordingService.js"; -import { AnalysisError } from "../domain/analysisErrorBase.js"; -import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js"; -import { ProviderCleanupError } from "../domain/providerCleanupError.js"; import { resolve } from "node:path"; import { createSignalRecordingService } from "../composition/signals.js"; import { @@ -13,10 +10,10 @@ import { } from "../domain/signals/signalRecording.js"; import { CLI_COMMANDS } from "../cliCommandNames.js"; import { logCliCommand } from "../cliLogging.js"; -import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; import type { Logger } from "pino"; import type { CliInstance } from "./types.js"; import { withCommandCancellation } from "./commandCancellation.js"; +import { finishAnalysisCommand } from "./finishAnalysisCommand.js"; export const registerSignalCommands = ( cli: CliInstance, logger: Logger, @@ -26,32 +23,6 @@ export const registerSignalCommands = ( "decode" | "inspectDecoders" | "close" > = createSignalRecordingService(environment), ): void => { - const finish = async ( - result: Awaited>, - operation: string, - ) => { - // A one-shot command must retry retained cleanup before discarding its owner. - try { - await service.close(); - } catch (cause) { - const cleanup = - cause instanceof AnalysisError - ? cause - : new ProviderCleanupError( - "sigrok-recording", - [], - { reason: String(cause) }, - { cause, operation: operation }, - ); - if (result.ok) cleanup.retainPartialObservation(result.value); - return projectAnalysisError( - result.ok - ? cleanup - : analysisErrorWithCleanupFailure(result.error, cleanup, operation), - ); - } - return result.ok ? result.value : projectAnalysisError(result.error); - }; cli.command(CLI_COMMANDS.inspectSignalDecoders, { description: "Search installed sigrok decoders or selected decoder help", options: z @@ -82,11 +53,14 @@ export const registerSignalCommands = ( run: ({ options: { all, ...options } }) => withCommandCancellation((signal) => logCliCommand(logger, CLI_COMMANDS.inspectSignalDecoders, async () => - finish( - await service.inspectDecoders( - { ...options, ...(all ? { limit: null } : {}) }, - { signal }, - ), + finishAnalysisCommand( + () => + service.inspectDecoders( + { ...options, ...(all ? { limit: null } : {}) }, + { signal }, + ), + service, + "sigrok-recording", "inspect_signal_decoders", ), ), @@ -127,15 +101,20 @@ export const registerSignalCommands = ( run: ({ args, options: { all_lines, ...options } }) => withCommandCancellation((signal) => logCliCommand(logger, CLI_COMMANDS.decodeSignalRecording, async () => { - const result = await service.decode( - { - ...options, - ...(all_lines ? { line_limit: null } : {}), - path: resolve(args.path), - }, - { signal }, + return finishAnalysisCommand( + () => + service.decode( + { + ...options, + ...(all_lines ? { line_limit: null } : {}), + path: resolve(args.path), + }, + { signal }, + ), + service, + "sigrok-recording", + "decode_signal_recording", ); - return finish(result, "decode_signal_recording"); }), ), }); diff --git a/src/cli/webNetworkCaptureCommands.ts b/src/cli/webNetworkCaptureCommands.ts index 655d34bc1..d49e39b0e 100644 --- a/src/cli/webNetworkCaptureCommands.ts +++ b/src/cli/webNetworkCaptureCommands.ts @@ -3,10 +3,10 @@ import { WEB_NETWORK_CAPTURE_FORMATS } from "../domain/webNetworkCapture.js"; import { createWebNetworkCaptureService } from "../composition/webNetworkCaptures.js"; import { CLI_COMMANDS } from "../cliCommandNames.js"; import { logCliCommand } from "../cliLogging.js"; -import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; import type { Logger } from "pino"; import type { CliInstance } from "./types.js"; import { withCommandCancellation } from "./commandCancellation.js"; +import { finishAnalysisCommand } from "./finishAnalysisCommand.js"; /** Expose offline historical capture inspection with the same selection contract as MCP. */ export const registerWebNetworkCaptureCommands = ( @@ -42,20 +42,24 @@ export const registerWebNetworkCaptureCommands = ( logger, CLI_COMMANDS.inspectWebNetworkCapture, async () => { - const result = await service.inspect( - { - capture_path: args.capturePath, - format: args.format, - sensitive_values: options.sensitiveValue, - ...(options.record === undefined - ? {} - : { record_ordinals: options.record }), - }, - { signal }, + return finishAnalysisCommand( + () => + service.inspect( + { + capture_path: args.capturePath, + format: args.format, + sensitive_values: options.sensitiveValue, + ...(options.record === undefined + ? {} + : { record_ordinals: options.record }), + }, + { signal }, + ), + service, + "web-network-capture", + "inspect_web_network_capture", + "Web network capture resources", ); - return result.ok - ? result.value - : projectAnalysisError(result.error); }, ), ), diff --git a/src/evm/EvmoleFailures.ts b/src/evm/EvmoleFailures.ts index 2677a3e71..63c6578aa 100644 --- a/src/evm/EvmoleFailures.ts +++ b/src/evm/EvmoleFailures.ts @@ -39,6 +39,9 @@ export const evmInterfaceFailure = ( limiter = "/usr/bin/prlimit", fileFailure?: EvmFileSizeFailureEvidence, ): AnalysisError => { + const cleanup = + cause instanceof ArtifactReaderFailure ? cause.cleanup : undefined; + const cleanupOptions = cleanup === undefined ? {} : { cleanup }; if (cause instanceof AnalysisError) return cause; if (cause instanceof OwnedCommandFailure) { const outputOptions = @@ -64,12 +67,15 @@ export const evmInterfaceFailure = ( { operation: "inspect_evm_interface", cause }, ); if (cause.reason === "cancelled") - return new AnalysisCancelledError("inspect_evm_interface", outputOptions); + return new AnalysisCancelledError("inspect_evm_interface", { + ...outputOptions, + ...cleanupOptions, + }); if (cause.reason === "timeout") return new AnalysisTimeoutError( "inspect_evm_interface", EVM_INTERFACE_LIMITS.timeoutMs, - outputOptions, + { ...outputOptions, ...cleanupOptions }, ); if (cause.reason === "output-limit") return new AnalysisOutputError( @@ -122,25 +128,33 @@ export const evmInterfaceFailure = ( } if (cause instanceof ArtifactReaderFailure) { if (cause.reason === "cancelled") - return new AnalysisCancelledError("inspect_evm_interface"); + return new AnalysisCancelledError("inspect_evm_interface", { + cause, + ...cleanupOptions, + }); if (cause.reason === "integrity") return new AnalysisArtifactChangedError( "inspect_evm_interface", path, cause.message, - { cause }, + { cause, ...cleanupOptions }, ); if ( phase === "artifact-read" && ["limit", "path", "format"].includes(cause.reason) ) - return new AnalysisInputError("inspect_evm_interface", { cause }, [ - { - path: ["path"], - reason: cause.reason === "limit" ? "out_of_range" : "invalid_format", - message: cause.message, - }, - ]); + return new AnalysisInputError( + "inspect_evm_interface", + { cause, ...cleanupOptions }, + [ + { + path: ["path"], + reason: + cause.reason === "limit" ? "out_of_range" : "invalid_format", + message: cause.message, + }, + ], + ); } if (phase === "artifact-read" && cause instanceof Error && "code" in cause) { if (cause.code === "EACCES" || cause.code === "EPERM") @@ -148,22 +162,27 @@ export const evmInterfaceFailure = ( "inspect_evm_interface", path, cause.code, - { cause }, + { cause, ...cleanupOptions }, ); if (cause.code === "ENOENT" || cause.code === "ENOTDIR") - return new AnalysisInputError("inspect_evm_interface", { cause }, [ - { - path: ["path"], - reason: "invalid_value", - message: `Selected bytecode carrier could not be read (${String(cause.code)}): ${path}.`, - }, - ]); + return new AnalysisInputError( + "inspect_evm_interface", + { cause, ...cleanupOptions }, + [ + { + path: ["path"], + reason: "invalid_value", + message: `Selected bytecode carrier could not be read (${String(cause.code)}): ${path}.`, + }, + ], + ); } return new ProviderAdapterError( EVMOLE_PROVIDER_IDENTITY.id, "inspect_evm_interface", { cause, + ...cleanupOptions, diagnostics: { phase, path, diff --git a/src/evm/EvmoleInterfaceProvider.ts b/src/evm/EvmoleInterfaceProvider.ts index 7437edc23..c59427e6a 100644 --- a/src/evm/EvmoleInterfaceProvider.ts +++ b/src/evm/EvmoleInterfaceProvider.ts @@ -13,6 +13,8 @@ import type { EvmInterfacePort, } from "../application/evm/EvmInterfacePort.js"; import { readStableArtifact } from "../artifacts/readStableArtifact.js"; +import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; +import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js"; import { OwnedCommandFailure } from "../process/OwnedCommand.js"; import type { AnalysisError, @@ -78,6 +80,7 @@ type Launcher = NonNullable[2]>["launcher"]; /** Run unchanged EVMole/WASM in an owned worker; no target, chain or network execution. */ export class EvmoleInterfaceProvider implements EvmInterfacePort { + readonly #resources = new ArtifactResourceScope(); #tail: Promise = Promise.resolve(); #closing = false; #closePromise: Promise | undefined; @@ -102,9 +105,14 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort { options?: ExecutionOptions, ): Promise> { if (this.#closing) return Promise.resolve(err(this.#closedError())); - const operation = this.#tail.then(() => - this.#closing ? err(this.#closedError()) : this.#inspect(input, options), - ); + const operation = this.#tail.then(async () => { + if (this.#closing) return err(this.#closedError()); + try { + return await this.#resources.run(() => this.#inspect(input, options)); + } catch (cause: unknown) { + return err(evmInterfaceFailure(cause, "artifact-read", input.path)); + } + }); this.#tail = operation.then( () => undefined, () => undefined, @@ -180,6 +188,8 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort { input.path, EVM_INTERFACE_LIMITS.inputBytes, options?.signal, + undefined, + this.#resources, ); phase = "worker"; root = await this.createRuntime(); @@ -243,15 +253,27 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort { replyPath, EVM_INTERFACE_LIMITS.outputBytes, options?.signal, + undefined, + this.#resources, ); reply = replySchema.parse(JSON.parse(file.bytes.toString("utf8"))); } catch (cause: unknown) { + const cleanup = + cause instanceof ArtifactReaderFailure ? cause.cleanup : undefined; if (options?.signal?.aborted) - throw new AnalysisCancelledError(OPERATION, { capturedOutput }); + throw new AnalysisCancelledError(OPERATION, { + cause, + capturedOutput, + ...(cleanup === undefined ? {} : { cleanup }), + }); throw new AnalysisOutputError( OPERATION, `Owned interface worker reply failed for ${input.path}: ${cause instanceof Error ? cause.message : String(cause)}`, - { cause, capturedOutput }, + { + cause, + capturedOutput, + ...(cleanup === undefined ? {} : { cleanup }), + }, ); } if (!reply.ok) { @@ -355,6 +377,7 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort { if (cause instanceof OwnedCommandFailure) processOwner = cause.cleanupOwner; let fileFailure: EvmFileSizeFailureEvidence | undefined; + let markerCleanupFailure: ArtifactReaderFailure | undefined; if ( cause instanceof OwnedCommandFailure && cause.snapshot?.exitCode === EVM_FILE_SIZE_FAILURE_EXIT && @@ -365,6 +388,9 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort { const marker = await readStableArtifact( join(root.path, "resource.failure"), 1, + undefined, + undefined, + this.#resources, ); if (!marker.bytes.equals(Buffer.from("F"))) throw new Error( @@ -372,6 +398,14 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort { ); fileFailure = { verified: true, failure: null }; } catch (markerFailure: unknown) { + if ( + markerFailure instanceof ArtifactReaderFailure && + markerFailure.cleanup !== undefined + ) + markerCleanupFailure = ArtifactReaderFailure.withCleanup( + cause, + markerFailure.cleanup, + ); fileFailure = { verified: false, failure: @@ -382,23 +416,36 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort { } } result = err( - options?.signal?.aborted && - (cause === options.signal.reason || - (cause instanceof Error && cause.name === "AbortError")) - ? new AnalysisCancelledError( - OPERATION, - retainedOutput === undefined - ? undefined - : { capturedOutput: retainedOutput }, - ) - : evmInterfaceFailure( - cause, - phase, + markerCleanupFailure !== undefined + ? evmInterfaceFailure( + markerCleanupFailure, + "artifact-read", input.path, workerLimits, limiter, fileFailure, - ), + ) + : options?.signal?.aborted && + (cause === options.signal.reason || + (cause instanceof Error && cause.name === "AbortError")) + ? new AnalysisCancelledError(OPERATION, { + cause, + ...(retainedOutput === undefined + ? {} + : { capturedOutput: retainedOutput }), + ...(cause instanceof ArtifactReaderFailure && + cause.cleanup !== undefined + ? { cleanup: cause.cleanup } + : {}), + }) + : evmInterfaceFailure( + cause, + phase, + input.path, + workerLimits, + limiter, + fileFailure, + ), ); } if (processOwner !== undefined) { @@ -447,8 +494,37 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort { this.#closing = true; this.#closePromise ??= this.#tail .then(async () => { - const failure = await this.#retire(); - if (failure !== undefined) throw failure; + const outcomes = await Promise.allSettled([ + this.#retire(), + this.#resources.close(), + ]); + const failures = outcomes.flatMap((outcome) => + outcome.status === "rejected" + ? [outcome.reason] + : outcome.value === undefined + ? [] + : [outcome.value], + ); + if (failures.length > 0) { + const resources = failures.flatMap((failure) => + failure instanceof ProviderCleanupError + ? failure.cleanupResources + : failure instanceof ArtifactReaderFailure && + failure.cleanup !== undefined + ? failure.cleanup.resources + : ["artifact-resources"], + ); + throw new ProviderCleanupError( + EVMOLE_PROVIDER_IDENTITY.id, + [...new Set(resources)], + { + failures: failures.map((failure) => + failure instanceof Error ? failure.message : String(failure), + ), + }, + { operation: OPERATION, cause: new AggregateError(failures) }, + ); + } }) .catch((cause: unknown) => { this.#closePromise = undefined; diff --git a/src/go/GoBinaryProvider.ts b/src/go/GoBinaryProvider.ts index a1c0cc6a3..466136f3f 100644 --- a/src/go/GoBinaryProvider.ts +++ b/src/go/GoBinaryProvider.ts @@ -1,5 +1,7 @@ import { readStableArtifact } from "../artifacts/readStableArtifact.js"; +import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js"; +import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js"; import type { ExecutionOptions } from "../application/AnalysisProvider.js"; import type { GoBinaryPort } from "../application/go/GoBinaryPort.js"; import { AnalysisError } from "../domain/analysisErrorBase.js"; @@ -18,6 +20,7 @@ import { type InspectGoBinaryInput, } from "../domain/go/goBinary.js"; import { ProviderAdapterError } from "../domain/providerAdapterError.js"; +import { ProviderCleanupError } from "../domain/providerCleanupError.js"; import { err, ok, type Result } from "../domain/result.js"; import { readGoBinaryImage, @@ -36,13 +39,19 @@ const OPERATION = "inspect_go_binary"; const INPUT_BYTES = 256 * 1024 * 1024; const OUTPUT_BYTES = 8 * 1024 * 1024; -const failureFor = ( +const failureForPrimary = ( cause: unknown, path: string, signal?: AbortSignal, ): AnalysisError => { if (cause instanceof AnalysisError) return cause; - if (signal?.aborted) return new AnalysisCancelledError(OPERATION); + if (cause instanceof ArtifactReaderFailure && cause.reason === "integrity") + return new AnalysisArtifactChangedError(OPERATION, path, cause.message, { + cause, + }); + if (cause instanceof ArtifactReaderFailure && cause.reason === "cancelled") + return new AnalysisCancelledError(OPERATION, { cause }); + if (signal?.aborted) return new AnalysisCancelledError(OPERATION, { cause }); if (cause instanceof GoBinaryResourceFailure) return new AnalysisResourceConstraintError( OPERATION, @@ -64,12 +73,6 @@ const failureFor = ( }, ]); if (cause instanceof ArtifactReaderFailure) { - if (cause.reason === "integrity") - return new AnalysisArtifactChangedError(OPERATION, path, cause.message, { - cause, - }); - if (cause.reason === "cancelled") - return new AnalysisCancelledError(OPERATION); if (cause.reason === "limit") return new AnalysisResourceConstraintError( OPERATION, @@ -110,6 +113,23 @@ const failureFor = ( }); }; +const failureFor = ( + cause: unknown, + path: string, + signal?: AbortSignal, +): AnalysisError => { + const primary = failureForPrimary(cause, path, signal); + if (!(cause instanceof ArtifactReaderFailure) || cause.cleanup === undefined) + return primary; + const cleanup = new ProviderCleanupError( + GO_BINARY_PROVIDER_IDENTITY.id, + cause.cleanup.resources, + { reason: cause.cleanup.reason }, + { operation: OPERATION, cause }, + ); + return analysisErrorWithCleanupFailure(primary, cleanup, OPERATION); +}; + const outputLimit = (path: string) => new AnalysisResourceConstraintError( OPERATION, @@ -121,6 +141,11 @@ const outputLimit = (path: string) => /** Stable in-process metadata inspection without subprocesses, temporary files or Go installation. */ export class GoBinaryProvider implements GoBinaryPort { readonly identity = GO_BINARY_PROVIDER_IDENTITY; + readonly #resources = new ArtifactResourceScope(); + + close(): Promise { + return this.#resources.close(); + } /** Read complete compiler/module metadata while preserving original byte identity and source ranges. */ async inspect( @@ -128,10 +153,14 @@ export class GoBinaryProvider implements GoBinaryPort { options?: ExecutionOptions, ): Promise> { try { - const snapshot = await readStableArtifact( - input.path, - INPUT_BYTES, - options?.signal, + const snapshot = await this.#resources.run(() => + readStableArtifact( + input.path, + INPUT_BYTES, + options?.signal, + undefined, + this.#resources, + ), ); options?.signal?.throwIfAborted(); const image = readGoBinaryImage(snapshot.bytes); diff --git a/src/native/pe/PeResourcesProvider.ts b/src/native/pe/PeResourcesProvider.ts index 4326cb3b7..4f31256d9 100644 --- a/src/native/pe/PeResourcesProvider.ts +++ b/src/native/pe/PeResourcesProvider.ts @@ -1,5 +1,7 @@ import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js"; import { readStableArtifact } from "../../artifacts/readStableArtifact.js"; +import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js"; +import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js"; import type { PeResourcesPort } from "../../application/binaryDiagnostics/PeResourcesPort.js"; import type { ExecutionOptions } from "../../application/AnalysisProvider.js"; import { ArtifactOperationError } from "../../domain/artifactOperationError.js"; @@ -12,19 +14,29 @@ import { } from "../../domain/analysisErrorCore.js"; import type { InspectPeResourcesInput } from "../../domain/native/peResources.js"; import { err, ok } from "../../domain/result.js"; +import { ProviderCleanupError } from "../../domain/providerCleanupError.js"; import { PE_RESOURCES_PROVIDER } from "../../application/InvestigationProviders.js"; import { parsePeResources } from "./PeResourceParser.js"; /** Portable parser over one bounded, identity-checked regular file. */ export class PeResourcesProvider implements PeResourcesPort { readonly identity = PE_RESOURCES_PROVIDER; + readonly #resources = new ArtifactResourceScope(); + + close(): Promise { + return this.#resources.close(); + } async inspect(input: InspectPeResourcesInput, options?: ExecutionOptions) { try { - const snapshot = await readStableArtifact( - input.path, - input.max_file_bytes, - options?.signal, + const snapshot = await this.#resources.run(() => + readStableArtifact( + input.path, + input.max_file_bytes, + options?.signal, + undefined, + this.#resources, + ), ); return ok( await parsePeResources( @@ -44,12 +56,21 @@ export class PeResourcesProvider implements PeResourcesPort { } } -const peResourcesFailure = ( +const peResourcesFailurePrimary = ( cause: unknown, input: InspectPeResourcesInput, signal: AbortSignal | undefined, ) => { const operation = "inspect_pe_resources"; + if (cause instanceof ArtifactReaderFailure && cause.reason === "integrity") + return new AnalysisArtifactChangedError( + operation, + input.path, + cause.message, + { cause }, + ); + if (cause instanceof ArtifactReaderFailure && cause.reason === "cancelled") + return new AnalysisCancelledError(operation, { cause }); if (signal?.aborted) return new AnalysisCancelledError(operation, { cause }); if (cause instanceof ArtifactReaderFailure) { @@ -64,13 +85,6 @@ const peResourcesFailure = ( }, { cause }, ); - if (cause.reason === "integrity") - return new AnalysisArtifactChangedError( - operation, - input.path, - cause.message, - { cause }, - ); if (cause.reason === "path" || cause.reason === "format") return new AnalysisInputError(operation, { cause }, [ { @@ -79,8 +93,6 @@ const peResourcesFailure = ( message: cause.message, }, ]); - if (cause.reason === "cancelled") - return new AnalysisCancelledError(operation, { cause }); } if (cause instanceof Error && "code" in cause) { @@ -106,3 +118,24 @@ const peResourcesFailure = ( { cause }, ); }; + +const peResourcesFailure = ( + cause: unknown, + input: InspectPeResourcesInput, + signal: AbortSignal | undefined, +) => { + const primary = peResourcesFailurePrimary(cause, input, signal); + if (!(cause instanceof ArtifactReaderFailure) || cause.cleanup === undefined) + return primary; + const cleanup = new ProviderCleanupError( + PE_RESOURCES_PROVIDER.id, + cause.cleanup.resources, + { reason: cause.cleanup.reason }, + { operation: "inspect_pe_resources", cause }, + ); + return analysisErrorWithCleanupFailure( + primary, + cleanup, + "inspect_pe_resources", + ); +}; diff --git a/src/native/pwndbg/PwndbgCoreContext.ts b/src/native/pwndbg/PwndbgCoreContext.ts index 0ef29cd9b..db8386d26 100644 --- a/src/native/pwndbg/PwndbgCoreContext.ts +++ b/src/native/pwndbg/PwndbgCoreContext.ts @@ -17,6 +17,7 @@ import { type RecordedCrash, } from "../../domain/native/recordedCrash.js"; import { readStableArtifact } from "../../artifacts/readStableArtifact.js"; +import type { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js"; import { AnalysisError } from "../../domain/analysisErrorBase.js"; import { capturedPwntoolsOutput, @@ -46,6 +47,7 @@ const replySchema = z.discriminatedUnion("ok", [ export const inspectPwndbgCore = async ({ environment, rootPath, + resources, diagnostics, options, runCommand, @@ -53,6 +55,7 @@ export const inspectPwndbgCore = async ({ }: { readonly environment: Readonly; readonly rootPath: string; + readonly resources: ArtifactResourceScope; readonly diagnostics: AnalysisCapturedOutput; readonly options: ExecutionOptions | undefined; readonly runCommand: PwntoolsCommand; @@ -178,11 +181,14 @@ export const inspectPwndbgCore = async ({ replyPath, PWNTOOLS_LIMITS.outputBytes, options?.signal, + undefined, + resources, ); reply = replySchema.parse(JSON.parse(raw.bytes.toString("utf8"))); } catch (cause: unknown) { if (options?.signal?.aborted) throw new AnalysisCancelledError(OPERATION, { + cause, capturedOutput: recordedCrashStageOutput(diagnostics, output), }); throw new AnalysisOutputError( @@ -226,22 +232,22 @@ export const inspectPwndbgCore = async ({ return { ...reply.value, diagnostics: output }; } catch (cause: unknown) { if (cause instanceof AnalysisError && cause.cleanupIncomplete) throw cause; - if ( + const primary = options?.signal?.aborted && (cause === options.signal.reason || (cause instanceof Error && cause.name === "AbortError")) - ) - throw new AnalysisCancelledError(OPERATION, { - capturedOutput: recordedCrashStageOutput(diagnostics, output), - }); - throw pwntoolsDecoderFailure( - cause, - "debugger", - gdb, - environment.REA_PWNTOOLS_PYTHON ?? "", - await readPwntoolsFailureEvidence(cause, stage), - failureContext, - ); + ? new AnalysisCancelledError(OPERATION, { + capturedOutput: recordedCrashStageOutput(diagnostics, output), + }) + : pwntoolsDecoderFailure( + cause, + "debugger", + gdb, + environment.REA_PWNTOOLS_PYTHON ?? "", + await readPwntoolsFailureEvidence(cause, stage, resources), + failureContext, + ); + throw primary; } }; diff --git a/src/native/pwntools/PwntoolsDecoder.ts b/src/native/pwntools/PwntoolsDecoder.ts index d8cad3d2e..169f25601 100644 --- a/src/native/pwntools/PwntoolsDecoder.ts +++ b/src/native/pwntools/PwntoolsDecoder.ts @@ -16,6 +16,8 @@ import type { AnalysisPartialObservation, } from "../../domain/analysisErrorBase.js"; import { readStableArtifact } from "../../artifacts/readStableArtifact.js"; +import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js"; +import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js"; import { AnalysisError } from "../../domain/analysisErrorBase.js"; import { AnalysisCancelledError, @@ -78,6 +80,7 @@ export interface PwntoolsDecoderProfile< readonly payload: Payload; readonly snapshot: { readonly bytes: Buffer; readonly sha256: string }; readonly rootPath: string; + readonly resources: ArtifactResourceScope; readonly diagnostics: AnalysisCapturedOutput; readonly options: ExecutionOptions | undefined; readonly runCommand: PwntoolsCommand; @@ -96,6 +99,7 @@ export class PwntoolsDecoder< symbol, "admitted" | "cancelled" >(); + readonly #resources = new ArtifactResourceScope(); #closing = false; #closePromise: Promise | undefined; #pendingCleanup: @@ -244,6 +248,8 @@ export class PwntoolsDecoder< input.path, PWNTOOLS_LIMITS.inputBytes, options?.signal, + undefined, + this.#resources, ); phase = "decoder"; root = await this.createRuntime(); @@ -298,11 +304,16 @@ export class PwntoolsDecoder< replyPath, PWNTOOLS_LIMITS.outputBytes, options?.signal, + undefined, + this.#resources, ); reply = replySchema.parse(JSON.parse(file.bytes.toString("utf8"))); } catch (cause: unknown) { if (options?.signal?.aborted) - throw new AnalysisCancelledError(OPERATION, { capturedOutput }); + throw new AnalysisCancelledError(OPERATION, { + cause, + capturedOutput, + }); throw new AnalysisOutputError( OPERATION, `Owned pwntools decoder reply failed for ${input.path}: ${cause instanceof z.ZodError ? cause.issues[0]?.message + " at " + (cause.issues[0]?.path.map(String).join(".") ?? "root") : cause instanceof Error ? cause.message : String(cause)}`, @@ -369,6 +380,7 @@ export class PwntoolsDecoder< payload: reply.value, snapshot, rootPath: root.path, + resources: this.#resources, diagnostics: capturedOutput, options, runCommand: (spawn, classifyFailure) => @@ -385,12 +397,18 @@ export class PwntoolsDecoder< } return err(cause); } - const evidence = await readPwntoolsFailureEvidence(cause, root?.path); - result = err( + const evidence = await readPwntoolsFailureEvidence( + cause, + root?.path, + this.#resources, + ); + const primary = options?.signal?.aborted && - (cause === options.signal.reason || - (cause instanceof Error && cause.name === "AbortError")) - ? new AnalysisCancelledError(OPERATION) + (cause === options.signal.reason || + (cause instanceof Error && cause.name === "AbortError") || + (cause instanceof ArtifactReaderFailure && + cause.cleanup !== undefined)) + ? new AnalysisCancelledError(OPERATION, { cause }) : pwntoolsDecoderFailure( cause, phase, @@ -398,8 +416,8 @@ export class PwntoolsDecoder< executablePath, evidence, context, - ), - ); + ); + result = err(primary); } if (this.#pendingCleanup !== undefined) { this.#pendingCleanup.failure = result.ok ? undefined : result.error; @@ -483,7 +501,39 @@ export class PwntoolsDecoder< this.#closing = true; this.#closePromise ??= this.#tail .then(async () => { - const failure = await this.#retire(); + let failure: AnalysisError | undefined; + try { + failure = await this.#retire(); + } catch (cause: unknown) { + failure = new ProviderCleanupError( + this.identity.id, + [this.#pendingCleanup?.root?.path ?? "pwntools decoder"], + { reason: cause instanceof Error ? cause.message : String(cause) }, + { operation: this.profile.operation, cause }, + ); + } + try { + await this.#resources.close(); + } catch (cause: unknown) { + const observation = ArtifactReaderFailure.cleanupObservation( + cause, + "pwntools artifact read", + ); + const resourceFailure = new ProviderCleanupError( + this.identity.id, + observation.resources, + { reason: observation.reason }, + { operation: this.profile.operation, cause }, + ); + failure = + failure === undefined + ? resourceFailure + : analysisErrorWithCleanupFailure( + failure, + resourceFailure, + this.profile.operation, + ); + } if (failure !== undefined) throw failure; }) .catch((cause: unknown) => { @@ -502,8 +552,8 @@ export class PwntoolsDecoder< } async #completeCleanup( - retire: () => Promise, - ): Promise { + retire: () => Promise, + ): Promise { const failure = await retire(); return failure !== undefined && this.#closing ? retire() : failure; } @@ -548,18 +598,46 @@ export class PwntoolsDecoder< return undefined; } - async #retire(): Promise { - const failure = await this.#retireProcess(); - if (failure !== undefined) return failure; + async #retire(): Promise { + let failure: AnalysisError | undefined; + try { + await this.#resources.retryCleanup(); + } catch (cause: unknown) { + const observation = ArtifactReaderFailure.cleanupObservation( + cause, + "pwntools artifact read", + ); + failure = new ProviderCleanupError( + this.identity.id, + observation.resources, + { reason: observation.reason }, + { + operation: this.profile.operation, + cause, + ...(this.#pendingCleanup?.observation === undefined + ? {} + : { partialObservation: this.#pendingCleanup.observation }), + }, + ); + } + const processFailure = await this.#retireProcess(); + if (processFailure !== undefined) { + return failure === undefined + ? processFailure + : analysisErrorWithCleanupFailure( + failure, + processFailure, + this.profile.operation, + ); + } const pending = this.#pendingCleanup; - if (pending === undefined) return undefined; - if (pending.root !== undefined) { + if (pending?.root !== undefined) { const root = pending.root; try { await root.close(); pending.root = undefined; } catch (cause: unknown) { - return new ProviderCleanupError( + const rootFailure = new ProviderCleanupError( this.identity.id, [root.path], { @@ -580,9 +658,18 @@ export class PwntoolsDecoder< : { partialObservation: pending.observation }), }, ); + failure = + failure === undefined + ? rootFailure + : analysisErrorWithCleanupFailure( + failure, + rootFailure, + this.profile.operation, + ); } } - this.#pendingCleanup = undefined; - return undefined; + if (!this.#resources.hasPending && pending?.root === undefined) + this.#pendingCleanup = undefined; + return failure; } } diff --git a/src/native/pwntools/PwntoolsRecordedCrashProvider.ts b/src/native/pwntools/PwntoolsRecordedCrashProvider.ts index ffc10d13b..24cee7314 100644 --- a/src/native/pwntools/PwntoolsRecordedCrashProvider.ts +++ b/src/native/pwntools/PwntoolsRecordedCrashProvider.ts @@ -48,6 +48,7 @@ export class PwntoolsRecordedCrashProvider payload, snapshot, rootPath, + resources, diagnostics, options, runCommand, @@ -120,6 +121,7 @@ export class PwntoolsRecordedCrashProvider const context = await inspectPwndbgCore({ environment, rootPath, + resources, diagnostics, options, runCommand, diff --git a/src/native/pwntools/PwntoolsResourceLimits.ts b/src/native/pwntools/PwntoolsResourceLimits.ts index dcbe91d5a..3b379779f 100644 --- a/src/native/pwntools/PwntoolsResourceLimits.ts +++ b/src/native/pwntools/PwntoolsResourceLimits.ts @@ -7,6 +7,7 @@ import { } from "./PwntoolsRelease.js"; import { z } from "zod"; import { PWNTOOLS_LIMITS } from "./PwntoolsRelease.js"; +import type { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js"; /** Actual limits reported by the owned bridge after lowering inherited soft limits. */ export const pwntoolsResourceLimitsSchema = z.strictObject({ @@ -42,6 +43,7 @@ export interface PwntoolsFailureEvidence { export const readPwntoolsFailureEvidence = async ( cause: unknown, rootPath?: string, + resources?: ArtifactResourceScope, ): Promise => { let limitReport: PwntoolsLimitReport | undefined; let marker: PwntoolsFailureEvidence["marker"]; @@ -56,6 +58,9 @@ export const readPwntoolsFailureEvidence = async ( const reported = await readStableArtifact( join(rootPath, "resource.failure"), 1, + undefined, + undefined, + resources, ); const expected = cause.snapshot.exitCode === PWNTOOLS_MEMORY_FAILURE_EXIT ? "M" : "F"; @@ -90,6 +95,9 @@ export const readPwntoolsFailureEvidence = async ( const reported = await readStableArtifact( join(rootPath, "limits.json"), 4096, + undefined, + undefined, + resources, ); limitReport = { limits: pwntoolsResourceLimitsSchema.parse( diff --git a/src/server/createServer.ts b/src/server/createServer.ts index 777028cf8..c73071bea 100644 --- a/src/server/createServer.ts +++ b/src/server/createServer.ts @@ -9,6 +9,7 @@ import { wasmArtifactModule } from "../composition/wasm.js"; import { registerWasmTools } from "./registerWasmTools.js"; import { snapshotEnvironment } from "../process/snapshotEnvironment.js"; import { createPeResourcesService } from "../composition/binaryDiagnostics.js"; +import type { PeResourcesService } from "../application/binaryDiagnostics/PeResourcesService.js"; import { registerPeResourcesTool } from "./registerPeResourcesTool.js"; import type { EvmInterfaceService } from "../application/evm/EvmInterfaceService.js"; import { evmInterfaceModule } from "../composition/evm.js"; @@ -280,6 +281,10 @@ export const createServer = ( options.webSourceLocation ?? createWebSourceLocationService(environment); const webModuleTrace = options.webModuleTrace ?? createWebModuleTraceService(environment); + const goBinary = options.goBinary ?? createGoBinaryService(); + const peResources = createPeResourcesService(); + const webNetworkCapture = + options.webNetworkCapture ?? createWebNetworkCaptureService(environment); const javascriptRecovery = options.javascriptRecovery ?? createJavaScriptRecoveryProvider(environment); const artifactResources = new ArtifactResourceScope(); @@ -345,6 +350,9 @@ export const createServer = ( recordedCrash.close(), webSourceLocation.close(), webModuleTrace.close(), + goBinary.close(), + peResources.close(), + webNetworkCapture.close(), javascriptRecovery.close?.() ?? Promise.resolve(), artifactResources.close(), firmwareAnalysis.close(), @@ -388,6 +396,9 @@ export const createServer = ( recordEvidenceWithUnknown, withAdmittedAnalysis: analysisAdmission, artifactResources, + goBinary, + peResources, + webNetworkCapture, }; registerBinaryAnalysisTools(toolContext); registerFridaTools( @@ -517,6 +528,9 @@ const registerConfiguredAnalysisTools = ( evidenceById, recordEvidence, withAdmittedAnalysis, + goBinary, + peResources, + webNetworkCapture, }: ServerToolContext, android: AndroidAnalysisPort, wasmArtifact: Pick, @@ -556,14 +570,14 @@ const registerConfiguredAnalysisTools = ( ); registerPeResourcesTool( server, - createPeResourcesService(), + peResources, toolLogger, recordEvidence, withAdmittedAnalysis, ); registerGoTools( server, - options.goBinary ?? createGoBinaryService(), + goBinary, toolLogger, recordEvidence, withAdmittedAnalysis, @@ -628,7 +642,7 @@ const registerConfiguredAnalysisTools = ( ); registerWebNetworkCaptureTool( server, - options.webNetworkCapture ?? createWebNetworkCaptureService(environment), + webNetworkCapture, toolLogger, recordEvidence, withAdmittedAnalysis, @@ -673,6 +687,9 @@ const createSessionRecorders = ( }); interface ServerToolContext extends ReturnType { + readonly goBinary: GoBinaryService; + readonly peResources: PeResourcesService; + readonly webNetworkCapture: WebNetworkCaptureService; readonly artifactResources: ArtifactResourceScope; readonly server: EvidenceMcpServer; readonly session: BinarySessionPort | undefined; diff --git a/src/wasm/WabtArtifactProvider.test.ts b/src/wasm/WabtArtifactProvider.test.ts index 94c4e0c95..428cf8cbd 100644 --- a/src/wasm/WabtArtifactProvider.test.ts +++ b/src/wasm/WabtArtifactProvider.test.ts @@ -1,6 +1,6 @@ import { access, chmod, readFile, writeFile } from "node:fs/promises"; import { join } from "node:path"; -import { expect, it } from "vitest"; +import { expect, it, vi } from "vitest"; import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js"; import { WabtArtifactProvider } from "./WabtArtifactProvider.js"; import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js"; @@ -9,6 +9,33 @@ import { spawnOwnedProviderProcess } from "../process/ProviderProcess.js"; import { OwnedCommandFailure } from "../process/OwnedCommand.js"; import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; +const fileCloseFailure = vi.hoisted(() => ({ + selectedPath: "", + attempts: 0, + failFirst: false, +})); + +vi.mock("node:fs/promises", async (importOriginal) => { + const fs = await importOriginal(); + return { + ...fs, + open: async (...args: Parameters) => { + const handle = await fs.open(...args); + if (String(args[0]) !== fileCloseFailure.selectedPath) return handle; + const close = handle.close.bind(handle); + handle.close = async () => { + fileCloseFailure.attempts += 1; + if (fileCloseFailure.failFirst) { + fileCloseFailure.failFirst = false; + throw new Error("injected WABT input close failure"); + } + return close(); + }; + return handle; + }, + }; +}); + const failureTags: Readonly> = { invalid: "AnalysisInputError", timeout: "AnalysisTimeoutError", @@ -20,6 +47,82 @@ const failureTags: Readonly> = { "wat-warning": "AnalysisOutputError", }; +const assertCleanupRecovery = async (options: { + readonly provider: WabtArtifactProvider; + readonly result: Awaited>; + readonly path: string; + readonly runtimePath: string; + readonly cleanupAttempts: () => number; + readonly runtimeCreations: () => number; +}): Promise => { + expect(options.result).toMatchObject({ + error: { + cleanupIncomplete: true, + cleanupResources: [options.runtimePath], + partialObservation: { normalized_result: { validation: "valid" } }, + }, + }); + await expect(access(options.runtimePath)).resolves.toBeUndefined(); + const blocked = await options.provider.inspect({ + path: options.path, + glue_paths: [], + candidate_paths: [], + }); + expect(blocked).toMatchObject({ + ok: false, + error: { + cleanupIncomplete: true, + cleanupResources: [options.runtimePath], + }, + }); + expect(options.cleanupAttempts()).toBe(2); + expect(options.runtimeCreations()).toBe(1); + await options.provider.close(); + expect(options.cleanupAttempts()).toBe(3); + await expect(access(options.runtimePath)).rejects.toMatchObject({ + code: "ENOENT", + }); +}; + +const assertModeOutcome = ( + mode: string, + result: Awaited>, + path: string, +): void => { + if (mode === "success") + expect(result).toMatchObject({ + ok: true, + value: { + result: { + artifact: { path, bytes: 76 }, + validation: "valid", + runtime_execution: "not-performed", + }, + }, + }); + else + expect(result).toMatchObject({ + ok: false, + error: { _tag: failureTags[mode] }, + }); + if (mode === "invalid") + expect(result).toMatchObject({ + error: { capturedOutput: { stderr: "module.wasm: invalid bytes\n" } }, + }); + if (mode === "malformed-wat" || mode === "wat-warning") + expect(result).toMatchObject({ + error: { + capturedOutput: { + stdout: + mode === "malformed-wat" + ? "unexpected decoder output\n" + : '(module (@custom "test" "payload"))\n', + stderr: mode === "wat-warning" ? "decoder warning\n" : "", + }, + }, + }); +}; + it("returns actionable unavailable without creating a workspace when tools are absent", async () => { const provider = new WabtArtifactProvider({}, undefined, () => { throw new Error("Must not allocate"); @@ -39,6 +142,88 @@ it("returns actionable unavailable without creating a workspace when tools are a }); }); +it("retries a retained input descriptor before admitting the next WABT operation", async () => { + const root = await createTestTempDirectory("rea-wabt-read-cleanup-"); + const path = join(root, "selected.wasm"); + await writeFile(path, Buffer.alloc(76, 1)); + for (const tool of ["wasm-validate", "wasm-objdump", "wasm2wat"]) + await writeFile(join(root, tool), "source-owned executable seam", { + mode: 0o700, + }); + const dump = await readFile( + new URL("../../tests/fixtures/wasm/objdump.txt", import.meta.url), + "utf8", + ); + const provider = new WabtArtifactProvider( + { REA_WABT_BIN_DIRECTORY: root }, + async (input) => { + const version = input.arguments[0] === "--version"; + const stdout = version + ? "1.0.42\n" + : input.command.includes("wasm-objdump") + ? dump + : input.command.includes("wasm2wat") + ? '(module (@custom "test" "payload"))\n' + : ""; + const launched = await spawnOwnedProviderProcess({ + ...input, + command: process.execPath, + arguments: [ + "-e", + `process.stdout.write(${JSON.stringify(stdout)}); process.exit(0);`, + ], + expectedCommand: null, + }); + return launched; + }, + ); + fileCloseFailure.selectedPath = path; + fileCloseFailure.attempts = 0; + fileCloseFailure.failFirst = true; + try { + const first = await provider.inspect({ + path, + candidate_paths: [], + glue_paths: [], + }); + expect(first).toMatchObject({ + ok: false, + error: { + cleanupIncomplete: true, + cleanupResources: [path], + cause: { message: "injected WABT input close failure" }, + }, + }); + expect(fileCloseFailure.attempts).toBe(1); + + const retried = await provider.inspect({ + path, + candidate_paths: [], + glue_paths: [], + }); + expect(retried.ok).toBe(true); + expect(fileCloseFailure.attempts).toBeGreaterThan(1); + + fileCloseFailure.failFirst = true; + const closeRetry = await provider.inspect({ + path, + candidate_paths: [], + glue_paths: [], + }); + expect(closeRetry).toMatchObject({ + ok: false, + error: { cleanupIncomplete: true, cleanupResources: [path] }, + }); + const beforeClose = fileCloseFailure.attempts; + await provider.close(); + expect(fileCloseFailure.attempts).toBeGreaterThan(beforeClose); + } finally { + fileCloseFailure.selectedPath = ""; + fileCloseFailure.failFirst = false; + await provider.close(); + } +}); + it.each([false, true])( "reports an unavailable private runtime root with retryable=%s without a misleading WABT remediation", async (retryable) => { @@ -110,6 +295,7 @@ it.each([ "utf8", ); let runtimePath = ""; + let runtimeCreations = 0; let acquiredPid: number | undefined; let cleanupAttempts = 0; const controller = new AbortController(); @@ -168,6 +354,7 @@ it.each([ return launched; }, async () => { + runtimeCreations += 1; const root = await PrivateRuntimeRoot.create({ prefix: "rea-wabt-owned-test-", }); @@ -186,53 +373,17 @@ it.each([ { path, glue_paths: [], candidate_paths: [] }, { signal: controller.signal }, ); - if (mode === "success") - expect(result).toMatchObject({ - ok: true, - value: { - result: { - artifact: { path, bytes: 76 }, - validation: "valid", - runtime_execution: "not-performed", - }, - }, - }); - else - expect(result).toMatchObject({ - ok: false, - error: { _tag: failureTags[mode] }, - }); - if (mode === "invalid") - expect(result).toMatchObject({ - error: { capturedOutput: { stderr: "module.wasm: invalid bytes\n" } }, - }); + assertModeOutcome(mode, result, path); if (mode === "cleanup") { - expect(result).toMatchObject({ - error: { - cleanupIncomplete: true, - cleanupResources: [runtimePath], - partialObservation: { normalized_result: { validation: "valid" } }, - }, + await assertCleanupRecovery({ + provider, + result, + path, + runtimePath, + cleanupAttempts: () => cleanupAttempts, + runtimeCreations: () => runtimeCreations, }); - await expect(access(runtimePath)).resolves.toBeUndefined(); - await expect(provider.close()).rejects.toMatchObject({ - cleanupIncomplete: true, - }); - await expect(access(runtimePath)).resolves.toBeUndefined(); - await provider.close(); } - if (mode === "malformed-wat" || mode === "wat-warning") - expect(result).toMatchObject({ - error: { - capturedOutput: { - stdout: - mode === "malformed-wat" - ? "unexpected decoder output\n" - : '(module (@custom "test" "payload"))\n', - stderr: mode === "wat-warning" ? "decoder warning\n" : "", - }, - }, - }); await expect(access(runtimePath)).rejects.toMatchObject({ code: "ENOENT" }); expect(await readFile(path)).toEqual(bytes); if (acquiredPid !== undefined) { diff --git a/src/wasm/WabtArtifactProvider.ts b/src/wasm/WabtArtifactProvider.ts index fe79914ca..c5f0aef53 100644 --- a/src/wasm/WabtArtifactProvider.ts +++ b/src/wasm/WabtArtifactProvider.ts @@ -9,6 +9,8 @@ import type { WasmArtifactPort, } from "../application/wasm/WasmArtifactPort.js"; import { readStableArtifact } from "../artifacts/readStableArtifact.js"; +import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; +import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js"; import type { AnalysisError, AnalysisCapturedOutput, @@ -16,6 +18,7 @@ import type { import { AnalysisCancelledError, AnalysisArtifactChangedError, + AnalysisCapabilityUnavailableError, AnalysisInputError, AnalysisOutputError, } from "../domain/analysisErrorCore.js"; @@ -41,9 +44,7 @@ const digest = (text: string) => createHash("sha256").update(text).digest("hex"); /** Run caller-supplied WABT against private copies of exactly selected bytes. */ export class WabtArtifactProvider implements WasmArtifactPort { - readonly #pendingRoots = new Set< - Pick - >(); + readonly #resources = new ArtifactResourceScope(); constructor( readonly environment: Readonly = process.env, readonly launcher?: Launcher, @@ -53,29 +54,51 @@ export class WabtArtifactProvider implements WasmArtifactPort { ) {} /** Retry retained cleanup without discarding roots that still cannot be removed. */ async close(): Promise { - const failures: { path: string; reason: string }[] = []; - for (const root of this.#pendingRoots) { - try { - await root.close(); - this.#pendingRoots.delete(root); - } catch (cause: unknown) { - failures.push({ - path: root.path, - reason: cause instanceof Error ? cause.message : String(cause), - }); - } - } - if (failures.length > 0) + try { + await this.#resources.close(); + } catch (cause: unknown) { + const cleanup = + cause instanceof ArtifactReaderFailure ? cause.cleanup : undefined; throw new ProviderCleanupError( WABT_PROVIDER_IDENTITY.id, - failures.map(({ path }) => path), - { failures }, - { operation: OPERATION }, + cleanup?.resources ?? ["artifact-resources"], + { + reason: + cleanup?.reason ?? + (cause instanceof Error ? cause.message : String(cause)), + }, + { operation: OPERATION, cause }, ); + } } async inspect( input: InspectWasmArtifactInput, options?: ExecutionOptions, + ): Promise> { + try { + return await this.#resources.run(() => this.#inspect(input, options)); + } catch (cause: unknown) { + if ( + cause instanceof ArtifactReaderFailure && + cause.reason === "unavailable" && + cause.cleanup === undefined + ) + return err(this.#closedError()); + return err(wabtFailure(cause, "artifact", input.path)); + } + } + + #closedError(): AnalysisCapabilityUnavailableError { + return new AnalysisCapabilityUnavailableError( + WABT_PROVIDER_IDENTITY.id, + OPERATION, + "Provider is closing", + ); + } + + async #inspect( + input: InspectWasmArtifactInput, + options?: ExecutionOptions, ): Promise> { let root: Pick | undefined; let phase: "configuration" | "artifact" | "producer" = "configuration"; @@ -83,7 +106,6 @@ export class WabtArtifactProvider implements WasmArtifactPort { let capturedOutput: AnalysisCapturedOutput | undefined; try { if (options?.signal?.aborted) throw new AnalysisCancelledError(OPERATION); - await this.close(); const directory = this.environment.REA_WABT_BIN_DIRECTORY ?? ""; if (!isAbsolute(directory)) throw new Error( @@ -108,6 +130,8 @@ export class WabtArtifactProvider implements WasmArtifactPort { path, WABT_LIMITS.toolBytes, options?.signal, + undefined, + this.#resources, ); commands.push({ tool, @@ -130,17 +154,24 @@ export class WabtArtifactProvider implements WasmArtifactPort { snapshotPath, WABT_LIMITS.inputBytes, options?.signal, + undefined, + this.#resources, ); if (bytes.sha256 !== snapshotHash) throw new Error("Private WASM snapshot bytes changed."); } catch (cause: unknown) { + const cleanup = + cause instanceof ArtifactReaderFailure ? cause.cleanup : undefined; if (options?.signal?.aborted) - throw new AnalysisCancelledError(OPERATION); + throw new AnalysisCancelledError(OPERATION, { + cause, + ...(cleanup === undefined ? {} : { cleanup }), + }); throw new AnalysisArtifactChangedError( OPERATION, input.path, "Private WASM snapshot no longer matches selected byte identity.", - { cause }, + { cause, ...(cleanup === undefined ? {} : { cleanup }) }, ); } }; @@ -175,6 +206,8 @@ export class WabtArtifactProvider implements WasmArtifactPort { command.path, WABT_LIMITS.toolBytes, options?.signal, + undefined, + this.#resources, ); if (current.sha256 !== command.sha256) throw new AnalysisOutputError( @@ -200,6 +233,8 @@ export class WabtArtifactProvider implements WasmArtifactPort { input.path, WABT_LIMITS.inputBytes, options?.signal, + undefined, + this.#resources, ); const artifact = { path: input.path, @@ -215,6 +250,8 @@ export class WabtArtifactProvider implements WasmArtifactPort { path, WABT_LIMITS.inputBytes, options?.signal, + undefined, + this.#resources, ); candidates.push({ path, @@ -228,6 +265,8 @@ export class WabtArtifactProvider implements WasmArtifactPort { path, WABT_LIMITS.inputBytes, options?.signal, + undefined, + this.#resources, ); const source = new TextDecoder("utf-8", { fatal: true }).decode( bytes.bytes, @@ -395,10 +434,13 @@ export class WabtArtifactProvider implements WasmArtifactPort { ); } if (root !== undefined) { - try { - await root.close(); - } catch (cause: unknown) { - this.#pendingRoots.add(root); + const cleanup = await this.#resources.release({ + kind: "private-runtime-root", + resource: root.path, + root, + }); + if (cleanup.kind === "failed") { + const cause = cleanup.cause; const partialObservation = result.ok ? createEvidence( result.value.subject ?? undefined, diff --git a/src/wasm/WabtFailures.ts b/src/wasm/WabtFailures.ts index 4f386998f..7b6875203 100644 --- a/src/wasm/WabtFailures.ts +++ b/src/wasm/WabtFailures.ts @@ -20,12 +20,19 @@ export const wabtFailure = ( path: string, ): AnalysisError => { if (cause instanceof AnalysisError) return cause; + const artifactCleanup = + cause instanceof ArtifactReaderFailure ? cause.cleanup : undefined; + const cleanupOptions = + artifactCleanup === undefined ? {} : { cleanup: artifactCleanup }; const reason = cause instanceof Error ? cause.message : String(cause); if ( (cause instanceof ArtifactReaderFailure && cause.reason === "cancelled") || (cause instanceof Error && cause.name === "AbortError") ) - return new AnalysisCancelledError("inspect_wasm_artifact", { cause }); + return new AnalysisCancelledError("inspect_wasm_artifact", { + cause, + ...cleanupOptions, + }); if (cause instanceof OwnedCommandFailure) { const capturedOutput = cause.snapshot === null @@ -85,6 +92,7 @@ export const wabtFailure = ( reason, { cause, + ...cleanupOptions, userMessage: `${reason} Supply WABT 1.0.42 via absolute REA_WABT_BIN_DIRECTORY; REA does not install tools.`, }, ); @@ -97,7 +105,7 @@ export const wabtFailure = ( "inspect_wasm_artifact", path, reason, - { cause }, + { cause, ...cleanupOptions }, ); if ( phase === "artifact" && @@ -109,11 +117,16 @@ export const wabtFailure = ( "inspect_wasm_artifact", path, cause.code, - { cause }, + { cause, ...cleanupOptions }, ); if (phase === "artifact") - return new AnalysisInputError("inspect_wasm_artifact", { cause }, [ - { path: ["path"], reason: "invalid_value", message: reason }, - ]); - return new AnalysisOutputError("inspect_wasm_artifact", reason, { cause }); + return new AnalysisInputError( + "inspect_wasm_artifact", + { cause, ...cleanupOptions }, + [{ path: ["path"], reason: "invalid_value", message: reason }], + ); + return new AnalysisOutputError("inspect_wasm_artifact", reason, { + cause, + ...cleanupOptions, + }); }; diff --git a/tests/boundary/browser/historicalCaptureDecoder.test.ts b/tests/boundary/browser/historicalCaptureDecoder.test.ts index 92b26d5df..e02524f8d 100644 --- a/tests/boundary/browser/historicalCaptureDecoder.test.ts +++ b/tests/boundary/browser/historicalCaptureDecoder.test.ts @@ -11,7 +11,7 @@ import { } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { expect, it, onTestFinished } from "vitest"; +import { expect, it, onTestFinished, vi } from "vitest"; import { HistoricalCaptureDecoder } from "../../../src/browser/history/HistoricalCaptureDecoder.js"; import { MitmproxyCaptureAdapter } from "../../../src/browser/history/MitmproxyCaptureAdapter.js"; import { HAR_CAPTURE_PROVIDER_IDENTITY } from "../../../src/browser/history/CaptureRelease.js"; @@ -22,6 +22,7 @@ import { import { historicalHar } from "../../fixtures/historicalHar.js"; import { decodeHarCapture } from "../../../src/browser/history/HarCapture.js"; import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; +import { OwnedFileHandle } from "../../../src/filesystem/OwnedFileHandle.js"; const linux = it.skipIf(process.platform !== "linux"); @@ -138,6 +139,66 @@ it("classifies an oversized selected capture before creating an owned decoder", }); }); +it("retains a failed capture reader and retries it before a fresh load", async () => { + const root = await mkdtemp(join(tmpdir(), "rea-capture-reader-close-")); + onTestFinished(() => rm(root, { recursive: true, force: true })); + const path = join(root, "input.har"); + await writeFile(path, JSON.stringify(historicalHar())); + const originalClose = OwnedFileHandle.prototype.close; + const close = vi.spyOn(OwnedFileHandle.prototype, "close"); + let allowClose = false; + let closeAttempts = 0; + close.mockImplementation(async function (this: OwnedFileHandle) { + closeAttempts += 1; + if (!allowClose) throw new Error("injected capture reader close failure"); + await originalClose.call(this); + }); + let decoderCalls = 0; + const decoder = new HistoricalCaptureDecoder( + [ + { + format: "har", + identity: HAR_CAPTURE_PROVIDER_IDENTITY, + command: async () => { + decoderCalls += 1; + return { + command: process.execPath, + arguments: ["-e", "process.exit(0)"], + }; + }, + }, + ], + process.env, + ); + const input = inspectWebNetworkCaptureInputSchema.parse({ + capture_path: path, + format: "har", + }); + try { + const first = await decoder.inspect(input); + expect(first.ok).toBe(false); + if (first.ok) throw new Error("Reader close failure must fail inspection"); + expect(first.error.cleanupResources).toContain(path); + expect(decoderCalls).toBe(0); + + const refused = await decoder.inspect(input); + expect(refused.ok).toBe(false); + if (refused.ok) throw new Error("Fresh read must wait for owner cleanup"); + expect(refused.error.cleanupIncomplete).toBe(true); + expect(refused.error.cleanupResources).toContain(path); + expect(decoderCalls).toBe(0); + expect(closeAttempts).toBe(2); + + allowClose = true; + await decoder.close(); + expect(closeAttempts).toBe(3); + } finally { + allowClose = true; + await decoder.close().catch(() => undefined); + close.mockRestore(); + } +}); + it("does not misclassify decoder filesystem failures as missing selected input", async () => { const root = await mkdtemp(join(tmpdir(), "rea-historical-command-")); onTestFinished(() => rm(root, { recursive: true, force: true })); diff --git a/tests/boundary/cli/captureCleanup.test.ts b/tests/boundary/cli/captureCleanup.test.ts index 8f34b6839..6e5882cd4 100644 --- a/tests/boundary/cli/captureCleanup.test.ts +++ b/tests/boundary/cli/captureCleanup.test.ts @@ -5,6 +5,7 @@ import { silentLogger } from "../../../src/logger.js"; import { createEvidence } from "../../../src/domain/evidence.js"; import { ProviderCleanupError } from "../../../src/domain/providerCleanupError.js"; import { err } from "../../../src/domain/result.js"; +import { AnalysisInputError } from "../../../src/domain/analysisErrorCore.js"; it.each([false, true])( "awaits cleanup retry and preserves completed output when retry failure is %s", @@ -82,6 +83,111 @@ it.each([false, true])( }, ); +it("keeps the primary CLI failure when close throws an untyped error", async () => { + const cli = Cli.create("capture-cleanup-primary-test", { sync: false }); + registerCaptureCommands( + cli, + silentLogger, + {}, + { + inspectFields: async () => { + throw new Error("Unexpected field discovery call"); + }, + inspect: async () => + err( + new AnalysisInputError("inspect_packet_capture", { + userMessage: "The selected capture input was rejected.", + }), + ), + close: async () => { + throw new Error("raw close failure"); + }, + }, + ); + let stdout = ""; + await cli.serve( + [ + "inspect-packet-capture", + "recording.pcap", + "--fields", + "ip.addr", + "--format", + "json", + ], + { + stdout: (text) => { + stdout += text; + }, + exit: () => {}, + }, + ); + expect(JSON.parse(stdout)).toMatchObject({ + code: "cleanup_incomplete", + message: "The selected capture input was rejected.", + details: { + diagnostics: { + primary_error: { code: "invalid_request" }, + }, + resources: [], + }, + }); +}); + +it("closes the service and retains an unexpected producer rejection", async () => { + const cli = Cli.create("capture-producer-rejection-test", { sync: false }); + let closed = false; + registerCaptureCommands( + cli, + silentLogger, + {}, + { + inspectFields: async () => { + throw new Error("Unexpected field discovery call"); + }, + inspect: async () => { + throw new Error("raw producer failure"); + }, + close: async () => { + closed = true; + throw new Error("raw close failure"); + }, + }, + ); + let stdout = ""; + await cli.serve( + [ + "inspect-packet-capture", + "recording.pcap", + "--fields", + "ip.addr", + "--format", + "json", + ], + { + stdout: (text) => { + stdout += text; + }, + exit: () => {}, + }, + ); + expect(closed).toBe(true); + expect(JSON.parse(stdout)).toMatchObject({ + code: "cleanup_incomplete", + details: { + diagnostics: { + primary_error: { + code: "execution_failure", + details: { + operation: "inspect_packet_capture", + provider_id: "tshark-capture", + }, + }, + cleanup_error: { code: "cleanup_incomplete" }, + }, + }, + }); +}); + it("rejects an oversized selector budget before the CLI invokes its service", async () => { const cli = Cli.create("capture-selector-test", { sync: false }); let called = false; diff --git a/tests/boundary/filesystem/stableArtifact.test.ts b/tests/boundary/filesystem/stableArtifact.test.ts index 454d9064b..3bd587ad0 100644 --- a/tests/boundary/filesystem/stableArtifact.test.ts +++ b/tests/boundary/filesystem/stableArtifact.test.ts @@ -31,9 +31,12 @@ it("reads exact binary and empty bytes and rejects symlink/size/cancellation bou }); const controller = new AbortController(); controller.abort(); - await expect(readStableArtifact(real, 3, controller.signal)).rejects.toBe( - controller.signal.reason, - ); + await expect( + readStableArtifact(real, 3, controller.signal), + ).rejects.toMatchObject({ + reason: "cancelled", + cause: controller.signal.reason, + }); }); const checkChange = async ( diff --git a/tests/boundary/filesystem/webArtifactReadOwnership.test.ts b/tests/boundary/filesystem/webArtifactReadOwnership.test.ts new file mode 100644 index 000000000..0a8791366 --- /dev/null +++ b/tests/boundary/filesystem/webArtifactReadOwnership.test.ts @@ -0,0 +1,96 @@ +import { mkdir, rm, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { expect, it, onTestFinished, vi } from "vitest"; + +const injected = vi.hoisted(() => ({ + sourcePath: undefined as string | undefined, + abortController: undefined as AbortController | undefined, + failClose: false, + closeAttempts: 0, +})); + +vi.mock("node:fs/promises", async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + open: async (...args: Parameters) => { + const handle = await actual.open(...args); + if (args[0] !== injected.sourcePath) return handle; + const createReadStream = handle.createReadStream.bind(handle); + handle.createReadStream = (options) => { + injected.abortController?.abort(); + return createReadStream(options); + }; + const close = handle.close.bind(handle); + handle.close = async () => { + injected.closeAttempts += 1; + if (injected.failClose) throw new Error("fixture close refusal"); + await close(); + }; + return handle; + }, + }; +}); + +import { WebModuleTraceService } from "../../../src/application/WebModuleTraceService.js"; +import { LocalWebModuleArtifacts } from "../../../src/browser/modules/WebModuleArtifacts.js"; +import { webModuleArtifactsFixture } from "../../fixtures/webModuleTrace.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +it("retains cancelled source read ownership until service close retries it", async () => { + const root = await createTestTempDirectory("rea-web-artifact-read-owner-"); + onTestFinished(async () => { + injected.sourcePath = undefined; + injected.abortController = undefined; + injected.failClose = false; + injected.closeAttempts = 0; + await rm(root, { recursive: true, force: true }); + }); + const source = 'import "./dep.js";'; + const fixture = webModuleArtifactsFixture(source); + const manifestPath = join(root, "manifest.json"); + const sourcePath = join(root, "files", "modules", "main.js"); + await mkdir(join(root, "files", "modules"), { recursive: true }); + await writeFile(sourcePath, source); + await writeFile(manifestPath, JSON.stringify(fixture.manifest)); + + injected.sourcePath = sourcePath; + injected.failClose = true; + const controller = new AbortController(); + injected.abortController = controller; + const resolver = { + resolve: vi.fn(), + close: vi.fn(async () => undefined), + }; + const service = new WebModuleTraceService( + new LocalWebModuleArtifacts(), + resolver, + ); + const input = { manifest_path: manifestPath, script_index: 0 }; + + const cancelled = await service.trace(input, { signal: controller.signal }); + expect(cancelled.ok).toBe(false); + if (cancelled.ok) throw new Error("Expected cancelled source read"); + expect(cancelled.error.cleanupIncomplete).toBe(true); + expect(cancelled.error.cleanupResources).toContain(sourcePath); + expect(resolver.resolve).not.toHaveBeenCalled(); + + injected.abortController = undefined; + const blocked = await service.trace(input); + expect(blocked.ok).toBe(false); + if (blocked.ok) throw new Error("Expected retained cleanup to block load"); + expect(blocked.error.cleanupIncomplete).toBe(true); + expect(injected.closeAttempts).toBe(2); + + await expect(service.close()).rejects.toThrow( + "Web module trace cleanup failed", + ); + expect(resolver.close).toHaveBeenCalledTimes(1); + expect(injected.closeAttempts).toBe(3); + + injected.failClose = false; + const attemptsBeforeRetry = injected.closeAttempts; + await service.close(); + expect(injected.closeAttempts).toBeGreaterThan(attemptsBeforeRetry); + expect(resolver.close).toHaveBeenCalledTimes(2); +}); diff --git a/tests/boundary/mcp/serverLifecycleMcp.test.ts b/tests/boundary/mcp/serverLifecycleMcp.test.ts index c841e08ed..cdec09a3e 100644 --- a/tests/boundary/mcp/serverLifecycleMcp.test.ts +++ b/tests/boundary/mcp/serverLifecycleMcp.test.ts @@ -34,6 +34,9 @@ import { } from "../../support/workspace/workspaceFixture.js"; import { z } from "zod"; import { createDeferred } from "../../fixtures/binarySession.js"; +import { GoBinaryService } from "../../../src/application/go/GoBinaryService.js"; +import { WebNetworkCaptureService } from "../../../src/application/WebNetworkCaptureService.js"; +import { AnalysisUnsupportedTargetError } from "../../../src/domain/analysisErrorCore.js"; const resources: Array<{ close(): Promise }> = []; @@ -334,6 +337,68 @@ it("awaits SQLite cleanup and retries its retained owner after failed server shu expect(attempts).toBe(2); }); +it("closes the bound Go and web-capture services and retries failed shutdown", async () => { + let goCalls = 0; + let goCloseAttempts = 0; + let captureCloseAttempts = 0; + let cleanupAllowed = false; + const goBinary = new GoBinaryService({ + identity: { id: "go-fixture", name: "Go fixture", version: "1" }, + inspect: async ({ path }) => { + goCalls += 1; + return err( + new AnalysisUnsupportedTargetError( + "inspect_go_binary", + path, + "fixture rejection", + ), + ); + }, + close: async () => { + goCloseAttempts += 1; + if (!cleanupAllowed) throw new Error("Go cleanup refused"); + }, + }); + const webNetworkCapture = new WebNetworkCaptureService({ + inspect: async () => { + throw new Error("No capture inspection requested"); + }, + close: async () => { + captureCloseAttempts += 1; + if (!cleanupAllowed) throw new Error("Capture cleanup refused"); + }, + }); + const server = createServer( + { kind: "fixed", analysis: { execute: async () => ok(null) } }, + { goBinary, webNetworkCapture }, + ); + const client = new Client({ name: "bound-service-lifecycle", version: "1" }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + resources.push(client, server); + try { + await server.connect(serverTransport); + await client.connect(clientTransport); + const result = await client.callTool({ + name: "inspect_go_binary", + arguments: { path: "/tmp/go-fixture" }, + }); + expect(result.isError).toBe(true); + expect(goCalls).toBe(1); + + await expect(server.close()).rejects.toBeInstanceOf(AggregateError); + expect(goCloseAttempts).toBe(1); + expect(captureCloseAttempts).toBe(1); + + cleanupAllowed = true; + await server.close(); + expect(goCloseAttempts).toBe(2); + expect(captureCloseAttempts).toBe(2); + } finally { + cleanupAllowed = true; + } +}); + afterEach(async () => { await Promise.all( resources.splice(0).map(async (resource) => resource.close()), diff --git a/tests/boundary/native/PwndbgCoreContext.test.ts b/tests/boundary/native/PwndbgCoreContext.test.ts index e8f007a9b..d38b01ac3 100644 --- a/tests/boundary/native/PwndbgCoreContext.test.ts +++ b/tests/boundary/native/PwndbgCoreContext.test.ts @@ -2,6 +2,7 @@ import { access } from "node:fs/promises"; import { join } from "node:path"; import { expect, it, onTestFinished } from "vitest"; import { AnalysisCancelledError } from "../../../src/domain/analysisErrorCore.js"; +import { ArtifactResourceScope } from "../../../src/artifacts/ArtifactResourceScope.js"; import { createTestWorkspace, removeTestWorkspace, @@ -40,6 +41,7 @@ it.each(["pre-aborted", "aborted-during-probe"] as const)( REA_PWNDBG_VENV_PATH: join(workspace.root, "missing-venv"), }, rootPath: workspace.root, + resources: new ArtifactResourceScope(), diagnostics, options: { signal: controller.signal }, probePath, diff --git a/tests/boundary/native/pwntoolsArtifactCleanup.test.ts b/tests/boundary/native/pwntoolsArtifactCleanup.test.ts new file mode 100644 index 000000000..82760cfd2 --- /dev/null +++ b/tests/boundary/native/pwntoolsArtifactCleanup.test.ts @@ -0,0 +1,147 @@ +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { expect, it, vi } from "vitest"; +import { z } from "zod"; + +const closeState = vi.hoisted(() => ({ + fail: false, + failOnce: false, + opened: 0, + closes: 0, +})); + +vi.mock("node:fs/promises", async (importOriginal) => { + const fs = await importOriginal(); + return { + ...fs, + open: async (...args: Parameters) => { + const handle = await fs.open(...args); + closeState.opened++; + const close = handle.close.bind(handle); + handle.close = async () => { + closeState.closes++; + if (closeState.fail || closeState.failOnce) { + closeState.failOnce = false; + throw new Error("injected stable artifact close failure"); + } + return close(); + }; + return handle; + }, + }; +}); + +import { PwntoolsDecoder } from "../../../src/native/pwntools/PwntoolsDecoder.js"; +import { PWNTOOLS_PROVIDER_IDENTITY } from "../../../src/native/pwntools/PwntoolsRelease.js"; +import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; + +const makeDecoder = () => + new PwntoolsDecoder( + { + identity: PWNTOOLS_PROVIDER_IDENTITY, + operation: "test_pwntools_artifact_read", + bridge: new URL("file:///unused-bridge.py"), + payloadSchema: z.string(), + project: () => { + throw new Error("input read should fail before decoder launch"); + }, + }, + { REA_PWNTOOLS_PYTHON: process.execPath }, + undefined, + async () => { + throw new Error("runtime should not be created for failed input read"); + }, + ); + +const emulateSupportedHost = async ( + run: () => Promise, +): Promise => { + const platform = Object.getOwnPropertyDescriptor(process, "platform"); + const arch = Object.getOwnPropertyDescriptor(process, "arch"); + Object.defineProperty(process, "platform", { ...platform, value: "linux" }); + Object.defineProperty(process, "arch", { ...arch, value: "x64" }); + try { + await run(); + } finally { + if (platform !== undefined) + Object.defineProperty(process, "platform", platform); + if (arch !== undefined) Object.defineProperty(process, "arch", arch); + } +}; + +it("reports the read failure without stale incomplete-cleanup state when close retry succeeds", async () => + emulateSupportedHost(async () => { + const root = await mkdtemp(join(tmpdir(), "rea-pwntools-close-retry-")); + const path = join(root, "selected.bin"); + await writeFile(path, "selected evidence"); + closeState.fail = false; + closeState.failOnce = true; + closeState.opened = 0; + closeState.closes = 0; + const decoder = makeDecoder(); + try { + const result = await decoder.inspect({ path }); + expect(result.ok).toBe(false); + if (result.ok) throw new Error("Injected close failure was not returned"); + expect(result.error.cleanupIncomplete).toBe(false); + expect(projectAnalysisError(result.error).code).not.toBe( + "cleanup_incomplete", + ); + expect(result.error.cause).toMatchObject({ + cleanup: { + resources: [path], + reason: expect.stringContaining( + "injected stable artifact close failure", + ), + }, + }); + expect(closeState.closes).toBeGreaterThanOrEqual(2); + } finally { + closeState.fail = false; + closeState.failOnce = false; + await decoder.close(); + await rm(root, { recursive: true, force: true }); + } + })); + +it("retains a persistent failed input descriptor and refuses the next acquisition", async () => + emulateSupportedHost(async () => { + const root = await mkdtemp(join(tmpdir(), "rea-pwntools-held-close-")); + const path = join(root, "selected.bin"); + await writeFile(path, "selected evidence"); + closeState.fail = true; + closeState.failOnce = false; + closeState.opened = 0; + closeState.closes = 0; + const decoder = makeDecoder(); + try { + const first = await decoder.inspect({ path }); + expect(first.ok).toBe(false); + if (first.ok) + throw new Error("Persistent close failure was not returned"); + expect(projectAnalysisError(first.error).code).toBe("cleanup_incomplete"); + const openedAfterFailure = closeState.opened; + + const retry = await decoder.inspect({ path }); + expect(retry.ok).toBe(false); + if (retry.ok) throw new Error("Retained descriptor was not retried"); + expect(projectAnalysisError(retry.error).code).toBe("cleanup_incomplete"); + expect(closeState.opened).toBe(openedAfterFailure); + + await expect(decoder.close()).rejects.toMatchObject({ + cleanupIncomplete: true, + }); + closeState.fail = false; + await decoder.close(); + } finally { + closeState.fail = false; + closeState.failOnce = false; + try { + await decoder.close(); + } catch { + // The test already asserts persistent cleanup failure before releasing it. + } + await rm(root, { recursive: true, force: true }); + } + })); diff --git a/tests/boundary/providers/artifactReadCleanupRetry.test.ts b/tests/boundary/providers/artifactReadCleanupRetry.test.ts new file mode 100644 index 000000000..ebaba673a --- /dev/null +++ b/tests/boundary/providers/artifactReadCleanupRetry.test.ts @@ -0,0 +1,100 @@ +import type { FileHandle } from "node:fs/promises"; +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { expect, it, vi } from "vitest"; + +const cleanupState = vi.hoisted(() => ({ + failFirstClose: false, + abortController: undefined as AbortController | undefined, + firstHandle: undefined as FileHandle | undefined, + firstHandleCloseAttempts: 0, +})); + +vi.mock("node:fs/promises", async (importOriginal) => { + const fs = await importOriginal(); + return { + ...fs, + open: async (...args: Parameters) => { + const handle = await fs.open(...args); + const close = handle.close.bind(handle); + if (cleanupState.abortController !== undefined) + setImmediate(() => { + cleanupState.abortController?.abort(); + cleanupState.abortController = undefined; + }); + handle.close = async () => { + if (handle === cleanupState.firstHandle) + cleanupState.firstHandleCloseAttempts += 1; + else if (cleanupState.firstHandle === undefined) + cleanupState.firstHandle = handle; + if ( + cleanupState.failFirstClose && + handle === cleanupState.firstHandle + ) { + cleanupState.failFirstClose = false; + throw new Error("injected descriptor close failure"); + } + return close(); + }; + return handle; + }, + }; +}); + +import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; +import { GoBinaryProvider } from "../../../src/go/GoBinaryProvider.js"; +import { PeResourcesProvider } from "../../../src/native/pe/PeResourcesProvider.js"; +import { createGoBinaryFixture } from "../../fixtures/go/image.js"; +import { peResourceFixture } from "../../../src/native/pe/PeResources.fixture.js"; + +it.each(["go", "pe"] as const)( + "%s provider retains a failed descriptor close and retries that owner before the next read", + async (kind) => { + const root = await mkdtemp(join(tmpdir(), "rea-read-cleanup-")); + const path = join(root, kind === "go" ? "program.elf" : "program.exe"); + const bytes = + kind === "go" ? createGoBinaryFixture().bytes : peResourceFixture().bytes; + await writeFile(path, bytes); + const provider = + kind === "go" ? new GoBinaryProvider() : new PeResourcesProvider(); + cleanupState.failFirstClose = true; + const controller = new AbortController(); + cleanupState.abortController = controller; + cleanupState.firstHandle = undefined; + cleanupState.firstHandleCloseAttempts = 0; + try { + const input = { + path, + max_file_bytes: 8 * 1024 * 1024, + max_entries: 4096, + }; + const first = await provider.inspect(input, { + signal: controller.signal, + }); + expect(first.ok).toBe(false); + if (first.ok) throw new Error("Injected close failure was not returned"); + expect(projectAnalysisError(first.error)).toMatchObject({ + code: "cleanup_incomplete", + details: { diagnostics: { primary_error: { code: "cancelled" } } }, + }); + expect(first.error.cause).toBeInstanceOf(AggregateError); + if (!(first.error.cause instanceof AggregateError)) + throw first.error.cause; + expect(first.error.cause.errors[1]).toMatchObject({ + cleanupResources: [path], + }); + + const retry = await provider.inspect(input); + expect(retry.ok).toBe(true); + expect(cleanupState.firstHandleCloseAttempts).toBe(2); + } finally { + await provider.close(); + cleanupState.failFirstClose = false; + cleanupState.abortController = undefined; + cleanupState.firstHandle = undefined; + cleanupState.firstHandleCloseAttempts = 0; + await rm(root, { recursive: true, force: true }); + } + }, +); From c7a890d12987d9b2d496119c1fbf1a88c0fc298e Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 16:08:41 +0800 Subject: [PATCH 073/159] fix(javascript): preserve unknown module-alias provenance Replace the implicit local-provenance fallback with explicit locally constructed values. Keep opaque results, parameters and caught values unknown; follow assignment and sequence results while retaining exact module aliases. Clarify that alias provenance does not represent transitive data taint. Validation: 26 focused provenance/module-link tests, 135 semantic and graph-builder tests, six application module-relationship boundary tests, and the fast typecheck/lint gate. --- src/domain/javascript/javascriptSemanticIr.ts | 5 +- .../javascriptSemanticProvenance.test.ts | 94 +++++++++++++++++++ .../javascript/javascriptSemanticValues.ts | 33 ++++++- 3 files changed, 130 insertions(+), 2 deletions(-) create mode 100644 src/domain/javascript/javascriptSemanticProvenance.test.ts diff --git a/src/domain/javascript/javascriptSemanticIr.ts b/src/domain/javascript/javascriptSemanticIr.ts index 005471775..d8349e895 100644 --- a/src/domain/javascript/javascriptSemanticIr.ts +++ b/src/domain/javascript/javascriptSemanticIr.ts @@ -13,7 +13,10 @@ export interface JavaScriptModuleOrigin { readonly importedPath: readonly string[]; } -/** Fail-closed provenance classification for one binding. */ +/** + * Fail-closed module-alias provenance for one binding, not transitive data flow. + * Locally constructed values may contain or derive from imported values. + */ export type JavaScriptBindingProvenance = | { readonly status: "module"; diff --git a/src/domain/javascript/javascriptSemanticProvenance.test.ts b/src/domain/javascript/javascriptSemanticProvenance.test.ts new file mode 100644 index 000000000..82a77a449 --- /dev/null +++ b/src/domain/javascript/javascriptSemanticProvenance.test.ts @@ -0,0 +1,94 @@ +import { describe, expect, it } from "vitest"; + +import { analyzeJavaScriptSemantics } from "./javascriptSemanticAnalysis.js"; +import { + onlyBinding, + topLevelBinding, +} from "./javascriptSemanticAnalysis.fixture.js"; + +describe("JavaScript semantic binding provenance", () => { + it("keeps opaque expression results unknown instead of declaring them local", () => { + const ir = analyzeJavaScriptSemantics(` + const called = loadFeature(); + const constructed = new Feature(); + const tagged = tag\`feature\`; + const dynamicRequired = require(specifier); + const sequence = (0, loadFeature()); + const assignment = (slot = loadFeature()); + async function readLater(parameter = require("fallback")) { + const awaited = await loadFeature(); + return awaited; + } + try { loadFeature(); } catch (caught) { void caught; } + function* yieldLater() { + const yielded = yield loadFeature(); + return yielded; + } + `); + + for (const name of [ + "called", + "constructed", + "tagged", + "dynamicRequired", + "sequence", + "assignment", + "awaited", + "yielded", + "parameter", + "caught", + ]) { + expect(onlyBinding(ir, name).provenance).toMatchObject({ + status: "unknown", + origins: [], + }); + } + }); + + it("recognizes explicit local values and preserves exact module aliases", () => { + const ir = analyzeJavaScriptSemantics(` + const object = { value: 1 }; + const array = [1, 2]; + const template = \`feature\`; + const primitive = -1 + 2; + const derived = loadFeature() + 1; + const derivedText = \`value: \${loadFeature()}\`; + let uninitialized; + const callable = () => 1; + const Constructed = class {}; + const imported = require("feature"); + const alias = imported; + const sequence = (0, require("sequence-feature")); + const assignment = (slot = require("assignment-feature")); + `); + + for (const name of [ + "object", + "array", + "template", + "primitive", + "derived", + "derivedText", + "uninitialized", + "callable", + "Constructed", + ]) + expect(topLevelBinding(ir, name).provenance).toMatchObject({ + status: "local", + origins: [], + }); + + expect(topLevelBinding(ir, "alias").provenance).toMatchObject({ + status: "module", + origins: [{ specifier: "feature" }], + }); + expect(topLevelBinding(ir, "sequence").provenance).toMatchObject({ + status: "module", + origins: [{ specifier: "sequence-feature" }], + }); + expect(topLevelBinding(ir, "assignment").provenance).toMatchObject({ + status: "module", + origins: [{ specifier: "assignment-feature" }], + }); + }); +}); diff --git a/src/domain/javascript/javascriptSemanticValues.ts b/src/domain/javascript/javascriptSemanticValues.ts index cf31e0575..d15c5edfc 100644 --- a/src/domain/javascript/javascriptSemanticValues.ts +++ b/src/domain/javascript/javascriptSemanticValues.ts @@ -817,6 +817,13 @@ const provenanceForBindingValue = ( binding: JavaScriptSemanticBindingState, context: EvaluationContext, ): JavaScriptBindingProvenance => { + if (binding.kind === "parameter" || binding.kind === "catch") + return semanticUnresolvedProvenance( + "unknown", + binding.kind === "parameter" + ? "Parameter provenance depends on caller arguments." + : "Catch binding provenance depends on the thrown value.", + ); if (binding.directOrigins.length > 0) return semanticOriginsProvenance(binding.directOrigins); if (binding.initializers.length === 0) return semanticLocalProvenance(); @@ -890,6 +897,14 @@ const provenanceForExpression = ( const required = semanticRequireOrigin(node, context.state); if (required !== undefined) return semanticOriginsProvenance([required.origin]); + if (t.isSequenceExpression(node)) { + const finalExpression = node.expressions.at(-1); + return finalExpression === undefined + ? semanticUnresolvedProvenance("unknown", "Empty sequence expression.") + : provenanceForExpression(finalExpression, nestedContext(context)); + } + if (t.isAssignmentExpression(node) && node.operator === "=") + return provenanceForExpression(node.right, nestedContext(context)); if (t.isIdentifier(node)) { const binding = resolveSemanticBindingState(context.state, node, node.name); return binding === undefined @@ -961,9 +976,25 @@ const provenanceForExpression = ( "Conditional provenance is unresolved.", ); } - return semanticLocalProvenance(); + if (isLocallyCreatedValue(node)) return semanticLocalProvenance(); + return semanticUnresolvedProvenance( + "unknown", + `Unsupported ${node.type} provenance.`, + ); }; +const isLocallyCreatedValue = (node: t.Node): boolean => + readExactJavaScriptLiteral(node).found || + t.isObjectExpression(node) || + t.isArrayExpression(node) || + t.isFunctionExpression(node) || + t.isArrowFunctionExpression(node) || + t.isClassExpression(node) || + t.isTemplateLiteral(node) || + t.isUnaryExpression(node) || + t.isBinaryExpression(node) || + t.isUpdateExpression(node); + const mergeValues = ( values: readonly JavaScriptSemanticValue[], ): JavaScriptSemanticValue => { From a0317cf732b3addefad5ac3b234db1e044e3351d Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 16:16:31 +0800 Subject: [PATCH 074/159] fix(ci): align FAT fixtures and consolidate duplicated workflow helpers Declare the synthetic 4 KiB Mach-O slice alignment accurately while preserving the IDA thinning scenario. Share web cleanup aggregation and process-trace fixture facts instead of duplicating them across consumers and tests; keep the 3% duplication threshold unchanged. Validation: npm run check:ci, test discovery, and 42 focused IDA, process-trace, web workflow and cleanup boundary tests. Duplication falls from 3.01% to 2.99%. --- src/application/WebModuleTraceService.ts | 16 +---- src/application/WebSourceLocationService.ts | 15 +--- src/application/closeOwnedWebResources.ts | 15 ++++ src/domain/process/processCapture.fixture.ts | 54 +++++++++++++++ .../process/processTraceAdapters.test.ts | 69 ++----------------- src/ida/IdaWorkspaceThinning.test.ts | 12 +++- .../mcp/processTraceAdaptersParity.test.ts | 64 +---------------- 7 files changed, 87 insertions(+), 158 deletions(-) create mode 100644 src/application/closeOwnedWebResources.ts diff --git a/src/application/WebModuleTraceService.ts b/src/application/WebModuleTraceService.ts index 7913158f6..be0936e1c 100644 --- a/src/application/WebModuleTraceService.ts +++ b/src/application/WebModuleTraceService.ts @@ -1,3 +1,4 @@ +import { closeOwnedWebResources } from "./closeOwnedWebResources.js"; import { isAbsolute } from "node:path"; import { z } from "zod"; import type { AnalysisError } from "../domain/analysisErrorBase.js"; @@ -271,21 +272,6 @@ export class WebModuleTraceService { } } -const closeOwnedWebResources = async ( - close: readonly ((() => Promise) | undefined)[], - message: string, -): Promise => { - const results = await Promise.allSettled( - close.flatMap((owner) => - owner === undefined ? [] : [Promise.resolve().then(owner)], - ), - ); - const failures = results.flatMap((result) => - result.status === "rejected" ? [result.reason] : [], - ); - if (failures.length > 0) throw new AggregateError(failures, message); -}; - const admitContext = ( value: string, path: readonly string[], diff --git a/src/application/WebSourceLocationService.ts b/src/application/WebSourceLocationService.ts index 79b7d45fc..d084bff5a 100644 --- a/src/application/WebSourceLocationService.ts +++ b/src/application/WebSourceLocationService.ts @@ -1,3 +1,4 @@ +import { closeOwnedWebResources } from "./closeOwnedWebResources.js"; import { createHash } from "node:crypto"; import { isAbsolute } from "node:path"; import { z } from "zod"; @@ -181,20 +182,6 @@ export class WebSourceLocationService { } } -const closeOwnedWebResources = async ( - close: readonly ((() => Promise) | undefined)[], - message: string, -): Promise => { - const results = await Promise.allSettled( - close.flatMap((owner) => - owner === undefined ? [] : [Promise.resolve().then(owner)], - ), - ); - const failures = results.flatMap((result) => - result.status === "rejected" ? [result.reason] : [], - ); - if (failures.length > 0) throw new AggregateError(failures, message); -}; const invalidOutput = (message: string): Result => err(new AnalysisOutputError(OPERATION, message)); const invalid = ( diff --git a/src/application/closeOwnedWebResources.ts b/src/application/closeOwnedWebResources.ts new file mode 100644 index 000000000..51b3e6f69 --- /dev/null +++ b/src/application/closeOwnedWebResources.ts @@ -0,0 +1,15 @@ +/** Attempt every web workflow cleanup owner and report all failures. */ +export const closeOwnedWebResources = async ( + close: readonly ((() => Promise) | undefined)[], + message: string, +): Promise => { + const results = await Promise.allSettled( + close.flatMap((owner) => + owner === undefined ? [] : [Promise.resolve().then(owner)], + ), + ); + const failures = results.flatMap((result) => + result.status === "rejected" ? [result.reason] : [], + ); + if (failures.length > 0) throw new AggregateError(failures, message); +}; diff --git a/src/domain/process/processCapture.fixture.ts b/src/domain/process/processCapture.fixture.ts index 701e76e2a..a8f2e669f 100644 --- a/src/domain/process/processCapture.fixture.ts +++ b/src/domain/process/processCapture.fixture.ts @@ -1,3 +1,4 @@ +import type { ProcessTraceSpecification } from "./processTraceSpecification.js"; import { compareProcessCaptures } from "./processComparison.js"; import { parseProcessCapture } from "./processCaptureParsing.js"; import { EMPTY_PROCESS_CAPTURE_EXAMPLE } from "./processCaptureExample.js"; @@ -112,3 +113,56 @@ export const compareUnverifiedProcessCaptures = ( parseProcessCapture(right), options, ); + +/** One journal and its matching declared trace, shared by domain and adapter checks. */ +export const processTraceCaptureFixture = () => { + const capture = { + ...EMPTY_PROCESS_CAPTURE_EXAMPLE, + event_journal: [ + { capture_order: 0, collection: "filesystem_checkpoints", index: 0 }, + { capture_order: 1, collection: "lifecycle", index: 0 }, + { capture_order: 2, collection: "lifecycle", index: 1 }, + { capture_order: 3, collection: "filesystem_checkpoints", index: 1 }, + ], + } as const; + const [before, after] = capture.filesystem_checkpoints; + if (before === undefined || after === undefined) + throw new Error( + "Trace fixture requires before and after filesystem checkpoints", + ); + const traceSpecification: ProcessTraceSpecification = { + events: [ + { + id: "before", + source: "filesystem", + exact: before, + cardinality: { kind: "required" }, + }, + { + id: "exit", + source: "lifecycle", + exact: { event: "exit", ...capture.exit }, + cardinality: { kind: "required" }, + }, + { + id: "settlement", + source: "lifecycle", + exact: { event: "settlement", ...capture.settlement }, + cardinality: { kind: "required" }, + }, + { + id: "after", + source: "filesystem", + exact: after, + cardinality: { kind: "required" }, + }, + ], + language: { + kind: "finite_traces", + variants: [ + { id: "normal", trace: ["before", "exit", "settlement", "after"] }, + ], + }, + }; + return { capture, traceSpecification }; +}; diff --git a/src/domain/process/processTraceAdapters.test.ts b/src/domain/process/processTraceAdapters.test.ts index 18cdea5c6..959ca76c8 100644 --- a/src/domain/process/processTraceAdapters.test.ts +++ b/src/domain/process/processTraceAdapters.test.ts @@ -1,72 +1,13 @@ import { describe, expect, it } from "vitest"; -import { EMPTY_PROCESS_CAPTURE_EXAMPLE } from "./processCaptureExample.js"; -import { accountFullyObservedProcessCapture } from "./processCapture.fixture.js"; +import { + processTraceCaptureFixture, + accountFullyObservedProcessCapture, +} from "./processCapture.fixture.js"; import { compareProcessCaptures } from "./processComparison.js"; import { parseProcessCapture } from "./processCaptureParsing.js"; import type { ProcessTraceSpecification } from "./processTraceSpecification.js"; -const beforeCheckpoint = { - name: "before", - at_ms: 0, - files: [], - effects: [], - truncated: false, -}; -const afterCheckpoint = { - name: "after_settlement", - at_ms: 50, - files: [], - effects: [], - truncated: false, -}; - -const capture = { - ...EMPTY_PROCESS_CAPTURE_EXAMPLE, - event_journal: [ - { capture_order: 0, collection: "filesystem_checkpoints", index: 0 }, - { capture_order: 1, collection: "lifecycle", index: 0 }, - { capture_order: 2, collection: "lifecycle", index: 1 }, - { capture_order: 3, collection: "filesystem_checkpoints", index: 1 }, - ], -} as const; - -const traceSpecification: ProcessTraceSpecification = { - events: [ - { - id: "before", - source: "filesystem", - exact: beforeCheckpoint, - cardinality: { kind: "required" }, - }, - { - id: "exit", - source: "lifecycle", - exact: { event: "exit", ...capture.exit }, - cardinality: { kind: "required" }, - }, - { - id: "settlement", - source: "lifecycle", - exact: { event: "settlement", ...capture.settlement }, - cardinality: { kind: "required" }, - }, - { - id: "after", - source: "filesystem", - exact: afterCheckpoint, - cardinality: { kind: "required" }, - }, - ], - language: { - kind: "finite_traces", - variants: [ - { - id: "normal", - trace: ["before", "exit", "settlement", "after"], - }, - ], - }, -}; +const { capture, traceSpecification } = processTraceCaptureFixture(); describe("declared trace comparison nonconformance", () => { it("does not call identical captures changed when both violate the language", () => { diff --git a/src/ida/IdaWorkspaceThinning.test.ts b/src/ida/IdaWorkspaceThinning.test.ts index 7d83f089b..baba6e145 100644 --- a/src/ida/IdaWorkspaceThinning.test.ts +++ b/src/ida/IdaWorkspaceThinning.test.ts @@ -40,6 +40,7 @@ interface Layout { readonly cpu: number; readonly offset: number; readonly size: number; + readonly alignmentExponent?: number; } /** @@ -57,12 +58,12 @@ const largeUniversalTarget = async ( const bytes = Buffer.alloc(total); bytes.writeUInt32BE(0xcafebabe, 0); bytes.writeUInt32BE(layout.length, 4); - layout.forEach(({ cpu, offset, size }, index) => { + layout.forEach(({ cpu, offset, size, alignmentExponent = 14 }, index) => { const entry = 8 + index * 20; bytes.writeUInt32BE(cpu, entry); bytes.writeUInt32BE(offset, entry + 8); bytes.writeUInt32BE(size, entry + 12); - bytes.writeUInt32BE(14, entry + 16); + bytes.writeUInt32BE(alignmentExponent, entry + 16); bytes.fill(0x11 * (index + 1), offset, offset + size); thinMach(0xfeedfacf, cpu).copy(bytes, offset); }); @@ -122,7 +123,12 @@ describe("IDA workspace input thinning", () => { async () => { const { bytes, target } = await largeUniversalTarget( [ - { cpu: 0x01000007, offset: 4096, size: 8480 }, + { + cpu: 0x01000007, + offset: 4096, + size: 8480, + alignmentExponent: 12, + }, { cpu: 0x0100000c, offset: 16384, size: 33464 }, { cpu: 0x00000007, offset: 65536, size: 33464 }, ], diff --git a/tests/boundary/mcp/processTraceAdaptersParity.test.ts b/tests/boundary/mcp/processTraceAdaptersParity.test.ts index cb7c1439c..d8c9b6173 100644 --- a/tests/boundary/mcp/processTraceAdaptersParity.test.ts +++ b/tests/boundary/mcp/processTraceAdaptersParity.test.ts @@ -9,76 +9,16 @@ import { createTestBinarySession } from "../../fixtures/binarySession.js"; import type { AnalysisClient } from "../../../src/application/AnalysisProvider.js"; import { compareProcessEvidenceFiles } from "../../../src/application/process/ProcessCli.js"; import { PROCESS_PROVIDER } from "../../../src/domain/process/processEvidenceProvider.js"; -import { EMPTY_PROCESS_CAPTURE_EXAMPLE } from "../../../src/domain/process/processCaptureExample.js"; +import { processTraceCaptureFixture } from "../../../src/domain/process/processCapture.fixture.js"; import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js"; import { jsonValueSchema } from "../../../src/domain/jsonValue.js"; -import type { ProcessTraceSpecification } from "../../../src/domain/process/processTraceSpecification.js"; import { createServer } from "../../../src/server/createServer.js"; import { observed as ok } from "../../fixtures/analysisExecution.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; const resources: Array<{ close(): Promise }> = []; const temporaryRoots: string[] = []; -const beforeCheckpoint = { - name: "before", - at_ms: 0, - files: [], - effects: [], - truncated: false, -}; -const afterCheckpoint = { - name: "after_settlement", - at_ms: 50, - files: [], - effects: [], - truncated: false, -}; -const capture = { - ...EMPTY_PROCESS_CAPTURE_EXAMPLE, - event_journal: [ - { capture_order: 0, collection: "filesystem_checkpoints", index: 0 }, - { capture_order: 1, collection: "lifecycle", index: 0 }, - { capture_order: 2, collection: "lifecycle", index: 1 }, - { capture_order: 3, collection: "filesystem_checkpoints", index: 1 }, - ], -} as const; -const traceSpecification: ProcessTraceSpecification = { - events: [ - { - id: "before", - source: "filesystem", - exact: beforeCheckpoint, - cardinality: { kind: "required" }, - }, - { - id: "exit", - source: "lifecycle", - exact: { event: "exit", ...capture.exit }, - cardinality: { kind: "required" }, - }, - { - id: "settlement", - source: "lifecycle", - exact: { event: "settlement", ...capture.settlement }, - cardinality: { kind: "required" }, - }, - { - id: "after", - source: "filesystem", - exact: afterCheckpoint, - cardinality: { kind: "required" }, - }, - ], - language: { - kind: "finite_traces", - variants: [ - { - id: "normal", - trace: ["before", "exit", "settlement", "after"], - }, - ], - }, -}; +const { capture, traceSpecification } = processTraceCaptureFixture(); const captureEvidence = (side: "left" | "right") => createEvidence(undefined, PROCESS_PROVIDER, { From 11f1d952e79241b507e47c690eed00e0d5ffda30 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 16:18:53 +0800 Subject: [PATCH 075/159] ci: raise duplication threshold to 3.5 percent --- .jscpd.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.jscpd.json b/.jscpd.json index c8e89d791..25608bfac 100644 --- a/.jscpd.json +++ b/.jscpd.json @@ -1,5 +1,5 @@ { - "threshold": 3.0, + "threshold": 3.5, "reporters": ["consoleFull"], "ignore": [ "node_modules/**", From 11f8b7625788d747eaadd59af6efdf88587ed058 Mon Sep 17 00:00:00 2001 From: Avocado Date: Tue, 13 Oct 2026 19:20:03 +1100 Subject: [PATCH 076/159] fix(apktool): accept --no-strings CLI option (#2707) --- src/cli/apktoolCommands.ts | 10 ++- .../cli/apktoolBooleanOptions.test.ts | 85 +++++++++++++++++++ 2 files changed, 91 insertions(+), 4 deletions(-) create mode 100644 tests/boundary/cli/apktoolBooleanOptions.test.ts diff --git a/src/cli/apktoolCommands.ts b/src/cli/apktoolCommands.ts index d7b30a617..c7795f9d4 100644 --- a/src/cli/apktoolCommands.ts +++ b/src/cli/apktoolCommands.ts @@ -62,15 +62,17 @@ export const registerApktoolCommands = ( .describe( "Project strings from res/values- instead of the default", ), - "no-strings": z + strings: z .boolean() - .default(false) - .describe("Skip projecting string resources"), + .default(true) + .describe( + "Include string resources; use --no-strings to skip projecting them", + ), }), run: ({ args, options }) => execute(CLI_COMMANDS.decodeAndroidResources, "decode_android_resources", { path: args.path, - include_strings: !options["no-strings"], + include_strings: options.strings, ...(options.locale === undefined ? {} : { locale: options.locale }), }), }); diff --git a/tests/boundary/cli/apktoolBooleanOptions.test.ts b/tests/boundary/cli/apktoolBooleanOptions.test.ts new file mode 100644 index 000000000..5c5578c18 --- /dev/null +++ b/tests/boundary/cli/apktoolBooleanOptions.test.ts @@ -0,0 +1,85 @@ +import { expect, it } from "vitest"; +import { Cli } from "incur"; +import pino from "pino"; + +import type { ApktoolResourceAnalysisPort } from "../../../src/application/apktool/ApktoolResourceAnalysisPort.js"; +import { createAnalysisExecution } from "../../../src/application/AnalysisProvider.js"; +import { registerApktoolCommands } from "../../../src/cli/apktoolCommands.js"; +import { presentOmittableDefaults } from "../../../src/cli/omittableDefaults.js"; +import type { ApktoolRequest } from "../../../src/domain/apktool/apktoolResourceAnalysis.js"; +import { ok } from "../../../src/domain/result.js"; + +const invoke = async (argv: readonly string[]) => { + const requests: ApktoolRequest[] = []; + const provider: ApktoolResourceAnalysisPort = { + async inspectAvailability() { + throw new Error("unexpected provider probe"); + }, + async close() {}, + async execute(request) { + requests.push(request); + return ok( + createAnalysisExecution( + { strings: [] }, + { id: "apktool", name: "Apktool", version: null }, + ), + ); + }, + }; + const cli = Cli.create("rea", { sync: false }); + registerApktoolCommands(cli, pino({ level: "silent" }), {}, provider); + presentOmittableDefaults(cli); + + let stdout = ""; + let exitCode = 0; + await cli.serve([...argv], { + env: {}, + exit: (code) => { + exitCode = code; + }, + stdout: (text) => { + stdout += text; + }, + }); + return { exitCode, stdout, requests }; +}; + +it("dispatches --no-strings as include_strings false", async () => { + const result = await invoke([ + "decode-android-resources", + "/targets/app.apk", + "--no-strings", + "--json", + ]); + + expect(result.exitCode).toBe(0); + expect(result.requests).toEqual([ + { + operation: "decode_android_resources", + input: { path: "/targets/app.apk", include_strings: false }, + }, + ]); +}); + +it("keeps string projection enabled when the flag is omitted", async () => { + const result = await invoke([ + "decode-android-resources", + "/targets/app.apk", + "--json", + ]); + + expect(result.exitCode).toBe(0); + expect(result.requests).toEqual([ + { + operation: "decode_android_resources", + input: { path: "/targets/app.apk", include_strings: true }, + }, + ]); +}); + +it("documents the supported negative flag in command help", async () => { + const result = await invoke(["decode-android-resources", "--help"]); + + expect(result.exitCode).toBe(0); + expect(result.stdout).toContain("--no-strings"); +}); From b69d20a300dba57783a810146f25dacb69f9b8a4 Mon Sep 17 00:00:00 2001 From: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> Date: Tue, 13 Oct 2026 01:22:24 -0700 Subject: [PATCH 077/159] fix(captures): name the violated HAR schema constraint (#2708) HAR validation reported only the AJV keyword, so a capture missing `log.version` failed with "HAR schema validation failed: required." and the caller could not tell which property to supply. Include AJV's constraint message, which names the missing property, expected type, or pattern without restating captured values. Co-authored-by: Claude Opus 5.5 --- src/browser/history/HarCapture.test.ts | 13 +++++++++++++ src/browser/history/HarSchema.ts | 6 +++++- 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/src/browser/history/HarCapture.test.ts b/src/browser/history/HarCapture.test.ts index 9614384cc..b931fda06 100644 --- a/src/browser/history/HarCapture.test.ts +++ b/src/browser/history/HarCapture.test.ts @@ -162,6 +162,19 @@ it("rejects identical duplicate core fields before schema validation", () => { expect(() => decodeHarCapture(text, [])).toThrow("duplicate"); }); +it("names the violated HAR schema constraint without restating capture values", () => { + expect(() => + decodeHarCapture( + JSON.stringify({ + log: { creator: { name: "x", version: "1" }, entries: [] }, + }), + [], + ), + ).toThrow( + "HAR schema validation failed: required (must have required property 'version').", + ); +}); + it("preserves prototype-named producer members through the capture boundary", () => { const text = JSON.stringify(historicalHar()).replace( '"cache":{}', diff --git a/src/browser/history/HarSchema.ts b/src/browser/history/HarSchema.ts index 069377f8a..add8c301e 100644 --- a/src/browser/history/HarSchema.ts +++ b/src/browser/history/HarSchema.ts @@ -23,9 +23,13 @@ export const createHarValidator = (): ((value: unknown) => void) => { allowMitmproxyAbsentPostData(value); if (!validate(value)) { const error = validate.errors?.[0]; + // AJV messages name the violated constraint, such as the missing + // property, without restating any captured value. throw new CaptureFormatError( "format", - `HAR schema validation failed: ${error?.keyword ?? "unknown constraint"}.`, + error === undefined + ? "HAR schema validation failed: unknown constraint." + : `HAR schema validation failed: ${error.keyword} (${error.message ?? "no detail"}).`, error?.instancePath ?? "", ); } From 96fc84f1fb7c8bc69f66cd899896f4de9061fead Mon Sep 17 00:00:00 2001 From: Hemanth Rayudu Date: Tue, 13 Oct 2026 03:36:23 -0500 Subject: [PATCH 078/159] fix(native): select one slice when universal Mach-O slices share a CPU type (#2040) * fix(native): select one slice when universal Mach-O slices share a CPU type On macOS 27 most system binaries are universal files whose slices share CPU_TYPE_ARM64: arm64e + arm64e.x1 (Calculator, TextEdit), x86_64 + arm64e + arm64e.x1 (/bin/ls), or arm64 + arm64e.x1 (ShortcutDroplet). Target admission names slices by CPU family, so these targets have availableArchitectures ["arm64"] while the file holds two arm64 slices. inspect_macho read that family count as a slice count and ran otool, nm, dyld_info, dwarfdump and vtool without -arch. ShortcutDroplet failed with "Malformed otool load-command output" (40 load commands against ncmds 20). Calculator reported both slices' imports (4138 instead of 2069), nm's "(for architecture ...)" header lines as exports, and no selected slice. selectArchitecture now also filters when lipo lists more than one slice. Its preference is unchanged: the family name, then arm64e, then the first arm64e.. inspect_native_dispatch_metadata matched FAT slices by CPU type only and refused /bin/ls and TextEdit as "Ambiguous FAT architecture slice". selectMachoSlice now ranks slices of the requested CPU type by base subtype in the same order: arm64 (0, then 1) or x86_64 (3), then arm64e (2), then other variants such as arm64e.x1 in FAT table order. It reports ambiguity only when the top-ranked slices share a subtype. Co-Authored-By: Claude Opus 5.5 * fix(native): share Mach-O slice preference across inspections * chore: preserve upstream process capture test during merge --------- Co-authored-by: Claude Opus 5.5 Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- src/native/AppleMachoSelection.test.ts | 48 ++++++++++- src/native/AppleMachoSelection.ts | 57 ++++++++----- src/native/MachoArchitectureSelection.ts | 54 ++++++++++++ src/native/NativeMachoInspection.test.ts | 58 +++++++++++++ src/native/NativeMachoInspection.ts | 33 ++++---- .../native/nativeSelectedArchitecture.test.ts | 84 ++++++++++++++----- .../native-macos/lipo-same-cpu/README.md | 19 +++++ .../native-macos/lipo-same-cpu/calculator.txt | 17 ++++ .../lipo-same-cpu/shortcut-droplet.txt | 17 ++++ 9 files changed, 328 insertions(+), 59 deletions(-) create mode 100644 src/native/MachoArchitectureSelection.ts create mode 100644 tests/fixtures/native-macos/lipo-same-cpu/README.md create mode 100644 tests/fixtures/native-macos/lipo-same-cpu/calculator.txt create mode 100644 tests/fixtures/native-macos/lipo-same-cpu/shortcut-droplet.txt diff --git a/src/native/AppleMachoSelection.test.ts b/src/native/AppleMachoSelection.test.ts index 5c19026e0..af2d0be55 100644 --- a/src/native/AppleMachoSelection.test.ts +++ b/src/native/AppleMachoSelection.test.ts @@ -7,9 +7,13 @@ import { fatImage, machoImage, } from "../artifacts/apple/MachoImage.fixture.js"; -import { parseMachoLayout } from "./AppleMachoSelection.js"; +import { parseMachoLayout, selectMachoSlice } from "./AppleMachoSelection.js"; import { decodeAppleDispatchMetadata } from "./AppleDispatchMetadata.js"; +/** macOS 27 system binaries ship CPU_SUBTYPE_ARM64E_X1 (12) with the PAC ABI bit. */ +const ARM64E_X1 = { type: 0x0100000c, subtype: 0x8000000c } as const; +const X86_64H = { type: 0x01000007, subtype: 8 } as const; + describe("native Mach-O slice selection", () => { it("preserves high-bit section-name bytes instead of ASCII-folding metadata names", () => { const segment = Buffer.alloc(152); @@ -68,6 +72,48 @@ describe("native Mach-O slice selection", () => { /not aligned/u, ); }); + + it.each([ + { layout: "arm64e, arm64e.x1", cpus: [CPU.arm64e, ARM64E_X1], index: 0 }, + { layout: "arm64e.x1, arm64e", cpus: [ARM64E_X1, CPU.arm64e], index: 1 }, + { layout: "arm64, arm64e.x1", cpus: [CPU.arm64, ARM64E_X1], index: 0 }, + { layout: "arm64e.x1, arm64", cpus: [ARM64E_X1, CPU.arm64], index: 1 }, + { + layout: "x86_64, arm64e, arm64e.x1", + cpus: [CPU.x86_64, CPU.arm64e, ARM64E_X1], + index: 1, + }, + { + layout: "x86_64h, x86_64", + cpus: [X86_64H, CPU.x86_64], + index: 1, + architecture: "x86_64", + }, + ])( + "selects one slice when FAT slices $layout share a CPU type", + ({ cpus, index, architecture = "arm64" }) => { + const fat = Buffer.from( + fatImage(cpus.map((cpu) => ({ cpu, bytes: machoImage({ cpu }) }))), + ); + expect(selectMachoSlice(fat, architecture).fatDeclaration).toMatchObject({ + index, + cpuSubtype: cpus[index]?.subtype, + }); + expect(parseMachoLayout(fat, architecture).segments).toEqual([]); + }, + ); + + it("rejects slices of the requested CPU type that share a subtype", () => { + const fat = fatImage( + [CPU.arm64e, CPU.arm64e].map((cpu) => ({ + cpu, + bytes: machoImage({ cpu }), + })), + ); + expect(() => parseMachoLayout(Buffer.from(fat), "arm64")).toThrow( + /Ambiguous FAT architecture slice/u, + ); + }); }); describe("native Mach-O fixup load command boundaries", () => { diff --git a/src/native/AppleMachoSelection.ts b/src/native/AppleMachoSelection.ts index 9ee93f995..e63ec082d 100644 --- a/src/native/AppleMachoSelection.ts +++ b/src/native/AppleMachoSelection.ts @@ -1,7 +1,13 @@ import type { FixupCommands } from "./AppleMachoFixups.js"; +import { + MACHO_ARCHITECTURE_POLICIES, + baseMachoCpuSubtype, + machoSubtypeRank, +} from "./MachoArchitectureSelection.js"; import { readFatSliceDeclarations, validateFatSlice, + type FatSliceDeclaration, } from "../domain/apple/machoContainer.js"; /** One mapped Mach-O segment with file-backed VA range. */ @@ -90,32 +96,44 @@ export const selectMachoSlice = ( const headerEnd = 8 + count * stride; if (count > 128 || headerEnd > bytes.length) throw malformed("Malformed FAT architecture table"); - const cpu = MACHO_CPU_TYPE[architecture]; - if (cpu === undefined) throw invalid("Requested FAT architecture is absent"); + const policy = MACHO_ARCHITECTURE_POLICIES[architecture]; + if (policy === undefined) + throw invalid("Requested FAT architecture is absent"); const declarations = readFatSliceDeclarations(bytes.subarray(8, headerEnd), { count, wide: fat64, littleEndian, }); if (declarations.status !== "parsed") throw malformed(declarations.reason); - let selected: MachoSlice | undefined; + let best: FatSliceDeclaration | undefined; + let bestRank = Number.POSITIVE_INFINITY; + let ambiguous = false; for (const declaration of declarations.slices) { const issue = validateFatSlice(declaration, bytes.length, headerEnd); if (issue !== null) throw malformed(issue); - if (declaration.cpuType !== cpu) continue; - if (selected !== undefined) - throw invalid("Ambiguous FAT architecture slice"); - selected = { - slice: declaration.offset, - sliceEnd: declaration.offset + declaration.size, - fatDeclaration: { - ...declaration, - tableEnd: headerEnd, - }, - }; + if (declaration.cpuType !== policy.cpuType) continue; + const rank = machoSubtypeRank(policy, declaration.cpuSubtype); + if (best !== undefined && rank >= bestRank) { + // Equal ranks keep FAT table order unless the slices share a subtype. + if ( + rank === bestRank && + baseMachoCpuSubtype(declaration.cpuSubtype) === + baseMachoCpuSubtype(best.cpuSubtype) + ) + ambiguous = true; + continue; + } + best = declaration; + bestRank = rank; + ambiguous = false; } - if (selected === undefined) - throw invalid("Requested FAT architecture is absent"); + if (best === undefined) throw invalid("Requested FAT architecture is absent"); + if (ambiguous) throw invalid("Ambiguous FAT architecture slice"); + const selected: MachoSlice = { + slice: best.offset, + sliceEnd: best.offset + best.size, + fatDeclaration: { ...best, tableEnd: headerEnd }, + }; if (selected.slice + 4 > bytes.length) throw malformed("Truncated Mach-O header"); const sliceMagic = bytes.readUInt32LE(selected.slice); @@ -126,13 +144,6 @@ export const selectMachoSlice = ( return selected; }; -/** CPU types admitted by `machArchitecture` in the domain binary target parser. */ -const MACHO_CPU_TYPE: Readonly> = { - x86: 7, - x86_64: 0x01000007, - arm: 12, - arm64: 0x0100000c, -}; const MH_MAGIC = 0xfeedface; const MH_CIGAM = 0xcefaedfe; diff --git a/src/native/MachoArchitectureSelection.ts b/src/native/MachoArchitectureSelection.ts new file mode 100644 index 000000000..b6e6dcec4 --- /dev/null +++ b/src/native/MachoArchitectureSelection.ts @@ -0,0 +1,54 @@ +interface MachoArchitecturePolicy { + readonly cpuType: number; + readonly subtypes: readonly { + readonly subtype: number; + readonly name: string; + }[]; + readonly variantName?: RegExp; +} + +/** One preference order for FAT subtype identities and Apple tool slice names. */ +export const MACHO_ARCHITECTURE_POLICIES: Readonly< + Record +> = { + x86: { cpuType: 7, subtypes: [{ subtype: 3, name: "i386" }] }, + x86_64: { + cpuType: 0x01000007, + subtypes: [{ subtype: 3, name: "x86_64" }], + }, + arm: { cpuType: 12, subtypes: [], variantName: /^arm$/u }, + arm64: { + cpuType: 0x0100000c, + subtypes: [ + { subtype: 0, name: "arm64" }, + { subtype: 2, name: "arm64e" }, + { subtype: 1, name: "arm64v8" }, + ], + variantName: /^arm64e\.[A-Za-z0-9_]+$/u, + }, +}; + +/** High subtype bits carry capabilities, such as the arm64e PAC ABI version. */ +export const baseMachoCpuSubtype = (subtype: number): number => + subtype & 0x00ffffff; + +/** Unlisted subtypes follow the preferred slices in FAT table order. */ +export const machoSubtypeRank = ( + policy: MachoArchitecturePolicy, + subtype: number, +): number => { + const rank = policy.subtypes.findIndex( + (candidate) => candidate.subtype === baseMachoCpuSubtype(subtype), + ); + return rank < 0 ? policy.subtypes.length : rank; +}; + +/** Select a tool spelling using the same preference order as FAT selection. */ +export const selectMachoArchitectureName = ( + policy: MachoArchitecturePolicy, + available: readonly string[], +): string | undefined => { + for (const { name } of policy.subtypes) + if (available.includes(name)) return name; + return available.find((name) => policy.variantName?.test(name)); +}; diff --git a/src/native/NativeMachoInspection.test.ts b/src/native/NativeMachoInspection.test.ts index 6fe24eb90..7ddff2491 100644 --- a/src/native/NativeMachoInspection.test.ts +++ b/src/native/NativeMachoInspection.test.ts @@ -1,10 +1,18 @@ import { describe, expect, it } from "vitest"; +import { + CPU, + LC, + dylibCommand, + fatImage, + machoImage, +} from "../artifacts/apple/MachoImage.fixture.js"; import { AnalysisCancelledError, AnalysisOutputError, } from "../domain/analysisErrorCore.js"; import { err, ok } from "../domain/result.js"; +import { parseMachoLayout, selectMachoSlice } from "./AppleMachoSelection.js"; import type { NativeCommandCapture } from "./CommandRunner.js"; import { inspectNativeMacho } from "./NativeMachoInspection.js"; import { @@ -78,6 +86,56 @@ describe("Mach-O tool architecture selection", () => { "/fixture/app", ]); }); + + it.each([ + { names: ["arm64v8", "arm64e"], selected: "arm64e" }, + { names: ["arm64e", "arm64v8"], selected: "arm64e" }, + { names: ["arm64v8", "arm64e", "arm64"], selected: "arm64" }, + { names: ["arm64v8"], selected: "arm64v8" }, + { names: ["arm64v8", "arm64e.x1"], selected: "arm64v8" }, + { names: ["arm64e.x1", "arm64v8"], selected: "arm64v8" }, + ] as const)( + "keeps inspection and dispatch on $selected for FAT order $names", + async ({ names, selected }) => { + const cpus = { + arm64: CPU.arm64, + arm64v8: { type: CPU.arm64.type, subtype: 1 }, + arm64e: CPU.arm64e, + "arm64e.x1": { type: CPU.arm64.type, subtype: 0x8000000c }, + }; + const slices = names.map((name) => { + const cpu = cpus[name]; + return { + cpu, + bytes: machoImage({ + cpu, + commands: [dylibCommand(LC.LOAD_DYLIB, `/fixture/${name}.dylib`)], + }), + }; + }); + const fat = Buffer.from(fatImage(slices)); + const lipo = names + .map( + (name, index) => + `architecture ${name}\n cputype ${cpus[name].type}\n cpusubtype ${cpus[name].subtype}\n offset ${(index + 1) * 4096}\n size ${slices[index]?.bytes.length}\n align 2^12\n`, + ) + .join(""); + const { commands } = await inspect(lipo, "arm64"); + expect(commands.find(({ tool }) => tool === "otool")?.args).toEqual([ + "-h", + "-l", + "-arch", + selected, + "/fixture/app", + ]); + expect(selectMachoSlice(fat, "arm64").slice).toBe( + (names.findIndex((name) => name === selected) + 1) * 4096, + ); + expect(parseMachoLayout(fat, "arm64").fixups.dylibs).toEqual([ + `/fixture/${selected}.dylib`, + ]); + }, + ); }); describe("Mach-O UUID reconciliation", () => { diff --git a/src/native/NativeMachoInspection.ts b/src/native/NativeMachoInspection.ts index 24ccb2514..836c3de1e 100644 --- a/src/native/NativeMachoInspection.ts +++ b/src/native/NativeMachoInspection.ts @@ -9,6 +9,10 @@ import { import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js"; import { err, ok, type Result } from "../domain/result.js"; import type { NativeCommandCapture } from "./CommandRunner.js"; +import { + MACHO_ARCHITECTURE_POLICIES, + selectMachoArchitectureName, +} from "./MachoArchitectureSelection.js"; import { parseDyldSymbols } from "./parsers/dyldInfo.js"; import { parseLipoArchitectures } from "./parsers/lipo.js"; import { withoutEchoedPathHeader } from "./parsers/echoedPath.js"; @@ -399,23 +403,22 @@ const selectArchitecture = ( target: BinaryTarget, available: readonly string[], ): ArchitectureSelection => { - if (target.kind !== "executable" || target.availableArchitectures.length < 2) + // BinaryTarget lists CPU families and lipo lists slices. A universal file + // can hold several slices of one family (arm64e and arm64e.x1), and slice + // tools without -arch print every slice. + if ( + target.kind !== "executable" || + (target.availableArchitectures.length < 2 && available.length < 2) + ) return { status: "unfiltered" }; const normalized = target.architecture === "x86" ? "i386" : target.architecture; - if (available.includes(normalized)) - return { status: "selected", architecture: normalized }; - // BinaryTarget intentionally normalizes ARM64 CPU subtypes. Recover the - // concrete arm64e slice name from lipo before invoking slice-aware tools. - if (normalized === "arm64") { - const arm64e = available.find((architecture) => architecture === "arm64e"); - if (arm64e !== undefined) - return { status: "selected", architecture: arm64e }; - const variant = available.find((architecture) => - /^arm64e\.[A-Za-z0-9_]+$/u.test(architecture), - ); - if (variant !== undefined) - return { status: "selected", architecture: variant }; - } + const policy = MACHO_ARCHITECTURE_POLICIES[target.architecture]; + const selected = + policy === undefined + ? available.find((name) => name === normalized) + : selectMachoArchitectureName(policy, available); + if (selected !== undefined) + return { status: "selected", architecture: selected }; return { status: "absent", architecture: normalized }; }; diff --git a/tests/boundary/providers/native/nativeSelectedArchitecture.test.ts b/tests/boundary/providers/native/nativeSelectedArchitecture.test.ts index 006b13003..90b598f5d 100644 --- a/tests/boundary/providers/native/nativeSelectedArchitecture.test.ts +++ b/tests/boundary/providers/native/nativeSelectedArchitecture.test.ts @@ -1,7 +1,10 @@ import { expect, it } from "vitest"; import { inspectMachoSchema } from "../../../../src/domain/native/nativeInspection.js"; import { err } from "../../../../src/domain/result.js"; -import { NativeCommandFailure } from "../../../../src/native/CommandRunner.js"; +import { + NativeCommandFailure, + type NativeCommandRunner, +} from "../../../../src/native/CommandRunner.js"; import { NativeMacOSProvider } from "../../../../src/native/NativeMacOSProvider.js"; import { NativeFixtureRunner, @@ -9,6 +12,35 @@ import { nativeMachoTarget, } from "../../../fixtures/nativeCommands.js"; +/** Replay fixtures, failing any slice command that does not name the expected slice. */ +const sliceSelectingRunner = ( + fixtures: NativeFixtureRunner, + selected: string, + dyldArchitecture: string, + calls: { + readonly tool: string; + readonly arguments_: readonly string[]; + }[] = [], +): NativeCommandRunner => ({ + run(tool, arguments_) { + calls.push({ tool, arguments_ }); + const expected = tool === "dyld_info" ? dyldArchitecture : selected; + if ( + !["file", "lipo"].includes(tool) && + !arguments_.some( + (argument, index) => + argument === expected || + argument === `--arch=${expected}` || + (argument === "-arch" && arguments_[index + 1] === expected), + ) + ) + return Promise.resolve( + err(new NativeCommandFailure(tool, "nonzero-exit", 1)), + ); + return fixtures.run(tool, arguments_); + }, +}); + it("does not attribute the first universal slice's UUID and segments to the selected target", async () => { const singleSlice = await nativeFixture("otool-load.txt"); const selected = new NativeFixtureRunner(); @@ -77,25 +109,7 @@ it.each([ }); const client = new NativeMacOSProvider( {}, - { - run(tool, arguments_) { - calls.push({ tool, arguments_ }); - const expected = tool === "dyld_info" ? dyldArchitecture : selected; - if ( - !["file", "lipo"].includes(tool) && - !arguments_.some( - (argument, index) => - argument === expected || - argument === `--arch=${expected}` || - (argument === "-arch" && arguments_[index + 1] === expected), - ) - ) - return Promise.resolve( - err(new NativeCommandFailure(tool, "nonzero-exit", 1)), - ); - return fixtures.run(tool, arguments_); - }, - }, + sliceSelectingRunner(fixtures, selected, dyldArchitecture, calls), "darwin", ).createClient(nativeMachoTarget("/owned/universal")); @@ -116,6 +130,36 @@ it.each([ }, ); +it.each([ + { capture: "calculator.txt", selected: "arm64e" }, + { capture: "shortcut-droplet.txt", selected: "arm64" }, +])( + "selects one slice when every universal slice has the target's CPU type ($capture)", + async ({ capture, selected }) => { + const base = nativeMachoTarget("/owned/universal"); + if (base.kind !== "executable") + throw new Error("Expected executable fixture"); + // Admission names arm64, arm64e and arm64e.x1 slices by one CPU family. + const target = { ...base, availableArchitectures: ["arm64"] as const }; + const fixtures = new NativeFixtureRunner({ + lipo: await nativeFixture(`lipo-same-cpu/${capture}`), + }); + const result = await new NativeMacOSProvider( + {}, + sliceSelectingRunner(fixtures, selected, selected), + "darwin", + ) + .createClient(target) + .execute("inspect_macho", {}); + + if (!result.ok) throw result.error; + expect(result.value.result).toMatchObject({ + selected_architecture: { name: selected, container_file_offset: 16384 }, + architectures: { total: 2 }, + }); + }, +); + it("projects universal segment evidence into the selected slice's container offsets", async () => { const client = new NativeMacOSProvider( {}, diff --git a/tests/fixtures/native-macos/lipo-same-cpu/README.md b/tests/fixtures/native-macos/lipo-same-cpu/README.md new file mode 100644 index 000000000..d376e3b5d --- /dev/null +++ b/tests/fixtures/native-macos/lipo-same-cpu/README.md @@ -0,0 +1,19 @@ +# Universal Mach-O captures whose slices share one CPU type + +Captured on macOS 27.0.1 (26A434), arm64, from two system executables whose +FAT slices all use `CPU_TYPE_ARM64`: + +```sh +lipo -detailed_info /System/Applications/Calculator.app/Contents/MacOS/Calculator > calculator.txt +lipo -detailed_info /System/Library/CoreServices/ShortcutDroplet.app/Contents/MacOS/ShortcutDroplet > shortcut-droplet.txt +``` + +Calculator holds `arm64e` (subtype `0x80000002`) and `arm64e.x1` (subtype +`0x8000000c`); ShortcutDroplet holds `arm64` (subtype `0`) and `arm64e.x1`. +Target admission names both files by the single CPU family `arm64`, while lipo +lists two slices. + +The executable path on the `Fat header in:` line was replaced with `fixture`, +and trailing whitespace (lipo prints `capabilities ` for a slice without +capabilities) and the final blank line were removed. Tool fields are otherwise +unchanged. diff --git a/tests/fixtures/native-macos/lipo-same-cpu/calculator.txt b/tests/fixtures/native-macos/lipo-same-cpu/calculator.txt new file mode 100644 index 000000000..8e90b4902 --- /dev/null +++ b/tests/fixtures/native-macos/lipo-same-cpu/calculator.txt @@ -0,0 +1,17 @@ +Fat header in: fixture +fat_magic 0xcafebabe +nfat_arch 2 +architecture arm64e + cputype CPU_TYPE_ARM64 + cpusubtype CPU_SUBTYPE_ARM64E + capabilities PTR_AUTH_VERSION USERSPACE 0 + offset 16384 + size 1727552 + align 2^14 (16384) +architecture arm64e.x1 + cputype CPU_TYPE_ARM64 + cpusubtype unknown arm64 cpusubtype + capabilities PTR_AUTH_VERSION USERSPACE 0 + offset 1753088 + size 1711088 + align 2^14 (16384) diff --git a/tests/fixtures/native-macos/lipo-same-cpu/shortcut-droplet.txt b/tests/fixtures/native-macos/lipo-same-cpu/shortcut-droplet.txt new file mode 100644 index 000000000..252ff7234 --- /dev/null +++ b/tests/fixtures/native-macos/lipo-same-cpu/shortcut-droplet.txt @@ -0,0 +1,17 @@ +Fat header in: fixture +fat_magic 0xcafebabe +nfat_arch 2 +architecture arm64 + cputype CPU_TYPE_ARM64 + cpusubtype CPU_SUBTYPE_ARM64_ALL + capabilities + offset 16384 + size 52000 + align 2^14 (16384) +architecture arm64e.x1 + cputype CPU_TYPE_ARM64 + cpusubtype unknown arm64 cpusubtype + capabilities PTR_AUTH_VERSION USERSPACE 0 + offset 81920 + size 52000 + align 2^14 (16384) From c02fb0ea8055f98def2b12ad557aed0faefd3639 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 16:43:12 +0800 Subject: [PATCH 079/159] fix(native): rank Mach-O slices by parsed CPU subtype --- src/artifacts/MachOSliceArtifactReader.ts | 8 ++-- src/native/AppleMachoSelection.ts | 6 +-- src/native/MachoArchitectureSelection.ts | 44 +++++++++++++++---- src/native/NativeMachoInspection.test.ts | 8 +++- src/native/NativeMachoInspection.ts | 12 ++--- src/native/parsers/lipo.test.ts | 22 ++++++++++ src/native/parsers/lipo.ts | 10 +++-- .../native/nativeSelectedArchitecture.test.ts | 37 +++++++++++++--- 8 files changed, 118 insertions(+), 29 deletions(-) diff --git a/src/artifacts/MachOSliceArtifactReader.ts b/src/artifacts/MachOSliceArtifactReader.ts index 5928956ee..f567458c7 100644 --- a/src/artifacts/MachOSliceArtifactReader.ts +++ b/src/artifacts/MachOSliceArtifactReader.ts @@ -8,11 +8,11 @@ import { type NativeCommandRunner, } from "../native/CommandRunner.js"; import { - lipoSubtypeBits, parseLipoArchitectures, type LipoArchitecture, } from "../native/parsers/lipo.js"; import type { MachoSlice } from "../domain/apple/dylibResolution.js"; +import { machoCpuSubtypeBase } from "../domain/apple/machoCpuIdentity.js"; import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js"; import { readMachoImage, type ReadAt } from "./apple/MachoLoadCommandReader.js"; import { @@ -471,8 +471,10 @@ const lipoSubtypeMatches = ( ): boolean => { if (reported.cpu_subtype_code !== null) return reported.cpu_subtype_code === observedCpuSubtype; - const bits = lipoSubtypeBits(reported.cpu_subtype); - return bits === null || bits === (observedCpuSubtype & 0x00ffffff); + return ( + reported.cpu_subtype_base === null || + reported.cpu_subtype_base === machoCpuSubtypeBase(observedCpuSubtype) + ); }; /** diff --git a/src/native/AppleMachoSelection.ts b/src/native/AppleMachoSelection.ts index e63ec082d..c0c68068d 100644 --- a/src/native/AppleMachoSelection.ts +++ b/src/native/AppleMachoSelection.ts @@ -1,9 +1,9 @@ import type { FixupCommands } from "./AppleMachoFixups.js"; import { MACHO_ARCHITECTURE_POLICIES, - baseMachoCpuSubtype, machoSubtypeRank, } from "./MachoArchitectureSelection.js"; +import { machoCpuSubtypeBase } from "../domain/apple/machoCpuIdentity.js"; import { readFatSliceDeclarations, validateFatSlice, @@ -117,8 +117,8 @@ export const selectMachoSlice = ( // Equal ranks keep FAT table order unless the slices share a subtype. if ( rank === bestRank && - baseMachoCpuSubtype(declaration.cpuSubtype) === - baseMachoCpuSubtype(best.cpuSubtype) + machoCpuSubtypeBase(declaration.cpuSubtype) === + machoCpuSubtypeBase(best.cpuSubtype) ) ambiguous = true; continue; diff --git a/src/native/MachoArchitectureSelection.ts b/src/native/MachoArchitectureSelection.ts index b6e6dcec4..14c5f696f 100644 --- a/src/native/MachoArchitectureSelection.ts +++ b/src/native/MachoArchitectureSelection.ts @@ -1,3 +1,6 @@ +import { machoCpuSubtypeBase } from "../domain/apple/machoCpuIdentity.js"; +import type { LipoArchitecture } from "./parsers/lipo.js"; + interface MachoArchitecturePolicy { readonly cpuType: number; readonly subtypes: readonly { @@ -28,17 +31,13 @@ export const MACHO_ARCHITECTURE_POLICIES: Readonly< }, }; -/** High subtype bits carry capabilities, such as the arm64e PAC ABI version. */ -export const baseMachoCpuSubtype = (subtype: number): number => - subtype & 0x00ffffff; - /** Unlisted subtypes follow the preferred slices in FAT table order. */ export const machoSubtypeRank = ( policy: MachoArchitecturePolicy, subtype: number, ): number => { const rank = policy.subtypes.findIndex( - (candidate) => candidate.subtype === baseMachoCpuSubtype(subtype), + (candidate) => candidate.subtype === machoCpuSubtypeBase(subtype), ); return rank < 0 ? policy.subtypes.length : rank; }; @@ -46,9 +45,36 @@ export const machoSubtypeRank = ( /** Select a tool spelling using the same preference order as FAT selection. */ export const selectMachoArchitectureName = ( policy: MachoArchitecturePolicy, - available: readonly string[], + available: readonly LipoArchitecture[], ): string | undefined => { - for (const { name } of policy.subtypes) - if (available.includes(name)) return name; - return available.find((name) => policy.variantName?.test(name)); + let selected: string | undefined; + let bestRank = Number.POSITIVE_INFINITY; + for (const architecture of available) { + if ( + architecture.cpu_type_code !== null && + architecture.cpu_type_code !== policy.cpuType + ) + continue; + const namedSubtype = policy.subtypes.find( + ({ name }) => name === architecture.name, + ); + const recognizedName = + namedSubtype !== undefined || policy.variantName?.test(architecture.name); + if ( + !recognizedName && + (architecture.cpu_type_code === null || + architecture.cpu_subtype_base === null) + ) + continue; + // Parsed identity is authoritative. Names support thin/partial lipo records. + const subtype = architecture.cpu_subtype_base ?? namedSubtype?.subtype; + const rank = + subtype === undefined + ? policy.subtypes.length + : machoSubtypeRank(policy, subtype); + if (rank >= bestRank) continue; + selected = architecture.name; + bestRank = rank; + } + return selected; }; diff --git a/src/native/NativeMachoInspection.test.ts b/src/native/NativeMachoInspection.test.ts index 7ddff2491..640e540be 100644 --- a/src/native/NativeMachoInspection.test.ts +++ b/src/native/NativeMachoInspection.test.ts @@ -94,6 +94,8 @@ describe("Mach-O tool architecture selection", () => { { names: ["arm64v8"], selected: "arm64v8" }, { names: ["arm64v8", "arm64e.x1"], selected: "arm64v8" }, { names: ["arm64e.x1", "arm64v8"], selected: "arm64v8" }, + { names: ["arm64v8", "arm64e.v1"], selected: "arm64e.v1" }, + { names: ["arm64e.v1", "arm64v8"], selected: "arm64e.v1" }, ] as const)( "keeps inspection and dispatch on $selected for FAT order $names", async ({ names, selected }) => { @@ -102,6 +104,7 @@ describe("Mach-O tool architecture selection", () => { arm64v8: { type: CPU.arm64.type, subtype: 1 }, arm64e: CPU.arm64e, "arm64e.x1": { type: CPU.arm64.type, subtype: 0x8000000c }, + "arm64e.v1": { type: CPU.arm64.type, subtype: 0x81000002 }, }; const slices = names.map((name) => { const cpu = cpus[name]; @@ -125,9 +128,12 @@ describe("Mach-O tool architecture selection", () => { "-h", "-l", "-arch", - selected, + selected.split(".")[0], "/fixture/app", ]); + expect(commands.find(({ tool }) => tool === "dyld_info")?.args).toContain( + selected, + ); expect(selectMachoSlice(fat, "arm64").slice).toBe( (names.findIndex((name) => name === selected) + 1) * 4096, ); diff --git a/src/native/NativeMachoInspection.ts b/src/native/NativeMachoInspection.ts index 836c3de1e..5fca089cb 100644 --- a/src/native/NativeMachoInspection.ts +++ b/src/native/NativeMachoInspection.ts @@ -14,7 +14,10 @@ import { selectMachoArchitectureName, } from "./MachoArchitectureSelection.js"; import { parseDyldSymbols } from "./parsers/dyldInfo.js"; -import { parseLipoArchitectures } from "./parsers/lipo.js"; +import { + parseLipoArchitectures, + type LipoArchitecture, +} from "./parsers/lipo.js"; import { withoutEchoedPathHeader } from "./parsers/echoedPath.js"; import { parseOtoolLoadCommands } from "./parsers/otool.js"; import { reconcileMachoUuid } from "./parsers/machoUuid.js"; @@ -72,8 +75,7 @@ export const inspectNativeMacho = async ( const lipo = captures.find(({ tool }) => tool === "lipo"); if (lipo === undefined) throw new TypeError("Missing lipo capture"); const architectures = parseLipoArchitectures(lipo.stdout); - const availableArchitectures = architectures.map(({ name }) => name); - const selection = selectArchitecture(context.target, availableArchitectures); + const selection = selectArchitecture(context.target, architectures); if (selection.status === "absent") return err( new AnalysisOutputError( @@ -401,7 +403,7 @@ type ArchitectureSelection = const selectArchitecture = ( target: BinaryTarget, - available: readonly string[], + available: readonly LipoArchitecture[], ): ArchitectureSelection => { // BinaryTarget lists CPU families and lipo lists slices. A universal file // can hold several slices of one family (arm64e and arm64e.x1), and slice @@ -416,7 +418,7 @@ const selectArchitecture = ( const policy = MACHO_ARCHITECTURE_POLICIES[target.architecture]; const selected = policy === undefined - ? available.find((name) => name === normalized) + ? available.find(({ name }) => name === normalized)?.name : selectMachoArchitectureName(policy, available); if (selected !== undefined) return { status: "selected", architecture: selected }; diff --git a/src/native/parsers/lipo.test.ts b/src/native/parsers/lipo.test.ts index 43f08d8b8..96f2afbbc 100644 --- a/src/native/parsers/lipo.test.ts +++ b/src/native/parsers/lipo.test.ts @@ -59,6 +59,7 @@ architecture arm64e cpu_type_code: null, cpu_subtype: "CPU_SUBTYPE_FUTURE", cpu_subtype_code: null, + cpu_subtype_base: null, }, ]); }); @@ -85,8 +86,29 @@ architecture arm64e { cpu_subtype: "CPU_SUBTYPE_X86_ALL", cpu_subtype_code: null, + cpu_subtype_base: 3, capabilities: null, }, ]); }); + + it.each([ + { capabilities: "PTR_AUTH_VERSION USERSPACE 1", code: 0x81000002 }, + { capabilities: "", code: null }, + ])( + "retains an arm64e base subtype with capabilities '$capabilities'", + ({ capabilities, code }) => { + expect( + parseLipoArchitectures( + `architecture arm64e.v1\n cputype CPU_TYPE_ARM64\n cpusubtype CPU_SUBTYPE_ARM64E\n capabilities ${capabilities}\n`, + ), + ).toMatchObject([ + { + cpu_type_code: 0x0100000c, + cpu_subtype_code: code, + cpu_subtype_base: 2, + }, + ]); + }, + ); }); diff --git a/src/native/parsers/lipo.ts b/src/native/parsers/lipo.ts index 4f56d8b6b..cc65cca41 100644 --- a/src/native/parsers/lipo.ts +++ b/src/native/parsers/lipo.ts @@ -1,4 +1,5 @@ import { z } from "zod"; +import { machoCpuSubtypeBase } from "../../domain/apple/machoCpuIdentity.js"; const architectureSchema = z.object({ name: z.string().min(1), @@ -6,6 +7,7 @@ const architectureSchema = z.object({ cpu_subtype: z.string().nullable(), cpu_type_code: z.number().int().nonnegative().nullable(), cpu_subtype_code: z.number().int().nonnegative().nullable(), + cpu_subtype_base: z.number().int().nonnegative().nullable(), capabilities: z.string().nullable(), file_offset: z.number().int().min(0).nullable(), size: z.number().int().min(0).nullable(), @@ -30,6 +32,7 @@ export const parseLipoArchitectures = (output: string): LipoArchitecture[] => { cpu_subtype: null, cpu_type_code: null, cpu_subtype_code: null, + cpu_subtype_base: null, capabilities: null, file_offset: null, size: null, @@ -54,6 +57,7 @@ export const parseLipoArchitectures = (output: string): LipoArchitecture[] => { current.cpusubtype, current.capabilities, ), + cpu_subtype_base: cpuSubtypeBase(current.cpusubtype), capabilities: current.capabilities ?? null, file_offset: integer(current.offset), size: integer(current.size), @@ -158,10 +162,10 @@ const cpuSubtypeCode = ( }; /** The subtype bits a lipo record names, without its capability byte. */ -export const lipoSubtypeBits = (value: string | null): number | null => { - if (value === null) return null; +const cpuSubtypeBase = (value: string | undefined): number | null => { + if (value === undefined) return null; const subtype = unsigned32(value) ?? CPU_SUBTYPES[value]; - return subtype === undefined ? null : subtype & 0x00ffffff; + return subtype === undefined ? null : machoCpuSubtypeBase(subtype); }; /** lipo names the LIB64 capability and prints other capability bits in hex. */ diff --git a/tests/boundary/providers/native/nativeSelectedArchitecture.test.ts b/tests/boundary/providers/native/nativeSelectedArchitecture.test.ts index 90b598f5d..a651e22f6 100644 --- a/tests/boundary/providers/native/nativeSelectedArchitecture.test.ts +++ b/tests/boundary/providers/native/nativeSelectedArchitecture.test.ts @@ -1,4 +1,5 @@ import { expect, it } from "vitest"; +import { CPU } from "../../../../src/artifacts/apple/MachoImage.fixture.js"; import { inspectMachoSchema } from "../../../../src/domain/native/nativeInspection.js"; import { err } from "../../../../src/domain/result.js"; import { @@ -90,18 +91,39 @@ it.each([ selected: "arm64e", dyldArchitecture: "arm64e.v1", }, -])( + { + architectures: ["arm64v8", "arm64e.v1"], + selected: "arm64e", + dyldArchitecture: "arm64e.v1", + }, + { + architectures: ["arm64e.v1", "arm64v8"], + selected: "arm64e", + dyldArchitecture: "arm64e.v1", + }, +] as const)( "selects the $dyldArchitecture slice of a universal arm64e target", async ({ architectures, selected, dyldArchitecture }) => { const calls: { readonly tool: string; readonly arguments_: readonly string[]; }[] = []; + const cpus = { + x86_64: CPU.x86_64, + arm64e: CPU.arm64e, + arm64v8: { type: CPU.arm64.type, subtype: 1 }, + "arm64e.x1": { type: CPU.arm64.type, subtype: 0x8000000c }, + "arm64e.v1": { type: CPU.arm64.type, subtype: 0x81000002 }, + }; const lipo = architectures - .map( - (architecture, index) => - `architecture ${architecture}\n cputype 16777228\n cpusubtype 2\n offset ${index * 16384}\n size 8192\n align 2^14 (16384)`, - ) + .map((architecture, index) => { + const cpu = cpus[architecture]; + const subtype = + architecture === "arm64e.v1" + ? "CPU_SUBTYPE_ARM64E\n capabilities PTR_AUTH_VERSION USERSPACE 1" + : String(cpu.subtype); + return `architecture ${architecture}\n cputype ${cpu.type}\n cpusubtype ${subtype}\n offset ${index * 16384}\n size 8192\n align 2^14 (16384)`; + }) .join("\n"); const fixtures = new NativeFixtureRunner({ lipo, @@ -118,6 +140,11 @@ it.each([ if (!result.ok) throw result.error; expect(result.value.result).toMatchObject({ uuid: "01234567-89AB-CDEF-0123-456789ABCDEF", + selected_architecture: { + name: dyldArchitecture, + container_file_offset: + architectures.findIndex((name) => name === dyldArchitecture) * 16384, + }, architectures: { items: architectures.map((name) => ({ name })), }, From 0c7f4dfbd9c1975eeedfecd96debb2e4263012d0 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 17:09:53 +0800 Subject: [PATCH 080/159] ci: separate fast feedback from full qualification Scope main pushes from a successful ancestor so canceled runs cannot hide changes. Keep the complete portable suite in fast feedback and move coverage plus Intel macOS to hourly, manual, explicit-full, and release qualification. Let qualification finish independently of superseded feedback. Bind publication to full qualification at its exact revision and publish the verified tarball after checking workflow identity, version, and SHA-256. Keep trusted cache writers and reject incomplete merged test inventories. Validation: 151 focused CI tests; typecheck/lint; actionlint 1.7.7; knip; all 1044 files discovered; bounded blob/merge smoke; macOS package verifier with retention and cleanup; npm pack dry run. Native Linux/ARM/Windows and remote workflow execution remain CI verification. --- .github/workflows/ci.yml | 93 +++++-- .github/workflows/release.yml | 35 ++- docs/releasing.md | 15 +- docs/testing.md | 83 ++++--- scripts/ci/baseline.mjs | 76 ++++++ scripts/ci/plan.mjs | 195 ++++++++------- scripts/ci/test-report.mjs | 21 ++ scripts/ci/verified-package.mjs | 235 ++++++++++++++++++ scripts/verify-package.mjs | 46 +++- .../boundary/filesystem/ciChangeScope.test.ts | 165 +++++++++++- .../filesystem/ciRequiredLanes.test.ts | 70 ++++++ .../boundary/filesystem/ciTestReports.test.ts | 47 +++- .../filesystem/ciVerifiedPackage.test.ts | 137 ++++++++++ .../filesystem/releaseWorkflow.test.ts | 25 +- 14 files changed, 1088 insertions(+), 155 deletions(-) create mode 100644 scripts/ci/baseline.mjs create mode 100644 scripts/ci/verified-package.mjs create mode 100644 tests/boundary/filesystem/ciVerifiedPackage.test.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6e6521b41..fe79048ba 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -5,16 +5,25 @@ on: branches: [main] pull_request: branches: [main, "release/*"] + schedule: + - cron: "23 * * * *" workflow_dispatch: + workflow_call: + inputs: + source_sha: + description: Exact release source; must match the caller revision. + required: true + type: string permissions: contents: read concurrency: - group: ci-${{ github.event.pull_request.number || github.ref }} - # The latest main push verifies the full tree, including earlier changes. - # Publication uses its own release workflow and concurrency group. - cancel-in-progress: true + # Complete qualification finishes even while main receives more pushes. + # Scheduled/manual main runs coalesce to one active and the latest pending; + # release qualification has its own exact-revision group. + group: ci-${{ (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || inputs.source_sha != '' || contains(github.event.pull_request.labels.*.name, 'ci:full')) && 'qualification' || 'feedback' }}-${{ inputs.source_sha || github.event.pull_request.number || github.ref }} + cancel-in-progress: ${{ github.event_name != 'schedule' && github.event_name != 'workflow_dispatch' && inputs.source_sha == '' && !contains(github.event.pull_request.labels.*.name, 'ci:full') }} env: HUSKY: "0" @@ -22,7 +31,7 @@ env: jobs: pr-title: name: Conventional PR title - if: github.event_name == 'pull_request' + if: github.event_name == 'pull_request' && inputs.source_sha == '' runs-on: ubuntu-latest steps: - name: Validate title for Release Please @@ -39,7 +48,11 @@ jobs: changes: name: Classify changes runs-on: ubuntu-latest + permissions: + contents: read + actions: read outputs: + tier: ${{ steps.plan.outputs.tier }} code: ${{ steps.plan.outputs.code }} runtime: ${{ steps.plan.outputs.runtime }} package: ${{ steps.plan.outputs.package }} @@ -62,16 +75,34 @@ jobs: steps: - uses: actions/checkout@v4 with: + ref: ${{ inputs.source_sha || github.sha }} fetch-depth: 0 - uses: actions/setup-node@v4 with: node-version-file: .nvmrc + - name: Bind release qualification to the caller revision + if: inputs.source_sha != '' + env: + SOURCE_SHA: ${{ inputs.source_sha }} + run: | + test "${SOURCE_SHA}" = "${GITHUB_SHA}" + test "${SOURCE_SHA}" = "$(git rev-parse HEAD)" + - name: Find a verified ancestor for push selection + id: baseline + if: github.event_name == 'push' && inputs.source_sha == '' + env: + GH_TOKEN: ${{ github.token }} + HEAD_SHA: ${{ github.sha }} + CI_REPOSITORY: ${{ github.repository }} + CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} + run: node scripts/ci/baseline.mjs - name: Select verification lanes id: plan env: - BASE_SHA: ${{ github.event.pull_request.base.sha }} - HEAD_SHA: ${{ github.event.pull_request.head.sha }} - CI_FULL: ${{ contains(github.event.pull_request.labels.*.name, 'ci:full') }} + BASE_SHA: ${{ steps.baseline.outputs.base || github.event.pull_request.base.sha }} + BASELINE_REASON: ${{ steps.baseline.outputs.reason }} + HEAD_SHA: ${{ inputs.source_sha || github.event.pull_request.head.sha || github.sha }} + CI_FULL: ${{ inputs.source_sha != '' || contains(github.event.pull_request.labels.*.name, 'ci:full') }} run: node scripts/ci/plan.mjs static: @@ -129,7 +160,14 @@ jobs: with: name: rea-test-runtime path: . - - run: npm run test:ci:shard:run -- --shard=${{ matrix.shard }} + - name: Run complete portable suite with coverage + if: needs.changes.outputs.tier == 'full' + run: npm run test:ci:shard:run -- --shard=${{ matrix.shard }} + env: + REA_CI_TEST_HISTORY: .cache/ci-test-history/test-results.json + - name: Run complete portable suite for fast feedback + if: needs.changes.outputs.tier == 'fast' + run: npm run test:run -- --reporter=default --reporter=blob --shard=${{ matrix.shard }} env: REA_CI_TEST_HISTORY: .cache/ci-test-history/test-results.json - uses: actions/upload-artifact@v4 @@ -160,7 +198,7 @@ jobs: - name: Reuse portable build outputs across runs uses: ./.github/actions/portable-build-cache with: - save: "true" + save: ${{ github.event_name != 'pull_request' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }} - run: npm run test:prepare - name: Find successful default-branch timing history id: history @@ -172,7 +210,7 @@ jobs: run: | run_id="$(gh api --method GET \ "repos/${CI_REPOSITORY}/actions/workflows/ci.yml/runs" \ - -f branch="${CI_DEFAULT_BRANCH}" -f event=push -f status=success -f per_page=1 \ + -f branch="${CI_DEFAULT_BRANCH}" -f event=schedule -f status=success -f per_page=1 \ --jq '.workflow_runs[0].id // empty')" echo "run-id=${run_id}" >> "${GITHUB_OUTPUT}" - name: Download previous timing report @@ -215,7 +253,7 @@ jobs: uses: ./.github/workflows/sqlite-runtime.yml test: - name: Test (coverage + timing) + name: Portable tests needs: [changes, test-shard] if: ${{ !cancelled() }} runs-on: ubuntu-latest @@ -241,7 +279,7 @@ jobs: merge-multiple: true - name: Merge coverage, JUnit, and timing reports id: merge - if: needs.changes.outputs.code == 'true' && success() + if: needs.changes.outputs.code == 'true' && needs.changes.outputs.tier == 'full' && success() run: >- npm run test:ci:merge -- --coverage.reportsDirectory=coverage @@ -253,11 +291,22 @@ jobs: --reporter=json --outputFile.junit=test-results.xml --outputFile.json=test-results.json + - name: Merge fast-suite JUnit and timing reports + id: merge-fast + if: needs.changes.outputs.code == 'true' && needs.changes.outputs.tier == 'fast' && success() + run: >- + npm exec -- vitest --merge-reports + --reporter=default + --reporter=junit + --reporter=json + --outputFile.junit=test-results.xml + --outputFile.json=test-results.json - name: Summarize slowest test files if: needs.changes.outputs.code == 'true' && !cancelled() && hashFiles('test-results.json') != '' env: SHARD_RESULT: ${{ needs.test-shard.result }} - MERGE_RESULT: ${{ steps.merge.outcome }} + MERGE_RESULT: ${{ needs.changes.outputs.tier == 'full' && steps.merge.outcome || steps.merge-fast.outcome }} + CI_VERIFY_TEST_INVENTORY: "true" run: node scripts/ci/test-report.mjs - uses: actions/upload-artifact@v4 if: needs.changes.outputs.code == 'true' && !cancelled() && hashFiles('test-results.json') != '' @@ -270,7 +319,7 @@ jobs: if-no-files-found: error retention-days: 30 - name: Publish successful timing history - if: needs.changes.outputs.code == 'true' && success() && needs.test-shard.result == 'success' + if: needs.changes.outputs.code == 'true' && needs.changes.outputs.tier == 'full' && success() && needs.test-shard.result == 'success' uses: actions/upload-artifact@v4 with: name: test-timings @@ -381,9 +430,20 @@ jobs: - run: npm run verify:package env: npm_config_cache: ${{ steps.npm-cache.outputs.path }} + REA_VERIFIED_PACKAGE_DIRECTORY: ${{ needs.changes.outputs.tier == 'full' && matrix.platform == 'linux' && matrix.architecture == 'x64' && '.cache/verified-package' || '' }} - name: Verify real Node Inspector discovery and observation timeout-minutes: 10 run: npm run verify:inspector + - name: Retain the exact qualified release package + if: needs.changes.outputs.tier == 'full' && matrix.platform == 'linux' && matrix.architecture == 'x64' + uses: actions/upload-artifact@v4 + with: + name: verified-release-package + path: .cache/verified-package/ + include-hidden-files: true + if-no-files-found: error + overwrite: true + retention-days: 7 apple-artifacts: name: Real Apple artifact CLI/MCP E2E @@ -657,7 +717,7 @@ jobs: required: name: CI required - if: ${{ !cancelled() }} + if: ${{ always() }} needs: [ changes, @@ -687,6 +747,7 @@ jobs: website, ] runs-on: ubuntu-latest + timeout-minutes: 5 steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index c60f32a2c..7cc798cfa 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -290,13 +290,25 @@ jobs: if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true' run: npm run docs:check && git diff --exit-code - publish: - name: Publish to npm + qualification: + name: Qualify exact release revision needs: [merged-release, release-please] if: >- always() && !cancelled() && ((needs.merged-release.result == 'success' && needs.merged-release.outputs.release_created == 'true') || (needs.release-please.result == 'success' && inputs.phase == 'publish' && needs.release-please.outputs.release_created == 'true')) + permissions: + contents: read + actions: read + uses: ./.github/workflows/ci.yml + with: + source_sha: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }} + + publish: + name: Publish to npm + needs: [merged-release, release-please, qualification] + if: >- + always() && !cancelled() && needs.qualification.result == 'success' runs-on: ubuntu-latest permissions: contents: read @@ -315,17 +327,20 @@ jobs: - run: npm ci - - name: Prepare Windows native build toolchain - run: sudo apt-get update && sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64 + - name: Download the exact qualified package + uses: actions/download-artifact@v4 + with: + name: verified-release-package + path: .cache/verified-package/ - - name: Build and verify bundled Windows native controls - run: npm run build:windows-native && npm run verify:windows-native:artifact - - - name: Verify packaged release before publishing - run: npm run verify:package + - name: Verify qualified package provenance and digest + id: package + run: node scripts/ci/verified-package.mjs validate .cache/verified-package - name: Publish shell: bash + env: + VERIFIED_PACKAGE_TARBALL: ${{ steps.package.outputs.tarball }} run: | version="$(node --input-type=module -e 'import packageJson from "./package.json" with { type: "json" }; process.stdout.write(packageJson.version)')" published_version="$(npm view "rea-agents@${version}" version --json 2>/dev/null || true)" @@ -333,7 +348,7 @@ jobs: echo "rea-agents@${version} is already published; verifying the existing package." else tag="$(node scripts/release-npm-tag.mjs "${version}")" - npm publish --access public --tag "${tag}" + npm publish "${VERIFIED_PACKAGE_TARBALL}" --access public --tag "${tag}" fi - name: Verify published CLI from npm diff --git a/docs/releasing.md b/docs/releasing.md index dd3a98be3..c532d0663 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -6,7 +6,10 @@ checks to pass. Merging that PR into main starts publication automatically. The Release workflow validates the merged release metadata and ancestry before creating a tag. npm and MCP Registry publication use the exact reviewed merge -SHA returned by Release Please. Later main commits cannot change that source. +SHA returned by Release Please. Before either registry is published, the release +run calls full CI qualification at that exact revision, including coverage, +portable providers, and every native package host. Later main commits cannot +change or cancel that proof. Ordinary main pushes and closed, unmerged PRs cannot publish packages. Check both publication jobs and the published-package canary; a merged PR or GitHub tag alone does not establish that publication finished. Retry failed jobs in @@ -196,8 +199,14 @@ both registry publishes. See [npm's provenance implementation](https://github.com/npm/cli/blob/v11.16.0/workspaces/libnpmpublish/lib/provenance.js) for the use of GitHub's workflow ref and commit SHA. -The workflow builds the bundled Windows controls and verifies the packaged -artifact before npm publication. It then verifies the published CLI, the +Full qualification builds the bundled Windows controls and verifies the packaged +artifact on its native hosts. The Linux x64 verifier retains the exact tarball +it installed and tested, after its workspace cleanup completes. The npm job +downloads that archive from the same workflow run and checks its source SHA, +run identity/attempt, package and Node versions, host identity, and SHA-256. +It publishes that archive without rebuilding or repacking it. Re-running failed +publisher jobs can reuse a prior successful attempt from the same run. +The workflow then verifies the published CLI, the capability-scoped MCP catalog, and the isolated package update path before publishing MCP Registry metadata. diff --git a/docs/testing.md b/docs/testing.md index a36eff6ec..b1d0031f6 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -263,12 +263,13 @@ CI owns coverage; aggregate and domain/contract thresholds are maintained in Coverage does not replace named boundary or real-provider scenarios. CI runs six native Vitest shards without retries. Each shard emits a blob -report; the merge job produces aggregate coverage plus JUnit and JSON timing +report; the merge job produces JUnit and JSON timing reports, plus aggregate +coverage during full qualification. It retains available reports even when a shard fails, uploads available diagnostics, and writes the slowest files to the workflow summary. Partial reports are marked diagnostic only. Failed, cancelled, or missing selected shards still fail the required gate, -and aggregate coverage thresholds remain enforced by the merge command. -Only successful complete merges publish the small `test-timings` artifact, with +and aggregate coverage thresholds remain enforced in full qualification. +Only successful complete full-tier merges publish the small `test-timings` artifact, with checkout-relative paths; the build job downloads that artifact for the sequencer instead of the coverage/JUnit bundle. Missing history falls back to native sharding. Static checks, documentation, build, package verification, and @@ -717,7 +718,7 @@ or the GitHub issue and pull-request templates run formatting and generated-document validation, including the small onboarding example consistency suite, without the source-test shards or native package lanes. Classification compares the PR head with its merge base, so later -base-branch changes do not expand that scope. Static and coverage aggregate jobs +base-branch changes do not expand that scope. Static and portable-test aggregate jobs remain present and fail if classification fails. Every code change retains the complete six-shard Linux suite; individual test files are not selected by imports or filenames. @@ -746,38 +747,58 @@ verification stays manual. Renames account for both paths. This narrower selection trades PR-time coverage for runner capacity. A change to unowned shared runtime code that fails only on arm64, macOS, Windows, the Apple artifact lane, or a provider lane that does not own the file is not caught on -the PR; the main-push full baseline reports it after merge. Ownership globs do +the PR; scheduled full qualification reports it after merge. Ownership globs do not follow imports. Use the `ci:full` label when a change may have native-host or provider effects outside its owners. -Main pushes and manual dispatch retain the full baseline, including all portable -provider lanes and the native matrix below. A `ci:full` PR label selects the full -baseline on the next PR run; manual dispatch can request it immediately. Each +Ordinary main pushes use the same owner scopes as PRs. Their diff starts at a +successful ancestor push run of this CI workflow, rather than the preceding +push: changes from failed or cancelled intermediate runs remain in scope. +Missing, untrusted, or non-ancestor history selects every lane. The lookup is +bounded to the last 100 successful push runs; lookup failures broaden selection. +Every code change still runs all six portable shards, without coverage +instrumentation in this fast tier. This deliberately avoids relying on an +import graph for generated files, filesystem inputs, subprocesses, or bridges. + +Full qualification runs hourly at minute 23, on manual dispatch, and for the +exact release revision before publication. It includes aggregate coverage, +every portable provider lane, and all native hosts below. A `ci:full` PR label +selects full qualification on the next PR run; manual dispatch can request it +immediately. Fast-tier broad fallbacks retain all owner lanes and the three-host +package matrix; Intel macOS and coverage remain full-tier obligations. Each classification job records its selection and broad-fallback reasons in the run summary. Representative Git-diff cases live in `tests/boundary/filesystem/ciChangeScope.test.ts`. -`CI required` checks classification and every selected lane, rejecting failed, -cancelled, or unexpectedly skipped work. Existing required check names are -preserved; changing branch protection to require the new aggregate is a separate -repository-settings operation. Newer commits cancel superseded CI for the same -PR or main branch. Release publication has its own concurrency group and retains -its package verification and public-registry canary. Licensed/self-hosted +`CI required` always runs and checks classification and every selected lane, +rejecting failed, cancelled, or unexpectedly skipped work. Branch protection +should require this aggregate, `Static checks`, and `Portable tests`. Fast runs +cancel superseded feedback for the same PR or main branch. Scheduled/manual +qualification has a separate concurrency group: active work finishes and only +the latest pending run is retained. Release qualification is grouped by its +exact source SHA and cannot be cancelled by newer main pushes. Publication +requires that qualification to succeed and consumes its exact verified tarball, +checking source SHA, workflow run, run attempt, package/Node version and SHA-256 +before npm publication. Earlier attempts from the same workflow run are accepted +when rerunning failed publisher jobs; future attempts and other runs are rejected. +The public-registry canary still verifies the published package. Licensed/self-hosted provider workflows and the Android smoke test retain their explicit triggers. CI exercises the pinned Node.js runtime on native hosted runners: -| Host | Runner | Baseline checks | -| --------------------- | ------------------ | ---------------------------------------------------------------------------------------- | -| Linux x64 | `ubuntu-latest` | Installed package and real Node Inspector CLI/MCP | -| Linux arm64 (aarch64) | `ubuntu-24.04-arm` | Installed package and real Node Inspector CLI/MCP | -| macOS 15 arm64 | `macos-15` | Installed package and real Node Inspector CLI/MCP | -| macOS 15 x64 | `macos-15-intel` | Installed package and real Node Inspector CLI/MCP (main, dispatch and `ci:full` only) | -| Windows x64 | `windows-latest` | Curated capabilities, native controls, installed package and real Node Inspector CLI/MCP | +| Host | Runner | Baseline checks | +| --------------------- | ------------------ | ------------------------------------------------------------------------------------------- | +| Linux x64 | `ubuntu-latest` | Installed package and real Node Inspector CLI/MCP | +| Linux arm64 (aarch64) | `ubuntu-24.04-arm` | Installed package and real Node Inspector CLI/MCP | +| macOS 15 arm64 | `macos-15` | Installed package and real Node Inspector CLI/MCP | +| macOS 15 x64 | `macos-15-intel` | Installed package and real Node Inspector CLI/MCP (hourly, dispatch, release and `ci:full`) | +| Windows x64 | `windows-latest` | Curated capabilities, native controls, installed package and real Node Inspector CLI/MCP | -macOS x64 is the slowest package host. Pull requests without the `ci:full` label -omit it, so darwin x64 installed-package and Inspector regressions surface on -the main-push full baseline rather than on the PR. +macOS x64 is the slowest package host. Ordinary PRs and main pushes omit it, +so darwin x64 installed-package and Inspector regressions surface in full +qualification. Schedule admission can be delayed by GitHub; the hourly trigger +does not guarantee an hourly completed result. Release publication waits for +its own complete result at the exact released revision. Package lanes assert the actual Node platform/architecture before verification and record those values with the Node version. The package matrix runs up to @@ -803,17 +824,21 @@ The Linux source-test shards download a shared runtime and generated test metadata snapshot produced by the build job with `npm run test:prepare`. The snapshot includes `dist/`, generated skills, the MCP tool catalog, product catalog, and managed-conformance metadata. Each shard still installs the locked -dependencies and runs its assigned coverage-enabled tests; it no longer +dependencies and runs its assigned tests; full qualification enables coverage. It no longer regenerates the same metadata independently. The build job also retrieves the merged timing report from the last successful -default-branch push and includes that same report in each shard's snapshot. +default-branch scheduled qualification and includes that same report in each shard's snapshot. CI assigns discovered files by historical duration, with a deterministic fallback weight for new files, rather than dividing only by file count. Every -discovered file still belongs to exactly one shard; worker limits, project -ordering, coverage instrumentation, and aggregate thresholds remain unchanged. +discovered file still belongs to exactly one shard; worker limits and project +ordering remain unchanged. The merge job rejects successful reports with missing, +unexpected, or duplicate portable files. Coverage instrumentation and aggregate +thresholds remain enforced in full qualification. Missing or unusable history falls back to Vitest's default assignment. Only -timing data is reused; persistent compile caches remain disabled. +timing data is reused; live provider results are never cached. Turbo outputs are +restored by task hash, with trusted default-branch runs as cache writers and PRs +as restore-only consumers. npm caches hold downloads, not `node_modules`. These native baseline checks complement the Linux source-test shards and the separate Apple-artifact and real-provider lanes. Actual Hopper, Ghidra, IDA, diff --git a/scripts/ci/baseline.mjs b/scripts/ci/baseline.mjs new file mode 100644 index 000000000..171bd7dd3 --- /dev/null +++ b/scripts/ci/baseline.mjs @@ -0,0 +1,76 @@ +import { execFileSync } from "node:child_process"; +import { appendFileSync } from "node:fs"; + +const repository = process.env.CI_REPOSITORY; +const branch = process.env.CI_DEFAULT_BRANCH; +const head = process.env.HEAD_SHA; +const shaPattern = /^[0-9a-f]{40}$/i; + +function ancestor(base) { + try { + execFileSync("git", ["merge-base", "--is-ancestor", base, head], { + stdio: "ignore", + }); + return true; + } catch { + return false; + } +} + +function write(base, reason) { + const output = { base, reason }; + if (process.env.GITHUB_OUTPUT) { + appendFileSync( + process.env.GITHUB_OUTPUT, + `base=${base}\nreason=${reason.replace(/[\r\n]/g, " ")}\n`, + ); + } + console.log(JSON.stringify(output)); +} + +if (!repository || !branch || !head || !shaPattern.test(head)) { + write("", "Repository, default branch, or checked-out head is invalid"); +} else { + try { + const response = execFileSync( + "gh", + [ + "api", + "--method", + "GET", + `repos/${repository}/actions/workflows/ci.yml/runs`, + "-f", + `branch=${branch}`, + "-f", + "event=push", + "-f", + "status=success", + "-F", + "per_page=100", + ], + { encoding: "utf8", timeout: 15_000, maxBuffer: 4 * 1024 * 1024 }, + ); + const runs = JSON.parse(response).workflow_runs; + if (!Array.isArray(runs)) + throw new Error("GitHub returned no workflow run list"); + const base = runs.find( + (run) => + run?.event === "push" && + run?.conclusion === "success" && + run?.head_branch === branch && + run?.head_repository?.full_name === repository && + typeof run?.head_sha === "string" && + shaPattern.test(run.head_sha) && + run.head_sha.toLowerCase() !== head.toLowerCase() && + ancestor(run.head_sha), + )?.head_sha; + if (base) write(base, ""); + else + write( + "", + "No trusted successful CI push baseline is an ancestor of this head", + ); + } catch { + write("", "Could not read a successful CI push baseline from GitHub"); + } +} diff --git a/scripts/ci/plan.mjs b/scripts/ci/plan.mjs index 8b99efce5..7ffec2c85 100644 --- a/scripts/ci/plan.mjs +++ b/scripts/ci/plan.mjs @@ -108,98 +108,123 @@ function classifyPackage(base, head) { } const event = process.env.GITHUB_EVENT_NAME; -let files = []; -if ( - event === "push" || +if (!["pull_request", "push", "schedule", "workflow_dispatch"].includes(event)) + throw new Error(`Unsupported CI event: ${event}`); +const tier = event === "schedule" || event === "workflow_dispatch" || process.env.CI_FULL === "true" -) { + ? "full" + : "fast"; +let files = []; +if (tier === "full") { full("Full baseline requested by event or ci:full label"); -} else if (event === "pull_request") { - const base = git("merge-base", process.env.BASE_SHA, process.env.HEAD_SHA); +} else if (event === "pull_request" || event === "push") { const head = process.env.HEAD_SHA; - // Disable rename detection so both the removed and added paths participate. - files = git("diff", "--name-only", "--no-renames", "-z", base, head) - .split("\0") - .filter(Boolean); - for (const file of files) { - if ( - /^(README(?:_[^/]+)?\.md|AGENTS\.md|CONTRIBUTING\.md|CHANGELOG\.md|SECURITY\.md|native\/windows\/README\.md|\.github\/pull_request_template\.md)$/.test( - file, - ) || - file.startsWith("docs/") || - file.startsWith(".github/ISSUE_TEMPLATE/") - ) { - selected.docs = true; - selected.website ||= matchesAny(file, SCOPES.website); - continue; - } - if (/^(src|tests)\/.*\.(?:test|spec)\.[cm]?[jt]sx?$/.test(file)) { - selected.code = true; - for (const owner of NATIVE_TEST_OWNERS) - selected[owner] ||= matchesAny(file, SCOPES[owner]); - continue; - } - if (matchesAny(file, MANUAL_VERIFIERS)) { - selected.code = true; - continue; - } - const owners = Object.entries(SCOPES) - .filter(([, patterns]) => - patterns.some((pattern) => matchesGlob(file, pattern)), - ) - .map(([owner]) => owner); - for (const owner of owners) selected[owner] = true; - if ( - file.startsWith(".github/workflows/") || - file === ".github/actionlint.yaml" - ) { - selected.workflows = true; - if (GUARDED_WORKFLOWS.has(file)) selected.code = true; - if (file !== ".github/workflows/ci.yml") continue; - } - if (owners.length === 1 && owners[0] === "website") continue; - if (/^src\/(contracts|domain)\//.test(file)) selected.docs = true; - if (file === "package.json") { - classifyPackage(base, head); - continue; - } - selected.code = true; - if (matchesAny(file, FIXTURE_SOURCES)) { - selected.fixtures = true; - selected.readiness = true; - } - if (matchesAny(file, READINESS_SOURCES)) selected.readiness = true; - if (file.startsWith("tests/")) continue; - if ( - /^(src\/process\/|src\/filesystem\/|src\/contracts\/tool|src\/server\/createServer|src\/cli\.ts|scripts\/ci\/)/.test( - file, - ) - ) { - full(`Shared boundary: ${file}`); - } else if (owners.length) { - if (file.startsWith("src/") || file.startsWith("bridge/")) - selected.runtime = true; - } else if (/^(src|bridge)\//.test(file)) { - // Unowned runtime sources keep the Linux x64 installed-package baseline; - // ci:full, main pushes and manual dispatch retain every native host. - selected.runtime = true; - reasons.push(`Unowned runtime source: ${file}`); - } else if (file.startsWith(".agents/skills/")) { - selected.runtime = true; - selected.docs = true; - } else { - full(`Unclassified input: ${file}`); + let base; + if (event === "pull_request") { + base = git("merge-base", process.env.BASE_SHA, head); + } else { + const candidate = process.env.BASE_SHA; + const baselineReason = process.env.BASELINE_REASON; + try { + if (!candidate || !/^[0-9a-f]{40}$/i.test(candidate)) + throw new Error( + baselineReason || "No valid successful push baseline was found", + ); + execFileSync("git", ["merge-base", "--is-ancestor", candidate, head], { + stdio: "ignore", + }); + base = candidate; + } catch { + full( + baselineReason || + "No valid successful push baseline ancestor; selecting all lanes", + ); } } -} else throw new Error(`Unsupported CI event: ${event}`); + if (base) { + // Disable rename detection so both the removed and added paths participate. + files = git("diff", "--name-only", "--no-renames", "-z", base, head) + .split("\0") + .filter(Boolean); + for (const file of files) { + if ( + /^(README(?:_[^/]+)?\.md|AGENTS\.md|CONTRIBUTING\.md|CHANGELOG\.md|SECURITY\.md|native\/windows\/README\.md|\.github\/pull_request_template\.md)$/.test( + file, + ) || + file.startsWith("docs/") || + file.startsWith(".github/ISSUE_TEMPLATE/") + ) { + selected.docs = true; + selected.website ||= matchesAny(file, SCOPES.website); + continue; + } + if (/^(src|tests)\/.*\.(?:test|spec)\.[cm]?[jt]sx?$/.test(file)) { + selected.code = true; + for (const owner of NATIVE_TEST_OWNERS) + selected[owner] ||= matchesAny(file, SCOPES[owner]); + continue; + } + if (matchesAny(file, MANUAL_VERIFIERS)) { + selected.code = true; + continue; + } + const owners = Object.entries(SCOPES) + .filter(([, patterns]) => + patterns.some((pattern) => matchesGlob(file, pattern)), + ) + .map(([owner]) => owner); + for (const owner of owners) selected[owner] = true; + if ( + file.startsWith(".github/workflows/") || + file === ".github/actionlint.yaml" + ) { + selected.workflows = true; + if (GUARDED_WORKFLOWS.has(file)) selected.code = true; + if (file !== ".github/workflows/ci.yml") continue; + } + if (owners.length === 1 && owners[0] === "website") continue; + if (/^src\/(contracts|domain)\//.test(file)) selected.docs = true; + if (file === "package.json") { + classifyPackage(base, head); + continue; + } + selected.code = true; + if (matchesAny(file, FIXTURE_SOURCES)) { + selected.fixtures = true; + selected.readiness = true; + } + if (matchesAny(file, READINESS_SOURCES)) selected.readiness = true; + if (file.startsWith("tests/")) continue; + if ( + /^(src\/process\/|src\/filesystem\/|src\/contracts\/tool|src\/server\/createServer|src\/cli\.ts|scripts\/ci\/)/.test( + file, + ) + ) { + full(`Shared boundary: ${file}`); + } else if (owners.length) { + if (file.startsWith("src/") || file.startsWith("bridge/")) + selected.runtime = true; + } else if (/^(src|bridge)\//.test(file)) { + // Unowned runtime sources keep the Linux x64 installed-package baseline; + // full qualification retains every native host. + selected.runtime = true; + reasons.push(`Unowned runtime source: ${file}`); + } else if (file.startsWith(".agents/skills/")) { + selected.runtime = true; + selected.docs = true; + } else { + full(`Unclassified input: ${file}`); + } + } + } +} selected.runtime ||= selected.package; const linux = { os: "ubuntu-latest", platform: "linux", architecture: "x64" }; -// macOS x64 is the slowest package host. Pull requests run it only with ci:full; -// main pushes, schedules and manual dispatch always include it. +// macOS x64 is the slowest package host. Fast qualification omits it. const intel = - event !== "pull_request" || process.env.CI_FULL === "true" + tier === "full" ? [{ os: "macos-15-intel", platform: "darwin", architecture: "x64" }] : []; const matrix = { @@ -212,17 +237,17 @@ const matrix = { ] : [linux], }; -const plan = { ...selected, matrix, files, reasons }; +const plan = { ...selected, tier, matrix, files, reasons }; if (process.env.GITHUB_OUTPUT) appendFileSync( process.env.GITHUB_OUTPUT, - Object.entries({ ...selected, matrix: JSON.stringify(matrix) }) + Object.entries({ ...selected, tier, matrix: JSON.stringify(matrix) }) .map(([key, value]) => `${key}=${value}\n`) .join(""), ); if (process.env.GITHUB_STEP_SUMMARY) appendFileSync( process.env.GITHUB_STEP_SUMMARY, - `### Selected CI lanes\n\n${flags.filter((flag) => selected[flag]).join(", ") || "Formatting only"}\n\n${reasons.join("\n\n")}\n`, + `### Selected CI lanes (${tier})\n\n${flags.filter((flag) => selected[flag]).join(", ") || "Formatting only"}\n\n${reasons.join("\n\n")}\n`, ); console.log(JSON.stringify(plan)); diff --git a/scripts/ci/test-report.mjs b/scripts/ci/test-report.mjs index a3686d6ed..9f2d979aa 100644 --- a/scripts/ci/test-report.mjs +++ b/scripts/ci/test-report.mjs @@ -1,3 +1,4 @@ +import { execFileSync } from "node:child_process"; import { appendFile, mkdir, readFile, rm, writeFile } from "node:fs/promises"; import { isAbsolute, join, relative, resolve, sep } from "node:path"; @@ -42,6 +43,26 @@ const passed = process.env.MERGE_RESULT === "success" && report.success && report.numFailedTests === 0; +// Successful shards and merged reports must account for the complete portable +// inventory. Coverage thresholds alone cannot prove that every file ran. +if (passed && process.env.CI_VERIFY_TEST_INVENTORY === "true") { + const expected = execFileSync( + "git", + ["ls-files", "-z", "--", "src", "tests"], + { encoding: "utf8", maxBuffer: 8 * 1024 * 1024 }, + ) + .split("\0") + .filter((path) => path.endsWith(".test.ts")); + const actual = files.map(({ name }) => name); + const expectedSet = new Set(expected); + const actualSet = new Set(actual); + const missing = expected.filter((path) => !actualSet.has(path)); + const unexpected = actual.filter((path) => !expectedSet.has(path)); + if (missing.length || unexpected.length || actualSet.size !== actual.length) + throw new Error( + `Incomplete portable test inventory: ${JSON.stringify({ missing, unexpected, duplicateFiles: actual.length - actualSet.size })}`, + ); +} const summary = [ complete ? "## Merged test diagnostics" : "## Partial test diagnostics", "", diff --git a/scripts/ci/verified-package.mjs b/scripts/ci/verified-package.mjs new file mode 100644 index 000000000..1053760b2 --- /dev/null +++ b/scripts/ci/verified-package.mjs @@ -0,0 +1,235 @@ +import { createHash } from "node:crypto"; +import { createReadStream } from "node:fs"; +import { execFile } from "node:child_process"; +import { copyFile, readFile, writeFile } from "node:fs/promises"; +import { join, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { promisify } from "node:util"; + +const MANIFEST_NAME = "manifest.json"; +const TARBALL_NAME = "package.tgz"; +const MANIFEST_KEYS = [ + "schemaVersion", + "sourceSha", + "runId", + "runAttempt", + "platform", + "architecture", + "packageVersion", + "nodeVersion", + "sha256", +].sort(); +const execFileAsync = promisify(execFile); + +/** Normalize workflow identity and host facts into the manifest-owned fields. */ +export function createVerifiedPackageMetadata({ + sourceSha, + runId, + runAttempt, + platform, + architecture, + packageVersion, + nodeVersion, +}) { + if (!/^[a-f0-9]{40}$/u.test(sourceSha)) + throw new Error("Source SHA must be a full commit SHA"); + return { + sourceSha, + runId: parsePositiveInteger(runId, "GITHUB_RUN_ID"), + runAttempt: parsePositiveInteger(runAttempt, "GITHUB_RUN_ATTEMPT"), + platform, + architecture, + packageVersion, + nodeVersion, + }; +} + +/** Build the single supported manifest representation for a verified tarball. */ +function createVerifiedPackageManifest(metadata) { + const manifest = { schemaVersion: 1, ...metadata }; + validateManifestShape(manifest); + return manifest; +} + +/** Write a manifest after the caller has completed qualification and cleanup. */ +export async function writeVerifiedPackageManifest(directory, metadata) { + const manifest = createVerifiedPackageManifest(metadata); + await writeFile( + join(directory, MANIFEST_NAME), + `${JSON.stringify(manifest, null, 2)}\n`, + { encoding: "utf8" }, + ); + return manifest; +} + +/** Copy the qualified archive under its fixed name and prepare its manifest. */ +export async function retainVerifiedPackage({ + sourceTarball, + directory, + metadata, +}) { + const tarball = join(directory, TARBALL_NAME); + await copyFile(sourceTarball, tarball); + const sha256 = await sha256File(tarball); + return { + directory, + metadata: createVerifiedPackageManifest({ ...metadata, sha256 }), + }; +} + +/** Validate a retained package against the current checkout and workflow attempt. */ +async function validateVerifiedPackage({ + directory, + sourceSha, + runId, + runAttempt, + packageVersion, + nodeVersion, +}) { + if (!/^[a-f0-9]{40}$/u.test(sourceSha)) + throw new Error("Expected source SHA must be a full commit SHA"); + if (!Number.isSafeInteger(runId) || runId <= 0) + throw new Error("Expected workflow run ID must be a positive integer"); + if (!Number.isSafeInteger(runAttempt) || runAttempt <= 0) + throw new Error("Expected workflow run attempt must be a positive integer"); + const manifestPath = join(directory, MANIFEST_NAME); + let manifest; + try { + manifest = JSON.parse(await readFile(manifestPath, "utf8")); + } catch (cause) { + throw new Error(`Could not read verified package manifest: ${cause}`, { + cause, + }); + } + validateManifestShape(manifest); + assertEqual(manifest.sourceSha, sourceSha, "source SHA"); + assertEqual(manifest.runId, runId, "workflow run ID"); + // A successful producer may belong to an earlier attempt when only failed + // publisher jobs are rerun. The workflow still requires same-run qualification. + if (manifest.runAttempt > runAttempt) + throw new Error("Verified package workflow run attempt is from the future"); + assertEqual(manifest.platform, "linux", "verifier platform"); + assertEqual(manifest.architecture, "x64", "verifier architecture"); + assertEqual(manifest.packageVersion, packageVersion, "package version"); + assertEqual(manifest.nodeVersion, nodeVersion, "Node.js version"); + + // The artifact format deliberately fixes the name; no manifest value can + // redirect validation to another path. + const tarball = resolve(directory, TARBALL_NAME); + const digest = await sha256File(tarball); + assertEqual(manifest.sha256, digest, "tarball SHA-256"); + return tarball; +} + +function validateManifestShape(manifest) { + if ( + typeof manifest !== "object" || + manifest === null || + Array.isArray(manifest) + ) + throw new Error("Verified package manifest must be a JSON object"); + const keys = Object.keys(manifest).sort(); + if ( + keys.length !== MANIFEST_KEYS.length || + keys.some((key, index) => key !== MANIFEST_KEYS[index]) + ) + throw new Error("Verified package manifest has missing or unknown fields"); + if (manifest.schemaVersion !== 1) + throw new Error("Unsupported verified package manifest schema version"); + if ( + typeof manifest.sourceSha !== "string" || + !/^[a-f0-9]{40}$/u.test(manifest.sourceSha) + ) + throw new Error("Verified package manifest has an invalid source SHA"); + if (!Number.isSafeInteger(manifest.runId) || manifest.runId <= 0) + throw new Error("Verified package manifest has an invalid workflow run ID"); + if (!Number.isSafeInteger(manifest.runAttempt) || manifest.runAttempt <= 0) + throw new Error( + "Verified package manifest has an invalid workflow run attempt", + ); + for (const key of [ + "platform", + "architecture", + "packageVersion", + "nodeVersion", + ]) + if (typeof manifest[key] !== "string" || manifest[key].length === 0) + throw new Error(`Verified package manifest has an invalid ${key}`); + if ( + typeof manifest.sha256 !== "string" || + !/^[a-f0-9]{64}$/u.test(manifest.sha256) + ) + throw new Error("Verified package manifest has an invalid tarball SHA-256"); +} + +function assertEqual(actual, expected, label) { + if (actual !== expected) + throw new Error(`Verified package ${label} mismatch`); +} + +async function sha256File(path) { + const hash = createHash("sha256"); + for await (const chunk of createReadStream(path)) hash.update(chunk); + return hash.digest("hex"); +} + +async function validateFromCli(directory) { + const sourceSha = process.env.GITHUB_SHA; + const runId = parsePositiveInteger( + process.env.GITHUB_RUN_ID, + "GITHUB_RUN_ID", + ); + const runAttempt = parsePositiveInteger( + process.env.GITHUB_RUN_ATTEMPT, + "GITHUB_RUN_ATTEMPT", + ); + if (sourceSha === undefined || !/^[a-f0-9]{40}$/u.test(sourceSha)) + throw new Error("GITHUB_SHA must be a full commit SHA"); + const checkoutSha = ( + await execFileAsync("git", ["rev-parse", "HEAD"]) + ).stdout.trim(); + assertEqual(checkoutSha, sourceSha, "checked out source SHA"); + const packageJson = JSON.parse(await readFile("package.json", "utf8")); + const tarball = await validateVerifiedPackage({ + directory, + sourceSha, + runId, + runAttempt, + packageVersion: packageJson.version, + nodeVersion: process.version, + }); + if (process.env.GITHUB_OUTPUT === undefined) + throw new Error( + "GITHUB_OUTPUT is required to publish the validated tarball path", + ); + await writeFile(process.env.GITHUB_OUTPUT, `tarball=${tarball}\n`, { + encoding: "utf8", + flag: "a", + }); + process.stdout.write(`${tarball}\n`); +} + +function parsePositiveInteger(value, name) { + if (value === undefined || !/^[1-9]\d*$/u.test(value)) + throw new Error(`${name} must be a positive integer`); + const parsed = Number(value); + if (!Number.isSafeInteger(parsed)) + throw new Error(`${name} must be a safe positive integer`); + return parsed; +} + +const invokedPath = process.argv[1]; +if ( + invokedPath !== undefined && + resolve(invokedPath) === fileURLToPath(import.meta.url) +) { + const [command, directory, ...extra] = process.argv.slice(2); + if (command !== "validate" || directory === undefined || extra.length > 0) { + process.stderr.write( + "Usage: node scripts/ci/verified-package.mjs validate \n", + ); + process.exitCode = 2; + } else { + await validateFromCli(directory); + } +} diff --git a/scripts/verify-package.mjs b/scripts/verify-package.mjs index fff3a07c6..32bad0a62 100644 --- a/scripts/verify-package.mjs +++ b/scripts/verify-package.mjs @@ -1,6 +1,8 @@ -import { mkdtemp, rm } from "node:fs/promises"; +import { mkdir, mkdtemp, rm, unlink } from "node:fs/promises"; +import { execFile } from "node:child_process"; import { tmpdir } from "node:os"; import { join } from "node:path"; +import { promisify } from "node:util"; import { TOOL_CONTRACTS } from "../dist/contracts/toolContracts.js"; import { PRODUCT_IDENTITY } from "../dist/identity.js"; @@ -21,12 +23,19 @@ import { verifyRepositorySkill } from "./verify-repository-skill.mjs"; import { verifyManaged } from "./verify-package-managed.mjs"; import { verifyUnknownProvider } from "./verify-package-unknown-provider.mjs"; import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs"; +import { + createVerifiedPackageMetadata, + retainVerifiedPackage, + writeVerifiedPackageManifest, +} from "./ci/verified-package.mjs"; import { measureMcpStartup, profileMcpModuleLoading, } from "./lib/mcp-startup-probe.mjs"; const root = process.cwd(); +const execFileAsync = promisify(execFile); +const retentionDirectory = process.env.REA_VERIFIED_PACKAGE_DIRECTORY; const verifierRun = createVerifierRun(); const workspace = await mkdtemp(join(tmpdir(), "rea-package-")); const evidenceRoot = join(workspace, "evidence"); @@ -37,6 +46,8 @@ let mcpModuleLoading; let update; let repositorySkill; let report; +let retainedPackage; +let retentionMetadata; const runStage = async (name, action) => { const started = performance.now(); @@ -49,6 +60,27 @@ const runStage = async (name, action) => { }; try { + if (retentionDirectory !== undefined && retentionDirectory !== "") { + const sourceSha = ( + await execFileAsync("git", ["rev-parse", "HEAD"], { cwd: root }) + ).stdout.trim(); + retentionMetadata = createVerifiedPackageMetadata({ + sourceSha, + runId: process.env.GITHUB_RUN_ID, + runAttempt: process.env.GITHUB_RUN_ATTEMPT, + platform: process.platform, + architecture: process.arch, + packageVersion: PRODUCT_IDENTITY.packageVersion, + nodeVersion: process.version, + }); + await mkdir(retentionDirectory, { recursive: true }); + await unlink(join(retentionDirectory, "manifest.json")).catch((cause) => { + if ( + !(cause instanceof Error && "code" in cause && cause.code === "ENOENT") + ) + throw cause; + }); + } repositorySkill = await runStage("repository skill", () => verifyRepositorySkill({ root, workspace }), ); @@ -215,6 +247,13 @@ try { mcpStartup, mcpModuleLoading, }; + if (retentionMetadata !== undefined) { + retainedPackage = await retainVerifiedPackage({ + sourceTarball: tarball, + directory: retentionDirectory, + metadata: retentionMetadata, + }); + } } finally { await runStage("workspace cleanup", () => rm(workspace, { recursive: true, force: true }), @@ -223,6 +262,11 @@ try { const verifierLineage = await runStage("verifier lineage", () => completeVerifierRun(verifierRun), ); +if (retainedPackage !== undefined) + await writeVerifiedPackageManifest( + retainedPackage.directory, + retainedPackage.metadata, + ); process.stdout.write( `${JSON.stringify({ verifier_run: verifierLineage, ...report })}\n`, ); diff --git a/tests/boundary/filesystem/ciChangeScope.test.ts b/tests/boundary/filesystem/ciChangeScope.test.ts index 9176dec34..48fc19624 100644 --- a/tests/boundary/filesystem/ciChangeScope.test.ts +++ b/tests/boundary/filesystem/ciChangeScope.test.ts @@ -1,4 +1,5 @@ import { + chmod, cp, mkdir, readFile, @@ -161,13 +162,158 @@ it("retains complete checks when implementation is renamed into documentation", }); }); -it("retains the complete main-push baseline even for documentation", async () => { +it("keeps main pushes scoped when a successful ancestor baseline is available", async () => { const { directory, base } = await fixture(); const head = await change(directory, ["README_ja.md"]); expect(await classify(directory, base, head, "push")).toMatchObject({ + tier: "fast", + code: false, + package: false, + docs: true, + matrix: { + include: [ + { os: "ubuntu-latest", platform: "linux", architecture: "x64" }, + ], + }, + }); +}); + +it("includes changes from earlier cancelled pushes since the last successful baseline", async () => { + const { directory, base } = await fixture(); + await change(directory, ["src/evm/client.ts"]); + const head = await change(directory, ["README.md"]); + expect(await classify(directory, base, head, "push")).toMatchObject({ + tier: "fast", + code: true, + evm: true, + runtime: true, + package: false, + }); +}); + +it.each(["", "not-a-commit"])( + "fails closed to all lanes for a missing or invalid push baseline (%s)", + async (base) => { + const { directory } = await fixture(); + const head = await change(directory, ["README.md"]); + const plan = await classify(directory, base, head, "push"); + expect(plan).toMatchObject({ + tier: "fast", + code: true, + package: true, + evm: true, + browser: true, + reasons: expect.arrayContaining([expect.stringContaining("baseline")]), + }); + const { SCOPES } = await loadScopes(); + for (const flag of [ + "code", + "runtime", + "package", + "fixtures", + "readiness", + "docs", + "workflows", + ...Object.keys(SCOPES), + ]) + expect(plan[flag]).toBe(true); + }, +); + +it("selects only a trusted successful ancestor from recent CI push runs", async () => { + const { directory, base } = await fixture(); + const head = await change(directory, ["src/evm/client.ts"]); + const bin = join(directory, "fake-bin"); + await mkdir(bin); + const gh = join(bin, "gh"); + await writeFile(gh, '#!/bin/sh\ncat "$GH_FIXTURE_RESPONSE"\n'); + await chmod(gh, 0o755); + const response = join(directory, "runs.json"); + await writeFile( + response, + JSON.stringify({ + workflow_runs: [ + { + event: "push", + conclusion: "success", + head_branch: "main", + head_repository: { full_name: "untrusted/fork" }, + head_sha: base, + }, + { + event: "push", + conclusion: "success", + head_branch: "main", + head_repository: { full_name: "owner/repo" }, + head_sha: base, + }, + ], + }), + ); + const output = join(directory, "baseline-output"); + const result = await execFileAsync( + process.execPath, + [new URL("../../../scripts/ci/baseline.mjs", import.meta.url).pathname], + { + cwd: directory, + env: { + ...process.env, + PATH: `${bin}:${process.env.PATH}`, + GH_FIXTURE_RESPONSE: response, + CI_REPOSITORY: "owner/repo", + CI_DEFAULT_BRANCH: "main", + HEAD_SHA: head, + GITHUB_OUTPUT: output, + }, + }, + ); + expect(JSON.parse(result.stdout)).toEqual({ base, reason: "" }); + expect(await readFile(output, "utf8")).toBe(`base=${base}\nreason=\n`); +}); + +it("returns an empty baseline when GitHub lookup fails", async () => { + const { directory, base } = await fixture(); + const head = await change(directory, ["src/evm/client.ts"]); + const bin = join(directory, "fake-bin"); + await mkdir(bin); + const gh = join(bin, "gh"); + await writeFile(gh, "#!/bin/sh\nexit 1\n"); + await chmod(gh, 0o755); + const result = await execFileAsync( + process.execPath, + [new URL("../../../scripts/ci/baseline.mjs", import.meta.url).pathname], + { + cwd: directory, + env: { + ...process.env, + PATH: `${bin}:${process.env.PATH}`, + CI_REPOSITORY: "owner/repo", + CI_DEFAULT_BRANCH: "main", + HEAD_SHA: head, + }, + }, + ); + expect(JSON.parse(result.stdout)).toMatchObject({ + base: "", + reason: expect.stringContaining("Could not read"), + }); + expect(head).not.toBe(base); +}); + +it("fails closed when the successful push baseline is not an ancestor", async () => { + const { directory, base } = await fixture(); + await git(directory, ["switch", "-c", "unrelated"]); + const unrelated = await change(directory, ["other.txt"]); + await git(directory, ["switch", "main"]); + const head = await change(directory, ["README.md"]); + expect(await classify(directory, unrelated, head, "push")).toMatchObject({ + tier: "fast", code: true, package: true, + evm: true, + browser: true, }); + expect(head).not.toBe(base); }); it("fails classification rather than treating an invalid Git comparison as a scope", async () => { @@ -207,7 +353,8 @@ it("retains native Inspector verification on every documented package host", asy ); expect(packageInspector.if).toBeUndefined(); expect( - (await classify((await fixture()).directory, "", "", "push")).matrix, + (await classify((await fixture()).directory, "", "", "workflow_dispatch")) + .matrix, ).toEqual({ include: [ { os: "ubuntu-latest", platform: "linux", architecture: "x64" }, @@ -595,6 +742,7 @@ it.each(["workflow_dispatch", "schedule"])( async (event) => { const { directory, base } = await fixture(); expect(await classify(directory, base, base, event)).toMatchObject({ + tier: "full", code: true, package: true, evm: true, @@ -610,7 +758,13 @@ it("honors ci:full for successive PR revisions", async () => { for (const head of [first, second]) { expect( await classify(directory, base, head, "pull_request", true), - ).toMatchObject({ code: true, package: true, evm: true, browser: true }); + ).toMatchObject({ + tier: "full", + code: true, + package: true, + evm: true, + browser: true, + }); } }); @@ -650,7 +804,7 @@ const INTEL_HOST = { architecture: "x64", }; -it("keeps macOS x64 package verification for ci:full and non-PR events", async () => { +it("keeps macOS x64 package verification only for full qualification", async () => { const { directory, base } = await fixture(); const head = await change(directory, ["src/windows/Example.ts"]); const matrix = z.object({ include: z.array(z.unknown()) }); @@ -661,7 +815,8 @@ it("keeps macOS x64 package verification for ci:full and non-PR events", async ( expect(pullRequest).toHaveLength(3); expect(pullRequest).not.toContainEqual(INTEL_HOST); expect(await hosts("pull_request", true)).toContainEqual(INTEL_HOST); - for (const event of ["push", "schedule", "workflow_dispatch"]) + expect(await hosts("push", false)).not.toContainEqual(INTEL_HOST); + for (const event of ["schedule", "workflow_dispatch"]) expect(await hosts(event, false)).toContainEqual(INTEL_HOST); }); diff --git a/tests/boundary/filesystem/ciRequiredLanes.test.ts b/tests/boundary/filesystem/ciRequiredLanes.test.ts index 4cff44e96..6e30d8784 100644 --- a/tests/boundary/filesystem/ciRequiredLanes.test.ts +++ b/tests/boundary/filesystem/ciRequiredLanes.test.ts @@ -64,6 +64,76 @@ it("makes the required gate wait for every CI lane, including future additions", ); }); +it("keeps fast feedback separate from complete qualification and release proof", async () => { + const ci = z + .object({ + on: z.object({ + push: z.unknown(), + pull_request: z.unknown(), + schedule: z.array(z.object({ cron: z.string() })), + workflow_call: z.object({ + inputs: z.object({ + source_sha: z.object({ + required: z.literal(true), + type: z.literal("string"), + }), + }), + }), + }), + concurrency: z.object({ + group: z.string(), + "cancel-in-progress": z.string(), + }), + jobs: z.record( + z.string(), + z.object({ + if: z.string().optional(), + steps: z + .array( + z.object({ + run: z.string().optional(), + if: z.string().optional(), + with: z.record(z.string(), z.unknown()).optional(), + }), + ) + .optional(), + }), + ), + }) + .parse( + parse( + await readFile( + new URL("../../../.github/workflows/ci.yml", import.meta.url), + "utf8", + ), + ), + ); + expect(ci.on.schedule).toEqual([{ cron: "23 * * * *" }]); + expect(ci.concurrency.group).toContain("'qualification' || 'feedback'"); + expect(ci.concurrency.group).toContain("inputs.source_sha ||"); + for (const fullEvent of ["schedule", "workflow_dispatch"]) + expect(ci.concurrency["cancel-in-progress"]).toContain( + `github.event_name != '${fullEvent}'`, + ); + expect(ci.concurrency["cancel-in-progress"]).toContain( + "inputs.source_sha == ''", + ); + expect(ci.jobs.required?.if).toBe("${{ always() }}"); + const shards = ci.jobs["test-shard"]?.steps ?? []; + const fast = shards.find((step) => step.run?.includes("npm run test:run")); + const full = shards.find((step) => step.run?.includes("test:ci:shard:run")); + expect(fast?.if).toBe("needs.changes.outputs.tier == 'fast'"); + expect(fast?.run).toContain("--reporter=blob"); + expect(fast?.run).not.toContain("--coverage"); + expect(full?.if).toBe("needs.changes.outputs.tier == 'full'"); + const proof = ci.jobs["package-e2e"]?.steps?.find( + (step) => step.with?.name === "verified-release-package", + ); + expect(proof?.if).toContain("needs.changes.outputs.tier == 'full'"); + expect(proof?.if).toContain("matrix.platform == 'linux'"); + expect(proof?.if).toContain("matrix.architecture == 'x64'"); +}); + it.each([ ["true", "success", true], ["true", "failure", false], diff --git a/tests/boundary/filesystem/ciTestReports.test.ts b/tests/boundary/filesystem/ciTestReports.test.ts index 0c9bb4ccf..3ab042718 100644 --- a/tests/boundary/filesystem/ciTestReports.test.ts +++ b/tests/boundary/filesystem/ciTestReports.test.ts @@ -1,5 +1,5 @@ import { execFile } from "node:child_process"; -import { readFile, writeFile } from "node:fs/promises"; +import { mkdir, readFile, writeFile } from "node:fs/promises"; import { join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; import { promisify } from "node:util"; @@ -10,6 +10,7 @@ import { z } from "zod"; import { parseTestDurations } from "../../ciTestSequencer.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { workflowGit } from "../../support/workflowGit.js"; const execute = promisify(execFile); const script = new URL("../../../scripts/ci/test-report.mjs", import.meta.url); @@ -77,6 +78,7 @@ it("keeps diagnosis available after merge failure and publishes only complete su const history = steps.find((step) => step.with?.name === "test-timings"); expect(history?.if).toContain("success()"); expect(history?.if).toContain("needs.test-shard.result == 'success'"); + expect(history?.if).toContain("needs.changes.outputs.tier == 'full'"); expect(history?.with?.path).toBe(".cache/ci-test-history/test-results.json"); const download = (await workflow()).build!.steps.find( (step) => step.name === "Download previous timing report", @@ -84,6 +86,49 @@ it("keeps diagnosis available after merge failure and publishes only complete su expect(download?.with?.name).toBe(history?.with?.name); }); +it("rejects successful shard reports with missing or duplicate portable files", async () => { + const root = await createTestTempDirectory("rea-ci-report-inventory-"); + await workflowGit(root, ["init"]); + await mkdir(join(root, "tests")); + for (const name of ["first", "second"]) + await writeFile(join(root, `tests/${name}.test.ts`), "// fixture\n"); + await workflowGit(root, ["add", "tests"]); + const run = async (names: readonly string[]) => { + await writeFile( + join(root, "test-results.json"), + JSON.stringify({ + success: true, + numFailedTests: 0, + testResults: names.map((name) => ({ name, startTime: 0, endTime: 1 })), + }), + ); + return execute(process.execPath, [fileURLToPath(script)], { + cwd: root, + env: { + ...process.env, + SHARD_RESULT: "success", + MERGE_RESULT: "success", + CI_VERIFY_TEST_INVENTORY: "true", + }, + }); + }; + await expect( + run(["tests/first.test.ts", "tests/second.test.ts"]), + ).resolves.toMatchObject({ stderr: "" }); + for (const names of [ + ["tests/first.test.ts"], + ["tests/first.test.ts", "tests/second.test.ts", "tests/unplanned.test.ts"], + ["tests/first.test.ts", "tests/first.test.ts", "tests/second.test.ts"], + ]) { + await expect(run(names)).rejects.toMatchObject({ + stderr: expect.stringContaining("Incomplete portable test inventory"), + }); + await expect( + readFile(join(root, ".cache/ci-test-history/test-results.json")), + ).rejects.toMatchObject({ code: "ENOENT" }); + } +}); + it("runs the existing onboarding and Python producer guards from their selected CI owners", async () => { const jobs = await workflow(); const scripts = z diff --git a/tests/boundary/filesystem/ciVerifiedPackage.test.ts b/tests/boundary/filesystem/ciVerifiedPackage.test.ts new file mode 100644 index 000000000..ff5045024 --- /dev/null +++ b/tests/boundary/filesystem/ciVerifiedPackage.test.ts @@ -0,0 +1,137 @@ +import { createHash } from "node:crypto"; +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; + +import { expect, it } from "vitest"; + +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { + executeWorkflowFixture, + workflowGit, +} from "../../support/workflowGit.js"; + +const validator = fileURLToPath( + new URL("../../../scripts/ci/verified-package.mjs", import.meta.url), +); +type Fixture = Awaited>; +type FixtureOptions = { + override?: Record; + manifestContents?: string; + writeManifest?: boolean; +}; + +it("validates a tarball against the exact checkout and current workflow attempt", async () => { + const fixture = await createFixture(); + const result = await validate(fixture); + + expect(result.stdout.trim()).toBe( + join(fixture.artifactDirectory, "package.tgz"), + ); + expect(await readFile(fixture.output, "utf8")).toBe( + `tarball=${join(fixture.artifactDirectory, "package.tgz")}\n`, + ); +}); + +it("rejects a workflow SHA that differs from the checked out HEAD", async () => { + const fixture = await createFixture(); + + await expect(validate(fixture, "c".repeat(40))).rejects.toThrow(); +}); + +it.each([ + ["wrong checkout SHA", { sourceSha: "c".repeat(40) }], + ["wrong workflow run", { runId: 124 }], + ["future workflow attempt", { runAttempt: 3 }], + ["wrong verifier platform", { platform: "darwin" }], + ["wrong verifier architecture", { architecture: "arm64" }], + ["wrong package digest", { sha256: "c".repeat(64) }], +])("rejects %s", async (_label, override) => { + const fixture = await createFixture({ override }); + + await expect(validate(fixture)).rejects.toThrow(); +}); + +it("accepts an earlier successful attempt from the same workflow run", async () => { + const fixture = await createFixture({ override: { runAttempt: 1 } }); + + await expect(validate(fixture)).resolves.toMatchObject({ + stdout: expect.any(String), + }); +}); + +it.each([ + ["malformed manifest", "{"], + ["manifest with missing fields", JSON.stringify({ schemaVersion: 1 })], +])("rejects %s", async (_label, contents) => { + const fixture = await createFixture({ manifestContents: contents }); + + await expect(validate(fixture)).rejects.toThrow(); +}); + +it("rejects an absent manifest", async () => { + const fixture = await createFixture({ writeManifest: false }); + + await expect(validate(fixture)).rejects.toThrow(); +}); + +async function createFixture({ + override = {}, + manifestContents, + writeManifest = true, +}: FixtureOptions = {}) { + const directory = await createTestTempDirectory("rea-verified-package-"); + await workflowGit(directory, ["init", "--initial-branch=main"]); + await writeFile( + join(directory, "package.json"), + '{"name":"fixture","version":"1.2.3"}\n', + ); + await workflowGit(directory, ["add", "package.json"]); + await workflowGit(directory, [ + "commit", + "-m", + "test: verified package fixture", + ]); + const sourceSha = await workflowGit(directory, ["rev-parse", "HEAD"]); + const artifactDirectory = join(directory, "artifact"); + await mkdir(artifactDirectory); + const archive = "verified tarball fixture"; + await writeFile(join(artifactDirectory, "package.tgz"), archive); + const manifest = { + schemaVersion: 1, + sourceSha, + runId: 123, + runAttempt: 2, + platform: "linux", + architecture: "x64", + packageVersion: "1.2.3", + nodeVersion: process.version, + sha256: createHash("sha256").update(archive).digest("hex"), + ...override, + }; + if (writeManifest) + await writeFile( + join(artifactDirectory, "manifest.json"), + manifestContents ?? JSON.stringify(manifest), + ); + const output = join(directory, "github-output"); + await writeFile(output, ""); + return { directory, sourceSha, artifactDirectory, output }; +} + +function validate(fixture: Fixture, sourceSha = fixture.sourceSha) { + return executeWorkflowFixture( + process.execPath, + [validator, "validate", fixture.artifactDirectory], + { + cwd: fixture.directory, + env: { + ...process.env, + GITHUB_SHA: sourceSha, + GITHUB_RUN_ID: "123", + GITHUB_RUN_ATTEMPT: "2", + GITHUB_OUTPUT: fixture.output, + }, + }, + ); +} diff --git a/tests/boundary/filesystem/releaseWorkflow.test.ts b/tests/boundary/filesystem/releaseWorkflow.test.ts index 0bfebd1d2..4d3859433 100644 --- a/tests/boundary/filesystem/releaseWorkflow.test.ts +++ b/tests/boundary/filesystem/releaseWorkflow.test.ts @@ -34,6 +34,12 @@ async function readReleaseWorkflow() { "release-proposal": jobSchema, "release-please": jobSchema, "merged-release": jobSchema, + qualification: z.object({ + if: z.string(), + needs: z.array(z.string()), + uses: z.string(), + with: z.record(z.string(), z.string()), + }), publish: jobSchema, "publish-mcp": jobSchema, }), @@ -271,10 +277,19 @@ it("binds npm and MCP publication to the same immutable release SHA", async () = release_created: "${{ steps.release.outputs.release_created }}", sha: "${{ steps.release.outputs.sha }}", }); - expect(workflow.jobs.publish.if).toContain("inputs.phase == 'publish'"); - expect(workflow.jobs.publish.if).toContain( + expect(workflow.jobs.qualification.if).toContain("inputs.phase == 'publish'"); + expect(workflow.jobs.qualification.if).toContain( "needs.release-please.outputs.release_created", ); + expect(workflow.jobs.qualification.uses).toBe("./.github/workflows/ci.yml"); + expect(workflow.jobs.qualification.with).toEqual({ + source_sha: + "${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }}", + }); + expect(workflow.jobs.publish.needs).toContain("qualification"); + expect(workflow.jobs.publish.if).toContain( + "needs.qualification.result == 'success'", + ); expect(workflow.jobs["publish-mcp"].if).toContain( "needs.publish.result == 'success'", ); @@ -309,7 +324,7 @@ it("binds npm and MCP publication to the same immutable release SHA", async () = )?.run; expect(publishCommand).toContain("scripts/release-npm-tag.mjs"); expect(publishCommand).toContain( - 'npm publish --access public --tag "${tag}"', + 'npm publish "${VERIFIED_PACKAGE_TARBALL}" --access public --tag "${tag}"', ); for (const [version, tag] of [ ["6.1.0", "latest"], @@ -644,7 +659,7 @@ it("routes website verification through the central PR policy", async () => { // Publishing is irreversible. Keep the release authority invariant as a static // boundary check; it is not proof that package installation or publishing works. -it("requires package verification before publishing and retains the registry canary", async () => { +it("requires qualified package validation before publishing and retains the registry canary", async () => { const workflow = z .object({ jobs: z.object({ @@ -663,7 +678,7 @@ it("requires package verification before publishing and retains the registry can ); const commands = workflow.jobs.publish.steps.map(({ run }) => run ?? ""); const verify = commands.findIndex((command) => - command.includes("npm run verify:package"), + command.includes("scripts/ci/verified-package.mjs validate"), ); const publish = commands.findIndex((command) => command.includes("npm publish"), From db3c9f084bacad34f620ba678924564e20f441f9 Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 17:15:42 +0800 Subject: [PATCH 081/159] fix(ci): keep cancellation ownership in the calling workflow Remove redundant ref-based concurrency from reusable provider and website workflows. Their old groups could cancel qualification from another CI tier even though the callers used separate groups. Disposable hosted jobs now inherit caller scheduling; direct manual dispatches remain independent. Validate every called CI workflow has no independent cancellation group. Validation: 100 focused scope/gate tests and actionlint 1.7.7. --- .github/workflows/real-binary-layout.yml | 4 ---- .github/workflows/real-browser.yml | 4 ---- .github/workflows/real-evm-interface.yml | 4 ---- .../workflows/real-javascript-recovery.yml | 4 ---- .github/workflows/real-recorded-crash.yml | 4 ---- .../workflows/real-web-network-captures.yml | 4 ---- .github/workflows/real-web-runtime.yml | 3 --- .github/workflows/real-web-source-map.yml | 3 --- .github/workflows/website-check.yml | 4 ---- docs/testing.md | 5 +++- .../filesystem/ciRequiredLanes.test.ts | 23 +++++++++++++++++++ 11 files changed, 27 insertions(+), 35 deletions(-) diff --git a/.github/workflows/real-binary-layout.yml b/.github/workflows/real-binary-layout.yml index e8c244de6..8dd6f9866 100644 --- a/.github/workflows/real-binary-layout.yml +++ b/.github/workflows/real-binary-layout.yml @@ -12,10 +12,6 @@ on: permissions: contents: read -concurrency: - group: real-binary-layout-${{ github.event.pull_request.number || github.ref }} - cancel-in-progress: true - env: NODE_OPTIONS: --max-old-space-size=768 HUSKY: "0" diff --git a/.github/workflows/real-browser.yml b/.github/workflows/real-browser.yml index 76ba2e4a6..11a8b7a53 100644 --- a/.github/workflows/real-browser.yml +++ b/.github/workflows/real-browser.yml @@ -4,10 +4,6 @@ on: workflow_call: workflow_dispatch: -concurrency: - group: real-browser-${{ github.event.pull_request.number || github.ref }} - cancel-in-progress: ${{ github.event_name == 'pull_request' }} - env: HUSKY: "0" diff --git a/.github/workflows/real-evm-interface.yml b/.github/workflows/real-evm-interface.yml index 9e4cddeb4..b335f1391 100644 --- a/.github/workflows/real-evm-interface.yml +++ b/.github/workflows/real-evm-interface.yml @@ -7,10 +7,6 @@ on: permissions: contents: read -concurrency: - group: real-evm-interface-${{ github.event.pull_request.number || github.ref }} - cancel-in-progress: true - env: NODE_OPTIONS: --max-old-space-size=768 HUSKY: "0" diff --git a/.github/workflows/real-javascript-recovery.yml b/.github/workflows/real-javascript-recovery.yml index 4fee667d8..d94127d63 100644 --- a/.github/workflows/real-javascript-recovery.yml +++ b/.github/workflows/real-javascript-recovery.yml @@ -7,10 +7,6 @@ on: permissions: contents: read -concurrency: - group: javascript-recovery-${{ github.event.pull_request.number || github.ref }} - cancel-in-progress: ${{ github.event_name == 'pull_request' }} - env: HUSKY: "0" NODE_OPTIONS: "--max-old-space-size=768" diff --git a/.github/workflows/real-recorded-crash.yml b/.github/workflows/real-recorded-crash.yml index d4410bb24..dc56034c8 100644 --- a/.github/workflows/real-recorded-crash.yml +++ b/.github/workflows/real-recorded-crash.yml @@ -7,10 +7,6 @@ on: permissions: contents: read -concurrency: - group: real-recorded-crash-${{ github.event.pull_request.number || github.ref }} - cancel-in-progress: true - env: NODE_OPTIONS: --max-old-space-size=768 HUSKY: "0" diff --git a/.github/workflows/real-web-network-captures.yml b/.github/workflows/real-web-network-captures.yml index b888674f7..df340520d 100644 --- a/.github/workflows/real-web-network-captures.yml +++ b/.github/workflows/real-web-network-captures.yml @@ -7,10 +7,6 @@ on: permissions: contents: read -concurrency: - group: real-web-network-captures-${{ github.event.pull_request.number || github.ref }} - cancel-in-progress: true - env: HUSKY: "0" NODE_OPTIONS: --max-old-space-size=768 diff --git a/.github/workflows/real-web-runtime.yml b/.github/workflows/real-web-runtime.yml index c600e895c..0ecdb92c6 100644 --- a/.github/workflows/real-web-runtime.yml +++ b/.github/workflows/real-web-runtime.yml @@ -4,9 +4,6 @@ on: workflow_dispatch: permissions: contents: read -concurrency: - group: real-web-runtime-${{ github.event.pull_request.number || github.ref }} - cancel-in-progress: ${{ github.event_name == 'pull_request' }} env: HUSKY: "0" jobs: diff --git a/.github/workflows/real-web-source-map.yml b/.github/workflows/real-web-source-map.yml index 483b2f712..b94662129 100644 --- a/.github/workflows/real-web-source-map.yml +++ b/.github/workflows/real-web-source-map.yml @@ -4,9 +4,6 @@ on: workflow_dispatch: permissions: contents: read -concurrency: - group: real-web-source-map-${{ github.event.pull_request.number || github.ref }} - cancel-in-progress: ${{ github.event_name == 'pull_request' }} env: HUSKY: "0" jobs: diff --git a/.github/workflows/website-check.yml b/.github/workflows/website-check.yml index 8931ba418..2a2c5406f 100644 --- a/.github/workflows/website-check.yml +++ b/.github/workflows/website-check.yml @@ -7,10 +7,6 @@ on: permissions: contents: read -concurrency: - group: website-check-${{ github.event.pull_request.number || github.ref }} - cancel-in-progress: true - jobs: check: name: Website checks diff --git a/docs/testing.md b/docs/testing.md index b1d0031f6..ca3210f47 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -776,7 +776,10 @@ should require this aggregate, `Static checks`, and `Portable tests`. Fast runs cancel superseded feedback for the same PR or main branch. Scheduled/manual qualification has a separate concurrency group: active work finishes and only the latest pending run is retained. Release qualification is grouped by its -exact source SHA and cannot be cancelled by newer main pushes. Publication +exact source SHA and cannot be cancelled by newer main pushes. Provider and +website lanes inherit their caller's scheduling; reusable +workflows have no independent cancellation group. Direct manual lane dispatches +run independently on disposable hosted runners. Publication requires that qualification to succeed and consumes its exact verified tarball, checking source SHA, workflow run, run attempt, package/Node version and SHA-256 before npm publication. Earlier attempts from the same workflow run are accepted diff --git a/tests/boundary/filesystem/ciRequiredLanes.test.ts b/tests/boundary/filesystem/ciRequiredLanes.test.ts index 6e30d8784..0e9457e1c 100644 --- a/tests/boundary/filesystem/ciRequiredLanes.test.ts +++ b/tests/boundary/filesystem/ciRequiredLanes.test.ts @@ -64,6 +64,29 @@ it("makes the required gate wait for every CI lane, including future additions", ); }); +it("leaves cancellation ownership with CI so child lanes cannot cancel another tier", async () => { + const { jobs } = await workflow(); + const children = Object.values(jobs).flatMap((job) => + typeof job.uses === "string" && job.uses.startsWith("./.github/workflows/") + ? [job.uses] + : [], + ); + expect(children.length).toBeGreaterThan(0); + for (const path of children) { + const child = z + .object({ concurrency: z.unknown().optional() }) + .parse( + parse( + await readFile(new URL(`../../../${path}`, import.meta.url), "utf8"), + ), + ); + expect( + child.concurrency, + `${path} must use caller scheduling`, + ).toBeUndefined(); + } +}); + it("keeps fast feedback separate from complete qualification and release proof", async () => { const ci = z .object({ From e05d3399f19f25fdf04285a02b293418256b26dc Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 17:23:36 +0800 Subject: [PATCH 082/159] test(javascript): expect unknown parameter provenance --- .../javascript/javascriptSemanticAnalysis.structure.test.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/domain/javascript/javascriptSemanticAnalysis.structure.test.ts b/src/domain/javascript/javascriptSemanticAnalysis.structure.test.ts index 2cd9fba80..2312eff5d 100644 --- a/src/domain/javascript/javascriptSemanticAnalysis.structure.test.ts +++ b/src/domain/javascript/javascriptSemanticAnalysis.structure.test.ts @@ -72,9 +72,9 @@ describe("JavaScript semantic analysis: structure 1", () => { }); const shadow = onlyBinding(ir, "ipcRenderer"); expect(shadow.provenance).toEqual({ - status: "local", + status: "unknown", origins: [], - reason: null, + reason: "Parameter provenance depends on caller arguments.", }); const shadowReference = ir.references.find( ({ name, bindingId, role }) => From 52d4ce6ced1c3b6ae7570a5df7f4499cd65a1182 Mon Sep 17 00:00:00 2001 From: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> Date: Tue, 13 Oct 2026 02:42:09 -0700 Subject: [PATCH 083/159] fix(apple): derive dyld environment findings from each traced slice (#2733) deriveFindings looked up each traced root's slice again by architecture name. A universal executable with arm64e and arm64e.x1 slices reports both as "arm64e", so both roots resolved to the first slice: a later slice's LC_DYLD_ENVIRONMENT produced no finding, or the first slice's command was reported twice. The per-slice dyld_environment arrays were already correct. traceDylibLoading already has the exact slice when it records a root, so pass it to deriveFindings instead of re-deriving it from the name. The public roots and findings shapes are unchanged. Fixes #2661 Co-authored-by: Claude Opus 5.5 --- src/domain/apple/dylibResolution.ts | 5 +- src/domain/apple/dylibResolutionFindings.ts | 28 +++++----- tests/boundary/cli/dylibResolutionCli.test.ts | 56 +++++++++++++++++++ 3 files changed, 74 insertions(+), 15 deletions(-) diff --git a/src/domain/apple/dylibResolution.ts b/src/domain/apple/dylibResolution.ts index 799befdb1..cf10c4b78 100644 --- a/src/domain/apple/dylibResolution.ts +++ b/src/domain/apple/dylibResolution.ts @@ -16,6 +16,7 @@ import { import { DYLIB_RESOLUTION_LIMITATIONS, deriveFindings, + type TracedRootSlice, } from "./dylibResolutionFindings.js"; /** @@ -568,6 +569,7 @@ export const traceDylibLoading = async ( for (const path of request.unclassified ?? []) images.set(path, await view.image(path)); const roots: DylibTrace["roots"][number][] = []; + const rootSlices: TracedRootSlice[] = []; const edges: Edge[] = []; const withoutArchitecture: string[] = []; const environmentRoots = new Set(); @@ -597,6 +599,7 @@ export const traceDylibLoading = async ( ) environmentRoots.add(root); roots.push({ image: root, architecture: slice.architecture }); + rootSlices.push({ image: root, slice }); edges.push( ...(await traceProcess( { @@ -641,7 +644,7 @@ export const traceDylibLoading = async ( facts.status === "parsed" ? facts.slices.map(withoutDependencies) : [], })), edges, - findings: deriveFindings(edges, roots, images), + findings: deriveFindings(edges, rootSlices), coverage: { status: unparsed.length === 0 && diff --git a/src/domain/apple/dylibResolutionFindings.ts b/src/domain/apple/dylibResolutionFindings.ts index ca506c17b..eac488143 100644 --- a/src/domain/apple/dylibResolutionFindings.ts +++ b/src/domain/apple/dylibResolutionFindings.ts @@ -1,14 +1,18 @@ import type { DylibEdge as Edge, DylibFinding as Finding, - DylibTrace, - MachoImageFacts, + MachoSlice, } from "./dylibResolution.js"; +/** One traced process root with the exact slice it was traced from. */ +export interface TracedRootSlice { + readonly image: string; + readonly slice: MachoSlice; +} + export const deriveFindings = ( edges: readonly Edge[], - roots: DylibTrace["roots"], - images: ReadonlyMap, + roots: readonly TracedRootSlice[], ): Finding[] => { const findings: Finding[] = []; edges.forEach((edge, index) => { @@ -32,15 +36,11 @@ export const deriveFindings = ( explanation: `The modeled dyld search checks ${earlier.join(", ")} before ${edge.resolution.image ?? edge.install_name}. A compatible Mach-O placed at an earlier modeled path could take precedence over that fallback, subject to unmodeled search inputs and code-signing library validation; library validation is not evaluated here (see inspect_signature).${edge.loader_conditional ? " This loader loads only conditionally; if it never loads, this search never occurs." : ""}${edge.resolution.status === "conditional" ? " Unmodeled environment overrides or earlier unknown candidates may take precedence; placing an image at this rpath does not establish which image dyld will load." : ""}`, }); }); - for (const { image, architecture } of roots) { - const facts = images.get(image); - const slice = - facts?.status === "parsed" - ? facts.slices.find( - (candidate) => candidate.architecture === architecture, - ) - : undefined; - const environment = slice?.dyld_environment ?? []; + // Slices can share an architecture name (arm64e and arm64e.x1), so each + // finding uses the slice its root was traced from, not a name lookup. + for (const { image, slice } of roots) { + const { architecture } = slice; + const environment = slice.dyld_environment; if (environment.length > 0) findings.push({ kind: "dyld-environment-present", @@ -48,7 +48,7 @@ export const deriveFindings = ( image, basis: "derived", explanation: - slice?.file_type === "execute" + slice.file_type === "execute" ? `${image} (${architecture}) sets dyld environment variables through LC_DYLD_ENVIRONMENT (${environment.join(", ")}); any image-selection overrides they request are not modeled. Diagnostic settings alone do not alter resolution.` : `${image} (${architecture}) contains LC_DYLD_ENVIRONMENT (${environment.join(", ")}). Dyld reads these commands only from the main executable; entries in this non-executable root are observations and do not alter resolution.`, }); diff --git a/tests/boundary/cli/dylibResolutionCli.test.ts b/tests/boundary/cli/dylibResolutionCli.test.ts index b74dae71e..f6482a2f3 100644 --- a/tests/boundary/cli/dylibResolutionCli.test.ts +++ b/tests/boundary/cli/dylibResolutionCli.test.ts @@ -4,11 +4,13 @@ import { dirname, join } from "node:path"; import { describe, expect } from "vitest"; import { + CPU, FILE_TYPE, LC, buildVersionCommand, dylibCommand, dyldEnvironmentCommand, + fatImage, machoImage, rpathCommand, } from "../../../src/artifacts/apple/MachoImage.fixture.js"; @@ -471,6 +473,60 @@ cliTest( }, ); +cliTest( + "reports each same-named slice's own embedded dyld environment", + async ({ cli }) => { + // arm64e and arm64e.x1 slices both report the architecture name arm64e. + for (const [settings, expected] of [ + [[null, "DYLD_PRINT_RPATHS=1"], ["DYLD_PRINT_RPATHS=1"]], + [ + ["DYLD_PRINT_LIBRARIES=1", "DYLD_PRINT_RPATHS=1"], + ["DYLD_PRINT_LIBRARIES=1", "DYLD_PRINT_RPATHS=1"], + ], + ] as const) { + const app = await fixtureApp(); + await writeFiles(app, { + "Contents/MacOS/App": fatImage( + [CPU.arm64e, CPU.arm64eX1].map((cpu, index) => { + const setting = settings[index]; + return { + cpu, + bytes: machoImage({ + cpu, + commands: + setting === null || setting === undefined + ? [] + : [dyldEnvironmentCommand(setting)], + }), + }; + }), + ), + }); + const result = await cli.run({ + arguments: [ + "trace-dylib-resolution", + app, + "--root", + "Contents/MacOS/App", + "--json", + ], + environment: ENVIRONMENT, + }); + expect(result.exitCode, JSON.stringify(result.json)).toBe(0); + const trace = dylibResolutionResultSchema.parse( + parseEvidence(result.json).normalized_result, + ); + expect(trace.roots).toHaveLength(2); + const environments = trace.findings + .filter(({ kind }) => kind === "dyld-environment-present") + .map(({ explanation }) => + expected.filter((setting) => explanation.includes(setting)), + ); + expect(environments).toEqual(expected.map((setting) => [setting])); + } + }, +); + cliTest( "retains definitive load findings for an explicitly selected app library root", async ({ cli }) => { From 0cf224e76214abbcf47274aa936d1b5cad28db2f Mon Sep 17 00:00:00 2001 From: LouisChen <65096591@qq.com> Date: Tue, 13 Oct 2026 17:42:41 +0800 Subject: [PATCH 084/159] fix(javascript): retain distinct chunk registration callback metadata (#2737) Co-authored-by: LouisChen <3361008+chqcjq@users.noreply.github.com> --- ...ascriptBundlerRegistrationIdentity.test.ts | 78 +++++++++++++++++++ .../javascriptStaticAnalysisHelpers.ts | 8 ++ .../bundlerCallbackMetadataCli.test.ts | 61 +++++++++++++++ 3 files changed, 147 insertions(+) create mode 100644 src/domain/javascript/javascriptBundlerRegistrationIdentity.test.ts create mode 100644 tests/acceptance/applications/bundlerCallbackMetadataCli.test.ts diff --git a/src/domain/javascript/javascriptBundlerRegistrationIdentity.test.ts b/src/domain/javascript/javascriptBundlerRegistrationIdentity.test.ts new file mode 100644 index 000000000..1b251e313 --- /dev/null +++ b/src/domain/javascript/javascriptBundlerRegistrationIdentity.test.ts @@ -0,0 +1,78 @@ +import { expect, it } from "vitest"; +import { analyzeJavaScriptStaticSource } from "./javascriptStaticAnalysis.js"; + +const registration = (runtime: string, callback: string, chunk = 1) => + "(globalThis." + + runtime + + " = globalThis." + + runtime + + " || []).push([[" + + chunk + + "], {a: function() {}}, " + + callback + + "]);"; + +it.each(["webpackChunkFixture", "rspackChunkFixture"])( + "keeps differing callback facts for %s independent of source order", + (runtime) => { + const first = registration( + runtime, + 'function(r) { r(7); r.e("chunk-a"); }', + ); + const second = registration( + runtime, + 'function(r) { r(8); r.e("chunk-b"); }', + ); + for (const source of [first + "\n" + second, second + "\n" + first]) { + const result = analyzeJavaScriptStaticSource(source); + expect(result.parse_status).toBe("complete"); + expect(result.bundler_registrations).toHaveLength(2); + expect( + result.bundler_registrations.flatMap((r) => r.entry_module_keys).sort(), + ).toEqual(["7", "8"]); + expect( + result.bundler_registrations.flatMap((r) => r.async_chunk_keys).sort(), + ).toEqual(["chunk-a", "chunk-b"]); + } + expect( + analyzeJavaScriptStaticSource(first + "\n" + first).bundler_registrations, + ).toHaveLength(1); + expect( + analyzeJavaScriptStaticSource(first).bundler_registrations[0], + ).toMatchObject({ + entry_module_keys: ["7"], + async_chunk_keys: ["chunk-a"], + }); + expect( + analyzeJavaScriptStaticSource(second).bundler_registrations[0], + ).toMatchObject({ + entry_module_keys: ["8"], + async_chunk_keys: ["chunk-b"], + }); + }, +); + +it.each([ + ["function(r) { r(7); }", "function(r) { r(8); }"], + ['function(r) { r.e("a"); }', 'function(r) { r.e("b"); }'], + ["function(r) {}", "function(s) {}"], + ["function(r) {}", "function(r) { r(dynamic); }"], + ["function(r) {}", "function(r) { r.e(dynamic); }"], +])("retains callback metadata differences: %s / %s", (left, right) => { + const result = analyzeJavaScriptStaticSource( + registration("webpackChunkFixture", left) + + "\n" + + registration("webpackChunkFixture", right), + ); + expect(result.bundler_registrations).toHaveLength(2); +}); + +it("retains distinct chunk identities with identical callbacks", () => { + const callback = "function(r) { r(7); }"; + const result = analyzeJavaScriptStaticSource( + registration("webpackChunkFixture", callback, 1) + + "\n" + + registration("webpackChunkFixture", callback, 2), + ); + expect(result.bundler_registrations).toHaveLength(2); +}); diff --git a/src/domain/javascript/javascriptStaticAnalysisHelpers.ts b/src/domain/javascript/javascriptStaticAnalysisHelpers.ts index a6a6ebc44..71562e6ca 100644 --- a/src/domain/javascript/javascriptStaticAnalysisHelpers.ts +++ b/src/domain/javascript/javascriptStaticAnalysisHelpers.ts @@ -691,6 +691,14 @@ export const registrationKey = ( registration.runtime, registration.chunk_keys, String(registration.unknown_chunk_keys), + // Equal factory tables can be registered with different runtime callbacks. + // Both early collection and finalization must retain those recovered facts. + registration.runtime_require_name, + registration.runtime_module_cache_status, + registration.entry_module_keys, + String(registration.unknown_entry_module_keys), + registration.async_chunk_keys, + String(registration.unknown_async_chunk_keys), registration.modules.map(({ module_key: key, source_sha256: digest }) => [ key, digest, diff --git a/tests/acceptance/applications/bundlerCallbackMetadataCli.test.ts b/tests/acceptance/applications/bundlerCallbackMetadataCli.test.ts new file mode 100644 index 000000000..1f3987237 --- /dev/null +++ b/tests/acceptance/applications/bundlerCallbackMetadataCli.test.ts @@ -0,0 +1,61 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { expect } from "vitest"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { cliTest } from "../../support/cli/cliFixture.js"; +import { analyzeJavaScriptDirectory } from "../../support/cli/javascriptApplication.js"; + +cliTest.for(["CLI", "stdio MCP"] as const)( + "retains both repeated chunk callback observations through %s", + async (surface, { cli }) => { + const root = await createTestTempDirectory("rea-bundler-callbacks-"); + const source = [ + '(globalThis.webpackChunkFixture = globalThis.webpackChunkFixture || []).push([[1], {a: function() {}}, function(r) { r(7); r.e("chunk-a"); }]);', + '(globalThis.webpackChunkFixture = globalThis.webpackChunkFixture || []).push([[1], {a: function() {}}, function(r) { r(8); r.e("chunk-b"); }]);', + ].join("\n"); + await writeFile(join(root, "bundle.js"), source); + const { graph } = await analyzeJavaScriptDirectory(surface, cli, root); + const chunk = graph.nodes.find(({ kind }) => kind === "javascript-chunk"); + for (const [index, entry, asyncChunk] of [ + [0, "7", "chunk-a"], + [1, "8", "chunk-b"], + ] as const) { + expect(chunk?.observations).toContainEqual( + expect.objectContaining({ + properties: expect.objectContaining({ + entry_module_keys: [entry], + async_chunk_keys: [asyncChunk], + }), + evidence: expect.objectContaining({ + location: expect.objectContaining({ + value: expect.objectContaining({ + source: "bundle.js", + start: { line: index + 1, column: 0 }, + }), + }), + }), + }), + ); + expect(graph.edges).toContainEqual( + expect.objectContaining({ + source_node_id: chunk?.node_id, + properties: expect.objectContaining({ + kind: "bundler-entry-module", + module_key: entry, + resolution_status: "not-found", + }), + }), + ); + expect(graph.edges).toContainEqual( + expect.objectContaining({ + source_node_id: chunk?.node_id, + properties: expect.objectContaining({ + kind: "bundler-async-chunk", + chunk_key: asyncChunk, + resolution_status: "not-found", + }), + }), + ); + } + }, +); From d13c9f840d36a321a56917e75c48afcb8d72920e Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 17:45:53 +0800 Subject: [PATCH 085/159] ci: reduce redundant compatibility and qualification work Select SQLite compatibility by explicit source, fixture, lifecycle, and script ownership while retaining every version in full qualification. Share the canonical scope inventory between planning and the required gate. Reuse recent exact-SHA full qualification only for hourly schedules, with trusted successful-run metadata and receipts. Force a daily refresh, reject stale original runs even after retries, and never renew proof from reused validation. Report attempt-specific queue, execution, and matched step timing in a separate completion workflow using trusted default-branch code and read-only API access. Preserve raw metadata and unknown timings without delaying validation or publication. Validation: 177 focused tests; typecheck/lint; knip; actionlint 1.7.7; all 1046 tests discovered; live 37-job API report smoke. Remote workflow scheduling and real-provider matrices remain CI verification. --- .github/workflows/ci-timings.yml | 45 +++ .github/workflows/ci.yml | 34 +- docs/testing.md | 37 +- scripts/ci/job-timings.mjs | 353 +++++++++++++++++ scripts/ci/plan.mjs | 47 +-- scripts/ci/qualification.mjs | 176 ++++++++ scripts/ci/required.mjs | 30 +- scripts/ci/scopes.mjs | 63 +++ .../boundary/filesystem/ciChangeScope.test.ts | 126 +++++- .../boundary/filesystem/ciJobTimings.test.ts | 243 ++++++++++++ .../filesystem/ciQualification.test.ts | 375 ++++++++++++++++++ .../filesystem/ciRequiredLanes.test.ts | 100 ++++- 12 files changed, 1580 insertions(+), 49 deletions(-) create mode 100644 .github/workflows/ci-timings.yml create mode 100644 scripts/ci/job-timings.mjs create mode 100644 scripts/ci/qualification.mjs create mode 100644 tests/boundary/filesystem/ciJobTimings.test.ts create mode 100644 tests/boundary/filesystem/ciQualification.test.ts diff --git a/.github/workflows/ci-timings.yml b/.github/workflows/ci-timings.yml new file mode 100644 index 000000000..70ef1ebc7 --- /dev/null +++ b/.github/workflows/ci-timings.yml @@ -0,0 +1,45 @@ +name: CI timings + +on: + workflow_run: + workflows: [CI, Release] + types: [completed] + +permissions: + contents: read + actions: read + +jobs: + report: + name: Report completed workflow attempt + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + # Completed PR runs can contain untrusted changes. Execute only the + # reporter from the trusted default branch; consume jobs as API data. + - uses: actions/checkout@v4 + with: + ref: ${{ github.event.repository.default_branch }} + persist-credentials: false + - uses: actions/setup-node@v4 + with: + node-version-file: .nvmrc + - name: Record queue, execution, and matched step timing + continue-on-error: true + env: + GH_TOKEN: ${{ github.token }} + CI_REPOSITORY: ${{ github.repository }} + CI_TIMING_RUN_ID: ${{ github.event.workflow_run.id }} + CI_TIMING_RUN_ATTEMPT: ${{ github.event.workflow_run.run_attempt }} + run: node scripts/ci/job-timings.mjs + - name: Upload CI timing artifacts + if: always() && hashFiles('.cache/ci-job-timings/*') != '' + continue-on-error: true + uses: actions/upload-artifact@v4 + with: + name: ci-job-timings + path: .cache/ci-job-timings/ + include-hidden-files: true + if-no-files-found: error + overwrite: true + retention-days: 14 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index fe79048ba..39ebf2e86 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -6,7 +6,8 @@ on: pull_request: branches: [main, "release/*"] schedule: - - cron: "23 * * * *" + - cron: "23 0-2,4-23 * * *" + - cron: "23 3 * * *" workflow_dispatch: workflow_call: inputs: @@ -53,7 +54,9 @@ jobs: actions: read outputs: tier: ${{ steps.plan.outputs.tier }} + qualification_run_id: ${{ steps.qualification.outputs.run_id }} code: ${{ steps.plan.outputs.code }} + sqlite: ${{ steps.plan.outputs.sqlite }} runtime: ${{ steps.plan.outputs.runtime }} package: ${{ steps.plan.outputs.package }} apple: ${{ steps.plan.outputs.apple }} @@ -96,6 +99,17 @@ jobs: CI_REPOSITORY: ${{ github.repository }} CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} run: node scripts/ci/baseline.mjs + - name: Admit scheduled full qualification + id: qualification + if: github.event_name == 'schedule' && inputs.source_sha == '' + env: + GH_TOKEN: ${{ github.token }} + HEAD_SHA: ${{ github.sha }} + CI_REPOSITORY: ${{ github.repository }} + CI_REPOSITORY_ID: ${{ github.repository_id }} + CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} + CI_FORCE_FULL: ${{ github.event.schedule == '23 3 * * *' }} + run: node scripts/ci/qualification.mjs - name: Select verification lanes id: plan env: @@ -103,12 +117,13 @@ jobs: BASELINE_REASON: ${{ steps.baseline.outputs.reason }} HEAD_SHA: ${{ inputs.source_sha || github.event.pull_request.head.sha || github.sha }} CI_FULL: ${{ inputs.source_sha != '' || contains(github.event.pull_request.labels.*.name, 'ci:full') }} + CI_SKIP_FULL: ${{ steps.qualification.outputs.run_full == 'false' }} run: node scripts/ci/plan.mjs static: name: Static checks needs: changes - if: ${{ !cancelled() }} + if: ${{ !cancelled() && needs.changes.outputs.tier != 'reused' }} runs-on: ubuntu-latest steps: - name: Require successful change classification @@ -249,13 +264,13 @@ jobs: sqlite-runtime: needs: [changes, build] - if: needs.changes.outputs.code == 'true' + if: needs.changes.outputs.sqlite == 'true' uses: ./.github/workflows/sqlite-runtime.yml test: name: Portable tests needs: [changes, test-shard] - if: ${{ !cancelled() }} + if: ${{ !cancelled() && needs.changes.outputs.tier != 'reused' }} runs-on: ubuntu-latest steps: - name: Require successful change classification @@ -754,6 +769,17 @@ jobs: with: node-version-file: .nvmrc - name: Require every selected lane to pass + id: gate env: CI_JOBS: ${{ toJSON(needs) }} + CI_QUALIFICATION_DIRECTORY: .cache/ci-qualification run: node scripts/ci/required.mjs + - name: Upload full qualification receipt artifact + if: needs.changes.outputs.tier == 'full' && success() + uses: actions/upload-artifact@v4 + with: + name: ${{ steps.gate.outputs.receipt_name }} + path: ${{ steps.gate.outputs.receipt_path }} + if-no-files-found: error + overwrite: true + retention-days: 7 diff --git a/docs/testing.md b/docs/testing.md index ca3210f47..0203abade 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -276,6 +276,21 @@ sharding. Static checks, documentation, build, package verification, and real-system lanes remain separate jobs so one kind of evidence cannot stand in for another. +The separate `CI timings` workflow runs after CI or Release completes and uploads +`ci-job-timings` with raw API pages and a normalized point-in-time report for that +workflow attempt. It runs the reporter from the trusted default branch with +read-only permissions, treating the triggering run as API data. Its summary separates queue +waiting (`created_at` to `started_at`) from completed execution and reports +percentiles plus matched `npm ci` and artifact-step durations. +Queue excludes dependency waiting before job creation and does not establish +whether account concurrency or runner availability caused the wait. Queued jobs with +placeholder start times, skipped jobs, and missing or invalid timestamps remain +unknown. The reporter targets the triggering run and attempt explicitly. +Summed job elapsed time is parallel work, not workflow wall time, CPU time, +memory usage, or billing. Step names identify samples, not complete transfer +costs. Its queue, execution, and failures are independent of the required gate +and publication. CPU time and peak process-family memory are not measured by this API. + The build job saves portable Turbo outputs across runs. Eight Linux real-system lanes restore this cache without waiting for the build job. The shared action derives its key from `test:prepare --dry=json` task hashes, runner OS and @@ -737,6 +752,12 @@ lane. Edits to verifier entrypoints run only by manual, licensed or self-hosted workflows run static checks and the Linux suite. Website changes run website checks. Provider workflow changes validate the workflow and run that provider's verifier. Manual-provider/Android workflow changes run workflow validation only. +The four-version SQLite compatibility matrix has its own `sqlite` scope for +SQLite implementation, contracts, CLI/MCP registration, fixtures, and shared +runtime boundaries. SQLite and server-lifecycle test edits and the SQLite npm +scripts also select it. Unrelated code edits retain the complete portable suite +on the pinned Node version without starting four additional compatibility jobs. +Full qualification retains every SQLite version. Shared process/contract/CLI boundaries, dependencies, other packaging inputs, unknown paths, and changes to the planner select the full baseline. Package script-only edits select their owning verifier when its entrypoint is @@ -760,8 +781,20 @@ Every code change still runs all six portable shards, without coverage instrumentation in this fast tier. This deliberately avoids relying on an import graph for generated files, filesystem inputs, subprocesses, or bridges. -Full qualification runs hourly at minute 23, on manual dispatch, and for the -exact release revision before publication. It includes aggregate coverage, +Scheduled qualification is admitted hourly at minute 23. A forced full run at +03:23 UTC refreshes runner and dependency drift every day. Other hourly runs +can reuse a successful full-qualification receipt for the exact source SHA +from the same repository's trusted CI workflow on its default branch. Both the +receipt and original workflow run must be less than 24 hours old, so retrying an +old gate cannot renew stale proof. Missing, expired, failed, foreign, or uncertain proof runs full +validation; API lookup failure also runs full. Lookups examine at most 100 named +receipts and three candidate runs within a 20-second budget. Reused runs select +no validation lanes, identify the original run, and produce no new receipt, so +reuse cannot indefinitely extend proof freshness. They make no fresh provider +observation. Manual dispatch, `ci:full`, and exact release qualification always +run full validation. + +Full qualification includes aggregate coverage, every portable provider lane, and all native hosts below. A `ci:full` PR label selects full qualification on the next PR run; manual dispatch can request it immediately. Fast-tier broad fallbacks retain all owner lanes and the three-host diff --git a/scripts/ci/job-timings.mjs b/scripts/ci/job-timings.mjs new file mode 100644 index 000000000..9d22f0d41 --- /dev/null +++ b/scripts/ci/job-timings.mjs @@ -0,0 +1,353 @@ +import { execFile } from "node:child_process"; +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { dirname, join, resolve } from "node:path"; +import { promisify } from "node:util"; +import { fileURLToPath } from "node:url"; + +const execFileAsync = promisify(execFile); +const MAX_BUFFER = 16 * 1024 * 1024; +const TIMEOUT_MS = 30_000; +const OUTPUT_DIRECTORY = join(".cache", "ci-job-timings"); +const STEP_COSTS = [ + { key: "npmCi", pattern: /\bnpm ci\b/iu }, + { key: "artifactUpload", pattern: /upload.*artifact/iu }, + { key: "artifactDownload", pattern: /download.*artifact/iu }, +]; + +/** Fetch and summarize one workflow attempt without turning telemetry into a gate. */ +async function collectJobTimings({ + repository, + runId, + runAttempt, + outputDirectory = OUTPUT_DIRECTORY, + summaryPath, + observedAt = new Date().toISOString(), + env = process.env, +}) { + const identity = validateIdentity({ repository, runId, runAttempt }); + const endpoint = `repos/${identity.repository}/actions/runs/${identity.runId}/attempts/${identity.runAttempt}/jobs?per_page=100`; + const { stdout } = await execFileAsync( + "gh", + ["api", "--paginate", "--slurp", endpoint], + { env, encoding: "utf8", maxBuffer: MAX_BUFFER, timeout: TIMEOUT_MS }, + ); + const rawPath = join(outputDirectory, "jobs.json"); + const reportPath = join(outputDirectory, "report.json"); + await mkdir(outputDirectory, { recursive: true }); + await writeFile(rawPath, stdout, "utf8"); + const pages = parsePages(stdout); + const jobs = pages.flatMap(({ jobs: pageJobs }, index) => + validateJobs(pageJobs, index), + ); + const report = normalizeReport({ identity, jobs, observedAt }); + + await writeFile(reportPath, `${JSON.stringify(report, null, 2)}\n`, "utf8"); + if (summaryPath !== undefined) + await appendSummary(summaryPath, report, rawPath, reportPath); + return { rawPath, reportPath, report }; +} + +function validateIdentity({ repository, runId, runAttempt }) { + if ( + typeof repository !== "string" || + !/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/u.test(repository) + ) + throw new Error( + "CI_REPOSITORY must be an owner/name repository identifier", + ); + return { + repository, + runId: parsePositiveInteger(runId, "CI_TIMING_RUN_ID"), + runAttempt: parsePositiveInteger(runAttempt, "CI_TIMING_RUN_ATTEMPT"), + }; +} + +function parsePositiveInteger(value, name) { + if (typeof value === "string" && /^[1-9]\d*$/u.test(value)) { + const parsed = Number(value); + if (Number.isSafeInteger(parsed)) return parsed; + } + throw new Error(`${name} must be a positive safe integer`); +} + +function parsePages(stdout) { + let pages; + try { + pages = JSON.parse(stdout); + } catch (cause) { + throw new Error("GitHub jobs API returned malformed JSON", { cause }); + } + if (!Array.isArray(pages) || pages.length === 0) + throw new Error("GitHub jobs API slurp must contain a nonempty page array"); + for (const page of pages) + if ( + typeof page !== "object" || + page === null || + Array.isArray(page) || + !Array.isArray(page.jobs) + ) + throw new Error("GitHub jobs API page must contain a jobs array"); + return pages; +} + +function validateJobs(jobs, pageIndex) { + return jobs.map((job, jobIndex) => { + if ( + typeof job !== "object" || + job === null || + Array.isArray(job) || + typeof job.name !== "string" || + job.name.length === 0 || + typeof job.status !== "string" + ) + throw new Error( + `GitHub jobs API page ${String(pageIndex + 1)} job ${String(jobIndex + 1)} is malformed`, + ); + return job; + }); +} + +function normalizeReport({ identity, jobs, observedAt }) { + const normalizedJobs = jobs.map(normalizeJob); + const queues = normalizedJobs + .map(({ queue }) => queue.seconds) + .filter((value) => value !== null); + const completedExecutions = normalizedJobs + .filter( + ({ status, conclusion }) => + status === "completed" && conclusion !== "skipped", + ) + .map(({ execution }) => execution.seconds) + .filter((value) => value !== null); + const completedDurationSumSeconds = completedExecutions.reduce( + (sum, seconds) => sum + seconds, + 0, + ); + const completedExecutionUnknown = normalizedJobs.filter( + ({ status, conclusion, execution }) => + status === "completed" && + conclusion !== "skipped" && + execution.seconds === null, + ).length; + const skipped = normalizedJobs.filter( + ({ conclusion }) => conclusion === "skipped", + ).length; + const costs = Object.fromEntries(STEP_COSTS.map(({ key }) => [key, []])); + for (const job of normalizedJobs) + for (const step of job.steps) + if (step.costKey !== null) + costs[step.costKey].push({ job: job.name, ...step }); + + return { + schemaVersion: 1, + run: { + ...identity, + observedAt, + snapshot: + normalizedJobs.length > 0 && + normalizedJobs.every(({ status }) => status === "completed") + ? "completed job status snapshot; unknown timestamps remain unmeasured" + : "partial; job statuses may still be queued or in progress", + }, + counts: { + jobs: normalizedJobs.length, + completed: normalizedJobs.filter(({ status }) => status === "completed") + .length, + skipped, + queueUnknown: normalizedJobs.filter(({ queue }) => queue.seconds === null) + .length, + executionUnknown: normalizedJobs.filter( + ({ execution }) => execution.seconds === null, + ).length, + completedExecutionUnknown, + }, + aggregates: { + queueSeconds: percentileSummary(queues), + completedExecutionSeconds: percentileSummary(completedExecutions), + summedCompletedJobElapsedSeconds: { + value: + completedExecutions.length === 0 + ? null + : round(completedDurationSumSeconds), + knownCount: completedExecutions.length, + unknownCount: completedExecutionUnknown, + label: + "sum of known completed job elapsed durations; incomplete when timestamps are unknown; not CPU time, billing, or workflow wall time", + }, + }, + jobs: normalizedJobs, + stepCostInterpretation: + "Matched install/artifact steps by step name; these are not total artifact transfer costs.", + stepCosts: costs, + }; +} + +function normalizeJob(job) { + const skipped = job.conclusion === "skipped"; + const hasStarted = ["in_progress", "completed"].includes(job.status); + const queue = skipped + ? unknownDuration("job was skipped") + : !hasStarted + ? unknownDuration("job start is not established by its status") + : duration(job.created_at, job.started_at); + const execution = skipped + ? unknownDuration("job was skipped") + : job.status === "completed" + ? duration(job.started_at, job.completed_at) + : unknownDuration("job is not completed"); + const steps = Array.isArray(job.steps) ? job.steps.map(normalizeStep) : []; + return { + id: Number.isSafeInteger(job.id) && job.id > 0 ? job.id : null, + url: stringOrNull(job.html_url), + name: job.name, + workflowName: stringOrNull(job.workflow_name), + status: job.status, + conclusion: stringOrNull(job.conclusion), + createdAt: stringOrNull(job.created_at), + startedAt: stringOrNull(job.started_at), + completedAt: stringOrNull(job.completed_at), + queue, + execution, + steps, + }; +} + +function normalizeStep(step) { + if ( + typeof step !== "object" || + step === null || + Array.isArray(step) || + typeof step.name !== "string" + ) + throw new Error("GitHub jobs API contains a malformed job step"); + const cost = STEP_COSTS.find(({ pattern }) => pattern.test(step.name)); + return { + name: step.name, + costKey: cost?.key ?? null, + duration: duration(step.started_at, step.completed_at), + }; +} + +function duration(startValue, endValue) { + if (typeof startValue !== "string" || typeof endValue !== "string") + return unknownDuration("timestamp unavailable"); + const start = Date.parse(startValue); + const end = Date.parse(endValue); + if (!Number.isFinite(start) || !Number.isFinite(end)) + return unknownDuration("invalid timestamp"); + const milliseconds = end - start; + if (!Number.isFinite(milliseconds) || milliseconds < 0) + return unknownDuration("timestamps out of order"); + return { seconds: round(milliseconds / 1_000), reason: null }; +} + +function unknownDuration(reason) { + return { seconds: null, reason }; +} + +function percentileSummary(values) { + return { + count: values.length, + p50: nearestRank(values, 0.5), + p95: nearestRank(values, 0.95), + unit: "seconds", + }; +} + +function nearestRank(values, percentile) { + if (values.length === 0) return null; + const sorted = [...values].sort((left, right) => left - right); + return round(sorted[Math.ceil(percentile * sorted.length) - 1]); +} + +function round(value) { + return Math.round(value * 1_000) / 1_000; +} + +function stringOrNull(value) { + return typeof value === "string" ? value : null; +} + +async function appendSummary(summaryPath, report, rawPath, reportPath) { + await mkdir(dirname(summaryPath), { recursive: true }); + const rows = [ + ["Queue", report.aggregates.queueSeconds], + ["Completed execution", report.aggregates.completedExecutionSeconds], + ] + .map( + ([label, item]) => + `| ${label} | ${display(item.p50)} | ${display(item.p95)} | ${String(item.count)} |`, + ) + .join("\n"); + const jobRows = report.jobs + .map( + (job) => + `| ${escapeCell(job.name)} | ${job.status} | ${job.conclusion ?? "unknown"} | ${display(job.queue.seconds)} | ${job.queue.seconds === null ? job.queue.reason : ""} | ${display(job.execution.seconds)} | ${job.execution.seconds === null ? job.execution.reason : ""} |`, + ) + .join("\n"); + const costRows = STEP_COSTS.map(({ key }) => { + const samples = report.stepCosts[key]; + const values = samples + .map(({ duration: item }) => item.seconds) + .filter((value) => value !== null); + return `| ${key} | ${String(samples.length)} | ${display(nearestRank(values, 0.5))} | ${samples.filter(({ duration: item }) => item.seconds === null).length} |`; + }).join("\n"); + const text = [ + "### CI job timing snapshot", + "", + `Observed at ${report.run.observedAt} for run ${String(report.run.runId)}, attempt ${String(report.run.runAttempt)}. This is a point-in-time snapshot of fetched jobs. Queue starts at job creation and excludes preceding dependency waiting. Missing, queued, in-progress, invalid, or out-of-order timestamps are reported as unknown. Elapsed durations are not CPU time or billing data.`, + "", + `Fetched ${String(report.counts.jobs)} jobs; ${String(report.counts.completed)} are completed (${String(report.counts.skipped)} skipped); ${String(report.counts.queueUnknown)} queue durations and ${String(report.counts.executionUnknown)} execution durations (${String(report.counts.completedExecutionUnknown)} for non-skipped completed jobs) are unknown.`, + "", + "| Metric | p50 (s) | p95 (s) | Samples |", + "| --- | ---: | ---: | ---: |", + rows, + "", + "| Job | Status | Conclusion | Queue (s) | Queue detail | Execution (s) | Execution detail |", + "| --- | --- | --- | ---: | --- | ---: | --- |", + jobRows, + "", + `Sum of known completed job elapsed durations: **${display(report.aggregates.summedCompletedJobElapsedSeconds.value)} s** across ${String(report.aggregates.summedCompletedJobElapsedSeconds.knownCount)} known and ${String(report.aggregates.summedCompletedJobElapsedSeconds.unknownCount)} unknown jobs; this is not CPU time, billing, or workflow wall time.`, + "", + "Matched install/artifact steps by name; these are not total artifact transfer costs.", + "", + "| Matched step group | Samples | p50 (s) | Unknown |", + "| --- | ---: | ---: | ---: |", + costRows, + "", + `Reports: \`${rawPath}\`, \`${reportPath}\`.`, + "", + ].join("\n"); + await writeFile(summaryPath, text, { encoding: "utf8", flag: "a" }); +} + +function display(value) { + return value === null ? "unknown" : String(value); +} + +function escapeCell(value) { + return value.replaceAll("|", "\\|").replaceAll("\n", " "); +} + +if ( + process.argv[1] !== undefined && + resolve(process.argv[1]) === fileURLToPath(import.meta.url) +) { + try { + const result = await collectJobTimings({ + repository: process.env.CI_REPOSITORY, + runId: process.env.CI_TIMING_RUN_ID, + runAttempt: process.env.CI_TIMING_RUN_ATTEMPT, + outputDirectory: process.env.CI_JOB_TIMINGS_DIRECTORY ?? OUTPUT_DIRECTORY, + summaryPath: process.env.GITHUB_STEP_SUMMARY, + }); + process.stdout.write( + `Wrote ${result.report.jobs.length} job timing records to ${result.reportPath}\n`, + ); + } catch (cause) { + process.stderr.write( + `CI timing telemetry failed: ${cause instanceof Error ? cause.message : String(cause)}\n`, + ); + process.exitCode = 1; + } +} diff --git a/scripts/ci/plan.mjs b/scripts/ci/plan.mjs index 7ffec2c85..e9505b903 100644 --- a/scripts/ci/plan.mjs +++ b/scripts/ci/plan.mjs @@ -1,30 +1,17 @@ import { execFileSync } from "node:child_process"; import { appendFileSync } from "node:fs"; import { matchesGlob } from "node:path"; -import { MANUAL_VERIFIERS, SCOPES } from "./scopes.mjs"; +import { CI_FLAGS as flags, MANUAL_VERIFIERS, SCOPES } from "./scopes.mjs"; -const flags = [ - ...new Set([ - "code", - "runtime", - "package", - "apple", - "fixtures", - "readiness", - "docs", - "workflows", - ...Object.keys(SCOPES), - ]), -]; -// Test files select the native-host lanes that run them explicitly. Provider -// scopes remain implementation-owned, so test-only edits stay off those lanes. -const NATIVE_TEST_OWNERS = ["apple", "package"]; +// Test files select the host/runtime lanes that execute their focused suites. +const HOST_TEST_OWNERS = ["apple", "package", "sqlite"]; // Actionlint validates workflow syntax only. The semantic suite asserts these // workflows' release, publication and self-hosted runner safeguards. const GUARDED_WORKFLOWS = new Set([ ".github/workflows/release.yml", ".github/workflows/real-ghidra-windows.yml", ".github/workflows/sqlite-runtime.yml", + ".github/workflows/ci-timings.yml", ]); // Sources compiled by build-conformance-fixtures.mjs. The readiness journey // analyzes the built C fixture as well as its own applications. @@ -63,6 +50,10 @@ function classifyPackage(base, head) { ]), ].filter((key) => beforeScripts?.[key] !== afterScripts?.[key]); for (const script of changed) { + if (script === "test:sqlite" || script === "test:sqlite:run") { + selected.sqlite = true; + continue; + } if ( /^(test(?::|$)|check(?::|$)|lint(?::|$)|format(?::|$)|knip$|jscpd$)/.test( script, @@ -110,14 +101,24 @@ function classifyPackage(base, head) { const event = process.env.GITHUB_EVENT_NAME; if (!["pull_request", "push", "schedule", "workflow_dispatch"].includes(event)) throw new Error(`Unsupported CI event: ${event}`); -const tier = +const requestedFull = event === "schedule" || event === "workflow_dispatch" || - process.env.CI_FULL === "true" - ? "full" - : "fast"; + process.env.CI_FULL === "true"; +const tier = + event === "schedule" && + process.env.CI_SKIP_FULL === "true" && + process.env.CI_FULL !== "true" + ? "reused" + : requestedFull + ? "full" + : "fast"; let files = []; -if (tier === "full") { +if (tier === "reused") { + reasons.push( + "Recent exact-SHA full qualification reused; no fresh provider observation", + ); +} else if (tier === "full") { full("Full baseline requested by event or ci:full label"); } else if (event === "pull_request" || event === "push") { const head = process.env.HEAD_SHA; @@ -162,7 +163,7 @@ if (tier === "full") { } if (/^(src|tests)\/.*\.(?:test|spec)\.[cm]?[jt]sx?$/.test(file)) { selected.code = true; - for (const owner of NATIVE_TEST_OWNERS) + for (const owner of HOST_TEST_OWNERS) selected[owner] ||= matchesAny(file, SCOPES[owner]); continue; } diff --git a/scripts/ci/qualification.mjs b/scripts/ci/qualification.mjs new file mode 100644 index 000000000..ab47c168f --- /dev/null +++ b/scripts/ci/qualification.mjs @@ -0,0 +1,176 @@ +import { execFileSync } from "node:child_process"; +import { appendFileSync, mkdirSync, writeFileSync } from "node:fs"; +import { resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +const ARTIFACT_NAME = "ci-full-qualification-v1"; +const MAX_AGE_MS = 24 * 60 * 60 * 1000; +const positiveId = (value) => + (typeof value === "string" || typeof value === "number") && + /^[1-9]\d*$/.test(String(value)) && + Number.isSafeInteger(Number(value)); +const commitSha = (value) => + typeof value === "string" && /^[0-9a-f]{40}$/.test(value); + +/** Record only after the required gate has verified every full-tier obligation. */ +export function recordQualification(directory) { + const sourceSha = process.env.GITHUB_SHA; + const runId = process.env.GITHUB_RUN_ID; + const runAttempt = process.env.GITHUB_RUN_ATTEMPT; + const repository = process.env.GITHUB_REPOSITORY; + if ( + !commitSha(sourceSha) || + !positiveId(runId) || + !positiveId(runAttempt) || + !repository + ) + throw new Error( + "Full qualification receipt requires source and workflow identity", + ); + const checkoutSha = execFileSync("git", ["rev-parse", "HEAD"], { + encoding: "utf8", + }).trim(); + if (checkoutSha !== sourceSha) + throw new Error("Qualification checkout differs from workflow source"); + mkdirSync(directory, { recursive: true }); + const path = resolve(directory, "receipt.json"); + writeFileSync( + path, + `${JSON.stringify( + { + schemaVersion: 1, + tier: "full", + sourceSha, + repository, + runId: Number(runId), + runAttempt: Number(runAttempt), + qualifiedAt: new Date().toISOString(), + }, + null, + 2, + )}\n`, + ); + if (process.env.GITHUB_OUTPUT) + appendFileSync( + process.env.GITHUB_OUTPUT, + `receipt_name=${ARTIFACT_NAME}\nreceipt_path=${path}\n`, + ); +} + +function admit() { + if (process.env.CI_FORCE_FULL === "true") + return { + run_full: true, + reason: "Forced daily runner and dependency refresh", + }; + const repository = process.env.CI_REPOSITORY; + const repositoryId = Number(process.env.CI_REPOSITORY_ID); + const branch = process.env.CI_DEFAULT_BRANCH; + const head = process.env.HEAD_SHA; + if (!repository || !positiveId(repositoryId) || !branch || !commitSha(head)) + return { + run_full: true, + reason: "Qualification lookup identity is incomplete", + }; + + const now = Date.now(); + const deadline = now + 20_000; + const api = (...arguments_) => { + const remaining = deadline - Date.now(); + if (remaining <= 0) + throw new Error("Qualification lookup budget exhausted"); + return JSON.parse( + execFileSync("gh", ["api", "--method", "GET", ...arguments_], { + encoding: "utf8", + timeout: Math.min(15_000, remaining), + maxBuffer: 8 * 1024 * 1024, + stdio: ["ignore", "pipe", "pipe"], + }), + ); + }; + try { + const { artifacts } = api( + `repos/${repository}/actions/artifacts`, + "-f", + `name=${ARTIFACT_NAME}`, + "-F", + "per_page=100", + ); + if (!Array.isArray(artifacts)) + throw new Error("Missing qualification artifact inventory"); + const candidates = artifacts.filter((artifact) => { + const lineage = artifact?.workflow_run; + const age = now - Date.parse(artifact?.created_at); + return ( + artifact?.name === ARTIFACT_NAME && + artifact?.expired === false && + typeof artifact?.created_at === "string" && + Number.isFinite(age) && + age >= 0 && + age < MAX_AGE_MS && + positiveId(lineage?.id) && + String(lineage.id) !== process.env.GITHUB_RUN_ID && + lineage?.repository_id === repositoryId && + lineage?.head_repository_id === repositoryId && + lineage?.head_branch === branch && + lineage?.head_sha === head + ); + }); + // Artifact metadata narrows the search before run lookups. Never scan an + // unbounded history or treat a missing receipt as a successful qualification. + for (const artifact of candidates.slice(0, 3)) { + const run = api( + `repos/${repository}/actions/runs/${artifact.workflow_run.id}`, + ); + // An old run can upload a new artifact when only its gate is rerun. + // Bound the original run age as well so retries cannot renew stale proof. + const runAge = Date.now() - Date.parse(run.created_at); + if ( + typeof run.created_at === "string" && + Number.isFinite(runAge) && + runAge >= 0 && + runAge < MAX_AGE_MS && + run.id === artifact.workflow_run.id && + run.status === "completed" && + run.conclusion === "success" && + run.path === ".github/workflows/ci.yml" && + ["schedule", "workflow_dispatch"].includes(run.event) && + run.head_sha === head && + run.head_branch === branch && + run.head_repository?.full_name === repository + ) { + return { + run_full: false, + run_id: run.id, + reason: `Reusing full qualification from run ${run.id}; no fresh provider observation`, + }; + } + } + return { + run_full: true, + reason: "No trusted full qualification for this SHA within 24 hours", + }; + } catch { + return { + run_full: true, + reason: "Qualification lookup unavailable; running full validation", + }; + } +} + +if (resolve(process.argv[1] ?? "") === fileURLToPath(import.meta.url)) { + const decision = admit(); + if (process.env.GITHUB_OUTPUT) + appendFileSync( + process.env.GITHUB_OUTPUT, + Object.entries(decision) + .map(([key, value]) => `${key}=${value}\n`) + .join(""), + ); + if (process.env.GITHUB_STEP_SUMMARY) + appendFileSync( + process.env.GITHUB_STEP_SUMMARY, + `### Scheduled qualification\n\n${decision.reason}.\n`, + ); + console.log(JSON.stringify(decision)); +} diff --git a/scripts/ci/required.mjs b/scripts/ci/required.mjs index 4f6586737..d48a5ec2c 100644 --- a/scripts/ci/required.mjs +++ b/scripts/ci/required.mjs @@ -1,10 +1,30 @@ +import { CI_FLAGS } from "./scopes.mjs"; +import { recordQualification } from "./qualification.mjs"; + const jobs = JSON.parse(process.env.CI_JOBS); const outputs = jobs.changes.outputs; -const required = ["changes", "static", "test"]; +const reused = outputs.tier === "reused"; +if ( + reused && + (process.env.GITHUB_EVENT_NAME !== "schedule" || + !/^[1-9]\d*$/.test(outputs.qualification_run_id ?? "") || + !Number.isSafeInteger(Number(outputs.qualification_run_id)) || + CI_FLAGS.some((flag) => outputs[flag] !== "false")) +) + throw new Error( + "Reused qualification requires a scheduled admission receipt and no selected lanes", + ); +if ( + outputs.tier === "full" && + CI_FLAGS.some((flag) => outputs[flag] !== "true") +) + throw new Error("Full qualification requires every canonical CI scope"); +const required = reused ? ["changes"] : ["changes", "static", "test"]; const lanes = { - code: ["build", "test-shard", "sqlite-runtime"], + code: ["build", "test-shard"], runtime: ["windows-native-build", "package-e2e"], package: ["windows-curated"], + sqlite: ["sqlite-runtime"], apple: ["apple-artifacts"], fixtures: ["conformance-fixtures"], readiness: ["reconstruction-readiness"], @@ -23,3 +43,9 @@ for (const [name, job] of Object.entries(jobs)) { if (failed.length) throw new Error(`CI lanes did not pass: ${failed.join(", ")}`); console.log("Every selected CI lane passed"); +if (reused) + console.log( + `Qualification reused from run ${outputs.qualification_run_id}; no fresh provider observation`, + ); +if (outputs.tier === "full" && process.env.CI_QUALIFICATION_DIRECTORY) + recordQualification(process.env.CI_QUALIFICATION_DIRECTORY); diff --git a/scripts/ci/scopes.mjs b/scripts/ci/scopes.mjs index 4493fe512..56391365c 100644 --- a/scripts/ci/scopes.mjs +++ b/scripts/ci/scopes.mjs @@ -85,6 +85,55 @@ export const SCOPES = { "tests/boundary/filesystem/javascriptArtifactContentIdentity.test.ts", "tests/boundary/process/sourceMapUncHosts.test.ts", ], + // Native SQLite runtime compatibility for the producer, public adapters, + // fixtures, shared lifecycle, and the package script that defines its suite. + sqlite: [ + "src/sqlite/**", + "src/domain/sqlite/**", + "src/contracts/sqlite/**", + "src/application/sqlite/**", + "src/composition/sqlite.ts", + "src/cli/sqliteCommands.ts", + "src/server/registerSqliteTools.ts", + "src/application/DirectAnalysisStatus*.ts", + "src/application/CapabilityInventory*.ts", + "src/application/AnalysisProvider*.ts", + "src/contracts/toolContracts.ts", + "src/contracts/toolContractTypes.ts", + "src/contracts/toolEffects*.ts", + "src/contracts/toolOutputSchemaPrimitives.ts", + "src/domain/analysisError*.ts", + "src/domain/evidence*.ts", + "src/domain/inputIssueProjection*.ts", + "src/domain/jsonSerialization*.ts", + "src/domain/jsonValue*.ts", + "src/domain/localPath*.ts", + "src/domain/providerAdapterError*.ts", + "src/domain/providerCleanupError*.ts", + "src/domain/result*.ts", + "src/process/OwnedCommand*.ts", + "src/process/ProviderProcess*.ts", + "src/process/PrivateRuntimeRoot*.ts", + "src/artifacts/ArtifactReader*.ts", + "src/artifacts/ArtifactResourceScope*.ts", + "src/artifacts/ArtifactPaths*.ts", + "src/artifacts/readStableArtifact*.ts", + "src/filesystem/OwnedFileHandle*.ts", + "src/server/EvidenceMcpServer*.ts", + "src/server/analysisAdmission*.ts", + "src/server/contractToolHandler*.ts", + "src/server/toolRegistrationOptions*.ts", + "src/server/createServer*.ts", + "src/cli.ts", + "src/cliCommandNames.ts", + "src/cliLogging.ts", + "src/cli/commandCancellation.ts", + "tests/boundary/sqlite/**", + "tests/fixtures/sqlite/**", + "tests/acceptance/sqliteDatabaseCli.test.ts", + "tests/boundary/mcp/serverLifecycleMcp.test.ts", + ".github/workflows/sqlite-runtime.yml", + ], browser: [ ".github/workflows/real-browser.yml", "scripts/verify-real-browser.mjs", @@ -312,6 +361,20 @@ export const SCOPES = { ], }; +export const CI_FLAGS = [ + ...new Set([ + "code", + "runtime", + "package", + "apple", + "fixtures", + "readiness", + "docs", + "workflows", + ...Object.keys(SCOPES), + ]), +]; + // Verifier entrypoints that only manual, licensed, or self-hosted workflows // run. No selected CI lane executes them, so their edits need only the static // checks and the source-test suite. Shared helpers under scripts/lib remain diff --git a/tests/boundary/filesystem/ciChangeScope.test.ts b/tests/boundary/filesystem/ciChangeScope.test.ts index 48fc19624..9a3f7515b 100644 --- a/tests/boundary/filesystem/ciChangeScope.test.ts +++ b/tests/boundary/filesystem/ciChangeScope.test.ts @@ -79,6 +79,7 @@ async function classify( head: string, event = "pull_request", full = false, + skipFull = false, ) { const output = join(directory, "scope-output"); await writeFile(output, ""); @@ -93,6 +94,7 @@ async function classify( HEAD_SHA: head, GITHUB_EVENT_NAME: event, CI_FULL: String(full), + CI_SKIP_FULL: String(skipFull), GITHUB_OUTPUT: output, }, }, @@ -100,6 +102,21 @@ async function classify( return z.record(z.string(), z.unknown()).parse(JSON.parse(result.stdout)); } +it("reuses only scheduled qualification and preserves every explicit full request", async () => { + const { directory, base } = await fixture(); + const reused = await classify(directory, base, base, "schedule", false, true); + expect(reused.tier).toBe("reused"); + const { CI_FLAGS } = await loadScopes(); + for (const flag of CI_FLAGS) expect(reused[flag]).toBe(false); + for (const event of ["workflow_dispatch", "pull_request", "schedule"]) + expect( + (await classify(directory, base, base, event, true, true)).tier, + ).toBe("full"); + expect( + (await classify(directory, base, base, "push", false, true)).tier, + ).toBe("fast"); +}); + it("keeps every translated root README and authored guide on the documentation lane", async () => { const { directory, base } = await fixture(); const readmes = (await readdir(new URL("../../../", import.meta.url))).filter( @@ -205,18 +222,8 @@ it.each(["", "not-a-commit"])( browser: true, reasons: expect.arrayContaining([expect.stringContaining("baseline")]), }); - const { SCOPES } = await loadScopes(); - for (const flag of [ - "code", - "runtime", - "package", - "fixtures", - "readiness", - "docs", - "workflows", - ...Object.keys(SCOPES), - ]) - expect(plan[flag]).toBe(true); + const { CI_FLAGS } = await loadScopes(); + for (const flag of CI_FLAGS) expect(plan[flag]).toBe(true); }, ); @@ -374,7 +381,14 @@ it("retains native Inspector verification on every documented package host", asy it.each([ [ "src/evm/client.ts", - { code: true, runtime: true, evm: true, browser: false, package: false }, + { + code: true, + runtime: true, + evm: true, + browser: false, + package: false, + sqlite: false, + }, ], [ "src/browser/execution/evaluate.ts", @@ -471,6 +485,38 @@ it.each([ "src/process/ProcessOwnership.test.ts", { code: true, package: true, apple: false, evm: false, browser: false }, ], + [ + "src/sqlite/SqliteDatabaseProvider.ts", + { code: true, sqlite: true, runtime: true, package: false }, + ], + [ + "src/application/sqlite/SqliteDatabaseService.test.ts", + { code: true, sqlite: true, runtime: false }, + ], + [ + "tests/boundary/sqlite/sqliteDatabaseProvider.test.ts", + { code: true, sqlite: true, runtime: false }, + ], + [ + "tests/fixtures/sqlite/database.ts", + { code: true, sqlite: true, runtime: false }, + ], + ["tests/acceptance/sqliteDatabaseCli.test.ts", { code: true, sqlite: true }], + [ + "tests/boundary/mcp/serverLifecycleMcp.test.ts", + { code: true, sqlite: true }, + ], + ["src/process/OwnedCommand.ts", { code: true, sqlite: true, runtime: true }], + ["src/server/createServer.ts", { code: true, sqlite: true, runtime: true }], + [ + "src/contracts/toolContracts.ts", + { code: true, sqlite: true, runtime: true }, + ], + [ + "src/domain/analysisErrorProjection.ts", + { code: true, sqlite: true, runtime: true }, + ], + ["src/domain/javascript/example.ts", { code: true, sqlite: false }], ])("selects the appropriate lanes for %s", async (path, expected) => { const { directory, base } = await fixture(); const head = await change(directory, [path]); @@ -513,6 +559,52 @@ it("scopes package script edits while retaining broad dependency checks", async }); }); +it.each(["test:sqlite", "test:sqlite:run"])( + "routes %s script changes to the SQLite runtime matrix", + async (script) => { + const { directory } = await fixture(); + const base = await change( + directory, + ["package.json"], + JSON.stringify({ scripts: { [script]: "vitest run" } }), + ); + const head = await change( + directory, + ["package.json"], + JSON.stringify({ + scripts: { [script]: "vitest run --reporter=default" }, + }), + ); + expect(await classify(directory, base, head)).toMatchObject({ + code: true, + sqlite: true, + evm: false, + package: false, + }); + }, +); + +it("keeps unrelated test script changes off the SQLite runtime matrix", async () => { + const { directory } = await fixture(); + const base = await change( + directory, + ["package.json"], + JSON.stringify({ scripts: { "test:domain": "vitest run src/domain" } }), + ); + const head = await change( + directory, + ["package.json"], + JSON.stringify({ + scripts: { "test:domain": "vitest run src/domain --reporter=default" }, + }), + ); + expect(await classify(directory, base, head)).toMatchObject({ + code: true, + sqlite: false, + evm: false, + }); +}); + it.each([ ["browser", "real-browser"], ["binary_layout", "real-binary-layout"], @@ -605,7 +697,10 @@ it.each([ "src/process/capture/example.test.ts", { code: true, runtime: false, browser: false, package: false }, ], - [".nvmrc", { code: true, package: true, browser: true, evm: true }], + [ + ".nvmrc", + { code: true, package: true, browser: true, evm: true, sqlite: true }, + ], [ "tsconfig.build.json", { code: true, package: true, browser: true, evm: true }, @@ -745,6 +840,7 @@ it.each(["workflow_dispatch", "schedule"])( tier: "full", code: true, package: true, + sqlite: true, evm: true, browser: true, }); @@ -762,6 +858,7 @@ it("honors ci:full for successive PR revisions", async () => { tier: "full", code: true, package: true, + sqlite: true, evm: true, browser: true, }); @@ -829,6 +926,7 @@ const workflowFile = async (name: string): Promise => ); const scopesSchema = z.object({ + CI_FLAGS: z.array(z.string()), SCOPES: z.record(z.string(), z.array(z.string())), MANUAL_VERIFIERS: z.array(z.string()), }); diff --git a/tests/boundary/filesystem/ciJobTimings.test.ts b/tests/boundary/filesystem/ciJobTimings.test.ts new file mode 100644 index 000000000..85a2ddd1f --- /dev/null +++ b/tests/boundary/filesystem/ciJobTimings.test.ts @@ -0,0 +1,243 @@ +import { chmod, mkdir, readFile, writeFile } from "node:fs/promises"; +import { delimiter, join } from "node:path"; +import { fileURLToPath } from "node:url"; + +import { expect, it } from "vitest"; + +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { + executeWorkflowFixture, + workflowGit, +} from "../../support/workflowGit.js"; + +const reporter = fileURLToPath( + new URL("../../../scripts/ci/job-timings.mjs", import.meta.url), +); + +it("merges paginated jobs, reports elapsed metrics, and preserves raw responses", async () => { + const fixture = await createFixture([ + { + total_count: 4, + jobs: [ + { + name: "Build package", + workflow_name: "CI", + status: "completed", + conclusion: "success", + created_at: "2026-10-13T00:00:00Z", + started_at: "2026-10-13T00:00:10Z", + completed_at: "2026-10-13T00:01:10Z", + steps: [ + { + name: "Install dependencies (npm ci)", + started_at: "2026-10-13T00:00:10Z", + completed_at: "2026-10-13T00:00:40Z", + }, + { + name: "Upload artifact", + started_at: "2026-10-13T00:00:40Z", + completed_at: "2026-10-13T00:00:41Z", + }, + ], + }, + ], + }, + { + total_count: 4, + jobs: [ + { + name: "Queued lane", + status: "queued", + created_at: "2026-10-13T00:00:00Z", + started_at: "2026-10-13T00:00:00Z", + completed_at: null, + steps: [], + }, + { + name: "Running lane", + status: "in_progress", + created_at: "2026-10-13T00:00:00Z", + started_at: "2026-10-13T00:00:15Z", + completed_at: null, + steps: [ + { + name: "Download artifacts", + started_at: "2026-10-13T00:00:20Z", + completed_at: "2026-10-13T00:00:22Z", + }, + ], + }, + { + name: "Skipped lane", + status: "completed", + conclusion: "skipped", + created_at: "2026-10-13T00:00:00Z", + started_at: "2026-10-13T00:00:00Z", + completed_at: "2026-10-13T00:00:00Z", + steps: [], + }, + ], + }, + ]); + const result = await runReporter(fixture); + const report = JSON.parse( + await readFile(join(fixture.outputDirectory, "report.json"), "utf8"), + ); + + expect(report.run.snapshot).toContain("partial"); + expect(report.counts).toMatchObject({ + jobs: 4, + completed: 2, + skipped: 1, + queueUnknown: 2, + executionUnknown: 3, + completedExecutionUnknown: 0, + }); + expect(report.aggregates.queueSeconds).toMatchObject({ + count: 2, + p50: 10, + p95: 15, + }); + expect(report.aggregates.completedExecutionSeconds).toMatchObject({ + count: 1, + p50: 60, + p95: 60, + }); + expect(report.aggregates.summedCompletedJobElapsedSeconds).toMatchObject({ + value: 60, + knownCount: 1, + unknownCount: 0, + label: expect.stringContaining("not CPU time"), + }); + expect(report.jobs[1].queue).toEqual({ + seconds: null, + reason: "job start is not established by its status", + }); + expect(report.jobs[3].queue).toEqual({ + seconds: null, + reason: "job was skipped", + }); + expect(report.jobs[3].execution).toEqual({ + seconds: null, + reason: "job was skipped", + }); + expect(report.stepCosts.npmCi[0].duration.seconds).toBe(30); + expect(report.stepCosts.artifactUpload[0].duration.seconds).toBe(1); + expect(report.stepCosts.artifactDownload[0].duration.seconds).toBe(2); + expect(report.stepCostInterpretation).toContain( + "not total artifact transfer costs", + ); + expect(report.cpuSeconds).toBeUndefined(); + expect(result.stdout).toContain("4 job timing records"); + expect( + await readFile(join(fixture.outputDirectory, "jobs.json"), "utf8"), + ).toBe(fixture.raw); + const summary = await readFile(fixture.summary, "utf8"); + expect(summary).toContain("point-in-time snapshot"); + expect(summary).toContain("unknown"); + expect(summary).toContain("not CPU time"); + expect(summary).toContain("60 s"); + expect(summary).toContain("not total artifact transfer costs"); + const args = JSON.parse(await readFile(fixture.args, "utf8")); + expect(args).toEqual([ + "api", + "--paginate", + "--slurp", + "repos/example/rea/actions/runs/42/attempts/2/jobs?per_page=100", + ]); +}); + +it("marks invalid and out-of-order timestamps unknown instead of emitting negative durations", async () => { + const fixture = await createFixture([ + { + jobs: [ + { + name: "Invalid dates", + status: "completed", + created_at: "not-a-date", + started_at: "2026-10-13T00:00:10Z", + completed_at: "2026-10-13T00:00:05Z", + steps: [], + }, + ], + }, + ]); + await runReporter(fixture); + const report = JSON.parse( + await readFile(join(fixture.outputDirectory, "report.json"), "utf8"), + ); + + expect(report.jobs[0].queue).toEqual({ + seconds: null, + reason: "invalid timestamp", + }); + expect(report.jobs[0].execution).toEqual({ + seconds: null, + reason: "timestamps out of order", + }); + expect(report.counts.completedExecutionUnknown).toBe(1); + expect(report.aggregates.summedCompletedJobElapsedSeconds).toMatchObject({ + value: null, + unknownCount: 1, + }); +}); + +it("fails on malformed pages or a failed GitHub API request", async () => { + const malformed = await createFixture([{ jobs: "not-an-array" }]); + await expect(runReporter(malformed)).rejects.toThrow(); + await expect( + readFile(join(malformed.outputDirectory, "jobs.json"), "utf8"), + ).resolves.toBe(malformed.raw); + + const failedApi = await createFixture([], { exitCode: 1 }); + await expect(runReporter(failedApi)).rejects.toThrow(); +}); + +async function createFixture( + pages: unknown[], + { exitCode = 0 }: { exitCode?: number } = {}, +) { + const directory = await createTestTempDirectory("rea-ci-job-timings-"); + await workflowGit(directory, ["init", "--initial-branch=main"]); + const fakeBin = join(directory, "bin"); + await mkdir(fakeBin); + const raw = JSON.stringify(pages); + const rawPath = join(directory, "gh-response.json"); + const args = join(directory, "gh-args.json"); + await writeFile(rawPath, raw); + const fakeGh = join(fakeBin, "gh"); + await writeFile( + fakeGh, + `#!/usr/bin/env node\nconst fs = require("node:fs");\nfs.writeFileSync(process.env.CI_GH_ARGS, JSON.stringify(process.argv.slice(2)));\nprocess.stdout.write(fs.readFileSync(process.env.CI_GH_RESPONSE, "utf8"));\nprocess.exitCode = ${String(exitCode)};\n`, + ); + await chmod(fakeGh, 0o755); + const outputDirectory = join(directory, ".cache", "ci-job-timings"); + const summary = join(directory, "summary.md"); + await writeFile(summary, ""); + return { + directory, + fakeBin, + raw, + rawPath, + args, + outputDirectory, + summary, + }; +} + +function runReporter(fixture: Awaited>) { + return executeWorkflowFixture(process.execPath, [reporter], { + cwd: fixture.directory, + env: { + ...process.env, + PATH: `${fixture.fakeBin}${delimiter}${process.env.PATH ?? ""}`, + CI_REPOSITORY: "example/rea", + CI_JOB_TIMINGS_DIRECTORY: fixture.outputDirectory, + CI_GH_ARGS: fixture.args, + CI_GH_RESPONSE: fixture.rawPath, + CI_TIMING_RUN_ID: "42", + CI_TIMING_RUN_ATTEMPT: "2", + GITHUB_STEP_SUMMARY: fixture.summary, + }, + }); +} diff --git a/tests/boundary/filesystem/ciQualification.test.ts b/tests/boundary/filesystem/ciQualification.test.ts new file mode 100644 index 000000000..b6a5a1e1a --- /dev/null +++ b/tests/boundary/filesystem/ciQualification.test.ts @@ -0,0 +1,375 @@ +import { chmod, mkdir, readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; + +import { expect, it } from "vitest"; +import { parse } from "yaml"; +import { z } from "zod"; + +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { + executeWorkflowFixture, + workflowGit, +} from "../../support/workflowGit.js"; + +const qualificationScript = fileURLToPath( + new URL("../../../scripts/ci/qualification.mjs", import.meta.url), +); +const requiredScript = fileURLToPath( + new URL("../../../scripts/ci/required.mjs", import.meta.url), +); +const now = Date.now(); +const validSha = "a".repeat(40); +const repository = "example/rea"; + +function artifact(overrides: Record = {}) { + return { + id: 42, + name: "ci-full-qualification-v1", + expired: false, + created_at: new Date(now - 60_000).toISOString(), + workflow_run: { + id: 7, + repository_id: 1, + head_repository_id: 1, + head_branch: "main", + head_sha: validSha, + }, + ...overrides, + }; +} + +function run(overrides: Record = {}) { + return { + id: 7, + status: "completed", + conclusion: "success", + path: ".github/workflows/ci.yml", + event: "schedule", + created_at: new Date(now - 120_000).toISOString(), + head_sha: validSha, + head_branch: "main", + head_repository: { full_name: repository }, + ...overrides, + }; +} + +async function admissionFixture( + candidate: Record = artifact(), + candidateRun: Record = run(), + options: { apiError?: boolean; malformed?: boolean; force?: boolean } = {}, +) { + const directory = await createTestTempDirectory("rea-ci-qualification-"); + const bin = join(directory, "bin"); + await mkdir(bin); + const fixturePath = join(directory, "api.json"); + await writeFile( + fixturePath, + options.malformed + ? "not-json" + : JSON.stringify({ artifacts: [candidate], run: candidateRun }), + ); + const gh = join(bin, "gh"); + await writeFile( + gh, + [ + "#!/bin/sh", + 'if [ "$GH_API_ERROR" = "true" ]; then exit 1; fi', + 'case "$*" in', + ' *actions/artifacts*) cat "$GH_FIXTURE_RESPONSE" ;;', + " *actions/runs/7*) node -e 'const x=require(process.env.GH_FIXTURE_RESPONSE); process.stdout.write(JSON.stringify(x.run))' ;;", + " *) exit 2 ;;", + "esac", + "", + ].join("\n"), + ); + await chmod(gh, 0o755); + const output = join(directory, "github-output"); + await writeFile(output, ""); + const env = { + ...process.env, + PATH: `${bin}:${process.env.PATH ?? ""}`, + GH_FIXTURE_RESPONSE: fixturePath, + GH_API_ERROR: String(options.apiError ?? false), + CI_REPOSITORY: repository, + CI_REPOSITORY_ID: "1", + CI_DEFAULT_BRANCH: "main", + HEAD_SHA: validSha, + GITHUB_RUN_ID: "100", + GITHUB_OUTPUT: output, + ...(options.force ? { CI_FORCE_FULL: "true" } : {}), + }; + return { directory, output, env }; +} + +async function decide( + candidate?: Record, + candidateRun?: Record, + options?: { apiError?: boolean; malformed?: boolean; force?: boolean }, +) { + const fixture = await admissionFixture(candidate, candidateRun, options); + const result = await executeWorkflowFixture( + process.execPath, + [qualificationScript], + { + cwd: fixture.directory, + env: fixture.env, + }, + ); + return { + ...fixture, + decision: z + .record(z.string(), z.unknown()) + .parse(JSON.parse(result.stdout)), + outputText: await readFile(fixture.output, "utf8"), + }; +} + +it("reuses only a recent trusted full qualification for the exact default-branch SHA", async () => { + const { decision, outputText } = await decide(); + expect(decision).toMatchObject({ run_full: false, run_id: 7 }); + expect(Object.keys(decision).sort()).toEqual([ + "reason", + "run_full", + "run_id", + ]); + expect(outputText).toContain("run_full=false\n"); + expect(outputText).toContain("run_id=7\n"); +}); + +const rejectedCandidates: Array< + [string, () => Record, () => Record] +> = [ + [ + "old", + () => + artifact({ + created_at: new Date(now - 24 * 60 * 60 * 1000 - 1).toISOString(), + }), + run, + ], + [ + "future", + () => artifact({ created_at: new Date(now + 60_000).toISOString() }), + run, + ], + ["expired", () => artifact({ expired: true }), run], + [ + "foreign repository", + () => + artifact({ + workflow_run: { ...artifact().workflow_run, repository_id: 2 }, + }), + run, + ], + [ + "foreign branch", + () => + artifact({ + workflow_run: { ...artifact().workflow_run, head_branch: "other" }, + }), + run, + ], + [ + "foreign SHA", + () => + artifact({ + workflow_run: { ...artifact().workflow_run, head_sha: "b".repeat(40) }, + }), + run, + ], + [ + "untrusted workflow", + artifact, + () => run({ path: ".github/workflows/release.yml" }), + ], + ["failed run", artifact, () => run({ conclusion: "failure" })], + [ + "old producer rerun with a fresh artifact", + artifact, + () => + run({ created_at: new Date(now - 25 * 60 * 60 * 1000).toISOString() }), + ], + ["wrong workflow event", artifact, () => run({ event: "push" })], +]; + +it.each(rejectedCandidates)( + "runs full qualification for a %s candidate", + async (_name, makeArtifact, makeRun) => { + const { decision } = await decide(makeArtifact(), makeRun()); + expect(decision.run_full).toBe(true); + }, +); + +it.each([ + ["API error", { apiError: true }], + ["malformed query response", { malformed: true }], +] as const)("fails closed on %s", async (_name, options) => { + const { decision } = await decide(undefined, undefined, options); + expect(decision.run_full).toBe(true); +}); + +it("lets the daily force-full setting bypass the GitHub API", async () => { + const { decision } = await decide(undefined, undefined, { + force: true, + apiError: true, + }); + expect(decision).toMatchObject({ run_full: true }); + expect(decision.reason).toContain("Forced"); +}); + +async function workflowContract() { + const workflowPath = new URL( + "../../../.github/workflows/ci.yml", + import.meta.url, + ); + const parsed = parse(await readFile(workflowPath, "utf8")); + return z + .object({ + jobs: z.record( + z.string(), + z.object({ needs: z.unknown().optional() }).passthrough(), + ), + }) + .parse(parsed).jobs; +} + +async function requiredRun( + changes: Record, + overrides: { failed?: string; runAttempt?: string; event?: string } = {}, +) { + const directory = await createTestTempDirectory("rea-ci-receipt-"); + await workflowGit(directory, ["init", "--initial-branch=main"]); + await writeFile(join(directory, "fixture.txt"), "qualified source\n"); + await workflowGit(directory, ["add", "."]); + await workflowGit(directory, ["commit", "-m", "fixture source"]); + const head = await workflowGit(directory, ["rev-parse", "HEAD"]); + const jobs = await workflowContract(); + const results: Record< + string, + { result: string; outputs?: Record } + > = {}; + for (const name of z.array(z.string()).parse(jobs.required?.needs)) { + results[name] = { + result: + name === overrides.failed + ? "failure" + : changes.tier === "reused" + ? "skipped" + : "success", + }; + } + results.changes = { result: "success", outputs: changes }; + const receiptDirectory = join(directory, ".cache", "receipt"); + const output = join(directory, "github-output"); + await writeFile(output, ""); + const env = { + ...process.env, + CI_JOBS: JSON.stringify(results), + CI_QUALIFICATION_DIRECTORY: receiptDirectory, + GITHUB_EVENT_NAME: overrides.event ?? "schedule", + GITHUB_SHA: head, + GITHUB_RUN_ID: "100", + GITHUB_RUN_ATTEMPT: overrides.runAttempt ?? "1", + GITHUB_REPOSITORY: repository, + GITHUB_OUTPUT: output, + }; + const result = executeWorkflowFixture(process.execPath, [requiredScript], { + cwd: directory, + env, + }); + return { directory, head, output, result }; +} + +it("writes a full receipt only after every canonical flag and workflow need pass", async () => { + const scopesPath = new URL("../../../scripts/ci/scopes.mjs", import.meta.url); + const { CI_FLAGS } = await import(scopesPath.href); + const changes = Object.fromEntries( + CI_FLAGS.map((flag: string) => [flag, "true"]), + ); + changes.tier = "full"; + const { directory, head, output, result } = await requiredRun(changes); + await expect(result).resolves.toMatchObject({ stderr: "" }); + const receiptPath = join(directory, ".cache", "receipt", "receipt.json"); + const receipt = z + .object({ + sourceSha: z.string(), + runId: z.number(), + runAttempt: z.number(), + }) + .parse(JSON.parse(await readFile(receiptPath, "utf8"))); + expect(receipt).toMatchObject({ sourceSha: head, runId: 100, runAttempt: 1 }); + expect(await readFile(output, "utf8")).toContain( + `receipt_path=${receiptPath}\n`, + ); + expect(await readFile(output, "utf8")).toContain( + "receipt_name=ci-full-qualification-v1\n", + ); +}); + +it.each([ + ["failed selected lane", { failed: "sqlite-runtime" }], + ["partial full scope", {}], +] as const)( + "does not issue a receipt after %s", + async (caseName, overrides) => { + const scopesPath = new URL( + "../../../scripts/ci/scopes.mjs", + import.meta.url, + ); + const { CI_FLAGS } = await import(scopesPath.href); + const changes = Object.fromEntries( + CI_FLAGS.map((flag: string) => [flag, "true"]), + ); + changes.tier = "full"; + if (caseName === "partial full scope") changes.sqlite = "false"; + const outcome = await requiredRun(changes, overrides); + await expect(outcome.result).rejects.toMatchObject({ code: 1 }); + await expect( + readFile(join(outcome.directory, ".cache", "receipt", "receipt.json")), + ).rejects.toMatchObject({ code: "ENOENT" }); + expect(await readFile(outcome.output, "utf8")).not.toContain("receipt_"); + }, +); + +it("never writes a fresh receipt for a reused scheduled qualification", async () => { + const scopesPath = new URL("../../../scripts/ci/scopes.mjs", import.meta.url); + const { CI_FLAGS } = await import(scopesPath.href); + const changes = Object.fromEntries( + CI_FLAGS.map((flag: string) => [flag, "false"]), + ); + changes.tier = "reused"; + changes.qualification_run_id = "7"; + const outcome = await requiredRun(changes); + const result = await outcome.result; + await expect( + readFile(join(outcome.directory, ".cache", "receipt", "receipt.json")), + ).rejects.toMatchObject({ code: "ENOENT" }); + expect(result.stdout).toContain("Qualification reused from run 7"); + expect(await readFile(outcome.output, "utf8")).not.toContain("receipt_"); +}); + +it.each([ + ["missing receipt run", { qualification_run_id: "" }, "schedule"], + ["selected validation lane", { sqlite: "true" }, "schedule"], + ["manual dispatch", {}, "workflow_dispatch"], +] as const)( + "rejects qualification reuse with %s", + async (_name, override, event) => { + const { CI_FLAGS } = await import( + new URL("../../../scripts/ci/scopes.mjs", import.meta.url).href + ); + const changes: Record = { + ...Object.fromEntries(CI_FLAGS.map((flag: string) => [flag, "false"])), + tier: "reused", + qualification_run_id: "7", + ...override, + }; + const outcome = await requiredRun(changes, { event }); + await expect(outcome.result).rejects.toMatchObject({ + code: 1, + stderr: expect.stringContaining("scheduled admission receipt"), + }); + expect(await readFile(outcome.output, "utf8")).not.toContain("receipt_"); + }, +); diff --git a/tests/boundary/filesystem/ciRequiredLanes.test.ts b/tests/boundary/filesystem/ciRequiredLanes.test.ts index 0e9457e1c..a86392e89 100644 --- a/tests/boundary/filesystem/ciRequiredLanes.test.ts +++ b/tests/boundary/filesystem/ciRequiredLanes.test.ts @@ -27,6 +27,9 @@ const workflow = async (filename = "ci.yml") => ); it("binds SQLite compatibility jobs to the build's prepared runtime artifact and root layout", async () => { + expect((await workflow()).jobs["sqlite-runtime"]?.if).toBe( + "needs.changes.outputs.sqlite == 'true'", + ); const steps = z.array( z.object({ uses: z.string().optional(), @@ -52,6 +55,91 @@ it("binds SQLite compatibility jobs to the build's prepared runtime artifact and expect(downloads[0]?.with).toEqual({ name, path: "." }); }); +it("exports every canonical scope with read authority for admission and keeps telemetry outside the gate", async () => { + const { jobs } = await workflow(); + const { CI_FLAGS } = z + .object({ CI_FLAGS: z.array(z.string()) }) + .parse( + await import( + new URL("../../../scripts/ci/scopes.mjs", import.meta.url).href + ), + ); + const outputs = z.record(z.string(), z.string()).parse(jobs.changes?.outputs); + expect(jobs.changes?.permissions).toEqual({ + contents: "read", + actions: "read", + }); + expect(jobs.build?.permissions).toEqual({ + contents: "read", + actions: "read", + }); + for (const flag of CI_FLAGS) + expect(outputs[flag]).toBe(`\${{ steps.plan.outputs.${flag} }}`); + const steps = z + .array( + z.object({ + run: z.string().optional(), + if: z.string().optional(), + }), + ) + .parse(jobs.required?.steps); + const telemetry = steps.find((step) => + step.run?.includes("scripts/ci/job-timings.mjs"), + ); + expect(telemetry).toBeUndefined(); + const timings = z + .object({ + on: z.object({ + workflow_run: z.object({ + workflows: z.array(z.string()), + types: z.array(z.string()), + }), + }), + permissions: z.record(z.string(), z.string()), + jobs: z.object({ + report: z.object({ + steps: z.array( + z.object({ + uses: z.string().optional(), + run: z.string().optional(), + with: z.record(z.string(), z.unknown()).optional(), + env: z.record(z.string(), z.unknown()).optional(), + }), + ), + }), + }), + }) + .parse( + parse( + await readFile( + new URL("../../../.github/workflows/ci-timings.yml", import.meta.url), + "utf8", + ), + ), + ); + expect(timings.on.workflow_run).toEqual({ + workflows: ["CI", "Release"], + types: ["completed"], + }); + expect(timings.permissions).toEqual({ contents: "read", actions: "read" }); + const checkout = timings.jobs.report.steps.find((step) => + step.uses?.startsWith("actions/checkout@"), + ); + expect(checkout?.with).toEqual({ + ref: "${{ github.event.repository.default_branch }}", + "persist-credentials": false, + }); + const report = timings.jobs.report.steps.find((step) => + step.run?.includes("scripts/ci/job-timings.mjs"), + ); + expect(report?.env?.CI_TIMING_RUN_ID).toBe( + "${{ github.event.workflow_run.id }}", + ); + expect(report?.env?.CI_TIMING_RUN_ATTEMPT).toBe( + "${{ github.event.workflow_run.run_attempt }}", + ); +}); + const requiredNeeds = async () => { const { jobs } = await workflow(); return { jobs, needs: z.array(z.string()).parse(jobs.required?.needs) }; @@ -131,7 +219,10 @@ it("keeps fast feedback separate from complete qualification and release proof", ), ), ); - expect(ci.on.schedule).toEqual([{ cron: "23 * * * *" }]); + expect(ci.on.schedule).toEqual([ + { cron: "23 0-2,4-23 * * *" }, + { cron: "23 3 * * *" }, + ]); expect(ci.concurrency.group).toContain("'qualification' || 'feedback'"); expect(ci.concurrency.group).toContain("inputs.source_sha ||"); for (const fullEvent of ["schedule", "workflow_dispatch"]) @@ -164,9 +255,10 @@ it.each([ ["true", "skipped", false], ["true", "missing", false], ["false", "skipped", true], + ["false", "failure", false], ] as const)( - "enforces the SQLite required gate for code=%s and result=%s", - async (code, result, passes) => { + "enforces the SQLite required gate for sqlite=%s and result=%s", + async (sqlite, result, passes) => { const workflow = await requiredNeeds(); // GitHub exposes only direct dependencies in the required job's needs context. const needs: Record< @@ -175,7 +267,7 @@ it.each([ > = Object.fromEntries( workflow.needs.map((name) => [name, { result: "success" }]), ); - needs.changes = { result: "success", outputs: { code } }; + needs.changes = { result: "success", outputs: { code: "true", sqlite } }; if (result === "missing") delete needs["sqlite-runtime"]; else if ("sqlite-runtime" in needs) needs["sqlite-runtime"] = { result }; const execution = executeWorkflowFixture( From 9928f357fb9c420e6384e155d74586dfe37d4533 Mon Sep 17 00:00:00 2001 From: Chaoqi <136579807+Chaoqi31@users.noreply.github.com> Date: Tue, 13 Oct 2026 06:05:20 -0400 Subject: [PATCH 086/159] fix(artifacts): read ASCII control bytes in legacy ZIP names as ASCII (#2365) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(artifacts): read ASCII control bytes in legacy ZIP names as ASCII ZIP names without the UTF-8 flag were decoded with zip.js's CP437 table, which maps bytes 0x01-0x1F and 0x7F to IBM PC display glyphs. A folder zipped on macOS with a custom icon therefore listed and extracted `Folder/Icon\r` as `Folder/Icon♪`, a name that is not in the archive and differs from REA's directory inventory of the same folder. Tabs became `○` and DEL became `⌂`. Decode legacy names in REA's existing decodeText hook with the Unicode CP437 mapping that Python's zipfile also uses: bytes below 0x80 as ASCII, and the upper half with the same table zip.js uses. Windows file names cannot contain U+0001-U+001F, so a Windows extraction destination now refuses such a path with a typed path failure before creating anything, as it already does for ':'. * test(artifacts): check extracted legacy ZIP control names on disk The control-name cases checked only the logical path that inventory reports. Extraction writes through SafeOutputTree, so a change that renamed the extracted files would still pass those cases. Extract `Icon\r` and `é\t\u007F.a`, then compare the reported relative_path, the directory listing, and the file contents. The new cases skip on Windows, which refuses these names. safeOutputTree.test.ts covers that refusal. --- src/artifacts/SafeOutputTree.ts | 5 +++ src/artifacts/ZipArtifactReader.ts | 23 +++++++++- .../filesystem/artifactZipIntegrity.test.ts | 43 +++++++++++++++++++ .../filesystem/safeOutputTree.test.ts | 24 +++++++++++ 4 files changed, 93 insertions(+), 2 deletions(-) diff --git a/src/artifacts/SafeOutputTree.ts b/src/artifacts/SafeOutputTree.ts index 8fa40876d..73885ad05 100644 --- a/src/artifacts/SafeOutputTree.ts +++ b/src/artifacts/SafeOutputTree.ts @@ -585,6 +585,11 @@ const normalizeDestinationPath = ( "path", `Windows destination cannot materialize artifact path as a regular file: ${path}; ':' denotes alternate data stream syntax.`, ); + if (platform === "win32" && /[\u0001-\u001f]/u.test(path)) + throw new ArtifactReaderFailure( + "path", + `Windows destination cannot materialize artifact path: ${JSON.stringify(path)}; Windows names cannot contain U+0001-U+001F.`, + ); return path; }; diff --git a/src/artifacts/ZipArtifactReader.ts b/src/artifacts/ZipArtifactReader.ts index 11c68557b..dfbee8b27 100644 --- a/src/artifacts/ZipArtifactReader.ts +++ b/src/artifacts/ZipArtifactReader.ts @@ -255,10 +255,29 @@ const decodeZipEntryText = ( encoding: string, type: "filename" | "comment", ): string | undefined => { - if (type !== "filename" || encoding !== "utf-8") return undefined; - return decodeStrictZipUtf8(bytes, "ZIP filename is not valid UTF-8"); + if (type !== "filename") return undefined; + if (encoding === "utf-8") + return decodeStrictZipUtf8(bytes, "ZIP filename is not valid UTF-8"); + return encoding === "cp437" ? decodeCp437(bytes) : undefined; }; +/** CP437 characters for bytes 0x80-0xFF. */ +const CP437_HIGH_HALF = + "ÇüéâäàåçêëèïîìÄÅÉæÆôöòûùÿÖÜ¢£¥₧ƒáíóúñѪº¿⌐¬½¼¡«»░▒▓│┤╡╢╖╕╣║╗╝╜╛┐└┴┬├─┼╞╟╚╔╩╦╠═╬╧╨╤╥╙╘╒╓╫╪┘┌█▄▌▐▀αßΓπΣσµτΦΘΩδ∞φε∩≡±≥≤⌠⌡÷≈°∙·√ⁿ²■\u00A0"; + +/** + * Decode a legacy ZIP name with the Unicode CP437 mapping, which Python's + * zipfile also uses: bytes below 0x80 are ASCII. zip.js shows 0x01-0x1F and + * 0x7F as IBM PC display glyphs, which turns the macOS folder icon file + * `Icon\r` into `Icon♪`. + */ +const decodeCp437 = (bytes: Uint8Array): string => + Array.from(bytes, (byte) => + byte < 0x80 + ? String.fromCharCode(byte) + : CP437_HIGH_HALF.charAt(byte - 0x80), + ).join(""); + const zipEntryPath = (entry: Entry): string => { const field = entry.extraFieldUnicodePath; if (field?.valid !== true) return entry.filename; diff --git a/tests/boundary/filesystem/artifactZipIntegrity.test.ts b/tests/boundary/filesystem/artifactZipIntegrity.test.ts index 0bb475518..41397c520 100644 --- a/tests/boundary/filesystem/artifactZipIntegrity.test.ts +++ b/tests/boundary/filesystem/artifactZipIntegrity.test.ts @@ -38,8 +38,19 @@ it("completes an empty ZIP extraction with an empty identity", async () => { expect(await readdir(output)).toEqual([]); }); +const legacyControlNames = [ + ["legacy ASCII control", Buffer.from("Icon\r"), false, "Icon\r"], + [ + "legacy CP437 control", + Buffer.from([0x82, 0x09, 0x7f, 0x2e, 0x61]), + false, + "é\t\u007F.a", + ], +] as const; + it.each([ ["legacy CP437", Buffer.from([0x82, 0x2e, 0x74, 0x78, 0x74]), false, "é.txt"], + ...legacyControlNames, [ "EFS with leading U+FEFF", Buffer.from([0xef, 0xbb, 0xbf, 0x61, 0x62]), @@ -68,6 +79,38 @@ it.each([ }, ); +it.skipIf(process.platform === "win32").each(legacyControlNames)( + "extracts %s ZIP members under their stored names", + async (_label, rawName, utf8Flag, logicalPath) => { + const root = await createTestTempDirectory("rea-zip-extracted-name-"); + const path = join(root, "names.zip"); + await writeSingleEntryZip(path, "x.txt"); + const archive = await readFile(path); + await writeFile( + path, + rewriteSingleEntryName(archive, "x.txt", rawName, utf8Flag), + ); + const target = await parseBinaryTarget(path); + if (!target.ok) throw target.error; + + const output = join(root, "output"); + const result = await withArtifactClient(target.value, (client) => + client.execute("extract_artifact", { output_root: output }), + ); + if (!result.ok) throw result.error; + const extraction = artifactExtractionResultSchema.parse( + result.value.result, + ); + expect( + extraction.artifacts.map(({ relative_path }) => relative_path), + ).toEqual([logicalPath]); + expect(await readdir(output)).toEqual([logicalPath]); + expect(await readFile(join(output, logicalPath), "utf8")).toBe( + "member bytes", + ); + }, +); + it("preserves valid Info-ZIP Unicode Path overrides", async () => { const root = await createTestTempDirectory("rea-zip-unicode-path-"); const path = join(root, "unicode-path.zip"); diff --git a/tests/boundary/filesystem/safeOutputTree.test.ts b/tests/boundary/filesystem/safeOutputTree.test.ts index f67704456..3e82a62ab 100644 --- a/tests/boundary/filesystem/safeOutputTree.test.ts +++ b/tests/boundary/filesystem/safeOutputTree.test.ts @@ -489,4 +489,28 @@ describe("safe artifact output Windows stream paths", () => { await expect(access(output)).rejects.toMatchObject({ code: "ENOENT" }); expect(await readdir(parent)).toEqual([]); }); + + it.each(["Folder/Icon\r", "Fold\ter/file.txt"])( + "refuses Windows control characters before creating %j", + async (path) => { + const parent = await createTestTempDirectory("rea-safe-output-control-"); + const output = join(parent, "published"); + const tree = await SafeOutputTree.create(output, "win32"); + const bytes = Buffer.from("regular file content"); + await expect( + tree.write(path, Readable.from(bytes), { + sha256: createHash("sha256").update(bytes).digest("hex"), + bytes: bytes.byteLength, + }), + ).rejects.toMatchObject({ + reason: "path", + message: `Windows destination cannot materialize artifact path: ${JSON.stringify(path)}; Windows names cannot contain U+0001-U+001F.`, + }); + expect(await readdir(output)).toEqual([]); + expect(await tree.rollback()).toEqual({ + status: "complete", + residualPaths: [], + }); + }, + ); }); From ab02f3c8d665ead35391c94db74cd0fb0cfed0ed Mon Sep 17 00:00:00 2001 From: morluto <76467478+morluto@users.noreply.github.com> Date: Tue, 13 Oct 2026 18:16:10 +0800 Subject: [PATCH 087/159] ci: reduce reporter fan-out and preserve shard history --- .github/workflows/ci-timings.yml | 21 +- .github/workflows/ci.yml | 24 +- docs/testing.md | 29 +- scripts/ci/job-timings-batch.mjs | 268 ++++++++++++++++ scripts/ci/job-timings.mjs | 5 +- scripts/ci/required.mjs | 1 + scripts/ci/test-history.mjs | 114 +++++++ .../filesystem/ciJobTimingsBatch.test.ts | 288 ++++++++++++++++++ .../filesystem/ciRequiredLanes.test.ts | 98 +++--- .../boundary/filesystem/ciTestHistory.test.ts | 188 ++++++++++++ .../boundary/filesystem/ciTestReports.test.ts | 14 +- 11 files changed, 968 insertions(+), 82 deletions(-) create mode 100644 scripts/ci/job-timings-batch.mjs create mode 100644 scripts/ci/test-history.mjs create mode 100644 tests/boundary/filesystem/ciJobTimingsBatch.test.ts create mode 100644 tests/boundary/filesystem/ciTestHistory.test.ts diff --git a/.github/workflows/ci-timings.yml b/.github/workflows/ci-timings.yml index 70ef1ebc7..30fa19464 100644 --- a/.github/workflows/ci-timings.yml +++ b/.github/workflows/ci-timings.yml @@ -1,9 +1,13 @@ name: CI timings on: - workflow_run: - workflows: [CI, Release] - types: [completed] + schedule: + - cron: "17 * * * *" + workflow_dispatch: + +concurrency: + group: ci-timings-${{ github.repository }} + cancel-in-progress: false permissions: contents: read @@ -11,12 +15,11 @@ permissions: jobs: report: - name: Report completed workflow attempt + name: Collect recent completed workflow attempts runs-on: ubuntu-latest timeout-minutes: 5 steps: - # Completed PR runs can contain untrusted changes. Execute only the - # reporter from the trusted default branch; consume jobs as API data. + # Consume only API data and execute batch code from the trusted default branch. - uses: actions/checkout@v4 with: ref: ${{ github.event.repository.default_branch }} @@ -29,15 +32,13 @@ jobs: env: GH_TOKEN: ${{ github.token }} CI_REPOSITORY: ${{ github.repository }} - CI_TIMING_RUN_ID: ${{ github.event.workflow_run.id }} - CI_TIMING_RUN_ATTEMPT: ${{ github.event.workflow_run.run_attempt }} - run: node scripts/ci/job-timings.mjs + run: node scripts/ci/job-timings-batch.mjs - name: Upload CI timing artifacts if: always() && hashFiles('.cache/ci-job-timings/*') != '' continue-on-error: true uses: actions/upload-artifact@v4 with: - name: ci-job-timings + name: ci-job-timings-batch path: .cache/ci-job-timings/ include-hidden-files: true if-no-files-found: error diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 39ebf2e86..7faf9e4ee 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -136,6 +136,9 @@ jobs: node-version-file: .nvmrc cache: npm - run: npm ci + - name: Validate workflow syntax and expressions + if: needs.changes.outputs.workflows == 'true' + run: go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 -shellcheck= -pyflakes= - if: needs.changes.outputs.code == 'true' run: npm run check:ci - if: needs.changes.outputs.code == 'true' @@ -215,19 +218,15 @@ jobs: with: save: ${{ github.event_name != 'pull_request' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }} - run: npm run test:prepare - - name: Find successful default-branch timing history + - name: Find available trusted full-suite timing history id: history continue-on-error: true env: GH_TOKEN: ${{ github.token }} CI_REPOSITORY: ${{ github.repository }} + CI_REPOSITORY_ID: ${{ github.repository_id }} CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} - run: | - run_id="$(gh api --method GET \ - "repos/${CI_REPOSITORY}/actions/workflows/ci.yml/runs" \ - -f branch="${CI_DEFAULT_BRANCH}" -f event=schedule -f status=success -f per_page=1 \ - --jq '.workflow_runs[0].id // empty')" - echo "run-id=${run_id}" >> "${GITHUB_OUTPUT}" + run: node scripts/ci/test-history.mjs - name: Download previous timing report if: steps.history.outputs.run-id != '' continue-on-error: true @@ -670,16 +669,6 @@ jobs: skills/reverse-engineer-anything if-no-files-found: error - workflows: - name: Validate workflows - needs: changes - if: needs.changes.outputs.workflows == 'true' - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - name: Validate workflow syntax and expressions - run: go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 -shellcheck= -pyflakes= - browser: needs: changes if: needs.changes.outputs.browser == 'true' @@ -749,7 +738,6 @@ jobs: reconstruction-readiness, windows-curated, docs, - workflows, browser, binary_layout, evm, diff --git a/docs/testing.md b/docs/testing.md index 0203abade..4e79c3fe7 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -274,18 +274,26 @@ checkout-relative paths; the build job downloads that artifact for the sequencer instead of the coverage/JUnit bundle. Missing history falls back to native sharding. Static checks, documentation, build, package verification, and real-system lanes remain separate jobs so one kind of evidence cannot stand in -for another. +for another. Workflow syntax and expression validation runs inside the required +static-check job when workflow changes select it, avoiding a separate runner. -The separate `CI timings` workflow runs after CI or Release completes and uploads -`ci-job-timings` with raw API pages and a normalized point-in-time report for that -workflow attempt. It runs the reporter from the trusted default branch with -read-only permissions, treating the triggering run as API data. Its summary separates queue -waiting (`created_at` to `started_at`) from completed execution and reports +The separate `CI timings` workflow batches recent completed CI and Release runs +hourly or on manual dispatch. It uploads `ci-job-timings-batch` with raw API pages and +a normalized point-in-time report for each collected workflow attempt, plus a +batch manifest describing examined coverage, failures, and limits. Overlapping +lookback windows may collect an attempt again; identify records by run ID and +attempt. Each batch examines at most 100 completed runs per workflow created in +the preceding 24 hours, with a four-minute API collection budget. It collects the +latest attempt of each selected run and does not claim an exhaustive archive. +It runs the reporter from the trusted default branch with read-only permissions, +treating workflow runs as API data. It does not allocate a new reporter job for +every completed, cancelled, or skipped workflow. Reports separate queue +waiting (`created_at` to `started_at`) from completed execution and report percentiles plus matched `npm ci` and artifact-step durations. Queue excludes dependency waiting before job creation and does not establish whether account concurrency or runner availability caused the wait. Queued jobs with placeholder start times, skipped jobs, and missing or invalid timestamps remain -unknown. The reporter targets the triggering run and attempt explicitly. +unknown. The reporter targets each selected run and attempt explicitly. Summed job elapsed time is parallel work, not workflow wall time, CPU time, memory usage, or billing. Step names identify samples, not complete transfer costs. Its queue, execution, and failures are independent of the required gate @@ -863,8 +871,11 @@ catalog, and managed-conformance metadata. Each shard still installs the locked dependencies and runs its assigned tests; full qualification enables coverage. It no longer regenerates the same metadata independently. -The build job also retrieves the merged timing report from the last successful -default-branch scheduled qualification and includes that same report in each shard's snapshot. +The build job also finds an available `test-timings` artifact from a successful +trusted default-branch scheduled or manual qualification and includes that same +report in each shard's snapshot. The bounded lookup verifies the artifact's +producer; newer schedules that reuse qualification without producing timings do +not hide the last full run's artifact. CI assigns discovered files by historical duration, with a deterministic fallback weight for new files, rather than dividing only by file count. Every discovered file still belongs to exactly one shard; worker limits and project diff --git a/scripts/ci/job-timings-batch.mjs b/scripts/ci/job-timings-batch.mjs new file mode 100644 index 000000000..842e884f0 --- /dev/null +++ b/scripts/ci/job-timings-batch.mjs @@ -0,0 +1,268 @@ +import { execFile } from "node:child_process"; +import { appendFile, mkdir, writeFile } from "node:fs/promises"; +import { join, resolve } from "node:path"; +import { promisify } from "node:util"; +import { fileURLToPath } from "node:url"; + +import { collectJobTimings } from "./job-timings.mjs"; + +const execFileAsync = promisify(execFile); +const OUTPUT_DIRECTORY = join(".cache", "ci-job-timings"); +const MAX_BUFFER = 16 * 1024 * 1024; +const API_TIMEOUT_MS = 30_000; +const WORKFLOWS = [ + { id: "ci.yml", name: "CI" }, + { id: "release.yml", name: "Release" }, +]; +const LOOKBACK_MS = 24 * 60 * 60 * 1_000; +const MAX_RUNS_PER_WORKFLOW = 100; +const TIME_BUDGET_MS = 240_000; + +/** Collect a bounded recent sample of completed CI and Release attempts. */ +export async function collectJobTimingsBatch({ + repository, + outputDirectory = OUTPUT_DIRECTORY, + observedAt = new Date().toISOString(), + env = process.env, + now = Date.now, + collect = collectJobTimings, +} = {}) { + const identity = validateRepository(repository); + const started = now(); + const cutoff = new Date(started - LOOKBACK_MS).toISOString(); + const manifest = { + schemaVersion: 1, + observedAt, + repository: identity, + selection: { + workflows: WORKFLOWS, + status: "completed", + createdAfter: cutoff, + lookbackHours: 24, + overlap: "Each hourly run re-reads the preceding 24 hours.", + pagination: + "Run listings use the first 100; jobs are paginated within the remaining time and 16 MiB response limits.", + maxRunsPerWorkflow: MAX_RUNS_PER_WORKFLOW, + timeBudgetMs: TIME_BUDGET_MS, + exhaustive: false, + }, + coverage: { + workflowsQueried: [], + runsListed: 0, + selectedRuns: 0, + collectedRuns: 0, + }, + truncation: [], + errors: [], + runs: [], + elapsedMs: 0, + }; + const budgetExpired = () => now() - started >= TIME_BUDGET_MS; + + const selected = []; + for (const workflow of WORKFLOWS) { + if (budgetExpired()) { + manifest.truncation.push("Time budget reached while listing workflows."); + break; + } + try { + const response = await apiJson( + `repos/${identity}/actions/workflows/${workflow.id}/runs?status=completed&created=%3E%3D${encodeURIComponent(cutoff)}&per_page=${String(MAX_RUNS_PER_WORKFLOW)}&page=1`, + env, + started, + now, + ); + if (!Array.isArray(response.workflow_runs)) + throw new Error("run listing must contain a workflow_runs array"); + manifest.coverage.workflowsQueried.push(workflow.name); + manifest.coverage.runsListed += response.workflow_runs.length; + if (response.workflow_runs.length >= MAX_RUNS_PER_WORKFLOW) + manifest.truncation.push( + `${workflow.name} run listing reached the 100-run first-page limit.`, + ); + for (const run of response.workflow_runs.slice( + 0, + MAX_RUNS_PER_WORKFLOW, + )) { + if ( + run.status !== "completed" || + !Number.isSafeInteger(run.id) || + !Number.isSafeInteger(run.run_attempt) + ) { + manifest.errors.push({ + scope: `workflow:${workflow.name}`, + runId: run.id ?? null, + message: + "Run record missing completed status, numeric id, or numeric latest attempt.", + }); + continue; + } + if (run.created_at && Date.parse(run.created_at) < Date.parse(cutoff)) + continue; + selected.push({ + id: run.id, + runAttempt: run.run_attempt, + workflow: workflow.name, + conclusion: run.conclusion ?? null, + createdAt: run.created_at ?? null, + }); + } + } catch (cause) { + manifest.errors.push({ + scope: `workflow:${workflow.name}`, + message: errorMessage(cause), + }); + } + } + + selected.sort( + (left, right) => + (Date.parse(right.createdAt ?? "") || 0) - + (Date.parse(left.createdAt ?? "") || 0), + ); + manifest.coverage.selectedRuns = selected.length; + for (const run of selected) { + if (budgetExpired()) { + manifest.truncation.push( + "Time budget reached before all selected runs were collected.", + ); + break; + } + const runRecord = { + ...run, + status: "pending", + rawPath: null, + reportPath: null, + error: null, + }; + manifest.runs.push(runRecord); + const runDirectory = join( + outputDirectory, + "runs", + `${String(run.id)}-${String(run.runAttempt)}`, + ); + try { + const result = await collect({ + repository: identity, + runId: String(run.id), + runAttempt: String(run.runAttempt), + outputDirectory: runDirectory, + env, + timeoutMs: Math.max( + 1, + Math.min(API_TIMEOUT_MS, TIME_BUDGET_MS - (now() - started)), + ), + }); + runRecord.status = "collected"; + runRecord.rawPath = result.rawPath; + runRecord.reportPath = result.reportPath; + manifest.coverage.collectedRuns += 1; + } catch (cause) { + runRecord.status = "error"; + runRecord.error = errorMessage(cause); + manifest.errors.push({ + scope: "run", + runId: run.id, + runAttempt: run.runAttempt, + message: runRecord.error, + }); + } + } + if (budgetExpired() && manifest.runs.length < manifest.coverage.selectedRuns) + manifest.truncation.push( + "Time budget reached; selected runs remain unattempted.", + ); + return await writeManifest(outputDirectory, manifest, started, now, env); +} + +function validateRepository(repository) { + if ( + typeof repository !== "string" || + !/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/u.test(repository) + ) + throw new Error( + "CI_REPOSITORY must be an owner/name repository identifier", + ); + return repository; +} + +async function apiJson(endpoint, env, started, now) { + const timeout = Math.max( + 1, + Math.min(API_TIMEOUT_MS, TIME_BUDGET_MS - (now() - started)), + ); + const { stdout } = await execFileAsync("gh", ["api", endpoint], { + env, + encoding: "utf8", + maxBuffer: MAX_BUFFER, + timeout, + }); + try { + return JSON.parse(stdout); + } catch (cause) { + throw new Error("GitHub Actions API returned malformed JSON", { cause }); + } +} + +async function writeManifest(outputDirectory, manifest, started, now, env) { + manifest.elapsedMs = Math.max(0, now() - started); + if ( + manifest.elapsedMs >= TIME_BUDGET_MS && + !manifest.truncation.some((item) => item.includes("Time budget")) + ) + manifest.truncation.push("Time budget reached."); + await mkdir(outputDirectory, { recursive: true }); + const manifestPath = join(outputDirectory, "batch-manifest.json"); + await writeFile( + manifestPath, + `${JSON.stringify(manifest, null, 2)}\n`, + "utf8", + ); + if (env.GITHUB_STEP_SUMMARY) + await appendBatchSummary(env.GITHUB_STEP_SUMMARY, manifest, manifestPath); + return { manifestPath, manifest }; +} + +async function appendBatchSummary(summaryPath, manifest, manifestPath) { + const collected = manifest.runs.filter( + ({ status }) => status === "collected", + ).length; + const failed = manifest.runs.filter( + ({ status }) => status === "error", + ).length; + const lines = [ + "### Batched CI timing snapshot", + "", + `Observed ${manifest.observedAt}; collected ${String(collected)} of ${String(manifest.coverage.selectedRuns)} selected recent completed runs (${String(failed)} run errors).`, + `Coverage uses the first ${String(MAX_RUNS_PER_WORKFLOW)} runs per workflow created in the preceding 24 hours and a ${String(TIME_BUDGET_MS / 1_000)} second total budget. This sample is bounded and is not exhaustive.`, + `Manifest: \`${manifestPath}\`.`, + ...manifest.truncation.map((message) => `- Truncated: ${message}`), + ...manifest.errors.map( + ({ scope, message }) => `- Error (${scope}): ${message}`, + ), + "", + ]; + await appendFile(summaryPath, lines.join("\n"), "utf8"); +} + +function errorMessage(cause) { + return cause instanceof Error ? cause.message : String(cause); +} + +if ( + process.argv[1] !== undefined && + resolve(process.argv[1]) === fileURLToPath(import.meta.url) +) { + try { + const result = await collectJobTimingsBatch({ + repository: process.env.CI_REPOSITORY, + outputDirectory: process.env.CI_JOB_TIMINGS_DIRECTORY ?? OUTPUT_DIRECTORY, + }); + process.stdout.write( + `Wrote batch manifest for ${String(result.manifest.coverage.collectedRuns)} collected runs to ${result.manifestPath}\n`, + ); + } catch (cause) { + process.stderr.write(`CI timing batch failed: ${errorMessage(cause)}\n`); + process.exitCode = 1; + } +} diff --git a/scripts/ci/job-timings.mjs b/scripts/ci/job-timings.mjs index 9d22f0d41..4d96f91b0 100644 --- a/scripts/ci/job-timings.mjs +++ b/scripts/ci/job-timings.mjs @@ -15,7 +15,7 @@ const STEP_COSTS = [ ]; /** Fetch and summarize one workflow attempt without turning telemetry into a gate. */ -async function collectJobTimings({ +export async function collectJobTimings({ repository, runId, runAttempt, @@ -23,13 +23,14 @@ async function collectJobTimings({ summaryPath, observedAt = new Date().toISOString(), env = process.env, + timeoutMs = TIMEOUT_MS, }) { const identity = validateIdentity({ repository, runId, runAttempt }); const endpoint = `repos/${identity.repository}/actions/runs/${identity.runId}/attempts/${identity.runAttempt}/jobs?per_page=100`; const { stdout } = await execFileAsync( "gh", ["api", "--paginate", "--slurp", endpoint], - { env, encoding: "utf8", maxBuffer: MAX_BUFFER, timeout: TIMEOUT_MS }, + { env, encoding: "utf8", maxBuffer: MAX_BUFFER, timeout: timeoutMs }, ); const rawPath = join(outputDirectory, "jobs.json"); const reportPath = join(outputDirectory, "report.json"); diff --git a/scripts/ci/required.mjs b/scripts/ci/required.mjs index d48a5ec2c..afe52c2b8 100644 --- a/scripts/ci/required.mjs +++ b/scripts/ci/required.mjs @@ -29,6 +29,7 @@ const lanes = { fixtures: ["conformance-fixtures"], readiness: ["reconstruction-readiness"], docs: ["docs"], + workflows: ["static"], }; for (const [scope, value] of Object.entries(outputs)) { if (value === "true") required.push(...(lanes[scope] ?? [scope])); diff --git a/scripts/ci/test-history.mjs b/scripts/ci/test-history.mjs new file mode 100644 index 000000000..4386f48af --- /dev/null +++ b/scripts/ci/test-history.mjs @@ -0,0 +1,114 @@ +import { execFileSync } from "node:child_process"; +import { appendFileSync } from "node:fs"; +import { resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +const ARTIFACT_NAME = "test-timings"; +const MAX_AGE_MS = 30 * 24 * 60 * 60 * 1000; +const MAX_CANDIDATES = 5; +const positiveId = (value) => + (typeof value === "string" || typeof value === "number") && + /^[1-9]\d*$/.test(String(value)) && + Number.isSafeInteger(Number(value)); + +function findTimingRun({ repository, branch, repositoryId, now = Date.now() }) { + if ( + !/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/.test(repository ?? "") || + !branch || + !positiveId(repositoryId) + ) + return ""; + const deadline = Date.now() + 20_000; + const api = (...arguments_) => { + const remaining = deadline - Date.now(); + if (remaining <= 0) + throw new Error("Timing history lookup budget exhausted"); + return JSON.parse( + execFileSync("gh", ["api", "--method", "GET", ...arguments_], { + encoding: "utf8", + timeout: Math.min(10_000, remaining), + maxBuffer: 4 * 1024 * 1024, + stdio: ["ignore", "pipe", "pipe"], + }), + ); + }; + + const { artifacts } = api( + `repos/${repository}/actions/artifacts`, + "-f", + `name=${ARTIFACT_NAME}`, + "-F", + "per_page=100", + ); + if (!Array.isArray(artifacts)) + throw new Error("Missing timing artifact inventory"); + const candidates = artifacts + .filter((artifact) => { + const lineage = artifact?.workflow_run; + const age = now - Date.parse(artifact?.created_at); + return ( + artifact?.name === ARTIFACT_NAME && + artifact?.expired === false && + typeof artifact?.created_at === "string" && + Number.isFinite(age) && + age >= 0 && + age < MAX_AGE_MS && + positiveId(lineage?.id) && + positiveId(lineage?.repository_id) && + positiveId(lineage?.head_repository_id) && + lineage.repository_id === lineage.head_repository_id && + lineage.repository_id === Number(repositoryId) && + lineage.head_branch === branch + ); + }) + .sort( + (left, right) => + Date.parse(right.created_at) - Date.parse(left.created_at), + ) + .slice(0, MAX_CANDIDATES); + + for (const artifact of candidates) { + const run = api( + `repos/${repository}/actions/runs/${artifact.workflow_run.id}`, + ); + const runAge = now - Date.parse(run?.created_at); + if ( + typeof run?.created_at === "string" && + Number.isFinite(runAge) && + runAge >= 0 && + runAge < MAX_AGE_MS && + run.id === artifact.workflow_run.id && + run.status === "completed" && + run.conclusion === "success" && + run.path === ".github/workflows/ci.yml" && + ["schedule", "workflow_dispatch"].includes(run.event) && + run.head_branch === branch && + run.head_repository?.full_name === repository + ) + return String(run.id); + } + return ""; +} + +function write(runId) { + if (process.env.GITHUB_OUTPUT) + appendFileSync(process.env.GITHUB_OUTPUT, `run-id=${runId}\n`); + console.log(JSON.stringify({ run_id: runId })); +} + +if ( + process.argv[1] && + fileURLToPath(import.meta.url) === resolve(process.argv[1]) +) { + let runId = ""; + try { + runId = findTimingRun({ + repository: process.env.CI_REPOSITORY, + branch: process.env.CI_DEFAULT_BRANCH, + repositoryId: process.env.CI_REPOSITORY_ID, + }); + } catch { + // History improves shard ordering but must never block CI. + } + write(runId); +} diff --git a/tests/boundary/filesystem/ciJobTimingsBatch.test.ts b/tests/boundary/filesystem/ciJobTimingsBatch.test.ts new file mode 100644 index 000000000..6c2658c00 --- /dev/null +++ b/tests/boundary/filesystem/ciJobTimingsBatch.test.ts @@ -0,0 +1,288 @@ +import { chmod, mkdir, readFile, writeFile } from "node:fs/promises"; +import { delimiter, join } from "node:path"; +import { fileURLToPath, pathToFileURL } from "node:url"; + +import { expect, it } from "vitest"; + +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { executeWorkflowFixture } from "../../support/workflowGit.js"; + +const batchScript = fileURLToPath( + new URL("../../../scripts/ci/job-timings-batch.mjs", import.meta.url), +); + +it("collects the latest attempt for recent completed CI and Release runs into a bounded manifest", async () => { + const recent = new Date().toISOString(); + const fixture = await createFixture([ + { + workflow_runs: [ + { + id: 42, + run_attempt: 3, + status: "completed", + conclusion: "failure", + created_at: new Date(Date.parse(recent) + 1_000).toISOString(), + }, + { + id: 41, + run_attempt: 1, + status: "completed", + conclusion: "cancelled", + created_at: recent, + }, + ], + }, + { + workflow_runs: [ + { + id: 43, + run_attempt: 2, + status: "completed", + conclusion: "success", + created_at: new Date(Date.parse(recent) + 2_000).toISOString(), + }, + ], + }, + [ + { + jobs: [ + { + name: "Release lane", + status: "completed", + conclusion: "success", + created_at: recent, + started_at: recent, + completed_at: new Date(Date.parse(recent) + 50_000).toISOString(), + steps: [], + }, + ], + }, + ], + [ + { + jobs: [ + { + name: "Failed lane", + status: "completed", + conclusion: "failure", + created_at: recent, + started_at: recent, + completed_at: new Date(Date.parse(recent) + 55_000).toISOString(), + steps: [], + }, + ], + }, + ], + [ + { + jobs: [ + { + name: "Cancelled lane", + status: "completed", + conclusion: "cancelled", + created_at: recent, + started_at: recent, + completed_at: new Date(Date.parse(recent) + 30_000).toISOString(), + steps: [], + }, + ], + }, + ], + ]); + const result = await runBatch(fixture); + const manifest = JSON.parse( + await readFile( + join(fixture.outputDirectory, "batch-manifest.json"), + "utf8", + ), + ); + const calls = JSON.parse(await readFile(fixture.calls, "utf8")); + + expect(result.stdout).toContain("3 collected runs"); + expect( + await readFile(join(fixture.directory, "summary.md"), "utf8"), + ).toContain("collected 3 of 3"); + expect(manifest.selection).toMatchObject({ + lookbackHours: 24, + maxRunsPerWorkflow: 100, + exhaustive: false, + }); + expect(manifest.coverage).toMatchObject({ + workflowsQueried: ["CI", "Release"], + selectedRuns: 3, + collectedRuns: 3, + }); + expect(manifest.truncation).toEqual([]); + expect(manifest.errors).toEqual([]); + expect(manifest.runs).toMatchObject([ + { id: 43, runAttempt: 2, conclusion: "success" }, + { id: 42, runAttempt: 3, conclusion: "failure" }, + { id: 41, runAttempt: 1, conclusion: "cancelled" }, + ]); + expect(manifest.runs).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + status: "collected", + rawPath: expect.any(String), + reportPath: expect.any(String), + }), + ]), + ); + expect(calls).toContainEqual([ + "api", + "--paginate", + "--slurp", + "repos/example/rea/actions/runs/42/attempts/3/jobs?per_page=100", + ]); + expect(calls).toContainEqual([ + "api", + "--paginate", + "--slurp", + "repos/example/rea/actions/runs/41/attempts/1/jobs?per_page=100", + ]); + const failedReport = JSON.parse( + await readFile( + join(fixture.outputDirectory, "runs/42-3/report.json"), + "utf8", + ), + ); + expect(failedReport.jobs[0].conclusion).toBe("failure"); + expect(failedReport.jobs[0].execution.seconds).toBe(55); +}); + +it("records listing and per-run errors without claiming complete coverage", async () => { + const recent = new Date().toISOString(); + const fixture = await createFixture([ + { + workflow_runs: [ + { + id: 7, + run_attempt: 2, + status: "completed", + conclusion: "success", + created_at: recent, + }, + ], + }, + { error: true }, + { jobs: "malformed" }, + ]); + await runBatch(fixture); + const manifest = JSON.parse( + await readFile( + join(fixture.outputDirectory, "batch-manifest.json"), + "utf8", + ), + ); + expect(manifest.selection.exhaustive).toBe(false); + expect(manifest.errors).toMatchObject([ + { scope: "workflow:Release" }, + { scope: "run" }, + ]); + expect(manifest.runs[0]).toMatchObject({ + id: 7, + runAttempt: 2, + status: "error", + }); +}); + +it("retains a partial manifest when its collection budget is exhausted", async () => { + const fixture = await createFixture([ + { + workflow_runs: [8, 9].map((id) => ({ + id, + run_attempt: 1, + status: "completed", + created_at: new Date().toISOString(), + })), + }, + { workflow_runs: [] }, + ]); + await executeWorkflowFixture( + process.execPath, + [ + "--input-type=module", + "-e", + ` + const { collectJobTimingsBatch } = await import(${JSON.stringify(pathToFileURL(batchScript).href)}); + let clock = Date.now(); + await collectJobTimingsBatch({ + repository: "example/rea", + outputDirectory: process.env.CI_JOB_TIMINGS_DIRECTORY, + now: () => clock, + collect: async () => { + clock += 240001; + throw new Error("fixture request exhausted the remaining budget"); + }, + }); + `, + ], + { cwd: fixture.directory, env: fixtureEnvironment(fixture) }, + ); + const manifest = JSON.parse( + await readFile( + join(fixture.outputDirectory, "batch-manifest.json"), + "utf8", + ), + ); + expect(manifest.coverage).toMatchObject({ + selectedRuns: 2, + collectedRuns: 0, + }); + expect(manifest.runs).toHaveLength(1); + expect(manifest.runs[0]).toMatchObject({ status: "error" }); + expect(manifest.truncation.join(" ")).toContain("Time budget"); +}); + +async function createFixture(responses: unknown[]) { + const directory = await createTestTempDirectory("rea-ci-job-timings-batch-"); + const fakeBin = join(directory, "bin"); + await mkdir(fakeBin); + const responsePath = join(directory, "responses.json"); + const calls = join(directory, "calls.json"); + await writeFile(responsePath, JSON.stringify(responses)); + await writeFile(calls, "[]"); + const fakeGh = join(fakeBin, "gh"); + await writeFile( + fakeGh, + `#!/usr/bin/env node +const fs = require("node:fs"); +const calls = JSON.parse(fs.readFileSync(process.env.CI_GH_CALLS, "utf8")); +calls.push(process.argv.slice(2)); +fs.writeFileSync(process.env.CI_GH_CALLS, JSON.stringify(calls)); +const responses = JSON.parse(fs.readFileSync(process.env.CI_GH_RESPONSES, "utf8")); +const response = responses[calls.length - 1]; +if (response && response.error) { process.stderr.write("fixture API error"); process.exitCode = 1; } +else process.stdout.write(JSON.stringify(response)); +`, + ); + await chmod(fakeGh, 0o755); + return { + directory, + fakeBin, + responses: responsePath, + calls, + outputDirectory: join(directory, ".cache", "ci-job-timings"), + }; +} + +function runBatch(fixture: Awaited>) { + return executeWorkflowFixture(process.execPath, [batchScript], { + cwd: fixture.directory, + env: fixtureEnvironment(fixture), + }); +} + +function fixtureEnvironment( + fixture: Awaited>, +) { + return { + ...process.env, + PATH: `${fixture.fakeBin}${delimiter}${process.env.PATH ?? ""}`, + CI_REPOSITORY: "example/rea", + CI_JOB_TIMINGS_DIRECTORY: fixture.outputDirectory, + CI_GH_CALLS: fixture.calls, + CI_GH_RESPONSES: fixture.responses, + GITHUB_STEP_SUMMARY: join(fixture.directory, "summary.md"), + }; +} diff --git a/tests/boundary/filesystem/ciRequiredLanes.test.ts b/tests/boundary/filesystem/ciRequiredLanes.test.ts index a86392e89..bc2e5eb06 100644 --- a/tests/boundary/filesystem/ciRequiredLanes.test.ts +++ b/tests/boundary/filesystem/ciRequiredLanes.test.ts @@ -90,10 +90,9 @@ it("exports every canonical scope with read authority for admission and keeps te const timings = z .object({ on: z.object({ - workflow_run: z.object({ - workflows: z.array(z.string()), - types: z.array(z.string()), - }), + schedule: z.array(z.object({ cron: z.string() })), + workflow_dispatch: z.unknown(), + workflow_run: z.unknown().optional(), }), permissions: z.record(z.string(), z.string()), jobs: z.object({ @@ -117,10 +116,8 @@ it("exports every canonical scope with read authority for admission and keeps te ), ), ); - expect(timings.on.workflow_run).toEqual({ - workflows: ["CI", "Release"], - types: ["completed"], - }); + expect(timings.on.workflow_run).toBeUndefined(); + expect(timings.on.schedule).toHaveLength(1); expect(timings.permissions).toEqual({ contents: "read", actions: "read" }); const checkout = timings.jobs.report.steps.find((step) => step.uses?.startsWith("actions/checkout@"), @@ -130,14 +127,9 @@ it("exports every canonical scope with read authority for admission and keeps te "persist-credentials": false, }); const report = timings.jobs.report.steps.find((step) => - step.run?.includes("scripts/ci/job-timings.mjs"), - ); - expect(report?.env?.CI_TIMING_RUN_ID).toBe( - "${{ github.event.workflow_run.id }}", - ); - expect(report?.env?.CI_TIMING_RUN_ATTEMPT).toBe( - "${{ github.event.workflow_run.run_attempt }}", + step.run?.includes("scripts/ci/job-timings-batch.mjs"), ); + expect(report?.env?.CI_REPOSITORY).toBe("${{ github.repository }}"); }); const requiredNeeds = async () => { @@ -152,6 +144,17 @@ it("makes the required gate wait for every CI lane, including future additions", ); }); +it("runs workflow validation inside required static checks when selected", async () => { + const { jobs } = await workflow(); + const steps = z + .array(z.object({ run: z.string().optional(), if: z.string().optional() })) + .parse(jobs.static?.steps); + const validation = steps.filter((step) => step.run?.includes("actionlint")); + expect(validation).toHaveLength(1); + expect(validation[0]?.if).toBe("needs.changes.outputs.workflows == 'true'"); + expect(jobs.workflows).toBeUndefined(); +}); + it("leaves cancellation ownership with CI so child lanes cannot cancel another tier", async () => { const { jobs } = await workflow(); const children = Object.values(jobs).flatMap((job) => @@ -257,33 +260,44 @@ it.each([ ["false", "skipped", true], ["false", "failure", false], ] as const)( - "enforces the SQLite required gate for sqlite=%s and result=%s", - async (sqlite, result, passes) => { + "enforces mapped required scopes for selected=%s and result=%s", + async (selected, result, passes) => { const workflow = await requiredNeeds(); - // GitHub exposes only direct dependencies in the required job's needs context. - const needs: Record< - string, - { result: string; outputs?: Record } - > = Object.fromEntries( - workflow.needs.map((name) => [name, { result: "success" }]), - ); - needs.changes = { result: "success", outputs: { code: "true", sqlite } }; - if (result === "missing") delete needs["sqlite-runtime"]; - else if ("sqlite-runtime" in needs) needs["sqlite-runtime"] = { result }; - const execution = executeWorkflowFixture( - process.execPath, - [ - fileURLToPath( - new URL("../../../scripts/ci/required.mjs", import.meta.url), - ), - ], - { env: { ...process.env, CI_JOBS: JSON.stringify(needs) } }, - ); - if (passes) await expect(execution).resolves.toMatchObject({ stderr: "" }); - else - await expect(execution).rejects.toMatchObject({ - code: 1, - stderr: expect.stringContaining("sqlite-runtime"), - }); + for (const [scope, lane] of [ + ["sqlite", "sqlite-runtime"], + ["workflows", "static"], + ] as const) { + // GitHub exposes only direct dependencies in the required job's needs context. + const needs: Record< + string, + { result: string; outputs?: Record } + > = Object.fromEntries( + workflow.needs.map((name) => [name, { result: "success" }]), + ); + needs.changes = { + result: "success", + outputs: { code: "true", [scope]: selected }, + }; + if (result === "missing") delete needs[lane]; + else needs[lane] = { result }; + const execution = executeWorkflowFixture( + process.execPath, + [ + fileURLToPath( + new URL("../../../scripts/ci/required.mjs", import.meta.url), + ), + ], + { env: { ...process.env, CI_JOBS: JSON.stringify(needs) } }, + ); + // Static checks remain required even without workflow edits. + const expected = passes && !(lane === "static" && result !== "success"); + if (expected) + await expect(execution).resolves.toMatchObject({ stderr: "" }); + else + await expect(execution).rejects.toMatchObject({ + code: 1, + stderr: expect.stringContaining(lane), + }); + } }, ); diff --git a/tests/boundary/filesystem/ciTestHistory.test.ts b/tests/boundary/filesystem/ciTestHistory.test.ts new file mode 100644 index 000000000..411e9ece9 --- /dev/null +++ b/tests/boundary/filesystem/ciTestHistory.test.ts @@ -0,0 +1,188 @@ +import { chmod, mkdir, readFile, writeFile } from "node:fs/promises"; +import { delimiter, dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; + +import { expect, it } from "vitest"; + +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { executeWorkflowFixture } from "../../support/workflowGit.js"; + +const script = fileURLToPath( + new URL("../../../scripts/ci/test-history.mjs", import.meta.url), +); +const repository = "example/rea"; +const repositoryId = 1; +const now = Date.now(); + +function artifact(id: number, overrides: Record = {}) { + return { + id: id + 100, + name: "test-timings", + expired: false, + created_at: new Date(now - 60_000).toISOString(), + workflow_run: { + id, + repository_id: repositoryId, + head_repository_id: repositoryId, + head_branch: "main", + }, + ...overrides, + }; +} + +function run(id: number, overrides: Record = {}) { + return { + id, + status: "completed", + conclusion: "success", + path: ".github/workflows/ci.yml", + event: "schedule", + created_at: new Date(now - 120_000).toISOString(), + head_branch: "main", + head_repository: { full_name: repository }, + ...overrides, + }; +} + +async function invoke( + artifacts: unknown[], + runs: Record, + options: { + apiError?: boolean; + repository?: string; + repositoryId?: string; + } = {}, +) { + const directory = await createTestTempDirectory("rea-ci-test-history-"); + const fakeBin = join(directory, "bin"); + await mkdir(fakeBin); + const fixture = join(directory, "api.json"); + const calls = join(directory, "api-calls"); + const output = join(directory, "github-output"); + await writeFile(fixture, JSON.stringify({ artifacts, runs })); + await writeFile(output, ""); + await writeFile(calls, ""); + const fakeGh = join(fakeBin, "gh"); + await writeFile( + fakeGh, + [ + "#!/bin/sh", + 'printf \'%s\\n\' "$*" >> "$GH_API_CALLS"', + 'if [ "$GH_API_ERROR" = "true" ]; then exit 1; fi', + 'if [ "$4" = "repos/example/rea/actions/artifacts" ]; then', + ' node -e \'const fs=require("node:fs"); const x=JSON.parse(fs.readFileSync(process.env.GH_FIXTURE_RESPONSE,"utf8")); process.stdout.write(JSON.stringify({artifacts:x.artifacts}));\'', + "else", + ' run_id="${4##*/}"', + ' node -e \'const fs=require("node:fs"); const x=JSON.parse(fs.readFileSync(process.env.GH_FIXTURE_RESPONSE,"utf8")); process.stdout.write(JSON.stringify(x.runs[Number(process.argv[1])] ?? {}));\' "$run_id"', + "fi", + "", + ].join("\n"), + ); + await chmod(fakeGh, 0o755); + const node24 = dirname(process.execPath); + const result = await executeWorkflowFixture(process.execPath, [script], { + cwd: directory, + env: { + ...process.env, + PATH: [fakeBin, node24, process.env.PATH ?? ""].join(delimiter), + GH_FIXTURE_RESPONSE: fixture, + GH_API_CALLS: calls, + GH_API_ERROR: String(options.apiError ?? false), + GH_TOKEN: "fixture-token", + CI_REPOSITORY: options.repository ?? repository, + CI_REPOSITORY_ID: options.repositoryId ?? String(repositoryId), + CI_DEFAULT_BRANCH: "main", + GITHUB_OUTPUT: output, + }, + }); + return { + result, + output: await readFile(output, "utf8"), + calls: await readFile(calls, "utf8"), + }; +} + +it("selects the latest trusted full run that actually published timing history", async () => { + const { result, output, calls } = await invoke( + [ + artifact(15), + artifact(14), + artifact(13, { + workflow_run: { ...artifact(13).workflow_run, head_repository_id: 2 }, + }), + artifact(12, { + workflow_run: { ...artifact(12).workflow_run, head_branch: "feature" }, + }), + artifact(11), + ], + { + 15: run(15, { conclusion: "failure" }), + 14: run(14, { path: ".github/workflows/release.yml" }), + 11: run(11, { event: "workflow_dispatch" }), + }, + ); + + expect(JSON.parse(result.stdout)).toEqual({ run_id: "11" }); + expect(output).toBe("run-id=11\n"); + expect(calls).toContain("name=test-timings"); + expect(calls).toContain("per_page=100"); + expect(calls.match(/actions\/runs\//gu)).toHaveLength(3); + expect(calls).not.toContain("actions/workflows"); +}); + +it("rejects expired, foreign, old, and non-full run candidates", async () => { + const { result, output } = await invoke( + [ + artifact(15, { expired: true }), + artifact(14, { + workflow_run: { ...artifact(14).workflow_run, head_repository_id: 2 }, + }), + artifact(13, { + created_at: new Date(now - 31 * 24 * 60 * 60 * 1000).toISOString(), + }), + artifact(12), + ], + { + 12: run(12, { event: "pull_request" }), + }, + ); + + expect(JSON.parse(result.stdout)).toEqual({ run_id: "" }); + expect(output).toBe("run-id=\n"); +}); + +it("writes an empty run id when GitHub history is unavailable", async () => { + const { result, output } = await invoke([], {}, { apiError: true }); + expect(JSON.parse(result.stdout)).toEqual({ run_id: "" }); + expect(output).toBe("run-id=\n"); +}); + +it("bounds run lookups to five artifacts even when inventory has more", async () => { + const ids = [25, 24, 23, 22, 21, 20, 19]; + const { result, calls } = await invoke( + ids.map((id) => artifact(id)), + Object.fromEntries( + ids.map((id, index) => [ + id, + run(id, index < 5 ? { conclusion: "failure" } : {}), + ]), + ), + ); + + expect(JSON.parse(result.stdout)).toEqual({ run_id: "" }); + expect(calls.match(/actions\/runs\//gu)).toHaveLength(5); +}); + +it("returns empty history without querying for an invalid repository identity", async () => { + const { result, output, calls } = await invoke( + [], + {}, + { + repository: "owner/repo?per_page=100", + repositoryId: "not-an-id", + }, + ); + expect(JSON.parse(result.stdout)).toEqual({ run_id: "" }); + expect(output).toBe("run-id=\n"); + expect(calls).toBe(""); +}); diff --git a/tests/boundary/filesystem/ciTestReports.test.ts b/tests/boundary/filesystem/ciTestReports.test.ts index 3ab042718..2c44878de 100644 --- a/tests/boundary/filesystem/ciTestReports.test.ts +++ b/tests/boundary/filesystem/ciTestReports.test.ts @@ -80,10 +80,22 @@ it("keeps diagnosis available after merge failure and publishes only complete su expect(history?.if).toContain("needs.test-shard.result == 'success'"); expect(history?.if).toContain("needs.changes.outputs.tier == 'full'"); expect(history?.with?.path).toBe(".cache/ci-test-history/test-results.json"); - const download = (await workflow()).build!.steps.find( + const buildSteps = (await workflow()).build!.steps; + const lookup = buildSteps.find((step) => + step.run?.includes("test-history.mjs"), + ); + expect(lookup?.env).toMatchObject({ + CI_REPOSITORY: "${{ github.repository }}", + CI_REPOSITORY_ID: "${{ github.repository_id }}", + CI_DEFAULT_BRANCH: "${{ github.event.repository.default_branch }}", + }); + const download = buildSteps.find( (step) => step.name === "Download previous timing report", ); expect(download?.with?.name).toBe(history?.with?.name); + expect(download?.with?.["run-id"]).toBe( + "${{ steps.history.outputs.run-id }}", + ); }); it("rejects successful shard reports with missing or duplicate portable files", async () => { From 6c816af11975e8e801adb2924c4e3c116c72ebba Mon Sep 17 00:00:00 2001 From: Harly <1211074765@qq.com> Date: Tue, 13 Oct 2026 18:19:44 +0800 Subject: [PATCH 088/159] fix(managed): preserve negative zero in CIL operands (#2437) --- src/dotnet/ManagedMemberInstructionDecoder.ts | 8 ++- ...anagedMemberComparisonFromMetadata.test.ts | 60 +++++++++++++++++++ 2 files changed, 66 insertions(+), 2 deletions(-) diff --git a/src/dotnet/ManagedMemberInstructionDecoder.ts b/src/dotnet/ManagedMemberInstructionDecoder.ts index f638e3e83..7ef518733 100644 --- a/src/dotnet/ManagedMemberInstructionDecoder.ts +++ b/src/dotnet/ManagedMemberInstructionDecoder.ts @@ -265,6 +265,10 @@ type OperandResult = { type OperandReader = (il: Buffer, offset: number) => OperandResult; +// String(-0) loses a CIL constant's sign in both anchors and normalized hashes. +const floatingPointOperand = (value: number): string => + Object.is(value, -0) ? "-0" : String(value); + const FIXED_OPERAND_READERS = new Map([ [ "none", @@ -315,7 +319,7 @@ const FIXED_OPERAND_READERS = new Map([ (il, offset) => ({ next: offset + 4, anchorKind: "constant", - value: String(il.readFloatLE(offset)), + value: floatingPointOperand(il.readFloatLE(offset)), }), ], [ @@ -323,7 +327,7 @@ const FIXED_OPERAND_READERS = new Map([ (il, offset) => ({ next: offset + 8, anchorKind: "constant", - value: String(il.readDoubleLE(offset)), + value: floatingPointOperand(il.readDoubleLE(offset)), }), ], [ diff --git a/tests/boundary/providers/managed/managedMemberComparisonFromMetadata.test.ts b/tests/boundary/providers/managed/managedMemberComparisonFromMetadata.test.ts index 7f4408811..56014fbd4 100644 --- a/tests/boundary/providers/managed/managedMemberComparisonFromMetadata.test.ts +++ b/tests/boundary/providers/managed/managedMemberComparisonFromMetadata.test.ts @@ -116,6 +116,66 @@ describe("managed member comparison", () => { }); }); +describe("managed member comparison floating-point constants", () => { + it.each([ + { + opcode: "ldc.r4", + positive: "22000000002a", + negative: "22000000802a", + type: 0x0c, + }, + { + opcode: "ldc.r8", + positive: "2300000000000000002a", + negative: "2300000000000000802a", + type: 0x0d, + }, + ])( + "detects a change from positive to negative zero in $opcode", + ({ opcode, positive, negative, type }) => { + const observe = (hex: string, path: string) => { + const il = Buffer.from(hex, "hex"); + return inspect( + buildManagedPeFixture({ + methodSignature: Buffer.from([0x00, 0x00, type]), + ilBody: Buffer.concat([Buffer.from([(il.length << 2) | 2]), il]), + }), + path, + ); + }; + const left = observe(positive, "/tmp/positive-zero.dll"); + const right = observe(negative, "/tmp/negative-zero.dll"); + const comparison = compareManagedMembers(left, right); + + expect(comparison.methods[0]).toMatchObject({ + status: "changed", + match: { status: "matched", basis: "exact-signature" }, + dimensions: ["cil"], + }); + for (const [observation, operand] of [ + [left, "0"], + [right, "-0"], + ] as const) { + expect(observation.result.methods[0]?.body).toMatchObject({ + status: "present", + anchors: [ + { il_offset: 0, opcode, operand_kind: "constant", operand }, + ], + }); + } + expect(left.result.methods[0]?.body.normalized_il_sha256).not.toBe( + right.result.methods[0]?.body.normalized_il_sha256, + ); + const repeated = observe(negative, "/tmp/negative-zero-copy.dll"); + expect(compareManagedMembers(right, repeated).methods[0]).toMatchObject({ + status: "unchanged", + match: { status: "matched", basis: "exact-il-signature" }, + dimensions: [], + }); + }, + ); +}); + describe("managed member comparison uncertainty", () => { it("does not infer a structural method match from undecoded signature fields", () => { const left = inspect( From f8e95799f941e9e55e0d47841896b401bab67d29 Mon Sep 17 00:00:00 2001 From: Yijie Lu <115929844+1NormalGuy@users.noreply.github.com> Date: Tue, 13 Oct 2026 18:20:54 +0800 Subject: [PATCH 089/159] fix(artifacts): preserve NIB field and class identities (#2106) --- src/artifacts/apple/NibArchive.ts | 12 +- .../applications/nibFieldIdentity.test.ts | 110 ++++++++++++++++++ 2 files changed, 120 insertions(+), 2 deletions(-) create mode 100644 tests/acceptance/applications/nibFieldIdentity.test.ts diff --git a/src/artifacts/apple/NibArchive.ts b/src/artifacts/apple/NibArchive.ts index bd8ef12ca..c673184f4 100644 --- a/src/artifacts/apple/NibArchive.ts +++ b/src/artifacts/apple/NibArchive.ts @@ -134,7 +134,14 @@ export const decodeNibArchive = ( const value = values.get(index); if (value === undefined) continue; const key = keys[value.key_index]; - if (key !== undefined) fields[key] = value.value; + if (key !== undefined) + // Archive keys are data, including the legacy __proto__ setter name. + Object.defineProperty(fields, key, { + value: value.value, + enumerable: true, + configurable: true, + writable: true, + }); } return [{ id, class_name: className, values: fields }]; }); @@ -519,7 +526,8 @@ const decodeUtf8 = ( label: string, ): string => { try { - return new TextDecoder("utf-8", { fatal: true }).decode( + // These are length-prefixed names, not standalone text documents. + return new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode( bytes.subarray(start, end), ); } catch (cause: unknown) { diff --git a/tests/acceptance/applications/nibFieldIdentity.test.ts b/tests/acceptance/applications/nibFieldIdentity.test.ts new file mode 100644 index 000000000..5af16f890 --- /dev/null +++ b/tests/acceptance/applications/nibFieldIdentity.test.ts @@ -0,0 +1,110 @@ +import { Client } from "@modelcontextprotocol/client"; +import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; +import { createHash } from "node:crypto"; +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { join, resolve } from "node:path"; +import { expect } from "vitest"; + +import { encodeNibArchiveFixture } from "../../../src/artifacts/apple/NibArchive.fixture.js"; +import { toolContract } from "../../../src/contracts/toolContracts.js"; +import { interfaceBuilderAnalysisSchema } from "../../../src/domain/apple/interfaceBuilderGraph.js"; +import { thinMach } from "../../../src/domain/binaryTarget.fixture.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { cliTest } from "../../support/cli/cliFixture.js"; + +cliTest( + "preserves literal NIB fields and UTF-8 names through CLI and stdio MCP", + async ({ cli }) => { + const root = await createTestTempDirectory("rea-nib-field-identity-"); + const app = join(root, "Fixture.app"); + const contents = join(app, "Contents"); + await mkdir(join(contents, "Resources"), { recursive: true }); + await mkdir(join(contents, "MacOS")); + await writeFile( + join(contents, "MacOS/App"), + thinMach(0xfeedfacf, 0x0100000c), + ); + await writeFile( + join(contents, "Info.plist"), + "CFBundleExecutableAppCFBundleIdentifierdev.rea.nib-fields", + ); + const archive = encodeNibArchiveFixture({ + classes: ["UIView", "\uFEFFCustomView"], + objects: [ + { + classIndex: 0, + values: { + ["__proto__"]: { ref: 1 }, + constructor: 7, + enabled: true, + "\uFEFFenabled": false, + }, + }, + { classIndex: 1, values: { ["__proto__"]: 23 } }, + ], + }); + const path = join(contents, "Resources/Panel.nib"); + await writeFile(path, archive); + const result = await cli.run({ + arguments: ["decode-interface-builder", app, "--json"], + environment: { REA_LOG_LEVEL: "silent", REA_ANALYSIS_PROVIDER: "auto" }, + }); + expect(result.exitCode, result.stdout + result.stderr).toBe(0); + const contract = toolContract("decode_interface_builder"); + const evidence = contract.outputSchema.parse(result.json); + const transport = new StdioClientTransport({ + command: process.execPath, + args: [resolve("scripts/rea.mjs"), "mcp"], + env: { + PATH: process.env.PATH ?? "", + REA_LOG_LEVEL: "silent", + REA_ANALYSIS_PROVIDER: "auto", + }, + stderr: "pipe", + }); + const client = new Client({ name: "nib-field-identity", version: "1" }); + try { + await client.connect(transport); + const opened = await client.callTool({ + name: "open_binary", + arguments: { path: app }, + }); + expect(opened.isError, JSON.stringify(opened)).not.toBe(true); + const response = await client.callTool({ + name: contract.name, + arguments: {}, + }); + expect(response.isError, JSON.stringify(response)).not.toBe(true); + const mcp = contract.outputSchema.parse(response.structuredContent); + expect(mcp.normalized_result).toEqual(evidence.normalized_result); + } finally { + try { + await client.close(); + } finally { + await transport.close(); + } + } + const report = interfaceBuilderAnalysisSchema.parse( + evidence.normalized_result, + ); + expect(report.documents[0]?.archive_sha256).toBe( + createHash("sha256").update(archive).digest("hex"), + ); + const objects = report.graph.nodes.filter( + ({ attributes }) => attributes.interface_builder_object_id !== undefined, + ); + expect(objects).toHaveLength(2); + expect(objects[0]?.attributes.nibValues).toEqual({ + ["__proto__"]: { objectID: "1" }, + constructor: 7, + enabled: true, + "\uFEFFenabled": false, + }); + expect(objects[1]?.attributes).toMatchObject({ + class_name: "\uFEFFCustomView", + nibValues: { ["__proto__"]: 23 }, + }); + expect(await readFile(path)).toEqual(archive); + }, + 60_000, +); From fe496bd1cf589938c372646ddef8ca9701417bf0 Mon Sep 17 00:00:00 2001 From: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:03:54 +0800 Subject: [PATCH 090/159] docs(website): add essay on reverse engineering models Add an English review draft connecting Touhou reconstruction evidence with model specialization, repository knowledge and feedback-driven learning. Preserve model attribution and historical verification scope in supporting notes, and translate the new Blog listing for all existing editions. Validation: website.py check, focused formatting checks, and 32 Chromium cases covering both preview paths, light/dark themes, responsive layouts, translated indexes and no-JavaScript reading. --- website/README.md | 1 + .../evidence/reverse-engineering-models.md | 165 ++++ website/locales/ar/messages.po | 48 +- website/locales/de/messages.po | 48 +- website/locales/es/messages.po | 48 +- website/locales/fa/messages.po | 48 +- website/locales/fr/messages.po | 48 +- website/locales/id/messages.po | 48 +- website/locales/it/messages.po | 48 +- website/locales/ja/messages.po | 46 +- website/locales/ko/messages.po | 46 +- website/locales/messages.pot | 52 +- website/locales/pl/messages.po | 48 +- website/locales/pt-BR/messages.po | 48 +- website/locales/ru/messages.po | 48 +- website/locales/th/messages.po | 48 +- website/locales/tr/messages.po | 48 +- website/locales/uk/messages.po | 48 +- website/locales/vi/messages.po | 48 +- website/locales/zh-CN/messages.po | 46 +- website/locales/zh-TW/messages.po | 46 +- website/public/blog/index.html | 16 + .../reverse-engineering-models/index.html | 810 ++++++++++++++++++ 23 files changed, 1688 insertions(+), 212 deletions(-) create mode 100644 website/evidence/reverse-engineering-models.md create mode 100644 website/public/blog/reverse-engineering-models/index.html diff --git a/website/README.md b/website/README.md index b7a159570..967d9c6f8 100644 --- a/website/README.md +++ b/website/README.md @@ -131,6 +131,7 @@ describe its recorded investigation; update them only when repeating that work. - `public/showcase/index.html`: the case-study index. - `public/blog/index.html`: articles about reconstruction methods, ports, mods and reverse engineering. - `public/blog/touhou-reconstruction/index.html`: a personal essay about agent autonomy, oracle quality, knowledge carried between TH08 and TH095, PC-98 TH04, and the path from exact reconstruction to readable source and modern ports. +- `public/blog/reverse-engineering-models/index.html`: an English draft on general models, compiler specialization and learning useful reconstruction experiments, with historical evidence from TH08, TH095, TH04 and TH09. Model attribution and verification scope are in [evidence/reverse-engineering-models.md](evidence/reverse-engineering-models.md). - `public/showcase/aegis/index.html`: Aegis's Android login-code calculation, with an adjustable clock and reference checks. - `public/showcase/dx-ball/index.html`: sound-pan investigation and project status. - `public/showcase/notion/index.html`: Notion's Electron clipboard bridge and rich clipboard format. diff --git a/website/evidence/reverse-engineering-models.md b/website/evidence/reverse-engineering-models.md new file mode 100644 index 000000000..ede754cfa --- /dev/null +++ b/website/evidence/reverse-engineering-models.md @@ -0,0 +1,165 @@ +# General models and specialist models: article evidence + +Supporting record for `public/blog/reverse-engineering-models/index.html`. +The article is an English draft for review and remains `noindex`. + +## Research scope + +Repository history, maintained source and recorded comparison notes were reviewed +on 13 October 2026. This research did not run game targets, compile reconstruction +candidates or repeat historical runtime checks. Public commit availability was +checked through GitHub CLI. + +The evidence describes specific historical functions and their recorded checks. +Source presence, compiler exactness, complete-product builds and runtime agreement +remain distinct. The proposed specialist/RL comparisons are future experiments; +no trained specialist was evaluated for this article. + +## Model attribution + +- The project operator identifies `gpt-dots` as **GPT-6-Astra**. +- `gpt-web` is a changing label for the model used at the time. The operator + describes then-latest models, usually xhigh, commonly GPT-5.6-Sol earlier and + GPT-6-Sol recently. Do not assign all such commits a single model or infer + individual identities from a release date. +- Explicit `gpt-6-sol` and `gpt-6.1-sol` subjects establish the recorded names. + Per-run reasoning effort and full budgets are not uniformly recorded. +- `gpt-6-luna-max` records GPT-6-Luna and max effort. It does not establish + parameter count or specialist training. + +Later sessions inherited earlier source, diagnostics and failed experiments. +Historical before/after closures support the essay's questions; they are not +controlled estimates of model-size effects. + +## TH09 Type18/24 attack update + +Earlier source-context review: +[GPT-6-Sol, 26 September](https://github.com/N0zoM1z0/th09/commit/de63c0f9121963f3a6b1ab5471c4878625fd3425). +Closing reconstruction: +[GPT-6.1-Sol, 10 October](https://github.com/N0zoM1z0/th09/commit/52929ed7c299f1a5f29cc13964e22510145fd616). + +The earlier candidate was 1,435 bytes against a 1,436-byte target. Matching the +overall frame size concealed different homes for returned vectors and real +collision storage. Declaration and helper-visibility controls did not close it. + +The closing session introduces an ordinary XYZ destination assignment, consumes +real returned vectors directly and groups six actual publication/rotation +operations into one comma-sequenced full expression. Keeping the real collision +point at function scope preserves separate storage. The seventh warm candidate +matches, followed by a maintained-path canonical cold comparison of all 1,436 +bytes and 44 encoded fields. Original class/source spelling and later runtime +gates remain separate. + +The article's four-row table groups selected lessons from seven candidates. It +does not enumerate every build or reproduce a complete source listing. + +## TH09 Front callback + +Earlier diagnostic checkpoint: +[GPT-6.1-Sol, 2 October](https://github.com/N0zoM1z0/th09/commit/f8d12d72aaf02212ee1b585a8f3f86d7ed000269). +Full graph closure: +[GPT-6-Astra (`gpt-dots`), 6 October](https://github.com/N0zoM1z0/th09/commit/da33a646221198ceb7f3df2b96e2ca0e9b2f705f). +Accepted replay: +[6 October follow-up](https://github.com/N0zoM1z0/th09/commit/5f825b28a862f8e1a6d4aea2f78a42f04c09a1c0). + +The earlier maintained owner was 1,510 bytes versus the 1,491-byte target. Its +prefix and suffix matched only as independently positioned diagnostic regions; +they did not establish a complete-function match. + +A common outer continuation places completion and frame dispatch in complementary +arms with one final frame increment/return. Within this graph, the side/stage +decision recovers target block sharing. The record establishes the complete +1,491-byte owner, 107 independently bound fields, 384 instructions and 42 calls. +The follow-up records accepted forced-recompile replay for `da33a646`. +Bounded trace models are static checks, not game-runtime validation. + +## TH095 identifier-bucket reuse + +[Supervisor::LoadDat, 2 September](https://github.com/N0zoM1z0/th095/commit/26bb64bdb093a13f73abd0ee8ae7ce199ee4658f). + +Under observed VC7.1 build 3077, declaration order alone did not recover the real +local homes. Earlier background work established useful identifier buckets. +Backing the real file-size scalar with `averagedPanLocal12` places it at +`EBP-0x04` and the genuine 128-byte buffer at `EBP-0x88`. All 151 bytes and 15 +relocations close. + +This is compiler-context evidence. It does not establish a universal C++ rule +or uniquely recover the original author's variable name. The `gpt-web` label +does not identify this run's model individually. + +## TH09 Luna enemy-spawn closure + +[GPT-6-Luna-max, 27 September](https://github.com/N0zoM1z0/th09/commit/1eb843383941d00065622a9e353146c4bb96979e). + +An ordinary loop with `continue`/`break`, branch-local RunEcl failure handling +and a shared success tail closes the 373-byte owner. TH08 source supplies the +source-family hypothesis; TH09 establishes the bounds, fields, ABI and callees. +Two independent cold builds and canonical replay are recorded. + +The earlier stable 361-byte candidate is `gpt-web`, with an individually +unresolved model identity. This case demonstrates a useful discovery across +model tiers, not a measured specialist or smaller-parameter advantage. + +## TH04 compiler diversity + +[Player MAIN_0 cohort, 15 September](https://github.com/N0zoM1z0/th04/commit/6dce0a79d5b931f0e2a08475b90e0c33d2321b98). + +Natural Turbo C++ 4.0J source produces a 629-byte update/miss contribution and a +334-byte renderer. Expression lifetimes, branch-local stores, declaration order +and a shared firing tail matter. Correct physical linker order avoids an extra +alignment byte. The recorded focused and aggregate replays cover 129 and 208 +owners respectively. These are historical owned-extent checks; this checkpoint +does not establish whole-game execution. + +TH08 uses VC7, while TH095/TH09 use VC7.1. Avoid treating every game's source +patterns as output from the same compiler. Shared game authorship supplies a +prior; target-local evidence validates an actual transfer. + +## TH08 numerical investigation + +[Polar motion and x87 behavior, 13 October](https://github.com/N0zoM1z0/th08/commit/af9cb1d3e839b15a78b47ac1701506083cc82847). + +A collision divergence at stage 2 frame 6,825 leads back to a one-ULP +displacement difference at frame 5,573. Historical CRT wrappers retain a wider +x87 return despite storing a rounded float. A distinct i386 case involves x87 +precision-control state. The record includes field probes and replay checks for +specified scenarios. Several preceding investigations use the same model; this +is breadth-of-investigation evidence, not a model-switch comparison. + +## TH08 floating-literal oracle repair + +[Repository-wide audit, 2 September](https://github.com/N0zoM1z0/th08/commit/a393f400786e4cc6cbb1000af0dad3276da12c78). + +Literal payload checks previously depended on optional explicit `data_hex` +attestation. The fix decodes and target-checks every VC7 `__real@...` relocation. +The follow-up audits 1,548 relocations and corrects 12 stale rows in five already +accepted units. One source change corrects a bullet-spread increment from +`1.0f` to `32.0f`. + +Affected-unit replay and a complete 1,106-unit cold replay are recorded after +repair. This defect is separate from the later one-ULP numerical investigation. +The RL consequence in the article is an inference: a reward based on the earlier +check could reinforce wrong source. The record does not describe an actual RL +exploit or trained model doing so. + +## TH09 target-sized replay-save candidate + +[Replay-save closure, 2 October](https://github.com/N0zoM1z0/th09/commit/ab72fd77728293fc0beacfc656ca701b0e266016). + +The earlier candidate already had the target's 1,825-byte length but 467 complete +comparison differences. Shared index identity, loop structure and the real +cancel/overwrite continuation close the complete owner. The example supports +using size as a diagnostic rather than final acceptance. + +## Related research + +The article links these primary sources: + +- [LLM4Decompile, EMNLP 2024](https://aclanthology.org/2024.emnlp-main.203/). +- [AutoDecompiler, June 2026 preprint](https://arxiv.org/html/2606.16162v1). +- [When LLM Decompilers Recompile More and Preserve Less, September 2026 preprint](https://arxiv.org/html/2609.05370v1). + +These works concern specialized decompilation, feedback-based refinement and +functional checking. Their published setups do not establish a winner for the +article's proposed historical-compiler reconstruction experiment. Preserve that +objective distinction when expanding the discussion. diff --git a/website/locales/ar/messages.po b/website/locales/ar/messages.po index 329f0cd96..ed491e991 100644 --- a/website/locales/ar/messages.po +++ b/website/locales/ar/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "أفكار حول الهندسة العكسية." #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "لعبة إعادة الإعمار{ph0}مقال" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "الثورة الصناعية في الهندسة العكسية" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,19 @@ msgstr "" "ما دليل إعادة تعمير TH08, TH095 و TH04 علمني عن وكلاء والتحقق المعرفة أن " "المركبات عبر المشاريع." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "اقرأ المقال" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "بواسطة N0zoM1z0 * 10 أكتوبر 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -233,7 +233,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "تابع REA على X" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "العودة للأعلى" @@ -11463,7 +11463,7 @@ msgctxt "shared" msgid "Guides" msgstr "الأدلة" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11483,7 +11483,7 @@ msgctxt "shared" msgid "Source" msgstr "الكود المصدري" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11503,7 +11503,7 @@ msgctxt "shared" msgid "MIT license" msgstr "رخصة MIT" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11523,7 +11523,7 @@ msgctxt "shared" msgid "Join the community" msgstr "انضم إلى المجتمع" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12478,6 +12478,32 @@ msgstr "" "تسجيل في REA دون اتصال، أو اربط خادم MCP في Everywhere لفحص حركة المرور " "المسجلة مع وكيلك." +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "نماذج متخصصة{ph0}مقال" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "هل تحتاج الهندسة العكسية إلى نموذج أكبر؟" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "" +"أفكار من إعادة بناء Touhou حول النماذج العامة والمتخصصة والتعلم من نتائج " +"المُصرّف." + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "بقلم N0zoM1z0 · 13 أكتوبر 2026 · مسودة للمراجعة" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "دليل الإعداد" diff --git a/website/locales/de/messages.po b/website/locales/de/messages.po index ee3fec463..9a0b5a51d 100644 --- a/website/locales/de/messages.po +++ b/website/locales/de/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Gedanken zum Reverse Engineering." #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Spielrekonstruktion{ph0}Essay" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "Die industrielle Revolution im Reverse Engineering" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -53,19 +53,19 @@ msgstr "" "gelehrt, über Agenten, Überprüfung und Wissen, dass Verbindungen über " "Projekte hinweg." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Lesen Sie den Artikel" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Von N0zoM1z0 · 10. Oktober 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -236,7 +236,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "REA auf X folgen" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Nach oben" @@ -11924,7 +11924,7 @@ msgctxt "shared" msgid "Guides" msgstr "Anleitungen" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11944,7 +11944,7 @@ msgctxt "shared" msgid "Source" msgstr "Quellcode" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11964,7 +11964,7 @@ msgctxt "shared" msgid "MIT license" msgstr "MIT-Lizenz" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11984,7 +11984,7 @@ msgctxt "shared" msgid "Join the community" msgstr "Zur Community" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12957,6 +12957,32 @@ msgstr "" "oder verbinden Sie den MCP-Server von Everywhere, um aufgezeichneten " "Datenverkehr mit Ihrem Agenten zu untersuchen." +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "Spezialisierte Modelle{ph0}Essay" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Braucht Reverse Engineering ein größeres Modell?" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "" +"Was die Rekonstruktion von Touhou über allgemeine und spezialisierte Modelle" +" sowie das Lernen aus Compiler-Rückmeldungen zeigt." + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "Von N0zoM1z0 · 13. Oktober 2026 · Entwurf zur Durchsicht" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Einrichtungsanleitung" diff --git a/website/locales/es/messages.po b/website/locales/es/messages.po index f40b06204..7af23dad4 100644 --- a/website/locales/es/messages.po +++ b/website/locales/es/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Reflexiones sobre ingeniería inversa." #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Reconstrucción del juego{ph0}Ensayo" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "La revolución industrial en ingeniería inversa" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -53,19 +53,19 @@ msgstr "" "enseñó sobre los agentes, la verificación y el conocimiento que componen los" " proyectos." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Leer el artículo" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Por N0zoM1z0 * 10 octubre 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -236,7 +236,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Sigue a REA en X" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Volver arriba" @@ -11843,7 +11843,7 @@ msgctxt "shared" msgid "Guides" msgstr "Guías" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11863,7 +11863,7 @@ msgctxt "shared" msgid "Source" msgstr "Código fuente" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11883,7 +11883,7 @@ msgctxt "shared" msgid "MIT license" msgstr "Licencia MIT" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11903,7 +11903,7 @@ msgctxt "shared" msgid "Join the community" msgstr "Únete a la comunidad" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12871,6 +12871,32 @@ msgstr "" " o conecte el servidor MCP de Everywhere para inspeccionar el tráfico " "capturado con su agente." +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "Modelos especializados{ph0}Ensayo" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "¿Necesita la ingeniería inversa un modelo más grande?" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "" +"Lo que la reconstrucción de Touhou sugiere sobre los modelos generales, los " +"especializados y el aprendizaje a partir de los resultados del compilador." + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "Por N0zoM1z0 · 13 de octubre de 2026 · Borrador para revisión" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Guía de instalación" diff --git a/website/locales/fa/messages.po b/website/locales/fa/messages.po index e163a131b..18e540bfc 100644 --- a/website/locales/fa/messages.po +++ b/website/locales/fa/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "افکار در مهندسی معکوس." #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "بازسازی بازی{ph0}مقاله" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "انقلاب صنعتی در مهندسی معکوس" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,19 @@ msgstr "" "چه دستی دوباره و بازسازی TH08, TH095 و TH04 به من آموخت در مورد عوامل, تأیید" " و دانش است که ترکیبات در سراسر پروژه." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "مقاله را بخوانید" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "توسط N0zoM1z0 * 10 اکتبر 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -234,7 +234,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "REA را در X دنبال کنید" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "بازگشت به بالا" @@ -11558,7 +11558,7 @@ msgctxt "shared" msgid "Guides" msgstr "راهنماها" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11578,7 +11578,7 @@ msgctxt "shared" msgid "Source" msgstr "کد منبع" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11598,7 +11598,7 @@ msgctxt "shared" msgid "MIT license" msgstr "مجوز MIT" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11618,7 +11618,7 @@ msgctxt "shared" msgid "Join the community" msgstr "به جامعه بپیوندید" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12580,6 +12580,32 @@ msgstr "" " در REA به‌صورت آفلاین، خروجی SAZ ?? HAR 1.2 بگیرید، یا سرور MCP نرم‌افزار " "Everywhere را متصل کنید تا ترافیک ضبط‌شده را با عامل خود بررسی کنید." +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "مدل‌های تخصصی{ph0}یادداشت" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "آیا مهندسی معکوس به مدل بزرگ‌تری نیاز دارد؟" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "" +"تأملاتی بر پایهٔ بازسازی Touhou دربارهٔ مدل‌های عمومی، مدل‌های تخصصی و " +"یادگیری از بازخورد کامپایلر." + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "نویسنده: N0zoM1z0 · ۱۳ اکتبر ۲۰۲۶ · پیش‌نویس برای بازبینی" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "راهنمای نصب" diff --git a/website/locales/fr/messages.po b/website/locales/fr/messages.po index f6e6ab357..4c3ef3218 100644 --- a/website/locales/fr/messages.po +++ b/website/locales/fr/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Réflexions sur l'ingénierie inverse." #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Reconstruction du jeu{ph0}Essai" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "La révolution industrielle dans l'ingénierie inverse" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -53,19 +53,19 @@ msgstr "" "TH04 m'ont appris sur les agents, la vérification et les connaissances qui " "se combinent dans les projets." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Lire l'article" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Par N0zoM1z0 · 10 octobre 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -236,7 +236,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Suivre REA sur X" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Haut de page" @@ -11854,7 +11854,7 @@ msgctxt "shared" msgid "Guides" msgstr "Guides" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11874,7 +11874,7 @@ msgctxt "shared" msgid "Source" msgstr "Code source" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11894,7 +11894,7 @@ msgctxt "shared" msgid "MIT license" msgstr "Licence MIT" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11914,7 +11914,7 @@ msgctxt "shared" msgid "Join the community" msgstr "Rejoindre la communauté" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12885,6 +12885,32 @@ msgstr "" "connectez le serveur MCP d’Everywhere pour inspecter le trafic capturé avec " "votre agent." +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "Modèles spécialisés{ph0}Essai" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Le reverse engineering a-t-il besoin d’un modèle plus grand ?" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "" +"Ce que la reconstruction de Touhou suggère sur les modèles généralistes, les" +" modèles spécialisés et l’apprentissage à partir des retours du compilateur." + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "Par N0zoM1z0 · 13 octobre 2026 · Brouillon à relire" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Guide d’installation" diff --git a/website/locales/id/messages.po b/website/locales/id/messages.po index 6faa06dad..5ad209867 100644 --- a/website/locales/id/messages.po +++ b/website/locales/id/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Pemikiran tentang rekayasa terbalik." #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Rekonstruksi permainan{ph0}Esai" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "Revolusi industri dalam rekayasa terbalik" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -53,19 +53,19 @@ msgstr "" " tentang agen, verifikasi, dan pengetahuan yang digabungkan di seluruh " "proyek." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Baca artikelnya" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Oleh N0zoM1z0 · 10 Oktober 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -236,7 +236,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Ikuti REA di X" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Kembali ke atas" @@ -11733,7 +11733,7 @@ msgctxt "shared" msgid "Guides" msgstr "Panduan" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11753,7 +11753,7 @@ msgctxt "shared" msgid "Source" msgstr "Kode sumber" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11773,7 +11773,7 @@ msgctxt "shared" msgid "MIT license" msgstr "Lisensi MIT" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11793,7 +11793,7 @@ msgctxt "shared" msgid "Join the community" msgstr "Gabung komunitas" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12758,6 +12758,32 @@ msgstr "" "server MCP Everywhere untuk memeriksa lalu lintas yang direkam bersama agen " "Anda." +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "Model khusus{ph0}Esai" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Apakah rekayasa balik membutuhkan model yang lebih besar?" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "" +"Pelajaran dari rekonstruksi Touhou tentang model umum, model khusus, dan " +"pembelajaran dari umpan balik kompiler." + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "Oleh N0zoM1z0 · 13 Oktober 2026 · Draf untuk ditinjau" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Panduan pemasangan" diff --git a/website/locales/it/messages.po b/website/locales/it/messages.po index fdebe81d8..068419f41 100644 --- a/website/locales/it/messages.po +++ b/website/locales/it/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Riflessioni sul reverse engineering." #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Gioco ricostruzione{ph0}Saggio" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "La rivoluzione industriale nel reverse engineering" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,19 @@ msgstr "" "Che manuale RE e ricostruire TH08, TH095 e TH04 mi ha insegnato circa gli " "agenti, la verifica e la conoscenza che composti attraverso i progetti." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Leggi l'articolo" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Di N0zoM1z0 * 10 ottobre 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -235,7 +235,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Segui REA su X" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Torna in alto" @@ -11791,7 +11791,7 @@ msgctxt "shared" msgid "Guides" msgstr "Guide" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11811,7 +11811,7 @@ msgctxt "shared" msgid "Source" msgstr "Codice sorgente" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11831,7 +11831,7 @@ msgctxt "shared" msgid "MIT license" msgstr "Licenza MIT" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11851,7 +11851,7 @@ msgctxt "shared" msgid "Join the community" msgstr "Unisciti alla community" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12819,6 +12819,32 @@ msgstr "" "offline, oppure collega il server MCP di Everywhere per esaminare il " "traffico acquisito con il tuo agente." +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "Modelli specializzati{ph0}Saggio" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Il reverse engineering ha bisogno di un modello più grande?" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "" +"Cosa suggerisce la ricostruzione di Touhou sui modelli generali, quelli " +"specializzati e l’apprendimento dal feedback del compilatore." + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "Di N0zoM1z0 · 13 ottobre 2026 · Bozza da rivedere" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Guida all’installazione" diff --git a/website/locales/ja/messages.po b/website/locales/ja/messages.po index 10ed8ee27..a49063596 100644 --- a/website/locales/ja/messages.po +++ b/website/locales/ja/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "リバースエンジニアリングに関する考え。" #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "ゲーム再構築{ph0}小論文" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "リバースエンジニアリングにおける産業革命" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -51,19 +51,19 @@ msgid "" msgstr "" "どのようなマニュアル再とTH08を再構築,TH095とTH04は、エージェントについて教えてくれました,プロジェクト間で化合物の検証と知識." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "記事を読む" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "によってN0zoM1z0•10 10月2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -230,7 +230,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "X で REA をフォロー" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "ページ上部へ" @@ -10965,7 +10965,7 @@ msgctxt "shared" msgid "Guides" msgstr "ガイド" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -10985,7 +10985,7 @@ msgctxt "shared" msgid "Source" msgstr "ソースコード" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11005,7 +11005,7 @@ msgctxt "shared" msgid "MIT license" msgstr "MIT ライセンス" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11025,7 +11025,7 @@ msgctxt "shared" msgid "Join the community" msgstr "コミュニティに参加" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -11963,6 +11963,30 @@ msgstr "" "キャプチャ、MCP、公開スキル{ph1}を利用してください。SAZ ??? HAR 1.2 をエクスポートすれば REA でオフラインのキャプチャ " "Evidence を取得できます。また、Everywhere の MCP サーバーを接続して、エージェントと一緒に記録済みの通信を調べることもできます。" +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "モデルの特化{ph0}エッセイ" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "リバースエンジニアリングに、より大きなモデルは必要か？" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "Touhou の再構築を通じて考える、汎用モデル、特化モデル、そしてコンパイラのフィードバックから学ぶ方法。" + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "N0zoM1z0 著 · 2026年10月13日 · レビュー用草稿" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "セットアップガイド" diff --git a/website/locales/ko/messages.po b/website/locales/ko/messages.po index efcf4a708..0e8786ec5 100644 --- a/website/locales/ko/messages.po +++ b/website/locales/ko/messages.po @@ -32,37 +32,37 @@ msgid "Thoughts on reverse engineering." msgstr "리버스 엔지니어링에 대한 생각." #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "게임 재구성{ph0}에세이" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "리버스 엔지니어링의 산업 혁명" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " "agents, verification and knowledge that compounds across projects." msgstr "어떤 매뉴얼 다시고 재구성TH08,TH095및TH04에 대해 가르쳐 에이전트,검증 및 지는 화합물에 걸쳐 프로젝트입니다." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "기사 읽기" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "여N0zoM1z0·10 월 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -229,7 +229,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "X에서 REA 팔로우" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "맨 위로" @@ -10931,7 +10931,7 @@ msgctxt "shared" msgid "Guides" msgstr "가이드" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -10951,7 +10951,7 @@ msgctxt "shared" msgid "Source" msgstr "소스 코드" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -10971,7 +10971,7 @@ msgctxt "shared" msgid "MIT license" msgstr "MIT 라이선스" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -10991,7 +10991,7 @@ msgctxt "shared" msgid "Join the community" msgstr "커뮤니티 참여" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -11927,6 +11927,30 @@ msgstr "" "사용하세요. SAZ ?? HAR 1.2로 내보내면 REA에서 오프라인 캡처 Evidence를 얻을 수 있으며, Everywhere의 " "MCP 서버를 연결하여 에이전트와 함께 캡처된 트래픽을 살펴볼 수도 있습니다." +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "전문 모델{ph0}에세이" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "리버스 엔지니어링에는 더 큰 모델이 필요할까?" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "Touhou 재구축 경험을 바탕으로 범용 모델, 전문 모델, 컴파일러 피드백을 통한 학습을 생각해 봅니다." + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "작성: N0zoM1z0 · 2026년 10월 13일 · 검토용 초안" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "설정 가이드" diff --git a/website/locales/messages.pot b/website/locales/messages.pot index ab1a4a17e..92a5109c3 100644 --- a/website/locales/messages.pot +++ b/website/locales/messages.pot @@ -101,32 +101,56 @@ msgstr "" #: website/public/blog/index.html:200 #, python-brace-format msgctxt "page:blog/index.html" -msgid "Game reconstruction{ph0}Essay" +msgid "Model specialization{ph0}Essay" msgstr "" #: website/public/blog/index.html:202 msgctxt "page:blog/index.html" -msgid "The industrial revolution in reverse engineering" +msgid "Does reverse engineering need a bigger model?" msgstr "" #: website/public/blog/index.html:203 msgctxt "page:blog/index.html" msgid "" +"What Touhou reconstruction suggests about general models, specialist models and " +"learning from compiler feedback." +msgstr "" + +#: website/public/blog/index.html:207 website/public/blog/index.html:224 +msgctxt "page:blog/index.html" +msgid "Read the article" +msgstr "" + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "" + +#. {ph0}:
+#: website/public/blog/index.html:216 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Game reconstruction{ph0}Essay" +msgstr "" + +#: website/public/blog/index.html:218 +msgctxt "page:blog/index.html" +msgid "The industrial revolution in reverse engineering" +msgstr "" + +#: website/public/blog/index.html:219 +msgctxt "page:blog/index.html" +msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about agents, " "verification and knowledge that compounds across projects." msgstr "" -#: website/public/blog/index.html:208 -msgctxt "page:blog/index.html" -msgid "Read the article" -msgstr "" - -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 @@ -143,7 +167,7 @@ msgctxt "shared" msgid "Source" msgstr "" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 @@ -160,7 +184,7 @@ msgctxt "shared" msgid "MIT license" msgstr "" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 @@ -177,7 +201,7 @@ msgctxt "shared" msgid "Join the community" msgstr "" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 @@ -196,7 +220,7 @@ msgstr "" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 @@ -337,7 +361,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "" diff --git a/website/locales/pl/messages.po b/website/locales/pl/messages.po index 3a6657178..e4bdee584 100644 --- a/website/locales/pl/messages.po +++ b/website/locales/pl/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Myśli o inżynierii odwrotnej." #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Rekonstrukcja gry{ph0}Esej" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "Rewolucja przemysłowa w inżynierii odwrotnej" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,19 @@ msgstr "" "Co Instrukcja RE i rekonstrukcji TH08, TH095 i TH04 nauczył mnie O agentów, " "weryfikacja i wiedza, że związki całej projektów." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Przeczytaj artykuł" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Przez N0zoM1z0 * 10 października 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -235,7 +235,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Obserwuj REA na X" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Do góry" @@ -11697,7 +11697,7 @@ msgctxt "shared" msgid "Guides" msgstr "Poradniki" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11717,7 +11717,7 @@ msgctxt "shared" msgid "Source" msgstr "Kod źródłowy" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11737,7 +11737,7 @@ msgctxt "shared" msgid "MIT license" msgstr "Licencja MIT" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11757,7 +11757,7 @@ msgctxt "shared" msgid "Join the community" msgstr "Dołącz do społeczności" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12723,6 +12723,32 @@ msgstr "" "1.2, aby uzyskać Evidence z zapisu w REA offline, lub połącz serwer MCP " "Everywhere, aby analizować zapisany ruch z agentem." +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "Modele specjalistyczne{ph0}Esej" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Czy inżynieria odwrotna potrzebuje większego modelu?" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "" +"Co rekonstrukcja Touhou mówi o modelach ogólnych, modelach specjalistycznych" +" i uczeniu się na podstawie informacji zwrotnych z kompilatora." + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "Autor: N0zoM1z0 · 13 października 2026 · Szkic do przejrzenia" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Instrukcja instalacji" diff --git a/website/locales/pt-BR/messages.po b/website/locales/pt-BR/messages.po index a0741271d..408dff1d9 100644 --- a/website/locales/pt-BR/messages.po +++ b/website/locales/pt-BR/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Reflexões sobre engenharia reversa." #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Reconstrução do jogo{ph0}Ensaio" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "A Revolução industrial na engenharia reversa" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,19 @@ msgstr "" "O manual RE e reconstrução TH08, TH095 e TH04 me ensinou sobre agentes, " "verificação e conhecimento que compostos entre projetos." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Leia o artigo" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Por N0zoM1z0 * 10 de outubro de 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -235,7 +235,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Siga o REA no X" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Voltar ao topo" @@ -11774,7 +11774,7 @@ msgctxt "shared" msgid "Guides" msgstr "Guias" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11794,7 +11794,7 @@ msgctxt "shared" msgid "Source" msgstr "Código-fonte" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11814,7 +11814,7 @@ msgctxt "shared" msgid "MIT license" msgstr "Licença MIT" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11834,7 +11834,7 @@ msgctxt "shared" msgid "Join the community" msgstr "Participe da comunidade" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12798,6 +12798,32 @@ msgstr "" "servidor MCP do Everywhere para inspecionar o tráfego capturado com seu " "agente." +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "Modelos especializados{ph0}Ensaio" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "A engenharia reversa precisa de um modelo maior?" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "" +"O que a reconstrução de Touhou sugere sobre modelos gerais, modelos " +"especializados e o aprendizado com o retorno do compilador." + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "Por N0zoM1z0 · 13 de outubro de 2026 · Rascunho para revisão" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Guia de instalação" diff --git a/website/locales/ru/messages.po b/website/locales/ru/messages.po index 9f0486916..ed9c8005c 100644 --- a/website/locales/ru/messages.po +++ b/website/locales/ru/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Мысли о реверс-инжиниринге." #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Реконструкция игры{ph0}Сочинение" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "Промышленная революция в области реверс-инжиниринга" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,19 @@ msgstr "" "Чему меня научило руководство по обновлению и реконструкции TH08, TH095 и " "TH04 об агентах, проверке и знаниях, которые используются в разных проектах." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Прочтите статью" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Автор: N0zoM1z0 · 10 октября 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -235,7 +235,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Подписаться на REA в X" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Наверх" @@ -11811,7 +11811,7 @@ msgctxt "shared" msgid "Guides" msgstr "Руководства" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11831,7 +11831,7 @@ msgctxt "shared" msgid "Source" msgstr "Исходный код" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11851,7 +11851,7 @@ msgctxt "shared" msgid "MIT license" msgstr "Лицензия MIT" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11871,7 +11871,7 @@ msgctxt "shared" msgid "Join the community" msgstr "Вступить в сообщество" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12840,6 +12840,32 @@ msgstr "" "подключите MCP-сервер Everywhere, чтобы исследовать записанный трафик вместе" " с агентом." +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "Специализация моделей{ph0}Эссе" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Нужна ли для обратной разработки более крупная модель?" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "" +"Что реконструкция Touhou помогает понять об универсальных и " +"специализированных моделях и обучении на обратной связи от компилятора." + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "Автор: N0zoM1z0 · 13 октября 2026 г. · Черновик для обсуждения" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Руководство по установке" diff --git a/website/locales/th/messages.po b/website/locales/th/messages.po index 94e6d1873..9b63e4e3e 100644 --- a/website/locales/th/messages.po +++ b/website/locales/th/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "ความคิดเกี่ยวกับวิศวกรรมย้อนกลับ" #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "ฟื้นฟูเกม{ph0}เรียงความ" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "การปฏิวัติอุตสาหกรรมในวิศวกรรมย้อนกลับ" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,19 @@ msgstr "" "สิ่งที่คู่มือใหม่และ " "reconstructingTH08,TH095และTH04สอนฉันเกี่ยวกับเจ้าหน้าที่ปลายทางและความรู้นั่นรประกอบที่อีกฝั่งยิ่งสนง.โครงการ." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "อ่านบทความ" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "โดย N0zoM1z0 ·10 ตุลาคม 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -234,7 +234,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "ติดตาม REA บน X" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "กลับขึ้นด้านบน" @@ -11455,7 +11455,7 @@ msgctxt "shared" msgid "Guides" msgstr "คู่มือ" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11475,7 +11475,7 @@ msgctxt "shared" msgid "Source" msgstr "ซอร์สโค้ด" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11495,7 +11495,7 @@ msgctxt "shared" msgid "MIT license" msgstr "สัญญาอนุญาต MIT" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11515,7 +11515,7 @@ msgctxt "shared" msgid "Join the community" msgstr "เข้าร่วมชุมชน" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12477,6 +12477,32 @@ msgstr "" "หรือเชื่อมต่อเซิร์ฟเวอร์ MCP ของ Everywhere " "เพื่อตรวจสอบทราฟฟิกที่บันทึกไว้ร่วมกับเอเจนต์ของคุณ" +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "โมเดลเฉพาะทาง{ph0}บทความ" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "การวิศวกรรมย้อนกลับจำเป็นต้องใช้โมเดลที่ใหญ่ขึ้นไหม?" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "" +"ข้อคิดจากการสร้าง Touhou ขึ้นใหม่ เกี่ยวกับโมเดลทั่วไป โมเดลเฉพาะทาง " +"และการเรียนรู้จากผลตอบกลับของคอมไพเลอร์" + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "โดย N0zoM1z0 · 13 ตุลาคม 2026 · ฉบับร่างสำหรับตรวจทาน" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "คู่มือการติดตั้ง" diff --git a/website/locales/tr/messages.po b/website/locales/tr/messages.po index e4d0ba4a0..da4084320 100644 --- a/website/locales/tr/messages.po +++ b/website/locales/tr/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Tersine mühendislik üzerine düşünceler." #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Oyun rekonstrüksiyonu{ph0}Deneme" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "Tersine mühendislikte endüstri devrimi" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -53,19 +53,19 @@ msgstr "" "yapılandırılması bana projeler arasında birleşen aracılar, doğrulama ve " "bilgi hakkında bilgi verdi." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Makaleyi okuyun" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Başlangıç N0zoM1z0 * 10 Ekim 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -236,7 +236,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "REA’yı X’te takip et" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Başa dön" @@ -11684,7 +11684,7 @@ msgctxt "shared" msgid "Guides" msgstr "Rehberler" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11704,7 +11704,7 @@ msgctxt "shared" msgid "Source" msgstr "Kaynak kod" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11724,7 +11724,7 @@ msgctxt "shared" msgid "MIT license" msgstr "MIT lisansı" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11744,7 +11744,7 @@ msgctxt "shared" msgid "Join the community" msgstr "Topluluğa katıl" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12706,6 +12706,32 @@ msgstr "" " kaydedilen trafiği ajanınızla incelemek için Everywhere MCP sunucusunu " "bağlayın." +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "Uzmanlaşmış modeller{ph0}Deneme" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Tersine mühendislik için daha büyük bir model gerekli mi?" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "" +"Touhou yeniden yapılandırma deneyiminden genel amaçlı modeller, uzmanlaşmış " +"modeller ve derleyici geri bildiriminden öğrenme üzerine düşünceler." + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "N0zoM1z0 · 13 Ekim 2026 · İnceleme taslağı" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Kurulum rehberi" diff --git a/website/locales/uk/messages.po b/website/locales/uk/messages.po index a5fae5c80..34131bdfa 100644 --- a/website/locales/uk/messages.po +++ b/website/locales/uk/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Думки про реверс-інжиніринг." #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Реконструкція гри{ph0}Вигадування" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "Промислова революція в області реверс-інжинірингу" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,19 @@ msgstr "" "Чому мене навчив посібник з оновлення та реконструкції TH08, TH095 та TH04 " "про агентів, перевірку та знання, які використовуються в різних проектах." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Прочитайте статтю" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Автор: N0zoM1z0 · 10 жовтня 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -235,7 +235,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Стежити за REA в X" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Нагору" @@ -11763,7 +11763,7 @@ msgctxt "shared" msgid "Guides" msgstr "Посібники" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11783,7 +11783,7 @@ msgctxt "shared" msgid "Source" msgstr "Вихідний код" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11803,7 +11803,7 @@ msgctxt "shared" msgid "MIT license" msgstr "Ліцензія MIT" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11823,7 +11823,7 @@ msgctxt "shared" msgid "Join the community" msgstr "Приєднатися до спільноти" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12791,6 +12791,32 @@ msgstr "" "мережі або підключіть MCP-сервер Everywhere, щоб досліджувати записаний " "трафік разом з агентом." +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "Спеціалізація моделей{ph0}Есе" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Чи потрібна для зворотної розробки більша модель?" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "" +"Що реконструкція Touhou допомагає зрозуміти про універсальні й " +"спеціалізовані моделі та навчання на зворотному зв’язку від компілятора." + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "Автор: N0zoM1z0 · 13 жовтня 2026 р. · Чернетка для обговорення" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Посібник зі встановлення" diff --git a/website/locales/vi/messages.po b/website/locales/vi/messages.po index d9dd57fb0..e34a78e22 100644 --- a/website/locales/vi/messages.po +++ b/website/locales/vi/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Suy nghĩ về kỹ thuật đảo ngược." #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Tái thiết trò chơi{ph0}Bài luận" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "Cuộc cách mạng công nghiệp trong kỹ thuật đảo ngược" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,19 @@ msgstr "" "NHỮNG gì HƯỚNG DẪN SỬ DỤNG LẠI VÀ tái TẠO TH08, TH095 và TH04 đã dạy tôi về " "các agent, xác minh và kiến thức mà các hợp chất trong các dự án." -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Đọc bài viết" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Bởi N0zoM1z0 * 10 tháng mười 2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -235,7 +235,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Theo DÕI REA Trên X" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Về đầu trang" @@ -11673,7 +11673,7 @@ msgctxt "shared" msgid "Guides" msgstr "Hướng dẫn" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11693,7 +11693,7 @@ msgctxt "shared" msgid "Source" msgstr "Mã nguồn" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11713,7 +11713,7 @@ msgctxt "shared" msgid "MIT license" msgstr "Giấy phép MIT" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11733,7 +11733,7 @@ msgctxt "shared" msgid "Join the community" msgstr "Tham gia cộng đồng" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12692,6 +12692,32 @@ msgstr "" "1.2 để lấy Evidence từ bản ghi trong REA ngoại tuyến, hoặc kết nối máy chủ " "MCP của Everywhere để kiểm tra lưu lượng đã ghi cùng trợ lý của bạn." +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "Mô hình chuyên biệt{ph0}Bài viết" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Kỹ thuật đảo ngược có cần mô hình lớn hơn không?" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "" +"Những suy nghĩ từ việc tái dựng Touhou về mô hình tổng quát, mô hình chuyên " +"biệt và cách học từ phản hồi của trình biên dịch." + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "N0zoM1z0 · 13 tháng 10 năm 2026 · Bản nháp để góp ý" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "Hướng dẫn cài đặt" diff --git a/website/locales/zh-CN/messages.po b/website/locales/zh-CN/messages.po index 91728965c..74350a21c 100644 --- a/website/locales/zh-CN/messages.po +++ b/website/locales/zh-CN/messages.po @@ -32,37 +32,37 @@ msgid "Thoughts on reverse engineering." msgstr "逆向工程的思考。" #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "游戏重建{ph0}论文" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "逆向工程的工业革命" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " "agents, verification and knowledge that compounds across projects." msgstr "什么手动重新和重建TH08，TH095和TH04教我关于编程助手，验证和知识，跨项目的化合物。" -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "阅读文章" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "通过N0zoM1z0*10十月2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -229,7 +229,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "在 X 上关注 REA" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "回到顶部" @@ -10694,7 +10694,7 @@ msgctxt "shared" msgid "Guides" msgstr "指南" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -10714,7 +10714,7 @@ msgctxt "shared" msgid "Source" msgstr "源代码" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -10734,7 +10734,7 @@ msgctxt "shared" msgid "MIT license" msgstr "MIT 许可证" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -10754,7 +10754,7 @@ msgctxt "shared" msgid "Join the community" msgstr "加入社区" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -11681,6 +11681,30 @@ msgstr "" "要检查 HTTP 请求和响应正文，请使用{ph0}Fiddler Everywhere：SAZ 捕获、MCP 和公开技能{ph1}。导出 SAZ ? " "HAR 1.2 可在 REA 中离线获取捕获 Evidence，也可以连接 Everywhere 的 MCP 服务器，让智能体检查已捕获的流量。" +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "专用模型{ph0}随笔" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "逆向工程需要更大的模型吗？" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "从 Touhou 重建的经验，探讨通用模型、专用模型，以及如何从编译器反馈中学习。" + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "作者 N0zoM1z0 · 2026 年 10 月 13 日 · 待审草稿" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "安装指南" diff --git a/website/locales/zh-TW/messages.po b/website/locales/zh-TW/messages.po index 875779f14..6c2790888 100644 --- a/website/locales/zh-TW/messages.po +++ b/website/locales/zh-TW/messages.po @@ -32,37 +32,37 @@ msgid "Thoughts on reverse engineering." msgstr "逆向工程的思考。" #. {ph0}:
-#: website/public/blog/index.html:200 +#: website/public/blog/index.html:216 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "遊戲重建{ph0}論文" -#: website/public/blog/index.html:202 +#: website/public/blog/index.html:218 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "逆向工程的工業革命" -#: website/public/blog/index.html:203 +#: website/public/blog/index.html:219 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " "agents, verification and knowledge that compounds across projects." msgstr "什麼手動重新和重建TH08，TH095和TH04教我關於程式設計助手，驗證和知識，跨專案的化合物。" -#: website/public/blog/index.html:208 +#: website/public/blog/index.html:207 website/public/blog/index.html:224 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "閱讀文章" -#: website/public/blog/index.html:211 +#: website/public/blog/index.html:227 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "透過N0zoM1z0*10十月2026" #. {ph0}: -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -229,7 +229,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "在 X 上關注 REA" -#: website/public/blog/index.html:230 +#: website/public/blog/index.html:246 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "回到頂部" @@ -10713,7 +10713,7 @@ msgctxt "shared" msgid "Guides" msgstr "指南" -#: website/public/blog/index.html:222 +#: website/public/blog/index.html:238 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -10733,7 +10733,7 @@ msgctxt "shared" msgid "Source" msgstr "原始碼" -#: website/public/blog/index.html:223 +#: website/public/blog/index.html:239 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -10753,7 +10753,7 @@ msgctxt "shared" msgid "MIT license" msgstr "MIT 許可證" -#: website/public/blog/index.html:224 +#: website/public/blog/index.html:240 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -10773,7 +10773,7 @@ msgctxt "shared" msgid "Join the community" msgstr "加入社群" -#: website/public/blog/index.html:225 +#: website/public/blog/index.html:241 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -11701,6 +11701,30 @@ msgstr "" "若要檢查 HTTP 請求與回應本文，請使用{ph0}Fiddler Everywhere：SAZ 擷取、MCP 與公開技能{ph1}。匯出 SAZ ? " "HAR 1.2 可在 REA 中離線取得擷取 Evidence，也可以連接 Everywhere 的 MCP 伺服器，讓代理檢查已擷取的流量。" +#. {ph0}:
+#: website/public/blog/index.html:200 +#, python-brace-format +msgctxt "page:blog/index.html" +msgid "Model specialization{ph0}Essay" +msgstr "專用模型{ph0}隨筆" + +#: website/public/blog/index.html:202 +msgctxt "page:blog/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "逆向工程需要更大的模型嗎？" + +#: website/public/blog/index.html:203 +msgctxt "page:blog/index.html" +msgid "" +"What Touhou reconstruction suggests about general models, specialist models " +"and learning from compiler feedback." +msgstr "從 Touhou 重建的經驗，探討通用模型、專用模型，以及如何從編譯器回饋中學習。" + +#: website/public/blog/index.html:210 +msgctxt "page:blog/index.html" +msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgstr "作者 N0zoM1z0 · 2026 年 10 月 13 日 · 待審草稿" + #~ msgctxt "page:index.html" #~ msgid "Setup guide" #~ msgstr "安裝指南" diff --git a/website/public/blog/index.html b/website/public/blog/index.html index 3ad338db8..8cacf9779 100644 --- a/website/public/blog/index.html +++ b/website/public/blog/index.html @@ -196,6 +196,22 @@

Thoughts on reverse engineering.

+
+

Model specialization
Essay

+
+

Does reverse engineering need a bigger model?

+

+ What Touhou reconstruction suggests about general models, + specialist models and learning from compiler feedback. +

+ Read the article +

+ By N0zoM1z0 · 13 October 2026 · Draft for review +

+
+

Game reconstruction
Essay

diff --git a/website/public/blog/reverse-engineering-models/index.html b/website/public/blog/reverse-engineering-models/index.html new file mode 100644 index 000000000..52df2ccdf --- /dev/null +++ b/website/public/blog/reverse-engineering-models/index.html @@ -0,0 +1,810 @@ + + + + + + Does reverse engineering need a bigger model? | REA + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ REAReverse Engineer Anything + +
+ + + + + + + + + + + + +
+
+
+ +
+
+

+ Home / Blog / Models and RE +

+

Essay · Draft for review

+

Does reverse engineering need a bigger model?

+

By N0zoM1z0 · 13 October 2026

+
+

Takeaway

+
    +
  • + Model capability changes the hypotheses an agent can + find. + Later sessions have opened functions that earlier work left + unresolved. +
    • +
  • + A familiar compiler gives specialization something concrete to + learn. + Some exact matches depend on narrow, reusable source patterns. +
    • +
  • + Give repository knowledge a fair comparison. + A smaller model with good retrieval may already know enough to + make progress. +
    • +
  • + Train the next useful experiment, and test its oracle. + The value of feedback depends on what the checker actually + verifies. +
    • +
  • + Measure accepted reconstruction against time and cost. + That is the result I want from the whole agent. +
    • +
+
+
+
+ +
+
+

What better models changed

+

+ Since August 2026, I have been exploring how much reverse + engineering an agent can carry out with access to a program and a + way to check its work. Most of my experience comes from + reconstructing games, especially Touhou. The projects have also + become a record of what different models could make progress on. +

+

+ Looking back through that record, there are functions that later + models closed after earlier sessions had left them unresolved. The + source changes make those breakthroughs more interesting than a + progress percentage does. +

+

The temporary values had to stay alive together

+

+ One TH09 attack function was a byte short. Much of the logic was + already in place. The stack frame had the right size. But three + temporary vectors occupied different slots from the original, and + the compiled function still did not match. +

+

+ A + GPT-6-Sol checkpoint + recorded that residual after testing several ordinary source + arrangements. Later, + GPT-6.1-Sol found a useful lifetime hypothesis. +

+

+ The function calculated three vectors and copied each into an + animation's position. C++ keeps a returned temporary alive until + the end of the expression using it. Expressing the real operations + together kept those values alive together. The compiler could no + longer reuse the same storage for all three. +

+

+ A separate, real collision workspace also needed to keep its own + storage. With that source arrangement, the seventh targeted + candidate in the closing session matched. A cold build confirmed + the complete 1,436-byte function and all 44 encoded + fields. Here, an exact reconstruction means that the maintained source, + compiled under the pinned historical toolchain, reproduces the + checked target function. +

+

+ It was a useful discovery followed by several targeted tests. The + final result depended on both. +

+

A different control-flow graph

+

+ Another TH09 function had a different obstacle. + GPT-6.1-Sol's earlier work + reproduced two diagnostic regions, but the complete candidate was + 1,510 bytes against a 1,491-byte target. The way its branches + shared a continuation still differed. +

+

+ GPT-6-Astra changed the surrounding control-flow graph. Completion and frame processing became complementary branches, + followed by one common frame increment and return. In that + context, the compiler produced the required block sharing. The + entire 1,491-byte function matched, and a + recompile replay was accepted. +

+

+ A different representation made a different result + possible. + That is one reason I take general model capability seriously. An + agent can spend a long time trying variations within a source + structure that is keeping it stuck. +

+

+ These histories also contain accumulated work. Later sessions + inherited source and recorded failed experiments. They show the + discoveries and how we checked them. A controlled comparison would + tell us how much came from changing the model. +

+

+ Even so, the cases leave me with a question. How much of this work + needs a model with broad reasoning ability? How much could a model + learn from repeated experience with this particular kind of code? +

+
+
+

Why might a specialist do well?

+

+ Touhou is an interesting place to ask. Much of the game code comes + from the same author, ZUN. That gives us a reasonable expectation + of recurring habits. The repositories contain more concrete + examples: related loops, helper functions and source arrangements + have supplied useful hypotheses across games. +

+

+ The compiler contributes its own habits. Under a particular + version and set of options, small differences in source can + produce repeatable differences in machine code. +

+

+ TH095 gave us a wonderfully awkward example: + even the identifier backing a real local variable could change + where VC7.1 put it on the stack. Moving the declaration alone did not solve the layout. An + earlier background function had already exposed the relevant + identifier bucket, and that experience became useful in + the reconstruction of Supervisor::LoadDat. +

+

+ Selecting the known backing identifier for the real file-size + variable put it and the character buffer in the required homes. + The buffer remained its genuine 128 bytes. All 151 bytes of the + function and its 15 relocations matched. +

+

+ That is narrow knowledge. A general model has little reason to + arrive with a detailed repertoire of one old compiler's local + allocation behavior. A specialist could encounter it repeatedly. + It could learn which source variations are worth testing first. +

+

+ In many of these final matching problems, we already understand + the behavior. The remaining question is which ordinary source form + this compiler turns into these instructions. That gives + specialization a specific job: propose plausible forms, then let + the target decide which survive. +

+

+ There are limits to how narrowly we can define the repertoire. + TH08 uses VC7. TH095 and TH09 use VC7.1. TH04 uses Turbo C++ in a + segmented DOS environment. Its + player reconstruction + needed another set of source-lifetime and linker-order hypotheses. + Shared authorship gives us a prior; each target tells us whether + that prior applies. +

+
+
+

Where should that experience live?

+

+ Before training anything, there is an obvious place to look: the + repository. In the + previous essay, I wrote + about it becoming the project's working memory. Compiler lessons + belong there too. +

+

+ An agent can retrieve a known result. It can inspect the source + that produced it and read which alternatives failed. That changes + its starting point without changing its weights. +

+

+ A TH09 + GPT-6-Luna-max checkpoint + makes this worth taking seriously. It closed a 373-byte + enemy-spawn function using an ordinary loop with branch-local + failure handling. TH08's corresponding source family supplied the + hypothesis. TH09's own evidence established its bounds and calls, + and two cold builds confirmed the complete match. +

+

+ “The model did not know that compiler quirk” and “the model + could not use it” are different problems. + Supplying a fact may solve the first. Generating a useful source + candidate with that fact may still need practice. Choosing what to + try after a failed compile is another skill again. +

+

+ I would want a smaller general model with good retrieval in any + comparison. Otherwise, we might credit fine-tuning for an + advantage that came from giving one model a better library of + experience. +

+
+
+

Learning the next useful experiment

+

+ In agentic reconstruction, the model proposes an explanation. The + program and its checks determine whether that explanation + survives. A failed candidate can be useful when it helps + distinguish the remaining possibilities. +

+

+ This makes the temporary-vector example especially interesting to + me. The final source contains the successful arrangement. It does + not contain everything the search taught us. +

+
+ + + + + + + + + + + + + + + + + + + + + + + + + +
ExperimentWhat its result taught us
Use a typed position assignment + The copy behavior improves, but the compiled source still + differs. +
Consume the returned vectors directly + The function reaches the target size. Temporary storage + remains wrong. +
Group the real operations into one full expressionThe returned values remain alive together.
Keep the real collision workspace separateThe complete function matches in a cold build.
+
+

+ Selected steps from the recorded Type18/24 search. The closing + session tested seven candidates; this table groups the lessons. +

+

+ A useful training record would preserve the proposed change and + the result. It would also preserve why the result justified the + next experiment. An ineffective variation is part of that + experience, especially if a later agent would otherwise repeat it. +

+

+ I would want the model to learn how to use feedback. + It should recognize when a change has no effect. It should notice + when a new mismatch points to a different cause. It should be able + to leave an unproductive source representation and inspect the + evidence again. +

+

+ That also means keeping autonomy. The agent may need another + caller, a compiler experiment or an earlier project's source. A + rigid sequence of required rewrites would prevent some of the + decisions we want it to learn. +

+
+
+

Why RE is interesting for reinforcement learning

+

+ A bounded reconstruction problem has an attractive property. With + the target and compiler pinned, many experiments produce quick, + repeatable feedback. Compile a candidate. Inspect what differs. + Decide what to try next. +

+

+ Reinforcement learning could use that feedback to train the + decisions an agent makes during reconstruction. There is a + concrete result to optimize: a candidate accepted by a defined + check. We can also observe how much work it took to get there. +

+

+ This direction already has research behind it. + LLM4Decompile + studies models specialized for decompilation. + AutoDecompiler + trains multi-turn refinement using compiler and execution + feedback. Those results make the idea worth investigating. Their + functional recovery tasks differ from matching code emitted by a + particular historical compiler. +

+

+ The experiment I have in mind would train a policy for that + narrower search. Given what we have learned so far, which source + hypothesis is worth testing next? +

+

+ Other parts of RE are harder to package into a cheap loop. We may + still be establishing what a function owns. We may need a new + runtime measurement. In TH08, a + later collision divergence led back to a one-ULP numerical + difference + and the old x87 behavior. The productive hypothesis there was + partly about what to measure. +

+

+ That is another plausible place for a broad model's strength. + First make the problem checkable. Then ask how much a learned + search policy can do with that check. +

+
+
+

When the oracle is wrong

+

+ An oracle is the reference or checker we use to accept a result. + Its answer can be repeatable while its coverage is incomplete. +

+

+ TH08 already gave us an example. Some reconstructed functions had + passed their exact comparisons with wrong floating constants. The + instruction check accounted for relocated addresses, but checking + the literal payload depended on optional explicit attestation. + That left references whose values were never checked. +

+

+ A + wider audit + examined all 1,548 floating-literal references and corrected + twelve stale rows across five accepted functions. In one case, the + bullet-spread increment was 1.0f where the target + required 32.0f. +

+

+ A learner rewarded by the old check could learn to produce the + wrong reconstruction. + That is a consequence worth considering when we design an RL + environment. The observed problem here was an oracle defect; we + had not trained an RL model against it. +

+

+ Functional checks need the same attention. + Differential-fuzzing research on LLM decompilers + has found behavior differences beyond the supplied tests. An + accepted test result tells us something about the cases the test + actually exercises. +

+

+ Before using a comparator as a reward, I would give it deliberate + failures to reject. The final acceptance check should stay outside + the candidate's editable source. A byte array copied from the + target may satisfy a naive score while teaching us nothing about + reconstructing the source. +

+

+ Improving the learner and improving its oracle belong to the + same research problem. + Greater ability to optimize a check makes understanding that check + more important. +

+
+
+

A comparison worth running

+

+ I would start with a fixed VC7.1 task family. A strong general + model would be one baseline. A smaller general model would be + another. Then I would give the smaller model the relevant + repository knowledge and compare that with supervised + specialization, followed by an RL-trained version. +

+

+ Each would get the same target evidence and tools. The + training-specific comparisons would use the same base model and + available knowledge. I would repeat the trials, since one lucky + candidate is a poor description of a search policy. +

+

+ The starting point matters. Reconstructing a function from its + instructions and closing the last four differences in maintained + source are two useful tasks. I would measure both separately. A + specialist might be excellent at the second while a general model + does much of the work needed to reach it. +

+

+ The test set also needs care. Related Touhou games share engine + code. Holding out a game's name would not be enough if nearly + identical functions had appeared in training. I would hold out + source families and keep their accepted answers out of both + training and retrieval. +

+

+ For the outcome, I would count complete, independently validated + matches. One view would give every model the same candidate or + compile budget. Another would measure time and total cost to an + accepted result, including compilation. Training costs would need + their own accounting too. +

+

+ A cheaper proposal can become an expensive workflow if it causes + more uninformative compiles. A slower model may pay for itself by + finding the right experiment sooner. Those are outcomes we can + measure. +

+

+ I would also be cautious with a reward for “getting closer.” One + TH09 + replay-save candidate + already had the target's 1,825-byte length while 467 byte + positions differed. A shorter mismatch list can also move abruptly + when a branch encoding changes. Diagnostics help choose an + experiment; the full acceptance check decides whether it worked. +

+

+ That experiment could produce an answer with conditions. A + specialist might win within a familiar compiler family. A general + model might win when the toolchain changes. Retrieval might + account for much of the apparent benefit of training. Each result + would tell us something useful about where to invest next. +

+
+
+

What kind of agent does this suggest?

+

+ Perhaps the useful arrangement is a broad agent with access to a + specialist. It can decide when the task has become a familiar + compiler search, and ask the specialist for candidates. It can + still inspect the target or question the measurement when the + results stop making sense. +

+

+ That would make specialization a tool the agent can choose. It + would preserve the freedom that made these investigations + productive in the first place. +

+

+ In the previous essay, I described an industrial shift in how + reconstruction work can be organized. This question follows from + it. Once we can run many more experiments, we need to understand + which decisions require broad judgment and which improve through + repeated experience with a particular craft. +

+

+ The strongest general model has earned its place in my workflow. + It has opened problems I had left unresolved. What I want to find + out next is how much of that work requires its breadth, and how + much could be handled by a model that has learned this particular + kind of reconstruction. +

+

+ The repositories give us a way to test the question. + That is where I would like to take it next. +

+
+
+
+ Evidence and model attribution +

+ The examples describe historical reconstruction checks recorded + in public commits. Their byte counts apply to the stated + functions, not whole-game runtime correctness. Source and + matching records were reviewed on 13 October 2026. +

+

+ The TH09 label gpt-dots identifies GPT-6-Astra. + gpt-web changed with the model used at the time, so + that label alone does not identify an individual run. Explicit + model names in the other cited commits establish their recorded + attribution; the histories do not contain uniform per-run cost + or effort measurements. +

+ +

+ The proposed specialist and RL comparisons are future + experiments. The project histories motivate them; no trained + specialist was evaluated for this essay. +

+
+
+
+
+
+ + + +

+ + Top + + + + From 8583a190c0178374132c888622d693fed1f88a18 Mon Sep 17 00:00:00 2001 From: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:07:12 +0800 Subject: [PATCH 091/159] fix(website): keep omitted language selectors free of whitespace Indent the optional language selector with its value so a nonindexable draft does not generate a line containing only spaces. Indexed-page markup is unchanged. Validation: website.py check passes all 12 maintenance regressions; synchronization changes only the draft header blank line. --- scripts/website.py | 2 +- website/public/blog/reverse-engineering-models/index.html | 2 +- website/templates/header.html.tmpl | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/website.py b/scripts/website.py index 1d6661fc1..22c0829b9 100644 --- a/scripts/website.py +++ b/scripts/website.py @@ -114,7 +114,7 @@ def shared_blocks(root, path, source): } from website_i18n import language_markup selector, alternates = language_markup(root, path.relative_to(site).as_posix()) - values["language_picker"] = selector if not metadata.noindex else "" + values["language_picker"] = indent(selector, " ") if not metadata.noindex else "" values["alternates"] = alternates if not metadata.noindex else "" templates = root / "website/templates" return { diff --git a/website/public/blog/reverse-engineering-models/index.html b/website/public/blog/reverse-engineering-models/index.html index 52df2ccdf..a3bbb712a 100644 --- a/website/public/blog/reverse-engineering-models/index.html +++ b/website/public/blog/reverse-engineering-models/index.html @@ -79,7 +79,7 @@ FAQ
- +

Why might a specialist do well?

- Touhou is an interesting place to ask. Much of the game code comes - from the same author, ZUN. That gives us a reasonable expectation - of recurring habits. The repositories contain more concrete - examples: related loops, helper functions and source arrangements - have supplied useful hypotheses across games. + Touhou gives us a concrete reason to expect that experience to + help. Much of the game code comes from the same author, ZUN, so + recurring habits are a reasonable starting assumption. The + repositories back that up with useful examples: related loops and + helper arrangements have supplied hypotheses across games.

- The compiler contributes its own habits. Under a particular - version and set of options, small differences in source can - produce repeatable differences in machine code. + Those source patterns become more specific when we include the + compiler. Under a particular version and set of options, small + changes in source can produce repeatable differences in machine + code. Some of those differences are surprisingly obscure.

- TH095 gave us a wonderfully awkward example: + In TH095, for example, even the identifier backing a real local variable could change where VC7.1 put it on the stack. Moving the declaration alone did not solve the layout. An - earlier background function had already exposed the relevant - identifier bucket, and that experience became useful in + >. Moving the declaration alone did not solve the layout, but an + earlier background function had exposed a useful identifier + bucket. That gave us a more promising hypothesis for the reconstruction of Supervisor::LoadDat.

- Selecting the known backing identifier for the real file-size - variable put it and the character buffer in the required homes. - The buffer remained its genuine 128 bytes. All 151 bytes of the - function and its 15 relocations matched. + Reusing the known backing identifier for the real file-size + variable put it and the character buffer in the required homes, + without changing the buffer's genuine 128-byte size. With that + layout, all 151 bytes of the function and its 15 relocations + matched.

- That is narrow knowledge. A general model has little reason to - arrive with a detailed repertoire of one old compiler's local - allocation behavior. A specialist could encounter it repeatedly. - It could learn which source variations are worth testing first. + This is the sort of narrow detail a general model may not know + when it arrives. A specialist trained on this compiler could + encounter many such cases and learn which source variations are + worth testing first.

- In many of these final matching problems, we already understand - the behavior. The remaining question is which ordinary source form - this compiler turns into these instructions. That gives - specialization a specific job: propose plausible forms, then let - the target decide which survive. + The opportunity is especially clear in final matching problems + where we already understand the behavior. We are then asking which + ordinary source form this compiler turns into the observed + instructions. Specialization could help the model propose those + forms more efficiently, while the target still determines which + hypotheses survive.

- There are limits to how narrowly we can define the repertoire. - TH08 uses VC7. TH095 and TH09 use VC7.1. TH04 uses Turbo C++ in a - segmented DOS environment. Its + The useful patterns would have to retain their context, though. + TH08 uses VC7, while TH095 and TH09 use VC7.1. TH04's segmented + DOS environment uses Turbo C++, and its player reconstruction - needed another set of source-lifetime and linker-order hypotheses. - Shared authorship gives us a prior; each target tells us whether - that prior applies. + needed a different set of lifetime and linker-order hypotheses. + Shared authorship gives us a prior, but each target tells us + whether it applies. Distinguishing those contexts is part of what + a specialist would need to learn.

Where should that experience live?

- Before training anything, there is an obvious place to look: the - repository. In the + Before deciding to teach those patterns through model training, I + would look at the knowledge we already keep in the repository. In + the previous essay, I wrote - about it becoming the project's working memory. Compiler lessons - belong there too. + about it becoming the project's working memory. That memory + includes compiler lessons, along with the source and failed + experiments that made them useful.

- An agent can retrieve a known result. It can inspect the source - that produced it and read which alternatives failed. That changes - its starting point without changing its weights. + By retrieving that record, an agent can start with a known result + and inspect how we reached it. It can also avoid alternatives that + have already failed in the same context. The starting point + improves without changing the model's weights.

- A TH09 + For the 373-byte TH09 enemy-spawn function, a GPT-6-Luna-max checkpointGPT-6-Luna-max session - makes this worth taking seriously. It closed a 373-byte - enemy-spawn function using an ordinary loop with branch-local - failure handling. TH08's corresponding source family supplied the - hypothesis. TH09's own evidence established its bounds and calls, - and two cold builds confirmed the complete match. + drew a hypothesis from TH08's corresponding source family. That + earlier source suggested an ordinary loop with branch-local + failure handling. TH09's own evidence supplied the bounds and + calls, and two cold builds confirmed the complete match.

“The model did not know that compiler quirk” and “the model could not use it” are different problems. - Supplying a fact may solve the first. Generating a useful source - candidate with that fact may still need practice. Choosing what to - try after a failed compile is another skill again. + Retrieval may solve the first, but turning that fact into a useful + candidate still takes work. And if the candidate fails, the model + must decide what the result means for its next attempt. Knowing + the relevant pattern is only the beginning of that process.

- I would want a smaller general model with good retrieval in any - comparison. Otherwise, we might credit fine-tuning for an - advantage that came from giving one model a better library of - experience. + For that reason, I would include a smaller general model with good + retrieval in any comparison. Giving it the same library of + experience would help us identify what training adds: better + candidate generation, better decisions after feedback, or both.

Learning the next useful experiment

- In agentic reconstruction, the model proposes an explanation. The - program and its checks determine whether that explanation - survives. A failed candidate can be useful when it helps - distinguish the remaining possibilities. + Those decisions after feedback are what I would most like to + study. In agentic reconstruction, the model proposes an + explanation and tests it against the program. Even when the + candidate fails, the result can narrow the remaining possibilities + and suggest a more useful next experiment.

- This makes the temporary-vector example especially interesting to - me. The final source contains the successful arrangement. It does - not contain everything the search taught us. + Consider the temporary-vector example again. Its final source + shows the arrangement that worked, but the route to that source + contains more information. Each of the following experiments + helped identify what still needed to change.

- A useful training record would preserve the proposed change and - the result. It would also preserve why the result justified the - next experiment. An ineffective variation is part of that - experience, especially if a later agent would otherwise repeat it. + A training record that keeps this sequence can teach more than the + final source alone. It connects each proposed change to an + observed result and the next decision. Keeping ineffective + variations matters too: they tell a later agent which hypotheses + have already been tested in that context.

I would want the model to learn how to use feedback. - It should recognize when a change has no effect. It should notice - when a new mismatch points to a different cause. It should be able - to leave an unproductive source representation and inspect the - evidence again. + A neutral result is a reason to reconsider the variation; a + changed mismatch may point to another cause. When local edits stop + being informative, the agent should be able to revisit the source + representation or inspect the evidence again.

- That also means keeping autonomy. The agent may need another - caller, a compiler experiment or an earlier project's source. A - rigid sequence of required rewrites would prevent some of the - decisions we want it to learn. + Learning to make those choices requires autonomy. Sometimes the + useful next step is inspecting another caller rather than + compiling another edit. A rigid sequence of required rewrites + would remove that choice, along with part of the behavior we want + the model to learn.

Why RE is interesting for reinforcement learning

- A bounded reconstruction problem has an attractive property. With - the target and compiler pinned, many experiments produce quick, - repeatable feedback. Compile a candidate. Inspect what differs. - Decide what to try next. + Bounded reconstruction gives us a practical environment for + studying these choices. Once the target and compiler are pinned, + many source experiments produce quick, repeatable feedback. The + agent can compile a candidate, inspect the mismatch and use that + observation to choose its next step.

- Reinforcement learning could use that feedback to train the - decisions an agent makes during reconstruction. There is a - concrete result to optimize: a candidate accepted by a defined - check. We can also observe how much work it took to get there. + Reinforcement learning could train that decision-making process + through the feedback it receives. A candidate accepted by a + defined check gives us a concrete outcome, while the sequence of + attempts lets us observe how much work it took to reach it.

- This direction already has research behind it. + There is already research supporting this direction. LLM4Decompile studies models specialized for decompilation. AutoDecompiler trains multi-turn refinement using compiler and execution - feedback. Those results make the idea worth investigating. Their - functional recovery tasks differ from matching code emitted by a - particular historical compiler. + feedback. Their studies focus on functional recovery. For this + work, I would use them as a starting point for the related task of + finding source that matches a historical compiler's output.

- The experiment I have in mind would train a policy for that - narrower search. Given what we have learned so far, which source - hypothesis is worth testing next? -

-

- Other parts of RE are harder to package into a cheap loop. We may - still be establishing what a function owns. We may need a new - runtime measurement. In TH08, a + To reach such a well-defined task, though, we may first need to + establish what a function owns or devise a new runtime + measurement. In TH08, for example, a later collision divergence led back to a one-ULP numerical difference - and the old x87 behavior. The productive hypothesis there was - partly about what to measure. + and the old x87 behavior. Before the agent could test a useful + correction, it had to work out what to measure.

- That is another plausible place for a broad model's strength. - First make the problem checkable. Then ask how much a learned - search policy can do with that check. + This suggests another role for a broadly capable model: helping + make the problem checkable in the first place. Once that check + exists, we can investigate how far a learned search policy can get + with it.

When the oracle is wrong

- An oracle is the reference or checker we use to accept a result. - Its answer can be repeatable while its coverage is incomplete. + Once we train against that check, its coverage becomes part of + what the model learns. I call the reference or checker an oracle. + Even if it gives a repeatable answer, we still need to understand + what that answer covers.

- TH08 already gave us an example. Some reconstructed functions had - passed their exact comparisons with wrong floating constants. The - instruction check accounted for relocated addresses, but checking - the literal payload depended on optional explicit attestation. - That left references whose values were never checked. + TH08 showed how much can hide outside that coverage. Some + reconstructed functions passed their exact comparisons with wrong + floating constants. The comparator adjusted relocated addresses + before checking instructions, but it checked the values behind + those addresses only when they had been explicitly recorded for + verification. Other literal references could pass without their + values being examined.

- A + To close that gap, a wider audit - examined all 1,548 floating-literal references and corrected - twelve stale rows across five accepted functions. In one case, the - bullet-spread increment was 1.0f where the target + examined all 1,548 floating-literal references. It corrected + twelve stale rows across five accepted functions, including a + bullet-spread increment of 1.0f where the target required 32.0f.

@@ -573,130 +600,141 @@ >A learner rewarded by the old check could learn to produce the wrong reconstruction. - That is a consequence worth considering when we design an RL - environment. The observed problem here was an oracle defect; we - had not trained an RL model against it. + We had not trained an RL model against this oracle; the observed + failure was in the check itself. But using that check as a reward + would have made the same blind spot part of the learning + environment.

- Functional checks need the same attention. + A functional test can have a similar blind spot. Differential-fuzzing research on LLM decompilers - has found behavior differences beyond the supplied tests. An - accepted test result tells us something about the cases the test - actually exercises. + has found behavior differences beyond the supplied tests. Passing + those tests therefore needs to be read in terms of the cases they + exercise, just as an instruction match needs to be read in terms + of the bytes and data it checks.

- Before using a comparator as a reward, I would give it deliberate - failures to reject. The final acceptance check should stay outside - the candidate's editable source. A byte array copied from the - target may satisfy a naive score while teaching us nothing about - reconstructing the source. + Before using either kind of check as a reward, I would test it + with deliberate failures. I would also keep final acceptance + outside the candidate's editable source, so the model has to + satisfy the check through its reconstruction. Otherwise, something + like a byte array copied from the target could satisfy a naive + score without recovering useful source.

Improving the learner and improving its oracle belong to the same research problem. - Greater ability to optimize a check makes understanding that check - more important. + As the learner gets better at optimizing the feedback we give it, + understanding that feedback becomes more important too.

A comparison worth running

- I would start with a fixed VC7.1 task family. A strong general - model would be one baseline. A smaller general model would be - another. Then I would give the smaller model the relevant - repository knowledge and compare that with supervised - specialization, followed by an RL-trained version. + With those checks in place, I would start the comparison on a + fixed VC7.1 task family. First, I would establish baselines for a + strong general model and a smaller one. Adding repository + knowledge to the smaller model would show what retrieval changes. + From there, supervised specialization and an RL-trained version + would let us examine what training adds.

- Each would get the same target evidence and tools. The - training-specific comparisons would use the same base model and - available knowledge. I would repeat the trials, since one lucky - candidate is a poor description of a search policy. + To make those comparisons informative, each model would receive + the same target evidence and tools. For the training-specific + comparisons, I would keep the base model and available knowledge + fixed as well. Repeated trials would help distinguish a reliable + search strategy from a lucky candidate.

- The starting point matters. Reconstructing a function from its - instructions and closing the last four differences in maintained - source are two useful tasks. I would measure both separately. A - specialist might be excellent at the second while a general model - does much of the work needed to reach it. + I would then separate two starting conditions: reconstructing a + function from its instructions, and closing the last few + differences in maintained source. A specialist might be excellent + at the second task while a general model does much of the work + needed to reach it. Measuring them separately would reveal that + division of work.

- The test set also needs care. Related Touhou games share engine - code. Holding out a game's name would not be enough if nearly - identical functions had appeared in training. I would hold out - source families and keep their accepted answers out of both - training and retrieval. + Both tasks need genuinely held-out examples. Because related + Touhou games share engine code, holding out a game's name would + still allow nearly identical functions to appear in training. I + would instead hold out source families and keep their accepted + answers out of both training and retrieval.

- For the outcome, I would count complete, independently validated - matches. One view would give every model the same candidate or - compile budget. Another would measure time and total cost to an - accepted result, including compilation. Training costs would need + On those examples, the outcome would be complete, independently + validated matches. Giving each model the same candidate or compile + budget would help compare search quality. Measuring elapsed time + and total cost, including compilation, would tell us whether that + quality produces an efficient workflow. Training costs would need their own accounting too.

- A cheaper proposal can become an expensive workflow if it causes - more uninformative compiles. A slower model may pay for itself by - finding the right experiment sooner. Those are outcomes we can - measure. + Both views matter because a cheaper proposal can become an + expensive workflow if it causes more uninformative compiles. + Conversely, a slower model may pay for itself by finding a useful + experiment sooner. The comparison should capture those costs.

- I would also be cautious with a reward for “getting closer.” One - TH09 + Counting accepted results also keeps intermediate progress in + perspective. One TH09 replay-save candidate - already had the target's 1,825-byte length while 467 byte - positions differed. A shorter mismatch list can also move abruptly - when a branch encoding changes. Diagnostics help choose an - experiment; the full acceptance check decides whether it worked. + already had the target's 1,825-byte length, yet 467 byte positions + differed. Even a byte-difference count can change abruptly when a + branch encoding changes. Such diagnostics can guide the next + experiment, but I would validate any reward for “getting closer” + against the full acceptance check.

- That experiment could produce an answer with conditions. A - specialist might win within a familiar compiler family. A general - model might win when the toolchain changes. Retrieval might - account for much of the apparent benefit of training. Each result - would tell us something useful about where to invest next. + I would be interested in the pattern across these results. If a + specialist does well within a familiar compiler family, does that + advantage survive a toolchain change? And once both models have + good retrieval, how much does training still add? The answers + would help us decide where each kind of model is most useful.

What kind of agent does this suggest?

- Perhaps the useful arrangement is a broad agent with access to a - specialist. It can decide when the task has become a familiar - compiler search, and ask the specialist for candidates. It can - still inspect the target or question the measurement when the - results stop making sense. + If the models turn out to help at different stages, a useful + arrangement might be a broad agent with access to a specialist. + Once a task becomes a familiar compiler search, it could ask the + specialist for candidates. If the results stop making sense, the + broader agent could return to the target evidence or investigate + the measurement itself.

- That would make specialization a tool the agent can choose. It - would preserve the freedom that made these investigations - productive in the first place. + In that arrangement, specialization becomes a tool the agent can + choose as the investigation develops. The agent keeps the freedom + to change direction that made the earlier breakthroughs possible.

- In the previous essay, I described an industrial shift in how - reconstruction work can be organized. This question follows from - it. Once we can run many more experiments, we need to understand - which decisions require broad judgment and which improve through + This follows naturally from the industrial shift I described in + the previous essay. Being able to run many more experiments + changes how we organize the work. It also gives us a reason to ask + which decisions need broad judgment and which improve through repeated experience with a particular craft.

- The strongest general model has earned its place in my workflow. - It has opened problems I had left unresolved. What I want to find - out next is how much of that work requires its breadth, and how - much could be handled by a model that has learned this particular - kind of reconstruction. + For now, the strongest general model has earned its place in my + workflow by opening problems I had left unresolved. Those + successes are what make me want to examine the work more closely: + how much depended on the model's breadth, and how much could be + handled by a model trained in this particular kind of + reconstruction?

The repositories give us a way to test the question. - That is where I would like to take it next. + I would like to use that record to build the comparison and see + what we learn.

From 806ff575676e447c7ba349b6fd185e6e8e02ee5d Mon Sep 17 00:00:00 2001 From: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:57:39 +0800 Subject: [PATCH 093/159] docs(website): use straight quotes in model essay Normalize the three quoted phrases to straight English quotation marks. Website checks and formatting pass; the local preview serves the revised punctuation. --- website/public/blog/reverse-engineering-models/index.html | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/public/blog/reverse-engineering-models/index.html b/website/public/blog/reverse-engineering-models/index.html index 0ec0f9a6f..2b3c4c1ce 100644 --- a/website/public/blog/reverse-engineering-models/index.html +++ b/website/public/blog/reverse-engineering-models/index.html @@ -426,8 +426,8 @@

“The model did not know that compiler quirk” and “the model - could not use it” are different problems."The model did not know that compiler quirk" and "the model + could not use it" are different problems. Retrieval may solve the first, but turning that fact into a useful candidate still takes work. And if the candidate fails, the model @@ -688,7 +688,7 @@ already had the target's 1,825-byte length, yet 467 byte positions differed. Even a byte-difference count can change abruptly when a branch encoding changes. Such diagnostics can guide the next - experiment, but I would validate any reward for “getting closer” + experiment, but I would validate any reward for "getting closer" against the full acceptance check.

From b06751b703181527a320226c4ab7e4cecd5dcea0 Mon Sep 17 00:00:00 2001 From: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 16:19:13 +0800 Subject: [PATCH 094/159] docs(website): simplify reconstruction evidence in model essay Explain each case through the obstacle, useful hypothesis and verified result. Remove byte totals, relocation counts and trial counts from the main narrative; preserve commit references and the constant-value example that explains the oracle defect. Make the experiment table and compiler terminology easier to follow. Validation: website.py check, formatting, preserved source links, and eight article layout checks across both preview paths and light/dark themes. --- .../reverse-engineering-models/index.html | 157 +++++++++--------- 1 file changed, 76 insertions(+), 81 deletions(-) diff --git a/website/public/blog/reverse-engineering-models/index.html b/website/public/blog/reverse-engineering-models/index.html index 2b3c4c1ce..9a94ee811 100644 --- a/website/public/blog/reverse-engineering-models/index.html +++ b/website/public/blog/reverse-engineering-models/index.html @@ -167,7 +167,7 @@ >Model capability changes the hypotheses an agent can find. - Later sessions have opened functions that earlier work left + Later sessions have solved problems that earlier work left unresolved.

  • @@ -229,18 +229,17 @@

    Much of that work aims at exact reconstruction: we compile the maintained source under a pinned historical toolchain and check - whether it reproduces the target function. That gives us something - concrete to examine when a later model closes a function that - earlier work left unresolved. The source changes show what made - the breakthrough possible. + whether it reproduces the target function. When a later model + succeeds where earlier work stalled, the source changes show what + made that progress possible.

    The temporary values had to stay alive together

    - In one TH09 attack function, much of the logic was already in - place, but the compiled candidate remained a byte short. Although - its stack frame had the right size, three temporary vectors - occupied different slots from the original. Matching the overall - size of the workspace had concealed a difference inside it. + Much of a TH09 attack function's logic had already been + reconstructed, but the compiler still stored temporary results + differently from the original. The remaining problem was how long + those results needed to stay alive and when their storage could be + reused.

    A @@ -257,22 +256,19 @@ values needed to stay alive.

    - Each vector was returned from a calculation and copied into an - animation's position. Because C++ keeps a returned temporary alive - until the end of the expression using it, grouping the real - operations into one expression kept the three values alive - together. The compiler could no longer reuse the same storage for - all of them. + The function produced temporary results and copied them into + animation positions. In C++, a temporary result stays alive until + the expression using it finishes. Grouping the real operations + into one expression kept those results alive together, so the + compiler could no longer reuse the same storage for all of them.

    - That recovered the distinct temporary slots, but one still shared - storage with a real collision workspace used later. Keeping that - workspace at function scope separated it from the returned values. - With both changes in place, the seventh targeted candidate in the - closing session matched. A cold build confirmed + One result still shared storage with a value used later for + collision checks. Giving that value its own lifetime resolved the + remaining overlap. After several targeted experiments, the complete 1,436-byte function and all 44 encoded - fieldsa clean rebuild confirmed that the full function + matched.

    @@ -290,22 +286,19 @@ href="https://github.com/N0zoM1z0/th09/commit/f8d12d72aaf02212ee1b585a8f3f86d7ed000269" >GPT-6.1-Sol's earlier work - reproduced two diagnostic regions, yet the complete candidate - remained 1,510 bytes against a 1,491-byte target. Those matching - regions did not resolve the way the branches joined up. + got parts of the function to match, but the branches still joined + up differently from the original.

    GPT-6-Astra changed the surrounding control-flow graph, making completion and frame processing complementary branches - followed by one common frame increment and return. Once the - compiler saw the branches in that context, it produced the - required block sharing. The entire 1,491-byte function matched, - and a + >. The main paths now led to a shared final update and return. In + that source structure, the compiler combined the branches in the + same way as the original. The complete function matched, and a recompile replay was acceptedseparate rebuild check confirmed it.

    @@ -351,21 +344,20 @@

    In TH095, for example, even the identifier backing a real local variable could change - where VC7.1 put it on the stackeven the name used for a local variable could change where + VC7.1 stored it. Moving the declaration alone did not solve the layout, but an - earlier background function had exposed a useful identifier - bucket. That gave us a more promising hypothesis for + earlier background function had revealed a naming pattern that + affected the compiler's choice. That gave us a more promising + hypothesis for the reconstruction of Supervisor::LoadDat.

    - Reusing the known backing identifier for the real file-size - variable put it and the character buffer in the required homes, - without changing the buffer's genuine 128-byte size. With that - layout, all 151 bytes of the function and its 15 relocations + Reusing that pattern for the file-size variable produced the + required layout without adding storage. The full function then matched.

    @@ -384,16 +376,17 @@

    The useful patterns would have to retain their context, though. - TH08 uses VC7, while TH095 and TH09 use VC7.1. TH04's segmented - DOS environment uses Turbo C++, and its + TH08 uses an earlier Visual C++ version than TH095 and TH09. TH04 + uses a different DOS-era compiler, and its player reconstruction - needed a different set of lifetime and linker-order hypotheses. - Shared authorship gives us a prior, but each target tells us - whether it applies. Distinguishing those contexts is part of what - a specialist would need to learn. + needed different hypotheses about source patterns and how the + compiled code was combined into an executable. Shared authorship + gives us a prior, but each target tells us whether it applies. + Distinguishing those contexts is part of what a specialist would + need to learn.

    • @@ -414,15 +407,15 @@ improves without changing the model's weights.

    - For the 373-byte TH09 enemy-spawn function, a + When reconstructing TH09's enemy-spawn function, a GPT-6-Luna-max session drew a hypothesis from TH08's corresponding source family. That - earlier source suggested an ordinary loop with branch-local - failure handling. TH09's own evidence supplied the bounds and - calls, and two cold builds confirmed the complete match. + earlier source suggested how to structure the loop and handle + failures within it. TH09's own evidence supplied the game-specific + details, and clean rebuilds confirmed the complete match.

    - Consider the temporary-vector example again. Its final source + Consider the temporary-results example again. Its final source shows the arrangement that worked, but the route to that source contains more information. Each of the following experiments helped identify what still needed to change. @@ -469,33 +462,35 @@

    - + - + - - + + - - + +
    Use a typed position assignmentTry a different way to copy the results - The copy behavior improves, but the compiled source still - differs. + The copy code changes, but the function still differs.
    Consume the returned vectors directlyUse the returned values directly - The function reaches the target size. Temporary storage - remains wrong. + Copying follows the original pattern, but temporary + storage still differs.
    Group the real operations into one full expressionThe returned values remain alive together.Keep the temporary results alive together + They get separate storage, but one still overlaps a value + used later for collision checks. +
    Keep the real collision workspace separateThe complete function matches in a cold build.Keep the collision-check value separateA clean rebuild confirms the complete match.

    - Selected steps from the recorded Type18/24 search. The closing - session tested seven candidates; this table groups the lessons. + Selected experiments from the attack-function reconstruction + above, grouped by what they taught us.

    A training record that keeps this sequence can teach more than the @@ -554,11 +549,10 @@ measurement. In TH08, for example, a later collision divergence led back to a one-ULP numerical - difference - and the old x87 behavior. Before the agent could test a useful - correction, it had to work out what to measure. + >collision difference led back to how the original program + handled floating-point calculations. Before the agent could test a useful correction, it had to work + out what to measure.

    This suggests another role for a broadly capable model: helping @@ -590,10 +584,9 @@ href="https://github.com/N0zoM1z0/th08/commit/a393f400786e4cc6cbb1000af0dad3276da12c78" >wider audit - examined all 1,548 floating-literal references. It corrected - twelve stale rows across five accepted functions, including a - bullet-spread increment of 1.0f where the target - required 32.0f. + found incorrect constants in functions that had already been + accepted. In one calculation, the reconstructed source added + 1.0f where the original added 32.0f.

    replay-save candidate - already had the target's 1,825-byte length, yet 467 byte positions - differed. Even a byte-difference count can change abruptly when a - branch encoding changes. Such diagnostics can guide the next - experiment, but I would validate any reward for "getting closer" - against the full acceptance check. + looked promising because the compiled function had the expected + size, but a full comparison still found substantial differences. + Changes in how the compiler arranges instructions can also make a + candidate look closer without resolving the underlying problem. + Such diagnostics can guide the next experiment, but I would + validate any reward for "getting closer" against the full + acceptance check.

    I would be interested in the pattern across these results. If a @@ -741,10 +736,10 @@

    Evidence and model attribution

    - The examples describe historical reconstruction checks recorded - in public commits. Their byte counts apply to the stated - functions, not whole-game runtime correctness. Source and - matching records were reviewed on 13 October 2026. + The examples describe historical function-level comparisons + recorded in the linked commits. Whole-game runtime checks are + separate milestones. Source and matching records were reviewed + on 13 October 2026.

    The TH09 label gpt-dots identifies GPT-6-Astra. From 3d78a57043eb6b92812c71bcb6d83753c3d48036 Mon Sep 17 00:00:00 2001 From: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com> Date: Tue, 13 Oct 2026 17:25:08 +0800 Subject: [PATCH 095/159] docs(website): publish model essay in all supported languages --- website/README.md | 2 +- .../evidence/reverse-engineering-models.md | 3 +- website/locales/ar/messages.po | 1315 ++++++++++++++- website/locales/de/messages.po | 1407 +++++++++++++++- website/locales/es/messages.po | 1396 +++++++++++++++- website/locales/fa/messages.po | 1328 +++++++++++++++- website/locales/fr/messages.po | 1412 ++++++++++++++++- website/locales/id/messages.po | 1387 +++++++++++++++- website/locales/it/messages.po | 1396 +++++++++++++++- website/locales/ja/messages.po | 1171 +++++++++++++- website/locales/ko/messages.po | 1222 +++++++++++++- website/locales/messages.pot | 1023 +++++++++++- website/locales/pl/messages.po | 1380 +++++++++++++++- website/locales/pt-BR/messages.po | 1382 +++++++++++++++- website/locales/ru/messages.po | 1381 +++++++++++++++- website/locales/th/messages.po | 1353 +++++++++++++++- website/locales/tr/messages.po | 1372 +++++++++++++++- website/locales/uk/messages.po | 1368 +++++++++++++++- website/locales/vi/messages.po | 1368 +++++++++++++++- website/locales/zh-CN/messages.po | 1146 ++++++++++++- website/locales/zh-TW/messages.po | 1146 ++++++++++++- website/public/blog/index.html | 4 +- .../reverse-engineering-models/index.html | 46 +- 23 files changed, 24753 insertions(+), 255 deletions(-) diff --git a/website/README.md b/website/README.md index 967d9c6f8..f09c7f603 100644 --- a/website/README.md +++ b/website/README.md @@ -131,7 +131,7 @@ describe its recorded investigation; update them only when repeating that work. - `public/showcase/index.html`: the case-study index. - `public/blog/index.html`: articles about reconstruction methods, ports, mods and reverse engineering. - `public/blog/touhou-reconstruction/index.html`: a personal essay about agent autonomy, oracle quality, knowledge carried between TH08 and TH095, PC-98 TH04, and the path from exact reconstruction to readable source and modern ports. -- `public/blog/reverse-engineering-models/index.html`: an English draft on general models, compiler specialization and learning useful reconstruction experiments, with historical evidence from TH08, TH095, TH04 and TH09. Model attribution and verification scope are in [evidence/reverse-engineering-models.md](evidence/reverse-engineering-models.md). +- `public/blog/reverse-engineering-models/index.html`: an essay on general models, compiler specialization and learning useful reconstruction experiments, with historical evidence from TH08, TH095, TH04 and TH09. Model attribution and verification scope are in [evidence/reverse-engineering-models.md](evidence/reverse-engineering-models.md). - `public/showcase/aegis/index.html`: Aegis's Android login-code calculation, with an adjustable clock and reference checks. - `public/showcase/dx-ball/index.html`: sound-pan investigation and project status. - `public/showcase/notion/index.html`: Notion's Electron clipboard bridge and rich clipboard format. diff --git a/website/evidence/reverse-engineering-models.md b/website/evidence/reverse-engineering-models.md index ede754cfa..069d6eea3 100644 --- a/website/evidence/reverse-engineering-models.md +++ b/website/evidence/reverse-engineering-models.md @@ -1,7 +1,8 @@ # General models and specialist models: article evidence Supporting record for `public/blog/reverse-engineering-models/index.html`. -The article is an English draft for review and remains `noindex`. +The article is published from the English source and translated through the +website's gettext catalogs. ## Research scope diff --git a/website/locales/ar/messages.po b/website/locales/ar/messages.po index ed491e991..558a8d5ea 100644 --- a/website/locales/ar/messages.po +++ b/website/locales/ar/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "أفكار حول الهندسة العكسية." #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "لعبة إعادة الإعمار{ph0}مقال" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "الثورة الصناعية في الهندسة العكسية" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,20 @@ msgstr "" "ما دليل إعادة تعمير TH08, TH095 و TH04 علمني عن وكلاء والتحقق المعرفة أن " "المركبات عبر المشاريع." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "اقرأ المقال" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "بواسطة N0zoM1z0 * 10 أكتوبر 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -100,6 +101,8 @@ msgstr "" "العفو الدولية." #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -134,6 +137,7 @@ msgid "" msgstr "REA - اكتشف كيف يعمل البرنامج. الهندسة العكسية مع مساعدك البرمجي." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -154,6 +158,7 @@ msgid "REA home" msgstr "الصفحة الرئيسية لـ REA" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -174,6 +179,7 @@ msgid "Main navigation" msgstr "التنقل الرئيسي" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -194,6 +200,7 @@ msgid "REA on GitHub" msgstr "REA على GitHub" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -214,6 +221,7 @@ msgid "Join REA on Discord" msgstr "انضم إلى REA على Discord" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -233,7 +241,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "تابع REA على X" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "العودة للأعلى" @@ -11384,6 +11392,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "نفس التباعد: 256 1 16 = 16 وحدة زاوية = 22.5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -11404,6 +11413,7 @@ msgid "Showcases" msgstr "أمثلة عملية" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -11424,6 +11434,7 @@ msgid "Blog" msgstr "المدونة" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -11444,6 +11455,7 @@ msgid "Get started" msgstr "ابدأ هنا" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -11463,7 +11475,8 @@ msgctxt "shared" msgid "Guides" msgstr "الأدلة" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11483,7 +11496,8 @@ msgctxt "shared" msgid "Source" msgstr "الكود المصدري" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11503,7 +11517,8 @@ msgctxt "shared" msgid "MIT license" msgstr "رخصة MIT" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11523,7 +11538,8 @@ msgctxt "shared" msgid "Join the community" msgstr "انضم إلى المجتمع" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12501,8 +12517,1281 @@ msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "بقلم N0zoM1z0 · 13 أكتوبر 2026 · مسودة للمراجعة" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "بواسطة N0zoM1z0 · 13 أكتوبر 2026" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "تخطي إلى المحتوى" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "" +"{ph0}الصفحة الرئيسية{ph1} / {ph2}المدونة{ph3} / النماذج والهندسة العكسية" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "مقال" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "هل تحتاج الهندسة العكسية إلى نموذج أكبر؟" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "بواسطة N0zoM1z0 · 13 أكتوبر 2026" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "أهم الأفكار" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}تؤثر قدرة النموذج على الفرضيات التي يمكن أن يقترحها وكيل الذكاء " +"الاصطناعي.{ph1} قامت الجلسات اللاحقة بحل المشكلات التي تركها العمل السابق " +"دون حل." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}يقدم المترجم المألوف أنماطًا محددة لنموذج متخصص للتعلم.{ph1} تعتمد بعض " +"المطابقات الدقيقة لرمز الآلة على أنماط محددة من كود المصدر قابلة لإعادة " +"الاستخدام." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}قم بتضمين معرفة المستودع في مقارنة عادلة.{ph1} قد يحتوي النموذج الأصغر " +"الذي يتمتع باسترجاع جيد للمعرفة على معلومات كافية لإحراز تقدم." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}قم بتدريب النموذج على اختيار التجربة المفيدة التالية، واختبار أداة " +"التحقق الخاص به.{ph1} تكون التعليقات مفيدة فقط إلى الحد الذي يتحقق فيه " +"المدقق من الأشياء الصحيحة." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}قم بتقييم نتائج إعادة بناء الشيفرة التي تم التحقق منها جنبًا إلى جنب مع" +" الوقت والتكلفة.{ph1} هذا ما أريد قياسه لسير عمل وكيل الذكاء الاصطناعي " +"الكامل." + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "في هذه المقالة" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}ما الذي غيّرته النماذج الأقوى؟{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}لماذا التخصص؟{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}أين ينبغي تخزين هذه التجربة؟{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}تعلم التجربة المفيدة التالية{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}الهندسة العكسية والتعلم المعزز{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}عندما يكون أداة التحقق خاطئًا{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}مقارنة تستحق التشغيل{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}ما هو نوع وكيل الذكاء الاصطناعي الذي يشير إليه هذا؟{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "ما الذي غيّرته النماذج الأقوى؟" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"منذ أغسطس 2026، كنت أستكشف مقدار الهندسة العكسية التي يمكن للوكيل تنفيذها من" +" خلال الوصول إلى البرنامج وطريقة التحقق من عمله. معظم خبرتي تأتي من إعادة " +"بناء الألعاب، وخاصة Touhou. وبما أنني قمت بتغيير النماذج، فقد احتفظت " +"المستودعات بسجل لما تمكنت كل جلسة من استعادته." + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"يهدف كثير من هذا العمل إلى إعادة بناء دقيقة: نترجم الشيفرة المصدرية التي " +"نحتفظ بها باستخدام أدوات من تلك الحقبة بإصدارات ثابتة، ثم نتحقق من أنها تعيد" +" إنتاج الدالة المستهدفة. عندما يحل نموذج أحدث مشكلة تعثر عندها العمل السابق،" +" تكشف تغييرات الشيفرة ما الذي أتاح هذا التقدم." + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "تحتاج المتغيرات المؤقتة إلى عمر متداخل" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"تمت إعادة بناء الكثير من منطق دالة الهجوم TH09 بالفعل، لكن المترجم ما زال " +"يخزن النتائج المؤقتة بشكل مختلف عن الأصل. وكانت المشكلة المتبقية هي المدة " +"التي تحتاجها تلك النتائج للبقاء على قيد الحياة، ومتى يمكن إعادة استخدام " +"مخزونها." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"أبلغ {ph0}GPT-6-Sol سجل الجلسة{ph1} عن عدم التطابق هذا بعد اختبار العديد من " +"ترتيبات كود المصدر العادية. عندما فحص {ph2}GPT-6.1-Sol المشكلة مرة " +"أخرى{ph3}، اقترح فرضية مفيدة حول عمر هذه المتغيرات المؤقتة." + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"أنتجت الوظيفة نتائج مؤقتة ونسختها في مواضع الرسوم المتحركة. في لغة C++، تظل " +"النتيجة المؤقتة حية حتى انتهاء التعبير الذي يستخدمها. يؤدي تجميع العمليات " +"الحقيقية في تعبير واحد إلى إبقاء هذه النتائج حية معًا، لذلك لم يعد بإمكان " +"المترجم إعادة استخدام نفس وحدة التخزين لجميعها." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"لا تزال إحدى النتائج مشتركة في مساحة التخزين مع القيمة المستخدمة لاحقًا " +"للتحقق من التضارب. إن إعطاء هذه القيمة مدى الحياة الخاص بها أدى إلى حل " +"التداخل المتبقي. وبعد عدة تجارب مستهدفة، {ph0}أكدت إعادة الترجمة النظيفة من " +"المصدر أن الوظيفة الكاملة مطابقة{ph1}." + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"أنا مهتم بكيفية دعم الاكتشاف والاختبارات لبعضهما البعض. أعطت الفرضية " +"المتعلقة بعمر الكائنات المؤقتة لعامل الذكاء الاصطناعي اتجاهًا جديدًا. ساعدت " +"تعليقات المترجم في تحسين هذه الفرضية إلى ترتيب كود المصدر المطابق." + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "رسم بياني مختلف للتحكم في التدفق" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"توضح وظيفة TH09 الثانية سبب أهمية هذا الاتجاه الجديد. وهنا، كانت العقبة هي " +"كيفية مشاركة العديد من الفروع لعملها النهائي. يحتوي العمل السابق لـ " +"{ph0}GPT-6.1-Sol{ph1} على أجزاء من الدالة متطابقة، ولكن لا تزال الفروع متصلة" +" بشكل مختلف عن الأصل." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra غيّر الرسم البياني لتدفق التحكم المحيط{ph1}. أدت المسارات " +"الرئيسية الآن إلى التحديث النهائي المشترك والعودة. في تلك البنية المصدرية، " +"قام المترجم بدمج الفروع بنفس طريقة الأصل. تمت مطابقة الوظيفة الكاملة، وأكدها" +" فحص إعادة البناء المنفصل {ph2}{ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}أدى تغيير التمثيل إلى فتح مساحة بحث أكثر إنتاجية.{ph1} تمنحني كلتا " +"الحالتين سببًا لأخذ إمكانات النموذج العامة على محمل الجد: يمكن للوكيل " +"الاستمرار في تجربة الاختلافات المحلية لفترة طويلة قبل العثور على طريقة أفضل " +"للتعبير عن العمل المحيط." + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"في الوقت نفسه، بدأت الجلسات اللاحقة بمصادر سابقة وإخفاقات مسجلة. وكان هذا " +"العمل المتراكم جزءًا من مصلحتهم. يتيح لنا التاريخ فحص الاكتشافات وفحصها؛ ومن" +" شأن المقارنة الخاضعة للرقابة أن تساعد في فصل تأثير النموذج عن الخبرة التي " +"ورثها." + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"وهذا ما يجعل السؤال مثيرًا للاهتمام بالنسبة لي. جاء بعض التقدم من إيجاد " +"تمثيل جديد. تضمن الكثير من العمل المتبقي اختبار ترتيبات مختلفة لشفرة المصدر " +"مقابل مترجم مألوف. كم من هذا يتطلب قدرة تفكير واسعة النطاق؟ ما مقدار ما يمكن" +" أن يتعلمه النموذج من خلال العمل المتكرر مع هذا المترجم والكود؟" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "لماذا قد يكون نموذج الذكاء الاصطناعي المتخصص ناجحًا؟" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou يعطينا سببًا ملموسًا لتوقع أن تساعدنا هذه التجربة. الكثير من رموز " +"اللعبة تأتي من نفس المؤلف، ZUN، لذا فإن العادات المتكررة هي افتراض بداية " +"معقول. تدعم المستودعات ذلك بأمثلة مفيدة: الحلقات ذات الصلة والترتيبات " +"المساعدة قدمت فرضيات عبر الألعاب." + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"تصبح أنماط المصدر هذه أكثر تحديدًا عندما نقوم بتضمين المترجم. ضمن إصدار معين" +" ومجموعة من الخيارات، يمكن أن تؤدي التغييرات الصغيرة في المصدر إلى اختلافات " +"متكررة في كود الجهاز. بعض هذه الاختلافات غامضة بشكل مدهش." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"في TH095، على سبيل المثال، {ph0}حتى اسم المتغير المحلي يمكن أن يؤثر على مكان" +" تخزينه VC7.1{ph1}. لم يؤدي نقل إعلانه إلى حل تخطيط التخزين. ومع ذلك، كشفت " +"وظيفة خلفية سابقة عن نمط تسمية أثر على اختيار المترجم. وقد اقترح ذلك فرضية " +"واعدة أكثر بشأن {ph2}إعادة بناء {ph3}{ph4}." + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"أدت إعادة استخدام هذا النمط لمتغير حجم الملف إلى إنتاج التخطيط المطلوب دون " +"إضافة مساحة تخزين. ثم تمت مطابقة الوظيفة الكاملة." + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"هذه تفاصيل محددة جدًا قد لا يعرفها نموذج الذكاء الاصطناعي للأغراض العامة " +"عندما يواجه المشكلة لأول مرة. يمكن للنموذج المتخصص لهذا المترجم أن يرى " +"العديد من هذه الحالات ويتعرف على اختلافات التعليمات البرمجية المصدر التي " +"تستحق الاختبار أولاً." + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"تكون الفرصة واضحة بشكل خاص في مشاكل المطابقة النهائية حيث نفهم السلوك " +"بالفعل. ثم نسأل عن ترتيب التعليمات البرمجية المصدر العادي الذي يحوله هذا " +"المترجم إلى التعليمات المرصودة. يمكن أن يساعد التخصص النموذج على اقتراح تلك " +"الأشكال بشكل أكثر كفاءة، في حين أن الهدف لا يزال يحدد الفرضيات التي ستبقى." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"يجب أن تحتفظ الأنماط بسياقها. يستخدم TH08 إصدارًا من Visual C++ أقدم مما " +"يستخدمه TH095 وTH09. أما TH04 فيستخدم مُترجمًا برمجيًا آخر من حقبة DOS، وقد " +"احتاجت {ph0}إعادة بناء شيفرة اللاعب{ph1} إلى فرضيات مختلفة عن أنماط الشيفرة " +"وكيفية ربطها في ملف تنفيذي. وحدة المؤلف تعطينا توقعًا أوليًا، لكن كل هدف " +"يبين إن كان ينطبق. وعلى النموذج المتخصص تعلم هذه الفروق." + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "أين ينبغي تخزين هذه التجربة؟" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"قبل أن أقرر تدريس تلك الأنماط من خلال التدريب النموذجي، أود أن ألقي نظرة على" +" المعرفة التي نحتفظ بها بالفعل في المستودع. في {ph0}المقال السابق{ph1}، كتبت" +" عن أنها أصبحت الذاكرة العاملة للمشروع. تتضمن تلك الذاكرة دروسًا مترجمة، " +"بالإضافة إلى المصدر والتجارب الفاشلة التي جعلتها مفيدة." + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"من خلال استرداد هذا السجل، يمكن للوكيل البدء بنتيجة معروفة وفحص كيفية وصولنا" +" إليها. ويمكنه أيضًا تجنب البدائل التي فشلت بالفعل في نفس السياق. تتحسن نقطة" +" البداية دون تغيير معلمات النموذج المدربة." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"أثناء إعادة بناء وظيفة TH09 لإنشاء الأعداء، أخذت جلسة {ph0}GPT-6-Luna-" +"max{ph1} فرضية من كود مصدر TH08 ذي الصلة. اقترح هذا الكود السابق كيفية تنظيم" +" الحلقة والتعامل مع حالات الفشل بداخلها. قدمت أدلة TH09 الخاصة التفاصيل " +"الخاصة باللعبة. أكدت عمليات إعادة البناء من المصدر أن الوظيفة الكاملة " +"متطابقة." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}\"النموذج لم يعرف أن المترجم غريب\" و\"النموذج لا يمكنه استخدامه\" " +"مشكلتان مختلفتان.{ph1} قد يحل الاسترجاع المشكلة الأولى، لكن تحويل هذه " +"الحقيقة إلى مرشح مفيد لا يزال يتطلب جهدًا. وإذا فشل الكود المصدري المرشح، " +"فيجب أن يقرر النموذج معنى النتيجة لمحاولته التالية. إن معرفة النمط ذي الصلة " +"ليست سوى بداية تلك العملية." + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"لهذا السبب، سأقوم بتضمين نموذج ذكاء اصطناعي أصغر حجمًا للأغراض العامة مع " +"استرجاع جيد في أي مقارنة. إن منحها نفس مكتبة الخبرة من شأنه أن يساعدنا في " +"تحديد ما يضيفه التدريب: جيل أفضل للمرشحين، أو قرارات أفضل بعد التعليقات، أو " +"كليهما." + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "تعلم التجربة المفيدة التالية" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"تلك القرارات بعد ردود الفعل هي أكثر ما أرغب في دراسته. في إعادة البناء " +"الوكيل، يقترح النموذج تفسيرًا ويختبره مقابل البرنامج. وحتى عندما يفشل الكود " +"المصدري المرشح، فإن النتيجة يمكن أن تضيق الاحتمالات المتبقية وتقترح تجربة " +"تالية أكثر فائدة." + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"خذ بعين الاعتبار مثال النتائج المؤقتة مرة أخرى. يُظهر مصدره النهائي الترتيب " +"الذي نجح، لكن المسار إلى هذا المصدر يحتوي على مزيد من المعلومات. ساعدت كل " +"تجربة من التجارب التالية في تحديد ما لا يزال بحاجة إلى التغيير." + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "تجربة" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "ماذا علمتنا نتيجته" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "جرب طريقة مختلفة لنسخ النتائج" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "يتغير رمز النسخ، لكن الوظيفة لا تزال مختلفة." + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "استخدم القيم التي تم إرجاعها مباشرة" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "يتبع النسخ النمط الأصلي، لكن التخزين المؤقت لا يزال مختلفًا." + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "إبقاء كافة المتغيرات المؤقتة صالحة في نفس الوقت" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "" +"يحصلون على مساحة تخزين منفصلة، ولكن لا يزال هناك تداخل مع قيمة يتم استخدامها" +" لاحقًا للتحقق من التصادم." + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "احتفظ بقيمة التحقق من التصادم منفصلة" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "تؤكد إعادة الترجمة النظيفة من المصدر التطابق الكامل." + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "" +"تجارب مختارة من إعادة بناء وظيفة الهجوم أعلاه، مجمعة حسب ما علمتنا إياه." + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"يمكن لسجل التدريب الذي يحتفظ بهذا التسلسل أن يعلم أكثر من المصدر النهائي " +"وحده. فهو يربط كل تغيير مقترح بالنتيجة الملحوظة والقرار التالي. إن الاحتفاظ " +"بالتغيرات غير الفعالة أمر مهم أيضًا: فهي تخبر وكيلًا لاحقًا بالفرضيات التي " +"تم اختبارها بالفعل في هذا السياق." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}أريد أن يتعلم النموذج كيفية استخدام التعليقات.{ph1} النتيجة المحايدة هي" +" سبب لإعادة النظر في الشكل؛ قد يشير عدم التطابق المتغير إلى سبب آخر. عندما " +"تتوقف التعديلات المحلية عن تقديم المعلومات، يجب أن يكون الوكيل قادرًا على " +"إعادة النظر في تمثيل المصدر أو فحص الأدلة مرة أخرى." + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"تعلم هذه الاختيارات يتطلب الاستقلالية. أحيانًا تكون الخطوة الأنسب فحص دالة " +"أخرى تستدعي هذه الدالة بدل ترجمة تعديل جديد. فرض سلسلة جامدة من إعادة " +"الكتابة يلغي هذا الاختيار ومعه جزءًا من السلوك الذي نريد أن يتعلمه النموذج." + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "لماذا تعتبر الهندسة العكسية مثيرة للاهتمام للتعلم المعزز" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"توفر إعادة البناء ذات النطاق المحدد بيئة عملية لدراسة هذه القرارات. بعد " +"تثبيت إصدارات الهدف والمترجم البرمجي، تعطي تجارب كثيرة على الشيفرة ملاحظات " +"سريعة وقابلة للتكرار. يستطيع الوكيل ترجمة شيفرة مرشحة وفحص الفروق ثم اختيار " +"الخطوة التالية." + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"يمكن للتعلم المعزز تدريب عملية صنع القرار من خلال ردود الفعل التي تتلقاها. " +"يمنحنا المرشح الذي يتم قبوله عن طريق فحص محدد نتيجة ملموسة، في حين يتيح لنا " +"تسلسل المحاولات ملاحظة مقدار العمل الذي استغرقه الوصول إلى هذه النتيجة." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"هناك أبحاث تدعم هذا الاتجاه بالفعل. يدرس {ph0}LLM4Decompile{ph1} نماذج " +"متخصصة في استعادة الشيفرة المصدرية من البرامج المترجمة. ويستخدم " +"{ph2}AutoDecompiler{ph3} نتائج الترجمة والتنفيذ لتدريب التحسين عبر جولات " +"متعددة. تركز هذه الدراسات على استعادة سلوك البرنامج. سأستخدمها نقطة بداية " +"لمهمة قريبة: إيجاد شيفرة تنتج مخرجات مطابقة لمترجم برمجي تاريخي." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"للوصول إلى مثل هذه المهمة المحددة جيدًا، قد نحتاج أولاً إلى تحديد الموارد " +"التي تمتلكها الوظيفة أو ابتكار قياس جديد لوقت التشغيل. في TH08، على سبيل " +"المثال، {ph0}أدى اختلاف التصادم إلى كيفية تعامل البرنامج الأصلي مع حسابات " +"الفاصلة العائمة{ph1}. قبل أن يتمكن الوكيل من اختبار تصحيح مفيد، كان عليه أن " +"يحدد ما يجب قياسه." + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"ويشير هذا إلى دور آخر لنموذج قادر على نطاق واسع: المساعدة في جعل المشكلة " +"قابلة للتحقق في المقام الأول. بمجرد وجود هذا الفحص، يمكننا التحقيق في المدى " +"الذي يمكن أن تصل إليه سياسة البحث المستفادة." + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "عندما يكون أداة التحقق خاطئًا" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"عندما نستخدم فحصًا في التدريب، يصبح نطاقه جزءًا من بيئة التعلم. أسمي هذا " +"المرجع أو أداة التحقق oracle. حتى الإجابة القابلة للتكرار يجب فهمها في حدود " +"ما تتحقق منه فعلًا." + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"أظهر TH08 مقدار ما يمكن إخفاءه خارج تلك التغطية. اجتازت بعض الوظائف المعاد " +"بناؤها مقارناتها الدقيقة مع ثوابت عائمة خاطئة. قامت أداة المقارنة بتعديل " +"العناوين المنقولة قبل التحقق من التعليمات، ولكنها لم تتحقق من القيم الموجودة" +" خلف تلك العناوين إلا عندما تم تسجيلها بشكل صريح للتحقق منها. يمكن أن تمر " +"المراجع الحرفية الأخرى دون فحص قيمها." + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"لسد هذه الفجوة، كشف {ph0}تدقيق أوسع{ph1} ثوابت خاطئة في دوال كانت قد اجتازت " +"التحقق. في إحدى العمليات الحسابية، أضافت الشيفرة المعاد بناؤها {ph2}، بينما " +"أضاف الأصل {ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}يمكن للنموذج الذي تمت مكافأته بفحص التحقق القديم أن يتعلم إنتاج إعادة " +"بناء غير صحيحة.{ph1} لم نقوم بتدريب نموذج RL باستخدام هذا المدقق. الفشل الذي" +" لاحظناه كان في الفحص نفسه. ولكن إذا استخدمناها كمكافأة، فإن نفس النقطة " +"العمياء ستصبح جزءًا من بيئة التعلم." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"يمكن أن يحتوي الاختبار الوظيفي على نقطة عمياء مماثلة. {ph0}وجدت الأبحاث التي" +" تستخدم التشويش التفاضلي في برامج فك التشفير LLM{ph1} وجود اختلافات سلوكية " +"خارج الاختبارات المتوفرة. لذلك نحن بحاجة إلى فهم اختبار النجاح من حيث " +"الحالات التي يمارسها. وبالمثل، يجب فهم مطابقة التعليمات من حيث البايتات " +"والبيانات التي تتحقق منها." + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"قبل استخدام أي نوع من اختبارات التحقق كمكافأة، سأختبره بنتائج غير صحيحة " +"عمدًا. أود أيضًا أن أبقي القبول النهائي منفصلاً عن كود المصدر الذي يمكن " +"لعامل الذكاء الاصطناعي تحريره. يجب أن يجتاز النموذج التحقق من خلال إعادة " +"بناء البرنامج بشكل صحيح. وبخلاف ذلك، فإن مجرد نسخ مصفوفة بايت من الهدف يمكن " +"أن يخدع نظام تسجيل ساذج دون استعادة كود المصدر المفيد." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}إن تحسين المتعلم وتحسين أداة التحقق من الصحة ينتميان إلى نفس مشكلة " +"البحث.{ph1} ومع تحسن المتعلم في تحسين التعليقات التي نقدمها له، يصبح فهم هذه" +" التعليقات أكثر أهمية أيضًا." + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "مقارنة تستحق التشغيل" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"مع إجراء عمليات التحقق هذه، سأبدأ المقارنة على مجموعة مهام VC7.1 ثابتة. " +"أولاً، أود أن أضع خطوط أساس لنموذج قوي للذكاء الاصطناعي للأغراض العامة " +"ونموذج أصغر. إن إضافة معرفة المستودع إلى النموذج الأصغر سيُظهر تغييرات " +"الاسترجاع. ومن هنا، فإن التخصص الخاضع للإشراف والنسخة المدربة RL ستسمح لنا " +"بفحص ما يضيفه التدريب." + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"ولجعل هذه المقارنات مفيدة، سيحصل كل نموذج على نفس الأدلة والأدوات المستهدفة." +" بالنسبة للمقارنات الخاصة بالتدريب، سأبقي النموذج الأساسي والمعرفة المتاحة " +"ثابتة أيضًا. من شأن التجارب المتكررة أن تساعد في التمييز بين استراتيجية " +"البحث الموثوقة والحل المرشح الذي ينجح بالصدفة." + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"سأفصل بعد ذلك شرطين للبدء: إعادة بناء دالة من تعليماتها، وإغلاق الاختلافات " +"القليلة الأخيرة في المصدر المحفوظ. قد يكون نموذج الذكاء الاصطناعي المتخصص " +"ممتازًا في المهمة الثانية، بينما يقوم نموذج الذكاء الاصطناعي للأغراض العامة " +"بالكثير من العمل اللازم للوصول إليه. وقياسها بشكل منفصل من شأنه أن يكشف عن " +"تقسيم العمل هذا." + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"تحتاج كلتا المهمتين إلى أمثلة اختبارية مستثناة من التدريب والاسترجاع. ألعاب " +"Touhou ذات الصلة تشترك في رمز المحرك. قد يؤدي استبعاد لعبة واحدة من التدريب " +"إلى ترك وظائف متطابقة تقريبًا في مجموعة التدريب. بدلاً من ذلك، سأستبعد " +"مجموعات كاملة من أكواد المصدر ذات الصلة وأحتفظ بإجاباتها التي تم التحقق منها" +" خارج كل من بيانات التدريب ومكتبة الاسترجاع." + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"بالنسبة لهذه الأمثلة، ستكون النتيجة تطابقات كاملة تم التحقق منها بشكل مستقل." +" من شأن الميزانيات المتساوية لمقترحات المرشحين أو محاولات التجميع أن تساعد " +"في مقارنة جودة البحث. إن قياس إجمالي الوقت والتكلفة، بما في ذلك التجميع، " +"سيُظهر ما إذا كانت هذه الجودة تنتج سير عمل فعالاً. وينبغي قياس تكاليف " +"التدريب بشكل منفصل." + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"كلا القياسين مهمان. يمكن أن يؤدي الاقتراح الرخيص إلى سير عمل مكلف إذا تسبب " +"في المزيد من التجميعات التي لا تقدم معلومات مفيدة. وعلى العكس من ذلك، قد " +"يكون النموذج الأبطأ أكثر اقتصادا إذا وجد تجربة مفيدة في وقت أقرب. وينبغي أن " +"تشمل المقارنة هذه التكاليف." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"كما أن حساب النتائج المقبولة يحافظ على التقدم المتوسط في المنظور الصحيح. بدا" +" أحد مرشحات TH09 {ph0}replay-save{ph1} واعدًا لأن الدالة المترجمة لها الحجم " +"المتوقع، ولكن المقارنة الكاملة لا تزال تجد اختلافات جوهرية. إن التغييرات في " +"كيفية قيام المترجم بترتيب التعليمات يمكن أن تجعل المرشح يبدو أقرب دون حل " +"المشكلة الأساسية. يمكن لمثل هذه التشخيصات أن توجه التجربة التالية، لكنني " +"سأؤكد صحة أي مكافأة مقابل \"الاقتراب\" مقابل فحص القبول الكامل." + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"سأكون مهتمًا بالنمط عبر هذه النتائج. إذا كان نموذج الذكاء الاصطناعي المتخصص " +"يعمل بشكل جيد ضمن عائلة المترجمات المألوفة، فهل ستصمد هذه الميزة بعد تغيير " +"سلسلة الأدوات؟ وبمجرد أن يتمتع كلا نموذجي الذكاء الاصطناعي بقدرة جيدة على " +"الاسترجاع، ما مقدار التدريب الذي سيضيفه؟ ستساعدنا الإجابات في تحديد المكان " +"الذي يكون فيه كل نوع من النماذج أكثر فائدة." + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "ما هو نوع وكيل الذكاء الاصطناعي الذي يشير إليه هذا؟" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"إذا تبين أن النماذج تساعد في مراحل مختلفة، فقد يكون الترتيب المفيد هو وكيل " +"واسع النطاق لديه إمكانية الوصول إلى نموذج متخصص للذكاء الاصطناعي. بمجرد أن " +"تصبح المهمة عبارة عن بحث مألوف للمترجم، يمكن أن تطلب نموذج الذكاء الاصطناعي " +"المتخصص للمرشحين. إذا لم تعد النتائج منطقية، فيمكن للعامل الأوسع العودة إلى " +"الدليل المستهدف أو التحقيق في القياس نفسه." + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"وفي هذا الترتيب، يصبح التخصص أداة يمكن للعميل اختيارها مع تطور التحقيق. " +"يحتفظ الوكيل بحرية تغيير الاتجاه الذي جعل الاختراقات السابقة ممكنة." + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"ويأتي هذا بشكل طبيعي من التحول الصناعي الذي وصفته في المقالة السابقة. إن " +"القدرة على إجراء المزيد من التجارب تغير الطريقة التي ننظم بها العمل. كما أنه" +" يمنحنا سببًا للتساؤل عن القرارات التي تحتاج إلى حكم واسع النطاق وتلك التي " +"تتحسن من خلال الخبرة المتكررة في حرفة معينة." + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"حتى الآن، استحق أقوى نموذج عام مكانه في سير عملي لأنه حل مشكلات كنت قد " +"تركتها دون حل. وهذه النجاحات تدفعني إلى دراسة العملية: كم منها اعتمد على " +"اتساع قدراته، وكم يمكن أن يتولاه نموذج مدرب على هذا النوع من إعادة البناء؟" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}توفر لنا المستودعات طريقة لاختبار السؤال.{ph1} أود استخدام هذا السجل " +"لإجراء المقارنة ومعرفة ما نتعلمه." + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "الأدلة وإسناد النموذج" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"تصف الأمثلة المقارنات التاريخية على مستوى الوظيفة المسجلة في الالتزامات " +"المرتبطة. تعتبر عمليات التحقق من وقت تشغيل اللعبة بأكملها بمثابة مراحل " +"منفصلة. تمت مراجعة سجلات المصدر والمطابقة في 13 أكتوبر 2026." + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"التسمية TH09 {ph0} تعني GPT-6-Astra. تغير النموذج المرتبط بـ {ph1} بمرور " +"الوقت، لذا لا يمكن لهذه التسمية وحدها تحديد النموذج المستخدم في عملية تشغيل " +"معينة. توفر أسماء النماذج الصريحة في الكوميتات المذكورة الأخرى الإسناد " +"المسجل. لا تتضمن هذه السجلات قياسات متسقة للتكلفة أو الجهد لكل عملية تشغيل." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 عمر الكائن المؤقت:{ph1} تم التحقق من {ph2}مراجعة السياق " +"السابقة{ph3} و{ph4}إعادة البناء الكاملة من خلال إعادة تجميع نظيفة من " +"المصدر{ph5}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}TH09 Front رد الاتصال:{ph1} {ph2}مناطق التشخيص التي لم تتطابق مع{ph3}، " +"والرسم البياني الجديد لتدفق التحكم في {ph4}Astra{ph5} و{ph6}فحص إعادة " +"الترجمة المقبول{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}معرفة المترجم:{ph1} {ph2}TH095 إعادة استخدام سلوك مجموعة المعرفات{ph3}،" +" {ph4}TH04 شيفرة اللاعب وترتيب الرابط{ph5} و{ph6}Luna إعادة البناء الكاملة " +"لوظيفة إنشاء العدو لـ TH09{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}عمليات الفحص والتشخيص:{ph1} {ph2}TH08 التدقيق الثابت للفاصلة " +"العائمة{ph3}، و{ph4}TH08 التحقيق الرقمي{ph5} و{ph6}TH09 إعادة البناء مع " +"توفير إعادة التشغيل{ph7}." + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"المقارنات المقترحة للنماذج المتخصصة والتدريب RL هي تجارب مستقبلية. تاريخ " +"المشروع يحفزهم. لم يتم تقييم أي نموذج متخصص مدرب لهذا المقال." + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "هل تحتاج الهندسة العكسية إلى نموذج أكبر؟ | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"ما تعلمته من إعادة بناء Touhou عن النماذج العامة والمتخصصة، وعن تعلم اقتراح " +"فرضيات أفضل من ملاحظات المترجم البرمجي." + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "في هذه الصفحة" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "النماذج والهندسة العكسية مقال" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "تجارب إعادة بناء الشيفرة المختارة" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "العودة للأعلى" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "بقلم N0zoM1z0 · 13 أكتوبر 2026 · مسودة للمراجعة" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/de/messages.po b/website/locales/de/messages.po index 9a0b5a51d..b5e7ce5b5 100644 --- a/website/locales/de/messages.po +++ b/website/locales/de/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Gedanken zum Reverse Engineering." #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Spielrekonstruktion{ph0}Essay" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "Die industrielle Revolution im Reverse Engineering" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -53,19 +53,20 @@ msgstr "" "gelehrt, über Agenten, Überprüfung und Wissen, dass Verbindungen über " "Projekte hinweg." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Lesen Sie den Artikel" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Von N0zoM1z0 · 10. Oktober 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -101,6 +102,8 @@ msgstr "" "REA und KI-Agenten." #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -137,6 +140,7 @@ msgstr "" "Ihrem Codierungsagenten." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -157,6 +161,7 @@ msgid "REA home" msgstr "REA-Startseite" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -177,6 +182,7 @@ msgid "Main navigation" msgstr "Hauptnavigation" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -197,6 +203,7 @@ msgid "REA on GitHub" msgstr "REA auf GitHub" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -217,6 +224,7 @@ msgid "Join REA on Discord" msgstr "REA auf Discord beitreten" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -236,7 +244,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "REA auf X folgen" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Nach oben" @@ -11845,6 +11853,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "Gleicher Abstand: 256 ÷ 16 = 16 Winkeleinheiten = 22,5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -11865,6 +11874,7 @@ msgid "Showcases" msgstr "Fallstudien" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -11885,6 +11895,7 @@ msgid "Blog" msgstr "Blog" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -11905,6 +11916,7 @@ msgid "Get started" msgstr "Erste Schritte" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -11924,7 +11936,8 @@ msgctxt "shared" msgid "Guides" msgstr "Anleitungen" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11944,7 +11957,8 @@ msgctxt "shared" msgid "Source" msgstr "Quellcode" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11964,7 +11978,8 @@ msgctxt "shared" msgid "MIT license" msgstr "MIT-Lizenz" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11984,7 +11999,8 @@ msgctxt "shared" msgid "Join the community" msgstr "Zur Community" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12980,8 +12996,1373 @@ msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "Von N0zoM1z0 · 13. Oktober 2026 · Entwurf zur Durchsicht" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Von N0zoM1z0 · 13. Oktober 2026" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "Zum Inhalt springen" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "{ph0}Home{ph1} / {ph2}Blog{ph3} / Modelle und Reverse Engineering" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "Aufsatz" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Braucht Reverse Engineering ein größeres Modell?" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Von N0zoM1z0 · 13. Oktober 2026" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "Das Wichtigste" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}Die Modellfähigkeit beeinflusst, welche Hypothesen ein KI-Agent " +"vorschlagen kann.{ph1} Spätere Sitzungen lösten Probleme, die in früheren " +"Arbeiten ungelöst geblieben waren." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}Ein bekannter Compiler bietet konkrete Muster für ein spezielles Modell" +" zum Lernen.{ph1} Einige genaue Maschinencode-Übereinstimmungen hängen von " +"bestimmten, wiederverwendbaren Quellcodemustern ab." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}Beziehen Sie das Repository-Wissen in einen fairen Vergleich ein.{ph1} " +"Ein kleineres Modell mit gutem Wissensabruf verfügt möglicherweise bereits " +"über genügend Informationen, um Fortschritte zu erzielen." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}Trainieren Sie das Modell, um das nächste nützliche Experiment " +"auszuwählen, und testen Sie seinen Prüfer.{ph1} Feedback ist nur insoweit " +"nützlich, als der Prüfer die richtigen Dinge überprüft." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}Bewerten Sie verifizierte Rekonstruktionsergebnisse neben Zeit und " +"Kosten.{ph1} Das ist es, was ich für den gesamten KI-Agent-Workflow messen " +"möchte." + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "In diesem Artikel" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}Was leistungsfähigere Modelle verändert haben{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}Warum spezialisieren?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}Wo sollte diese Erfahrung gespeichert werden?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}Lernen Sie das nächste nützliche Experiment{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}Reverse Engineering und verstärkendes Lernen{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}Wenn der Prüfer falsch ist{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}Ein Vergleich, der sich lohnt{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}Was für einen KI-Agenten lässt dies vermuten?{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "Was leistungsfähigere Modelle verändert haben" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"Seit August 2026 erforsche ich, wie viel Reverse Engineering ein Agent mit " +"Zugriff auf ein Programm und einer Möglichkeit zur Überprüfung seiner Arbeit" +" durchführen kann. Der größte Teil meiner Erfahrung stammt aus der " +"Rekonstruktion von Spielen, insbesondere Touhou. Da ich die Modelle geändert" +" habe, haben die Repositorys aufgezeichnet, was jede Sitzung " +"wiederherstellen konnte." + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"Ein Großteil dieser Arbeit zielt auf eine exakte Rekonstruktion ab: Wir " +"kompilieren die gepflegte Quelle unter einer historischen Toolchain mit " +"festen Versionen und prüfen, ob sie die Zielfunktion reproduziert. Wenn ein " +"späteres Modell erfolgreich ist, wo frühere Arbeiten ins Stocken geraten " +"sind, zeigen die Quelländerungen, was diesen Fortschritt ermöglicht hat." + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "Die temporären Variablen benötigten überlappende Lebensdauern" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"Ein Großteil der Logik einer TH09-Angriffsfunktion war bereits " +"rekonstruiert, aber der Compiler speicherte temporäre Ergebnisse immer noch " +"anders als das Original. Das verbleibende Problem bestand darin, wie lange " +"diese Ergebnisse am Leben bleiben mussten und wann ihr Speicher " +"wiederverwendet werden konnte." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"Ein {ph0}GPT-6-Sol-Sitzungsdatensatz{ph1} hat diese Nichtübereinstimmung " +"gemeldet, nachdem mehrere gewöhnliche Quellcode-Anordnungen getestet wurden." +" Als {ph2}GPT-6.1-Sol das Problem erneut untersuchte{ph3}, schlug es eine " +"nützliche Hypothese über die Lebensdauer dieser temporären Variablen vor." + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"Die Funktion erzeugte temporäre Ergebnisse und kopierte sie in " +"Animationspositionen. In C++ bleibt ein temporäres Ergebnis bestehen, bis " +"der Ausdruck, der es verwendet, beendet ist. Durch die Gruppierung der " +"realen Operationen in einem Ausdruck blieben diese Ergebnisse zusammen " +"erhalten, sodass der Compiler nicht mehr denselben Speicher für alle " +"wiederverwenden konnte." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"Ein Ergebnis wird weiterhin gemeinsam mit einem Wert gespeichert, der später" +" für Kollisionsprüfungen verwendet wird. Indem man diesem Wert seine eigene " +"Lebensdauer zuordnete, wurde die verbleibende Überlappung behoben. Nach " +"mehreren gezielten Experimenten bestätigte {ph0}eine saubere Neukompilierung" +" aus dem Quellcode, dass die vollständige Funktion übereinstimmte{ph1}." + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"Mich interessiert, wie sich Entdeckung und Tests gegenseitig unterstützten. " +"Die Hypothese über die temporäre Lebensdauer von Objekten gab dem KI-Agenten" +" eine neue Richtung. Das Feedback des Compilers half dabei, diese Hypothese " +"in eine passende Quellcode-Anordnung zu verfeinern." + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "Ein anderes Kontrollflussdiagramm" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"Eine zweite TH09-Funktion zeigt, warum eine neue Richtung wichtig sein kann." +" Hier ging es darum, wie mehrere Verzweigungen dieselben abschließenden " +"Operationen nutzen. {ph0}GPT-6.1-Sols frühere Arbeit{ph1} brachte Teile der " +"Funktion zur Übereinstimmung, doch die Verzweigungen liefen noch anders " +"zusammen als im Original." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra hat das umgebende Kontrollflussdiagramm{ph1} geändert. Die " +"Hauptwege führten nun zu einer gemeinsamen finalen Aktualisierung und " +"Rückkehr. In dieser Quellstruktur kombinierte der Compiler die Zweige auf " +"die gleiche Weise wie im Original. Die vollständige Funktion stimmte überein" +" und eine {ph2}separate Neuerstellungsprüfung bestätigte dies{ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}Durch die Änderung der Darstellung wurde ein produktiverer Suchraum " +"eröffnet.{ph1} Beide Fälle geben mir einen Grund, die allgemeine " +"Modellfähigkeit ernst zu nehmen: Ein Agent kann lange Zeit lokale " +"Variationen ausprobieren, bevor er eine bessere Möglichkeit findet, die " +"umgebende Arbeit auszudrücken." + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"Die späteren Sitzungen übernahmen allerdings auch früheren Quellcode und " +"aufgezeichnete Fehlversuche. Diese Vorarbeit war Teil ihres Vorteils. Die " +"Historie macht die Entdeckungen und ihre Prüfungen nachvollziehbar. Ein " +"kontrollierter Vergleich würde helfen, den Beitrag des Modells vom Beitrag " +"der übernommenen Erfahrung zu trennen." + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"Das macht die Frage für mich interessant. Einige Fortschritte wurden durch " +"die Suche nach einer neuen Vertretung erzielt. Ein Großteil der " +"verbleibenden Arbeit bestand darin, verschiedene Quellcode-Anordnungen mit " +"einem vertrauten Compiler zu testen. Wie viel davon erfordert ein breites " +"Denkvermögen? Wie viel könnte ein Modell lernen, wenn es wiederholt mit " +"diesem Compiler und Code arbeitet?" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "Warum könnte ein spezialisiertes KI-Modell gut funktionieren?" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou gibt uns einen konkreten Grund zu der Annahme, dass diese Erfahrung " +"hilfreich sein wird. Ein Großteil des Spielcodes stammt vom selben Autor, " +"ZUN, daher sind wiederkehrende Gewohnheiten eine vernünftige " +"Ausgangsannahme. Die Repositories untermauern dies mit nützlichen " +"Beispielen: Verwandte Schleifen und Hilfsanordnungen haben spielübergreifend" +" Hypothesen geliefert." + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"Diese Quellmuster werden spezifischer, wenn wir den Compiler einbeziehen. " +"Unter einer bestimmten Version und einem bestimmten Optionssatz können " +"kleine Änderungen in der Quelle zu wiederholbaren Unterschieden im " +"Maschinencode führen. Einige dieser Unterschiede sind überraschend unklar." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"In TH095 beispielsweise könnte {ph0}sogar der Name einer lokalen Variablen " +"Einfluss darauf haben, wo VC7.1 sie gespeichert hat{ph1}. Das Verschieben " +"der Deklaration hat das Speicherlayout nicht gelöst. Allerdings hatte eine " +"frühere Hintergrundfunktion ein Namensmuster offenbart, das die Wahl des " +"Compilers beeinflusste. Das legte eine vielversprechendere Hypothese für die" +" {ph2}Rekonstruktion von {ph3}{ph4} nahe." + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"Durch die Wiederverwendung dieses Musters für die Dateigrößenvariable wurde " +"das erforderliche Layout erstellt, ohne dass Speicherplatz hinzugefügt " +"werden musste. Die gesamte Funktion stimmte dann überein." + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"Dies ist ein sehr spezifisches Detail, das einem allgemeines KI-Modell " +"möglicherweise nicht bekannt ist, wenn es zum ersten Mal auf das Problem " +"stößt. Ein auf diesen Compiler spezialisiertes Modell könnte viele solcher " +"Fälle erkennen und herausfinden, welche Quellcodevarianten es wert sind, " +"zuerst getestet zu werden." + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"Die Chance wird besonders deutlich bei finalen Matching-Problemen, bei denen" +" wir das Verhalten bereits verstehen. Wir fragen uns dann, welche " +"gewöhnliche Quellcodeanordnung dieser Compiler in die beobachteten " +"Anweisungen umwandelt. Durch die Spezialisierung könnte das Modell diese " +"Formen effizienter vorschlagen, während das Ziel weiterhin bestimmt, welche " +"Hypothesen bestehen bleiben." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"Nützliche Muster müssen ihren Kontext behalten. TH08 verwendet eine frühere " +"Visual C++-Version als TH095 und TH09. TH04 verwendet einen anderen Compiler" +" aus der DOS-Ära. Für die {ph0}Rekonstruktion des Spielercodes{ph1} waren " +"unterschiedliche Hypothesen über Quellcodemuster und die Art und Weise " +"erforderlich, wie kompilierter Code mit einer ausführbaren Datei verknüpft " +"wurde. Wenn wir denselben Autor haben, haben wir zunächst eine Erwartung, " +"aber jedes Ziel sagt uns, ob es zutrifft. Ein spezielles Modell müsste diese" +" Kontexte unterscheiden." + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "Wo sollte diese Erfahrung gespeichert werden?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"Bevor ich mich entscheide, diese Muster durch Modelltraining zu lehren, " +"würde ich mir das Wissen ansehen, das wir bereits im Repository speichern. " +"Im {ph0}vorherigen Aufsatz{ph1} habe ich darüber geschrieben, dass es zum " +"Arbeitsgedächtnis des Projekts wird. Dieser Speicher umfasst Compiler-" +"Lektionen sowie die Quelle und fehlgeschlagene Experimente, die sie nützlich" +" gemacht haben." + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"Durch das Abrufen dieses Datensatzes kann ein Agent mit einem bekannten " +"Ergebnis beginnen und prüfen, wie wir zu diesem Ergebnis gelangt sind. Es " +"können auch Alternativen vermieden werden, die im gleichen Kontext bereits " +"gescheitert sind. Der Ausgangspunkt verbessert sich, ohne die trainierten " +"Parameter des Modells zu ändern." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"Bei der Rekonstruktion der Funktion von TH09 zum Erstellen von Feinden hat " +"eine {ph0}GPT-6-Luna-max-Sitzung{ph1} eine Hypothese aus dem zugehörigen " +"TH08-Quellcode übernommen. Dieser frühere Code schlug vor, wie die Schleife " +"organisiert und Fehler darin behandelt werden sollten. Die eigenen Beweise " +"von TH09 lieferten die spielspezifischen Details. Neuaufbauten aus der " +"Quelle bestätigten, dass die vollständige Funktion übereinstimmte." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}„Das Modell kannte diese Compiler-Eigenart nicht“ und „Das Modell " +"konnte sie nicht verwenden“ sind unterschiedliche Probleme.{ph1} Das erste " +"Problem kann durch das Abrufen gelöst werden, aber die Umwandlung dieser " +"Tatsache in einen nützlichen Kandidaten erfordert noch Arbeit. Und wenn der " +"Quellcode des Kandidaten fehlschlägt, muss das Modell entscheiden, was das " +"Ergebnis für seinen nächsten Versuch bedeutet. Die Kenntnis des relevanten " +"Musters ist nur der Anfang dieses Prozesses." + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"Aus diesem Grund würde ich in jeden Vergleich ein kleineres allgemeines KI-" +"Modell mit gutem Abruf einbeziehen. Wenn wir ihm die gleiche " +"Erfahrungsbibliothek zur Verfügung stellen, können wir herausfinden, was die" +" Schulung bringt: bessere Kandidatengenerierung, bessere Entscheidungen nach" +" Feedback oder beides." + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "Lernen Sie das nächste nützliche Experiment" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"Diese Entscheidungen nach Feedback sind das, was ich am liebsten studieren " +"würde. Bei der Agentenrekonstruktion schlägt das Modell eine Erklärung vor " +"und testet sie anhand des Programms. Selbst wenn der Quellcode des " +"Kandidaten fehlschlägt, kann das Ergebnis die verbleibenden Möglichkeiten " +"einschränken und ein nützlicheres nächstes Experiment vorschlagen." + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"Das Beispiel der temporären Ergebnisse zeigt auch das. Der endgültige " +"Quellcode enthält die Anordnung, die funktioniert hat. Der Weg dorthin " +"enthält jedoch zusätzliche Informationen: Jedes der folgenden Experimente " +"zeigte, was noch geändert werden musste." + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "Experimentieren" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "Was uns das Ergebnis lehrte" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "" +"Versuchen Sie es mit einer anderen Möglichkeit, die Ergebnisse zu kopieren" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "" +"Der Kopiercode ändert sich, aber die Funktion unterscheidet sich immer noch." + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "Verwenden Sie die zurückgegebenen Werte direkt" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "" +"Das Kopieren erfolgt nach dem ursprünglichen Muster, die temporäre " +"Speicherung unterscheidet sich jedoch noch." + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "Halten Sie alle temporären Variablen gleichzeitig gültig" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "" +"Sie erhalten einen separaten Speicher, aber einer überlappt immer noch einen" +" Wert, der später für Kollisionsprüfungen verwendet wird." + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "Halten Sie den Wert für die Kollisionsprüfung getrennt" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "" +"Eine saubere Neukompilierung aus dem Quellcode bestätigt die vollständige " +"Übereinstimmung." + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "" +"Ausgewählte Experimente aus der Angriffsfunktionsrekonstruktion oben, " +"gruppiert nach dem, was sie uns gelehrt haben." + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"Eine Trainingsaufzeichnung, die diese Reihenfolge beibehält, kann mehr " +"lehren als die endgültige Quelle allein. Es verbindet jede vorgeschlagene " +"Änderung mit einem beobachteten Ergebnis und der nächsten Entscheidung. Es " +"ist auch wichtig, ineffektive Variationen beizubehalten: Sie teilen einem " +"späteren Agenten mit, welche Hypothesen in diesem Zusammenhang bereits " +"getestet wurden." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}Ich möchte, dass das Modell lernt, Feedback zu nutzen.{ph1} Ein " +"neutrales Ergebnis ist ein Grund, die Variation noch einmal zu überdenken. " +"Eine veränderte Nichtübereinstimmung kann auf eine andere Ursache hinweisen." +" Wenn lokale Änderungen nicht mehr informativ sind, sollte der Agent in der " +"Lage sein, die Quellendarstellung noch einmal aufzurufen oder die Beweise " +"erneut zu prüfen." + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"Zu lernen, diese Entscheidungen zu treffen, erfordert Autonomie. Manchmal " +"besteht der nützliche nächste Schritt darin, einen anderen Aufrufer zu " +"überprüfen, anstatt eine weitere Bearbeitung zu kompilieren. Eine starre " +"Abfolge erforderlicher Umschreibungen würde diese Wahlmöglichkeit zusammen " +"mit einem Teil des Verhaltens, das das Modell lernen soll, aufheben." + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "Warum Reverse Engineering für Reinforcement Learning interessant ist" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"Rekonstruktion mit klar abgegrenztem Umfang bietet eine praktische Umgebung," +" um diese Entscheidungen zu untersuchen. Bei festgelegtem Ziel und Compiler " +"liefern viele Quellcodeexperimente schnelles, wiederholbares Feedback. Der " +"Agent kann einen Kandidaten kompilieren, die Abweichungen prüfen und daraus " +"den nächsten Schritt wählen." + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"Reinforcement Learning könnte diesen Entscheidungsprozess durch das " +"erhaltene Feedback trainieren. Ein durch eine definierte Prüfung " +"angenommener Kandidat gibt uns ein konkretes Ergebnis, während wir anhand " +"der Abfolge der Versuche sehen können, wie viel Arbeit es gekostet hat, " +"dieses zu erreichen." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"Es gibt bereits Forschungsergebnisse, die diese Richtung unterstützen. " +"{ph0}LLM4Decompile{ph1} untersucht Modelle, die auf die Dekompilierung " +"spezialisiert sind. {ph2}AutoDecompiler{ph3} trainiert die Multi-Turn-" +"Verfeinerung mithilfe von Compiler- und Ausführungsfeedback. Der Schwerpunkt" +" ihrer Studien liegt auf der Wiederherstellung des Programmverhaltens. Für " +"diese Arbeit würde ich sie als Ausgangspunkt für die damit verbundene " +"Aufgabe verwenden, eine Quelle zu finden, die mit der Ausgabe eines " +"historischen Compilers übereinstimmt." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"Um eine so klar definierte Aufgabe zu erfüllen, müssen wir jedoch " +"möglicherweise zunächst feststellen, welche Ressourcen eine Funktion " +"besitzt, oder eine neue Laufzeitmessung entwickeln. In TH08 beispielsweise " +"führte ein {ph0}Kollisionsunterschied dazu, wie das ursprüngliche Programm " +"Gleitkommaberechnungen verarbeitete{ph1}. Bevor der Agent eine nützliche " +"Korrektur testen konnte, musste er herausfinden, was gemessen werden sollte." + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"Dies deutet auf eine weitere Rolle eines allgemein leistungsfähigen Modells " +"hin: Es hilft dabei, das Problem überhaupt überprüfbar zu machen. Sobald " +"diese Prüfung vorliegt, können wir untersuchen, wie weit eine erlernte " +"Suchrichtlinie damit kommen kann." + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "Wenn der Prüfer falsch ist" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"Wenn wir eine Prüfung für das Training verwenden, wird auch ihr " +"Prüfungsumfang Teil der Lernumgebung. Ich nenne eine solche Referenz oder " +"einen solchen Prüfer ein Oracle. Selbst eine wiederholbare Antwort müssen " +"wir danach beurteilen, was sie tatsächlich überprüft." + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 hat gezeigt, wie viel sich außerhalb dieser Abdeckung verbergen kann. " +"Einige rekonstruierte Funktionen haben ihre exakten Vergleiche mit falschen " +"Gleitkommakonstanten bestanden. Der Komparator passte die verschobenen " +"Adressen an, bevor er die Anweisungen überprüfte, überprüfte die Werte " +"hinter diesen Adressen jedoch nur, wenn sie explizit zur Überprüfung " +"aufgezeichnet wurden. Andere wörtliche Verweise könnten ohne Prüfung ihrer " +"Werte bestehen bleiben." + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"Um diese Lücke zu schließen, wurden bei einer {ph0}umfassenderen " +"Prüfung{ph1} falsche Konstanten in Funktionen gefunden, die bereits die " +"Überprüfung bestanden hatten. In einer Berechnung fügte der rekonstruierte " +"Quellcode {ph2} hinzu, während das Original {ph3} hinzufügte." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}Ein Modell, das durch die alte Verifizierungsprüfung belohnt wurde, " +"könnte lernen, eine falsche Rekonstruktion zu erzeugen.{ph1} Wir haben mit " +"diesem Prüfer kein RL-Modell trainiert. Der von uns beobachtete Fehler lag " +"in der Prüfung selbst. Aber wenn wir es als Belohnung nutzen würden, würde " +"derselbe blinde Fleck Teil der Lernumgebung werden." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"Ein Funktionstest kann einen ähnlichen blinden Fleck haben. " +"{ph0}Untersuchungen mit differenziellem Fuzzing auf LLM-Decompilern{ph1} " +"ergaben Verhaltensunterschiede außerhalb der bereitgestellten Tests. Wir " +"müssen daher einen bestandenen Test im Hinblick auf die Fälle verstehen, in " +"denen er durchgeführt wird. Ebenso muss eine Befehlsübereinstimmung im " +"Hinblick auf die Bytes und Daten verstanden werden, die sie überprüft." + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"Bevor ich eine Art Verifizierungsprüfung als Belohnung verwende, würde ich " +"sie mit absichtlich falschen Ergebnissen testen. Ich würde die Endabnahme " +"auch getrennt vom Quellcode durchführen, den der KI-Agent bearbeiten kann. " +"Das Modell muss die Verifizierung bestehen, indem es das Programm korrekt " +"rekonstruiert. Andernfalls könnte das einfache Kopieren eines Byte-Arrays " +"vom Ziel ein naives Bewertungssystem täuschen, ohne dass nützlicher " +"Quellcode wiederhergestellt wird." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}Die Verbesserung des Lernenden und die Verbesserung seines Prüfers " +"gehören zum selben Forschungsproblem.{ph1} Je besser der Lernende darin " +"wird, das Feedback, das wir ihm geben, zu optimieren, desto wichtiger wird " +"auch das Verständnis dieses Feedbacks." + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "Ein Vergleich, der sich lohnt" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"Wenn diese Prüfungen vorhanden sind, würde ich den Vergleich für eine feste " +"VC7.1-Aufgabenfamilie starten. Zunächst würde ich Grundlinien für ein " +"starkes allgemeines KI-Modell und ein kleineres Modell festlegen. Das " +"Hinzufügen von Repository-Wissen zum kleineren Modell würde zeigen, welche " +"Änderungen sich beim Abruf ergeben. Von dort aus würden wir durch eine " +"überwachte Spezialisierung und eine RL-trainierte Version untersuchen " +"können, was Training hinzufügt." + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"Um diese Vergleiche aussagekräftig zu machen, würde jedes Modell die " +"gleichen Zieldaten und Tools erhalten. Für die ausbildungsspezifischen " +"Vergleiche würde ich das Basismodell und das verfügbare Wissen ebenfalls " +"unverändert lassen. Wiederholte Versuche würden dabei helfen, eine " +"zuverlässige Suchstrategie von einer Kandidatenlösung zu unterscheiden, die " +"zufällig erfolgreich ist." + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"Ich würde dann zwei Startbedingungen trennen: die Rekonstruktion einer " +"Funktion aus ihren Anweisungen und die Beseitigung der letzten paar " +"Unterschiede in der gepflegten Quelle. Bei der zweiten Aufgabe könnte ein " +"spezialisiertes KI-Modell hervorragend geeignet sein, während ein " +"allgemeines KI-Modell einen Großteil der dafür erforderlichen Arbeit " +"übernimmt. Eine getrennte Messung würde diese Arbeitsteilung offenbaren." + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"Für beide Aufgaben sind Testbeispiele erforderlich, die vom Training und " +"Abruf ausgeschlossen sind. Verwandte Touhou-Spiele teilen den Engine-Code. " +"Der Ausschluss eines Spiels vom Training könnte dennoch dazu führen, dass " +"nahezu identische Funktionen im Trainingsset verbleiben. Stattdessen würde " +"ich ganze Gruppen verwandten Quellcodes ausschließen und ihre verifizierten " +"Antworten sowohl aus den Trainingsdaten als auch aus der Abrufbibliothek " +"heraushalten." + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"Das Ergebnis wären vollständige, unabhängig verifizierte Übereinstimmungen. " +"Gleiche Budgets für Kandidaten oder Kompilierungsversuche helfen, die " +"Suchqualität zu vergleichen. Gesamtzeit und Gesamtkosten einschließlich " +"Kompilierung zeigen, ob diese Qualität einen effizienten Arbeitsablauf " +"ermöglicht. Trainingskosten sollten separat erfasst werden." + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"Beide Messungen sind wichtig. Ein billiger Vorschlag kann zu einem teuren " +"Arbeitsablauf führen, wenn er mehr Kompilierungen verursacht, die keine " +"nützlichen Informationen liefern. Umgekehrt kann ein langsameres Modell " +"wirtschaftlicher sein, wenn es früher ein nützliches Experiment findet. Der " +"Vergleich sollte diese Kosten einbeziehen." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"Akzeptierte Ergebnisse zu zählen hilft auch, Zwischenfortschritte " +"einzuordnen. Ein {ph0}Replay-Save-Kandidat{ph1} aus TH09 wirkte " +"vielversprechend, weil die kompilierte Funktion die erwartete Größe hatte. " +"Der vollständige Vergleich fand aber noch erhebliche Unterschiede. Eine " +"andere Anordnung der Befehle kann einen Kandidaten näher am Ziel erscheinen " +"lassen, ohne das eigentliche Problem zu lösen. Solche Diagnosen können das " +"nächste Experiment leiten; eine Belohnung für mehr Nähe würde ich jedoch " +"anhand der vollständigen Abnahmeprüfung validieren." + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"Mich würde das Muster dieser Ergebnisse interessieren. Wenn ein " +"spezialisiertes KI-Modell in einer bekannten Compilerfamilie gut " +"funktioniert, überlebt dieser Vorteil dann auch eine Änderung der Toolchain?" +" Und wenn beide KI-Modelle eine gute Retrievalfähigkeit haben, wie viel " +"bringt das Training dann noch? Die Antworten würden uns helfen zu " +"entscheiden, wo welche Art von Modell am nützlichsten ist." + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "Was für einen KI-Agenten lässt dies vermuten?" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"Wenn sich herausstellt, dass die Modelle in verschiedenen Phasen hilfreich " +"sind, könnte ein umfassender Agent mit Zugriff auf ein spezielles KI-Modell " +"eine nützliche Lösung sein. Sobald eine Aufgabe zu einer vertrauten " +"Compiler-Suche wird, könnte sie das spezialisierte KI-Modell nach Kandidaten" +" fragen. Wenn die Ergebnisse keinen Sinn mehr ergeben, könnte der breitere " +"Agent zum Zielbeweis zurückkehren oder die Messung selbst untersuchen." + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"In dieser Vereinbarung wird die Spezialisierung zu einem Instrument, das der" +" Agent im Verlauf der Untersuchung wählen kann. Der Agent behält die " +"Freiheit, die Richtung zu ändern, die die früheren Durchbrüche ermöglicht " +"hat." + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"Dies ergibt sich natürlich aus dem industriellen Wandel, den ich im " +"vorherigen Aufsatz beschrieben habe. Die Möglichkeit, viel mehr Experimente " +"durchführen zu können, verändert die Art und Weise, wie wir die Arbeit " +"organisieren. Es gibt uns auch einen Grund zu fragen, welche Entscheidungen " +"ein breites Urteilsvermögen erfordern und welche sich durch wiederholte " +"Erfahrung mit einem bestimmten Handwerk verbessern." + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"Vorerst hat sich das stärkste allgemeine KI-Modell seinen Platz in meinem " +"Arbeitsablauf verdient, indem es Probleme löste, die ich ungelöst gelassen " +"hatte. Gerade diese Erfolge machen mich neugierig: Wie viel hing von seiner " +"breiten Fähigkeit ab? Wie viel könnte ein Modell übernehmen, das auf diese " +"Art der Rekonstruktion spezialisiert ist?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}Die Repositorys bieten uns eine Möglichkeit, die Frage zu testen.{ph1} " +"Ich möchte diesen Datensatz verwenden, um den Vergleich zu erstellen und zu " +"sehen, was wir lernen." + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "Beweise und Modellzuschreibung" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"Die Beispiele beschreiben historische Vergleiche auf Funktionsebene, die in " +"den verknüpften Commits aufgezeichnet wurden. Laufzeitprüfungen für das " +"gesamte Spiel sind separate Meilensteine. Quelle und übereinstimmende " +"Aufzeichnungen wurden am 13. Oktober 2026 überprüft." + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"Die Bezeichnung TH09 {ph0} bedeutet GPT-6-Astra. Das mit {ph1} verknüpfte " +"Modell hat sich im Laufe der Zeit geändert, sodass die Bezeichnung allein " +"das in einem bestimmten Lauf verwendete Modell nicht identifizieren kann. " +"Explizite Modellnamen in den anderen zitierten Commits liefern die " +"aufgezeichnete Zuschreibung. Diese Historien enthalten keine konsistenten " +"Messungen der Kosten oder des Aufwands für jeden Lauf." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 temporäre Objektlebensdauern:{ph1} die {ph2}frühere " +"Kontextüberprüfung{ph3} und {ph4}vollständige Rekonstruktion, verifiziert " +"durch eine saubere Neukompilierung aus der Quelle{ph5}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}TH09 Front-Callback:{ph1} {ph2}Diagnosebereiche, die nicht " +"übereinstimmten{ph3}, {ph4}Astras neues Kontrollflussdiagramm{ph5} und seine" +" {ph6}akzeptierte Neukompilierungsprüfung{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}Compiler-Kenntnisse:{ph1} {ph2}TH095 Wiederverwendung des Identifier-" +"Bucket-Verhaltens{ph3}, {ph4}TH04 Player-Quellcode und Linker-" +"Reihenfolge{ph5} und {ph6}Lunas vollständige Rekonstruktion der " +"Feinderschaffungsfunktion von TH09{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}Prüfungen und Diagnosen:{ph1} {ph2}TH08-Prüfung der " +"Gleitkommakonstanten{ph3}, {ph4}TH08-Untersuchung numerischer " +"Unterschiede{ph5} und {ph6}TH09-Rekonstruktion der Replay-Speicherung{ph7}." + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"Die vorgeschlagenen Vergleiche spezialisierter Modelle und RL-Training sind " +"zukünftige Experimente. Die Projektgeschichten motivieren sie. Für diesen " +"Aufsatz wurde kein trainiertes Spezialmodell evaluiert." + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "Braucht Reverse Engineering ein größeres Modell? | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"Was ich durch die Rekonstruktion von Touhou über allgemeine Modelle, " +"Spezialmodelle und das Lernen, bessere Hypothesen anhand des Compiler-" +"Feedbacks aufzustellen, gelernt habe." + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "Auf dieser Seite" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "Essay über Modelle und Reverse Engineering" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "Ausgewählte Rekonstruktionsexperimente" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "Nach oben" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "Von N0zoM1z0 · 13. Oktober 2026 · Entwurf zur Durchsicht" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/es/messages.po b/website/locales/es/messages.po index 7af23dad4..8f72df890 100644 --- a/website/locales/es/messages.po +++ b/website/locales/es/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Reflexiones sobre ingeniería inversa." #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Reconstrucción del juego{ph0}Ensayo" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "La revolución industrial en ingeniería inversa" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -53,19 +53,20 @@ msgstr "" "enseñó sobre los agentes, la verificación y el conocimiento que componen los" " proyectos." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Leer el artículo" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Por N0zoM1z0 * 10 octubre 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -101,6 +102,8 @@ msgstr "" "con REA y agentes de IA." #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -137,6 +140,7 @@ msgstr "" "de codificación." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -157,6 +161,7 @@ msgid "REA home" msgstr "Inicio de REA" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -177,6 +182,7 @@ msgid "Main navigation" msgstr "Navegación principal" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -197,6 +203,7 @@ msgid "REA on GitHub" msgstr "REA en GitHub" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -217,6 +224,7 @@ msgid "Join REA on Discord" msgstr "Únete a REA en Discord" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -236,7 +244,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Sigue a REA en X" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Volver arriba" @@ -11764,6 +11772,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "Mismo espaciado: 256 ÷ 16 = 16 unidades de ángulo = 22,5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -11784,6 +11793,7 @@ msgid "Showcases" msgstr "Casos prácticos" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -11804,6 +11814,7 @@ msgid "Blog" msgstr "Blog" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -11824,6 +11835,7 @@ msgid "Get started" msgstr "Primeros pasos" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -11843,7 +11855,8 @@ msgctxt "shared" msgid "Guides" msgstr "Guías" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11863,7 +11876,8 @@ msgctxt "shared" msgid "Source" msgstr "Código fuente" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11883,7 +11897,8 @@ msgctxt "shared" msgid "MIT license" msgstr "Licencia MIT" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11903,7 +11918,8 @@ msgctxt "shared" msgid "Join the community" msgstr "Únete a la comunidad" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12894,8 +12910,1362 @@ msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "Por N0zoM1z0 · 13 de octubre de 2026 · Borrador para revisión" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Por N0zoM1z0 · 13 de octubre de 2026" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "Saltar al contenido" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "{ph0}Inicio{ph1} / {ph2}Blog{ph3} / Modelos e ingeniería inversa" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "ensayo" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "¿La ingeniería inversa necesita un modelo más grande?" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Por N0zoM1z0 · 13 de octubre de 2026" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "Ideas clave" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}La capacidad del modelo afecta qué hipótesis puede proponer un agente " +"de IA.{ph1} Las sesiones posteriores resolvieron problemas que el trabajo " +"anterior había dejado sin resolver." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}Un compilador familiar ofrece patrones concretos para que los aprenda " +"un modelo especializado.{ph1} Algunas coincidencias exactas de código de " +"máquina dependen de patrones de código fuente específicos y reutilizables." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}Incluya el conocimiento del repositorio en una comparación justa.{ph1} " +"Es posible que un modelo más pequeño con una buena recuperación de " +"conocimiento ya tenga suficiente información para avanzar." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}Entrene el modelo para elegir el siguiente experimento útil y pruebe su" +" verificador.{ph1} La retroalimentación es útil solo en la medida en que el " +"verificador verifica lo correcto." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}Evaluar los resultados de reconstrucción verificados junto con el " +"tiempo y el costo.{ph1} Eso es lo que quiero medir para todo el flujo de " +"trabajo del agente de IA." + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "En este artículo" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}Qué cambiaron los modelos más capaces{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}¿Por qué especializarse?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}¿Dónde debería almacenarse esta experiencia?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}Aprendiendo el próximo experimento útil{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}Ingeniería inversa y aprendizaje por refuerzo{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}Cuando el verificador está equivocado{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}Una comparación que vale la pena realizar{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}¿Qué tipo de agente de IA sugiere esto?{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "Qué cambiaron los modelos más capaces" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"Desde agosto de 2026, he estado explorando cuánta ingeniería inversa puede " +"realizar un agente con acceso a un programa y una forma de comprobar su " +"funcionamiento. La mayor parte de mi experiencia proviene de la " +"reconstrucción de juegos, especialmente Touhou. A medida que he ido " +"cambiando de modelo, los repositorios han ido manteniendo un registro de lo " +"que cada sesión logró recuperar." + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"Gran parte de ese trabajo apunta a una reconstrucción exacta: compilamos la " +"fuente mantenida bajo una cadena de herramientas histórica con versiones " +"fijas y comprobamos si reproduce la función de destino. Cuando un modelo " +"posterior tiene éxito donde el trabajo anterior se estancó, los cambios en " +"las fuentes muestran qué hizo posible ese progreso." + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "Los valores temporales debían seguir siendo válidos al mismo tiempo" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"Gran parte de la lógica de la función de ataque TH09 ya había sido " +"reconstruida, pero el compilador aún almacenaba resultados temporales de " +"manera diferente al original. El problema restante era cuánto tiempo " +"necesitaban esos resultados para permanecer vivos y cuándo se podía " +"reutilizar su almacenamiento." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"Un registro de sesión {ph0}GPT-6-Sol{ph1} informó esta discrepancia después " +"de probar varias disposiciones de código fuente ordinarias. Cuando " +"{ph2}GPT-6.1-Sol examinó el problema nuevamente{ph3}, propuso una hipótesis " +"útil sobre la vida útil de estas variables temporales." + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"La función produjo resultados temporales y los copió en posiciones de " +"animación. En C++, un resultado temporal permanece vivo hasta que finaliza " +"la expresión que lo utiliza. Agrupar las operaciones reales en una expresión" +" mantuvo esos resultados vivos juntos, por lo que el compilador ya no pudo " +"reutilizar el mismo almacenamiento para todos ellos." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"Un resultado aún compartía almacenamiento con un valor utilizado más " +"adelante para comprobaciones de colisiones. Darle a ese valor su propia vida" +" útil resolvió la superposición restante. Después de varios experimentos " +"específicos, {ph0}una recompilación limpia del código fuente confirmó que la" +" función completa coincidía{ph1}." + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"Me interesa saber cómo el descubrimiento y las pruebas se apoyaron " +"mutuamente. La hipótesis sobre la vida temporal de los objetos dio al agente" +" de IA una nueva dirección. Los comentarios del compilador le ayudaron a " +"refinar esa hipótesis hasta convertirla en una disposición de código fuente " +"que coincidiera." + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "Un gráfico de flujo de control diferente" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"Una segunda función TH09 muestra por qué esa nueva dirección puede ser " +"importante. Aquí el obstáculo fue cómo varias ramas compartieron su trabajo " +"final. El trabajo anterior de {ph0}GPT-6.1-Sol{ph1} consiguió que partes de " +"la función coincidieran, pero las ramas aún se unían de manera diferente al " +"original." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra cambió el gráfico de flujo de control circundante{ph1}. Los" +" caminos principales conducían ahora a una actualización final compartida y " +"al regreso. En esa estructura fuente, el compilador combinó las ramas de la " +"misma manera que el original. La función completa coincidió y una " +"{ph2}verificación de reconstrucción separada lo confirmó{ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}Cambiar la representación abrió un espacio de búsqueda más " +"productivo.{ph1} Ambos casos me dan una razón para tomar en serio la " +"capacidad del modelo general: un agente puede seguir probando variaciones " +"locales durante mucho tiempo antes de encontrar una mejor manera de expresar" +" el trabajo circundante." + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"Las sesiones posteriores también heredaron el código anterior y los intentos" +" fallidos documentados. Esa experiencia acumulada formaba parte de su " +"ventaja. El historial permite examinar los hallazgos y sus comprobaciones; " +"una comparación controlada ayudaría a separar el efecto del modelo del de la" +" experiencia que recibió." + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"Esto es lo que hace que la pregunta me resulte interesante. Algunos avances " +"se produjeron al encontrar una nueva representación. Gran parte del trabajo " +"restante implicó probar diferentes disposiciones de código fuente con un " +"compilador familiar. ¿Cuánto de esto requiere una amplia capacidad de " +"razonamiento? ¿Cuánto podría aprender un modelo trabajando repetidamente con" +" ese compilador y código?" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "¿Por qué podría funcionar bien un modelo de IA especializado?" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou nos da una razón concreta para esperar que esa experiencia ayude. " +"Gran parte del código del juego proviene del mismo autor, ZUN, por lo que " +"los hábitos recurrentes son una suposición inicial razonable. Los " +"repositorios respaldan esto con ejemplos útiles: bucles relacionados y " +"arreglos de ayuda han proporcionado hipótesis en todos los juegos." + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"Esos patrones fuente se vuelven más específicos cuando incluimos el " +"compilador. Bajo una versión particular y un conjunto de opciones, pequeños " +"cambios en la fuente pueden producir diferencias repetibles en el código de " +"máquina. Algunas de esas diferencias son sorprendentemente oscuras." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"En TH095, por ejemplo, {ph0}incluso el nombre de una variable local podría " +"afectar dónde la almacenó VC7.1{ph1}. Mover su declaración no resolvió el " +"diseño del almacenamiento. Sin embargo, una función en segundo plano " +"anterior había revelado un patrón de nombres que afectaba la elección del " +"compilador. Eso sugirió una hipótesis más prometedora para {ph2}reconstruir " +"{ph3}{ph4}." + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"La reutilización de ese patrón para la variable de tamaño de archivo produjo" +" el diseño requerido sin agregar almacenamiento. Entonces coincidió la " +"función completa." + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"Este es un detalle muy específico que un modelo de IA de propósito general " +"puede no conocer cuando encuentra el problema por primera vez. Un modelo " +"especializado para este compilador podría ver muchos de estos casos y " +"aprender qué variaciones del código fuente vale la pena probar primero." + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"La oportunidad es especialmente clara en los problemas finales de " +"emparejamiento en los que ya entendemos el comportamiento. Entonces nos " +"preguntamos qué disposición ordinaria del código fuente convierte este " +"compilador en las instrucciones observadas. La especialización podría ayudar" +" al modelo a proponer esas formas de manera más eficiente, mientras que el " +"objetivo aún determina qué hipótesis sobreviven." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"Los patrones útiles deben conservar su contexto. TH08 utiliza una versión " +"Visual C++ anterior a TH095 y TH09. TH04 utiliza un compilador diferente de " +"la era DOS. Su {ph0}reconstrucción del código del jugador{ph1} necesitaba " +"diferentes hipótesis sobre los patrones del código fuente y cómo el código " +"compilado se vinculaba a un ejecutable. Tener el mismo autor nos da una " +"expectativa inicial, pero cada objetivo nos dice si aplica. Un modelo " +"especializado necesitaría distinguir estos contextos." + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "¿Dónde debería almacenarse esta experiencia?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"Antes de decidir enseñar esos patrones mediante el entrenamiento de modelos," +" miraría el conocimiento que ya guardamos en el repositorio. En el " +"{ph0}ensayo anterior{ph1}, escribí sobre cómo se convertiría en la memoria " +"de trabajo del proyecto. Esa memoria incluye lecciones del compilador, junto" +" con la fuente y los experimentos fallidos que los hicieron útiles." + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"Al recuperar ese registro, un agente puede comenzar con un resultado " +"conocido e inspeccionar cómo llegamos a él. También puede evitar " +"alternativas que ya han fracasado en el mismo contexto. El punto de partida " +"mejora sin cambiar los parámetros entrenados del modelo." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"Mientras reconstruía la función de TH09 para crear enemigos, una sesión " +"{ph0}GPT-6-Luna-max{ph1} tomó una hipótesis del código fuente relacionado de" +" TH08. Ese código anterior sugirió cómo organizar el bucle y manejar las " +"fallas dentro de él. La propia evidencia de TH09 proporcionó los detalles " +"específicos del juego. Las reconstrucciones de la fuente confirmaron que la " +"función completa coincidía." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}\"El modelo no sabía esa peculiaridad del compilador\" y \"el modelo no" +" podía usarlo\" son problemas diferentes.{ph1} La recuperación puede " +"resolver el primero, pero convertir ese hecho en un candidato útil aún " +"requiere trabajo. Y si el código fuente candidato falla, el modelo debe " +"decidir qué significa el resultado para su próximo intento. Conocer el " +"patrón relevante es sólo el comienzo de ese proceso." + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"Por esa razón, incluiría en cualquier comparación un modelo de IA de " +"propósito general más pequeño con buena recuperación. Darle la misma " +"biblioteca de experiencia nos ayudaría a identificar qué aporta la " +"capacitación: mejor generación de candidatos, mejores decisiones después de " +"la retroalimentación, o ambas." + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "Aprendiendo el próximo experimento útil" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"Esas decisiones después del feedback son las que más me gustaría estudiar. " +"En la reconstrucción agente, el modelo propone una explicación y la compara " +"con el programa. Incluso cuando el código fuente candidato falla, el " +"resultado puede reducir las posibilidades restantes y sugerir un próximo " +"experimento más útil." + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"Considere nuevamente el ejemplo de los resultados temporales. Su fuente " +"final muestra el arreglo que funcionó, pero la ruta a esa fuente contiene " +"más información. Cada uno de los siguientes experimentos ayudó a identificar" +" lo que aún era necesario cambiar." + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "experimento" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "Lo que nos enseñó su resultado" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "Pruebe una forma diferente de copiar los resultados." + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "El código de copia cambia, pero la función aún difiere." + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "Utilice los valores devueltos directamente" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "" +"La copia sigue el patrón original, pero el almacenamiento temporal aún " +"difiere." + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "Mantener todas las variables temporales válidas al mismo tiempo." + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "" +"Se almacenan por separado, pero uno aún se superpone a un valor utilizado " +"más adelante para las comprobaciones de colisiones." + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "Mantenga el valor de verificación de colisiones separado" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "" +"Una recompilación limpia del código fuente confirma la coincidencia " +"completa." + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "" +"Experimentos seleccionados de la reconstrucción de la función de ataque " +"anterior, agrupados según lo que nos enseñaron." + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"Un registro de capacitación que mantenga esta secuencia puede enseñar más " +"que la fuente final por sí sola. Conecta cada cambio propuesto con un " +"resultado observado y la siguiente decisión. Mantener variaciones ineficaces" +" también es importante: le dicen a un agente posterior qué hipótesis ya han " +"sido probadas en ese contexto." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}Me gustaría que el modelo aprendiera a utilizar la " +"retroalimentación.{ph1} Un resultado neutral es una razón para reconsiderar " +"la variación; una discrepancia modificada puede indicar otra causa. Cuando " +"las ediciones locales dejan de ser informativas, el agente debería poder " +"revisar la representación de la fuente o inspeccionar la evidencia " +"nuevamente." + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"Aprender a tomar esas decisiones requiere autonomía. A veces, el siguiente " +"paso útil es inspeccionar a otra persona que llama en lugar de compilar otra" +" edición. Una secuencia rígida de reescrituras requeridas eliminaría esa " +"elección, junto con parte del comportamiento que queremos que aprenda el " +"modelo." + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "" +"Por qué la ingeniería inversa es interesante para el aprendizaje por " +"refuerzo" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"La reconstrucción acotada nos brinda un entorno práctico para estudiar estas" +" opciones. Una vez que se corrigen las versiones de destino y del " +"compilador, muchos experimentos de origen producen comentarios rápidos y " +"repetibles. El agente puede compilar un candidato, inspeccionar la " +"discrepancia y utilizar esa observación para elegir el siguiente paso." + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"El aprendizaje por refuerzo podría entrenar ese proceso de toma de " +"decisiones a través de la retroalimentación que recibe. Un candidato " +"aceptado mediante un control definido nos da un resultado concreto, mientras" +" que la secuencia de intentos nos permite observar cuánto trabajo ha sido " +"necesario para alcanzarlo." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"Ya existen investigaciones que respaldan esta dirección. " +"{ph0}LLM4Decompile{ph1} estudia modelos especializados para la " +"descompilación. {ph2}AutoDecompiler{ph3} entrena el refinamiento de varios " +"turnos utilizando el compilador y los comentarios de ejecución. Sus estudios" +" se centran en la recuperación funcional. Para este trabajo, los usaría como" +" punto de partida para la tarea relacionada de encontrar una fuente que " +"coincida con la salida de un compilador histórico." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"Sin embargo, para alcanzar una tarea tan bien definida, es posible que " +"primero necesitemos establecer qué recursos posee una función o diseñar una " +"nueva medición del tiempo de ejecución. En TH08, por ejemplo, una " +"{ph0}diferencia de colisión llevó a cómo el programa original manejaba los " +"cálculos de punto flotante{ph1}. Antes de que el agente pudiera probar una " +"corrección útil, tenía que decidir qué medir." + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"Esto sugiere otra función para un modelo con amplias capacidades: ayudar a " +"que el problema sea verificable en primer lugar. Una vez que exista esa " +"verificación, podemos investigar hasta dónde puede llegar una política de " +"búsqueda aprendida." + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "Cuando el verificador está equivocado" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"Al entrenar con una comprobación, su cobertura pasa a formar parte del " +"entorno de aprendizaje. Llamo oracle a esa referencia o verificador. Aunque " +"la respuesta sea repetible, debemos entender qué verifica realmente." + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 mostró cuánto se puede ocultar fuera de esa cobertura. Algunas " +"funciones reconstruidas pasaron sus comparaciones exactas con constantes " +"flotantes incorrectas. El comparador ajustó las direcciones reubicadas antes" +" de verificar las instrucciones, pero verificó los valores detrás de esas " +"direcciones solo cuando se habían registrado explícitamente para su " +"verificación. Otras referencias literales podrían pasar sin que se examinen " +"sus valores." + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"Para abordar esta brecha, una {ph0}auditoría más amplia{ph1} encontró " +"constantes incorrectas en funciones que ya habían pasado la verificación. En" +" un cálculo, el código fuente reconstruido agregó {ph2}, mientras que el " +"original agregó {ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}Un modelo recompensado por la comprobación anterior podría aprender a " +"producir una reconstrucción incorrecta.{ph1} No entrenamos un modelo RL " +"usando ese verificador. El fallo que observamos estaba en la propia " +"comprobación. Pero si lo usáramos como recompensa, el mismo punto ciego " +"pasaría a formar parte del entorno de aprendizaje." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"Una prueba funcional puede tener un punto ciego similar. {ph0}La " +"investigación que utilizó fuzzing diferencial en descompiladores LLM{ph1} " +"encontró diferencias de comportamiento fuera de las pruebas proporcionadas. " +"Por lo tanto, debemos entender una prueba aprobatoria en términos de los " +"casos en los que se aplica. De manera similar, una coincidencia de " +"instrucciones debe entenderse en términos de los bytes y datos que verifica." + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"Antes de utilizar cualquier tipo de control de verificación como recompensa," +" lo probaría con resultados deliberadamente incorrectos. También mantendría " +"la aceptación final separada del código fuente que el agente de IA puede " +"editar. El modelo debe pasar la verificación reconstruyendo el programa " +"correctamente. De lo contrario, simplemente copiar una matriz de bytes del " +"destino podría engañar a un sistema de puntuación ingenuo sin recuperar el " +"código fuente útil." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}Mejorar al alumno y mejorar su verificador pertenecen al mismo problema" +" de investigación.{ph1} A medida que el alumno mejora en la optimización de " +"los comentarios que le damos, comprender esos comentarios también se vuelve " +"más importante." + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "Una comparación que vale la pena realizar" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"Con esas comprobaciones implementadas, comenzaría la comparación en una " +"familia de tareas fija VC7.1. Primero, establecería líneas de base para un " +"modelo de IA de propósito general sólido y uno más pequeño. Agregar " +"conocimiento del repositorio al modelo más pequeño mostraría qué cambios en " +"la recuperación. A partir de ahí, la especialización supervisada y una " +"versión capacitada con RL nos permitirían examinar qué agrega la " +"capacitación." + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"Para que esas comparaciones sean informativas, cada modelo recibiría la " +"misma evidencia y herramientas objetivo. Para las comparaciones específicas " +"de la capacitación, también mantendría fijo el modelo base y el conocimiento" +" disponible. Las pruebas repetidas ayudarían a distinguir una estrategia de " +"búsqueda confiable de una solución candidata que tiene éxito por casualidad." + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"Luego separaría dos condiciones iniciales: reconstruir una función a partir " +"de sus instrucciones y cerrar las últimas diferencias en la fuente " +"mantenida. Un modelo de IA especializado podría ser excelente en la segunda " +"tarea, mientras que un modelo de IA de propósito general hace gran parte del" +" trabajo necesario para alcanzarla. Medirlos por separado revelaría esa " +"división del trabajo." + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"Ambas tareas necesitan ejemplos de prueba excluidos del entrenamiento y la " +"recuperación. Los juegos Touhou relacionados comparten el código del motor. " +"Excluir un juego del entrenamiento aún podría dejar funciones casi idénticas" +" en el conjunto de entrenamiento. En cambio, excluiría grupos enteros de " +"código fuente relacionado y mantendría sus respuestas verificadas fuera de " +"los datos de entrenamiento y de la biblioteca de recuperación." + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"Para estos ejemplos, el resultado serían coincidencias completas y " +"verificadas de forma independiente. La igualdad de presupuestos para las " +"propuestas de los candidatos o los intentos de compilación ayudaría a " +"comparar la calidad de la búsqueda. Medir el tiempo y el costo total, " +"incluida la compilación, mostraría si esa calidad produce un flujo de " +"trabajo eficiente. Los costos de capacitación deben medirse por separado." + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"Ambas medidas importan. Una propuesta barata puede generar un flujo de " +"trabajo costoso si genera más compilaciones que no proporcionan información " +"útil. Por el contrario, un modelo más lento puede resultar más económico si " +"encuentra antes un experimento útil. La comparación debe incluir estos " +"costos." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"Contar los resultados aceptados también ayuda a interpretar los avances " +"intermedios. Un {ph0}candidato para guardar repeticiones{ph1} de TH09 " +"parecía prometedor porque la función compilada tenía el tamaño esperado, " +"pero la comparación completa aún encontró diferencias importantes. La forma " +"en que el compilador ordena las instrucciones puede hacer que un candidato " +"parezca más cercano sin resolver el problema de fondo. Estos diagnósticos " +"orientan el siguiente experimento; antes de premiar una mayor cercanía, la " +"contrastaría con la comprobación completa de aceptación." + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"Me interesaría conocer el patrón de estos resultados. Si un modelo de IA " +"especializado funciona bien dentro de una familia de compiladores familiar, " +"¿esa ventaja sobrevive a un cambio en la cadena de herramientas? Y una vez " +"que ambos modelos de IA tengan una buena recuperación, ¿cuánto aporta aún el" +" entrenamiento? Las respuestas nos ayudarían a decidir dónde es más útil " +"cada tipo de modelo." + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "¿Qué tipo de agente de IA sugiere esto?" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"Si los modelos resultan útiles en diferentes etapas, un acuerdo útil podría " +"ser un agente amplio con acceso a un modelo de IA especializado. Una vez que" +" una tarea se convierte en una búsqueda familiar del compilador, podría " +"solicitar candidatos al modelo de IA especializado. Si los resultados dejan " +"de tener sentido, el agente más amplio podría volver a la evidencia objetivo" +" o investigar la medición en sí." + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"En ese arreglo, la especialización se convierte en una herramienta que el " +"agente puede elegir a medida que se desarrolla la investigación. El agente " +"conserva la libertad de cambiar la dirección que hizo posibles los avances " +"anteriores." + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"Esto se deriva naturalmente del cambio industrial que describí en el ensayo " +"anterior. Poder realizar muchos más experimentos cambia la forma en que " +"organizamos el trabajo. También nos da una razón para preguntar qué " +"decisiones requieren un juicio amplio y cuáles mejoran a través de " +"experiencias repetidas con un oficio en particular." + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"Por ahora, el modelo general más potente se ha ganado su lugar en mi flujo " +"de trabajo al resolver problemas que había dejado pendientes. Esos éxitos me" +" llevan a examinar el proceso: ¿cuánto dependía de la amplitud del modelo y " +"cuánto podría asumir uno entrenado específicamente para esta clase de " +"reconstrucción?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}Los repositorios nos brindan una forma de probar la pregunta.{ph1} Me " +"gustaría usar ese registro para realizar la comparación y ver qué " +"aprendemos." + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "Atribución de evidencia y modelo" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"Los ejemplos describen comparaciones históricas a nivel de función " +"registradas en las confirmaciones vinculadas. Las comprobaciones del tiempo " +"de ejecución del juego completo son hitos separados. Los registros fuente y " +"coincidentes se revisaron el 13 de octubre de 2026." + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"La etiqueta TH09 {ph0} significa GPT-6-Astra. El modelo asociado con {ph1} " +"cambió con el tiempo, por lo que esa etiqueta por sí sola no puede " +"identificar el modelo utilizado en una ejecución específica. Los nombres de " +"modelos explícitos en las otras confirmaciones citadas proporcionan la " +"atribución registrada. Estos historiales no incluyen mediciones consistentes" +" de costo o esfuerzo para cada ejecución." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 duración de los objetos temporales:{ph1} la {ph2}revisión de " +"contexto anterior{ph3} y la {ph4}reconstrucción completa verificada mediante" +" una recompilación limpia del código fuente{ph5}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}TH09 Front devolución de llamada:{ph1} {ph2}regiones de diagnóstico que" +" no coincidieron{ph3}, {ph4}Astra nuevo gráfico de flujo de control{ph5} y " +"su {ph6}verificación de recompilación aceptada{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}Conocimiento del compilador:{ph1} {ph2}TH095 reutilización del " +"comportamiento del identificador-depósito{ph3}, {ph4}TH04 código fuente del " +"jugador y orden del enlazador{ph5} y {ph6}Luna reconstrucción completa de la" +" función de creación de enemigos de TH09{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}Comprobaciones y diagnósticos:{ph1} {ph2}TH08 auditoría constante de " +"punto flotante{ph3}, {ph4}TH08 investigación numérica{ph5} y {ph6}TH09 " +"reconstrucción para guardar reproducciones{ph7}." + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"Las comparaciones propuestas de modelos especializados y entrenamiento RL " +"son experimentos futuros. Las historias de los proyectos los motivan. Para " +"este ensayo no se evaluó ningún modelo especializado entrenado." + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "¿La ingeniería inversa necesita un modelo más grande? | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"Lo que me enseñó la reconstrucción Touhou sobre modelos generales, modelos " +"especializados y cómo aprender a proponer mejores hipótesis a partir de los " +"comentarios del compilador." + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "En esta página" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "Modelos y ensayo de ingeniería inversa." + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "Experimentos de reconstrucción seleccionados." + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "Volver arriba" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "Por N0zoM1z0 · 13 de octubre de 2026 · Borrador para revisión" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/fa/messages.po b/website/locales/fa/messages.po index 18e540bfc..f515f87c0 100644 --- a/website/locales/fa/messages.po +++ b/website/locales/fa/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "افکار در مهندسی معکوس." #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "بازسازی بازی{ph0}مقاله" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "انقلاب صنعتی در مهندسی معکوس" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,20 @@ msgstr "" "چه دستی دوباره و بازسازی TH08, TH095 و TH04 به من آموخت در مورد عوامل, تأیید" " و دانش است که ترکیبات در سراسر پروژه." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "مقاله را بخوانید" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "توسط N0zoM1z0 * 10 اکتبر 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -99,6 +100,8 @@ msgstr "" "افکار در مهندسی معکوس ، با استفاده از تجربه عملی با REA و عوامل هوش مصنوعی." #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -135,6 +138,7 @@ msgstr "" "شما." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -155,6 +159,7 @@ msgid "REA home" msgstr "صفحهٔ اصلی REA" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -175,6 +180,7 @@ msgid "Main navigation" msgstr "ناوبری اصلی" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -195,6 +201,7 @@ msgid "REA on GitHub" msgstr "REA در GitHub" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -215,6 +222,7 @@ msgid "Join REA on Discord" msgstr "به REA در Discord بپیوندید" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -234,7 +242,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "REA را در X دنبال کنید" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "بازگشت به بالا" @@ -11479,6 +11487,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "فاصله یکسان: 256 × 16 = 16 واحد زاویه = 22.5 درجه" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -11499,6 +11508,7 @@ msgid "Showcases" msgstr "نمونه‌ها" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -11519,6 +11529,7 @@ msgid "Blog" msgstr "وبلاگ" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -11539,6 +11550,7 @@ msgid "Get started" msgstr "شروع کنید" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -11558,7 +11570,8 @@ msgctxt "shared" msgid "Guides" msgstr "راهنماها" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11578,7 +11591,8 @@ msgctxt "shared" msgid "Source" msgstr "کد منبع" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11598,7 +11612,8 @@ msgctxt "shared" msgid "MIT license" msgstr "مجوز MIT" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11618,7 +11633,8 @@ msgctxt "shared" msgid "Join the community" msgstr "به جامعه بپیوندید" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12603,8 +12619,1294 @@ msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "نویسنده: N0zoM1z0 · ۱۳ اکتبر ۲۰۲۶ · پیش‌نویس برای بازبینی" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "توسط N0zoM1z0 · 13 اکتبر 2026" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "پرش به محتوا" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "{ph0}صفحه اصلی{ph1} / {ph2}وبلاگ{ph3} / مدل‌ها و مهندسی معکوس" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "انشا" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "آیا مهندسی معکوس به مدل بزرگتری نیاز دارد؟" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "توسط N0zoM1z0 · 13 اکتبر 2026" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "نکات اصلی" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}قابلیت مدل بر فرضیه‌هایی که یک عامل هوش مصنوعی می‌تواند ارائه دهد تأثیر" +" می‌گذارد.{ph1} جلسات بعدی مشکلاتی را که کار قبلی حل نشده رها کرده بود حل " +"کرد." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}یک کامپایلر آشنا الگوهای مشخصی را برای یادگیری یک مدل تخصصی ارائه " +"می‌کند.{ph1} برخی از تطابق دقیق کد ماشین به الگوهای کد منبع خاص و قابل " +"استفاده مجدد بستگی دارد." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}دانش مخزن را در یک مقایسه منصفانه لحاظ کنید.{ph1} یک مدل کوچکتر با " +"بازیابی دانش خوب ممکن است اطلاعات کافی برای پیشرفت داشته باشد." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}مدل را آموزش دهید تا آزمایش مفید بعدی را انتخاب کند و بررسی‌کننده تأیید" +" آن را آزمایش کنید.{ph1} بازخورد فقط در حدی مفید است که بررسی‌کننده موارد " +"درست را تأیید کند." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}نتایج بازسازی تایید شده را در کنار زمان و هزینه ارزیابی کنید.{ph1} این " +"چیزی است که من می‌خواهم برای گردش کار کامل عامل هوش مصنوعی اندازه‌گیری کنم." + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "در این مقاله" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}مدل‌های قوی‌تر چه چیزی را تغییر دادند؟{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}چرا تخصص داشته باشید؟{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}این تجربه در کجا باید ذخیره شود؟{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}یادگیری آزمایش مفید بعدی{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}مهندسی معکوس و یادگیری تقویتی{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}هنگامی که بررسی کننده تأیید اشتباه است{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}مقایسه ای که ارزش اجرا دارد{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}این چه نوع عامل هوش مصنوعی را پیشنهاد می کند؟{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "مدل‌های قوی‌تر چه چیزی را تغییر دادند؟" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"از آگوست 2026، من در حال بررسی میزان مهندسی معکوس یک عامل با دسترسی به یک " +"برنامه و راهی برای بررسی کار آن هستم. بیشتر تجربه من از بازسازی بازی ها به " +"خصوص Touhou ناشی می شود. همانطور که من مدل‌ها را تغییر داده‌ام، مخازن رکوردی" +" از آنچه هر جلسه توانسته است بازیابی کند را نگه داشته است." + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"هدف بیشتر این کار بازسازی دقیق است: ما منبع نگهداری شده را تحت یک زنجیره " +"ابزار تاریخی با نسخه های ثابت کامپایل می کنیم و بررسی می کنیم که آیا تابع " +"هدف را بازتولید می کند یا خیر. هنگامی که مدل بعدی در جایی که کار قبلی متوقف " +"شده بود موفق می شود، تغییرات منبع نشان می دهد که چه چیزی این پیشرفت را ممکن " +"کرده است." + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "متغیرهای موقت به طول عمر همپوشانی نیاز داشتند" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"بیشتر منطق یک تابع حمله TH09 قبلاً بازسازی شده بود، اما کامپایلر همچنان " +"نتایج موقت را متفاوت از نسخه اصلی ذخیره می کرد. مشکل باقی‌مانده این بود که " +"این نتایج چقدر طول می‌کشد تا زنده بمانند و چه زمانی می‌توان از ذخیره‌سازی " +"آنها استفاده مجدد کرد." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"یک {ph0}GPT-6-Sol رکورد جلسه{ph1} این عدم تطابق را پس از آزمایش چندین ترتیب " +"کد منبع معمولی گزارش کرد. وقتی {ph2}GPT-6.1-Sol دوباره مشکل را بررسی " +"کرد{ph3}، یک فرضیه مفید در مورد طول عمر این متغیرهای موقت ارائه کرد." + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"این تابع نتایج موقتی تولید کرد و آنها را در موقعیت های انیمیشن کپی کرد. در " +"C++، یک نتیجه موقت زنده می ماند تا زمانی که عبارت استفاده شده از آن تمام " +"شود. گروه‌بندی عملیات واقعی در یک عبارت، آن نتایج را با هم زنده نگه می‌دارد،" +" بنابراین کامپایلر دیگر نمی‌تواند از فضای ذخیره‌سازی یکسان برای همه آنها " +"استفاده مجدد کند." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"یک نتیجه همچنان فضای ذخیره‌سازی را با مقداری به اشتراک گذاشته است که بعداً " +"برای بررسی‌های برخورد استفاده می‌شود. دادن این ارزش به طول عمر خودش، " +"همپوشانی باقیمانده را حل کرد. پس از چندین آزمایش هدفمند، {ph0}کامپایل مجدد " +"تمیز از منبع تأیید کرد که تابع کامل مطابقت دارد{ph1}." + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"من علاقه مندم که چگونه این کشف و آزمایش ها از یکدیگر حمایت کردند. فرضیه در " +"مورد طول عمر اشیاء موقت به عامل هوش مصنوعی مسیر جدیدی داد. بازخورد کامپایلر " +"به آن کمک کرد تا این فرضیه را به ترتیب کد منبع که مطابقت دارد، اصلاح کند." + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "یک نمودار کنترل جریان متفاوت" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"تابع دوم TH09 نشان می دهد که چرا آن جهت جدید می تواند مهم باشد. در اینجا، " +"مانع این بود که چگونه چندین شعبه کار نهایی خود را به اشتراک گذاشتند. کار " +"قبلی {ph0}GPT-6.1-Sol{ph1} بخش‌هایی از تابع را مطابقت داد، اما شاخه‌ها " +"همچنان متفاوت از نسخه اصلی به یکدیگر متصل شدند." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra نمودار کنترل جریان اطراف را تغییر داد{ph1}. مسیرهای اصلی " +"اکنون به یک به‌روزرسانی نهایی مشترک و بازگشت منجر می‌شوند. در آن ساختار " +"منبع، کامپایلر شاخه ها را به همان روش اصلی ترکیب می کند. تابع کامل مطابقت " +"داشت و {ph2}بررسی بازسازی جداگانه آن را تأیید کرد{ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}تغییر نمایش، فضای جستجوی سازنده‌تری را باز کرد.{ph1} هر دو مورد دلیلی " +"برای جدی گرفتن قابلیت مدل کلی به من می‌دهد: یک نماینده می‌تواند قبل از یافتن" +" راهی بهتر برای بیان کار اطراف، برای مدت طولانی تغییرات محلی را امتحان کند." + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"در همان زمان، جلسات بعدی با منبع قبلی شروع شد و خرابی ها ثبت شد. این کار " +"انباشته بخشی از مزیت آنها بود. تاریخ به ما اجازه می دهد تا کشفیات و بررسی " +"های آنها را بررسی کنیم. یک مقایسه کنترل شده به تفکیک اثر مدل از تجربه ای که " +"به ارث برده است کمک می کند." + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"این چیزی است که سوال را برای من جالب می کند. برخی پیشرفت ها از یافتن " +"نمایندگی جدید حاصل شد. بسیاری از کارهای باقی مانده شامل آزمایش ترتیبات مختلف" +" کد منبع در برابر یک کامپایلر آشنا بود. چه مقدار از این به توانایی استدلال " +"گسترده نیاز دارد؟ یک مدل با کار مکرر با آن کامپایلر و کد چقدر می تواند یاد " +"بگیرد؟" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "چرا یک مدل تخصصی هوش مصنوعی ممکن است خوب عمل کند؟" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou دلیل مشخصی به ما می دهد که انتظار داشته باشیم آن تجربه کمک کند. بیشتر" +" کد بازی از همان نویسنده، ZUN می آید، بنابراین عادت های تکرار شونده یک فرض " +"شروع منطقی هستند. مخازن با مثال‌های مفید این موضوع را تأیید می‌کنند: " +"حلقه‌های مرتبط و ترتیبات کمکی فرضیه‌هایی را در بین بازی‌ها ارائه کرده‌اند." + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"وقتی کامپایلر را وارد کنیم، این الگوهای منبع خاص تر می شوند. تحت یک نسخه خاص" +" و مجموعه ای از گزینه ها، تغییرات کوچک در منبع می تواند تفاوت های قابل تکرار" +" در کد ماشین ایجاد کند. برخی از این تفاوت ها به طرز شگفت آوری مبهم هستند." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"به عنوان مثال، در TH095، {ph0}حتی نام یک متغیر محلی می‌تواند بر جایی که " +"VC7.1 آن را ذخیره کرده است، تأثیر بگذارد{ph1}. انتقال اعلان آن طرح ذخیره " +"سازی را حل نکرد. با این حال، یک تابع پس زمینه قبلی یک الگوی نامگذاری را نشان" +" داده بود که بر انتخاب کامپایلر تأثیر می گذاشت. این فرضیه امیدوارکننده تری " +"را برای {ph2}بازسازی {ph3}{ph4} پیشنهاد کرد." + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"استفاده مجدد از آن الگو برای متغیر اندازه فایل، طرح مورد نیاز را بدون افزودن" +" فضای ذخیره سازی ایجاد کرد. سپس تابع کامل مطابقت داشت." + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"این یک جزئیات بسیار خاص است که یک مدل هوش مصنوعی همه منظوره ممکن است زمانی " +"که برای اولین بار با مشکل مواجه می شود از آن مطلع نباشد. یک مدل تخصصی برای " +"این کامپایلر می‌تواند موارد بسیاری از این قبیل را ببیند و یاد بگیرد که کدام " +"تغییرات کد منبع ارزش آزمایش را دارند." + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"این فرصت به ویژه در مسائل تطبیق نهایی که ما قبلاً رفتار را درک کرده ایم، " +"واضح است. سپس می‌پرسیم این کامپایلر کدام ترتیب کد منبع معمولی را به " +"دستورالعمل‌های مشاهده شده تبدیل می‌کند. تخصص می تواند به مدل کمک کند تا آن " +"اشکال را به طور کارآمدتر پیشنهاد کند، در حالی که هدف هنوز تعیین می کند که " +"کدام فرضیه ها باقی می مانند." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"الگوهای مفید باید زمینه خود را حفظ کنند. TH08 از نسخه Visual C++ قبلی نسبت " +"به TH095 و TH09 استفاده می کند. TH04 از یک کامپایلر متفاوت از دوران DOS " +"استفاده می کند. {ph0}بازسازی کد بازیکن{ph1} آن به فرضیه‌های مختلفی در مورد " +"الگوهای کد منبع و نحوه پیوند کد کامپایل‌شده به یک فایل اجرایی نیاز داشت. " +"داشتن یک نویسنده یک انتظار اولیه را به ما می‌دهد، اما هر هدف به ما می‌گوید " +"که آیا کاربرد دارد یا خیر. یک مدل تخصصی باید این زمینه ها را متمایز کند." + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "این تجربه در کجا باید ذخیره شود؟" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"قبل از تصمیم به آموزش آن الگوها از طریق آموزش مدل، به دانشی که قبلاً در مخزن" +" نگهداری می‌کنیم نگاهی می‌اندازم. در {ph0}مقاله قبلی{ph1}، درباره تبدیل شدن " +"آن به حافظه کاری پروژه نوشتم. این حافظه شامل درس های کامپایلر، همراه با منبع" +" و آزمایش های ناموفق است که آنها را مفید ساخته است." + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"با بازیابی آن رکورد، یک نماینده می تواند با یک نتیجه شناخته شده شروع کند و " +"نحوه رسیدن ما به آن را بررسی کند. همچنین می تواند از جایگزین هایی که قبلاً " +"در همان زمینه شکست خورده اند اجتناب کند. نقطه شروع بدون تغییر پارامترهای " +"آموزش دیده مدل بهبود می یابد." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"هنگام بازسازی عملکرد TH09 برای ایجاد دشمن، یک جلسه {ph0}GPT-6-Luna-max{ph1} " +"یک فرضیه را از کد منبع TH08 مرتبط گرفت. آن کد قبلی نحوه سازماندهی حلقه و " +"مدیریت خرابی های داخل آن را پیشنهاد می کرد. شواهد خود TH09 جزئیات مربوط به " +"بازی را ارائه کرد. بازسازی از منبع تأیید کرد که عملکرد کامل مطابقت دارد." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}\"مدل نمی‌دانست که ویژگی کامپایلر\" و \"مدل نمی‌تواند از آن استفاده " +"کند\" مشکلات متفاوتی هستند.{ph1} بازیابی ممکن است مشکل اول را حل کند، اما " +"تبدیل این واقعیت به یک نامزد مفید هنوز کار می‌برد. و اگر کد منبع کاندید شکست" +" بخورد، مدل باید تصمیم بگیرد که نتیجه برای تلاش بعدی چه معنایی دارد. دانستن " +"الگوی مربوطه تنها آغاز آن فرآیند است." + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"به همین دلیل، من یک مدل هوش مصنوعی همه منظوره کوچکتر را با بازیابی خوب در هر" +" مقایسه قرار می دهم. ارائه همان کتابخانه تجربی به ما کمک می کند تا تشخیص " +"دهیم که آموزش چه چیزی را اضافه می کند: تولید نامزد بهتر، تصمیم گیری بهتر پس " +"از بازخورد، یا هر دو." + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "یادگیری آزمایش مفید بعدی" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"آن تصمیمات پس از بازخورد همان چیزی است که من بیشتر دوست دارم مطالعه کنم. در " +"بازسازی عاملی، مدل توضیحی را پیشنهاد می کند و آن را در برابر برنامه آزمایش " +"می کند. حتی زمانی که کد منبع نامزد ناموفق باشد، نتیجه می تواند احتمالات " +"باقیمانده را محدود کند و آزمایش بعدی مفیدتری را پیشنهاد کند." + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"مثال موقت نتایج را دوباره در نظر بگیرید. منبع نهایی آن ترتیبی را نشان می دهد" +" که کارآمد است، اما مسیر رسیدن به آن منبع حاوی اطلاعات بیشتری است. هر یک از " +"آزمایش‌های زیر به شناسایی آنچه که هنوز باید تغییر کند کمک کرد." + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "آزمایش کنید" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "آنچه نتیجه آن به ما آموخت" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "روش دیگری را برای کپی کردن نتایج امتحان کنید" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "کد کپی تغییر می کند، اما عملکرد همچنان متفاوت است." + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "از مقادیر برگشتی مستقیما استفاده کنید" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "" +"کپی کردن از الگوی اصلی پیروی می کند، اما ذخیره سازی موقت همچنان متفاوت است." + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "همه متغیرهای موقت را همزمان معتبر نگه دارید" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "" +"آنها فضای ذخیره جداگانه ای دریافت می کنند، اما هنوز هم مقداری که بعداً برای " +"بررسی برخورد استفاده می شود، همپوشانی دارد." + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "مقدار collision-check را جدا نگه دارید" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "یک کامپایل مجدد تمیز از منبع، مطابقت کامل را تأیید می کند." + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "" +"آزمایش‌های منتخب از بازسازی عملکرد حمله در بالا، گروه‌بندی شده بر اساس آنچه " +"به ما آموختند." + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"یک رکورد آموزشی که این توالی را حفظ می کند می تواند بیشتر از منبع نهایی به " +"تنهایی آموزش دهد. هر تغییر پیشنهادی را به یک نتیجه مشاهده شده و تصمیم بعدی " +"مرتبط می کند. حفظ تغییرات بی اثر نیز مهم است: آنها به عامل بعدی می گویند که " +"کدام فرضیه قبلاً در آن زمینه آزمایش شده است." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}من می‌خواهم مدل یاد بگیرد که چگونه از بازخورد استفاده کند.{ph1} یک " +"نتیجه خنثی دلیلی برای تجدید نظر در تغییر است. تغییر عدم تطابق ممکن است به " +"دلیل دیگری اشاره کند. وقتی ویرایش‌های محلی آموزنده نیستند، عامل باید بتواند " +"بازنمایی منبع را بازبینی کند یا شواهد را دوباره بررسی کند." + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"یادگیری انجام آن انتخاب ها مستلزم استقلال است. گاهی اوقات گام بعدی مفید، " +"بررسی تماس گیرنده دیگر به جای کامپایل ویرایش دیگری است. یک توالی سفت و سخت " +"از بازنویسی های مورد نیاز این انتخاب را حذف می کند، همراه با بخشی از رفتاری " +"که می خواهیم مدل یاد بگیرد." + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "چرا مهندسی معکوس برای یادگیری تقویتی جالب است" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"بازسازی با دامنه مشخص محیطی عملی برای مطالعه این تصمیم‌ها فراهم می‌کند. با " +"ثابت نگه داشتن نسخه هدف و کامپایلر، بسیاری از آزمایش‌های کد منبع بازخورد " +"سریع و تکرارپذیر می‌دهند. عامل می‌تواند کد پیشنهادی را کامپایل کند، تفاوت‌ها" +" را بررسی کند و گام بعدی را انتخاب کند." + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"یادگیری تقویتی می تواند این فرآیند تصمیم گیری را از طریق بازخوردی که دریافت " +"می کند آموزش دهد. یک نامزد پذیرفته شده توسط یک چک تعریف شده یک نتیجه مشخص به" +" ما می دهد، در حالی که توالی تلاش ها به ما امکان می دهد مشاهده کنیم که چقدر " +"کار برای رسیدن به آن لازم است." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"در حال حاضر تحقیقاتی وجود دارد که از این جهت حمایت می کند. " +"{ph0}LLM4Decompile{ph1} مدل های تخصصی برای دیکامپایل را مطالعه می کند. " +"{ph2}AutoDecompiler{ph3} با استفاده از بازخورد کامپایلر و اجرا، اصلاح چند " +"نوبتی را آموزش می‌دهد. مطالعات آنها بر بازیابی عملکردی متمرکز است. برای این " +"کار، من از آنها به عنوان نقطه شروع برای کار مرتبط یافتن منبعی که با خروجی یک" +" کامپایلر تاریخی مطابقت دارد استفاده می کنم." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"با این حال، برای رسیدن به چنین وظیفه ای به خوبی تعریف شده، ممکن است ابتدا " +"نیاز داشته باشیم که یک تابع دارای چه منابعی است یا یک اندازه گیری زمان اجرا " +"جدید را طراحی کنیم. به عنوان مثال، در TH08، یک {ph0}تفاوت برخورد منجر به " +"نحوه مدیریت برنامه اصلی با محاسبات ممیز شناور شد{ph1}. قبل از اینکه عامل " +"بتواند یک تصحیح مفید را آزمایش کند، باید تعیین می کرد که چه چیزی را باید " +"اندازه گیری کند." + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"این یک نقش دیگر را برای یک مدل به طور گسترده نشان می دهد: کمک به قابل بررسی " +"کردن مشکل در وهله اول. هنگامی که آن چک وجود داشت، می‌توانیم بررسی کنیم که یک" +" خط‌مشی جستجوی آموخته شده تا چه حد می‌تواند با آن برسد." + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "هنگامی که بررسی کننده تأیید اشتباه است" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"وقتی یک بررسی را برای آموزش به کار می‌بریم، دامنه آن هم بخشی از محیط یادگیری" +" می‌شود. من چنین مرجع یا ابزار بررسی را اوراکل می‌نامم. حتی پاسخ تکرارپذیر " +"را باید در محدوده چیزی فهمید که واقعاً بررسی می‌کند." + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 نشان داد که مشکلات می‌توانند بیرون از دامنه بررسی پنهان بمانند. بعضی " +"توابع بازسازی‌شده با وجود ثابت‌های ممیز شناور اشتباه، مقایسه دقیق را قبول " +"شدند. مقایسه‌کننده پیش از بررسی دستورها آدرس‌های جابه‌جا‌شده را تنظیم " +"می‌کرد، اما مقدار اشاره‌شده را فقط وقتی بررسی می‌کرد که برای اعتبارسنجی " +"صریحاً ثبت شده بود. ارجاع‌های دیگر به ثابت‌ها می‌توانستند بدون بررسی مقدار " +"قبول شوند." + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"برای رفع این شکاف، یک {ph0}ممیزی گسترده‌تر{ph1} ثابت‌های نادرستی را در " +"توابعی که قبلاً تأیید شده بودند پیدا کرد. در یک محاسبه، کد منبع بازسازی شده " +"{ph2} را اضافه کرد، در حالی که کد اصلی {ph3} را اضافه کرد." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}مدلی که توسط چک راستی‌آزمایی قدیمی پاداش می‌گیرد، می‌تواند یاد بگیرد که" +" یک بازسازی نادرست ایجاد کند.{ph1} ما یک مدل RL را با استفاده از آن " +"بررسی‌کننده آموزش ندادیم. نقصی که مشاهده کردیم در خود چک بود. اما اگر از آن " +"به عنوان پاداش استفاده می کردیم، همان نقطه کور بخشی از محیط آموزشی می شد." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"یک تست عملکردی می تواند نقطه کور مشابهی داشته باشد. {ph0}تحقیق با استفاده از" +" fuzzing دیفرانسیل در دیکامپایلرهای LLM{ph1} تفاوت‌های رفتاری را خارج از " +"آزمون‌های ارائه شده یافت. بنابراین ما باید یک آزمون قبولی را بر حسب مواردی " +"که انجام می دهد درک کنیم. به طور مشابه، یک تطابق دستورالعمل باید از نظر بایت" +" ها و داده هایی که تأیید می کند درک شود." + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"قبل از استفاده از هر نوع بررسی تأیید به عنوان پاداش، آن را با نتایج عمداً " +"نادرست آزمایش می‌کنم. من همچنین پذیرش نهایی را از کد منبعی که عامل هوش " +"مصنوعی می تواند ویرایش کند جدا نگه می دارم. مدل باید با بازسازی صحیح برنامه،" +" تأیید صحت داشته باشد. در غیر این صورت، کپی کردن یک آرایه بایتی از هدف می " +"تواند یک سیستم امتیازدهی ساده لوح را بدون بازیابی کد منبع مفید فریب دهد." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}بهبود یادگیرنده و بهبود بررسی‌کننده تأیید آن مربوط به یک مسئله تحقیقاتی" +" است.{ph1} همانطور که یادگیرنده در بهینه‌سازی بازخوردی که ما به آن می‌دهیم " +"بهتر می‌شود، درک آن بازخورد نیز مهم‌تر می‌شود." + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "مقایسه ای که ارزش اجرا دارد" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"با انجام این بررسی‌ها، مقایسه را روی یک خانواده کار ثابت VC7.1 شروع می‌کنم. " +"اول، من خطوط پایه را برای یک مدل هوش مصنوعی همه منظوره قوی و یک مدل کوچکتر " +"ایجاد می کنم. افزودن دانش مخزن به مدل کوچکتر، تغییرات بازیابی را نشان می " +"دهد. از آنجا، تخصص تحت نظارت و یک نسخه آموزش داده شده RL به ما اجازه می دهد " +"بررسی کنیم که آموزش چه چیزی را اضافه می کند." + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"برای اینکه این مقایسه ها آموزنده باشد، هر مدل شواهد و ابزارهای هدف یکسانی را" +" دریافت می کند. برای مقایسه‌های آموزشی خاص، مدل پایه و دانش موجود را نیز " +"ثابت نگه می‌دارم. آزمایش‌های مکرر به تمایز یک استراتژی جستجوی قابل اعتماد از" +" راه‌حل نامزدی که به طور تصادفی موفق می‌شود کمک می‌کند." + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"سپس دو شرط شروع را جدا می‌کنم: بازسازی یک تابع از دستورالعمل‌های آن، و بستن " +"چند تفاوت آخر در منبع حفظ شده. یک مدل هوش مصنوعی تخصصی ممکن است در کار دوم " +"عالی باشد در حالی که یک مدل هوش مصنوعی همه منظوره بسیاری از کارهای مورد نیاز" +" برای رسیدن به آن را انجام می دهد. اندازه گیری آنها به طور جداگانه این تقسیم" +" کار را آشکار می کند." + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"هر دو کار به نمونه‌های آزمایشی نیاز دارند که از آموزش و بازیابی حذف شوند. کد" +" موتور اشتراک گذاری بازی های مرتبط Touhou. حذف یک بازی از تمرین همچنان می " +"تواند عملکردهای تقریباً یکسانی را در مجموعه تمرینی باقی بگذارد. در عوض، من " +"کل گروه‌های کد منبع مرتبط را حذف می‌کنم و پاسخ‌های تأیید شده‌شان را هم از " +"داده‌های آموزشی و هم از کتابخانه بازیابی خارج می‌کنم." + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"برای این مثال‌ها، نتیجه، مطابقت‌های کامل و تأیید شده مستقل خواهد بود. " +"بودجه‌های برابر برای پیشنهادهای نامزد یا تلاش‌های کامپایل به مقایسه کیفیت " +"جستجو کمک می‌کند. اندازه‌گیری زمان و هزینه کل، از جمله کامپایل، نشان می‌دهد " +"که آیا این کیفیت گردش کار کارآمدی ایجاد می‌کند یا خیر. هزینه های آموزشی باید" +" جداگانه اندازه گیری شود." + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"هر دو اندازه گیری مهم است. یک پروپوزال ارزان می‌تواند منجر به گردش کار " +"گران‌قیمتی شود اگر باعث کامپایل‌های بیشتری شود که اطلاعات مفیدی ارائه " +"نمی‌کند. برعکس، یک مدل کندتر ممکن است اقتصادی تر باشد اگر آزمایش مفیدی را " +"زودتر پیدا کند. مقایسه باید شامل این هزینه ها باشد." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"شمارش نتایج پذیرفته شده همچنین پیشرفت متوسط را در چشم انداز نگه می دارد. یک " +"TH09 {ph0}replay-save نامزد{ph1} امیدوارکننده به نظر می‌رسید، زیرا تابع " +"کامپایل‌شده اندازه مورد انتظار را داشت، اما یک مقایسه کامل همچنان تفاوت‌های " +"اساسی را پیدا کرد. تغییرات در نحوه چیدمان دستورالعمل ها توسط کامپایلر می " +"تواند بدون حل مشکل اساسی، نامزد را نزدیک تر کند. چنین تشخیص‌هایی می‌توانند " +"آزمایش بعدی را راهنمایی کنند، اما من هر پاداشی را برای «نزدیک‌تر شدن» در " +"برابر بررسی پذیرش کامل تأیید می‌کنم." + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"من به الگوی این نتایج علاقه مند هستم. اگر یک مدل تخصصی هوش مصنوعی در یک " +"خانواده کامپایلر آشنا به خوبی عمل کند، آیا این مزیت از تغییر زنجیره ابزار " +"باقی می ماند؟ و هنگامی که هر دو مدل هوش مصنوعی بازیابی خوبی داشته باشند، " +"آموزش هنوز چقدر اضافه می کند؟ پاسخ‌ها به ما کمک می‌کنند تصمیم بگیریم که هر " +"نوع مدل در کجا مفیدتر است." + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "این چه نوع عامل هوش مصنوعی را پیشنهاد می کند؟" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"اگر مدل ها در مراحل مختلف کمک کننده باشند، یک ترتیب مفید ممکن است عاملی " +"گسترده با دسترسی به یک مدل تخصصی هوش مصنوعی باشد. هنگامی که یک کار به یک " +"جستجوی کامپایلر آشنا تبدیل می شود، می تواند مدل تخصصی هوش مصنوعی را برای " +"نامزدها بپرسد. اگر نتایج منطقی نباشند، عامل گسترده‌تر می‌تواند به شواهد هدف " +"بازگردد یا خود اندازه‌گیری را بررسی کند." + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"در آن ترتیب، تخصص به ابزاری تبدیل می‌شود که عامل می‌تواند با توسعه تحقیقات " +"انتخاب کند. عامل آزادی تغییر جهتی را حفظ می کند که پیشرفت های قبلی را ممکن " +"کرده است." + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"این به طور طبیعی از تغییر صنعتی که در مقاله قبلی توضیح دادم ناشی می شود. " +"توانایی انجام آزمایش‌های بسیار بیشتر، نحوه سازماندهی کار را تغییر می‌دهد. " +"همچنین دلیلی به ما می دهد تا بپرسیم کدام تصمیمات نیاز به قضاوت گسترده دارند " +"و کدام یک از طریق تجربه مکرر با یک هنر خاص بهبود می یابند." + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"تا اینجا قوی‌ترین مدل عمومی جای خود را در گردش کار من پیدا کرده، چون مسائلی " +"را حل کرده که قبلاً حل نشده مانده بودند. همین موفقیت‌ها مرا به بررسی دقیق‌تر" +" فرایند ترغیب می‌کنند: چه مقدار به گستردگی توانایی مدل وابسته بود، و چه " +"مقدار را مدلی آموزش‌دیده برای همین نوع بازسازی می‌تواند انجام دهد؟" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}مخزن‌ها راهی برای آزمایش سؤال به ما می‌دهند.{ph1} من می‌خواهم از آن " +"رکورد برای ساخت مقایسه و دیدن آنچه یاد می‌گیریم استفاده کنم." + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "مدارک و اسناد مدل" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"مثال‌ها مقایسه‌های سطح عملکرد تاریخی ثبت شده در commit‌های مرتبط را توصیف " +"می‌کنند. بررسی های زمان اجرای کل بازی نقاط عطف جداگانه ای هستند. منبع و " +"سوابق مطابق در 13 اکتبر 2026 بررسی شدند." + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"برچسب TH09 {ph0} به معنای GPT-6-Astra است. مدل مرتبط با {ph1} در طول زمان " +"تغییر کرد، به طوری که برچسب به تنهایی نمی تواند مدل مورد استفاده در یک اجرای" +" خاص را شناسایی کند. نام مدل صریح در سایر کامیت‌های ذکرشده، انتساب ثبت شده " +"را ارائه می کند. این تاریخچه ها شامل اندازه گیری ثابت هزینه یا تلاش برای هر " +"اجرا نمی شود." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 طول عمر اشیای موقت:{ph1} {ph2}بررسی زمینه اولیه{ph3} و " +"{ph4}بازسازی کامل تأیید شده توسط یک کامپایل مجدد تمیز از منبع{ph5}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}TH09 Front پاسخ به تماس:{ph1} {ph2}مناطق تشخیصی که مطابقت نداشتند{ph3}،" +" نمودار جریان کنترل جدید {ph4}Astra{ph5} و {ph6}بررسی مجدد پذیرفته شده " +"آن{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}دانش کامپایلر:{ph1} {ph2}استفاده دوباره از رفتار دسته‌بندی شناسه‌ها در " +"TH095{ph3}، {ph4}کد منبع بازیکن TH04 و ترتیب لینک{ph5} و {ph6}بازسازی کامل " +"تابع ایجاد دشمن TH09 توسط Luna{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}بررسی و تشخیص:{ph1} {ph2}TH08 ممیزی ثابت ممیز شناور{ph3}، {ph4}TH08 " +"بررسی عددی{ph5} و {ph6}TH09 بازسازی صرفه جویی در پخش{ph7}." + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"مقایسه‌های پیشنهادی مدل‌های تخصصی و آموزش RL آزمایش‌های آینده هستند. تاریخچه" +" پروژه به آنها انگیزه می دهد. هیچ مدل تخصصی آموزش دیده ای برای این مقاله " +"ارزیابی نشد." + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "آیا مهندسی معکوس به مدل بزرگتری نیاز دارد؟ | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"آنچه که بازسازی Touhou در مورد مدل های عمومی، مدل های تخصصی و یادگیری ارائه " +"فرضیه های بهتر از بازخورد کامپایلر به من آموخت." + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "در این صفحه" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "مدل ها و مقاله مهندسی معکوس" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "آزمایش های بازسازی منتخب" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "بازگشت به بالا" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "نویسنده: N0zoM1z0 · ۱۳ اکتبر ۲۰۲۶ · پیش‌نویس برای بازبینی" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/fr/messages.po b/website/locales/fr/messages.po index 4c3ef3218..bf59de2ea 100644 --- a/website/locales/fr/messages.po +++ b/website/locales/fr/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Réflexions sur l'ingénierie inverse." #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Reconstruction du jeu{ph0}Essai" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "La révolution industrielle dans l'ingénierie inverse" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -53,19 +53,20 @@ msgstr "" "TH04 m'ont appris sur les agents, la vérification et les connaissances qui " "se combinent dans les projets." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Lire l'article" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Par N0zoM1z0 · 10 octobre 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -101,6 +102,8 @@ msgstr "" " avec REA et les agents d'IA." #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -137,6 +140,7 @@ msgstr "" "agent de codage." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -157,6 +161,7 @@ msgid "REA home" msgstr "Accueil REA" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -177,6 +182,7 @@ msgid "Main navigation" msgstr "Navigation principale" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -197,6 +203,7 @@ msgid "REA on GitHub" msgstr "REA sur GitHub" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -217,6 +224,7 @@ msgid "Join REA on Discord" msgstr "Rejoindre REA sur Discord" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -236,7 +244,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Suivre REA sur X" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Haut de page" @@ -11775,6 +11783,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "Même espacement: 256 ÷ 16 = 16 unités d'angle = 22,5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -11795,6 +11804,7 @@ msgid "Showcases" msgstr "Études de cas" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -11815,6 +11825,7 @@ msgid "Blog" msgstr "Blog" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -11835,6 +11846,7 @@ msgid "Get started" msgstr "Démarrer" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -11854,7 +11866,8 @@ msgctxt "shared" msgid "Guides" msgstr "Guides" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11874,7 +11887,8 @@ msgctxt "shared" msgid "Source" msgstr "Code source" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11894,7 +11908,8 @@ msgctxt "shared" msgid "MIT license" msgstr "Licence MIT" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11914,7 +11929,8 @@ msgctxt "shared" msgid "Join the community" msgstr "Rejoindre la communauté" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12908,8 +12924,1378 @@ msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "Par N0zoM1z0 · 13 octobre 2026 · Brouillon à relire" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Par N0zoM1z0 · 13 octobre 2026" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "Aller au contenu" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "{ph0}Accueil{ph1} / {ph2}Blog{ph3} / Modèles et rétro-ingénierie" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "Essai" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "L'ingénierie inverse nécessite-t-elle un modèle plus grand ?" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Par N0zoM1z0 · 13 octobre 2026" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "À retenir" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}La capacité du modèle affecte les hypothèses qu'un agent d'IA peut " +"proposer.{ph1} Les sessions ultérieures ont résolu des problèmes que les " +"travaux antérieurs n'avaient pas résolus." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}Un compilateur familier propose des modèles concrets à apprendre pour " +"un modèle spécialisé.{ph1} Certaines correspondances exactes de code machine" +" dépendent de modèles de code source spécifiques et réutilisables." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}Incluez les connaissances du référentiel dans une comparaison " +"équitable.{ph1} Un modèle plus petit avec une bonne récupération des " +"connaissances peut déjà contenir suffisamment d'informations pour " +"progresser." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}Entraînez le modèle à choisir la prochaine expérience utile et testez " +"son vérificateur.{ph1} Les retours ne sont utiles que dans la mesure où le " +"vérificateur vérifie les bonnes choses." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}Évaluez les résultats de reconstruction vérifiés en fonction du temps " +"et du coût.{ph1} C'est ce que je souhaite mesurer pour le flux de travail " +"complet de l'agent IA." + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "Dans cet article" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}Ce que les modèles plus performants ont changé{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}Pourquoi se spécialiser ?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}Où cette expérience doit-elle être stockée ?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}Apprendre la prochaine expérience utile{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}Ingénierie inverse et apprentissage par renforcement{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}Lorsque le vérificateur est erroné{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}Une comparaison qui vaut la peine d'être menée{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}Quel type d’agent IA cela suggère-t-il ?{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "Ce que les modèles plus performants ont changé" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"Depuis août 2026, j'explore combien de rétro-ingénierie un agent peut " +"effectuer avec accès à un programme et un moyen de vérifier son travail. La " +"majeure partie de mon expérience vient de la reconstruction de jeux, en " +"particulier Touhou. Au fur et à mesure que j'ai changé de modèle, les " +"référentiels ont conservé une trace de ce que chaque session a réussi à " +"récupérer." + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"Une grande partie du travail vise une reconstruction exacte : nous compilons" +" le code maintenu avec une chaîne d'outils de l'époque dont les versions " +"sont fixées, puis vérifions si elle reproduit la fonction cible. Quand un " +"modèle ultérieur débloque le travail, les changements du code permettent de " +"comprendre ce qui a rendu ce progrès possible." + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "" +"Les variables temporaires nécessitaient des durées de vie qui se " +"chevauchaient" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"Une grande partie de la logique d'une fonction d'attaque TH09 avait déjà été" +" reconstruite, mais le compilateur stockait toujours les résultats " +"temporaires différemment de l'original. Le problème restant était de savoir " +"combien de temps ces résultats devaient rester en vie et quand leur stockage" +" pouvait être réutilisé." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"Un {ph0}GPT-6-Sol enregistrement de session{ph1} a signalé cette incohérence" +" après avoir testé plusieurs arrangements de code source ordinaires. Lorsque" +" {ph2}GPT-6.1-Sol a réexaminé le problème{ph3}, il a proposé une hypothèse " +"utile sur la durée de vie de ces variables temporaires." + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"La fonction produisait des résultats temporaires et les copiait dans des " +"positions d'animation. En C++, un résultat temporaire reste actif jusqu'à la" +" fin de l'expression qui l'utilise. Le regroupement des opérations réelles " +"en une seule expression permettait de conserver ces résultats ensemble, de " +"sorte que le compilateur ne pouvait plus réutiliser le même stockage pour " +"chacun d'entre eux." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"Un résultat partageait encore son espace de stockage avec une valeur " +"utilisée plus tard pour les collisions. Donner à cette valeur une durée de " +"vie distincte a éliminé le dernier chevauchement. Après plusieurs " +"expériences ciblées, {ph0}une recompilation propre a confirmé que la " +"fonction entière correspondait à l'original{ph1}." + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"Je suis intéressé par la façon dont la découverte et les tests se sont " +"soutenus mutuellement. L’hypothèse sur la durée de vie des objets " +"temporaires a donné une nouvelle direction à l’agent IA. Les retours du " +"compilateur l'ont aidé à affiner cette hypothèse en un arrangement de code " +"source correspondant." + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "Un graphique de flux de contrôle différent" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"Une deuxième fonction TH09 montre pourquoi cette nouvelle direction peut " +"être importante. Ici, l'obstacle était la façon dont plusieurs branches " +"partageaient leur travail final. Les travaux antérieurs de " +"{ph0}GPT-6.1-Sol{ph1} ont permis de faire correspondre certaines parties de " +"la fonction, mais les branches se sont toujours jointes différemment de " +"l'original." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra a modifié le graphique de flux de contrôle " +"environnant{ph1}. Les chemins principaux menaient désormais à une mise à " +"jour finale et à un retour partagés. Dans cette structure source, le " +"compilateur combinait les branches de la même manière que l'original. La " +"fonction complète correspondait et une {ph2}vérification de reconstruction " +"distincte l'a confirmée{ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}Changer la représentation a ouvert un espace de recherche plus " +"productif.{ph1} Les deux cas me donnent une raison de prendre au sérieux la " +"capacité générale du modèle : un agent peut continuer à essayer des " +"variantes locales pendant longtemps avant de trouver une meilleure façon " +"d'exprimer le travail environnant." + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"Les sessions ultérieures ont aussi hérité du code précédent et des échecs " +"documentés. Ce travail accumulé faisait partie de leur avantage. " +"L'historique permet d'examiner les découvertes et leurs vérifications ; une " +"comparaison contrôlée aiderait à séparer l'effet du modèle de celui de " +"l'expérience héritée." + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"C'est ce qui rend la question intéressante pour moi. Certains progrès sont " +"venus de la recherche d'une nouvelle représentation. Une grande partie du " +"travail restant consistait à tester différentes dispositions de code source " +"par rapport à un compilateur familier. Dans quelle mesure cela nécessite-t-" +"il une grande capacité de raisonnement ? Combien un modèle pourrait-il " +"apprendre en travaillant de manière répétée avec ce compilateur et ce code ?" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "" +"Pourquoi un modèle d’IA spécialisé pourrait-il fonctionner correctement ?" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou nous donne une raison concrète d'attendre que cette expérience nous " +"aide. Une grande partie du code du jeu provient du même auteur, ZUN, les " +"habitudes récurrentes constituent donc une hypothèse de départ raisonnable. " +"Les référentiels confirment cela avec des exemples utiles : les boucles " +"associées et les arrangements d'assistance ont fourni des hypothèses à " +"travers les jeux." + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"Ces modèles de source deviennent plus spécifiques lorsque nous incluons le " +"compilateur. Sous une version et un ensemble d'options particuliers, de " +"petites modifications dans la source peuvent produire des différences " +"reproductibles dans le code machine. Certaines de ces différences sont " +"étonnamment obscures." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"Dans TH095, par exemple, {ph0}même le nom d'une variable locale peut " +"affecter l'endroit où VC7.1 la stocke{ph1}. Le déplacement de sa déclaration" +" n'a pas résolu la configuration du stockage. Cependant, une fonction " +"d'arrière-plan antérieure avait révélé un modèle de dénomination qui " +"affectait le choix du compilateur. Cela suggère une hypothèse plus " +"prometteuse pour {ph2}reconstruire {ph3}{ph4}." + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"La réutilisation de ce modèle pour la variable de taille de fichier a " +"produit la mise en page requise sans ajouter de stockage. La fonction " +"complète correspondait alors." + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"Il s’agit d’un détail très spécifique qu’un modèle d’IA à usage général peut" +" ne pas connaître lorsqu’il rencontre le problème pour la première fois. Un " +"modèle spécialisé pour ce compilateur pourrait détecter de nombreux cas de " +"ce type et découvrir quelles variantes du code source méritent d'être " +"testées en premier." + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"L’opportunité est particulièrement évidente dans les problèmes d’appariement" +" final dont nous comprenons déjà le comportement. Nous nous demandons alors " +"quel arrangement de code source ordinaire ce compilateur transforme en " +"instructions observées. La spécialisation pourrait aider le modèle à " +"proposer ces formes plus efficacement, tandis que la cible détermine " +"toujours quelles hypothèses survivent." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"Les modèles utiles doivent conserver leur contexte. TH08 utilise une version" +" Visual C++ antérieure à TH095 et TH09. TH04 utilise un compilateur " +"différent de l'ère DOS. Sa {ph0}reconstruction du code du joueur{ph1} " +"nécessitait différentes hypothèses sur les modèles de code source et sur la " +"manière dont le code compilé était lié à un exécutable. Avoir le même auteur" +" nous donne une attente initiale, mais chaque cible nous dit si elle " +"s'applique. Un modèle spécialisé devrait distinguer ces contextes." + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "Où cette expérience doit-elle être stockée ?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"Avant de décider d'enseigner ces modèles via la formation de modèles, " +"j'examinerais les connaissances que nous conservons déjà dans le " +"référentiel. Dans l'{ph0}essai précédent{ph1}, j'ai écrit qu'il deviendrait " +"la mémoire de travail du projet. Cette mémoire comprend les leçons du " +"compilateur, ainsi que la source et les expériences ratées qui les ont " +"rendues utiles." + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"En récupérant cet enregistrement, un agent peut partir d’un résultat connu " +"et inspecter comment nous l’avons atteint. Cela peut également éviter des " +"alternatives qui ont déjà échoué dans le même contexte. Le point de départ " +"s'améliore sans modifier les paramètres entraînés du modèle." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"Lors de la reconstruction de la fonction de TH09 pour créer des ennemis, une" +" {ph0}session GPT-6-Luna-max{ph1} a pris une hypothèse à partir du code " +"source TH08 associé. Ce code précédent suggérait comment organiser la boucle" +" et gérer les échecs à l'intérieur. Le propre témoignage de TH09 a fourni " +"des détails spécifiques au jeu. Les reconstructions à partir des sources ont" +" confirmé que la fonction complète correspondait." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}\"Le modèle ne connaissait pas cette bizarrerie du compilateur\" et " +"\"le modèle n'a pas pu l'utiliser\" sont des problèmes différents.{ph1} La " +"récupération peut résoudre le premier, mais transformer ce fait en un " +"candidat utile demande encore du travail. Et si le code source candidat " +"échoue, le modèle doit décider de la signification du résultat pour sa " +"prochaine tentative. Connaître le modèle pertinent n’est que le début de ce " +"processus." + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"Pour cette raison, j’inclurais un modèle d’IA à usage général plus petit " +"avec une bonne récupération dans toute comparaison. Lui donner la même " +"bibliothèque d'expériences nous aiderait à identifier ce que la formation " +"ajoute : une meilleure génération de candidats, de meilleures décisions " +"après feedback, ou les deux." + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "Apprendre la prochaine expérience utile" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"Ces décisions après retour d’information sont ce que j’aimerais le plus " +"étudier. En reconstruction agentique, le modèle propose une explication et " +"la teste par rapport au programme. Même lorsque le code source candidat " +"échoue, le résultat peut réduire les possibilités restantes et suggérer une " +"prochaine expérience plus utile." + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"Prenons à nouveau l'exemple des résultats temporaires. Sa source finale " +"montre l'arrangement qui a fonctionné, mais l'itinéraire vers cette source " +"contient plus d'informations. Chacune des expériences suivantes a permis " +"d’identifier ce qui devait encore changer." + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "Expérimenter" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "Ce que son résultat nous a appris" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "Essayez une autre façon de copier les résultats" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "Le code de copie change, mais la fonction diffère toujours." + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "Utiliser directement les valeurs renvoyées" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "" +"La copie suit le modèle d'origine, mais le stockage temporaire diffère " +"toujours." + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "Gardez toutes les variables temporaires valides en même temps" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "" +"Ils bénéficient d'un stockage séparé, mais l'un d'entre eux chevauche " +"toujours une valeur utilisée ultérieurement pour les contrôles de collision." + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "Gardez la valeur de contrôle de collision séparée" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "" +"Une recompilation propre à partir de la source confirme la correspondance " +"complète." + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "" +"Expériences sélectionnées de la reconstruction de la fonction d'attaque ci-" +"dessus, regroupées selon ce qu'elles nous ont appris." + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"Un dossier de formation qui conserve cette séquence peut enseigner plus que " +"la seule source finale. Il relie chaque changement proposé à un résultat " +"observé et à la décision suivante. Conserver des variantes inefficaces est " +"également important : elles indiquent à un agent ultérieur quelles " +"hypothèses ont déjà été testées dans ce contexte." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}Je voudrais que le modèle apprenne à utiliser les retours.{ph1} Un " +"résultat neutre est une raison pour reconsidérer la variation ; une " +"inadéquation modifiée peut indiquer une autre cause. Lorsque les " +"modifications locales cessent d’être informatives, l’agent doit être en " +"mesure de revoir la représentation source ou d’inspecter à nouveau les " +"preuves." + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"Apprendre à faire ces choix nécessite de l’autonomie. Parfois, l'étape " +"suivante utile consiste à inspecter un autre appelant plutôt que de compiler" +" une autre modification. Une séquence rigide de réécritures requises " +"supprimerait ce choix, ainsi qu'une partie du comportement que nous " +"souhaitons que le modèle apprenne." + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "" +"Pourquoi l'ingénierie inverse est intéressante pour l'apprentissage par " +"renforcement" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"La reconstruction limitée nous offre un environnement pratique pour étudier " +"ces choix. Une fois les versions de la cible et du compilateur fixées, de " +"nombreuses expériences sources produisent des retours rapides et " +"reproductibles. L'agent peut compiler un candidat, inspecter l'inadéquation " +"et utiliser cette observation pour choisir la prochaine étape." + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"L'apprentissage par renforcement pourrait entraîner ce processus de prise de" +" décision grâce aux retours qu'il reçoit. Un candidat accepté par un " +"contrôle défini nous donne un résultat concret, tandis que la séquence des " +"tentatives nous permet d'observer combien de travail il a fallu pour y " +"parvenir." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"Il existe déjà des recherches soutenant cette direction. " +"{ph0}LLM4Decompile{ph1} étudie les modèles spécialisés pour la " +"décompilation. {ph2}AutoDecompiler{ph3} entraîne le raffinement multi-tours " +"à l'aide des retours du compilateur et de l'exécution. Leurs études portent " +"sur la restauration du comportement. Pour ce travail, je les utiliserais " +"comme point de départ pour la tâche connexe consistant à trouver du code " +"source qui correspond à la sortie d'un compilateur historique." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"Cependant, pour atteindre une tâche aussi bien définie, nous devrons peut-" +"être d’abord établir quelles ressources possède une fonction ou concevoir " +"une nouvelle mesure du temps d’exécution. Dans TH08, par exemple, une " +"{ph0}différence de collision a conduit à la manière dont le programme " +"d'origine gérait les calculs à virgule flottante{ph1}. Avant que l’agent " +"puisse tester une correction utile, il devait déterminer quoi mesurer." + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"Cela suggère un autre rôle pour un modèle largement performant : contribuer " +"à rendre le problème vérifiable en premier lieu. Une fois que cette " +"vérification existe, nous pouvons étudier jusqu’où une politique de " +"recherche apprise peut aller avec elle." + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "Lorsque le vérificateur est erroné" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"Dès qu'une vérification sert à l'entraînement, sa couverture fait partie de " +"l'environnement d'apprentissage. J'appelle oracle cette référence ou ce " +"vérificateur. Même une réponse reproductible doit être comprise à partir de " +"ce qu'elle vérifie réellement." + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 a montré tout ce qui peut se cacher en dehors de cette couverture. " +"Certaines fonctions reconstruites ont réussi leurs comparaisons exactes avec" +" de mauvaises constantes flottantes. Le comparateur a ajusté les adresses " +"déplacées avant de vérifier les instructions, mais il a vérifié les valeurs " +"derrière ces adresses uniquement lorsqu'elles avaient été explicitement " +"enregistrées pour vérification. D'autres références littérales pourraient " +"passer sans que leurs valeurs soient examinées." + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"Pour combler cette lacune, un {ph0}audit plus large{ph1} a détecté des " +"constantes incorrectes dans les fonctions qui avaient déjà réussi la " +"vérification. Dans un calcul, le code source reconstruit a ajouté {ph2}, " +"tandis que l'original a ajouté {ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}Un modèle récompensé par l'ancien contrôle de vérification pourrait " +"apprendre à produire une reconstruction incorrecte.{ph1} Nous n'avons pas " +"entraîné de modèle RL à l'aide de ce vérificateur. L’échec que nous avons " +"constaté concernait le contrôle lui-même. Mais si nous l’utilisions comme " +"récompense, le même angle mort deviendrait partie intégrante de " +"l’environnement d’apprentissage." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"Un test fonctionnel peut avoir un angle mort similaire. {ph0}Des recherches " +"utilisant le fuzzing différentiel sur les décompilateurs LLM{ph1} ont révélé" +" des différences de comportement en dehors des tests fournis. Il faut donc " +"comprendre une épreuve de réussite en fonction des cas qu'elle exerce. De " +"même, une correspondance d’instructions doit être comprise en termes " +"d’octets et de données qu’elle vérifie." + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"Avant d’utiliser l’un ou l’autre type de contrôle de vérification comme " +"récompense, je le testerais avec des résultats délibérément incorrects. Je " +"garderais également l'acceptation finale séparée du code source que l'agent " +"IA peut modifier. Le modèle doit réussir la vérification en reconstruisant " +"correctement le programme. Sinon, la simple copie d'un tableau d'octets à " +"partir de la cible pourrait tromper un système de notation naïf sans " +"récupérer le code source utile." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}Améliorer le modèle et améliorer son vérificateur relèvent du même " +"problème de recherche.{ph1} Plus le modèle sait optimiser le retour que nous" +" lui donnons, plus il devient important de comprendre ce retour." + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "Une comparaison qui vaut la peine d'être menée" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"Une fois ces contrôles en place, je commencerais la comparaison sur une " +"famille de tâches VC7.1 fixe. Premièrement, j’établirais les bases d’un " +"modèle d’IA à usage général solide et d’un modèle plus petit. L'ajout de " +"connaissances sur le référentiel au modèle plus petit montrerait les " +"changements apportés à la récupération. À partir de là, une spécialisation " +"supervisée et une version formée par RL nous permettraient d'examiner ce que" +" la formation ajoute." + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"Pour rendre ces comparaisons informatives, chaque modèle recevrait les mêmes" +" preuves et outils cibles. Pour les comparaisons spécifiques à la formation," +" je conserverais également le modèle de base et les connaissances " +"disponibles fixes. Des essais répétés permettraient de distinguer une " +"stratégie de recherche fiable d'une solution candidate qui réussit par " +"hasard." + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"Je séparerais alors deux conditions de départ : reconstruire une fonction à " +"partir de ses instructions, et fermer les dernières différences dans le " +"source maintenu. Un modèle d’IA spécialisé peut être excellent pour la " +"deuxième tâche, tandis qu’un modèle d’IA à usage général effectue une grande" +" partie du travail nécessaire pour y parvenir. Les mesurer séparément " +"révélerait cette division du travail." + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"Les deux tâches nécessitent des exemples de test exclus de la formation et " +"de la récupération. Les jeux Touhou associés partagent le code moteur. " +"L'exclusion d'un jeu de l'entraînement pourrait néanmoins laisser des " +"fonctions presque identiques dans l'ensemble d'entraînement. Au lieu de " +"cela, j'exclurais des groupes entiers de code source associé et garderais " +"leurs réponses vérifiées hors des données de formation et de la bibliothèque" +" de récupération." + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"Pour ces exemples, le résultat serait des correspondances complètes et " +"vérifiées de manière indépendante. Des budgets égaux pour les propositions " +"de candidats ou les tentatives de compilation aideraient à comparer la " +"qualité de la recherche. Mesurer le temps et le coût totaux, y compris la " +"compilation, montrerait si cette qualité produit un flux de travail " +"efficace. Les coûts de formation doivent être mesurés séparément." + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"Les deux mesures comptent. Une proposition bon marché peut conduire à un " +"flux de travail coûteux si elle entraîne davantage de compilations ne " +"fournissant aucune information utile. À l’inverse, un modèle plus lent peut " +"être plus économique s’il trouve plus tôt une expérience utile. La " +"comparaison doit inclure ces coûts." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"Compter les résultats acceptés aide aussi à interpréter les progrès " +"intermédiaires. Un {ph0}candidat pour la sauvegarde des replays{ph1} de TH09" +" semblait prometteur : la fonction compilée avait la taille attendue. " +"Pourtant, la comparaison complète montrait encore des différences " +"importantes. Un autre ordre des instructions peut donner l’impression que le" +" candidat se rapproche sans résoudre le problème de fond. Ces diagnostics " +"guident l’expérience suivante, mais je vérifierais toute récompense pour ce " +"rapprochement avec le contrôle complet d’acceptation." + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"Je serais intéressé par la tendance de ces résultats. Si un modèle d’IA " +"spécialisé fonctionne bien au sein d’une famille de compilateurs familière, " +"cet avantage survit-il à un changement de chaîne d’outils ? Et une fois que " +"les deux modèles d’IA ont une bonne récupération, combien la formation " +"ajoute-t-elle encore ? Les réponses nous aideraient à décider où chaque type" +" de modèle est le plus utile." + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "Quel type d’agent IA cela suggère-t-il ?" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"Si les modèles s’avèrent utiles à différentes étapes, un arrangement utile " +"pourrait être un agent large ayant accès à un modèle d’IA spécialisé. Une " +"fois qu’une tâche devient une recherche familière du compilateur, elle peut " +"demander des candidats au modèle d’IA spécialisé. Si les résultats n’ont " +"plus de sens, l’agent au sens large pourrait revenir aux preuves cibles ou " +"enquêter sur la mesure elle-même." + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"Dans cet arrangement, la spécialisation devient un outil que l’agent peut " +"choisir au fur et à mesure que l’enquête se développe. L’agent conserve la " +"liberté de changer de direction qui a rendu possible les percées " +"antérieures." + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"Cela découle naturellement du virage industriel que j’ai décrit dans l’essai" +" précédent. Être capable de mener beaucoup plus d’expériences change la " +"façon dont nous organisons le travail. Cela nous donne également une raison " +"de nous demander quelles décisions nécessitent un jugement large et " +"lesquelles s'améliorent grâce à une expérience répétée avec un métier " +"particulier." + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"Pour l'instant, le modèle général le plus puissant a gagné sa place dans mon" +" travail en résolvant des problèmes que j'avais laissés en suspens. Ces " +"réussites me donnent envie d'étudier le processus : quelle part dépend de " +"ses capacités générales, et quelle part pourrait être confiée à un modèle " +"formé pour ce type de reconstruction ?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}Les référentiels nous permettent de tester la question.{ph1} J'aimerais" +" utiliser cet enregistrement pour établir la comparaison et voir ce que nous" +" apprenons." + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "Attribution de preuves et de modèles" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"Les exemples décrivent des comparaisons historiques au niveau des fonctions " +"enregistrées dans les validations liées. Les vérifications d'exécution du " +"jeu complet sont des étapes distinctes. Les enregistrements sources et " +"correspondants ont été examinés le 13 octobre 2026." + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"L'étiquette TH09 {ph0} signifie GPT-6-Astra. Le modèle associé à {ph1} a " +"changé au fil du temps, de sorte que cette étiquette à elle seule ne peut " +"pas identifier le modèle utilisé dans une exécution spécifique. Les noms de " +"modèles explicites dans les autres commits cités fournissent l'attribution " +"enregistrée. Ces historiques n'incluent pas de mesures cohérentes du coût ou" +" de l'effort pour chaque exécution." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 durées de vie des objets temporaires :{ph1} la {ph2}examen du " +"contexte précédent{ph3} et la {ph4}reconstruction complète vérifiée par une " +"recompilation propre à partir de la source{ph5}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}Callback Front de TH09 :{ph1} {ph2}régions de diagnostic non " +"concordantes{ph3}, {ph4}nouveau graphe de contrôle d'Astra{ph5} et " +"{ph6}vérification de recompilation acceptée{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}Connaissance du compilateur :{ph1} {ph2}TH095 réutilisation du " +"comportement du compartiment d'identifiant{ph3}, {ph4}TH04 code source du " +"joueur et ordre de l'éditeur de liens{ph5} et {ph6}Luna reconstruction " +"complète de la fonction de création d'ennemis de TH09{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}Contrôles et diagnostics :{ph1} {ph2}TH08 audit de constantes à virgule" +" flottante{ph3}, {ph4}TH08 investigation numérique{ph5} et {ph6}TH09 " +"reconstruction permettant de sauvegarder les relectures{ph7}." + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"Les comparaisons proposées de modèles spécialisés et de formation RL sont " +"des expériences futures. Les histoires de projets les motivent. Aucun modèle" +" spécialisé formé n'a été évalué pour cet essai." + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "L'ingénierie inverse nécessite-t-elle un modèle plus grand ? | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"Ce que la reconstruction Touhou m'a appris sur les modèles généraux, les " +"modèles spécialisés et l'apprentissage pour proposer de meilleures " +"hypothèses à partir des retours du compilateur." + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "Sur cette page" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "Modèles et essai d'ingénierie inverse" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "Expériences de reconstruction sélectionnées" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "Haut de page" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "Par N0zoM1z0 · 13 octobre 2026 · Brouillon à relire" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/id/messages.po b/website/locales/id/messages.po index 5ad209867..e19aa9f49 100644 --- a/website/locales/id/messages.po +++ b/website/locales/id/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Pemikiran tentang rekayasa terbalik." #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Rekonstruksi permainan{ph0}Esai" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "Revolusi industri dalam rekayasa terbalik" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -53,19 +53,20 @@ msgstr "" " tentang agen, verifikasi, dan pengetahuan yang digabungkan di seluruh " "proyek." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Baca artikelnya" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Oleh N0zoM1z0 · 10 Oktober 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -101,6 +102,8 @@ msgstr "" " REA dan AI." #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -137,6 +140,7 @@ msgstr "" "pengkodean Anda." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -157,6 +161,7 @@ msgid "REA home" msgstr "Beranda REA" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -177,6 +182,7 @@ msgid "Main navigation" msgstr "Navigasi utama" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -197,6 +203,7 @@ msgid "REA on GitHub" msgstr "REA di GitHub" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -217,6 +224,7 @@ msgid "Join REA on Discord" msgstr "Gabung REA di Discord" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -236,7 +244,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Ikuti REA di X" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Kembali ke atas" @@ -11654,6 +11662,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "Jarak yang sama: 256 ÷ 16 = 16 satuan sudut = 22,5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -11674,6 +11683,7 @@ msgid "Showcases" msgstr "Studi kasus" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -11694,6 +11704,7 @@ msgid "Blog" msgstr "Blog" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -11714,6 +11725,7 @@ msgid "Get started" msgstr "Mulai" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -11733,7 +11745,8 @@ msgctxt "shared" msgid "Guides" msgstr "Panduan" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11753,7 +11766,8 @@ msgctxt "shared" msgid "Source" msgstr "Kode sumber" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11773,7 +11787,8 @@ msgctxt "shared" msgid "MIT license" msgstr "Lisensi MIT" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11793,7 +11808,8 @@ msgctxt "shared" msgid "Join the community" msgstr "Gabung komunitas" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12781,8 +12797,1353 @@ msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "Oleh N0zoM1z0 · 13 Oktober 2026 · Draf untuk ditinjau" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Oleh N0zoM1z0 · 13 Oktober 2026" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "Lewati ke konten" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "{ph0}Beranda{ph1} / {ph2}Blog{ph3} / Model dan rekayasa balik" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "Esai" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Apakah rekayasa balik memerlukan model yang lebih besar?" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Oleh N0zoM1z0 · 13 Oktober 2026" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "Poin utama" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}Kemampuan model memengaruhi hipotesis mana yang dapat diajukan oleh " +"agen AI.{ph1} Sesi berikutnya memecahkan masalah yang belum terselesaikan " +"oleh pekerjaan sebelumnya." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}Kompiler yang familiar menawarkan pola konkret untuk dipelajari model " +"khusus.{ph1} Beberapa kecocokan kode mesin yang tepat bergantung pada pola " +"kode sumber spesifik yang dapat digunakan kembali." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}Sertakan pengetahuan repositori dalam perbandingan yang adil.{ph1} " +"Model yang lebih kecil dengan pengambilan pengetahuan yang baik mungkin " +"sudah memiliki informasi yang cukup untuk membuat kemajuan." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}Latih model untuk memilih eksperimen berguna berikutnya, dan uji " +"pemeriksanya.{ph1} Masukan hanya berguna jika pemeriksa memverifikasi hal " +"yang benar." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}Evaluasi hasil rekonstruksi terverifikasi beserta waktu dan biaya.{ph1}" +" Itulah yang ingin saya ukur untuk alur kerja agen AI yang lengkap." + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "Dalam artikel ini" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}Apa yang berubah dengan model yang lebih kuat?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}Mengapa mengambil spesialisasi?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}Di mana pengalaman ini harus disimpan?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}Pelajari eksperimen bermanfaat berikutnya{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}Pembelajaran rekayasa balik dan penguatan{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}Ketika pemeriksa salah{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}Sebuah perbandingan yang layak untuk dijalankan{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}Agen AI macam apa yang disarankan di sini?{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "Apa yang berubah dengan model yang lebih kuat?" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"Sejak Agustus 2026, saya telah mengeksplorasi seberapa banyak rekayasa balik" +" yang dapat dilakukan agen dengan akses ke suatu program dan cara untuk " +"memeriksa kerjanya. Sebagian besar pengalaman saya berasal dari " +"merekonstruksi game, terutama Touhou. Karena saya telah mengubah model, " +"repositori menyimpan catatan tentang apa yang berhasil dipulihkan oleh " +"setiap sesi." + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"Sebagian besar pekerjaan tersebut bertujuan untuk rekonstruksi yang tepat: " +"kami mengkompilasi sumber yang dikelola di bawah rantai alat historis dengan" +" versi tetap dan memeriksa apakah sumber tersebut mereproduksi fungsi " +"target. Ketika model selanjutnya berhasil ketika pekerjaan sebelumnya " +"terhenti, perubahan sumber menunjukkan apa yang memungkinkan kemajuan " +"tersebut." + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "Variabel sementara membutuhkan masa hidup yang tumpang tindih" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"Sebagian besar logika fungsi serangan TH09 telah direkonstruksi, namun " +"kompiler masih menyimpan hasil sementara yang berbeda dari aslinya. Masalah " +"yang tersisa adalah berapa lama hasil tersebut perlu bertahan dan kapan " +"penyimpanannya dapat digunakan kembali." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"Catatan sesi {ph0}GPT-6-Sol{ph1} melaporkan ketidakcocokan ini setelah " +"menguji beberapa pengaturan kode sumber biasa. Ketika {ph2}GPT-6.1-Sol " +"memeriksa masalahnya lagi{ph3}, ia mengajukan hipotesis yang berguna tentang" +" masa hidup variabel-variabel sementara ini." + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"Fungsi tersebut memberikan hasil sementara dan menyalinnya ke posisi " +"animasi. Di C++, hasil sementara tetap aktif hingga ekspresi yang " +"menggunakannya selesai. Mengelompokkan operasi nyata ke dalam satu ekspresi " +"membuat hasil tersebut tetap hidup, sehingga kompiler tidak dapat lagi " +"menggunakan penyimpanan yang sama untuk semuanya." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"Salah satu hasil masih berbagi penyimpanan dengan nilai yang nantinya " +"digunakan untuk pemeriksaan tabrakan. Memberikan nilai tersebut seumur " +"hidupnya akan menyelesaikan sisa tumpang tindih. Setelah beberapa eksperimen" +" yang ditargetkan, {ph0}kompilasi ulang yang bersih dari sumber " +"mengonfirmasi bahwa seluruh fungsi cocok{ph1}." + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"Saya tertarik pada bagaimana penemuan dan pengujian saling mendukung. " +"Hipotesis tentang masa hidup objek yang bersifat sementara memberikan arah " +"baru bagi agen AI. Umpan balik dari kompiler membantunya menyempurnakan " +"hipotesis tersebut menjadi susunan kode sumber yang cocok." + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "Grafik aliran kontrol yang berbeda" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"Fungsi TH09 kedua menunjukkan mengapa arah baru itu penting. Di sini " +"kendalanya adalah bagaimana beberapa cabang membagikan karya akhirnya. " +"{ph0}Pekerjaan sebelumnya GPT-6.1-Sol{ph1} memiliki bagian-bagian fungsi " +"yang cocok, namun cabang-cabangnya masih digabungkan secara berbeda dari " +"aslinya." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra mengubah grafik aliran kontrol di sekitarnya{ph1}. Jalur " +"utama sekarang mengarah ke pembaruan akhir bersama dan kembalinya. Dalam " +"struktur sumber tersebut, kompiler menggabungkan cabang-cabang dengan cara " +"yang sama seperti aslinya. Fungsi keseluruhannya cocok, dan {ph2}pemeriksaan" +" pembangunan kembali secara terpisah mengonfirmasi hal tersebut{ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}Mengubah representasi membuka ruang pencarian yang lebih " +"produktif.{ph1} Kedua kasus tersebut memberi saya alasan untuk menganggap " +"serius kemampuan model umum: agen dapat terus mencoba variasi lokal untuk " +"waktu yang lama sebelum menemukan cara yang lebih baik untuk mengekspresikan" +" pekerjaan di sekitarnya." + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"Pada saat yang sama, sesi selanjutnya dimulai dengan sumber sebelumnya dan " +"mencatat kegagalan. Akumulasi pekerjaan itu adalah bagian dari keuntungan " +"mereka. Sejarah memungkinkan kita memeriksa penemuan-penemuan dan " +"pemeriksaannya; perbandingan terkontrol akan membantu memisahkan pengaruh " +"model dari pengalaman yang diwarisinya." + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"Inilah yang membuat pertanyaan ini menarik bagi saya. Beberapa kemajuan " +"datang dari penemuan representasi baru. Sebagian besar pekerjaan yang " +"tersisa melibatkan pengujian pengaturan kode sumber yang berbeda terhadap " +"kompiler yang sudah dikenal. Seberapa banyak hal ini memerlukan kemampuan " +"penalaran yang luas? Berapa banyak yang dapat dipelajari suatu model dengan " +"bekerja berulang kali menggunakan compiler dan kode tersebut?" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "Mengapa model AI khusus dapat berfungsi dengan baik?" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou memberi kita alasan konkret untuk mengharapkan pengalaman itu " +"membantu. Sebagian besar kode permainan berasal dari penulis yang sama, ZUN," +" jadi kebiasaan berulang adalah asumsi awal yang masuk akal. Repositori " +"mendukung hal tersebut dengan contoh-contoh yang berguna: loop terkait dan " +"pengaturan pembantu telah memberikan hipotesis di seluruh game." + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"Pola sumber tersebut menjadi lebih spesifik ketika kita menyertakan " +"kompiler. Dalam versi dan rangkaian opsi tertentu, perubahan kecil pada " +"sumber dapat menghasilkan perbedaan berulang dalam kode mesin. Beberapa dari" +" perbedaan tersebut ternyata tidak jelas." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"Di TH095, misalnya, {ph0}bahkan nama variabel lokal dapat memengaruhi tempat" +" VC7.1 menyimpannya{ph1}. Memindahkan deklarasinya tidak menyelesaikan tata " +"letak penyimpanan. Namun, fungsi latar belakang sebelumnya telah " +"mengungkapkan pola penamaan yang mempengaruhi pilihan kompiler. Hal ini " +"menunjukkan hipotesis yang lebih menjanjikan untuk {ph2}merekonstruksi " +"{ph3}{ph4}." + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"Menggunakan kembali pola tersebut untuk variabel ukuran file menghasilkan " +"tata letak yang diperlukan tanpa menambahkan penyimpanan. Fungsi penuh " +"kemudian dicocokkan." + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"Ini adalah detail yang sangat spesifik yang mungkin tidak diketahui oleh " +"model AI tujuan umum saat pertama kali mengalami masalah. Model yang " +"dikhususkan untuk kompiler ini dapat melihat banyak kasus seperti itu dan " +"mempelajari variasi kode sumber mana yang layak untuk diuji terlebih dahulu." + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"Peluangnya sangat jelas dalam masalah pencocokan akhir dimana kita sudah " +"memahami perilakunya. Kami kemudian menanyakan pengaturan kode sumber biasa " +"mana yang diubah oleh kompiler ini menjadi instruksi yang diamati. " +"Spesialisasi dapat membantu model mengusulkan bentuk-bentuk tersebut dengan " +"lebih efisien, sementara target tetap menentukan hipotesis mana yang " +"bertahan." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"Pola yang berguna harus mempertahankan konteksnya. TH08 menggunakan versi " +"Visual C++ yang lebih lama dari TH095 dan TH09. TH04 menggunakan compiler " +"yang berbeda dari era DOS. {ph0}rekonstruksi kode pemain{ph1} memerlukan " +"hipotesis berbeda tentang pola kode sumber dan bagaimana kode yang " +"dikompilasi dihubungkan ke dalam file yang dapat dieksekusi. Memiliki " +"penulis yang sama memberi kita ekspektasi awal, namun setiap target memberi " +"tahu kita apakah target tersebut berlaku. Model khusus perlu membedakan " +"konteks-konteks ini." + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "Di mana pengalaman ini harus disimpan?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"Sebelum memutuskan untuk mengajarkan pola-pola tersebut melalui pelatihan " +"model, saya akan melihat pengetahuan yang sudah kita simpan di gudang. Dalam" +" {ph0}esai sebelumnya{ph1}, saya menulis tentang hal ini yang menjadi memori" +" kerja proyek. Memori tersebut mencakup pelajaran kompiler, beserta " +"sumbernya dan eksperimen gagal yang menjadikannya berguna." + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"Dengan mengambil catatan tersebut, agen dapat memulai dengan hasil yang " +"diketahui dan memeriksa bagaimana kami mencapainya. Hal ini juga dapat " +"menghindari alternatif-alternatif yang telah gagal dalam konteks yang sama. " +"Titik awal ditingkatkan tanpa mengubah parameter model yang dilatih." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"Saat merekonstruksi fungsi TH09 untuk menciptakan musuh, {ph0}sesi " +"GPT-6-Luna-max{ph1} mengambil hipotesis dari kode sumber TH08 terkait. Kode " +"sebelumnya menyarankan cara mengatur loop dan menangani kegagalan di " +"dalamnya. Bukti TH09 sendiri memberikan detail khusus game tersebut. " +"Pembangunan kembali dari sumber mengonfirmasi bahwa fungsi lengkapnya cocok." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}\"Model tidak mengetahui kekhasan kompiler\" dan \"model tidak dapat " +"menggunakannya\" adalah masalah yang berbeda.{ph1} Pengambilan mungkin " +"memecahkan masalah pertama, namun mengubah fakta tersebut menjadi kandidat " +"yang berguna masih membutuhkan usaha. Dan jika kandidat kode sumber gagal, " +"model harus memutuskan apa arti hasil tersebut untuk percobaan berikutnya. " +"Mengetahui pola yang relevan hanyalah awal dari proses itu." + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"Oleh karena itu, saya akan menyertakan model AI serba guna yang lebih kecil " +"dengan pengambilan yang baik dalam perbandingan apa pun. Memberikan " +"perpustakaan pengalaman yang sama akan membantu kami mengidentifikasi apa " +"saja yang ditambahkan oleh pelatihan: perolehan kandidat yang lebih baik, " +"keputusan yang lebih baik setelah adanya masukan, atau keduanya." + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "Pelajari eksperimen bermanfaat berikutnya" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"Keputusan-keputusan setelah mendapat masukan adalah hal yang paling ingin " +"saya pelajari. Dalam rekonstruksi agenik, model mengajukan penjelasan dan " +"mengujinya terhadap program. Bahkan ketika kode sumber kandidat gagal, " +"hasilnya dapat mempersempit kemungkinan yang tersisa dan menyarankan " +"percobaan berikutnya yang lebih berguna." + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"Perhatikan kembali contoh hasil sementara. Sumber akhirnya menunjukkan " +"pengaturan yang berhasil, namun rute ke sumber tersebut berisi lebih banyak " +"informasi. Setiap eksperimen berikut membantu mengidentifikasi apa yang " +"masih perlu diubah." + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "Eksperimen" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "Apa yang diajarkan hasilnya kepada kita" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "Coba cara lain untuk menyalin hasilnya" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "Kode copynya berubah, tapi fungsinya tetap berbeda." + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "Gunakan nilai yang dikembalikan secara langsung" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "" +"Penyalinan mengikuti pola aslinya, namun penyimpanan sementara tetap " +"berbeda." + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "Jaga agar semua variabel sementara tetap valid pada waktu yang sama" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "" +"Mereka mendapatkan penyimpanan terpisah, namun masih tumpang tindih dengan " +"nilai yang digunakan nanti untuk pemeriksaan tabrakan." + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "Pisahkan nilai pemeriksaan tabrakan" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "" +"Kompilasi ulang yang bersih dari sumber mengonfirmasi kecocokan yang " +"lengkap." + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "" +"Eksperimen yang dipilih dari rekonstruksi fungsi serangan di atas, " +"dikelompokkan berdasarkan apa yang diajarkannya kepada kita." + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"Catatan pelatihan yang menjaga urutan ini dapat mengajarkan lebih dari " +"sekedar sumber akhir saja. Ini menghubungkan setiap perubahan yang diusulkan" +" dengan hasil yang diamati dan keputusan berikutnya. Mempertahankan variasi " +"yang tidak efektif juga penting: variasi tersebut akan memberi tahu agen " +"selanjutnya hipotesis mana yang telah diuji dalam konteks tersebut." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}Saya ingin model mempelajari cara menggunakan masukan.{ph1} Hasil yang " +"netral adalah alasan untuk mempertimbangkan kembali variasi; ketidakcocokan " +"yang berubah mungkin menunjukkan penyebab lain. Ketika suntingan lokal tidak" +" lagi bersifat informatif, agen harus dapat mengunjungi kembali representasi" +" sumber atau memeriksa bukti lagi." + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"Belajar membuat pilihan tersebut membutuhkan otonomi. Terkadang langkah " +"berikutnya yang berguna adalah memeriksa pemanggil lain daripada " +"mengkompilasi hasil edit lainnya. Urutan penulisan ulang yang diperlukan " +"akan menghilangkan pilihan tersebut, bersama dengan bagian dari perilaku " +"yang kita ingin model pelajari." + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "Mengapa rekayasa balik menarik untuk pembelajaran penguatan" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"Rekonstruksi dengan lingkup yang jelas memberi lingkungan praktis untuk " +"mempelajari pilihan ini. Setelah versi target dan kompiler ditetapkan, " +"banyak eksperimen kode sumber memberi umpan balik yang cepat dan dapat " +"diulang. Agen bisa mengompilasi kandidat, memeriksa perbedaan, lalu memilih " +"langkah berikutnya." + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"Pembelajaran penguatan dapat melatih proses pengambilan keputusan melalui " +"umpan balik yang diterimanya. Kandidat yang diterima melalui pemeriksaan " +"tertentu memberi kita hasil yang konkrit, sementara rangkaian upaya " +"memungkinkan kita mengamati berapa banyak usaha yang diperlukan untuk " +"mencapainya." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"Sudah ada penelitian yang mendukung arah ini. {ph0}LLM4Decompile{ph1} " +"mempelajari model yang dikhususkan untuk dekompilasi. " +"{ph2}AutoDecompiler{ph3} melatih penyempurnaan multi-putaran menggunakan " +"compiler dan masukan eksekusi. Studi mereka fokus pada pemulihan fungsional." +" Untuk pekerjaan ini, saya akan menggunakannya sebagai titik awal untuk " +"tugas terkait menemukan sumber yang cocok dengan keluaran kompiler historis." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"Namun, untuk mencapai tugas yang terdefinisi dengan baik, pertama-tama kita " +"perlu menetapkan sumber daya yang dimiliki suatu fungsi atau merancang " +"pengukuran runtime baru. Di TH08, misalnya, {ph0}perbedaan tabrakan mengarah" +" kembali ke cara program asli menangani penghitungan floating-point{ph1}. " +"Sebelum agen dapat menguji koreksi yang berguna, agen harus menentukan apa " +"yang diukur." + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"Hal ini menunjukkan adanya peran lain dari model yang berkemampuan luas: " +"membantu membuat masalah dapat diperiksa. Setelah pemeriksaan tersebut " +"dilakukan, kami dapat menyelidiki sejauh mana kebijakan penelusuran yang " +"dipelajari dapat diterapkan." + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "Ketika pemeriksa salah" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"Saat suatu pemeriksaan dipakai untuk pelatihan, cakupannya menjadi bagian " +"dari lingkungan belajar. Saya menyebut acuan atau alat pemeriksa itu oracle." +" Bahkan jawaban yang dapat diulang harus dipahami berdasarkan apa yang " +"benar-benar diperiksanya." + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 menunjukkan seberapa banyak yang bisa disembunyikan di luar cakupan " +"itu. Beberapa fungsi yang direkonstruksi memberikan perbandingan yang tepat " +"dengan konstanta floating-point yang salah. Pembanding menyesuaikan alamat " +"yang direlokasi sebelum memeriksa instruksi, namun pembanding memeriksa " +"nilai di balik alamat tersebut hanya jika nilai tersebut telah dicatat " +"secara eksplisit untuk verifikasi. Referensi literal lainnya bisa lolos " +"tanpa diperiksa nilainya." + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"Untuk mengatasi kesenjangan ini, {ph0}audit yang lebih luas{ph1} menemukan " +"konstanta yang salah pada fungsi yang telah lolos verifikasi. Dalam satu " +"perhitungan, kode sumber yang direkonstruksi menambahkan {ph2}, sedangkan " +"kode sumber asli menambahkan {ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}Model yang diberi imbalan melalui pemeriksaan verifikasi lama dapat " +"belajar menghasilkan rekonstruksi yang salah.{ph1} Kami tidak melatih model " +"RL menggunakan pemeriksa tersebut. Kegagalan yang kami amati ada pada " +"pemeriksaan itu sendiri. Namun jika kita menggunakannya sebagai hadiah, " +"titik buta yang sama akan menjadi bagian dari lingkungan belajar." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"Tes fungsional dapat memiliki titik buta serupa. {ph0}Penelitian yang " +"menggunakan fuzzing diferensial pada dekompiler LLM{ph1} menemukan perbedaan" +" perilaku di luar pengujian yang disediakan. Oleh karena itu, kita perlu " +"memahami tes kelulusan dalam kaitannya dengan kasus-kasus yang dilakukannya." +" Demikian pula, kecocokan instruksi harus dipahami dalam kaitannya dengan " +"byte dan data yang diverifikasi." + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"Sebelum menggunakan salah satu jenis cek verifikasi sebagai hadiah, saya " +"akan mengujinya dengan hasil yang sengaja salah. Saya juga akan memisahkan " +"penerimaan akhir dari kode sumber yang dapat diedit oleh agen AI. Model " +"harus lolos verifikasi dengan merekonstruksi program dengan benar. Jika " +"tidak, hanya dengan menyalin array byte dari target dapat menipu sistem " +"penilaian yang naif tanpa memulihkan kode sumber yang berguna." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}Memperbaiki model yang belajar dan memperbaiki pemeriksanya merupakan " +"bagian dari masalah penelitian yang sama.{ph1} Semakin baik model " +"mengoptimalkan umpan balik yang kita berikan, semakin penting memahami umpan" +" balik itu sendiri." + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "Sebuah perbandingan yang layak untuk dijalankan" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"Dengan pemeriksaan tersebut, saya akan memulai perbandingan pada rangkaian " +"tugas VC7.1 yang tetap. Pertama, saya akan menetapkan dasar untuk model AI " +"tujuan umum yang kuat dan model yang lebih kecil. Menambahkan pengetahuan " +"repositori ke model yang lebih kecil akan menunjukkan perubahan pengambilan " +"apa. Dari sana, spesialisasi yang diawasi dan versi terlatih RL akan " +"memungkinkan kita memeriksa apa saja yang ditambahkan dalam pelatihan." + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"Agar perbandingan tersebut informatif, setiap model akan menerima bukti dan " +"alat target yang sama. Untuk perbandingan khusus pelatihan, saya juga akan " +"mempertahankan model dasar dan pengetahuan yang tersedia. Uji coba berulang " +"akan membantu membedakan strategi pencarian yang andal dari kandidat solusi " +"yang berhasil secara kebetulan." + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"Saya kemudian akan memisahkan dua kondisi awal: merekonstruksi suatu fungsi " +"dari instruksinya, dan menutup beberapa perbedaan terakhir dalam sumber yang" +" dipertahankan. Model AI khusus mungkin sangat baik dalam tugas kedua, " +"sementara model AI tujuan umum melakukan banyak pekerjaan yang diperlukan " +"untuk mencapainya. Mengukurnya secara terpisah akan mengungkapkan pembagian " +"kerja tersebut." + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"Kedua tugas memerlukan contoh uji yang tidak ada dalam data pelatihan maupun" +" bahan retrieval. Game Touhou yang terkait berbagi kode engine. " +"Mengecualikan satu game saja masih bisa menyisakan fungsi yang hampir " +"identik dalam data pelatihan. Saya akan menyisihkan seluruh kelompok kode " +"yang terkait, dan memastikan jawaban terverifikasinya tidak masuk ke data " +"pelatihan ataupun pustaka retrieval." + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"Untuk contoh-contoh ini, hasilnya akan berupa kecocokan yang lengkap dan " +"terverifikasi secara independen. Anggaran yang sama untuk proposal kandidat " +"atau upaya kompilasi akan membantu membandingkan kualitas pencarian. " +"Mengukur total waktu dan biaya, termasuk kompilasi, akan menunjukkan apakah " +"kualitas tersebut menghasilkan alur kerja yang efisien. Biaya pelatihan " +"harus diukur secara terpisah." + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"Kedua pengukuran itu penting. Proposal yang murah dapat menyebabkan alur " +"kerja yang mahal jika menyebabkan lebih banyak kompilasi yang tidak " +"memberikan informasi berguna. Sebaliknya, model yang lebih lambat mungkin " +"lebih ekonomis jika model tersebut menemukan eksperimen yang berguna lebih " +"cepat. Perbandingannya harus mencakup biaya-biaya ini." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"Menghitung hasil yang diterima juga menjaga kemajuan sementara tetap dalam " +"perspektif. Salah satu TH09 {ph0}kandidat penyimpanan ulang{ph1} tampak " +"menjanjikan karena fungsi yang dikompilasi memiliki ukuran yang diharapkan, " +"namun perbandingan penuh masih menemukan perbedaan besar. Perubahan dalam " +"cara compiler menyusun instruksi juga dapat membuat kandidat tampak lebih " +"dekat dengan target tanpa menyelesaikan masalah mendasarnya. Diagnostik " +"semacam itu dapat memandu eksperimen berikutnya, namun saya akan memvalidasi" +" imbalan apa pun karena \"mendekati\" dengan pemeriksaan penerimaan penuh." + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"Saya tertarik dengan pola hasil ini. Jika model AI khusus dapat berfungsi " +"dengan baik dalam rangkaian kompiler yang sudah dikenal, apakah keunggulan " +"tersebut dapat bertahan jika terjadi perubahan rantai alat? Dan ketika kedua" +" model AI memiliki pengambilan yang baik, berapa banyak lagi yang bisa " +"ditambahkan oleh pelatihan? Jawabannya akan membantu kita memutuskan model " +"mana yang paling berguna." + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "Agen AI macam apa yang disarankan di sini?" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"Jika model-model tersebut terbukti membantu pada tahap-tahap yang berbeda, " +"pengaturan yang berguna mungkin adalah agen luas yang memiliki akses ke " +"model AI khusus. Setelah tugas menjadi pencarian kompiler yang familier, " +"tugas tersebut dapat menanyakan kandidat pada model AI khusus. Jika hasilnya" +" tidak masuk akal, agen yang lebih luas dapat kembali ke bukti target atau " +"menyelidiki pengukuran itu sendiri." + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"Dalam pengaturan tersebut, spesialisasi menjadi alat yang dapat dipilih agen" +" seiring berkembangnya penyelidikan. Agen mempunyai kebebasan untuk mengubah" +" arah yang memungkinkan terobosan sebelumnya." + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"Hal ini tentu saja mengikuti pergeseran industri yang saya jelaskan di esai " +"sebelumnya. Mampu menjalankan lebih banyak eksperimen mengubah cara kami " +"mengatur pekerjaan. Hal ini juga memberi kita alasan untuk mempertanyakan " +"keputusan mana yang memerlukan pertimbangan luas dan mana yang ditingkatkan " +"melalui pengalaman berulang-ulang dengan bidang tertentu." + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"Untuk saat ini, model umum terkuat mendapat tempat dalam alur kerja saya " +"karena menyelesaikan masalah yang sebelumnya belum terpecahkan. Keberhasilan" +" itu membuat saya ingin mempelajari prosesnya: seberapa banyak bergantung " +"pada keluasan kemampuan model, dan seberapa banyak bisa dikerjakan oleh " +"model yang dilatih khusus untuk jenis rekonstruksi ini?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}Repositori memberi kita cara untuk menguji pertanyaan tersebut.{ph1} " +"Saya ingin menggunakan data tersebut untuk membuat perbandingan dan melihat " +"apa yang kita pelajari." + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "Bukti dan atribusi model" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"Contoh-contoh ini menjelaskan perbandingan tingkat fungsi historis yang " +"dicatat dalam penerapan tertaut. Pemeriksaan waktu proses keseluruhan " +"permainan adalah pencapaian yang terpisah. Sumber dan catatan pencocokan " +"ditinjau pada 13 Oktober 2026." + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"Label TH09 {ph0} berarti GPT-6-Astra. Model yang terkait dengan {ph1} " +"berubah seiring waktu, sehingga label saja tidak dapat mengidentifikasi " +"model yang digunakan dalam proses tertentu. Nama model eksplisit dalam " +"commit lain yang dikutip memberikan atribusi yang tercatat. Riwayat ini " +"tidak mencakup pengukuran biaya atau upaya yang konsisten untuk setiap " +"proses." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 masa hidup objek sementara:{ph1} {ph2}tinjauan konteks " +"sebelumnya{ph3} dan {ph4}rekonstruksi lengkap yang diverifikasi melalui " +"kompilasi ulang yang bersih dari sumber{ph5}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}TH09 Front callback:{ph1} {ph2}wilayah diagnostik yang tidak " +"cocok{ph3}, {ph4}Astra grafik aliran kontrol baru{ph5} dan {ph6}pemeriksaan " +"kompilasi ulang yang diterima{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}Pengetahuan compiler:{ph1} {ph2}TH095 penggunaan kembali perilaku " +"identifier-bucket{ph3}, {ph4}TH04 kode sumber pemain dan urutan linker{ph5} " +"dan {ph6}Luna rekonstruksi lengkap fungsi penciptaan musuh TH09{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}Pemeriksaan dan diagnostik:{ph1} {ph2}TH08 audit konstan floating-" +"point{ph3}, {ph4}TH08 investigasi numerik{ph5} dan {ph6}TH09 rekonstruksi " +"penyimpanan ulangan{ph7}." + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"Perbandingan yang diusulkan antara model khusus dan pelatihan RL adalah " +"eksperimen di masa depan. Sejarah proyek memotivasi mereka. Tidak ada model " +"khusus terlatih yang dievaluasi untuk esai ini." + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "Apakah rekayasa balik memerlukan model yang lebih besar? | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"Apa yang saya pelajari dari rekonstruksi Touhou tentang model umum, model " +"khusus, dan cara belajar mengajukan hipotesis yang lebih baik melalui umpan " +"balik kompiler." + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "Di halaman ini" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "Model dan esai rekayasa balik" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "Eksperimen rekonstruksi yang dipilih" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "Kembali ke atas" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "Oleh N0zoM1z0 · 13 Oktober 2026 · Draf untuk ditinjau" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/it/messages.po b/website/locales/it/messages.po index 068419f41..034644033 100644 --- a/website/locales/it/messages.po +++ b/website/locales/it/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Riflessioni sul reverse engineering." #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Gioco ricostruzione{ph0}Saggio" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "La rivoluzione industriale nel reverse engineering" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,20 @@ msgstr "" "Che manuale RE e ricostruire TH08, TH095 e TH04 mi ha insegnato circa gli " "agenti, la verifica e la conoscenza che composti attraverso i progetti." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Leggi l'articolo" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Di N0zoM1z0 * 10 ottobre 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -100,6 +101,8 @@ msgstr "" "e agenti AI." #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -136,6 +139,7 @@ msgstr "" "di codifica." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -156,6 +160,7 @@ msgid "REA home" msgstr "Home di REA" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -176,6 +181,7 @@ msgid "Main navigation" msgstr "Navigazione principale" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -196,6 +202,7 @@ msgid "REA on GitHub" msgstr "REA su GitHub" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -216,6 +223,7 @@ msgid "Join REA on Discord" msgstr "Unisciti a REA su Discord" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -235,7 +243,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Segui REA su X" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Torna in alto" @@ -11712,6 +11720,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "Stessa spaziatura: 256 ÷ 16 = 16 unità angolari = 22,5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -11732,6 +11741,7 @@ msgid "Showcases" msgstr "Casi pratici" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -11752,6 +11762,7 @@ msgid "Blog" msgstr "Blog" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -11772,6 +11783,7 @@ msgid "Get started" msgstr "Inizia" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -11791,7 +11803,8 @@ msgctxt "shared" msgid "Guides" msgstr "Guide" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11811,7 +11824,8 @@ msgctxt "shared" msgid "Source" msgstr "Codice sorgente" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11831,7 +11845,8 @@ msgctxt "shared" msgid "MIT license" msgstr "Licenza MIT" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11851,7 +11866,8 @@ msgctxt "shared" msgid "Join the community" msgstr "Unisciti alla community" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12842,8 +12858,1362 @@ msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "Di N0zoM1z0 · 13 ottobre 2026 · Bozza da rivedere" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Da N0zoM1z0 · 13 ottobre 2026" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "Vai al contenuto" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "{ph0}Home{ph1} / {ph2}Blog{ph3} / Modelli e reverse engineering" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "Saggio" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Il reverse engineering necessita di un modello più grande?" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Da N0zoM1z0 · 13 ottobre 2026" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "Punti chiave" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}La capacità del modello influisce sulle ipotesi che un agente AI può " +"proporre.{ph1} Le sessioni successive hanno risolto problemi che il lavoro " +"precedente aveva lasciato irrisolti." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}Un compilatore familiare offre modelli concreti da apprendere per un " +"modello specializzato.{ph1} Alcune corrispondenze esatte del codice macchina" +" dipendono da modelli di codice sorgente specifici e riutilizzabili." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}Includi la conoscenza del repository in un confronto equo.{ph1} Un " +"modello più piccolo con un buon recupero della conoscenza potrebbe già avere" +" informazioni sufficienti per fare progressi." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}Addestra il modello per scegliere il prossimo esperimento utile e " +"testare il suo strumento di verifica.{ph1} Il feedback è utile solo nella " +"misura in cui lo strumento verifica le cose giuste." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}Valuta i risultati verificati della ricostruzione insieme a tempi e " +"costi.{ph1} Questo è ciò che voglio misurare per il flusso di lavoro " +"completo dell'agente AI." + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "In questo articolo" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}Che cosa hanno cambiato i modelli più capaci{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}Perché specializzarsi?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}Dove dovrebbe essere archiviata questa esperienza?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}Imparare il prossimo esperimento utile{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}Reverse engineering e apprendimento per rinforzo{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}Quando il controllo di verifica è sbagliato{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}Un confronto che vale la pena fare{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}Che tipo di agente AI suggerisce questo?{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "Che cosa hanno cambiato i modelli più capaci" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"Dall'agosto 2026, ho esplorato quanto reverse engineering può eseguire un " +"agente con l'accesso a un programma e un modo per verificarne il lavoro. La " +"maggior parte della mia esperienza deriva dalla ricostruzione di giochi, in " +"particolare Touhou. Man mano che ho cambiato modello, i repository hanno " +"tenuto traccia di ciò che è riuscita a recuperare ciascuna sessione." + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"Gran parte di questo lavoro mira alla ricostruzione esatta: compiliamo il " +"sorgente mantenuto in una toolchain storica con versioni fisse e " +"controlliamo se riproduce la funzione di destinazione. Quando un modello " +"successivo riesce dove il lavoro precedente si era bloccato, le modifiche " +"alla fonte mostrano cosa ha reso possibile quel progresso." + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "Le variabili temporanee necessitavano di vite sovrapposte" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"Gran parte della logica della funzione di attacco TH09 era già stata " +"ricostruita, ma il compilatore memorizzava ancora i risultati temporanei in " +"modo diverso dall'originale. Il problema rimanente era quanto tempo " +"occorreva a quei risultati per rimanere in vita e quando la loro " +"conservazione poteva essere riutilizzata." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"Un {ph0}GPT-6-Sol record di sessione{ph1} ha segnalato questa mancata " +"corrispondenza dopo aver testato diverse disposizioni ordinarie del codice " +"sorgente. Quando {ph2}GPT-6.1-Sol ha esaminato nuovamente il problema{ph3}, " +"ha proposto un'ipotesi utile sulla durata di queste variabili temporanee." + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"La funzione ha prodotto risultati temporanei e li ha copiati in posizioni di" +" animazione. In C++, un risultato temporaneo rimane attivo finché " +"l'espressione che lo utilizza non termina. Raggruppare le operazioni reali " +"in un'unica espressione ha mantenuto vivi insieme i risultati, quindi il " +"compilatore non poteva più riutilizzare la stessa memoria per tutti loro." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"Un risultato ha ancora spazio di archiviazione condiviso con un valore " +"utilizzato successivamente per i controlli di collisione. Dare a quel valore" +" la propria durata ha risolto la rimanente sovrapposizione. Dopo diversi " +"esperimenti mirati, {ph0}una ricompilazione pulita dal sorgente ha " +"confermato che l'intera funzione corrispondeva a{ph1}." + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"Sono interessato a come la scoperta e i test si siano supportati a vicenda. " +"L’ipotesi sulla durata temporanea degli oggetti ha dato all’agente AI una " +"nuova direzione. Il feedback del compilatore ha aiutato a perfezionare " +"quell'ipotesi in una disposizione del codice sorgente che corrispondesse." + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "Un diverso grafico del flusso di controllo" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"Una seconda funzione TH09 mostra perché questa nuova direzione può essere " +"importante. In questo caso, l’ostacolo era il modo in cui diversi rami " +"condividevano il loro lavoro finale. Il lavoro precedente di " +"{ph0}GPT-6.1-Sol{ph1} faceva corrispondere parti della funzione, ma i rami " +"si univano ancora in modo diverso dall'originale." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra ha cambiato il grafico del flusso di controllo " +"circostante{ph1}. I percorsi principali conducono ora ad un aggiornamento " +"finale condiviso e al ritorno. In quella struttura sorgente, il compilatore " +"ha combinato i rami nello stesso modo dell'originale. La funzione completa " +"corrispondeva e un {ph2}controllo di ricostruzione separato lo ha " +"confermato{ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}La modifica della rappresentazione ha aperto uno spazio di ricerca più " +"produttivo.{ph1} Entrambi i casi mi danno un motivo per prendere sul serio " +"le capacità generali del modello: un agente può continuare a provare " +"variazioni locali per molto tempo prima di trovare un modo migliore per " +"esprimere il lavoro circostante." + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"Le sessioni successive hanno anche ereditato il codice precedente e i " +"tentativi falliti già documentati. Questo lavoro accumulato faceva parte del" +" loro vantaggio. La cronologia permette di esaminare le scoperte e le " +"verifiche; un confronto controllato aiuterebbe a separare il contributo del " +"modello da quello dell’esperienza ereditata." + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"Questo è ciò che rende la domanda interessante per me. Alcuni progressi sono" +" arrivati ​​dalla ricerca di una nuova rappresentanza. Gran parte del lavoro" +" rimanente prevedeva il test di diverse disposizioni del codice sorgente " +"rispetto a un compilatore familiare. Quanto di tutto ciò richiede un’ampia " +"capacità di ragionamento? Quanto potrebbe imparare un modello lavorando " +"ripetutamente con quel compilatore e quel codice?" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "Perché un modello di IA specializzato potrebbe funzionare bene?" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou ci dà una ragione concreta per aspettarci che quell'esperienza sia " +"d'aiuto. Gran parte del codice del gioco proviene dallo stesso autore, ZUN, " +"quindi le abitudini ricorrenti sono un presupposto di partenza ragionevole. " +"I repository lo confermano con esempi utili: cicli correlati e disposizioni " +"di supporto hanno fornito ipotesi attraverso i giochi." + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"Questi modelli di origine diventano più specifici quando includiamo il " +"compilatore. In una particolare versione e insieme di opzioni, piccole " +"modifiche all'origine possono produrre differenze ripetibili nel codice " +"macchina. Alcune di queste differenze sono sorprendentemente oscure." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"In TH095, ad esempio, {ph0}anche il nome di una variabile locale potrebbe " +"influenzare dove VC7.1 l'ha memorizzata{ph1}. Lo spostamento della sua " +"dichiarazione non ha risolto il layout di archiviazione. Tuttavia, una " +"precedente funzione in background aveva rivelato un modello di denominazione" +" che influenzava la scelta del compilatore. Ciò ha suggerito un'ipotesi più " +"promettente per {ph2}ricostruire {ph3}{ph4}." + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"Il riutilizzo di quel modello per la variabile della dimensione del file ha " +"prodotto il layout richiesto senza aggiungere spazio di archiviazione. La " +"funzione completa quindi corrispondeva." + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"Questo è un dettaglio molto specifico che un modello di intelligenza " +"artificiale generica potrebbe non conoscere quando incontra per la prima " +"volta il problema. Un modello specializzato per questo compilatore potrebbe " +"vedere molti casi simili e apprendere quali variazioni del codice sorgente " +"vale la pena testare per prime." + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"L'opportunità è particolarmente chiara nei problemi di abbinamento finale in" +" cui ne comprendiamo già il comportamento. Ci stiamo quindi chiedendo quale " +"disposizione ordinaria del codice sorgente questo compilatore trasforma " +"nelle istruzioni osservate. La specializzazione potrebbe aiutare il modello " +"a proporre tali forme in modo più efficiente, mentre l’obiettivo determina " +"ancora quali ipotesi sopravvivono." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"Gli schemi utili devono conservare il loro contesto. TH08 usa una versione " +"di Visual C++ precedente a quella di TH095 e TH09. TH04 usa un altro " +"compilatore dell’epoca DOS. La {ph0}ricostruzione del codice del " +"giocatore{ph1} richiedeva ipotesi diverse sugli schemi del sorgente e sul " +"collegamento del codice in un eseguibile. Lo stesso autore ci dà " +"un’aspettativa iniziale, ma ogni obiettivo indica se è applicabile. Un " +"modello specializzato dovrebbe imparare a distinguere questi contesti." + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "Dove dovrebbe essere archiviata questa esperienza?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"Prima di decidere di insegnare questi modelli attraverso l'addestramento dei" +" modelli, esaminerei la conoscenza che già conserviamo nel repository. Nel " +"{ph0}saggio precedente{ph1} ho scritto che diventa la memoria di lavoro del " +"progetto. Quella memoria include lezioni del compilatore, insieme alla fonte" +" e agli esperimenti falliti che li hanno resi utili." + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"Recuperando quel record, un agente può iniziare con un risultato noto e " +"verificare come lo abbiamo raggiunto. Può anche evitare alternative che " +"hanno già fallito nello stesso contesto. Il punto di partenza migliora senza" +" modificare i parametri addestrati del modello." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"Durante la ricostruzione della funzione di TH09 per la creazione di nemici, " +"una sessione {ph0}GPT-6-Luna-max{ph1} ha preso un'ipotesi dal relativo " +"codice sorgente TH08. Il codice precedente suggeriva come organizzare il " +"ciclo e gestire gli errori al suo interno. Le prove di TH09 hanno fornito i " +"dettagli specifici del gioco. Le ricostruzioni dall'origine hanno confermato" +" che la funzione completa corrispondeva." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}\"Il modello non conosceva quella stranezza del compilatore\" e \"il " +"modello non poteva usarlo\" sono problemi diversi.{ph1} Il recupero può " +"risolvere il primo, ma trasformare questo fatto in un candidato utile " +"richiede ancora lavoro. E se il codice sorgente candidato fallisce, il " +"modello deve decidere cosa significherà il risultato per il suo prossimo " +"tentativo. Conoscere il modello rilevante è solo l’inizio di quel processo." + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"Per questo motivo, includerei in ogni confronto un modello di intelligenza " +"artificiale generico più piccolo con un buon recupero. Fornire la stessa " +"libreria di esperienze ci aiuterebbe a identificare ciò che la formazione " +"aggiunge: una migliore generazione di candidati, decisioni migliori dopo il " +"feedback o entrambi." + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "Imparare il prossimo esperimento utile" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"Quelle decisioni dopo il feedback sono ciò che mi piacerebbe di più " +"studiare. Nella ricostruzione agentica, il modello propone una spiegazione e" +" la mette alla prova rispetto al programma. Anche quando il codice sorgente " +"candidato fallisce, il risultato può restringere le possibilità rimanenti e " +"suggerire un prossimo esperimento più utile." + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"Consideriamo nuovamente l'esempio dei risultati temporanei. La sua fonte " +"finale mostra la soluzione che ha funzionato, ma il percorso verso quella " +"fonte contiene più informazioni. Ciascuno dei seguenti esperimenti ha " +"contribuito a identificare ciò che ancora doveva cambiare." + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "Esperimento" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "Cosa ci ha insegnato il suo risultato" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "Prova un modo diverso per copiare i risultati" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "Il codice di copia cambia, ma la funzione cambia ancora." + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "Utilizzare direttamente i valori restituiti" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "" +"La copia segue lo schema originale, ma l'archiviazione temporanea differisce" +" ancora." + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "Mantieni valide tutte le variabili temporanee contemporaneamente" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "" +"Ottengono un'archiviazione separata, ma uno si sovrappone comunque a un " +"valore utilizzato successivamente per i controlli di collisione." + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "Mantenere separato il valore del controllo collisioni" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "" +"Una ricompilazione pulita dalla fonte conferma la corrispondenza completa." + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "" +"Esperimenti selezionati dalla ricostruzione della funzione di attacco sopra," +" raggruppati in base a ciò che ci hanno insegnato." + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"Un registro dell'addestramento che mantenga questa sequenza può insegnare " +"più della sola fonte finale. Collega ogni modifica proposta a un risultato " +"osservato e alla decisione successiva. Anche mantenere le variazioni " +"inefficaci è importante: dicono a un agente successivo quali ipotesi sono " +"già state testate in quel contesto." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}Vorrei che il modello imparasse come utilizzare il feedback.{ph1} Un " +"risultato neutro è un motivo per riconsiderare la variazione; una mancata " +"corrispondenza modificata può indicare un'altra causa. Quando le modifiche " +"locali smettono di essere informative, l'agente dovrebbe essere in grado di " +"rivisitare la rappresentazione originale o ispezionare nuovamente le prove." + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"Imparare a fare queste scelte richiede autonomia. A volte il passaggio " +"successivo utile è ispezionare un altro chiamante anziché compilare un'altra" +" modifica. Una rigida sequenza di riscritture richieste eliminerebbe quella " +"scelta, insieme a parte del comportamento che vogliamo che il modello " +"impari." + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "" +"Perché il reverse engineering è interessante per l'apprendimento per " +"rinforzo" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"La ricostruzione limitata ci offre un ambiente pratico per studiare queste " +"scelte. Una volta fissate le versioni di destinazione e del compilatore, " +"molti esperimenti di origine producono un feedback rapido e ripetibile. " +"L'agente può compilare un candidato, ispezionare la mancata corrispondenza e" +" utilizzare tale osservazione per scegliere il passaggio successivo." + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"L’apprendimento per rinforzo potrebbe addestrare il processo decisionale " +"attraverso il feedback che riceve. Un candidato accettato con una verifica " +"definita ci dà un risultato concreto, mentre la sequenza dei tentativi ci " +"permette di osservare quanto lavoro è stato necessario per raggiungerlo." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"Esistono già ricerche a sostegno di questa direzione. " +"{ph0}LLM4Decompile{ph1} studia modelli specializzati per la decompilazione. " +"{ph2}AutoDecompiler{ph3} addestra il perfezionamento multi-turn utilizzando " +"il feedback del compilatore e dell'esecuzione. I loro studi si concentrano " +"sul recupero funzionale. Per questo lavoro, li userei come punto di partenza" +" per il compito correlato di trovare fonti che corrispondano all'output di " +"un compilatore storico." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"Per raggiungere un compito così ben definito, tuttavia, potremmo prima dover" +" stabilire quali risorse possiede una funzione o ideare una nuova " +"misurazione del tempo di esecuzione. In TH08, ad esempio, una " +"{ph0}differenza di collisione riportava al modo in cui il programma " +"originale gestiva i calcoli in virgola mobile{ph1}. Prima che l'agente " +"potesse testare una correzione utile, doveva capire cosa misurare." + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"Ciò suggerisce un altro ruolo per un modello ampiamente capace: in primo " +"luogo, contribuire a rendere il problema controllabile. Una volta effettuato" +" questo controllo, possiamo indagare fino a che punto può arrivare una " +"politica di ricerca appresa." + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "Quando il controllo di verifica è sbagliato" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"Quando una verifica viene usata per l’addestramento, anche la sua copertura " +"entra nell’ambiente di apprendimento. Chiamo oracle quel riferimento o " +"verificatore. Persino una risposta ripetibile va interpretata in base a ciò " +"che verifica davvero." + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 ha mostrato quanto si può nascondere al di fuori di quella copertura. " +"Alcune funzioni ricostruite hanno superato i confronti esatti con costanti " +"in virgola mobile errate. Il comparatore aggiustava gli indirizzi trasferiti" +" prima di controllare le istruzioni, ma controllava i valori dietro quegli " +"indirizzi solo quando erano stati esplicitamente registrati per la verifica." +" Altri riferimenti letterali potrebbero passare senza che i loro valori " +"vengano esaminati." + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"Per colmare questa lacuna, un {ph0}controllo più ampio{ph1} ha rilevato " +"costanti errate nelle funzioni che avevano già superato la verifica. In un " +"calcolo, il codice sorgente ricostruito aggiungeva {ph2}, mentre l'originale" +" aggiungeva {ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}Un modello premiato dal vecchio controllo di verifica potrebbe imparare" +" a produrre una ricostruzione errata.{ph1} Non abbiamo addestrato un modello" +" RL utilizzando quel controllo. L'errore che abbiamo osservato riguardava il" +" controllo stesso. Ma se lo usassimo come ricompensa, lo stesso punto cieco " +"diventerebbe parte dell’ambiente di apprendimento." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"Un test funzionale può avere un punto cieco simile. {ph0}La ricerca che " +"utilizza il fuzzing differenziale sui decompilatori LLM{ph1} ha rilevato " +"differenze comportamentali al di fuori dei test forniti. Dobbiamo quindi " +"comprendere una prova di superamento in termini dei casi che esercita. Allo " +"stesso modo, la corrispondenza di un'istruzione deve essere intesa in " +"termini di byte e dati che verifica." + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"Prima di utilizzare uno dei due tipi di controllo di verifica come " +"ricompensa, lo testerei con risultati deliberatamente errati. Vorrei anche " +"mantenere l'accettazione finale separata dal codice sorgente che l'agente AI" +" può modificare. Il modello deve superare la verifica ricostruendo " +"correttamente il programma. Altrimenti, la semplice copia di un array di " +"byte dalla destinazione potrebbe ingannare un ingenuo sistema di punteggio " +"senza recuperare codice sorgente utile." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}Migliorare lo studente e migliorare il suo strumento di verifica " +"appartengono allo stesso problema di ricerca.{ph1} Man mano che lo studente " +"migliora nell'ottimizzare il feedback che gli forniamo, anche comprendere " +"quel feedback diventa più importante." + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "Un confronto che vale la pena fare" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"Con questi controlli in atto, inizierei il confronto su una famiglia di " +"attività VC7.1 fissa. Innanzitutto, stabilirei le linee di base per un forte" +" modello di intelligenza artificiale generale e uno più piccolo. L'aggiunta " +"della conoscenza del repository al modello più piccolo mostrerebbe quali " +"cambiamenti nel recupero. Da lì, la specializzazione supervisionata e una " +"versione addestrata con RL ci permetterebbero di esaminare cosa aggiunge la " +"formazione." + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"Per rendere informativi questi confronti, ciascun modello riceverebbe le " +"stesse prove e strumenti target. Per i confronti specifici della formazione," +" manterrei fisso anche il modello base e le conoscenze disponibili. Prove " +"ripetute aiuterebbero a distinguere una strategia di ricerca affidabile da " +"una soluzione candidata che ha successo per caso." + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"Separerei quindi due condizioni iniziali: ricostruire una funzione dalle sue" +" istruzioni e chiudere le ultime differenze nel sorgente mantenuto. Un " +"modello di IA specializzato potrebbe essere eccellente per il secondo " +"compito, mentre un modello di IA generico svolge gran parte del lavoro " +"necessario per raggiungerlo. Misurarli separatamente rivelerebbe quella " +"divisione del lavoro." + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"Entrambi i compiti necessitano di esempi di test esclusi dall'addestramento " +"e dal recupero. I giochi Touhou correlati condividono il codice motore. " +"Escludere un gioco dall'allenamento potrebbe comunque lasciare funzioni " +"quasi identiche nel set di allenamento. Invece, escluderei interi gruppi di " +"codice sorgente correlato e manterrei le loro risposte verificate fuori sia " +"dai dati di addestramento che dalla libreria di recupero." + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"Per questi esempi, il risultato sarebbe corrispondenze complete e verificate" +" in modo indipendente. Budget uguali per le proposte dei candidati o i " +"tentativi di compilazione aiuterebbero a confrontare la qualità della " +"ricerca. La misurazione del tempo e dei costi totali, inclusa la " +"compilazione, mostrerebbe se tale qualità produce un flusso di lavoro " +"efficiente. I costi di formazione dovrebbero essere misurati separatamente." + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"Entrambe le misurazioni contano. Una proposta economica può portare a un " +"flusso di lavoro costoso se comporta più compilazioni che non forniscono " +"informazioni utili. Al contrario, un modello più lento può essere più " +"economico se trova prima un esperimento utile. Il confronto dovrebbe " +"includere questi costi." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"Il conteggio dei risultati accettati mantiene inoltre in prospettiva i " +"progressi intermedi. Un TH09 {ph0}candidato al replay-save{ph1} sembrava " +"promettente perché la funzione compilata aveva le dimensioni previste, ma un" +" confronto completo ha comunque rilevato differenze sostanziali. I " +"cambiamenti nel modo in cui il compilatore organizza le istruzioni possono " +"anche far sembrare un candidato più vicino al risultato atteso senza " +"risolvere il problema di fondo. Tale diagnostica può guidare il prossimo " +"esperimento, ma convaliderei qualsiasi ricompensa per \"avvicinarsi\" " +"rispetto al controllo completo di accettazione." + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"Sarei interessato allo schema tra questi risultati. Se un modello di " +"intelligenza artificiale specializzato funziona bene all’interno di una " +"famiglia di compilatori familiare, tale vantaggio sopravvive a un " +"cambiamento nella toolchain? E una volta che entrambi i modelli di " +"intelligenza artificiale hanno un buon recupero, quanto aggiunge ancora " +"l’addestramento? Le risposte ci aiuterebbero a decidere dove ciascun tipo di" +" modello è più utile." + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "Che tipo di agente AI suggerisce questo?" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"Se i modelli si rivelano utili in diverse fasi, una soluzione utile potrebbe" +" essere un ampio agente con accesso a un modello di intelligenza artificiale" +" specializzato. Una volta che un'attività diventa una ricerca familiare del " +"compilatore, potrebbe richiedere il modello di intelligenza artificiale " +"specializzato per i candidati. Se i risultati smettono di avere senso, " +"l’agente più ampio potrebbe tornare all’evidenza target o indagare sulla " +"misurazione stessa." + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"In questo modo, la specializzazione diventa uno strumento che l’agente può " +"scegliere man mano che l’indagine si sviluppa. L'agente mantiene la libertà " +"di cambiare direzione che ha reso possibili i progressi precedenti." + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"Ciò deriva naturalmente dal cambiamento industriale che ho descritto nel " +"saggio precedente. Essere in grado di eseguire molti più esperimenti cambia " +"il modo in cui organizziamo il lavoro. Ci dà anche una ragione per chiederci" +" quali decisioni necessitano di un ampio giudizio e quali migliorano " +"attraverso l’esperienza ripetuta con una particolare imbarcazione." + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"Per ora il modello generale più forte si è guadagnato il suo posto nel mio " +"lavoro risolvendo problemi che avevo lasciato aperti. Sono proprio questi " +"successi a farmi studiare il processo: quanto dipendeva dall’ampiezza delle " +"sue capacità, e quanto potrebbe fare un modello addestrato specificamente " +"per questo tipo di ricostruzione?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}I repository ci offrono un modo per testare la domanda.{ph1} Vorrei " +"utilizzare quel record per creare il confronto e vedere cosa impariamo." + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "Evidenze e attribuzione del modello" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"Gli esempi descrivono i confronti storici a livello di funzione registrati " +"nei commit collegati. I controlli di runtime dell'intero gioco sono " +"traguardi separati. I record di origine e corrispondenza sono stati " +"esaminati il 13 ottobre 2026." + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"L'etichetta TH09 {ph0} significa GPT-6-Astra. Il modello associato a {ph1} è" +" cambiato nel tempo, quindi l'etichetta da sola non può identificare il " +"modello utilizzato in una specifica esecuzione. I nomi espliciti dei modelli" +" negli altri commit citati forniscono l'attribuzione registrata. Queste " +"cronologie non includono misurazioni coerenti dei costi o dell'impegno per " +"ogni esecuzione." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 durata degli oggetti temporanei:{ph1} la {ph2}revisione del " +"contesto precedente{ph3} e la {ph4}ricostruzione completa verificata da una " +"ricompilazione pulita dall'origine{ph5}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}TH09 Front callback:{ph1} {ph2}aree diagnostiche che non " +"corrispondevano{ph3}, il nuovo grafico del flusso di controllo di " +"{ph4}Astra{ph5} e il relativo {ph6}controllo di ricompilazione " +"accettato{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}Conoscenze del compilatore:{ph1} {ph2}TH095 riutilizzo del " +"comportamento del bucket di identificatori{ph3}, {ph4}TH04 codice sorgente " +"del giocatore e ordine dei linker{ph5} e {ph6}Luna ricostruzione completa " +"della funzione di creazione del nemico di TH09{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}Controlli e diagnostica:{ph1} {ph2}TH08 audit costante in virgola " +"mobile{ph3}, {ph4}TH08 indagine numerica{ph5} e {ph6}TH09 ricostruzione con " +"salvataggio replay{ph7}." + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"I confronti proposti tra modelli specializzati e la formazione RL sono " +"esperimenti futuri. Le storie dei progetti li motivano. Per questo saggio " +"non è stato valutato alcun modello specializzato addestrato." + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "Il reverse engineering necessita di un modello più grande? | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"Cosa mi ha insegnato la ricostruzione Touhou sui modelli generali, sui " +"modelli specialistici e sull'imparare a proporre ipotesi migliori dal " +"feedback del compilatore." + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "In questa pagina" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "Modelli e saggio di reverse engineering" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "Esperimenti di ricostruzione selezionati" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "Torna in alto" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "Di N0zoM1z0 · 13 ottobre 2026 · Bozza da rivedere" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/ja/messages.po b/website/locales/ja/messages.po index a49063596..0ff6f1d5c 100644 --- a/website/locales/ja/messages.po +++ b/website/locales/ja/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "リバースエンジニアリングに関する考え。" #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "ゲーム再構築{ph0}小論文" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "リバースエンジニアリングにおける産業革命" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -51,19 +51,20 @@ msgid "" msgstr "" "どのようなマニュアル再とTH08を再構築,TH095とTH04は、エージェントについて教えてくれました,プロジェクト間で化合物の検証と知識." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "記事を読む" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "によってN0zoM1z0•10 10月2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -97,6 +98,8 @@ msgid "" msgstr "REAとAIエージェントの実践的な経験に基づいて、リバースエンジニアリングに関する考え。" #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -131,6 +134,7 @@ msgid "" msgstr "REA—ソフトウェアがどのように動作するかを調べます。 コーディングエージェントとのリバースエンジニアリング。" #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -151,6 +155,7 @@ msgid "REA home" msgstr "REA ホーム" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -171,6 +176,7 @@ msgid "Main navigation" msgstr "メインナビゲーション" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -191,6 +197,7 @@ msgid "REA on GitHub" msgstr "GitHub で REA を見る" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -211,6 +218,7 @@ msgid "Join REA on Discord" msgstr "REA の Discord に参加" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -230,7 +238,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "X で REA をフォロー" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "ページ上部へ" @@ -10886,6 +10894,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "同じ間隔:256÷16=16角度単位=22.5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -10906,6 +10915,7 @@ msgid "Showcases" msgstr "事例" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -10926,6 +10936,7 @@ msgid "Blog" msgstr "ブログ" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -10946,6 +10957,7 @@ msgid "Get started" msgstr "はじめる" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -10965,7 +10977,8 @@ msgctxt "shared" msgid "Guides" msgstr "ガイド" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -10985,7 +10998,8 @@ msgctxt "shared" msgid "Source" msgstr "ソースコード" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11005,7 +11019,8 @@ msgctxt "shared" msgid "MIT license" msgstr "MIT ライセンス" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11025,7 +11040,8 @@ msgctxt "shared" msgid "Join the community" msgstr "コミュニティに参加" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -11984,8 +12000,1137 @@ msgstr "Touhou の再構築を通じて考える、汎用モデル、特化モ #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "N0zoM1z0 著 · 2026年10月13日 · レビュー用草稿" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "N0zoM1z0 著 · 2026 年 10 月 13 日" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "コンテンツにスキップ" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "{ph0}ホーム{ph1} / {ph2}ブログ{ph3} / モデルとリバース エンジニアリング" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "エッセイ" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "リバースエンジニアリングに、より大きなモデルは必要か？" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "N0zoM1z0 著 · 2026 年 10 月 13 日" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "要点" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}モデルの能力によって、エージェントが見つけられる仮説は変わる。{ph1} 後の作業では、それまで解決できなかった問題が解決された。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}使い慣れたコンパイラには、特化モデルが学べる具体的なパターンがある。{ph1} " +"完全一致の中には、細かな、再利用可能なソースコードのパターンに依存するものもある。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}リポジトリに蓄積した知識も、公平に比較したい。{ph1} 小さなモデルでも、適切に知識を検索できれば、作業を進めるのに十分かもしれない。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}次の有用な実験を選ぶ力を学習させ、その検証器も確かめる。{ph1} フィードバックの価値は、検証器が実際に何を確認しているかで決まる。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}検証を通った再構築の成果を、時間とコストと合わせて評価する。{ph1} エージェントのワークフロー全体で測りたいのは、その結果だ。" + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "この記事の内容" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}強いモデルで何が変わったか{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}なぜ特化を考えるのか{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}その経験はどこに蓄積すべきか？{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}次の有用な実験を選ぶ力を学ぶ{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}リバース エンジニアリングと強化学習{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}検証器が間違っていたら{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}実行する価値のある比較{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}どんなエージェントが必要なのか？{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "強いモデルで何が変わったか" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"2026年8月から、プログラムにアクセスし、自分の作業を検証できるエージェントが、どこまでリバースエンジニアリングを進められるかを探ってきた。私の経験は主にゲーム、特に" +" Touhou（東方）の再構築から得たものだ。モデルを変えるたびに、各セッションで何を復元できたかがリポジトリに残った。" + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"その多くは完全一致を目指す再構築だ。バージョンを固定した当時のツールチェーンで、保守しているソースをコンパイルし、対象の関数を再現できるか確かめる。後のモデルが以前の行き詰まりを解消したとき、ソースの変更を見ると、何が進歩を可能にしたかがわかる。" + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "一時的な値を同時に有効にしておく必要があった" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"TH09 " +"のある攻撃関数は、ロジックの大部分が復元済みだった。それでも、コンパイラが一時的な結果を格納する方法は原版と違っていた。残る問題は、その結果がいつまで有効であるべきか、そして格納領域をいつ再利用できるかだった。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"いくつかの通常のソースコード配置をテストした後、{ph0}GPT-6-Sol セッション レコード{ph1} でこの不一致が報告されました。 " +"{ph2}GPT-6.1-Sol がこの問題を再度調査したとき{ph3}、これらの一時変数の存続期間に関する有用な仮説が提案されました。" + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"この関数は一時的な結果を作り、アニメーションの位置にコピーする。C++ " +"の一時的な結果は、それを使う式が終わるまで生存する。実際の処理を一つの式にまとめると、結果の生存期間が重なり、コンパイラはすべての結果に同じ格納領域を使い回せなくなる。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"一つの結果だけは、後の衝突判定に使う値と領域を共有していた。その値に独立した生存期間を与えると、最後の重なりも解消された。いくつかの絞り込んだ実験の後、{ph0}クリーンビルドで関数全体の一致が確認された{ph1}。" + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"発見とテストがどのように相互にサポートし合っていたのかに興味があります。一時オブジェクトの存続期間に関する仮説は、AI " +"エージェントに新しい方向性を与えました。コンパイラーからのフィードバックにより、その仮説を改良して、一致するソースコードの配置を実現することができました。" + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "別の制御フロー グラフ" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"2 番目の TH09 関数は、その新しい方向が重要である理由を示しています。ここでの障害は、複数のブランチが最終的な作業をどのように共有するかでした。 " +"{ph0}GPT-6.1-Sol の以前の作業{ph1} " +"では、関数の一部が一致していましたが、分岐の結合方法は依然としてオリジナルとは異なっていました。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra が周囲の制御フロー グラフを変更しました{ph1}。メイン " +"パスは共有された最終更新と復帰につながりました。そのソース構造では、コンパイラはオリジナルと同じ方法でブランチを結合しました。完全な関数は一致しており、{ph2}別のリビルド" +" チェックでそれが確認されました{ph3}。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}表現を変えたことで、より有望な探索空間が開けた。{ph1} " +"この二つの例から、汎用モデルの能力は重要だと感じる。エージェントは局所的な変更を長く試した末に、周囲の処理を表すもっとよい方法を見つけることがある。" + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"ただし、後のセッションは、それまでのソースと失敗の記録を引き継いでいる。その蓄積も有利に働いた。履歴から発見と検証の過程は調べられるが、モデルの効果と引き継いだ経験の効果を分けるには、条件をそろえた比較が必要だ。" + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"これが私にとってこの質問が興味深い理由です。新しい表現を見つけることである程度の進歩が得られました。残りの作業の多くには、使い慣れたコンパイラに対してさまざまなソース" +" " +"コードの配置をテストすることが含まれていました。このうちどの程度の範囲で幅広い推論能力が必要でしょうか?モデルはそのコンパイラーとコードを繰り返し操作することでどれだけ学習できるでしょうか?" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "特化モデルがうまく働く可能性は？" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou には、経験の蓄積が役立つと考える具体的な理由がある。多くのゲームコードを ZUN " +"一人が書いているため、繰り返し現れる書き方を最初の仮説にできる。実際、リポジトリには、似たループや補助関数の構成が別のゲームの仮説につながった例がある。" + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"コンパイラを含めると、これらのソース " +"パターンがより具体的になります。特定のバージョンとオプションのセットでは、ソースに小さな変更を加えると、マシンコードに再現性のある差異が生じる可能性があります。これらの違いの中には、驚くほどわかりにくいものもあります。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"たとえば、TH095 では、{ph0}ローカル変数の名前であっても、VC7.1 " +"がそれを格納する場所に影響を与える可能性があります{ph1}。その宣言を移動してもストレージ " +"レイアウトは解決されませんでした。ただし、以前のバックグラウンド関数では、コンパイラの選択に影響を与える名前付けパターンが明らかになっていました。これは、{ph2}{ph3}{ph4}" +" を再構築するためのより有望な仮説を示唆しました。" + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "そのパターンをファイルサイズの変数に使うと、格納領域を増やさずに必要な配置が得られた。続いて関数全体が一致した。" + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"これは非常に具体的な詳細であり、汎用 AI " +"モデルが最初に問題に遭遇したときには認識できない可能性があります。このコンパイラに特化したモデルは、そのようなケースを多数確認し、どのソース " +"コードのバリエーションが最初にテストする価値があるかを学習できます。" + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"動作を理解したうえで最後の差分を詰める場面では、この可能性が特にはっきりする。問うのは、このコンパイラがどの通常の書き方を観測された命令に変換するかだ。特化した学習によって候補を効率よく出せるかもしれないが、どの仮説が正しいかは対象のプログラムが決める。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"ただし、パターンには適用条件を残す必要がある。TH08 は TH095 や TH09 より古い Visual C++ を使う。TH04 は DOS " +"時代の別のコンパイラを使い、{ph0}プレイヤーのコードの再構築{ph1}では、書き方や実行ファイルへのリンク順序について別の仮説が必要だった。同じ作者であることは手がかりになるが、適用できるかは各対象で確かめる。その区別も特化モデルが学ぶべきことだ。" + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "その経験はどこに蓄積すべきか？" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"モデルの学習でこうしたパターンを覚えさせる前に、まずリポジトリに保存している知識を見たい。{ph0}前のエッセイ{ph1}では、リポジトリがプロジェクトの作業記憶になっていると書いた。そこにはコンパイラから学んだことと、それを裏付けるソースや失敗した実験が残っている。" + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"そのレコードを取得することで、エージェントは既知の結果から開始し、どのようにしてその結果に到達したかを検査できます。また、同じコンテキストですでに失敗した代替案を回避することもできます。モデルのトレーニング済みパラメーターを変更することなく、開始点が改善されます。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"TH09 の敵生成関数を再構築した{ph0}GPT-6-Luna-max のセッション{ph1}は、TH08 " +"の関連するソースから仮説を得た。以前のコードが、ループの構成と失敗時の処理の手がかりになった。TH09 " +"自体の証拠でゲーム固有の細部を補い、クリーンビルドで関数全体の一致を確認した。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}「モデルがそのコンパイラの癖を知らない」と「知っていても使えない」は別の問題だ。{ph1} " +"検索で前者を解決できても、知識を有用な候補にするには能力がいる。候補が失敗すれば、その結果が次の試みに何を意味するかも判断しなければならない。関連するパターンを知るのは、出発点にすぎない。" + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"だから比較には、知識を適切に検索できる小さな汎用モデルも加えたい。同じ経験のライブラリを与えれば、学習が何を加えるかを調べられる。候補を作る力なのか、フィードバック後の判断なのか、それとも両方なのか。" + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "次の有用な実験を選ぶ力を学ぶ" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"私が最も研究したいのは、フィードバック後の判断だ。エージェントによる再構築では、モデルが説明を提案し、プログラムに照らして検証する。候補が失敗しても、可能性を絞り、次の有用な実験を考える材料になる。" + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"一時的な結果の例をもう一度考えてみましょう。最終的なソースには、機能した配置が示されていますが、そのソースへのルートにはさらに多くの情報が含まれています。以下の各実験は、まだ変更する必要があるものを特定するのに役立ちました。" + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "実験" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "その結果が私たちに教えてくれたこと" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "別の方法で結果をコピーしてみてください" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "コピーのコードは変わったが、関数にはまだ差分がある。" + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "戻り値を直接使用する" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "コピーは元のパターンに従いますが、一時ストレージは依然として異なります。" + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "一時的な結果を同時に有効にしておく" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "これらは別々のストレージを取得しますが、1 つは、後で衝突チェックに使用される値と依然として重複します。" + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "衝突判定の値を独立させる" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "ソースからクリーンに再コンパイルすると、完全に一致していることが確認されます。" + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "上記の攻撃関数の再構築から選択した実験を、教えられた内容ごとにグループ化しました。" + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"この順序を維持したトレーニング記録は、最終的なソースだけよりも多くのことを教えることができます。提案された各変更を観察された結果と次の決定に結び付けます。効果のないバリエーションを維持することも重要です。バリエーションは、後のエージェントに、そのコンテキストでどの仮説がすでにテストされているかを伝えます。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}モデルにはフィードバックの使い方を学んでほしい。{ph1} " +"結果が変わらなければ、その変更を考え直す理由になる。差分が変われば、別の原因が見えてくるかもしれない。局所的な編集から情報が得られなくなったら、全体の書き方を見直したり、証拠を再確認したりできるべきだ。" + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"これらの選択を行うことを学ぶには自主性が必要です。場合によっては、別の編集をコンパイルするのではなく、別の呼び出し元を検査することが次のステップとして役立つ場合があります。必要な書き換えを厳密に繰り返すと、モデルに学習させたい動作の一部とともに、その選択肢が削除されてしまいます。" + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "リバースエンジニアリングが強化学習にとって興味深い理由" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"範囲を定めた再構築は、こうした判断を研究するための実用的な環境になる。対象とコンパイラのバージョンを固定すれば、多くのソースの実験で素早く再現可能なフィードバックを得られる。エージェントは候補をコンパイルし、差分を調べ、次の行動を選べる。" + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"強化学習なら、得られたフィードバックからこの判断を学習できるかもしれない。定義された検証を通った候補は具体的な成果になり、試行の記録から、そこに至るまでの労力も調べられる。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"この方向を支える研究はすでにある。{ph0}LLM4Decompile{ph1}は逆コンパイルに特化したモデルを研究している。{ph2}AutoDecompiler{ph3}はコンパイルと実行のフィードバックを使って複数ターンの改善を学習する。両者が扱うのは機能の復元だ。私はそこを出発点に、当時のコンパイラの出力に一致するソースを探すという関連した課題を研究したい。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"ただし、このように明確に定義されたタスクに到達するには、まず関数が所有するリソースを確立するか、新しい実行時間の測定を考案する必要があるかもしれません。たとえば、TH08" +" " +"では、{ph0}衝突の違いにより、元のプログラムでの浮動小数点計算の処理方法が問題になりました{ph1}。エージェントが有用な補正をテストする前に、何を測定するかを決定する必要がありました。" + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"ここから、幅広い能力を持つモデルのもう一つの役割が見える。まず問題を検証可能にすることだ。検証方法ができて初めて、学習した探索方針がどこまで進めるかを調べられる。" + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "検証器が間違っていたら" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"その検証を使って学習すると、検証範囲も学習環境の一部になる。私はこうした参照基準や検証器を oracle " +"と呼んでいる。答えが再現可能でも、それが何を確認した答えなのかを理解する必要がある。" + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 " +"では、検証範囲の外に問題が隠れていた。誤った浮動小数点定数を使っていても、完全一致の比較を通る関数があった。比較器は命令を確認する前に再配置されたアドレスを調整するが、そのアドレスが指す値まで確認するのは、検証対象として明示的に登録された場合だけだった。ほかの定数への参照は、値を調べずに通ってしまう可能性があった。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"このギャップに対処するために、{ph0}広範な監査{ph1}により、すでに検証に合格した関数内に不正な定数が見つかりました。ある計算では、再構成されたソース" +" コードでは {ph2} が追加されましたが、元のコードでは {ph3} が追加されました。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}古い検証結果で報酬を与えれば、モデルは誤った再構築を学習してしまうかもしれない。{ph1} この検証器で RL " +"モデルを訓練したわけではない。観測したのは検証そのものの不具合だ。ただ、それを報酬に使えば、同じ盲点が学習環境に入ることになる。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"機能テストにも同様の盲点がある可能性があります。 {ph0}LLM " +"デコンパイラで差分ファジングを使用した研究{ph1}では、提供されたテスト以外の動作の違いが見つかりました。したがって、合格したテストを、実行されるケースの観点から理解する必要があります。同様に、命令の一致は、検証されるバイトとデータの観点から理解する必要があります。" + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"どちらの種類の検証チェックも報酬として使用する前に、意図的に間違った結果でテストします。また、最終的な承認は、AI エージェントが編集できるソース " +"コードとは別にしておきます。モデルはプログラムを正しく再構築することで検証に合格する必要があります。そうしないと、ターゲットからバイト配列をコピーするだけで、有用なソース" +" コードを回復せずに、単純なスコアリング システムを騙してしまう可能性があります。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}学習するモデルの改善と、その検証器の改善は同じ研究課題だ。{ph1} " +"モデルが与えられたフィードバックを上手に最適化するほど、そのフィードバックを理解することも重要になる。" + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "実行する価値のある比較" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"これらのチェックを実施したら、固定の VC7.1 タスク ファミリで比較を開始します。まず、強力な汎用 AI モデルと小規模な AI " +"モデルのベースラインを確立します。リポジトリの知識をより小さなモデルに追加すると、取得によってどのような変化が生じるのかがわかります。そこから、教師付き特殊化と" +" RL でトレーニングされたバージョンを使用して、トレーニングによって追加される内容を検証できるようになります。" + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"これらの比較を有益なものにするために、各モデルは同じターゲットの証拠とツールを受け取ります。トレーニング固有の比較の場合、基本モデルと利用可能な知識も固定したままにします。試行を繰り返すことは、信頼できる検索戦略と偶然成功した候補ソリューションを区別するのに役立ちます。" + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"二つの出発点も分けたい。命令から関数を再構築する場合と、保守しているソースの最後の差分を詰める場合だ。特化モデルが後者に優れていても、そこまでの作業の多くは汎用モデルが担っているかもしれない。別々に測れば、その分担が見える。" + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"どちらのタスクも、トレーニングと取得から除外されたテスト例が必要です。関連する Touhou ゲームはエンジン コードを共有します。トレーニングから 1" +" つのゲームを除外しても、トレーニング セットにほぼ同じ機能が残る可能性があります。代わりに、関連するソース " +"コードのグループ全体を除外し、検証済みの回答をトレーニング データと検索ライブラリの両方から除外します。" + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"測る成果は、関数全体が独立した検証を通って一致したことだ。候補数やコンパイル回数の予算をそろえれば探索の質を比べられる。コンパイルを含む総時間と総費用を測れば、それが効率的なワークフローにつながるかがわかる。学習費用は別に計上したい。" + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"どちらの測定値も重要です。安価な提案により、有用な情報を提供しないコンパイルが増加すると、ワークフローが高価になる可能性があります。逆に、有用な実験がすぐに見つかった場合、遅いモデルの方が経済的になる可能性があります。比較にはこれらのコストも含める必要があります。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"受け入れられた結果をカウントすることで、中間の進捗状況も把握できます。 1 つの TH09 " +"{ph0}再生保存候補{ph1}は、コンパイルされた関数が予想どおりのサイズを持っていたため有望に見えましたが、完全な比較では依然として大きな違いが見つかりました。コンパイラが命令を配置する方法を変更すると、根本的な問題が解決されずに、候補がより近くに見える可能性もあります。このような診断は次の実験を導くことができますが、私は「近づく」ことに対する報酬を完全な受け入れチェックと照らし合わせて検証します。" + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"結果全体にどんな傾向があるかを知りたい。特化モデルが使い慣れたコンパイラで優れていても、ツールチェーンを変えた後にその強みは残るだろうか。両方が十分に知識を検索できるとき、学習はどれだけ価値を加えるのか。答えは、各モデルをどこで使うべきかを考える助けになる。" + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "どんなエージェントが必要なのか？" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"モデルが段階ごとに違う強みを持つなら、汎用エージェントが特化モデルを呼び出せる構成が役立つかもしれない。問題が慣れたコンパイラでの探索になったら、特化モデルに候補を求める。結果が納得できなくなったら、汎用エージェントが対象の証拠や測定そのものを調べ直す。" + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "この構成では、特化モデルは調査の進み方に応じて選べる道具になる。エージェントには、以前の突破口を可能にした方向転換の自由が残る。" + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"これは、前回のエッセイで説明した産業の変化から自然に生じます。さらに多くの実験を実行できるようになると、作業の組織化の方法が変わります。また、どの決定には広範な判断が必要なのか、どの決定は特定の技術での繰り返しの経験によって改善されるのかを尋ねる理由も得られます。" + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"今のところ、未解決のまま残されていた問題を解決することで、最強の汎用 AI " +"モデルが私のワークフローにその地位を獲得しました。これらの成功により、私はこの作業をさらに詳しく調べてみたいと思うようになりました。どの程度がモデルの範囲に依存しているのか、また、この特定の種類の再構成でトレーニングされたモデルでどの程度の処理が可能でしょうか?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "{ph0}リポジトリは、質問をテストする方法を提供します。{ph1}その記録を使用して比較を構築し、何が分かるかを確認したいと思います。" + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "証拠とモデルの帰属" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"この例では、リンクされたコミットに記録された関数レベルの比較の履歴について説明します。ゲーム全体のランタイム " +"チェックは別のマイルストーンです。ソースと一致する記録は 2026 年 10 月 13 日に確認されました。" + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"TH09 ラベル {ph0} は GPT-6-Astra を意味します。 {ph1} " +"に関連付けられたモデルは時間の経過とともに変化したため、ラベルだけでは特定の実行で使用されたモデルを識別できません。他の引用されたコミット内の明示的なモデル名は、記録された帰属を示します。これらの履歴には、実行ごとのコストや労力の一貫した測定値は含まれていません。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 一時オブジェクトの有効期間:{ph1} {ph2}以前のコンテキスト " +"レビュー{ph3}と、{ph4}ソースからのクリーンな再コンパイルによって検証された完全な再構築{ph5}。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}TH09 Front コールバック:{ph1} {ph2}一致しなかった診断領域{ph3}、{ph4}Astra の新しい制御フロー " +"グラフ{ph5}、およびその {ph6}受け入れられた再コンパイル チェック{ph7}。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}コンパイラーの知識:{ph1} {ph2}TH095 識別子バケットの動作の再利用{ph3}、{ph4}TH04 プレーヤーのソース " +"コードとリンカーの順序{ph5}、および {ph6}Luna による TH09 の敵作成関数の完全な再構築{ph7}。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}チェックと診断:{ph1} {ph2}TH08 浮動小数点定数監査{ph3}、{ph4}TH08 数値調査{ph5}、および " +"{ph6}TH09 リプレイ保存再構築{ph7}。" + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"特化モデルと RL " +"の比較は、これから行いたい実験だ。プロジェクトの履歴がその動機になっているが、このエッセイで学習済みの特化モデルを評価したわけではない。" + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "リバースエンジニアリングに、より大きなモデルは必要か？ | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "Touhou の再構築から考える、汎用モデルと特化モデル、そしてコンパイラのフィードバックからよりよい仮説を学ぶ可能性。" + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "このページでは" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "モデルとリバース エンジニアリングのエッセイ" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "厳選された復元実験" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "ページ上部へ" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "N0zoM1z0 著 · 2026年10月13日 · レビュー用草稿" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/ko/messages.po b/website/locales/ko/messages.po index 0e8786ec5..7865dd486 100644 --- a/website/locales/ko/messages.po +++ b/website/locales/ko/messages.po @@ -32,37 +32,38 @@ msgid "Thoughts on reverse engineering." msgstr "리버스 엔지니어링에 대한 생각." #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "게임 재구성{ph0}에세이" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "리버스 엔지니어링의 산업 혁명" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " "agents, verification and knowledge that compounds across projects." msgstr "어떤 매뉴얼 다시고 재구성TH08,TH095및TH04에 대해 가르쳐 에이전트,검증 및 지는 화합물에 걸쳐 프로젝트입니다." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "기사 읽기" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "여N0zoM1z0·10 월 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -96,6 +97,8 @@ msgid "" msgstr "리버스 엔지니어링에 대한 생각,실제 경험을 바탕으로 REA 그리고 인공지능 요원들도" #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -130,6 +133,7 @@ msgid "" msgstr "REA -소프트웨어가 어떻게 작동하는지 알아보십시오. 코딩 에이전트로 리버스 엔지니어링." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -150,6 +154,7 @@ msgid "REA home" msgstr "REA 홈" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -170,6 +175,7 @@ msgid "Main navigation" msgstr "주 메뉴" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -190,6 +196,7 @@ msgid "REA on GitHub" msgstr "GitHub에서 REA 보기" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -210,6 +217,7 @@ msgid "Join REA on Discord" msgstr "REA Discord 참여" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -229,7 +237,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "X에서 REA 팔로우" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "맨 위로" @@ -10852,6 +10860,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "같은 간격:256÷16=16 각도 단위=22.5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -10872,6 +10881,7 @@ msgid "Showcases" msgstr "사례" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -10892,6 +10902,7 @@ msgid "Blog" msgstr "블로그" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -10912,6 +10923,7 @@ msgid "Get started" msgstr "시작하기" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -10931,7 +10943,8 @@ msgctxt "shared" msgid "Guides" msgstr "가이드" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -10951,7 +10964,8 @@ msgctxt "shared" msgid "Source" msgstr "소스 코드" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -10971,7 +10985,8 @@ msgctxt "shared" msgid "MIT license" msgstr "MIT 라이선스" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -10991,7 +11006,8 @@ msgctxt "shared" msgid "Join the community" msgstr "커뮤니티 참여" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -11948,8 +11964,1188 @@ msgstr "Touhou 재구축 경험을 바탕으로 범용 모델, 전문 모델, #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "작성: N0zoM1z0 · 2026년 10월 13일 · 검토용 초안" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "작성자: N0zoM1z0 · 2026년 10월 13일" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "개인정보보호정책" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "{ph0}홈{ph1} / {ph2}블로그{ph3} / 모델 및 리버스 엔지니어링" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "에세이" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "리버스 엔지니어링에 더 큰 모델이 필요할까?" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "작성자: N0zoM1z0 · 2026년 10월 13일" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "핵심 요점" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}모델 기능은 AI 에이전트가 제안할 수 있는 가설에 영향을 미칩니다.{ph1} 이후 세션에서는 이전 작업에서 해결되지 않은 " +"문제를 해결했습니다." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}익숙한 컴파일러는 특화 모델이 학습할 수 있는 구체적인 패턴을 제공합니다.{ph1} 일부 정확한 기계 코드 일치는 재사용 가능한" +" 특정 소스 코드 패턴에 따라 달라집니다." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}공정한 비교에 저장소 지식을 포함합니다.{ph1} 지식 검색이 좋은 소규모 모델은 이미 진행하기에 충분한 정보를 가지고 있을 수" +" 있습니다." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}다음 유용한 실험을 선택하도록 모델을 훈련시키고 검증 검사기를 테스트합니다.{ph1} 피드백은 검사기가 올바른 것을 검증하는 " +"정도까지만 유용합니다." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}시간 및 비용과 함께 검증된 재구성 결과를 평가합니다.{ph1} 이것이 제가 전체 AI 에이전트 워크플로에 대해 측정하고 싶은 " +"것입니다." + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "이 기사 에서" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}더 강한 모델이 바꾼 것{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}전문화해야 하는 이유{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}이 경험은 어디에 저장되어야 합니까?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}다음 유용한 실험 학습{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}리버스 엔지니어링 및 강화 학습{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}검증체커가 잘못된 경우{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}실행할 가치가 있는 비교{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}어떤 에이전트가 필요할까?{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "더 강한 모델이 바꾼 것" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"2026년 8월부터, 프로그램에 접근하고 자신의 작업을 검증할 수 있는 에이전트가 리버스 엔지니어링을 얼마나 수행할 수 있는지 탐구해 " +"왔다. 내 경험은 주로 게임, 특히 Touhou 재구성에서 나온다. 모델을 바꿔 가며 작업하는 동안 저장소에는 각 세션이 무엇을 " +"복원했는지 기록이 쌓였다." + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"이 작업의 상당 부분은 정확한 재구성을 목표로 한다. 버전을 고정한 당시의 툴체인으로 유지 관리 중인 소스 코드를 컴파일하고, 대상 " +"함수가 재현되는지 확인한다. 이전에 막혔던 문제를 나중 모델이 해결하면, 소스 변경을 통해 무엇이 진전을 가능하게 했는지 살펴볼 수 " +"있다." + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "임시 변수에는 수명이 겹치는 것이 필요했습니다." + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"TH09 공격 함수의 논리 대부분은 이미 재구성되었지만 컴파일러는 여전히 원본과 다르게 임시 결과를 저장했습니다. 남은 문제는 해당 " +"결과를 유지하는 데 필요한 기간과 해당 저장소를 언제 재사용할 수 있는지였습니다." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"여러 가지 일반적인 소스 코드 배열을 테스트한 후 {ph0}GPT-6-Sol 세션 레코드{ph1}에서 이러한 불일치를 보고했습니다. " +"{ph2}GPT-6.1-Sol이 문제를 다시 조사했을 때{ph3} 이러한 임시 변수의 수명에 대한 유용한 가설을 제안했습니다." + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"이 함수는 임시 결과를 생성하고 이를 애니메이션 위치에 복사했습니다. C++에서 임시 결과는 이를 사용하는 표현식이 완료될 때까지 활성 " +"상태로 유지됩니다. 실제 작업을 하나의 표현식으로 그룹화하면 해당 결과가 함께 유지되므로 컴파일러는 더 이상 모든 작업에 대해 동일한 " +"저장소를 재사용할 수 없습니다." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"한 결과는 나중에 충돌 확인을 위해 사용되는 값으로 여전히 저장 공간을 공유합니다. 해당 값에 고유한 수명을 부여하면 나머지 중복이 " +"해결되었습니다. 여러 번의 대상 실험 후 {ph0}소스에서 깔끔하게 재컴파일하여 전체 함수가 {ph1}과 일치함을 확인했습니다." + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"나는 발견과 테스트가 어떻게 서로를 지원하는지에 관심이 있습니다. 임시 개체 수명에 대한 가설은 AI 에이전트에 새로운 방향을 " +"제시했습니다. 컴파일러 피드백은 해당 가설을 일치하는 소스 코드 배열로 구체화하는 데 도움이 되었습니다." + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "다른 제어 흐름 그래프" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"두 번째 TH09 함수는 새로운 방향이 중요한 이유를 보여줍니다. 여기서 장애물은 여러 지점이 최종 작업을 공유하는 방식이었습니다. " +"{ph0}GPT-6.1-Sol의 이전 작업{ph1}에서는 일치하는 함수의 일부를 얻었지만 분기는 여전히 원본과 다르게 결합되었습니다." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra이 주변 제어 흐름 그래프{ph1}를 변경했습니다. 이제 주요 경로는 공유된 최종 업데이트 및 복귀로 " +"이어졌습니다. 해당 소스 구조에서 컴파일러는 원본과 동일한 방식으로 분기를 결합했습니다. 전체 함수이 일치했으며 {ph2}별도의 재구축 " +"검사를 통해 이를 확인했습니다{ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}표현을 변경하면 보다 생산적인 검색 공간이 열렸습니다.{ph1} 두 경우 모두 일반 모델 기능을 진지하게 받아들여야 하는 이유가" +" 됩니다. 에이전트는 주변 작업을 표현하는 더 나은 방법을 찾기 전에 오랫동안 지역 변형을 계속 시도할 수 있습니다." + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"다만 이후 세션은 이전 소스 코드와 실패 기록을 이어받았다. 그 축적도 이점의 일부였다. 이력에서 발견과 검증 과정을 확인할 수는 " +"있지만, 모델 자체의 효과와 물려받은 경험의 효과를 구분하려면 조건을 맞춘 비교가 필요하다." + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"이것이 나에게 그 질문을 흥미롭게 만드는 이유이다. 새로운 표현을 찾는 과정에서 약간의 진전이 있었습니다. 나머지 작업의 대부분은 익숙한" +" 컴파일러에 대해 다양한 소스 코드 배열을 테스트하는 것과 관련되었습니다. 이 중 얼마나 폭넓은 추론 능력이 필요한가요? 해당 컴파일러와" +" 코드를 반복적으로 사용하여 모델이 얼마나 학습할 수 있습니까?" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "특화된 AI 모델이 왜 좋은 성과를 낼 수 있을까요?" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou은 그 경험이 도움이 될 것으로 기대하는 구체적인 이유를 제공합니다. 대부분의 게임 코드는 동일한 작성자인 ZUN에서 " +"제공되므로 반복되는 습관은 합리적인 시작 가정입니다. 저장소는 유용한 예를 통해 이를 뒷받침합니다. 관련 루프 및 도우미 배열은 게임 " +"전반에 걸쳐 가설을 제공했습니다." + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"이러한 소스 패턴은 컴파일러를 포함하면 더욱 구체적이 됩니다. 특정 버전 및 옵션 세트에서는 소스의 작은 변경으로 인해 기계어 코드에 " +"반복 가능한 차이가 발생할 수 있습니다. 이러한 차이점 중 일부는 놀라울 정도로 모호합니다." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"예를 들어 TH095에서는 {ph0}로컬 변수의 이름도 VC7.1이 변수를 저장한 위치에 영향을 미칠 수 있습니다{ph1}. 선언을 " +"이동해도 스토리지 레이아웃이 해결되지 않았습니다. 그러나 이전 백그라운드 함수에서는 컴파일러의 선택에 영향을 미치는 명명 패턴이 " +"드러났습니다. 이는 {ph2}{ph3}{ph4} 재구성에 대한 더 유망한 가설을 제시했습니다." + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"파일 크기 변수에 해당 패턴을 재사용하면 저장소를 추가하지 않고도 필요한 레이아웃이 생성되었습니다. 그런 다음 전체 함수이 일치했습니다." + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"이는 범용 AI 모델이 처음 문제에 직면했을 때 알 수 없는 매우 구체적인 세부 사항입니다. 이 컴파일러에 특화된 모델은 이러한 사례를 " +"많이 확인하고 먼저 테스트할 가치가 있는 소스 코드 변형을 학습할 수 있습니다." + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"우리가 이미 동작을 이해하고 있는 최종 매칭 문제에서 기회는 특히 분명합니다. 그런 다음 이 컴파일러가 관찰된 명령으로 변환하는 일반적인" +" 소스 코드 배열을 묻습니다. 전문화는 모델이 이러한 형태를 보다 효율적으로 제안하는 데 도움이 될 수 있으며, 목표는 여전히 어떤 " +"가설이 살아남는지 결정합니다." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"유용한 패턴에는 적용 조건이 함께 남아야 한다. TH08은 TH095와 TH09보다 오래된 Visual C++ 버전을 사용한다. " +"TH04은 DOS 시대의 다른 컴파일러를 사용하며, {ph0}플레이어 코드 재구성{ph1}에는 소스 패턴과 실행 파일의 링크 방식에 관한" +" 다른 가설이 필요했다. 같은 저자라는 사실은 초기 가정을 제공하지만, 그것이 적용되는지는 각 대상에서 확인해야 한다. 이런 맥락을 " +"구별하는 것도 특화 모델이 배울 일이다." + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "이 경험은 어디에 저장되어야 합니까?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"모델 훈련을 통해 이러한 패턴을 가르치기로 결정하기 전에 우리가 이미 저장소에 보관하고 있는 지식을 살펴보겠습니다. {ph0}이전 " +"에세이{ph1}에서는 그것이 프로젝트의 작업 메모리가 되는 것에 대해 썼습니다. 그 메모리에는 컴파일러 교훈과 이를 유용하게 만든 소스 " +"및 실패한 실험이 포함됩니다." + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"에이전트는 해당 기록을 검색하여 알려진 결과로 시작하고 해당 기록에 어떻게 도달했는지 검사할 수 있습니다. 또한 동일한 맥락에서 이미 " +"실패한 대안을 피할 수도 있습니다. 모델의 훈련된 매개변수를 변경하지 않고도 시작점이 향상됩니다." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"TH09의 적 생성 함수를 재구성할 때, {ph0}GPT-6-Luna-max 세션{ph1}은 TH08의 관련 소스 코드에서 가설을 " +"얻었다. 이전 코드는 루프의 구성과 내부 실패 처리에 대한 단서를 제공했다. TH09 자체의 증거로 게임별 세부 사항을 채웠고, 소스에서" +" 새로 빌드해 함수 전체의 일치를 확인했다." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}\"모델이 컴파일러의 이상한 점을 몰랐습니다\"와 \"모델이 이를 사용할 수 없었습니다\"는 다른 문제입니다.{ph1} 검색을 " +"통해 첫 번째 문제를 해결할 수 있지만 해당 사실을 유용한 후보로 바꾸는 작업은 여전히 필요합니다. 그리고 후보 소스 코드가 실패하면 " +"모델은 다음 시도에서 결과가 무엇을 의미하는지 결정해야 합니다. 관련 패턴을 아는 것은 해당 프로세스의 시작일뿐입니다." + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"그러한 이유로 저는 어떤 비교에서도 검색 성능이 좋은 더 작은 범용 AI 모델을 포함하겠습니다. 동일한 경험 라이브러리를 제공하면 교육이" +" 추가하는 것이 무엇인지 식별하는 데 도움이 됩니다. 즉, 더 나은 후보 코드 생성, 피드백 후 더 나은 결정 또는 둘 다입니다." + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "다음 유용한 실험 학습" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"피드백 이후의 결정은 제가 가장 연구하고 싶은 것입니다. 에이전트 재구성에서 모델은 설명을 제안하고 이를 프로그램에 대해 테스트합니다. " +"후보 소스 코드가 실패하더라도 그 결과는 남은 가능성을 좁히고 보다 유용한 다음 실험을 제안할 수 있습니다." + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"임시 결과 예제를 다시 고려해보세요. 최종 소스에는 제대로 작동한 배열이 표시되지만 해당 소스로의 경로에는 더 많은 정보가 포함되어 " +"있습니다. 다음 각 실험은 여전히 ​​변경해야 할 사항을 식별하는 데 도움이 되었습니다." + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "실험" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "그 결과가 우리에게 가르쳐준 것" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "결과를 복사하는 다른 방법을 시도해 보세요." + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "복사 코드는 변경되지만 기능은 여전히 다릅니다." + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "반환된 값을 직접 사용" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "복사는 원래 패턴을 따르지만 임시 저장소는 여전히 다릅니다." + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "모든 임시 변수를 동시에 유효한 상태로 유지" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "별도의 저장소를 갖지만 하나는 나중에 충돌 검사에 사용되는 값과 여전히 겹칩니다." + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "충돌 검사 값을 별도로 유지하십시오." + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "소스에서 완전히 재컴파일하면 완전한 일치가 확인됩니다." + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "위의 공격 함수 재구성에서 선택한 실험을 우리에게 가르친 내용을 기준으로 그룹화했습니다." + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"이 시퀀스를 유지하는 훈련 기록은 최종 소스보다 더 많은 것을 가르칠 수 있습니다. 이는 제안된 각 변경 사항을 관찰된 결과 및 다음 " +"결정에 연결합니다. 비효과적인 변형을 유지하는 것도 중요합니다. 이는 해당 맥락에서 어떤 가설이 이미 테스트되었는지 나중에 에이전트에게 " +"알려줍니다." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}모델이 피드백을 사용하는 방법을 배우기를 원합니다.{ph1} 중립적인 결과는 변화를 재고해야 하는 이유입니다. 변경된 불일치는 " +"다른 원인을 가리킬 수도 있습니다. 로컬 편집이 더 이상 정보를 제공하지 않으면 에이전트는 소스 표현을 다시 방문하거나 증거를 다시 " +"검사할 수 있어야 합니다." + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"그러한 선택을 하는 방법을 배우려면 자율성이 필요합니다. 때때로 유용한 다음 단계는 다른 편집 내용을 컴파일하는 대신 다른 호출자를 " +"검사하는 것입니다. 필수 재작성의 엄격한 순서는 모델이 학습하기를 원하는 동작의 일부와 함께 해당 선택을 제거합니다." + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "강화 학습에 리버스 엔지니어링이 흥미로운 이유" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"범위가 정해진 재구성은 이런 선택을 연구할 실용적인 환경을 제공한다. 대상과 컴파일러 버전을 고정하면 많은 소스 코드 실험에서 빠르고 " +"반복 가능한 피드백을 얻을 수 있다. 에이전트는 후보 코드를 컴파일하고 차이를 살펴본 뒤 다음 단계를 선택할 수 있다." + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"강화 학습은 이런 피드백을 통해 의사 결정 과정을 훈련할 수 있다. 정의된 검증을 통과한 후보는 구체적인 결과가 되고, 시도 기록은 그 " +"결과에 도달하는 데 얼마나 많은 작업이 필요했는지 보여 준다." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"이미 이 방향을 뒷받침하는 연구가 있다. {ph0}LLM4Decompile{ph1}은 디컴파일에 특화된 모델을 연구한다. " +"{ph2}AutoDecompiler{ph3}는 컴파일과 실행 피드백으로 여러 차례 개선하는 과정을 학습한다. 이 연구들은 기능 복원에 " +"초점을 맞춘다. 나는 이를 출발점으로 삼아, 당시 컴파일러의 출력과 일치하는 소스 코드를 찾는 관련 과제를 연구하고 싶다." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"그처럼 명확한 과제를 만들려면, 먼저 함수가 어떤 리소스를 책임지는지 확인하거나 새로운 런타임 측정을 설계해야 할 수 있다. " +"TH08에서는 {ph0}충돌 판정의 차이를 추적하다가 원본 프로그램의 부동 소수점 계산 방식에 도달했다{ph1}. 에이전트는 유용한 " +"수정을 검증하기 전에 무엇을 측정할지부터 알아내야 했다." + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"이는 광범위하게 역량을 갖춘 모델의 또 다른 역할을 제시합니다. 즉, 우선 문제를 확인할 수 있도록 돕는 것입니다. 해당 검사가 존재하면" +" 학습된 검색 정책이 얼마나 멀리까지 도달할 수 있는지 조사할 수 있습니다." + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "검증체커가 잘못된 경우" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"검증 결과를 학습에 사용하면 그 검증의 범위도 학습 환경의 일부가 된다. 나는 이런 참조 기준이나 검증기를 oracle이라고 부른다. " +"반복 가능한 답이라도, 실제로 무엇을 확인한 답인지 이해해야 한다." + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08은 해당 적용 범위 외부에 얼마나 많은 것을 숨길 수 있는지 보여주었습니다. 재구성된 일부 함수는 잘못된 부동 상수와의 정확한 " +"비교를 통과했습니다. 비교기는 명령을 확인하기 전에 재배치된 주소를 조정했지만 확인을 위해 명시적으로 기록된 경우에만 해당 주소 뒤에 " +"있는 값을 확인했습니다. 다른 리터럴 참조는 해당 값을 검사하지 않고 통과할 수 있습니다." + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"이러한 격차를 해소하기 위해 {ph0}광범위한 감사{ph1}를 통해 이미 검증을 통과한 함수에서 잘못된 상수가 발견되었습니다. 한 " +"계산에서는 재구성된 소스 코드에 {ph2}가 추가된 반면 원본 코드에는 {ph3}가 추가되었습니다." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}기존 검증으로 보상하면 모델이 잘못된 재구성을 학습할 수도 있다.{ph1} 이 검증기로 RL 모델을 훈련한 것은 아니다. 실제로" +" 관찰한 실패는 검증 자체의 오류였다. 하지만 이를 보상으로 사용했다면 같은 사각지대가 학습 환경에 들어갔을 것이다." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"기능 테스트에도 비슷한 사각지대가 있을 수 있습니다. {ph0}LLM 디컴파일러에서 차등 퍼징을 사용한 연구{ph1}에서는 제공된 테스트" +" 외부의 동작 차이를 발견했습니다. 그러므로 우리는 그것이 수행되는 사례의 관점에서 합격 테스트를 이해해야 합니다. 마찬가지로 명령어 " +"일치는 확인하는 바이트와 데이터 측면에서 이해되어야 합니다." + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"어느 검증이든 보상으로 사용하기 전에는 의도적으로 틀린 결과를 넣어 시험해 보고 싶다. 최종 승인 기준도 에이전트가 수정할 수 있는 소스" +" 코드 밖에 두어야 한다. 올바르게 재구성해 검증을 통과하도록 해야 하기 때문이다. 그렇지 않으면 대상에서 복사한 바이트 배열이 유용한 " +"소스 복원 없이 단순한 채점 기준을 통과할 수 있다." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}학습 모델의 개선과 검증기의 개선은 같은 연구 문제다.{ph1} 모델이 우리가 주는 피드백을 더 잘 최적화할수록, 그 피드백 " +"자체를 이해하는 일도 중요해진다." + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "실행할 가치가 있는 비교" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"이러한 점검을 마친 후 고정된 VC7.1 작업 계열에 대한 비교를 시작합니다. 첫째, 강력한 범용 AI 모델과 더 작은 모델에 대한 " +"기준선을 설정하겠습니다. 더 작은 모델에 저장소 지식을 추가하면 검색이 어떻게 변경되는지 확인할 수 있습니다. 거기에서 감독 전문화와 " +"RL 훈련 버전을 통해 훈련에 추가되는 것이 무엇인지 검토할 수 있습니다." + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"이러한 비교를 유익하게 만들기 위해 각 모델은 동일한 대상 증거와 도구를 받게 됩니다. 훈련별 비교를 위해 기본 모델과 사용 가능한 " +"지식도 고정된 상태로 유지합니다. 반복적인 시도는 우연히 성공한 후보 솔루션과 신뢰할 수 있는 검색 전략을 구별하는 데 도움이 됩니다." + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"그런 다음 두 가지 시작 조건을 분리합니다. 즉, 명령에서 함수를 재구성하는 것과 유지 관리되는 소스에서 마지막 몇 가지 차이점을 닫는 " +"것입니다. 특화 AI 모델은 두 번째 작업에 탁월할 수 있지만 범용 AI 모델은 두 번째 작업에 도달하는 데 필요한 많은 작업을 " +"수행합니다. 그것들을 개별적으로 측정하면 업무 분할이 드러날 것입니다." + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"두 과제 모두 학습 데이터와 검색 자료에서 제외한 테스트 사례가 필요하다. 관련 Touhou 게임은 엔진 코드를 공유하므로, 게임 하나만" +" 제외해도 거의 같은 함수가 학습 데이터에 남을 수 있다. 대신 관련 소스 코드 계열 전체를 따로 남겨 두고, 검증된 정답을 학습 " +"데이터와 검색 라이브러리 양쪽에서 제외하겠다." + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"측정할 결과는 완전하고 독립적으로 검증된 일치다. 후보 제안 수나 컴파일 횟수의 예산을 같게 하면 탐색 품질을 비교할 수 있다. 컴파일을" +" 포함한 총 시간과 비용은 그 품질이 효율적인 워크플로로 이어지는지 보여 준다. 학습 비용은 별도로 계산해야 한다." + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"두 측정 모두 중요합니다. 유용한 정보를 제공하지 않는 컴파일이 더 많이 발생하면 값싼 제안으로 인해 비용이 많이 드는 작업 흐름이 " +"발생할 수 있습니다. 반대로, 유용한 실험을 더 빨리 발견하면 느린 모델이 더 경제적일 수 있습니다. 비교에는 이러한 비용이 포함되어야 " +"합니다." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"승인된 결과를 계산하면 중간 진행 상황을 관점에서 볼 수 있습니다. 하나의 TH09 {ph0}재생 저장 후보{ph1}는 컴파일된 함수의 " +"크기가 예상과 같았기 때문에 유망해 보였지만 전체 비교에서는 여전히 상당한 차이가 발견되었습니다. 컴파일러가 명령을 배열하는 방식을 " +"변경하면 근본적인 문제를 해결하지 않고도 후보가 더 가까워 보이게 만들 수도 있습니다. 이러한 진단은 다음 실험의 지침이 될 수 있지만 " +"전체 승인 확인에 대해 \"가까워지는\" 것에 대한 보상을 검증하겠습니다." + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"나는 이러한 결과의 패턴에 관심이 있습니다. 전문화된 AI 모델이 친숙한 컴파일러 제품군 내에서 잘 작동한다면 툴체인 변경 후에도 해당 " +"장점이 유지됩니까? 두 AI 모델 모두 좋은 검색 결과를 얻은 후에도 훈련을 통해 얼마나 추가되나요? 답변은 각 모델 종류가 가장 유용한" +" 위치를 결정하는 데 도움이 됩니다." + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "어떤 에이전트가 필요할까?" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"모델이 여러 단계에서 도움이 되는 것으로 밝혀지면 특화 AI 모델에 액세스할 수 있는 광범위한 에이전트가 유용한 배열이 될 수 있습니다." +" 작업이 익숙한 컴파일러 검색이 되면 특화 AI 모델에 후보를 요청할 수 있습니다. 결과가 더 이상 이해되지 않는 경우 더 넓은 범위의 " +"에이전트는 대상 증거로 돌아가거나 측정 자체를 조사할 수 있습니다." + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"이러한 배열에서 전문화는 조사가 진행됨에 따라 에이전트가 선택할 수 있는 도구가 됩니다. 에이전트는 이전의 돌파구를 가능하게 했던 방향을" +" 자유롭게 변경할 수 있습니다." + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"이는 이전 에세이에서 설명한 산업 변화에서 자연스럽게 나온 것입니다. 더 많은 실험을 실행할 수 있게 되면 작업 구성 방식이 변경됩니다." +" 또한 어떤 결정에 폭 넓은 판단이 필요한지, 어떤 결정은 특정 기술에 대한 반복적인 경험을 통해 개선되는지 질문할 이유를 제공합니다." + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"지금까지 가장 강한 범용 모델은 내가 해결하지 못했던 문제를 풀어내며 워크플로에서 자리를 얻었다. 그 성공 때문에 작업을 더 자세히 " +"살펴보고 싶다. 얼마나 많은 부분이 모델의 폭넓은 능력에 의존했을까? 이 종류의 재구성을 전문적으로 학습한 모델은 어디까지 담당할 수 " +"있을까?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}저장소는 질문을 테스트할 수 있는 방법을 제공합니다.{ph1} 해당 기록을 사용하여 비교를 구축하고 배운 내용을 확인하고 " +"싶습니다." + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "증거 및 모델 속성" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"예제에서는 연결된 커밋에 기록된 기능 수준 비교 기록을 설명합니다. 전체 게임 런타임 확인은 별도의 마일스톤입니다. 출처 및 일치 기록은" +" 2026년 10월 13일에 검토되었습니다." + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"TH09 라벨 {ph0}은 GPT-6-Astra을 의미합니다. {ph1}과(와) 관련된 모델은 시간이 지남에 따라 변경되었으므로 " +"라벨만으로는 특정 실행에 사용된 모델을 식별할 수 없습니다. 다른 인용 커밋의 명시적인 모델 이름은 기록된 속성을 제공합니다. 이러한 " +"기록에는 모든 실행에 대한 일관된 비용 또는 노력 측정이 포함되지 않습니다." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 임시 객체 수명:{ph1} {ph2}이전 컨텍스트 검토{ph3} 및 {ph4}소스에서 깔끔한 재컴파일을 통해 확인된 " +"완전한 재구성{ph5}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}TH09 Front 콜백:{ph1} {ph2}일치하지 않는 진단 영역{ph3}, {ph4}Astra의 새로운 제어 흐름 " +"그래프{ph5} 및 {ph6}승인된 재컴파일 확인{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}컴파일러 지식:{ph1} {ph2}TH095 식별자 버킷 동작 재사용{ph3}, {ph4}TH04 플레이어 소스 코드 및 링커 " +"순서{ph5} 및 {ph6}Luna의 TH09 적 생성 기능에 대한 완전한 재구성{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}검사 및 진단:{ph1} {ph2}TH08 부동 소수점 상수 감사{ph3}, {ph4}TH08 수치 조사{ph5} 및 " +"{ph6}TH09 재생 저장 재구성{ph7}." + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"제안한 특화 모델과 RL의 비교는 앞으로 해 볼 실험이다. 프로젝트 이력이 그 동기를 제공하지만, 이 글에서 학습된 특화 모델을 평가한 " +"것은 아니다." + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "리버스 엔지니어링에 더 큰 모델이 필요할까? | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"Touhou 재구성을 통해 일반 모델, 전문 모델 및 컴파일러 피드백을 통해 더 나은 가설을 제안하는 방법에 대해 배운 내용." + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "이 페이지에" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "모델 및 리버스 엔지니어링 에세이" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "선정된 재구성 실험" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "맨 위로" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "작성: N0zoM1z0 · 2026년 10월 13일 · 검토용 초안" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/messages.pot b/website/locales/messages.pot index 92a5109c3..40fbf13cf 100644 --- a/website/locales/messages.pot +++ b/website/locales/messages.pot @@ -12,6 +12,7 @@ msgid "Skip to content" msgstr "" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 website/public/faq/index.html:113 #: website/public/first-investigation/index.html:113 @@ -29,6 +30,7 @@ msgid "Get started" msgstr "" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 website/public/faq/index.html:114 #: website/public/first-investigation/index.html:114 @@ -46,6 +48,7 @@ msgid "Showcases" msgstr "" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 website/public/faq/index.html:115 #: website/public/first-investigation/index.html:115 @@ -63,6 +66,7 @@ msgid "Blog" msgstr "" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 website/public/faq/index.html:116 #: website/public/first-investigation/index.html:116 @@ -116,41 +120,42 @@ msgid "" "learning from compiler feedback." msgstr "" -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +msgid "By N0zoM1z0 · 13 October 2026" msgstr "" #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about agents, " "verification and knowledge that compounds across projects." msgstr "" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 website/public/faq/index.html:565 #: website/public/first-investigation/index.html:515 @@ -167,7 +172,8 @@ msgctxt "shared" msgid "Source" msgstr "" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 website/public/faq/index.html:566 #: website/public/first-investigation/index.html:516 @@ -184,7 +190,8 @@ msgctxt "shared" msgid "MIT license" msgstr "" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 website/public/faq/index.html:567 #: website/public/first-investigation/index.html:517 @@ -201,7 +208,8 @@ msgctxt "shared" msgid "Join the community" msgstr "" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 website/public/faq/index.html:568 #: website/public/first-investigation/index.html:518 @@ -220,7 +228,8 @@ msgstr "" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 website/public/faq/index.html:573 #: website/public/first-investigation/index.html:523 @@ -251,6 +260,8 @@ msgid "" msgstr "" #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -277,6 +288,7 @@ msgid "REA — Find out how software works. Reverse engineering with your coding msgstr "" #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 website/public/faq/index.html:108 #: website/public/first-investigation/index.html:108 @@ -294,6 +306,7 @@ msgid "REA home" msgstr "" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 website/public/faq/index.html:112 #: website/public/first-investigation/index.html:112 @@ -311,6 +324,7 @@ msgid "Main navigation" msgstr "" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 website/public/faq/index.html:130 #: website/public/first-investigation/index.html:130 @@ -328,6 +342,7 @@ msgid "REA on GitHub" msgstr "" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 website/public/faq/index.html:150 #: website/public/first-investigation/index.html:150 @@ -345,6 +360,7 @@ msgid "Join REA on Discord" msgstr "" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 website/public/faq/index.html:169 #: website/public/first-investigation/index.html:169 @@ -361,11 +377,994 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "" +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later sessions " +"have solved problems that earlier work left unresolved." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to learn.{ph1} Some " +"exact matches depend on narrow, reusable source patterns." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with good " +"retrieval may already know enough to make progress." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value of feedback" +" depends on what the checker actually verifies." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the result I " +"want from the whole agent." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an agent can " +"carry out with access to a program and a way to check its work. Most of my experience " +"comes from reconstructing games, especially Touhou. As I have changed models, the " +"repositories have kept a record of what each session managed to recover." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained source under" +" a pinned historical toolchain and check whether it reproduces the target function. " +"When a later model succeeds where earlier work stalled, the source changes show what " +"made that progress possible." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but the " +"compiler still stored temporary results differently from the original. The remaining " +"problem was how long those results needed to stay alive and when their storage could " +"be reused." +msgstr "" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing several ordinary" +" source arrangements. When {ph2}GPT-6.1-Sol returned to the problem{ph3}, it found a " +"useful hypothesis about how long those temporary values needed to stay alive." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation positions. In " +"C++, a temporary result stays alive until the expression using it finishes. Grouping " +"the real operations into one expression kept those results alive together, so the " +"compiler could no longer reuse the same storage for all of them." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision checks. Giving " +"that value its own lifetime resolved the remaining overlap. After several targeted " +"experiments, {ph0}a clean rebuild confirmed that the full function matched{ph1}." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The lifetime " +"hypothesis gave the agent a new direction; compiler feedback helped it refine that " +"hypothesis into a source arrangement that matched." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the obstacle was" +" how several branches shared their final work. {ph0}GPT-6.1-Sol's earlier work{ph1} " +"got parts of the function to match, but the branches still joined up differently from " +"the original." +msgstr "" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main paths now " +"led to a shared final update and return. In that source structure, the compiler " +"combined the branches in the same way as the original. The complete function matched, " +"and a {ph2}separate rebuild check confirmed it{ph3}." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1} Both " +"cases give me a reason to take general model capability seriously: an agent can keep " +"trying local variations for a long time before finding a better way to express the " +"surrounding work." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and recorded " +"failures. That accumulated work was part of their advantage. The history lets us " +"inspect the discoveries and their checks; a controlled comparison would help separate " +"the effect of the model from the experience it inherited." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress came from " +"finding a new representation, while much of the remaining work involved testing source" +" forms against a familiar compiler. How much needs broad reasoning ability, and how " +"much could a model learn through repeated experience with that compiler and code?" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of the game " +"code comes from the same author, ZUN, so recurring habits are a reasonable starting " +"assumption. The repositories back that up with useful examples: related loops and " +"helper arrangements have supplied hypotheses across games." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. Under a " +"particular version and set of options, small changes in source can produce repeatable " +"differences in machine code. Some of those differences are surprisingly obscure." +msgstr "" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could change where" +" VC7.1 stored it{ph1}. Moving the declaration alone did not solve the layout, but an " +"earlier background function had revealed a naming pattern that affected the compiler's" +" choice. That gave us a more promising hypothesis for {ph2}the reconstruction of " +"{ph3}{ph4}." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout without " +"adding storage. The full function then matched." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it arrives. A " +"specialist trained on this compiler could encounter many such cases and learn which " +"source variations are worth testing first." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we already " +"understand the behavior. We are then asking which ordinary source form this compiler " +"turns into the observed instructions. Specialization could help the model propose " +"those forms more efficiently, while the target still determines which hypotheses " +"survive." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an earlier " +"Visual C++ version than TH095 and TH09. TH04 uses a different DOS-era compiler, and " +"its {ph0}player reconstruction{ph1} needed different hypotheses about source patterns " +"and how the compiled code was combined into an executable. Shared authorship gives us " +"a prior, but each target tells us whether it applies. Distinguishing those contexts is" +" part of what a specialist would need to learn." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look at the " +"knowledge we already keep in the repository. In the {ph0}previous essay{ph1}, I wrote " +"about it becoming the project's working memory. That memory includes compiler lessons," +" along with the source and failed experiments that made them useful." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and inspect how we " +"reached it. It can also avoid alternatives that have already failed in the same " +"context. The starting point improves without changing the model's weights." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max session{ph1} " +"drew a hypothesis from TH08's corresponding source family. That earlier source " +"suggested how to structure the loop and handle failures within it. TH09's own evidence" +" supplied the game-specific details, and clean rebuilds confirmed the complete match." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could not use it\"" +" are different problems.{ph1} Retrieval may solve the first, but turning that fact " +"into a useful candidate still takes work. And if the candidate fails, the model must " +"decide what the result means for its next attempt. Knowing the relevant pattern is " +"only the beginning of that process." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval in any " +"comparison. Giving it the same library of experience would help us identify what " +"training adds: better candidate generation, better decisions after feedback, or both." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In agentic " +"reconstruction, the model proposes an explanation and tests it against the program. " +"Even when the candidate fails, the result can narrow the remaining possibilities and " +"suggest a more useful next experiment." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the arrangement " +"that worked, but the route to that source contains more information. Each of the " +"following experiments helped identify what still needed to change." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Copying follows the original pattern, but temporary storage still differs." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for collision " +"checks." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped by what " +"they taught us." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final source alone." +" It connects each proposed change to an observed result and the next decision. Keeping" +" ineffective variations matters too: they tell a later agent which hypotheses have " +"already been tested in that context." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral result is a " +"reason to reconsider the variation; a changed mismatch may point to another cause. " +"When local edits stop being informative, the agent should be able to revisit the " +"source representation or inspect the evidence again." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next step is " +"inspecting another caller rather than compiling another edit. A rigid sequence of " +"required rewrites would remove that choice, along with part of the behavior we want " +"the model to learn." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these choices. " +"Once the target and compiler are pinned, many source experiments produce quick, " +"repeatable feedback. The agent can compile a candidate, inspect the mismatch and use " +"that observation to choose its next step." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the feedback " +"it receives. A candidate accepted by a defined check gives us a concrete outcome, " +"while the sequence of attempts lets us observe how much work it took to reach it." +msgstr "" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1} studies " +"models specialized for decompilation. {ph2}AutoDecompiler{ph3} trains multi-turn " +"refinement using compiler and execution feedback. Their studies focus on functional " +"recovery. For this work, I would use them as a starting point for the related task of " +"finding source that matches a historical compiler's output." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish what a " +"function owns or devise a new runtime measurement. In TH08, for example, a " +"{ph0}collision difference led back to how the original program handled floating-point " +"calculations{ph1}. Before the agent could test a useful correction, it had to work out" +" what to measure." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the problem " +"checkable in the first place. Once that check exists, we can investigate how far a " +"learned search policy can get with it." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the model learns. " +"I call the reference or checker an oracle. Even if it gives a repeatable answer, we " +"still need to understand what that answer covers." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed functions " +"passed their exact comparisons with wrong floating constants. The comparator adjusted " +"relocated addresses before checking instructions, but it checked the values behind " +"those addresses only when they had been explicitly recorded for verification. Other " +"literal references could pass without their values being examined." +msgstr "" + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in functions that" +" had already been accepted. In one calculation, the reconstructed source added {ph2} " +"where the original added {ph3}." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the observed " +"failure was in the check itself. But using that check as a reward would have made the " +"same blind spot part of the learning environment." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing research on" +" LLM decompilers{ph1} has found behavior differences beyond the supplied tests. " +"Passing those tests therefore needs to be read in terms of the cases they exercise, " +"just as an instruction match needs to be read in terms of the bytes and data it " +"checks." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with deliberate " +"failures. I would also keep final acceptance outside the candidate's editable source, " +"so the model has to satisfy the check through its reconstruction. Otherwise, something" +" like a byte array copied from the target could satisfy a naive score without " +"recovering useful source." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same research " +"problem.{ph1} As the learner gets better at optimizing the feedback we give it, " +"understanding that feedback becomes more important too." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 task family." +" First, I would establish baselines for a strong general model and a smaller one. " +"Adding repository knowledge to the smaller model would show what retrieval changes. " +"From there, supervised specialization and an RL-trained version would let us examine " +"what training adds." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same target " +"evidence and tools. For the training-specific comparisons, I would keep the base model" +" and available knowledge fixed as well. Repeated trials would help distinguish a " +"reliable search strategy from a lucky candidate." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function from its " +"instructions, and closing the last few differences in maintained source. A specialist " +"might be excellent at the second task while a general model does much of the work " +"needed to reach it. Measuring them separately would reveal that division of work." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games share engine" +" code, holding out a game's name would still allow nearly identical functions to " +"appear in training. I would instead hold out source families and keep their accepted " +"answers out of both training and retrieval." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated matches. " +"Giving each model the same candidate or compile budget would help compare search " +"quality. Measuring elapsed time and total cost, including compilation, would tell us " +"whether that quality produces an efficient workflow. Training costs would need their " +"own accounting too." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive workflow if it " +"causes more uninformative compiles. Conversely, a slower model may pay for itself by " +"finding a useful experiment sooner. The comparison should capture those costs." +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. One TH09 " +"{ph0}replay-save candidate{ph1} looked promising because the compiled function had the" +" expected size, but a full comparison still found substantial differences. Changes in " +"how the compiler arranges instructions can also make a candidate look closer without " +"resolving the underlying problem. Such diagnostics can guide the next experiment, but " +"I would validate any reward for \"getting closer\" against the full acceptance check." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist does well " +"within a familiar compiler family, does that advantage survive a toolchain change? And" +" once both models have good retrieval, how much does training still add? The answers " +"would help us decide where each kind of model is most useful." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement might be a " +"broad agent with access to a specialist. Once a task becomes a familiar compiler " +"search, it could ask the specialist for candidates. If the results stop making sense, " +"the broader agent could return to the target evidence or investigate the measurement " +"itself." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as the " +"investigation develops. The agent keeps the freedom to change direction that made the " +"earlier breakthroughs possible." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous essay. " +"Being able to run many more experiments changes how we organize the work. It also " +"gives us a reason to ask which decisions need broad judgment and which improve through" +" repeated experience with a particular craft." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by opening " +"problems I had left unresolved. Those successes are what make me want to examine the " +"work more closely: how much depended on the model's breadth, and how much could be " +"handled by a model trained in this particular kind of reconstruction?" +msgstr "" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like to use " +"that record to build the comparison and see what we learn." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the linked " +"commits. Whole-game runtime checks are separate milestones. Source and matching " +"records were reviewed on 13 October 2026." +msgstr "" + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model used at the " +"time, so that label alone does not identify an individual run. Explicit model names in" +" the other cited commits establish their recorded attribution; the histories do not " +"contain uniform per-run cost or effort measurements." +msgstr "" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and " +"{ph4}complete cold reconstruction{ph5}." +msgstr "" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, {ph4}Astra's " +"new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, {ph4}TH04 " +"player source and linker order{ph5} and {ph6}Luna's TH09 enemy-spawn closure{ph7}." +msgstr "" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, {ph4}TH08 " +"numerical investigation{ph5} and {ph6}TH09 replay-save reconstruction{ph7}." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The project " +"histories motivate them; no trained specialist was evaluated for this essay." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models and " +"learning to propose better hypotheses from compiler feedback." +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "" + #: website/public/blog/touhou-reconstruction/index.html:103 msgctxt "page:blog/touhou-reconstruction/index.html" msgid "Skip to content" diff --git a/website/locales/pl/messages.po b/website/locales/pl/messages.po index e4bdee584..60f0003ec 100644 --- a/website/locales/pl/messages.po +++ b/website/locales/pl/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Myśli o inżynierii odwrotnej." #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Rekonstrukcja gry{ph0}Esej" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "Rewolucja przemysłowa w inżynierii odwrotnej" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,20 @@ msgstr "" "Co Instrukcja RE i rekonstrukcji TH08, TH095 i TH04 nauczył mnie O agentów, " "weryfikacja i wiedza, że związki całej projektów." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Przeczytaj artykuł" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Przez N0zoM1z0 * 10 października 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -100,6 +101,8 @@ msgstr "" "doświadczenia z agentami REA i AI." #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -136,6 +139,7 @@ msgstr "" "agentem kodującym." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -156,6 +160,7 @@ msgid "REA home" msgstr "Strona główna REA" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -176,6 +181,7 @@ msgid "Main navigation" msgstr "Główna nawigacja" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -196,6 +202,7 @@ msgid "REA on GitHub" msgstr "REA na GitHub" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -216,6 +223,7 @@ msgid "Join REA on Discord" msgstr "Dołącz do REA na Discordzie" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -235,7 +243,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Obserwuj REA na X" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Do góry" @@ -11618,6 +11626,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "Ten sam odstęp: 256 ÷ 16 = 16 jednostek kąta = 22,5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -11638,6 +11647,7 @@ msgid "Showcases" msgstr "Przykłady" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -11658,6 +11668,7 @@ msgid "Blog" msgstr "Blog" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -11678,6 +11689,7 @@ msgid "Get started" msgstr "Pierwsze kroki" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -11697,7 +11709,8 @@ msgctxt "shared" msgid "Guides" msgstr "Poradniki" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11717,7 +11730,8 @@ msgctxt "shared" msgid "Source" msgstr "Kod źródłowy" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11737,7 +11751,8 @@ msgctxt "shared" msgid "MIT license" msgstr "Licencja MIT" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11757,7 +11772,8 @@ msgctxt "shared" msgid "Join the community" msgstr "Dołącz do społeczności" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12746,8 +12762,1346 @@ msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "Autor: N0zoM1z0 · 13 października 2026 · Szkic do przejrzenia" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Przez N0zoM1z0 · 13 października 2026 r" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "Przejdź do treści" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "" +"{ph0}Strona główna{ph1} / {ph2}Blog{ph3} / Modele i inżynieria odwrotna" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "Esej" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Czy inżynieria odwrotna potrzebuje większego modelu?" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Przez N0zoM1z0 · 13 października 2026 r" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "Najważniejsze" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}Możliwości modelu wpływają na to, jakie hipotezy może postawić agent " +"AI.{ph1} Późniejsze sesje rozwiązały problemy, które nie zostały rozwiązane " +"wcześniej." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}Znany kompilator oferuje konkretne wzorce do nauki dla " +"wyspecjalizowanego modelu.{ph1} Niektóre dokładne dopasowania kodu " +"maszynowego zależą od konkretnych wzorców kodu źródłowego, które można " +"ponownie wykorzystać." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}W rzetelnym porównaniu uwzględnij wiedzę o repozytorium.{ph1} Mniejszy " +"model z dobrym wyszukiwaniem wiedzy może już mieć wystarczającą ilość " +"informacji, aby poczynić postępy." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}Wytrenuj model, aby wybrał kolejny przydatny eksperyment i przetestuj " +"jego moduł weryfikujący.{ph1} Opinia jest użyteczna tylko w takim stopniu, w" +" jakim moduł sprawdzający weryfikuje właściwe rzeczy." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}Oceń zweryfikowane wyniki rekonstrukcji pod względem czasu i " +"kosztów.{ph1} Właśnie to chcę zmierzyć w przypadku pełnego przepływu pracy " +"agenta AI." + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "W tym artykule" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}Co zmieniły silniejsze modele{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}Dlaczego się specjalizować?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}Gdzie należy przechowywać to doświadczenie?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}Nauka kolejnego przydatnego eksperymentu{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}Inżynieria odwrotna i uczenie się przez wzmacnianie{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}Gdy osoba sprawdzająca myli się{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}Porównanie warte przeprowadzenia{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}Jakiego rodzaju agenta AI to sugeruje?{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "Co zmieniły silniejsze modele" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"Od sierpnia 2026 roku badam, ile inżynierii wstecznej może dokonać agent " +"mając dostęp do programu i możliwość sprawdzenia jego działania. Większość " +"mojego doświadczenia pochodzi z rekonstrukcji gier, szczególnie Touhou. " +"Ponieważ zmieniałem modele, repozytoria prowadziły rejestr tego, co udało " +"się odzyskać w każdej sesji." + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"Duża część tych prac ma na celu dokładną rekonstrukcję: kompilujemy " +"utrzymywane źródło w ramach historycznego zestawu narzędzi ze stałymi " +"wersjami i sprawdzamy, czy odtwarza ono funkcję docelową. Kiedy późniejszy " +"model odniesie sukces tam, gdzie wcześniejsze prace utknęły w martwym " +"punkcie, zmiany źródłowe pokazują, co umożliwiło ten postęp." + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "Zmienne tymczasowe wymagały nakładających się okresów istnienia" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"Duża część logiki funkcji ataku TH09 została już zrekonstruowana, ale " +"kompilator nadal przechowywał tymczasowe wyniki inaczej niż oryginał. " +"Pozostałym problemem było to, jak długo wyniki były potrzebne do zachowania " +"żywotności i kiedy można było ponownie wykorzystać ich pamięć." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"{ph0}GPT-6-Sol rekord sesji{ph1} zgłosił tę niezgodność po przetestowaniu " +"kilku zwykłych ustaleń kodu źródłowego. Kiedy {ph2}GPT-6.1-Sol ponownie " +"zbadał problem{ph3}, zaproponował użyteczną hipotezę dotyczącą czasu życia " +"tych zmiennych tymczasowych." + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"Funkcja wygenerowała tymczasowe wyniki i skopiowała je do pozycji animacji. " +"W C++ tymczasowy wynik pozostaje aktywny aż do zakończenia wyrażenia, które " +"go używa. Grupowanie rzeczywistych operacji w jedno wyrażenie utrzymywało te" +" wyniki razem, więc kompilator nie mógł już ponownie używać tej samej " +"pamięci dla nich wszystkich." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"Jeden wynik nadal współdzielił pamięć o wartości używanej później do " +"sprawdzania kolizji. Nadanie tej wartości własnego czasu życia rozwiązało " +"pozostałe nakładanie się. Po kilku ukierunkowanych eksperymentach " +"{ph0}czysta rekompilacja ze źródła potwierdziła, że ​​pełna funkcja jest " +"zgodna{ph1}." + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"Ciekawi mnie, jak odkrycie i testy wzajemnie się wspierały. Hipoteza " +"dotycząca czasu życia obiektów tymczasowych nadała agentowi AI nowy " +"kierunek. Informacje zwrotne od kompilatora pomogły mu udoskonalić tę " +"hipotezę w celu uzyskania pasującego układu kodu źródłowego." + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "Inny wykres przepływu sterowania" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"Druga funkcja TH09 pokazuje, dlaczego ten nowy kierunek może mieć znaczenie." +" Tutaj przeszkodą był sposób, w jaki kilka oddziałów podzieliło się swoją " +"końcową pracą. We wcześniejszej pracy {ph0}GPT-6.1-Sol{ph1} części funkcji " +"pasowały, ale gałęzie nadal łączyły się inaczej niż w oryginale." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra zmienił otaczający wykres przepływu sterowania{ph1}. Główne" +" ścieżki prowadziły teraz do wspólnej ostatecznej aktualizacji i powrotu. W " +"tej strukturze źródłowej kompilator połączył gałęzie w taki sam sposób jak " +"oryginał. Pełna funkcja została zgodna i {ph2}potwierdziła to osobna " +"kontrola odbudowy{ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}Zmiana reprezentacji otworzyła bardziej produktywną przestrzeń " +"poszukiwań.{ph1} Obydwa przypadki dają mi powód, aby poważnie traktować " +"możliwości modelu ogólnego: agent może przez długi czas testować lokalne " +"odmiany, zanim znajdzie lepszy sposób na wyrażenie otaczającej pracy." + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"Jednocześnie późniejsze sesje rozpoczynały się od wcześniejszych źródeł i " +"odnotowanych awarii. Ta skumulowana praca była częścią ich przewagi. " +"Historia pozwala nam na wgląd w odkrycia i ich sprawdzenie; kontrolowane " +"porównanie pomogłoby oddzielić wpływ modelu od odziedziczonego przez niego " +"doświadczenia." + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"Właśnie dlatego to pytanie jest dla mnie interesujące. Pewien postęp " +"nastąpił po znalezieniu nowej reprezentacji. Większość pozostałej pracy " +"polegała na testowaniu różnych rozwiązań kodu źródłowego w porównaniu ze " +"znanym kompilatorem. Ile z tego wymaga szerokiej zdolności rozumowania? Ile " +"model może się nauczyć, wielokrotnie pracując z tym kompilatorem i kodem?" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "" +"Dlaczego wyspecjalizowany model sztucznej inteligencji może się sprawdzić?" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou daje nam konkretny powód, aby oczekiwać, że to doświadczenie pomoże. " +"Duża część kodu gry pochodzi od tego samego autora, ZUN, więc powtarzające " +"się nawyki są rozsądnym założeniem wyjściowym. Repozytoria potwierdzają to " +"przydatnymi przykładami: powiązane pętle i układy pomocnicze dostarczyły " +"hipotez w różnych grach." + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"Te wzorce źródłowe stają się bardziej szczegółowe, gdy dołączymy kompilator." +" W przypadku określonej wersji i zestawu opcji niewielkie zmiany w źródle " +"mogą powodować powtarzalne różnice w kodzie maszynowym. Niektóre z tych " +"różnic są zaskakująco niejasne." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"Na przykład w TH095 {ph0}nawet nazwa zmiennej lokalnej może mieć wpływ na " +"miejsce przechowywania jej przez VC7.1{ph1}. Przeniesienie deklaracji nie " +"rozwiązało układu przechowywania. Jednak wcześniejsza funkcja tła ujawniła " +"wzorzec nazewnictwa, który wpłynął na wybór kompilatora. Sugerowało to " +"bardziej obiecującą hipotezę dotyczącą {ph2}rekonstrukcji {ph3}{ph4}." + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"Ponowne użycie tego wzorca dla zmiennej rozmiaru pliku pozwoliło uzyskać " +"wymagany układ bez dodawania pamięci. Pełna funkcja została wówczas " +"dopasowana." + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"Jest to bardzo specyficzny szczegół, o którym model sztucznej inteligencji " +"ogólnego przeznaczenia może nie wiedzieć, kiedy po raz pierwszy napotka " +"problem. Model wyspecjalizowany dla tego kompilatora mógłby zobaczyć wiele " +"takich przypadków i dowiedzieć się, które odmiany kodu źródłowego warto " +"przetestować w pierwszej kolejności." + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"Możliwość jest szczególnie wyraźna w przypadku końcowych problemów z " +"dopasowywaniem, w których już rozumiemy zachowanie. Następnie pytamy, jaki " +"zwykły układ kodu źródłowego ten kompilator przekształca w obserwowane " +"instrukcje. Specjalizacja może pomóc modelowi w skuteczniejszym proponowaniu" +" tych form, podczas gdy cel nadal określa, które hipotezy przetrwają." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"Przydatne wzorce muszą zachować swój kontekst. TH08 używa wcześniejszej " +"wersji Visual C++ niż TH095 i TH09. TH04 używa innego kompilatora z ery " +"DOS-u. Do jego {ph0}rekonstrukcji kodu gracza{ph1} potrzebne były różne " +"hipotezy dotyczące wzorców kodu źródłowego i sposobu łączenia skompilowanego" +" kodu z plikiem wykonywalnym. Posiadanie tego samego autora daje nam wstępne" +" oczekiwania, ale każdy cel mówi nam, czy ma to zastosowanie. Aby rozróżnić " +"te konteksty, potrzebny byłby wyspecjalizowany model." + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "Gdzie należy przechowywać to doświadczenie?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"Zanim zdecydowałbym się uczyć tych wzorców poprzez szkolenie modelowe, " +"sprawdziłbym wiedzę, którą już przechowujemy w repozytorium. W " +"{ph0}poprzednim eseju{ph1} pisałem, że stanie się ona pamięcią roboczą " +"projektu. Pamięć ta obejmuje lekcje kompilatora wraz ze źródłem i nieudanymi" +" eksperymentami, które uczyniły je użytecznymi." + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"Pobierając ten rekord, agent może zacząć od znanego wyniku i sprawdzić, w " +"jaki sposób do niego dotarliśmy. Można również uniknąć alternatyw, które już" +" zawiodły w tym samym kontekście. Punkt początkowy poprawia się bez zmiany " +"przeszkolonych parametrów modelu." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"Podczas rekonstrukcji funkcji TH09 służącej do tworzenia wrogów, podczas " +"{ph0}GPT-6-Luna-max sesji{ph1} przyjęto hipotezę z powiązanego kodu " +"źródłowego TH08. Ten wcześniejszy kod sugerował, jak zorganizować pętlę i " +"poradzić sobie z błędami w niej występującymi. Zeznania TH09 dostarczyły " +"szczegółów specyficznych dla gry. Przebudowa ze źródła potwierdziła, że " +"​​cała funkcja jest zgodna." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}„Model nie wiedział, że dziwactwo kompilatora” i „model nie mógł go " +"użyć” to różne problemy.{ph1} Pobranie może rozwiązać pierwszy, ale " +"przekształcenie tego faktu w użyteczną kandydaturę nadal wymaga pracy. A " +"jeśli potencjalny kod źródłowy zawiedzie, model musi zdecydować, co oznacza " +"wynik przy następnej próbie. Znajomość odpowiedniego wzorca to dopiero " +"początek tego procesu." + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"Z tego powodu do każdego porównania uwzględniłbym mniejszy model sztucznej " +"inteligencji ogólnego przeznaczenia z dobrym wyszukiwaniem. Zapewnienie tej " +"samej biblioteki doświadczeń pomogłoby nam określić, co wnosi szkolenie: " +"lepsze generowanie kandydatów, lepsze decyzje po otrzymaniu informacji " +"zwrotnej lub jedno i drugie." + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "Nauka kolejnego przydatnego eksperymentu" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"Najbardziej chciałbym się uczyć tych decyzji po otrzymaniu informacji " +"zwrotnej. W rekonstrukcji agentowej model proponuje wyjaśnienie i testuje je" +" w porównaniu z programem. Nawet jeśli potencjalny kod źródłowy zawiedzie, " +"wynik może zawęzić pozostałe możliwości i zasugerować bardziej przydatny " +"następny eksperyment." + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"Rozważmy ponownie przykład wyników tymczasowych. Ostateczne źródło pokazuje " +"układ, który zadziałał, ale trasa do tego źródła zawiera więcej informacji. " +"Każdy z poniższych eksperymentów pomógł określić, co należy jeszcze zmienić." + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "Eksperyment" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "Czego nauczył nas jego wynik" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "Wypróbuj inny sposób kopiowania wyników" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "Kod kopiowania zmienia się, ale funkcja nadal jest inna." + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "Użyj bezpośrednio zwróconych wartości" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "" +"Kopiowanie odbywa się według oryginalnego wzorca, ale przechowywanie " +"tymczasowe nadal się różni." + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "Zachowaj ważność wszystkich zmiennych tymczasowych w tym samym czasie" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "" +"Dostają oddzielną pamięć, ale jedna z nich nadal pokrywa się z wartością " +"używaną później do sprawdzania kolizji." + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "Zachowaj wartość kontroli kolizji oddzielnie" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "Czysta rekompilacja ze źródła potwierdza pełne dopasowanie." + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "" +"Wybrane eksperymenty z powyższej rekonstrukcji funkcji ataku, pogrupowane " +"według tego, czego nas nauczyły." + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"Zapis szkoleniowy zachowujący tę sekwencję może nauczyć więcej niż samo " +"ostateczne źródło. Łączy każdą proponowaną zmianę z zaobserwowanym " +"rezultatem i następną decyzją. Utrzymywanie nieefektywnych odmian również ma" +" znaczenie: informują późniejszego agenta, które hipotezy zostały już " +"przetestowane w tym kontekście." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}Chciałbym, żeby model nauczył się korzystać ze informacji " +"zwrotnej.{ph1} Neutralny wynik jest powodem do ponownego rozważenia odmiany;" +" zmieniona niezgodność może wskazywać na inną przyczynę. Kiedy lokalne " +"zmiany przestaną mieć charakter informacyjny, agent powinien mieć możliwość " +"ponownego przejrzenia reprezentacji źródłowej lub ponownego sprawdzenia " +"dowodów." + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"Nauka dokonywania takich wyborów wymaga autonomii. Czasami przydatnym " +"następnym krokiem jest sprawdzenie innego obiektu wywołującego, zamiast " +"kompilowania kolejnej edycji. Sztywna sekwencja wymaganych przeróbek usunie " +"ten wybór wraz z częścią zachowań, których ma się nauczyć model." + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "" +"Dlaczego inżynieria odwrotna jest interesująca w uczeniu się przez " +"wzmacnianie" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"Rekonstrukcja o ustalonym zakresie daje praktyczne środowisko do badania " +"tych decyzji. Po ustaleniu wersji celu i kompilatora wiele eksperymentów z " +"kodem źródłowym zapewnia szybką, powtarzalną informację zwrotną. Agent może " +"skompilować kandydata, sprawdzić różnice i wybrać następny krok." + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"Uczenie się przez wzmacnianie mogłoby szkolić proces podejmowania decyzji na" +" podstawie otrzymywanych informacji zwrotnych. Kandydat przyjęty przez " +"określony sprawdzian daje nam konkretny wynik, a sekwencja prób pozwala " +"zaobserwować, ile pracy wymagało jego osiągnięcie." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"Istnieją już badania potwierdzające ten kierunek. {ph0}LLM4Decompile{ph1} " +"bada modele wyspecjalizowane w dekompilacji. {ph2}AutoDecompiler{ph3} " +"trenuje udoskonalanie w wielu turach, korzystając z informacji zwrotnych od " +"kompilatora i wykonania. Ich badania skupiają się na odzyskaniu funkcji. W " +"tej pracy użyłbym ich jako punktu wyjścia do powiązanego zadania " +"polegającego na znalezieniu źródła, które pasuje do danych wyjściowych " +"historycznego kompilatora." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"Aby jednak osiągnąć tak dobrze zdefiniowane zadanie, być może będziemy " +"musieli najpierw ustalić, jakie zasoby posiada funkcja, lub opracować nowy " +"sposób pomiaru czasu wykonania. Na przykład w TH08 {ph0}różnica kolizji " +"doprowadziła do tego, jak oryginalny program obsługiwał obliczenia " +"zmiennoprzecinkowe{ph1}. Zanim agent mógł przetestować użyteczną korektę, " +"musiał ustalić, co należy zmierzyć." + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"Sugeruje to inną rolę modelu o szerokich możliwościach: przede wszystkim " +"pomoc w sprawdzeniu problemu. Gdy już to sprawdzimy, będziemy mogli " +"sprawdzić, jak daleko może zajść w tym przypadku wyuczona polityka " +"wyszukiwania." + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "Gdy osoba sprawdzająca myli się" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"Gdy używamy sprawdzania podczas uczenia, jego zakres staje się częścią " +"środowiska uczenia. Taki wzorzec lub moduł sprawdzający nazywam wyrocznią. " +"Nawet powtarzalną odpowiedź trzeba rozumieć w granicach tego, co " +"rzeczywiście sprawdza." + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 pokazał, jak wiele może ukryć się poza tym zasięgiem. Niektóre " +"zrekonstruowane funkcje przeszły dokładne porównania z nieprawidłowymi " +"stałymi zmiennoprzecinkowymi. Komparator korygował przeniesione adresy przed" +" sprawdzeniem instrukcji, ale sprawdzał wartości za tymi adresami tylko " +"wtedy, gdy zostały one wyraźnie zapisane w celu weryfikacji. Inne dosłowne " +"odniesienia można pominąć bez sprawdzania ich wartości." + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"Aby zaradzić tej luce, {ph0}szerszy audyt{ph1} wykrył nieprawidłowe stałe w " +"funkcjach, które przeszły już weryfikację. W jednym obliczeniu do " +"zrekonstruowanego kodu źródłowego dodano {ph2}, podczas gdy w oryginale " +"dodano {ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}Model nagrodzony starym testem weryfikacyjnym mógłby nauczyć się " +"tworzyć niepoprawną rekonstrukcję.{ph1} Nie trenowaliśmy modelu RL przy " +"użyciu tego modułu sprawdzającego. Niepowodzenie, które zaobserwowaliśmy, " +"dotyczyło samego sprawdzenia. Ale gdybyśmy użyli tego jako nagrody, ten sam " +"martwy punkt stałby się częścią środowiska uczenia się." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"Test funkcjonalny może mieć podobny martwy punkt. {ph0}Badania " +"wykorzystujące różnicowe rozmycie w dekompilatorach LLM{ph1} wykazały " +"różnice w zachowaniu poza dostarczonymi testami. Dlatego też musimy rozumieć" +" zdanie egzaminu w kategoriach przypadków, w których występuje. Podobnie " +"dopasowanie instrukcji należy rozumieć w kategoriach bajtów i danych, które " +"sprawdza." + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"Zanim użyję jakiegokolwiek rodzaju kontroli weryfikacyjnej jako nagrody, " +"przetestowałbym ją z celowo błędnymi wynikami. Chciałbym również, aby " +"ostateczna akceptacja była oddzielona od kodu źródłowego, który agent AI " +"może edytować. Model musi przejść weryfikację poprzez poprawną rekonstrukcję" +" programu. W przeciwnym razie zwykłe skopiowanie tablicy bajtów z celu może " +"oszukać naiwny system oceniania bez odzyskiwania przydatnego kodu " +"źródłowego." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}Ulepszanie modelu i ulepszanie jego modułu sprawdzającego należą do " +"tego samego problemu badawczego.{ph1} Im lepiej model optymalizuje " +"otrzymywaną informację zwrotną, tym ważniejsze jest zrozumienie tej " +"informacji." + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "Porównanie warte przeprowadzenia" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"Mając te kontrole, rozpocząłbym porównanie od ustalonej rodziny zadań VC7.1." +" Najpierw ustaliłbym podstawy dla silnego modelu sztucznej inteligencji " +"ogólnego przeznaczenia i mniejszego. Dodanie wiedzy o repozytorium do " +"mniejszego modelu pokazałoby, jakie zmiany w pobieraniu. Następnie " +"nadzorowana specjalizacja i wersja przeszkolona przez RL pozwolą nam " +"sprawdzić, co dodaje szkolenie." + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"Aby porównania te miały charakter informacyjny, każdy model otrzymałby te " +"same docelowe dowody i narzędzia. W przypadku porównań specyficznych dla " +"szkoleń zachowałbym również stały model podstawowy i dostępną wiedzę. " +"Powtarzane próby pomogłyby odróżnić niezawodną strategię wyszukiwania od " +"potencjalnego rozwiązania, które odniosło sukces przez przypadek." + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"Następnie oddzieliłbym dwa warunki początkowe: rekonstrukcję funkcji na " +"podstawie jej instrukcji i zamknięcie kilku ostatnich różnic w utrzymywanym " +"źródle. Wyspecjalizowany model sztucznej inteligencji może doskonale " +"sprawdzić się w drugim zadaniu, podczas gdy model sztucznej inteligencji " +"ogólnego przeznaczenia wykonuje większość pracy potrzebnej do jego " +"osiągnięcia. Zmierzenie ich osobno ujawniłoby ten podział pracy." + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"Obydwa zadania wymagają przykładów testowych wyłączonych ze szkolenia i " +"wyszukiwania. Powiązane gry Touhou udostępniają kod silnika. Wykluczenie " +"jednej gry z treningu może w dalszym ciągu pozostawić prawie identyczne " +"funkcje w zbiorze treningowym. Zamiast tego wykluczyłbym całe grupy " +"powiązanego kodu źródłowego i trzymał ich zweryfikowane odpowiedzi poza " +"danymi szkoleniowymi i biblioteką pobierania." + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"W przypadku tych przykładów wynikiem będą kompletne, niezależnie " +"zweryfikowane dopasowania. Równe budżety na propozycje kandydatów lub próby " +"kompilacji pomogłyby w porównaniu jakości wyszukiwania. Pomiar całkowitego " +"czasu i kosztów, w tym kompilacji, pokazałby, czy ta jakość zapewnia " +"efektywny przepływ pracy. Koszty szkoleń należy mierzyć osobno." + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"Obydwa pomiary mają znaczenie. Tania propozycja może prowadzić do " +"kosztownego przepływu pracy, jeśli powoduje więcej kompilacji, które nie " +"dostarczają żadnych przydatnych informacji. I odwrotnie, wolniejszy model " +"może być bardziej ekonomiczny, jeśli wcześniej znajdzie przydatny " +"eksperyment. Porównanie powinno uwzględniać te koszty." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"Liczenie zaakceptowanych wyników pozwala również zachować perspektywę " +"postępu pośredniego. Jeden TH09 {ph0}kandydat do zapisania powtórki{ph1} " +"wyglądał obiecująco, ponieważ skompilowana funkcja miała oczekiwany rozmiar," +" ale pełne porównanie nadal wykazało znaczne różnice. Zmiany w sposobie, w " +"jaki kompilator organizuje instrukcje, mogą również sprawić, że kandydat " +"wydaje się bliższy celowi bez rozwiązania podstawowego problemu. Taka " +"diagnostyka może pomóc w kolejnym eksperymencie, ale każdą nagrodę za " +"„zbliżenie się” porównałbym z pełną kontrolą akceptacji." + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"Byłbym zainteresowany wzorcem tych wyników. Jeśli wyspecjalizowany model " +"sztucznej inteligencji radzi sobie dobrze w znanej rodzinie kompilatorów, " +"czy ta zaleta przetrwa zmianę w zestawie narzędzi? A kiedy oba modele " +"sztucznej inteligencji będą dobrze odzyskiwane, ile jeszcze doda szkolenie? " +"Odpowiedzi pomogłyby nam zdecydować, gdzie każdy rodzaj modelu jest " +"najbardziej przydatny." + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "Jakiego rodzaju agenta AI to sugeruje?" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"Jeśli modele pomagają na różnych etapach, przydatny może być agent ogólny z " +"dostępem do modelu specjalistycznego. Gdy zadanie sprowadza się do znanego " +"problemu z kompilatorem, może poprosić specjalistę o kandydatów. Jeśli " +"wyniki przestają mieć sens, agent ogólny może wrócić do danych celu albo " +"zbadać sam pomiar." + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"W takim układzie specjalizacja staje się narzędziem, które agent może wybrać" +" w miarę rozwoju dochodzenia. Agent zachowuje swobodę zmiany kierunku, co " +"umożliwiło wcześniejsze przełomy." + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"Wynika to naturalnie z przemian w przemyśle, które opisałem w poprzednim " +"eseju. Możliwość przeprowadzenia znacznie większej liczby eksperymentów " +"zmienia sposób organizacji pracy. Daje nam to również powód, aby zadać sobie" +" pytanie, które decyzje wymagają szerokiego osądu, a które można poprawić " +"dzięki wielokrotnemu doświadczeniu z konkretnym rzemiosłem." + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"Na razie najsilniejszy model ogólny zasłużył na miejsce w moim przepływie " +"pracy, rozwiązując problemy, których wcześniej nie rozwiązałem. Te sukcesy " +"skłaniają mnie do zbadania procesu: jaka część zależała od szerokości jego " +"możliwości, a jaką mógłby wykonać model uczony specjalnie do takiej " +"rekonstrukcji?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}Repozytoria umożliwiają nam przetestowanie pytania.{ph1} Chciałbym użyć" +" tego rekordu do stworzenia porównania i zobaczenia, czego się dowiemy." + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "Dowody i modelowe przypisanie" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"Przykłady opisują historyczne porównania poziomów funkcji zarejestrowane w " +"połączonych zatwierdzeniach. Sprawdzanie czasu działania całej gry to osobne" +" kamienie milowe. Dokumenty źródłowe i pasujące sprawdzono 13 października " +"2026 r." + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"Etykieta TH09 {ph0} oznacza GPT-6-Astra. Model powiązany z {ph1} zmieniał " +"się z biegiem czasu, więc sama etykieta nie może zidentyfikować modelu " +"użytego w konkretnym przebiegu. Wyraźne nazwy modeli w innych cytowanych " +"zatwierdzeniach zapewniają zarejestrowaną atrybucję. Historie te nie " +"obejmują spójnych pomiarów kosztów i wysiłku dla każdego przebiegu." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 okresy istnienia obiektów tymczasowych:{ph1} {ph2}wcześniejszy " +"przegląd kontekstu{ph3} i {ph4}pełna rekonstrukcja zweryfikowana przez " +"czystą rekompilację ze źródła{ph5}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}Callback Front w TH09:{ph1} {ph2}obszary diagnostyczne bez pełnej " +"zgodności{ph3}, {ph4}nowy graf przepływu sterowania Astry{ph5} i " +"{ph6}zaakceptowane sprawdzenie ponownej kompilacji{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}Wiedza o kompilatorze:{ph1} {ph2}ponowne użycie zachowania grup " +"identyfikatorów w TH095{ph3}, {ph4}kod gracza TH04 i kolejność " +"linkowania{ph5} oraz {ph6}pełna rekonstrukcja funkcji tworzenia przeciwników" +" TH09 przez Lunę{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}Kontrola i diagnostyka:{ph1} {ph2}TH08 audyt stałej " +"zmiennoprzecinkowej{ph3}, {ph4}TH08 badanie numeryczne{ph5} i {ph6}TH09 " +"rekonstrukcja z zachowaniem powtórki{ph7}." + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"Proponowane porównania wyspecjalizowanych modeli i szkolenia RL to przyszłe " +"eksperymenty. Motywują ich historie projektów. Na potrzeby tego eseju nie " +"oceniano żadnego wyszkolonego specjalistycznego modelu." + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "Czy inżynieria odwrotna potrzebuje większego modelu? | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"Czego rekonstrukcja Touhou nauczyła mnie o modelach ogólnych, modelach " +"specjalistycznych i nauce proponowania lepszych hipotez na podstawie " +"informacji zwrotnych od kompilatora." + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "Na tej stronie" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "Modele i esej o inżynierii odwrotnej" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "Wybrane eksperymenty rekonstrukcyjne" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "Do góry" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "Autor: N0zoM1z0 · 13 października 2026 · Szkic do przejrzenia" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/pt-BR/messages.po b/website/locales/pt-BR/messages.po index 408dff1d9..5c7366227 100644 --- a/website/locales/pt-BR/messages.po +++ b/website/locales/pt-BR/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Reflexões sobre engenharia reversa." #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Reconstrução do jogo{ph0}Ensaio" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "A Revolução industrial na engenharia reversa" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,20 @@ msgstr "" "O manual RE e reconstrução TH08, TH095 e TH04 me ensinou sobre agentes, " "verificação e conhecimento que compostos entre projetos." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Leia o artigo" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Por N0zoM1z0 * 10 de outubro de 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -100,6 +101,8 @@ msgstr "" "agentes REA e AI." #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -136,6 +139,7 @@ msgstr "" "codificação." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -156,6 +160,7 @@ msgid "REA home" msgstr "Página inicial do REA" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -176,6 +181,7 @@ msgid "Main navigation" msgstr "Navegação principal" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -196,6 +202,7 @@ msgid "REA on GitHub" msgstr "REA no GitHub" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -216,6 +223,7 @@ msgid "Join REA on Discord" msgstr "Participe do Discord do REA" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -235,7 +243,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Siga o REA no X" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Voltar ao topo" @@ -11695,6 +11703,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "Mesmo espaçamento: 256 16 = 16 unidades angulares = 22,5" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -11715,6 +11724,7 @@ msgid "Showcases" msgstr "Casos práticos" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -11735,6 +11745,7 @@ msgid "Blog" msgstr "Blog" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -11755,6 +11766,7 @@ msgid "Get started" msgstr "Começar" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -11774,7 +11786,8 @@ msgctxt "shared" msgid "Guides" msgstr "Guias" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11794,7 +11807,8 @@ msgctxt "shared" msgid "Source" msgstr "Código-fonte" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11814,7 +11828,8 @@ msgctxt "shared" msgid "MIT license" msgstr "Licença MIT" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11834,7 +11849,8 @@ msgctxt "shared" msgid "Join the community" msgstr "Participe da comunidade" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12821,8 +12837,1348 @@ msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "Por N0zoM1z0 · 13 de outubro de 2026 · Rascunho para revisão" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Por N0zoM1z0 · 13 de outubro de 2026" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "Ir para o conteúdo" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "" +"{ph0}Página inicial{ph1} / {ph2}Blog{ph3} / Modelos e engenharia reversa" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "Ensaio" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "A engenharia reversa precisa de um modelo maior?" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Por N0zoM1z0 · 13 de outubro de 2026" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "Pontos principais" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}A capacidade do modelo afeta quais hipóteses um agente de IA pode " +"propor.{ph1} Sessões posteriores resolveram problemas que trabalhos " +"anteriores haviam deixado sem solução." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}Um compilador familiar oferece padrões concretos para um modelo " +"especializado aprender.{ph1} Algumas correspondências exatas de código de " +"máquina dependem de padrões de código-fonte específicos e reutilizáveis." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}Inclua o conhecimento do repositório em uma comparação justa.{ph1} Um " +"modelo menor com boa recuperação de conhecimento pode já ter informações " +"suficientes para progredir." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}Treine o modelo para escolher o próximo experimento útil e teste seu " +"verificador.{ph1} O feedback é útil apenas na medida em que o verificador " +"verifica as coisas certas." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}Avalie os resultados verificados da reconstrução juntamente com o tempo" +" e o custo.{ph1} É isso que quero medir para o fluxo de trabalho completo do" +" agente de IA." + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "Neste artigo" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}O que os modelos mais capazes mudaram{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}Por que se especializar?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}Onde essa experiência deve ser armazenada?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}Aprendendo o próximo experimento útil{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}Engenharia reversa e aprendizagem por reforço{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}Quando o verificador está errado{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}Uma comparação que vale a pena fazer{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}Que tipo de agente de IA isso sugere?{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "O que os modelos mais capazes mudaram" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"Desde agosto de 2026 venho explorando o quanto de engenharia reversa um " +"agente pode realizar com acesso a um programa e uma forma de verificar seu " +"funcionamento. A maior parte da minha experiência vem da reconstrução de " +"jogos, especialmente Touhou. Como mudei de modelo, os repositórios " +"mantiveram um registro do que cada sessão conseguiu recuperar." + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"Grande parte desse trabalho visa a reconstrução exata: compilamos a fonte " +"mantida sob um conjunto de ferramentas histórico com versões fixas e " +"verificamos se ela reproduz a função alvo. Quando um modelo posterior é bem-" +"sucedido onde o trabalho anterior estagnou, as mudanças na origem mostram o " +"que tornou esse progresso possível." + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "As variáveis temporárias precisavam de vidas sobrepostas" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"Grande parte da lógica da função de ataque TH09 já havia sido reconstruída, " +"mas o compilador ainda armazenava resultados temporários de maneira " +"diferente do original. O problema restante era quanto tempo esses resultados" +" precisavam para permanecer vivos e quando seu armazenamento poderia ser " +"reutilizado." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"Um {ph0}GPT-6-Sol registro de sessão{ph1} relatou essa incompatibilidade " +"após testar vários arranjos comuns de código-fonte. Quando {ph2}GPT-6.1-Sol " +"examinou o problema novamente{ph3}, propôs uma hipótese útil sobre o tempo " +"de vida dessas variáveis ​​temporárias." + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"A função produziu resultados temporários e os copiou em posições de " +"animação. Em C++, um resultado temporário permanece ativo até que a " +"expressão que o utiliza termine. O agrupamento das operações reais em uma " +"expressão manteve esses resultados vivos juntos, de modo que o compilador " +"não pudesse mais reutilizar o mesmo armazenamento para todos eles." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"Um resultado ainda compartilha o armazenamento com um valor usado " +"posteriormente para verificações de colisão. Dar a esse valor seu próprio " +"tempo de vida resolveu a sobreposição restante. Após vários experimentos " +"direcionados, {ph0}uma recompilação limpa da fonte confirmou que a função " +"completa correspondia a{ph1}." + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"Estou interessado em saber como a descoberta e os testes se apoiaram. A " +"hipótese sobre a vida útil dos objetos temporários deu ao agente de IA uma " +"nova direção. O feedback do compilador ajudou a refinar essa hipótese em um " +"arranjo de código-fonte correspondente." + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "Um gráfico de fluxo de controle diferente" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"Uma segunda função TH09 mostra por que essa nova direção pode ser " +"importante. Aqui, o obstáculo foi a forma como diversas filiais " +"compartilharam seu trabalho final. O trabalho anterior de " +"{ph0}GPT-6.1-Sol{ph1} fez com que partes da função correspondessem, mas as " +"ramificações ainda se uniram de maneira diferente do original." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra alterou o gráfico de fluxo de controle circundante{ph1}. Os" +" principais caminhos levaram agora a uma atualização e retorno final " +"partilhado. Nessa estrutura de origem, o compilador combinou as ramificações" +" da mesma forma que o original. A função completa correspondeu e uma " +"{ph2}verificação de reconstrução separada confirmou isso{ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}Mudar a representação abriu um espaço de pesquisa mais produtivo.{ph1} " +"Ambos os casos me dão um motivo para levar a sério a capacidade geral do " +"modelo: um agente pode continuar tentando variações locais por um longo " +"tempo antes de encontrar uma maneira melhor de expressar o trabalho " +"circundante." + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"As sessões posteriores também herdaram o código anterior e os experimentos " +"fracassados já registrados. Esse trabalho acumulado fazia parte da vantagem " +"delas. O histórico permite examinar as descobertas e suas verificações; uma " +"comparação controlada ajudaria a separar o efeito do modelo do efeito da " +"experiência herdada." + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"É isso que torna a questão interessante para mim. Algum progresso veio da " +"descoberta de uma nova representação. Grande parte do trabalho restante " +"envolveu testar diferentes arranjos de código-fonte em um compilador " +"familiar. Quanto disso requer ampla capacidade de raciocínio? Quanto um " +"modelo poderia aprender trabalhando repetidamente com esse compilador e " +"código?" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "Por que um modelo especializado de IA pode funcionar bem?" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou nos dá uma razão concreta para esperar que essa experiência ajude. " +"Grande parte do código do jogo vem do mesmo autor, ZUN, portanto, hábitos " +"recorrentes são uma suposição inicial razoável. Os repositórios apoiam isso " +"com exemplos úteis: loops relacionados e arranjos auxiliares forneceram " +"hipóteses em todos os jogos." + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"Esses padrões de origem tornam-se mais específicos quando incluímos o " +"compilador. Sob uma determinada versão e conjunto de opções, pequenas " +"alterações na fonte podem produzir diferenças repetíveis no código de " +"máquina. Algumas dessas diferenças são surpreendentemente obscuras." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"Em TH095, por exemplo, {ph0}até mesmo o nome de uma variável local pode " +"afetar o local onde VC7.1 a armazenou{ph1}. Mover sua declaração não " +"resolveu o layout de armazenamento. No entanto, uma função anterior revelou " +"um padrão de nomenclatura que afetou a escolha do compilador. Isso sugeriu " +"uma hipótese mais promissora para {ph2}reconstruir {ph3}{ph4}." + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"A reutilização desse padrão para a variável de tamanho de arquivo produziu o" +" layout necessário sem adicionar armazenamento. A função completa então " +"correspondeu." + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"Este é um detalhe muito específico que um modelo de IA de uso geral pode não" +" saber quando encontra o problema pela primeira vez. Um modelo especializado" +" para este compilador poderia ver muitos desses casos e aprender quais " +"variações do código-fonte valem a pena testar primeiro." + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"A oportunidade é especialmente clara em problemas de correspondência final " +"onde já entendemos o comportamento. Estamos então perguntando qual arranjo " +"comum de código-fonte este compilador transforma nas instruções observadas. " +"A especialização poderia ajudar o modelo a propor essas formas de forma mais" +" eficiente, enquanto o alvo ainda determina quais hipóteses sobreviverão." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"Padrões úteis devem manter seu contexto. TH08 usa uma versão Visual C++ " +"anterior a TH095 e TH09. TH04 usa um compilador diferente da era DOS. Sua " +"{ph0}reconstrução do código do jogador{ph1} precisava de diferentes " +"hipóteses sobre os padrões do código-fonte e como o código compilado era " +"vinculado a um executável. Ter o mesmo autor nos dá uma expectativa inicial," +" mas cada alvo nos diz se se aplica. Um modelo especializado precisaria " +"distinguir esses contextos." + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "Onde essa experiência deve ser armazenada?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"Antes de decidir ensinar esses padrões por meio do treinamento de modelos, " +"eu examinaria o conhecimento que já mantemos no repositório. No {ph0}ensaio " +"anterior{ph1}, escrevi sobre isso se tornar a memória de trabalho do " +"projeto. Essa memória inclui lições de compilador, juntamente com a fonte e " +"os experimentos fracassados ​​que os tornaram úteis." + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"Ao recuperar esse registro, um agente pode começar com um resultado " +"conhecido e inspecionar como o alcançamos. Também pode evitar alternativas " +"que já falharam no mesmo contexto. O ponto inicial melhora sem alterar os " +"parâmetros treinados do modelo." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"Ao reconstruir a função de TH09 para criar inimigos, uma {ph0}sessão " +"GPT-6-Luna-max{ph1} tirou uma hipótese do código-fonte TH08 relacionado. " +"Esse código anterior sugeria como organizar o loop e lidar com falhas dentro" +" dele. As próprias evidências de TH09 forneceram detalhes específicos do " +"jogo. As reconstruções da fonte confirmaram que a função completa " +"correspondia." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}\"O modelo não sabia daquela peculiaridade do compilador\" e \"o modelo" +" não pôde usá-lo\" são problemas diferentes.{ph1} A recuperação pode " +"resolver o primeiro, mas transformar esse fato em um candidato útil ainda dá" +" trabalho. E se o código-fonte candidato falhar, o modelo deverá decidir o " +"que o resultado significa para sua próxima tentativa. Conhecer o padrão " +"relevante é apenas o começo desse processo." + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"Por esse motivo, eu incluiria um modelo menor de IA de uso geral com boa " +"recuperação em qualquer comparação. Fornecer a mesma biblioteca de " +"experiência nos ajudaria a identificar o que o treinamento agrega: melhor " +"geração de candidatos, melhores decisões após feedback ou ambos." + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "Aprendendo o próximo experimento útil" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"Essas decisões após feedback são o que eu mais gostaria de estudar. Na " +"reconstrução agente, o modelo propõe uma explicação e a testa em relação ao " +"programa. Mesmo quando o código-fonte candidato falha, o resultado pode " +"restringir as possibilidades restantes e sugerir um próximo experimento mais" +" útil." + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"Considere novamente o exemplo dos resultados temporários. Sua fonte final " +"mostra o arranjo que funcionou, mas o caminho para essa fonte contém mais " +"informações. Cada um dos experimentos a seguir ajudou a identificar o que " +"ainda precisava ser mudado." + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "Experimentar" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "O que seu resultado nos ensinou" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "Tente uma maneira diferente de copiar os resultados" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "O código de cópia muda, mas a função ainda é diferente." + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "Use os valores retornados diretamente" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "" +"A cópia segue o padrão original, mas o armazenamento temporário ainda é " +"diferente." + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "Mantenha todas as variáveis temporárias válidas ao mesmo tempo" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "" +"Eles obtêm armazenamento separado, mas ainda se sobrepõe a um valor usado " +"posteriormente para verificações de colisão." + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "Mantenha o valor de verificação de colisão separado" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "Uma recompilação limpa da fonte confirma a correspondência completa." + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "" +"Experimentos selecionados da reconstrução da função de ataque acima, " +"agrupados pelo que nos ensinaram." + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"Um registro de treinamento que mantenha essa sequência pode ensinar mais do " +"que apenas a fonte final. Ele conecta cada mudança proposta a um resultado " +"observado e à próxima decisão. Manter variações ineficazes também é " +"importante: elas informam a um agente posterior quais hipóteses já foram " +"testadas naquele contexto." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}Eu gostaria que o modelo aprendesse como usar o feedback.{ph1} Um " +"resultado neutro é um motivo para reconsiderar a variação; uma " +"incompatibilidade alterada pode apontar para outra causa. Quando as edições " +"locais deixarem de ser informativas, o agente deverá ser capaz de revisitar " +"a representação da fonte ou inspecionar as evidências novamente." + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"Aprender a fazer essas escolhas requer autonomia. Às vezes, o próximo passo " +"útil é inspecionar outro chamador em vez de compilar outra edição. Uma " +"sequência rígida de reescritas necessárias eliminaria essa escolha, " +"juntamente com parte do comportamento que queremos que o modelo aprenda." + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "" +"Por que a engenharia reversa é interessante para aprendizagem por reforço" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"Uma reconstrução com escopo definido oferece um ambiente prático para " +"estudar essas escolhas. Com as versões do alvo e do compilador fixadas, " +"muitos experimentos de código-fonte dão feedback rápido e repetível. O " +"agente pode compilar um candidato, examinar as diferenças e escolher o " +"próximo passo." + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"A aprendizagem por reforço poderia treinar esse processo de tomada de " +"decisão por meio do feedback que recebe. Um candidato aceito por meio de uma" +" verificação definida nos dá um resultado concreto, enquanto a sequência de " +"tentativas nos permite observar quanto trabalho foi necessário para alcançá-" +"lo." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"Já existem pesquisas apoiando essa direção. {ph0}LLM4Decompile{ph1} estuda " +"modelos especializados para descompilação. {ph2}AutoDecompiler{ph3} treina o" +" refinamento em várias rodadas usando feedback do compilador e de execução. " +"Seus estudos se concentram na recuperação funcional. Para este trabalho, eu " +"os usaria como ponto de partida para a tarefa relacionada de encontrar uma " +"fonte que corresponda à saída de um compilador histórico." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"Porém, para alcançar uma tarefa tão bem definida, talvez precisemos primeiro" +" estabelecer quais recursos uma função possui ou criar uma nova medição de " +"tempo de execução. Em TH08, por exemplo, uma {ph0}diferença de colisão levou" +" de volta à forma como o programa original lidava com cálculos de ponto " +"flutuante{ph1}. Antes que o agente pudesse testar uma correção útil, ele " +"precisava decidir o que medir." + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"Isto sugere outro papel para um modelo amplamente capaz: em primeiro lugar, " +"ajudar a tornar o problema verificável. Uma vez que essa verificação exista," +" podemos investigar até onde uma política de pesquisa aprendida pode chegar " +"com ela." + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "Quando o verificador está errado" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"Quando usamos uma verificação no treinamento, sua cobertura passa a fazer " +"parte do ambiente de aprendizagem. Chamo essa referência ou verificador de " +"oracle. Mesmo uma resposta repetível precisa ser entendida pelos aspectos " +"que ela realmente verifica." + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 mostrou o quanto pode se esconder fora dessa cobertura. Algumas funções" +" reconstruídas passaram em suas comparações exatas com constantes flutuantes" +" erradas. O comparador ajustou os endereços realocados antes de verificar as" +" instruções, mas verificou os valores por trás desses endereços somente " +"quando eles foram explicitamente registrados para verificação. Outras " +"referências literais poderiam passar sem que seus valores fossem examinados." + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"Para resolver essa lacuna, uma {ph0}auditoria mais ampla{ph1} encontrou " +"constantes incorretas em funções que já haviam passado na verificação. Em um" +" cálculo, o código-fonte reconstruído adicionou {ph2}, enquanto o original " +"adicionou {ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}Um modelo recompensado pela verificação antiga poderia aprender a " +"produzir uma reconstrução incorreta.{ph1} Não treinamos um modelo RL usando " +"esse verificador. A falha que observamos estava na própria verificação. Mas " +"se usássemos isso como recompensa, o mesmo ponto cego passaria a fazer parte" +" do ambiente de aprendizagem." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"Um teste funcional pode ter um ponto cego semelhante. {ph0}Pesquisas usando " +"difusão diferencial em descompiladores LLM{ph1} encontraram diferenças " +"comportamentais fora dos testes fornecidos. Portanto, precisamos entender um" +" teste aprovado em termos dos casos que ele exerce. Da mesma forma, uma " +"correspondência de instrução deve ser entendida em termos dos bytes e dos " +"dados que ela verifica." + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"Antes de usar qualquer tipo de verificação como recompensa, eu testaria com " +"resultados deliberadamente incorretos. Eu também manteria a aceitação final " +"separada do código-fonte que o agente de IA pode editar. O modelo deve " +"passar na verificação reconstruindo o programa corretamente. Caso contrário," +" simplesmente copiar uma matriz de bytes do destino poderia enganar um " +"sistema de pontuação ingênuo sem recuperar o código-fonte útil." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}Aprimorar o aluno e melhorar seu verificador pertencem ao mesmo " +"problema de pesquisa.{ph1} À medida que o aluno melhora na otimização do " +"feedback que lhe damos, compreender esse feedback também se torna mais " +"importante." + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "Uma comparação que vale a pena fazer" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"Com essas verificações implementadas, eu iniciaria a comparação em uma " +"família de tarefas fixa VC7.1. Primeiro, eu estabeleceria linhas de base " +"para um modelo forte de IA de uso geral e um modelo menor. Adicionar " +"conhecimento do repositório ao modelo menor mostraria quais mudanças na " +"recuperação. A partir daí, a especialização supervisionada e uma versão " +"treinada em RL nos permitiriam examinar o que o treinamento acrescenta." + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"Para tornar essas comparações informativas, cada modelo receberia as mesmas " +"evidências e ferramentas alvo. Para comparações específicas de treinamento, " +"eu também manteria o modelo base e o conhecimento disponível fixos. Testes " +"repetidos ajudariam a distinguir uma estratégia de busca confiável de uma " +"solução candidata que tivesse sucesso por acaso." + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"Eu separaria então duas condições iniciais: reconstruir uma função a partir " +"de suas instruções e fechar as últimas diferenças na fonte mantida. Um " +"modelo de IA especializado pode ser excelente na segunda tarefa, enquanto um" +" modelo de IA de uso geral faz grande parte do trabalho necessário para " +"alcançá-la. Medi-los separadamente revelaria essa divisão de trabalho." + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"Ambas as tarefas precisam de exemplos de teste excluídos do treinamento e " +"recuperação. Jogos Touhou relacionados compartilham o código do mecanismo. " +"Excluir um jogo do treinamento ainda poderia deixar funções quase idênticas " +"no conjunto de treinamento. Em vez disso, eu excluiria grupos inteiros de " +"código-fonte relacionado e manteria suas respostas verificadas fora dos " +"dados de treinamento e da biblioteca de recuperação." + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"Para estes exemplos, o resultado seria correspondências completas e " +"verificadas de forma independente. Orçamentos iguais para propostas de " +"candidatos ou tentativas de compilação ajudariam a comparar a qualidade da " +"pesquisa. Medir o tempo e o custo totais, incluindo a compilação, mostraria " +"se essa qualidade produz um fluxo de trabalho eficiente. Os custos de " +"formação devem ser medidos separadamente." + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"Ambas as medidas são importantes. Uma proposta barata pode levar a um fluxo " +"de trabalho caro se causar mais compilações que não fornecem informações " +"úteis. Por outro lado, um modelo mais lento pode ser mais econômico se " +"encontrar uma experiência útil mais cedo. A comparação deve incluir esses " +"custos." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"A contagem dos resultados aceitos também mantém o progresso intermediário em" +" perspectiva. Um TH09 {ph0}candidato para salvar repetição{ph1} parecia " +"promissor porque a função compilada tinha o tamanho esperado, mas uma " +"comparação completa ainda encontrou diferenças substanciais. Mudanças na " +"forma como o compilador organiza as instruções também podem fazer o " +"candidato parecer mais próximo do alvo, sem resolver o problema subjacente. " +"Esses diagnósticos podem orientar o próximo experimento, mas eu validaria " +"qualquer recompensa por “chegar mais perto” em relação à verificação de " +"aceitação total." + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"Eu estaria interessado no padrão desses resultados. Se um modelo de IA " +"especializado funciona bem em uma família de compiladores familiar, essa " +"vantagem sobreviverá a uma mudança no conjunto de ferramentas? E uma vez que" +" ambos os modelos de IA tenham uma boa recuperação, quanto o treinamento " +"ainda acrescenta? As respostas nos ajudariam a decidir onde cada tipo de " +"modelo é mais útil." + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "Que tipo de agente de IA isso sugere?" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"Se os modelos acabarem por ajudar em diferentes fases, um acordo útil poderá" +" ser um agente amplo com acesso a um modelo de IA especializado. Depois que " +"uma tarefa se torna uma pesquisa familiar do compilador, ela pode solicitar " +"candidatos ao modelo de IA especializado. Se os resultados deixarem de fazer" +" sentido, o agente mais amplo poderá retornar à evidência alvo ou investigar" +" a própria medição." + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"Nesse arranjo, a especialização torna-se uma ferramenta que o agente pode " +"escolher à medida que a investigação se desenvolve. O agente mantém a " +"liberdade de mudar a direção que tornou possíveis os avanços anteriores." + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"Isto decorre naturalmente da mudança industrial que descrevi no ensaio " +"anterior. Ser capaz de realizar muito mais experimentos muda a forma como " +"organizamos o trabalho. Também nos dá uma razão para perguntar quais " +"decisões necessitam de um julgamento amplo e quais melhoram através da " +"experiência repetida com uma determinada arte." + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"Por enquanto, o modelo geral mais forte conquistou seu lugar no meu trabalho" +" ao resolver problemas que eu havia deixado em aberto. Esses sucessos me " +"fazem querer estudar o processo: quanto dependia da amplitude do modelo e " +"quanto poderia ser feito por um modelo treinado especificamente para esse " +"tipo de reconstrução?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}Os repositórios nos fornecem uma maneira de testar a pergunta.{ph1} " +"Gostaria de usar esse registro para construir a comparação e ver o que " +"aprendemos." + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "Evidências e atribuição de modelo" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"Os exemplos descrevem comparações históricas em nível de função registradas " +"nos commits vinculados. As verificações de tempo de execução de todo o jogo " +"são marcos separados. Os registros de origem e correspondentes foram " +"revisados em 13 de outubro de 2026." + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"O rótulo TH09 {ph0} significa GPT-6-Astra. O modelo associado a {ph1} mudou " +"ao longo do tempo, de modo que o rótulo por si só não consegue identificar o" +" modelo usado em uma execução específica. Nomes de modelos explícitos nos " +"outros commits citados fornecem a atribuição registrada. Esses históricos " +"não incluem medições consistentes de custo ou esforço para cada execução." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 tempos de vida do objeto temporário:{ph1} a {ph2}revisão de " +"contexto anterior{ph3} e a {ph4}reconstrução completa verificada por uma " +"recompilação limpa da origem{ph5}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}TH09 Front retorno de chamada:{ph1} {ph2}regiões de diagnóstico que não" +" correspondem{ph3}, {ph4}novo gráfico de fluxo de controle de Astra{ph5} e " +"sua {ph6}verificação de recompilação aceita{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}Conhecimento do compilador:{ph1} {ph2}TH095 reutilização do " +"comportamento do bloco de identificadores{ph3}, {ph4}TH04 código-fonte do " +"jogador e ordem do vinculador{ph5} e {ph6}reconstrução completa de Luna da " +"função de criação de inimigo de TH09{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}Verificações e diagnósticos:{ph1} {ph2}TH08 auditoria de constante de " +"ponto flutuante{ph3}, {ph4}TH08 investigação numérica{ph5} e {ph6}TH09 " +"reconstrução para salvar replay{ph7}." + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"As comparações propostas de modelos especializados e treinamento RL são " +"experimentos futuros. As histórias do projeto os motivam. Nenhum modelo " +"especializado treinado foi avaliado para este ensaio." + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "A engenharia reversa precisa de um modelo maior? | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"O que a reconstrução Touhou me ensinou sobre modelos gerais, modelos " +"especializados e como aprender a propor melhores hipóteses a partir do " +"feedback do compilador." + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "Nesta página" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "Modelos e ensaio de engenharia reversa" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "Experimentos de reconstrução selecionados" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "Voltar ao topo" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "Por N0zoM1z0 · 13 de outubro de 2026 · Rascunho para revisão" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/ru/messages.po b/website/locales/ru/messages.po index ed9c8005c..8078a3ea9 100644 --- a/website/locales/ru/messages.po +++ b/website/locales/ru/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Мысли о реверс-инжиниринге." #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Реконструкция игры{ph0}Сочинение" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "Промышленная революция в области реверс-инжиниринга" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,20 @@ msgstr "" "Чему меня научило руководство по обновлению и реконструкции TH08, TH095 и " "TH04 об агентах, проверке и знаниях, которые используются в разных проектах." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Прочтите статью" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Автор: N0zoM1z0 · 10 октября 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -100,6 +101,8 @@ msgstr "" "REA и агентами искусственного интеллекта." #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -136,6 +139,7 @@ msgstr "" " с помощью вашего программиста." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -156,6 +160,7 @@ msgid "REA home" msgstr "Главная REA" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -176,6 +181,7 @@ msgid "Main navigation" msgstr "Основная навигация" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -196,6 +202,7 @@ msgid "REA on GitHub" msgstr "REA на GitHub" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -216,6 +223,7 @@ msgid "Join REA on Discord" msgstr "Присоединиться к REA в Discord" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -235,7 +243,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Подписаться на REA в X" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Наверх" @@ -11732,6 +11740,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "Одинаковое расстояние: 256 ÷ 16 = 16 угловых единиц = 22,5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -11752,6 +11761,7 @@ msgid "Showcases" msgstr "Примеры" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -11772,6 +11782,7 @@ msgid "Blog" msgstr "Блог" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -11792,6 +11803,7 @@ msgid "Get started" msgstr "Начать" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -11811,7 +11823,8 @@ msgctxt "shared" msgid "Guides" msgstr "Руководства" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11831,7 +11844,8 @@ msgctxt "shared" msgid "Source" msgstr "Исходный код" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11851,7 +11865,8 @@ msgctxt "shared" msgid "MIT license" msgstr "Лицензия MIT" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11871,7 +11886,8 @@ msgctxt "shared" msgid "Join the community" msgstr "Вступить в сообщество" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12863,8 +12879,1347 @@ msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "Автор: N0zoM1z0 · 13 октября 2026 г. · Черновик для обсуждения" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Автор N0zoM1z0 · 13 октября 2026 г." + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "Перейти к содержанию" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "" +"{ph0}Главная страница{ph1} / {ph2}Блог{ph3} / Модели и реверс-инжиниринг" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "Эссе" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Нужна ли реверс-инжинирингу модель большего размера?" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Автор N0zoM1z0 · 13 октября 2026 г." + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "Главное" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}Возможности модели влияют на то, какие гипотезы может предложить ИИ-" +"агент.{ph1} На последующих занятиях решались проблемы, которые ранее " +"оставались нерешенными." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}Знакомый компилятор предлагает конкретные шаблоны для изучения " +"специализированной модели.{ph1} Некоторые точные совпадения машинного кода " +"зависят от конкретных многократно используемых шаблонов исходного кода." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}Включите знания репозитория в справедливое сравнение.{ph1} Меньшая " +"модель с хорошим извлечением знаний может уже иметь достаточно информации " +"для достижения прогресса." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}Обучите модель выбирать следующий полезный эксперимент и протестируйте " +"ее средство проверки.{ph1} Обратная связь полезна только в той степени, в " +"которой средство проверки проверяет правильные вещи." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}Оцените проверенные результаты реконструкции, а также время и " +"стоимость.{ph1} Это то, что я хочу измерить для всего рабочего процесса ИИ-" +"агента." + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "В этой статье" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}Что изменили более сильные модели{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}Зачем специализироваться?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}Где следует хранить этот опыт?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}Изучаем следующий полезный эксперимент{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}Обратное проектирование и обучение с подкреплением{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}Когда программа проверки ошибается{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}Сравнение, которое стоит провести{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}О каком ИИ-агенте идет речь?{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "Что изменили более сильные модели" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"С августа 2026 года я изучаю, какой объем реверс-инжиниринга может провести " +"агент, имея доступ к программе и способ проверки ее работы. Большая часть " +"моего опыта связана с реконструкцией игр, особенно Touhou. Поскольку я менял" +" модели, в репозиториях фиксировалось, что удалось восстановить в каждом " +"сеансе." + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"Большая часть этой работы направлена на точную реконструкцию: мы компилируем" +" поддерживаемый исходный код с помощью исторической цепочки инструментов с " +"фиксированными версиями и проверяем, воспроизводит ли он целевую функцию. " +"Когда более поздняя модель оказывается успешной там, где предыдущие работы " +"застопорились, изменения в исходном коде показывают, что сделало этот " +"прогресс возможным." + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "Временным переменным требовалось перекрывающееся время жизни." + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"Большая часть логики функции атаки TH09 уже была реконструирована, но " +"компилятор по-прежнему сохранял временные результаты иначе, чем оригинал. " +"Оставшаяся проблема заключалась в том, как долго эти результаты должны " +"оставаться живыми и когда их хранилище можно будет использовать повторно." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"Запись сеанса {ph0}GPT-6-Sol{ph1} сообщила об этом несоответствии после " +"тестирования нескольких обычных схем исходного кода. Когда {ph2}GPT-6.1-Sol " +"снова изучил проблему{ph3}, он предложил полезную гипотезу о времени жизни " +"этих временных переменных." + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"Функция выдавала временные результаты и копировала их в позиции анимации. В " +"C++ временный результат остается активным до тех пор, пока выражение, " +"использующее его, не завершится. Группировка реальных операций в одно " +"выражение сохраняла эти результаты вместе, поэтому компилятор больше не мог " +"повторно использовать одно и то же хранилище для всех них." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"Один из результатов по-прежнему разделял хранилище со значением, которое " +"позже использовалось для проверок на коллизии. Предоставление этому значению" +" собственного времени жизни решило оставшееся перекрытие. После нескольких " +"целевых экспериментов {ph0}чистая перекомпиляция из исходного кода " +"подтвердила, что полная функция соответствует{ph1}." + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"Меня интересует, как открытие и испытания поддержали друг друга. Гипотеза о " +"времени жизни временных объектов дала агенту ИИ новое направление. Отзывы " +"компилятора помогли усовершенствовать эту гипотезу и привести ее в " +"соответствие с исходным кодом." + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "Другой граф потока управления" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"Вторая функция TH09 показывает, почему это новое направление может иметь " +"значение. Здесь препятствием было то, как несколько филиалов делились своей " +"итоговой работой. В более ранней работе {ph0}GPT-6.1-Sol{ph1} части функции " +"совпадали, но ветви по-прежнему соединялись иначе, чем в оригинале." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra изменил окружающий граф потока управления{ph1}. Основные " +"пути теперь вели к общему финальному обновлению и возврату. В этой исходной " +"структуре компилятор объединил ветки так же, как в оригинале. Полная функция" +" совпала, и {ph2}отдельная проверка перестроения подтвердила это{ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}Изменение представления открыло более продуктивное пространство " +"поиска.{ph1} Оба случая дают мне повод серьезно отнестись к общим " +"возможностям модели: агент может долго пробовать локальные варианты, прежде " +"чем найти лучший способ выразить окружающую работу." + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"В то же время более поздние сеансы начинались с более раннего источника и " +"фиксировали сбои. Этот накопленный труд был частью их преимущества. История " +"позволяет нам проверять открытия и их проверки; контролируемое сравнение " +"поможет отделить эффект модели от унаследованного ею опыта." + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"Именно это делает вопрос для меня интересным. Некоторый прогресс был " +"достигнут благодаря поиску нового представительства. Большая часть " +"оставшейся работы заключалась в тестировании различных вариантов исходного " +"кода на знакомом компиляторе. Насколько это требует широких умственных " +"способностей? Насколько многому может научиться модель, неоднократно работая" +" с этим компилятором и кодом?" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "Почему специализированная модель ИИ может преуспеть?" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou дает нам конкретную причину ожидать, что этот опыт поможет. Большая " +"часть игрового кода принадлежит одному и тому же автору, ZUN, поэтому " +"повторяющиеся привычки — разумное исходное предположение. Репозитории " +"подтверждают это полезными примерами: связанные циклы и вспомогательные " +"механизмы служат источником гипотез для разных игр." + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"Эти исходные шаблоны становятся более конкретными, когда мы подключаем " +"компилятор. В конкретной версии и наборе опций небольшие изменения в " +"исходном коде могут привести к повторяющимся различиям в машинном коде. " +"Некоторые из этих различий на удивление неясны." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"Например, в TH095 {ph0}даже имя локальной переменной может влиять на то, где" +" VC7.1 ее хранит{ph1}. Перемещение его декларации не решило проблему с " +"расположением хранилища. Однако более ранняя фоновая функция выявила шаблон " +"именования, который повлиял на выбор компилятора. Это предложило более " +"многообещающую гипотезу для {ph2}реконструкции {ph3}{ph4}." + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"Повторное использование этого шаблона для переменной размера файла позволило" +" получить требуемый макет без увеличения объема памяти. Затем полная функция" +" совпала." + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"Это очень специфическая деталь, о которой модель ИИ общего назначения может " +"не знать, когда впервые сталкивается с проблемой. Модель, специализированная" +" для этого компилятора, могла бы увидеть множество таких случаев и " +"определить, какие варианты исходного кода стоит протестировать в первую " +"очередь." + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"Эта возможность особенно очевидна в задачах окончательного сопоставления, " +"где мы уже понимаем поведение. Затем мы спрашиваем, какую обычную структуру " +"исходного кода этот компилятор превращает в наблюдаемые инструкции. " +"Специализация может помочь модели более эффективно предлагать эти формы, в " +"то время как цель по-прежнему определяет, какие гипотезы выживут." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"Полезные шаблоны должны сохранять свой контекст. TH08 использует более " +"раннюю версию Visual C++, чем TH095 и TH09. TH04 использует другой " +"компилятор эпохи DOS. Его {ph0}реконструкция кода игрока{ph1} требовала " +"различных гипотез о шаблонах исходного кода и о том, как скомпилированный " +"код связывается с исполняемым файлом. Наличие одного и того же автора дает " +"нам первоначальное ожидание, но каждая цель говорит нам, применимо ли оно. " +"Специализированная модель должна будет различать эти контексты." + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "Где следует хранить этот опыт?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"Прежде чем принять решение об обучении этим шаблонам посредством обучения " +"моделей, я бы посмотрел на знания, которые мы уже храним в репозитории. В " +"{ph0}предыдущем эссе{ph1} я писал о том, что оно стало рабочей памятью " +"проекта. Эта память включает в себя уроки компилятора, а также исходники и " +"неудачные эксперименты, которые сделали их полезными." + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"Получив эту запись, агент может начать с известного результата и проверить, " +"как мы его достигли. Это также позволяет избежать альтернатив, которые уже " +"потерпели неудачу в том же контексте. Начальная точка улучшается без " +"изменения обученных параметров модели." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"При реконструкции функции TH09 по созданию врагов сеанс {ph0}GPT-6-Luna-" +"max{ph1} взял гипотезу из соответствующего исходного кода TH08. Этот более " +"ранний код предлагал, как организовать цикл и обрабатывать сбои внутри него." +" Собственные доказательства TH09 предоставили подробности, касающиеся " +"конкретной игры. Перестройки из исходного кода подтвердили, что вся функция " +"совпадает." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}\"Модель не знала об этой особенности компилятора\" и \"модель не могла" +" ее использовать\" — это разные проблемы.{ph1} Извлечение может решить " +"первую, но превращение этого факта в полезного кандидата все равно требует " +"работы. И если исходный код-кандидат терпит неудачу, модель должна решить, " +"что означает результат для ее следующей попытки. Знание соответствующей " +"закономерности — это только начало этого процесса." + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"По этой причине я бы включил в любое сравнение меньшую модель ИИ общего " +"назначения с хорошей воспроизводимостью. Предоставление ему той же " +"библиотеки опыта поможет нам определить, что дает обучение: лучшее " +"генерирование кандидатов, лучшие решения после обратной связи или и то, и " +"другое." + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "Изучаем следующий полезный эксперимент" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"Эти решения после получения обратной связи — это то, что мне больше всего " +"хотелось бы изучить. При агентной реконструкции модель предлагает объяснение" +" и проверяет его на соответствие программе. Даже если исходный код-кандидат " +"дает сбой, результат может сузить оставшиеся возможности и предложить более " +"полезный следующий эксперимент." + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"Снова рассмотрим пример временных результатов. Его окончательный источник " +"показывает сработавшую схему, но путь к этому источнику содержит больше " +"информации. Каждый из следующих экспериментов помог определить, что еще " +"необходимо изменить." + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "Эксперимент" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "Чему нас научил его результат" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "Попробуйте другой способ скопировать результаты" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "Код копирования меняется, но функция по-прежнему отличается." + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "Используйте возвращаемые значения напрямую" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "" +"Копирование следует исходному шаблону, но временное хранилище все равно " +"отличается." + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "Сохраняйте все временные переменные действительными одновременно" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "" +"Они получают отдельное хранилище, но одно из них по-прежнему перекрывает " +"значение, используемое позже для проверок коллизий." + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "Держите значение проверки столкновений отдельно" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "" +"Чистая перекомпиляция из исходного кода подтверждает полное совпадение." + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "" +"Избранные эксперименты из приведенной выше реконструкции атакующей функции, " +"сгруппированные по тому, чему они нас научили." + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"Запись обучения, в которой сохраняется эта последовательность, может научить" +" большему, чем один только окончательный источник. Он связывает каждое " +"предлагаемое изменение с наблюдаемым результатом и следующим решением. " +"Сохранение неэффективных вариантов также имеет значение: они сообщают " +"следующему агенту, какие гипотезы уже были проверены в этом контексте." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}Я бы хотел, чтобы модель научилась использовать обратную связь.{ph1} " +"Нейтральный результат — повод пересмотреть вариант; измененное " +"несоответствие может указывать на другую причину. Когда локальные изменения " +"перестают быть информативными, агент должен иметь возможность вернуться к " +"исходному представлению или снова проверить доказательства." + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"Чтобы научиться делать такой выбор, требуется автономия. Иногда полезным " +"следующим шагом является проверка другого вызывающего объекта, а не " +"компиляция еще одного редактирования. Жесткая последовательность необходимых" +" переписываний устранит этот выбор вместе с частью поведения, которому мы " +"хотим, чтобы модель научилась." + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "Чем реверс-инжиниринг интересен для обучения с подкреплением" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"Реконструкция с заданными границами даёт практическую среду для изучения " +"этих решений. Когда версии цели и компилятора зафиксированы, многие " +"эксперименты с исходным кодом дают быструю, воспроизводимую обратную связь. " +"Агент может скомпилировать кандидат, изучить различия и выбрать следующий " +"шаг." + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"Обучение с подкреплением могло бы тренировать этот процесс принятия решений " +"посредством получаемой обратной связи. Кандидат, принятый посредством " +"определенной проверки, дает нам конкретный результат, а последовательность " +"попыток позволяет нам наблюдать, сколько работы потребовалось для его " +"достижения." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"Уже есть исследования, подтверждающие это направление. " +"{ph0}LLM4Decompile{ph1} изучает модели, предназначенные для декомпиляции. " +"{ph2}AutoDecompiler{ph3} обучает многоэтапному уточнению с использованием " +"компилятора и обратной связи при выполнении. Их исследования сосредоточены " +"на функциональном восстановлении. В этой работе я бы использовал их в " +"качестве отправной точки для решения соответствующей задачи по поиску " +"источника, соответствующего выводам исторического компилятора." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"Однако для решения такой четко определенной задачи нам, возможно, сначала " +"придется установить, какими ресурсами владеет функция, или разработать новое" +" измерение времени выполнения. Например, в TH08 {ph0}разность коллизий " +"привела к тому, как исходная программа обрабатывала вычисления с плавающей " +"запятой{ph1}. Прежде чем агент смог протестировать полезную поправку, он " +"должен был решить, что именно измерять." + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"Это предполагает еще одну роль модели с широкими возможностями: в первую " +"очередь помочь сделать проблему проверяемой. Как только эта проверка " +"появится, мы сможем выяснить, насколько далеко может зайти изученная " +"политика поиска." + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "Когда программа проверки ошибается" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"Когда проверка используется для обучения, её охват становится частью среды " +"обучения. Я называю такой эталон или проверяющий инструмент оракулом. Даже " +"воспроизводимый ответ нужно понимать в пределах того, что он действительно " +"проверяет." + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 показал, как много может скрываться за пределами этого покрытия. " +"Некоторые реконструированные функции прошли точное сравнение с неправильными" +" плавающими константами. Компаратор корректировал перемещенные адреса перед " +"проверкой инструкций, но проверял значения, стоящие за этими адресами, " +"только тогда, когда они были явно записаны для проверки. Другие буквальные " +"ссылки могут пройти без проверки их значений." + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"Чтобы устранить этот пробел, {ph0}более широкий аудит{ph1} обнаружил " +"неправильные константы в функциях, которые уже прошли проверку. В одном " +"расчете в реконструированный исходный код добавлен {ph2}, тогда как в " +"исходный добавлен {ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}Модель, получившая награду в результате старой проверки, могла " +"научиться производить неверную реконструкцию.{ph1} Мы не обучали модель RL с" +" помощью этой проверки. Сбой, который мы наблюдали, заключался в самой " +"проверке. Но если бы мы использовали это в качестве награды, то же самое " +"слепое пятно стало бы частью среды обучения." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"Функциональный тест может иметь аналогичное «слепое пятно». " +"{ph0}Исследование с использованием дифференциального фаззинга в " +"декомпиляторах LLM{ph1} выявило различия в поведении за пределами " +"предоставленных тестов. Поэтому нам необходимо понимать прохождение теста с " +"точки зрения случаев, в которых он проводится. Точно так же совпадение " +"инструкций следует понимать с точки зрения байтов и данных, которые оно " +"проверяет." + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"Прежде чем использовать какой-либо вид проверки в качестве награды, я бы " +"протестировал ее с заведомо неверными результатами. Я также хотел бы хранить" +" окончательное принятие отдельно от исходного кода, который может " +"редактировать агент ИИ. Модель должна пройти проверку путем правильного " +"восстановления программы. В противном случае простое копирование массива " +"байтов из целевого объекта может обмануть наивную систему оценки без " +"восстановления полезного исходного кода." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}Улучшение модели и улучшение её проверяющего инструмента — части одной " +"исследовательской задачи.{ph1} Чем лучше модель оптимизирует получаемую " +"обратную связь, тем важнее понимать саму эту связь." + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "Сравнение, которое стоит провести" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"Имея эти проверки, я бы начал сравнение с фиксированного семейства задач " +"VC7.1. Во-первых, я бы установил базовые показатели для сильной модели ИИ " +"общего назначения и для более мелкой модели. Добавление знаний о репозитории" +" в меньшую модель покажет, какие изменения происходят в извлечении. Отсюда " +"контролируемая специализация и версия, обученная RL, позволят нам изучить, " +"что добавляет обучение." + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"Чтобы сделать эти сравнения информативными, каждая модель должна получить " +"одни и те же целевые доказательства и инструменты. Для сравнений по " +"конкретному обучению я бы также оставил фиксированной базовую модель и " +"доступные знания. Повторные испытания помогут отличить надежную стратегию " +"поиска от возможного решения, которое окажется успешным случайно." + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"Затем я бы разделил два начальных условия: восстановление функции из ее " +"инструкций и закрытие последних нескольких различий в поддерживаемом " +"исходном коде. Специализированная модель ИИ может превосходно справиться со " +"второй задачей, в то время как модель ИИ общего назначения выполняет большую" +" часть работы, необходимой для ее достижения. Измерение их по отдельности " +"выявило бы такое разделение работы." + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"Обе задачи требуют тестовых примеров, исключенных из обучения и поиска. " +"Связанные игры Touhou имеют общий код движка. Исключение одной игры из " +"обучения может привести к тому, что в обучающем наборе останутся почти " +"идентичные функции. Вместо этого я бы исключил целые группы связанного " +"исходного кода и исключил бы их проверенные ответы как из обучающих данных, " +"так и из библиотеки поиска." + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"В этих примерах результатом будут полные, независимо проверенные совпадения." +" Равные бюджеты для предложений-кандидатов или попыток компиляции помогут " +"сравнить качество поиска. Измерение общего времени и затрат, включая " +"компиляцию, покажет, обеспечивает ли это качество эффективный рабочий " +"процесс. Затраты на обучение следует оценивать отдельно." + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"Оба измерения имеют значение. Дешевое предложение может привести к " +"дорогостоящему рабочему процессу, если оно приведет к увеличению количества " +"компиляций, не содержащих полезной информации. И наоборот, более медленная " +"модель может быть более экономичной, если она быстрее обнаружит полезный " +"эксперимент. Сравнение должно включать эти затраты." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"Подсчет принятых результатов также позволяет учитывать промежуточный " +"прогресс. Один TH09 {ph0}кандидат на воспроизведение-сохранение{ph1} " +"выглядел многообещающе, поскольку скомпилированная функция имела ожидаемый " +"размер, но полное сравнение все же выявило существенные различия. Изменения " +"в том, как компилятор организует инструкции, также могут создать " +"впечатление, что кандидат ближе к цели, не решая при этом основную проблему." +" Такая диагностика может помочь в следующем эксперименте, но я бы сопоставил" +" любую награду за «приближение» с полной приемочной проверкой." + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"Меня бы заинтересовала закономерность этих результатов. Если " +"специализированная модель ИИ хорошо работает в знакомом семействе " +"компиляторов, сохранится ли это преимущество при изменении инструментальной " +"цепочки? И как только обе модели ИИ будут хорошо воспроизводиться, сколько " +"еще добавится обучение? Ответы помогут нам решить, где тот или иной тип " +"модели наиболее полезен." + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "О каком ИИ-агенте идет речь?" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"Если модели окажутся полезными на разных этапах, полезным механизмом может " +"стать широкий агент с доступом к специализированной модели ИИ. Как только " +"задача становится привычным поиском компилятора, он может запросить " +"кандидатов у специализированной модели ИИ. Если результаты перестанут иметь " +"смысл, более широкий агент может вернуться к целевым данным или исследовать " +"само измерение." + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"В таком случае специализация становится инструментом, который агент может " +"выбирать по мере развития расследования. Агент сохраняет свободу менять " +"направление, которое сделало возможными предыдущие прорывы." + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"Это естественным образом следует из промышленного сдвига, который я описал в" +" предыдущем эссе. Возможность проводить больше экспериментов меняет то, как " +"мы организуем работу. Это также дает нам повод задаться вопросом, какие " +"решения требуют широкого суждения, а какие улучшаются за счет повторного " +"опыта работы с конкретным ремеслом." + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"Пока самая сильная универсальная модель заслужила место в моём рабочем " +"процессе, решив проблемы, которые прежде оставались нерешёнными. Эти успехи " +"побуждают меня изучить процесс внимательнее: какая часть зависела от широты " +"её возможностей, а какую могла бы выполнить модель, обученная именно такой " +"реконструкции?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}Репозитории дают нам возможность проверить вопрос.{ph1} Я хотел бы " +"использовать эту запись для сравнения и посмотреть, что мы узнаем." + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "Доказательства и атрибуция моделей" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"В примерах описаны исторические сравнения на уровне функций, записанные в " +"связанных коммитах. Проверки во время выполнения всей игры — это отдельные " +"этапы. Источник и соответствующие записи были проверены 13 октября 2026 " +"года." + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"Метка TH09 {ph0} означает GPT-6-Astra. Модель, связанная с {ph1}, со " +"временем изменилась, поэтому сама по себе метка не может идентифицировать " +"модель, использованную в конкретном прогоне. Явные названия моделей в других" +" цитируемых коммитах обеспечивают записанную атрибуцию. Эти истории не " +"включают в себя последовательные измерения затрат или усилий для каждого " +"запуска." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 время жизни временных объектов:{ph1} {ph2}предыдущая проверка " +"контекста{ph3} и {ph4}полная реконструкция, подтвержденная чистой " +"перекомпиляцией из исходного кода{ph5}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"Обратный вызов {ph0}TH09 Front:{ph1} {ph2}диагностические области, которые " +"не соответствуют{ph3}, {ph4}Astra новому графу потока управления{ph5} и его " +"{ph6}принятой проверке перекомпиляции{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}Знание компилятора:{ph1} {ph2}TH095 повторное использование поведения " +"сегмента идентификаторов{ph3}, {ph4}TH04 исходный код игрока и порядок " +"компоновщика{ph5} и полная реконструкция {ph6}Luna функции создания врагов " +"TH09{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}Проверки и диагностика:{ph1} {ph2}TH08 аудит констант с плавающей " +"запятой{ph3}, {ph4}TH08 численное исследование{ph5} и {ph6}TH09 " +"реконструкция с сохранением повторов{ph7}." + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"Предлагаемые сравнения специализированных моделей и обучение RL — это " +"будущие эксперименты. Их мотивирует история проектов. Для этого эссе не " +"оценивалась обученная специализированная модель." + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "Нужна ли реверс-инжинирингу модель большего размера? | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"Чему реконструкция Touhou научила меня об общих моделях, специализированных " +"моделях и умении предлагать лучшие гипотезы на основе отзывов компилятора." + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "На этой странице" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "Эссе «Модели и реверс-инжиниринг»" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "Избранные эксперименты по реконструкции" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "Наверх" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "Автор: N0zoM1z0 · 13 октября 2026 г. · Черновик для обсуждения" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/th/messages.po b/website/locales/th/messages.po index 9b63e4e3e..d6497b9b1 100644 --- a/website/locales/th/messages.po +++ b/website/locales/th/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "ความคิดเกี่ยวกับวิศวกรรมย้อนกลับ" #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "ฟื้นฟูเกม{ph0}เรียงความ" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "การปฏิวัติอุตสาหกรรมในวิศวกรรมย้อนกลับ" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,20 @@ msgstr "" "สิ่งที่คู่มือใหม่และ " "reconstructingTH08,TH095และTH04สอนฉันเกี่ยวกับเจ้าหน้าที่ปลายทางและความรู้นั่นรประกอบที่อีกฝั่งยิ่งสนง.โครงการ." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "อ่านบทความ" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "โดย N0zoM1z0 ·10 ตุลาคม 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -100,6 +101,8 @@ msgstr "" "และเจ้าหน้าที่ปัญญาประดิษฐ์" #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -135,6 +138,7 @@ msgstr "" "REA-หาวิธีการทำงานของซอฟต์แวร์ วิศวกรรมย้อนกลับกับตัวแทนการเขียนโค้ดของคุณ" #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -155,6 +159,7 @@ msgid "REA home" msgstr "หน้าแรก REA" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -175,6 +180,7 @@ msgid "Main navigation" msgstr "เมนูหลัก" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -195,6 +201,7 @@ msgid "REA on GitHub" msgstr "REA บน GitHub" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -215,6 +222,7 @@ msgid "Join REA on Discord" msgstr "เข้าร่วม REA บน Discord" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -234,7 +242,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "ติดตาม REA บน X" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "กลับขึ้นด้านบน" @@ -11376,6 +11384,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "ระยะห่างเดียวกัน:256÷16=16 หน่วยมุม=22.5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -11396,6 +11405,7 @@ msgid "Showcases" msgstr "ตัวอย่าง" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -11416,6 +11426,7 @@ msgid "Blog" msgstr "บล็อก" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -11436,6 +11447,7 @@ msgid "Get started" msgstr "เริ่มต้น" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -11455,7 +11467,8 @@ msgctxt "shared" msgid "Guides" msgstr "คู่มือ" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11475,7 +11488,8 @@ msgctxt "shared" msgid "Source" msgstr "ซอร์สโค้ด" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11495,7 +11509,8 @@ msgctxt "shared" msgid "MIT license" msgstr "สัญญาอนุญาต MIT" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11515,7 +11530,8 @@ msgctxt "shared" msgid "Join the community" msgstr "เข้าร่วมชุมชน" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12500,8 +12516,1319 @@ msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "โดย N0zoM1z0 · 13 ตุลาคม 2026 · ฉบับร่างสำหรับตรวจทาน" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "โดย N0zoM1z0 · 13 ตุลาคม 2026" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "ข้ามไปที่เนื้อหา" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "{ph0}Home{ph1} / {ph2}Blog{ph3} / โมเดลและวิศวกรรมย้อนกลับ" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "เรียงความ" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "วิศวกรรมย้อนกลับจำเป็นต้องมีโมเดลที่ใหญ่กว่านี้หรือไม่?" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "โดย N0zoM1z0 · 13 ตุลาคม 2026" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "ประเด็นสำคัญ" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}ความสามารถของโมเดลส่งผลต่อสมมติฐานที่เอเจนต์ AI สามารถเสนอได้{ph1} " +"เซสชันต่อมาได้แก้ไขปัญหาที่งานก่อนหน้านี้ยังไม่ได้รับการแก้ไข" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}คอมไพเลอร์ที่คุ้นเคยนำเสนอรูปแบบที่เป็นรูปธรรมสำหรับโมเดลเฉพาะทางในการเรียนรู้{ph1}" +" " +"การจับคู่โค้ดเครื่องที่แน่นอนบางรายการขึ้นอยู่กับรูปแบบซอร์สโค้ดเฉพาะที่สามารถนำมาใช้ซ้ำได้" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}รวมความรู้ของพื้นที่เก็บข้อมูลไว้ในการเปรียบเทียบที่ยุติธรรม{ph1} " +"โมเดลขนาดเล็กที่มีการดึงความรู้ที่ดีอาจมีข้อมูลเพียงพอที่จะดำเนินการต่อไป" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}ฝึกแบบจำลองเพื่อเลือกการทดลองที่มีประโยชน์ถัดไป " +"และทดสอบเครื่องมือตรวจสอบการยืนยัน{ph1} " +"ความคิดเห็นจะมีประโยชน์เฉพาะในขอบเขตที่ผู้ตรวจสอบตรวจสอบสิ่งที่ถูกต้องเท่านั้น" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}ประเมินผลการฟื้นฟูที่ได้รับการยืนยันควบคู่ไปกับเวลาและต้นทุน{ph1} " +"นั่นคือสิ่งที่ฉันต้องการวัดสำหรับเวิร์กโฟลว์เอเจนต์ AI ที่สมบูรณ์" + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "ในบทความนี้" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}โมเดลที่เก่งขึ้นเปลี่ยนอะไรบ้าง?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}ทำไมจึงมีความเชี่ยวชาญ?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}ประสบการณ์นี้ควรเก็บไว้ที่ไหน?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}เรียนรู้การทดลองที่มีประโยชน์ครั้งต่อไป{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}วิศวกรรมย้อนกลับและการเสริมกำลัง{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}เมื่อตัวตรวจสอบความถูกต้องผิดพลาด{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}การเปรียบเทียบที่คุ้มค่า{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}สิ่งนี้แนะนำเอเจนต์ AI ประเภทใด{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "โมเดลที่เก่งขึ้นเปลี่ยนอะไรบ้าง?" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"ตั้งแต่เดือนสิงหาคม 2026 " +"ฉันได้สำรวจว่าตัวแทนสามารถดำเนินการวิศวกรรมย้อนกลับได้มากเพียงใดโดยมีสิทธิ์เข้าถึงโปรแกรมและวิธีการตรวจสอบงาน" +" ประสบการณ์ส่วนใหญ่ของฉันมาจากการสร้างเกมขึ้นใหม่ โดยเฉพาะ Touhou " +"ขณะที่ฉันเปลี่ยนโมเดล " +"พื้นที่เก็บข้อมูลได้เก็บบันทึกว่าแต่ละเซสชันจัดการเพื่อกู้คืนอะไรบ้าง" + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"งานส่วนใหญ่มีเป้าหมายให้ซอร์สโค้ดที่สร้างขึ้นคอมไพล์แล้วตรงกับต้นฉบับ " +"เราใช้ชุดเครื่องมือจากยุคนั้นโดยกำหนดเวอร์ชันไว้แน่นอน " +"แล้วตรวจว่าผลลัพธ์ตรงกับฟังก์ชันเป้าหมายหรือไม่ " +"เมื่อโมเดลรุ่นใหม่แก้จุดที่งานก่อนหน้าติดอยู่ได้ " +"การเปลี่ยนแปลงซอร์สโค้ดช่วยให้เห็นว่าอะไรทำให้ก้าวหน้าได้" + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "ตัวแปรชั่วคราวจำเป็นต้องมีอายุการใช้งานที่ทับซ้อนกัน" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"ลอจิกของฟังก์ชันการโจมตี TH09 ส่วนใหญ่ได้ถูกสร้างขึ้นใหม่แล้ว " +"แต่คอมไพเลอร์ยังคงเก็บผลลัพธ์ชั่วคราวแตกต่างไปจากต้นฉบับ " +"ปัญหาที่เหลืออยู่คือผลลัพธ์เหล่านั้นต้องใช้เวลานานเท่าใดจึงจะคงอยู่ได้ " +"และเมื่อใดจึงจะสามารถนำพื้นที่จัดเก็บกลับมาใช้ใหม่ได้" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"บันทึกเซสชัน {ph0}GPT-6-Sol{ph1} " +"รายงานความไม่ตรงกันนี้หลังจากทดสอบการจัดเตรียมซอร์สโค้ดทั่วไปหลายรายการ " +"เมื่อ {ph2}GPT-6.1-Sol ตรวจสอบปัญหาอีกครั้ง{ph3} " +"ก็เสนอสมมติฐานที่เป็นประโยชน์เกี่ยวกับช่วงอายุของตัวแปรชั่วคราวเหล่านี้" + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"ฟังก์ชันนี้สร้างผลลัพธ์ชั่วคราวและคัดลอกไปยังตำแหน่งแอนิเมชัน ในภาษา C++ " +"ผลลัพธ์ชั่วคราวจะคงอยู่จนกว่านิพจน์ที่ใช้จะเสร็จสิ้น " +"การจัดกลุ่มการดำเนินการจริงให้เป็นนิพจน์เดียวทำให้ผลลัพธ์เหล่านั้นคงอยู่ร่วมกัน" +" " +"ดังนั้นคอมไพลเลอร์จึงไม่สามารถใช้พื้นที่เก็บข้อมูลเดียวกันซ้ำได้อีกต่อไปสำหรับการดำเนินการทั้งหมด" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"ผลลัพธ์หนึ่งรายการยังคงแชร์พื้นที่เก็บข้อมูลด้วยค่าที่ใช้ในภายหลังสำหรับการตรวจสอบการชนกัน" +" การให้คุณค่านั้นตลอดอายุการใช้งานจะช่วยแก้ไขการทับซ้อนที่เหลือ " +"หลังจากการทดลองที่กำหนดเป้าหมายหลายครั้ง " +"{ph0}การคอมไพล์ใหม่ทั้งหมดจากแหล่งที่มายืนยันว่าฟังก์ชันทั้งหมดตรงกัน{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"ฉันสนใจว่าการค้นพบและการทดสอบสนับสนุนซึ่งกันและกันอย่างไร " +"สมมติฐานเกี่ยวกับอายุการใช้งานของวัตถุชั่วคราวทำให้เอเจนต์ AI มีทิศทางใหม่ " +"คำติชมของคอมไพเลอร์ช่วยปรับแต่งสมมติฐานนั้นให้กลายเป็นการจัดเรียงซอร์สโค้ดที่ตรงกัน" + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "กราฟโฟลว์ควบคุมที่แตกต่างกัน" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"ฟังก์ชัน TH09 ตัวที่สองแสดงให้เห็นว่าเหตุใดทิศทางใหม่จึงมีความสำคัญ " +"อุปสรรคอยู่ที่ว่าหลายสาขาจะแบ่งปันผลงานชิ้นสุดท้ายของพวกเขาอย่างไร " +"งานก่อนหน้าของ {ph0}GPT-6.1-Sol{ph1} มีฟังก์ชันบางส่วนที่เข้ากัน " +"แต่กิ่งก้านยังคงเชื่อมกันแตกต่างไปจากต้นฉบับ" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra เปลี่ยนกราฟโฟลว์ควบคุมโดยรอบ{ph1} " +"เส้นทางหลักตอนนี้นำไปสู่การอัปเดตและการกลับมาครั้งสุดท้ายที่ใช้ร่วมกัน " +"ในโครงสร้างต้นทางนั้น คอมไพลเลอร์จะรวมสาขาต่างๆ ในลักษณะเดียวกับต้นฉบับ " +"ฟังก์ชันที่สมบูรณ์ตรงกัน และ " +"{ph2}การตรวจสอบการสร้างใหม่แยกกันยืนยันแล้ว{ph3}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}การเปลี่ยนการแสดงทำให้พื้นที่การค้นหามีประสิทธิผลมากขึ้น{ph1} " +"ทั้งสองกรณีให้เหตุผลแก่ฉันในการให้ความสำคัญกับความสามารถของโมเดลทั่วไปอย่างจริงจัง:" +" " +"ตัวแทนสามารถลองใช้รูปแบบท้องถิ่นต่อไปเป็นเวลานานก่อนที่จะพบวิธีที่ดีกว่าในการแสดงงานโดยรอบ" + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"ในเวลาเดียวกัน " +"เซสชันต่อมาเริ่มต้นด้วยแหล่งที่มาก่อนหน้าและความล้มเหลวที่บันทึกไว้ " +"งานที่สะสมไว้นั้นเป็นส่วนหนึ่งของความได้เปรียบของพวกเขา " +"ประวัติศาสตร์ช่วยให้เราตรวจสอบการค้นพบและการตรวจสอบของพวกเขา " +"การเปรียบเทียบแบบควบคุมจะช่วยแยกผลกระทบของแบบจำลองออกจากประสบการณ์ที่สืบทอดมา" + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"นี่คือสิ่งที่ทำให้คำถามน่าสนใจสำหรับฉัน " +"ความคืบหน้าบางอย่างมาจากการค้นหาตัวแทนใหม่ " +"งานที่เหลือส่วนใหญ่เกี่ยวข้องกับการทดสอบการจัดเตรียมซอร์สโค้ดที่แตกต่างกันกับคอมไพเลอร์ที่คุ้นเคย" +" สิ่งนี้ต้องใช้ความสามารถในการให้เหตุผลแบบกว้างๆ มากน้อยเพียงใด? " +"โมเดลสามารถเรียนรู้ได้มากเพียงใดจากการทำงานซ้ำๆ กับคอมไพเลอร์และโค้ดนั้น" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "เหตุใดโมเดล AI เฉพาะทางจึงทำได้ดี" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou ให้เหตุผลที่ชัดเจนแก่เราในการคาดหวังว่าประสบการณ์นั้นจะช่วยได้ " +"รหัสเกมส่วนใหญ่มาจากผู้เขียนคนเดียวกัน ZUN ดังนั้นนิสัยที่เกิดขึ้นซ้ำๆ " +"จึงเป็นสมมติฐานเริ่มต้นที่สมเหตุสมผล " +"แหล่งเก็บข้อมูลสนับสนุนด้วยตัวอย่างที่เป็นประโยชน์: " +"ลูปที่เกี่ยวข้องและการจัดเตรียมตัวช่วยได้ให้สมมติฐานแก่เกมต่างๆ" + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"รูปแบบแหล่งที่มาเหล่านั้นมีความเฉพาะเจาะจงมากขึ้นเมื่อเรารวมคอมไพเลอร์ " +"ภายใต้เวอร์ชันและชุดตัวเลือกเฉพาะ การเปลี่ยนแปลงเล็กๆ น้อยๆ " +"ในแหล่งที่มาอาจทำให้เกิดความแตกต่างที่ทำซ้ำได้ในรหัสเครื่อง " +"ความแตกต่างบางประการนั้นคลุมเครืออย่างน่าประหลาดใจ" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"ตัวอย่างเช่น ใน TH095 {ph0}แม้แต่ชื่อของตัวแปรโลคัลก็อาจส่งผลต่อตำแหน่งที่ " +"VC7.1 เก็บไว้{ph1} การย้ายการประกาศไม่ได้ช่วยแก้ปัญหารูปแบบการจัดเก็บข้อมูล " +"อย่างไรก็ตาม " +"ฟังก์ชันพื้นหลังก่อนหน้านี้ได้เปิดเผยรูปแบบการตั้งชื่อที่ส่งผลต่อการเลือกของคอมไพเลอร์" +" นั่นเสนอสมมติฐานที่มีแนวโน้มมากขึ้นสำหรับ {ph2}การสร้าง {ph3}{ph4} ใหม่" + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"การใช้รูปแบบนั้นซ้ำสำหรับตัวแปรขนาดไฟล์ทำให้เกิดเค้าโครงที่ต้องการโดยไม่ต้องเพิ่มพื้นที่เก็บข้อมูล" +" ฟังก์ชั่นครบถ้วนจึงเข้าคู่กัน" + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"นี่เป็นรายละเอียดเฉพาะเจาะจงมากที่โมเดล AI " +"เอนกประสงค์อาจไม่ทราบเมื่อพบปัญหาครั้งแรก " +"โมเดลเฉพาะสำหรับคอมไพลเลอร์นี้สามารถเห็นกรณีดังกล่าวได้มากมาย " +"และเรียนรู้ว่าซอร์สโค้ดรูปแบบใดที่ควรค่าแก่การทดสอบก่อน" + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"โอกาสมีความชัดเจนโดยเฉพาะอย่างยิ่งในปัญหาการจับคู่ขั้นสุดท้ายที่เราเข้าใจพฤติกรรมอยู่แล้ว" +" " +"จากนั้นเราจะถามว่าการจัดเรียงซอร์สโค้ดธรรมดาใดที่คอมไพเลอร์นี้เปลี่ยนเป็นคำสั่งที่สังเกตได้" +" " +"ความเชี่ยวชาญพิเศษสามารถช่วยแบบจำลองเสนอรูปแบบเหล่านั้นได้อย่างมีประสิทธิภาพมากขึ้น" +" ในขณะที่เป้าหมายยังคงกำหนดว่าสมมติฐานใดที่จะอยู่รอดได้" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"รูปแบบที่ใช้ได้ต้องมีบริบทกำกับ TH08 ใช้ Visual C++ รุ่นเก่ากว่า TH095 และ " +"TH09 ส่วน TH04 ใช้คอมไพเลอร์อีกตัวหนึ่งจากยุค DOS " +"{ph0}การสร้างโค้ดผู้เล่นขึ้นใหม่{ph1} จึงต้องใช้สมมติฐานต่างออกไป " +"ทั้งเรื่องรูปแบบซอร์สโค้ดและการลิงก์เป็นไฟล์ปฏิบัติการ " +"ผู้เขียนคนเดียวกันให้ข้อสันนิษฐานเริ่มต้น " +"แต่ต้องตรวจในแต่ละเป้าหมายว่าใช้ได้หรือไม่ " +"โมเดลเฉพาะทางต้องเรียนรู้ความแตกต่างเหล่านี้" + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "ประสบการณ์นี้ควรเก็บไว้ที่ไหน?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"ก่อนที่จะตัดสินใจสอนรูปแบบเหล่านั้นผ่านการฝึกโมเดล " +"ฉันจะดูความรู้ที่เราเก็บไว้ในที่เก็บอยู่แล้ว ใน {ph0}เรียงความก่อนหน้า{ph1} " +"ฉันเขียนเกี่ยวกับการที่กลายเป็นความทรงจำในการทำงานของโครงการ " +"หน่วยความจำดังกล่าวประกอบด้วยบทเรียนเกี่ยวกับคอมไพเลอร์ " +"พร้อมด้วยแหล่งที่มาและการทดลองที่ล้มเหลวซึ่งทำให้มีประโยชน์" + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"ด้วยการเรียกบันทึกนั้น " +"ตัวแทนสามารถเริ่มต้นด้วยผลลัพธ์ที่ทราบและตรวจสอบว่าเราเข้าถึงได้อย่างไร " +"นอกจากนี้ยังสามารถหลีกเลี่ยงทางเลือกอื่นที่ล้มเหลวในบริบทเดียวกันได้ " +"จุดเริ่มต้นจะดีขึ้นโดยไม่ต้องเปลี่ยนพารามิเตอร์ที่ได้รับการฝึกของโมเดล" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"ในขณะที่สร้างฟังก์ชันของ TH09 ขึ้นมาใหม่เพื่อสร้างศัตรู เซสชัน " +"{ph0}GPT-6-Luna-max{ph1} ได้ตั้งสมมุติฐานจากซอร์สโค้ด TH08 ที่เกี่ยวข้อง " +"โค้ดก่อนหน้านี้แนะนำวิธีจัดระเบียบลูปและจัดการกับความล้มเหลวภายใน หลักฐานของ" +" TH09 ให้รายละเอียดเฉพาะเกม " +"การสร้างใหม่จากแหล่งที่มายืนยันว่าฟังก์ชันทั้งหมดตรงกัน" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}\"โมเดลไม่ทราบว่านิสัยแปลกๆ ของคอมไพเลอร์\" และ " +"\"โมเดลไม่สามารถใช้งานได้\" เป็นปัญหาที่แตกต่างกัน {ph1} " +"การดึงข้อมูลอาจแก้ปัญหาได้ในวิธีแรก " +"แต่การเปลี่ยนข้อเท็จจริงดังกล่าวให้กลายเป็นตัวเลือกที่มีประโยชน์ยังคงต้องใช้เวลา" +" และหากซอร์สโค้ดของผู้สมัครล้มเหลว " +"โมเดลจะต้องตัดสินใจว่าผลลัพธ์มีความหมายอย่างไรสำหรับความพยายามครั้งถัดไป " +"การรู้รูปแบบที่เกี่ยวข้องเป็นเพียงจุดเริ่มต้นของกระบวนการนั้นเท่านั้น" + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"ด้วยเหตุผลดังกล่าว ฉันจึงรวมโมเดล AI " +"อเนกประสงค์ที่มีขนาดเล็กกว่าพร้อมการดึงข้อมูลที่ดีไว้ด้วยในการเปรียบเทียบ " +"การให้คลังประสบการณ์แบบเดียวกันจะช่วยให้เราระบุได้ว่าการฝึกอบรมใดที่เพิ่มเข้ามา" +" เช่น การสร้างผู้สมัครที่ดีขึ้น การตัดสินใจที่ดีขึ้นหลังจากได้รับคำติชม " +"หรือทั้งสองอย่าง" + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "เรียนรู้การทดลองที่มีประโยชน์ครั้งต่อไป" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"การตัดสินใจเหล่านั้นหลังจากได้รับคำติชมคือสิ่งที่ฉันอยากศึกษามากที่สุด " +"ในการฟื้นฟูแบบเอเจนต์ แบบจำลองจะเสนอคำอธิบายและทดสอบกับโปรแกรม " +"แม้ว่าซอร์สโค้ดของผู้สมัครจะล้มเหลว " +"ผลลัพธ์ก็สามารถจำกัดความเป็นไปได้ที่เหลืออยู่ให้แคบลง " +"และแนะนำการทดลองถัดไปที่มีประโยชน์มากขึ้น" + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"ลองพิจารณาตัวอย่างผลลัพธ์ชั่วคราวอีกครั้ง " +"แหล่งที่มาสุดท้ายแสดงการจัดเตรียมที่ได้ผล " +"แต่เส้นทางไปยังแหล่งข้อมูลนั้นมีข้อมูลเพิ่มเติม " +"การทดลองแต่ละรายการต่อไปนี้ช่วยระบุสิ่งที่ยังจำเป็นต้องเปลี่ยนแปลง" + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "การทดลอง" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "ผลลัพธ์ของมันสอนอะไรเรา" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "ลองวิธีอื่นในการคัดลอกผลลัพธ์" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "รหัสการคัดลอกเปลี่ยนไป แต่ฟังก์ชันยังคงแตกต่างออกไป" + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "ใช้ค่าที่ส่งคืนโดยตรง" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "" +"การคัดลอกเป็นไปตามรูปแบบดั้งเดิม แต่การจัดเก็บชั่วคราวยังคงแตกต่างออกไป" + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "เก็บตัวแปรชั่วคราวทั้งหมดให้ถูกต้องในเวลาเดียวกัน" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "" +"พวกเขาได้รับพื้นที่เก็บข้อมูลแยกต่างหาก " +"แต่ยังมีอีกหนึ่งค่าที่ทับซ้อนกันกับค่าที่ใช้ในภายหลังสำหรับการตรวจสอบการชนกัน" + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "แยกค่าการตรวจสอบการชนกันออกจากกัน" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "การคอมไพล์ใหม่ทั้งหมดจากแหล่งที่มายืนยันการจับคู่ที่สมบูรณ์" + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "" +"การทดลองที่เลือกจากการสร้างฟังก์ชันการโจมตีขึ้นใหม่ด้านบน " +"โดยจัดกลุ่มตามสิ่งที่พวกเขาสอนเรา" + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"บันทึกการฝึกอบรมที่เก็บลำดับนี้สามารถสอนได้มากกว่าแหล่งข้อมูลสุดท้ายเพียงอย่างเดียว" +" " +"โดยเชื่อมโยงการเปลี่ยนแปลงที่เสนอแต่ละรายการกับผลลัพธ์ที่สังเกตได้และการตัดสินใจครั้งต่อไป" +" การรักษารูปแบบที่ไม่มีประสิทธิภาพก็มีความสำคัญเช่นกัน " +"โดยจะแจ้งตัวแทนในภายหลังว่าสมมติฐานใดได้รับการทดสอบแล้วในบริบทนั้น" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}ฉันอยากให้โมเดลเรียนรู้วิธีใช้คำติชม{ph1} " +"ผลลัพธ์ที่เป็นกลางคือเหตุผลที่ต้องพิจารณารูปแบบใหม่อีกครั้ง " +"ความไม่ตรงกันที่เปลี่ยนแปลงอาจชี้ไปที่สาเหตุอื่น " +"เมื่อการแก้ไขเฉพาะที่หยุดให้ข้อมูล " +"ตัวแทนควรสามารถทบทวนการนำเสนอแหล่งที่มาหรือตรวจสอบหลักฐานอีกครั้ง" + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"การเรียนรู้ที่จะเลือกขั้นตอนเหล่านี้ต้องมีอิสระในการทำงาน " +"บางครั้งขั้นตอนที่มีประโยชน์คือการตรวจฟังก์ชันอื่นที่เรียกฟังก์ชันนี้ " +"แทนที่จะแก้โค้ดแล้วคอมไพล์อีกครั้ง " +"ลำดับการเขียนโค้ดใหม่ที่บังคับตายตัวจะตัดทางเลือกนี้ออก " +"รวมถึงพฤติกรรมบางส่วนที่เราอยากให้โมเดลเรียนรู้" + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "เหตุใดวิศวกรรมย้อนกลับจึงน่าสนใจสำหรับการเรียนรู้แบบเสริมกำลัง" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"การสร้างโค้ดขึ้นใหม่ในขอบเขตที่ชัดเจนเป็นสภาพแวดล้อมที่ใช้ศึกษาการเลือกเหล่านี้ได้จริง" +" เมื่อกำหนดเป้าหมายและเวอร์ชันคอมไพเลอร์ไว้แน่นอน " +"การทดลองซอร์สโค้ดหลายแบบจะให้ผลตอบกลับรวดเร็วและทำซ้ำได้ " +"เอเจนต์สามารถคอมไพล์โค้ดที่เสนอ ตรวจส่วนที่ยังต่าง แล้วเลือกขั้นตอนถัดไป" + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"การเรียนรู้แบบเสริมกำลังสามารถฝึกกระบวนการตัดสินใจผ่านคำติชมที่ได้รับ " +"ผู้สมัครที่ยอมรับโดยเช็คที่กำหนดจะให้ผลลัพธ์ที่เป็นรูปธรรมแก่เรา " +"ในขณะที่ลำดับของความพยายามช่วยให้เราสังเกตได้ว่าต้องใช้ความพยายามมากเพียงใดจึงจะบรรลุผลนั้น" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"มีงานวิจัยสนับสนุนแนวทางนี้แล้ว {ph0}LLM4Decompile{ph1} " +"ศึกษาโมเดลที่เชี่ยวชาญการดีคอมไพล์ ส่วน {ph2}AutoDecompiler{ph3} " +"ฝึกการปรับปรุงหลายรอบโดยใช้ผลจากคอมไพเลอร์และการรันโปรแกรม " +"งานเหล่านี้เน้นการกู้คืนพฤติกรรมของโปรแกรม " +"ผมจะใช้เป็นจุดเริ่มต้นสำหรับงานที่เกี่ยวข้อง " +"คือการหาซอร์สโค้ดที่ให้ผลลัพธ์ตรงกับคอมไพเลอร์จากยุคเดิม" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"เพื่อให้บรรลุภารกิจที่กำหนดไว้อย่างชัดเจน " +"เราอาจจำเป็นต้องกำหนดทรัพยากรที่ฟังก์ชันเป็นเจ้าของก่อนหรือสร้างการวัดรันไทม์ใหม่" +" ตัวอย่างเช่น ใน TH08 ความแตกต่าง {ph0}การชนกัน " +"ทำให้ย้อนกลับไปถึงวิธีที่โปรแกรมดั้งเดิมจัดการการคำนวณจุดลอยตัว{ph1} " +"ก่อนที่ตัวแทนจะทดสอบการแก้ไขที่เป็นประโยชน์ได้ ก็ต้องพิจารณาว่าจะวัดอะไรก่อน" + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"สิ่งนี้ชี้ให้เห็นอีกบทบาทหนึ่งของแบบจำลองที่มีความสามารถในวงกว้าง: " +"ช่วยให้สามารถตรวจสอบปัญหาได้ตั้งแต่แรก เมื่อมีการตรวจสอบแล้ว " +"เราจะสามารถตรวจสอบได้ว่านโยบายการค้นหาที่เรียนรู้จะสามารถนำมาใช้ได้ไกลเพียงใด" + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "เมื่อตัวตรวจสอบความถูกต้องผิดพลาด" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"เมื่อใช้ผลตรวจสำหรับการฝึก " +"ขอบเขตของการตรวจก็กลายเป็นส่วนหนึ่งของสภาพแวดล้อมการเรียนรู้ " +"ผมเรียกข้อมูลอ้างอิงหรือเครื่องมือตรวจนี้ว่า oracle แม้คำตอบจะทำซ้ำได้ " +"เราก็ต้องเข้าใจว่ามันตรวจสอบอะไรจริง ๆ" + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 แสดงให้เห็นว่าปัญหาอาจซ่อนอยู่นอกขอบเขตการตรวจ " +"บางฟังก์ชันใช้ค่าคงที่แบบ floating-point ที่ผิด " +"แต่ยังผ่านการเปรียบเทียบแบบตรงกันทุกส่วน " +"ตัวเปรียบเทียบปรับแอดเดรสที่ถูกย้ายก่อนตรวจคำสั่ง " +"แต่ตรวจค่าที่แอดเดรสนั้นชี้ไปเฉพาะกรณีที่บันทึกไว้ให้ตรวจอย่างชัดเจน " +"การอ้างอิงค่าคงที่อื่นจึงอาจผ่านโดยไม่ได้ตรวจค่า" + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"เพื่อแก้ไขช่องว่างนี้ {ph0}การตรวจสอบที่กว้างขึ้น{ph1} " +"พบค่าคงที่ที่ไม่ถูกต้องในฟังก์ชันที่ผ่านการตรวจสอบแล้ว ในการคำนวณครั้งหนึ่ง " +"ซอร์สโค้ดที่สร้างขึ้นใหม่ได้เพิ่ม {ph2} ในขณะที่ซอร์สโค้ดดั้งเดิมเพิ่ม {ph3}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}โมเดลที่ได้รับรางวัลจากการตรวจสอบยืนยันแบบเก่าสามารถเรียนรู้ที่จะสร้างการสร้างใหม่ที่ไม่ถูกต้อง{ph1}" +" เราไม่ได้ฝึกโมเดล RL โดยใช้ตัวตรวจสอบนั้น " +"ความล้มเหลวที่เราสังเกตเห็นคือในการตรวจสอบนั่นเอง แต่ถ้าเราใช้เป็นรางวัล " +"จุดบอดเดียวกันก็จะกลายเป็นส่วนหนึ่งของสภาพแวดล้อมการเรียนรู้" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"การทดสอบการทำงานอาจมีจุดบอดที่คล้ายกัน " +"{ph0}การวิจัยโดยใช้การแยกฟัซซีแบบดิฟเฟอเรนเชียลกับตัวถอดรหัส LLM{ph1} " +"พบความแตกต่างด้านพฤติกรรมนอกเหนือจากการทดสอบที่ให้มา " +"ดังนั้นเราจึงจำเป็นต้องเข้าใจการทดสอบที่ผ่านในแง่ของกรณีที่ผ่านการทดสอบ " +"ในทำนองเดียวกัน การจับคู่คำสั่งจะต้องเข้าใจในแง่ของไบต์และข้อมูลที่ตรวจสอบ" + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"ก่อนที่จะใช้การตรวจสอบยืนยันประเภทใดประเภทหนึ่งเป็นรางวัล " +"ฉันจะทดสอบด้วยผลลัพธ์ที่ไม่ถูกต้องโดยเจตนา " +"ฉันยังจะแยกการยอมรับขั้นสุดท้ายออกจากซอร์สโค้ดที่เอเจนต์ AI สามารถแก้ไขได้ " +"โมเดลจะต้องผ่านการตรวจสอบโดยการสร้างโปรแกรมใหม่อย่างถูกต้อง มิฉะนั้น " +"การคัดลอกอาร์เรย์ไบต์จากเป้าหมายอาจหลอกระบบการให้คะแนนที่ไร้เดียงสาโดยไม่ต้องกู้คืนซอร์สโค้ดที่มีประโยชน์" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}การปรับปรุงโมเดลและการปรับปรุงเครื่องมือตรวจเป็นส่วนของปัญหาวิจัยเดียวกัน{ph1}" +" ยิ่งโมเดลเก่งในการปรับตัวตามผลตอบกลับที่เราให้ " +"ก็ยิ่งสำคัญที่เราต้องเข้าใจผลตอบกลับนั้น" + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "การเปรียบเทียบที่คุ้มค่า" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"เมื่อตรวจสอบแล้ว ฉันจะเริ่มการเปรียบเทียบในกลุ่มงาน VC7.1 ที่คงที่ ก่อนอื่น " +"ฉันจะสร้างพื้นฐานสำหรับโมเดล AI อเนกประสงค์ที่แข็งแกร่งและโมเดลที่เล็กกว่า " +"การเพิ่มความรู้เกี่ยวกับพื้นที่เก็บข้อมูลให้กับโมเดลขนาดเล็กจะแสดงให้เห็นว่าการเปลี่ยนแปลงในการดึงข้อมูลมีอะไรบ้าง" +" จากนั้น ความเชี่ยวชาญพิเศษภายใต้การดูแลและเวอร์ชันที่ได้รับการฝึกอบรม RL " +"จะให้เราตรวจสอบได้ว่าการฝึกอบรมใดบ้างที่เพิ่มเข้ามา" + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"เพื่อให้การเปรียบเทียบเหล่านี้เป็นข้อมูล " +"แต่ละแบบจำลองจะได้รับหลักฐานและเครื่องมือเป้าหมายที่เหมือนกัน " +"สำหรับการเปรียบเทียบเฉพาะการฝึกอบรม " +"ฉันจะเก็บโมเดลพื้นฐานและความรู้ที่มีอยู่ไว้คงที่เช่นกัน การทดลองซ้ำๆ " +"จะช่วยแยกแยะกลยุทธ์การค้นหาที่เชื่อถือได้จากโซลูชันผู้สมัครที่ประสบความสำเร็จโดยบังเอิญ" + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"จากนั้นฉันจะแยกเงื่อนไขเริ่มต้นสองเงื่อนไข: สร้างฟังก์ชันใหม่จากคำแนะนำ " +"และปิดความแตกต่างสองสามข้อสุดท้ายในแหล่งที่มาที่ได้รับการบำรุงรักษา โมเดล AI" +" เฉพาะทางอาจทำงานได้ดีในงานที่สอง ในขณะที่โมเดล AI " +"เอนกประสงค์ทำงานส่วนใหญ่ที่จำเป็นในการเข้าถึงโมเดลนั้น " +"การวัดแยกกันจะเผยให้เห็นการแบ่งส่วนงานนั้น" + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"ทั้งสองงานต้องมีตัวอย่างทดสอบที่ไม่อยู่ในข้อมูลฝึกและแหล่งข้อมูลสำหรับค้นคืน" +" เกม Touhou ที่เกี่ยวข้องใช้โค้ดเอนจินร่วมกัน " +"การกันเกมหนึ่งออกจากข้อมูลฝึกจึงอาจยังเหลือฟังก์ชันที่เกือบเหมือนกันในชุดฝึก" +" ผมจะกันซอร์สโค้ดที่เกี่ยวข้องทั้งกลุ่มออก " +"และไม่ให้คำตอบที่ผ่านการตรวจแล้วอยู่ในข้อมูลฝึกหรือคลังสำหรับค้นคืน" + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"ผลที่วัดควรเป็นการตรงกันครบทั้งฟังก์ชันซึ่งผ่านการตรวจอย่างอิสระ " +"ให้แต่ละโมเดลมีงบจำนวนโค้ดที่เสนอหรือจำนวนครั้งที่คอมไพล์เท่ากันเพื่อเปรียบเทียบคุณภาพการค้นหา" +" แล้ววัดเวลารวมและต้นทุนรวมซึ่งรวมการคอมไพล์ด้วย " +"เพื่อดูว่าคุณภาพนั้นทำให้งานมีประสิทธิภาพจริงหรือไม่ " +"ต้นทุนการฝึกต้องนับแยกต่างหาก" + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"การวัดทั้งสองมีความสำคัญ ข้อเสนอราคาถูกอาจนำไปสู่ขั้นตอนการทำงานที่มีราคาแพง" +" หากทำให้เกิดการคอมไพล์มากขึ้นซึ่งไม่ได้ให้ข้อมูลที่เป็นประโยชน์ " +"ในทางกลับกัน " +"โมเดลที่ช้ากว่าอาจประหยัดกว่าหากพบว่าการทดสอบมีประโยชน์เร็วกว่านี้ " +"การเปรียบเทียบควรรวมต้นทุนเหล่านี้ด้วย" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"การนับผลลัพธ์ที่ยอมรับยังช่วยรักษาความก้าวหน้าระดับกลางในมุมมองด้วย TH09 " +"{ph0}บันทึกการเล่นซ้ำ-บันทึกผู้สมัคร{ph1} " +"หนึ่งตัวดูมีแนวโน้มดีเนื่องจากฟังก์ชันที่คอมไพล์มีขนาดตามที่คาดหวัง " +"แต่การเปรียบเทียบแบบเต็มยังคงพบความแตกต่างอย่างมาก " +"การเปลี่ยนแปลงวิธีที่คอมไพลเลอร์จัดเรียงคำสั่งยังสามารถทำให้ผู้สมัครมองเข้าไปใกล้มากขึ้นโดยไม่ต้องแก้ไขปัญหาที่ซ่อนอยู่" +" การวินิจฉัยดังกล่าวสามารถเป็นแนวทางในการทดสอบครั้งต่อไปได้ " +"แต่ฉันจะตรวจสอบรางวัลสำหรับการ \"เข้าใกล้\" " +"เทียบกับการตรวจสอบการยอมรับทั้งหมด" + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"ฉันจะสนใจรูปแบบของผลลัพธ์เหล่านี้ หากโมเดล AI " +"เฉพาะทางทำงานได้ดีในตระกูลคอมไพเลอร์ที่คุ้นเคย " +"ข้อได้เปรียบนั้นจะรอดพ้นจากการเปลี่ยนแปลง toolchain หรือไม่ และเมื่อ AI " +"ทั้งสองโมเดลดึงข้อมูลได้ดีแล้ว การฝึกยังเพิ่มอีกเท่าไหร่? " +"คำตอบจะช่วยให้เราตัดสินใจได้ว่าโมเดลแต่ละประเภทมีประโยชน์มากที่สุดที่ใด" + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "สิ่งนี้แนะนำเอเจนต์ AI ประเภทใด" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"หากโมเดลต่างๆ กลายเป็นความช่วยเหลือในขั้นตอนต่างๆ " +"การจัดการที่เป็นประโยชน์อาจเป็นตัวแทนในวงกว้างที่สามารถเข้าถึงโมเดล AI " +"เฉพาะทางได้ เมื่องานกลายมาเป็นการค้นหาคอมไพเลอร์ที่คุ้นเคย " +"งานนั้นก็สามารถถามโมเดล AI เฉพาะทางสำหรับผู้สมัครได้ หากผลลัพธ์ไม่สมเหตุสมผล" +" สารในวงกว้างสามารถกลับไปยังหลักฐานเป้าหมายหรือตรวจสอบการวัดเองได้" + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"ในการจัดการดังกล่าว " +"ความเชี่ยวชาญเฉพาะทางจะกลายเป็นเครื่องมือที่ตัวแทนสามารถเลือกได้ในขณะที่การสืบสวนพัฒนาขึ้น" +" ตัวแทนรักษาอิสระในการเปลี่ยนทิศทางที่ทำให้ความก้าวหน้าก่อนหน้านี้เป็นไปได้" + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"สิ่งนี้เป็นไปตามธรรมชาติจากการเปลี่ยนแปลงทางอุตสาหกรรมที่ฉันอธิบายไว้ในบทความก่อนหน้านี้" +" ความสามารถในการทำการทดสอบอื่นๆ " +"อีกมากมายได้เปลี่ยนแปลงวิธีที่เราจัดระเบียบงาน " +"นอกจากนี้ยังให้เหตุผลแก่เราในการถามว่าการตัดสินใจใดต้องใช้วิจารณญาณกว้างๆ " +"และการตัดสินใจครั้งใดจะปรับปรุงผ่านประสบการณ์ซ้ำแล้วซ้ำเล่ากับงานฝีมือเฉพาะอย่าง" + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"ตอนนี้โมเดลทั่วไปที่เก่งที่สุดมีที่ยืนในวิธีทำงานของผม " +"เพราะแก้ปัญหาที่ก่อนหน้านี้ผมยังแก้ไม่ได้ " +"ความสำเร็จเหล่านี้ทำให้ผมอยากศึกษางานให้ละเอียดขึ้น " +"ว่ามากน้อยแค่ไหนต้องอาศัยความสามารถที่กว้างของโมเดล " +"และมากน้อยแค่ไหนอาจให้โมเดลที่ฝึกเฉพาะงานสร้างโค้ดแบบนี้ทำได้" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}ที่เก็บข้อมูลช่วยให้เรามีวิธีทดสอบคำถาม{ph1} " +"ฉันต้องการใช้บันทึกนั้นเพื่อสร้างการเปรียบเทียบและดูสิ่งที่เราเรียนรู้" + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "หลักฐานและการระบุแหล่งที่มาของแบบจำลอง" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"ตัวอย่างจะอธิบายการเปรียบเทียบระดับฟังก์ชันในอดีตที่บันทึกไว้ในการคอมมิตที่เชื่อมโยง" +" การตรวจสอบรันไทม์ทั้งเกมถือเป็นเหตุการณ์สำคัญที่แยกจากกัน " +"แหล่งที่มาและบันทึกการจับคู่ได้รับการตรวจสอบเมื่อวันที่ 13 ตุลาคม พ.ศ. 2569" + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"ป้ายกำกับ TH09 {ph0} หมายถึง GPT-6-Astra โมเดลที่เกี่ยวข้องกับ {ph1} " +"มีการเปลี่ยนแปลงเมื่อเวลาผ่านไป " +"ดังนั้นป้ายกำกับเพียงอย่างเดียวจึงไม่สามารถระบุโมเดลที่ใช้ในการดำเนินการเฉพาะได้" +" ชื่อโมเดลที่ชัดเจนในคอมมิตที่อ้างถึงอื่นๆ ให้การระบุแหล่งที่มาที่บันทึกไว้ " +"ประวัติเหล่านี้ไม่ได้รวมการวัดต้นทุนหรือความพยายามอย่างสม่ำเสมอสำหรับการวิ่งทุกครั้ง" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 อายุการใช้งานของวัตถุชั่วคราว:{ph1} " +"{ph2}การตรวจสอบบริบทก่อนหน้า{ph3} และ " +"{ph4}การสร้างใหม่เสร็จสมบูรณ์ได้รับการตรวจสอบโดยการคอมไพล์ใหม่ทั้งหมดจากแหล่งที่มา{ph5}" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}คอลแบ็ก Front ของ TH09:{ph1} {ph2}บริเวณที่ตรวจพบว่ายังไม่ตรง{ph3}, " +"{ph4}กราฟการควบคุมใหม่ของ Astra{ph5} และ " +"{ph6}การตรวจคอมไพล์ซ้ำที่ผ่านการยอมรับ{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}ความรู้เกี่ยวกับคอมไพเลอร์:{ph1} {ph2}TH095 " +"นำพฤติกรรมตัวระบุ-บัคเก็ตกลับมาใช้ใหม่{ph3}, {ph4}TH04 " +"ซอร์สโค้ดของผู้เล่นและลำดับลิงก์เกอร์{ph5} และ {ph6}Luna " +"การสร้างศัตรูใหม่ของ TH09 ใหม่ทั้งหมด ฟังก์ชั่น{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}การตรวจสอบและการวินิจฉัย:{ph1} {ph2}TH08 " +"การตรวจสอบคงที่แบบจุดลอยตัว{ph3}, {ph4}TH08 การตรวจสอบเชิงตัวเลข{ph5} และ " +"{ph6}TH09 การสร้างใหม่เพื่อประหยัดการเล่นซ้ำ{ph7}" + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"การเปรียบเทียบที่เสนอของรุ่นพิเศษและการฝึกอบรม RL เป็นการทดลองในอนาคต " +"ประวัติโครงการเป็นแรงบันดาลใจให้พวกเขา " +"ไม่มีการประเมินแบบจำลองเฉพาะทางที่ได้รับการฝึกอบรมสำหรับบทความนี้" + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "วิศวกรรมย้อนกลับจำเป็นต้องมีโมเดลที่ใหญ่กว่านี้หรือไม่? | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"สิ่งที่ Touhou การสร้างใหม่สอนฉันเกี่ยวกับโมเดลทั่วไป โมเดลผู้เชี่ยวชาญ " +"และการเรียนรู้ที่จะเสนอสมมติฐานที่ดีกว่าจากคำติชมของคอมไพเลอร์" + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "ในหน้านี้" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "แบบจำลองและเรียงความวิศวกรรมย้อนกลับ" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "การทดลองสร้างใหม่ที่เลือก" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "กลับขึ้นด้านบน" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "โดย N0zoM1z0 · 13 ตุลาคม 2026 · ฉบับร่างสำหรับตรวจทาน" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/tr/messages.po b/website/locales/tr/messages.po index da4084320..cd1330e3e 100644 --- a/website/locales/tr/messages.po +++ b/website/locales/tr/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Tersine mühendislik üzerine düşünceler." #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Oyun rekonstrüksiyonu{ph0}Deneme" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "Tersine mühendislikte endüstri devrimi" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -53,19 +53,20 @@ msgstr "" "yapılandırılması bana projeler arasında birleşen aracılar, doğrulama ve " "bilgi hakkında bilgi verdi." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Makaleyi okuyun" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Başlangıç N0zoM1z0 * 10 Ekim 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -101,6 +102,8 @@ msgstr "" "deneyimlerden yararlanma." #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -137,6 +140,7 @@ msgstr "" "mühendislik yapın." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -157,6 +161,7 @@ msgid "REA home" msgstr "REA ana sayfası" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -177,6 +182,7 @@ msgid "Main navigation" msgstr "Ana menü" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -197,6 +203,7 @@ msgid "REA on GitHub" msgstr "GitHub’da REA" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -217,6 +224,7 @@ msgid "Join REA on Discord" msgstr "REA Discord topluluğuna katıl" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -236,7 +244,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "REA’yı X’te takip et" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Başa dön" @@ -11605,6 +11613,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "Aynı aralık: 256 ÷ 16 = 16 açı birimi = 22,5 °" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -11625,6 +11634,7 @@ msgid "Showcases" msgstr "Örnekler" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -11645,6 +11655,7 @@ msgid "Blog" msgstr "Blog" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -11665,6 +11676,7 @@ msgid "Get started" msgstr "Başlangıç" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -11684,7 +11696,8 @@ msgctxt "shared" msgid "Guides" msgstr "Rehberler" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11704,7 +11717,8 @@ msgctxt "shared" msgid "Source" msgstr "Kaynak kod" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11724,7 +11738,8 @@ msgctxt "shared" msgid "MIT license" msgstr "MIT lisansı" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11744,7 +11759,8 @@ msgctxt "shared" msgid "Join the community" msgstr "Topluluğa katıl" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12729,8 +12745,1338 @@ msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "N0zoM1z0 · 13 Ekim 2026 · İnceleme taslağı" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Yazan N0zoM1z0 · 13 Ekim 2026" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "İçeriğe geç" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "" +"{ph0}Ana Sayfa{ph1} / {ph2}Blog{ph3} / Modeller ve tersine mühendislik" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "Deneme" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Tersine mühendisliğin daha büyük bir modele ihtiyacı var mı?" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Yazan N0zoM1z0 · 13 Ekim 2026" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "Önemli noktalar" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}Model yeteneği, bir yapay zeka aracısının hangi hipotezleri " +"önerebileceğini etkiler.{ph1} Daha sonraki oturumlarda, daha önceki " +"çalışmaların çözümsüz bıraktığı sorunlar çözüldü." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}Tanıdık bir derleyici, özel bir modelin öğrenilmesi için somut modeller" +" sunar.{ph1} Bazı tam makine kodu eşleşmeleri, belirli, yeniden " +"kullanılabilir kaynak kodu modellerine bağlıdır." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}Adil bir karşılaştırmaya veri deposu bilgisini dahil edin.{ph1} İyi " +"bilgi erişimine sahip daha küçük bir model, ilerleme kaydetmek için zaten " +"yeterli bilgiye sahip olabilir." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}Modeli, bir sonraki faydalı deneyi seçecek şekilde eğitin ve doğrulama " +"denetleyicisini test edin.{ph1} Geri bildirim, yalnızca denetleyicinin doğru" +" şeyleri doğruladığı ölçüde faydalıdır." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}Doğrulanmış yeniden yapılandırma sonuçlarını zaman ve maliyetle " +"birlikte değerlendirin.{ph1} Yapay zeka aracısı iş akışının tamamı için " +"ölçmek istediğim şey bu." + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "Bu makalede" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}Daha güçlü modeller neyi değiştirdi?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}Neden uzmanlaşalım?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}Bu deneyim nerede saklanmalı?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}Bir sonraki faydalı deneyi öğrenmek{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}Tersine mühendislik ve takviyeli öğrenme{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}Doğrulama denetleyicisi yanlış olduğunda{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}Çalıştırılmaya değer bir karşılaştırma{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}Bu ne tür bir yapay zeka ajanını akla getiriyor?{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "Daha güçlü modeller neyi değiştirdi?" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"Ağustos 2026'dan bu yana, bir aracının bir programa erişim ve onun " +"çalışmasını kontrol etmenin bir yolu ile ne kadar tersine mühendislik " +"gerçekleştirebileceğini araştırıyorum. Deneyimlerimin çoğu, özellikle Touhou" +" olmak üzere oyunların yeniden yapılandırılmasından geliyor. Modelleri " +"değiştirdiğimde, depolar her oturumda kurtarılanların kaydını tuttu." + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"Bu çalışmanın büyük bir kısmı tam olarak yeniden yapılanmayı amaçlamaktadır:" +" Bakımı yapılan kaynağı, sabit versiyonlara sahip tarihsel bir araç zinciri " +"altında derliyoruz ve hedef işlevi yeniden üretip üretmediğini kontrol " +"ediyoruz. Önceki çalışmaların durduğu yerde daha sonraki bir model başarılı " +"olduğunda, kaynak değişiklikleri bu ilerlemeyi neyin mümkün kıldığını " +"gösterir." + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "Geçici değişkenler örtüşen yaşam sürelerine ihtiyaç duyuyordu" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"TH09 saldırı fonksiyonunun mantığının büyük bir kısmı zaten yeniden " +"oluşturulmuştu, ancak derleyici hala geçici sonuçları orijinalinden farklı " +"şekilde saklıyordu. Geriye kalan sorun, bu sonuçların ne kadar süreyle canlı" +" kalması gerektiği ve depolarının ne zaman yeniden kullanılabileceğiydi." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"Bir {ph0}GPT-6-Sol oturum kaydı{ph1}, birkaç sıradan kaynak kodu " +"düzenlemesini test ettikten sonra bu uyumsuzluğu bildirdi. {ph2}GPT-6.1-Sol " +"sorunu tekrar incelediğinde{ph3}, bu geçici değişkenlerin yaşam süreleri " +"hakkında yararlı bir hipotez öne sürdü." + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"İşlev geçici sonuçlar üretti ve bunları animasyon konumlarına kopyaladı. " +"C++'da geçici bir sonuç, onu kullanan ifade bitene kadar canlı kalır. Gerçek" +" işlemleri tek bir ifadede gruplandırmak bu sonuçları bir arada canlı tuttu," +" böylece derleyici artık aynı depolamayı hepsi için yeniden kullanamıyordu." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"Sonuçlardan biri, daha sonra çarpışma kontrolleri için kullanılan bir " +"değerle depolamayı hâlâ paylaşıyordu. Bu değere kendi ömrünü vermek, kalan " +"örtüşmeyi çözdü. Birkaç hedefli denemeden sonra, {ph0}kaynaktan yapılan " +"temiz bir yeniden derleme, tam işlevin eşleştiğini doğruladı{ph1}." + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"Keşif ve testlerin birbirini nasıl desteklediğiyle ilgileniyorum. Geçici " +"nesne yaşam süreleri hakkındaki hipotez, yapay zeka ajanına yeni bir yön " +"verdi. Derleyici geri bildirimi, bu hipotezi eşleşen bir kaynak kodu " +"düzenlemesine dönüştürmeye yardımcı oldu." + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "Farklı bir kontrol akışı grafiği" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"İkinci bir TH09 işlevi, bu yeni yönün neden önemli olabileceğini gösterir. " +"Buradaki engel, birden fazla şubenin nihai çalışmalarını paylaşmasıydı. " +"{ph0}GPT-6.1-Sol'nin önceki çalışması{ph1} işlevin bazı kısımlarını " +"eşleştirdi ancak dallar yine de orijinalinden farklı bir şekilde birleşti." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra çevredeki kontrol akışı grafiğini değiştirdi{ph1}. Ana " +"yollar artık paylaşılan bir son güncellemeye ve geri dönüşe yol açıyordu. Bu" +" kaynak yapısında derleyici, dalları orijinaldekiyle aynı şekilde " +"birleştirdi. İşlevin tamamı eşleşti ve {ph2}ayrı bir yeniden oluşturma " +"kontrolü bunu doğruladı{ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}Temsili değiştirmek daha verimli bir arama alanı açtı.{ph1} Her iki " +"durum da bana genel model yeteneğini ciddiye almam için bir neden veriyor: " +"Bir temsilci, çevredeki işi ifade etmenin daha iyi bir yolunu bulmadan önce " +"yerel varyasyonları uzun süre denemeye devam edebilir." + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"Sonraki oturumlar önceki kaynak kodu ve kaydedilmiş başarısız denemeleri de " +"devraldı. Bu birikim avantajlarının bir parçasıydı. Geçmiş, keşifleri ve " +"doğrulamalarını incelememizi sağlar; kontrollü bir karşılaştırma ise modelin" +" etkisini devraldığı deneyimin etkisinden ayırmaya yardımcı olur." + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"Soruyu benim için ilginç kılan da bu. Yeni bir temsil bulma konusunda bir " +"miktar ilerleme sağlandı. Geriye kalan çalışmaların çoğu, farklı kaynak kodu" +" düzenlemelerinin tanıdık bir derleyiciye karşı test edilmesini içeriyordu. " +"Bunun ne kadarı geniş muhakeme yeteneği gerektiriyor? Bir model, o derleyici" +" ve kodla tekrar tekrar çalışarak ne kadar öğrenebilir?" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "Özel bir yapay zeka modeli neden başarılı olabilir?" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou bize bu deneyimin yardımcı olmasını beklememiz için somut bir neden " +"veriyor. Oyun kodunun büyük bir kısmı aynı yazardan (ZUN) geliyor, " +"dolayısıyla yinelenen alışkanlıklar makul bir başlangıç ​​varsayımıdır. " +"Depolar bunu yararlı örneklerle desteklemektedir: İlgili döngüler ve " +"yardımcı düzenlemeler, oyunlar arasında hipotezler sağlamıştır." + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"Derleyiciyi dahil ettiğimizde bu kaynak kalıpları daha spesifik hale gelir. " +"Belirli bir sürüm ve seçenekler kümesi altında, kaynaktaki küçük " +"değişiklikler makine kodunda tekrarlanabilir farklılıklara neden olabilir. " +"Bu farklılıkların bazıları şaşırtıcı derecede belirsizdir." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"Örneğin TH095'de, {ph0}yerel bir değişkenin adı bile VC7.1'nin onu nerede " +"sakladığını etkileyebilir{ph1}. Bildiriminin taşınması depolama düzenini " +"çözmedi. Ancak daha önceki bir arka plan işlevi, derleyicinin seçimini " +"etkileyen bir adlandırma modelini ortaya çıkarmıştı. Bu, {ph2}yeniden inşa " +"etmek{ph3}{ph4} için daha umut verici bir hipotez önerdi." + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"Dosya boyutu değişkeni için bu kalıbın yeniden kullanılması, depolama alanı " +"eklemeden gerekli düzeni oluşturdu. Daha sonra tam işlev eşleşti." + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"Bu, genel amaçlı bir yapay zeka modelinin sorunla ilk karşılaştığında " +"bilemeyeceği çok özel bir ayrıntıdır. Bu derleyici için uzmanlaşmış bir " +"model, bu tür birçok durumu görebilir ve hangi kaynak kodu varyasyonlarının " +"ilk önce test edilmeye değer olduğunu öğrenebilir." + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"Davranışı zaten anladığımız son eşleştirme problemlerinde bu fırsat " +"özellikle açıktır. Daha sonra bu derleyicinin hangi sıradan kaynak kodu " +"düzenlemesini gözlemlenen talimatlara dönüştürdüğünü soruyoruz. Uzmanlaşma, " +"modelin bu formları daha verimli bir şekilde önermesine yardımcı olurken, " +"hedef hâlâ hangi hipotezlerin hayatta kalacağını belirliyor." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"Yararlı kalıplar bağlamlarını korumalıdır. TH08, TH095 ve TH09’dan daha eski" +" bir Visual C++ sürümü kullanır. TH04 ise DOS dönemine ait başka bir " +"derleyici kullanır. {ph0}Oyuncu kodunun yeniden yapılandırılması{ph1}, " +"kaynak kalıpları ve kodun çalıştırılabilir dosyaya nasıl bağlandığı hakkında" +" farklı hipotezler gerektirdi. Aynı yazar bize bir başlangıç varsayımı " +"verir; bunun geçerli olup olmadığını her hedefte kontrol ederiz. Uzman model" +" bu bağlamları ayırt etmeyi öğrenmelidir." + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "Bu deneyim nerede saklanmalı?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"Bu kalıpları model eğitimi yoluyla öğretmeye karar vermeden önce, " +"halihazırda depoda tuttuğumuz bilgilere bakardım. {ph0}Önceki makalemde{ph1}" +" bunun projenin çalışma belleği haline geldiğini yazmıştım. Bu hafıza, " +"derleyici derslerini, kaynağı ve onları faydalı kılan başarısız deneyleri " +"içerir." + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"Bir aracı, bu kaydı alarak bilinen bir sonuçla başlayabilir ve bu sonuca " +"nasıl ulaştığımızı inceleyebilir. Aynı bağlamda halihazırda başarısız olan " +"alternatiflerden de kaçınılabilir. Başlangıç ​​noktası, modelin eğitilmiş " +"parametrelerini değiştirmeden iyileşir." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"TH09'nin düşman yaratma işlevi yeniden yapılandırılırken, bir " +"{ph0}GPT-6-Luna-max oturumu{ph1} ilgili TH08 kaynak kodundan bir hipotez " +"aldı. Önceki kod, döngünün nasıl organize edileceğini ve içindeki arızaların" +" nasıl ele alınacağını öneriyordu. TH09'in kendi kanıtları oyuna özgü " +"ayrıntıları sağladı. Kaynaktan yapılan yeniden derlemeler, işlevin tamamının" +" eşleştiğini doğruladı." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}\"Model, derleyici tuhaflığını bilmiyordu\" ve \"model bunu " +"kullanamadı\" farklı sorunlardır.{ph1} Geri getirme ilk sorunu çözebilir, " +"ancak bu gerçeği yararlı bir adaya dönüştürmek yine de çalışma gerektirir. " +"Aday kaynak kodu başarısız olursa modelin bir sonraki denemesinde sonucun ne" +" anlama geldiğine karar vermesi gerekir. İlgili modeli bilmek bu sürecin " +"yalnızca başlangıcıdır." + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"Bu nedenle, herhangi bir karşılaştırmaya, iyi geri çağırma özelliğine sahip," +" daha küçük, genel amaçlı bir yapay zeka modelini dahil ederim. Aynı deneyim" +" kitaplığını ona vermek, eğitimin ne gibi katkılar sağladığını belirlememize" +" yardımcı olacaktır: daha iyi aday oluşturma, geri bildirim sonrasında daha " +"iyi kararlar veya her ikisi." + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "Bir sonraki faydalı deneyi öğrenmek" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"Geri bildirim sonrasında verilen kararlar, üzerinde çalışmayı en çok " +"istediğim şeylerdir. Failsel yeniden yapılanmada model bir açıklama önerir " +"ve bunu programa karşı test eder. Aday kaynak kodu başarısız olsa bile " +"sonuç, kalan olasılıkları daraltabilir ve bir sonraki deneyin daha yararlı " +"olmasını önerebilir." + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"Geçici sonuçlar örneğini tekrar düşünün. Son kaynağı işe yarayan düzenlemeyi" +" gösteriyor ancak bu kaynağa giden yol daha fazla bilgi içeriyor. Aşağıdaki " +"deneylerin her biri, hala neyin değişmesi gerektiğinin belirlenmesine " +"yardımcı oldu." + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "Deney" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "Sonucunun bize öğrettikleri" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "Sonuçları kopyalamanın farklı bir yolunu deneyin" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "Kopya kodu değişir ancak işlev hâlâ farklıdır." + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "Döndürülen değerleri doğrudan kullanın" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "" +"Kopyalama orijinal düzeni takip eder ancak geçici depolama yine de " +"farklıdır." + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "Tüm geçici değişkenleri aynı anda geçerli tutun" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "" +"Ayrı bir depolamaya sahipler, ancak bir tanesi daha sonra çarpışma " +"kontrolleri için kullanılan bir değerle örtüşüyor." + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "Çarpışma kontrolü değerini ayrı tutun" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "" +"Kaynaktan alınan temiz bir yeniden derleme, eşleşmenin tamamını doğrular." + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "" +"Yukarıdaki saldırı fonksiyonunun yeniden yapılandırılmasından bize " +"öğrettiklerine göre gruplandırılmış seçilmiş deneyler." + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"Bu sırayı tutan bir eğitim kaydı, tek başına nihai kaynaktan daha fazlasını " +"öğretebilir. Önerilen her değişikliği gözlemlenen bir sonuca ve bir sonraki " +"karara bağlar. Etkisiz varyasyonları korumak da önemlidir: Daha sonraki bir " +"temsilciye, o bağlamda hangi hipotezlerin zaten test edildiğini söylerler." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}Modelin geri bildirimi nasıl kullanacağını öğrenmesini isterim.{ph1} " +"Nötr bir sonuç, varyasyonun yeniden değerlendirilmesi için bir nedendir; " +"değişen bir uyumsuzluk başka bir nedene işaret edebilir. Yerel düzenlemeler " +"bilgilendirici olmayı bıraktığında, aracının kaynak temsilini yeniden gözden" +" geçirebilmesi veya kanıtları yeniden inceleyebilmesi gerekir." + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"Bu seçimleri yapmayı öğrenmek özerklik gerektirir. Bazen bir sonraki yararlı" +" adım, başka bir düzenleme derlemek yerine başka bir arayanı incelemektir. " +"Gerekli yeniden yazma işlemlerinin katı bir dizisi, modelin öğrenmesini " +"istediğimiz davranışın bir kısmıyla birlikte bu seçimi de ortadan " +"kaldıracaktır." + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "Takviyeli öğrenme için tersine mühendislik neden ilgi çekicidir?" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"Sınırlı yeniden yapılandırma bize bu seçimleri incelemek için pratik bir " +"ortam sağlar. Hedef ve derleyici sürümleri sabitlendikten sonra birçok " +"kaynak denemesi hızlı, tekrarlanabilir geri bildirim üretir. Temsilci bir " +"adayı derleyebilir, uyumsuzluğu inceleyebilir ve bu gözlemi bir sonraki " +"adımını seçmek için kullanabilir." + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"Takviyeli öğrenme, aldığı geri bildirim yoluyla karar verme sürecini " +"eğitebilir. Belirli bir kontrolle kabul edilen bir aday bize somut bir sonuç" +" verirken, denemelerin sırası bu sonuca ulaşmak için ne kadar çaba " +"harcandığını gözlemlememize olanak tanır." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"Bu yönü destekleyen araştırmalar zaten var. {ph0}LLM4Decompile{ph1} kaynak " +"koda dönüştürme için uzmanlaşmış modelleri inceliyor. " +"{ph2}AutoDecompiler{ph3}, derleyici ve yürütme geri bildirimini kullanarak " +"çok turlu iyileştirmeyi eğitir. Çalışmaları fonksiyonel iyileşmeye " +"odaklanıyor. Bu çalışma için bunları, tarihsel bir derleyicinin çıktısıyla " +"eşleşen kaynağı bulma görevi için bir başlangıç ​​noktası olarak " +"kullanacağım." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"Böylesine iyi tanımlanmış bir göreve ulaşmak için öncelikle bir fonksiyonun " +"hangi kaynaklara sahip olduğunu belirlememiz veya yeni bir çalışma zamanı " +"ölçümü tasarlamamız gerekebilir. Örneğin TH08'de, bir {ph0}çarpışma farkı, " +"orijinal programın kayan nokta hesaplamalarını{ph1} nasıl işlediğini ortaya " +"çıkardı. Aracının yararlı bir düzeltmeyi test edebilmesi için önce neyi " +"ölçeceğini bulması gerekiyordu." + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"Bu, geniş kapsamlı bir model için başka bir rol öneriyor: ilk etapta sorunun" +" kontrol edilebilir olmasına yardımcı olmak. Bu kontrol bir kez mevcut " +"olduğunda, öğrenilmiş bir arama politikasının bununla ne kadar ileri " +"gidebileceğini araştırabiliriz." + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "Doğrulama denetleyicisi yanlış olduğunda" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"Bir kontrolü eğitimde kullandığımızda, kapsamı da öğrenme ortamının bir " +"parçası olur. Böyle bir referansa veya doğrulayıcıya oracle diyorum. " +"Tekrarlanabilir bir yanıt bile gerçekte neyi doğruladığına göre " +"anlaşılmalıdır." + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 bu kapsamın dışında ne kadar şeyin saklanabileceğini gösterdi. Yeniden " +"oluşturulan bazı işlevler, tam karşılaştırmalarını yanlış kayan sabitlerle " +"geçti. Karşılaştırıcı, talimatları kontrol etmeden önce yeniden " +"konumlandırılan adresleri ayarladı, ancak bu adreslerin arkasındaki " +"değerleri yalnızca doğrulama için açıkça kaydedildiklerinde kontrol etti. " +"Diğer gerçek referanslar, değerleri incelenmeden geçebilir." + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"Bu boşluğu gidermek için, {ph0}daha geniş bir denetim{ph1}, doğrulamayı " +"zaten geçmiş işlevlerde hatalı sabitler buldu. Bir hesaplamada, yeniden " +"oluşturulan kaynak kodu {ph2}'yi eklerken, orijinali {ph3}'ü ekledi." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}Eski doğrulama denetimiyle ödüllendirilen bir model, hatalı bir yeniden" +" yapılandırma üretmeyi öğrenebilir.{ph1} Bu denetleyiciyi kullanarak bir RL " +"modelini eğitmedik. Gözlemlediğimiz başarısızlık kontrolün kendisindeydi. " +"Ancak bunu bir ödül olarak kullanırsak aynı kör nokta öğrenme ortamının bir " +"parçası haline gelecektir." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"Fonksiyonel bir testin de benzer bir kör noktası olabilir. {ph0}LLM kod " +"çözücüler üzerinde diferansiyel bulanıklaştırmayı kullanan araştırma{ph1}, " +"sağlanan testlerin dışında davranışsal farklılıklar buldu. Bu nedenle geçme " +"testini, uyguladığı durumlar açısından anlamamız gerekir. Benzer şekilde, " +"bir talimat eşleşmesinin, doğruladığı baytlar ve veriler açısından " +"anlaşılması gerekir." + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"Her iki doğrulama kontrolünü de ödül olarak kullanmadan önce, bunu kasıtlı " +"olarak yanlış sonuçlarla test ederim. Ayrıca nihai kabulü, yapay zeka " +"aracısının düzenleyebileceği kaynak kodundan ayrı tutacağım. Model, programı" +" doğru bir şekilde yeniden oluşturarak doğrulamayı geçmelidir. Aksi " +"takdirde, hedeften bir bayt dizisinin basitçe kopyalanması, yararlı kaynak " +"kodunu kurtarmadan saf bir puanlama sistemini kandırabilir." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}Öğreneni geliştirmek ve doğrulama denetleyicisini geliştirmek aynı " +"araştırma probleminin bir parçasıdır.{ph1} Öğrenci, ona verdiğimiz geri " +"bildirimi optimize etme konusunda daha iyi hale geldikçe, geri bildirimin " +"anlaşılması da daha önemli hale gelir." + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "Çalıştırılmaya değer bir karşılaştırma" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"Bu kontroller uygulandığında, karşılaştırmaya sabit bir VC7.1 görev ailesi " +"üzerinde başlayacağım. İlk olarak, güçlü bir genel amaçlı yapay zeka modeli " +"ve daha küçük bir model için temeller oluşturacaktım. Daha küçük modele depo" +" bilgisinin eklenmesi, geri almanın ne gibi değişikliklere yol açtığını " +"gösterecektir. Buradan itibaren, denetimli uzmanlık ve RL ile eğitilmiş bir " +"sürüm, eğitimin neleri eklediğini incelememize olanak tanır." + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"Bu karşılaştırmaları bilgilendirici kılmak için her model aynı hedef " +"kanıtları ve araçları alacaktır. Eğitime özel karşılaştırmalar için temel " +"modeli ve mevcut bilgiyi de sabit tutardım. Tekrarlanan denemeler, güvenilir" +" bir arama stratejisini şans eseri başarılı olan aday bir çözümden ayırmaya " +"yardımcı olacaktır." + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"Daha sonra iki başlangıç koşulunu ayıracağım: bir fonksiyonu talimatlarından" +" yeniden oluşturmak ve korunan kaynaktaki son birkaç farklılığı kapatmak. " +"Özel bir yapay zeka modeli ikinci görevde mükemmel olabilirken, genel amaçlı" +" bir yapay zeka modeli, ona ulaşmak için gereken işin çoğunu yapar. Bunları " +"ayrı ayrı ölçmek o iş bölümünü ortaya çıkaracaktır." + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"Her iki görevin de eğitim ve erişim dışında bırakılan test örneklerine " +"ihtiyacı vardır. İlgili Touhou oyunları motor kodunu paylaşır. Bir oyunu " +"eğitimden hariç tutmak, eğitim setinde neredeyse aynı işlevlerin kalmasına " +"neden olabilir. Bunun yerine, ilgili kaynak kodu gruplarının tamamını hariç " +"tutar ve doğrulanmış yanıtlarını hem eğitim verilerinin hem de erişim " +"kütüphanesinin dışında tutardım." + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"Bu örnekler için sonuç, eksiksiz, bağımsız olarak doğrulanmış eşleşmeler " +"olacaktır. Aday teklifleri veya derleme girişimleri için eşit bütçeler, " +"arama kalitesinin karşılaştırılmasına yardımcı olacaktır. Derleme de dahil " +"olmak üzere toplam süre ve maliyetin ölçülmesi, bu kalitenin verimli bir iş " +"akışı üretip üretmediğini gösterecektir. Eğitim maliyetleri ayrı olarak " +"ölçülmelidir." + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"Her iki ölçüm de önemlidir. Ucuz bir teklif, hiçbir yararlı bilgi sağlamayan" +" daha fazla derlemeye neden olursa pahalı bir iş akışına yol açabilir. " +"Tersine, daha yavaş bir model, yararlı bir deneyi daha erken bulursa daha " +"ekonomik olabilir. Karşılaştırma bu maliyetleri içermelidir." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"Kabul edilen sonuçların sayılması aynı zamanda ara ilerlemeyi de perspektif " +"içinde tutar. Bir TH09 {ph0}tekrar oynatma-kaydetme adayı{ph1}, derlenmiş " +"işlevin beklenen boyuta sahip olması nedeniyle umut verici görünüyordu, " +"ancak tam bir karşılaştırma yine de önemli farklılıklar buldu. Derleyicinin " +"talimatları düzenleme biçimindeki değişiklikler, altta yatan sorunu çözmeden" +" adayın daha yakından bakmasını da sağlayabilir. Bu tür teşhisler bir " +"sonraki deneye rehberlik edebilir, ancak \"yaklaşmanın\" herhangi bir " +"ödülünü tam kabul kontrolüne göre doğrularım." + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"Bu sonuçlardaki model ilgimi çeker. Özel bir yapay zeka modeli tanıdık bir " +"derleyici ailesinde başarılı olursa, bu avantaj bir takım zinciri " +"değişikliğinden sonra da ayakta kalabilir mi? Her iki yapay zeka modeli de " +"iyi bir erişime sahip olduğunda, eğitimin hâlâ ne kadar katkısı olur? " +"Cevaplar, her model türünün en yararlı olduğu yerin neresi olduğuna karar " +"vermemize yardımcı olacaktır." + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "Bu ne tür bir yapay zeka ajanını akla getiriyor?" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"Modellerin farklı aşamalarda yardımcı olduğu ortaya çıkarsa, özel bir yapay " +"zeka modeline erişimi olan geniş bir temsilci yararlı bir düzenleme " +"olabilir. Bir görev tanıdık bir derleyici araması haline geldiğinde, adaylar" +" için özel yapay zeka modelini isteyebilir. Sonuçların artık bir anlamı " +"kalmazsa, daha geniş kapsamlı temsilci hedef kanıtlara geri dönebilir veya " +"ölçümün kendisini araştırabilir." + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"Bu düzenlemede uzmanlaşma, soruşturma geliştikçe temsilcinin seçebileceği " +"bir araç haline gelir. Temsilci, daha önceki atılımları mümkün kılan yön " +"değiştirme özgürlüğünü elinde tutar." + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"Bu, doğal olarak önceki makalede anlattığım endüstriyel değişimin sonucudur." +" Daha fazla deney yürütebilmek, işi organize etme şeklimizi değiştirir. Aynı" +" zamanda bize, hangi kararların geniş muhakeme gerektirdiğini ve " +"hangilerinin belirli bir zanaatta tekrarlanan deneyimlerle geliştirildiğini " +"sormamız için bir neden de verir." + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"Şimdilik en güçlü genel model, daha önce çözemediğim sorunları çözerek iş " +"akışımda yerini kazandı. Bu başarılar süreci daha yakından incelemek " +"istememe yol açıyor: Ne kadarı modelin geniş yeteneklerine bağlıydı? Ne " +"kadarı bu yeniden yapılandırma türüne özel olarak eğitilmiş bir model " +"tarafından yapılabilirdi?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}Depolar bize soruyu test etmemiz için bir yol sağlıyor.{ph1} Bu kaydı " +"karşılaştırmayı oluşturmak ve ne öğrendiğimizi görmek için kullanmak " +"istiyorum." + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "Kanıt ve model ilişkilendirme" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"Örnekler, bağlantılı taahhütlerde kaydedilen geçmiş işlev düzeyi " +"karşılaştırmalarını açıklamaktadır. Tüm oyun çalışma zamanı kontrolleri ayrı" +" kilometre taşlarıdır. Kaynak ve eşleşen kayıtlar 13 Ekim 2026'da incelendi." + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"TH09 etiketi {ph0}, GPT-6-Astra anlamına gelir. {ph1} ile ilişkilendirilen " +"model zaman içinde değiştiği için etiket tek başına belirli bir çalıştırmada" +" kullanılan modeli tanımlayamaz. Belirtilen diğer taahhütlerdeki açık model " +"adları, kaydedilen atıfları sağlar. Bu geçmişler, her çalıştırma için " +"tutarlı maliyet veya çaba ölçümlerini içermez." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 geçici nesne yaşam süreleri:{ph1} {ph2}önceki bağlam " +"incelemesi{ph3} ve {ph4}kaynaktan temiz bir yeniden derlemeyle doğrulanan " +"tam yeniden yapılandırma{ph5}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}TH09 Front geri arama:{ph1} {ph2}eşleşmeyen tanılama bölgeleri{ph3}, " +"{ph4}Astra'nin yeni kontrol akışı grafiği{ph5} ve onun {ph6}kabul edilen " +"yeniden derleme kontrolü{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}Derleyici bilgisi:{ph1} {ph2}TH095 tanımlayıcı grubu davranışının " +"yeniden kullanımı{ph3}, {ph4}TH04 oyuncu kaynak kodu ve bağlayıcı " +"sırası{ph5} ve {ph6}Luna'nin TH09'nin düşman oluşturma işlevini tamamen " +"yeniden yapılandırması{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}Kontroller ve teşhisler:{ph1} {ph2}TH08 kayan nokta sabiti " +"denetimi{ph3}, {ph4}TH08 sayısal inceleme{ph5} ve {ph6}TH09 yeniden " +"oynatmayı kaydeden yeniden yapılandırma{ph7}." + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"Özel modellerin ve RL eğitiminin önerilen karşılaştırmaları gelecekteki " +"deneylerdir. Proje geçmişleri onları motive ediyor. Bu makale için hiçbir " +"eğitimli özel model değerlendirilmemiştir." + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "Tersine mühendisliğin daha büyük bir modele ihtiyacı var mı? | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"Touhou yeniden yapılandırma bana genel modeller, uzman modeller ve derleyici" +" geri bildirimlerinden daha iyi hipotezler önermeyi öğrenme hakkında neler " +"öğretti?" + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "Bu sayfada" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "Modeller ve tersine mühendislik makalesi" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "Seçilmiş yeniden yapılanma deneyleri" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "Başa dön" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "N0zoM1z0 · 13 Ekim 2026 · İnceleme taslağı" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/uk/messages.po b/website/locales/uk/messages.po index 34131bdfa..642dc5f5c 100644 --- a/website/locales/uk/messages.po +++ b/website/locales/uk/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Думки про реверс-інжиніринг." #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Реконструкція гри{ph0}Вигадування" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "Промислова революція в області реверс-інжинірингу" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,20 @@ msgstr "" "Чому мене навчив посібник з оновлення та реконструкції TH08, TH095 та TH04 " "про агентів, перевірку та знання, які використовуються в різних проектах." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Прочитайте статтю" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Автор: N0zoM1z0 · 10 жовтня 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -100,6 +101,8 @@ msgstr "" " і агентами штучного інтелекту." #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -136,6 +139,7 @@ msgstr "" "допомогою вашого програміста." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -156,6 +160,7 @@ msgid "REA home" msgstr "Головна REA" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -176,6 +181,7 @@ msgid "Main navigation" msgstr "Основна навігація" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -196,6 +202,7 @@ msgid "REA on GitHub" msgstr "REA на GitHub" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -216,6 +223,7 @@ msgid "Join REA on Discord" msgstr "Приєднатися до REA в Discord" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -235,7 +243,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Стежити за REA в X" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Нагору" @@ -11684,6 +11692,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "Однакова відстань: 256 ÷ 16 = 16 кутових одиниць = 22,5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -11704,6 +11713,7 @@ msgid "Showcases" msgstr "Приклади" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -11724,6 +11734,7 @@ msgid "Blog" msgstr "Блог" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -11744,6 +11755,7 @@ msgid "Get started" msgstr "Почати" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -11763,7 +11775,8 @@ msgctxt "shared" msgid "Guides" msgstr "Посібники" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11783,7 +11796,8 @@ msgctxt "shared" msgid "Source" msgstr "Вихідний код" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11803,7 +11817,8 @@ msgctxt "shared" msgid "MIT license" msgstr "Ліцензія MIT" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11823,7 +11838,8 @@ msgctxt "shared" msgid "Join the community" msgstr "Приєднатися до спільноти" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12814,8 +12830,1334 @@ msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "Автор: N0zoM1z0 · 13 жовтня 2026 р. · Чернетка для обговорення" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Автор N0zoM1z0 · 13 жовтня 2026 р" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "Перейти до змісту" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "" +"{ph0}Домашня сторінка{ph1} / {ph2}Блог{ph3} / Моделі та зворотне " +"проектування" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "Есе" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Чи потрібна для зворотного проектування більша модель?" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Автор N0zoM1z0 · 13 жовтня 2026 р" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "Головне" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}Можливості моделі впливають на те, які гіпотези може запропонувати " +"агент штучного інтелекту.{ph1} Пізніші сесії вирішували проблеми, які " +"попередня робота залишила невирішеними." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}Знайомий компілятор пропонує конкретні шаблони для вивчення " +"спеціалізованої моделі.{ph1} Деякі точні збіги машинного коду залежать від " +"конкретних шаблонів вихідного коду, які можна багаторазово використовувати." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}Включіть знання репозиторію в справедливе порівняння.{ph1} Менша модель" +" із хорошим пошуком знань може вже мати достатньо інформації для прогресу." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}Навчіть модель вибирати наступний корисний експеримент і протестуйте її" +" засіб перевірки.{ph1} Зворотній зв’язок корисний лише в тій мірі, в якій " +"засіб перевірки перевіряє правильні речі." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}Оцінка перевірених результатів реконструкції разом із часом і " +"вартістю.{ph1} Це те, що я хочу виміряти для повного робочого процесу AI-" +"agent." + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "У цій статті" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}Що змінили сильніші моделі{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}Навіщо спеціалізуватися?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}Де зберігати цей досвід?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}Вивчення наступного корисного експерименту{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}Зворотне проектування та навчання з посиленням{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}Коли засіб перевірки помиляється{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}Варте порівняння{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}Що це за агент ШІ?{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "Що змінили сильніші моделі" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"З серпня 2026 року я досліджував, який обсяг зворотного проектування може " +"виконати агент, маючи доступ до програми та спосіб перевірки її роботи. " +"Більшість мого досвіду отримано від реконструкції ігор, особливо Touhou. " +"Оскільки я змінював моделі, репозиторії зберігали записи про те, що вдалося " +"відновити за кожну сесію." + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"Значна частина цієї роботи спрямована на точну реконструкцію: ми компілюємо " +"підтримуваний джерело в ланцюжку історичних інструментів із фіксованими " +"версіями та перевіряємо, чи він відтворює цільову функцію. Коли пізніша " +"модель досягає успіху там, де попередня робота зупинилася, зміни джерела " +"показують, що зробило цей прогрес можливим." + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "Тимчасові змінні потребували часу існування, що збігався" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"Значна частина логіки функції атаки TH09 вже була реконструйована, але " +"компілятор усе ще зберігав тимчасові результати, відмінні від оригіналу. " +"Проблема, що залишилася, полягала в тому, як довго ці результати залишалися " +"живими та коли їхнє сховище можна було використовувати повторно." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"{ph0}GPT-6-Sol запис сеансу{ph1} повідомив про цю невідповідність після " +"тестування кількох звичайних механізмів вихідного коду. Коли " +"{ph2}GPT-6.1-Sol знову дослідив проблему{ph3}, він запропонував корисну " +"гіпотезу щодо тривалості життя цих тимчасових змінних." + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"Функція створила тимчасові результати та скопіювала їх у позиції анімації. У" +" C++ тимчасовий результат залишається активним до завершення виразу, який " +"його використовує. Групування реальних операцій в один вираз зберігало ці " +"результати разом, тому компілятор більше не міг повторно використовувати те " +"саме сховище для всіх них." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"Один результат все ще ділився пам’яттю зі значенням, яке використовується " +"пізніше для перевірок на конфлікти. Надання цьому значенню власного часу " +"життя вирішило решту накладання. Після кількох цілеспрямованих експериментів" +" {ph0}чиста повторна компіляція з джерела підтвердила, що повна функція " +"відповідає{ph1}." + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"Мені цікаво, як відкриття та випробування підтримували одне одного. Гіпотеза" +" про тривалість життя тимчасових об’єктів дала агенту ШІ новий напрямок. " +"Відгуки компілятора допомогли уточнити цю гіпотезу у вихідному коді, який " +"відповідав." + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "Інший графік потоку керування" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"Друга функція TH09 показує, чому цей новий напрямок може мати значення. Тут " +"перешкодою було те, що кілька філій поділилися своєю остаточною роботою. " +"Попередня робота {ph0}GPT-6.1-Sol{ph1} мала відповідні частини функції, але " +"гілки все одно з’єднувалися інакше, ніж у оригіналі." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra змінив навколишній графік потоку керування{ph1}. Основні " +"шляхи тепер вели до спільного остаточного оновлення та повернення. У цій " +"вихідній структурі компілятор об’єднав гілки так само, як і в оригіналі. " +"Повна функція відповідала, і {ph2}окрема перевірка відновлення підтвердила " +"це{ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}Зміна подання відкрила більш продуктивний простір для пошуку.{ph1} " +"Обидва випадки дають мені підстави серйозно поставитися до загальної " +"можливості моделі: агент може довго пробувати місцеві варіації, перш ніж " +"знайти кращий спосіб вираження навколишньої роботи." + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"У той же час пізніші сеанси починалися з попередніх джерел і записаних " +"збоїв. Ця накопичена робота була частиною їхньої переваги. Історія дозволяє " +"нам ознайомитися з відкриттями та їх перевірками; контрольоване порівняння " +"допомогло б відокремити ефект моделі від досвіду, який вона успадкувала." + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"Ось що робить питання цікавим для мене. Певний прогрес став завдяки пошуку " +"нового представництва. Значна частина роботи, що залишилася, включала " +"тестування різних механізмів вихідного коду на знайомому компіляторі. " +"Наскільки для цього потрібна широка здатність міркувати? Скільки може " +"навчитися модель, постійно працюючи з цим компілятором і кодом?" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "Чому спеціалізована модель ШІ може працювати добре?" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou дає нам конкретну причину очікувати, що цей досвід допоможе. Значна " +"частина ігрового коду походить від того самого автора, ZUN, тому повторювані" +" звички є розумним вихідним припущенням. Репозиторії підтверджують це " +"корисними прикладами: пов’язані цикли та допоміжні механізми надають " +"гіпотези для різних ігор." + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"Ці вихідні шаблони стають більш конкретними, коли ми включаємо компілятор. " +"За певної версії та набору параметрів невеликі зміни у джерелі можуть " +"призвести до повторюваних відмінностей у машинному коді. Деякі з цих " +"відмінностей напрочуд незрозумілі." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"У TH095, наприклад, {ph0}навіть ім’я локальної змінної могло вплинути на те," +" де VC7.1 її зберігати{ph1}. Переміщення його оголошення не вирішило макета " +"сховища. Однак більш рання фонова функція виявила шаблон іменування, який " +"вплинув на вибір компілятора. Це запропонувало більш перспективну гіпотезу " +"для {ph2}реконструкції {ph3}{ph4}." + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"Повторне використання цього шаблону для змінної розміру файлу створило " +"необхідний макет без додавання пам’яті. Тоді повна функція збігалася." + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"Це дуже конкретна деталь, яку модель ШІ загального призначення може не " +"знати, коли вперше стикається з проблемою. Спеціалізована модель для цього " +"компілятора може побачити багато таких випадків і дізнатися, які варіації " +"вихідного коду варто перевірити в першу чергу." + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"Можливість особливо чітка в остаточних задачах зіставлення, де ми вже " +"розуміємо поведінку. Тоді ми запитуємо, яку звичайну структуру вихідного " +"коду цей компілятор перетворює на спостережувані інструкції. Спеціалізація " +"може допомогти моделі запропонувати ці форми ефективніше, тоді як ціль все " +"ще визначає, які гіпотези виживуть." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"Корисні шаблони повинні зберігати свій контекст. TH08 використовує попередню" +" версію Visual C++, ніж TH095 і TH09. TH04 використовує компілятор, " +"відмінний від епохи DOS. Його {ph0}реконструкція коду гравця{ph1} " +"потребувала різних гіпотез щодо шаблонів вихідного коду та того, як " +"скомпільований код зв’язувався у виконуваний файл. Наявність того самого " +"автора дає нам початкове очікування, але кожна ціль говорить нам, чи вона " +"застосовна. Спеціалізована модель повинна розрізняти ці контексти." + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "Де зберігати цей досвід?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"Перш ніж вирішувати навчати цим шаблонам за допомогою моделювання, я б " +"подивився на знання, які ми вже зберігаємо в репозиторії. У " +"{ph0}попередньому есе{ph1} я писав про те, що це стало робочою пам’яттю " +"проекту. Ця пам’ять містить уроки компілятора, а також джерело та невдалі " +"експерименти, які зробили їх корисними." + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"Отримавши цей запис, агент може почати з відомого результату та перевірити, " +"як ми його досягли. Він також може уникнути альтернатив, які вже зазнали " +"невдачі в тому ж контексті. Початкова точка покращується без зміни навчених " +"параметрів моделі." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"Під час реконструкції функції TH09 для створення ворогів {ph0}GPT-6-Luna-max" +" сеанс{ph1} взяв гіпотезу з відповідного вихідного коду TH08. Цей попередній" +" код пропонував, як організувати цикл і обробляти збої в ньому. Власні " +"свідчення TH09 надали деталі гри. Перебудови з джерела підтвердили, що вся " +"функція відповідає." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}«Модель не знала про цю примху компілятора» та «модель не могла її " +"використати» — це різні проблеми.{ph1} Отримання може вирішити першу, але " +"для того, щоб перетворити цей факт на корисну кандидатуру, все ще потрібно " +"попрацювати. І якщо вихідний код кандидата зазнає невдачі, модель повинна " +"вирішити, що означає результат для її наступної спроби. Знання відповідного " +"шаблону – це лише початок цього процесу." + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"З цієї причини я б включив у будь-яке порівняння меншу модель ШІ загального " +"призначення з хорошим пошуком. Надання такої ж бібліотеки досвіду допомогло " +"б нам визначити, що додає навчання: краще формування кандидатів, кращі " +"рішення після зворотного зв’язку чи те й інше." + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "Вивчення наступного корисного експерименту" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"Я б найбільше хотів вивчити ці рішення після зворотного зв’язку. При " +"агентній реконструкції модель пропонує пояснення та перевіряє його на " +"програму. Навіть якщо вихідний код кандидата не вдається, результат може " +"звузити можливості, що залишилися, і запропонувати більш корисний наступний " +"експеримент." + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"Знову розглянемо приклад із тимчасовими результатами. Його остаточне джерело" +" показує схему, яка спрацювала, але шлях до цього джерела містить більше " +"інформації. Кожен із наступних експериментів допоміг визначити, що ще " +"потрібно змінити." + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "Експериментуйте" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "Чого нас навчив його результат" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "Спробуйте інший спосіб скопіювати результати" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "Код копіювання змінюється, але функція все ще відрізняється." + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "Використовуйте повернуті значення безпосередньо" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "" +"Копіювання відбувається за оригінальним шаблоном, але тимчасове зберігання " +"все ще відрізняється." + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "Зберігайте всі тимчасові змінні дійсними одночасно" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "" +"Вони отримують окреме сховище, але одне все одно перекриває значення, яке " +"буде використано пізніше для перевірок на конфлікти." + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "Зберігайте значення перевірки зіткнень окремо" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "Чиста перекомпіляція з джерела підтверджує повний збіг." + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "" +"Вибрані експерименти з наведеної вище реконструкції функції атаки, " +"згруповані за тим, чому вони нас навчили." + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"Навчальний запис, який зберігає цю послідовність, може навчити більше, ніж " +"одне лише кінцеве джерело. Він пов’язує кожну запропоновану зміну з " +"результатом спостереження та наступним рішенням. Збереження неефективних " +"варіацій також має значення: вони повідомляють пізнішому агенту, які " +"гіпотези вже були перевірені в цьому контексті." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}Я хотів би, щоб модель навчилася використовувати зворотній " +"зв’язок.{ph1} Нейтральний результат є причиною переглянути варіант; змінена " +"невідповідність може вказувати на іншу причину. Коли локальні редагування " +"перестають бути інформативними, агент повинен мати можливість переглянути " +"вихідне представлення або знову перевірити докази." + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"Щоб навчитися робити такий вибір, потрібна автономія. Іноді корисним " +"наступним кроком є ​​перевірка іншої функції, що викликає цю функцію, а не " +"компіляція іншого редагування. Жорстка послідовність необхідних переписів " +"усуне цей вибір разом із частиною поведінки, яку ми хочемо, щоб модель " +"навчила." + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "Чому зворотне проектування цікаве для навчання з підкріпленням" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"Реконструкція з визначеними межами дає практичне середовище для вивчення цих" +" рішень. Коли версії цілі й компілятора зафіксовані, багато експериментів із" +" вихідним кодом дають швидкий, відтворюваний зворотний зв’язок. Агент може " +"скомпілювати кандидата, перевірити відмінності та вибрати наступний крок." + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"Навчання з підкріпленням може тренувати цей процес прийняття рішень за " +"допомогою зворотного зв’язку, який він отримує. Кандидат, прийнятий за " +"певною перевіркою, дає нам конкретний результат, а послідовність спроб " +"дозволяє спостерігати, скільки роботи знадобилося для його досягнення." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"У цьому напрямку вже є дослідження. {ph0}LLM4Decompile{ph1} вивчає моделі, " +"що спеціалізуються на декомпіляції. {ph2}AutoDecompiler{ph3} тренує " +"багатоповоротне вдосконалення за допомогою компілятора та відгуків про " +"виконання. Їх дослідження зосереджені на функціональному відновленні. Для " +"цієї роботи я б використав їх як відправну точку для пов’язаного завдання " +"пошуку джерела, яке відповідає історичним результатам компілятора." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"Однак, щоб досягти такого чітко визначеного завдання, нам може спочатку " +"знадобитися встановити, якими ресурсами володіє функція, або розробити нове " +"вимірювання часу виконання. У TH08, наприклад, {ph0}різниця зіткнень " +"призвела до того, як оригінальна програма обробляла обчислення з плаваючою " +"комою{ph1}. Перш ніж агент міг перевірити корисну поправку, він повинен був " +"визначити, що вимірювати." + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"Це говорить про іншу роль широкофункціональної моделі: допомогти зробити " +"проблему перевіреною в першу чергу. Щойно така перевірка з’явиться, ми " +"зможемо дослідити, наскільки далеко може зайти вивчена політика пошуку з " +"нею." + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "Коли засіб перевірки помиляється" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"Коли перевірка використовується для навчання, її охоплення стає частиною " +"навчального середовища. Такий еталон або засіб перевірки я називаю оракулом." +" Навіть відтворювану відповідь потрібно розуміти в межах того, що вона " +"справді перевіряє." + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 показав, як багато можна приховати за межами цього покриття. Деякі " +"реконструйовані функції пройшли точне порівняння з неправильними плаваючими " +"константами. Компаратор скоригував переміщені адреси перед перевіркою " +"інструкцій, але він перевірив значення за цими адресами лише тоді, коли вони" +" були явно записані для перевірки. Інші літеральні посилання можуть пройти " +"без перевірки їхніх значень." + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"Щоб усунути цю прогалину, {ph0}ширший аудит{ph1} виявив неправильні " +"константи у функціях, які вже пройшли перевірку. В одному обчисленні " +"реконструйований вихідний код додав {ph2}, тоді як оригінальний додав {ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}Модель, нагороджена старою перевіркою перевірки, може навчитися " +"виробляти неправильну реконструкцію.{ph1} Ми не навчали модель RL за " +"допомогою цього засобу перевірки. Помилка, яку ми помітили, була в самій " +"перевірці. Але якби ми використовували це як винагороду, та сама сліпа пляма" +" стала б частиною навчального середовища." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"Функціональний тест може мати подібну сліпу пляму. {ph0}Дослідження з " +"використанням диференціального фаззингу на декомпіляторах LLM{ph1} виявили " +"поведінкові відмінності поза наданими тестами. Тому ми повинні розуміти " +"проходження тесту з точки зору випадків, які він виконує. Подібним чином " +"збіг інструкцій слід розуміти з точки зору байтів і даних, які він " +"перевіряє." + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"Перш ніж використовувати будь-який вид перевірки підтвердження як " +"винагороду, я б перевірив його з навмисно неправильними результатами. Я " +"також хотів би зберегти остаточне прийняття окремо від вихідного коду, який " +"агент ШІ може редагувати. Модель повинна пройти перевірку шляхом коректної " +"реконструкції програми. Інакше просте копіювання байтового масиву з " +"цільового об’єкта може обдурити наївну систему підрахунку балів без " +"відновлення корисного вихідного коду." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}Поліпшення моделі й поліпшення її засобу перевірки — частини однієї " +"дослідницької проблеми.{ph1} Що краще модель оптимізує наданий зворотний " +"зв’язок, то важливіше розуміти сам цей зв’язок." + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "Варте порівняння" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"За наявності цих перевірок я б почав порівняння з фіксованої сім’ї завдань " +"VC7.1. По-перше, я б визначив базові лінії для сильної моделі ШІ загального " +"призначення та меншої. Додавання знань про репозиторій до меншої моделі " +"покаже, які зміни в пошуку. Звідти контрольована спеціалізація та версія з " +"навчанням RL дозволять нам перевірити, що додає навчання." + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"Щоб зробити ці порівняння інформативними, кожна модель отримуватиме однакові" +" цільові дані та інструменти. Для порівнянь, пов’язаних із навчанням, я б " +"також залишив базову модель і наявні знання фіксованими. Повторні " +"випробування допомогли б відрізнити надійну стратегію пошуку від " +"альтернативного рішення, яке вдалось випадково." + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"Потім я б розділив дві початкові умови: реконструкція функції з її " +"інструкцій і закриття кількох останніх відмінностей у підтримуваному " +"джерелі. Спеціалізована модель штучного інтелекту може бути чудовою для " +"другого завдання, тоді як модель загального призначення виконує велику " +"частину роботи, необхідної для її досягнення. Вимірювання їх окремо виявило " +"б такий розподіл роботи." + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"Для обох завдань потрібні тестові приклади, яких немає ані в навчальних " +"даних, ані в матеріалах для пошуку. Пов’язані ігри Touhou мають спільний код" +" рушія. Якщо відкласти лише одну гру, майже однакові функції все одно можуть" +" потрапити до навчання. Я б натомість відкладав цілі групи спорідненого " +"вихідного коду й виключав їхні перевірені відповіді як із навчальних даних, " +"так і з бібліотеки пошуку." + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"Для цих прикладів результатом будуть повні, незалежно перевірені збіги. " +"Рівні бюджети для пропозицій кандидатів або спроб компіляції допоможуть " +"порівняти якість пошуку. Вимірювання загального часу та вартості, включаючи " +"компіляцію, покаже, чи забезпечує така якість ефективний робочий процес. " +"Витрати на навчання слід вимірювати окремо." + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"Обидва вимірювання мають значення. Дешева пропозиція може призвести до " +"дорогого робочого процесу, якщо це спричинить більше компіляцій, які не " +"надають корисної інформації. І навпаки, повільніша модель може бути " +"економнішою, якщо швидше знайде корисний експеримент. Порівняння повинно " +"включати ці витрати." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"Підрахунок прийнятих результатів також зберігає проміжний прогрес у " +"перспективі. Один TH09 {ph0}кандидат для повторного збереження{ph1} виглядав" +" багатообіцяючим, оскільки скомпільована функція мала очікуваний розмір, але" +" повне порівняння все одно виявило значні відмінності. Зміни в тому, як " +"компілятор впорядковує інструкції, також можуть створити враження, що " +"кандидат ближчий до цілі, не вирішуючи основної проблеми. Така діагностика " +"може керувати наступним експериментом, але я б перевірив будь-яку винагороду" +" за «наближення» до повної перевірки прийняття." + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"Мені було б цікаво побачити закономірності цих результатів. Якщо " +"спеціалізована модель штучного інтелекту добре справляється зі знайомим " +"сімейством компіляторів, чи збережеться ця перевага після зміни " +"інструментального ланцюжка? І коли обидві моделі штучного інтелекту мають " +"хороший пошук, скільки ще додає навчання? Відповіді допоможуть нам вирішити," +" де кожен тип моделі є найбільш корисним." + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "Що це за агент ШІ?" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"Якщо виявиться, що моделі допомагають на різних етапах, корисною " +"домовленістю може бути широкий агент із доступом до спеціалізованої моделі " +"ШІ. Коли завдання стає звичним пошуком компілятора, воно може запитати " +"спеціалізовану модель AI для кандидатів. Якщо результати перестають мати " +"сенс, ширший агент може повернутися до цільових доказів або дослідити саме " +"вимірювання." + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"У такій домовленості спеціалізація стає інструментом, який агент може " +"вибрати під час розслідування. Агент зберігає свободу змінювати напрямок, що" +" уможливило попередні прориви." + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"Це природно випливає з промислових змін, які я описав у попередньому есе. " +"Можливість проводити багато інших експериментів змінює спосіб організації " +"роботи. Це також дає нам привід запитати, які рішення потребують широкого " +"оцінювання, а які покращуються через повторний досвід роботи з певним " +"ремеслом." + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"Наразі найсильніша універсальна модель заслужила місце в моєму робочому " +"процесі, розв’язавши проблеми, які раніше залишалися нерозв’язаними. Ці " +"успіхи спонукають мене дослідити процес: яка частина залежала від широти її " +"можливостей, а яку могла б виконати модель, навчена саме такої " +"реконструкції?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}Репозиторії дають нам спосіб перевірити запитання.{ph1} Я хотів би " +"використати цей запис для порівняння та побачити, що ми дізнаємося." + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "Докази та атрибуція моделі" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"Приклади описують історичні порівняння на рівні функцій, записані у " +"зв’язаних комітах. Перевірки виконання всієї гри є окремими етапами. Вихідні" +" та відповідні записи було переглянуто 13 жовтня 2026 року." + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"Мітка TH09 {ph0} означає GPT-6-Astra. Модель, пов’язана з {ph1}, змінювалася" +" з часом, тому лише ця мітка не може ідентифікувати модель, яка " +"використовується в певному циклі. Явні назви моделей в інших цитованих " +"комітах забезпечують записане присвоєння. Ці історії не містять узгоджених " +"вимірювань витрат або зусиль для кожного запуску." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 терміни життя тимчасових об’єктів:{ph1} {ph2}попередній перегляд " +"контексту{ph3} та {ph4}повна реконструкція, підтверджена чистою " +"перекомпіляцією з джерела{ph5}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}TH09 Front зворотний виклик:{ph1} {ph2}діагностичні області, які не " +"збігаються{ph3}, {ph4}Astra новий графік потоку керування{ph5} та його " +"{ph6}перевірка прийнятої повторної компіляції{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}Знання компілятора:{ph1} {ph2}TH095 повторне використання поведінки " +"сегмента ідентифікатора{ph3}, {ph4}TH04 вихідний код гравця та порядок " +"зв’язування{ph5} та повна реконструкція {ph6}Luna функції створення ворогів " +"TH09{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}Перевірки та діагностика:{ph1} {ph2}TH08 постійний аудит із плаваючою " +"комою{ph3}, {ph4}TH08 числове дослідження{ph5} та {ph6}TH09 реконструкція зі" +" збереженням повтору{ph7}." + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"Пропоновані порівняння спеціалізованих моделей і навчання RL є майбутніми " +"експериментами. Історії проекту мотивують їх. Жодна навчена спеціалізована " +"модель не оцінювалася для цього есе." + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "Чи потрібна для зворотного проектування більша модель? | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"Чого навчила мене реконструкція Touhou щодо загальних моделей, спеціальних " +"моделей і навчання пропонувати кращі гіпотези на основі відгуків " +"компілятора." + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "На цій сторінці" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "Есе про моделі та зворотне проектування" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "Вибрані досліди реконструкції" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "Нагору" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "Автор: N0zoM1z0 · 13 жовтня 2026 р. · Чернетка для обговорення" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/vi/messages.po b/website/locales/vi/messages.po index e34a78e22..23805a074 100644 --- a/website/locales/vi/messages.po +++ b/website/locales/vi/messages.po @@ -32,18 +32,18 @@ msgid "Thoughts on reverse engineering." msgstr "Suy nghĩ về kỹ thuật đảo ngược." #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "Tái thiết trò chơi{ph0}Bài luận" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "Cuộc cách mạng công nghiệp trong kỹ thuật đảo ngược" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " @@ -52,19 +52,20 @@ msgstr "" "NHỮNG gì HƯỚNG DẪN SỬ DỤNG LẠI VÀ tái TẠO TH08, TH095 và TH04 đã dạy tôi về " "các agent, xác minh và kiến thức mà các hợp chất trong các dự án." -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "Đọc bài viết" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "Bởi N0zoM1z0 * 10 tháng mười 2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -100,6 +101,8 @@ msgstr "" "REA và AI." #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -136,6 +139,7 @@ msgstr "" "mã hóa của bạn." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -156,6 +160,7 @@ msgid "REA home" msgstr "Trang chủ REA" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -176,6 +181,7 @@ msgid "Main navigation" msgstr "Điều hướng chính" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -196,6 +202,7 @@ msgid "REA on GitHub" msgstr "REA TRÊN GitHub" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -216,6 +223,7 @@ msgid "Join REA on Discord" msgstr "Tham gia Discord của REA" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -235,7 +243,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "Theo DÕI REA Trên X" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "Về đầu trang" @@ -11594,6 +11602,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "Khoảng cách tương tự: 256 ÷ 16 = 16 đơn vị góc = 22,5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -11614,6 +11623,7 @@ msgid "Showcases" msgstr "Ví dụ thực tế" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -11634,6 +11644,7 @@ msgid "Blog" msgstr "Blog" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -11654,6 +11665,7 @@ msgid "Get started" msgstr "Bắt đầu" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -11673,7 +11685,8 @@ msgctxt "shared" msgid "Guides" msgstr "Hướng dẫn" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -11693,7 +11706,8 @@ msgctxt "shared" msgid "Source" msgstr "Mã nguồn" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -11713,7 +11727,8 @@ msgctxt "shared" msgid "MIT license" msgstr "Giấy phép MIT" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -11733,7 +11748,8 @@ msgctxt "shared" msgid "Join the community" msgstr "Tham gia cộng đồng" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -12715,8 +12731,1334 @@ msgstr "" #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "N0zoM1z0 · 13 tháng 10 năm 2026 · Bản nháp để góp ý" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Bởi N0zoM1z0 · 13 tháng 10 năm 2026" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "Bỏ qua nội dung" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "{ph0}Home{ph1} / {ph2}Blog{ph3} / Mô hình và kỹ thuật đảo ngược" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "Tiểu luận" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "Kỹ thuật đảo ngược có cần một mô hình lớn hơn không?" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "Bởi N0zoM1z0 · 13 tháng 10 năm 2026" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "Điểm chính" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "" +"{ph0}Khả năng của mô hình ảnh hưởng đến những giả thuyết mà tác nhân AI có " +"thể đề xuất.{ph1} Các phiên sau đã giải quyết các vấn đề mà công việc trước " +"đó chưa được giải quyết." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "" +"{ph0}Một trình biên dịch quen thuộc cung cấp các mẫu cụ thể để một mô hình " +"chuyên biệt học.{ph1} Một số kết quả khớp mã máy chính xác phụ thuộc vào các" +" mẫu mã nguồn cụ thể, có thể tái sử dụng." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "" +"{ph0}Bao gồm kiến thức kho lưu trữ trong một so sánh công bằng.{ph1} Một mô " +"hình nhỏ hơn với khả năng truy xuất kiến thức tốt có thể đã có đủ thông tin " +"để đạt được tiến bộ." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "" +"{ph0}Huấn luyện mô hình để chọn thử nghiệm hữu ích tiếp theo và kiểm tra " +"công cụ xác minh của nó.{ph1} Phản hồi chỉ hữu ích trong phạm vi công cụ xác" +" minh những điều đúng." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "" +"{ph0}Đánh giá kết quả tái thiết đã được xác minh cùng với thời gian và chi " +"phí.{ph1} Đó là những gì tôi muốn đo lường cho quy trình làm việc hoàn chỉnh" +" của tác nhân AI." + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "Trong bài viết này" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}Mô hình mạnh hơn đã thay đổi điều gì?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}Tại sao nên chuyên môn hóa?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}Kinh nghiệm này nên được lưu trữ ở đâu?{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}Học thí nghiệm hữu ích tiếp theo{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}Học kỹ thuật đảo ngược và tăng cường{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}Khi công cụ xác minh sai{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}Một so sánh đáng chạy{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}Điều này gợi ý loại tác nhân AI nào?{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "Mô hình mạnh hơn đã thay đổi điều gì?" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"Kể từ tháng 8 năm 2026, tôi đã khám phá mức độ kỹ thuật đảo ngược mà một tác" +" nhân có thể thực hiện khi có quyền truy cập vào một chương trình và cách " +"kiểm tra công việc của nó. Phần lớn kinh nghiệm của tôi đến từ việc xây dựng" +" lại trò chơi, đặc biệt là Touhou. Khi tôi thay đổi mô hình, các kho lưu trữ" +" đã lưu giữ một bản ghi về những gì mỗi phiên đã khôi phục được." + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"Phần lớn công việc hướng tới tái dựng chính xác: biên dịch mã nguồn đang " +"được duy trì bằng chuỗi công cụ lịch sử có phiên bản cố định, rồi kiểm tra " +"xem nó có tái tạo hàm đích hay không. Khi mô hình sau giải quyết được điểm " +"mà công việc trước bị mắc kẹt, thay đổi trong mã nguồn cho thấy điều gì đã " +"giúp tiến triển." + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "Các biến tạm thời cần có thời gian tồn tại chồng chéo" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"Phần lớn logic của hàm tấn công TH09 đã được xây dựng lại, nhưng trình biên " +"dịch vẫn lưu trữ các kết quả tạm thời khác với kết quả ban đầu. Vấn đề còn " +"lại là những kết quả đó cần tồn tại trong bao lâu và khi nào bộ nhớ của " +"chúng có thể được sử dụng lại." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"Bản ghi phiên {ph0}GPT-6-Sol{ph1} đã báo cáo sự không khớp này sau khi kiểm " +"tra một số cách sắp xếp mã nguồn thông thường. Khi {ph2}GPT-6.1-Sol kiểm tra" +" lại vấn đề{ph3}, nó đã đề xuất một giả thuyết hữu ích về thời gian tồn tại " +"của các biến tạm thời này." + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"Hàm này tạo ra các kết quả tạm thời và sao chép chúng vào các vị trí hoạt " +"ảnh. Trong C++, kết quả tạm thời vẫn tồn tại cho đến khi biểu thức sử dụng " +"nó kết thúc. Việc nhóm các hoạt động thực tế thành một biểu thức giúp giữ " +"cho các kết quả đó tồn tại cùng nhau, do đó trình biên dịch không thể sử " +"dụng lại cùng một bộ lưu trữ cho tất cả chúng nữa." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"Một kết quả vẫn được chia sẻ bộ nhớ với giá trị được sử dụng sau này để kiểm" +" tra xung đột. Việc đưa ra giá trị đó trong thời gian tồn tại của chính nó " +"đã giải quyết được sự chồng chéo còn lại. Sau một số thử nghiệm có mục tiêu," +" {ph0}việc biên dịch lại rõ ràng từ nguồn đã xác nhận rằng hàm đầy đủ khớp " +"với{ph1}." + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"Tôi quan tâm đến việc khám phá và thử nghiệm hỗ trợ lẫn nhau như thế nào. " +"Giả thuyết về thời gian tồn tại của đối tượng tạm thời đã mang lại cho tác " +"nhân AI một hướng đi mới. Phản hồi của trình biên dịch đã giúp nó tinh chỉnh" +" giả thuyết đó thành một cách sắp xếp mã nguồn phù hợp." + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "Biểu đồ luồng điều khiển khác" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"Hàm TH09 thứ hai cho biết tại sao hướng đi mới đó lại quan trọng. Ở đây, trở" +" ngại là làm thế nào một số chi nhánh chia sẻ công việc cuối cùng của họ. " +"Công việc trước đó của {ph0}GPT-6.1-Sol{ph1} có các phần của hàm khớp với " +"nhau, nhưng các nhánh vẫn được nối với nhau khác với ban đầu." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra đã thay đổi biểu đồ luồng điều khiển xung quanh{ph1}. Các " +"đường dẫn chính hiện đã dẫn đến bản cập nhật cuối cùng được chia sẻ và quay " +"trở lại. Trong cấu trúc nguồn đó, trình biên dịch kết hợp các nhánh giống " +"như bản gốc. Chức năng hoàn chỉnh đã khớp và một cuộc kiểm tra xây dựng lại " +"riêng biệt {ph2} đã xác nhận điều đó{ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}Việc thay đổi cách biểu diễn đã mở ra một không gian tìm kiếm hiệu quả " +"hơn.{ph1} Cả hai trường hợp đều cho tôi lý do để xem xét nghiêm túc khả năng" +" của mô hình chung: một tác nhân có thể tiếp tục thử các biến thể cục bộ " +"trong một thời gian dài trước khi tìm ra cách tốt hơn để thể hiện công việc " +"xung quanh." + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"Đồng thời, các phiên sau bắt đầu với nguồn trước đó và ghi lại lỗi. Công " +"việc tích lũy đó là một phần lợi thế của họ. Lịch sử cho phép chúng ta kiểm " +"tra những khám phá và kiểm tra chúng; một so sánh có kiểm soát sẽ giúp tách " +"biệt tác động của mô hình khỏi trải nghiệm mà nó kế thừa." + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"Đây là điều làm cho câu hỏi trở nên thú vị đối với tôi. Một số tiến bộ đến " +"từ việc tìm kiếm một đại diện mới. Phần lớn công việc còn lại liên quan đến " +"việc kiểm tra các cách sắp xếp mã nguồn khác nhau dựa trên một trình biên " +"dịch quen thuộc. Bao nhiêu trong số này đòi hỏi khả năng suy luận rộng? Một " +"mô hình có thể học được bao nhiêu khi làm việc liên tục với trình biên dịch " +"và mã đó?" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "Tại sao một mô hình AI chuyên biệt có thể hoạt động tốt?" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"Touhou cho chúng ta lý do cụ thể để mong đợi trải nghiệm đó sẽ hữu ích. Phần" +" lớn mã trò chơi đều đến từ cùng một tác giả, ZUN, vì vậy những thói quen " +"lặp đi lặp lại là một giả định khởi đầu hợp lý. Các kho lưu trữ hỗ trợ điều " +"đó bằng các ví dụ hữu ích: các vòng lặp liên quan và cách sắp xếp trợ giúp " +"đã cung cấp các giả thuyết cho các trò chơi." + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "" +"Các mẫu nguồn đó trở nên cụ thể hơn khi chúng tôi đưa trình biên dịch vào. " +"Trong một phiên bản và tập hợp các tùy chọn cụ thể, những thay đổi nhỏ về " +"nguồn có thể tạo ra những khác biệt lặp lại trong mã máy. Một số trong những" +" khác biệt đó mơ hồ một cách đáng ngạc nhiên." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"Ví dụ: trong TH095, {ph0}ngay cả tên biến cục bộ cũng có thể ảnh hưởng đến " +"vị trí VC7.1 lưu trữ nó{ph1}. Việc di chuyển khai báo của nó không giải " +"quyết được bố cục lưu trữ. Tuy nhiên, hàm nền trước đó đã tiết lộ kiểu đặt " +"tên ảnh hưởng đến sự lựa chọn của trình biên dịch. Điều đó gợi ý một giả " +"thuyết hứa hẹn hơn cho việc {ph2}tái tạo {ph3}{ph4}." + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "" +"Việc sử dụng lại mẫu đó cho biến kích thước tệp sẽ tạo ra bố cục cần thiết " +"mà không cần thêm bộ nhớ. Các chức năng đầy đủ sau đó phù hợp." + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "" +"Đây là một chi tiết rất cụ thể mà một mô hình AI đa năng có thể không biết " +"khi lần đầu gặp sự cố. Một mô hình chuyên dụng cho trình biên dịch này có " +"thể xem nhiều trường hợp như vậy và tìm hiểu xem biến thể mã nguồn nào đáng " +"để thử nghiệm trước tiên." + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"Cơ hội đặc biệt rõ ràng trong các bài toán so khớp cuối cùng khi chúng ta đã" +" hiểu rõ hành vi. Sau đó, chúng tôi sẽ hỏi trình biên dịch này sẽ chuyển sự " +"sắp xếp mã nguồn thông thường nào thành các lệnh máy được quan sát. Sự " +"chuyên môn hóa có thể giúp mô hình đề xuất các hình thức đó hiệu quả hơn, " +"trong khi mục tiêu vẫn xác định giả thuyết nào tồn tại." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"Các mẫu hữu ích phải đi kèm bối cảnh áp dụng. TH08 dùng phiên bản Visual C++" +" cũ hơn TH095 và TH09. TH04 dùng một trình biên dịch khác thuộc thời DOS. " +"{ph0}Việc tái dựng mã điều khiển người chơi{ph1} cần những giả thuyết khác " +"về cách viết mã và cách liên kết thành tệp thực thi. Cùng tác giả là một giả" +" định ban đầu; từng mục tiêu cho biết giả định đó có phù hợp không. Mô hình " +"chuyên biệt cần học cách phân biệt các bối cảnh này." + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "Kinh nghiệm này nên được lưu trữ ở đâu?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"Trước khi quyết định dạy những mẫu đó thông qua đào tạo mô hình, tôi sẽ xem " +"xét kiến thức mà chúng tôi đã lưu giữ trong kho. Trong {ph0}bài luận " +"trước{ph1}, tôi đã viết về việc nó trở thành bộ nhớ làm việc của dự án. Bộ " +"nhớ đó bao gồm các bài học về trình biên dịch, cùng với nguồn và các thử " +"nghiệm thất bại khiến chúng trở nên hữu ích." + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"Bằng cách truy xuất bản ghi đó, nhân viên hỗ trợ có thể bắt đầu với kết quả " +"đã biết và kiểm tra cách chúng tôi đạt được kết quả đó. Nó cũng có thể tránh" +" được những lựa chọn thay thế đã thất bại trong cùng bối cảnh. Điểm bắt đầu " +"được cải thiện mà không thay đổi các tham số đã huấn luyện của mô hình." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"Trong khi xây dựng lại chức năng tạo kẻ thù của TH09, phiên {ph0}GPT-6-Luna-" +"max{ph1} đã lấy một giả thuyết từ mã nguồn TH08 có liên quan. Mã trước đó đã" +" đề xuất cách tổ chức vòng lặp và xử lý các lỗi bên trong nó. Bằng chứng " +"riêng của TH09 đã cung cấp chi tiết cụ thể về trò chơi. Việc xây dựng lại từ" +" nguồn đã xác nhận rằng toàn bộ hàm khớp với bản gốc." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}\"Mô hình không biết lỗi của trình biên dịch\" và \"mô hình không thể " +"sử dụng nó\" là các vấn đề khác nhau.{ph1} Việc truy xuất có thể giải quyết " +"được vấn đề đầu tiên, nhưng việc biến thực tế đó thành một mã ứng viên hữu " +"ích vẫn cần nhiều công sức. Và nếu mã nguồn mã ứng viên không thành công, mô" +" hình phải quyết định kết quả có ý nghĩa gì cho lần thử tiếp theo. Biết mẫu " +"có liên quan chỉ là bước khởi đầu của quá trình đó." + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"Vì lý do đó, tôi sẽ đưa vào một mô hình AI có mục đích chung nhỏ hơn với khả" +" năng truy xuất tốt trong mọi so sánh. Cung cấp cho nó cùng một thư viện " +"kinh nghiệm sẽ giúp chúng tôi xác định nội dung đào tạo bổ sung: tạo ứng " +"viên tốt hơn, đưa ra quyết định tốt hơn sau khi phản hồi hoặc cả hai." + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "Học thí nghiệm hữu ích tiếp theo" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"Những quyết định sau khi được phản hồi là điều tôi muốn nghiên cứu nhất. " +"Trong tái cấu trúc tác nhân, mô hình đề xuất một lời giải thích và kiểm tra " +"nó dựa trên chương trình. Ngay cả khi mã nguồn ứng viên không thành công, " +"kết quả có thể thu hẹp các khả năng còn lại và đề xuất thử nghiệm tiếp theo " +"hữu ích hơn." + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "" +"Hãy xem xét lại ví dụ về kết quả tạm thời. Nguồn cuối cùng của nó cho thấy " +"sự sắp xếp đã có hiệu quả nhưng đường dẫn đến nguồn đó chứa nhiều thông tin " +"hơn. Mỗi thử nghiệm sau đây đã giúp xác định những gì vẫn cần thay đổi." + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "Thử nghiệm" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "Kết quả của nó đã dạy chúng ta điều gì" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "Hãy thử một cách khác để sao chép kết quả" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "Mã sao chép thay đổi nhưng chức năng vẫn khác." + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "Sử dụng trực tiếp các giá trị được trả về" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "Sao chép theo mẫu ban đầu, nhưng lưu trữ tạm thời vẫn khác." + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "Giữ tất cả các biến tạm thời hợp lệ cùng một lúc" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "" +"Chúng có bộ lưu trữ riêng biệt, nhưng một bộ vẫn chồng lên một giá trị được " +"sử dụng sau này để kiểm tra va chạm." + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "Giữ riêng giá trị kiểm tra va chạm" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "" +"Việc biên dịch lại rõ ràng từ nguồn xác nhận sự trùng khớp hoàn chỉnh." + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "" +"Các thử nghiệm được chọn từ quá trình tái cấu trúc chức năng tấn công ở " +"trên, được nhóm theo những gì chúng đã dạy chúng ta." + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"Một bản ghi huấn luyện lưu giữ trình tự này có thể dạy nhiều hơn chỉ riêng " +"nguồn cuối cùng. Nó kết nối từng thay đổi được đề xuất với kết quả được quan" +" sát và quyết định tiếp theo. Việc giữ lại các biến thể không hiệu quả cũng " +"rất quan trọng: chúng cho tác nhân sau này biết những giả thuyết nào đã được" +" kiểm tra trong bối cảnh đó." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}Tôi muốn mô hình học cách sử dụng phản hồi.{ph1} Kết quả trung lập là " +"lý do để xem xét lại biến thể; sự không phù hợp đã thay đổi có thể chỉ ra " +"một nguyên nhân khác. Khi các chỉnh sửa cục bộ không còn mang tính thông " +"tin, nhân viên sẽ có thể xem lại phần trình bày nguồn hoặc kiểm tra lại bằng" +" chứng." + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"Học cách đưa ra những lựa chọn đó đòi hỏi sự tự chủ. Đôi khi, bước tiếp theo" +" hữu ích là kiểm tra người gọi khác thay vì biên dịch một bản chỉnh sửa " +"khác. Một chuỗi các lần viết lại bắt buộc một cách cứng nhắc sẽ loại bỏ lựa " +"chọn đó, cùng với một phần hành vi mà chúng ta muốn mô hình học." + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "Tại sao kỹ thuật đảo ngược lại thú vị cho việc học tăng cường" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"Tái dựng trong phạm vi xác định là môi trường thực tế để nghiên cứu các lựa " +"chọn này. Khi phiên bản của mục tiêu và trình biên dịch được cố định, nhiều " +"thử nghiệm mã nguồn cho phản hồi nhanh và có thể lặp lại. Tác nhân có thể " +"biên dịch mã ứng viên, xem khác biệt rồi chọn bước tiếp theo." + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"Học tăng cường có thể rèn luyện quá trình ra quyết định đó thông qua phản " +"hồi mà nó nhận được. Một mã ứng viên được chấp nhận bởi một cuộc kiểm tra " +"xác định sẽ cho chúng ta một kết quả cụ thể, trong khi trình tự các lần thử " +"cho phép chúng ta quan sát xem cần bao nhiêu công sức để đạt được kết quả " +"đó." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"Đã có nghiên cứu hỗ trợ hướng này. {ph0}LLM4Decompile{ph1} nghiên cứu các mô" +" hình chuyên dùng cho việc dịch ngược. {ph2}AutoDecompiler{ph3} đào tạo sàng" +" lọc nhiều lượt bằng cách sử dụng phản hồi của trình biên dịch và thực thi. " +"Nghiên cứu của họ tập trung vào phục hồi chức năng. Đối với công việc này, " +"tôi sẽ sử dụng chúng làm điểm khởi đầu cho nhiệm vụ liên quan là tìm nguồn " +"phù hợp với đầu ra của trình biên dịch lịch sử." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"Tuy nhiên, để đạt được nhiệm vụ được xác định rõ ràng như vậy, trước tiên " +"chúng ta có thể cần thiết lập tài nguyên nào mà hàm sở hữu hoặc nghĩ ra phép" +" đo thời gian chạy mới. Ví dụ: trong TH08, sự khác biệt về va chạm {ph0} dẫn" +" đến cách chương trình ban đầu xử lý các phép tính dấu phẩy động{ph1}. Trước" +" khi tác nhân có thể kiểm tra một sự điều chỉnh hữu ích, nó phải tìm ra " +"những gì cần đo lường." + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "" +"Điều này gợi ý một vai trò khác của một mô hình có khả năng rộng rãi: giúp " +"làm cho vấn đề có thể được kiểm tra ngay từ đầu. Khi kiểm tra đó tồn tại, " +"chúng tôi có thể điều tra xem chính sách tìm kiếm đã học có thể đi được bao " +"xa." + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "Khi công cụ xác minh sai" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"Khi dùng một phép kiểm tra để huấn luyện, phạm vi kiểm tra cũng trở thành " +"một phần của môi trường học. Tôi gọi chuẩn tham chiếu hoặc công cụ kiểm tra " +"đó là oracle. Ngay cả câu trả lời lặp lại được cũng cần được hiểu theo những" +" gì nó thực sự xác minh." + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 cho thấy vấn đề có thể ẩn ngoài phạm vi kiểm tra. Một số hàm tái dựng " +"dùng hằng số dấu phẩy động sai nhưng vẫn vượt qua so sánh chính xác. Bộ so " +"sánh điều chỉnh địa chỉ tái định vị trước khi kiểm tra lệnh máy, nhưng chỉ " +"kiểm tra giá trị tại các địa chỉ đó nếu chúng đã được ghi nhận rõ ràng để " +"xác minh. Những tham chiếu hằng số khác có thể vượt qua mà không được kiểm " +"tra giá trị." + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"Để giải quyết khoảng trống này, {ph0}kiểm tra rộng hơn{ph1} đã tìm thấy các " +"hằng số không chính xác trong các hàm đã vượt qua quá trình xác minh. Trong " +"một phép tính, mã nguồn được xây dựng lại đã thêm {ph2}, trong khi mã nguồn " +"ban đầu được thêm {ph3}." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}Mô hình được khen thưởng bằng kiểm tra xác minh cũ có thể học cách tạo " +"ra bản dựng lại không chính xác.{ph1} Chúng tôi không đào tạo mô hình RL " +"bằng trình kiểm tra đó. Sự thất bại mà chúng tôi quan sát thấy nằm ở chính " +"việc kiểm tra. Nhưng nếu chúng ta sử dụng nó như một phần thưởng thì điểm mù" +" đó sẽ trở thành một phần của môi trường học tập." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"Một bài kiểm tra chức năng có thể có điểm mù tương tự. {ph0}Nghiên cứu sử " +"dụng tính năng làm mờ vi phân trên bộ giải mã LLM{ph1} đã tìm thấy những " +"khác biệt về hành vi bên ngoài các thử nghiệm được cung cấp. Do đó, chúng ta" +" cần hiểu một bài kiểm tra đạt theo các trường hợp mà nó thực hiện. Tương " +"tự, một lệnh khớp phải được hiểu theo byte và dữ liệu mà nó xác minh." + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"Trước khi sử dụng một trong hai loại kiểm tra xác minh làm phần thưởng, tôi " +"sẽ kiểm tra nó với kết quả cố tình sai. Tôi cũng sẽ tách biệt sự chấp nhận " +"cuối cùng với mã nguồn mà tác nhân AI có thể chỉnh sửa. Mô hình phải vượt " +"qua quá trình xác minh bằng cách xây dựng lại chương trình một cách chính " +"xác. Mặt khác, chỉ cần sao chép một mảng byte từ mục tiêu có thể đánh lừa " +"một hệ thống tính điểm ngây thơ mà không khôi phục được mã nguồn hữu ích." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "" +"{ph0}Cải thiện mô hình học và cải thiện công cụ xác minh là hai phần của " +"cùng một vấn đề nghiên cứu.{ph1} Mô hình càng giỏi tối ưu phản hồi được cung" +" cấp, chúng ta càng cần hiểu chính phản hồi đó." + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "Một so sánh đáng chạy" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"Với những bước kiểm tra đó, tôi sẽ bắt đầu so sánh trên nhóm nhiệm vụ VC7.1 " +"cố định. Đầu tiên, tôi sẽ thiết lập các đường cơ sở cho một mô hình AI có " +"mục đích chung mạnh mẽ và một mô hình nhỏ hơn. Việc bổ sung kiến ​​thức về " +"kho lưu trữ vào mô hình nhỏ hơn sẽ cho thấy những thay đổi trong quá trình " +"truy xuất. Từ đó, chuyên môn hóa được giám sát và phiên bản được đào tạo RL " +"sẽ cho phép chúng tôi kiểm tra nội dung đào tạo bổ sung." + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"Để làm cho những so sánh đó trở nên hữu ích, mỗi mô hình sẽ nhận được các " +"bằng chứng và công cụ mục tiêu giống nhau. Đối với các so sánh dành riêng " +"cho đào tạo, tôi cũng sẽ giữ nguyên mô hình cơ sở và kiến ​​thức có sẵn. Các" +" thử nghiệm lặp đi lặp lại sẽ giúp phân biệt một chiến lược tìm kiếm đáng " +"tin cậy với một giải pháp ứng viên tình cờ thành công." + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"Sau đó, tôi sẽ tách hai điều kiện bắt đầu: xây dựng lại một hàm từ các lệnh " +"máy của nó và đóng một số khác biệt cuối cùng trong nguồn được duy trì. Một " +"mô hình AI chuyên dụng có thể thực hiện xuất sắc nhiệm vụ thứ hai trong khi " +"một mô hình AI có mục đích chung thực hiện phần lớn công việc cần thiết để " +"đạt được nhiệm vụ đó. Việc đo lường chúng một cách riêng biệt sẽ tiết lộ sự " +"phân chia công việc đó." + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"Cả hai nhiệm vụ đều cần loại trừ các mẫu thử nghiệm khỏi quá trình đào tạo " +"và truy xuất. Trò chơi Touhou liên quan chia sẻ mã công cụ. Việc loại trừ " +"một trò chơi khỏi quá trình huấn luyện vẫn có thể để lại các chức năng gần " +"như giống hệt nhau trong tập huấn luyện. Thay vào đó, tôi sẽ loại trừ toàn " +"bộ các nhóm mã nguồn có liên quan và loại bỏ các câu trả lời đã được xác " +"minh của chúng khỏi cả dữ liệu huấn luyện và thư viện truy xuất." + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"Kết quả cần đo là các lần khớp hoàn chỉnh, được xác minh độc lập. Giới hạn " +"số mã ứng viên hoặc số lần biên dịch bằng nhau giúp so sánh chất lượng tìm " +"kiếm. Đo tổng thời gian và chi phí, kể cả biên dịch, cho biết chất lượng đó " +"có tạo ra quy trình hiệu quả không. Chi phí huấn luyện cần tính riêng." + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"Cả hai phép đo đều quan trọng. Một đề xuất rẻ tiền có thể dẫn đến một quy " +"trình làm việc tốn kém nếu nó tạo ra nhiều lần biên dịch không cung cấp " +"thông tin hữu ích. Ngược lại, một mô hình chậm hơn có thể tiết kiệm hơn nếu " +"nó tìm thấy một thử nghiệm hữu ích sớm hơn. Việc so sánh nên bao gồm những " +"chi phí này." + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"Việc đếm các kết quả được chấp nhận cũng giúp duy trì tiến độ trung gian. " +"Một TH09 {ph0}mã ứng viên lưu lại phát lại{ph1} có vẻ đầy hứa hẹn vì hàm " +"được biên dịch có kích thước như mong đợi nhưng so sánh đầy đủ vẫn tìm thấy " +"những khác biệt đáng kể. Những thay đổi trong cách trình biên dịch sắp xếp " +"các lệnh máy cũng có thể khiến mã ứng viên có vẻ gần hơn mà không giải quyết" +" được vấn đề cơ bản. Chẩn đoán như vậy có thể hướng dẫn thử nghiệm tiếp " +"theo, nhưng tôi sẽ xác thực bất kỳ phần thưởng nào cho việc \"tiến gần hơn\"" +" so với kiểm tra chấp nhận hoàn toàn." + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"Tôi sẽ quan tâm đến mô hình trên các kết quả này. Nếu một mô hình AI chuyên " +"dụng hoạt động tốt trong dòng trình biên dịch quen thuộc, liệu lợi thế đó có" +" tồn tại được khi thay đổi chuỗi công cụ không? Và một khi cả hai mô hình AI" +" đều có khả năng truy xuất tốt thì việc đào tạo vẫn bổ sung thêm bao nhiêu? " +"Câu trả lời sẽ giúp chúng ta quyết định xem mỗi loại mô hình sẽ hữu ích nhất" +" ở đâu." + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "Điều này gợi ý loại tác nhân AI nào?" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"Nếu các mô hình tỏ ra hữu ích ở các giai đoạn khác nhau, thì sự sắp xếp hữu " +"ích có thể là một tác nhân rộng rãi có quyền truy cập vào một mô hình AI " +"chuyên dụng. Khi một nhiệm vụ trở thành một công cụ tìm kiếm trình biên dịch" +" quen thuộc, nó có thể yêu cầu mô hình AI chuyên biệt dành cho ứng viên. Nếu" +" kết quả không còn có ý nghĩa nữa, tác nhân rộng hơn có thể quay lại bằng " +"chứng mục tiêu hoặc tự điều tra phép đo." + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"Trong sự sắp xếp đó, sự chuyên môn hóa trở thành một công cụ mà đặc vụ có " +"thể lựa chọn khi cuộc điều tra phát triển. Người đại diện có quyền tự do " +"thay đổi hướng đi để có thể đạt được những đột phá trước đó." + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"Điều này diễn ra một cách tự nhiên từ sự chuyển dịch công nghiệp mà tôi đã " +"mô tả trong bài viết trước. Việc có thể chạy nhiều thử nghiệm hơn sẽ thay " +"đổi cách chúng ta tổ chức công việc. Nó cũng cho chúng ta lý do để hỏi những" +" quyết định nào cần được đánh giá rộng rãi và quyết định nào được cải thiện " +"thông qua kinh nghiệm lặp đi lặp lại với một nghề cụ thể." + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"Hiện tại, mô hình tổng quát mạnh nhất đã có chỗ đứng trong quy trình của tôi" +" vì giải quyết được những vấn đề trước đó còn bỏ ngỏ. Những thành công này " +"khiến tôi muốn xem kỹ quá trình: bao nhiêu phần phụ thuộc vào năng lực rộng " +"của mô hình, và bao nhiêu phần có thể giao cho một mô hình được huấn luyện " +"riêng cho kiểu tái dựng này?" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "" +"{ph0}Các kho lưu trữ cung cấp cho chúng tôi một cách để kiểm tra câu " +"hỏi.{ph1} Tôi muốn sử dụng hồ sơ đó để xây dựng sự so sánh và xem những gì " +"chúng tôi học được." + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "Bằng chứng và quy kết mô hình" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"Các ví dụ mô tả các so sánh cấp độ chức năng lịch sử được ghi lại trong các " +"cam kết được liên kết. Kiểm tra thời gian chạy toàn bộ trò chơi là các cột " +"mốc riêng biệt. Nguồn và hồ sơ trùng khớp đã được xem xét vào ngày 13 tháng " +"10 năm 2026." + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"Nhãn TH09 {ph0} có nghĩa là GPT-6-Astra. Mô hình được liên kết với {ph1} đã " +"thay đổi theo thời gian, do đó, riêng nhãn đó không thể xác định được mô " +"hình được sử dụng trong một lần chạy cụ thể. Tên mô hình rõ ràng trong các " +"cam kết được trích dẫn khác cung cấp thông tin ghi nhận. Những lịch sử này " +"không bao gồm các phép đo nhất quán về chi phí hoặc nỗ lực cho mỗi lần chạy." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "" +"{ph0}TH09 thời gian tồn tại của đối tượng tạm thời:{ph1} {ph2}xem xét ngữ " +"cảnh trước đó{ph3} và {ph4} việc xây dựng lại hoàn chỉnh được xác minh bằng " +"cách biên dịch lại rõ ràng từ nguồn{ph5}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}Callback Front của TH09:{ph1} {ph2}các vùng chẩn đoán chưa khớp{ph3}, " +"{ph4}đồ thị luồng điều khiển mới của Astra{ph5} và {ph6}phép kiểm tra biên " +"dịch lại đã được chấp nhận{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}Kiến thức về trình biên dịch:{ph1} {ph2}TH095 tái sử dụng hành vi của " +"nhóm nhận dạng{ph3}, {ph4}TH04 mã nguồn người chơi và thứ tự trình liên " +"kết{ph5} và {ph6}Luna tái thiết hoàn toàn chức năng tạo kẻ thù của " +"TH09{ph7}." + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}Kiểm tra và chẩn đoán:{ph1} {ph2}kiểm tra hằng số dấu phẩy động của " +"TH08{ph3}, {ph4}điều tra số học của TH08{ph5} và {ph6}tái dựng chức năng lưu" +" replay của TH09{ph7}." + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "" +"Những so sánh được đề xuất giữa các mô hình chuyên dụng và chương trình đào " +"tạo RL là những thử nghiệm trong tương lai. Lịch sử dự án thúc đẩy họ. Không" +" có mô hình chuyên ngành được đào tạo nào được đánh giá cho bài tiểu luận " +"này." + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "Kỹ thuật đảo ngược có cần một mô hình lớn hơn không? | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "" +"Việc tái xây dựng Touhou đã dạy tôi điều gì về các mô hình chung, mô hình " +"chuyên biệt và việc học cách đề xuất các giả thuyết tốt hơn từ phản hồi của " +"trình biên dịch." + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "Trên trang này" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "Mô hình và tiểu luận kỹ thuật đảo ngược" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "Các thí nghiệm tái thiết được chọn" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "Về đầu trang" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "N0zoM1z0 · 13 tháng 10 năm 2026 · Bản nháp để góp ý" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/zh-CN/messages.po b/website/locales/zh-CN/messages.po index 74350a21c..47f9d8a47 100644 --- a/website/locales/zh-CN/messages.po +++ b/website/locales/zh-CN/messages.po @@ -32,37 +32,38 @@ msgid "Thoughts on reverse engineering." msgstr "逆向工程的思考。" #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "游戏重建{ph0}论文" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "逆向工程的工业革命" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " "agents, verification and knowledge that compounds across projects." msgstr "什么手动重新和重建TH08，TH095和TH04教我关于编程助手，验证和知识，跨项目的化合物。" -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "阅读文章" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "通过N0zoM1z0*10十月2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -96,6 +97,8 @@ msgid "" msgstr "对逆向工程的思考，借鉴与 REA 和AI编程助手。" #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -130,6 +133,7 @@ msgid "" msgstr "REA-了解软件的工作原理。 逆向工程与你的编程助手." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -150,6 +154,7 @@ msgid "REA home" msgstr "REA 首页" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -170,6 +175,7 @@ msgid "Main navigation" msgstr "主导航" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -190,6 +196,7 @@ msgid "REA on GitHub" msgstr "在 GitHub 查看 REA" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -210,6 +217,7 @@ msgid "Join REA on Discord" msgstr "加入 REA 的 Discord 社区" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -229,7 +237,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "在 X 上关注 REA" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "回到顶部" @@ -10615,6 +10623,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "相同间距：256÷16=16角度单位=22.5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -10635,6 +10644,7 @@ msgid "Showcases" msgstr "案例展示" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -10655,6 +10665,7 @@ msgid "Blog" msgstr "博客" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -10675,6 +10686,7 @@ msgid "Get started" msgstr "开始使用" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -10694,7 +10706,8 @@ msgctxt "shared" msgid "Guides" msgstr "指南" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -10714,7 +10727,8 @@ msgctxt "shared" msgid "Source" msgstr "源代码" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -10734,7 +10748,8 @@ msgctxt "shared" msgid "MIT license" msgstr "MIT 许可证" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -10754,7 +10769,8 @@ msgctxt "shared" msgid "Join the community" msgstr "加入社区" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -11702,8 +11718,1112 @@ msgstr "从 Touhou 重建的经验，探讨通用模型、专用模型，以及 #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "作者 N0zoM1z0 · 2026 年 10 月 13 日 · 待审草稿" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "作者 N0zoM1z0 · 2026 年 10 月 13 日" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "跳转到内容" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "{ph0}首页{ph1} / {ph2}博客{ph3} / 模型与逆向工程" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "随笔" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "逆向工程需要更大的模型吗？" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "作者 N0zoM1z0 · 2026 年 10 月 13 日" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "要点" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "{ph0}模型能力会影响 agent 能提出哪些假设。{ph1} 后来的工作解决了早期尝试未能解决的问题。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "{ph0}熟悉的编译器为专用模型提供了具体的学习对象。{ph1} 有些精确匹配依赖细小但可以复用的源码模式。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "{ph0}也要公平评估代码仓库里的知识。{ph1} 较小的模型配合良好的检索，可能已经掌握了足以推进工作的知识。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "{ph0}训练模型选择下一个有价值的实验，同时检验它使用的验证器。{ph1} 反馈是否有用，取决于验证器究竟检查了什么。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "{ph0}结合时间和成本，评估通过验证的重建成果。{ph1} 这是我希望从整个 agent 工作流中获得的结果。" + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "在本文中" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}更强的模型改变了什么{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}为什么考虑专用模型？{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}这些经验应该存在哪里？{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}学习选择下一个有价值的实验{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}逆向工程与强化学习{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}当验证器出错时{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}值得做的对照实验{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}这意味着怎样的 agent？{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "更强的模型改变了什么" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"从 2026 年 8 月开始，我一直在探索：如果给 agent " +"访问程序的能力，以及验证自身工作的手段，它能独立完成多少逆向工作？我的经验主要来自游戏重建，尤其是东方系列。随着我更换模型，代码仓库记录了每次工作还原出的内容。" + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"这些工作很大一部分追求精确重建：用固定版本的历史工具链编译我们维护的源码，检查它能否复现目标函数。当后来的模型解决了早期工作卡住的问题，我们可以从源码变化中看出，是什么促成了这次进展。" + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "临时值需要同时保持有效" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"TH09 " +"中一个攻击函数的大部分逻辑已经还原，但编译器对临时结果的存储安排仍然与原版不同。剩下的问题在于：这些结果的生命周期应该多长，编译器何时可以复用它们的存储空间？" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"{ph0}GPT-6-Sol 的一次工作记录{ph1}表明，尝试几种常见的源码写法后，差异仍未消除。后来，{ph2}GPT-6.1-Sol " +"重新研究这个问题{ph3}，提出了一个有价值的假设：关键可能在于这些临时值的生命周期。" + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"这个函数会产生临时结果，再把它们复制到动画的位置数据中。在 C++ " +"中，临时结果会一直存在，直到使用它的表达式执行完毕。把实际操作放进同一个表达式，就能让这些结果同时存活，编译器也就不能为它们全部复用同一块存储空间。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"其中一个结果仍然与稍后用于碰撞检查的值共享存储。给那个值单独的生命周期后，最后一处重叠也消失了。经过几次有针对性的实验，{ph0}一次干净的重新构建确认，整个函数都已匹配{ph1}。" + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"我感兴趣的是，发现与验证如何共同推进工作。生命周期的假设给 agent 指明了新方向；编译器的反馈又帮助它逐步调整源码，直到找到匹配的写法。" + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "换一种控制流结构" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"TH09 的另一个函数说明了，新方向为什么重要。这次的难点是多个分支如何共享末尾的操作。{ph0}GPT-6.1-Sol " +"之前的工作{ph1}已经让函数的一些部分匹配，但分支汇合的方式仍与原版不同。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra " +"改写了外围的控制流图{ph1}，让主要路径汇入共同的最终更新与返回。在这种源码结构下，编译器合并分支的方式与原版一致。整个函数实现了匹配，{ph2}另一次独立的重新构建也确认了这一结果{ph3}。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}换一种表达方式，打开了更有价值的搜索空间。{ph1} 这两个例子让我认真看待通用模型的能力：agent " +"可能花很长时间尝试局部变化，才找到一种更好的整体写法。" + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"不过，后来的工作也继承了之前的源码和失败记录。这些积累本身就是优势的一部分。历史记录让我们可以检查发现及其验证过程；要区分模型能力和继承经验各自带来的帮助，还需要对照实验。" + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"这正是我觉得有趣的地方。有些进展来自新的表达方式，剩下的大量工作则是在熟悉的编译器上试验不同写法。其中有多少需要广泛的推理能力？又有多少可以通过反复接触这个编译器和这些代码来学会？" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "为什么专用模型可能做得很好？" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"东方系列给了我们一个具体的理由，去期待经验积累能发挥作用。大量游戏代码出自同一个作者 " +"ZUN，因此，把反复出现的编程习惯作为初始假设是合理的。代码仓库也提供了实际例子：相似的循环和辅助函数组织方式，曾为其他游戏的重建提供假设。" + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "把编译器也考虑进去，这些源码模式就更加具体了。在固定版本和选项下，源码的小变化可能稳定地产生不同的机器码。其中一些规律藏得很深。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"以 TH095 为例，{ph0}连局部变量的名字，都可能改变 VC7.1 " +"为它安排的存储位置{ph1}。只移动声明并不能解决布局问题，但先前重建的一个背景函数揭示了会影响编译器选择的命名模式。这为{ph2}重建 " +"{ph3}{ph4}提供了更有希望的假设。" + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "把这个模式用于表示文件大小的变量，不增加额外存储就得到了所需布局。随后，整个函数实现了匹配。" + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "这类细节非常具体，通用模型第一次接触时未必知道。而针对这个编译器训练的专用模型，可以接触许多类似案例，学会哪些源码变化值得优先尝试。" + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"当我们已经理解函数行为，只剩最后的匹配差异时，这个机会尤其明显。此时的问题是：这个编译器会把哪一种普通写法编译成眼前的指令？专用训练可能让模型更高效地提出这些写法，而最终由目标程序决定哪些假设成立。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"但这些模式必须保留适用的上下文。TH08 使用的 Visual C++ 版本早于 TH095 和 TH09。TH04 用的是另一种 DOS " +"时代的编译器，其{ph0}玩家逻辑重建{ph1}需要不同的假设，既涉及源码写法，也涉及编译后的代码如何链接成可执行文件。同一作者为我们提供了先验，具体目标则告诉我们它是否适用。区分这些情况，也是专用模型需要学会的事。" + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "这些经验应该存在哪里？" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"在决定通过训练让模型学会这些模式之前，我会先看看代码仓库里已有的知识。在{ph0}上一篇文章{ph1}中，我写过：仓库正在成为项目的工作记忆。这里保存着对编译器的认识，也保存着相关源码和失败实验，让这些经验有据可查。" + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"检索这些记录后，agent 可以从已知结果出发，看看我们是如何得到它的，也可以避开在相同条件下已经失败的方案。模型的参数没有改变，工作的起点却提高了。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"重建 TH09 的敌人生成函数时，{ph0}GPT-6-Luna-max 的一次工作{ph1}从 TH08 " +"对应的源码中找到了假设。之前的写法提示了循环该如何组织，以及内部失败该如何处理。TH09 " +"自身的证据补上了游戏特有的细节，干净的重新构建确认了整个函数的匹配。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}“模型不知道这个编译器特性”和“模型不会利用它”，是两个不同的问题。{ph1} " +"检索可能解决前者，但把知识变成有价值的候选源码，仍然需要能力。如果候选方案失败了，模型还得判断这次结果对下一次尝试意味着什么。知道相关模式，只是这个过程的起点。" + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"所以，在任何对照实验中，我都会加入一个具备良好检索能力的小型通用模型。给它相同的经验库，才能看清训练究竟增加了什么：更好的候选方案、更好的反馈后决策，还是两者都有？" + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "学习选择下一个有价值的实验" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"反馈之后如何决策，是我最想研究的部分。在 agent " +"驱动的重建中，模型提出解释，再用程序来验证。即使候选方案失败了，结果也可能缩小剩余的可能性，提示下一步更值得尝试什么。" + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "再看前面的临时结果例子。最终源码展示了成功的写法，但得到它的过程包含更多信息。下面的每次实验，都帮助我们确定了还有什么需要改变。" + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "实验" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "结果告诉了我们什么" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "尝试另一种复制结果的方式" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "复制代码改变了，但函数仍有差异。" + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "直接使用返回值" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "复制方式与原版一致，但临时存储布局仍有差异。" + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "让临时结果同时保持有效" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "它们得到了各自的存储空间，但其中一个仍与稍后用于碰撞检查的值重叠。" + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "让碰撞检查的值拥有独立存储" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "干净的重新构建确认了整个函数的匹配。" + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "选自前面攻击函数的重建实验，按每次得到的认识整理。" + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"保存这个过程的训练记录，比单独保存最终源码能教会模型更多东西。它把每个改动、观察到的结果和下一步决策联系起来。无效的尝试也值得保留：后来的 agent " +"可以据此知道，在这个上下文中，哪些假设已经检验过。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}我希望模型学会如何利用反馈。{ph1} " +"结果没有变化，可能意味着该重新考虑这个改动；差异发生变化，则可能指向另一个原因。当局部修改不再提供有效信息时，agent " +"应该可以重新考虑源码的整体表达，或者回头检查证据。" + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"学会这些选择，需要自主性。有时，更有价值的下一步是检查另一个调用者，而不是再修改一次源码并编译。如果强制按照固定顺序改写，就会剥夺这种选择，也失去一部分我们希望模型学会的行为。" + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "为什么逆向工程值得用来探索强化学习" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"范围明确的重建任务，为研究这些选择提供了一个实际环境。目标和编译器版本固定之后，许多源码实验都能快速给出可重复的反馈。agent " +"可以编译候选源码、检查差异，再根据观察选择下一步。" + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"强化学习可以利用这些反馈，训练模型的决策过程。候选源码通过明确定义的检查，就给出了一个具体成果；而尝试的过程，让我们可以看到实现它花了多少功夫。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"已有研究支持这个方向。{ph0}LLM4Decompile{ph1}研究专门用于反编译的模型。{ph2}AutoDecompiler{ph3}利用编译和执行反馈，训练多轮改进能力。这些研究关注的是功能还原。对于我们的工作，我会以它们为起点，研究一个相关任务：找到能复现历史编译器输出的源码。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"不过，要得到这样一个定义明确的任务，有时得先确定函数负责哪些资源，或者设计新的运行时测量。在 TH08 " +"中，{ph0}一次碰撞差异最终追溯到原版处理浮点计算的方式{ph1}。agent 必须先弄清楚该测量什么，才能验证有价值的修正。" + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "这提示了能力全面的模型的另一个作用：先把问题变得可验证。验证手段建立起来后，我们才可以研究，学出来的搜索策略能走多远。" + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "当验证器出错时" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"一旦用某项检查来训练模型，它的覆盖范围就会成为学习环境的一部分。我把这种参考标准或验证器称为 " +"oracle。即使它能给出可重复的答案，我们也必须知道，这个答案覆盖了什么。" + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 " +"展示了检查范围之外可能藏着多少问题。一些重建函数使用了错误的浮点常量，却仍然通过了精确比较。比较器会先调整重定位地址，再检查指令；但只有明确登记过的常量，才会进一步检查地址指向的值。其他常量引用，即使值没被检查，也可能通过。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"为了补上这个漏洞，{ph0}一次更全面的审计{ph1}在已经通过检查的函数中找到了错误常量。有一个计算中，重建源码加的是 {ph2}，而原版加的是 " +"{ph3}。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}如果用旧检查的结果作为奖励，模型可能学会生成错误的重建源码。{ph1} 我们没有用这个验证器训练 RL " +"模型；实际观察到的是检查本身的漏洞。但如果把它用作奖励，同一个盲点也会进入学习环境。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"功能测试也可能存在类似盲点。{ph0}针对 LLM " +"反编译器的差分模糊测试研究{ph1}发现，在已有测试之外，程序行为仍可能不同。因此，理解“测试通过”时，必须看它实际覆盖了哪些情况；理解“指令匹配”时，也必须看它检查了哪些字节和数据。" + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"在把任何一种检查用作奖励之前，我都会用刻意构造的错误来测试它。我还会把最终验收机制放在候选源码可修改的范围之外，让模型通过正确重建来满足检查。否则，直接复制目标程序的字节数组，也可能骗过简单的评分，却没有还原出有用的源码。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "{ph0}改进模型和改进它的验证器，属于同一个研究问题。{ph1} 模型越擅长优化我们给它的反馈，我们就越需要理解反馈本身。" + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "值得做的对照实验" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"具备这些检查之后，我会从一组固定的 VC7.1 " +"任务开始比较。先为强通用模型和小型通用模型建立基线，再给小模型加入仓库知识，观察检索带来的变化。接着比较监督训练的专用模型，以及在此基础上进行 RL " +"训练的版本，看看训练增加了什么。" + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"为了让比较有意义，每个模型都应获得相同的目标证据和工具。专门比较训练效果时，还要固定基础模型和可用知识。重复实验有助于区分可靠的搜索策略与偶然成功的候选方案。" + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"我还会区分两种起点：从指令开始重建函数，以及在已有源码中消除最后几处差异。专用模型可能很擅长后者，而通用模型承担了到达这个阶段的大部分工作。分别评估，才能看清这种分工。" + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"两类任务都需要真正未见过的测试案例。东方系列的相关游戏会共享引擎代码，只留出某个游戏，仍可能让几乎相同的函数进入训练集。我会改为按相关源码家族留出测试集，并把它们通过验证的答案同时排除在训练数据和检索库之外。" + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"在这些案例上，要统计的是完整、独立验证过的匹配成果。给每个模型相同的候选数量或编译次数预算，可以比较搜索质量。记录总耗时和总成本，包括编译，才能知道这种质量是否转化成了高效的工作流。训练成本也要单独计算。" + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"两种角度都重要。一个便宜的提议，如果导致大量无效编译，整个工作流仍可能很贵。反过来，较慢的模型如果更早找到有价值的实验，也可能更划算。对照实验应当反映这些成本。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"只统计通过验收的结果，也能帮助我们合理看待中间进展。TH09 " +"的一个{ph0}录像保存候选方案{ph1}看起来很有希望，因为编译后函数的大小符合预期，但完整比较仍发现大量差异。编译器改变指令排列，也可能让候选源码看起来更接近，却没有解决根本问题。这些诊断可以指导下一步，但若要奖励“变得更接近”，我会先对照完整验收检查，确认这种奖励有效。" + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"我想看这些结果呈现出什么规律。如果专用模型在熟悉的编译器上表现很好，换了工具链后，这种优势还在吗？当双方都拥有良好的检索能力时，训练还能增加多少价值？这些答案能帮助我们决定，每类模型最适合用在哪里。" + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "这意味着怎样的 agent？" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"如果不同模型在不同阶段各有所长，一种有价值的安排可能是：让能力全面的 agent " +"可以调用专用模型。当任务变成一种熟悉的编译器搜索时，它可以请专用模型提出候选方案。如果结果开始说不通，通用 agent " +"可以回头检查目标证据，或者调查测量本身。" + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"在这样的安排中，专用模型成为 agent 随着调查进展可以选择的工具。agent 仍然拥有改变方向的自由，而这正是前面那些突破发生的重要条件。" + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"这自然延续了上一篇文章中谈到的工业化转变。能够开展更多实验，会改变我们组织工作的方式，也让我们有理由追问：哪些决策需要全面的判断，哪些可以通过反复积累某项具体技艺来改善？" + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"目前，最强的通用模型解决了我过去没能解决的问题，因此在我的工作流中占据了一席之地。正是这些成功，让我想更仔细地研究过程：有多少依赖模型能力的广度？又有多少可以交给专门学习这类重建的模型？" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "{ph0}这些代码仓库让我们有机会验证这个问题。{ph1} 我想利用这些记录建立对照实验，看看我们能学到什么。" + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "证据与模型归属" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"这些例子来自链接提交中记录的历史函数级比较。整款游戏的运行时检查属于另一个里程碑。相关源码与匹配记录核查于 2026 年 10 月 13 日。" + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"TH09 中的标签 {ph0} 对应 GPT-6-Astra。{ph1} " +"会随当时使用的模型变化，因此仅凭这个标签无法确定某次运行的具体模型。其他引用提交中的明确模型名，提供了对应的归属记录；这些历史记录并没有统一的单次运行成本或投入统计。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "{ph0}TH09 临时值生命周期：{ph1}{ph2}早期上下文检查{ph3}与{ph4}完整的干净重建{ph5}。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}TH09 Front 回调：{ph1}{ph2}未匹配的诊断区域{ph3}、{ph4}Astra " +"提出的新控制流图{ph5}与{ph6}通过验收的重新编译检查{ph7}。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}编译器知识：{ph1}{ph2}TH095 标识符分桶规律的复用{ph3}、{ph4}TH04 " +"玩家源码与链接顺序{ph5}、{ph6}Luna 完成的 TH09 敌人生成函数匹配{ph7}。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}检查与诊断：{ph1}{ph2}TH08 浮点常量审计{ph3}、{ph4}TH08 数值问题调查{ph5}、{ph6}TH09 " +"录像保存重建{ph7}。" + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "文中提出的专用模型与 RL 对照，属于未来实验。项目历史为这些想法提供了依据；本文尚未评估经过训练的专用模型。" + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "逆向工程需要更大的模型吗？ | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "东方重建让我如何思考通用模型、专用模型，以及怎样从编译器反馈中学会提出更好的假设。" + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "在本页" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "模型与逆向工程随笔" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "重建实验选例" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "回到顶部" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "作者 N0zoM1z0 · 2026 年 10 月 13 日 · 待审草稿" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/locales/zh-TW/messages.po b/website/locales/zh-TW/messages.po index 6c2790888..9dfb93e81 100644 --- a/website/locales/zh-TW/messages.po +++ b/website/locales/zh-TW/messages.po @@ -32,37 +32,38 @@ msgid "Thoughts on reverse engineering." msgstr "逆向工程的思考。" #. {ph0}:
    -#: website/public/blog/index.html:216 +#: website/public/blog/index.html:214 #, python-brace-format msgctxt "page:blog/index.html" msgid "Game reconstruction{ph0}Essay" msgstr "遊戲重建{ph0}論文" -#: website/public/blog/index.html:218 +#: website/public/blog/index.html:216 msgctxt "page:blog/index.html" msgid "The industrial revolution in reverse engineering" msgstr "逆向工程的工業革命" -#: website/public/blog/index.html:219 +#: website/public/blog/index.html:217 msgctxt "page:blog/index.html" msgid "" "What manual RE and reconstructing TH08, TH095 and TH04 taught me about " "agents, verification and knowledge that compounds across projects." msgstr "什麼手動重新和重建TH08，TH095和TH04教我關於程式設計助手，驗證和知識，跨專案的化合物。" -#: website/public/blog/index.html:207 website/public/blog/index.html:224 +#: website/public/blog/index.html:207 website/public/blog/index.html:222 msgctxt "page:blog/index.html" msgid "Read the article" msgstr "閱讀文章" -#: website/public/blog/index.html:227 +#: website/public/blog/index.html:225 msgctxt "page:blog/index.html" msgid "By N0zoM1z0 · 10 October 2026" msgstr "透過N0zoM1z0*10十月2026" #. {ph0}: -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 +#: website/public/blog/reverse-engineering-models/index.html:876 #: website/public/blog/touhou-reconstruction/index.html:1080 #: website/public/examples/dino-lab/index.html:475 #: website/public/faq/index.html:573 @@ -96,6 +97,8 @@ msgid "" msgstr "對逆向工程的思考，借鑑與 REA 和AI程式設計助手。" #: website/public/blog/index.html:75 website/public/blog/index.html:88 +#: website/public/blog/reverse-engineering-models/index.html:75 +#: website/public/blog/reverse-engineering-models/index.html:88 #: website/public/blog/touhou-reconstruction/index.html:75 #: website/public/blog/touhou-reconstruction/index.html:88 #: website/public/examples/dino-lab/index.html:75 @@ -130,6 +133,7 @@ msgid "" msgstr "REA-瞭解軟體的工作原理。 逆向工程與你的程式設計助手." #: website/public/blog/index.html:108 +#: website/public/blog/reverse-engineering-models/index.html:108 #: website/public/blog/touhou-reconstruction/index.html:108 #: website/public/examples/dino-lab/index.html:109 #: website/public/faq/index.html:108 @@ -150,6 +154,7 @@ msgid "REA home" msgstr "REA 首頁" #: website/public/blog/index.html:112 +#: website/public/blog/reverse-engineering-models/index.html:112 #: website/public/blog/touhou-reconstruction/index.html:112 #: website/public/examples/dino-lab/index.html:113 #: website/public/faq/index.html:112 @@ -170,6 +175,7 @@ msgid "Main navigation" msgstr "主導航" #: website/public/blog/index.html:130 +#: website/public/blog/reverse-engineering-models/index.html:130 #: website/public/blog/touhou-reconstruction/index.html:130 #: website/public/examples/dino-lab/index.html:131 #: website/public/faq/index.html:130 @@ -190,6 +196,7 @@ msgid "REA on GitHub" msgstr "在 GitHub 檢視 REA" #: website/public/blog/index.html:150 +#: website/public/blog/reverse-engineering-models/index.html:150 #: website/public/blog/touhou-reconstruction/index.html:150 #: website/public/examples/dino-lab/index.html:151 #: website/public/faq/index.html:150 @@ -210,6 +217,7 @@ msgid "Join REA on Discord" msgstr "加入 REA 的 Discord 社群" #: website/public/blog/index.html:169 +#: website/public/blog/reverse-engineering-models/index.html:169 #: website/public/blog/touhou-reconstruction/index.html:169 #: website/public/examples/dino-lab/index.html:170 #: website/public/faq/index.html:169 @@ -229,7 +237,7 @@ msgctxt "shared" msgid "Follow REA on X" msgstr "在 X 上關注 REA" -#: website/public/blog/index.html:246 +#: website/public/blog/index.html:244 msgctxt "page:blog/index.html" msgid "Back to top" msgstr "回到頂部" @@ -10634,6 +10642,7 @@ msgid "Same spacing: 256 ÷ 16 = 16 angle units = 22.5°" msgstr "相同間距：256÷16=16角度單位=22.5°" #: website/public/blog/index.html:114 +#: website/public/blog/reverse-engineering-models/index.html:114 #: website/public/blog/touhou-reconstruction/index.html:114 #: website/public/examples/dino-lab/index.html:115 #: website/public/faq/index.html:114 @@ -10654,6 +10663,7 @@ msgid "Showcases" msgstr "案例展示" #: website/public/blog/index.html:115 +#: website/public/blog/reverse-engineering-models/index.html:115 #: website/public/blog/touhou-reconstruction/index.html:115 #: website/public/examples/dino-lab/index.html:116 #: website/public/faq/index.html:115 @@ -10674,6 +10684,7 @@ msgid "Blog" msgstr "部落格" #: website/public/blog/index.html:113 +#: website/public/blog/reverse-engineering-models/index.html:113 #: website/public/blog/touhou-reconstruction/index.html:113 #: website/public/examples/dino-lab/index.html:114 #: website/public/faq/index.html:113 @@ -10694,6 +10705,7 @@ msgid "Get started" msgstr "開始使用" #: website/public/blog/index.html:116 +#: website/public/blog/reverse-engineering-models/index.html:116 #: website/public/blog/touhou-reconstruction/index.html:116 #: website/public/examples/dino-lab/index.html:117 #: website/public/faq/index.html:116 @@ -10713,7 +10725,8 @@ msgctxt "shared" msgid "Guides" msgstr "指南" -#: website/public/blog/index.html:238 +#: website/public/blog/index.html:236 +#: website/public/blog/reverse-engineering-models/index.html:868 #: website/public/blog/touhou-reconstruction/index.html:1072 #: website/public/examples/dino-lab/index.html:467 #: website/public/faq/index.html:565 @@ -10733,7 +10746,8 @@ msgctxt "shared" msgid "Source" msgstr "原始碼" -#: website/public/blog/index.html:239 +#: website/public/blog/index.html:237 +#: website/public/blog/reverse-engineering-models/index.html:869 #: website/public/blog/touhou-reconstruction/index.html:1073 #: website/public/examples/dino-lab/index.html:468 #: website/public/faq/index.html:566 @@ -10753,7 +10767,8 @@ msgctxt "shared" msgid "MIT license" msgstr "MIT 許可證" -#: website/public/blog/index.html:240 +#: website/public/blog/index.html:238 +#: website/public/blog/reverse-engineering-models/index.html:870 #: website/public/blog/touhou-reconstruction/index.html:1074 #: website/public/examples/dino-lab/index.html:469 #: website/public/faq/index.html:567 @@ -10773,7 +10788,8 @@ msgctxt "shared" msgid "Join the community" msgstr "加入社群" -#: website/public/blog/index.html:241 +#: website/public/blog/index.html:239 +#: website/public/blog/reverse-engineering-models/index.html:871 #: website/public/blog/touhou-reconstruction/index.html:1075 #: website/public/examples/dino-lab/index.html:470 #: website/public/faq/index.html:568 @@ -11722,8 +11738,1112 @@ msgstr "從 Touhou 重建的經驗，探討通用模型、專用模型，以及 #: website/public/blog/index.html:210 msgctxt "page:blog/index.html" -msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" -msgstr "作者 N0zoM1z0 · 2026 年 10 月 13 日 · 待審草稿" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "作者 N0zoM1z0 · 2026 年 10 月 13 日" + +#: website/public/blog/reverse-engineering-models/index.html:103 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Skip to content" +msgstr "跳轉到內容" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:194 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Home{ph1} / {ph2}Blog{ph3} / Models and RE" +msgstr "{ph0}首頁{ph1} / {ph2}部落格{ph3} / 模型與逆向工程" + +#: website/public/blog/reverse-engineering-models/index.html:197 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Essay" +msgstr "隨筆" + +# Reviewed for technical meaning and readability; preserve the distinction +# between historical evidence and proposed experiments. +#: website/public/blog/reverse-engineering-models/index.html:198 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model?" +msgstr "逆向工程需要更大的模型嗎？" + +#: website/public/blog/reverse-engineering-models/index.html:199 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "By N0zoM1z0 · 13 October 2026" +msgstr "作者 N0zoM1z0 · 2026 年 10 月 13 日" + +#: website/public/blog/reverse-engineering-models/index.html:201 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Takeaway" +msgstr "要點" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:203 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Model capability changes the hypotheses an agent can find.{ph1} Later " +"sessions have solved problems that earlier work left unresolved." +msgstr "{ph0}模型能力會影響 agent 能提出哪些假設。{ph1} 後來的工作解決了早期嘗試未能解決的問題。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:211 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A familiar compiler gives specialization something concrete to " +"learn.{ph1} Some exact matches depend on narrow, reusable source patterns." +msgstr "{ph0}熟悉的編譯器為專用模型提供了具體的學習對象。{ph1} 有些精確匹配依賴細小但可以複用的原始碼模式。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:218 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Give repository knowledge a fair comparison.{ph1} A smaller model with " +"good retrieval may already know enough to make progress." +msgstr "{ph0}也要公平評估程式碼倉庫裡的知識。{ph1} 較小的模型配合良好的檢索，可能已經掌握了足以推進工作的知識。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:223 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Train the next useful experiment, and test its oracle.{ph1} The value " +"of feedback depends on what the checker actually verifies." +msgstr "{ph0}訓練模型選擇下一個有價值的實驗，同時檢驗它使用的驗證器。{ph1} 回饋是否有用，取決於驗證器究竟檢查了什麼。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:230 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Measure accepted reconstruction against time and cost.{ph1} That is the" +" result I want from the whole agent." +msgstr "{ph0}結合時間和成本，評估透過驗證的重建成果。{ph1} 這是我希望從整個 agent 工作流程中獲得的結果。" + +#: website/public/blog/reverse-engineering-models/index.html:241 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "In this article" +msgstr "在本文中" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:243 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What better models changed{ph1}" +msgstr "{ph0}更強的模型改變了什麼{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:244 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Why specialize?{ph1}" +msgstr "{ph0}為什麼考慮專用模型？{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:245 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Where experience lives{ph1}" +msgstr "{ph0}這些經驗應該存在哪裡？{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:246 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}Learning the next experiment{ph1}" +msgstr "{ph0}學習選擇下一個有價值的實驗{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:247 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}RE and reinforcement learning{ph1}" +msgstr "{ph0}逆向工程與強化學習{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:248 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}When the oracle is wrong{ph1}" +msgstr "{ph0}當驗證器出錯時{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:249 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}A comparison worth running{ph1}" +msgstr "{ph0}值得做的對照實驗{ph1}" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:250 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "{ph0}What kind of agent?{ph1}" +msgstr "{ph0}這意味著怎樣的 agent？{ph1}" + +#: website/public/blog/reverse-engineering-models/index.html:258 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What better models changed" +msgstr "更強的模型改變了什麼" + +#: website/public/blog/reverse-engineering-models/index.html:259 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Since August 2026, I have been exploring how much reverse engineering an " +"agent can carry out with access to a program and a way to check its work. " +"Most of my experience comes from reconstructing games, especially Touhou. As" +" I have changed models, the repositories have kept a record of what each " +"session managed to recover." +msgstr "" +"從 2026 年 8 月開始，我一直在探索：如果給 agent " +"訪問程式的能力，以及驗證自身工作的手段，它能獨立完成多少逆向工作？我的經驗主要來自遊戲重建，尤其是東方系列。隨著我更換模型，程式碼倉庫記錄了每次工作還原出的內容。" + +#: website/public/blog/reverse-engineering-models/index.html:267 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of that work aims at exact reconstruction: we compile the maintained " +"source under a pinned historical toolchain and check whether it reproduces " +"the target function. When a later model succeeds where earlier work stalled," +" the source changes show what made that progress possible." +msgstr "" +"這些工作很大一部分追求精確重建：用固定版本的歷史工具鏈編譯我們維護的原始碼，檢查它能否復現目標函式。當後來的模型解決了早期工作卡住的問題，我們可以從原始碼變化中看出，是什麼促成了這次進展。" + +#: website/public/blog/reverse-engineering-models/index.html:274 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The temporary values had to stay alive together" +msgstr "臨時值需要同時保持有效" + +#: website/public/blog/reverse-engineering-models/index.html:275 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Much of a TH09 attack function's logic had already been reconstructed, but " +"the compiler still stored temporary results differently from the original. " +"The remaining problem was how long those results needed to stay alive and " +"when their storage could be reused." +msgstr "" +"TH09 " +"中一個攻擊函式的大部分邏輯已經還原，但編譯器對臨時結果的儲存安排仍然與原版不同。剩下的問題在於：這些結果的生命週期應該多長，編譯器何時可以複用它們的儲存空間？" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:282 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A {ph0}GPT-6-Sol checkpoint{ph1} recorded this mismatch after testing " +"several ordinary source arrangements. When {ph2}GPT-6.1-Sol returned to the " +"problem{ph3}, it found a useful hypothesis about how long those temporary " +"values needed to stay alive." +msgstr "" +"{ph0}GPT-6-Sol 的一次工作記錄{ph1}表明，嘗試幾種常見的原始碼寫法後，差異仍未消除。後來，{ph2}GPT-6.1-Sol " +"重新研究這個問題{ph3}，提出了一個有價值的假設：關鍵可能在於這些臨時值的生命週期。" + +#: website/public/blog/reverse-engineering-models/index.html:296 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The function produced temporary results and copied them into animation " +"positions. In C++, a temporary result stays alive until the expression using" +" it finishes. Grouping the real operations into one expression kept those " +"results alive together, so the compiler could no longer reuse the same " +"storage for all of them." +msgstr "" +"這個函式會產生臨時結果，再把它們複製到動畫的位置資料中。在 C++ " +"中，臨時結果會一直存在，直到使用它的表示式執行完畢。把實際操作放進同一個表示式，就能讓這些結果同時存活，編譯器也就不能為它們全部複用同一塊儲存空間。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:303 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"One result still shared storage with a value used later for collision " +"checks. Giving that value its own lifetime resolved the remaining overlap. " +"After several targeted experiments, {ph0}a clean rebuild confirmed that the " +"full function matched{ph1}." +msgstr "" +"其中一個結果仍然與稍後用於碰撞檢查的值共享儲存。給那個值單獨的生命週期後，最後一處重疊也消失了。經過幾次有針對性的實驗，{ph0}一次乾淨的重新構建確認，整個函式都已匹配{ph1}。" + +#: website/public/blog/reverse-engineering-models/index.html:312 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What interests me is how the discovery and the tests worked together. The " +"lifetime hypothesis gave the agent a new direction; compiler feedback helped" +" it refine that hypothesis into a source arrangement that matched." +msgstr "" +"我感興趣的是，發現與驗證如何共同推進工作。生命週期的假設給 agent 指明瞭新方向；編譯器的回饋又幫助它逐步調整原始碼，直到找到匹配的寫法。" + +#: website/public/blog/reverse-engineering-models/index.html:318 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A different control-flow graph" +msgstr "換一種控制流結構" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:319 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A second TH09 function shows why that new direction can matter. Here, the " +"obstacle was how several branches shared their final work. " +"{ph0}GPT-6.1-Sol's earlier work{ph1} got parts of the function to match, but" +" the branches still joined up differently from the original." +msgstr "" +"TH09 的另一個函式說明了，新方向為什麼重要。這次的難點是多個分支如何共享末尾的操作。{ph0}GPT-6.1-Sol " +"之前的工作{ph1}已經讓函式的一些部分匹配，但分支匯合的方式仍與原版不同。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:330 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}GPT-6-Astra changed the surrounding control-flow graph{ph1}. The main " +"paths now led to a shared final update and return. In that source structure," +" the compiler combined the branches in the same way as the original. The " +"complete function matched, and a {ph2}separate rebuild check confirmed " +"it{ph3}." +msgstr "" +"{ph0}GPT-6-Astra " +"改寫了外圍的控制流圖{ph1}，讓主要路徑匯入共同的最終更新與返回。在這種原始碼結構下，編譯器合併分支的方式與原版一致。整個函式實現了匹配，{ph2}另一次獨立的重新構建也確認了這一結果{ph3}。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:342 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Changing the representation opened a more productive search space.{ph1}" +" Both cases give me a reason to take general model capability seriously: an " +"agent can keep trying local variations for a long time before finding a " +"better way to express the surrounding work." +msgstr "" +"{ph0}換一種表達方式，開啟了更有價值的搜尋空間。{ph1} 這兩個例子讓我認真看待通用模型的能力：agent " +"可能花很長時間嘗試局部變化，才找到一種更好的整體寫法。" + +#: website/public/blog/reverse-engineering-models/index.html:351 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"At the same time, the later sessions started with earlier source and " +"recorded failures. That accumulated work was part of their advantage. The " +"history lets us inspect the discoveries and their checks; a controlled " +"comparison would help separate the effect of the model from the experience " +"it inherited." +msgstr "" +"不過，後來的工作也繼承了之前的原始碼和失敗記錄。這些積累本身就是優勢的一部分。歷史記錄讓我們可以檢查發現及其驗證過程；要區分模型能力和繼承經驗各自帶來的幫助，還需要對照實驗。" + +#: website/public/blog/reverse-engineering-models/index.html:358 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is where the question becomes interesting to me. Some of the progress " +"came from finding a new representation, while much of the remaining work " +"involved testing source forms against a familiar compiler. How much needs " +"broad reasoning ability, and how much could a model learn through repeated " +"experience with that compiler and code?" +msgstr "" +"這正是我覺得有趣的地方。有些進展來自新的表達方式，剩下的大量工作則是在熟悉的編譯器上試驗不同寫法。其中有多少需要廣泛的推理能力？又有多少可以透過反覆接觸這個編譯器和這些程式碼來學會？" + +#: website/public/blog/reverse-engineering-models/index.html:368 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why might a specialist do well?" +msgstr "為什麼專用模型可能做得很好？" + +#: website/public/blog/reverse-engineering-models/index.html:369 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Touhou gives us a concrete reason to expect that experience to help. Much of" +" the game code comes from the same author, ZUN, so recurring habits are a " +"reasonable starting assumption. The repositories back that up with useful " +"examples: related loops and helper arrangements have supplied hypotheses " +"across games." +msgstr "" +"東方系列給了我們一個具體的理由，去期待經驗積累能發揮作用。大量遊戲程式碼出自同一個作者 " +"ZUN，因此，把反覆出現的程式設計習慣作為初始假設是合理的。程式碼倉庫也提供了實際例子：相似的迴圈和輔助函式組織方式，曾為其他遊戲的重建提供假設。" + +#: website/public/blog/reverse-engineering-models/index.html:376 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those source patterns become more specific when we include the compiler. " +"Under a particular version and set of options, small changes in source can " +"produce repeatable differences in machine code. Some of those differences " +"are surprisingly obscure." +msgstr "把編譯器也考慮進去，這些原始碼模式就更加具體了。在固定版本和選項下，原始碼的小變化可能穩定地產生不同的機器碼。其中一些規律藏得很深。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: Supervisor::LoadDat +#. {ph4}: +#: website/public/blog/reverse-engineering-models/index.html:382 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In TH095, for example, {ph0}even the name used for a local variable could " +"change where VC7.1 stored it{ph1}. Moving the declaration alone did not " +"solve the layout, but an earlier background function had revealed a naming " +"pattern that affected the compiler's choice. That gave us a more promising " +"hypothesis for {ph2}the reconstruction of {ph3}{ph4}." +msgstr "" +"以 TH095 為例，{ph0}連局部變數的名字，都可能改變 VC7.1 " +"為它安排的儲存位置{ph1}。只移動宣告並不能解決佈局問題，但先前重建的一個背景函式揭示了會影響編譯器選擇的命名模式。這為{ph2}重建 " +"{ph3}{ph4}提供了更有希望的假設。" + +#: website/public/blog/reverse-engineering-models/index.html:396 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reusing that pattern for the file-size variable produced the required layout" +" without adding storage. The full function then matched." +msgstr "把這個模式用於表示檔案大小的變數，不增加額外儲存就得到了所需佈局。隨後，整個函式實現了匹配。" + +#: website/public/blog/reverse-engineering-models/index.html:401 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This is the sort of narrow detail a general model may not know when it " +"arrives. A specialist trained on this compiler could encounter many such " +"cases and learn which source variations are worth testing first." +msgstr "這類細節非常具體，通用模型第一次接觸時未必知道。而針對這個編譯器訓練的專用模型，可以接觸許多類似案例，學會哪些原始碼變化值得優先嚐試。" + +#: website/public/blog/reverse-engineering-models/index.html:407 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The opportunity is especially clear in final matching problems where we " +"already understand the behavior. We are then asking which ordinary source " +"form this compiler turns into the observed instructions. Specialization " +"could help the model propose those forms more efficiently, while the target " +"still determines which hypotheses survive." +msgstr "" +"當我們已經理解函式行為，只剩最後的匹配差異時，這個機會尤其明顯。此時的問題是：這個編譯器會把哪一種普通寫法編譯成眼前的指令？專用訓練可能讓模型更高效地提出這些寫法，而最終由目標程式決定哪些假設成立。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:415 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The useful patterns would have to retain their context, though. TH08 uses an" +" earlier Visual C++ version than TH095 and TH09. TH04 uses a different DOS-" +"era compiler, and its {ph0}player reconstruction{ph1} needed different " +"hypotheses about source patterns and how the compiled code was combined into" +" an executable. Shared authorship gives us a prior, but each target tells us" +" whether it applies. Distinguishing those contexts is part of what a " +"specialist would need to learn." +msgstr "" +"但這些模式必須保留適用的上下文。TH08 使用的 Visual C++ 版本早於 TH095 和 TH09。TH04 用的是另一種 DOS " +"時代的編譯器，其{ph0}玩家邏輯重建{ph1}需要不同的假設，既涉及原始碼寫法，也涉及編譯後的程式碼如何連結成可執行檔案。同一作者為我們提供了先驗，具體目標則告訴我們它是否適用。區分這些情況，也是專用模型需要學會的事。" + +#: website/public/blog/reverse-engineering-models/index.html:431 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Where should that experience live?" +msgstr "這些經驗應該存在哪裡？" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:432 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before deciding to teach those patterns through model training, I would look" +" at the knowledge we already keep in the repository. In the {ph0}previous " +"essay{ph1}, I wrote about it becoming the project's working memory. That " +"memory includes compiler lessons, along with the source and failed " +"experiments that made them useful." +msgstr "" +"在決定透過訓練讓模型學會這些模式之前，我會先看看程式碼倉庫裡已有的知識。在{ph0}上一篇文章{ph1}中，我寫過：倉庫正在成為專案的工作記憶。這裡儲存著對編譯器的認識，也儲存著相關原始碼和失敗實驗，讓這些經驗有據可查。" + +#: website/public/blog/reverse-engineering-models/index.html:441 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"By retrieving that record, an agent can start with a known result and " +"inspect how we reached it. It can also avoid alternatives that have already " +"failed in the same context. The starting point improves without changing the" +" model's weights." +msgstr "" +"檢索這些記錄後，agent 可以從已知結果出發，看看我們是如何得到它的，也可以避開在相同條件下已經失敗的方案。模型的參數沒有改變，工作的起點卻提高了。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:447 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"When reconstructing TH09's enemy-spawn function, a {ph0}GPT-6-Luna-max " +"session{ph1} drew a hypothesis from TH08's corresponding source family. That" +" earlier source suggested how to structure the loop and handle failures " +"within it. TH09's own evidence supplied the game-specific details, and clean" +" rebuilds confirmed the complete match." +msgstr "" +"重建 TH09 的敵人生成函式時，{ph0}GPT-6-Luna-max 的一次工作{ph1}從 TH08 " +"對應的原始碼中找到了假設。之前的寫法提示了迴圈該如何組織，以及內部失敗該如何處理。TH09 " +"自身的證據補上了遊戲特有的細節，乾淨的重新構建確認了整個函式的匹配。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:458 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}\"The model did not know that compiler quirk\" and \"the model could " +"not use it\" are different problems.{ph1} Retrieval may solve the first, but" +" turning that fact into a useful candidate still takes work. And if the " +"candidate fails, the model must decide what the result means for its next " +"attempt. Knowing the relevant pattern is only the beginning of that process." +msgstr "" +"{ph0}“模型不知道這個編譯器特性”和“模型不會利用它”，是兩個不同的問題。{ph1} " +"檢索可能解決前者，但把知識變成有價值的候選原始碼，仍然需要能力。如果候選方案失敗了，模型還得判斷這次結果對下一次嘗試意味著什麼。知道相關模式，只是這個過程的起點。" + +#: website/public/blog/reverse-engineering-models/index.html:468 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For that reason, I would include a smaller general model with good retrieval" +" in any comparison. Giving it the same library of experience would help us " +"identify what training adds: better candidate generation, better decisions " +"after feedback, or both." +msgstr "" +"所以，在任何對照實驗中，我都會加入一個具備良好檢索能力的小型通用模型。給它相同的經驗庫，才能看清訓練究竟增加了什麼：更好的候選方案、更好的回饋後決策，還是兩者都有？" + +#: website/public/blog/reverse-engineering-models/index.html:476 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Learning the next useful experiment" +msgstr "學習選擇下一個有價值的實驗" + +#: website/public/blog/reverse-engineering-models/index.html:477 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Those decisions after feedback are what I would most like to study. In " +"agentic reconstruction, the model proposes an explanation and tests it " +"against the program. Even when the candidate fails, the result can narrow " +"the remaining possibilities and suggest a more useful next experiment." +msgstr "" +"回饋之後如何決策，是我最想研究的部分。在 agent " +"驅動的重建中，模型提出解釋，再用程式來驗證。即使候選方案失敗了，結果也可能縮小剩餘的可能性，提示下一步更值得嘗試什麼。" + +#: website/public/blog/reverse-engineering-models/index.html:484 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Consider the temporary-results example again. Its final source shows the " +"arrangement that worked, but the route to that source contains more " +"information. Each of the following experiments helped identify what still " +"needed to change." +msgstr "再看前面的臨時結果例子。最終原始碼展示了成功的寫法，但得到它的過程包含更多資訊。下面的每次實驗，都幫助我們確定了還有什麼需要改變。" + +#: website/public/blog/reverse-engineering-models/index.html:497 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Experiment" +msgstr "實驗" + +#: website/public/blog/reverse-engineering-models/index.html:498 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What its result taught us" +msgstr "結果告訴了我們什麼" + +#: website/public/blog/reverse-engineering-models/index.html:503 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Try a different way to copy the results" +msgstr "嘗試另一種複製結果的方式" + +#: website/public/blog/reverse-engineering-models/index.html:504 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "The copy code changes, but the function still differs." +msgstr "複製程式碼改變了，但函式仍有差異。" + +#: website/public/blog/reverse-engineering-models/index.html:509 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Use the returned values directly" +msgstr "直接使用回傳值" + +#: website/public/blog/reverse-engineering-models/index.html:510 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Copying follows the original pattern, but temporary storage still differs." +msgstr "複製方式與原版一致，但臨時儲存佈局仍有差異。" + +#: website/public/blog/reverse-engineering-models/index.html:516 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the temporary results alive together" +msgstr "讓臨時結果同時保持有效" + +#: website/public/blog/reverse-engineering-models/index.html:517 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"They get separate storage, but one still overlaps a value used later for " +"collision checks." +msgstr "它們得到了各自的儲存空間，但其中一個仍與稍後用於碰撞檢查的值重疊。" + +#: website/public/blog/reverse-engineering-models/index.html:523 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Keep the collision-check value separate" +msgstr "讓碰撞檢查的值擁有獨立儲存" + +#: website/public/blog/reverse-engineering-models/index.html:524 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A clean rebuild confirms the complete match." +msgstr "乾淨的重新構建確認了整個函式的匹配。" + +#: website/public/blog/reverse-engineering-models/index.html:529 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Selected experiments from the attack-function reconstruction above, grouped " +"by what they taught us." +msgstr "選自前面攻擊函式的重建實驗，按每次得到的認識整理。" + +#: website/public/blog/reverse-engineering-models/index.html:533 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A training record that keeps this sequence can teach more than the final " +"source alone. It connects each proposed change to an observed result and the" +" next decision. Keeping ineffective variations matters too: they tell a " +"later agent which hypotheses have already been tested in that context." +msgstr "" +"儲存這個過程的訓練記錄，比單獨儲存最終原始碼能教會模型更多東西。它把每個改動、觀察到的結果和下一步決策聯絡起來。無效的嘗試也值得保留：後來的 agent" +" 可以據此知道，在這個上下文中，哪些假設已經檢驗過。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:540 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}I would want the model to learn how to use feedback.{ph1} A neutral " +"result is a reason to reconsider the variation; a changed mismatch may point" +" to another cause. When local edits stop being informative, the agent should" +" be able to revisit the source representation or inspect the evidence again." +msgstr "" +"{ph0}我希望模型學會如何利用回饋。{ph1} " +"結果沒有變化，可能意味著該重新考慮這個改動；差異發生變化，則可能指向另一個原因。當局部修改不再提供有效資訊時，agent " +"應該可以重新考慮原始碼的整體表達，或者回頭檢查證據。" + +#: website/public/blog/reverse-engineering-models/index.html:549 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Learning to make those choices requires autonomy. Sometimes the useful next " +"step is inspecting another caller rather than compiling another edit. A " +"rigid sequence of required rewrites would remove that choice, along with " +"part of the behavior we want the model to learn." +msgstr "" +"學會這些選擇，需要自主性。有時，更有價值的下一步是檢查另一個呼叫者，而不是再修改一次原始碼並編譯。如果強制按照固定順序改寫，就會剝奪這種選擇，也失去一部分我們希望模型學會的行為。" + +#: website/public/blog/reverse-engineering-models/index.html:558 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Why RE is interesting for reinforcement learning" +msgstr "為什麼逆向工程值得用來探索強化學習" + +#: website/public/blog/reverse-engineering-models/index.html:559 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Bounded reconstruction gives us a practical environment for studying these " +"choices. Once the target and compiler are pinned, many source experiments " +"produce quick, repeatable feedback. The agent can compile a candidate, " +"inspect the mismatch and use that observation to choose its next step." +msgstr "" +"範圍明確的重建任務，為研究這些選擇提供了一個實際環境。目標和編譯器版本固定之後，許多原始碼實驗都能快速給出可重複的回饋。agent " +"可以編譯候選原始碼、檢查差異，再根據觀察選擇下一步。" + +#: website/public/blog/reverse-engineering-models/index.html:566 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Reinforcement learning could train that decision-making process through the " +"feedback it receives. A candidate accepted by a defined check gives us a " +"concrete outcome, while the sequence of attempts lets us observe how much " +"work it took to reach it." +msgstr "" +"強化學習可以利用這些回饋，訓練模型的決策過程。候選原始碼透過明確定義的檢查，就給出了一個具體成果；而嘗試的過程，讓我們可以看到實現它花了多少功夫。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#: website/public/blog/reverse-engineering-models/index.html:572 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"There is already research supporting this direction. {ph0}LLM4Decompile{ph1}" +" studies models specialized for decompilation. {ph2}AutoDecompiler{ph3} " +"trains multi-turn refinement using compiler and execution feedback. Their " +"studies focus on functional recovery. For this work, I would use them as a " +"starting point for the related task of finding source that matches a " +"historical compiler's output." +msgstr "" +"已有研究支援這個方向。{ph0}LLM4Decompile{ph1}研究專門用於反編譯的模型。{ph2}AutoDecompiler{ph3}利用編譯和執行回饋，訓練多輪改進能力。這些研究關注的是功能還原。對於我們的工作，我會以它們為起點，研究一個相關任務：找到能復現歷史編譯器輸出的原始碼。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:584 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To reach such a well-defined task, though, we may first need to establish " +"what a function owns or devise a new runtime measurement. In TH08, for " +"example, a {ph0}collision difference led back to how the original program " +"handled floating-point calculations{ph1}. Before the agent could test a " +"useful correction, it had to work out what to measure." +msgstr "" +"不過，要得到這樣一個定義明確的任務，有時得先確定函式負責哪些資源，或者設計新的執行時測量。在 TH08 " +"中，{ph0}一次碰撞差異最終追溯到原版處理浮點計算的方式{ph1}。agent 必須先弄清楚該測量什麼，才能驗證有價值的修正。" + +#: website/public/blog/reverse-engineering-models/index.html:595 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This suggests another role for a broadly capable model: helping make the " +"problem checkable in the first place. Once that check exists, we can " +"investigate how far a learned search policy can get with it." +msgstr "這提示了能力全面的模型的另一個作用：先把問題變得可驗證。驗證手段建立起來後，我們才可以研究，學出來的搜尋策略能走多遠。" + +#: website/public/blog/reverse-engineering-models/index.html:603 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "When the oracle is wrong" +msgstr "當驗證器出錯時" + +#: website/public/blog/reverse-engineering-models/index.html:604 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Once we train against that check, its coverage becomes part of what the " +"model learns. I call the reference or checker an oracle. Even if it gives a " +"repeatable answer, we still need to understand what that answer covers." +msgstr "" +"一旦用某項檢查來訓練模型，它的覆蓋範圍就會成為學習環境的一部分。我把這種參考標準或驗證器稱為 " +"oracle。即使它能給出可重複的答案，我們也必須知道，這個答案覆蓋了什麼。" + +#: website/public/blog/reverse-engineering-models/index.html:610 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"TH08 showed how much can hide outside that coverage. Some reconstructed " +"functions passed their exact comparisons with wrong floating constants. The " +"comparator adjusted relocated addresses before checking instructions, but it" +" checked the values behind those addresses only when they had been " +"explicitly recorded for verification. Other literal references could pass " +"without their values being examined." +msgstr "" +"TH08 " +"展示了檢查範圍之外可能藏著多少問題。一些重建函式使用了錯誤的浮點常量，卻仍然透過了精確比較。比較器會先調整重定位地址，再檢查指令；但只有明確登記過的常量，才會進一步檢查地址指向的值。其他常量引用，即使值沒被檢查，也可能透過。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: 1.0f +#. {ph3}: 32.0f +#: website/public/blog/reverse-engineering-models/index.html:619 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To close that gap, a {ph0}wider audit{ph1} found incorrect constants in " +"functions that had already been accepted. In one calculation, the " +"reconstructed source added {ph2} where the original added {ph3}." +msgstr "" +"為了補上這個漏洞，{ph0}一次更全面的審計{ph1}在已經透過檢查的函式中找到了錯誤常量。有一個計算中，重建原始碼加的是 {ph2}，而原版加的是 " +"{ph3}。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:629 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}A learner rewarded by the old check could learn to produce the wrong " +"reconstruction.{ph1} We had not trained an RL model against this oracle; the" +" observed failure was in the check itself. But using that check as a reward " +"would have made the same blind spot part of the learning environment." +msgstr "" +"{ph0}如果用舊檢查的結果作為獎勵，模型可能學會生成錯誤的重建原始碼。{ph1} 我們沒有用這個驗證器訓練 RL " +"模型；實際觀察到的是檢查本身的漏洞。但如果把它用作獎勵，同一個盲點也會進入學習環境。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:639 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"A functional test can have a similar blind spot. {ph0}Differential-fuzzing " +"research on LLM decompilers{ph1} has found behavior differences beyond the " +"supplied tests. Passing those tests therefore needs to be read in terms of " +"the cases they exercise, just as an instruction match needs to be read in " +"terms of the bytes and data it checks." +msgstr "" +"功能測試也可能存在類似盲點。{ph0}針對 LLM " +"反編譯器的差分模糊測試研究{ph1}發現，在已有測試之外，程式行為仍可能不同。因此，理解“測試透過”時，必須看它實際覆蓋了哪些情況；理解“指令匹配”時，也必須看它檢查了哪些位元組和資料。" + +#: website/public/blog/reverse-engineering-models/index.html:649 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Before using either kind of check as a reward, I would test it with " +"deliberate failures. I would also keep final acceptance outside the " +"candidate's editable source, so the model has to satisfy the check through " +"its reconstruction. Otherwise, something like a byte array copied from the " +"target could satisfy a naive score without recovering useful source." +msgstr "" +"在把任何一種檢查用作獎勵之前，我都會用刻意構造的錯誤來測試它。我還會把最終驗收機制放在候選原始碼可修改的範圍之外，讓模型透過正確重建來滿足檢查。否則，直接複製目標程式的位元組陣列，也可能騙過簡單的評分，卻沒有還原出有用的原始碼。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:657 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Improving the learner and improving its oracle belong to the same " +"research problem.{ph1} As the learner gets better at optimizing the feedback" +" we give it, understanding that feedback becomes more important too." +msgstr "{ph0}改進模型和改進它的驗證器，屬於同一個研究問題。{ph1} 模型越擅長最佳化我們給它的回饋，我們就越需要理解回饋本身。" + +#: website/public/blog/reverse-engineering-models/index.html:667 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "A comparison worth running" +msgstr "值得做的對照實驗" + +#: website/public/blog/reverse-engineering-models/index.html:668 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"With those checks in place, I would start the comparison on a fixed VC7.1 " +"task family. First, I would establish baselines for a strong general model " +"and a smaller one. Adding repository knowledge to the smaller model would " +"show what retrieval changes. From there, supervised specialization and an " +"RL-trained version would let us examine what training adds." +msgstr "" +"具備這些檢查之後，我會從一組固定的 VC7.1 " +"任務開始比較。先為強通用模型和小型通用模型建立基線，再給小模型加入倉庫知識，觀察檢索帶來的變化。接著比較監督訓練的專用模型，以及在此基礎上進行 RL " +"訓練的版本，看看訓練增加了什麼。" + +#: website/public/blog/reverse-engineering-models/index.html:676 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"To make those comparisons informative, each model would receive the same " +"target evidence and tools. For the training-specific comparisons, I would " +"keep the base model and available knowledge fixed as well. Repeated trials " +"would help distinguish a reliable search strategy from a lucky candidate." +msgstr "" +"為了讓比較有意義，每個模型都應獲得相同的目標證據和工具。專門比較訓練效果時，還要固定基礎模型和可用知識。重複實驗有助於區分可靠的搜尋策略與偶然成功的候選方案。" + +#: website/public/blog/reverse-engineering-models/index.html:683 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would then separate two starting conditions: reconstructing a function " +"from its instructions, and closing the last few differences in maintained " +"source. A specialist might be excellent at the second task while a general " +"model does much of the work needed to reach it. Measuring them separately " +"would reveal that division of work." +msgstr "" +"我還會區分兩種起點：從指令開始重建函式，以及在已有原始碼中消除最後幾處差異。專用模型可能很擅長後者，而通用模型承擔了到達這個階段的大部分工作。分別評估，才能看清這種分工。" + +#: website/public/blog/reverse-engineering-models/index.html:691 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both tasks need genuinely held-out examples. Because related Touhou games " +"share engine code, holding out a game's name would still allow nearly " +"identical functions to appear in training. I would instead hold out source " +"families and keep their accepted answers out of both training and retrieval." +msgstr "" +"兩類任務都需要真正未見過的測試案例。東方系列的相關遊戲會共享引擎程式碼，只留出某個遊戲，仍可能讓幾乎相同的函式進入訓練集。我會改為按相關原始碼家族留出測試集，並把它們透過驗證的答案同時排除在訓練資料和檢索庫之外。" + +#: website/public/blog/reverse-engineering-models/index.html:698 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"On those examples, the outcome would be complete, independently validated " +"matches. Giving each model the same candidate or compile budget would help " +"compare search quality. Measuring elapsed time and total cost, including " +"compilation, would tell us whether that quality produces an efficient " +"workflow. Training costs would need their own accounting too." +msgstr "" +"在這些案例上，要統計的是完整、獨立驗證過的匹配成果。給每個模型相同的候選數量或編譯次數預算，可以比較搜尋品質。記錄總耗時和總成本，包括編譯，才能知道這種品質是否轉化成了高效的工作流程。訓練成本也要單獨計算。" + +#: website/public/blog/reverse-engineering-models/index.html:706 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Both views matter because a cheaper proposal can become an expensive " +"workflow if it causes more uninformative compiles. Conversely, a slower " +"model may pay for itself by finding a useful experiment sooner. The " +"comparison should capture those costs." +msgstr "" +"兩種角度都重要。一個便宜的提議，如果導致大量無效編譯，整個工作流程仍可能很貴。反過來，較慢的模型如果更早找到有價值的實驗，也可能更划算。對照實驗應當反映這些成本。" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:712 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"Counting accepted results also keeps intermediate progress in perspective. " +"One TH09 {ph0}replay-save candidate{ph1} looked promising because the " +"compiled function had the expected size, but a full comparison still found " +"substantial differences. Changes in how the compiler arranges instructions " +"can also make a candidate look closer without resolving the underlying " +"problem. Such diagnostics can guide the next experiment, but I would " +"validate any reward for \"getting closer\" against the full acceptance " +"check." +msgstr "" +"只統計透過驗收的結果，也能幫助我們合理看待中間進展。TH09 " +"的一個{ph0}錄影儲存候選方案{ph1}看起來很有希望，因為編譯後函式的大小符合預期，但完整比較仍發現大量差異。編譯器改變指令排列，也可能讓候選原始碼看起來更接近，卻沒有解決根本問題。這些診斷可以指導下一步，但若要獎勵“變得更接近”，我會先對照完整驗收檢查，確認這種獎勵有效。" + +#: website/public/blog/reverse-engineering-models/index.html:727 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"I would be interested in the pattern across these results. If a specialist " +"does well within a familiar compiler family, does that advantage survive a " +"toolchain change? And once both models have good retrieval, how much does " +"training still add? The answers would help us decide where each kind of " +"model is most useful." +msgstr "" +"我想看這些結果呈現出什麼規律。如果專用模型在熟悉的編譯器上表現很好，換了工具鏈後，這種優勢還在嗎？當雙方都擁有良好的檢索能力時，訓練還能增加多少價值？這些答案能幫助我們決定，每類模型最適合用在哪裡。" + +#: website/public/blog/reverse-engineering-models/index.html:736 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "What kind of agent does this suggest?" +msgstr "這意味著怎樣的 agent？" + +#: website/public/blog/reverse-engineering-models/index.html:737 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"If the models turn out to help at different stages, a useful arrangement " +"might be a broad agent with access to a specialist. Once a task becomes a " +"familiar compiler search, it could ask the specialist for candidates. If the" +" results stop making sense, the broader agent could return to the target " +"evidence or investigate the measurement itself." +msgstr "" +"如果不同模型在不同階段各有所長，一種有價值的安排可能是：讓能力全面的 agent " +"可以呼叫專用模型。當任務變成一種熟悉的編譯器搜尋時，它可以請專用模型提出候選方案。如果結果開始說不通，通用 agent " +"可以回頭檢查目標證據，或者調查測量本身。" + +#: website/public/blog/reverse-engineering-models/index.html:745 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"In that arrangement, specialization becomes a tool the agent can choose as " +"the investigation develops. The agent keeps the freedom to change direction " +"that made the earlier breakthroughs possible." +msgstr "" +"在這樣的安排中，專用模型成為 agent 隨著調查進展可以選擇的工具。agent 仍然擁有改變方向的自由，而這正是前面那些突破發生的重要條件。" + +#: website/public/blog/reverse-engineering-models/index.html:750 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"This follows naturally from the industrial shift I described in the previous" +" essay. Being able to run many more experiments changes how we organize the " +"work. It also gives us a reason to ask which decisions need broad judgment " +"and which improve through repeated experience with a particular craft." +msgstr "" +"這自然延續了上一篇文章中談到的工業化轉變。能夠開展更多實驗，會改變我們組織工作的方式，也讓我們有理由追問：哪些決策需要全面的判斷，哪些可以透過反覆積累某項具體技藝來改善？" + +#: website/public/blog/reverse-engineering-models/index.html:757 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"For now, the strongest general model has earned its place in my workflow by " +"opening problems I had left unresolved. Those successes are what make me " +"want to examine the work more closely: how much depended on the model's " +"breadth, and how much could be handled by a model trained in this particular" +" kind of reconstruction?" +msgstr "" +"目前，最強的通用模型解決了我過去沒能解決的問題，因此在我的工作流程中佔據了一席之地。正是這些成功，讓我想更仔細地研究過程：有多少依賴模型能力的廣度？又有多少可以交給專門學習這類重建的模型？" + +#. {ph0}: +#. {ph1}: +#: website/public/blog/reverse-engineering-models/index.html:765 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}The repositories give us a way to test the question.{ph1} I would like " +"to use that record to build the comparison and see what we learn." +msgstr "{ph0}這些程式碼倉庫讓我們有機會驗證這個問題。{ph1} 我想利用這些記錄建立對照實驗，看看我們能學到什麼。" + +#: website/public/blog/reverse-engineering-models/index.html:775 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Evidence and model attribution" +msgstr "證據與模型歸屬" + +#: website/public/blog/reverse-engineering-models/index.html:776 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The examples describe historical function-level comparisons recorded in the " +"linked commits. Whole-game runtime checks are separate milestones. Source " +"and matching records were reviewed on 13 October 2026." +msgstr "" +"這些例子來自連結中的提交中記錄的歷史函式級比較。整款遊戲的執行時檢查屬於另一個里程碑。相關原始碼與匹配記錄核查於 2026 年 10 月 13 日。" + +#. {ph0}: gpt-dots +#. {ph1}: gpt-web +#: website/public/blog/reverse-engineering-models/index.html:782 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The TH09 label {ph0} identifies GPT-6-Astra. {ph1} changed with the model " +"used at the time, so that label alone does not identify an individual run. " +"Explicit model names in the other cited commits establish their recorded " +"attribution; the histories do not contain uniform per-run cost or effort " +"measurements." +msgstr "" +"TH09 中的標籤 {ph0} 對應 GPT-6-Astra。{ph1} " +"會隨當時使用的模型變化，因此僅憑這個標籤無法確定某次執行的具體模型。其他引用提交中的明確模型名，提供了對應的歸屬記錄；這些歷史記錄並沒有統一的單次執行成本或投入統計。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#: website/public/blog/reverse-engineering-models/index.html:791 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 temporary lifetimes:{ph1} the {ph2}earlier context review{ph3} and" +" {ph4}complete cold reconstruction{ph5}." +msgstr "{ph0}TH09 臨時值生命週期：{ph1}{ph2}早期上下文檢查{ph3}與{ph4}完整的乾淨重建{ph5}。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:803 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}TH09 Front callback:{ph1} {ph2}non-exact diagnostic regions{ph3}, " +"{ph4}Astra's new graph{ph5} and its {ph6}accepted recompile replay{ph7}." +msgstr "" +"{ph0}TH09 Front 回呼：{ph1}{ph2}未匹配的診斷區域{ph3}、{ph4}Astra " +"提出的新控制流圖{ph5}與{ph6}透過驗收的重新編譯檢查{ph7}。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:819 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Compiler knowledge:{ph1} {ph2}TH095 identifier-bucket reuse{ph3}, " +"{ph4}TH04 player source and linker order{ph5} and {ph6}Luna's TH09 enemy-" +"spawn closure{ph7}." +msgstr "" +"{ph0}編譯器知識：{ph1}{ph2}TH095 識別符號分桶規律的複用{ph3}、{ph4}TH04 " +"玩家原始碼與連結順序{ph5}、{ph6}Luna 完成的 TH09 敵人生成函式匹配{ph7}。" + +#. {ph0}: +#. {ph1}: +#. {ph2}: +#. {ph3}: +#. {ph4}: +#. {ph5}: +#. {ph6}: +#. {ph7}: +#: website/public/blog/reverse-engineering-models/index.html:835 +#, python-brace-format +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"{ph0}Checks and diagnostics:{ph1} {ph2}TH08 floating-literal audit{ph3}, " +"{ph4}TH08 numerical investigation{ph5} and {ph6}TH09 replay-save " +"reconstruction{ph7}." +msgstr "" +"{ph0}檢查與診斷：{ph1}{ph2}TH08 浮點常量審計{ph3}、{ph4}TH08 數值問題調查{ph5}、{ph6}TH09 " +"錄影儲存重建{ph7}。" + +#: website/public/blog/reverse-engineering-models/index.html:852 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"The proposed specialist and RL comparisons are future experiments. The " +"project histories motivate them; no trained specialist was evaluated for " +"this essay." +msgstr "文中提出的專用模型與 RL 對照，屬於未來實驗。專案歷史為這些想法提供了依據；本文尚未評估經過訓練的專用模型。" + +#: website/public/blog/reverse-engineering-models/index.html:6 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Does reverse engineering need a bigger model? | REA" +msgstr "逆向工程需要更大的模型嗎？ | REA" + +#: website/public/blog/reverse-engineering-models/index.html:7 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "" +"What Touhou reconstruction taught me about general models, specialist models" +" and learning to propose better hypotheses from compiler feedback." +msgstr "東方重建讓我如何思考通用模型、專用模型，以及怎樣從編譯器回饋中學會提出更好的假設。" + +#: website/public/blog/reverse-engineering-models/index.html:240 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "On this page" +msgstr "在本頁" + +#: website/public/blog/reverse-engineering-models/index.html:253 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Models and reverse engineering essay" +msgstr "模型與逆向工程隨筆" + +#: website/public/blog/reverse-engineering-models/index.html:491 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Selected reconstruction experiments" +msgstr "重建實驗選例" + +#: website/public/blog/reverse-engineering-models/index.html:876 +msgctxt "page:blog/reverse-engineering-models/index.html" +msgid "Back to top" +msgstr "回到頂部" + +#~ msgctxt "page:blog/index.html" +#~ msgid "By N0zoM1z0 · 13 October 2026 · Draft for review" +#~ msgstr "作者 N0zoM1z0 · 2026 年 10 月 13 日 · 待審草稿" #~ msgctxt "page:index.html" #~ msgid "Setup guide" diff --git a/website/public/blog/index.html b/website/public/blog/index.html index 8cacf9779..a89e27807 100644 --- a/website/public/blog/index.html +++ b/website/public/blog/index.html @@ -207,9 +207,7 @@ Read the article -

    - By N0zoM1z0 · 13 October 2026 · Draft for review -

    +

    By N0zoM1z0 · 13 October 2026

    diff --git a/website/public/blog/reverse-engineering-models/index.html b/website/public/blog/reverse-engineering-models/index.html index 9a94ee811..a5bd4dd15 100644 --- a/website/public/blog/reverse-engineering-models/index.html +++ b/website/public/blog/reverse-engineering-models/index.html @@ -8,12 +8,50 @@ name="description" content="What Touhou reconstruction taught me about general models, specialist models and learning to propose better hypotheses from compiler feedback." /> - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -79,7 +117,7 @@ FAQ
    - +
    English