merge: preserve current MCP paging contracts in organized documentation

Retain list_unknowns paging guidance in docs/reference/mcp-contracts.md and verify the public routes and affected MCP regressions.

Co-Authored-By: Codex <noreply@openai.com>
This commit is contained in:
Matheus Breguêz 2026-10-12 21:24:41 -03:00
commit 51aa6eeca3
No known key found for this signature in database
GPG Key ID: B6FA8458D5176E83
54 changed files with 1778 additions and 285 deletions

View File

@ -78,6 +78,12 @@ unknown metadata. Missing omission counts remain null. Summary output carries
every matching record; actual transport admission still applies. Omitted
`detail` preserves complete canonical bundle delivery; filters require summary.
`list_unknowns` returns every matching current head by default. For a smaller
inline response, supply `offset` and `limit`, then follow `next_offset` until it
is null. `total` counts all matching heads; each page preserves complete records
in stable unknown ID order. Filters apply before paging. Pages reflect current
session state, so keep the unknown ledger unchanged while traversing it.
For a portable exported bundle, the CLI shares this projection:
`rea inspect-evidence-bundle '{"path":"/tmp/evidence.json","detail":"summary","filters":{"operation":"analyze_function"}}' --format json`.
The CLI validates and materializes that local bundle under its existing memory

View File

@ -1,7 +1,7 @@
# Verification after upstream integration
Integrated upstream 82cfcee19f18b0eeae9a330ba04e0411cdf85225, preserving its
new signal-recording guide, managed summary documentation and current translations.
Integrated upstream c462a4ebd, preserving its new signal-recording guide, managed
summary documentation, current translations and unknown-head pagination contract.
- Documentation facts and README consistency checks passed.
- Production VitePress build passed.
@ -15,6 +15,9 @@ new signal-recording guide, managed summary documentation and current translatio
ten files. CLI runs need process-table access for actual ownership/cleanup
verification; restricted execution failed with spawn EPERM, while the normal
host run passed without changing timeouts or assertions.
- After the subsequent upstream advance, 14 tests across unknown-head pagination,
workflow replay and target-content framing passed. Static checks, documentation
facts and all 56 published routes passed again against that base.
- Documentation CI and the manual build require the built-page verifier.
An actual CI-planner regression ensures edits to the verifier select that lane.

View File

@ -0,0 +1,60 @@
# Selected unknown retrieval
`list_unknowns` now accepts optional paging. Omitted limits still return all
matching heads. Selection happens before payload cloning in the ledger owner;
MCP returns complete selected records and the total matching count.
## Bounded workflow comparison
The task is to retrieve the first ten complete current unknown heads, in stable
ID order, from a session containing 200 heads. The complete-list workflow reads
all heads and selects ten locally. The selected-page workflow requests
`{"offset":0,"limit":10}`. Both use one real SDK MCP call through
`Client`/`InMemoryTransport`, with the same imported session and no provider.
All six executions returned identical selected records and the expected total.
These are two workflows on the candidate implementation, **not** a historical
before/after binary comparison. The candidate was built from `f09bb8a99` plus
the paging changes. Runtime file-table SHA-256 for the preserved complete-list
copy: `06a457dac90912dceba88f945573897372701f6c6b2e0934d7028023b4646648`.
Node 24.18.0, locked MCP SDK 2.3.1, tokenx 1.3.0, macOS arm64.
Flameox 0.2.9 ran three randomized paired blocks, seed 731, with a 30-second
and 1-GiB limit per execution. Each fresh process performed a first read and
five identical warm reads. The semantic oracle checked the selected answer
SHA-256, `c18d718a71323eb53712040b4b09ffebfc3497000d1ff8f58d57665638077310`.
Every execution and oracle passed.
| Measurement | Complete list | Selected page |
| ----------------------------------------- | ------------: | ------------: |
| Calls / errors per task | 1 / 0 | 1 / 0 |
| Serialized response bytes | 646,855 | 32,603 |
| Estimated request tokens | 12 | 21 |
| Estimated response tokens | 151,097 | 7,647 |
| Estimated full catalog tokens | 764,512 | 764,512 |
| Median first-read elapsed milliseconds | 1,084.05 | 1,082.65 |
| Median warm-read elapsed milliseconds | 3.943 | 0.426 |
| Median first-read CPU milliseconds | 1,333.23 | 1,328.47 |
| Median warm-read CPU milliseconds | 5.012 | 0.591 |
| Median whole-process elapsed milliseconds | 1,752.40 | 1,664.42 |
| Median peak process-family MiB | 598.59 | 554.09 |
Read timings include the MCP call and answer validation. First-read timings
exclude startup, imports, session preparation, handshake and catalog discovery;
the whole-process elapsed measurement includes those, all six reads, token
estimation and cleanup. Whole-process CPU was not measured. Peak RSS includes
preparation and imports; this workload has no child processes. Host caches were
not flushed. Three pairs and fifteen warm reads per workflow describe this
fixture only; they do not establish broad speed or memory improvements.
Token counts estimate serialized request, response and catalog JSON, including
both MCP response representations. They are not measured model tokens. Host
schema injection, prompts, reasoning, retries and actual agent behavior are
unmeasured. The full catalog cost is unchanged between these workflows and can
dominate a first task. Smaller selected delivery does not remove that cost.
Native Flameox evidence was preserved locally as
`14ea4c0ef6595a1b89645299d4e184a582d5e975f7916294d68c1f1b312608d0`,
analysis `f66e125179623591ce00b1ccf0d917c2e90512de55db1061269c042d65e1125d`.
An earlier 512-MiB capture exceeded its memory limit and is excluded from these
measurements. No production provider or model trial was run.

View File

@ -124,9 +124,7 @@ async function verifyFixture(byteOrder, kernelModule) {
});
const call = async (name, args = {}) =>
requireMcpOperationResult(
await client.callTool({ name, arguments: args }, undefined, {
timeout: 180000,
}),
await client.callTool({ name, arguments: args }, { timeout: 180000 }),
name,
);
try {

View File

@ -171,7 +171,6 @@ try {
const call = async (name, args = {}) => {
const reply = await client.callTool(
{ name, arguments: args },
undefined,
{ timeout: 180000 },
);
return requireMcpOperationResult(reply, name);
@ -184,7 +183,6 @@ try {
name: "open_binary",
arguments: { path: fixture.path, provider_id: "ghidra" },
},
undefined,
{ timeout: 60000 },
);
const error = requireMcpToolError(rejected);
@ -423,7 +421,6 @@ try {
snapshot_path: snapshotPath,
},
},
undefined,
{ timeout: 180000 },
);
requireMcpToolError(wrongSnapshot);

View File

@ -41,9 +41,10 @@ const cli = async (...args) => {
return parseEvidence(JSON.parse(stdout));
};
const call = async (name, args) => {
const result = await client.callTool({ name, arguments: args }, undefined, {
timeout: 150000,
});
const result = await client.callTool(
{ name, arguments: args },
{ timeout: 150000 },
);
assert.notEqual(result.isError, true, mcpTextValue(result));
return parseEvidence(JSON.parse(mcpTextValue(result)));
};

View File

@ -3,6 +3,7 @@ import type {
MipsAbiFlags,
MipsElfMetadata,
} from "../domain/binaryTargetTypes.js";
import { readElfFixedHeader } from "../domain/elfFixedHeader.js";
import { err, ok, type Result } from "../domain/result.js";
import {
readMipsElfSharedLoad,
@ -64,21 +65,22 @@ export const readMipsElfMetadata = async (
const header = await read(0, 52, "ELF32 header");
if (!header.ok) return header;
const bytes = header.value;
const fixedHeader = readElfFixedHeader(bytes, fileSize);
if (!fixedHeader.ok) return fixedHeader;
const little = metadata.byteOrder === "little";
const u16 = (data: Buffer, offset: number): number =>
little ? data.readUInt16LE(offset) : data.readUInt16BE(offset);
const u32 = (data: Buffer, offset: number): number =>
little ? data.readUInt32LE(offset) : data.readUInt32BE(offset);
if (
bytes.toString("hex", 0, 4) !== "7f454c46" ||
bytes[4] !== 1 ||
bytes[5] !== (little ? 1 : 2) ||
fixedHeader.value.elfClass !== 32 ||
fixedHeader.value.byteOrder !== metadata.byteOrder ||
u16(bytes, 18) !== 8 ||
u16(bytes, 16) !== metadata.type ||
u32(bytes, 36) !== metadata.flags
)
return err("MIPS ELF header no longer matches the resolved identity");
if (bytes[6] !== 1 || u32(bytes, 20) !== 1 || u16(bytes, 40) !== 52)
if (fixedHeader.value.headerSize !== 52)
return err("invalid MIPS ELF32 version or header size");
const tables = [

View File

@ -19,6 +19,10 @@ import type {
UnknownStatus,
UpdateUnknownInput,
} from "../../domain/residualUnknown.js";
import type {
UnknownPage,
UnknownQuery,
} from "../investigation/InvestigationRecordPort.js";
import { err, ok, type Result } from "../../domain/result.js";
import type {
AnalysisExecution,
@ -350,6 +354,10 @@ export abstract class BinarySessionRecords {
return this.#records.listUnknowns(filters);
}
listUnknownsPage(query: UnknownQuery = {}): UnknownPage {
return this.#records.listUnknownsPage(query);
}
verifyUnknownResolution(unknownId: string): Result<
{
readonly valid: boolean;

View File

@ -26,6 +26,7 @@ import {
} from "../../domain/residualUnknown.js";
import { err, ok, type Result } from "../../domain/result.js";
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
import type { UnknownPage, UnknownQuery } from "./InvestigationRecordPort.js";
type EvidenceLedgerFailure = EvidenceIntegrityError;
type RecordResult = Result<"added" | "duplicate", EvidenceLedgerFailure>;
@ -326,18 +327,39 @@ export class EvidenceLedger {
readonly domain?: string;
} = {},
): ResidualUnknown[] {
return [...this.#unknownHeads.values()]
return this.listUnknownsPage(filters).items;
}
/** Select a stable page of current heads before detaching record payloads. */
listUnknownsPage(query: UnknownQuery = {}): UnknownPage {
const offset = query.offset ?? 0;
const matching = [...this.#unknownHeads.values()]
.filter(
(unknown) =>
(filters.status === undefined || unknown.status === filters.status) &&
(filters.severity === undefined ||
unknown.severity === filters.severity) &&
(filters.domain === undefined || unknown.domain === filters.domain),
(query.status === undefined || unknown.status === query.status) &&
(query.severity === undefined ||
unknown.severity === query.severity) &&
(query.domain === undefined || unknown.domain === query.domain),
)
.sort((left, right) =>
compareUnicodeCodePoints(left.unknown_id, right.unknown_id),
)
.map((unknown) => structuredClone(unknown));
);
const start = Math.min(offset, matching.length);
const end =
query.limit === undefined
? undefined
: start + Math.min(query.limit, matching.length - start);
const page = matching.slice(start, end);
const nextOffset =
offset <= matching.length && start + page.length < matching.length
? start + page.length
: null;
return {
items: page.map((unknown) => structuredClone(unknown)),
total: matching.length,
offset,
next_offset: nextOffset,
};
}
/** Return current resolution validity; imported invalid states are rejected. */

View File

@ -38,6 +38,21 @@ export interface EvidenceUnknownWriter {
): Result<ResidualUnknown | null, AnalysisError>;
}
export type UnknownPage = {
readonly items: ResidualUnknown[];
readonly total: number;
readonly offset: number;
readonly next_offset: number | null;
};
export interface UnknownQuery {
readonly status?: UnknownStatus;
readonly severity?: ResidualUnknown["severity"];
readonly domain?: string;
readonly offset?: number;
readonly limit?: number;
}
/** Optimistic Unknown revisions and consistency verification. */
export interface UnknownRegistryPort {
recordUnknown(
@ -51,6 +66,7 @@ export interface UnknownRegistryPort {
readonly severity?: ResidualUnknown["severity"];
readonly domain?: string;
}): ResidualUnknown[];
listUnknownsPage(query?: UnknownQuery): UnknownPage;
verifyUnknownResolution(unknownId: string): Result<
{
readonly valid: boolean;

View File

@ -19,6 +19,8 @@ import type {
EvidenceWriter,
EvidenceUnknownWriter,
UnknownRegistryPort,
UnknownPage,
UnknownQuery,
} from "./InvestigationRecordPort.js";
/** One investigation's validated record owner, independent of binary execution. */
@ -124,6 +126,10 @@ export class InvestigationRecords
return this.#evidence.listUnknowns(filters);
}
listUnknownsPage(query: UnknownQuery = {}): UnknownPage {
return this.#evidence.listUnknownsPage(query);
}
/** Verify registry consistency without asserting the answer is true. */
verifyUnknownResolution(unknownId: string): Result<
{

View File

@ -6,6 +6,7 @@ import type { OwnedStagedOutput } from "../filesystem/OwnedStagedOutput.js";
import type { ArtifactReader } from "./ArtifactReader.js";
import { ArtifactReaderFailure } from "./ArtifactReader.js";
import type { SafeOutputTree } from "./SafeOutputTree.js";
import type { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
/** One artifact resource and its format-specific cleanup owner. */
export type ArtifactResourceOwner =
@ -38,6 +39,11 @@ export type ArtifactResourceOwner =
readonly kind: "staged-output";
readonly resource: string;
readonly output: OwnedStagedOutput;
}
| {
readonly kind: "private-runtime-root";
readonly resource: string;
readonly root: Pick<PrivateRuntimeRoot, "path" | "close">;
};
export type ArtifactCleanupAttempt =
@ -161,5 +167,8 @@ const cleanup = async (owner: ArtifactResourceOwner): Promise<void> => {
return;
case "staged-output":
await owner.output.close();
return;
case "private-runtime-root":
await owner.root.close();
}
};

View File

@ -170,8 +170,8 @@ export interface BinaryPlistExpansionOptions {
readonly fail?: (kind: BinaryPlistExpansionFailure, message: string) => Error;
/**
* Structural bounds: "strict" keeps objects and the offset table before the
* trailer; "decoder" admits exactly what plist.parseBinary can read, so
* expansion is bounded without rejecting archives that decoder accepts.
* trailer; "decoder" follows the decoder's object-table boundary policy
* while still rejecting structures whose coercion would lose plist facts.
*/
readonly bounds?: "strict" | "decoder";
}
@ -267,6 +267,8 @@ export const estimateBinaryPlistExpansion = (
throw new TypeError("binary plist object offset is invalid");
const marker = bytes[offset] ?? 0;
const type = marker >> 4;
if (entry.dictionaryKey && type !== 5 && type !== 6)
throw new TypeError("binary plist dictionary key is not a string");
let size = marker & 0x0f;
let cursor = offset + 1;
if (size === 0x0f && type !== 0 && type !== 8) {

View File

@ -219,6 +219,37 @@ const keyedRoot = (payload: readonly Buffer[]): Buffer =>
describe("binary plist reference preflight", () => {
const selection = { offset: 0, limit: 10 };
it("rejects an array key that coerces to __proto__ before parsing", () => {
expect(() =>
decodeKeyedArchiveBytes(
keyedRoot([
plistArray([9]),
Buffer.from([0xd1, 0, 10, 0, 11]),
plistArray([12]),
Buffer.from([0xd1, 0, 13, 0, 14]),
plistString("__proto__"),
plistString("lost"),
plistString("value"),
]),
selection,
),
).toThrow("binary plist dictionary key is not a string");
});
it("rejects binary data dictionary keys", () => {
expect(() =>
decodeKeyedArchiveBytes(
keyedRoot([
plistArray([9]),
Buffer.from([0xd1, 0, 10, 0, 11]),
Buffer.from([0x41, 0xab]),
plistString("value"),
]),
selection,
),
).toThrow("binary plist dictionary key is not a string");
});
it("reports a reference cycle as malformed, not as an oversized archive", () => {
expect(() =>
decodeKeyedArchiveBytes(keyedRoot([plistArray([8])]), selection),

View File

@ -80,6 +80,49 @@ const baseScenario = () => ({
},
});
describe("browser scenario secret redaction", () => {
it("matches percent-escape hex case without case-folding raw secrets", () => {
const secrets = BrowserScenarioSecrets.resolve(
browserScenarioSchema.parse(baseScenario()),
{
REA_TEST_SESSION: "a/b?",
REA_TEST_PASSWORD: "password",
REA_TEST_COOKIE: "cookie",
},
);
if (secrets === undefined) throw new Error("Expected resolved secrets");
expect(secrets.redact("logged a%2fb%3f, a%2Fb%3F and a%2fb%3F")).toBe(
"logged [REDACTED:session], [REDACTED:session] and [REDACTED:session]",
);
expect(secrets.redact("raw A/B? remains case-sensitive")).toBe(
"raw A/B? remains case-sensitive",
);
expect(
secrets.redactBytes(
Buffer.concat([
Buffer.from([0xff, 0]),
Buffer.from("a%2fb%3F"),
Buffer.from([0x80]),
]),
),
).toEqual(
Buffer.concat([
Buffer.from([0xff, 0]),
Buffer.from("[REDACTED:session]"),
Buffer.from([0x80]),
]),
);
expect(
secrets.sanitizeUrl("https://app.example.test/?token=a%2fb%3F").url,
).toBe("https://app.example.test/?token=[REDACTED:session]");
expect(secrets.fingerprint("stored a%2fb%3F")).toEqual({
value_state: "redacted-secret",
value_sha256: null,
});
});
});
describe("browserScenarioSchema", () => {
it("accepts and normalizes a declared scenario", () => {
const parsed = browserScenarioSchema.parse(baseScenario());

View File

@ -15,84 +15,229 @@ const REDACTION_PREFIX = "[REDACTED:";
const encodedSecretValues = (secret: string): ReadonlySet<string> =>
new Set([
secret,
encodeURIComponent(secret),
new URLSearchParams([["value", secret]]).toString().slice("value=".length),
]);
type SecretSpellings = "raw" | "url" | "bytes";
type SecretSpellings = "url" | "text";
interface SecretPattern {
readonly text: string;
readonly searchText: string;
readonly percentInsensitive: boolean;
}
interface SecretReplacement extends SecretPattern {
readonly id: string;
}
interface SecretNeedle {
readonly id: string;
readonly needle: Buffer;
readonly percentInsensitive: boolean;
readonly marker: Buffer;
}
const secretSpellings = (
secret: string,
spellings: SecretSpellings,
): readonly string[] => {
if (secret === "") return [];
if (spellings === "raw") return [secret];
const values = [...encodedSecretValues(secret)];
if (spellings === "bytes") values.push(JSON.stringify(secret).slice(1, -1));
return [...new Set(values)].filter((value) => value !== "");
const normalizePercentEscapes = (value: string): string => {
if (!value.includes("%")) return value;
return value.replace(/%[0-9a-f]{2}/giu, (escape) => escape.toLowerCase());
};
const declaredLiterals = (
const secretPatterns = (
secret: string,
spellings: SecretSpellings,
): readonly SecretPattern[] => {
if (secret === "") return [];
const patterns: SecretPattern[] = [
{ text: secret, searchText: secret, percentInsensitive: false },
...[...encodedSecretValues(secret)].map((text) => ({
text,
searchText: normalizePercentEscapes(text),
percentInsensitive: true,
})),
];
if (spellings === "text")
patterns.push({
text: JSON.stringify(secret).slice(1, -1),
searchText: JSON.stringify(secret).slice(1, -1),
percentInsensitive: false,
});
const seen = new Set<string>();
return patterns.filter(({ text, percentInsensitive }) => {
if (text === "") return false;
const key = `${percentInsensitive ? "percent" : "literal"}:${text}`;
if (seen.has(key)) return false;
seen.add(key);
return true;
});
};
const declaredPatterns = (
values: ReadonlyMap<string, string>,
spellings: SecretSpellings,
): readonly string[] => [
...new Set(
[...values.values()].flatMap((secret) =>
secretSpellings(secret, spellings),
),
),
];
): readonly SecretPattern[] => {
const seen = new Set<string>();
return [...values.values()]
.flatMap((secret) => secretPatterns(secret, spellings))
.filter(({ text, percentInsensitive }) => {
const key = `${percentInsensitive ? "percent" : "literal"}:${text}`;
if (seen.has(key)) return false;
seen.add(key);
return true;
});
};
/** Pick a marker that does not itself contain a declared secret spelling. */
const redactionMarker = (id: string, literals: readonly string[]): string =>
[`${REDACTION_PREFIX}${id}]`, "[REDACTED]", "…", ""].find((candidate) =>
literals.every((literal) => !candidate.includes(literal)),
) ?? "";
const replaceSecretLiterals = (
value: string,
const secretReplacements = (
values: ReadonlyMap<string, string>,
spellings: SecretSpellings,
literals: readonly string[],
): string => {
const replacements = [...values]
): readonly SecretReplacement[] =>
[...values]
.flatMap(([id, secret]) =>
secretSpellings(secret, spellings).map((text) => ({ id, text })),
secretPatterns(secret, spellings).map((pattern) => ({ id, ...pattern })),
)
.sort(
(left, right) =>
right.text.length - left.text.length ||
(left.id < right.id ? -1 : left.id > right.id ? 1 : 0),
);
let output = value;
for (const replacement of replacements)
output = output.replaceAll(
replacement.text,
redactionMarker(replacement.id, literals),
);
return output;
const normalizePercentEscapesBytes = (value: Buffer): Buffer => {
if (value.indexOf(37) < 0) return value;
const normalized = Buffer.from(value);
for (let index = 0; index + 2 < normalized.length; index += 1)
if (
normalized[index] === 37 &&
isHexByte(normalized[index + 1]) &&
isHexByte(normalized[index + 2])
) {
normalized[index + 1] = lowercaseHexByte(normalized[index + 1]!);
normalized[index + 2] = lowercaseHexByte(normalized[index + 2]!);
index += 2;
}
return normalized;
};
const isHexByte = (value: number | undefined): boolean =>
value !== undefined &&
((value >= 48 && value <= 57) ||
(value >= 65 && value <= 70) ||
(value >= 97 && value <= 102));
const lowercaseHexByte = (value: number): number =>
value >= 65 && value <= 70 ? value + 32 : value;
const patternMatches = (
value: string,
normalizedValue: string,
pattern: SecretPattern,
): boolean =>
(pattern.percentInsensitive ? normalizedValue : value).includes(
pattern.searchText,
);
const containsSecretPattern = (
value: string,
normalizedValue: string,
patterns: readonly SecretPattern[],
): boolean => {
if (patterns.length === 0) return false;
return patterns.some((pattern) =>
patternMatches(value, normalizedValue, pattern),
);
};
const containsSecretBytePattern = (
value: Buffer,
normalizedValue: Buffer,
needles: readonly SecretNeedle[],
): boolean =>
needles.some((needle) =>
(needle.percentInsensitive ? normalizedValue : value).includes(
needle.needle,
),
);
/** Pick a marker that does not itself contain a declared secret spelling. */
const redactionMarker = (
id: string,
patterns: readonly SecretPattern[],
): string => {
const candidates = [`${REDACTION_PREFIX}${id}]`, "[REDACTED]", "…", ""];
return (
candidates.find((candidate) =>
patterns.every(
(pattern) =>
!patternMatches(
candidate,
normalizePercentEscapes(candidate),
pattern,
),
),
) ?? ""
);
};
const replaceSecretLiterals = (
value: string,
replacements: readonly SecretReplacement[],
patterns: readonly SecretPattern[],
): string => {
const normalizedValue = replacements.some(
({ percentInsensitive }) => percentInsensitive,
)
? normalizePercentEscapes(value)
: value;
const parts: string[] = [];
let offset = 0;
while (offset < value.length) {
let next:
| ((typeof replacements)[number] & { readonly index: number })
| undefined;
for (const replacement of replacements) {
const search = replacement.searchText;
const view = replacement.percentInsensitive ? normalizedValue : value;
const index = view.indexOf(search, offset);
if (
index >= 0 &&
(next === undefined ||
index < next.index ||
(index === next.index && replacement.text.length > next.text.length))
)
next = { ...replacement, index };
}
if (next === undefined) break;
const marker = redactionMarker(next.id, patterns);
parts.push(value.slice(offset, next.index), marker);
offset = next.index + next.text.length;
}
if (parts.length === 0) return value;
parts.push(value.slice(offset));
return parts.join("");
};
const secretNeedles = (
values: ReadonlyMap<string, string>,
literals: readonly string[],
): Array<{
readonly id: string;
readonly needle: Buffer;
readonly marker: Buffer;
}> =>
[...values].flatMap(([id, secret]) =>
secretSpellings(secret, "bytes").map((text) => ({
id,
needle: Buffer.from(text),
marker: Buffer.from(redactionMarker(id, literals)),
})),
);
replacements: readonly SecretReplacement[],
patterns: readonly SecretPattern[],
): SecretNeedle[] =>
replacements.map((pattern) => ({
id: pattern.id,
needle: Buffer.from(pattern.searchText),
percentInsensitive: pattern.percentInsensitive,
marker: Buffer.from(redactionMarker(pattern.id, patterns)),
}));
/** Resolved secret values kept only for one in-memory scenario session. */
export class BrowserScenarioSecrets {
private constructor(private readonly values: ReadonlyMap<string, string>) {}
private readonly urlPatterns: readonly SecretPattern[];
private readonly textPatterns: readonly SecretPattern[];
private readonly urlReplacements: readonly SecretReplacement[];
private readonly textReplacements: readonly SecretReplacement[];
private readonly textNeedles: readonly SecretNeedle[];
private constructor(private readonly values: ReadonlyMap<string, string>) {
this.urlPatterns = declaredPatterns(values, "url");
this.textPatterns = declaredPatterns(values, "text");
this.urlReplacements = secretReplacements(values, "url");
this.textReplacements = secretReplacements(values, "text");
this.textNeedles = secretNeedles(this.textReplacements, this.textPatterns);
}
static resolve(
scenario: BrowserScenario,
@ -127,18 +272,40 @@ export class BrowserScenarioSecrets {
}
redact(value: string): string {
const literals = declaredLiterals(this.values, "raw");
const output = replaceSecretLiterals(value, this.values, "raw", literals);
return literals.some((literal) => output.includes(literal)) ? "" : output;
const patterns = this.textPatterns;
if (patterns.length === 0) return value;
const output = replaceSecretLiterals(
value,
this.textReplacements,
patterns,
);
return containsSecretPattern(
output,
normalizePercentEscapes(output),
patterns,
)
? ""
: output;
}
/** Replace declared UTF-8, URI/form, and JSON-escaped secrets without decoding binary data. */
/** Replace declared UTF-8, URI/form, and JSON-escaped secret spellings. */
redactBytes(value: Buffer): Buffer {
const literals = declaredLiterals(this.values, "bytes");
const candidates = secretNeedles(this.values, literals)
const patterns = this.textPatterns;
if (patterns.length === 0) return value;
const candidates = this.textNeedles;
const hasPercentPatterns = candidates.some(
({ percentInsensitive }) => percentInsensitive,
);
const normalizedValue =
hasPercentPatterns && value.indexOf(37) >= 0
? normalizePercentEscapesBytes(value)
: value;
const sortedCandidates = candidates
.map((candidate) => ({
...candidate,
index: value.indexOf(candidate.needle),
index: (candidate.percentInsensitive ? normalizedValue : value).indexOf(
candidate.needle,
),
}))
.sort(
(left, right) =>
@ -148,13 +315,18 @@ export class BrowserScenarioSecrets {
const parts: Buffer[] = [];
let offset = 0;
while (true) {
let next: (typeof candidates)[number] | undefined;
for (const candidate of candidates) {
let next: (typeof sortedCandidates)[number] | undefined;
for (const candidate of sortedCandidates) {
if (candidate.index >= 0 && candidate.index < offset)
candidate.index = value.indexOf(candidate.needle, offset);
candidate.index = (
candidate.percentInsensitive ? normalizedValue : value
).indexOf(candidate.needle, offset);
if (
candidate.index >= 0 &&
(next === undefined || candidate.index < next.index)
(next === undefined ||
candidate.index < next.index ||
(candidate.index === next.index &&
candidate.needle.length > next.needle.length))
)
next = candidate;
}
@ -165,16 +337,23 @@ export class BrowserScenarioSecrets {
if (parts.length === 0) return value;
parts.push(value.subarray(offset));
const redacted = Buffer.concat(parts);
return literals.some((literal) => redacted.includes(Buffer.from(literal)))
const normalizedRedacted =
hasPercentPatterns && redacted.indexOf(37) >= 0
? normalizePercentEscapesBytes(redacted)
: redacted;
return containsSecretBytePattern(redacted, normalizedRedacted, candidates)
? Buffer.alloc(0)
: redacted;
}
/** Remove only declared secret values from one URL and report that action. */
sanitizeUrl(value: string): SanitizedBrowserUrl {
const literals = declaredLiterals(this.values, "url");
const output = replaceSecretLiterals(value, this.values, "url", literals);
if (literals.some((literal) => output.includes(literal)))
const patterns = this.urlPatterns;
if (patterns.length === 0) return sanitizeBrowserUrl(value);
const output = replaceSecretLiterals(value, this.urlReplacements, patterns);
if (
containsSecretPattern(output, normalizePercentEscapes(output), patterns)
)
return {
url: "",
origin: null,
@ -189,14 +368,15 @@ export class BrowserScenarioSecrets {
}
fingerprint(value: string): BrowserStorageValueFingerprint {
for (const secret of this.values.values())
if (secret !== "" && value.includes(secret))
return { value_state: "redacted-secret", value_sha256: null };
const patterns = this.textPatterns;
if (
patterns.length > 0 &&
containsSecretPattern(value, normalizePercentEscapes(value), patterns)
)
return { value_state: "redacted-secret", value_sha256: null };
return {
value_state: "hashed",
value_sha256: createHash("sha256")
.update(this.redact(value))
.digest("hex"),
value_sha256: createHash("sha256").update(value).digest("hex"),
};
}
}

View File

@ -1,4 +1,4 @@
import { request } from "node:http";
import { request, type ClientRequest, type IncomingMessage } from "node:http";
import { z } from "zod";
@ -29,6 +29,17 @@ const endpointTargetSchema = z.object({
});
const endpointTargetsSchema = z.array(endpointTargetSchema);
/** Per-response bounds for loopback CDP discovery documents. */
const CDP_DISCOVERY_RESPONSE_LIMITS = {
bytes: 8 * 1024 * 1024,
timeoutMs: 10_000,
} as const;
interface CdpDiscoveryResponseLimits {
readonly bytes: number;
readonly timeoutMs: number;
}
/** Validated direct CDP WebSocket bound to one discovered page target. */
interface CdpPageWebSocketEndpoint {
readonly scope: "page";
@ -291,29 +302,71 @@ export const readCdpJson = async (
url: URL,
operation: BrowserObservationOperation,
signal?: AbortSignal,
limits: CdpDiscoveryResponseLimits = CDP_DISCOVERY_RESPONSE_LIMITS,
): Promise<unknown> =>
await new Promise((resolve, reject) => {
if (signal?.aborted === true) {
reject(new AnalysisCancelledError(operation));
return;
}
const request_ = request(
let response: IncomingMessage | undefined;
let request_: ClientRequest | undefined;
let timer: NodeJS.Timeout | undefined;
let settled = false;
const removeAbort = (): void =>
signal?.removeEventListener("abort", onAbort);
const finish = (error?: Error, value?: unknown): void => {
if (settled) return;
settled = true;
if (timer !== undefined) clearTimeout(timer);
removeAbort();
if (error !== undefined) {
reject(error);
response?.destroy();
request_?.destroy();
} else resolve(value);
};
const onAbort = (): void => {
if (signal?.aborted === true)
finish(new AnalysisCancelledError(operation));
};
request_ = request(
url,
{ method: "GET", signal, headers: { Accept: "application/json" } },
(response) => {
if (response.statusCode !== 200) {
response.resume();
reject(
{ method: "GET", headers: { Accept: "application/json" } },
(received) => {
response = received;
if (settled) {
received.destroy();
return;
}
if (received.statusCode !== 200) {
finish(
new BrowserObservationError(operation, "invalid_endpoint_response"),
);
return;
}
const chunks: Buffer[] = [];
response.on("data", (chunk: Buffer) => chunks.push(chunk));
response.on("end", () => {
const parsed = safeParseJson(Buffer.concat(chunks).toString("utf8"));
let bytes = 0;
received.on("data", (chunk: Buffer) => {
if (settled) return;
bytes += chunk.byteLength;
if (bytes > limits.bytes) {
finish(
new BrowserObservationError(operation, "payload_limit", {
detail: `CDP discovery response exceeds its ${String(limits.bytes)} byte limit.`,
}),
);
return;
}
chunks.push(chunk);
});
received.on("end", () => {
if (settled) return;
const parsed = safeParseJson(
Buffer.concat(chunks, bytes).toString("utf8"),
);
if (!parsed.ok) {
reject(
finish(
new BrowserObservationError(
operation,
"invalid_endpoint_response",
@ -322,15 +375,41 @@ export const readCdpJson = async (
);
return;
}
resolve(parsed.value);
finish(undefined, parsed.value);
});
response.on("error", (cause: unknown) =>
reject(endpointFailure(cause, operation, signal)),
received.on("aborted", () =>
finish(
new BrowserObservationError(operation, "invalid_endpoint_response"),
),
);
received.on("close", () => {
if (!settled && !received.complete)
finish(
new BrowserObservationError(
operation,
"invalid_endpoint_response",
),
);
});
received.on("error", (cause: unknown) =>
finish(endpointFailure(cause, operation, signal)),
);
},
);
request_.on("error", (cause: unknown) =>
reject(endpointFailure(cause, operation, signal)),
finish(endpointFailure(cause, operation, signal)),
);
signal?.addEventListener("abort", onAbort, { once: true });
onAbort();
if (settled) return;
timer = setTimeout(
() =>
finish(
new BrowserObservationError(operation, "timeout", {
detail: `CDP discovery response exceeded its ${String(limits.timeoutMs)} ms deadline.`,
}),
),
limits.timeoutMs,
);
request_.end();
});

View File

@ -5,6 +5,77 @@ import { browserScenarioSchema } from "../domain/browserScenario.js";
import { BrowserScenarioSecrets } from "./BrowserScenarioSecrets.js";
import { PlaywrightScenarioEvents } from "./PlaywrightScenarioEvents.js";
describe("declared secrets in scenario events", () => {
it("redacts encoded declared secrets from text evidence and fingerprints", () => {
const listeners = new Map<string, (value: unknown) => void>();
const socketListeners = new Map<string, (value: unknown) => void>();
const page = {
on: (name: string, listener: (value: unknown) => void) => {
listeners.set(name, listener);
},
} as unknown as Page;
const secret = "a/b&c\n d";
const scenario = browserScenarioSchema.parse({
browser: { mode: "launch", executable_path: "/opt/chromium" },
start_url: { url: "https://app.example.test/" },
actions: [
{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 },
],
secrets: [
{ secret_id: "login_input", environment_variable: "REA_TEST_TOKEN" },
],
capture: { events: ["console", "page-errors", "websockets"] },
});
const secrets = BrowserScenarioSecrets.resolve(scenario, {
REA_TEST_TOKEN: secret,
});
if (secrets === undefined) throw new Error("Expected resolved secrets");
const events = new PlaywrightScenarioEvents({
page,
enabled: new Set(["console", "page-errors", "websockets"]),
secrets,
});
listeners.get("console")?.({
type: () => "log",
text: () => encodeURIComponent(secret),
location: () => ({ url: "https://app.example.test/app.js" }),
});
const escapedSecret = JSON.stringify(secret).slice(1, -1);
const error = new Error(`failure ${escapedSecret}`);
error.stack = `failure ${escapedSecret}`;
listeners.get("pageerror")?.(error);
listeners.get("websocket")?.({
url: () => "wss://app.example.test/socket",
on: (name: string, listener: (value: unknown) => void) => {
socketListeners.set(name, listener);
},
});
const formEncodedSecret = new URLSearchParams([["value", secret]])
.toString()
.slice("value=".length);
socketListeners.get("framesent")?.({ payload: formEncodedSecret });
const items = events.result().items;
expect(items).toMatchObject([
{ kind: "console", text: "[REDACTED:login_input]" },
{
kind: "page-error",
message: "failure [REDACTED:login_input]",
stack: "failure [REDACTED:login_input]",
},
{ kind: "websocket-opened" },
{
kind: "websocket-frame-sent",
payload_text: "[REDACTED:login_input]",
},
]);
expect(secrets.fingerprint(`stored ${encodeURIComponent(secret)}`)).toEqual(
{ value_state: "redacted-secret", value_sha256: null },
);
});
});
describe("PlaywrightScenarioEvents", () => {
it("observes requested popup page errors independently of popup lifecycle events", () => {
const listeners = new Map<string, (value: unknown) => void>();
@ -42,7 +113,7 @@ describe("PlaywrightScenarioEvents", () => {
]);
});
it("bounds oversized page errors before validating captured events", () => {
it("retains oversized page errors without truncating their text", () => {
const listeners = new Map<string, (value: Error) => void>();
const page = {
url: () => "https://app.example.test/",
@ -88,7 +159,7 @@ describe("PlaywrightScenarioEvents", () => {
const error = new Error("m".repeat(70_000));
error.stack = "s".repeat(300_000);
expect(() => listeners.get("pageerror")?.(error)).not.toThrow();
listeners.get("pageerror")?.(error);
expect(events.result().items[0]).toMatchObject({
kind: "page-error",
message: "m".repeat(70_000),

View File

@ -139,7 +139,7 @@ export const SESSION_TOOL_CONTRACTS = [
),
session(
"list_unknowns",
"List every current residual-unknown head in deterministic ID order, with optional exact status, severity, and domain filters. Results are complete and inline. This is read-only; unresolved, contradicted, and non-truth dispositions remain distinct.",
"List current residual-unknown heads in deterministic ID order, with optional exact status, severity, and domain filters. Results are complete inline by default; supply offset and limit to page the filtered order. total is the matching-head count and next_offset is null when the page exhausts the result. This is read-only; unresolved, contradicted, and non-truth dispositions remain distinct.",
listUnknownsInputSchema,
),
session(

View File

@ -52,6 +52,22 @@ export const listUnknownsInputSchema = z.strictObject({
.optional(),
severity: z.enum(["low", "medium", "high", "critical"]).optional(),
domain: z.string().trim().min(1).optional(),
offset: z
.number()
.int()
.min(0)
.max(Number.MAX_SAFE_INTEGER)
.default(0)
.describe("Zero-based offset in the stable filtered unknown ID order."),
limit: z
.number()
.int()
.positive()
.max(Number.MAX_SAFE_INTEGER)
.optional()
.describe(
"Optional maximum number of heads to return. Omit to return every matching head from offset.",
),
});
/** Exact residual-unknown identity to revalidate. */

View File

@ -555,6 +555,8 @@ export const sessionOutputSchemas = {
z.object({
items: z.array(residualUnknownSchema),
total: z.number().int().min(0),
offset: z.number().int().min(0),
next_offset: z.number().int().min(0).nullable(),
}),
),
record_unknown: lifecycleResultOf(residualUnknownSchema),

View File

@ -2,7 +2,7 @@ import type { JsonValue } from "../domain/jsonValue.js";
/** Exact example inputs for residual-unknown lifecycle contracts. */
export const UNKNOWN_CONTRACT_EXAMPLES = {
list_unknowns: {},
list_unknowns: { offset: 0, limit: 50 },
record_unknown: {
question: "Does this branch require an unavailable external service?",
severity: "medium",

View File

@ -1,6 +1,9 @@
import { z } from "zod";
import { parseArtifactInventoryEvidence } from "../artifactInventoryEvidence.js";
import {
hasCompleteInventoryCoverage,
parseArtifactInventoryEvidence,
} from "../artifactInventoryEvidence.js";
import { digestSchema } from "../digests.js";
import { prefixedDigestSchema } from "../digests.js";
import { nativeRuntimeConvention } from "../nativeConvention.js";
@ -62,6 +65,7 @@ export const androidApplicationProjectionResultSchema = z.strictObject({
bridge_candidate_coverage: bridgeCandidateCoverageSchema,
coverage: z.strictObject({
status: z.enum(["complete-within-inventory", "partial"]),
// Count completeness is separate from the declared coverage status.
inventory_complete: z.boolean(),
}),
limitations: z.array(z.string().min(1)),
@ -95,8 +99,8 @@ export const projectAndroidApplication = (
);
const components = classified;
const limitations = [
...(!inventory.complete
? ["Source inventory pages are incomplete; absence is unknown."]
...(!hasCompleteInventoryCoverage(inventory)
? ["Source inventory coverage is partial; absence is unknown."]
: []),
"Manifest, resource, signing, and bytecode semantics require a dedicated Android provider; this projection reports exact inventory paths and hashes only.",
"Runtime families are inferred from inventory formats and paths; filename suffixes do not establish valid DEX or JVM class bytes.",

View File

@ -138,6 +138,7 @@ const inventoryEvidence = (
occurrences: sortedOccurrences,
edges: [],
provenance: [],
coverage_status: "complete",
limitations: [],
}),
},

View File

@ -17,7 +17,10 @@ import {
isSwiftRuntimeLibrary,
nativeRuntimeConvention,
} from "../nativeConvention.js";
import { parseArtifactInventoryEvidence } from "../artifactInventoryEvidence.js";
import {
hasCompleteInventoryCoverage,
parseArtifactInventoryEvidence,
} from "../artifactInventoryEvidence.js";
import { evidenceSchema } from "../evidence.js";
import { digestSchema } from "../digests.js";
import { prefixedDigestSchema } from "../digests.js";
@ -88,6 +91,7 @@ export const appleApplicationProjectionResultSchema = z.strictObject({
bridge_candidate_coverage: bridgeCandidateCoverageSchema,
coverage: z.strictObject({
status: z.enum(["complete-within-inventory", "partial"]),
// Count completeness is separate from the declared coverage status.
inventory_complete: z.boolean(),
}),
limitations: z.array(z.string().min(1)),
@ -165,9 +169,10 @@ export const projectAppleApplication = (
);
const uninventoriedImage =
rootFormat === "dmg" && inventory.occurrences.length === 1;
const complete = inventory.complete && !uninventoriedImage;
const inventoryComplete = hasCompleteInventoryCoverage(inventory);
const complete = inventoryComplete && !uninventoriedImage;
const limitations = projectionLimitations({
complete: inventory.complete,
complete: inventoryComplete,
uninventoriedImage,
roots,
bundles,
@ -390,7 +395,7 @@ const projectionLimitations = (facts: {
readonly bridgeCandidateCoverage: AppleApplicationProjectionResult["bridge_candidate_coverage"];
}): string[] => [
...(!facts.complete
? ["Source inventory pages are incomplete; absence is unknown."]
? ["Source inventory coverage is partial; absence is unknown."]
: []),
...(facts.uninventoriedImage
? [

View File

@ -8,6 +8,7 @@ import type {
} from "./artifactGraph.js";
import { evidenceSchema } from "./evidence.js";
import {
hasCompleteInventoryCoverage,
parseArtifactInventoryEvidence,
type InventorySet,
} from "./artifactInventoryEvidence.js";
@ -103,10 +104,8 @@ export const compareArtifacts = (
const right = parseArtifactInventoryEvidence(rightEvidence);
const leftComplete = left.inventory.complete;
const rightComplete = right.inventory.complete;
const leftCovered =
leftComplete && left.inventory.coverage_status === "complete";
const rightCovered =
rightComplete && right.inventory.coverage_status === "complete";
const leftCovered = hasCompleteInventoryCoverage(left.inventory);
const rightCovered = hasCompleteInventoryCoverage(right.inventory);
const limitations = [
...left.inventory.limitations.map((item) => `Left: ${item}`),
...right.inventory.limitations.map((item) => `Right: ${item}`),

View File

@ -28,10 +28,16 @@ export interface InventorySet {
readonly edges: readonly ArtifactInventoryResult["edges"][number][];
readonly integrityContradictions: readonly IntegrityContradiction[];
readonly limitations: readonly string[];
/** True when merged node, occurrence, and edge counts match the manifest. */
readonly complete: boolean;
readonly coverage_status: ArtifactInventoryCoverage;
}
/** Whether all graph members are present and the producer reports full coverage. */
export const hasCompleteInventoryCoverage = (
inventory: InventorySet,
): boolean => inventory.complete && inventory.coverage_status === "complete";
export interface ParsedInventorySet {
readonly evidence: readonly Evidence[];
readonly inventory: InventorySet;

View File

@ -5,9 +5,16 @@ export const elf = (
machine: number,
): Buffer => {
const bytes = Buffer.alloc(class_ === 1 ? 52 : 64);
bytes.set([0x7f, 0x45, 0x4c, 0x46, class_, endian]);
if (endian === 1) bytes.writeUInt16LE(machine, 18);
else bytes.writeUInt16BE(machine, 18);
bytes.set([0x7f, 0x45, 0x4c, 0x46, class_, endian, 1]);
if (endian === 1) {
bytes.writeUInt16LE(machine, 18);
bytes.writeUInt32LE(1, 20);
bytes.writeUInt16LE(class_ === 1 ? 52 : 64, class_ === 1 ? 40 : 52);
} else {
bytes.writeUInt16BE(machine, 18);
bytes.writeUInt32BE(1, 20);
bytes.writeUInt16BE(class_ === 1 ? 52 : 64, class_ === 1 ? 40 : 52);
}
return bytes;
};

View File

@ -7,10 +7,14 @@ const header = (little: boolean, bits: 32 | 64 = 32): Buffer => {
if (little) {
bytes.writeUInt16LE(2, 16);
bytes.writeUInt16LE(8, 18);
bytes.writeUInt32LE(1, 20);
bytes.writeUInt16LE(bits === 32 ? 52 : 64, bits === 32 ? 40 : 52);
bytes.writeUInt32LE(0x70001001, bits === 32 ? 36 : 48);
} else {
bytes.writeUInt16BE(2, 16);
bytes.writeUInt16BE(8, 18);
bytes.writeUInt32BE(1, 20);
bytes.writeUInt16BE(bits === 32 ? 52 : 64, bits === 32 ? 40 : 52);
bytes.writeUInt32BE(0x70001001, bits === 32 ? 36 : 48);
}
return bytes;

View File

@ -13,10 +13,14 @@ const header = (
if (little) {
bytes.writeUInt16LE(type, 16);
bytes.writeUInt16LE(machine, 18);
bytes.writeUInt32LE(1, 20);
bytes.writeUInt16LE(bits === 32 ? 52 : 64, bits === 32 ? 40 : 52);
bytes.writeUInt32LE(flags, bits === 32 ? 36 : 48);
} else {
bytes.writeUInt16BE(type, 16);
bytes.writeUInt16BE(machine, 18);
bytes.writeUInt32BE(1, 20);
bytes.writeUInt16BE(bits === 32 ? 52 : 64, bits === 32 ? 40 : 52);
bytes.writeUInt32BE(flags, bits === 32 ? 36 : 48);
}
return bytes;

View File

@ -26,6 +26,13 @@ describe("executable header parsing", () => {
expect(result.ok && result.value).toMatchObject({ format, architecture });
});
it("retains ELF32 x86-64 identity for the x32 ABI", () => {
expect(parseExecutableHeader(elf(1, 1, 62), "x64")).toMatchObject({
ok: true,
value: { format: "elf", architecture: "x86_64" },
});
});
it("distinguishes native PE applications, DLLs, and managed assemblies", () => {
const application = parseExecutableHeader(pe(0x8664), "x64");
const library = parseExecutableHeader(pe(0x8664, 64, 0x2002), "x64");

View File

@ -7,6 +7,7 @@ import type {
UniversalMachOSlice,
} from "./binaryTargetTypes.js";
import { err, ok, type Result } from "./result.js";
import { readElfFixedHeader } from "./elfFixedHeader.js";
import { observeMzWindowsHeader, parseDosMzHeader } from "./dosMz.js";
import { decodePeMachine } from "./peMachine.js";
import {
@ -191,20 +192,14 @@ const parseElf = (
bytes: Buffer,
fileSize: number,
): Result<ExecutableMetadata, string> => {
if (bytes.length < 6) return err("truncated ELF header");
if (bytes[4] !== 1 && bytes[4] !== 2) return err("unsupported ELF class");
const little = bytes[5] === 1;
if (!little && bytes[5] !== 2) return err("unsupported ELF endianness");
const headerSize = bytes[4] === 1 ? 52 : 64;
if (bytes.length < headerSize)
return err(
`truncated ELF header: an ELF${bytes[4] === 1 ? 32 : 64} header needs ${headerSize} bytes; the file has ${fileSize}`,
);
const fixedHeader = readElfFixedHeader(bytes, fileSize);
if (!fixedHeader.ok) return fixedHeader;
const { elfClass, byteOrder } = fixedHeader.value;
const little = byteOrder === "little";
const machine = little ? bytes.readUInt16LE(18) : bytes.readUInt16BE(18);
const architecture = elfArchitecture(machine);
if (architecture === undefined) return err("unsupported ELF architecture");
if (architecture === "mips" || architecture === "riscv") {
const elfClass = bytes[4] === 1 ? 32 : 64;
const flagsOffset = elfClass === 32 ? 36 : 48;
const metadata: ElfFamilyMetadata = {
elfClass,

View File

@ -0,0 +1,61 @@
import { expect, it } from "vitest";
import { readElfFixedHeader } from "./elfFixedHeader.js";
import { elf } from "./binaryTarget.fixture.js";
it.each([
["ELF32 little endian", 1, 1],
["ELF32 big endian", 1, 2],
["ELF64 little endian", 2, 1],
["ELF64 big endian", 2, 2],
] as const)(
"reads common fixed header commitments for %s",
(_name, cls, data) => {
const bytes = elf(cls, data, cls === 1 ? 3 : 62);
expect(readElfFixedHeader(bytes)).toEqual({
ok: true,
value: {
elfClass: cls === 1 ? 32 : 64,
byteOrder: data === 1 ? "little" : "big",
headerSize: cls === 1 ? 52 : 64,
},
});
},
);
it.each([
[
"identification version",
(bytes: Buffer) => {
bytes[6] = 0;
},
],
[
"header version",
(bytes: Buffer) => {
bytes.writeUInt32LE(0, 20);
},
],
[
"undersized declared header",
(bytes: Buffer) => {
bytes.writeUInt16LE(51, 40);
},
],
[
"declared header beyond file",
(bytes: Buffer) => {
bytes.writeUInt16LE(53, 40);
},
],
])("rejects an invalid %s", (_name, corrupt) => {
const bytes = elf(1, 1, 3);
corrupt(bytes);
expect(readElfFixedHeader(bytes)).toMatchObject({ ok: false });
});
it("checks declared header size against the whole file, not the prefix", () => {
const bytes = elf(2, 1, 62);
bytes.writeUInt16LE(128, 52);
expect(readElfFixedHeader(bytes, 128)).toMatchObject({ ok: true });
expect(readElfFixedHeader(bytes, 127)).toMatchObject({ ok: false });
});

View File

@ -0,0 +1,55 @@
import { err, ok, type Result } from "./result.js";
export interface ElfFixedHeader {
readonly elfClass: 32 | 64;
readonly byteOrder: "little" | "big";
readonly headerSize: number;
}
/**
* Validate the common, fixed ELF header commitments.
*
* `bytes` may be a bounded prefix. `totalBytes` is the containing file size,
* so the declared header extent is checked against the file without requiring
* the whole declared extent to be present in this prefix.
*/
export const readElfFixedHeader = (
bytes: Buffer,
totalBytes = bytes.length,
): Result<ElfFixedHeader, string> => {
if (bytes.length >= 4 && bytes.toString("hex", 0, 4) !== "7f454c46")
return err("not an ELF image");
if (bytes.length < 16) return err("ELF identification is truncated");
const classByte = bytes[4];
if (classByte !== 1 && classByte !== 2) return err("unsupported ELF class");
const dataByte = bytes[5];
if (dataByte !== 1 && dataByte !== 2)
return err("unsupported ELF endianness");
if (bytes[6] !== 1) return err("unsupported ELF identification version");
const elfClass = classByte === 1 ? 32 : 64;
const minimumSize = elfClass === 32 ? 52 : 64;
if (
!Number.isSafeInteger(totalBytes) ||
totalBytes < bytes.length ||
totalBytes < minimumSize ||
bytes.length < minimumSize
)
return err(
`truncated ELF header: an ELF${elfClass} header needs ${minimumSize} bytes; the file has ${totalBytes}`,
);
const littleEndian = dataByte === 1;
const readU16 = (offset: number): number =>
littleEndian ? bytes.readUInt16LE(offset) : bytes.readUInt16BE(offset);
const readU32 = (offset: number): number =>
littleEndian ? bytes.readUInt32LE(offset) : bytes.readUInt32BE(offset);
if (readU32(20) !== 1) return err("unsupported ELF header version");
const headerSize = readU16(elfClass === 32 ? 40 : 52);
if (headerSize < minimumSize || headerSize > totalBytes)
return err("invalid ELF header size");
return ok({
elfClass,
byteOrder: littleEndian ? "little" : "big",
headerSize,
});
};

View File

@ -1,6 +1,9 @@
import { z } from "zod";
import { parseArtifactInventoryEvidence } from "../artifactInventoryEvidence.js";
import {
hasCompleteInventoryCoverage,
parseArtifactInventoryEvidence,
} from "../artifactInventoryEvidence.js";
import { digestSchema } from "../digests.js";
import { prefixedDigestSchema } from "../digests.js";
import { nativeRuntimeConvention } from "../nativeConvention.js";
@ -66,6 +69,7 @@ export const harmonyApplicationProjectionResultSchema = z.strictObject({
bridge_candidate_coverage: bridgeCandidateCoverageSchema,
coverage: z.strictObject({
status: z.enum(["complete-within-inventory", "partial"]),
// Count completeness is separate from the declared coverage status.
inventory_complete: z.boolean(),
}),
limitations: z.array(z.string().min(1)),
@ -108,8 +112,8 @@ export const projectHarmonyApplication = (
classified.native_libraries,
);
const limitations = [
...(!inventory.complete
? ["Source inventory pages are incomplete; absence is unknown."]
...(!hasCompleteInventoryCoverage(inventory)
? ["Source inventory coverage is partial; absence is unknown."]
: []),
...(unavailable.length > 0
? [

View File

@ -3,7 +3,10 @@ import { createHash } from "node:crypto";
import canonicalize from "canonicalize";
import { z } from "zod";
import type { ParsedInventorySet } from "./artifactInventoryEvidence.js";
import {
hasCompleteInventoryCoverage,
type ParsedInventorySet,
} from "./artifactInventoryEvidence.js";
import { evidenceSchema } from "./evidence.js";
import { digestSchema } from "./digests.js";
import { prefixedDigestSchema } from "./digests.js";
@ -49,10 +52,11 @@ export const projectionCoverage = (
budgetComplete: boolean,
): {
status: "complete-within-inventory" | "partial";
/** True when all manifest graph-member counts are present, independent of source coverage. */
inventory_complete: boolean;
} => ({
status:
inventory.complete && budgetComplete
hasCompleteInventoryCoverage(inventory) && budgetComplete
? ("complete-within-inventory" as const)
: ("partial" as const),
inventory_complete: inventory.complete,

View File

@ -61,6 +61,13 @@ it.each([
functions: [{ selector: "aabbccdd", bytecodeOffset: -1, dispatch: "abi" }],
},
{ metadata: {} },
{
metadata: {
bytecodeOffset: 1,
cborLength: 4,
entries: [{ key: "reported", value: { type: "integer", value: 1.5 } }],
},
},
])("rejects producer representations outside the selected profile", (raw) => {
expect(() => projectEvmoleInterface(raw)).toThrow();
});

View File

@ -9,7 +9,11 @@ const reportedMetadata = z.strictObject({
key: z.string(),
value: z.discriminatedUnion("type", [
z.strictObject({ type: z.literal("string"), value: z.string() }),
z.strictObject({ type: z.literal("integer"), value: z.number() }),
z.strictObject({
type: z.literal("integer"),
// Reported integers can exceed the safe range; fractions still violate the tag.
value: z.number().refine(Number.isInteger, "Expected an integer"),
}),
z.strictObject({ type: z.literal("bytes"), value: z.string() }),
z.strictObject({ type: z.literal("bool"), value: z.boolean() }),
z.strictObject({ type: z.literal("undecoded"), value: z.string() }),

View File

@ -5,6 +5,7 @@ import {
SNAPSHOT_HASH_OFFSET,
SNAPSHOT_HASH_PATTERN,
} from "./FlutterPayloadScan.js";
import { readElfFixedHeader } from "../domain/elfFixedHeader.js";
/** Bounds-checked little-endian reader over one loaded buffer. */
class BoundedReader {
@ -80,24 +81,13 @@ export const readElfHeader = (
totalBytes: number = bytes.length,
) => {
const reader = new BoundedReader(bytes);
const magic = reader.u32(0);
if (magic !== 0x464c457f) return { failure: { reason: "not an ELF image" } };
const elfClass = reader.u8(4);
if (elfClass !== 1 && elfClass !== 2)
return { failure: { reason: `unsupported ELF class ${String(elfClass)}` } };
const is64 = elfClass === 2;
const fixedHeader = readElfFixedHeader(bytes, totalBytes);
if (!fixedHeader.ok) return { failure: { reason: fixedHeader.error } };
const is64 = fixedHeader.value.elfClass === 64;
const headerBytes = is64 ? ELF_HEADER_MAX_BYTES : 52;
if (
bytes.length < headerBytes ||
!Number.isSafeInteger(totalBytes) ||
totalBytes < headerBytes
)
return { failure: { reason: "ELF header is truncated" } };
if (reader.u8(5) !== 1)
if (fixedHeader.value.byteOrder !== "little")
return { failure: { reason: "unsupported ELF byte order" } };
if (reader.u8(6) !== 1 || reader.u32(0x14) !== 1)
return { failure: { reason: "unsupported ELF version" } };
if (reader.u16(is64 ? 0x34 : 0x28) !== headerBytes)
if (fixedHeader.value.headerSize !== headerBytes)
return { failure: { reason: "unsupported ELF header size" } };
const e_type = reader.u16(0x10);
if (e_type !== 1 && e_type !== 2 && e_type !== ET_DYN)

View File

@ -6,31 +6,19 @@ import {
type GoFileMapping,
type GoZeroFillMapping,
} from "./GoBinaryContainer.js";
import { readElfFixedHeader } from "../domain/elfFixedHeader.js";
/** Read ELF build-info sections and file-backed program mappings. */
export const readGoElfImage = (bytes: Buffer): GoBinaryContainer => {
if (bytes.length < 16)
throw new GoBinaryFormatFailure(
"malformed",
"ELF identification is truncated",
);
const bits = bytes[4] === 1 ? 32 : bytes[4] === 2 ? 64 : null;
if (bits === null || ![1, 2].includes(bytes[5] ?? 0) || bytes[6] !== 1)
throw new GoBinaryFormatFailure(
"malformed",
"ELF class, byte order, or version is invalid",
);
const reader = new GoBinaryReader(bytes, bytes[5] === 1);
const fixedHeader = readElfFixedHeader(bytes);
if (!fixedHeader.ok)
throw new GoBinaryFormatFailure("malformed", fixedHeader.error);
const { elfClass: bits, headerSize } = fixedHeader.value;
const reader = new GoBinaryReader(
bytes,
fixedHeader.value.byteOrder === "little",
);
const wide = bits === 64;
const minimum = wide ? 64 : 52;
reader.range(0, minimum, "ELF header");
if (reader.u32(20) !== 1 || reader.u16(wide ? 52 : 40) < minimum)
throw new GoBinaryFormatFailure(
"malformed",
"ELF header version or size is invalid",
);
const headerSize = reader.u16(wide ? 52 : 40);
reader.range(0, headerSize, "ELF declared header");
const phOffset = reader.word(wide ? 32 : 28, bits);
const shOffset = reader.word(wide ? 40 : 32, bits);
const phWidth = reader.u16(wide ? 54 : 42);

View File

@ -84,16 +84,24 @@ import {
inspectNativeMacho,
} from "./NativeMachoInspection.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import type { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
import {
createNativeUiHelperRuntime,
type NativeUiHelperRuntime,
} from "./NativeUiHelperRuntime.js";
type SignatureRuntimeRootFactory = () => Promise<
Pick<PrivateRuntimeRoot, "path" | "close">
>;
/** Read-only semantic provider composed from Xcode command-line utilities. */
export class NativeMacOSProvider implements AnalysisProvider {
readonly #capabilities: readonly CapabilityDescriptor[];
readonly #environment: NodeJS.ProcessEnv;
readonly #tracerFactory: () => NativeCallTracer;
readonly #privateRuntimeRootFactory: SignatureRuntimeRootFactory | undefined;
constructor(
environment: Readonly<NodeJS.ProcessEnv>,
@ -102,10 +110,12 @@ export class NativeMacOSProvider implements AnalysisProvider {
),
platform: NodeJS.Platform = process.platform,
tracerFactory?: () => NativeCallTracer,
privateRuntimeRootFactory?: SignatureRuntimeRootFactory,
) {
this.#environment = snapshotEnvironment(environment, platform);
this.#tracerFactory =
tracerFactory ?? (() => new LldbCallTracer(this.#environment));
this.#privateRuntimeRootFactory = privateRuntimeRootFactory;
this.#capabilities = nativeMacOSCapabilities(platform);
}
@ -123,6 +133,7 @@ export class NativeMacOSProvider implements AnalysisProvider {
this.runner,
this.#tracerFactory(),
this.#environment,
this.#privateRuntimeRootFactory,
);
}
}
@ -130,6 +141,7 @@ export class NativeMacOSProvider implements AnalysisProvider {
class NativeMacOSClient implements AnalysisClient {
readonly #pendingNativeUiRuntimes = new Set<NativeUiHelperRuntime>();
readonly #pendingCommands = new Set<Promise<unknown>>();
readonly #signatureResources = new ArtifactResourceScope();
#nativeUiTail: Promise<void> = Promise.resolve();
#closed = false;
@ -138,6 +150,9 @@ class NativeMacOSClient implements AnalysisClient {
private readonly runner: NativeCommandRunner,
private readonly tracer: NativeCallTracer,
private readonly environment: Readonly<NodeJS.ProcessEnv>,
private readonly privateRuntimeRootFactory:
| SignatureRuntimeRootFactory
| undefined,
) {}
async execute(
@ -270,6 +285,21 @@ class NativeMacOSClient implements AnalysisClient {
const uiCleanupFailures = await this.#serializeNativeUi(() =>
this.#retryNativeUiCleanup(),
);
let signatureCleanup: AnalysisError | undefined;
try {
await this.#signatureResources.close();
} catch (cause: unknown) {
const observation = ArtifactReaderFailure.cleanupObservation(
cause,
"signature-snapshot",
);
signatureCleanup = new ProviderCleanupError(
IDENTITY.id,
observation.resources,
{ reason: observation.reason },
{ operation: "close_binary", cause },
);
}
const commandCleanup = await this.#retryNativeCommandCleanup();
let result = await this.tracer.close();
const cleanupErrors = [
@ -277,6 +307,7 @@ class NativeMacOSClient implements AnalysisClient {
? []
: [this.#nativeUiCleanupFailure(uiCleanupFailures, "close_binary")]),
...(commandCleanup === undefined ? [] : [commandCleanup]),
...(signatureCleanup === undefined ? [] : [signatureCleanup]),
];
for (const cleanupError of cleanupErrors)
result = result.ok
@ -532,8 +563,10 @@ class NativeMacOSClient implements AnalysisClient {
): Promise<Result<NativeObservation, AnalysisError>> {
return withSignatureTarget(
this.target,
this.#signatureResources,
(binding) => this.#inspectBoundSignature(binding, signal),
signal,
this.privateRuntimeRootFactory,
);
}

View File

@ -0,0 +1,198 @@
import { createHash } from "node:crypto";
import { mkdtemp, rm, stat, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, expect, it } from "vitest";
import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { ok } from "../domain/result.js";
import type { NativeCommandRunner } from "./CommandRunner.js";
import type { NativeCallTracer } from "./LldbCallTracer.js";
import { NativeMacOSProvider } from "./NativeMacOSProvider.js";
import { withSignatureTarget } from "./SignatureTargetBinding.js";
const roots: string[] = [];
const targets: string[] = [];
const unusedRunner: NativeCommandRunner = {
async run() {
throw new Error("Changed target must be rejected before a command runs");
},
};
const unusedTracer: NativeCallTracer = {
async trace() {
throw new Error("Signature inspection must not start a trace");
},
async close() {
return ok(null);
},
};
afterEach(async () => {
await Promise.all(
[...roots, ...targets].map((path) =>
rm(path, { recursive: true, force: true }),
),
);
roots.length = 0;
targets.length = 0;
});
const createTarget = async (
bytes: string,
digest = bytes,
): Promise<BinaryTarget> => {
const directory = await mkdtemp(join(tmpdir(), "rea-signature-target-"));
targets.push(directory);
const path = join(directory, "target.bin");
await writeFile(path, bytes);
return {
path,
sha256: createHash("sha256").update(digest).digest("hex"),
kind: "executable",
format: "mach-o",
architecture: "arm64",
availableArchitectures: ["arm64"],
};
};
const createRoot = async (failedAttempts = 0) => {
const path = await mkdtemp(join(tmpdir(), "rea-signature-root-"));
roots.push(path);
let attempts = 0;
return {
path,
async close() {
if (attempts++ < failedAttempts) {
throw new Error("injected root cleanup failure");
}
await rm(path, { recursive: true, force: true });
},
};
};
it("retains a snapshot root after cleanup fails alongside a digest mismatch", async () => {
const target = await createTarget("observed", "expected");
const resources = new ArtifactResourceScope();
const result = await withSignatureTarget(
target,
resources,
async () => ok("unused"),
undefined,
() => createRoot(1),
);
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.cleanupIncomplete).toBe(true);
expect(result.error.cleanupResources).toEqual(expect.arrayContaining(roots));
expect(projectAnalysisError(result.error)).toMatchObject({
details: { diagnostics: { primary_error: { code: "artifact_changed" } } },
});
await resources.close();
await expect(stat(roots[0]!)).rejects.toMatchObject({ code: "ENOENT" });
});
it("returns a retained cleanup failure when the next call cannot retry its root", async () => {
const target = await createTarget("observed", "expected");
const resources = new ArtifactResourceScope();
const createFailingRoot = async () => {
const path = await mkdtemp(join(tmpdir(), "rea-signature-root-sticky-"));
roots.push(path);
return {
path,
close: async () => {
throw new Error("still blocked");
},
};
};
const first = await withSignatureTarget(
target,
resources,
async () => ok("unused"),
undefined,
createFailingRoot,
);
expect(first.ok).toBe(false);
const second = await withSignatureTarget(
target,
resources,
async () => ok("must not run"),
undefined,
createRoot,
);
expect(second.ok).toBe(false);
if (second.ok) return;
expect(second.error.cleanupIncomplete).toBe(true);
expect(second.error.cleanupResources).toEqual(expect.arrayContaining(roots));
});
it("retries a retained signature root when the provider client closes", async () => {
const target = await createTarget("observed", "expected");
const provider = new NativeMacOSProvider(
{},
unusedRunner,
"darwin",
() => unusedTracer,
() => createRoot(2),
);
const client = provider.createClient(target);
const result = await client.execute("inspect_signature", {});
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.cleanupIncomplete).toBe(true);
expect(result.error.cleanupResources).toEqual(expect.arrayContaining(roots));
const firstClose = await client.close();
expect(firstClose).toMatchObject({
ok: false,
error: { cleanupIncomplete: true, cleanupResources: roots },
});
expect((await stat(roots[0]!)).isDirectory()).toBe(true);
await expect(client.close()).resolves.toMatchObject({
ok: true,
value: null,
});
await expect(stat(roots[0]!)).rejects.toMatchObject({ code: "ENOENT" });
});
it("waits for an in-flight signature binding before provider close", async () => {
const target = await createTarget("observed", "expected");
let rootFactoryEntered!: () => void;
let releaseRootFactory!: () => void;
const entered = new Promise<void>(
(resolve) => (rootFactoryEntered = resolve),
);
const gate = new Promise<void>((resolve) => (releaseRootFactory = resolve));
const provider = new NativeMacOSProvider(
{},
unusedRunner,
"darwin",
() => unusedTracer,
async () => {
rootFactoryEntered();
await gate;
return createRoot();
},
);
const client = provider.createClient(target);
const operation = client.execute("inspect_signature", {});
await entered;
let closeSettled = false;
const closing = client.close().finally(() => {
closeSettled = true;
});
await Promise.resolve();
expect(closeSettled).toBe(false);
releaseRootFactory();
await expect(operation).resolves.toMatchObject({
ok: false,
error: { _tag: "AnalysisArtifactChangedError" },
});
await expect(closing).resolves.toMatchObject({ ok: true, value: null });
expect(closeSettled).toBe(true);
await expect(stat(roots[0]!)).rejects.toMatchObject({ code: "ENOENT" });
});

View File

@ -1,7 +1,6 @@
import { createHash } from "node:crypto";
import { constants, type BigIntStats } from "node:fs";
import { lstat, mkdtemp, open, rm, type FileHandle } from "node:fs/promises";
import { tmpdir } from "node:os";
import { lstat, open, type FileHandle } from "node:fs/promises";
import { basename, join } from "node:path";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import { AnalysisError } from "../domain/analysisErrorBase.js";
@ -9,10 +8,17 @@ import {
AnalysisAccessDeniedError,
AnalysisArtifactChangedError,
AnalysisCancelledError,
AnalysisOutputError,
AnalysisResourceConstraintError,
} from "../domain/analysisErrorCore.js";
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { err, ok, type Result } from "../domain/result.js";
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import { NATIVE_MACOS_PROVIDER_IDENTITY } from "./NativeMacOSProviderMetadata.js";
/** Binding of commands to the selected executable's registered content/version. */
@ -68,41 +74,126 @@ const signatureReadFailure = (
/** Inspect a digest-verified private copy, releasing it on every exit path. */
export const withSignatureTarget = async <T>(
target: BinaryTarget,
resources: ArtifactResourceScope,
inspect: (
binding: SignatureTargetBinding,
) => Promise<Result<T, AnalysisError>>,
signal?: AbortSignal,
createRoot: () => Promise<Pick<PrivateRuntimeRoot, "path" | "close">> = () =>
PrivateRuntimeRoot.create({ prefix: "rea-signature-" }),
): Promise<Result<T, AnalysisError>> => {
let directory: string;
try {
signal?.throwIfAborted();
directory = await mkdtemp(join(tmpdir(), "rea-signature-"));
return await resources.run(async () => {
let root: Pick<PrivateRuntimeRoot, "path" | "close">;
try {
signal?.throwIfAborted();
root = await createRoot();
} catch (cause: unknown) {
return err(
signal?.aborted
? new AnalysisCancelledError("inspect_signature")
: snapshotFailure(target, "create", cause),
);
}
const owner = {
kind: "private-runtime-root" as const,
root,
resource: root.path,
};
let result: Result<T, AnalysisError>;
try {
const binding = await bindSignatureTarget(
target,
join(root.path, basename(target.path)),
resources,
signal,
);
if (!binding.ok) result = binding;
else {
try {
result = await inspect(binding.value);
} catch (cause: unknown) {
result = err(
cause instanceof AnalysisError
? cause
: new AnalysisOutputError(
"inspect_signature",
"Native output parsing failed",
{ cause },
),
);
}
}
} catch (cause: unknown) {
result = err(
cause instanceof AnalysisError
? cause
: signatureReadFailure(target, cause),
);
}
const cleanup = await resources.release(owner);
if (cleanup.kind === "failed")
return combineCleanupFailure(
target,
result,
[root.path],
cleanup.cause,
"cleanup",
);
return result;
});
} catch (cause: unknown) {
if (cause instanceof AnalysisError) return err(cause);
const retained =
cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined
? cause.cleanup
: undefined;
if (retained !== undefined)
return combineCleanupFailure(
target,
err(
signal?.aborted
? new AnalysisCancelledError("inspect_signature")
: signatureReadFailure(target, cause),
),
retained.resources,
cause,
"retry",
);
return err(
signal?.aborted
? new AnalysisCancelledError("inspect_signature")
: snapshotFailure(target, "create", cause),
: signatureReadFailure(target, cause),
);
}
let result: Result<T, AnalysisError>;
let cleanup: Result<void, AnalysisError>;
try {
const binding = await bindSignatureTarget(
target,
join(directory, basename(target.path)),
signal,
);
result = binding.ok ? await inspect(binding.value) : binding;
} finally {
try {
await rm(directory, { recursive: true, force: true });
cleanup = ok(undefined);
} catch (cause: unknown) {
cleanup = err(snapshotFailure(target, "cleanup", cause));
}
}
// Cleanup cannot replace the operation's original actionable failure.
return result.ok && !cleanup.ok ? cleanup : result;
};
const combineCleanupFailure = <T>(
target: BinaryTarget,
primary: Result<T, AnalysisError>,
resources: readonly string[],
cause: unknown,
phase: string,
): Result<T, AnalysisError> => {
const cleanup = new ProviderCleanupError(
NATIVE_MACOS_PROVIDER_IDENTITY.id,
resources,
{
reason: cause instanceof Error ? cause.message : String(cause),
phase: `signature-snapshot-${phase}`,
path: target.path,
},
{ operation: "inspect_signature", cause },
);
return primary.ok
? err(cleanup)
: err(
analysisErrorWithCleanupFailure(
primary.error,
cleanup,
"inspect_signature",
),
);
};
const snapshotFailure = (
@ -180,22 +271,28 @@ const copySignatureBytes = async (
const bindSignatureTarget = async (
target: BinaryTarget,
snapshotPath: string,
resources: ArtifactResourceScope,
signal?: AbortSignal,
): Promise<Result<SignatureTargetBinding, AnalysisError>> => {
try {
signal?.throwIfAborted();
const before = await lstat(target.path, { bigint: true });
signal?.throwIfAborted();
if (!before.isFile() || before.isSymbolicLink())
return changedTarget(
target,
`Signature target is no longer a regular file: ${target.path}`,
);
const file = await open(
target.path,
constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK,
);
let sourceOwner: OwnedFileHandle | undefined;
let snapshotOwner: OwnedFileHandle | undefined;
const perform = async (): Promise<
Result<SignatureTargetBinding, AnalysisError>
> => {
try {
signal?.throwIfAborted();
const before = await lstat(target.path, { bigint: true });
signal?.throwIfAborted();
if (!before.isFile() || before.isSymbolicLink())
return changedTarget(
target,
`Signature target is no longer a regular file: ${target.path}`,
);
const file = await open(
target.path,
constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK,
);
sourceOwner = new OwnedFileHandle(file);
const opened = await file.stat({ bigint: true });
signal?.throwIfAborted();
if (identity(opened) !== identity(before))
@ -206,44 +303,71 @@ const bindSignatureTarget = async (
const snapshot = await snapshotIO(target, "create-file", () =>
open(snapshotPath, "wx", 0o600),
);
try {
const observed = await copySignatureBytes(
snapshotOwner = new OwnedFileHandle(snapshot);
const observed = await copySignatureBytes(
target,
{ file, stat: opened },
snapshot,
signal,
);
if (observed !== target.sha256)
return changedTarget(
target,
{ file, stat: opened },
snapshot,
signal,
`Signature target digest changed: ${target.path}; expected ${target.sha256}, observed ${observed}`,
);
if (observed !== target.sha256)
return changedTarget(
target,
`Signature target digest changed: ${target.path}; expected ${target.sha256}, observed ${observed}`,
);
const after = await file.stat({ bigint: true });
const current = await lstat(target.path, { bigint: true });
signal?.throwIfAborted();
if (
identity(after) !== identity(opened) ||
identity(current) !== identity(opened)
)
return changedTarget(
target,
`Signature target changed while establishing its version: ${target.path}`,
);
await snapshotIO(target, "seal", () => snapshot.chmod(0o400));
return ok({ identity: identity(opened), snapshotPath });
} finally {
await snapshotIO(target, "close", () => snapshot.close());
}
} finally {
await file.close();
const after = await file.stat({ bigint: true });
const current = await lstat(target.path, { bigint: true });
signal?.throwIfAborted();
if (
identity(after) !== identity(opened) ||
identity(current) !== identity(opened)
)
return changedTarget(
target,
`Signature target changed while establishing its version: ${target.path}`,
);
await snapshotIO(target, "seal", () => snapshot.chmod(0o400));
return ok({ identity: identity(opened), snapshotPath });
} catch (cause: unknown) {
return err(
signal?.aborted
? new AnalysisCancelledError("inspect_signature")
: signatureReadFailure(target, cause),
);
}
} catch (cause: unknown) {
return err(
signal?.aborted
? new AnalysisCancelledError("inspect_signature")
: signatureReadFailure(target, cause),
};
let result = await perform();
const cleanupFailures: {
readonly path: string;
readonly cause: unknown;
}[] = [];
for (const [owner, path] of [
[snapshotOwner, snapshotPath],
[sourceOwner, target.path],
] as const) {
if (owner === undefined) continue;
const cleanup = await resources.release({
kind: "file-handle",
handle: owner,
resource: path,
});
if (cleanup.kind === "failed")
cleanupFailures.push({ path, cause: cleanup.cause });
}
if (cleanupFailures.length > 0) {
const cause = new AggregateError(
cleanupFailures.map(({ cause: failure }) => failure),
"Signature snapshot file handles could not be closed",
);
result = combineCleanupFailure(
target,
result,
cleanupFailures.map(({ path }) => path),
cause,
"close",
);
}
return result;
};
/** Reject target drift across signature command captures. */

View File

@ -151,18 +151,14 @@ const registerListUnknownsTool = ({
listContract.name,
toolRegistrationOptions(listContract),
(input) => {
const all = session.listUnknowns({
const page = session.listUnknownsPage({
...(input.status === undefined ? {} : { status: input.status }),
...(input.severity === undefined ? {} : { severity: input.severity }),
...(input.domain === undefined ? {} : { domain: input.domain }),
offset: input.offset,
...(input.limit === undefined ? {} : { limit: input.limit }),
});
return server.delivery.toCallToolResult(
ok({
items: all,
total: all.length,
}),
listContract,
);
return server.delivery.toCallToolResult(ok(page), listContract);
},
);
};

View File

@ -0,0 +1,158 @@
import { createServer } from "node:http";
import type { RequestListener, Server } from "node:http";
import { afterEach, describe, expect, it } from "vitest";
import { readCdpJson } from "../../../src/browser/CdpEndpoint.js";
const servers: Server[] = [];
afterEach(async () => {
await Promise.all(
servers.splice(0).map(
(server) =>
new Promise<void>((resolve, reject) => {
server.closeAllConnections();
server.close((error) =>
error === undefined ? resolve() : reject(error),
);
}),
),
);
});
describe("CDP discovery response bounds", () => {
it("accepts a complete JSON response exactly at the byte limit", async () => {
const body = JSON.stringify("x".repeat(14));
expect(Buffer.byteLength(body)).toBe(16);
const server = await startServer((_request, response) =>
response.end(body),
);
await expect(
readCdpJson(server.url, "list_browser_targets", undefined, {
bytes: 16,
timeoutMs: 1_000,
}),
).resolves.toBe("x".repeat(14));
});
it("rejects and closes an oversized chunked response without parsing a prefix", async () => {
let responseClosed: (() => void) | undefined;
const closed = new Promise<void>((resolve) => {
responseClosed = resolve;
});
const server = await startServer((_request, response) => {
response.once("close", () => responseClosed?.());
response.write(JSON.stringify("x".repeat(15)));
});
await expect(
readCdpJson(server.url, "list_browser_targets", undefined, {
bytes: 16,
timeoutMs: 1_000,
}),
).rejects.toMatchObject({
_tag: "BrowserObservationError",
reason: "payload_limit",
});
await closed;
});
it("ends a response that exceeds the total elapsed deadline", async () => {
let responseClosed: (() => void) | undefined;
let requestStarted: (() => void) | undefined;
const closed = new Promise<void>((resolve) => {
responseClosed = resolve;
});
const started = new Promise<void>((resolve) => {
requestStarted = resolve;
});
const server = await startServer((_request, response) => {
response.once("close", () => responseClosed?.());
requestStarted?.();
});
const pending = readCdpJson(server.url, "list_browser_targets", undefined, {
bytes: 16,
timeoutMs: 50,
});
const timeoutAssertion = expect(pending).rejects.toMatchObject({
_tag: "BrowserObservationError",
reason: "timeout",
});
await started;
await timeoutAssertion;
await closed;
});
it("preserves caller cancellation and closes the response", async () => {
let responseClosed: (() => void) | undefined;
let requestStarted: (() => void) | undefined;
const closed = new Promise<void>((resolve) => {
responseClosed = resolve;
});
const started = new Promise<void>((resolve) => {
requestStarted = resolve;
});
const server = await startServer((_request, response) => {
response.once("close", () => responseClosed?.());
requestStarted?.();
});
const controller = new AbortController();
const pending = readCdpJson(
server.url,
"list_browser_targets",
controller.signal,
{
bytes: 16,
timeoutMs: 1_000,
},
);
await started;
controller.abort();
await expect(pending).rejects.toMatchObject({
_tag: "AnalysisCancelledError",
operation: "list_browser_targets",
});
await closed;
});
it("destroys an unsuccessful HTTP response", async () => {
let responseClosed: (() => void) | undefined;
const closed = new Promise<void>((resolve) => {
responseClosed = resolve;
});
const server = await startServer((_request, response) => {
response.once("close", () => responseClosed?.());
response.writeHead(503).flushHeaders();
});
await expect(
readCdpJson(server.url, "list_browser_targets", undefined, {
bytes: 16,
timeoutMs: 1_000,
}),
).rejects.toMatchObject({
_tag: "BrowserObservationError",
reason: "invalid_endpoint_response",
});
await closed;
});
});
const startServer = async (
handler: RequestListener,
): Promise<{ readonly url: URL }> => {
const server = createServer(handler);
servers.push(server);
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(0, "127.0.0.1", resolve);
});
const address = server.address();
if (address === null || typeof address === "string")
throw new TypeError("Expected a TCP listener address");
return { url: new URL(`http://127.0.0.1:${String(address.port)}`) };
};

View File

@ -227,6 +227,9 @@ describe("case-distinct APK resource names", () => {
const inventory = parseEvidence(
await runProviderAnalysis(path, "inventory_artifact", {}),
);
expect(inventory.normalized_result).toMatchObject({
coverage_status: "partial",
});
const result = projectAndroidApplicationEvidence({
inventory_evidence: [inventory],
});
@ -234,6 +237,13 @@ describe("case-distinct APK resource names", () => {
requireSuccessfulProjection(result).normalized_result,
);
expect(projection.root_format).toBe("apk");
expect(projection.coverage).toEqual({
status: "partial",
inventory_complete: true,
});
expect(projection.limitations).toContain(
"Source inventory coverage is partial; absence is unknown.",
);
expect(projection.components.manifests).toHaveLength(1);
expect(projection.components.dex).toHaveLength(1);
expect(JSON.stringify(inventory)).toContain("res/2F.xml");

View File

@ -75,6 +75,42 @@ async function createCompleteAppleProjection() {
}
describe("Apple application projection", () => {
it("keeps a case-colliding real IPA inventory partial", async () => {
const root = await createTestTempDirectory("rea-apple-case-collision-");
const path = join(root, "CaseCollision.ipa");
const writer = new ZipWriter(new Uint8ArrayWriter());
await writer.add(
"Payload/Example.app/Contents/Info.plist",
new TextReader("upper"),
);
await writer.add(
"Payload/Example.app/contents/Info.plist",
new TextReader("lower"),
);
await writeFile(path, await writer.close());
const inventory = parseEvidence(
await runProviderAnalysis(path, "inventory_artifact", {}),
);
expect(inventory.normalized_result).toMatchObject({
coverage_status: "partial",
});
const result = projectAppleApplicationEvidence({
inventory_evidence: [inventory],
});
const projection = appleApplicationProjectionResultSchema.parse(
requireSuccessfulProjection(result).normalized_result,
);
expect(projection.coverage).toEqual({
status: "partial",
inventory_complete: true,
});
expect(projection.limitations).toContain(
"Source inventory coverage is partial; absence is unknown.",
);
});
it("projects deterministic IPA components and bridge hypotheses from exact inventory Evidence", async () => {
const root = await createTestTempDirectory("rea-apple-");
const path = join(root, "Fixture.ipa");

View File

@ -172,6 +172,9 @@ describe("artifact inventory coverage", () => {
authority: "shipped-artifact",
},
);
expect(parseArtifactInventoryEvidence(partial).inventory.complete).toBe(
true,
);
expect(compareArtifacts(partial, partial)).toMatchObject({
status: "unknown",
summary: { unchanged: 0, unknown: 1 },

View File

@ -20,6 +20,7 @@ import {
} from "../../../src/application/AppBundleExecutable.js";
import {
dosMz,
elf,
fat,
pe,
thinMach,
@ -846,3 +847,40 @@ describe("DOS binary target I/O", () => {
});
});
});
it.each([
[
"EI_VERSION",
(bytes: Buffer) => {
bytes[6] = 0;
},
],
[
"e_version",
(bytes: Buffer) => {
bytes.writeUInt32LE(0, 20);
},
],
[
"e_ehsize",
(bytes: Buffer) => {
bytes.writeUInt16LE(51, 40);
},
],
])(
"rejects an ELF with malformed %s during public target opening",
async (_name, corrupt) => {
const directory = await createTestTempDirectory("rea-invalid-elf-header-");
const path = join(directory, "invalid.elf");
const bytes = elf(1, 1, 3);
corrupt(bytes);
await writeFile(path, bytes);
expect(await parseBinaryTarget(path)).toMatchObject({
ok: false,
error: {
_tag: "BinaryTargetError",
constraint: "file_format_rejected",
},
});
},
);

View File

@ -214,6 +214,34 @@ describe("HarmonyOS packaging boundaries", () => {
});
describe("HarmonyOS inventory coverage", () => {
it("keeps a case-colliding inventory partial in the application projection", async () => {
const path = await writeApplicationZip(
"rea-harmony-case-collision-",
"Fixture.hap",
[
{ path: "module.json", content: '{"module":{}}' },
{ path: "MODULE.JSON", content: '{"module":{}}' },
],
);
const inventory = await inventoryApplicationPackage(path);
expect(inventory.normalized_result).toMatchObject({
coverage_status: "partial",
});
const projection = requireDeterministicProjection(
projectHarmonyApplicationEvidence,
inventory,
harmonyApplicationProjectionResultSchema,
);
expect(projection.coverage).toEqual({
status: "partial",
inventory_complete: true,
});
expect(projection.limitations).toContain(
"Source inventory coverage is partial; absence is unknown.",
);
});
it.each(ZIP_PACKAGE_FORMATS)(
"retains archive identity and ZIP coverage for %s",
async (format) => {

View File

@ -227,11 +227,6 @@ it("rejects ELF header tables overlapping the native header", () => {
expect(() => readGoBinaryImage(fixture.bytes)).toThrowError(
/header|overlap/i,
);
const declaredSize = createGoBinaryFixture();
declaredSize.bytes.writeUInt16LE(8192, 52);
expect(() => readGoBinaryImage(declaredSize.bytes)).toThrowError(
/truncated|range/i,
);
});
it("rejects contradictory aliases when a metadata address maps to different file bytes", () => {

View File

@ -4,6 +4,7 @@ import type { CallToolResult } from "@modelcontextprotocol/server";
import { afterEach, expect, it } from "vitest";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { toolContract } from "../../../src/contracts/toolContracts.js";
import type {
AnalysisClient,
AnalysisProvider,
@ -249,3 +250,158 @@ it("records capture disagreement as a contradicted unknown", async () => {
),
).toEqual(beforeRepeat);
});
it("pages filtered unknown heads through the public MCP contract", async () => {
const session = createTestBinarySession(providerWithCapabilities([]));
const server = createServer({ kind: "session", session });
const client = new Client({ name: "unknown-pagination", version: "1" });
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
resources.push(client, server);
await server.connect(serverTransport);
await client.connect(clientTransport);
const created = [
["matching c", "high", "pagination"],
["matching a", "high", "pagination"],
["matching b", "high", "pagination"],
["filtered severity", "medium", "pagination"],
["filtered domain", "high", "other"],
] as const;
for (const [question, severity, domain] of created) {
const result = session.recordUnknown({
question,
severity,
domain,
required_authority: "analyst-inference",
required_confidence: "observed",
required_environment: null,
recommended_probes: [],
relationships: [],
supporting_evidence_ids: [],
contradicting_evidence_ids: [],
});
expect(result.ok).toBe(true);
}
const defaultPage = structured(
await client.callTool({ name: "list_unknowns", arguments: {} }),
).result as {
items: { unknown_id: string }[];
total: number;
offset: number;
next_offset: number | null;
};
expect(defaultPage.items.map(({ unknown_id }) => unknown_id)).toEqual(
defaultPage.items.map(({ unknown_id }) => unknown_id).sort(),
);
expect(defaultPage).toMatchObject({ total: 5, offset: 0, next_offset: null });
const detached = session.listUnknownsPage({ limit: 1 });
const detachedHead = detached.items[0];
if (detachedHead === undefined) throw new Error("Expected one unknown head");
const originalQuestion = detachedHead.question;
Reflect.set(detachedHead, "question", "changed outside the ledger");
expect(
session
.listUnknowns()
.find(({ unknown_id }) => unknown_id === detachedHead.unknown_id)
?.question,
).toBe(originalQuestion);
const offsetWithoutLimit = structured(
await client.callTool({
name: "list_unknowns",
arguments: { domain: "pagination", offset: 1 },
}),
).result as typeof defaultPage;
expect(offsetWithoutLimit).toMatchObject({
total: 4,
offset: 1,
next_offset: null,
});
expect(offsetWithoutLimit.items).toHaveLength(3);
const noMatches = structured(
await client.callTool({
name: "list_unknowns",
arguments: { domain: "no-such-domain", limit: 2 },
}),
).result as typeof defaultPage;
expect(noMatches).toMatchObject({
items: [],
total: 0,
offset: 0,
next_offset: null,
});
const beyondEnd = structured(
await client.callTool({
name: "list_unknowns",
arguments: { domain: "pagination", offset: 100, limit: 2 },
}),
).result as typeof defaultPage;
expect(beyondEnd).toMatchObject({
total: 4,
offset: 100,
next_offset: null,
});
expect(beyondEnd.items).toEqual([]);
const maximumSafePage = structured(
await client.callTool({
name: "list_unknowns",
arguments: {
offset: Number.MAX_SAFE_INTEGER,
limit: Number.MAX_SAFE_INTEGER,
},
}),
).result as typeof defaultPage;
expect(maximumSafePage).toMatchObject({
total: 5,
offset: Number.MAX_SAFE_INTEGER,
next_offset: null,
});
expect(maximumSafePage.items).toEqual([]);
const request = {
name: "list_unknowns",
arguments: { domain: "pagination", severity: "high", limit: 2 },
} as const;
const first = structured(await client.callTool(request))
.result as typeof defaultPage;
expect(first).toMatchObject({ total: 3, offset: 0, next_offset: 2 });
expect(first.items).toHaveLength(2);
const second = structured(
await client.callTool({
...request,
arguments: { ...request.arguments, offset: first.next_offset ?? 0 },
}),
).result as typeof defaultPage;
expect(second).toMatchObject({ total: 3, offset: 2, next_offset: null });
expect(second.items).toHaveLength(1);
expect(
[...first.items, ...second.items].map(({ unknown_id }) => unknown_id),
).toEqual(
session
.listUnknowns({ domain: "pagination", severity: "high" })
.map(({ unknown_id }) => unknown_id),
);
expect(
toolContract("list_unknowns").outputSchema.safeParse(
(await client.callTool(request)).structuredContent,
).success,
).toBe(true);
for (const arguments_ of [
{ limit: 0 },
{ offset: -1 },
{ offset: Number.MAX_SAFE_INTEGER + 1 },
{ unexpected: true },
]) {
expect(
(await client.callTool({ name: "list_unknowns", arguments: arguments_ }))
.isError,
).toBe(true);
}
});

View File

@ -412,7 +412,12 @@ describe("MCP investigation history snapshot replay", () => {
toolContract("list_unknowns").outputSchema.parse(
listed.structuredContent,
).result,
).toEqual({ items: [resolved], total: 1 });
).toEqual({
items: [resolved],
total: 1,
offset: 0,
next_offset: null,
});
},
);
});

View File

@ -15,6 +15,7 @@ import {
import { dirname, join } from "node:path";
import { describe, expect, it } from "vitest";
import { NativeMacOSProvider } from "../../../../src/native/NativeMacOSProvider.js";
import { ArtifactResourceScope } from "../../../../src/artifacts/ArtifactResourceScope.js";
import {
withSignatureTarget,
verifySignatureTarget,
@ -277,6 +278,7 @@ it("returns typed cancellation at binding entry and final verification", async (
await expect(
withSignatureTarget(
target,
new ArtifactResourceScope(),
async () => {
throw new Error("Cancelled acquisition must not invoke inspection");
},
@ -286,14 +288,18 @@ it("returns typed cancellation at binding entry and final verification", async (
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
await withSignatureTarget(target, async (binding) => {
const result = await verifySignatureTarget(target, binding, signal);
expect(result).toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
return result;
});
await withSignatureTarget(
target,
new ArtifactResourceScope(),
async (binding) => {
const result = await verifySignatureTarget(target, binding, signal);
expect(result).toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
return result;
},
);
});
it("preserves cancellation after a capture instead of classifying it as invalid output", async () => {
@ -348,21 +354,31 @@ it("copies and hashes the same bytes across multiple snapshot chunks", async ()
const bytes = Buffer.alloc(128 * 1024 + 13, 42);
await writeFile(path, bytes);
const target = await nativeMachoTargetForFile(path);
await withSignatureTarget(target, async (binding) => {
expect(await readFile(binding.snapshotPath)).toEqual(bytes);
return verifySignatureTarget(target, binding);
});
await withSignatureTarget(
target,
new ArtifactResourceScope(),
async (binding) => {
expect(await readFile(binding.snapshotPath)).toEqual(bytes);
return verifySignatureTarget(target, binding);
},
);
});
it("cleans the private snapshot when inspection throws", async () => {
const { target } = await fixture();
let snapshotPath: string | undefined;
await expect(
withSignatureTarget(target, async (binding) => {
const result = await withSignatureTarget(
target,
new ArtifactResourceScope(),
async (binding) => {
snapshotPath = binding.snapshotPath;
throw new Error("Invalid parsed output");
}),
).rejects.toThrow("Invalid parsed output");
},
);
expect(result).toMatchObject({
ok: false,
error: { _tag: "AnalysisOutputError" },
});
if (snapshotPath === undefined) throw new Error("Expected snapshot");
await expect(lstat(dirname(snapshotPath))).rejects.toMatchObject({
code: "ENOENT",