From 91830f1e0e68232e991902b0fa7617651d2eee15 Mon Sep 17 00:00:00 2001 From: Nelglor Date: Tue, 13 Oct 2026 09:12:15 -0400 Subject: [PATCH 02/24] feat(managed): name the declaring type of call and field targets (#2163) * feat(managed): name the declaring type of call and field targets Call edges named external targets by member name alone (Join, ToJson, WriteAllText), and method-spec edges had no name at all, so an analyst could not tell which library a method calls or find every use of a type without a decompiler. Field accesses through MemberRefs had no name either. Name each TypeDef, TypeRef, and generic-instantiation TypeSpec once, in the namespace-qualified form TypeDef full_name uses, and add member_refs parent_name, call_edges target_declaring_type, and field_accesses field_declaring_type. A method-spec target reports the generic method it instantiates. Fields, member references, methods, and method specs now resolve through one token-keyed MemberCore lookup instead of three maps. Evidence recorded before these fields reads them as null. Co-Authored-By: Claude Opus 5.5 * fix(managed): validate spec blobs and member kinds before naming targets Address review findings on declaring-type naming: - A generic instantiation must name a TypeDef or TypeRef, so a TypeSpec base is reported as malformed instead of followed. Naming no longer recurses. - TypeSpec and MethodSpec instantiation blobs are decoded with the member-signature grammar. Malformed blobs and out-of-range TypeDef rows are reported against their owning row and make coverage partial. - A MethodSpec reads both columns. An invalid Method coded index is reported; a readable Method keeps its name beside a malformed instantiation. - MemberCore carries its signature kind, so a field operand is named only by a field or field-signature MemberRef. This replaces the token-table guard. Co-Authored-By: Claude Opus 5.5 * fix(managed): validate MethodSpec arity and report unsupported spec blobs - A MethodSpec now checks that its method signature is generic and that the instantiation supplies as many type arguments as it declares. A non-generic target, a field reference, or a count mismatch is reported as invalid-row; the readable method keeps its name. - Unsupported TypeSpec and MethodSpec forms raise an owner-scoped unsupported-signature issue instead of reading as an intentional unknown. - MethodSpec reading moves to ManagedMethodSpecs.ts, and its tests get their own describe block. Co-Authored-By: Claude Opus 5.5 * fix(managed): reject non-method signature headers and invalid TypeRef scopes The method signature header reader now owns the calling-convention rule that only FNPTR applied: FIELD, LOCAL_SIG, PROPERTY, and GENERICINST headers, the reserved bit, and EXPLICITTHIS without HASTHIS are malformed. A MethodSpec whose method has a PROPERTY header is no longer accepted as a generic method, and member signatures with such headers no longer decode as methods. The type-name lookup validates a TypeRef's ResolutionScope coded index and reports an out-of-range scope as invalid-row, keeping the readable name. Co-Authored-By: Claude Opus 5.5 * fix(managed): hold member rows to MethodDefSig and FieldSig headers Method signature headers are now checked in context. MethodDef and MemberRef rows accept only the MethodDefSig/MethodRefSig headers: DEFAULT, VARARG, or GENERIC with DEFAULT. FNPTR keeps the unmanaged conventions compilers emit for function pointers. A field signature must be exactly FIELD, so a flagged FIELD header is malformed and never names a field. Co-Authored-By: Claude Opus 5.5 * fix(managed): accept MethodSpec arity only from a fully decoded signature The arity check read only the target's signature header, so a truncated generic method signature passed. Under type selection a MethodDef target outside the selection is never decoded elsewhere, so coverage stayed complete. The arity now comes from a transient full decode, and only a decoded method signature supplies it. signature() keeps its raw length and digest around the shared decodeSignature. Co-Authored-By: Claude Opus 5.5 * perf(managed): decode each MethodSpec target signature once edgeTargets caches the generic arity, including a failed decode, by target method token, so many instantiations of one generic method share a single signature decode. Unity.InputSystem has 472 distinct MethodSpecs over about 133 target methods. Co-Authored-By: Claude Opus 5.5 * fix(managed): validate spec type operands and transitive reference scopes * fix(managed): retain readable names when a scope row is unreadable * test(managed): pin owner-scoped spec operand and scope-chain checks e59217c83 validates spec type operands inside specSignature, so a malformed generic TypeSpec base is now invalid-row under its owning TypeSpec row; the three stale expectations follow that. New cases cover a TypeSpec argument and a MethodSpec argument that name a missing TypeRef, and a TypeRef whose ResolutionScope names itself. The scope cases move to their own describe block. Co-Authored-By: Claude Opus 5.5 * fix(managed): keep field targets from naming MethodSpecs A MethodSpec over a field-signature MemberRef inherited kind "field", so an ldsfld on that MethodSpec published a field name and declaring type for malformed IL. A MethodSpec now borrows its target's name only when the target is not a field; the arity check still reports it. Co-Authored-By: Claude Opus 5.5 * fix(managed): reuse owner-decoded signatures for MethodSpec arity MemberRef signatures, and MethodDef signatures in the type selection, are already fully decoded, but the MethodSpec arity check decoded them again and admitted a second transient allocation, which could exhaust the budget. MemberCore now carries the owner's genericArity, and only a target nothing decoded, such as an unselected MethodDef, falls back to the raw signature. signatureGenericArity is the one rule for both paths. Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- docs/managed-code-analysis.md | 22 ++ src/domain/managed/managedArtifact.ts | 19 +- src/dotnet/ManagedMemberInspector.test.ts | 394 +++++++++++++++++++++- src/dotnet/ManagedMemberInspector.ts | 18 +- src/dotnet/ManagedMemberInspectorCore.ts | 230 ++++++++++--- src/dotnet/ManagedMemberRows.ts | 113 +++++-- src/dotnet/ManagedMethodSpecs.ts | 154 +++++++++ src/dotnet/ManagedPe.fixture.ts | 81 ++++- src/dotnet/ManagedTypeNames.ts | 137 ++++++++ 9 files changed, 1060 insertions(+), 108 deletions(-) create mode 100644 src/dotnet/ManagedMethodSpecs.ts create mode 100644 src/dotnet/ManagedTypeNames.ts diff --git a/docs/managed-code-analysis.md b/docs/managed-code-analysis.md index 060b44bc2..d60603ce3 100644 --- a/docs/managed-code-analysis.md +++ b/docs/managed-code-analysis.md @@ -147,6 +147,28 @@ opposite observation's selection unknown, and application graphs report their selected scope as partial. Neither makes an absence claim about the whole assembly. Older member Evidence without `selection` remains readable. +### Edge target declaring types + +`member_refs[].parent_name`, `call_edges[].target_declaring_type`, and +`field_accesses[].field_declaring_type` name the type that declares each +target. Referenced spec signatures validate encoded type coordinates against +the metadata tables, including nested type arguments and custom modifiers. +TypeRef resolution scopes are checked transitively with cycle detection; +readable names survive malformed scopes as partial evidence. +TypeDef and TypeRef names use the namespace-qualified `full_name` +form, and nested types keep bare names. A generic-instantiation TypeSpec +parent names its generic type, such as ``System.Collections.Generic.List`1``. +Other well-formed TypeSpec forms and ModuleRef or MethodDef parents stay +`null`. A `method-spec` call edge reports the name and declaring type of the +generic method it instantiates. A field access is named only by a field or a +field-signature MemberRef. TypeSpec and MethodSpec blobs are decoded with the +member-signature grammar, and a MethodSpec must instantiate a generic method +with as many type arguments as it declares. Malformed or unsupported rows and +blobs, and arity mismatches, make coverage partial. A readable MethodSpec +method column that names a method, or a TypeRef whose resolution scope is +invalid, keeps its name. +Older member Evidence without these fields reads them as `null`: unknown. + ## Analysis objective Start by identifying the artifact with `inspect_managed_artifact`, then inspect diff --git a/src/domain/managed/managedArtifact.ts b/src/domain/managed/managedArtifact.ts index 297b60a4e..94eff2398 100644 --- a/src/domain/managed/managedArtifact.ts +++ b/src/domain/managed/managedArtifact.ts @@ -203,10 +203,18 @@ const managedMethodSchema = z.object({ body: managedMethodBodySchema, }); +// Declaring types are namespace-qualified like TypeDef `full_name`, nested +// types keep bare names, and a generic instantiation names its generic type. +// Evidence recorded before declaring types were named reads them as null. +const declaringTypeNameSchema = z.string().nullable().default(null); + const managedMemberReferenceSchema = z.object({ token: tokenSchema, row_offset: offsetSchema, parent_token: tokenSchema.nullable(), + parent_name: declaringTypeNameSchema.describe( + "Name of a TypeDef, TypeRef, or generic-instantiation TypeSpec parent; null for other parents or when unknown", + ), name: z.string(), signature: managedSignatureSchema, }); @@ -281,7 +289,13 @@ const managedCallEdgeSchema = z.object({ opcode: z.string(), target_token: tokenSchema, target_kind: z.enum(["method-def", "member-ref", "method-spec", "unknown"]), - target_name: z.string().nullable(), + target_name: z + .string() + .nullable() + .describe("Method name; a method-spec target reports its generic method"), + target_declaring_type: declaringTypeNameSchema.describe( + "Declaring type of the called method; null when unknown", + ), }); const managedFieldAccessSchema = z.object({ @@ -290,6 +304,9 @@ const managedFieldAccessSchema = z.object({ opcode: z.string(), field_token: tokenSchema, field_name: z.string().nullable(), + field_declaring_type: declaringTypeNameSchema.describe( + "Declaring type of the accessed field; null when unknown", + ), }); const managedParseIssueSchema = z.object({ diff --git a/src/dotnet/ManagedMemberInspector.test.ts b/src/dotnet/ManagedMemberInspector.test.ts index 81d64f4a1..9ada1bf55 100644 --- a/src/dotnet/ManagedMemberInspector.test.ts +++ b/src/dotnet/ManagedMemberInspector.test.ts @@ -110,6 +110,8 @@ describe("managed member inspection", () => { expect(result.member_refs).toEqual([ expect.objectContaining({ token: "0x0a000001", + parent_token: "0x01000001", + parent_name: "System.Runtime.Versioning.TargetFrameworkAttribute", name: ".ctor", signature: expect.objectContaining({ kind: "method", @@ -126,6 +128,8 @@ describe("managed member inspection", () => { target_token: "0x0a000001", target_kind: "member-ref", target_name: ".ctor", + target_declaring_type: + "System.Runtime.Versioning.TargetFrameworkAttribute", }, ]); expect(result.field_accesses).toEqual([ @@ -135,12 +139,366 @@ describe("managed member inspection", () => { opcode: "ldfld", field_token: "0x04000001", field_name: "counter", + field_declaring_type: "Fixture.Program", }, ]); expect(result.coverage).toMatchObject({ state: "complete", issues: [] }); }); }); +/** A tiny-header body of token-operand instructions followed by `ret`. */ +const tokenBody = ( + instructions: readonly (readonly [opcode: number, token: number])[], +): Buffer => { + const il = Buffer.concat([ + ...instructions.map(([opcode, token]) => { + const instruction = Buffer.alloc(5); + instruction.writeUInt8(opcode, 0); + instruction.writeUInt32LE(token, 1); + return instruction; + }), + Buffer.from([0x2a]), + ]); + return Buffer.concat([Buffer.from([(il.length << 2) | 2]), il]); +}; + +const attribute = "System.Runtime.Versioning.TargetFrameworkAttribute"; +const inspect = (options: Parameters[0]) => { + const bytes = buildManagedPeFixture(options); + return inspectManagedMembersBytes(bytes, managedPeFixtureTarget(bytes)); +}; +const call = 0x28; +const ldsfld = 0x7e; +const memberRefOnTypeSpec = (signature: Buffer) => + inspect({ + typeSpecSignatures: [signature], + memberRefParentRaw: (1 << 3) | 4, + }); + +// HASTHIS | GENERIC, one generic parameter, no parameters, void. +const genericRef = Buffer.from([0x30, 0x01, 0x00, 0x01]); +// GENERIC, one generic parameter, no parameters, void. +const genericDef = Buffer.from([0x10, 0x01, 0x00, 0x01]); + +describe("managed edge target declaring types", () => { + it("names a generic TypeSpec parent by its generic type and leaves other TypeSpecs unknown", () => { + // GENERICINST CLASS TypeRef(1) + const generic = memberRefOnTypeSpec( + Buffer.from([0x15, 0x12, (1 << 2) | 1, 0x01, 0x0e]), + ); + expect(generic.member_refs[0]?.parent_name).toBe(attribute); + expect(generic.call_edges[0]?.target_declaring_type).toBe(attribute); + + // SZARRAY i4 + const array = memberRefOnTypeSpec(Buffer.from([0x1d, 0x08])); + expect(array.member_refs[0]?.parent_name).toBeNull(); + expect(array.coverage).toEqual({ state: "complete", issues: [] }); + }); + + it.each([ + [ + "names a TypeSpec as its generic type", + [0x15, 0x12, (1 << 2) | 2, 0x01, 0x0e], + "invalid-row", + "metadata.TypeSpec:0x1b000001", + ], + [ + "has no type arguments", + [0x15, 0x12, (1 << 2) | 1], + "invalid-blob", + "metadata.TypeSpec:0x1b000001", + ], + [ + "names a missing TypeRef", + [0x15, 0x12, (9 << 2) | 1, 0x01, 0x0e], + "invalid-row", + "metadata.TypeSpec:0x1b000001", + ], + [ + "names a missing TypeDef", + [0x15, 0x12, (9 << 2) | 0, 0x01, 0x0e], + "invalid-row", + "metadata.TypeSpec:0x1b000001", + ], + [ + "has a type argument naming a missing TypeRef", + [0x15, 0x12, (1 << 2) | 1, 0x01, 0x12, (9 << 2) | 1], + "invalid-row", + "metadata.TypeSpec:0x1b000001", + ], + ])( + "reports a generic TypeSpec parent that %s and leaves it unknown", + (_case, signature, code, scope) => { + const result = memberRefOnTypeSpec(Buffer.from(signature)); + + expect(result.member_refs[0]?.parent_name).toBeNull(); + expect(result.coverage).toMatchObject({ + state: "partial", + issues: [{ code, scope }], + }); + }, + ); + + it("reports unsupported TypeSpec and MethodSpec forms instead of treating them as unknown", () => { + const typeSpec = memberRefOnTypeSpec(Buffer.from([0xff])); + const methodSpec = inspect({ + memberRefSignature: genericRef, + methodSpecMethods: [(1 << 1) | 1], + methodSpecInstantiation: Buffer.from([0x0a, 0x01, 0xff]), + ilBody: tokenBody([[call, 0x2b00_0001]]), + }); + + expect(typeSpec.member_refs[0]?.parent_name).toBeNull(); + expect(typeSpec.coverage).toMatchObject({ + state: "partial", + issues: [ + { + code: "unsupported-signature", + scope: "metadata.TypeSpec:0x1b000001", + }, + ], + }); + expect(methodSpec.coverage).toMatchObject({ + state: "partial", + issues: [ + { + code: "unsupported-signature", + scope: "metadata.MethodSpec:0x2b000001", + }, + ], + }); + }); + + it("names field MemberRefs as fields and leaves mismatched member kinds unnamed", () => { + const result = inspect({ + memberRefName: "Empty", + // FIELD string + memberRefSignature: Buffer.from([0x06, 0x0e]), + ilBody: tokenBody([ + [ldsfld, 0x0a00_0001], + [call, 0x0400_0001], + ]), + }); + const methodRef = inspect({ ilBody: tokenBody([[ldsfld, 0x0a00_0001]]) }); + const flaggedField = inspect({ + memberRefName: "Empty", + // GENERIC | FIELD string: FieldSig allows no flags. + memberRefSignature: Buffer.from([0x16, 0x0e]), + ilBody: tokenBody([[ldsfld, 0x0a00_0001]]), + }); + + expect(result.field_accesses).toMatchObject([ + { field_name: "Empty", field_declaring_type: attribute }, + ]); + expect(methodRef.field_accesses).toMatchObject([ + { field_name: null, field_declaring_type: null }, + ]); + expect(flaggedField.field_accesses).toMatchObject([ + { field_name: null, field_declaring_type: null }, + ]); + expect(flaggedField.coverage.state).toBe("partial"); + expect(result.call_edges).toMatchObject([ + { + target_kind: "unknown", + target_name: null, + target_declaring_type: null, + }, + ]); + }); + + it("reads Evidence recorded before declaring types were named as unknown", () => { + const current = inspect({}); + const parsed = managedMemberInspectionSchema.parse({ + ...current, + member_refs: current.member_refs.map( + ({ parent_name: _omitted, ...legacy }) => legacy, + ), + call_edges: current.call_edges.map( + ({ target_declaring_type: _omitted, ...legacy }) => legacy, + ), + field_accesses: current.field_accesses.map( + ({ field_declaring_type: _omitted, ...legacy }) => legacy, + ), + }); + + expect(parsed.member_refs[0]?.parent_name).toBeNull(); + expect(parsed.call_edges[0]?.target_declaring_type).toBeNull(); + expect(parsed.field_accesses[0]?.field_declaring_type).toBeNull(); + }); +}); + +describe("managed TypeRef resolution scopes", () => { + it.each([ + // AssemblyRef 9 does not exist. + ["an invalid", (9 << 2) | 2, "is invalid"], + // The TypeRef is its own enclosing type. + ["a cyclic", (1 << 2) | 3, "contains a cycle"], + ])( + "keeps a TypeRef name beside %s resolution scope", + (_case, typeRefScopeRaw, detail) => { + const result = inspect({ typeRefScopeRaw }); + + expect(result.member_refs[0]?.parent_name).toBe(attribute); + expect(result.call_edges[0]?.target_declaring_type).toBe(attribute); + expect(result.coverage).toMatchObject({ + state: "partial", + issues: [ + { + code: "invalid-row", + scope: "metadata.TypeRef:0x01000001", + detail: expect.stringContaining(detail), + }, + ], + }); + }, + ); +}); + +describe("managed MethodSpec call targets", () => { + it("names generic method calls after the method they instantiate", () => { + const result = inspect({ + memberRefSignature: genericRef, + methodSignature: genericDef, + methodSpecMethods: [(1 << 1) | 1, (1 << 1) | 0], + ilBody: tokenBody([ + [call, 0x2b00_0001], + [call, 0x2b00_0002], + ]), + }); + + expect( + result.call_edges.map((edge) => [ + edge.target_kind, + edge.target_name, + edge.target_declaring_type, + ]), + ).toEqual([ + ["method-spec", ".ctor", attribute], + ["method-spec", "Main", "Fixture.Program"], + ]); + expect(result.coverage).toEqual({ state: "complete", issues: [] }); + }); + + it.each([ + [ + "a non-generic method", + {}, + Buffer.from([0x0a, 0x01, 0x0e]), + "but its method declares 0 generic parameter(s)", + ], + [ + "the wrong number of type arguments", + { memberRefSignature: genericRef }, + Buffer.from([0x0a, 0x02, 0x0e, 0x08]), + "supplies 2 type argument(s), but its method declares 1", + ], + [ + "a field reference", + { memberRefSignature: Buffer.from([0x06, 0x0e]) }, + Buffer.from([0x0a, 0x01, 0x0e]), + "does not have a decodable method signature", + ], + [ + "a property signature", + // PROPERTY | GENERIC, one generic parameter, no parameters, void. + { memberRefSignature: Buffer.from([0x18, 0x01, 0x00, 0x01]) }, + Buffer.from([0x0a, 0x01, 0x0e]), + "does not have a decodable method signature", + ], + [ + "a truncated signature", + // HASTHIS | GENERIC, one generic parameter, no parameters, no return type. + { memberRefSignature: Buffer.from([0x30, 0x01, 0x00]) }, + Buffer.from([0x0a, 0x01, 0x0e]), + "does not have a decodable method signature", + ], + [ + "a type argument naming a missing TypeRef", + { memberRefSignature: genericRef }, + Buffer.from([0x0a, 0x01, 0x12, (9 << 2) | 1]), + "TypeDefOrRefOrSpec operand 0x25 is invalid", + ], + [ + "an unmanaged-convention signature", + // C | GENERIC: a function pointer convention, not a MethodRefSig one. + { memberRefSignature: Buffer.from([0x11, 0x01, 0x00, 0x01]) }, + Buffer.from([0x0a, 0x01, 0x0e]), + "does not have a decodable method signature", + ], + ])( + "reports a MethodSpec that instantiates %s", + (_case, options, methodSpecInstantiation, detail) => { + const result = inspect({ + ...options, + methodSpecMethods: [(1 << 1) | 1], + methodSpecInstantiation, + ilBody: tokenBody([[call, 0x2b00_0001]]), + }); + + expect(result.coverage).toMatchObject({ + state: "partial", + issues: [ + { + code: "invalid-row", + scope: "metadata.MethodSpec:0x2b000001", + detail: expect.stringContaining(detail), + }, + ], + }); + }, + ); + + it("never names a field through a MethodSpec", () => { + const result = inspect({ + memberRefName: "Empty", + // FIELD string: a MethodSpec on it is malformed. + memberRefSignature: Buffer.from([0x06, 0x0e]), + methodSpecMethods: [(1 << 1) | 1], + ilBody: tokenBody([ + [ldsfld, 0x2b00_0001], + [call, 0x2b00_0001], + ]), + }); + + expect(result.field_accesses).toMatchObject([ + { field_name: null, field_declaring_type: null }, + ]); + expect(result.call_edges).toMatchObject([ + { target_name: null, target_declaring_type: null }, + ]); + expect(result.coverage.state).toBe("partial"); + }); + + it("keeps a readable MethodSpec method beside a malformed instantiation", () => { + const result = inspect({ + methodSpecMethods: [(1 << 1) | 1, (9 << 1) | 1], + methodSpecInstantiation: Buffer.from([0x0a, 0x00]), + ilBody: tokenBody([ + [call, 0x2b00_0001], + [call, 0x2b00_0002], + ]), + }); + + expect( + result.call_edges.map(({ target_name, target_declaring_type }) => [ + target_name, + target_declaring_type, + ]), + ).toEqual([ + [".ctor", attribute], + [null, null], + ]); + expect(result.coverage.state).toBe("partial"); + expect( + result.coverage.issues.map(({ code, scope }) => [code, scope]), + ).toEqual([ + ["invalid-blob", "metadata.MethodSpec:0x2b000001"], + ["invalid-row", "metadata.MethodSpec:0x2b000002"], + ["invalid-blob", "metadata.MethodSpec:0x2b000002"], + ]); + }); +}); + describe("managed exported types", () => { it("reports type forwarders and nested exported types with their implementation scope", () => { const bytes = buildManagedPeFixture({ @@ -373,15 +731,9 @@ describe("managed derived edges", () => { body: { anchors: Array.from({ length: 40_000 }, () => anchor) }, } as unknown as ManagedMethod; - expect(() => - edges( - [method], - new Map(), - new Map(), - new Map(), - new ManagedDecodeBudget(), - ), - ).toThrow(ManagedReaderFailure); + expect(() => edges([method], new Map(), new ManagedDecodeBudget())).toThrow( + ManagedReaderFailure, + ); }); }); @@ -413,15 +765,9 @@ describe("managed derived caller names", () => { }, } as unknown as ManagedMethod; - expect(() => - edges( - [method], - new Map(), - new Map(), - new Map(), - new ManagedDecodeBudget(), - ), - ).toThrow(ManagedReaderFailure); + expect(() => edges([method], new Map(), new ManagedDecodeBudget())).toThrow( + ManagedReaderFailure, + ); expect(built).toBe(false); }); }); @@ -671,6 +1017,18 @@ it.each([ }); }); +it.each([ + ["a property header", [0x08, 0x00, 0x08]], + ["an explicit-this header without has-this", [0x40, 0x00, 0x01]], + ["a C calling convention header", [0x01, 0x00, 0x01]], + ["a generic vararg header", [0x15, 0x01, 0x00, 0x01]], +])("reports %s as a malformed member signature", (_name, blob) => { + expect(signature(Buffer.from(blob))).toMatchObject({ + kind: "unknown", + parse_status: "malformed", + }); +}); + it("rejects array ranks beyond the CLR limit before rendering dimensions", () => { expect( signature( diff --git a/src/dotnet/ManagedMemberInspector.ts b/src/dotnet/ManagedMemberInspector.ts index 24b97f9eb..dba20ac57 100644 --- a/src/dotnet/ManagedMemberInspector.ts +++ b/src/dotnet/ManagedMemberInspector.ts @@ -27,6 +27,8 @@ import { typeRanges, type TypeRange, } from "./ManagedMemberInspectorCore.js"; +import { edgeTargets } from "./ManagedMethodSpecs.js"; +import { typeNameLookup } from "./ManagedTypeNames.js"; import { edges, parseExportedTypes, @@ -171,7 +173,11 @@ export const inspectManagedMembersBytes = ( const exportedTypes = parseExportedTypes(bytes, layout); issues.push(...exportedTypes.issues); const fields = parseFields(bytes, layout, ranges, selected); - const memberRefs = parseMemberRefs(bytes, layout); + const memberRefs = parseMemberRefs( + bytes, + layout, + typeNameLookup(bytes, layout, ranges, issues), + ); issues.push(...memberRefs.issues); const methods = parseMethods({ bytes, @@ -181,6 +187,12 @@ export const inspectManagedMembersBytes = ( selected, }); issues.push(...fields.issues, ...methods.issues); + const targets = edgeTargets(bytes, layout, methods.methods, [ + fields.core, + memberRefs.core, + methods.core, + ]); + issues.push(...targets.issues); const coverageIssues = uniqueManagedParseIssues(issues); const incompleteMembers = [...fields.fields, ...memberRefs.refs, ...methods.methods].some( @@ -193,9 +205,7 @@ export const inspectManagedMembersBytes = ( ); const related = edges( methods.methods, - methods.core, - fields.core, - memberRefs.core, + targets.members, managedDecodeBudget(layout), ); return managedMemberInspectionSchema.parse( diff --git a/src/dotnet/ManagedMemberInspectorCore.ts b/src/dotnet/ManagedMemberInspectorCore.ts index f54a11769..7776c0090 100644 --- a/src/dotnet/ManagedMemberInspectorCore.ts +++ b/src/dotnet/ManagedMemberInspectorCore.ts @@ -7,6 +7,7 @@ import type { ManagedMemberInspection, ManagedParseIssue, } from "../domain/managed/managedArtifact.js"; +import { managedDecodeBudget } from "./ManagedDecodeBudget.js"; import type { ManagedMetadataLayout, MetadataTableLayout, @@ -49,20 +50,22 @@ export interface TypeRange { readonly methodEnd: number; } -export interface MethodCore { +/** + * Name and declaring type of a field, method, member reference, or method + * specification, keyed by its token so edges resolve any target uniformly. + */ +export interface MemberCore { readonly token: string; readonly name: string; readonly declaringType: string | null; -} - -export interface FieldCore { - readonly token: string; - readonly name: string; -} - -export interface MemberRefCore { - readonly token: string; - readonly name: string; + /** Fields and methods by table; member references by decoded signature. */ + readonly kind: ManagedSignature["kind"]; + /** + * Generic arity from the owner's decoded signature (see + * `signatureGenericArity`); absent when no owner decoded it, as for a + * MethodDef outside the type selection. + */ + readonly genericArity?: number | null; } export interface ParsedInstruction { @@ -88,7 +91,7 @@ const rowRange = ( return { first: start, last: end - 1, count: end - start }; }; -const fullName = (namespace: string, name: string): string => +export const fullName = (namespace: string, name: string): string => namespace.length === 0 ? name : `${namespace}.${name}`; const readTypeRange = ( @@ -242,7 +245,7 @@ export const parseTypes = ( return { types: items, issues }; }; -const readCompressed = ( +export const readCompressed = ( bytes: Buffer, offset: number, ): { readonly value: number; readonly next: number } => { @@ -341,13 +344,38 @@ interface MethodSignatureHeader { readonly next: number; } +// MethodDefSig and MethodRefSig headers (ECMA-335 II.23.2.1-2) without +// HASTHIS and EXPLICITTHIS: DEFAULT, VARARG, or GENERIC, which implies DEFAULT. +const MEMBER_METHOD_HEADERS = new Set([0x00, 0x05, 0x10]); + +// FNPTR also carries the unmanaged C, STDCALL, THISCALL, FASTCALL, and +// UNMANAGED conventions that compilers emit for unmanaged function pointers. +const FUNCTION_POINTER_CONVENTIONS = new Set([0, 1, 2, 3, 4, 5, 9]); + +/** + * Read a method signature's calling convention and counts. A header that is + * not a method signature in its context (a MethodDef or MemberRef row, or an + * FNPTR type), sets the reserved bit, or sets EXPLICITTHIS without HASTHIS is + * malformed, so no caller consumes its counts. FIELD, LOCAL_SIG, PROPERTY, + * and GENERICINST headers are never method signatures. + */ const readMethodSignatureHeader = ( blob: Buffer, offset: number, + context: "member" | "function-pointer", ): MethodSignatureHeader => { const first = blob[offset]; if (first === undefined) return malformedSignature("truncated method signature", offset); + const allowed = + context === "member" + ? MEMBER_METHOD_HEADERS.has(first & 0x9f) + : (first & 0x80) === 0 && FUNCTION_POINTER_CONVENTIONS.has(first & 0x0f); + if (!allowed || ((first & 0x40) !== 0 && (first & 0x20) === 0)) + return malformedSignature( + `calling convention 0x${first.toString(16)} does not head a ${context} method signature`, + offset, + ); let next = offset + 1; let genericParameterCount: number | null = null; if ((first & 0x10) !== 0) { @@ -366,12 +394,6 @@ const readMethodSignatureHeader = ( const MAX_ARRAY_RANK = 32; -// FNPTR carries a method signature: managed DEFAULT/VARARG, or the unmanaged -// C, STDCALL, THISCALL, FASTCALL, and UNMANAGED conventions that compilers -// emit for unmanaged function pointers. FIELD, LOCAL_SIG, PROPERTY, and -// GENERICINST headers are not method signatures. -const FUNCTION_POINTER_CONVENTIONS = new Set([0, 1, 2, 3, 4, 5, 9]); - const arrayShape = ( blob: Buffer, offset: number, @@ -449,10 +471,40 @@ const POSITION_ELEMENTS: ReadonlyMap = [0x10, ["return", "parameter", "field"]], ]); +/** Decode a type operand once; spec callers also validate its metadata coordinate. */ +const readTypeOperand = ( + blob: Buffer, + offset: number, + layout?: ManagedMetadataLayout, + tables: readonly number[] = [2, 1, 27], +): ReturnType => { + const token = readCompressed(blob, offset); + if (layout !== undefined) { + const reason = + token.value === 0 + ? "type operand is null" + : metadataCodedTokenInvalidReason( + token.value, + 2, + tables, + layout.rowCounts, + ); + if (reason !== null) + throw managedFailure( + "invalid-row", + "signature", + `TypeDefOrRefOrSpec operand 0x${token.value.toString(16)} is invalid: ${reason}`, + offset, + ); + } + return token; +}; + const readTypeSignature = ( blob: Buffer, offset: number, base: SignaturePosition, + layout?: ManagedMetadataLayout, ): { readonly value: string; readonly next: number } => { const stack: TypeContinuation[] = []; let cursor = offset; @@ -479,7 +531,7 @@ const readTypeSignature = ( stack.push({ kind: "suffix", suffix }); cursor += 1; } else if (kind === 0x1f || kind === 0x20) { - const token = readCompressed(blob, cursor + 1); + const token = readTypeOperand(blob, cursor + 1, layout); const modifier = `${kind === 0x1f ? "modreq" : "modopt"}:${String(token.value)}`; // A modifier frame on top was pushed by the directly preceding byte; // keep an adjacent run together and in source order. @@ -488,7 +540,7 @@ const readTypeSignature = ( else stack.push({ kind: "modifiers", modifiers: [modifier] }); cursor = token.next; } else if (kind === 0x11 || kind === 0x12) { - const token = readCompressed(blob, cursor + 1); + const token = readTypeOperand(blob, cursor + 1, layout); value = `${kind === 0x11 ? "valuetype" : "class"}:${String(token.value)}`; cursor = token.next; } else if (kind === 0x1e || kind === 0x13) { @@ -508,7 +560,7 @@ const readTypeSignature = ( "generic instantiation does not name a class or value type", cursor + 1, ); - const token = readCompressed(blob, cursor + 2); + const token = readTypeOperand(blob, cursor + 2, layout, [2, 1]); const argumentCount = readCompressed(blob, token.next); if (argumentCount.value === 0) return malformedSignature( @@ -525,16 +577,11 @@ const readTypeSignature = ( }); cursor = argumentCount.next; } else if (kind === 0x1b) { - const header = readMethodSignatureHeader(blob, cursor + 1); - if ( - !FUNCTION_POINTER_CONVENTIONS.has(header.first & 0x0f) || - (header.first & 0x80) !== 0 || - ((header.first & 0x40) !== 0 && (header.first & 0x20) === 0) - ) - return malformedSignature( - `function pointer calling convention 0x${header.first.toString(16)} does not head a method signature`, - cursor + 1, - ); + const header = readMethodSignatureHeader( + blob, + cursor + 1, + "function-pointer", + ); const convention = callingConvention(header.first); const head = header.genericParameterCount === null @@ -602,22 +649,81 @@ const callingConvention = (value: number): string => { return flags.length === 0 ? name : `${name} ${flags.join(" ")}`; }; -export const signature = ( +/** + * Decode a whole TypeSpec blob, or a MethodSpec instantiation (GENERICINST, + * argument count, Types), with the member-signature grammar and return the + * rendered types. A malformed or unsupported blob throws invalid-blob or + * unsupported-signature under the owning row, so coverage records it. + */ +export const specSignature = ( + layout: ManagedMetadataLayout, blob: Buffer, - rawSha256?: string, -): ManagedSignature => { - const raw = { - raw_length: blob.length, - raw_sha256: rawSha256 ?? sha256Bytes(blob), - }; + owner: { + readonly form: "type" | "instantiation"; + readonly scope: string; + readonly offset: number; + }, +): readonly string[] => { + const budget = managedDecodeBudget(layout); + // Rendering is transient, so charge it as member signatures are charged. + budget.admitTransient(blob.length * 256, owner.offset); + try { + let cursor = 0; + let count = 1; + if (owner.form === "instantiation") { + if (blob[0] !== 0x0a) + malformedSignature("MethodSpec instantiation is not GENERICINST", 0); + const header = readCompressed(blob, 1); + if (header.value === 0) + malformedSignature("MethodSpec instantiation has no type arguments", 1); + count = header.value; + cursor = header.next; + } + const types: string[] = []; + // Each Type consumes at least one byte, so a false count fails on truncation. + for (let index = 0; index < count; index += 1) { + const type = readTypeSignature(blob, cursor, "type", layout); + types.push(type.value); + cursor = type.next; + } + if (cursor !== blob.length) + malformedSignature("Trailing signature data", cursor); + return types; + } catch (cause: unknown) { + if ( + !(cause instanceof ManagedReaderFailure) || + (cause.issue.code !== "invalid-blob" && + cause.issue.code !== "invalid-row" && + cause.issue.code !== "unsupported-signature") + ) + throw cause; + throw managedFailure( + cause.issue.code, + owner.scope, + cause.issue.detail, + owner.offset, + ); + } finally { + budget.releaseTransient(); + } +}; + +type DecodedSignature = Omit; + +/** + * Decode a field or member method signature. A malformed or unsupported blob + * decodes as kind `unknown` with its reason. + */ +const decodeSignature = (blob: Buffer): DecodedSignature => { try { if (blob.length === 0) throw managedFailure("invalid-blob", "signature", "empty signature", 0); const first = blob[0] ?? 0; - if ((first & 0x0f) === 6) { + // FieldSig (II.23.2.4) is exactly FIELD; a flagged FIELD header is not a + // method header either, so it is malformed below. + if (first === 0x06) { const fieldType = readTypeSignature(blob, 1, "field"); return { - ...raw, kind: "field", parse_status: fieldType.next === blob.length ? "decoded" : "partial", calling_convention: callingConvention(first), @@ -630,7 +736,7 @@ export const signature = ( fieldType.next === blob.length ? null : "Trailing signature data", }; } - const header = readMethodSignatureHeader(blob, 0); + const header = readMethodSignatureHeader(blob, 0, "member"); const returnType = readTypeSignature(blob, header.next, "return"); let offset = returnType.next; const parameters: string[] = []; @@ -640,7 +746,6 @@ export const signature = ( offset = parameter.next; } return { - ...raw, kind: "method", parse_status: offset === blob.length ? "decoded" : "partial", calling_convention: callingConvention(first), @@ -659,7 +764,6 @@ export const signature = ( ) throw cause; return { - ...raw, kind: "unknown", parse_status: cause.issue.code === "unsupported-signature" @@ -675,3 +779,41 @@ export const signature = ( }; } }; + +/** + * Generic parameter count of a method signature that decodes completely, 0 + * when it is not generic, or null when the blob is not a decodable method + * signature. The rendered decode is transient. + */ +export const methodGenericArity = ( + layout: ManagedMetadataLayout, + blob: Buffer, +): number | null => { + const budget = managedDecodeBudget(layout); + budget.admitTransient(blob.length * 256, layout.blob.offset); + try { + return signatureGenericArity(decodeSignature(blob)); + } finally { + budget.releaseTransient(); + } +}; + +/** + * Generic parameter count of a completely decoded method signature, 0 when it + * is not generic, or null for any other signature. + */ +export const signatureGenericArity = ( + decoded: DecodedSignature, +): number | null => + decoded.kind === "method" && decoded.parse_status === "decoded" + ? (decoded.generic_parameter_count ?? 0) + : null; + +export const signature = ( + blob: Buffer, + rawSha256?: string, +): ManagedSignature => ({ + raw_length: blob.length, + raw_sha256: rawSha256 ?? sha256Bytes(blob), + ...decodeSignature(blob), +}); diff --git a/src/dotnet/ManagedMemberRows.ts b/src/dotnet/ManagedMemberRows.ts index f4a63db5a..201ff1c45 100644 --- a/src/dotnet/ManagedMemberRows.ts +++ b/src/dotnet/ManagedMemberRows.ts @@ -9,15 +9,14 @@ import type { ManagedParseIssue } from "../domain/managed/managedArtifact.js"; import type { ManagedMetadataLayout } from "./ManagedMetadataLayout.js"; import { signature, - type FieldCore, + signatureGenericArity, type ManagedCallEdge, type ManagedExportedType, type ManagedField, type ManagedFieldAccess, type ManagedMemberRef, type ManagedMethod, - type MemberRefCore, - type MethodCore, + type MemberCore, type TypeRange, } from "./ManagedMemberInspectorCore.js"; import { @@ -51,11 +50,11 @@ export const parseFields = ( selected: SelectedTypeTokens = null, ): { readonly fields: readonly ManagedField[]; - readonly core: ReadonlyMap; + readonly core: ReadonlyMap; readonly issues: readonly ManagedParseIssue[]; } => { const fields: ManagedField[] = []; - const core = new Map(); + const core = new Map(); const issues: ManagedParseIssue[] = []; const table = layout.table(4); const ownerRow = readManagedValue( @@ -78,7 +77,12 @@ export const parseFields = ( : (readManagedValue(() => ownerRow(row), issues) ?? 0); const declared = ranges[owner - 1]; const token = metadataToken(4, row); - core.set(token, { token, name }); + core.set(token, { + token, + name, + declaringType: declared?.fullName ?? null, + kind: "field", + }); if (!isSelected(selected, declared)) return; const sig = readMetadataBlob( bytes, @@ -174,13 +178,14 @@ export const parseExportedTypes = ( export const parseMemberRefs = ( bytes: Buffer, layout: ManagedMetadataLayout, + typeName: (token: string | null) => string | null, ): { readonly refs: readonly ManagedMemberRef[]; - readonly core: ReadonlyMap; + readonly core: ReadonlyMap; readonly issues: readonly ManagedParseIssue[]; } => { const refs: ManagedMemberRef[] = []; - const core = new Map(); + const core = new Map(); const issues: ManagedParseIssue[] = []; const table = layout.table(10); for (let row = 1; row <= (table?.rowCount ?? 0); row += 1) { @@ -218,19 +223,29 @@ export const parseMemberRefs = ( ? "MemberRef parent coded index 0x0 is null, but the Class column must reference a row" : `MemberRef parent coded index 0x${parentRaw.toString(16)} is invalid: ${parentReason}`, }); + const parentToken = metadataCodedToken( + parentRaw, + 3, + [2, 1, 26, 6, 27], + layout.rowCounts, + ); + const parentName = typeName(parentToken); + const decoded = budgetedSignature(sig, layout); refs.push({ token, row_offset: cursor.start, - parent_token: metadataCodedToken( - parentRaw, - 3, - [2, 1, 26, 6, 27], - layout.rowCounts, - ), + parent_token: parentToken, + parent_name: parentName, name, - signature: budgetedSignature(sig, layout), + signature: decoded, + }); + core.set(token, { + token, + name, + declaringType: parentName, + kind: decoded.kind, + genericArity: signatureGenericArity(decoded), }); - core.set(token, { token, name }); }, issues); } return { refs, core, issues }; @@ -252,11 +267,11 @@ export const parseMethods = ({ selected = null, }: ParseMethodsInput): { readonly methods: readonly ManagedMethod[]; - readonly core: ReadonlyMap; + readonly core: ReadonlyMap; readonly issues: readonly ManagedParseIssue[]; } => { const methods: ManagedMethod[] = []; - const core = new Map(); + const core = new Map(); const issues: ManagedParseIssue[] = []; const table = layout.table(6); const ownerRow = readManagedValue( @@ -281,11 +296,13 @@ export const parseMethods = ({ : (readManagedValue(() => ownerRow(row), issues) ?? 0); const declared = ranges[owner - 1]; const token = metadataToken(6, row); - core.set(token, { + const member = { token, name, declaringType: declared?.fullName ?? null, - }); + kind: "method", + } as const; + core.set(token, member); if (!isSelected(selected, declared)) return; const sig = readMetadataBlob( bytes, @@ -294,6 +311,11 @@ export const parseMethods = ({ layout.blob.size, ); cursor.readIndex(layout.tableIndexSize(8)); + const decoded = budgetedSignature(sig, layout); + core.set(token, { + ...member, + genericArity: signatureGenericArity(decoded), + }); methods.push({ token, row_offset: cursor.start, @@ -303,7 +325,7 @@ export const parseMethods = ({ rva, impl_flags: implFlags, flags, - signature: budgetedSignature(sig, layout), + signature: decoded, body: methodBody( bytes, pe, @@ -317,6 +339,15 @@ export const parseMethods = ({ return { methods, core, issues }; }; +/** + * Member kinds each operand kind may name. A MemberRef whose signature does + * not decode keeps its name on calls, as before, but is never named a field. + */ +const NAMEABLE_KINDS = new Map>([ + ["method", new Set(["method", "unknown"])], + ["field", new Set(["field"])], +]); + const targetKind = (token: string): ManagedCallEdge["target_kind"] => token.startsWith("0x06") ? "method-def" @@ -326,11 +357,22 @@ const targetKind = (token: string): ManagedCallEdge["target_kind"] => ? "method-spec" : "unknown"; +/** The member an operand names, if the operand kind can name that member's kind. */ +const namedTarget = ( + members: ReadonlyMap, + operandKind: string, + operand: string, +): MemberCore | undefined => { + const member = members.get(operand); + return member !== undefined && + NAMEABLE_KINDS.get(operandKind)?.has(member.kind) === true + ? member + : undefined; +}; + export const edges = ( methods: readonly ManagedMethod[], - methodCore: ReadonlyMap, - fieldCore: ReadonlyMap, - refCore: ReadonlyMap, + members: ReadonlyMap, budget: ManagedDecodeBudget, ): { readonly callEdges: readonly ManagedCallEdge[]; @@ -350,11 +392,10 @@ export const edges = ( const caller = derives ? callerName(method) : method.name; for (const anchor of method.body.anchors) { if (anchor.opcode === "ldtoken") continue; - if (anchor.operand_kind === "method" && anchor.operand !== null) { - const named = - methodCore.get(anchor.operand)?.name ?? - refCore.get(anchor.operand)?.name ?? - null; + const { operand } = anchor; + if (operand === null) continue; + const target = namedTarget(members, anchor.operand_kind, operand); + if (anchor.operand_kind === "method") { // Each derived record is admitted before the graph grows further. callEdges.push( admitManagedProjection( @@ -362,23 +403,25 @@ export const edges = ( caller_token: method.token, caller, opcode: anchor.opcode, - target_token: anchor.operand, - target_kind: targetKind(anchor.operand), - target_name: named, + target_token: operand, + target_kind: targetKind(operand), + target_name: target?.name ?? null, + target_declaring_type: target?.declaringType ?? null, }, budget, ), ); } - if (anchor.operand_kind === "field" && anchor.operand !== null) { + if (anchor.operand_kind === "field") { fieldAccesses.push( admitManagedProjection( { method_token: method.token, method: caller, opcode: anchor.opcode, - field_token: anchor.operand, - field_name: fieldCore.get(anchor.operand)?.name ?? null, + field_token: operand, + field_name: target?.name ?? null, + field_declaring_type: target?.declaringType ?? null, }, budget, ), diff --git a/src/dotnet/ManagedMethodSpecs.ts b/src/dotnet/ManagedMethodSpecs.ts new file mode 100644 index 000000000..057090862 --- /dev/null +++ b/src/dotnet/ManagedMethodSpecs.ts @@ -0,0 +1,154 @@ +import type { ManagedParseIssue } from "../domain/managed/managedArtifact.js"; +import { + methodGenericArity, + specSignature, + type ManagedMethod, + type MemberCore, +} from "./ManagedMemberInspectorCore.js"; +import type { ManagedMetadataLayout } from "./ManagedMetadataLayout.js"; +import { + metadataCodedToken, + metadataCodedTokenInvalidReason, + metadataRowCursor, + readMetadataBlob, +} from "./ManagedMetadataHeaps.js"; +import { managedFailure, readManagedValue } from "./ManagedReaderFailure.js"; + +/** + * Index fields, member references, and methods by token, then name each + * MethodSpec an inspected body calls after the generic method it instantiates, + * so every edge target resolves through one lookup. + */ +export const edgeTargets = ( + bytes: Buffer, + layout: ManagedMetadataLayout, + methods: readonly ManagedMethod[], + cores: readonly ReadonlyMap[], +): { + readonly members: ReadonlyMap; + readonly issues: readonly ManagedParseIssue[]; +} => { + const members = new Map(cores.flatMap((core) => [...core])); + const issues: ManagedParseIssue[] = []; + const seen = new Set(); + // Generic arity by method token, so instantiations of one method decode it once. + const arities = new Map(); + for (const method of methods) + for (const { operand } of method.body.anchors) { + if (operand?.startsWith("0x2b") !== true || seen.has(operand)) continue; + seen.add(operand); + readManagedValue( + () => + readMethodSpec({ bytes, layout, members, issues, arities }, operand), + issues, + ); + } + return { members, issues }; +}; + +/** + * Map one MethodSpec to its generic method, then check its instantiation + * decodes and matches that method's generic arity. A readable Method column + * keeps its name when the rest is malformed; the failure makes coverage partial. + */ +const readMethodSpec = ( + { + bytes, + layout, + members, + issues, + arities, + }: { + readonly bytes: Buffer; + readonly layout: ManagedMetadataLayout; + readonly members: Map; + readonly issues: ManagedParseIssue[]; + readonly arities: Map; + }, + token: string, +): void => { + const scope = `metadata.MethodSpec:${token}`; + const cursor = metadataRowCursor( + bytes, + layout, + 0x2b, + Number.parseInt(token.slice(4), 16), + ); + const methodRaw = cursor.readIndex(layout.codedIndexSize("MethodDefOrRef")); + const instantiation = cursor.readIndex(layout.blobIndexSize); + const methodReason = metadataCodedTokenInvalidReason( + methodRaw, + 1, + [6, 10], + layout.rowCounts, + ); + if (methodReason !== null || methodRaw === 0) + issues.push({ + code: "invalid-row", + scope, + offset: cursor.start, + detail: + methodReason === null + ? "MethodSpec method coded index 0x0 is null, but the Method column must reference a row" + : `MethodSpec method coded index 0x${methodRaw.toString(16)} is invalid: ${methodReason}`, + }); + const generic = metadataCodedToken(methodRaw, 1, [6, 10], layout.rowCounts); + const target = generic === null ? undefined : members.get(generic); + // A MethodSpec instantiates a method, so a field target never lends it a + // name, even to a field operand. + if (target !== undefined && target.kind !== "field") + members.set(token, { ...target, token }); + const types = specSignature( + layout, + readMetadataBlob(bytes, layout, instantiation, layout.blob.size), + { form: "instantiation", scope, offset: cursor.start }, + ); + if (generic === null) return; + // Prefer the arity the target's owner decoded; read the raw signature only + // for a target nothing decoded, such as a MethodDef outside the selection. + if (target?.genericArity === undefined && !arities.has(generic)) + arities.set( + generic, + methodGenericArity(layout, methodSignatureBlob(bytes, layout, generic)), + ); + const arity = + target?.genericArity === undefined + ? (arities.get(generic) ?? null) + : target.genericArity; + if (arity !== types.length) + throw managedFailure( + "invalid-row", + scope, + arity === null + ? "MethodSpec method does not have a decodable method signature" + : `MethodSpec supplies ${String(types.length)} type argument(s), but its method declares ${String(arity)} generic parameter(s)`, + cursor.start, + ); +}; + +/** The Signature column of a MethodDef or MemberRef row. */ +const methodSignatureBlob = ( + bytes: Buffer, + layout: ManagedMetadataLayout, + token: string, +): Buffer => { + const table = Number.parseInt(token.slice(2, 4), 16); + const cursor = metadataRowCursor( + bytes, + layout, + table, + Number.parseInt(token.slice(4), 16), + ); + if (table === 6) { + cursor.readUInt32(); + cursor.readUInt16(); + cursor.readUInt16(); + } else cursor.readIndex(layout.codedIndexSize("MemberRefParent")); + cursor.readIndex(layout.stringIndexSize); + return readMetadataBlob( + bytes, + layout, + cursor.readIndex(layout.blobIndexSize), + layout.blob.size, + ); +}; diff --git a/src/dotnet/ManagedPe.fixture.ts b/src/dotnet/ManagedPe.fixture.ts index 25f819925..7ea2b06b4 100644 --- a/src/dotnet/ManagedPe.fixture.ts +++ b/src/dotnet/ManagedPe.fixture.ts @@ -29,6 +29,16 @@ interface ManagedPeFixtureOptions { readonly moduleRowCount?: number; readonly assemblyRowCount?: number; readonly memberRefParentRaw?: number; + readonly memberRefName?: string; + readonly memberRefSignature?: Buffer; + /** Raw MethodDefOrRef coded values, one MethodSpec row each. */ + readonly methodSpecMethods?: readonly number[]; + /** Instantiation blob shared by every MethodSpec row. */ + readonly methodSpecInstantiation?: Buffer; + /** TypeSpec signature blobs, one TypeSpec row each. */ + readonly typeSpecSignatures?: readonly Buffer[]; + /** Raw ResolutionScope coded value of the TypeRef row; null by default. */ + readonly typeRefScopeRaw?: number; readonly mvid?: Buffer; readonly pinvoke?: { readonly importName?: string; @@ -174,10 +184,10 @@ const typeRefRow = ( name: number, namespace: number, stringSize: MetadataIndexSize, - resolutionScopeSize: MetadataIndexSize, + resolutionScope: Buffer, ): Buffer => Buffer.concat([ - metadataIndex(0, resolutionScopeSize), + resolutionScope, metadataIndex(name, stringSize), metadataIndex(namespace, stringSize), ]); @@ -524,6 +534,8 @@ interface FixtureMetadataRowContext { readonly methodSignature: number; readonly constructorSignature: number; readonly attributeBlob: number; + readonly typeSpecSignatures: readonly number[]; + readonly methodSpecInstantiation: number; }; readonly heapSizes: { readonly strings: MetadataIndexSize; @@ -595,6 +607,37 @@ const exportedTypeTable = ( ], ]; +const specTables = ( + options: ManagedPeFixtureOptions, + indexes: FixtureMetadataRowContext["indexes"], + heapSizes: FixtureMetadataRowContext["heapSizes"], + widths: FixtureIndexWidths, +): readonly (readonly [number, readonly Buffer[]])[] => [ + ...(indexes.typeSpecSignatures.length === 0 + ? [] + : ([ + [ + 27, + indexes.typeSpecSignatures.map((signature) => + metadataIndex(signature, heapSizes.blobs), + ), + ], + ] as const)), + ...(options.methodSpecMethods === undefined + ? [] + : ([ + [ + 43, + options.methodSpecMethods.map((method) => + Buffer.concat([ + metadataIndex(method, widths.coded([6, 10], 1)), + metadataIndex(indexes.methodSpecInstantiation, heapSizes.blobs), + ]), + ), + ], + ] as const)), +]; + const fixtureIndexWidths = ({ options, referenceStringIndexes, @@ -616,10 +659,12 @@ const fixtureIndexWidths = ({ ], ...(indexes.pinvokeModuleName === null ? [] : ([[26, 1]] as const)), ...(indexes.pinvokeImportName === null ? [] : ([[28, 1]] as const)), + [27, indexes.typeSpecSignatures.length], [32, options.assemblyRowCount ?? 1], [35, referenceStringIndexes.length], [39, indexes.exportedTypes.length], [40, 1], + [43, options.methodSpecMethods?.length ?? 0], ]); const count = (table: number): number => rowCounts.get(table) ?? 0; const table = (index: number): MetadataIndexSize => indexSize(count(index)); @@ -673,7 +718,7 @@ const fixtureMetadataRows = ({ indexes.attributeName, indexes.attributeNamespace, heapSizes.strings, - widths.resolutionScope, + metadataIndex(options.typeRefScopeRaw ?? 0, widths.resolutionScope), ), ], ], @@ -780,6 +825,7 @@ const fixtureMetadataRows = ({ ), ], ...exportedTypeTable(options, indexes, heapSizes, widths), + ...specTables(options, indexes, heapSizes, widths), [ 40, [ @@ -795,6 +841,29 @@ const fixtureMetadataRows = ({ return rows; }; +/** Blob indexes for the MemberRef and optional spec rows; defaults leave the heap unchanged. */ +const referenceBlobIndexes = ( + blobs: BlobHeap, + options: ManagedPeFixtureOptions, +): Pick< + FixtureMetadataRowContext["indexes"], + "constructorSignature" | "typeSpecSignatures" | "methodSpecInstantiation" +> => ({ + constructorSignature: blobs.add( + options.memberRefSignature ?? Buffer.from([0x20, 0x01, 0x01, 0x0e]), + ), + typeSpecSignatures: (options.typeSpecSignatures ?? []).map((signature) => + blobs.add(signature), + ), + // Defaults to GENERICINST with one string type argument. + methodSpecInstantiation: + options.methodSpecMethods === undefined + ? 0 + : blobs.add( + options.methodSpecInstantiation ?? Buffer.from([0x0a, 0x01, 0x0e]), + ), +}); + const buildManagedFixtureMetadata = ( options: ManagedPeFixtureOptions, methodRva: number, @@ -805,7 +874,7 @@ const buildManagedFixtureMetadata = ( const assemblyName = strings.add("Fixture.Managed"); const attributeName = strings.add("TargetFrameworkAttribute"); const attributeNamespace = strings.add("System.Runtime.Versioning"); - const constructorName = strings.add(".ctor"); + const constructorName = strings.add(options.memberRefName ?? ".ctor"); const typeName = strings.add(options.typeName ?? "Program"); const additionalTypeNames = Array.from( { length: options.additionalEmptyTypes ?? 0 }, @@ -840,7 +909,7 @@ const buildManagedFixtureMetadata = ( const methodSignature = blobs.add( options.methodSignature ?? Buffer.from([0x00, 0x00, 0x01]), ); - const constructorSignature = blobs.add(Buffer.from([0x20, 0x01, 0x01, 0x0e])); + const referenceBlobs = referenceBlobIndexes(blobs, options); const attributeBlob = blobs.add( fixedStringAttributeBlob( options.targetFramework === null @@ -873,8 +942,8 @@ const buildManagedFixtureMetadata = ( tokenBlob, fieldSignature, methodSignature, - constructorSignature, attributeBlob, + ...referenceBlobs, }, heapSizes: { strings: stringSize, blobs: blobSize, guids: guidSize }, }); diff --git a/src/dotnet/ManagedTypeNames.ts b/src/dotnet/ManagedTypeNames.ts new file mode 100644 index 000000000..ab3b34c8f --- /dev/null +++ b/src/dotnet/ManagedTypeNames.ts @@ -0,0 +1,137 @@ +import { managedDecodeBudget } from "./ManagedDecodeBudget.js"; +import type { ManagedParseIssue } from "../domain/managed/managedArtifact.js"; +import { + fullName, + readCompressed, + specSignature, + type TypeRange, +} from "./ManagedMemberInspectorCore.js"; +import { + metadataRowOffset, + type ManagedMetadataLayout, +} from "./ManagedMetadataLayout.js"; +import { + metadataCodedTokenInvalidReason, + metadataRowCursor, + metadataToken, + readMetadataBlob, + readMetadataString, +} from "./ManagedMetadataHeaps.js"; +import { managedFailure, readManagedValue } from "./ManagedReaderFailure.js"; + +/** + * Name the type a TypeDef, TypeRef, or generic-instantiation TypeSpec token + * denotes, in the namespace-qualified form TypeDef `full_name` uses, once per + * token. Other decoded TypeSpecs stay null: the type is unknown. Malformed or + * unsupported rows and blobs are reported to `issues` and also stay null, + * except that a TypeRef with an invalid ResolutionScope keeps its readable name. + */ +export const typeNameLookup = ( + bytes: Buffer, + layout: ManagedMetadataLayout, + ranges: readonly TypeRange[], + issues: ManagedParseIssue[], +): ((token: string | null) => string | null) => { + const names = new Map(); + const checkedScopes = new Set(); + const validateScopeChain = (row: number): void => { + const path = new Set(); + let current = row; + while (!checkedScopes.has(current)) { + const cursor = metadataRowCursor(bytes, layout, 1, current); + if (path.has(current)) { + issues.push({ + code: "invalid-row", + scope: `metadata.TypeRef:${metadataToken(1, current)}`, + offset: cursor.start, + detail: "TypeRef ResolutionScope chain contains a cycle", + }); + break; + } + // Admit the chain's visited entry and retained validation-cache entry. + managedDecodeBudget(layout).reserve(128, cursor.start); + path.add(current); + const raw = cursor.readIndex(layout.codedIndexSize("ResolutionScope")); + const reason = metadataCodedTokenInvalidReason( + raw, + 2, + [0, 26, 35, 1], + layout.rowCounts, + ); + if (reason !== null) { + issues.push({ + code: "invalid-row", + scope: `metadata.TypeRef:${metadataToken(1, current)}`, + offset: cursor.start, + detail: `TypeRef ResolutionScope coded index 0x${raw.toString(16)} is invalid: ${reason}`, + }); + break; + } + // Null scopes are legal; non-TypeRef scopes terminate local validation. + if (raw === 0 || (raw & 3) !== 3) break; + current = Math.floor(raw / 4); + } + for (const visited of path) checkedScopes.add(visited); + }; + const definitionName = (table: 1 | 2, row: number): string | null => { + if (table === 2) { + metadataRowOffset(layout, 2, row); + return ranges[row - 1]?.fullName ?? null; + } + const cursor = metadataRowCursor(bytes, layout, 1, row); + cursor.readIndex(layout.codedIndexSize("ResolutionScope")); + readManagedValue(() => validateScopeChain(row), issues); + const name = cursor.readIndex(layout.stringIndexSize); + const namespace = cursor.readIndex(layout.stringIndexSize); + return fullName( + readMetadataString(bytes, layout, namespace, layout.strings.size), + readMetadataString(bytes, layout, name, layout.strings.size), + ); + }; + const specName = (row: number): string | null => { + const cursor = metadataRowCursor(bytes, layout, 27, row); + const spec = readMetadataBlob( + bytes, + layout, + cursor.readIndex(layout.blobIndexSize), + layout.blob.size, + ); + const owner = { + form: "type", + scope: `metadata.TypeSpec:${metadataToken(27, row)}`, + offset: cursor.start, + } as const; + specSignature(layout, spec, owner); + if (spec[0] !== 0x15) return null; + // GENERICINST (CLASS | VALUETYPE) TypeDefOrRefEncoded: never a TypeSpec, + // so naming never recurses. + const encoded = readCompressed(spec, 2).value; + const table = ([2, 1] as const)[encoded & 3]; + if (table === undefined) + throw managedFailure( + "invalid-blob", + owner.scope, + "Generic instantiation does not name a TypeDef or TypeRef", + owner.offset, + ); + return definitionName(table, Math.floor(encoded / 4)); + }; + return (token) => { + if (token === null) return null; + if (names.has(token)) return names.get(token) ?? null; + const table = Number.parseInt(token.slice(2, 4), 16); + const row = Number.parseInt(token.slice(4), 16); + const name = + readManagedValue( + () => + table === 27 + ? specName(row) + : table === 1 || table === 2 + ? definitionName(table, row) + : null, + issues, + ) ?? null; + names.set(token, name); + return name; + }; +}; From 352c46aa7567afcd142524e5f583379c27a1075d Mon Sep 17 00:00:00 2001 From: Aravind Rao <135997804+rekcilyssup@users.noreply.github.com> Date: Tue, 13 Oct 2026 18:42:50 +0530 Subject: [PATCH 03/24] fix(javascript): keep CommonJS require file URLs unresolved (#2077) * fix(javascript): keep CommonJS require file URLs unresolved require("file:///dep.cjs") was reported as a resolved dependency of the inventoried dep.cjs although Node rejects the string with MODULE_NOT_FOUND. The shared module-specifier resolver decoded file URLs before the declared CommonJS loader semantics. File-URL decoding now applies only to non-require module specifiers, so a CommonJS file URL reports external with its declared string and source evidence preserved. ESM file URLs and ordinary relative require paths keep their established lookup. * test(javascript): consolidate CommonJS file URL public workflow coverage --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- tests/acceptance/applications/packageSubpathsCli.test.ts | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tests/acceptance/applications/packageSubpathsCli.test.ts b/tests/acceptance/applications/packageSubpathsCli.test.ts index 15f734e22..869a8deba 100644 --- a/tests/acceptance/applications/packageSubpathsCli.test.ts +++ b/tests/acceptance/applications/packageSubpathsCli.test.ts @@ -141,6 +141,13 @@ cliTest.for(["CLI", "stdio MCP"] as const)( ).toBe("node_modules/fi%xture/actual.cjs"); const { graph } = await analyzeJavaScriptDirectory(surface, cli, root); + expect(graph.coverage.status).toBe("complete"); + const requireFileUrl = graph.edges.find( + ({ properties }) => + properties.module_link_kind === "require" && + properties.specifier === "file:///src/uri-dep.cjs", + ); + expect(requireFileUrl?.evidence.limitations.length).toBeGreaterThan(0); const imports = graph.edges.filter( ({ properties }) => properties.module_link_kind === "require", ); From 9c14999efd94ae04bfaf6532a74613b7a914810c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=AE=B0=E5=BE=97=E5=90=83=E4=B8=AD=E9=A5=AD?= <17373302683@163.com> Date: Tue, 13 Oct 2026 21:13:01 +0800 Subject: [PATCH 04/24] feat(ghidra): support raw NMOS 6502 analysis (#2079) * feat(ghidra): admit raw NMOS 6502 analysis * test(ghidra): require verified POSIX verifier cleanup * test(ghidra): verify 6502 low-memory raw imports * docs(skill): document raw NMOS 6502 support * fix(ghidra): preserve requested Windows raw source paths * docs(ghidra): include raw 6502 in Windows installation scope * test(ghidra): verify Windows raw junction rejection through CLI and MCP * fix(validation): make fixtures portable and preserve explicit Codex home * fix(ghidra): attest raw mapping state and address space * fix(ghidra): align diagnostics and regression groups * fix(ghidra): bind raw attestation to language address space * chore(ghidra): avoid verifier insertion conflict * docs(ghidra): align linked raw admission guides * fix(ghidra): retain 6502 verifier workspace on failure * test(ghidra): isolate Windows authority doctor fixture * fix(binary): re-admit Windows raw paths on session reopen * test(binary): require fresh provider execution after path change * Constrain 6502 to raw lifecycle and snapshot targets * Bypass saved reads for unadmitted Windows paths * Guard direct CLI snapshot replay on Windows paths * fix(snapshot): gate Windows Ghidra replay on native admission * fix(snapshot): constrain nested 6502 subjects to raw images * test: make dyld slice fixtures portable --- .../references/native-and-artifacts.md | 7 +- bridge/ghidra/ReaGhidraPrepareRaw.java | 17 +- docs/cli.md | 4 +- docs/ghidra-mips.md | 2 +- docs/ghidra-riscv.md | 4 +- docs/installation.md | 5 +- docs/native-investigation.md | 7 +- docs/raw-binary.md | 63 ++- docs/testing.md | 10 + docs/windows-ghidra-p0.md | 14 +- .../analysis-efficiency/agent-trial.py | 4 +- .../analysis-efficiency/test_codex_context.py | 5 +- package.json | 1 + scripts/verify-real-ghidra-6502.mjs | 479 ++++++++++++++++++ src/application/BinaryTargetResolver.ts | 12 +- src/application/DirectAnalysis.ts | 6 + .../binary/AnalysisSnapshotCache.ts | 5 + src/application/binary/BinarySession.ts | 12 + .../binary/BinarySessionExecution.ts | 16 +- src/application/binary/BinarySessionOpen.ts | 18 +- src/contracts/toolOutputReferences.test.ts | 28 + src/contracts/toolOutputSchemaGroups.ts | 2 +- src/domain/analysisSnapshot.test.ts | 67 ++- src/domain/analysisSnapshot.ts | 75 ++- src/domain/binaryTarget.ts | 4 +- src/domain/binaryTargetTypes.ts | 16 +- src/domain/evidence.test.ts | 27 + src/domain/evidence.ts | 76 +-- src/domain/evidenceBundle.ts | 23 +- src/domain/native/nativeLoadImage.ts | 3 +- src/domain/rawBinary.test.ts | 38 ++ src/domain/rawBinary.ts | 13 +- src/ghidra/GhidraAnalysisProfile.ts | 2 +- src/ghidra/GhidraLoadImageValues.ts | 33 +- src/ghidra/GhidraProvider.behavior.test.ts | 73 ++- src/ghidra/GhidraProvider.ts | 9 +- src/ghidra/GhidraProviderCapabilities.ts | 2 +- src/ghidra/GhidraProviderClient.ts | 2 +- src/ghidra/GhidraRawBinaryLanguage.ts | 37 +- src/ghidra/GhidraRawBinaryLoadImage.test.ts | 138 +++++ src/ghidra/GhidraRiscvProfile.test.ts | 11 +- src/hopper/HopperAnalysisProfile.ts | 3 +- src/hopper/HopperMachOImage.ts | 11 +- tests/boundary/cli/dylibResolutionCli.test.ts | 16 +- tests/boundary/cli/rawBinaryCli.test.ts | 30 ++ .../directAnalysisSnapshotReplay.test.ts | 95 +++- .../javascriptArtifactReconstruction.test.ts | 6 +- .../filesystem/rawBinaryTarget.test.ts | 39 +- tests/boundary/mcp/rawBinaryMcp.test.ts | 231 ++++++++- ...actProvider.extractionCompleteness.test.ts | 2 +- tests/conformance/ghidra/6502.cfg | 9 + tests/conformance/ghidra/6502.s | 19 + tests/conformance/readmeConsistency.test.ts | 14 + tests/fixtures/strippedAsarAddon.ts | 2 +- 54 files changed, 1650 insertions(+), 197 deletions(-) create mode 100644 scripts/verify-real-ghidra-6502.mjs create mode 100644 tests/conformance/ghidra/6502.cfg create mode 100644 tests/conformance/ghidra/6502.s diff --git a/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md b/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md index 6a942c73c..a4fc6d18f 100644 --- a/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md +++ b/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md @@ -224,10 +224,13 @@ context. PSP/stack/device state remains unmodeled. See [DOS guide](https://githu For other headerless images (embedded firmware without ELF/Mach-O/PE/MZ headers), open with `format: "raw-binary"`, `architecture` (`x86`, `x86_64`, -`arm`, or `arm64`), and `entry_point` (`0x` hex). File offset 0 is mapped at +`arm`, `arm64`, or `6502`), and `entry_point` (`0x` hex). File offset 0 is mapped at that address. Hopper and IDA refuse this interpretation; Ghidra uses `BinaryLoader` with a little-endian generic language. Big-endian, Thumb-only, -and Cortex-M variants are unsupported. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/raw-binary.md). +and Cortex-M variants are unsupported. `6502` selects NMOS 6502; the entry and +complete file mapping must fit within `0x0000..0xffff`. 65C02, 65816, cartridge +containers, bank switching, and mapper semantics are unsupported. Windows +Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/raw-binary.md). ### Function annotations in Ghidra diff --git a/bridge/ghidra/ReaGhidraPrepareRaw.java b/bridge/ghidra/ReaGhidraPrepareRaw.java index 1a5db9627..143331390 100644 --- a/bridge/ghidra/ReaGhidraPrepareRaw.java +++ b/bridge/ghidra/ReaGhidraPrepareRaw.java @@ -1,6 +1,8 @@ import ghidra.app.script.GhidraScript; import ghidra.program.model.address.Address; import ghidra.program.model.listing.Program; +import ghidra.program.model.mem.MemoryBlock; +import ghidra.program.model.mem.MemoryBlockSourceInfo; /** Seed the caller-supplied raw-binary entry before default auto-analysis. */ public final class ReaGhidraPrepareRaw extends GhidraScript { @@ -21,8 +23,19 @@ public final class ReaGhidraPrepareRaw extends GhidraScript { ); } Address entry = currentProgram.getAddressFactory().getDefaultAddressSpace().getAddress(entryHex); - Address start = currentProgram.getMinAddress(); - if (entry == null || !entry.equals(start)) { + MemoryBlock block = entry == null ? null : currentProgram.getMemory().getBlock(entry); + boolean fileStart = false; + if (block != null) { + // Processor-defined zero page/stack blocks can precede the imported file. + for (MemoryBlockSourceInfo source : block.getSourceInfos()) { + if (source.getFileBytes().isPresent() && source.getFileBytesOffset() == 0 && + entry.equals(source.getMinAddress())) { + fileStart = true; + break; + } + } + } + if (!fileStart) { throw new IllegalArgumentException( "Raw binary import must map file offset 0 at the caller-supplied entry" ); diff --git a/docs/cli.md b/docs/cli.md index dcd949e11..6d81ec7c1 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -174,7 +174,9 @@ rea analyze /absolute/path/to/program --provider ghidra --snapshot /absolute/pat rea analyze /absolute/path/to/program --provider ghidra --snapshot /absolute/path/to/analysis/program.json ``` -An exact CLI cache hit is read before starting a provider process. +An exact CLI cache hit is normally read before starting a provider process. +On Windows, Ghidra first admits the selected source through its native snapshot +owner, so replay can start that provider. In MCP, `open_binary` accepts `snapshot_path` to import a snapshot atomically for its matching target; an MCP provider may still start before returning a cached result. `close_binary` accepts `snapshot_path` and optional diff --git a/docs/ghidra-mips.md b/docs/ghidra-mips.md index 59d04a8b0..d5b58f00b 100644 --- a/docs/ghidra-mips.md +++ b/docs/ghidra-mips.md @@ -111,7 +111,7 @@ MIPS header and ABI record facts are committed in the analysis profile so interpretation changes invalidate profile-bound snapshots. The support-lane identity is versioned independently of file identity. Evidence, lifecycle outputs and saved snapshots preserve the `mips` family. Hopper and IDA adapters -do not implicitly gain MIPS support. Windows Ghidra P0 remains PE x86/x86-64 only. +do not implicitly gain MIPS support. Windows Ghidra P0 does not admit MIPS targets. ### Soft-float prototype diff --git a/docs/ghidra-riscv.md b/docs/ghidra-riscv.md index 5b3407361..afae01948 100644 --- a/docs/ghidra-riscv.md +++ b/docs/ghidra-riscv.md @@ -11,8 +11,8 @@ ELF `EM_RISCV` (243) is retained as the provider-neutral `riscv` family. The target also retains the raw ELF declarations that decide a lane: class, byte order, `e_type`, and `e_flags`. Recognition does not mean every provider or RISC-V variant is supported. Existing PE, managed and Mach-O architecture sets -are unchanged, and caller-supplied headerless raw-binary entry still accepts only -its existing four families. +are unchanged. Headerless inputs use the separate [raw-binary admission +boundary](raw-binary.md); raw RISC-V images are not supported. The Ghidra lane admits little-endian ELF32 and ELF64 `ET_EXEC` images that declare `EF_RISCV_FLOAT_ABI_DOUBLE` in `e_flags`, with no `EF_RISCV_RVE`, diff --git a/docs/installation.md b/docs/installation.md index 3bd4e0928..d6dc8e3b0 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -512,7 +512,10 @@ and `set_addresses_names` name a function entry or any other mapped address session metadata edits leave executable bytes unchanged and are discarded on close. GUI controls require Hopper; Windows P0 remains read-only. -Windows P0 admits native x86 and x86-64 PE applications and DLLs on fixed local NTFS volumes. +Windows P0 admits native x86 and x86-64 PE applications and DLLs, plus explicitly +selected [raw NMOS 6502 images](raw-binary.md#nmos-6502) with a caller-supplied +entry at file offset 0, on fixed local NTFS volumes. Other raw CPU families remain +unsupported on Windows. The npm package bundles native Job Object ownership, protected private runtime DACLs, and handle-based path admission; no separate addon installation is needed. See the [Windows Ghidra P0 guide](windows-ghidra-p0.md) for verified scope and diff --git a/docs/native-investigation.md b/docs/native-investigation.md index 27381246b..5a2fcbd31 100644 --- a/docs/native-investigation.md +++ b/docs/native-investigation.md @@ -476,8 +476,11 @@ the tool fails with a `debugger-attach-denied` reason. Install a Ghidra 12.1.x release and the 64-bit full JDK it declares, then configure REA to use them. Ghidra analysis supports Linux x64/arm64 and macOS x64/arm64; each installation requires the matching native decompiler. Experimental Windows x64 P0 admits native -x86 and x86-64 PE applications and DLLs on local NTFS using bundled Job Object ownership, -protected runtime DACLs, and handle-based path admission. See [Windows Ghidra P0](windows-ghidra-p0.md) and +x86 and x86-64 PE applications and DLLs, plus explicitly selected +[raw NMOS 6502 images](raw-binary.md#nmos-6502) with a caller-supplied load address, +on fixed local NTFS volumes. Other raw CPU families remain unsupported on Windows. +The boundary uses bundled Job Object ownership, protected runtime DACLs, and +handle-based path admission. See [Windows Ghidra P0](windows-ghidra-p0.md) and [issue #527](https://github.com/morluto/rea/issues/527). On Linux and macOS, `annotate_native_function` atomically edits a function name and entry comments in the ephemeral database, returning refreshed analysis diff --git a/docs/raw-binary.md b/docs/raw-binary.md index 8e277568b..8d251d010 100644 --- a/docs/raw-binary.md +++ b/docs/raw-binary.md @@ -25,10 +25,12 @@ rea inspect-native-load-image /absolute/path/to/ecu.bin --target-format raw-bina } ``` -`architecture` is one of `x86`, `x86_64`, `arm`, or `arm64`. `entry_point` is a +`architecture` is one of `x86`, `x86_64`, `arm`, `arm64`, or `6502`. `entry_point` is a hexadecimal address of the form `0x…`. The complete file is mapped starting at that address, and Ghidra seeds an `entry` function there before auto-analysis. -ARM and AArch64 entries must be 4-byte aligned; x86 entries may be byte-aligned. +ARM and AArch64 entries must be 4-byte aligned; x86 and 6502 entries may be byte-aligned. +For 6502, the entry and complete mapped file must fit within `0x0000..0xffff`; +images that cross the 16-bit boundary are rejected before provider startup. `open_binary` and an open `binary_session` return the canonical `entry_point` for raw targets so callers can verify the selected mapping directly. Omitted `format` keeps header-based detection; architecture and entry without @@ -37,12 +39,13 @@ Omitted `format` keeps header-based detection; architecture and entry without Ghidra imports a private snapshot with `BinaryLoader` and these little-endian generic languages and compiler specifications: -| architecture | language_id | compiler_spec_id | -| ------------ | ------------------- | ---------------- | -| x86 | `x86:LE:32:default` | `gcc` | -| x86_64 | `x86:LE:64:default` | `gcc` | -| arm | `ARM:LE:32:v7` | `default` | -| arm64 | `AARCH64:LE:64:v8A` | `default` | +| architecture | language_id | compiler_spec_id | +| ------------ | -------------------- | ---------------- | +| x86 | `x86:LE:32:default` | `gcc` | +| x86_64 | `x86:LE:64:default` | `gcc` | +| arm | `ARM:LE:32:v7` | `default` | +| arm64 | `AARCH64:LE:64:v8A` | `default` | +| 6502 | `6502:LE:16:default` | `default` | Compiler specifications select analysis conventions; they do not identify the original compiler or ABI of a headerless target. Ghidra's x86 languages do not @@ -57,11 +60,51 @@ The mapped start is independent of Ghidra's program image base, which remains `0x0` for BinaryLoader. Ghidra may report additional language-defined entry points, including unmapped ARM vectors. Load-image verification requires the caller-supplied entry and preserves those other entries as observations. +For 6502, BinaryLoader maps the file before adding its language-defined zero-page +and stack blocks. Conflicting default blocks are skipped; low-address files stay +in the default address space rather than an overlay. Non-conflicting uninitialized +blocks remain observations, not file-backed bytes. Independent raw verification streams the complete private snapshot digest and extent with bounded memory and supports cancellation during the read. +It also requires the source-backed block to be initialized, loaded, and +non-overlay in the bound language's default address space. Contradictory mapping +observations are retained with `mismatch`, even when the source hashes agree. -Hopper and IDA refuse raw-binary targets. Windows Ghidra P0 continues to admit -native PE applications and DLLs only; raw-binary targets are unsupported. +Hopper and IDA refuse raw-binary targets. Windows Ghidra P0 admits native PE +applications/DLLs and explicitly selected raw 6502 images. Other raw CPU families +remain outside the Windows boundary. Native path admission, private DACLs, Job +Object ownership, immutable snapshots, and the no-mutation policy still apply. + +## NMOS 6502 + +```bash +rea function /absolute/path/to/image.bin 0x8001 --target-format raw-binary --architecture 6502 --entry-point 0x8001 --provider ghidra --json +rea inspect-native-load-image /absolute/path/to/image.bin --target-format raw-binary --architecture 6502 --entry-point 0x8001 --provider ghidra --json +``` + +Select only headerless NMOS 6502 code with a known entry at file offset 0. +The built-in Ghidra processor supplies static instructions, functions, +pseudocode, and references. Evidence preserves `architecture: "6502"`, the +selected language, entry, file digest, and verified source mapping. Register and +calling conventions are analysis conventions, not a recovered source ABI. + +This lane does not interpret iNES headers, cartridge mappers, bank switching, +or other console-specific memory layouts. 65C02 and 65816 programs need different +processor semantics and are not admitted as those variants. REA does not infer +the CPU from the file suffix, guess the reset vector, or emulate the target. + +`npm run verify:ghidra:6502` assembles and links the source-owned fixture with +`ca65` and `ld65` (tested from cc65 tag `V2.19`, whose tools report `V2.18`). +Set `REA_CA65` and `REA_LD65` to explicit executable paths if needed. The lane +checks real CLI/MCP load-image parity, mapped bytes, decoded calls, decompilation, +invalid mappings, unchanged source bytes and runtime cleanup. +Low-memory checks include a file spanning the zero-page/stack boundary at `0xff` +and a complete 64-KiB image at `0x0`, with verified default-space mappings through +both CLI and MCP. On Windows, the lane also checks that a caller-selected +junction path is rejected at snapshot admission through CLI and MCP, with +unchanged source bytes and complete runtime cleanup. Windows requires the +matching native addon; the POSIX provider path has not been exercised by the +Windows run. ## Limitations diff --git a/docs/testing.md b/docs/testing.md index e8f36dc0c..7aa80d22a 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -734,6 +734,16 @@ truncation constraint, while leading-zero coordinates still resolve correctly. It has the same Ghidra/JDK prerequisites as the MZ lane. Neither lane claims DOS runtime or PC-98 device execution. +`npm run verify:ghidra:6502` uses `ca65` and `ld65` to build the source-owned +NMOS 6502 fixture from `tests/conformance/ghidra/6502.s` and `6502.cfg`. Set +`REA_CA65` and `REA_LD65` for non-PATH installations (tested with cc65 `V2.19`). +With an installed Ghidra 12.1.4 and full JDK, it verifies real CLI/stdio MCP +mapping parity, exact source bytes, functions, a direct call reference, +decompilation, 16-bit overflow rejection without replacing the active session, +and runtime cleanup. Windows additionally requires the matching native addon; +its verifier lineage remains explicitly unavailable outside the provider's +retained Job Object, as in the other Windows lanes. + `npm run verify:ghidra:raw` uses a generated ARM little-endian headerless image through the compiled CLI and stdio MCP server. With the same Ghidra/JDK prerequisites, it checks preparation and bridge startup, verified load mappings, diff --git a/docs/windows-ghidra-p0.md b/docs/windows-ghidra-p0.md index 9cd3bb6d8..5def0dae4 100644 --- a/docs/windows-ghidra-p0.md +++ b/docs/windows-ghidra-p0.md @@ -17,14 +17,16 @@ additional permission flag or degraded mode is required. - Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+; - an operator-installed Ghidra 12.1.x distribution (verified with 12.1.4); - a 64-bit full JDK inside that installation's declared Java range (JDK 21 or newer, with no maximum, for current 12.1 releases); -- an explicit native, non-managed x86 or x86-64 PE application or DLL; and +- an explicit native, non-managed x86 or x86-64 PE application/DLL, or a + [raw NMOS 6502 image](raw-binary.md#nmos-6502) with a caller-supplied entry; and - the 25 read-only Ghidra inventory, memory, and function-analysis operations. Loaded memory reads and file offsets preserve Ghidra's observed source mappings. PE load-image inspection returns an explicit unsupported attestation status. Independent verification covers admitted DOS MZ, explicit COM, and explicit -[raw-binary imports](raw-binary.md) on their supported hosts. This does not -extend Windows P0 admission beyond native PE. For x64 PE +[raw-binary imports](raw-binary.md) on their supported hosts, including the +admitted raw NMOS 6502 mapping on Windows. Windows P0 admission remains limited +to native PE and raw NMOS 6502 images. For x64 PE NativeAOT RTR 9.1, the read-only snapshot parser can report metadata through `inspect_native_load_image` and `inspect_native_data_type` without a JAR or database mutation. A real Windows-host NativeAOT Ghidra run is still required @@ -35,6 +37,12 @@ state, annotations, other target architectures and formats, and Hopper-only operations remain unavailable through this provider. Managed PE/CLI inspection uses its separate execution-free provider. +Raw 6502 imports retain the same native filesystem and process ownership +requirements. The complete image must fit in the 16-bit address space; no +console-specific cartridge loader, CPU variant, or bank-switching model is +implied. Run `npm run verify:ghidra:6502` with caller-installed `ca65`/`ld65` +to check the source-built fixture through real CLI and MCP workflows. + REA does not install or upgrade Ghidra, Java, Python, npm, Node.js, Hopper, or a compiler. The adapter uses the packaged Java `HeadlessScript`; Python and PyGhidra are not prerequisites. Users do not build the native addon. diff --git a/experiments/analysis-efficiency/agent-trial.py b/experiments/analysis-efficiency/agent-trial.py index 6dcc7d7f8..dc13e8781 100644 --- a/experiments/analysis-efficiency/agent-trial.py +++ b/experiments/analysis-efficiency/agent-trial.py @@ -94,6 +94,9 @@ def main(): if undeclared: raise ValueError('Declare inherited provider settings explicitly in --providers: ' + ', '.join(undeclared) + '. Values are not logged.') + source_auth_home = (Path(os.environ['CODEX_HOME']) if 'CODEX_HOME' in os.environ + else Path.home() / '.codex') + source_auth = source_auth_home / 'auth.json' args.output.mkdir(parents=True, exist_ok=False, mode=0o700) runtime_files, runtime_digest = runtime_fingerprint(repository) (args.output / 'runtime-files.json').write_text(json.dumps(runtime_files)) @@ -103,7 +106,6 @@ def main(): workspace = profile / 'workspace' for directory in [trial_codex_home, account, workspace]: directory.mkdir(mode=0o700) - source_auth = Path(os.environ.get('CODEX_HOME', str(Path.home() / '.codex'))) / 'auth.json' auth = trial_codex_home / 'auth.json' started = time.monotonic() timeout = False diff --git a/experiments/analysis-efficiency/test_codex_context.py b/experiments/analysis-efficiency/test_codex_context.py index 37df83a7c..0b8e0247c 100644 --- a/experiments/analysis-efficiency/test_codex_context.py +++ b/experiments/analysis-efficiency/test_codex_context.py @@ -71,7 +71,9 @@ class ContextCaptureTests(unittest.TestCase): self.assertNotIn(marker, text) self.assertIn('caller-selected evidence and paths', text) self.assertIn('original-response', text) - self.assertEqual((output / 'codex-tool-context.jsonl').stat().st_mode & 0o777, 0o600) + # Windows mode bits do not describe access control; this is a POSIX assertion. + if os.name == 'posix': + self.assertEqual((output / 'codex-tool-context.jsonl').stat().st_mode & 0o777, 0o600) def test_missing_rollouts_are_unknown_and_malformed_lines_remain_partial(self): with tempfile.TemporaryDirectory() as directory: @@ -124,6 +126,7 @@ class ContextCaptureTests(unittest.TestCase): patch.object(agent.subprocess, 'check_output', return_value='v24.18.0\n'), \ patch.object(agent.subprocess, 'Popen', return_value=child), \ patch.object(agent, 'capture_tool_context', side_effect=OSError('fixture capture failure')), \ + patch.object(agent.Path, 'home', side_effect=AssertionError('Explicit CODEX_HOME must not resolve the default home')), \ patch('builtins.print'): with self.assertRaisesRegex(OSError, 'fixture capture failure'): agent.main() diff --git a/package.json b/package.json index b2e2a51d3..7ee2b42ab 100644 --- a/package.json +++ b/package.json @@ -174,6 +174,7 @@ "verify:ghidra:seeds": "npm run build:cached && node scripts/verify/ghidra/seeds.mjs", "verify:ghidra:mips": "npm run build:cached && node scripts/verify-real-ghidra-mips.mjs", "verify:ghidra:riscv": "npm run build:cached && node scripts/verify-real-ghidra-riscv.mjs", + "verify:ghidra:6502": "npm run build:cached && node scripts/verify-real-ghidra-6502.mjs", "verify:ghidra:mips-shared": "npm run build:cached && node scripts/verify-real-ghidra-mips-shared.mjs", "verify:ghidra:mips-relocatable": "npm run build:cached && node scripts/verify-real-ghidra-mips-relocatable.mjs", "verify:ghidra:swift": "npm run build:cached && node scripts/verify-real-ghidra-swift.mjs", diff --git a/scripts/verify-real-ghidra-6502.mjs b/scripts/verify-real-ghidra-6502.mjs new file mode 100644 index 000000000..87fb01d41 --- /dev/null +++ b/scripts/verify-real-ghidra-6502.mjs @@ -0,0 +1,479 @@ +import assert from "node:assert/strict"; +import { execFile } from "node:child_process"; +import { createHash } from "node:crypto"; +import { + mkdir, + mkdtemp, + readFile, + readdir, + rm, + symlink, + unlink, + writeFile, +} from "node:fs/promises"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { promisify } from "node:util"; + +import { Client } from "@modelcontextprotocol/client"; +import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; +import { parseEvidence } from "../dist/domain/evidence.js"; +import { requireMcpOperationResult } from "./lib/mcp-verifier-results.mjs"; +import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs"; + +if (process.argv.length !== 2) + throw new Error("Usage: node scripts/verify-real-ghidra-6502.mjs"); +if (process.env.GHIDRA_INSTALL_DIR === undefined) + throw new Error( + "Set GHIDRA_INSTALL_DIR to a bring-your-own Ghidra installation.", + ); +const exec = promisify(execFile); +const run = createVerifierRun(); +const root = fileURLToPath(new URL("..", import.meta.url)); +const ca65 = process.env.REA_CA65 ?? "ca65"; +const ld65 = process.env.REA_LD65 ?? "ld65"; +const toolchain = {}; +for (const [name, command] of Object.entries({ ca65, ld65 })) { + const version = await exec(command, ["--version"], { timeout: 10000 }); + toolchain[name] = (version.stdout + version.stderr).trim(); +} +await mkdir(join(root, "build"), { recursive: true }); +const workspace = await mkdtemp(join(root, "build", "rea-6502-proof-")); +const runtime = join(workspace, "runtime"); +await mkdir(runtime); +const path = join(workspace, "image.bin"); +const object = join(workspace, "image.o"); +const entrypoint = join(root, "scripts", "rea.mjs"); +const env = { + ...process.env, + TMPDIR: runtime, + TMP: runtime, + TEMP: runtime, + REA_LOG_LEVEL: "silent", + REA_ANALYSIS_PROVIDER: "ghidra", + REA_PROCESS_RUN_ID: run.run_id, +}; +const transport = new StdioClientTransport({ + command: process.execPath, + args: [entrypoint, "mcp"], + env, + stderr: "pipe", +}); +const client = new Client({ name: "rea-real-6502-proof", version: "1" }); +let stderr = ""; +transport.stderr?.on("data", (chunk) => { + stderr = (stderr + chunk.toString()).slice(-65536); +}); +let report; +try { + await exec( + ca65, + [join(root, "tests/conformance/ghidra/6502.s"), "-o", object], + { timeout: 10000 }, + ); + await exec( + ld65, + ["-C", join(root, "tests/conformance/ghidra/6502.cfg"), "-o", path, object], + { timeout: 10000 }, + ); + const bytes = await readFile(path); + // Independently known opcodes and link addresses from the source fixture. + assert.equal(bytes.toString("hex"), "a907201080c90ad003a92a60a9006018690360"); + const sha256 = createHash("sha256").update(bytes).digest("hex"); + await client.connect(transport); + const opened = await call("open_binary", { + path, + format: "raw-binary", + architecture: "6502", + entry_point: "0x008001", + provider_id: "ghidra", + }); + assert.equal(opened.architecture, "6502"); + assert.equal(opened.entry_point, "0x8001"); + const status = await call("binary_session"); + const image = evidence(await call("inspect_native_load_image"), sha256); + assert.equal( + image.normalized_result.status, + "verified", + JSON.stringify(image.normalized_result), + ); + assert.equal(image.normalized_result.architecture, "6502"); + assert.equal(image.normalized_result.base_address, "0x8001"); + assert.equal(image.normalized_result.language_id, "6502:LE:16:default"); + assert.equal(image.normalized_result.compiler_spec_id, "default"); + assert.equal(image.normalized_result.file_bytes, bytes.length); + const memory = evidence( + await call("read_bytes", { address: "0x8001", length: bytes.length }), + sha256, + ); + assert.equal(memory.normalized_result.bytes_hex, bytes.toString("hex")); + const procedures = evidence(await call("list_procedures"), sha256); + for (const address of ["0x8001", "0x8010"]) + assert.ok( + procedures.normalized_result.some( + (procedure) => procedure.address === address, + ), + ); + const instruction = evidence( + await call("inspect_native_instruction", { address: "0x8003" }), + sha256, + ); + assert.equal(instruction.normalized_result.mnemonic.toUpperCase(), "JSR"); + assert.ok( + instruction.normalized_result.references.some( + (reference) => reference.call && reference.target_address === "0x8010", + ), + ); + const pseudo = evidence( + await call("procedure_pseudo_code", { procedure: "0x8010" }), + sha256, + ); + assert.match(pseudo.normalized_result, /\+\s*(?:3|0x3|'\\x03')/u); + const filePseudo = evidence( + await call("procedure_pseudo_code", { procedure: "file-offset:0xf" }), + sha256, + ); + assert.equal(filePseudo.normalized_result, pseudo.normalized_result); + assert.ok( + filePseudo.locations.some( + (location) => location.kind === "file-offset" && location.offset === 15, + ), + ); + const rejected = await client.callTool({ + name: "open_binary", + arguments: { + path, + format: "raw-binary", + architecture: "6502", + entry_point: "0xffff", + }, + }); + assert.equal(rejected.isError, true); + assert.match(JSON.stringify(rejected.content), /6502 address width/u); + const preserved = await call("binary_session"); + assert.equal(preserved.analysis_run.run_id, status.analysis_run.run_id); + assert.equal(preserved.entry_point, "0x8001"); + await call("close_binary"); + await client.ping(); + const { stdout } = await exec( + process.execPath, + [ + entrypoint, + "inspect-native-load-image", + path, + "--target-format", + "raw-binary", + "--architecture", + "6502", + "--entry-point", + "0x8001", + "--provider", + "ghidra", + "--json", + ], + { env, timeout: 240000, maxBuffer: 16 * 1024 * 1024 }, + ); + const cli = parseEvidence(JSON.parse(stdout)); + assert.deepEqual(cli.normalized_result, image.normalized_result); + assert.equal(cli.analysis_profile.digest, image.analysis_profile.digest); + assert.equal(cli.subject.digest.sha256, sha256); + const fileCli = await exec( + process.execPath, + [ + entrypoint, + "decompile", + path, + "--file-offset=0xf", + "--target-format", + "raw-binary", + "--architecture", + "6502", + "--entry-point", + "0x8001", + "--provider", + "ghidra", + "--json", + ], + { env, timeout: 240000, maxBuffer: 16 * 1024 * 1024 }, + ); + const fileCliEvidence = parseEvidence(JSON.parse(fileCli.stdout)); + assert.equal(fileCliEvidence.normalized_result, filePseudo.normalized_result); + assert.deepEqual(fileCliEvidence.locations, filePseudo.locations); + assert.equal(fileCliEvidence.subject.digest.sha256, sha256); + assert.deepEqual(await readFile(path), bytes); + assert.deepEqual(await readdir(runtime), []); + const lowMemory = []; + // BinaryLoader maps file bytes before trying processor-defined blocks. Prove + // that overlapping zero-page/stack definitions never overlay the selected image. + for (const [entry, fullAddressSpace] of [ + [0xff, false], + [0, true], + ]) { + const lowPath = join(workspace, `low-${entry}.bin`); + await exec( + ld65, + [ + "-C", + join(root, "tests/conformance/ghidra/6502.cfg"), + "--start-addr", + String(entry), + "-o", + lowPath, + object, + ], + { timeout: 10000 }, + ); + const linked = await readFile(lowPath); + const expectedCode = Buffer.from( + "a907200000c90ad003a92a60a9006018690360", + "hex", + ); + expectedCode.writeUInt16LE(entry + 15, 3); + assert.deepEqual(linked, expectedCode); + const lowBytes = fullAddressSpace + ? Buffer.concat([linked, Buffer.alloc(65536 - linked.length)]) + : linked; + await writeFile(lowPath, lowBytes); + const lowSha256 = createHash("sha256").update(lowBytes).digest("hex"); + const lowEntry = `0x${entry.toString(16)}`; + const lowOpened = await call("open_binary", { + path: lowPath, + format: "raw-binary", + architecture: "6502", + entry_point: lowEntry, + provider_id: "ghidra", + }); + assert.equal(lowOpened.entry_point, lowEntry); + const lowImage = evidence( + await call("inspect_native_load_image"), + lowSha256, + ); + assert.equal( + lowImage.normalized_result.status, + "verified", + JSON.stringify(lowImage.normalized_result), + ); + assert.equal(lowImage.normalized_result.base_address, lowEntry); + assert.equal(lowImage.normalized_result.file_bytes, lowBytes.length); + const fileMappings = + lowImage.normalized_result.observations.mappings.filter( + (mapping) => mapping.source_file_index !== null, + ); + assert.equal(fileMappings.length, 1); + assert.equal( + fileMappings[0].address_space, + lowImage.normalized_result.observations.default_address_space, + ); + assert.equal(fileMappings[0].overlay, false); + assert.equal(fileMappings[0].start, lowEntry); + assert.equal(fileMappings[0].length, lowBytes.length); + const lowRead = evidence( + await call("read_bytes", { address: lowEntry, length: linked.length }), + lowSha256, + ); + assert.equal(lowRead.normalized_result.bytes_hex, linked.toString("hex")); + const lowCall = evidence( + await call("inspect_native_instruction", { + address: `0x${(entry + 2).toString(16)}`, + }), + lowSha256, + ); + assert.equal(lowCall.normalized_result.mnemonic.toUpperCase(), "JSR"); + assert.ok( + lowCall.normalized_result.references.some( + (reference) => + reference.call && + reference.target_address === `0x${(entry + 15).toString(16)}`, + ), + ); + await call("close_binary"); + const lowCli = await exec( + process.execPath, + [ + entrypoint, + "inspect-native-load-image", + lowPath, + "--target-format", + "raw-binary", + "--architecture", + "6502", + "--entry-point", + lowEntry, + "--provider", + "ghidra", + "--json", + ], + { env, timeout: 240000, maxBuffer: 16 * 1024 * 1024 }, + ); + const lowCliEvidence = evidence(JSON.parse(lowCli.stdout), lowSha256); + assert.deepEqual( + lowCliEvidence.normalized_result, + lowImage.normalized_result, + ); + assert.equal( + lowCliEvidence.analysis_profile.digest, + lowImage.analysis_profile.digest, + ); + assert.deepEqual(await readFile(lowPath), lowBytes); + assert.deepEqual(await readdir(runtime), []); + lowMemory.push({ + entry: lowEntry, + bytes: lowBytes.length, + sha256: lowSha256, + }); + } + let windowsReparseAdmission = "not-applicable"; + if (process.platform === "win32") { + const junction = join(workspace, "linked"); + await symlink(workspace, junction, "junction"); + try { + const linkedPath = join(junction, "image.bin"); + // Reopening the same bytes through a junction must replace the direct session. + await call("open_binary", { + path, + format: "raw-binary", + architecture: "6502", + entry_point: "0x8001", + provider_id: "ghidra", + }); + const directImage = evidence( + await call("inspect_native_load_image"), + sha256, + ); + assert.equal(directImage.normalized_result.status, "verified"); + // Canonical file identity must not bypass admission of the new path. + await call("open_binary", { + path: linkedPath, + format: "raw-binary", + architecture: "6502", + entry_point: "0x8001", + provider_id: "ghidra", + }); + // open_binary selects a lazy session; the first analysis admits the snapshot. + const linkedImage = await client.callTool( + { name: "inspect_native_load_image", arguments: {} }, + { timeout: 240000 }, + ); + assert.equal(linkedImage.isError, true); + assert.match( + JSON.stringify(linkedImage), + /Reparse path component is unsupported/u, + ); + await call("close_binary"); + const snapshotPath = join(workspace, "direct-snapshot.json"); + const cliArgs = (selectedPath) => [ + entrypoint, + "analyze", + selectedPath, + "--target-format", + "raw-binary", + "--architecture", + "6502", + "--entry-point", + "0x8001", + "--provider", + "ghidra", + "--snapshot", + snapshotPath, + "--json", + ]; + const saved = await exec(process.execPath, cliArgs(path), { + env, + timeout: 240000, + maxBuffer: 16 * 1024 * 1024, + }); + assert.equal( + evidence(JSON.parse(saved.stdout), sha256).operation, + "binary_overview", + ); + const savedSnapshot = JSON.parse(await readFile(snapshotPath, "utf8")); + assert.ok( + savedSnapshot.workflow_entries.some( + ({ operation }) => operation === "binary_overview", + ), + ); + assert.deepEqual(await readdir(runtime), []); + // Replaying a matching snapshot must still reach native path admission. + await assert.rejects( + exec(process.execPath, cliArgs(linkedPath), { + env, + timeout: 240000, + maxBuffer: 16 * 1024 * 1024, + }), + (cause) => { + assert.match( + cause.stdout + cause.stderr, + /Reparse path component is unsupported/u, + ); + return true; + }, + ); + assert.deepEqual(await readFile(path), bytes); + assert.deepEqual(await readdir(runtime), []); + windowsReparseAdmission = "cli-snapshot-mcp-rejected"; + } finally { + await unlink(junction); + } + } + report = { + ok: true, + mocked: false, + lane: "ghidra-6502", + host: process.platform, + toolchain, + provider: image.provider, + target: { + sha256, + bytes: bytes.length, + architecture: "6502", + entry: "0x8001", + }, + cli_mcp_load_image_parity: true, + instructions_and_call_reference: true, + decompilation: true, + file_offset_decompilation_cli_mcp: true, + invalid_mapping_preserves_session: true, + low_memory_cli_mcp_parity: lowMemory, + windows_reparse_admission: windowsReparseAdmission, + source_unchanged: true, + runtime_cleanup: "complete", + }; +} catch (cause) { + if (stderr) process.stderr.write(stderr); + throw cause; +} finally { + try { + await client.close(); + } finally { + await transport.close(); + } +} +const verifierRun = await completeVerifierRun(run); +if (process.platform !== "win32") { + assert.equal(verifierRun.process_lineage.status, "verified"); + assert.deepEqual(verifierRun.process_lineage.descendants, []); +} +// Keep the workspace for diagnosis unless verification and cleanup both pass. +await rm(workspace, { recursive: true, force: true }); +process.stdout.write( + JSON.stringify({ verifier_run: verifierRun, ...report }) + "\n", +); + +async function call(name, args = {}) { + const result = await client.callTool( + { name, arguments: args }, + { timeout: 240000 }, + ); + assert.notEqual(result.isError, true, JSON.stringify(result)); + if (["open_binary", "binary_session", "close_binary"].includes(name)) + return requireMcpOperationResult(result, name); + return result.structuredContent; +} + +function evidence(value, sha256) { + const result = parseEvidence(value); + assert.equal(result.subject.digest.sha256, sha256); + assert.equal(result.subject.architecture, "6502"); + assert.equal(result.subject.format, "raw-binary"); + return result; +} diff --git a/src/application/BinaryTargetResolver.ts b/src/application/BinaryTargetResolver.ts index ac691b55e..6f1bc4366 100644 --- a/src/application/BinaryTargetResolver.ts +++ b/src/application/BinaryTargetResolver.ts @@ -76,6 +76,8 @@ export const parseBinaryTarget = async ( await access(candidate, constants.R_OK); throwIfTargetResolutionCancelled(signal); const canonical = await realpath(candidate); + // Windows native admission must inspect the caller-selected reparse components. + const sourcePath = process.platform === "win32" ? candidate : canonical; throwIfTargetResolutionCancelled(signal); const resolved = await resolveAppBundle(canonical); throwIfTargetResolutionCancelled(signal); @@ -113,7 +115,7 @@ export const parseBinaryTarget = async ( ); return ok({ path, - sourcePath: canonical, + sourcePath, sha256: await sha256Handle(handle, path, metadata, signal), kind: "executable", format: "raw-binary", @@ -152,7 +154,7 @@ export const parseBinaryTarget = async ( ); return ok({ path, - sourcePath: canonical, + sourcePath, sha256: await sha256Handle(handle, path, metadata, signal), kind: "executable", format: "dos-com", @@ -166,7 +168,7 @@ export const parseBinaryTarget = async ( ) return ok({ path, - sourcePath: process.platform === "win32" ? candidate : canonical, + sourcePath, sha256: await sha256Handle(handle, path, metadata, signal), kind: "database", format: "analysis-database", @@ -175,7 +177,7 @@ export const parseBinaryTarget = async ( if (artifactFormat !== undefined) { const identity = { path, - sourcePath: process.platform === "win32" ? candidate : canonical, + sourcePath, sha256: await sha256Handle(handle, path, metadata, signal), }; return isArchiveFormat(artifactFormat) @@ -195,7 +197,7 @@ export const parseBinaryTarget = async ( ); return ok({ path, - sourcePath: process.platform === "win32" ? candidate : canonical, + sourcePath, ...bundle, sha256: await sha256Handle(handle, path, metadata, signal), kind: "executable", diff --git a/src/application/DirectAnalysis.ts b/src/application/DirectAnalysis.ts index c4d130f63..0bdb69864 100644 --- a/src/application/DirectAnalysis.ts +++ b/src/application/DirectAnalysis.ts @@ -23,6 +23,7 @@ import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; import type { AnalysisError } from "../domain/analysisErrorBase.js"; import { access } from "node:fs/promises"; import { readAnalysisSnapshot } from "./binary/AnalysisSnapshotFiles.js"; +import { requiresNativeSourceAdmissionBeforeReplay } from "./binary/AnalysisSnapshotCache.js"; import { parseBinaryTarget } from "./BinaryTargetResolver.js"; import type { BinaryTarget } from "../domain/binaryTargetTypes.js"; import { @@ -254,9 +255,14 @@ const runAnalysis = async ( const route = preview.value.route; const bindingProfile = route.profile ?? undefined; const evidenceProfile = analysisProfileForRoute(route, tool); + // A parsed Windows path does not prove Ghidra's native snapshot owner + // admitted the selected source; CLI replay must wait for that client. if ( bindingProfile !== undefined && evidenceProfile !== undefined && + !requiresNativeSourceAdmissionBeforeReplay( + route.binding?.identity.id, + ) && allowsSnapshotReplay(route, tool) ) { const cached = snapshotEvidenceForQuery(snapshot, { diff --git a/src/application/binary/AnalysisSnapshotCache.ts b/src/application/binary/AnalysisSnapshotCache.ts index 7e307baa0..79e212cac 100644 --- a/src/application/binary/AnalysisSnapshotCache.ts +++ b/src/application/binary/AnalysisSnapshotCache.ts @@ -84,6 +84,11 @@ export const isSnapshotCacheable = ( descriptor.effects.mayWriteFilesystem === false) && descriptor.effects.changesPermissions === false; +/** Windows Ghidra must admit the selected source through its native snapshot owner. */ +export const requiresNativeSourceAdmissionBeforeReplay = ( + providerId: string | undefined, +): boolean => process.platform === "win32" && providerId === "ghidra"; + /** Bounded in-memory cache for one immutable binary identity. */ export class AnalysisSnapshotCache { readonly #entries = new Map(); diff --git a/src/application/binary/BinarySession.ts b/src/application/binary/BinarySession.ts index f7b2d3529..3da571832 100644 --- a/src/application/binary/BinarySession.ts +++ b/src/application/binary/BinarySession.ts @@ -58,6 +58,7 @@ interface SessionBinding { readonly profile: AnalysisProfileCommitment | null; readonly route: SessionProviderRoute; readonly runId: string; + nativeSourceAdmitted: boolean; } /** @@ -323,6 +324,7 @@ export class BinarySession profile, route, runId, + nativeSourceAdmitted: false, }; this.#clearRuntimeAvailability(); this.resetSnapshotInvalidation(); @@ -609,6 +611,7 @@ export class BinarySession active: this.#active, operation: name, parameters: arguments_, + nativeSourceAdmitted: this.#active?.nativeSourceAdmitted === true, unboundOperationError: (operation, route) => this.#providerRouter.unboundOperationError(operation, route), lookupSnapshot: (target, profile, operation, parameters) => @@ -645,6 +648,14 @@ export class BinarySession }, }) : profiled; + // A successful Ghidra operation proves this client's native source snapshot + // was admitted; a composite health response and parsed path do not. + if ( + execution.ok && + capability.provider.id === "ghidra" && + this.#active === active + ) + this.#active.nativeSourceAdmitted = true; this.#observeRuntimeAvailability(name, execution); if ( execution.ok && @@ -838,6 +849,7 @@ export class BinarySession profile: previous.profile, route: previous.route, runId, + nativeSourceAdmitted: false, }; return ok(null); } diff --git a/src/application/binary/BinarySessionExecution.ts b/src/application/binary/BinarySessionExecution.ts index 996749aae..73dfe4eea 100644 --- a/src/application/binary/BinarySessionExecution.ts +++ b/src/application/binary/BinarySessionExecution.ts @@ -13,7 +13,10 @@ import type { AnalysisOperation, CapabilityDescriptor, } from "../AnalysisProvider.js"; -import { isSnapshotCacheable } from "./AnalysisSnapshotCache.js"; +import { + isSnapshotCacheable, + requiresNativeSourceAdmissionBeforeReplay, +} from "./AnalysisSnapshotCache.js"; import type { SessionProviderRoute } from "./SessionProviderRouter.js"; interface ActiveExecutionBinding { @@ -35,6 +38,7 @@ interface PrepareSessionExecutionInput { readonly active: ActiveExecutionBinding | undefined; readonly operation: AnalysisOperation; readonly parameters: Readonly>; + readonly nativeSourceAdmitted: boolean; readonly unboundOperationError: ( operation: AnalysisOperation, route: SessionProviderRoute, @@ -55,6 +59,7 @@ export const prepareSessionExecution = ( active, operation, parameters, + nativeSourceAdmitted, unboundOperationError, lookupSnapshot, } = input; @@ -79,9 +84,12 @@ export const prepareSessionExecution = ( const cacheable = profile !== undefined && isSnapshotCacheable(operation, capability, parameters); - const cached = cacheable - ? lookupSnapshot(active.target, profile, operation, parameters) - : undefined; + const cached = + cacheable && + (!requiresNativeSourceAdmissionBeforeReplay(capability.provider.id) || + nativeSourceAdmitted) + ? lookupSnapshot(active.target, profile, operation, parameters) + : undefined; return ok({ active, capability, profile, cacheable, cached }); }; diff --git a/src/application/binary/BinarySessionOpen.ts b/src/application/binary/BinarySessionOpen.ts index 8d810e671..cd5f6be4a 100644 --- a/src/application/binary/BinarySessionOpen.ts +++ b/src/application/binary/BinarySessionOpen.ts @@ -92,9 +92,7 @@ export const resolveSessionTarget = async ( input: ResolveSessionTargetInput, ): Promise> => { const { router, current, target, options } = input; - const sameTarget = - current?.target.path === target.path && - snapshotMatchesTarget(snapshotTarget(current.target), target); + const sameTarget = sameSessionTarget(current?.target, target); const resolvedRoute = sameTarget && options.providerId === undefined && current !== undefined ? ({ ok: true, value: current.route } as const) @@ -113,9 +111,7 @@ export const validateResolvedSessionTarget = ( }, ): Result => { const { current, target, route, options, stagedSnapshotMatches } = input; - const sameTarget = - current?.target.path === target.path && - snapshotMatchesTarget(snapshotTarget(current.target), target); + const sameTarget = sameSessionTarget(current?.target, target); const snapshotError = validateSnapshot({ snapshot: options.snapshot, target, @@ -127,6 +123,16 @@ export const validateResolvedSessionTarget = ( return { ok: true, value: { target, route, sameTarget } }; }; +const sameSessionTarget = ( + current: BinaryTarget | undefined, + target: BinaryTarget, +): boolean => + current !== undefined && + current.path === target.path && + // Native admission is bound to the requested path, not just its realpath. + (current.sourcePath ?? current.path) === (target.sourcePath ?? target.path) && + snapshotMatchesTarget(snapshotTarget(current), target); + interface ValidateSnapshotInput { readonly snapshot: AnalysisSnapshot | undefined; readonly target: BinaryTarget; diff --git a/src/contracts/toolOutputReferences.test.ts b/src/contracts/toolOutputReferences.test.ts index cda49f1d5..1cd890bf3 100644 --- a/src/contracts/toolOutputReferences.test.ts +++ b/src/contracts/toolOutputReferences.test.ts @@ -6,6 +6,34 @@ import { z } from "zod"; import { toolContract } from "./toolContracts.js"; import { toolOutputSchemaWithMetadata } from "./toolSchemaMetadata.js"; +import { createEvidence } from "../domain/evidence.js"; + +it.each(["draft-2020-12", "draft-07"] as const)( + "advertises the raw-only 6502 Evidence subject constraint in %s", + (target) => { + const valid = createEvidence( + { + path: "/fixture/image.bin", + sha256: "a".repeat(64), + format: "raw-binary", + architecture: "6502", + }, + { id: "fixture", name: "Fixture", version: "1" }, + { operation: "address_name", parameters: {}, result: "entry" }, + ); + const advertised = toolOutputSchemaWithMetadata( + toolContract("address_name"), + )["~standard"].jsonSchema.output({ target }); + const validate = new AjvJsonSchemaValidator().getValidator(advertised); + expect(validate(valid).valid).toBe(true); + expect( + validate({ + ...valid, + subject: { ...valid.subject, format: "pe" }, + }).valid, + ).toBe(false); + }, +); it("keeps dialect-specific validators distinct after JSON schema serialization", () => { const contract = { diff --git a/src/contracts/toolOutputSchemaGroups.ts b/src/contracts/toolOutputSchemaGroups.ts index dcf7ad033..76294e2db 100644 --- a/src/contracts/toolOutputSchemaGroups.ts +++ b/src/contracts/toolOutputSchemaGroups.ts @@ -444,11 +444,11 @@ const lifecycleTargetFields = { path: z.string(), kind: targetKindSchema, sha256: z.string().regex(/^[a-f0-9]{64}$/u), - architecture: z.enum(BINARY_ARCHITECTURES).nullable(), }; const detectedLifecycleTargetSchema = z.object({ ...lifecycleTargetFields, format: targetFormatSchema.exclude(["raw-binary"]), + architecture: z.enum(BINARY_ARCHITECTURES).exclude(["6502"]).nullable(), }); const rawLifecycleTargetSchema = z.object({ ...lifecycleTargetFields, diff --git a/src/domain/analysisSnapshot.test.ts b/src/domain/analysisSnapshot.test.ts index ee3920be5..d83ddac30 100644 --- a/src/domain/analysisSnapshot.test.ts +++ b/src/domain/analysisSnapshot.test.ts @@ -52,7 +52,7 @@ const snapshotWithHistoricalEvidence = () => { }; return { current, - snapshot: { + snapshot: analysisSnapshotSchema.parse({ target, binding, entries: [entry], @@ -66,11 +66,32 @@ const snapshotWithHistoricalEvidence = () => { result: { summary: "unprofiled" }, }), ]), - } satisfies AnalysisSnapshot, + }), }; }; describe("analysis snapshot target identity", () => { + it("keeps 6502 confined to executable raw snapshots", () => { + const target = { + sha256: ANALYSIS_SNAPSHOT_TARGET.sha256, + kind: "executable", + architecture: "6502", + }; + expect( + analysisSnapshotSchema.shape.target.safeParse({ + ...target, + format: "pe", + }).success, + ).toBe(false); + expect( + analysisSnapshotSchema.shape.target.safeParse({ + ...target, + format: "raw-binary", + entry_point: "0x8001", + }).success, + ).toBe(true); + }); + it("partitions raw snapshots and query identities by the canonical entry", () => { const target: BinaryTarget = { path: "/fixture/image.bin", @@ -120,6 +141,48 @@ describe("analysis snapshot target identity", () => { }); }); +describe("analysis snapshot nested subject contract", () => { + it("rejects raw-only 6502 in nested detected execution subjects", () => { + const { snapshot } = snapshotWithHistoricalEvidence(); + const entry = snapshot.entries[0]; + if (entry === undefined) throw new Error("Fixture has no snapshot entry"); + const malformedSubject = { + ...entry.execution.subject, + format: "pe", + architecture: "6502", + }; + expect( + analysisSnapshotSchema.safeParse({ + ...snapshot, + entries: [ + { + ...entry, + execution: { + ...entry.execution, + subject: malformedSubject, + }, + }, + ], + }).success, + ).toBe(false); + expect( + analysisSnapshotSchema.safeParse({ + ...snapshot, + workflow_entries: [ + { + ...entry, + execution: { + ...entry.execution, + analysis_profile: snapshot.binding.analysis_profile, + subject: malformedSubject, + }, + }, + ], + }).success, + ).toBe(false); + }); +}); + describe("analysis snapshot contract", () => { it.each([ ["dos-mz", "x86"], diff --git a/src/domain/analysisSnapshot.ts b/src/domain/analysisSnapshot.ts index 57e876653..4e1af5fa8 100644 --- a/src/domain/analysisSnapshot.ts +++ b/src/domain/analysisSnapshot.ts @@ -41,7 +41,10 @@ import { digestSchema } from "./../domain/digests.js"; import { prefixedDigestSchema } from "./../domain/digests.js"; import { canonicalJsonDigestSteps } from "./canonicalJsonDigestSteps.js"; import { freezeJsonSnapshot } from "./immutableJson.js"; -import { rawBinaryEntryPointSchema } from "./rawBinary.js"; +import { + binaryArchitectureSchema, + rawBinaryEntryPointSchema, +} from "./rawBinary.js"; import { relationshipCoverageSchema, type RelationshipCoverage, @@ -73,20 +76,30 @@ const formatSchema = z.enum([ "source-map", ]); const kindSchema = z.enum(["executable", "database", "archive", "artifact"]); -const subjectSchema = z.object({ +const subjectBaseSchema = z.object({ path: localPathStringSchema, sha256: digestSchema, - format: z.enum([ - ...formatSchema.options, - "hopper", - "directory", - "file", - "unknown", - "javascript-bundle", - "entitlements", - ]), - architecture: architectureSchema.nullable(), }); +const subjectFormatSchema = z.enum([ + ...formatSchema.options, + "hopper", + "directory", + "file", + "unknown", + "javascript-bundle", + "entitlements", +]); +// Snapshot entries retain the same raw-only architecture boundary as targets. +const subjectSchema = z.discriminatedUnion("format", [ + subjectBaseSchema.extend({ + format: subjectFormatSchema.exclude(["raw-binary"]), + architecture: architectureSchema.exclude(["6502"]).nullable(), + }), + subjectBaseSchema.extend({ + format: z.literal("raw-binary"), + architecture: architectureSchema.nullable(), + }), +]); const targetIdentitySchema = z.object({ sha256: digestSchema, kind: kindSchema, @@ -97,9 +110,12 @@ const targetSchema = z.discriminatedUnion("format", [ format: z.enum( formatSchema.options.filter((format) => format !== "raw-binary"), ), + architecture: architectureSchema.exclude(["6502"]).nullable(), }), targetIdentitySchema.extend({ format: z.literal("raw-binary"), + kind: z.literal("executable"), + architecture: binaryArchitectureSchema, entry_point: rawBinaryEntryPointSchema, }), ]); @@ -332,18 +348,21 @@ const canonicalJson = (value: unknown): string => { }; /** Project a binary into the path-free identity used for cache invalidation. */ -export const snapshotTarget = ( - target: BinaryTarget, -): AnalysisSnapshotTarget => { - const identity = { - sha256: target.sha256, - kind: target.kind, - architecture: target.architecture ?? null, - }; - return target.format === "raw-binary" - ? { ...identity, format: target.format, entry_point: target.entryPoint } - : { ...identity, format: target.format }; -}; +export const snapshotTarget = (target: BinaryTarget): AnalysisSnapshotTarget => + target.format === "raw-binary" + ? { + sha256: target.sha256, + kind: target.kind, + architecture: target.architecture, + format: target.format, + entry_point: target.entryPoint, + } + : { + sha256: target.sha256, + kind: target.kind, + architecture: target.architecture ?? null, + format: target.format, + }; /** Build the immutable provider/profile binding used by a cached snapshot. */ export const snapshotBinding = ( @@ -475,10 +494,10 @@ export const createAnalysisSnapshotWorkflowEntry = (input: { subject: execution.subject === null ? null - : { + : subjectSchema.parse({ ...execution.subject, architecture: execution.subject.architecture ?? null, - }, + }), }, }; }; @@ -538,10 +557,10 @@ export const createAnalysisSnapshotEntry = (input: { subject: execution.subject === null ? null - : { + : subjectSchema.parse({ ...execution.subject, architecture: execution.subject.architecture ?? null, - }, + }), }, }; }; diff --git a/src/domain/binaryTarget.ts b/src/domain/binaryTarget.ts index 99bc6dd24..94828ee6e 100644 --- a/src/domain/binaryTarget.ts +++ b/src/domain/binaryTarget.ts @@ -351,7 +351,9 @@ const machArchitecture = (cpu: number): HeaderArchitecture | undefined => { return undefined; }; -const elfArchitecture = (machine: number): BinaryArchitecture | undefined => { +const elfArchitecture = ( + machine: number, +): Exclude | undefined => { switch (machine) { case 3: return "x86"; diff --git a/src/domain/binaryTargetTypes.ts b/src/domain/binaryTargetTypes.ts index ab5046549..99ad8487f 100644 --- a/src/domain/binaryTargetTypes.ts +++ b/src/domain/binaryTargetTypes.ts @@ -1,12 +1,13 @@ import type { ZipPackageFormat } from "./zipPackageFormat.js"; -/** Provider-neutral CPU families detected from supported executable headers. */ +/** Provider-neutral CPU families detected from headers or explicitly selected raw loads. */ export const BINARY_ARCHITECTURES = [ "x86", "x86_64", "arm", "arm64", "mips", + "6502", "riscv", ] as const; export type BinaryArchitecture = (typeof BINARY_ARCHITECTURES)[number]; @@ -40,7 +41,7 @@ export const isElfMetadataArchitecture = ( /** Recognized family with no ELF metadata member; the header alone is the fact. */ export type HeaderArchitecture = Exclude< BinaryArchitecture, - ElfMetadataArchitecture + ElfMetadataArchitecture | "6502" >; /** Raw ELF facts; family recognition does not establish ISA or ABI support. */ @@ -144,8 +145,17 @@ export type BinaryTarget = readonly executableRole?: never; readonly managed?: never; }) - | (ExecutableTarget & { + | (BinaryTargetIdentity & { + readonly kind: "executable"; readonly format: "raw-binary"; + readonly architecture: Exclude< + BinaryArchitecture, + ElfMetadataArchitecture + >; + readonly availableArchitectures: readonly Exclude< + BinaryArchitecture, + ElfMetadataArchitecture + >[]; readonly entryPoint: string; readonly entryPointSource: string; readonly executableRole?: never; diff --git a/src/domain/evidence.test.ts b/src/domain/evidence.test.ts index 5cdf30a27..be923e3a6 100644 --- a/src/domain/evidence.test.ts +++ b/src/domain/evidence.test.ts @@ -19,6 +19,33 @@ const TARGET: BinaryTarget = { const PROVIDER = { id: "fixture", name: "Fixture provider", version: "1" }; const PROFILE = createAnalysisProfile(PROVIDER, { loader: "default" }); +it("rejects a detected Evidence subject with the raw-only 6502 architecture", () => { + const selected = { + path: TARGET.path, + sha256: TARGET.sha256, + format: "raw-binary" as const, + architecture: "6502" as const, + }; + expect( + createEvidence(selected, PROVIDER, { + operation: "inspect", + parameters: {}, + result: null, + }).subject, + ).toMatchObject({ format: "raw-binary", architecture: "6502" }); + const malformed = { + ...selected, + format: "pe" as const, + }; + expect(() => + createEvidence(malformed, PROVIDER, { + operation: "inspect", + parameters: {}, + result: null, + }), + ).toThrow(); +}); + it("preserves omitted relationship diagnostics and authenticates reported diagnostics", () => { const observation = { operation: "procedure_callees", diff --git a/src/domain/evidence.ts b/src/domain/evidence.ts index 8a450f478..49cb2a229 100644 --- a/src/domain/evidence.ts +++ b/src/domain/evidence.ts @@ -38,42 +38,52 @@ export const providerSchema = z.strictObject({ name: z.string().min(1), version: z.string().nullable(), }); -const subjectSchema = z.strictObject({ +const subjectFormatSchema = z.enum([ + "hopper", + "analysis-database", + "mach-o", + "elf", + "pe", + "dos-mz", + "dos-com", + "raw-binary", + "zip", + "ipa", + "apk", + "msix", + "appx", + "hap", + "hsp", + "app-pack", + "asar", + "dmg", + "pkg", + "plist", + "javascript", + "source-map", + "directory", + "file", + "unknown", + "mach-o-universal", + "javascript-bundle", + "entitlements", +]); +const subjectBaseSchema = z.strictObject({ name: z.string().min(1), digest: z.strictObject({ sha256: digestSchema }), - format: z.enum([ - "hopper", - "analysis-database", - "mach-o", - "elf", - "pe", - "dos-mz", - "dos-com", - "raw-binary", - "zip", - "ipa", - "apk", - "msix", - "appx", - "hap", - "hsp", - "app-pack", - "asar", - "dmg", - "pkg", - "plist", - "javascript", - "source-map", - "directory", - "file", - "unknown", - "mach-o-universal", - "javascript-bundle", - "entitlements", - ]), - architecture: z.enum(BINARY_ARCHITECTURES).nullable(), local_path: nulFreeStringSchema, }); +// The format owns the architecture claim, including nested Evidence subjects. +const subjectSchema = z.discriminatedUnion("format", [ + subjectBaseSchema.extend({ + format: subjectFormatSchema.exclude(["raw-binary"]), + architecture: z.enum(BINARY_ARCHITECTURES).exclude(["6502"]).nullable(), + }), + subjectBaseSchema.extend({ + format: z.literal("raw-binary"), + architecture: z.enum(BINARY_ARCHITECTURES).nullable(), + }), +]); /** Source location attached to an evidence observation. */ /** Source-location schema shared by evidence-bearing persistence formats. */ export const evidenceLocationSchema = z.discriminatedUnion("kind", [ @@ -186,7 +196,7 @@ const immutableResultSchema = z }); export type EvidenceLocation = z.infer; -/** Minimal immutable local artifact identity accepted by Evidence. */ +/** Candidate local artifact identity validated by the Evidence subject schema. */ export interface EvidenceSubjectTarget { readonly path: string; readonly sha256: string; diff --git a/src/domain/evidenceBundle.ts b/src/domain/evidenceBundle.ts index 7e6fd2b3b..a70969b8e 100644 --- a/src/domain/evidenceBundle.ts +++ b/src/domain/evidenceBundle.ts @@ -16,14 +16,17 @@ import { } from "./residualUnknown.js"; import { freezeJsonSnapshot } from "./immutableJson.js"; -const artifactManifestSchema = evidenceSchema.shape.subject - .unwrap() - .pick({ - digest: true, - format: true, - architecture: true, - }) - .strict(); +const [detectedSubjectSchema, rawSubjectSchema] = + evidenceSchema.shape.subject.unwrap().options; +const manifestFields = { + digest: true, + format: true, + architecture: true, +} as const; +const artifactManifestSchema = z.discriminatedUnion("format", [ + detectedSubjectSchema.pick(manifestFields), + rawSubjectSchema.pick(manifestFields), +]); const providerManifestSchema = providerSchema; const environmentManifestSchema = evidenceSchema.shape.environment.unwrap(); const scenarioManifestSchema = z.strictObject({ @@ -85,11 +88,11 @@ export const createEvidenceBundle = ( subject === null ? [] : [ - { + artifactManifestSchema.parse({ digest: subject.digest, format: subject.format, architecture: subject.architecture, - }, + }), ], ), ), diff --git a/src/domain/native/nativeLoadImage.ts b/src/domain/native/nativeLoadImage.ts index 6a3be4421..a4efc2847 100644 --- a/src/domain/native/nativeLoadImage.ts +++ b/src/domain/native/nativeLoadImage.ts @@ -1,6 +1,7 @@ import { z } from "zod"; import { nativeMetadataRecoverySummarySchema } from "./nativeMetadataRecovery.js"; import { jsonValueSchema } from "../jsonValue.js"; +import { binaryArchitectureSchema } from "../rawBinary.js"; const integer = z.number().int().min(0).max(Number.MAX_SAFE_INTEGER); const digest = z.string().regex(/^[a-f0-9]{64}$/u); @@ -112,7 +113,7 @@ export const nativeLoadImageSchema = z.union([ status: z.enum(["verified", "mismatch"]), format: z.literal("raw-binary"), target_sha256: digest, - architecture: z.enum(["x86", "x86_64", "arm", "arm64"]), + architecture: binaryArchitectureSchema, language_id: z.string().min(1), compiler_spec_id: z.string().min(1), base_address: address, diff --git a/src/domain/rawBinary.test.ts b/src/domain/rawBinary.test.ts index f22e5fced..5cb6ed129 100644 --- a/src/domain/rawBinary.test.ts +++ b/src/domain/rawBinary.test.ts @@ -39,6 +39,44 @@ describe("raw-binary entry addresses", () => { }); describe("raw-binary admission", () => { + it("admits byte-aligned 6502 images up to the 16-bit address boundary", () => { + for (const [entryPoint, fileBytes] of [ + ["0x8001", 8], + ["0xffff", 1], + ["0x0", 65536], + ] as const) { + const result = admitRawBinary({ + architecture: "6502", + entryPoint, + fileBytes, + }); + if (!result.ok) throw new Error(result.error.reason); + expect(result.value).toEqual({ + architecture: "6502", + entryPoint, + entryPointSource: entryPoint, + }); + } + }); + it.each([ + ["0x10000", 1], + ["0xffff", 2], + ["0x0", 65537], + ])( + "rejects a 6502 mapping at %s with %i bytes that exceeds 16 bits", + (entryPoint, fileBytes) => { + const result = admitRawBinary({ + architecture: "6502", + entryPoint, + fileBytes, + }); + if (result.ok) throw new Error("expected 16-bit overflow rejection"); + expect(result.error).toEqual({ + reason: `raw-binary mapping from ${entryPoint} must fit in the 6502 address width`, + constraint: "load_selection_rejected", + }); + }, + ); it("admits a little-endian ARM image mapped at the caller entry", () => { expect( admitRawBinary({ diff --git a/src/domain/rawBinary.ts b/src/domain/rawBinary.ts index 94fdf6ce7..22d94f4d6 100644 --- a/src/domain/rawBinary.ts +++ b/src/domain/rawBinary.ts @@ -13,6 +13,7 @@ export const BINARY_ARCHITECTURES = [ "x86_64", "arm", "arm64", + "6502", ] as const satisfies readonly BinaryArchitecture[]; /** Provider-neutral CPU family for a caller-supplied raw-binary load. */ @@ -58,7 +59,7 @@ export const validateRawBinaryLength = ( /** Admitted headerless load identity used by target resolution and profiles. */ export interface RawBinaryAdmission { - readonly architecture: HeaderArchitecture; + readonly architecture: HeaderArchitecture | "6502"; readonly entryPoint: string; readonly entryPointSource: string; } @@ -74,9 +75,11 @@ export interface RawBinaryAdmissionRejection { } const addressWidth = (architecture: BinaryArchitecture): bigint => - architecture === "x86" || architecture === "arm" - ? 0xffffffffn - : 0xffffffffffffffffn; + architecture === "6502" + ? 0xffffn + : architecture === "x86" || architecture === "arm" + ? 0xffffffffn + : 0xffffffffffffffffn; /** Validate architecture, entry, and that the mapped image fits the CPU width. */ export const admitRawBinary = (input: { @@ -88,7 +91,7 @@ export const admitRawBinary = (input: { if (!architecture.success) return err({ reason: - "raw-binary format requires architecture (x86, x86_64, arm, or arm64)", + "raw-binary format requires architecture (x86, x86_64, arm, arm64, or 6502)", constraint: "load_selection_rejected", }); if (typeof input.entryPoint !== "string") diff --git a/src/ghidra/GhidraAnalysisProfile.ts b/src/ghidra/GhidraAnalysisProfile.ts index fd7cdff11..8cc5f2807 100644 --- a/src/ghidra/GhidraAnalysisProfile.ts +++ b/src/ghidra/GhidraAnalysisProfile.ts @@ -136,7 +136,7 @@ export const resolveGhidraAnalysisProfile = async ( installation.platform === "win32" ? "official-headless-script-v1" : "inspected-jvm-launch-support-v1", - load_image_evidence: "independent-raw-mapping-context-v2", + load_image_evidence: "independent-raw-mapping-context-v4", jump_table_evidence: "typed-case-default-blocks-v1", decompiler_jump_loads: true, loader: "BinaryLoader", diff --git a/src/ghidra/GhidraLoadImageValues.ts b/src/ghidra/GhidraLoadImageValues.ts index 1c80bb635..f06f4fdaa 100644 --- a/src/ghidra/GhidraLoadImageValues.ts +++ b/src/ghidra/GhidraLoadImageValues.ts @@ -361,7 +361,10 @@ export const attestGhidraRawBinaryLoadImage = ( }); }; const source = observations.source_files[0]; - const mapping = observations.mappings[0]; + const fileMappings = observations.mappings.filter( + (item) => item.source_file_index !== null, + ); + const mapping = fileMappings[0]; check("executable_format", "Raw Binary", observations.executable_format); check("language_id", language.languageId, observations.language_id); check( @@ -371,11 +374,34 @@ export const attestGhidraRawBinaryLoadImage = ( ); // BinaryLoader's block base is independent of the Program image base. check("image_base", "0x0", observations.image_base); + check( + "default_address_space", + language.defaultAddressSpace, + observations.default_address_space, + ); check("source_file_count", 1, observations.source_files.length); check("source_file_size", snapshot.bytes, source?.size ?? null); check("original_file_sha256", target.sha256, source?.original_sha256 ?? null); check("modified_file_sha256", target.sha256, source?.modified_sha256 ?? null); - check("mapping_count", 1, observations.mappings.length); + // Language-defined uninitialized zero page/stack blocks are observations, not file bytes. + check("mapping_count", 1, fileMappings.length); + check( + "initialized_unbacked_mappings", + 0, + observations.mappings.filter( + (item) => item.initialized && item.source_file_index === null, + ).length, + ); + check("mapping.source_file_index", 0, mapping?.source_file_index ?? null); + // Matching bytes alone do not establish the admitted default-space load. + check("mapping.initialized", true, mapping?.initialized ?? null); + check("mapping.loaded", true, mapping?.loaded ?? null); + check("mapping.overlay", false, mapping?.overlay ?? null); + check( + "mapping.address_space", + observations.default_address_space, + mapping?.address_space ?? null, + ); check( "mapping.file_offset", 0, @@ -427,8 +453,9 @@ export const attestGhidraRawBinaryLoadImage = ( checks, limitations: [ "Raw binary is a headerless interpretation explicitly selected by the caller, not format identification from file bytes.", - "Verification covers complete original/modified source identity, mapped file bytes, the caller-supplied entry, and the bound little-endian Ghidra language. File offset 0 is mapped at the entry address.", + "Verification covers complete original/modified source identity, initialized and loaded non-overlay file bytes in the default address space, the caller-supplied entry, and the bound little-endian Ghidra language. File offset 0 is mapped at the entry address.", "Ghidra may report additional language-defined external entries outside the mapped image. They are preserved as observations; verification requires the caller-supplied entry and does not attest other entries.", + "Language-defined uninitialized memory, such as 6502 zero page and stack, is preserved as observations but is not attested as file-backed bytes.", "Reset vectors, memory-mapped I/O, big-endian and Thumb-only variants, and runtime boot behavior are not modeled or verified. No original executable bytes are changed.", "Mapping ends are inclusive; a mismatching digest identifies its source range, not the first differing byte.", ], diff --git a/src/ghidra/GhidraProvider.behavior.test.ts b/src/ghidra/GhidraProvider.behavior.test.ts index 3979222d9..a41bf096f 100644 --- a/src/ghidra/GhidraProvider.behavior.test.ts +++ b/src/ghidra/GhidraProvider.behavior.test.ts @@ -8,7 +8,7 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; import { fileURLToPath } from "node:url"; import { promisify } from "node:util"; -import { describe, expect, it } from "vitest"; +import { describe, expect, it, onTestFinished, vi } from "vitest"; import { fixtureDosLoadImage } from "./GhidraLoadImage.fixture.js"; import { fixtureRawBinaryLoadImage } from "./GhidraRawBinaryLoadImage.fixture.js"; import { @@ -30,6 +30,8 @@ import { err, ok } from "../domain/result.js"; import type { Result } from "../domain/result.js"; import { GhidraSessionError } from "./GhidraSessionError.js"; import { silentLogger } from "../logger.js"; +import * as WindowsNativeLoader from "../windows/WindowsNativeLoader.js"; +import { WINDOWS_NATIVE_AUTHORITY_UNAVAILABLE_REASON } from "../process/WindowsAuthority.js"; import { ProviderCleanupError } from "../domain/providerCleanupError.js"; import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; import { ProviderStartupDeadline } from "../process/ProviderDeadline.js"; @@ -190,6 +192,36 @@ describe("Ghidra provider", () => { }); describe("Ghidra raw-binary profile", () => { + it("admits raw 6502 on Windows and binds its 16-bit language without mutation authority", async () => { + const ghidra = provider({ ...installationHost(), platform: "win32" }); + const target: BinaryTarget = { + path: "/tmp/6502.bin", + sha256: "a".repeat(64), + kind: "executable", + format: "raw-binary", + architecture: "6502", + availableArchitectures: ["6502"], + entryPoint: "0x8001", + entryPointSource: "0x008001", + }; + expect(ghidra.inspectTargetSupport(target).status).toBe("supported"); + const resolved = await ghidra.resolveAnalysisProfile(target); + if (!resolved.ok) throw resolved.error; + expect(resolved.value.profile?.parameters).toMatchObject({ + architecture: "6502", + loader: "BinaryLoader", + language_id: "6502:LE:16:default", + compiler_spec_id: "default", + entry_point: "0x8001", + language_selection: "caller-supplied-v1", + load_image_evidence: "independent-raw-mapping-context-v4", + }); + expect( + ghidra + .capabilities() + .some(({ available, effects }) => available && effects.mutatesArtifact), + ).toBe(false); + }); it("commits BinaryLoader language and caller-supplied entry", async () => { const ghidra = provider(); const target: BinaryTarget = { @@ -324,6 +356,13 @@ describe("Ghidra platform support", () => { }); it("keeps Windows P0 unavailable until native isolation authority exists", () => { + const native = vi + .spyOn(WindowsNativeLoader, "systemWindowsNativeAuthority") + .mockReturnValue({ + available: false, + reason: WINDOWS_NATIVE_AUTHORITY_UNAVAILABLE_REASON, + }); + onTestFinished(() => native.mockRestore()); const ghidra = provider({ ...installationHost(), platform: "win32" }); const nativeApplication = peTarget("x86_64"); @@ -812,11 +851,22 @@ describe("Ghidra relationship coverage", () => { }); }); -describe("Ghidra measured load-image projection", () => { - it.each(["verified", "changed-snapshot", "cancelled"] as const)( +describe("Ghidra measured raw load-image projection", () => { + it.each([ + "verified", + "mapping-mismatch", + "changed-snapshot", + "cancelled", + ] as const)( "streams raw snapshot identity and preserves %s outcomes", async (state) => { const fixture = fixtureRawBinaryLoadImage(); + if (state === "mapping-mismatch") { + const mapping = fixture.observation.mappings[0]; + if (mapping === undefined) + throw new Error("Missing source mapping fixture"); + mapping.overlay = true; + } const controller = new AbortController(); const signals: (AbortSignal | undefined)[] = []; const ghidra = provider(installationHost(), () => ({ @@ -872,12 +922,23 @@ describe("Ghidra measured load-image projection", () => { { signal: controller.signal }, ); expect(signals).toEqual([controller.signal]); - if (state === "verified") { + if (state === "verified" || state === "mapping-mismatch") { if (!result.ok) throw result.error; expect(result.value.result).toMatchObject({ - status: "verified", + status: state === "verified" ? "verified" : "mismatch", observations: fixture.observation, }); + if (state === "mapping-mismatch") { + expect(result.value.result).toMatchObject({ + checks: expect.arrayContaining([ + expect.objectContaining({ + name: "mapping.overlay", + observed: true, + matched: false, + }), + ]), + }); + } } else { expect(result).toMatchObject({ ok: false, @@ -894,7 +955,9 @@ describe("Ghidra measured load-image projection", () => { } }, ); +}); +describe("Ghidra measured DOS load-image projection", () => { it.each([ "verified", "mismatch", diff --git a/src/ghidra/GhidraProvider.ts b/src/ghidra/GhidraProvider.ts index c48f58957..0a1cde0f0 100644 --- a/src/ghidra/GhidraProvider.ts +++ b/src/ghidra/GhidraProvider.ts @@ -66,6 +66,7 @@ const SUPPORTED_ARCHITECTURES = new Set([ "arm", "arm64", "mips", + "6502", "riscv", ]); @@ -184,7 +185,7 @@ export class GhidraProvider implements AnalysisProviderCandidate { status: "unsupported", code: "architecture_unsupported", reason: - "Ghidra v1 requires a supported x86, x86_64, arm, arm64, or MIPS target profile.", + "Ghidra v1 requires a supported x86, x86_64, arm, arm64, MIPS, RISC-V, or raw 6502 target profile.", diagnostics, }; return { @@ -304,11 +305,15 @@ const inspectWindowsP0TargetSupport = ( target: BinaryTarget, diagnostics: Readonly>, ): ProviderTargetSupport => { + // The raw 6502 lane uses the same immutable snapshot and native ownership boundary. + if (target.format === "raw-binary" && target.architecture === "6502") + return { status: "supported", code: null, reason: null, diagnostics }; if (target.format !== "pe") return { status: "unsupported", code: "target_format_unsupported", - reason: "Windows Ghidra P0 accepts PE targets only.", + reason: + "Windows Ghidra P0 accepts native PE and explicitly selected raw 6502 targets only.", diagnostics, }; if (target.architecture !== "x86" && target.architecture !== "x86_64") diff --git a/src/ghidra/GhidraProviderCapabilities.ts b/src/ghidra/GhidraProviderCapabilities.ts index ea863e8a7..68c3045ce 100644 --- a/src/ghidra/GhidraProviderCapabilities.ts +++ b/src/ghidra/GhidraProviderCapabilities.ts @@ -33,7 +33,7 @@ export const healthLimitations = Object.freeze([ /** Additional limitations applied to the experimental Windows x64 P0 boundary. */ export const windowsP0Limitations = Object.freeze([ - "Windows Ghidra P0 accepts approved native x86 and x86-64 PE applications and DLLs; managed, hostile, sensitive, and mutable-path targets are unsupported.", + "Windows Ghidra P0 accepts native x86 and x86-64 PE applications and DLLs, and explicitly selected raw 6502 images; managed, hostile, sensitive, and mutable-path targets are unsupported.", "The Windows bridge uses authenticated IPv4 loopback because Node path-based IPC does not expose Java AF_UNIX sockets; the endpoint file contains no bearer token.", "Windows sessions require the matching packaged Windows x64 native addon, local NTFS targets and runtimes, and Windows 10 or later. Native handles enforce path admission, private DACLs, and Job Object ownership automatically.", ]); diff --git a/src/ghidra/GhidraProviderClient.ts b/src/ghidra/GhidraProviderClient.ts index 5911985df..3a15dbc4e 100644 --- a/src/ghidra/GhidraProviderClient.ts +++ b/src/ghidra/GhidraProviderClient.ts @@ -516,7 +516,7 @@ export const createGhidraProviderClient = (input: { ...(target.format === "raw-binary" ? [ "Headerless raw-binary analysis uses the caller-supplied architecture and entry address; file offset 0 is mapped at that address. Headers are not discovered from file bytes.", - "Little-endian generic Ghidra languages are used for x86, x86_64, arm, and arm64. Big-endian, Thumb-only, Cortex-M, and other processor variants are unsupported in this increment.", + "Little-endian generic Ghidra languages are used for x86, x86_64, arm, arm64, and NMOS 6502. Big-endian, Thumb-only, Cortex-M, 65C02, 65816, and other processor variants are unsupported in this increment.", "Static analysis does not emulate reset vectors, memory-mapped I/O, or firmware boot sequences.", ] : []), diff --git a/src/ghidra/GhidraRawBinaryLanguage.ts b/src/ghidra/GhidraRawBinaryLanguage.ts index ea024bdba..e9046975d 100644 --- a/src/ghidra/GhidraRawBinaryLanguage.ts +++ b/src/ghidra/GhidraRawBinaryLanguage.ts @@ -1,28 +1,51 @@ -import type { HeaderArchitecture } from "../domain/binaryTargetTypes.js"; +import type { RawBinaryAdmission } from "../domain/rawBinary.js"; /** Bound Ghidra language for a caller-supplied raw-binary architecture. */ export interface GhidraRawBinaryLanguage { readonly languageId: string; readonly compilerSpecId: "default" | "gcc"; + readonly defaultAddressSpace: "ram" | "RAM"; } /** - * Map the four caller-visible CPU families to little-endian generic Ghidra + * Map caller-visible CPU families to little-endian generic Ghidra * languages with a supported generic compiler specification. These are * analysis conventions, not evidence of the target's original compiler. * Big-endian, Thumb-only, and Cortex-M variants are out of scope. */ export const ghidraRawBinaryLanguage = ( - architecture: HeaderArchitecture, + architecture: RawBinaryAdmission["architecture"], ): GhidraRawBinaryLanguage => { switch (architecture) { + case "6502": + return { + languageId: "6502:LE:16:default", + compilerSpecId: "default", + defaultAddressSpace: "RAM", + }; case "x86": - return { languageId: "x86:LE:32:default", compilerSpecId: "gcc" }; + return { + languageId: "x86:LE:32:default", + compilerSpecId: "gcc", + defaultAddressSpace: "ram", + }; case "x86_64": - return { languageId: "x86:LE:64:default", compilerSpecId: "gcc" }; + return { + languageId: "x86:LE:64:default", + compilerSpecId: "gcc", + defaultAddressSpace: "ram", + }; case "arm": - return { languageId: "ARM:LE:32:v7", compilerSpecId: "default" }; + return { + languageId: "ARM:LE:32:v7", + compilerSpecId: "default", + defaultAddressSpace: "ram", + }; case "arm64": - return { languageId: "AARCH64:LE:64:v8A", compilerSpecId: "default" }; + return { + languageId: "AARCH64:LE:64:v8A", + compilerSpecId: "default", + defaultAddressSpace: "ram", + }; } }; diff --git a/src/ghidra/GhidraRawBinaryLoadImage.test.ts b/src/ghidra/GhidraRawBinaryLoadImage.test.ts index 2fc11502e..d09b3a504 100644 --- a/src/ghidra/GhidraRawBinaryLoadImage.test.ts +++ b/src/ghidra/GhidraRawBinaryLoadImage.test.ts @@ -9,23 +9,161 @@ describe("raw-binary Ghidra language binding", () => { expect(ghidraRawBinaryLanguage("x86")).toEqual({ languageId: "x86:LE:32:default", compilerSpecId: "gcc", + defaultAddressSpace: "ram", }); expect(ghidraRawBinaryLanguage("x86_64")).toEqual({ languageId: "x86:LE:64:default", compilerSpecId: "gcc", + defaultAddressSpace: "ram", }); expect(ghidraRawBinaryLanguage("arm")).toEqual({ languageId: "ARM:LE:32:v7", compilerSpecId: "default", + defaultAddressSpace: "ram", }); expect(ghidraRawBinaryLanguage("arm64")).toEqual({ languageId: "AARCH64:LE:64:v8A", compilerSpecId: "default", + defaultAddressSpace: "ram", + }); + }); +}); + +describe("raw-binary mapped-block state", () => { + it.each(["RAM", "ram"])( + "requires the bound 6502 address space for %s observations", + (space) => { + const f = fixtureRawBinaryLoadImage(); + const target = { + ...f.target, + architecture: "6502" as const, + availableArchitectures: ["6502" as const], + entryPoint: "0x8001", + entryPointSource: "0x8001", + }; + f.observation.language_id = "6502:LE:16:default"; + f.observation.default_address_space = space; + f.observation.entry_points = [target.entryPoint]; + const mapping = f.observation.mappings[0]; + if (mapping === undefined) + throw new Error("Missing source mapping fixture"); + mapping.start = target.entryPoint; + mapping.end = "0x8004"; + mapping.address_space = space; + expect( + attestGhidraRawBinaryLoadImage(f.snapshot, target, f.observation), + ).toMatchObject({ + ok: true, + value: { + status: space === "RAM" ? "verified" : "mismatch", + checks: expect.arrayContaining([ + expect.objectContaining({ + name: "default_address_space", + expected: "RAM", + observed: space, + matched: space === "RAM", + }), + ]), + }, + }); + }, + ); + it.each([ + ["initialized", false], + ["loaded", false], + ["overlay", true], + ["address_space", "overlay"], + ] as const)( + "retains a mismatch when the file mapping declares %s=%s", + (property, value) => { + const f = fixtureRawBinaryLoadImage(); + const mapping = f.observation.mappings[0]; + if (mapping === undefined) + throw new Error("Missing source mapping fixture"); + Object.assign(mapping, { [property]: value }); + expect( + attestGhidraRawBinaryLoadImage(f.snapshot, f.target, f.observation), + ).toMatchObject({ + ok: true, + value: { + status: "mismatch", + observations: f.observation, + checks: expect.arrayContaining([ + expect.objectContaining({ + name: `mapping.${property}`, + matched: false, + observed: value, + }), + ]), + }, + }); + }, + ); + it("does not verify a foreign default address space even when its mapping agrees", () => { + const f = fixtureRawBinaryLoadImage(); + f.observation.default_address_space = "foreign"; + const mapping = f.observation.mappings[0]; + if (mapping === undefined) + throw new Error("Missing source mapping fixture"); + mapping.address_space = "foreign"; + expect( + attestGhidraRawBinaryLoadImage(f.snapshot, f.target, f.observation), + ).toMatchObject({ + ok: true, + value: { + status: "mismatch", + observations: f.observation, + checks: expect.arrayContaining([ + expect.objectContaining({ + name: "default_address_space", + matched: false, + observed: "foreign", + }), + ]), + }, }); }); }); describe("raw-binary independent load-image checks", () => { + it("retains uninitialized language memory without treating it as file-backed bytes", () => { + const f = fixtureRawBinaryLoadImage(); + const zeroPage = { + block: "ZERO_PAGE", + start: "0x0", + end: "0xff", + address_space: "ram", + initialized: false, + loaded: true, + overlay: false, + length: 256, + source_file_index: null, + file_offset: null, + sha256: null, + }; + f.observation.mappings.unshift(zeroPage); + expect( + attestGhidraRawBinaryLoadImage(f.snapshot, f.target, f.observation), + ).toMatchObject({ + ok: true, + value: { status: "verified", observations: f.observation }, + }); + zeroPage.initialized = true; + expect( + attestGhidraRawBinaryLoadImage(f.snapshot, f.target, f.observation), + ).toMatchObject({ + ok: true, + value: { + status: "mismatch", + checks: expect.arrayContaining([ + expect.objectContaining({ + name: "initialized_unbacked_mappings", + matched: false, + }), + ]), + }, + }); + }); it("verifies mapped bytes at the caller-supplied entry", () => { const f = fixtureRawBinaryLoadImage(); const result = attestGhidraRawBinaryLoadImage( diff --git a/src/ghidra/GhidraRiscvProfile.test.ts b/src/ghidra/GhidraRiscvProfile.test.ts index 881ba0078..718fe8d46 100644 --- a/src/ghidra/GhidraRiscvProfile.test.ts +++ b/src/ghidra/GhidraRiscvProfile.test.ts @@ -1,6 +1,6 @@ import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; -import { join } from "node:path"; +import { join, parse, relative } from "node:path"; import { afterAll, beforeAll, describe, expect, it } from "vitest"; import { parseConfig } from "../config/parseConfig.js"; import type { @@ -92,7 +92,14 @@ const target = (change: Partial = {}): BinaryTarget => ({ let installDir = ""; beforeAll(async () => { - installDir = await mkdtemp(join(tmpdir(), "rea-ghidra-riscv-languages-")); + const root = + process.platform === "win32" ? join(process.cwd(), "build") : tmpdir(); + await mkdir(root, { recursive: true }); + installDir = await mkdtemp(join(root, "rea-ghidra-riscv-languages-")); + // The simulated Linux host needs a POSIX absolute path. A current-drive root + // path also reaches this real fixture on Windows without changing admission. + if (process.platform === "win32") + installDir = `/${relative(parse(installDir).root, installDir).replaceAll("\\", "/")}`; const languages = join(installDir, "Ghidra/Processors/RISCV/data/languages"); await mkdir(languages, { recursive: true }); await writeFile(join(languages, "riscv.ldefs"), RISCV_LDEFS); diff --git a/src/hopper/HopperAnalysisProfile.ts b/src/hopper/HopperAnalysisProfile.ts index 299862a97..a4848533f 100644 --- a/src/hopper/HopperAnalysisProfile.ts +++ b/src/hopper/HopperAnalysisProfile.ts @@ -123,6 +123,8 @@ export const hopperLoaderArgsForTarget = ( if (target.kind === "database") return ok([]); if (target.kind !== "executable") return err(new ProviderAdapterError("hopper", "resolve_analysis_profile")); + if (target.format === "raw-binary") + return err(new ProviderAdapterError("hopper", "resolve_analysis_profile")); const architecture = target.architecture; if (isElfMetadataArchitecture(architecture)) return err( @@ -155,7 +157,6 @@ export const hopperLoaderArgsForTarget = ( return ok(["-l", "WinPE", flag]); case "dos-mz": case "dos-com": - case "raw-binary": return err( new ProviderAdapterError("hopper", "resolve_analysis_profile"), ); diff --git a/src/hopper/HopperMachOImage.ts b/src/hopper/HopperMachOImage.ts index 20a53105c..ed81ea749 100644 --- a/src/hopper/HopperMachOImage.ts +++ b/src/hopper/HopperMachOImage.ts @@ -2,11 +2,7 @@ import { createHash } from "node:crypto"; import { open, type FileHandle } from "node:fs/promises"; import { z } from "zod"; -import { - isMachOFormat, - isElfMetadataArchitecture, - type BinaryTarget, -} from "../domain/binaryTargetTypes.js"; +import type { BinaryTarget } from "../domain/binaryTargetTypes.js"; import { BinaryTargetError } from "../domain/configurationErrors.js"; import { AnalysisCancelledError } from "../domain/analysisErrorCore.js"; import { @@ -48,10 +44,7 @@ export const resolveHopperMachOImage = async ( ), ); if (signal?.aborted) return err(new AnalysisCancelledError("open_binary")); - if ( - !isMachOFormat(target.format) || - isElfMetadataArchitecture(target.architecture) - ) + if (target.format !== "mach-o" && target.format !== "mach-o-universal") return invalid("Hopper FAT64 image preparation requires a Mach-O target"); let file: FileHandle; try { diff --git a/tests/boundary/cli/dylibResolutionCli.test.ts b/tests/boundary/cli/dylibResolutionCli.test.ts index 66fa21ba3..21ea99a9c 100644 --- a/tests/boundary/cli/dylibResolutionCli.test.ts +++ b/tests/boundary/cli/dylibResolutionCli.test.ts @@ -180,14 +180,17 @@ describe("trace-dylib-resolution CLI", () => { // A root below a symlinked directory that points outside the bundle. const outside = await createTestTempDirectory("rea-dylib-outside-"); await writeFiles(outside, { Tool: machoImage({}) }); - await symlink(outside, join(app, "Contents/Linked")); + await symlink(outside, join(app, "Contents/Linked"), "dir"); for (const [root, reason, message] of [ ["../outside", "invalid_format", "Use a normalized path"], ["Contents/Info.plist", "invalid_value", "is not a Mach-O image"], [ "Contents/Linked/Tool", "invalid_value", - "resolves outside the analyzed root", + // Windows link text is outside the POSIX dyld path representation. + process.platform === "win32" + ? "is not a regular file in the analyzed root" + : "resolves outside the analyzed root", ], ] as const) { await expectRootInputError(cli, { app, root, reason, message }); @@ -378,7 +381,8 @@ cliTest( }, ); -describe.skipIf(process.getuid?.() === 0)( +// This permission fixture requires POSIX mode bits and a non-root caller. +describe.skipIf(process.platform === "win32" || process.getuid?.() === 0)( "trace-dylib-resolution permissions", () => { cliTest( @@ -510,6 +514,8 @@ cliTest( app, "--root", "Contents/MacOS/App", + "--architecture", + "arm64e", "--json", ], environment: ENVIRONMENT, @@ -518,9 +524,7 @@ cliTest( const trace = dylibResolutionResultSchema.parse( parseEvidence(result.json).normalized_result, ); - expect( - trace.roots.filter(({ architecture }) => architecture === "arm64e"), - ).toHaveLength(2); + expect(trace.roots).toHaveLength(2); const environments = trace.findings .filter(({ kind }) => kind === "dyld-environment-present") .map(({ explanation }) => diff --git a/tests/boundary/cli/rawBinaryCli.test.ts b/tests/boundary/cli/rawBinaryCli.test.ts index 1cff4cac6..f12312d02 100644 --- a/tests/boundary/cli/rawBinaryCli.test.ts +++ b/tests/boundary/cli/rawBinaryCli.test.ts @@ -38,6 +38,36 @@ cliTest( }, ); +cliTest( + "rejects a 6502 image crossing the 16-bit boundary before provider startup", + async ({ cli }) => { + const root = await createTestTempDirectory("rea-6502-cli-overflow-"); + const path = join(root, "image.bin"); + await writeFile(path, Buffer.from("a92a60", "hex")); + const result = await cli.run({ + arguments: [ + "inspect", + path, + "--target-format", + "raw-binary", + "--architecture", + "6502", + "--entry-point", + "0xffff", + "--provider", + "ghidra", + "--json", + ], + environment: { HOME: root, XDG_CONFIG_HOME: root, XDG_CACHE_HOME: root }, + }); + expect(result.exitCode).toBe(1); + expect(result.json).toMatchObject({ + code: "invalid_request", + details: { reason: expect.stringContaining("6502 address width") }, + }); + }, +); + cliTest( "rejects an unaligned AArch64 entry before provider startup", async ({ cli }) => { diff --git a/tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts b/tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts index 6687b0ef7..7adf193f4 100644 --- a/tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts +++ b/tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts @@ -1,5 +1,5 @@ import { parseConfig } from "../../../src/config/parseConfig.js"; -import { writeFile } from "node:fs/promises"; +import { symlink, writeFile } from "node:fs/promises"; import { basename, dirname, join } from "node:path"; import { describe, expect, it, vi } from "vitest"; @@ -14,7 +14,7 @@ import type { import type { DirectAnalysisDependencies } from "../../../src/application/DirectAnalysisDependencies.js"; import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js"; import { createAnalysisExecution } from "../../../src/application/AnalysisProvider.js"; -import { ok } from "../../../src/domain/result.js"; +import { err, ok } from "../../../src/domain/result.js"; import { readAnalysisSnapshot, writeAnalysisSnapshot, @@ -40,6 +40,7 @@ import { } from "../../fixtures/binarySession.js"; import { BinarySession } from "../../../src/application/binary/BinarySession.js"; import { OwnedStagedOutput } from "../../../src/filesystem/OwnedStagedOutput.js"; +import { ProviderAdapterError } from "../../../src/domain/providerAdapterError.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { CAPABILITIES as GHIDRA_CAPABILITIES, @@ -396,7 +397,85 @@ describe("snapshot output cleanup lifecycle", () => { }); }); +const verifyWindowsSnapshotAdmission = async ( + selectedPath: "direct" | "junction", +): Promise => { + const fixture = await createSnapshotReplayFixture( + "rea-direct-snapshot-junction-", + ); + const rawPath = join(dirname(fixture.path), "image.bin"); + await writeFile(rawPath, Buffer.from("a92a60", "hex")); + const linked = join(dirname(fixture.path), "linked"); + const linkedPath = join(linked, basename(rawPath)); + const identity = { ...GHIDRA_PROVIDER_IDENTITY, version: "12.1.4" }; + const profile = createAnalysisProfile(identity, { + fixture: true, + }); + const baseProvider = fixture.createProvider({ + identity, + profile, + supportedOperations: ["search_strings"], + }); + let rejectAdmission = false; + const provider: AnalysisProvider = { + ...baseProvider, + createClient: (target, profile, context) => { + const client = baseProvider.createClient(target, profile, context); + return { + ...client, + execute: (operation, parameters, options) => + rejectAdmission && operation === "search_strings" + ? Promise.resolve( + err(new ProviderAdapterError("ghidra", operation)), + ) + : client.execute(operation, parameters, options), + }; + }, + }; + const dependencies = fixture.dependencies(provider); + const arguments_ = { pattern: "fixture", document: "fixture" }; + const options = { + snapshotPath: fixture.snapshotPath, + formatHint: "raw-binary" as const, + architecture: "6502" as const, + entryPoint: "0x8001", + }; + const first = await runDirectAnalysis( + dependencies, + rawPath, + "search_strings", + arguments_, + options, + ); + expect(first).toMatchObject({ operation: "search_strings" }); + expect(fixture.starts).toHaveLength(1); + if (selectedPath === "junction") + await symlink(dirname(fixture.path), linked, "junction"); + rejectAdmission = true; + const second = await runDirectAnalysis( + dependencies, + selectedPath === "junction" ? linkedPath : rawPath, + "search_strings", + arguments_, + options, + ); + expect(second).toMatchObject({ error: "Analysis failed" }); + expect(fixture.starts).toHaveLength(2); + expect( + fixture.calls.filter((call) => call === "search_strings"), + ).toHaveLength(1); +}; + describe("direct analysis composed snapshot replay", () => { + it.skipIf(process.platform !== "win32")( + "does not replay a CLI snapshot through a Windows junction", + () => verifyWindowsSnapshotAdmission("junction"), + ); + it.skipIf(process.platform !== "win32")( + "does not replay a CLI snapshot before native admission on a canonical path", + () => verifyWindowsSnapshotAdmission("direct"), + ); + it.each([ { tool: "search_strings", @@ -516,7 +595,7 @@ describe("Ghidra composed workflow snapshot replay", () => { { tool: "inspect_native_api", arguments: { procedure: "0x1000" } }, { tool: "trace_feature", arguments: { query: "fixture" } }, ] as const)( - "persists and replays $tool with production Ghidra capability policies", + "persists $tool with production Ghidra capability policies", async ({ tool, arguments: parameters }) => { const fixture = await createSnapshotReplayFixture("rea-ghidra-workflow-"); const profile = createAnalysisProfile( @@ -554,8 +633,14 @@ describe("Ghidra composed workflow snapshot replay", () => { { snapshotPath: fixture.snapshotPath }, ); expect(replay).toEqual(first); - expect(fixture.starts).toHaveLength(1); - expect(fixture.calls).toEqual(initialCalls); + if (process.platform === "win32") { + // The first provider call in a new CLI session admits its source. + expect(fixture.starts).toHaveLength(2); + expect(fixture.calls.length).toBeGreaterThan(initialCalls.length); + } else { + expect(fixture.starts).toHaveLength(1); + expect(fixture.calls).toEqual(initialCalls); + } }, ); }); diff --git a/tests/boundary/filesystem/javascript-applications/javascriptArtifactReconstruction.test.ts b/tests/boundary/filesystem/javascript-applications/javascriptArtifactReconstruction.test.ts index 6b7253b6d..416337097 100644 --- a/tests/boundary/filesystem/javascript-applications/javascriptArtifactReconstruction.test.ts +++ b/tests/boundary/filesystem/javascript-applications/javascriptArtifactReconstruction.test.ts @@ -388,7 +388,7 @@ it("produces deterministic ASAR graphs with unpacked native linkage and complete await mkdir(source); await writeJavaScriptArtifactFixture(source); const archive = join(root, "app.asar"); - await createPackageWithOptions(source, archive, { unpack: "**/*.node" }); + await createPackageWithOptions(source, archive, { unpack: "*.node" }); const first = await reconstructJavaScriptArtifact({ input_path: archive, @@ -487,7 +487,7 @@ it("keeps ASAR analysis usable when unpacked native companion bytes are absent", await mkdir(source); await writeJavaScriptArtifactFixture(source); const archive = join(root, "app.asar"); - await createPackageWithOptions(source, archive, { unpack: "**/*.node" }); + await createPackageWithOptions(source, archive, { unpack: "*.node" }); await rm(join(`${archive}.unpacked`, "native", "addon.node")); const snapshot = await scanArtifactInventory(archive); @@ -515,7 +515,7 @@ it("recurses into filesystem-backed ASAR containers without losing container-rel const resources = join(outer, "resources"); await mkdir(resources, { recursive: true }); await createPackageWithOptions(source, join(resources, "app.asar"), { - unpack: "**/*.node", + unpack: "*.node", }); const result = await reconstructJavaScriptArtifact({ diff --git a/tests/boundary/filesystem/rawBinaryTarget.test.ts b/tests/boundary/filesystem/rawBinaryTarget.test.ts index 5fd171d20..2757f0507 100644 --- a/tests/boundary/filesystem/rawBinaryTarget.test.ts +++ b/tests/boundary/filesystem/rawBinaryTarget.test.ts @@ -1,5 +1,11 @@ import { createHash } from "node:crypto"; -import { readFile, writeFile } from "node:fs/promises"; +import { + mkdir, + readFile, + realpath, + symlink, + writeFile, +} from "node:fs/promises"; import { join } from "node:path"; import { describe, expect, it } from "vitest"; @@ -7,6 +13,37 @@ import { parseBinaryTarget } from "../../../src/application/BinaryTargetResolver import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; describe("raw-binary admission", () => { + it("preserves the requested raw path for Windows reparse admission", async () => { + const directory = await createTestTempDirectory("rea-raw-linked-"); + const actual = join(directory, "actual"); + const linked = join(directory, "linked"); + await mkdir(actual); + const bytes = Buffer.from("a90160", "hex"); + await writeFile(join(actual, "fixture.bin"), bytes); + await symlink( + actual, + linked, + process.platform === "win32" ? "junction" : "dir", + ); + const requested = join(linked, "fixture.bin"); + const canonical = await realpath(requested); + const target = await parseBinaryTarget("linked/fixture.bin", { + cwd: directory, + formatHint: "raw-binary", + architecture: "6502", + entryPoint: "0x8000", + }); + expect(target).toMatchObject({ + ok: true, + value: { + path: canonical, + sourcePath: process.platform === "win32" ? requested : canonical, + sha256: createHash("sha256").update(bytes).digest("hex"), + format: "raw-binary", + architecture: "6502", + }, + }); + }); it("requires explicit architecture and entry even with a misleading suffix", async () => { const directory = await createTestTempDirectory("rea-raw-target-"); const path = join(directory, "fixture.js"); diff --git a/tests/boundary/mcp/rawBinaryMcp.test.ts b/tests/boundary/mcp/rawBinaryMcp.test.ts index c1d2029c2..26a0e6501 100644 --- a/tests/boundary/mcp/rawBinaryMcp.test.ts +++ b/tests/boundary/mcp/rawBinaryMcp.test.ts @@ -1,4 +1,4 @@ -import { writeFile } from "node:fs/promises"; +import { mkdir, symlink, writeFile } from "node:fs/promises"; import { join } from "node:path"; import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; @@ -10,7 +10,8 @@ import { createServer } from "../../../src/server/createServer.js"; import { toolContract } from "../../../src/contracts/toolContracts.js"; import type { AnalysisProvider } from "../../../src/application/AnalysisProvider.js"; import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js"; -import { ok } from "../../../src/domain/result.js"; +import { ProviderAdapterError } from "../../../src/domain/providerAdapterError.js"; +import { err, ok } from "../../../src/domain/result.js"; import { observed } from "../../fixtures/analysisExecution.js"; import { createCacheProvider, @@ -18,6 +19,7 @@ import { } from "../../fixtures/binarySession.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { silentLogger } from "../../../src/logger.js"; +import { GHIDRA_PROVIDER_IDENTITY } from "../../../src/ghidra/GhidraProviderCapabilities.js"; import type { CallToolResult } from "@modelcontextprotocol/server"; const connect = async ( @@ -50,29 +52,97 @@ const structured = (result: CallToolResult): Record => { return z.record(z.string(), z.unknown()).parse(result.structuredContent); }; -it("opens a headerless image when architecture and entry are supplied", async () => { - const root = await createTestTempDirectory("rea-raw-mcp-"); - const path = join(root, "ecu.bin"); - await writeFile(path, Buffer.from("1eff2fe1", "hex")); - const { client } = await connect(); - const opened = await client.callTool({ - name: "open_binary", - arguments: { +const withGhidraFixtureIdentity = ( + provider: AnalysisProvider, +): AnalysisProvider => { + const identity = { ...GHIDRA_PROVIDER_IDENTITY, version: "12.1.4" }; + const profile = createAnalysisProfile(identity, { fixture: true }); + return { + ...provider, + identity: () => identity, + capabilities: () => + provider.capabilities().map((capability) => ({ + ...capability, + provider: identity, + })), + resolveAnalysisProfile: () => Promise.resolve(ok({ profile })), + createClient: (target, _profile, context) => { + const client = provider.createClient(target, profile, context); + return { + ...client, + execute: async (operation, parameters, options) => { + const result = await client.execute(operation, parameters, options); + return result.ok + ? ok({ + ...result.value, + provider: identity, + analysisProfile: profile, + }) + : result; + }, + }; + }, + }; +}; + +it("keeps 6502 out of detected lifecycle output contracts", () => { + const schema = toolContract("open_binary").outputSchema; + const result = { + path: "/fixture/image.bin", + kind: "executable", + sha256: "a".repeat(64), + architecture: "6502", + }; + expect( + schema.safeParse({ result: { ...result, format: "pe" } }).success, + ).toBe(false); + expect( + schema.safeParse({ + result: { ...result, format: "raw-binary", entry_point: "0x8001" }, + }).success, + ).toBe(true); + const validateAdvertised = new Ajv2020({ strict: false }).compile( + z.toJSONSchema(schema), + ); + expect(validateAdvertised({ result: { ...result, format: "pe" } })).toBe( + false, + ); + expect( + validateAdvertised({ + result: { ...result, format: "raw-binary", entry_point: "0x8001" }, + }), + ).toBe(true); +}); + +it.each([ + ["arm", "0x08000000", "0x8000000", "1eff2fe1"], + ["6502", "0x008001", "0x8001", "a92a60"], +])( + "opens a headerless %s image when architecture and entry are supplied", + async (architecture, entryPoint, canonical, bytes) => { + const root = await createTestTempDirectory("rea-raw-mcp-"); + const path = join(root, "ecu.bin"); + await writeFile(path, Buffer.from(bytes, "hex")); + const { client } = await connect(); + const opened = await client.callTool({ + name: "open_binary", + arguments: { + path, + format: "raw-binary", + architecture, + entry_point: entryPoint, + }, + }); + expect(opened.isError).not.toBe(true); + expect(structured(opened).result).toMatchObject({ path, format: "raw-binary", - architecture: "arm", - entry_point: "0x08000000", - }, - }); - expect(opened.isError).not.toBe(true); - expect(structured(opened).result).toMatchObject({ - path, - format: "raw-binary", - kind: "executable", - architecture: "arm", - entry_point: "0x8000000", - }); -}); + kind: "executable", + architecture, + entry_point: canonical, + }); + }, +); it("rejects an unaligned AArch64 entry before selecting or starting a provider", async () => { const root = await createTestTempDirectory("rea-raw-mcp-alignment-"); @@ -217,3 +287,118 @@ it("reuses a canonical raw entry and replaces the session when the entry changes expect(created).toEqual(["0x8000000", "0x9000000"]); expect(closed).toEqual(["0x8000000"]); }); + +it.skipIf(process.platform !== "win32")( + "does not replay a saved read after switching to a Windows junction", + async () => { + const root = await createTestTempDirectory("rea-raw-mcp-linked-reopen-"); + const actual = join(root, "actual"); + const linked = join(root, "linked"); + await mkdir(actual); + const path = join(actual, "image.bin"); + await writeFile(path, Buffer.from("a92a60", "hex")); + await symlink(actual, linked, "junction"); + const created: string[] = []; + const executed: string[] = []; + const fixture = createCacheProvider([]); + const { client } = await connect( + withGhidraFixtureIdentity({ + ...fixture, + createClient: (target) => { + created.push(target.sourcePath ?? target.path); + return { + execute: (operation) => { + if (operation === "address_name") + executed.push(target.sourcePath ?? target.path); + if ( + operation === "address_name" && + target.sourcePath === join(linked, "image.bin") + ) + return Promise.resolve( + err(new ProviderAdapterError("fixture", operation)), + ); + return Promise.resolve( + observed(operation === "address_name" ? "entry" : null), + ); + }, + close: () => Promise.resolve(ok(null)), + }; + }, + }), + ); + const linkedPath = join(linked, "image.bin"); + for (const requested of [path, linkedPath]) { + const opened = await client.callTool({ + name: "open_binary", + arguments: { + path: requested, + format: "raw-binary", + architecture: "6502", + entry_point: "0x8001", + }, + }); + expect(opened.isError, JSON.stringify(opened)).not.toBe(true); + const address = await client.callTool({ + name: "address_name", + arguments: { address: "0x8001", document: "image.bin" }, + }); + expect(address.isError === true, JSON.stringify(address)).toBe( + requested === linkedPath, + ); + } + expect(created).toEqual([path, linkedPath]); + expect(executed).toEqual([path, linkedPath]); + }, +); + +it.skipIf(process.platform !== "win32")( + "reuses saved reads after a fixture admits a noncanonical Windows path", + async () => { + const root = await createTestTempDirectory("rea-raw-mcp-admitted-path-"); + const actual = join(root, "actual"); + const linked = join(root, "linked"); + await mkdir(actual); + const path = join(actual, "image.bin"); + await writeFile(path, Buffer.from("a92a60", "hex")); + await symlink(actual, linked, "junction"); + const calls: string[] = []; + const fixture = createCacheProvider([]); + // This fixture accepts the alias to isolate cache behavior after admission. + // The real Ghidra verifier checks that a junction itself is rejected. + const { client, session } = await connect( + withGhidraFixtureIdentity({ + ...fixture, + createClient: () => ({ + execute: (operation) => { + if (operation === "address_name") calls.push(operation); + return Promise.resolve( + observed(operation === "health" ? null : "entry"), + ); + }, + close: () => Promise.resolve(ok(null)), + }), + }), + ); + const linkedPath = join(linked, "image.bin"); + const opened = await client.callTool({ + name: "open_binary", + arguments: { + path: linkedPath, + format: "raw-binary", + architecture: "6502", + entry_point: "0x8001", + }, + }); + expect(opened.isError, JSON.stringify(opened)).not.toBe(true); + expect(session.activeTarget()?.sourcePath).toBe(linkedPath); + expect(session.activeTarget()?.path).toBe(path); + for (const _ of [0, 1]) { + const address = await client.callTool({ + name: "address_name", + arguments: { address: "0x8001", document: "image.bin" }, + }); + expect(address.isError, JSON.stringify(address)).not.toBe(true); + } + expect(calls).toEqual(["address_name"]); + }, +); diff --git a/tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts b/tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts index 0bcf12bb2..53ffcd7d3 100644 --- a/tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts +++ b/tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts @@ -135,7 +135,7 @@ it("ignores unavailable members of a nested ASAR but refuses them as active entr const source = join(root, "bundle"); await mkdir(source); const archive = join(source, "app.asar"); - await createPackageWithOptions(contents, archive, { unpack: "**/*.node" }); + await createPackageWithOptions(contents, archive, { unpack: "*.node" }); await rm(join(`${archive}.unpacked`, "native", "addon.node")); const nestedOutput = join(root, "nested-output"); diff --git a/tests/conformance/ghidra/6502.cfg b/tests/conformance/ghidra/6502.cfg new file mode 100644 index 000000000..cb85a4e76 --- /dev/null +++ b/tests/conformance/ghidra/6502.cfg @@ -0,0 +1,9 @@ +FEATURES { + STARTADDRESS: default = $8001; +} +MEMORY { + IMAGE: start = %S, size = $10000 - %S, file = %O; +} +SEGMENTS { + CODE: load = IMAGE, type = ro; +} diff --git a/tests/conformance/ghidra/6502.s b/tests/conformance/ghidra/6502.s new file mode 100644 index 000000000..201f8bcd2 --- /dev/null +++ b/tests/conformance/ghidra/6502.s @@ -0,0 +1,19 @@ +; Source-owned NMOS 6502 fixture: direct call, conditional branch, and arithmetic. +.setcpu "6502" +.segment "CODE" +.export entry, add_three + +entry: + lda #7 + jsr add_three + cmp #10 + bne mismatch + lda #42 + rts +mismatch: + lda #0 + rts +add_three: + clc + adc #3 + rts diff --git a/tests/conformance/readmeConsistency.test.ts b/tests/conformance/readmeConsistency.test.ts index 79a1fc518..90f8e5886 100644 --- a/tests/conformance/readmeConsistency.test.ts +++ b/tests/conformance/readmeConsistency.test.ts @@ -35,6 +35,20 @@ const shellExamples = (content: string): string[] => ); describe("onboarding documentation product facts", () => { + it.each([ + ["docs/native-investigation.md", "raw-binary.md#nmos-6502"], + ["docs/ghidra-riscv.md", "raw-binary.md"], + ])( + "routes raw admission guidance in %s to the owning guide", + async (path, target) => { + const content = await readFile(resolve(path), "utf8"); + const links = [...content.matchAll(/\]\(([^)\s]+)\)/gu)].map( + (match) => match[1], + ); + expect(links, path).toContain(target); + }, + ); + it("keeps the installation MCP example aligned with the package registration command", async () => { const content = await readFile(resolve("docs/installation.md"), "utf8"); expect(jsonExamples(content)).toContainEqual( diff --git a/tests/fixtures/strippedAsarAddon.ts b/tests/fixtures/strippedAsarAddon.ts index 5ff76356c..3422ac2ca 100644 --- a/tests/fixtures/strippedAsarAddon.ts +++ b/tests/fixtures/strippedAsarAddon.ts @@ -29,7 +29,7 @@ export const createStrippedAsarAddon = async ( await mkdir(dirname(path), { recursive: true }); await writeFile(path, file.contents); } - await createPackageWithOptions(source, archive, { unpack: "**/*.node" }); + await createPackageWithOptions(source, archive, { unpack: "*.node" }); await writeFile( join(`${archive}.unpacked`, "node_modules", "btime", "binding.node"), stripped, From cdc41fc86c440317160652c1000fe8b082ba76f1 Mon Sep 17 00:00:00 2001 From: TonyWu <75903587+TonyWu2333@users.noreply.github.com> Date: Tue, 13 Oct 2026 21:13:18 +0800 Subject: [PATCH 05/24] fix(managed): classify ReadyToRun images without ILONLY as modern .NET (#2075) * fix(managed): classify ReadyToRun images without ILONLY as modern .NET crossgen2 clears COMIMAGE_FLAGS_ILONLY and sets IL_LIBRARY on ReadyToRun output. inspect_managed_artifact checked the clear ILONLY flag before the ReadyToRun signature, so real ReadyToRun assemblies were reported as runtime_family mixed-clr-native and implementation cpp-cli-mixed, and cil-and-ready-to-run was unreachable for crossgen2 output. Check the ReadyToRun signature first; a clear ILONLY flag without it still reports mixed CLR/native code. * test(managed): exhaust capacity through inspected assembly references --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- src/dotnet/ManagedArtifactInspector.test.ts | 57 +++++++++++++++++++++ src/dotnet/ManagedArtifactInspector.ts | 18 +++++-- 2 files changed, 71 insertions(+), 4 deletions(-) diff --git a/src/dotnet/ManagedArtifactInspector.test.ts b/src/dotnet/ManagedArtifactInspector.test.ts index 3b734c176..50de3a39b 100644 --- a/src/dotnet/ManagedArtifactInspector.test.ts +++ b/src/dotnet/ManagedArtifactInspector.test.ts @@ -547,6 +547,63 @@ describe("managed fixture data layout", () => { }); }); +describe("managed implementation classification", () => { + it("classifies crossgen2 ReadyToRun output without ILONLY as modern .NET", () => { + // crossgen2 writes IL_LIBRARY | STRONGNAMESIGNED and clears ILONLY. + const bytes = buildManagedPeFixture({ readyToRun: true, cliFlags: 0xc }); + const result = inspectManagedArtifactBytes( + bytes, + managedPeFixtureTarget(bytes), + ); + + expect(result.classification).toMatchObject({ + runtime_family: "modern-dotnet", + implementation: "cil-and-ready-to-run", + }); + expect(result.classification.evidence).toContainEqual( + expect.objectContaining({ code: "ready-to-run-signature-observed" }), + ); + expect(result.classification.evidence).not.toContainEqual( + expect.objectContaining({ code: "cli-il-only-clear" }), + ); + }); + + it("preserves ReadyToRun runtime identity after metadata capacity is exhausted", () => { + const bytes = buildManagedPeFixture({ + readyToRun: true, + cliFlags: 0xc, + references: ["x".repeat(12 * 1024 * 1024)], + }); + const result = inspectManagedArtifactBytes( + bytes, + managedPeFixtureTarget(bytes), + ); + expect(result.classification).toMatchObject({ + status: "managed", + runtime_family: "modern-dotnet", + }); + expect(result.coverage.issues).toContainEqual( + expect.objectContaining({ code: "resource-limit" }), + ); + }); + + it("keeps a clear ILONLY flag without ReadyToRun as mixed CLR/native", () => { + const bytes = buildManagedPeFixture({ cliFlags: 0 }); + const result = inspectManagedArtifactBytes( + bytes, + managedPeFixtureTarget(bytes), + ); + + expect(result.classification).toMatchObject({ + runtime_family: "mixed-clr-native", + implementation: "cpp-cli-mixed", + }); + expect(result.classification.evidence).toContainEqual( + expect.objectContaining({ code: "cli-il-only-clear" }), + ); + }); +}); + describe("managed artifact metadata identity", () => { it("accepts CLI metadata GUIDs without RFC UUID version or variant bits", () => { const bytes = buildManagedPeFixture({ diff --git a/src/dotnet/ManagedArtifactInspector.ts b/src/dotnet/ManagedArtifactInspector.ts index 7e7acb087..625969aba 100644 --- a/src/dotnet/ManagedArtifactInspector.ts +++ b/src/dotnet/ManagedArtifactInspector.ts @@ -158,6 +158,15 @@ interface RuntimeClassificationInput { readonly inventory: ManagedMetadataInventory; } +const runtimeFamilyFromCliHeader = ( + cli: ManagedCliHeader | null, +): ManagedArtifactInspection["classification"]["runtime_family"] => + cli?.readyToRunSignature === true + ? "modern-dotnet" + : cli !== null && (cli.flags & 1) === 0 + ? "mixed-clr-native" + : "unknown"; + const runtimeFamilyFor = ( cli: ManagedCliHeader, inventory: ManagedMetadataInventory, @@ -180,7 +189,9 @@ const runtimeFamilyFor = ( ); let runtimeFamily: ManagedArtifactInspection["classification"]["runtime_family"] = "unknown"; - if ((cli.flags & 1) === 0) { + // crossgen2 clears ILONLY on ReadyToRun output, so only a clear flag + // without the ReadyToRun signature indicates mixed CLR/native code. + if (runtimeFamilyFromCliHeader(cli) === "mixed-clr-native") { runtimeFamily = "mixed-clr-native"; evidence.push({ code: "cli-il-only-clear", @@ -239,8 +250,8 @@ const implementationFor = ( cli: ManagedCliHeader, metadata: ManagedMetadataLayout, ): ManagedArtifactInspection["classification"]["implementation"] => { - if ((cli.flags & 1) === 0) return "cpp-cli-mixed"; if (cli.readyToRunSignature) return "cil-and-ready-to-run"; + if ((cli.flags & 1) === 0) return "cpp-cli-mixed"; return (metadata.rowCounts[6] ?? 0) === 0 ? "metadata-only" : "cil"; }; @@ -303,8 +314,7 @@ const partialMetadataResult = ({ classification: { status: issue.code === "resource-limit" ? "managed" : "malformed", container: "pe", - runtime_family: - (layout.cli?.flags ?? 1) & 1 ? "unknown" : "mixed-clr-native", + runtime_family: runtimeFamilyFromCliHeader(layout.cli), implementation: "unknown", managed_architecture: "unknown", evidence: [ From 97dc6581f45adfe5fcaa9c98e77e7dc6bc423a6e Mon Sep 17 00:00:00 2001 From: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> Date: Tue, 13 Oct 2026 06:14:31 -0700 Subject: [PATCH 06/24] fix(dotnet): name the instruction when a CIL operand leaves the method body (#2697) The decoder already reports "Instruction X at IL offset N leaves method body" when an operand's computed end passes the IL body, but only after the operand was read. Operand readers use Buffer reads, so an operand cut off by the body end threw a RangeError first. The catch-all stored Node's message ("Attempt to access memory outside buffer bounds" or an offset RangeError) as body.issue, dropping the instruction and offset. A body ending in a lone 0xFE two-byte opcode prefix failed the same way. Treat a RangeError from the operand read as the operand leaving the body so the existing diagnostic applies, and check the second opcode byte before reading it. Malformed status, partial coverage and earlier decoded instructions are unchanged. Fixes #2677 Co-authored-by: Claude Opus 5.5 --- src/dotnet/ManagedCilDecoding.test.ts | 38 +++++++++++++++++++ src/dotnet/ManagedMemberInstructionDecoder.ts | 28 +++++++++++++- 2 files changed, 64 insertions(+), 2 deletions(-) diff --git a/src/dotnet/ManagedCilDecoding.test.ts b/src/dotnet/ManagedCilDecoding.test.ts index c1ed21130..55a008b72 100644 --- a/src/dotnet/ManagedCilDecoding.test.ts +++ b/src/dotnet/ManagedCilDecoding.test.ts @@ -510,3 +510,41 @@ describe("managed exception section decoding", () => { }); }); }); + +describe("managed CIL truncated operands", () => { + const tinyBody = (il: readonly number[]) => { + const bytes = buildManagedPeFixture({ + ilBody: Buffer.from([(il.length << 2) | 0x02, ...il]), + }); + const inspection = inspectManagedMembersBytes( + bytes, + managedPeFixtureTarget(bytes), + ); + return { body: inspection.methods[0]?.body, coverage: inspection.coverage }; + }; + + it.each([ + [[0x20], 0, "Instruction ldc.i4 at IL offset 0 leaves method body"], + [ + [0x00, 0x20, 0x01], + 1, + "Instruction ldc.i4 at IL offset 1 leaves method body", + ], + [[0x2b], 0, "Instruction br.s at IL offset 0 leaves method body"], + [[0x45, 0x02], 0, "Instruction switch at IL offset 0 leaves method body"], + [ + [0xfe, 0x09, 0x00], + 0, + "Instruction ldarg at IL offset 0 leaves method body", + ], + [[0x00, 0xfe], 1, "Two-byte CIL opcode at IL offset 1 leaves method body"], + ])("names the instruction and IL offset for IL %j", (il, decoded, issue) => { + const { body, coverage } = tinyBody(il); + expect(body).toMatchObject({ + status: "malformed", + decoded_instruction_count: decoded, + issue, + }); + expect(coverage.state).toBe("partial"); + }); +}); diff --git a/src/dotnet/ManagedMemberInstructionDecoder.ts b/src/dotnet/ManagedMemberInstructionDecoder.ts index 7ef518733..b05dd083f 100644 --- a/src/dotnet/ManagedMemberInstructionDecoder.ts +++ b/src/dotnet/ManagedMemberInstructionDecoder.ts @@ -384,6 +384,22 @@ const readOperand = ( return { next: offset, anchorKind: "unknown", value: null }; }; +/** An operand cut off by the end of the IL body has no decodable value. */ +const readOperandWithin = ( + il: Buffer, + offset: number, + kind: string, +): OperandResult | undefined => { + try { + return readOperand(il, offset, kind); + } catch (cause: unknown) { + // Buffer reads past the IL body throw RangeError; report it as the + // instruction leaving the method body rather than as a Buffer message. + if (cause instanceof RangeError) return undefined; + throw cause; + } +}; + export const decodeInstructions = ( il: Buffer, admitInstruction?: (offset: number) => void, @@ -404,14 +420,22 @@ export const decodeInstructions = ( const start = offset; const first = il.readUInt8(offset); offset += 1; + if (first === 0xfe && offset >= il.length) { + issue = `Two-byte CIL opcode at IL offset ${String(start)} leaves method body`; + break; + } const code = first === 0xfe ? 0xfe00 + il.readUInt8(offset++) : first; const descriptor = OPCODES.get(code); if (descriptor === undefined) { issue = `Unsupported CIL opcode 0x${code.toString(16)} at IL offset ${String(start)}`; break; } - const operand = readOperand(il, offset, descriptor.operand); - if (operand.next < offset || operand.next > il.length) { + const operand = readOperandWithin(il, offset, descriptor.operand); + if ( + operand === undefined || + operand.next < offset || + operand.next > il.length + ) { issue = `Instruction ${descriptor.name} at IL offset ${String(start)} leaves method body`; break; } From ece15ed6a18122f95e59d7d9f8bf9f0d3589f9f6 Mon Sep 17 00:00:00 2001 From: xin zhuang <65798732+1052326311@users.noreply.github.com> Date: Tue, 13 Oct 2026 21:15:33 +0800 Subject: [PATCH 07/24] fix(electron): identify each unresolved scenario path (#2279) Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- .../cli/scenarioCliInputDiagnostics.test.ts | 41 +++++++++++++++++ tests/boundary/mcp/electronMcp.test.ts | 46 +++++++++++++++++++ 2 files changed, 87 insertions(+) diff --git a/tests/boundary/cli/scenarioCliInputDiagnostics.test.ts b/tests/boundary/cli/scenarioCliInputDiagnostics.test.ts index bc98d9d9f..ae03568f4 100644 --- a/tests/boundary/cli/scenarioCliInputDiagnostics.test.ts +++ b/tests/boundary/cli/scenarioCliInputDiagnostics.test.ts @@ -1,3 +1,5 @@ +import { join } from "node:path"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { describe, expect } from "vitest"; import { cliTest } from "../../support/cli/cliFixture.js"; @@ -62,3 +64,42 @@ describe("scenario CLI schema diagnostics", () => { }); }); }); + +for (const field of [ + "executable_path", + "application_path", + "application_root", +] as const) { + cliTest(`names the unresolved Electron ${field}`, async ({ cli }) => { + const root = await createTestTempDirectory("rea-electron-path-"); + const missing = join(root, "missing"); + const result = await cli.run({ + arguments: [ + "capture-electron-scenario", + JSON.stringify({ + executable_path: process.execPath, + application_path: root, + application_root: root, + actions: [], + [field]: missing, + }), + "--json", + ], + environment: { REA_LOG_LEVEL: "silent" }, + }); + expect(result.exitCode).toBe(1); + expect(result.json).toMatchObject({ + code: "invalid_request", + details: { + issues: [ + { + path: [field], + reason: "invalid_value", + message: expect.stringContaining("ENOENT"), + }, + ], + }, + }); + expect(JSON.stringify(result.json)).toContain(missing); + }); +} diff --git a/tests/boundary/mcp/electronMcp.test.ts b/tests/boundary/mcp/electronMcp.test.ts index fef9839aa..72d9fdaf6 100644 --- a/tests/boundary/mcp/electronMcp.test.ts +++ b/tests/boundary/mcp/electronMcp.test.ts @@ -471,3 +471,49 @@ it("exposes the target-free static JavaScript application workflow", async () => const evidenceFor = (value: unknown) => { return parseEvidence(value); }; + +it("reports every unresolved Electron path without starting a provider", async () => { + const root = await createTestTempDirectory("rea-electron-invalid-paths-"); + const capture = vi.fn(); + const session = createTestBinarySession(() => { + throw new Error("must not start"); + }); + const server = createServer( + { kind: "session", session }, + { + electronActiveObservation: { + capture, + identity: () => ({ id: "test", name: "test", version: "1" }), + }, + }, + ); + const client = new Client({ name: "electron-path-test", version: "1" }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + resources.push(client, server, session); + await server.connect(serverTransport); + await client.connect(clientTransport); + const fields = ["executable_path", "application_path", "application_root"]; + const result = await client.callTool({ + name: "capture_electron_scenario", + arguments: { + ...Object.fromEntries(fields.map((field) => [field, join(root, field)])), + actions: [], + }, + }); + expect(result.isError).toBe(true); + expect(parseMcpToolError(result)).toMatchObject({ + error: { + code: "invalid_request", + details: { + issues: fields.map((field) => ({ + path: [field], + reason: "invalid_value", + message: expect.stringContaining("ENOENT"), + })), + }, + }, + }); + expect(capture).not.toHaveBeenCalled(); + await client.ping(); +}); From e46bed73f190484984c42efcb6fae0ccc42da4ee Mon Sep 17 00:00:00 2001 From: Angelo Perez <102934797+Angelopgit@users.noreply.github.com> Date: Tue, 13 Oct 2026 09:16:00 -0400 Subject: [PATCH 08/24] fix(managed): decode the CIL no. prefix (#2241) The CIL opcode table held five of the six ECMA-335 prefix opcodes and omitted no. (FE 19 , III.2.2). A method body using it was reported as malformed with "Unsupported CIL opcode 0xfe19", losing its instruction counts and normalized IL digest. Decode it with its one-byte check mask and treat it as a prefix, so it shares an instruction boundary with the opcode it modifies. Claude-Session: https://claude.ai/code/session_012pRGskbPGjT444CgJfuhhj Co-authored-by: Claude Fable 5.1 --- src/dotnet/ManagedCilDecoding.test.ts | 25 +++++++++++++++++++ src/dotnet/ManagedMemberInstructionDecoder.ts | 2 ++ 2 files changed, 27 insertions(+) diff --git a/src/dotnet/ManagedCilDecoding.test.ts b/src/dotnet/ManagedCilDecoding.test.ts index 55a008b72..4ba33f4c9 100644 --- a/src/dotnet/ManagedCilDecoding.test.ts +++ b/src/dotnet/ManagedCilDecoding.test.ts @@ -345,6 +345,31 @@ describe("managed exception region prefix boundaries", () => { }); }); + it("decodes the no. prefix and keeps it with its opcode", () => { + // ldarg.0; no. nullcheck; ldfld 0x04000001; pop; ret + const il = Buffer.from([2, 0xfe, 0x19, 4, 0x7b, 1, 0, 0, 4, 0x26, 0x2a]); + expect( + managedBodyWithRegion({ il, tryLength: 10, handlerOffset: 10 }), + ).toMatchObject({ + status: "present", + issue: null, + decoded_instruction_count: 5, + opcode_counts: { "ldarg.0": 1, "no.": 1, ldfld: 1, pop: 1, ret: 1 }, + }); + expect( + managedBodyWithRegion({ + il, + tryOffset: 4, + tryLength: 5, + handlerOffset: 10, + }), + ).toMatchObject({ + status: "malformed", + issue: + "Exception clause try range does not align with CIL instruction boundaries", + }); + }); + it.each([ ["start", { tryOffset: 5, tryLength: 1, handlerOffset: 6 }], [ diff --git a/src/dotnet/ManagedMemberInstructionDecoder.ts b/src/dotnet/ManagedMemberInstructionDecoder.ts index b05dd083f..01c2b7814 100644 --- a/src/dotnet/ManagedMemberInstructionDecoder.ts +++ b/src/dotnet/ManagedMemberInstructionDecoder.ts @@ -12,6 +12,7 @@ const PREFIX_OPCODES = new Set([ "tail.", "constrained.", "readonly.", + "no.", ]); const OPCODES = new Map< @@ -232,6 +233,7 @@ const OPCODES = new Map< [0xfe16, { name: "constrained.", operand: "type" }], [0xfe17, { name: "cpblk", operand: "none" }], [0xfe18, { name: "initblk", operand: "none" }], + [0xfe19, { name: "no.", operand: "short-i" }], [0xfe1a, { name: "rethrow", operand: "none" }], [0xfe1c, { name: "sizeof", operand: "type" }], [0xfe1d, { name: "refanytype", operand: "none" }], From 11268fe241a86fde0b462270dd8708934c74ab06 Mon Sep 17 00:00:00 2001 From: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com> Date: Tue, 13 Oct 2026 06:16:29 -0700 Subject: [PATCH 09/24] perf(artifacts): share sealed inventory and report in inspect_artifact (#2690) inspect_artifact parsed a fresh inventory and inspection report but left both mutable, so each later Evidence boundary cloned and re-digested the complete payload: createArtifactInspection re-authenticated the nested inventory Evidence, createAnalysisExecution re-parsed the report, and recordEvidence re-authenticated the outer Evidence. Seal both exclusively owned values with freezeJsonSnapshot so those boundaries take the existing immutable-snapshot path. Output is byte-identical. Co-authored-by: Claude Opus 5.5 --- src/artifacts/ArtifactProvider.ts | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/src/artifacts/ArtifactProvider.ts b/src/artifacts/ArtifactProvider.ts index c95cec6d5..47d5906ea 100644 --- a/src/artifacts/ArtifactProvider.ts +++ b/src/artifacts/ArtifactProvider.ts @@ -40,6 +40,7 @@ import { ArtifactResourceScope } from "./ArtifactResourceScope.js"; import { artifactCapabilities } from "./ArtifactProviderMetadata.js"; import { ARTIFACT_GRAPH_PROVIDER } from "../application/InvestigationProviders.js"; import { createEvidence } from "../domain/evidence.js"; +import { freezeJsonSnapshot } from "../domain/immutableJson.js"; import { createArtifactInspection } from "../domain/artifactInspection.js"; import { ProviderCleanupError } from "../domain/providerCleanupError.js"; @@ -341,7 +342,11 @@ class ArtifactClient implements AnalysisClient { total: 1, message: "inventory substep started", }); - const inventory = await this.inventory(inventoryParameters, options); + // The freshly parsed inventory is exclusively owned here. Sealing it lets + // the nested Evidence share it instead of cloning and re-digesting it. + const inventory = freezeJsonSnapshot( + await this.inventory(inventoryParameters, options), + ); const subject = subjectFor( this.target.sourcePath ?? this.target.path, inventory.manifest, @@ -355,7 +360,10 @@ class ArtifactClient implements AnalysisClient { // The nested inventory already names every occurrence's logical path. locations: [], }); - const result = createArtifactInspection(inventoryEvidence); + // The returned execution and its Evidence then reuse this owned report. + const result = freezeJsonSnapshot( + createArtifactInspection(inventoryEvidence), + ); await options?.progress?.report({ phase: "inspect_artifact.inventory", completed: 1, From 59d33a2733ce50ed0ac3d0371b4b646237d33619 Mon Sep 17 00:00:00 2001 From: FengCi <119940611+FengCi0@users.noreply.github.com> Date: Tue, 13 Oct 2026 21:16:49 +0800 Subject: [PATCH 10/24] fix(native): keep lipo path echoes out of FAT records (#2605) * fix(native): keep lipo path echoes out of FAT records Parse FAT records only after separating the exact echoed operand path. Pass the same lipo operand at all three consumers and cover multiline paths through parser, slice-reader and compiled CLI regressions. * fix(native): use parsed lipo framing for container offsets * test(native): use explicit FAT fixtures for CLI regressions --- src/artifacts/MachOSliceArtifactReader.ts | 4 +- src/native/NativeMacOSProvider.ts | 7 +- src/native/NativeMachoInspection.ts | 14 +-- src/native/parsers/lipo.test.ts | 115 ++++++++++++++++-- src/native/parsers/lipo.ts | 55 ++++++--- .../cli/nativeMachoEchoedPathCli.test.ts | 91 ++++++++++---- .../providers/native/machoSliceReader.test.ts | 54 +++++++- .../providers/native/nativeProvider.test.ts | 23 ++-- .../native/nativeSelectedArchitecture.test.ts | 72 ++++++----- .../native/nativeTextRobustness.test.ts | 18 ++- 10 files changed, 347 insertions(+), 106 deletions(-) diff --git a/src/artifacts/MachOSliceArtifactReader.ts b/src/artifacts/MachOSliceArtifactReader.ts index f567458c7..b57b0247e 100644 --- a/src/artifacts/MachOSliceArtifactReader.ts +++ b/src/artifacts/MachOSliceArtifactReader.ts @@ -8,7 +8,7 @@ import { type NativeCommandRunner, } from "../native/CommandRunner.js"; import { - parseLipoArchitectures, + parseLipoOutput, type LipoArchitecture, } from "../native/parsers/lipo.js"; import type { MachoSlice } from "../domain/apple/dylibResolution.js"; @@ -114,7 +114,7 @@ export class MachOSliceArtifactReader implements ArtifactReader { }; await this.#verifySource(source); const fileSize = source.initial.size; - const architectures = parseLipoArchitectures(captured.value.stdout); + const { architectures } = parseLipoOutput(captured.value.stdout, this.path); const structural = await readStructuralSlices(source.handle, fileSize); await this.#verifySource(source); if (structural.status === "malformed") diff --git a/src/native/NativeMacOSProvider.ts b/src/native/NativeMacOSProvider.ts index a59ec90a7..691e830d9 100644 --- a/src/native/NativeMacOSProvider.ts +++ b/src/native/NativeMacOSProvider.ts @@ -73,7 +73,7 @@ import { } from "./SignatureTargetBinding.js"; import { parseCodeSignature } from "./parsers/codesign.js"; import { parseDemangledSymbols } from "./parsers/demangle.js"; -import { parseLipoArchitectures } from "./parsers/lipo.js"; +import { parseLipoOutput } from "./parsers/lipo.js"; import { parsePlistJson, parsePlistXml, @@ -507,7 +507,10 @@ class NativeMacOSClient implements AnalysisClient { ); if (!capture.ok) return capture; const provenance = [invocation(capture.value, this.target.path)]; - const architectures = parseLipoArchitectures(capture.value.stdout); + const { architectures } = parseLipoOutput( + capture.value.stdout, + this.target.path, + ); const result = listArchitecturesSchema.parse({ architectures: { items: architectures, diff --git a/src/native/NativeMachoInspection.ts b/src/native/NativeMachoInspection.ts index 5fca089cb..bb1966c3d 100644 --- a/src/native/NativeMachoInspection.ts +++ b/src/native/NativeMachoInspection.ts @@ -14,10 +14,7 @@ import { selectMachoArchitectureName, } from "./MachoArchitectureSelection.js"; import { parseDyldSymbols } from "./parsers/dyldInfo.js"; -import { - parseLipoArchitectures, - type LipoArchitecture, -} from "./parsers/lipo.js"; +import { parseLipoOutput, type LipoArchitecture } from "./parsers/lipo.js"; import { withoutEchoedPathHeader } from "./parsers/echoedPath.js"; import { parseOtoolLoadCommands } from "./parsers/otool.js"; import { reconcileMachoUuid } from "./parsers/machoUuid.js"; @@ -74,7 +71,10 @@ export const inspectNativeMacho = async ( } const lipo = captures.find(({ tool }) => tool === "lipo"); if (lipo === undefined) throw new TypeError("Missing lipo capture"); - const architectures = parseLipoArchitectures(lipo.stdout); + const { format, architectures } = parseLipoOutput( + lipo.stdout, + context.target.path, + ); const selection = selectArchitecture(context.target, architectures); if (selection.status === "absent") return err( @@ -145,7 +145,7 @@ export const inspectNativeMacho = async ( const containerOffset = selectedArchitectureName === null ? undefined - : /^Non-fat file:/mu.test(lipo.stdout) + : format === "thin" ? 0 : (architectures.find(({ name }) => name === selectedArchitectureName) ?.file_offset ?? undefined); @@ -185,7 +185,7 @@ const normalizeMacho = ( toInvocation: (capture: NativeCommandCapture) => NativeCommandInvocation, additionalLimitations: readonly string[], load: ReturnType, - architectures: ReturnType, + architectures: readonly LipoArchitecture[], artifactPath: string, selectedArchitecture: { readonly name: string | null; diff --git a/src/native/parsers/lipo.test.ts b/src/native/parsers/lipo.test.ts index 96f2afbbc..b5c97a415 100644 --- a/src/native/parsers/lipo.test.ts +++ b/src/native/parsers/lipo.test.ts @@ -1,10 +1,28 @@ import { describe, expect, it } from "vitest"; -import { parseLipoArchitectures } from "./lipo.js"; +import { parseLipoOutput } from "./lipo.js"; + +const fatRecords = `fat_magic 0xcafebabe +nfat_arch 2 +architecture arm64 + cputype CPU_TYPE_ARM64 + cpusubtype CPU_SUBTYPE_ARM64_ALL + capabilities 0x0 + offset 4096 + size 32 + align 2^12 (4096) +architecture x86_64 + cputype CPU_TYPE_X86_64 + cpusubtype CPU_SUBTYPE_X86_64_ALL + capabilities 0x0 + offset 8192 + size 32 + align 2^12 (4096) +`; describe("lipo architecture facts", () => { it("normalizes symbolic CPU fields and arm64e capability bits", () => { - const architectures = parseLipoArchitectures(`Fat header in: /bin/ls + const { format, architectures } = parseLipoOutput(`Fat header in: /bin/ls fat_magic 0xcafebabe nfat_arch 2 architecture x86_64 @@ -22,6 +40,7 @@ architecture arm64e size 89088 align 2^14 (16384) `); + expect(format).toBe("fat"); expect(architectures).toEqual([ expect.objectContaining({ name: "x86_64", @@ -50,9 +69,9 @@ architecture arm64e it("keeps unknown symbolic CPU values as source text without a code", () => { expect( - parseLipoArchitectures( + parseLipoOutput( "architecture future\n cputype CPU_TYPE_FUTURE\n cpusubtype CPU_SUBTYPE_FUTURE\n offset 4096\n size 64\n align 2^12\n", - ), + ).architectures, ).toMatchObject([ { cpu_type: "CPU_TYPE_FUTURE", @@ -70,18 +89,18 @@ architecture arm64e ["0x0", 3], ])("applies printed x86_64 capabilities %s", (capabilities, code) => { expect( - parseLipoArchitectures( + parseLipoOutput( `architecture x86_64\n cpusubtype CPU_SUBTYPE_X86_64_ALL\n capabilities ${capabilities}\n`, - )[0]?.cpu_subtype_code, + ).architectures[0]?.cpu_subtype_code, ).toBe(code); }); it("leaves the subtype code unknown when lipo prints no capabilities", () => { // macOS 26 lipo prints this blank line for LIB64 and plain x86_64 slices. expect( - parseLipoArchitectures( + parseLipoOutput( "architecture x86_64\n cputype CPU_TYPE_X86_64\n cpusubtype CPU_SUBTYPE_X86_ALL\n capabilities \n offset 16384\n", - ), + ).architectures, ).toMatchObject([ { cpu_subtype: "CPU_SUBTYPE_X86_ALL", @@ -99,9 +118,9 @@ architecture arm64e "retains an arm64e base subtype with capabilities '$capabilities'", ({ capabilities, code }) => { expect( - parseLipoArchitectures( + parseLipoOutput( `architecture arm64e.v1\n cputype CPU_TYPE_ARM64\n cpusubtype CPU_SUBTYPE_ARM64E\n capabilities ${capabilities}\n`, - ), + ).architectures, ).toMatchObject([ { cpu_type_code: 0x0100000c, @@ -112,3 +131,79 @@ architecture arm64e }, ); }); + +describe("lipo FAT operand framing", () => { + it.each([ + "/fixture/\narchitecture forged\n offset 123\n size 64", + "/fixture/\nfat_magic 0xcafebabe\nnfat_arch 1\narchitecture forged", + "/fixture/\nNon-fat file: fake is architecture: forged", + "/fixture/first\nsecond\nthird", + "/fixture/first\r\narchitecture forged\r\n offset 123", + "/fixture/[app](arm64).*+$^?\\name", + ])( + "keeps the complete echoed operand out of FAT slice records: %j", + (path) => { + const output = `Fat header in: ${path}\n${fatRecords}`; + + expect(parseLipoOutput(output, path)).toEqual( + parseLipoOutput(fatRecords), + ); + expect( + parseLipoOutput(output, path).architectures.map(({ name }) => name), + ).toEqual(["arm64", "x86_64"]); + }, + ); + + it("keeps a CRLF pathname separate from a CRLF record terminator", () => { + const path = "/fixture/first\r\narchitecture forged\r\n size 64"; + const output = `Fat header in: ${path}\r\n${fatRecords.replaceAll("\n", "\r\n")}`; + + expect(parseLipoOutput(output, path)).toEqual(parseLipoOutput(fatRecords)); + }); + + it("preserves ordinary FAT metadata with and without an operand", () => { + const path = "/fixture/app"; + const output = `Fat header in: ${path}\n${fatRecords}`; + + expect(parseLipoOutput(output, path)).toEqual(parseLipoOutput(output)); + }); + + it("does not require a FAT path header in captured fixtures", () => { + expect(parseLipoOutput(fatRecords, "/fixture/app")).toEqual( + parseLipoOutput(fatRecords), + ); + }); + + it("does not strip a header for a different operand that shares a prefix", () => { + const path = "/fixture/app-extra\narchitecture forged"; + const output = `Fat header in: ${path}\n${fatRecords}`; + + expect(parseLipoOutput(output, "/fixture/app")).toEqual( + parseLipoOutput(output), + ); + }); + + it("preserves thin-file parsing when the operand contains FAT-like lines", () => { + const path = "/fixture/\nFat header in: fake\narchitecture forged"; + const output = `Non-fat file: ${path} is architecture: arm64\n`; + + expect(parseLipoOutput(output, path)).toEqual({ + format: "thin", + architectures: [ + { + name: "arm64", + cpu_type: null, + cpu_subtype: null, + cpu_type_code: null, + cpu_subtype_code: null, + cpu_subtype_base: null, + capabilities: null, + file_offset: null, + size: null, + alignment: null, + }, + ], + }); + expect(parseLipoOutput(output, path)).toEqual(parseLipoOutput(output)); + }); +}); diff --git a/src/native/parsers/lipo.ts b/src/native/parsers/lipo.ts index cc65cca41..7f74a51b9 100644 --- a/src/native/parsers/lipo.ts +++ b/src/native/parsers/lipo.ts @@ -16,8 +16,26 @@ const architectureSchema = z.object({ export type LipoArchitecture = z.infer; -/** Parse `lipo -detailed_info` into deterministic slice metadata. */ -export const parseLipoArchitectures = (output: string): LipoArchitecture[] => { +interface LipoOutput { + readonly format: "thin" | "fat"; + readonly architectures: LipoArchitecture[]; +} + +/** Parse `lipo -detailed_info` framing and deterministic slice metadata. */ +export const parseLipoOutput = ( + output: string, + operandPath?: string, +): LipoOutput => { + // The echoed FAT operand is one value, even when its pathname contains lines. + if (operandPath !== undefined) { + const header = `Fat header in: ${operandPath}`; + if (output.startsWith(header)) { + if (output.startsWith("\r\n", header.length)) + output = output.slice(header.length + 2); + else if (output.startsWith("\n", header.length)) + output = output.slice(header.length + 1); + } + } // lipo echoes a thin file's pathname verbatim, including embedded newlines. // Match its final architecture field before splitting records into lines. const thinArchitecture = @@ -25,20 +43,23 @@ export const parseLipoArchitectures = (output: string): LipoArchitecture[] => { output, )?.[1]; if (thinArchitecture !== undefined) - return [ - architectureSchema.parse({ - name: thinArchitecture, - cpu_type: null, - cpu_subtype: null, - cpu_type_code: null, - cpu_subtype_code: null, - cpu_subtype_base: null, - capabilities: null, - file_offset: null, - size: null, - alignment: null, - }), - ]; + return { + format: "thin", + architectures: [ + architectureSchema.parse({ + name: thinArchitecture, + cpu_type: null, + cpu_subtype: null, + cpu_type_code: null, + cpu_subtype_code: null, + cpu_subtype_base: null, + capabilities: null, + file_offset: null, + size: null, + alignment: null, + }), + ], + }; const architectures: LipoArchitecture[] = []; let current: Record | undefined; const flush = (): void => { @@ -90,7 +111,7 @@ export const parseLipoArchitectures = (output: string): LipoArchitecture[] => { flush(); if (architectures.length === 0) throw new TypeError("lipo output contained no architectures"); - return architectures; + return { format: "fat", architectures }; }; const integer = (value: string | undefined): number | null => { diff --git a/tests/boundary/cli/nativeMachoEchoedPathCli.test.ts b/tests/boundary/cli/nativeMachoEchoedPathCli.test.ts index 0c362d263..a3118c1f0 100644 --- a/tests/boundary/cli/nativeMachoEchoedPathCli.test.ts +++ b/tests/boundary/cli/nativeMachoEchoedPathCli.test.ts @@ -1,9 +1,13 @@ import { copyFile } from "node:fs/promises"; -import { join } from "node:path"; +import { join, resolve } from "node:path"; import { describe, expect } from "vitest"; import { z } from "zod"; +import { + inspectMachoSchema, + listArchitecturesSchema, +} from "../../../src/domain/native/nativeInspection.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { cliTest } from "../../support/cli/cliFixture.js"; @@ -34,27 +38,68 @@ const resultSchema = z.object({ describe.skipIf(process.platform !== "darwin")( "native Mach-O CLI with echoed path text", () => { - cliTest( - "does not read load commands or symbols from the file name", - async ({ cli }) => { - const directory = await createTestTempDirectory("rea-macho-echo-"); - const inspect = async (name: string) => { - const path = join(directory, name); - await copyFile("/usr/bin/true", path); - const result = await cli.run({ - arguments: ["inspect-macho", path, "--json"], - environment, - }); - expect(result.exitCode).toBe(0); - const { provenance: _provenance, ...facts } = resultSchema.parse( - result.json, - ).normalized_result; - return facts; - }; - const forged = await inspect(FORGED_NAME); - expect(forged).toEqual(await inspect("tool")); - expect(JSON.stringify(forged)).not.toMatch(/forged|4242/u); - }, - ); + const lipoName = "tool\narchitecture forged\n offset 123\n size 32"; + for (const [facts, command, filename] of [ + ["load commands or symbols", "inspect-macho", FORGED_NAME], + ["FAT architecture records", "inspect-macho", lipoName], + ["FAT architecture inventory", "list-architectures", lipoName], + ["thin-file framing", "inspect-macho", "tool\nNon-fat file: forged"], + ] as const) { + cliTest( + `does not read ${facts} from the file name`, + async ({ cli, processes }) => { + const directory = await createTestTempDirectory( + "rea-native-cli-echo-", + ); + const fixture = join(directory, "fat-fixture"); + const compiled = await processes.run( + "/usr/bin/xcrun", + [ + "clang", + "-arch", + "arm64", + "-arch", + "x86_64", + resolve("tests/conformance/c/fixture.c"), + "-o", + fixture, + ], + { timeoutMs: 30_000 }, + ); + expect(compiled.exitCode, compiled.stderr).toBe(0); + const inspect = async (name: string) => { + const path = join(directory, name); + await copyFile(fixture, path); + const result = await cli.run({ + arguments: [command, path, "--json"], + environment, + }); + expect(result.exitCode).toBe(0); + const { provenance: _provenance, ...facts } = resultSchema.parse( + result.json, + ).normalized_result; + return facts; + }; + const ordinary = await inspect("tool"); + // Prove this comparison exercised FAT framing, independent of host binaries. + const architectures = + listArchitecturesSchema.shape.architectures.parse( + ordinary.architectures, + ); + expect(architectures.total).toBeGreaterThan(1); + if (command === "inspect-macho") { + const selected = + inspectMachoSchema.shape.selected_architecture.parse( + ordinary.selected_architecture, + ); + expect(selected.container_file_offset).toBeGreaterThan(0); + } + const forged = await inspect(filename); + expect(forged).toEqual(ordinary); + expect(JSON.stringify(forged)).not.toMatch(/forged|4242/u); + }, + 60_000, + ); + } }, ); diff --git a/tests/boundary/providers/native/machoSliceReader.test.ts b/tests/boundary/providers/native/machoSliceReader.test.ts index 5891eaf53..b2f1867a1 100644 --- a/tests/boundary/providers/native/machoSliceReader.test.ts +++ b/tests/boundary/providers/native/machoSliceReader.test.ts @@ -1,10 +1,12 @@ -import { readFile } from "node:fs/promises"; +import { readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; import { describe, expect, it, onTestFinished } from "vitest"; import { MachOSliceArtifactReader } from "../../../../src/artifacts/MachOSliceArtifactReader.js"; import { XcrunCommandRunner } from "../../../../src/native/CommandRunner.js"; import { DarwinProcessOwnershipInspectionError } from "../../../../src/process/DarwinProcessRunTokenReader.js"; +import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js"; /** * Read a 32-bit FAT table directly. The host's /bin/ls changes between macOS @@ -42,6 +44,56 @@ describe("Mach-O slice reader with system lipo", () => { }); }); + it.skipIf(process.platform !== "darwin")( + "keeps FAT slice records separate from the echoed source pathname", + async () => { + const directory = await createTestTempDirectory("rea-lipo-fat-path-"); + const path = join( + directory, + "source\narchitecture forged\n offset 123\n size 32", + ); + const bytes = Buffer.alloc(8224); + bytes.writeUInt32BE(0xca_fe_ba_be, 0); + bytes.writeUInt32BE(2, 4); + const slices = [ + { cpu: 0x0100_000c, subtype: 0, offset: 4096 }, + { cpu: 0x0100_0007, subtype: 3, offset: 8192 }, + ]; + slices.forEach(({ cpu, subtype, offset }, index) => { + const entry = 8 + index * 20; + bytes.writeUInt32BE(cpu, entry); + bytes.writeUInt32BE(subtype, entry + 4); + bytes.writeUInt32BE(offset, entry + 8); + bytes.writeUInt32BE(32, entry + 12); + bytes.writeUInt32BE(12, entry + 16); + bytes.writeUInt32LE(0xfeed_facf, offset); + bytes.writeUInt32LE(cpu, offset + 4); + bytes.writeUInt32LE(subtype, offset + 8); + bytes.writeUInt32LE(1, offset + 12); + }); + await writeFile(path, bytes); + const reader = new MachOSliceArtifactReader(path, {}); + try { + const entries = []; + for await (const entry of reader.entries()) entries.push(entry); + expect( + entries.map(({ byteOffset, declaredSize }) => [ + byteOffset, + declaredSize, + ]), + ).toEqual([ + [4096, 32], + [8192, 32], + ]); + expect(entries.every(({ path }) => !path.includes("forged"))).toBe( + true, + ); + } finally { + await reader.close(); + } + }, + ); + it.skipIf(process.platform !== "darwin")( "checks real /bin/ls slice ranges against its FAT table", async () => { diff --git a/tests/boundary/providers/native/nativeProvider.test.ts b/tests/boundary/providers/native/nativeProvider.test.ts index 0d1860b16..81766e22d 100644 --- a/tests/boundary/providers/native/nativeProvider.test.ts +++ b/tests/boundary/providers/native/nativeProvider.test.ts @@ -17,7 +17,7 @@ import { type NativeCommandRunner, } from "../../../../src/native/CommandRunner.js"; import { err, ok } from "../../../../src/domain/result.js"; -import { parseLipoArchitectures } from "../../../../src/native/parsers/lipo.js"; +import { parseLipoOutput } from "../../../../src/native/parsers/lipo.js"; import { parseCodeSignature } from "../../../../src/native/parsers/codesign.js"; import { prepareProcessOwnershipInspection } from "../../../../src/process/ProcessOwnershipObservation.js"; import { projectAnalysisError } from "../../../../src/domain/analysisErrorProjection.js"; @@ -960,15 +960,17 @@ describe("native macOS provider failures and parsing", () => { }); it("parses thin and universal lipo fixtures", async () => { - expect( - parseLipoArchitectures(await fixture("lipo-thin.txt")), - ).toMatchObject([{ name: "arm64" }]); - expect(parseLipoArchitectures(await fixture("lipo-fat.txt"))).toMatchObject( - [ + expect(parseLipoOutput(await fixture("lipo-thin.txt"))).toMatchObject({ + format: "thin", + architectures: [{ name: "arm64" }], + }); + expect(parseLipoOutput(await fixture("lipo-fat.txt"))).toMatchObject({ + format: "fat", + architectures: [ { name: "x86_64", file_offset: 16384, size: 4096 }, { name: "arm64", file_offset: 32768, size: 8192 }, ], - ); + }); }); it("rejects trailing text in lipo numeric fields instead of truncating", () => { @@ -980,7 +982,7 @@ describe("native macOS provider failures and parsing", () => { " size 0x10", " align 2^14 (16384)", ].join("\n"); - expect(parseLipoArchitectures(output)).toMatchObject([ + expect(parseLipoOutput(output).architectures).toMatchObject([ { name: "arm64", file_offset: null, size: null, alignment: 16384 }, ]); }); @@ -993,10 +995,11 @@ describe("native macOS provider failures and parsing", () => { " align 2^14 (16384)", ].join("\n"); expect( - parseLipoArchitectures(output.replace("(16384)", "(8192)"))[0]?.alignment, + parseLipoOutput(output.replace("(16384)", "(8192)")).architectures[0] + ?.alignment, ).toBeNull(); expect( - parseLipoArchitectures(output.replace("2^14 (16384)", "2^14"))[0] + parseLipoOutput(output.replace("2^14 (16384)", "2^14")).architectures[0] ?.alignment, ).toBe(16384); }); diff --git a/tests/boundary/providers/native/nativeSelectedArchitecture.test.ts b/tests/boundary/providers/native/nativeSelectedArchitecture.test.ts index a651e22f6..182c33612 100644 --- a/tests/boundary/providers/native/nativeSelectedArchitecture.test.ts +++ b/tests/boundary/providers/native/nativeSelectedArchitecture.test.ts @@ -157,6 +157,50 @@ it.each([ }, ); +it.each([ + ["ordinary", "/owned/universal", "\n"], + ["LF Non-fat text", "/owned/\nNon-fat file: forged", "\n"], + ["CRLF Non-fat text", "/owned/\r\nNon-fat file: forged", "\r\n"], +])( + "projects universal segment evidence into the selected slice's container offsets with %s path framing", + async (_description, path, lineEnding) => { + const records = (await nativeFixture("lipo-fat.txt")).replace( + /^Fat header in:[^\r\n]*\r?\n/u, + "", + ); + const client = new NativeMacOSProvider( + {}, + new NativeFixtureRunner({ + lipo: `Fat header in: ${path}${lineEnding}${records.replaceAll("\n", lineEnding)}`, + }), + "darwin", + ).createClient(nativeMachoTarget(path)); + const result = await client.execute("inspect_macho", {}); + if (!result.ok) throw result.error; + expect(result.value.result).toMatchObject({ + selected_architecture: { container_file_offset: 32768 }, + }); + expect(result.value.locations).toContainEqual({ + kind: "file-offset-range", + start: 32768, + end: 40960, + }); + expect(result.value.locations).toContainEqual({ + kind: "file-offset", + offset: 36864, + }); + expect(result.value.locations).not.toContainEqual({ + kind: "file-offset-range", + start: 0, + end: 8192, + }); + // Normalized tool fields keep their slice-relative meaning. + const normalized = inspectMachoSchema.parse(result.value.result); + expect(normalized.segments.items[0]?.file_offset).toBe(0); + expect(normalized.entrypoints.items[0]?.file_offset).toBe(4096); + }, +); + it.each([ { capture: "calculator.txt", selected: "arm64e" }, { capture: "shortcut-droplet.txt", selected: "arm64" }, @@ -187,34 +231,6 @@ it.each([ }, ); -it("projects universal segment evidence into the selected slice's container offsets", async () => { - const client = new NativeMacOSProvider( - {}, - new NativeFixtureRunner(), - "darwin", - ).createClient(nativeMachoTarget("/owned/universal")); - const result = await client.execute("inspect_macho", {}); - if (!result.ok) throw result.error; - expect(result.value.locations).toContainEqual({ - kind: "file-offset-range", - start: 32768, - end: 40960, - }); - expect(result.value.locations).toContainEqual({ - kind: "file-offset", - offset: 36864, - }); - expect(result.value.locations).not.toContainEqual({ - kind: "file-offset-range", - start: 0, - end: 8192, - }); - // Normalized tool fields keep their slice-relative meaning. - const normalized = inspectMachoSchema.parse(result.value.result); - expect(normalized.segments.items[0]?.file_offset).toBe(0); - expect(normalized.entrypoints.items[0]?.file_offset).toBe(4096); -}); - it.each([ { description: "thin", diff --git a/tests/boundary/providers/native/nativeTextRobustness.test.ts b/tests/boundary/providers/native/nativeTextRobustness.test.ts index 256aa0a98..ce82bd520 100644 --- a/tests/boundary/providers/native/nativeTextRobustness.test.ts +++ b/tests/boundary/providers/native/nativeTextRobustness.test.ts @@ -2,7 +2,7 @@ import { expect, it } from "vitest"; import { parseOtoolLoadCommands } from "../../../../src/native/parsers/otool.js"; import { parsePlistJson } from "../../../../src/native/parsers/plist.js"; -import { parseLipoArchitectures } from "../../../../src/native/parsers/lipo.js"; +import { parseLipoOutput } from "../../../../src/native/parsers/lipo.js"; import { nativeFixture } from "../../../fixtures/nativeCommands.js"; const toCrlf = (value: string): string => value.replaceAll("\n", "\r\n"); @@ -15,7 +15,10 @@ it.each([ "retains the reported thin architecture when lipo echoes filename %j", (filename) => { const output = `input file /owned/${filename} is not a fat file\nNon-fat file: /owned/${filename} is architecture: arm64\n`; - expect(parseLipoArchitectures(output)).toMatchObject([{ name: "arm64" }]); + expect(parseLipoOutput(output)).toMatchObject({ + format: "thin", + architectures: [{ name: "arm64" }], + }); }, ); @@ -38,10 +41,13 @@ it("does not mistake fat slice output for a thin Non-fat file record", () => { " align 2^12 (4096)", "", ].join("\n"); - expect(parseLipoArchitectures(output)).toMatchObject([ - { name: "arm64", file_offset: 16384 }, - { name: "x86_64", file_offset: 16793600 }, - ]); + expect(parseLipoOutput(output)).toMatchObject({ + format: "fat", + architectures: [ + { name: "arm64", file_offset: 16384 }, + { name: "x86_64", file_offset: 16793600 }, + ], + }); }); it("keeps section blocks separate under CRLF captures", async () => { From c44d17fd3c4c71b86fcae401d9ff1088b4e27170 Mon Sep 17 00:00:00 2001 From: The one -Niu <912906590@qq.com> Date: Tue, 13 Oct 2026 21:18:12 +0800 Subject: [PATCH 11/24] fix(pwntools): let callers widen the artifact input bound (#2486) * fix(pwntools): let callers widen the artifact input bound The pwntools adapters rejected every artifact over 32 MiB before any analysis ran, and nothing in configuration, environment, CLI or MCP could raise that bound, so real recorded core dumps stayed unanalyzable. Resolve the bound from REA_PWNTOOLS_MAX_INPUT_BYTES in front of the shipped default. An absent value keeps 32 MiB, and a value outside 1..1073741824 bytes is refused as unusable provider configuration before the artifact is read instead of falling back silently. * fix(pwntools): name the bound variable in its own refusal An unusable REA_PWNTOOLS_MAX_INPUT_BYTES travelled through the shared engine-absence helper, which always stamped configuration_key REA_PWNTOOLS_PYTHON, so a caller following that field was pointed at the Python executable instead of the bound it had just set. The guide also described the refusal as an invalid request while the decoder reports provider_unavailable. Let the failure context carry the key under review and pass the bound's own name at that call site, then state the code callers actually receive in the guide. Assert the key on the projected rejection so the engine default and the bound override each stay attributed to the right setting. --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- docs/binary-diagnostics.md | 6 +- docs/recorded-crashes.md | 10 +- src/native/pwntools/PwntoolsDecoder.ts | 15 ++- src/native/pwntools/PwntoolsFailures.ts | 3 +- .../pwntools/PwntoolsInputLimit.test.ts | 96 +++++++++++++++++++ src/native/pwntools/PwntoolsInputLimit.ts | 40 ++++++++ .../pwntoolsInputBoundAdmission.test.ts | 89 +++++++++++++++++ 7 files changed, 252 insertions(+), 7 deletions(-) create mode 100644 src/native/pwntools/PwntoolsInputLimit.test.ts create mode 100644 src/native/pwntools/PwntoolsInputLimit.ts create mode 100644 tests/boundary/binaryDiagnostics/pwntoolsInputBoundAdmission.test.ts diff --git a/docs/binary-diagnostics.md b/docs/binary-diagnostics.md index 29e10a33c..0e6a3f196 100644 --- a/docs/binary-diagnostics.md +++ b/docs/binary-diagnostics.md @@ -98,8 +98,10 @@ section, symbol, mitigation or linkage views. The default `complete` detail and CLI output are unchanged. Complete results have a 32 MiB input, 64 MiB reply, 1 MiB combined diagnostics -and 30-second owned command deadline. The reply budget applies to the decoder -record; the CLI/MCP Evidence envelope adds copies and encoding overhead. The +and 30-second owned command deadline. `REA_PWNTOOLS_MAX_INPUT_BYTES` raises the +input bound alone, up to 1 GiB; the reply and deadline bounds are unchanged. The +reply budget applies to the decoder record; the CLI/MCP Evidence envelope adds +copies and encoding overhead. The pinned MCP SDK defaults to a 10 MiB receive buffer. Large complete results need a caller-configured `StdioClientTransport({ maxBufferSize: ... })` and a matching server `REA_MCP_MAX_RESPONSE_BYTES`. See [MCP delivery budgets](mcp-contracts.md#tool-results). diff --git a/docs/recorded-crashes.md b/docs/recorded-crashes.md index d68db74fd..8b1a51662 100644 --- a/docs/recorded-crashes.md +++ b/docs/recorded-crashes.md @@ -72,9 +72,13 @@ debugger context is `observed`. Initial Linux x64 limits: 32 MiB input, 64 MiB decoder reply, 1 MiB diagnostics per stage, 30 seconds owned wall deadline per stage, 3 GiB address-space soft ceiling -and 30 seconds CPU soft ceiling. Tighter inherited soft/hard limits are retained; -hard limits are not raised. Complete oversized results fail explicitly. These -process limits do not constitute a sandbox for caller-supplied engine code. +and 30 seconds CPU soft ceiling. `REA_PWNTOOLS_MAX_INPUT_BYTES` raises the input +bound alone, up to 1 GiB, and returns `provider_unavailable` when it is not a +decimal integer in range; the reply, diagnostic and deadline bounds stay fixed, so a +large core that outruns them still fails explicitly instead of partially. Tighter +inherited soft/hard limits are retained; hard limits are not raised. Complete +oversized results fail explicitly. These process limits do not constitute a sandbox +for caller-supplied engine code. Resource exit statuses need matching private failure-branch markers. Unmarked launcher failures retain their actual status/output and unknown resource cause. diff --git a/src/native/pwntools/PwntoolsDecoder.ts b/src/native/pwntools/PwntoolsDecoder.ts index 6aa456631..6aa9a3a96 100644 --- a/src/native/pwntools/PwntoolsDecoder.ts +++ b/src/native/pwntools/PwntoolsDecoder.ts @@ -51,6 +51,7 @@ import { PWNTOOLS_PROVIDER_IDENTITY, PWNTOOLS_LIMITS, } from "./PwntoolsRelease.js"; +import { pwntoolsInputBytes } from "./PwntoolsInputLimit.js"; const HOST_REQUIREMENT = "The initial real-verified pwntools artifact profile supports Linux x64 only."; @@ -245,11 +246,23 @@ export class PwntoolsDecoder< undefined, context, ); + const inputLimit = pwntoolsInputBytes(this.environment); + if (!inputLimit.configured) + throw pwntoolsUnavailable( + inputLimit.error, + selectedPath, + undefined, + undefined, + { + ...context, + configurationKey: "REA_PWNTOOLS_MAX_INPUT_BYTES", + }, + ); phase = "artifact-read"; selectedPath = input.path; const snapshot = await readStableArtifact( input.path, - PWNTOOLS_LIMITS.inputBytes, + inputLimit.bytes, options?.signal, undefined, this.#resources, diff --git a/src/native/pwntools/PwntoolsFailures.ts b/src/native/pwntools/PwntoolsFailures.ts index f5e1b2654..e36ae9503 100644 --- a/src/native/pwntools/PwntoolsFailures.ts +++ b/src/native/pwntools/PwntoolsFailures.ts @@ -30,6 +30,7 @@ export interface PwntoolsFailureContext { readonly operation: string; readonly providerId: string; readonly precedingOutput?: AnalysisCapturedOutput; + readonly configurationKey?: string; } const DEFAULT_CONTEXT = { operation: "inspect_binary_layout", @@ -274,7 +275,7 @@ export const pwntoolsUnavailable = ( code: "provider_unavailable", reason, diagnostics: { - configuration_key: "REA_PWNTOOLS_PYTHON", + configuration_key: context.configurationKey ?? "REA_PWNTOOLS_PYTHON", executable_path: path, ...(systemCode === undefined ? {} : { system_code: systemCode }), }, diff --git a/src/native/pwntools/PwntoolsInputLimit.test.ts b/src/native/pwntools/PwntoolsInputLimit.test.ts new file mode 100644 index 000000000..d60c27b80 --- /dev/null +++ b/src/native/pwntools/PwntoolsInputLimit.test.ts @@ -0,0 +1,96 @@ +import { describe, expect, it } from "vitest"; + +import { pwntoolsInputBytes } from "./PwntoolsInputLimit.js"; +import { pwntoolsUnavailable } from "./PwntoolsFailures.js"; +import { + PWNTOOLS_LIMITS, + PWNTOOLS_PROVIDER_IDENTITY, +} from "./PwntoolsRelease.js"; + +const CONSTRAINT = + "REA_PWNTOOLS_MAX_INPUT_BYTES must be a decimal integer from 1 to " + + "1073741824 bytes, and REA never installs Python or packages."; + +describe("pwntools input bound", () => { + it("keeps the shipped default when the override is absent", () => { + expect(pwntoolsInputBytes({})).toEqual({ + configured: true, + bytes: PWNTOOLS_LIMITS.inputBytes, + }); + expect(PWNTOOLS_LIMITS.inputBytes).toBe(32 * 1024 * 1024); + }); + + it.each([ + ["1", 1], + ["67108864", 64 * 1024 * 1024], + ["172879872", 172_879_872], + ["1073741824", 1024 * 1024 * 1024], + ] as const)("accepts %s as %s bytes", (value, expected) => { + expect(pwntoolsInputBytes({ REA_PWNTOOLS_MAX_INPUT_BYTES: value })).toEqual( + { configured: true, bytes: expected }, + ); + }); + + it.each([ + "", + "0", + "-1", + "33554432.0", + "1e8", + " 67108864", + "67_108_864", + "0x2000000", + "1073741825", + "99999999999999999999", + "not-a-number", + ])("rejects %s instead of widening the bound silently", (value) => { + expect(pwntoolsInputBytes({ REA_PWNTOOLS_MAX_INPUT_BYTES: value })).toEqual( + { + configured: false, + error: CONSTRAINT, + }, + ); + }); + + it("rejects a value above the ceiling that a float would round", () => { + const ceiling = pwntoolsInputBytes({ + REA_PWNTOOLS_MAX_INPUT_BYTES: "1073741824", + }); + const above = pwntoolsInputBytes({ + REA_PWNTOOLS_MAX_INPUT_BYTES: "10737418241", + }); + expect(ceiling.configured).toBe(true); + expect(above).toEqual({ configured: false, error: CONSTRAINT }); + }); + + it("never narrows the bound below the shipped default implicitly", () => { + const accepted = [1, 1024, PWNTOOLS_LIMITS.inputBytes].map((bytes) => + pwntoolsInputBytes({ REA_PWNTOOLS_MAX_INPUT_BYTES: String(bytes) }), + ); + expect(accepted.every((limit) => limit.configured)).toBe(true); + }); +}); + +describe("pwntools unavailability configuration key", () => { + const context = { + operation: "inspect_binary_layout", + providerId: PWNTOOLS_PROVIDER_IDENTITY.id, + }; + const reportedKey = (configurationKey?: string) => + pwntoolsUnavailable( + CONSTRAINT, + "/caller-selected/python", + undefined, + undefined, + configurationKey === undefined + ? context + : { ...context, configurationKey }, + ).rejections[0]?.diagnostics["configuration_key"]; + + it("keeps the engine key unless the failure is about the bound", () => { + expect(reportedKey()).toBe("REA_PWNTOOLS_PYTHON"); + expect(reportedKey("REA_PWNTOOLS_MAX_INPUT_BYTES")).toBe( + "REA_PWNTOOLS_MAX_INPUT_BYTES", + ); + }); +}); diff --git a/src/native/pwntools/PwntoolsInputLimit.ts b/src/native/pwntools/PwntoolsInputLimit.ts new file mode 100644 index 000000000..e6867e80f --- /dev/null +++ b/src/native/pwntools/PwntoolsInputLimit.ts @@ -0,0 +1,40 @@ +import { z } from "zod"; + +import { PWNTOOLS_LIMITS } from "./PwntoolsRelease.js"; + +// The artifact is read into one in-memory buffer and staged again as a private +// snapshot before the owned bridge parses it under a 3 GiB address-space +// reservation, so an override stays within a third of that reservation. +const MAXIMUM_INPUT_BYTES = 1024 * 1024 * 1024; + +/** Optional caller-supplied input bound, in bytes, without a decimal point. */ +export const pwntoolsInputBytesSchema = z + .string() + .regex(/^[1-9]\d*$/u, "must be a positive decimal integer") + .transform(Number) + .pipe(z.number().int().safe().max(MAXIMUM_INPUT_BYTES)); + +const CONSTRAINT = + `REA_PWNTOOLS_MAX_INPUT_BYTES must be a decimal integer from 1 to ` + + `${String(MAXIMUM_INPUT_BYTES)} bytes, and REA never installs Python or packages.`; + +/** Resolved input bound, or the reason the caller's override cannot be used. */ +export type PwntoolsInputBytes = + | { readonly configured: true; readonly bytes: number } + | { readonly configured: false; readonly error: string }; + +/** + * Resolve the admitted artifact size for the pwntools adapters. An unset value + * keeps the shipped default, so the bound is only ever widened deliberately. + */ +export const pwntoolsInputBytes = ( + environment: Readonly, +): PwntoolsInputBytes => { + const value = environment.REA_PWNTOOLS_MAX_INPUT_BYTES; + if (value === undefined) + return { configured: true, bytes: PWNTOOLS_LIMITS.inputBytes }; + const parsed = pwntoolsInputBytesSchema.safeParse(value); + return parsed.success + ? { configured: true, bytes: parsed.data } + : { configured: false, error: CONSTRAINT }; +}; diff --git a/tests/boundary/binaryDiagnostics/pwntoolsInputBoundAdmission.test.ts b/tests/boundary/binaryDiagnostics/pwntoolsInputBoundAdmission.test.ts new file mode 100644 index 000000000..ca9f0bc2a --- /dev/null +++ b/tests/boundary/binaryDiagnostics/pwntoolsInputBoundAdmission.test.ts @@ -0,0 +1,89 @@ +import { open } from "node:fs/promises"; +import { join } from "node:path"; + +import { expect, it, onTestFinished } from "vitest"; + +import { PwntoolsLayoutProvider } from "../../../src/native/pwntools/PwntoolsLayoutProvider.js"; +import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; +import { spawnOwnedProviderProcess } from "../../../src/process/ProviderProcess.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +const unsupportedHost = process.platform !== "linux" || process.arch !== "x64"; +const DEFAULT_INPUT_BYTES = 33_554_432; +// One byte above the shipped bound, so the admission decision is the only +// difference between the first two cases. +const ARTIFACT_BYTES = DEFAULT_INPUT_BYTES + 1; + +/** Sparse regular file just larger than the shipped input bound. */ +const oversizedArtifact = async (): Promise => { + const root = await createTestTempDirectory("rea-pwntools-input-bound-"); + const path = join(root, "recorded-core"); + const handle = await open(path, "w"); + await handle.truncate(ARTIFACT_BYTES); + await handle.close(); + return path; +}; + +/** One admission attempt, reported as the projected caller-visible failure. */ +const inspect = async (override?: string) => { + const path = await oversizedArtifact(); + const provider = new PwntoolsLayoutProvider( + { + REA_PWNTOOLS_PYTHON: process.execPath, + ...(override === undefined + ? {} + : { REA_PWNTOOLS_MAX_INPUT_BYTES: override }), + }, + async (spawn) => + spawnOwnedProviderProcess({ + ...spawn, + command: process.execPath, + arguments: ["-e", "process.stdout.write('no reply')"], + }), + ); + onTestFinished(async () => { + await provider.close(); + }); + const result = await provider.inspect({ path }); + if (result.ok) throw new Error("Expected the bounded decoder to fail"); + const projected = projectAnalysisError(result.error); + return { projected, text: JSON.stringify(projected) }; +}; + +it.runIf(!unsupportedHost)( + "refuses an artifact above the shipped bound when no override is set", + async () => { + const { projected, text } = await inspect(); + expect(projected.code).toBe("invalid_request"); + expect(text).toContain( + `Selected artifact exceeds ${String(DEFAULT_INPUT_BYTES)} bytes`, + ); + }, +); + +it.runIf(!unsupportedHost)( + "admits the same artifact once the override widens the bound", + async () => { + const { projected, text } = await inspect(String(ARTIFACT_BYTES)); + // Admission passed, so the only remaining failure is that the owned decoder + // never wrote a reply, which also proves the process really ran. + expect(text).not.toContain("Selected artifact exceeds"); + expect(projected.code).toBe("unreadable_output"); + expect(text).toContain("no reply"); + }, +); + +it.runIf(!unsupportedHost)( + "refuses an unusable override before reading the artifact", + async () => { + const { projected, text } = await inspect("0"); + expect(projected.code).toBe("provider_unavailable"); + expect(projected.details).toMatchObject({ + rejections: [ + { diagnostics: { configuration_key: "REA_PWNTOOLS_MAX_INPUT_BYTES" } }, + ], + }); + expect(text).toContain("REA_PWNTOOLS_MAX_INPUT_BYTES must be a decimal"); + expect(text).not.toContain("Selected artifact exceeds"); + }, +); From cae7620b77ec2893c944197603cc9158e2224930 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=90=B4=E4=BD=B3=E6=B5=A9Alben?= <35291372+XiaoBinGan@users.noreply.github.com> Date: Tue, 13 Oct 2026 21:19:17 +0800 Subject: [PATCH 12/24] fix(apple): report a recognized connector that cannot name its owner (#2484) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(apple): report a connector that cannot name its owner as an omission A connector is recognized by class name and then dropped by an early `continue` when its owner cannot be resolved. Nothing recorded the drop, so connection coverage derived `complete` with `examined: 0` and `omitted: 0` for a connector that was present, recognized and discarded. Both producers of that coverage contract are affected and are changed together, because they share the facet and the reason value: - the keyed-archive parser records the drop as `unresolvedConnections` and reports it through the `connections:` facet, the document `connection_count` and the graph omission total; - the NIBArchive projection in the artifact adapter recorded nothing at all, and the projected dictionary carried no trace of the dropped connector for a downstream consumer to find. Its count is taken where the connector is discarded and plumbed through the same per-document omission channel that already carries incomplete hierarchies, so the reason follows the cause. An unresolved source cannot be represented as a connection at all, because connection identity requires a non-empty source; that is unlike an unresolved destination, which the graph already retains as an unresolved edge. `connection_limit_reached` stays reserved for connections dropped by the hard cap and `serialized_view_hierarchy_incomplete` for hierarchy omissions. `connection_count` also includes the unresolved connectors. That field already counts connectors the parser recognized but could not emit, because connections dropped by the hard cap are counted there too, so unresolved connectors follow the existing rule rather than a new one. Verified with 98 focused Interface Builder and catalog tests, check:fast and verify:test-discovery on macOS arm64, and through the public CLI: a rooted archive holding one ownerless UIRuntimeOutletConnection reported complete with examined 0 and omitted 0 before this change and reports partial with the reason connection_unresolved and omitted 1 after it. * fix(apple): count ownerless NIBArchive connectors and keep the limit reason Greptile found two defects in the NIBArchive half of this change. The document summary was left unchanged, so the same recognized connector counted as one in a keyed archive and zero in a NIBArchive. The dictionary parser only sees connectors that were projected, so the omission recorded where the connector was dropped is added to the matching document here. The facet reason was overwritten unconditionally, so a NIBArchive holding both a connection dropped by the limit and an ownerless connector reported connection_unresolved and lost the cause a higher limit could recover. The existing reason is kept when the graph already named one, which is the precedence the keyed-archive producer applies. Verified with `npm run test:focused` over the bundle suite (42 passed, 1 skipped), `npm run lint` (0 errors) and `npm run check:fast` on macOS arm64. Reverting the reason precedence alone fails only the limit case; reverting the document count fails the count assertion in both cases. The new case moved into its own describe block because the enclosing callback crossed the repository's max-lines-per-function bound otherwise. --------- Co-authored-by: 吴佳浩 --- .../apple/InterfaceBuilderAnalysis.ts | 56 +++++++- .../apple/interfaceBuilderGraph.test.ts | 100 +++++++++++++ src/domain/apple/interfaceBuilderGraph.ts | 23 ++- .../apple/interfaceBuilderKeyedArchive.ts | 12 +- .../filesystem/interfaceBuilderBundle.test.ts | 136 ++++++++++++++++++ 5 files changed, 315 insertions(+), 12 deletions(-) diff --git a/src/artifacts/apple/InterfaceBuilderAnalysis.ts b/src/artifacts/apple/InterfaceBuilderAnalysis.ts index fb90594b3..02ecd9821 100644 --- a/src/artifacts/apple/InterfaceBuilderAnalysis.ts +++ b/src/artifacts/apple/InterfaceBuilderAnalysis.ts @@ -56,6 +56,7 @@ export const analyzeInterfaceBuilderBundle = async (input: { const omittedArchives: Array<{ path: string; reason: string }> = []; let unreportedOmittedArchives = 0; const incompleteHierarchies = new Map(); + const unresolvedConnections = new Map(); const prototypeKeyOmissions = new Map(); let omitted = 0; let attempted = 0; @@ -136,6 +137,8 @@ export const analyzeInterfaceBuilderBundle = async (input: { : { value: nib.raw, omittedPrototypeKeys: 0 }; if (nib !== null && nib.omitted > 0) incompleteHierarchies.set(entry.path, nib.omitted); + if (nib !== null && nib.unresolvedConnections > 0) + unresolvedConnections.set(entry.path, nib.unresolvedConnections); if (omittedPrototypeKeys !== 0) prototypeKeyOmissions.set(entry.path, omittedPrototypeKeys); const documentHash = createHash("sha256").update(bytes).digest("hex"); @@ -170,6 +173,7 @@ export const analyzeInterfaceBuilderBundle = async (input: { }); return finalizeHierarchyCoverage(result, { incompleteHierarchies, + unresolvedConnections, prototypeKeyOmissions, invalid, omittedArchives, @@ -237,6 +241,7 @@ const finalizeHierarchyCoverage = ( result: ReturnType, counts: { incompleteHierarchies: ReadonlyMap; + unresolvedConnections: ReadonlyMap; prototypeKeyOmissions: ReadonlyMap; invalid: readonly string[]; omittedArchives: readonly { path: string; reason: string }[]; @@ -247,6 +252,7 @@ const finalizeHierarchyCoverage = ( ) => { const { incompleteHierarchies, + unresolvedConnections, prototypeKeyOmissions, invalid, omittedArchives, @@ -260,6 +266,13 @@ const finalizeHierarchyCoverage = ( ...result, documents: result.documents.map((document) => ({ ...document, + // The dictionary parser only sees connectors that were projected, so the + // omission recorded where an ownerless connector was dropped has to be + // added here for this producer to report the same document count as the + // keyed-archive producer for the same recognized connector. + connection_count: + document.connection_count + + (unresolvedConnections.get(document.relative_path) ?? 0), hierarchy_complete: document.hierarchy_complete && !incompleteHierarchies.has(document.relative_path), @@ -268,15 +281,32 @@ const finalizeHierarchyCoverage = ( ...result.graph, coverage: [ ...result.graph.coverage.map((facet) => { - const omitted = incompleteHierarchies.get( + const omittedHierarchy = incompleteHierarchies.get( facet.facet.replace(/^hierarchy:/u, ""), ); - return facet.facet.startsWith("hierarchy:") && omitted !== undefined + if ( + facet.facet.startsWith("hierarchy:") && + omittedHierarchy !== undefined + ) + return { + ...facet, + status: "partial" as const, + reason: "serialized_view_hierarchy_incomplete", + omitted: facet.omitted + omittedHierarchy, + }; + const omittedConnection = unresolvedConnections.get( + facet.facet.replace(/^connections:/u, ""), + ); + return facet.facet.startsWith("connections:") && + omittedConnection !== undefined ? { ...facet, status: "partial" as const, - reason: "serialized_view_hierarchy_incomplete", - omitted: facet.omitted + omitted, + // A connection dropped by the limit is recoverable by raising + // it, so that cause keeps the reason the graph already named. + // Only a facet with no other cause reports the omission. + reason: facet.reason ?? "connection_unresolved", + omitted: facet.omitted + omittedConnection, } : facet; }), @@ -306,7 +336,8 @@ const finalizeHierarchyCoverage = ( result.graph.truncated || counts.omitted > 0 || archiveDecodePartial || - incompleteHierarchies.size > 0, + incompleteHierarchies.size > 0 || + unresolvedConnections.size > 0, }, limitations: [ ...result.limitations, @@ -316,6 +347,11 @@ const finalizeHierarchyCoverage = ( : [ "Some serialized hierarchy links could not be projected within the bounded hierarchy; their parentage remains unknown.", ]), + ...(unresolvedConnections.size === 0 + ? [] + : [ + "Some recognized NIBArchive connectors declare no resolvable owner; they carry no connection evidence and are reported as omissions.", + ]), ...(invalid.length === 0 ? [] : [ @@ -496,6 +532,10 @@ const projectNibArchive = ( ); const connections: Record = {}; const hierarchy: NibHierarchyNode[] = []; + // A recognized connector whose owner reference cannot be resolved produces no + // connection entry, so the omission has to be counted here: the projected + // dictionary carries no trace of it for a downstream consumer to find. + let unresolvedConnections = 0; const reference = (value: JsonValue | undefined): number | null => { if (typeof value !== "object" || value === null || Array.isArray(value)) return null; @@ -512,7 +552,10 @@ const projectNibArchive = ( const values = object.values; const connectionFields = interfaceBuilderConnectionFieldNames(className); const source = reference(values[connectionFields.source]); - if (source === null) continue; + if (source === null) { + unresolvedConnections += 1; + continue; + } const destination = reference(values[connectionFields.destination]); const label = referencedString(values[connectionFields.label]); const type = className.includes("Outlet") @@ -608,6 +651,7 @@ const projectNibArchive = ( ); return { omitted: viewHierarchy.omitted + hierarchyOmitted + merged.omitted, + unresolvedConnections, raw: jsonValueSchema.parse({ "com.apple.ibtool.document.objects": objects, "com.apple.ibtool.document.connections": connections, diff --git a/src/domain/apple/interfaceBuilderGraph.test.ts b/src/domain/apple/interfaceBuilderGraph.test.ts index bfa2c71de..96c6b4915 100644 --- a/src/domain/apple/interfaceBuilderGraph.test.ts +++ b/src/domain/apple/interfaceBuilderGraph.test.ts @@ -510,6 +510,106 @@ describe("AppKit keyed archive connectors", () => { }); }); +describe("keyed archive unresolved connectors", () => { + // A rooted UIKit archive whose only connector is UID 3, with UID 1 as the + // owner, UID 2 as the destination and UID 7 as the "view" label. + const rootedOutletArchive = (connection: Record) => ({ + $archiver: "NSKeyedArchiver", + $version: 100000, + $objects: [ + "$null", + { $class: { CF$UID: 4 } }, + { $class: { CF$UID: 5 } }, + { $class: { CF$UID: 6 }, ...connection }, + { + $classname: "UIViewController", + $classes: ["UIViewController", "UIResponder", "NSObject"], + }, + { $classname: "UIView", $classes: ["UIView", "UIResponder", "NSObject"] }, + { + $classname: "UIRuntimeOutletConnection", + $classes: [ + "UIRuntimeOutletConnection", + "UIRuntimeConnection", + "NSObject", + ], + }, + "view", + { + $class: { CF$UID: 11 }, + "NS.objects": [{ CF$UID: 1 }, { CF$UID: 2 }, { CF$UID: 3 }], + }, + { $class: { CF$UID: 11 }, "NS.objects": [{ CF$UID: 3 }] }, + { + $class: { CF$UID: 11 }, + "NS.objects": [{ CF$UID: 1 }, { CF$UID: 2 }], + }, + { $classname: "NSArray", $classes: ["NSArray", "NSObject"] }, + ], + $top: { + root: { CF$UID: 8 }, + UINibConnectionsKey: { CF$UID: 9 }, + UINibObjectsKey: { CF$UID: 10 }, + UINibTopLevelObjectsKey: { CF$UID: 10 }, + }, + }); + + const project = (raw: unknown) => + buildInterfaceBuilderAnalysis({ + targetSha256: hash, + toolVersion: "test", + documents: [ + { + relativePath: "Main.nib", + archiveSha256: hash, + documentKind: "nib", + raw, + }, + ], + limits: interfaceBuilderLimitsSchema.parse({}), + }); + + it("covers a resolvable connector as a complete examined connection", () => { + const raw = rootedOutletArchive({ + UISource: { CF$UID: 1 }, + UIDestination: { CF$UID: 2 }, + UILabel: { CF$UID: 7 }, + }); + expect(parseInterfaceBuilderRecords(raw).unresolvedConnections).toBe(0); + expect(project(raw).graph.coverage).toContainEqual( + expect.objectContaining({ + facet: "connections:Main.nib", + status: "complete", + examined: 1, + omitted: 0, + }), + ); + }); + + it("reports a recognized connector with no resolvable owner as unresolved", () => { + const raw = rootedOutletArchive({ + UIDestination: { CF$UID: 2 }, + UILabel: { CF$UID: 7 }, + }); + expect(parseInterfaceBuilderRecords(raw).unresolvedConnections).toBe(1); + + const analysis = project(raw); + expect(analysis.graph.nodes.some(({ kind }) => kind === "outlet")).toBe( + false, + ); + expect(analysis.graph.coverage).toContainEqual( + expect.objectContaining({ + facet: "connections:Main.nib", + status: "partial", + reason: "connection_unresolved", + examined: 0, + omitted: 1, + }), + ); + expect(analysis.graph.truncated).toBe(true); + }); +}); + it("names the archive once per evidence record", () => { const relativePath = "Views/Main.storyboardc/scene.nib/objects.nib"; const result = buildInterfaceBuilderAnalysis({ diff --git a/src/domain/apple/interfaceBuilderGraph.ts b/src/domain/apple/interfaceBuilderGraph.ts index f25c8b9b1..b3e5216dc 100644 --- a/src/domain/apple/interfaceBuilderGraph.ts +++ b/src/domain/apple/interfaceBuilderGraph.ts @@ -410,18 +410,26 @@ export const buildInterfaceBuilderAnalysis = (input: { const connectionsTruncated = parserOmittedConnections > 0 || parsed.connections.length > input.limits.max_connections; + // A recognized connector the parser could not attach to an owner produces + // no connection record, so its coverage is partial with its own reason + // rather than absent or truncated. + const connectionsUnresolved = parsed.unresolvedConnections > 0; omittedObjects += parsed.omittedObjects + Math.max(0, parsed.objects.length - documentObjectCount); omittedConnections += Math.max(0, parsed.connections.length - input.limits.max_connections) + - parserOmittedConnections; + parserOmittedConnections + + parsed.unresolvedConnections; summaries.push({ relative_path: document.relativePath, archive_sha256: document.archiveSha256, document_kind: document.documentKind, object_count: parsed.objectCount, - connection_count: parsed.connections.length + parserOmittedConnections, + connection_count: + parsed.connections.length + + parserOmittedConnections + + parsed.unresolvedConnections, hierarchy_complete: parsed.hierarchy.length > 0 && !truncated && @@ -440,13 +448,17 @@ export const buildInterfaceBuilderAnalysis = (input: { { facet: `connections:${document.relativePath}`, status: - connectionsTruncated || documentOmittedConnections > 0 + connectionsTruncated || + documentOmittedConnections > 0 || + connectionsUnresolved ? "partial" : "complete", reason: connectionsTruncated || documentOmittedConnections > 0 ? "connection_limit_reached" - : null, + : connectionsUnresolved + ? "connection_unresolved" + : null, examined: Math.min( parsed.connections.length, input.limits.max_connections, @@ -457,7 +469,8 @@ export const buildInterfaceBuilderAnalysis = (input: { parsed.connections.length - input.limits.max_connections, ) + documentOmittedConnections + - parserOmittedConnections, + parserOmittedConnections + + parsed.unresolvedConnections, }, { facet: `hierarchy:${document.relativePath}`, diff --git a/src/domain/apple/interfaceBuilderKeyedArchive.ts b/src/domain/apple/interfaceBuilderKeyedArchive.ts index 207950bf8..57ebda803 100644 --- a/src/domain/apple/interfaceBuilderKeyedArchive.ts +++ b/src/domain/apple/interfaceBuilderKeyedArchive.ts @@ -55,6 +55,7 @@ export const parseInterfaceBuilderRecords = ( readonly objectCount: number; readonly omittedObjects: number; readonly omittedConnections: number; + readonly unresolvedConnections: number; readonly omittedHierarchyReferences: number; } => { const root = record(value); @@ -135,6 +136,7 @@ export const parseInterfaceBuilderRecords = ( objectCount: objectEntries.length, omittedObjects: Math.max(0, objectEntries.length - 20_000), omittedConnections, + unresolvedConnections: 0, omittedHierarchyReferences: 0, }; }; @@ -320,6 +322,7 @@ const parseKeyedArchive = ( const connections: InterfaceBuilderConnection[] = []; let objectCount = 0; let omittedConnections = 0; + let unresolvedConnections = 0; for (const [index, raw] of objectTable.entries()) { if (index === 0) continue; const resolved = record(resolve(raw)); @@ -371,7 +374,13 @@ const parseKeyedArchive = ( controlAction ? fields.to : fields.from, controlAction ? fields.target : fields.owner, ); - if (source === null) continue; + if (source === null) { + // A recognized connector that cannot name its owner is an unresolved + // connection, not an absent one, and cannot be represented as a + // connection because connection identity requires a source. + unresolvedConnections += 1; + continue; + } const kind = controlAction ? "action" : classifyConnection(className); const parsedConnection = connection.parse({ id: `archive:${objectId}`, @@ -426,6 +435,7 @@ const parseKeyedArchive = ( objectCount, omittedObjects: Math.max(0, objectCount - 20_000), omittedConnections, + unresolvedConnections, omittedHierarchyReferences: hierarchy.omittedReferences, }; }; diff --git a/tests/boundary/filesystem/interfaceBuilderBundle.test.ts b/tests/boundary/filesystem/interfaceBuilderBundle.test.ts index 4446c80dc..d584aaf3c 100644 --- a/tests/boundary/filesystem/interfaceBuilderBundle.test.ts +++ b/tests/boundary/filesystem/interfaceBuilderBundle.test.ts @@ -1060,6 +1060,142 @@ describe("NIBArchive UIKit event connections", () => { "UIRuntimeEventConnection", ); }); + + it("reports a runtime connection with no resolvable owner as an omission", async () => { + const root = await createTestTempDirectory("rea-ib-ownerless-"); + const bundle = join(root, "Example.app"); + await mkdir(bundle, { recursive: true }); + // The connector is recognized by class, but carries no UISource reference, + // so no connection can be projected from it. + await writeFile( + join(bundle, "Cell.nib"), + encodeNibArchiveFixture({ + classes: [ + "UIProxyObject", + "UIClassSwapper", + "UIButton", + "UIRuntimeEventConnection", + "NSString", + ], + objects: [ + { classIndex: 0, values: { UIProxiedObjectIdentifier: { ref: 5 } } }, + { classIndex: 1, values: { UIClassName: { ref: 6 } } }, + { classIndex: 2, values: {} }, + { + classIndex: 3, + values: { UIDestination: { ref: 1 }, UIEventMask: 64 }, + }, + { classIndex: 2, values: {} }, + { classIndex: 4, values: { "NS.bytes": "IBFilesOwner" } }, + { classIndex: 4, values: { "NS.bytes": "BuildCell" } }, + ], + }), + ); + + const analysis = await analyzeInterfaceBuilderBundle({ + bundlePath: bundle, + targetSha256: "e".repeat(64), + }); + + expect(analysis.graph.coverage).toContainEqual( + expect.objectContaining({ + facet: "connections:Cell.nib", + status: "partial", + reason: "connection_unresolved", + examined: 0, + omitted: 1, + }), + ); + // The keyed-archive producer counts a recognized connector it cannot emit, + // so this producer has to report the same document total for the same + // connector rather than zero. + expect(analysis.documents).toContainEqual( + expect.objectContaining({ + relative_path: "Cell.nib", + connection_count: 1, + }), + ); + expect(analysis.graph.truncated).toBe(true); + expect(analysis.limitations).toContain( + "Some recognized NIBArchive connectors declare no resolvable owner; they carry no connection evidence and are reported as omissions.", + ); + }); +}); + +describe("NIBArchive ownerless connector accounting", () => { + it("keeps a connection dropped by the limit reported as a limit", async () => { + const root = await createTestTempDirectory("rea-ib-ownerless-limit-"); + const bundle = join(root, "Example.app"); + await mkdir(bundle, { recursive: true }); + // Two resolvable connectors and one ownerless connector with the limit + // below the resolvable count, so this facet has both causes at once. Only a + // higher limit recovers the dropped connection, so that is the reason the + // caller needs, exactly as the keyed-archive producer reports it. + await writeFile( + join(bundle, "Cell.nib"), + encodeNibArchiveFixture({ + classes: [ + "UIProxyObject", + "UIClassSwapper", + "UIButton", + "UIRuntimeEventConnection", + "NSString", + ], + objects: [ + { classIndex: 0, values: { UIProxiedObjectIdentifier: { ref: 5 } } }, + { classIndex: 1, values: { UIClassName: { ref: 6 } } }, + { classIndex: 2, values: {} }, + { + classIndex: 3, + values: { + UISource: { ref: 2 }, + UIDestination: { ref: 1 }, + UILabel: { ref: 7 }, + UIEventMask: 64, + }, + }, + { classIndex: 2, values: {} }, + { classIndex: 4, values: { "NS.bytes": "IBFilesOwner" } }, + { classIndex: 4, values: { "NS.bytes": "BuildCell" } }, + { classIndex: 4, values: { "NS.bytes": "buildTapped:" } }, + { + classIndex: 3, + values: { + UISource: { ref: 4 }, + UIDestination: { ref: 1 }, + UILabel: { ref: 7 }, + }, + }, + { + classIndex: 3, + values: { UIDestination: { ref: 1 }, UIEventMask: 64 }, + }, + ], + }), + ); + + const analysis = await analyzeInterfaceBuilderBundle({ + bundlePath: bundle, + targetSha256: "e".repeat(64), + limits: { max_connections: 1 }, + }); + + expect(analysis.graph.coverage).toContainEqual( + expect.objectContaining({ + facet: "connections:Cell.nib", + status: "partial", + reason: "connection_limit_reached", + examined: 1, + omitted: 2, + }), + ); + expect(analysis.documents).toContainEqual( + expect.objectContaining({ + relative_path: "Cell.nib", + connection_count: 3, + }), + ); + }); }); // This exact file is already selected by the unchanged native Apple CI lane. From 5eb296649d2cd53b0bb74df7b3d8081e964e027d Mon Sep 17 00:00:00 2001 From: Mason Date: Tue, 13 Oct 2026 21:22:42 +0800 Subject: [PATCH 13/24] fix(native): refuse inspect_signature on non-Mach-O targets (#2262) * fix(native): refuse inspect_signature on non-Mach-O targets This provider's signature inspection covers Mach-O code signatures only. On a xar pkg, codesign's "code object is not signed" verdict cannot see the archive-level signature layer, so a signed or invalidly-signed package was reported as signed:false with "Artifact is not signed." Match the existing inspect_macho/list_architectures gate and report capability_unavailable instead of an unsigned claim for a layer that was never evaluated. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(native): scope the signature-gate comment to this provider Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(native): align signature capability discovery with Mach-O support Bind inspect_signature availability to Mach-O targets in capability discovery and align the public tool description. Source inspection and git diff --check; tests/builds/typechecks/CI reruns skipped at explicit user request. --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- src/application/CapabilityInventory.ts | 1 + src/contracts/native/nativeToolContracts.ts | 2 +- src/native/NativeMacOSProvider.ts | 13 ++++++++ .../providers/native/nativeProvider.test.ts | 32 +++++++++++++++++++ 4 files changed, 47 insertions(+), 1 deletion(-) diff --git a/src/application/CapabilityInventory.ts b/src/application/CapabilityInventory.ts index 4e364a1aa..f4584eb29 100644 --- a/src/application/CapabilityInventory.ts +++ b/src/application/CapabilityInventory.ts @@ -141,6 +141,7 @@ const FRIDA_OPERATION_NAMES: ReadonlySet = new Set( /** Native contracts defined only for an active Mach-O executable. */ const MACHO_TARGET_OPERATIONS: ReadonlySet = new Set([ + "inspect_signature", "inspect_macho", "list_architectures", "observe_native_calls", diff --git a/src/contracts/native/nativeToolContracts.ts b/src/contracts/native/nativeToolContracts.ts index 16101368b..d45b62cc6 100644 --- a/src/contracts/native/nativeToolContracts.ts +++ b/src/contracts/native/nativeToolContracts.ts @@ -107,7 +107,7 @@ export const NATIVE_TOOL_CONTRACTS = [ ), native( "inspect_signature", - "Inspect the active artifact's code-signing identity, hashes, authorities, requirements, entitlements, hardened-runtime state, and exact command provenance.", + "Inspect the active Mach-O executable's code-signing identity, hashes, authorities, requirements, entitlements, hardened-runtime state, and exact command provenance.", z.object({}), ), native( diff --git a/src/native/NativeMacOSProvider.ts b/src/native/NativeMacOSProvider.ts index 691e830d9..333569e47 100644 --- a/src/native/NativeMacOSProvider.ts +++ b/src/native/NativeMacOSProvider.ts @@ -564,6 +564,19 @@ class NativeMacOSClient implements AnalysisClient { async #inspectSignature( signal?: AbortSignal, ): Promise> { + // This provider's signature inspection covers Mach-O code signatures + // only. On another container, codesign's "not signed" verdict cannot + // see a format-level signature such as a xar pkg signature, so + // reporting it would claim unsigned for a layer that was never + // evaluated. + if (this.target.kind !== "executable" || !isMachOFormat(this.target.format)) + return err( + new AnalysisCapabilityUnavailableError( + IDENTITY.id, + "inspect_signature", + "Active artifact is not Mach-O; its container-level signature layer is not evaluated by codesign.", + ), + ); return withSignatureTarget( this.target, this.#signatureResources, diff --git a/tests/boundary/providers/native/nativeProvider.test.ts b/tests/boundary/providers/native/nativeProvider.test.ts index 81766e22d..a1e7d63ce 100644 --- a/tests/boundary/providers/native/nativeProvider.test.ts +++ b/tests/boundary/providers/native/nativeProvider.test.ts @@ -1024,6 +1024,38 @@ describe("native macOS provider failures and parsing", () => { }); }); +describe("native signature inspection target scope", () => { + // This provider's inspect_signature path is scoped to Mach-O targets; on + // another container a codesign verdict cannot see a format-level signature + // such as a xar pkg signature. + it.each([ + { + path: "/fixture.pkg", + sha256: "0".repeat(64), + kind: "archive" as const, + format: "pkg" as const, + }, + { + ...machoTarget("/fixture.elf"), + format: "elf" as const, + }, + ])( + "refuses inspect_signature on a $format target without running codesign", + async (target) => { + const runner = new CountingRunner(); + const result = await new NativeMacOSProvider({}, runner, "darwin") + .createClient(target) + .execute("inspect_signature", {}); + if (result.ok) throw new Error("Expected a non-Mach-O refusal"); + expect(result.error._tag).toBe("AnalysisCapabilityUnavailableError"); + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "capability_unavailable", + }); + expect(runner.calls).toBe(0); + }, + ); +}); + class FailingRunner implements NativeCommandRunner { constructor(private readonly reason: NativeCommandFailure["reason"]) {} From 84ec40f1af8ea7d74b757d7b50b391aec8482f8a Mon Sep 17 00:00:00 2001 From: xingzhou Date: Tue, 13 Oct 2026 21:23:33 +0800 Subject: [PATCH 14/24] fix(browser): distinguish SVG links from HTML XLink attributes (#2480) * fix(browser): respect SVG desc and title HTML integration points * fix(browser): preserve moved SVG XLink destinations * fix(browser): bound concurrent SVG identity reads --- .../CdpCaptureDocuments.baseUrl.test.ts | 284 ++++++++++++++-- src/browser/CdpCaptureDocuments.ts | 160 +++++++-- src/browser/CdpCaptureDomSvg.ts | 74 ++-- src/browser/CdpPageCapture.ts | 54 ++- .../browser/cdpDomSvgIdentity.test.ts | 315 ++++++++++++++++++ .../boundary/browser/cdpSvgXlinkHref.test.ts | 57 +++- 6 files changed, 821 insertions(+), 123 deletions(-) create mode 100644 tests/boundary/browser/cdpDomSvgIdentity.test.ts diff --git a/src/browser/CdpCaptureDocuments.baseUrl.test.ts b/src/browser/CdpCaptureDocuments.baseUrl.test.ts index a5481c161..e3201eade 100644 --- a/src/browser/CdpCaptureDocuments.baseUrl.test.ts +++ b/src/browser/CdpCaptureDocuments.baseUrl.test.ts @@ -2,6 +2,8 @@ import { describe, expect, it } from "vitest"; import { inspectWebPageInputSchema } from "../domain/browserObservation.js"; import { captureDom } from "./CdpCaptureDocuments.js"; +import { capturedSvgNodeKind } from "./CdpCaptureDomSvg.js"; +import { CdpCaptureCompleteness } from "./CdpCaptureCompleteness.js"; const origin = "https://app.example.test"; const documentUrl = `${origin}/screens/app`; @@ -50,8 +52,11 @@ const snapshot = ( ], }); -const capture = (value: ReturnType) => - captureDom(value, new Set([origin]), input); +const capture = (value: ReturnType) => { + const step = captureDom(value, new Set([origin]), input).next(); + if (!step.done) throw new Error("Unexpected SVG identity query"); + return step.value; +}; describe("DOM metadata document base URLs", () => { it("resolves links, actions and agent hints against the captured document base", () => { @@ -282,6 +287,7 @@ interface SvgSnapshotNode { readonly name: string; readonly parent: number; readonly attributes?: readonly (readonly [string, string])[]; + readonly svg?: boolean; } const svgDocument = ( @@ -300,7 +306,13 @@ const svgDocument = ( const nodeValue = [intern("")]; const parentIndex = [-1]; const attributes: number[][] = [[]]; + const backendNodeId = [1]; + const svgNodeKinds = new Map(); for (const element of elements) { + const id = backendNodeId.length + 1; + backendNodeId.push(id); + if (element.svg !== undefined) + svgNodeKinds.set(`${String(id)}:${element.name}`, element.svg); nodeType.push(1); nodeName.push(intern(element.name)); nodeValue.push(intern("")); @@ -316,14 +328,35 @@ const svgDocument = ( { documentURL: 0, baseURL: 1, - nodes: { nodeType, nodeName, nodeValue, parentIndex, attributes }, + nodes: { + nodeType, + nodeName, + nodeValue, + parentIndex, + attributes, + backendNodeId, + }, + svgNodeKinds, }, ], }; }; -const captureSvg = (value: ReturnType) => - captureDom(value, new Set([origin]), input); +const captureSvg = (value: ReturnType) => { + const kinds = new Map( + value.documents.flatMap((document) => [...document.svgNodeKinds]), + ); + const capture = captureDom(value, new Set([origin]), input); + let step = capture.next(); + while (!step.done) { + step = capture.next( + step.value.map((query) => + kinds.get(`${String(query.backendNodeId)}:${query.nodeName}`), + ), + ); + } + return step.value; +}; const urlFor = ( result: ReturnType, @@ -331,6 +364,136 @@ const urlFor = ( ): (typeof result.urls)[number] | undefined => result.urls.find((url) => url.node_index === nodeIndex); +describe("browser SVG identity", () => { + const query = { backendNodeId: 42, nodeName: "a" }; + const node = { ...query, nodeType: 1 }; + + it("uses only a matching described element's SVG flag", () => { + expect(capturedSvgNodeKind(query, { node: { ...node, isSVG: true } })).toBe( + true, + ); + expect( + capturedSvgNodeKind(query, { node: { ...node, isSVG: false } }), + ).toBe(false); + expect(capturedSvgNodeKind(query, { node })).toBe(false); + for (const invalid of [ + {}, + { node: { ...node, backendNodeId: 43, isSVG: true } }, + { node: { ...node, nodeName: "A", isSVG: true } }, + { node: { ...node, nodeType: 3, isSVG: true } }, + { node: { ...node, isSVG: "true" } }, + ]) + expect(capturedSvgNodeKind(query, invalid)).toBeUndefined(); + }); + + it("distinguishes identically named SVG and MathML elements regardless of parent", () => { + const result = captureSvg( + svgDocument([ + { name: "svg", parent: 0 }, + { name: "desc", parent: 1 }, + { + name: "a", + svg: true, + parent: 2, + attributes: [["xlink:href", "/svg"]], + }, + { + name: "a", + svg: false, + parent: 2, + attributes: [["xlink:href", "/math"]], + }, + { + name: "A", + svg: false, + parent: 1, + attributes: [["xlink:href", "/html"]], + }, + { + name: "a", + svg: true, + parent: 0, + attributes: [["xlink:href", "/orphan"]], + }, + ]), + ); + expect(result.urls.map(({ url }) => url)).toEqual([ + `${origin}/svg`, + `${origin}/orphan`, + ]); + }); + + it("records unresolved identity without inventing a destination or hiding attributes", () => { + const value = svgDocument([ + { name: "a", parent: 0, attributes: [["xlink:href", "/unknown"]] }, + ]); + const completeness = new CdpCaptureCompleteness(); + const capture = captureDom(value, new Set([origin]), input, completeness); + expect(capture.next()).toEqual({ + done: false, + value: [{ backendNodeId: 2, nodeName: "a" }], + }); + const step = capture.next([undefined]); + expect(step.done).toBe(true); + if (!step.done) throw new Error("Unexpected second query"); + expect(step.value.unresolvedSvgNodes).toBe(1); + expect(step.value.urls).toEqual([]); + expect(step.value.nodes[1]?.attribute_names).toEqual(["xlink:href"]); + expect(completeness.snapshot().excluded).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + section: "metadata", + reason: "provider_unavailable", + count: 1, + }), + ]), + ); + }); +}); + +it("keeps batched SVG identities aligned around missing IDs and ordinary links", () => { + const value = svgDocument([ + { name: "a", parent: 0, attributes: [["xlink:href", "/svg"]] }, + { name: "A", parent: 0, attributes: [["href", "/ordinary"]] }, + { name: "a", parent: 0, attributes: [["xlink:href", "/missing-id"]] }, + { name: "a", parent: 0, attributes: [["xlink:href", "/non-svg"]] }, + ]); + const completeness = new CdpCaptureCompleteness(); + const capture = captureDom( + { + ...value, + documents: value.documents.map((document) => ({ + ...document, + nodes: { ...document.nodes, backendNodeId: [1, 2, 3, null, 5] }, + })), + }, + new Set([origin]), + input, + completeness, + ); + expect(capture.next()).toEqual({ + done: false, + value: [ + { backendNodeId: 2, nodeName: "a" }, + { backendNodeId: 5, nodeName: "a" }, + ], + }); + const step = capture.next([true, false]); + if (!step.done) throw new Error("Unexpected second batch"); + expect( + step.value.urls.map(({ node_index, url }) => [node_index, url]), + ).toEqual([ + [1, `${origin}/svg`], + [2, `${origin}/ordinary`], + ]); + expect(step.value.unresolvedSvgNodes).toBe(1); + expect(completeness.snapshot().excluded).toContainEqual({ + section: "metadata", + reason: "provider_unavailable", + count: 1, + }); +}); + describe("SVG xlink:href destinations", () => { it("reports an SVG anchor that only has xlink:href, keeping the first value", () => { const result = captureSvg( @@ -339,6 +502,7 @@ describe("SVG xlink:href destinations", () => { { name: "g", parent: 1 }, { name: "a", + svg: true, parent: 2, attributes: [ ["xlink:href", "/xl-s2h"], @@ -371,6 +535,7 @@ describe("SVG xlink:href destinations", () => { { name: "svg", parent: 0 }, { name: "a", + svg: true, parent: 1, attributes: [ [first, valueFor(first)], @@ -395,18 +560,21 @@ describe("SVG xlink:href destinations", () => { svgDocument([ { name: "A", + svg: false, parent: 0, attributes: [["xlink:href", "/xl-html"]], }, { name: "math", parent: 0 }, { name: "mrow", + svg: false, parent: 2, attributes: [["xlink:href", "/xl-math"]], }, { name: "svg", parent: 0 }, { name: "a", + svg: true, parent: 4, attributes: [["xlink:href", "/xl-s2h"]], }, @@ -425,6 +593,7 @@ describe("SVG xlink:href destinations", () => { { name: "svg", parent: 0 }, { name: "a", + svg: true, parent: 1, attributes: [["xlink:href", "https://cdn.example.test/out"]], }, @@ -444,8 +613,18 @@ describe("SVG xlink:href destinations", () => { const result = captureSvg( svgDocument([ { name: "svg", parent: 0 }, - { name: "image", parent: 1, attributes: [["xlink:href", "guide"]] }, - { name: "use", parent: 1, attributes: [["xlink:href", "icon.svg"]] }, + { + name: "image", + svg: true, + parent: 1, + attributes: [["xlink:href", "guide"]], + }, + { + name: "use", + svg: true, + parent: 1, + attributes: [["xlink:href", "icon.svg"]], + }, ]), ); expect(result.urls).toEqual([ @@ -466,14 +645,24 @@ describe("SVG xlink:href destinations", () => { }); describe("SVG xlink:href context", () => { - it("keeps forward ancestor context local to each document", () => { + it("binds SVG identity to the selected element across documents", () => { const svg = svgDocument([ - { name: "a", parent: 2, attributes: [["xlink:href", "/svg-link"]] }, + { + name: "a", + svg: true, + parent: 2, + attributes: [["xlink:href", "/svg-link"]], + }, { name: "g", parent: 3 }, { name: "svg", parent: 0 }, ]); const html = svgDocument([ - { name: "A", parent: 2, attributes: [["xlink:href", "/html-link"]] }, + { + name: "A", + svg: false, + parent: 2, + attributes: [["xlink:href", "/html-link"]], + }, { name: "DIV", parent: 3 }, { name: "BODY", parent: 0 }, ]); @@ -507,32 +696,39 @@ describe("SVG xlink:href context", () => { ]); }); - it("stops SVG context at foreignObject and resumes it for a nested svg", () => { - const result = captureSvg( - svgDocument([ - { name: "svg", parent: 0 }, - { name: "foreignObject", parent: 1 }, - { - name: "A", - parent: 2, - attributes: [["xlink:href", "/xl-html-fo"]], - }, - { name: "svg", parent: 2 }, - { - name: "a", - parent: 4, - attributes: [["xlink:href", "/xl-nested"]], - }, - ]), - ); - expect(urlFor(result, 3)).toBeUndefined(); - expect(urlFor(result, 5)).toMatchObject({ - attribute: "href", - url: `${origin}/xl-nested`, - destination_scope: "approved", - }); - }); + it.each(["foreignObject", "desc", "title"])( + "preserves browser SVG identity across %s HTML integration points", + (integrationPoint) => { + const result = captureSvg( + svgDocument([ + { name: "svg", parent: 0 }, + { name: integrationPoint, parent: 1 }, + { + name: "A", + svg: false, + parent: 2, + attributes: [["xlink:href", "/xl-html-fo"]], + }, + { name: "svg", parent: 2 }, + { + name: "a", + svg: true, + parent: 4, + attributes: [["xlink:href", "/xl-nested"]], + }, + ]), + ); + expect(urlFor(result, 3)).toBeUndefined(); + expect(urlFor(result, 5)).toMatchObject({ + attribute: "href", + url: `${origin}/xl-nested`, + destination_scope: "approved", + }); + }, + ); +}); +describe("SVG XLink precedence and unknown identity", () => { it("keeps an empty href ahead of xlink:href", () => { const baseUrl = `${origin}/screens/assets/`; const result = captureSvg( @@ -541,6 +737,7 @@ describe("SVG xlink:href context", () => { { name: "svg", parent: 0 }, { name: "a", + svg: true, parent: 1, attributes: [ ["href", ""], @@ -567,6 +764,7 @@ describe("SVG xlink:href context", () => { { name: "svg", parent: 0 }, { name: "a", + svg: true, parent: 1, attributes: [["xlink:href", "javascript:void(0)"]], }, @@ -582,7 +780,7 @@ describe("SVG xlink:href context", () => { ]); }); - it("does not treat a cyclic or dangling parent chain as SVG", () => { + it("keeps an unavailable SVG identity unknown despite ancestor names", () => { const cyclic = captureSvg( svgDocument([ { name: "g", parent: 2 }, @@ -652,7 +850,12 @@ describe("XML SVG roots and namespace scope restoration", () => { ["xmlns:p", XLINK_NAMESPACE], ], }, - { name: "a", parent: 1, attributes: [["p:href", "/linked"]] }, + { + name: "a", + svg: true, + parent: 1, + attributes: [["p:href", "/linked"]], + }, ]), ); expect(result.urls).toEqual([approvedHref(2, `${origin}/linked`)]); @@ -764,6 +967,7 @@ describe("XHTML and XML SVG namespace prefixes", () => { { name: "svg:foreignObject", parent: 2 }, { name: "a", + svg: true, parent: 3, attributes: [["xlink:href", "/xl-html-fo"]], }, @@ -799,6 +1003,7 @@ describe("XHTML and XML SVG namespace prefixes", () => { }, { name: "a", + svg: true, parent: 2, attributes: [ ["xlink:href", "/not-xlink"], @@ -918,11 +1123,13 @@ describe("XHTML SVG exclusions, HTML documents, and prefix scope", () => { }, { name: "a", + svg: true, parent: 2, attributes: [["xlink:href", "/xl-literal"]], }, { name: "a", + svg: true, parent: 2, attributes: [["x:href", "/xl-custom-html"]], }, @@ -937,6 +1144,7 @@ describe("XHTML SVG exclusions, HTML documents, and prefix scope", () => { }, { name: "A", + svg: false, parent: 1, attributes: [["xlink:href", "/xl-html-anchor"]], }, @@ -944,7 +1152,9 @@ describe("XHTML SVG exclusions, HTML documents, and prefix scope", () => { ); expect(result.urls).toEqual([approvedHref(3, `${origin}/xl-literal`)]); }); +}); +describe("XML namespace resolution ordering and capacity", () => { it("applies an element's own xmlns declarations to its name and href", () => { const result = captureSvg( svgDocument([ diff --git a/src/browser/CdpCaptureDocuments.ts b/src/browser/CdpCaptureDocuments.ts index 3c7fb98ed..f7eb4fe31 100644 --- a/src/browser/CdpCaptureDocuments.ts +++ b/src/browser/CdpCaptureDocuments.ts @@ -11,7 +11,11 @@ import { } from "./CdpCaptureValues.js"; import { exclusionReasonForUrl } from "./CdpCaptureEventHelpers.js"; import type { CdpCaptureCompleteness } from "./CdpCaptureCompleteness.js"; -import { createDocumentSvgAncestry, domSvgLink } from "./CdpCaptureDomSvg.js"; +import { + createDocumentSvgAncestry, + domSvgLink, + type DomSvgNodeQuery, +} from "./CdpCaptureDomSvg.js"; import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js"; export type CapturedResource = Omit< @@ -135,18 +139,39 @@ export const walkFrameTrees = function* ( } }; -export const captureDom = ( +interface DomMetadataCapture { + readonly urls: WebPageInspection["metadata"]["dom_urls"]; + readonly agentHints: WebPageInspection["metadata"]["agent_hints"]; + readonly excludedUrls: number; + readonly unresolvedSvgNodes: number; +} + +export interface DomCapture extends DomMetadataCapture { + readonly total: number; + readonly nodes: WebPageInspection["dom"]["nodes"]; +} + +// Bound retained projections and outstanding CDP reads independently of page size. +// Eight leaves room for the session's other cancellation listeners. +const DOM_METADATA_BATCH_SIZE = 8; + +interface PendingDomMetadata { + readonly options: DomMetadataOptions; + // Undefined means no query is needed; null means the candidate has no valid ID. + readonly svgQuery: DomSvgNodeQuery | null | undefined; +} + +/** Project one snapshot, requesting bounded batches of SVG identity facts. */ +export function* captureDom( result: unknown, allowedOrigins: ReadonlySet, input: InspectWebPageInput, completeness?: CdpCaptureCompleteness, -): { - readonly total: number; - readonly nodes: WebPageInspection["dom"]["nodes"]; - readonly urls: WebPageInspection["metadata"]["dom_urls"]; - readonly agentHints: WebPageInspection["metadata"]["agent_hints"]; - readonly excludedUrls: number; -} => { +): Generator< + readonly DomSvgNodeQuery[], + DomCapture, + readonly (boolean | undefined)[] +> { const root = requiredRecord(result); const strings = Array.isArray(root.strings) ? root.strings.map((value) => cdpStringValue(value) ?? "") @@ -156,6 +181,7 @@ export const captureDom = ( const agentHints: WebPageInspection["metadata"]["agent_hints"] = []; let excludedUrls = 0; let total = 0; + let unresolvedSvgNodes = 0; for (const document of recordsValue(root.documents)) { const documentUrl = indexedString(strings, document.documentURL); if (allowedSanitizedUrl(documentUrl, allowedOrigins) === undefined) { @@ -172,6 +198,9 @@ export const captureDom = ( const attributes = Array.isArray(documentNodes.attributes) ? documentNodes.attributes : []; + const backendNodeIds = Array.isArray(documentNodes.backendNodeId) + ? documentNodes.backendNodeId + : []; const baseIndex = nodes.length; const ancestry = createDocumentSvgAncestry({ nodeTypes, @@ -181,12 +210,29 @@ export const captureDom = ( attributes, }); total += nodeTypes.length; + const pendingMetadata: PendingDomMetadata[] = []; for (let index = 0; index < nodeTypes.length; index += 1) { const attributeIndexes = numberArray(attributes[index]); const parent = Math.trunc(parents[index] ?? -1); const nodeIndex = nodes.length; const nodeName = indexedString(strings, nodeNames[index]); const attributeNames = uniqueAttributeNames(attributeIndexes, strings); + const svgLink = domSvgLink(ancestry, index); + let svgQuery: DomSvgNodeQuery | null | undefined; + if ( + !svgLink.xmlDocument && + nodeTypes[index] === 1 && + attributeNames.includes("xlink:href") && + !attributeNames.includes("href") + ) { + const backendNodeId = numberValue(backendNodeIds[index]); + svgQuery = + backendNodeId !== undefined && + Number.isSafeInteger(backendNodeId) && + backendNodeId > 0 + ? { backendNodeId, nodeName } + : null; + } nodes.push({ index: nodeIndex, parent_index: parent < 0 ? -1 : baseIndex + parent, @@ -195,30 +241,34 @@ export const captureDom = ( node_value_length: indexedString(strings, nodeValues[index]).length, attribute_names: attributeNames, }); - const metadata = domMetadata({ - strings, - attributes: attributeIndexes, - // The BASE element itself resolves against the document fallback URL. - baseUrl: nodeName.toLowerCase() === "base" ? documentUrl : baseUrl, - documentUrl, - nodeIndex, - nodeName, - allowedOrigins, - ...domSvgLink(ancestry, index), + pendingMetadata.push({ + svgQuery, + options: { + strings, + attributes: attributeIndexes, + // The BASE element itself resolves against the document fallback URL. + baseUrl: nodeName.toLowerCase() === "base" ? documentUrl : baseUrl, + documentUrl, + nodeIndex, + nodeName, + allowedOrigins, + ...svgLink, + }, }); - for (const url of metadata.urls) { - urls.push(url); - if (url.destination_scope !== "approved") { - excludedUrls += 1; - completeness?.exclude( - "metadata", - url.destination_scope === "outside_policy" - ? "disallowed_origin" - : "unsupported_url", - ); - } + if ( + pendingMetadata.length === DOM_METADATA_BATCH_SIZE || + index === nodeTypes.length - 1 + ) { + const metadata = yield* captureDomMetadata( + pendingMetadata, + completeness, + ); + urls.push(...metadata.urls); + agentHints.push(...metadata.agentHints); + excludedUrls += metadata.excludedUrls; + unresolvedSvgNodes += metadata.unresolvedSvgNodes; + pendingMetadata.length = 0; } - agentHints.push(...metadata.agentHints); } } return { @@ -227,8 +277,54 @@ export const captureDom = ( urls, agentHints, excludedUrls, + unresolvedSvgNodes, }; -}; +} + +function* captureDomMetadata( + pending: readonly PendingDomMetadata[], + completeness: CdpCaptureCompleteness | undefined, +): Generator< + readonly DomSvgNodeQuery[], + DomMetadataCapture, + readonly (boolean | undefined)[] +> { + const queries = pending.flatMap(({ svgQuery }) => + svgQuery == null ? [] : [svgQuery], + ); + const kinds = queries.length > 0 ? yield queries : []; + const urls: DomMetadataCapture["urls"] = []; + const agentHints: DomMetadataCapture["agentHints"] = []; + let queryIndex = 0; + let unresolvedSvgNodes = 0; + let excludedUrls = 0; + for (const { options, svgQuery } of pending) { + let svgElement = options.svgElement; + if (svgQuery !== undefined) { + const kind = svgQuery === null ? undefined : kinds[queryIndex++]; + svgElement = kind === true; + if (kind === undefined) { + unresolvedSvgNodes += 1; + completeness?.exclude("metadata", "provider_unavailable"); + } + } + const metadata = domMetadata({ ...options, svgElement }); + for (const url of metadata.urls) { + urls.push(url); + if (url.destination_scope !== "approved") { + excludedUrls += 1; + completeness?.exclude( + "metadata", + url.destination_scope === "outside_policy" + ? "disallowed_origin" + : "unsupported_url", + ); + } + } + agentHints.push(...metadata.agentHints); + } + return { urls, agentHints, unresolvedSvgNodes, excludedUrls }; +} export const captureAccessibility = ( results: readonly unknown[], diff --git a/src/browser/CdpCaptureDomSvg.ts b/src/browser/CdpCaptureDomSvg.ts index 3d1a5c977..4d021766c 100644 --- a/src/browser/CdpCaptureDomSvg.ts +++ b/src/browser/CdpCaptureDomSvg.ts @@ -1,4 +1,4 @@ -import { numberValue } from "./CdpCaptureValues.js"; +import { numberValue, recordValue } from "./CdpCaptureValues.js"; const SVG_NAMESPACE = "http://www.w3.org/2000/svg"; const XLINK_NAMESPACE = "http://www.w3.org/1999/xlink"; @@ -18,11 +18,34 @@ interface XmlSvgLink { interface DocumentSvgAncestry { readonly xml: boolean; - readonly nodes: DomSnapshotNodes; - readonly svgContexts: Map; readonly xmlLinks: Map; } +export interface DomSvgNodeQuery { + readonly backendNodeId: number; + readonly nodeName: string; +} + +/** Bind the browser's SVG flag to the exact snapshotted element. */ +export const capturedSvgNodeKind = ( + query: DomSvgNodeQuery, + result: unknown, +): boolean | undefined => { + const node = recordValue(recordValue(result)?.node); + if ( + node?.backendNodeId !== query.backendNodeId || + node.nodeName !== query.nodeName || + node.nodeType !== 1 + ) + return undefined; + // CDP omits isSVG for non-SVG nodes. Other values are malformed, not false. + return node.isSVG === undefined + ? false + : typeof node.isSVG === "boolean" + ? node.isSVG + : undefined; +}; + interface DomSvgLink { readonly svgElement: boolean; readonly xmlDocument: boolean; @@ -79,8 +102,6 @@ export const createDocumentSvgAncestry = ( const xml = xmlDocument(nodes); return { xml, - nodes, - svgContexts: new Map(), xmlLinks: xml ? resolveXmlElements(nodes) : new Map(), }; }; @@ -91,11 +112,7 @@ export const domSvgLink = ( ): DomSvgLink => { if (!ancestry.xml) { return { - svgElement: elementInSvgContext( - index, - ancestry.nodes, - ancestry.svgContexts, - ), + svgElement: false, xmlDocument: false, xlinkHref: undefined, }; @@ -108,43 +125,6 @@ export const domSvgLink = ( }; }; -// Chrome's HTML parser records SVG local names (`svg`, `foreignObject`). -// Descendants of foreignObject are HTML until a nested `svg`. -const elementInSvgContext = ( - index: number, - nodes: DomSnapshotNodes, - contexts: Map, -): boolean => { - let current = index; - const seen = new Set(); - let inSvg = false; - while ( - current >= 0 && - current < nodes.nodeNames.length && - !seen.has(current) - ) { - const cached = contexts.get(current); - if (cached !== undefined) { - inSvg = cached; - break; - } - seen.add(current); - if (stringAt(nodes.strings, nodes.nodeNames[current]) === "svg") { - inSvg = true; - break; - } - const parent = Math.trunc(nodes.parents[current] ?? -1); - if (parent < 0 || parent === current) break; - if (stringAt(nodes.strings, nodes.nodeNames[parent]) === "foreignObject") - break; - current = parent; - } - // Cache the entire examined chain, including forward references and cycles, - // so shared ancestors are examined once per document. - for (const node of seen) contexts.set(node, inSvg); - return inSvg; -}; - interface NamespaceRestore { readonly prefix: string; readonly previous: string | undefined; diff --git a/src/browser/CdpPageCapture.ts b/src/browser/CdpPageCapture.ts index 0d97576b5..f49d4e186 100644 --- a/src/browser/CdpPageCapture.ts +++ b/src/browser/CdpPageCapture.ts @@ -17,8 +17,10 @@ import { captureFrames, captureResources, type CapturedResource, + type DomCapture, mainFrameUrl, } from "./CdpCaptureDocuments.js"; +import { capturedSvgNodeKind } from "./CdpCaptureDomSvg.js"; import { allowedSanitizedUrl, delayWithCancellation, @@ -52,7 +54,7 @@ interface CaptureState { interface CapturedSections { readonly attachedUrl: string; readonly frames: WebPageInspection["frames"]; - readonly dom: ReturnType; + readonly dom: DomCapture; readonly accessibility: ReturnType; readonly scripts: WebPageInspection["scripts"]; readonly resources: WebPageInspection["resources"]; @@ -260,9 +262,7 @@ const authorizeObservationWindow = async ( throw new BrowserObservationError("inspect_web_page", "target_not_allowed"); }; -const capturePageDom = async ( - state: CaptureState, -): Promise> => { +const capturePageDom = async (state: CaptureState): Promise => { const { context, allowedOrigins } = state; const result = await context.connection.send( "DOMSnapshot.captureSnapshot", @@ -274,12 +274,56 @@ const capturePageDom = async ( context.sessionId, context.signal, ); - return captureDom( + const capture = captureDom( result, allowedOrigins, context.input, state.events.completeness, ); + let step = capture.next(); + let descriptionUnavailable = false; + const pendingController = new AbortController(); + const signal = + context.signal === undefined + ? pendingController.signal + : AbortSignal.any([context.signal, pendingController.signal]); + while (!step.done) { + const pending = step.value.map(async (query) => { + if (descriptionUnavailable) return undefined; + try { + return capturedSvgNodeKind( + query, + await context.connection.send( + "DOM.describeNode", + { backendNodeId: query.backendNodeId, depth: 0 }, + context.sessionId, + signal, + ), + ); + } catch (cause: unknown) { + // A detached node can disappear between snapshot and description. + // Preserve partial metadata, but propagate transport/cancellation errors. + if (!(cause instanceof CdpCommandRejection)) throw cause; + if (cause.code === -32_601) descriptionUnavailable = true; + else if (cause.code !== -32_000) throw cause; + return undefined; + } + }); + try { + // Promise.all preserves snapshot order even when CDP replies out of order. + step = capture.next(await Promise.all(pending)); + } catch (cause: unknown) { + // Stop and settle this bounded batch before the session can be detached. + pendingController.abort(); + await Promise.allSettled(pending); + throw cause; + } + } + if (step.value.unresolvedSvgNodes > 0) + state.limitations.push( + `SVG identity was unavailable for ${String(step.value.unresolvedSvgNodes)} snapshotted XLink candidate nodes; their URL destinations remain unknown.`, + ); + return step.value; }; const normalizedInspection = ( diff --git a/tests/boundary/browser/cdpDomSvgIdentity.test.ts b/tests/boundary/browser/cdpDomSvgIdentity.test.ts new file mode 100644 index 000000000..7d42ae336 --- /dev/null +++ b/tests/boundary/browser/cdpDomSvgIdentity.test.ts @@ -0,0 +1,315 @@ +import { describe, expect, it } from "vitest"; + +import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js"; +import { inspectWebPageInputSchema } from "../../../src/domain/browserObservation.js"; +import { + startFakeCdpBrowser, + type FakeCdpBrowser, + type FakeOptions, +} from "../../fixtures/fakeCdpBrowser.js"; +import { trackBrowser } from "./cdpBrowserProvider.support.js"; + +const startBrowser = async (options: FakeOptions = {}) => { + const browser = await startFakeCdpBrowser({ + ...options, + commandResult(command, origin, frameReads) { + const override = options.commandResult?.(command, origin, frameReads); + if (override !== undefined) return override; + if (command.method === "DOMSnapshot.captureSnapshot") { + const nodes = { + nodeType: [9, 1, 1, 1, 1], + nodeName: [1, 2, 3, 3, 3], + nodeValue: [4, 4, 4, 4, 4], + parentIndex: [-1, 0, 1, 1, 1], + backendNodeId: [1, 2, 3, 4, 5], + attributes: [[], [], [5, 6], [5, 7], [5, 6, 8, 9]], + }; + return { + strings: [ + `${origin}/app`, + "#document", + "HTML", + "a", + "", + "xlink:href", + "/first", + "/second", + "href", + "/ordinary", + "https://excluded.example/page", + ], + documents: [ + { documentURL: 0, nodes }, + { + documentURL: 10, + nodes: { ...nodes, backendNodeId: [11, 12, 13, 14, 15] }, + }, + ], + }; + } + if (command.method === "DOM.describeNode") + return { + node: { + backendNodeId: command.params.backendNodeId, + nodeType: 1, + nodeName: "a", + isSVG: true, + }, + }; + return undefined; + }, + }); + trackBrowser(browser); + return browser; +}; + +const inspect = (browser: FakeCdpBrowser, signal?: AbortSignal) => + new CdpBrowserProvider().inspectPage( + inspectWebPageInputSchema.parse({ + cdp_endpoint: browser.endpoint, + target_id: "allowed-page", + allowed_origins: [browser.allowedOrigin], + observation_ms: 0, + }), + signal === undefined ? undefined : { signal }, + ); + +describe("CDP SVG identity at the snapshot boundary", () => { + it("describes only in-scope XLink candidates without a winning href", async () => { + const browser = await startBrowser(); + const result = await inspect(browser); + if (!result.ok) throw result.error; + expect(result.value.metadata.dom_urls.map(({ url }) => url)).toEqual( + ["/first", "/second", "/ordinary"].map( + (path) => browser.allowedOrigin + path, + ), + ); + expect( + browser.commands + .filter(({ method }) => method === "DOM.describeNode") + .map(({ params }) => params), + ).toEqual([ + { backendNodeId: 3, depth: 0 }, + { backendNodeId: 4, depth: 0 }, + ]); + }); + + it.each(["removed", "mismatched"])( + "retains partial metadata when a candidate is %s", + async (failure) => { + const browser = await startBrowser({ + commandError: (command) => + failure === "removed" && + command.method === "DOM.describeNode" && + command.params.backendNodeId === 3 + ? { code: -32_000, message: "No node found for given backend id" } + : undefined, + commandResult: (command) => + failure === "mismatched" && + command.method === "DOM.describeNode" && + command.params.backendNodeId === 3 + ? { + node: { + backendNodeId: 99, + nodeType: 1, + nodeName: "a", + isSVG: true, + }, + } + : undefined, + }); + const result = await inspect(browser); + if (!result.ok) throw result.error; + expect(result.value.metadata.dom_urls.map(({ url }) => url)).toEqual( + ["/second", "/ordinary"].map((path) => browser.allowedOrigin + path), + ); + expect(result.value.completeness.excluded).toContainEqual({ + section: "metadata", + reason: "provider_unavailable", + count: 1, + }); + expect(result.value.limitations).toContain( + "SVG identity was unavailable for 1 snapshotted XLink candidate nodes; their URL destinations remain unknown.", + ); + }, + ); + + it("stops querying after the first batch reports an unsupported method", async () => { + const browser = await startBrowser({ + commandResult: manyCandidates, + unsupportedMethods: ["DOM.describeNode"], + }); + const result = await inspect(browser); + if (!result.ok) throw result.error; + expect(result.value.metadata.dom_urls).toEqual([]); + expect(result.value.completeness.excluded).toContainEqual({ + section: "metadata", + reason: "provider_unavailable", + count: 25, + }); + expect(descriptions(browser).length).toBeGreaterThan(0); + expect(descriptions(browser).length).toBeLessThanOrEqual(8); + }); + + it.each([ + { options: { closeOnMethod: "DOM.describeNode" }, reason: "disconnected" }, + { + options: { + commandError: (command: { method: string }) => + command.method === "DOM.describeNode" + ? { code: -32_602, message: "Invalid parameters" } + : undefined, + }, + reason: "protocol_error", + }, + ])( + "propagates $reason during node description", + async ({ options, reason }) => { + const browser = await startBrowser(options); + expect(await inspect(browser)).toMatchObject({ + ok: false, + error: { _tag: "BrowserObservationError", reason }, + }); + }, + ); + + it("cancels a pending description and detaches without closing the target", async () => { + const browser = await startBrowser({ hangOnMethod: "DOM.describeNode" }); + const controller = new AbortController(); + const pending = inspect(browser, controller.signal); + await expect.poll(() => descriptions(browser).length).toBeGreaterThan(1); + controller.abort(); + expect(await pending).toMatchObject({ + ok: false, + error: { _tag: "AnalysisCancelledError", operation: "inspect_web_page" }, + }); + const methods = browser.commands.map(({ method }) => method); + expect(methods).toContain("Target.detachFromTarget"); + expect(methods).not.toContain("Target.closeTarget"); + expect(methods).not.toContain("Browser.close"); + }); +}); + +const manyCandidates: NonNullable = ( + command, + origin, +) => { + if (command.method !== "DOMSnapshot.captureSnapshot") return undefined; + const count = 25; + return { + strings: [ + origin, + "#document", + "HTML", + "a", + "", + "xlink:href", + ...Array.from({ length: count }, (_, index) => `/link-${String(index)}`), + ], + documents: [ + { + documentURL: 0, + nodes: { + nodeType: [9, 1, ...Array(count).fill(1)], + nodeName: [1, 2, ...Array(count).fill(3)], + nodeValue: Array(count + 2).fill(4), + parentIndex: [-1, 0, ...Array(count).fill(1)], + backendNodeId: Array.from( + { length: count + 2 }, + (_, index) => index + 1, + ), + attributes: [ + [], + [], + ...Array.from({ length: count }, (_, index) => [5, index + 6]), + ], + }, + }, + ], + }; +}; + +const descriptions = (browser: FakeCdpBrowser) => + browser.commands.filter(({ method }) => method === "DOM.describeNode"); + +describe("CDP SVG query concurrency", () => { + it("bounds in-flight reads and preserves metadata order across reversed replies", async () => { + const browser = await startBrowser({ + commandResult: manyCandidates, + hangOnMethod: "DOM.describeNode", + }); + const controller = new AbortController(); + const pending = inspect(browser, controller.signal); + try { + await expect.poll(() => descriptions(browser).length).toBeGreaterThan(1); + let replied = 0; + while (replied < 25) { + await expect + .poll(() => descriptions(browser).length) + .toBeGreaterThan(replied); + const batch = descriptions(browser).slice(replied); + expect(batch.length).toBeLessThanOrEqual(8); + for (const command of [...batch].reverse()) { + const id = Number(command.params.backendNodeId); + browser.emitRawMessage( + JSON.stringify({ + id: command.id, + result: { + node: { + backendNodeId: id, + nodeType: 1, + nodeName: "a", + isSVG: id % 2 === 1, + }, + }, + }), + ); + } + replied += batch.length; + } + const result = await pending; + if (!result.ok) throw result.error; + expect(result.value.metadata.dom_urls.map(({ url }) => url)).toEqual( + Array.from( + { length: 13 }, + (_, index) => `${browser.allowedOrigin}/link-${String(index * 2)}`, + ), + ); + expect(descriptions(browser)).toHaveLength(25); + } finally { + controller.abort(); + await pending; + } + }); + + it("cancels sibling reads on a fatal reply without waiting for hung nodes", async () => { + const browser = await startBrowser({ + commandResult: manyCandidates, + hangOnMethod: "DOM.describeNode", + }); + const controller = new AbortController(); + const pending = inspect(browser, controller.signal); + try { + await expect.poll(() => descriptions(browser).length).toBeGreaterThan(1); + const batch = descriptions(browser); + browser.emitRawMessage( + JSON.stringify({ + id: batch.at(-1)?.id, + error: { code: -32_602, message: "Invalid parameters" }, + }), + ); + await expect(pending).resolves.toMatchObject({ + ok: false, + error: { _tag: "BrowserObservationError", reason: "protocol_error" }, + }); + expect(descriptions(browser)).toHaveLength(batch.length); + const methods = browser.commands.map(({ method }) => method); + expect(methods).toContain("Target.detachFromTarget"); + expect(methods).not.toContain("Target.closeTarget"); + expect(methods).not.toContain("Browser.close"); + } finally { + controller.abort(); + await pending; + } + }); +}); diff --git a/tests/boundary/browser/cdpSvgXlinkHref.test.ts b/tests/boundary/browser/cdpSvgXlinkHref.test.ts index 56c3fc3cf..38048861b 100644 --- a/tests/boundary/browser/cdpSvgXlinkHref.test.ts +++ b/tests/boundary/browser/cdpSvgXlinkHref.test.ts @@ -45,8 +45,38 @@ const pageHtml = `x out + move desc + move title + + not an SVG link + HTML link + nested + + + <a xlink:href="/xl-html-title">not an SVG link</a> + <a href="/html-title">HTML link</a> + <svg><a xlink:href="/xl-nested-title"><text>nested</text></a></svg> + html + `; it.skipIf(chromeExecutable === undefined)( @@ -143,17 +173,36 @@ it.skipIf(chromeExecutable === undefined)( url: null, destination_scope: "outside_policy", }), + ...[ + "/html-desc", + "/html-title", + "/xl-nested-desc", + "/xl-nested-title", + "/xl-moved-desc", + "/xl-moved-title", + "/xl-created-desc", + "/xl-created-title", + "/xl-created-nested-desc", + "/xl-created-nested-title", + ].map((path) => + expect.objectContaining({ + attribute: "href", + url: `${origin}${path}`, + destination_scope: "approved", + }), + ), ]), ); expect( urls.some( (url) => - url.url?.endsWith("/xl-html") === true || + url.url?.includes("/xl-html") === true || + url.url?.includes("/xl-math") === true || url.url?.endsWith("/xlink-loses") === true, ), ).toBe(false); expect(urls.every((url) => url.attribute === "href")).toBe(true); - expect(urls).toHaveLength(5); + expect(urls).toHaveLength(15); } finally { child.kill("SIGTERM"); if (!(await waitForExit(child, 2_000))) child.kill("SIGKILL"); @@ -173,6 +222,8 @@ const xhtmlPage = ` p + desc + title no no @@ -202,6 +253,8 @@ for (const { format, page, contentType, nodeNames, link, destinations } of [ destinations: [ "/xl-pa1", "/xl-image", + "/xl-xml-desc", + "/xl-xml-title", "/xl-fo-svg", "/xl-orphan", "/xl-custom", From dd4bfed6141efcb57aef81cf653962272efdf866 Mon Sep 17 00:00:00 2001 From: Jin Date: Tue, 13 Oct 2026 09:23:58 -0400 Subject: [PATCH 15/24] fix(javascript): keep ASAR-named files without an ASAR header opaque (#2347) Inventory expands an ASAR-named filesystem file only when its bytes are an ASAR. JavaScript reconstruction expanded it on the suffix alone, so a directory holding any other file named *.asar failed the whole analysis with an invalid ASAR header. Co-authored-by: Claude Opus 5.5 Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- .../javascript/JavaScriptArtifactFiles.ts | 6 ++++ .../javascriptIntegrityPolicy.test.ts | 28 +++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/src/artifacts/javascript/JavaScriptArtifactFiles.ts b/src/artifacts/javascript/JavaScriptArtifactFiles.ts index e3849f814..78d8cf750 100644 --- a/src/artifacts/javascript/JavaScriptArtifactFiles.ts +++ b/src/artifacts/javascript/JavaScriptArtifactFiles.ts @@ -37,6 +37,7 @@ interface ExpectedFile { interface ExpectedContainer { readonly hash_status: ArtifactOccurrence["hash_status"]; + readonly format: ArtifactOccurrence["artifact_format"]; readonly inventory: JavaScriptArtifactContainer | null; } @@ -219,6 +220,9 @@ const visitReader = async ( "integrity", `Nested ASAR was not integrity-verified: ${path}`, ); + // Inventory expands an ASAR-named file only when its bytes are an + // ASAR. One it recorded as ordinary bytes has no members to read. + if (container.format !== "asar") continue; const inventory = container.inventory; if (inventory === null) throw new ArtifactReaderFailure( @@ -395,6 +399,7 @@ const expectedInventory = ( if (isNestedAsar) containers.set(occurrence.logical_path, { hash_status: occurrence.hash_status, + format: occurrence.artifact_format, inventory: null, }); continue; @@ -412,6 +417,7 @@ const expectedInventory = ( if (isNestedAsar) containers.set(occurrence.logical_path, { hash_status: occurrence.hash_status, + format: occurrence.artifact_format, inventory: { path: occurrence.logical_path, sha256: node.sha256, diff --git a/tests/boundary/filesystem/javascriptIntegrityPolicy.test.ts b/tests/boundary/filesystem/javascriptIntegrityPolicy.test.ts index 47b77114a..67b167bdf 100644 --- a/tests/boundary/filesystem/javascriptIntegrityPolicy.test.ts +++ b/tests/boundary/filesystem/javascriptIntegrityPolicy.test.ts @@ -105,6 +105,34 @@ it("keeps a contradicted nested ASAR opaque during JavaScript reconstruction", a expect(result.graph.coverage.omitted_count).toBeNull(); }); +it("keeps an ASAR-named file without an ASAR header opaque during JavaScript reconstruction", async () => { + const root = await createTestTempDirectory("rea-opaque-asar-name-"); + await writeFile(join(root, "main.js"), "export const outer = true;\n"); + // Inventory records this file as ordinary bytes and does not expand it. + await writeFile(join(root, "ordinary.asar"), "ordinary bytes"); + + const snapshot = await scanCanonicalArtifactInventory(root, {}); + const reader = createJavaScriptArtifactReader(root, "directory"); + try { + const read = await readJavaScriptArtifactFiles(reader, snapshot); + expect(read.files.map(({ path }) => path)).toEqual(["main.js"]); + expect(read.containers).toEqual([]); + } finally { + await reader.close(); + } + + const analyzed = await analyzeJavaScriptApplication({ + input_path: root, + format: "directory", + }); + if (!analyzed.ok) throw analyzed.error; + const result = javascriptApplicationAnalysisResultSchema.parse( + analyzed.value.normalized_result, + ); + expect(result.statistics.nested_asar_containers).toBe(0); + expect(result.integrity_contradictions).toEqual([]); +}); + it("retains semantic resource limits alongside recorded integrity contradictions", async () => { const { archive } = await createStrippedAsarAddon([ { path: "app.js", contents: primitiveCandidateExpansionSource() }, From 003707f712d06e49b84a48e04d58b50f9a097469 Mon Sep 17 00:00:00 2001 From: Luis <86675342+LuisTaffelli@users.noreply.github.com> Date: Tue, 13 Oct 2026 10:24:40 -0300 Subject: [PATCH 16/24] fix(verification): align Linux Hopper setup lane with current contracts (#1942) * fix(verification): align Linux Hopper setup lane with current contracts * fix(verification): retain orphan checks on clean Linux runners * fix(verification): detect Hopper through alternate launcher paths * fix(verification): report identified Hopper leaks before proc errors * fix(verification): distinguish Hopper string recognition from byte fidelity * fix(verification): report full literal decoding coverage explicitly * test(verification): require a clean host before orphan checks --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- tests/process-global/cleanLinuxHopperRuntime.test.ts | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/process-global/cleanLinuxHopperRuntime.test.ts b/tests/process-global/cleanLinuxHopperRuntime.test.ts index d356f5abf..f6b9fc677 100644 --- a/tests/process-global/cleanLinuxHopperRuntime.test.ts +++ b/tests/process-global/cleanLinuxHopperRuntime.test.ts @@ -30,6 +30,9 @@ it await chmod(launcher, 0o700); await symlink(launcher, alternate); } + // A rejection must come from the fixture we are about to launch. An + // existing Xvfb would otherwise mask a broken executable-identity check. + await assertCleanLinuxHopperRuntime(alternateMode ? [launcher] : []); const command = alternateMode ? alternate : process.execPath; const arguments_ = alternateMode ? ["30"] : ["-e", childCode]; const parentCode = ` From 48b92e1be1110fd200cc6d2fd0cdf7c6e2f9e21a Mon Sep 17 00:00:00 2001 From: Joshwong1908 <136307740+Joshwong1908@users.noreply.github.com> Date: Tue, 13 Oct 2026 21:25:32 +0800 Subject: [PATCH 17/24] fix(go): classify JVM class files instead of universal Mach-O for CAFEBABE bytes (#2458) * test(go): pin JVM class files as unsupported Go images The regression fails on main: readGoBinaryImage(javaClassFile()) reports "Universal/fat Mach-O containers require selecting a thin slice" for a JVM class file whose bytes 4..7 are minor/major version, not nfat_arch. * fix(go): classify JVM class files instead of universal Mach-O Route the four FAT magics through the shared readFatContainerHeader and classifyFatContainerIdentity facts added for the same disambiguation in the artifact and target lanes (921bb9575). Keep the thin-slice failure only when the container identity is confirmed Mach-O; a JVM class file now falls through to the generic only-ELF/PE/thin-Mach-O failure. * test(go): pin universal Mach-O thin-slice failure with FAT fixtures Cover the kept behavior with the repository's own fatImage/machoImage builders (narrow and FAT64 tables) so the regression also proves real universal containers keep the slice-selection failure. * test(go): cover little-endian FAT magics in the universal container control * fix(go): require positive universal Mach-O identity Require positive Mach-O identity before reporting a universal thin-slice remedy; malformed FAT-shaped bytes are not proof. Add the JVM constant-pool CPU lookalike to the existing fixture control. Source inspection and git diff --check; tests/builds/typechecks/CI reruns skipped at explicit user request. * fix(go): retain malformed FAT header diagnostics Preserve the owning FAT header parser reason for malformed headers while requiring positive Mach-O proof before the universal-slice remedy. Source inspection and git diff --check only. Tests, builds and typechecks skipped by explicit user request. GitHub review writes paused globally. --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- src/go/GoBinaryImage.ts | 30 +++++++++++++++++---- tests/boundary/go/goBinaryImage.test.ts | 35 +++++++++++++++++++++++++ tests/fixtures/javaClassFile.ts | 3 ++- 3 files changed, 62 insertions(+), 6 deletions(-) diff --git a/src/go/GoBinaryImage.ts b/src/go/GoBinaryImage.ts index 03d0866ab..4a845c197 100644 --- a/src/go/GoBinaryImage.ts +++ b/src/go/GoBinaryImage.ts @@ -8,6 +8,10 @@ import { type GoBinaryContainer, type GoFileMapping, } from "./GoBinaryContainer.js"; +import { + classifyFatContainerIdentity, + readFatContainerHeader, +} from "../domain/apple/machoContainer.js"; import { readGoElfImage } from "./GoElfImage.js"; import { readGoMachoImage } from "./GoMachoImage.js"; import { readGoPeImage } from "./GoPeImage.js"; @@ -187,11 +191,27 @@ const readContainer = (bytes: Buffer): GoBinaryContainer => { return readGoPeImage(bytes); if (bytes.length >= 4) { const magic = bytes.readUInt32BE(); - if ([0xcafebabe, 0xcafebabf, 0xbebafeca, 0xbfbafeca].includes(magic)) - throw new GoBinaryFormatFailure( - "unsupported", - "Universal/fat Mach-O containers require selecting a thin slice", - ); + if ([0xcafebabe, 0xcafebabf, 0xbebafeca, 0xbfbafeca].includes(magic)) { + const fatHeader = readFatContainerHeader(bytes, bytes.length); + if (fatHeader.state === "malformed") + throw new GoBinaryFormatFailure("malformed", fatHeader.reason); + const identity = + fatHeader.state === "parsed" + ? classifyFatContainerIdentity(fatHeader, (offset) => { + if (offset >= bytes.length) return undefined; + if (offset + 4 > bytes.length) return null; + return bytes.readUInt32BE(offset); + }) + : undefined; + // An invalid FAT-shaped table is not proof of a Mach-O container: JVM + // constant-pool bytes can share both the magic and a known CPU value. + const confirmedFatMachO = identity?.state === "mach-o"; + if (confirmedFatMachO) + throw new GoBinaryFormatFailure( + "unsupported", + "Universal/fat Mach-O containers require selecting a thin slice", + ); + } if ([0xfeedface, 0xfeedfacf, 0xcefaedfe, 0xcffaedfe].includes(magic)) return readGoMachoImage(bytes); } diff --git a/tests/boundary/go/goBinaryImage.test.ts b/tests/boundary/go/goBinaryImage.test.ts index 24321ecc8..29f0e952c 100644 --- a/tests/boundary/go/goBinaryImage.test.ts +++ b/tests/boundary/go/goBinaryImage.test.ts @@ -1,4 +1,10 @@ import { expect, it } from "vitest"; +import { javaClassFile } from "../../fixtures/javaClassFile.js"; +import { + CPU, + fatImage, + machoImage, +} from "../../../src/artifacts/apple/MachoImage.fixture.js"; import { readGoBinaryImage } from "../../../src/go/GoBinaryImage.js"; import { createGoBinaryFixture, @@ -221,6 +227,35 @@ it("distinguishes unsupported containers from malformed recognized images", () = expect(() => readGoBinaryImage(pe.bytes)).toThrowError(/range|truncated/i); }); +it("reports JVM class files as unsupported images instead of fat Mach-O containers", () => { + expect(() => readGoBinaryImage(javaClassFile())).toThrowError( + /Only ELF, PE, and thin Mach-O native images are supported/, + ); + expect(() => readGoBinaryImage(javaClassFile(4000))).toThrowError( + /Only ELF, PE, and thin Mach-O native images are supported/, + ); + expect(() => readGoBinaryImage(javaClassFile(4000, true))).toThrowError( + /Only ELF, PE, and thin Mach-O native images are supported/, + ); +}); + +it("keeps the thin-slice failure for confirmed universal Mach-O containers", () => { + const slices = [ + { cpu: CPU.x86_64, bytes: machoImage({ cpu: CPU.x86_64 }) }, + { cpu: CPU.arm64, bytes: machoImage({ cpu: CPU.arm64 }) }, + ]; + for (const universal of [ + fatImage(slices), + fatImage(slices, true), + fatImage(slices, false, true), + fatImage(slices, true, true), + ]) { + expect(() => readGoBinaryImage(Buffer.from(universal))).toThrowError( + /fat Mach-O containers require selecting a thin slice/, + ); + } +}); + it("rejects ELF header tables overlapping the native header", () => { const fixture = createGoBinaryFixture({ sectionless: true }); fixture.bytes.writeBigUInt64LE(0n, 32); diff --git a/tests/fixtures/javaClassFile.ts b/tests/fixtures/javaClassFile.ts index 4f7727193..30f5ffc37 100644 --- a/tests/fixtures/javaClassFile.ts +++ b/tests/fixtures/javaClassFile.ts @@ -1,5 +1,5 @@ /** Build a valid minimal JVM class file, optionally with an unused UTF-8 constant. */ -export const javaClassFile = (extraUtf8Bytes = 0): Buffer => { +export const javaClassFile = (extraUtf8Bytes = 0, fatCpuLookalike = false): Buffer => { const utf8 = (value: string): Buffer => { const bytes = Buffer.from(value, "utf8"); const entry = Buffer.alloc(3 + bytes.length); @@ -13,6 +13,7 @@ export const javaClassFile = (extraUtf8Bytes = 0): Buffer => { utf8("Hello"), Buffer.from([7, 0, 4]), utf8("java/lang/Object"), + ...(fatCpuLookalike ? [utf8("x"), Buffer.from([3, 0, 0, 0, 7])] : []), ...(extraUtf8Bytes === 0 ? [] : [utf8("x".repeat(extraUtf8Bytes))]), ]; const headerAndPool = Buffer.alloc(10); From dea770d5d9e4af92bc3fa6a8ffad9ff414f6e4e6 Mon Sep 17 00:00:00 2001 From: To3akaRin <131595336+To3akaRin@users.noreply.github.com> Date: Tue, 13 Oct 2026 21:26:13 +0800 Subject: [PATCH 18/24] fix(javascript): retain getter receivers through local aliases (#2778) * fix(javascript): retain getter receivers through local aliases * fix(javascript): resolve projected private getter receivers --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- docs/javascript-application-workflows.md | 12 + ...iptSemanticAnalysis.getter_aliases.test.ts | 351 +++++++++++++++ .../javascriptSemanticMemberMutations.ts | 69 +-- .../javascriptSemanticMutationInitializers.ts | 61 ++- .../javascriptSemanticObjectReferences.ts | 424 +++++++++++++++--- .../javascriptSemanticReferenceSources.ts | 53 +++ .../getterLocalAliasesCli.test.ts | 85 ++++ 7 files changed, 942 insertions(+), 113 deletions(-) create mode 100644 src/domain/javascript/javascriptSemanticAnalysis.getter_aliases.test.ts create mode 100644 src/domain/javascript/javascriptSemanticReferenceSources.ts create mode 100644 tests/acceptance/applications/getterLocalAliasesCli.test.ts diff --git a/docs/javascript-application-workflows.md b/docs/javascript-application-workflows.md index 041015b9c..e9fed813f 100644 --- a/docs/javascript-application-workflows.md +++ b/docs/javascript-application-workflows.md @@ -208,6 +208,18 @@ arrays, and later explicit data properties and methods replace earlier object spread references. A known array prefix does not expose values from a following spread. Accessors, custom or uncertain iteration, unknown computed keys, and unknown spread lengths retain conservative uncertainty. +Getter-local aliases of `this`, including direct destructuring and assignments, +retain their receiver when a returned shared reference is written. These writes +invalidate the earlier literal instead of claiming the getter executed or the +new value is known. Straight-line alias reads retain their capture order; +uncertain control flow remains conservative. A binding outside the getter or +`this` in a nested ordinary function is not attributed to the getter's receiver. +Private reads keep receiver projections separate from the private value's path. +Receiver projections follow known data fields initialized from `this` or earlier +self-referencing fields; static fields retain their declaring class identity. +An alias cycle that repeatedly selects deeper children retains escape uncertainty +at its recovered roots rather than claiming a fixed number of loop iterations. + Writes through an object getter's returned receiver retain that uncertainty when the object is an array element, nested inside mixed object/array containers, or assigned to a previously declared identifier. Saving the getter diff --git a/src/domain/javascript/javascriptSemanticAnalysis.getter_aliases.test.ts b/src/domain/javascript/javascriptSemanticAnalysis.getter_aliases.test.ts new file mode 100644 index 000000000..1eeaec350 --- /dev/null +++ b/src/domain/javascript/javascriptSemanticAnalysis.getter_aliases.test.ts @@ -0,0 +1,351 @@ +import { describe, expect, it } from "vitest"; +import { analyzeJavaScriptSemantics } from "./javascriptSemanticAnalysis.js"; +import { onlyCallable } from "./javascriptSemanticAnalysis.fixture.js"; + +const returnedValue = (getter: string, write = "box.v.x = 2;") => + onlyCallable( + analyzeJavaScriptSemantics(`export function result() { + const shared = { x: 1, y: 1 }; + const box = { s: shared, get v() { ${getter} } }; + ${write} + return shared.x; + }`), + "result", + ).returnSites[0]?.value; + +describe("projected private getter receivers", () => { + it.each([ + [ + "destructured self field", + "shared", + "peer = this;", + "const { peer: self } = this; return self.#s;", + true, + "unknown", + ], + [ + "unrelated private child", + "{ peer: shared }", + "peer = this;", + "const { peer: self } = this; return self.#s;", + true, + "literal", + ], + [ + "read without write", + "shared", + "peer = this;", + "const { peer: self } = this; return self.#s;", + false, + "literal", + ], + [ + "two self fields", + "shared", + "peer = this; next = this.peer;", + "const { next: self } = this; return self.#s;", + true, + "unknown", + ], + [ + "nested receiver projection", + "shared", + "peer = this;", + "const { peer: { peer: self } } = this; return self.#s;", + true, + "unknown", + ], + [ + "member receiver alias", + "shared", + "peer = this;", + "const self = this.peer.peer; return self.#s;", + true, + "unknown", + ], + [ + "projected result suffix", + "{ child: shared }", + "peer = this;", + "const { peer: self } = this; return self.#s.child;", + true, + "unknown", + ], + [ + "external receiver", + "shared", + "peer = external;", + "const { peer: self } = this; return self.#s;", + true, + "literal", + ], + [ + "field receiver cycle", + "shared", + "peer = this.other; other = this.peer;", + "const { peer: self } = this; return self.#s;", + true, + "literal", + ], + ] as const)( + "keeps receiver and result paths separate for %s", + (_name, value, fields, getter, write, status) => { + const ir = analyzeJavaScriptSemantics(`export function result() { + const shared = { x: 1 }; + class Box { + #s = ${value}; ${fields} + get v() { ${getter} } + } + const box = new Box(); + ${write ? "box.v.x = 2;" : "const saved = box.v;"} + return shared.x; + }`); + expect(onlyCallable(ir, "result").returnSites[0]?.value).toMatchObject( + status === "literal" ? { status, value: 1 } : { status }, + ); + }, + ); +}); + +describe("private receiver field ownership", () => { + it("bounds growing receiver projections under cyclic class inheritance", () => { + const ir = analyzeJavaScriptSemantics(`export function result() { + const shared = { x: 1 }; + class A extends B { + #s = shared; + peer = this.other.other; + get v() { const { peer: self } = this; return self.#s; } + } + class B extends A { other = this.peer.peer; } + new A().v.x = 2; + return shared.x; + }`); + expect(onlyCallable(ir, "result").returnSites[0]?.value).toMatchObject({ + status: "literal", + value: 1, + }); + }, 1000); + + it.each([ + [ + "own static receiver", + "class Box { static #s=shared; static peer=this; static get v(){const {peer:self}=this;return self.#s;} } Box.v.x=2;", + "unknown", + ], + [ + "inherited instance receiver", + "class Base {peer=this;} class Box extends Base {#s=shared;get v(){const {peer:self}=this;return self.#s;}} new Box().v.x=2;", + "unknown", + ], + [ + "inherited static field retains its declaring class", + "class Base {static peer=this;} class Box extends Base {static #s=shared;static get v(){const {peer:self}=this;return self.#s;}} Box.v.x=2;", + "literal", + ], + [ + "inherited static getter's own private field", + "class Base {static #s=shared;static peer=this;static get v(){const {peer:self}=this;return self.#s;}} class Box extends Base {} Box.v.x=2;", + "unknown", + ], + [ + "a field that is not initialized yet", + "class Box {#s=shared;peer=this.later;later=this;get v(){const {peer:self}=this;return self.#s;}} new Box().v.x=2;", + "literal", + ], + [ + "remaining getter projection after a captured field", + "class Box {#s=shared;peer=this;next=this.peer;later=this;get v(){const {next:{later:self}}=this;return self.#s;}} new Box().v.x=2;", + "unknown", + ], + [ + "a replaced self field", + "class Box {#s=shared;peer=this;peer=external;get v(){const {peer:self}=this;return self.#s;}} new Box().v.x=2;", + "literal", + ], + ] as const)("respects %s", (_name, body, status) => { + const ir = analyzeJavaScriptSemantics(`export function result() { + const shared = { x: 1 }; + ${body} + return shared.x; + }`); + expect(onlyCallable(ir, "result").returnSites[0]?.value).toMatchObject( + status === "literal" ? { status, value: 1 } : { status }, + ); + }); +}); + +describe("getter-local receiver aliases (#2653)", () => { + it.each([ + ["initialized alias", "const self = this; return self.s;"], + ["assigned alias", "let self; self = this; return self.s;"], + [ + "chained assignment", + "let self; let alias; self = alias = this; return self.s;", + ], + ["destructured receiver", "const { s: value } = this; return value;"], + ["alias chain", "const self = this; const saved = self; return saved.s;"], + [ + "capture before reassignment", + "let self = this; const saved = self; self = { s: { x: 1 } }; return saved.s;", + ], + [ + "conditional receiver alias", + "const self = flag ? this : { s: { x: 1 } }; return self.s;", + ], + ])("invalidates shared values through %s", (_name, getter) => { + expect(returnedValue(getter)?.status).toBe("unknown"); + }); + + it.each([ + ["no write", "const self = this; return self.s;", "const saved = box.v;"], + [ + "an unrelated property write", + "const self = this; return self.s;", + "box.v.y = 2;", + ], + [ + "a cycle that does not grow the selected path", + "let self = this; while (flag) self = self; return self.s;", + "box.v.y = 2;", + ], + [ + "an alias replaced before return", + "let self = this; self = { s: { x: 1 } }; return self.s;", + "box.v.x = 2;", + ], + [ + "a shadowing block binding", + "const self = this; { const self = { s: { x: 1 } }; return self.s; }", + "box.v.x = 2;", + ], + [ + "a nested function's receiver", + "function other() { const self = this; return self.s; } return other();", + "box.v.x = 2;", + ], + [ + "an alias assigned in a different receiver scope", + "let self; function assign() { self = this; } assign(); return self.s;", + "box.v.x = 2;", + ], + [ + "an excluded destructuring default", + "const { value = shared } = { value: { x: 1 } }; return value;", + "box.v.x = 2;", + ], + ])("preserves known values with %s", (_name, getter, write) => { + expect(returnedValue(getter, write)).toMatchObject({ + status: "literal", + value: 1, + }); + }); +}); + +describe("getter receiver alias boundaries", () => { + it("does not reinterpret an outer this binding as the getter receiver", () => { + const ir = analyzeJavaScriptSemantics(`export function result() { + const shared = { x: 1 }; + const outside = this; + const box = { s: shared, get v() { return outside.s; } }; + box.v.x = 2; + return shared.x; + }`); + expect(onlyCallable(ir, "result").returnSites[0]?.value).toMatchObject({ + status: "literal", + value: 1, + }); + }); + + it("retains a private field's lexical owner through a receiver alias", () => { + const ir = analyzeJavaScriptSemantics(`export function result() { + const shared = { x: 1 }; + class Box { + #s = shared; + get v() { const self = this; return self.#s; } + } + new Box().v.x = 2; + return shared.x; + }`); + expect(onlyCallable(ir, "result").returnSites[0]?.value.status).toBe( + "unknown", + ); + }); + + it("does not append a receiver projection to a private field's result path", () => { + const ir = analyzeJavaScriptSemantics(`export function result() { + const shared = { x: 1 }; + class Box { + #s = { peer: shared }; + peer = this; + get v() { const { peer: self } = this; return self.#s; } + } + new Box().v.x = 2; + return shared.x; + }`); + expect(onlyCallable(ir, "result").returnSites[0]?.value).toMatchObject({ + status: "literal", + value: 1, + }); + }); + + it("resolves a long local alias chain without recursive expansion", () => { + const aliases = ["const a0 = this;"]; + for (let index = 1; index <= 2000; index++) + aliases.push(`const a${String(index)} = a${String(index - 1)};`); + expect(returnedValue(`${aliases.join("\n")} return a2000.s;`)?.status).toBe( + "unknown", + ); + }); + + it("bounds a growing alias cycle without losing the original receiver", () => { + expect( + returnedValue( + "let self = this; while (flag) self = self.s; return self.s;", + )?.status, + ).toBe("unknown"); + }); + + it.each([true, false])( + "retains deeper shared references through an alias cycle (write=%s)", + (write) => { + const ir = analyzeJavaScriptSemantics(`export function result() { + const shared = { x: 1 }; + const box = { + start: { child: { child: shared } }, + get v() { + let self = this.start; + while (flag) self = self.child; + return self; + } + }; + ${write ? "box.v.x = 2;" : "const saved = box.v;"} + return shared.x; + }`); + expect(onlyCallable(ir, "result").returnSites[0]?.value).toMatchObject( + write ? { status: "unknown" } : { status: "literal", value: 1 }, + ); + }, + ); + + it("does not widen a growing cycle to unrelated receiver properties", () => { + const ir = analyzeJavaScriptSemantics(`export function result() { + const shared = { x: 1 }; + const unrelated = { x: 1 }; + const box = { + start: { child: { child: shared } }, + unrelated, + get v() { + let self = this.start; + while (flag) self = self.child; + return self; + } + }; + box.v.x = 2; + return unrelated.x; + }`); + expect(onlyCallable(ir, "result").returnSites[0]?.value).toMatchObject({ + status: "literal", + value: 1, + }); + }); +}); diff --git a/src/domain/javascript/javascriptSemanticMemberMutations.ts b/src/domain/javascript/javascriptSemanticMemberMutations.ts index 35b63c815..7b988af88 100644 --- a/src/domain/javascript/javascriptSemanticMemberMutations.ts +++ b/src/domain/javascript/javascriptSemanticMemberMutations.ts @@ -10,10 +10,7 @@ import { evaluateSemanticBinding, evaluateSemanticExpression, } from "./javascriptSemanticValues.js"; -import { - semanticContainer, - semanticSlotAtPath, -} from "./javascriptSemanticSlots.js"; +import { semanticContainer } from "./javascriptSemanticSlots.js"; import { semanticClassCandidates, semanticClassPropertyReferences, @@ -24,6 +21,10 @@ import { semanticIterationSources } from "./javascriptSemanticIterationSources.j import { semanticCallableResultExpressions } from "./javascriptSemanticCallableResults.js"; import { createSemanticReturnedReferences } from "./javascriptSemanticReturnedReferences.js"; import { semanticMutationInitializers } from "./javascriptSemanticMutationInitializers.js"; +import { + semanticReferenceSourcesInactive, + semanticSelectedReferenceSlot, +} from "./javascriptSemanticReferenceSources.js"; import { semanticArrayIndex, SemanticPropertyPathCoverage, @@ -329,7 +330,17 @@ export function* collectSemanticMemberMutationsSteps( for (const value of effect === "escape" ? escapedReferenceLeaves(expression, current.path) : expandedValues(expression, current.path, effect)) { - if (value.iterableFallbackPath !== undefined) + if (value.escapes && effect !== "escape") + markValue( + value.node, + value.path, + new Set(), + "escape", + mutation, + current.originAt, + value.ownerInitializer, + ); + else if (value.iterableFallbackPath !== undefined) deferIterable({ node: value.node, projection: [], @@ -504,7 +515,7 @@ export function* collectSemanticMemberMutationsSteps( for (const [index, reference] of pendingReferences.splice(0).entries()) { if (index % 64 === 0) yield; const { initializer } = reference; - if (referenceSourcesInactive(initializer, state)) + if (semanticReferenceSourcesInactive(initializer, state)) pendingReferences.push(reference); else if (t.isForOfStatement(initializer.node)) deferIterable({ @@ -549,7 +560,7 @@ export function* collectSemanticMemberMutationsSteps( } for (const [index, reference] of [...iterableReferences].entries()) { if (index % 64 === 0) yield; - if (referenceSourcesInactive(reference, state)) continue; + if (semanticReferenceSourcesInactive(reference, state)) continue; if (reference.iterationOnly) { const expanded = expandedIterationReferences(reference, state); if (expanded !== null) { @@ -557,7 +568,7 @@ export function* collectSemanticMemberMutationsSteps( continue; } } - const slot = selectedReferenceSlot(reference, state); + const slot = semanticSelectedReferenceSlot(reference, state); // Custom iteration can yield the iterable itself as well as any child. // Treat its root as escaped, including writes through yielded references. const knownArray = @@ -610,27 +621,6 @@ interface IterableReference extends ReferencedValue { }[]; } -const referenceSourcesInactive = ( - reference: Pick, - state: JavaScriptSemanticAnalysisState, -): boolean => - reference.requiredSources?.some( - (source) => - selectedReferenceSlot(source, state, source.node)?.presence === "absent", - ) === true || - reference.fallbackSources?.some((source) => { - // Selection precedes effects through the chosen reference. A later escape - // cannot make a fallback that was excluded at destructuring reachable. - const slot = selectedReferenceSlot(source, state, source.node); - return ( - slot?.presence === "present" && - (slot.value.status === "literal" || - slot.value.status === "union" || - slot.value.status === "object" || - slot.value.status === "array") - ); - }) === true; - const expandedIterationReferences = ( reference: IterableReference, state: JavaScriptSemanticAnalysisState, @@ -645,7 +635,10 @@ const expandedIterationReferences = ( ...(reference.fallbackSources ?? []), ].find((source) => source.node === iterable && source.projection[0] === null); if (source === undefined) return null; - const slot = selectedReferenceSlot({ node: iterable, projection: [] }, state); + const slot = semanticSelectedReferenceSlot( + { node: iterable, projection: [] }, + state, + ); const container = slot === undefined ? null : semanticContainer(slot.value); if (container?.coverage.status !== "complete") return null; const select = ( @@ -1141,19 +1134,3 @@ const copiedReferencePath = ( } return [...initializer.projection.slice(0, offset), key, ...remaining]; }; - -const selectedReferenceSlot = ( - source: { readonly node: t.Node; readonly projection: PropertyPath }, - state: JavaScriptSemanticAnalysisState, - capturePoint?: t.Node, -) => { - const path: string[] = []; - for (const key of source.projection) { - if (key === null || typeof key === "object") return undefined; - path.push(String(key)); - } - return semanticSlotAtPath( - evaluateSemanticExpression(source.node, state, capturePoint), - path, - ); -}; diff --git a/src/domain/javascript/javascriptSemanticMutationInitializers.ts b/src/domain/javascript/javascriptSemanticMutationInitializers.ts index cfaaf0ef5..6de7217c1 100644 --- a/src/domain/javascript/javascriptSemanticMutationInitializers.ts +++ b/src/domain/javascript/javascriptSemanticMutationInitializers.ts @@ -149,7 +149,50 @@ export const semanticMutationInitializers = ( const mutationPosition = directMutationPosition(mutation, parents) ?? referenceInitializerPosition(mutation, parents); - if (mutationPosition === null) return binding; + return selectInitializers(binding, mutationPosition, parents); +}; + +/** Select a local alias at its read, before any later straight-line reassignment. */ +export const semanticReadInitializers = ( + binding: JavaScriptSemanticBindingState, + read: t.Node, + parents: WeakMap, +): Pick< + JavaScriptSemanticBindingState, + "initializers" | "referenceInitializers" +> => { + let position: StatementPosition | null = null; + let current = read; + for ( + let parent = parents.get(current); + parent; + parent = parents.get(current) + ) { + if (t.isLoop(parent)) return binding; + if (t.isFunction(parent)) break; + if (position === null) { + if (t.isVariableDeclarator(parent)) { + const declaration = parents.get(parent); + if (t.isVariableDeclaration(declaration)) + position = statementPosition(declaration, parents, parent); + } else if (t.isStatement(parent)) + position = statementPosition(parent, parents); + } + current = parent; + } + return selectInitializers(binding, position, parents, true); +}; + +const selectInitializers = ( + binding: JavaScriptSemanticBindingState, + position: StatementPosition | null, + parents: WeakMap, + atRead = false, +): Pick< + JavaScriptSemanticBindingState, + "initializers" | "referenceInitializers" +> => { + if (position === null) return binding; const origins = [ ...binding.initializers.map((initializer) => ({ initializer, @@ -169,12 +212,20 @@ export const semanticMutationInitializers = ( ]; if ( origins.some( - ({ position }) => - position === null || !positionPrecedes(position, mutationPosition), + ({ position: origin }) => + origin === null || + origin.body !== position.body || + (!atRead && !positionPrecedes(origin, position)), ) ) return binding; - const latest = origins.reduce( + const preceding = atRead + ? origins.filter( + ({ position: origin }) => + origin !== null && positionPrecedes(origin, position), + ) + : origins; + const latest = preceding.reduce( (latest, { position }) => position !== null && (latest === null || positionPrecedes(latest, position)) @@ -183,7 +234,7 @@ export const semanticMutationInitializers = ( null, ); const retained = new Set( - origins + preceding .filter( ({ position }) => position?.index === latest?.index && diff --git a/src/domain/javascript/javascriptSemanticObjectReferences.ts b/src/domain/javascript/javascriptSemanticObjectReferences.ts index 4d1bf3e76..425e5fea5 100644 --- a/src/domain/javascript/javascriptSemanticObjectReferences.ts +++ b/src/domain/javascript/javascriptSemanticObjectReferences.ts @@ -4,6 +4,8 @@ import { type JavaScriptSemanticAnalysisState, } from "./javascriptSemanticState.js"; import { semanticCallableResultExpressions } from "./javascriptSemanticCallableResults.js"; +import { semanticReadInitializers } from "./javascriptSemanticMutationInitializers.js"; +import { semanticReferenceSourcesInactive } from "./javascriptSemanticReferenceSources.js"; import { semanticStaticPropertyKey, unwrapJavaScriptExpression, @@ -17,6 +19,7 @@ import { compareUnicodeCodePoints } from "../unicodeCodePointOrder.js"; export interface JavaScriptSemanticObjectReference { readonly node: t.Node; readonly path: PropertyPath; + readonly escapes?: true; /** The allocation owned by this binding, rather than a read of its prior value. */ readonly ownerInitializer?: t.Node; } @@ -73,6 +76,7 @@ const instanceFieldReferences = ( propertyName: string | null, path: PropertyPath, state: JavaScriptSemanticAnalysisState, + initializedBefore?: t.ClassProperty, ): { readonly references: readonly JavaScriptSemanticObjectReference[]; readonly definiteNames: ReadonlySet; @@ -94,7 +98,15 @@ const instanceFieldReferences = ( if (previous !== undefined && previous.size === excluded.size) continue; selections.set(current.klass, excluded); const shadowed = new Set(excluded); - for (const member of [...current.klass.body.body].reverse()) { + const before = + initializedBefore === undefined + ? -1 + : current.klass.body.body.indexOf(initializedBefore); + const members = + before < 0 + ? current.klass.body.body + : current.klass.body.body.slice(0, before); + for (const member of [...members].reverse()) { if (!t.isClassProperty(member) || member.static) continue; const key = semanticStaticPropertyKey(member.key, member.computed); if (key !== null && shadowed.has(key)) continue; @@ -123,12 +135,15 @@ interface ClassPropertyLookup { readonly owner: t.ClassDeclaration | t.ClassExpression; readonly receiver?: t.ClassDeclaration | t.ClassExpression; readonly kind: "instance" | "static" | "prototype"; + readonly fieldsOnly?: true; + readonly initializedBefore?: t.ClassProperty; } const classPropertyDescriptors = ( klass: t.ClassDeclaration | t.ClassExpression, propertyName: string | null, kind: ClassPropertyLookup["kind"], + initializedBefore?: t.ClassProperty, ) => { const isStatic = kind === "static"; const selectedGetters: t.ClassMethod[] = []; @@ -138,7 +153,12 @@ const classPropertyDescriptors = ( readonly index: number; readonly key: string | null; }[] = []; + const before = + initializedBefore === undefined + ? -1 + : klass.body.body.indexOf(initializedBefore); klass.body.body.forEach((member, index) => { + if (before >= 0 && index >= before && t.isClassProperty(member)) return; if ( (t.isClassMethod(member) || (kind === "static" && t.isClassProperty(member))) && @@ -223,7 +243,13 @@ export const semanticClassPropertyReferences = ( // constructors. They take precedence over every prototype descriptor. let instanceFields: ReadonlySet = new Set(); if (lookup.kind === "instance") { - const fields = instanceFieldReferences(root, propertyName, path, state); + const fields = instanceFieldReferences( + root, + propertyName, + path, + state, + lookup.initializedBefore, + ); possibleResults.push(...fields.references); instanceFields = fields.definiteNames; if (propertyName !== null && instanceFields.has(propertyName)) @@ -237,11 +263,12 @@ export const semanticClassPropertyReferences = ( klass, propertyName, lookup.kind, + lookup.initializedBefore, ); for (const field of descriptors.fields) if (field.value != null) possibleResults.push({ node: field.value, path }); - for (const getter of descriptors.getters) { + for (const getter of lookup.fieldsOnly ? [] : descriptors.getters) { const key = semanticStaticPropertyKey(getter.key, getter.computed); if (key !== null && instanceFields.has(key)) continue; possibleResults.push( @@ -345,6 +372,102 @@ interface GetterInvocation { | t.ClassExpression; } +/** Follow only data fields whose initializers retain a class receiver. */ +const privateReceiverProjection = ( + receiver: t.ClassDeclaration | t.ClassExpression, + owner: t.ClassDeclaration | t.ClassExpression, + path: PropertyPath, + isStatic: boolean, + state: JavaScriptSemanticAnalysisState, +): boolean => { + if (path.some((key) => key === null || typeof key === "object")) return false; + type Selection = { readonly key: string; readonly before?: t.ClassProperty }; + const pending: { + receiver: typeof receiver; + path: readonly Selection[]; + visits: ReadonlyMap; + }[] = [ + { + receiver, + path: path.map((key) => ({ key: String(key) })), + visits: new Map(), + }, + ]; + const seen = new Set(); + const declaringClass = (field: t.ClassProperty) => { + const body = state.parentsByNode.get(field); + return body === undefined ? undefined : state.parentsByNode.get(body); + }; + for (let cursor = 0; cursor < pending.length; cursor++) { + const current = pending[cursor]; + if (current === undefined) break; + const [selection, ...remaining] = current.path; + if (selection === undefined) { + if (!isStatic || current.receiver === owner) return true; + continue; + } + const origin = JSON.stringify([ + current.receiver.start, + selection.key, + selection.before?.start, + ]); + const previous = current.visits.get(origin); + // Repeated fields may consume a finite getter path. An initializer cycle + // that preserves or grows it cannot establish another receiver identity. + if (previous !== undefined && previous <= current.path.length) continue; + const visits = new Map(current.visits).set(origin, current.path.length); + const identity = JSON.stringify([ + current.receiver.start, + current.path.map(({ key, before }) => [key, before?.start]), + ]); + if (seen.has(identity)) continue; + seen.add(identity); + const lookupOwner = + selection.before === undefined + ? current.receiver + : declaringClass(selection.before); + if (!t.isClass(lookupOwner)) continue; + for (const reference of semanticClassPropertyReferences( + { + owner: lookupOwner, + receiver: current.receiver, + kind: isStatic ? "static" : "instance", + fieldsOnly: true, + ...(selection.before === undefined + ? {} + : { initializedBefore: selection.before }), + }, + [selection.key], + state, + )) { + const field = state.parentsByNode.get(reference.node); + if (!t.isClassProperty(field)) continue; + const fieldOwner = declaringClass(field); + if (!t.isClass(fieldOwner)) continue; + const projection: Selection[] = []; + let value = unwrapJavaScriptExpression(reference.node).node; + while ( + t.isMemberExpression(value) || + t.isOptionalMemberExpression(value) + ) { + const key = semanticStaticPropertyKey(value.property, value.computed); + if (key === null || t.isPrivateName(value.property)) break; + projection.unshift({ key, before: field }); + value = unwrapJavaScriptExpression(value.object).node; + } + if (!t.isThisExpression(value)) continue; + // Instance fields retain the actual instance. Static field `this` belongs + // to its declaring class, even when a subclass inherits the field. + pending.push({ + receiver: isStatic ? fieldOwner : current.receiver, + path: [...projection, ...remaining], + visits, + }); + } + } + return false; +}; + const getterResultReferences = ( invocation: GetterInvocation, path: PropertyPath, @@ -358,40 +481,206 @@ const getterResultReferences = ( // path; recursive getters that preserve or grow that path cannot progress. const active = new Map(activeGetters).set(getter, path.length); const isStatic = t.isClassMethod(getter) && getter.static; - const resolve = ( - node: t.Node, - remaining: PropertyPath, - ): readonly JavaScriptSemanticObjectReference[] => { - const expression = unwrapJavaScriptExpression(node).node; - if (t.isThisExpression(expression)) { - if (t.isClass(receiver)) { - return semanticClassPropertyReferences( - { owner: receiver, kind: isStatic ? "static" : "instance" }, - remaining, - state, - active, - ); + interface AliasVisit { + readonly identity: string; + readonly pathLength: number; + readonly parent: AliasVisit | undefined; + } + const pending: { + readonly node: t.Node; + readonly path: PropertyPath; + readonly aliases: AliasVisit | undefined; + readonly escapes?: true; + readonly privateRead?: { + readonly node: t.MemberExpression | t.OptionalMemberExpression; + readonly name: string; + readonly path: PropertyPath; + }; + }[] = semanticCallableResultExpressions(getter).flatMap(({ node }) => + node === null ? [] : [{ node, path, aliases: undefined }], + ); + const references: JavaScriptSemanticObjectReference[] = []; + const expandedAliases = new Map>(); + const getterScope = state.scopeByNode.get(getter)?.scopeId; + for (let cursor = 0; cursor < pending.length; cursor++) { + const current = pending[cursor]; + if (current === undefined) break; + const { path: remaining } = current; + const expression = unwrapJavaScriptExpression(current.node).node; + const retain = (reference: JavaScriptSemanticObjectReference): void => { + references.push( + current.escapes ? { ...reference, escapes: true } : reference, + ); + }; + const retainPrivateRead = (): void => { + if (current.privateRead !== undefined) + retain({ + node: current.privateRead.node, + path: current.escapes ? [] : current.privateRead.path, + }); + }; + if (t.isIdentifier(expression)) { + const binding = resolveSemanticBindingState( + state, + expression, + expression.name, + ); + let scope = + binding === undefined + ? undefined + : state.scopesById.get(binding.scopeId); + while (scope !== undefined && scope.scopeId !== getterScope) { + if (scope.kind === "function" || scope.parentScopeId === null) { + scope = undefined; + break; + } + scope = state.scopesById.get(scope.parentScopeId); } - return objectReceiverReference(receiver, remaining, state); + if ( + binding !== undefined && + scope !== undefined && + binding.referenceInitializers.every( + (initializer) => initializer.copyKind === undefined, + ) + ) { + const identity = JSON.stringify([ + binding.bindingId, + expression.start, + expression.end, + ]); + const known = expandedAliases.get(identity); + const key = JSON.stringify([ + remaining, + current.privateRead?.name, + current.privateRead?.path, + current.escapes, + ]); + let cyclic = false; + let growing = false; + if (known !== undefined) + for ( + let visit = current.aliases; + visit !== undefined; + visit = visit.parent + ) + if (visit.identity === identity) { + cyclic = true; + growing = remaining.length > visit.pathLength; + break; + } + if (cyclic) { + // A repeated alias can select arbitrarily deep children. Revisit its + // roots as escapes rather than enumerating iterations or dropping them. + if (growing && !current.escapes) + pending.push({ + ...current, + path: [], + aliases: undefined, + escapes: true, + }); + continue; + } + if (known?.has(key)) continue; + const selections = known ?? new Set(); + selections.add(key); + expandedAliases.set(identity, selections); + const aliases = { + identity, + pathLength: remaining.length, + parent: current.aliases, + }; + const origins = semanticReadInitializers( + binding, + expression, + state.parentsByNode, + ); + for (const initializer of [ + ...origins.initializers, + ...origins.referenceInitializers.filter( + (initializer) => + !semanticReferenceSourcesInactive(initializer, state), + ), + ]) + pending.push({ + ...current, + node: initializer.node, + path: [...initializer.projection, ...remaining], + aliases, + }); + continue; + } + } + if (t.isThisExpression(expression)) { + // An outer binding or a nested ordinary function has a different receiver. + let ownerNode = state.parentsByNode.get(expression); + while ( + ownerNode !== undefined && + !t.isClass(ownerNode) && + !(t.isFunction(ownerNode) && !t.isArrowFunctionExpression(ownerNode)) + ) + ownerNode = state.parentsByNode.get(ownerNode); + if (ownerNode !== getter) continue; + if (current.privateRead !== undefined && t.isClass(owner)) { + if ( + !t.isClass(receiver) || + !privateReceiverProjection( + receiver, + owner, + remaining, + isStatic, + state, + ) + ) { + retainPrivateRead(); + continue; + } + for (const member of owner.body.body) + if ( + t.isClassPrivateProperty(member) && + member.static === isStatic && + member.key.id.name === current.privateRead.name && + member.value != null + ) + retain({ + node: member.value, + path: current.escapes ? [] : current.privateRead.path, + }); + continue; + } + for (const reference of t.isClass(receiver) + ? semanticClassPropertyReferences( + { owner: receiver, kind: isStatic ? "static" : "instance" }, + remaining, + state, + active, + ) + : objectReceiverReference(receiver, remaining, state)) + retain(reference); + continue; } if ( t.isMemberExpression(expression) || t.isOptionalMemberExpression(expression) ) { if ( - t.isPrivateName(expression.property) && - t.isThisExpression(expression.object) && - t.isClass(owner) + current.privateRead !== undefined && + t.isPrivateName(expression.property) ) { - const privateName = expression.property.id.name; - return owner.body.body.flatMap((member) => - t.isClassPrivateProperty(member) && - member.static === isStatic && - member.key.id.name === privateName && - member.value != null - ? [{ node: member.value, path: remaining }] - : [], - ); + retainPrivateRead(); + continue; + } + if (t.isPrivateName(expression.property) && t.isClass(owner)) { + pending.push({ + ...current, + node: expression.object, + path: [], + privateRead: { + node: expression, + name: expression.property.id.name, + path: remaining, + }, + }); + continue; } const key = semanticStaticPropertyKey( expression.property, @@ -402,41 +691,52 @@ const getterResultReferences = ( t.isClass(owner) && owner.superClass != null ) { - return semanticClassCandidates(owner.superClass, state).flatMap( - (base) => - semanticClassPropertyReferences( - { - owner: base, - receiver: t.isClass(receiver) ? receiver : owner, - kind: isStatic ? "static" : "prototype", - }, - [key, ...remaining], - state, - active, - ), - ); + for (const base of semanticClassCandidates(owner.superClass, state)) + for (const reference of semanticClassPropertyReferences( + { + owner: base, + receiver: t.isClass(receiver) ? receiver : owner, + kind: isStatic ? "static" : "prototype", + }, + [key, ...remaining], + state, + active, + )) + retain(reference); + continue; } - return resolve(expression.object, [key, ...remaining]); - } - if (t.isConditionalExpression(expression)) - return [ - ...resolve(expression.consequent, remaining), - ...resolve(expression.alternate, remaining), - ]; - if (t.isLogicalExpression(expression)) - return [ - ...resolve(expression.left, remaining), - ...resolve(expression.right, remaining), - ]; - if (t.isSequenceExpression(expression)) { + pending.push({ + ...current, + node: expression.object, + path: [key, ...remaining], + }); + } else if (t.isConditionalExpression(expression)) + pending.push( + { ...current, node: expression.consequent }, + { ...current, node: expression.alternate }, + ); + else if (t.isLogicalExpression(expression)) + pending.push( + { ...current, node: expression.left }, + { ...current, node: expression.right }, + ); + else if (t.isSequenceExpression(expression)) { const last = expression.expressions.at(-1); - return last === undefined ? [] : resolve(last, remaining); - } - return [{ node: expression, path: remaining }]; - }; - return semanticCallableResultExpressions(getter).flatMap(({ node }) => - node === null ? [] : resolve(node, path), - ); + if (last !== undefined) pending.push({ ...current, node: last }); + } else if (t.isAssignmentExpression(expression, { operator: "=" })) + pending.push({ ...current, node: expression.right }); + else if ( + t.isAssignmentExpression(expression) && + ["&&=", "||=", "??="].includes(expression.operator) + ) + pending.push( + { ...current, node: expression.left }, + { ...current, node: expression.right }, + ); + else if (current.privateRead !== undefined) retainPrivateRead(); + else retain({ node: expression, path: remaining }); + } + return references; }; export const semanticObjectPropertyReferences = ( diff --git a/src/domain/javascript/javascriptSemanticReferenceSources.ts b/src/domain/javascript/javascriptSemanticReferenceSources.ts new file mode 100644 index 000000000..54e1d3ba9 --- /dev/null +++ b/src/domain/javascript/javascriptSemanticReferenceSources.ts @@ -0,0 +1,53 @@ +import type * as t from "@babel/types"; +import type { JavaScriptSemanticAnalysisState } from "./javascriptSemanticState.js"; +import type { JavaScriptSemanticPropertyPath as PropertyPath } from "./javascriptSemanticPropertyPaths.js"; +import { evaluateSemanticExpression } from "./javascriptSemanticValues.js"; +import { semanticSlotAtPath } from "./javascriptSemanticSlots.js"; + +interface ReferenceSource { + readonly node: t.Node; + readonly projection: PropertyPath; +} + +/** Exclude references whose source slot or destructuring fallback is absent. */ +export const semanticReferenceSourcesInactive = ( + reference: { + readonly requiredSources?: readonly ReferenceSource[]; + readonly fallbackSources?: readonly ReferenceSource[]; + }, + state: JavaScriptSemanticAnalysisState, +): boolean => + reference.requiredSources?.some( + (source) => + semanticSelectedReferenceSlot(source, state, source.node)?.presence === + "absent", + ) === true || + reference.fallbackSources?.some((source) => { + // Selection precedes effects through the chosen reference. A later escape + // cannot make a fallback that was excluded at destructuring reachable. + const slot = semanticSelectedReferenceSlot(source, state, source.node); + return ( + slot?.presence === "present" && + (slot.value.status === "literal" || + slot.value.status === "union" || + slot.value.status === "object" || + slot.value.status === "array") + ); + }) === true; + +/** Select a retained source slot at its capture, before later effects. */ +export const semanticSelectedReferenceSlot = ( + source: ReferenceSource, + state: JavaScriptSemanticAnalysisState, + capturePoint?: t.Node, +) => { + const path: string[] = []; + for (const key of source.projection) { + if (key === null || typeof key === "object") return undefined; + path.push(String(key)); + } + return semanticSlotAtPath( + evaluateSemanticExpression(source.node, state, capturePoint), + path, + ); +}; diff --git a/tests/acceptance/applications/getterLocalAliasesCli.test.ts b/tests/acceptance/applications/getterLocalAliasesCli.test.ts new file mode 100644 index 000000000..2b2b0196c --- /dev/null +++ b/tests/acceptance/applications/getterLocalAliasesCli.test.ts @@ -0,0 +1,85 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { expect } from "vitest"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { cliTest } from "../../support/cli/cliFixture.js"; +import { analyzeJavaScriptDirectory } from "../../support/cli/javascriptApplication.js"; + +cliTest.for(["CLI", "stdio MCP"] as const)( + "retains getter-local alias mutations in exported Evidence through %s", + async (surface, { cli }) => { + const root = await createTestTempDirectory("rea-getter-local-aliases-"); + const cases = [ + ["initialized", "const self = this; return self.s;", true], + ["assigned", "let self; self = this; return self.s;", true], + ["destructured", "const { s: value } = this; return value;", true], + ["readOnly", "const self = this; return self.s;", false], + [ + "cyclic", + "let self = this.start; while (flag) self = self.child; return self;", + true, + "start: { child: { child: shared } }", + ], + ] as const; + const privateCases = [ + ["privateProjection", "shared", true, "unknown"], + ["privateUnrelated", "{ peer: shared }", true, "literal"], + ["privateReadOnly", "shared", false, "literal"], + ] as const; + await writeFile( + join(root, "app.mjs"), + cases + .map( + ([ + name, + getter, + write, + members = "s: shared", + ]) => `export function ${name}() { + const shared = { x: 1 }; + const box = { ${members}, get v() { ${getter} } }; + ${write ? "box.v.x = 2;" : "const saved = box.v;"} + return shared.x; + }`, + ) + .join("\n") + + privateCases + .map( + ([name, value, write]) => `export function ${name}() { + const shared = { x: 1 }; + class Box { + #s = ${value}; + peer = this; + get v() { const { peer: self } = this; return self.#s; } + } + const box = new Box(); + ${write ? "box.v.x = 2;" : "const saved = box.v;"} + return shared.x; + }`, + ) + .join("\n"), + ); + const { graph } = await analyzeJavaScriptDirectory(surface, cli, root); + for (const [name, status] of [ + ...cases.map(([name, , write]) => [name, write ? "unknown" : "literal"]), + ...privateCases.map(([name, , , status]) => [name, status]), + ]) { + const observation = graph.nodes + .flatMap(({ observations }) => observations) + .find( + ({ properties }) => + properties.semantic_role === "export-return-shapes" && + properties.exported_name === name, + ); + expect(observation, name).toBeDefined(); + expect(observation?.properties.static_return_shapes).toMatchObject([ + { + value_status: status, + fields: [ + { path: "", state: status, value: status === "unknown" ? null : 1 }, + ], + }, + ]); + } + }, +); From 6637161e0272513ccc11c33aa6d8915bce6171ac Mon Sep 17 00:00:00 2001 From: Saim Shafique Date: Tue, 13 Oct 2026 18:26:57 +0500 Subject: [PATCH 19/24] fix(ghidra): warn when callee/caller reads stop at a truncated function body (#2360) procedure_callees and procedure_callers returned a bare address list, so an inferred no-return call that truncates Ghidra's function body silently dropped real edges (fixes #2326). Wrap both views in the function-observation envelope with noReturnCallLimitations, matching procedure_info, procedure_assembly, procedure_pseudo_code, read_function_instructions, and analyze_function (precedent: 69a8b3a95). Version the boundary-observation contract to ghidra-terminal-call-limitations-v2 so older snapshots cannot replay unqualified results. Validation: red->green unit test for the envelope requirement, MCP boundary evidence test extended for both views (limitations retained through Evidence and snapshot replay), 8 harness test files + ghidra behavior tests green, check:fast, docs:check, verify:test-discovery green. Real Ghidra host lane unavailable here (no Ghidra installed); Java change mirrors the existing procedureInfo call pattern. --- bridge/ghidra/ReaGhidraBridge.java | 9 +++++-- docs/ghidra-provider.md | 7 +++--- src/contracts/officialToolContracts.ts | 4 ++-- src/ghidra/GhidraAnalysisProfile.ts | 4 ++-- src/ghidra/GhidraFunctionValues.test.ts | 25 ++++++++++++++++++++ src/ghidra/GhidraFunctionValues.ts | 2 ++ src/ghidra/GhidraProvider.behavior.test.ts | 6 ++++- tests/boundary/mcp/ghidraEvidenceMcp.test.ts | 4 +++- tests/boundary/mcp/ghidraMcpHarness.ts | 2 ++ 9 files changed, 52 insertions(+), 11 deletions(-) diff --git a/bridge/ghidra/ReaGhidraBridge.java b/bridge/ghidra/ReaGhidraBridge.java index 70132eeae..c180c5866 100644 --- a/bridge/ghidra/ReaGhidraBridge.java +++ b/bridge/ghidra/ReaGhidraBridge.java @@ -812,7 +812,7 @@ public final class ReaGhidraBridge extends HeadlessScript { return functionObservation(result, boundaryLimitations); } - private JsonArray procedureCalls(JsonObject params, boolean callers) throws Exception { + private JsonObject procedureCalls(JsonObject params, boolean callers) throws Exception { requireKeys(params, Set.of("document", "procedure")); requireDocument(params); Function function = resolveProcedure(requireString(params, "procedure")); @@ -825,7 +825,12 @@ public final class ReaGhidraBridge extends HeadlessScript { for (Function item : ordered) { result.add(canonicalAddress(item.getEntryPoint())); } - return result; + return functionObservation( + result, + noReturnCallLimitations( + currentProgram.getListing().getInstructions(function.getBody(), true) + ) + ); } private JsonObject procedureInfo(JsonObject params) throws Exception { diff --git a/docs/ghidra-provider.md b/docs/ghidra-provider.md index 32edbd320..5b83ee152 100644 --- a/docs/ghidra-provider.md +++ b/docs/ghidra-provider.md @@ -131,9 +131,10 @@ overview merely because its source artifact is unchanged. Function extents and instruction completeness refer to Ghidra's analyzed body. An inferred no-return call can terminate that body before subsequent bytes. -Pseudocode, assembly, procedure info, instruction windows, and dossiers report -observed terminal call sites in Evidence limitations, including resolved callee -names, addresses, and FunctionManager flags when available. Instruction windows +Pseudocode, assembly, procedure info, instruction windows, callee and caller +reads, and dossiers report observed terminal call sites in Evidence limitations, +including resolved callee names, addresses, and FunctionManager flags when +available. Instruction windows and dossiers also retain these warnings in their result limitations. An unresolved callee or unestablished terminal-flow basis remains explicit. These observations qualify the analysis; they do not prove that a no-return flag is diff --git a/src/contracts/officialToolContracts.ts b/src/contracts/officialToolContracts.ts index 82f1dc83d..4750528ad 100644 --- a/src/contracts/officialToolContracts.ts +++ b/src/contracts/officialToolContracts.ts @@ -149,12 +149,12 @@ export const OFFICIAL_TOOL_CONTRACTS = [ ), official( "procedure_callees", - "Return the provider's resolved direct callees for one procedure identified by symbol or address. Unresolved indirect calls may be absent; use analyze_function and typed references to preserve available edge uncertainty.", + "Return the provider's resolved direct callees for one procedure identified by symbol or address. Unresolved indirect calls may be absent; use analyze_function and typed references to preserve available edge uncertainty. Ghidra terminal-call assumptions that may cut the analyzed body are reported in Evidence limitations.", z.object({ procedure, document }), ), official( "procedure_callers", - "Return the provider's resolved direct callers for one procedure identified by symbol or address. Results reflect completed static analysis and may omit unresolved indirect references.", + "Return the provider's resolved direct callers for one procedure identified by symbol or address. Results reflect completed static analysis and may omit unresolved indirect references. Ghidra terminal-call assumptions that may cut the analyzed body are reported in Evidence limitations.", z.object({ procedure, document }), ), official( diff --git a/src/ghidra/GhidraAnalysisProfile.ts b/src/ghidra/GhidraAnalysisProfile.ts index 8cc5f2807..81defabd3 100644 --- a/src/ghidra/GhidraAnalysisProfile.ts +++ b/src/ghidra/GhidraAnalysisProfile.ts @@ -127,7 +127,7 @@ export const resolveGhidraAnalysisProfile = async ( "address-first-unique-target-source-mapping-v2", instruction_flow_evidence: "decoded-return-pcode-v1", instruction_references: "exact-address-incoming-v1", - function_boundary_observations: "ghidra-terminal-call-limitations-v1", + function_boundary_observations: "ghidra-terminal-call-limitations-v2", no_return_repair: installation.platform === "win32" ? "disabled-windows-p0" @@ -218,7 +218,7 @@ export const resolveGhidraAnalysisProfile = async ( "address-first-unique-target-source-mapping-v2", instruction_flow_evidence: "decoded-return-pcode-v1", instruction_references: "exact-address-incoming-v1", - function_boundary_observations: "ghidra-terminal-call-limitations-v1", + function_boundary_observations: "ghidra-terminal-call-limitations-v2", no_return_repair: installation.platform === "win32" ? "disabled-windows-p0" diff --git a/src/ghidra/GhidraFunctionValues.test.ts b/src/ghidra/GhidraFunctionValues.test.ts index 333d2c913..e54692840 100644 --- a/src/ghidra/GhidraFunctionValues.test.ts +++ b/src/ghidra/GhidraFunctionValues.test.ts @@ -23,6 +23,8 @@ import { const observationOperations: ReadonlySet = new Set([ "analyze_function", "procedure_assembly", + "procedure_callees", + "procedure_callers", "procedure_info", "procedure_pseudo_code", "read_function_instructions", @@ -93,6 +95,29 @@ describe("Ghidra function-analysis result values", () => { }); }); + it.each(["procedure_callees", "procedure_callers"] as const)( + "requires the function-observation envelope to carry terminal-call limitations for %s", + (operation) => { + // A bare address list is the pre-fix bridge shape; without the envelope + // a truncated function body reports no warning, so it must be rejected. + expect(parseGhidraFunctionResult(operation, ["0x2000"])).toMatchObject({ + ok: false, + error: { _tag: "AnalysisOutputError" }, + }); + const limitation = + "Ghidra Listing reports a terminal call at 0x401000 to 0x402000 abort; fallthrough is excluded by Ghidra's flow model."; + expect( + parseGhidraFunctionResult(operation, { + value: ["0x2000"], + limitations: [limitation], + }), + ).toEqual({ + ok: true, + value: { value: ["0x2000"], limitations: [limitation] }, + }); + }, + ); + it.each(["reference_kinds_available", "unresolved_calls"])( "rejects omitted %s instead of inventing complete reference observations", (field) => { diff --git a/src/ghidra/GhidraFunctionValues.ts b/src/ghidra/GhidraFunctionValues.ts index 4e4c5054a..a6e62c93c 100644 --- a/src/ghidra/GhidraFunctionValues.ts +++ b/src/ghidra/GhidraFunctionValues.ts @@ -409,6 +409,8 @@ const functionBoundaryObservationOperations: ReadonlySet { start: () => Promise.resolve(ok(sessionInfo())), callTool: (operation) => Promise.resolve( - ok(operation === "procedure_callees" ? ["0x2000"] : []), + ok( + operation === "procedure_callees" + ? { value: ["0x2000"], limitations: [] } + : { value: [], limitations: [] }, + ), ), close: () => Promise.resolve(ok(null)), })); diff --git a/tests/boundary/mcp/ghidraEvidenceMcp.test.ts b/tests/boundary/mcp/ghidraEvidenceMcp.test.ts index c28fcca39..ed82400aa 100644 --- a/tests/boundary/mcp/ghidraEvidenceMcp.test.ts +++ b/tests/boundary/mcp/ghidraEvidenceMcp.test.ts @@ -46,6 +46,8 @@ it("retains terminal-call qualifications across function views and snapshot repl limitations: [limitation], }, ], + ["procedure_callees", ["0x402000"]], + ["procedure_callers", [identity.address]], ["analyze_function", jsonValueSchema.parse(dossier)], ]); const harness = await connectGhidraMcp( @@ -81,7 +83,7 @@ it("retains terminal-call qualifications across function views and snapshot repl }); expect(evidence.limitations, name).toContain(limitation); expect(evidence.analysis_profile?.parameters).toMatchObject({ - function_boundary_observations: "ghidra-terminal-call-limitations-v1", + function_boundary_observations: "ghidra-terminal-call-limitations-v2", }); const replay = await harness.session.execute(name, evidence.parameters); if (!replay.ok) throw replay.error; diff --git a/tests/boundary/mcp/ghidraMcpHarness.ts b/tests/boundary/mcp/ghidraMcpHarness.ts index ad3565992..a9b2248dc 100644 --- a/tests/boundary/mcp/ghidraMcpHarness.ts +++ b/tests/boundary/mcp/ghidraMcpHarness.ts @@ -263,6 +263,8 @@ const resultFor = ( const functionObservationOperations: ReadonlySet = new Set([ "analyze_function", "procedure_assembly", + "procedure_callees", + "procedure_callers", "procedure_info", "procedure_pseudo_code", "read_function_instructions", From 117bd23dd5084a0a15286d749a46bf1f7bc8fadb Mon Sep 17 00:00:00 2001 From: AICodeLion <2401112573@stu.pku.edu.cn> Date: Tue, 13 Oct 2026 21:27:45 +0800 Subject: [PATCH 20/24] test(clients): verify Qwen Code on native Windows (#2103) Co-authored-by: AICodeLion <190163814+AICodeLion@users.noreply.github.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- docs/client-verification.md | 16 ++++++++++-- scripts/verify-qwen-client.mjs | 47 ++++++++++++++++++++++------------ 2 files changed, 44 insertions(+), 19 deletions(-) diff --git a/docs/client-verification.md b/docs/client-verification.md index 42e059382..d9ece9aad 100644 --- a/docs/client-verification.md +++ b/docs/client-verification.md @@ -18,14 +18,26 @@ pass `-- chat` to check plain chat with REA enabled. Qwen also reads the full result that its client offloads, and Pi exercises default codemode execution. Use `REA_VERIFY_RUNTIME_ROOT` to select a production-only installed REA package. Receipts retain client versions, result digests, host coverage and owned-process -cleanup. These POSIX lanes use a deterministic loopback model, so they do not -prove live model-provider or native Windows compatibility. Qwen, Pi and Hermes +cleanup. These lanes use a deterministic loopback model and do not prove live +model-provider compatibility. Pi and Hermes remain POSIX-only. Qwen, Pi and Hermes bind both `HOME` and `USERPROFILE` to the disposable account. Qwen and Pi discover the shared personal skill through their default search paths without an explicit skill setting; Hermes discovers it through the selected `HERMES_HOME`. Set `REA_VERIFY_HERMES_STICKY_PROFILE=1` to exercise native Hermes selection of a named sticky profile with `HERMES_HOME` still pointing to its root. +The Qwen lane also supports native Windows (verified with Qwen Code 0.25.1 and +Node 24.19.0 for both `chat` and `call`). With Node 24.18.0, the same `call` +journey reached full Evidence readback but the client aborted during exit with +`UV_HANDLE_CLOSING`; that run remains a failure, not a compatibility claim. +Select the npm install's +`node_modules/@qwen-code/qwen-code/cli-entry.js` with `REA_VERIFY_QWEN_COMMAND`, +because `execFile` cannot launch the `qwen.cmd` shim directly. A selected `.js`, +`.mjs` or `.cjs` entry point runs under the verifier's Node. Windows path checks +use native separators; persisted output is read through Qwen's real shell tool +with a shell-independent encoded Node payload. Windows receipts report process +lineage as unavailable when the verifier has no retained native Job Object lease. + `verify:gemini-client` requires an installed Gemini CLI (verified with `@google/gemini-cli@0.63.0` on POSIX and native Windows); select it with `REA_VERIFY_GEMINI_COMMAND`. A selected `.js`, `.mjs` or `.cjs` entry point runs diff --git a/scripts/verify-qwen-client.mjs b/scripts/verify-qwen-client.mjs index ed60ccd4c..fff4338b7 100644 --- a/scripts/verify-qwen-client.mjs +++ b/scripts/verify-qwen-client.mjs @@ -8,7 +8,7 @@ import { realpath, } from "node:fs/promises"; import { tmpdir } from "node:os"; -import { join } from "node:path"; +import { join, resolve, sep } from "node:path"; import { fileURLToPath } from "node:url"; import { createHash } from "node:crypto"; import { exec } from "./lib/verify-package-core.mjs"; @@ -18,10 +18,6 @@ import { toolContract, } from "../dist/contracts/toolContracts.js"; -assert( - process.platform !== "win32", - "The Qwen real-client verifier currently requires a POSIX host; native Windows is unverified.", -); const repo = fileURLToPath(new URL("..", import.meta.url)); const runtimeRoot = process.env.REA_VERIFY_RUNTIME_ROOT ?? repo; const lab = await mkdtemp(join(tmpdir(), "rea-qwen-client-")); @@ -32,6 +28,13 @@ assert( "Usage: node scripts/verify-qwen-client.mjs [chat|call]", ); const command = process.env.REA_VERIFY_QWEN_COMMAND ?? "qwen"; +// npm exposes only qwen.cmd on Windows. Resolve a selected JavaScript entry +// before the verifier switches to its disposable workspace. +const entryPoint = /\.(c|m)?js$/iu.test(command) ? resolve(command) : undefined; +const launch = (args) => + entryPoint === undefined + ? [command, args] + : [process.execPath, [entryPoint, ...args]]; const verifier = createVerifierRun(); const profile = join(lab, "selected profile"); const account = join(lab, "account"); @@ -54,14 +57,18 @@ const baseEnvironment = { let clientVersion; try { clientVersion = ( - await exec(command, ["--version"], { + await exec(...launch(["--version"]), { env: baseEnvironment, timeout: 30_000, }) ).stdout.trim(); } catch (cause) { + const windowsHint = + process.platform === "win32" + ? " On Windows, select node_modules/@qwen-code/qwen-code/cli-entry.js." + : ""; throw new Error( - `Qwen client preflight failed for ${command}; install Qwen Code or set REA_VERIFY_QWEN_COMMAND. ${String(cause)}`, + `Qwen client preflight failed for ${command}; install Qwen Code or set REA_VERIFY_QWEN_COMMAND.${windowsHint} ${String(cause)}`, ); } const original = JSON.stringify( @@ -93,8 +100,8 @@ for (const step of [ assert( result.plannedActions.every( (a) => - a.target.startsWith(profile + "/") || - a.target.startsWith(sharedSkills + "/"), + a.target.startsWith(profile + sep) || + a.target.startsWith(sharedSkills + sep), ), ); await writeFile( @@ -163,7 +170,14 @@ const fixture = await createOpenAiModelFixture({ skillSeen ||= JSON.stringify(body.messages ?? []).includes( "reverse-engineer-anything", ); - selectedSkillLoaded ||= JSON.stringify(body.messages ?? []).includes( + const messageText = (body.messages ?? []) + .map((message) => + typeof message.content === "string" + ? message.content + : (message.content ?? []).map((part) => part.text ?? "").join("\n"), + ) + .join("\n"); + selectedSkillLoaded ||= messageText.includes( join(sharedSkills, "reverse-engineer-anything"), ); const toolText = JSON.stringify( @@ -243,9 +257,7 @@ const fixture = await createOpenAiModelFixture({ const invoke = tools.find((t) => t.name === "tool_call"); const skill = tools.find((t) => t.name === "skill"); const shell = tools.find((t) => t.name === "run_shell_command"); - const persisted = toolText.match( - /Full output saved to: ([^\\"\n]+)(?:\\n|\n)/u, - ); + const persisted = messageText.match(/Full output saved to: ([^\r\n]+)/u); const call = (name, args) => ({ role: "assistant", content: null, @@ -268,13 +280,14 @@ const fixture = await createOpenAiModelFixture({ artifactPath = persisted[1]; assert( (await realpath(artifactPath)).startsWith( - (await realpath(runtime)) + "/", + (await realpath(runtime)) + sep, ), ); const script = `const fs=require('node:fs');const e=JSON.parse(fs.readFileSync(${JSON.stringify(artifactPath)},'utf8'));console.log(JSON.stringify({REA_CLIENT_FIXTURE:JSON.stringify(e.normalized_result).includes('REA_CLIENT_FIXTURE'),evidence_id:e.evidence_id,graph_nodes:e.normalized_result.graph.nodes.length}));`; - const quote = (value) => "'" + value.replaceAll("'", "'\\''") + "'"; + // Keep paths and JSON quotes out of shell syntax on Windows and POSIX. + const encodedScript = Buffer.from(script).toString("base64"); message = call(shell.name, { - command: `node -e ${quote(script)}`, + command: `node -e "eval(Buffer.from('${encodedScript}','base64').toString('utf8'))"`, description: "Read the complete saved REA Evidence and verify the fixture facts.", }); @@ -329,7 +342,7 @@ const args = [ let response; let failure; try { - response = await exec(command, args, { + response = await exec(...launch(args), { env, cwd: workspace, timeout: 120_000, From 9e019ad27dbad078122f87a9bb41ff6d8687d217 Mon Sep 17 00:00:00 2001 From: Danish Rafie Ekaputra <155859586+0xRafie@users.noreply.github.com> Date: Tue, 13 Oct 2026 20:34:38 +0700 Subject: [PATCH 21/24] feat(clients): register Factory Droid during setup (#2309) --- .../references/connection-and-recovery.md | 2 +- docs/client-configuration.md | 8 + docs/installation.md | 1 + src/application/SupportedClients.ts | 13 ++ .../filesystem/clientLifecycle.test.ts | 1 + .../droidClientConfiguration.test.ts | 167 ++++++++++++++++++ 6 files changed, 191 insertions(+), 1 deletion(-) create mode 100644 tests/boundary/filesystem/droidClientConfiguration.test.ts diff --git a/.agents/skills/reverse-engineer-anything/references/connection-and-recovery.md b/.agents/skills/reverse-engineer-anything/references/connection-and-recovery.md index ed8393bd9..0feae3ac4 100644 --- a/.agents/skills/reverse-engineer-anything/references/connection-and-recovery.md +++ b/.agents/skills/reverse-engineer-anything/references/connection-and-recovery.md @@ -46,7 +46,7 @@ rea setup --client codex --dry-run --json Use the current supported client's ID: `claude_code`, `claude_desktop`, `codex`, `cursor`, `gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`, `copilot_cli`, `commandcode`, `qwen_code`, `vscode`, `trae`, `grok_build`, `omp`, `pi`, -`hermes`, `qoder`, `zcode`, `dsh`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a +`hermes`, `qoder`, `droid`, `zcode`, `dsh`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a scope. `grok_bot` uses a chat registration step rather than local `mcp.json`. For unsupported clients, use version-pinned manual stdio registration or CLI. diff --git a/docs/client-configuration.md b/docs/client-configuration.md index e72ca191f..e27866f7d 100644 --- a/docs/client-configuration.md +++ b/docs/client-configuration.md @@ -165,6 +165,14 @@ For [Qoder](https://docs.qoder.com/cli/mcp-reference), setup writes from `~/.qoder/skills` (following `QODER_CONFIG_DIR` when set), so setup installs its skill copy there. +## Factory Droid + +For [Factory Droid](https://docs.factory.com/harness/mcp), setup writes +`mcpServers.rea` into the user-level `~/.factory/mcp.json`. Entries carry +`command`, `args`, and `env`. Droid discovers personal skills from +`~/.factory/skills` ([skills](https://docs.factory.com/harness/skills)), so +setup installs its skill copy there. + ## Trae For [Trae](https://docs.trae.ai/ide/add-mcp-servers), setup writes diff --git a/docs/installation.md b/docs/installation.md index d6dc8e3b0..9285883e3 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -283,6 +283,7 @@ hosted connector; see [its configuration](client-configuration.md#other-client-s | Pi | `pi` | | Hermes | `hermes` | | Qoder | `qoder` | +| Factory Droid | `droid` | | Trae | `trae` | | ZCode | `zcode` | | DeepSeek Harness | `dsh` | diff --git a/src/application/SupportedClients.ts b/src/application/SupportedClients.ts index 5839409d4..0a7352627 100644 --- a/src/application/SupportedClients.ts +++ b/src/application/SupportedClients.ts @@ -135,6 +135,9 @@ const codexDirectory = ({ home, env }: ClientPathContext): string => const qoderDirectory = ({ home, env }: ClientPathContext): string => env.QODER_CONFIG_DIR || join(home, ".qoder"); +const factoryDirectory = ({ home }: ClientPathContext): string => + join(home, ".factory"); + /** Match Qwen Code's home override, including tilde and cwd-relative paths. */ const qwenCodeDirectory = ({ home, env }: ClientPathContext): string => { const override = env.QWEN_HOME; @@ -558,6 +561,16 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [ markerPath: qoderDirectory, format: "json", }, + { + name: "droid", + displayName: "Factory Droid", + skillPath: (context: ClientPathContext) => + join(factoryDirectory(context), "skills"), + configPath: (context: ClientPathContext) => + join(factoryDirectory(context), "mcp.json"), + markerPath: factoryDirectory, + format: "json", + }, { name: "zcode", zcodeFallbackPath: ({ home }: ClientPathContext) => diff --git a/tests/boundary/filesystem/clientLifecycle.test.ts b/tests/boundary/filesystem/clientLifecycle.test.ts index 506a6d6df..63ca0667a 100644 --- a/tests/boundary/filesystem/clientLifecycle.test.ts +++ b/tests/boundary/filesystem/clientLifecycle.test.ts @@ -71,6 +71,7 @@ describe("client configuration filesystem lifecycle", () => { "pi", "hermes", "qoder", + "droid", "zcode", "dsh", "grok_bot", diff --git a/tests/boundary/filesystem/droidClientConfiguration.test.ts b/tests/boundary/filesystem/droidClientConfiguration.test.ts new file mode 100644 index 000000000..6f9b26423 --- /dev/null +++ b/tests/boundary/filesystem/droidClientConfiguration.test.ts @@ -0,0 +1,167 @@ +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +import { readClientRegistrationStatuses } from "../../../src/application/ClientRegistrationStatus.js"; +import { npxRegistrationCommand } from "../../../src/application/ClientRegistrationIdentity.js"; +import { configureClientConfiguration } from "../../../src/application/SetupClientConfiguration.js"; +import { skillDestinations } from "../../../src/application/SetupSkill.js"; +import { supportedClients } from "../../../src/application/SupportedClients.js"; +import { systemUninstallHost } from "../../../src/application/Uninstall.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { clearClientLocationEnvironment } from "../../fixtures/clientEnvironment.js"; + +/** + * The registration command setup writes on the host running the tests. Windows + * wraps it in `cmd.exe /d /c`, so the expected entry follows the host too. + */ +const command = npxRegistrationCommand(process.platform); + +beforeEach(clearClientLocationEnvironment); +afterEach(() => vi.unstubAllEnvs()); + +const droidPath = ( + home: string, + platform: NodeJS.Platform = "linux", + env: Parameters[2] = {}, +) => supportedClients(home, platform, env).find(({ name }) => name === "droid"); + +describe("Factory Droid configuration paths", () => { + it("resolves the user MCP config file and marker directory", () => { + expect(droidPath("/home/a")).toEqual({ + name: "droid", + displayName: "Factory Droid", + configPath: join("/home/a", ".factory", "mcp.json"), + markerPath: join("/home/a", ".factory"), + format: "json", + }); + expect(droidPath("/Users/a", "darwin")?.configPath).toBe( + join("/Users/a", ".factory", "mcp.json"), + ); + expect(droidPath("C:\\Users\\a", "win32")?.configPath).toBe( + join("C:\\Users\\a", ".factory", "mcp.json"), + ); + }); + + it("resolves the personal skills directory beside the config root", () => { + expect( + skillDestinations("/home/a", ["droid"]).map(({ path }) => path), + ).toEqual([ + join("/home/a", ".factory", "skills", "reverse-engineer-anything"), + ]); + }); +}); + +describe("Factory Droid JSON registration", () => { + const droidClient = async () => { + const home = await createTestTempDirectory("rea-droid-"); + const client = droidPath(home, process.platform); + if (client?.markerPath === undefined) + throw new Error("missing Factory Droid client"); + await mkdir(client.markerPath, { recursive: true }); + return { home, client }; + }; + + it("upserts rea into mcpServers while preserving unrelated servers and permissions", async () => { + const { home, client } = await droidClient(); + await writeFile( + client.configPath, + `{ + "mcpServers": { + "other": { + "type": "stdio", + "command": "other", + "args": [], + "disabled": false + } + }, + "persistentPermissions": { + "servers": { + "other": { "approvedAt": "2026-01-01T00:00:00.000Z" } + } + } +}`, + ); + + expect( + (await configureClientConfiguration(client, {}, command)).status, + ).toBe("configured"); + const parsed = JSON.parse(await readFile(client.configPath, "utf8")); + expect(parsed.mcpServers.other).toEqual({ + type: "stdio", + command: "other", + args: [], + disabled: false, + }); + expect(parsed.persistentPermissions).toEqual({ + servers: { other: { approvedAt: "2026-01-01T00:00:00.000Z" } }, + }); + expect(parsed.mcpServers.rea).toEqual({ + command: command[0], + args: command.slice(1), + }); + + expect(await configureClientConfiguration(client, {}, command)).toEqual({ + status: "unchanged", + }); + expect( + ( + await readClientRegistrationStatuses(home, undefined, { + platform: process.platform, + environment: {}, + }) + ).find(({ client }) => client === "droid"), + ).toMatchObject({ state: "aligned", command }); + }); + + it("creates the mcpServers table in an existing config file without servers", async () => { + const { client } = await droidClient(); + await writeFile( + client.configPath, + JSON.stringify({ persistentPermissions: { servers: {} } }, null, 2), + ); + expect( + (await configureClientConfiguration(client, {}, command)).status, + ).toBe("configured"); + const parsed = JSON.parse(await readFile(client.configPath, "utf8")); + expect(parsed.persistentPermissions).toEqual({ servers: {} }); + expect(parsed.mcpServers.rea).toEqual({ + command: command[0], + args: command.slice(1), + }); + }); + + it("registers into an empty config file", async () => { + const { client } = await droidClient(); + await writeFile(client.configPath, ""); + expect( + (await configureClientConfiguration(client, {}, command)).status, + ).toBe("configured"); + const parsed = JSON.parse(await readFile(client.configPath, "utf8")); + expect(parsed.mcpServers.rea).toEqual({ + command: command[0], + args: command.slice(1), + }); + }); + + it("removes only the rea entry on uninstall", async () => { + const { home, client } = await droidClient(); + await writeFile( + client.configPath, + JSON.stringify( + { mcpServers: { other: { command: "other-server" } } }, + null, + 2, + ), + ); + await configureClientConfiguration(client, {}, command); + + expect((await systemUninstallHost(home).removeClient(client)).status).toBe( + "removed", + ); + const parsed = JSON.parse(await readFile(client.configPath, "utf8")); + expect(parsed.mcpServers.other).toEqual({ command: "other-server" }); + expect(parsed.mcpServers.rea).toBeUndefined(); + }); +}); From 811bd9f6353748d01023260681f138fdf81313c9 Mon Sep 17 00:00:00 2001 From: Jose Camargo Date: Tue, 13 Oct 2026 10:34:59 -0300 Subject: [PATCH 22/24] docs(contributing): explain Windows native prerequisites for source checkouts (#2295) --- CONTRIBUTING.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a8d5f3635..50674afa7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -142,6 +142,21 @@ caches deterministic builds and static checks across Git worktrees. After a pack Repository invariants live in [AGENTS.md](AGENTS.md). Keep provider-specific protocols in adapters and shared CLI/MCP workflows in the application layer. +### Windows source checkouts + +The published npm package includes the Windows x64 native addon and its +manifest. A source checkout does not: `native/windows/build/` is ignored by +Git, and neither `npm ci` nor `npm run build:cached` (or `npm run build`) +produces `rea-windows-x64.node` or `manifest.json` in that directory. +Windows workflows and tests that require native controls need these artifacts; +without them, those controls are unavailable even after a successful runtime build. + +Follow the [Windows native build guide](native/windows/README.md#build) before +running those workflows from source. The artifact build requires Linux, +MinGW-w64, and `tar`; the build script does not run on Windows. See that guide +for the build and artifact-verification commands and the separate real Windows +verification prerequisites. Keep the generated addon and manifest uncommitted. + ## Documentation website The VitePress site uses the Markdown files in `docs/`. Run `npm run docs:dev` From 5dd2ce3125b0c7918bffe99011ccacbba6592853 Mon Sep 17 00:00:00 2001 From: Max <68940212+AS9929@users.noreply.github.com> Date: Tue, 13 Oct 2026 21:36:52 +0800 Subject: [PATCH 23/24] fix(ida): let one-shot commands exit after retaining cleanup (#1841) * fix(ida): let one-shot commands exit after retaining cleanup When a headless IDA request outlives timeoutMs (for example idb_open on a large binary still running auto-analysis), REA cannot prove the worker stopped and correctly retains the upstream connection and private workspace. The retained stdio child and its pipes stayed referenced, so a one-shot CLI printed its cleanup_incomplete result and then never exited, keeping the supervisor, worker and database alive. Detach the retained connection's child process and pipes from the event loop when cleanup fails. A long-lived MCP server can still retry through the retained connection; a one-shot CLI exits with status 1, and the supervisor observes end of input. Co-Authored-By: Claude Opus 5.5 * fix(ida): report an unverified upstream worker after failed cleanup Upstream ida-pro-mcp spawns headless workers in their own session and deliberately leaves them running when its supervisor exits, so neither a retained connection nor REA exiting proves that a timed-out worker stopped. When cleanup fails before the owned worker's release is confirmed, the cleanup diagnostics now name its private session and state that it may still be running until it finishes and reaches its idle timeout. Failures after a confirmed release, or before any open was attempted, do not report a worker. Co-Authored-By: Claude Opus 5.5 * fix(ida): name the inventory sessions that block worker release When idb_close confirmed the requested worker gone but the follow-up inventory still found a different peer on the private input, the upstream_worker diagnostic named the released session and could misdirect manual cleanup. Record the session ids of the inventory entries that actually blocked release and report those; fall back to the requested private session only when no inventory evidence exists (an incomplete open or an unanswered request). The field becomes session_ids, since more than one entry can block release. Co-Authored-By: Claude Opus 5.5 * fix(ida): keep blocker evidence and backend in cleanup diagnostics The release diagnostic reset its observed blockers on every close, so a retry whose inventory was unavailable fell back to the already-released session; and it kept only session ids, describing a blocking GUI session with detached-worker guidance. Blocker evidence is now replaced only by a successful inventory, and the diagnostic becomes upstream_sessions with each session's id and backend, with guidance that distinguishes workers from GUI sessions. Co-Authored-By: Claude Opus 5.5 * fix(ida): stop naming a worker after its close was confirmed When idb_close confirmed the requested worker's release but the following inventory failed, no blocker evidence existed yet and the diagnostic fell back to the confirmed-absent worker. After a confirmed close the diagnostic now reports an empty session list, meaning any remaining same-input sessions are unknown. Co-Authored-By: Claude Opus 5.5 * test(ida): group cleanup diagnostic cases in their own suite The added cases pushed the reaped-worker describe block past the 150-line limit enforced by lint:built. Co-Authored-By: Claude Opus 5.5 * fix(ida): keep ownership and pre-close peers in cleanup diagnostics Blocking sessions dropped the parsed ownership fact, so an external unowned worker looked the same as REA's own; and recovery inventory recorded same-input peers only when the requested worker was missing, so a peer seen beside it was lost if the following close failed. Each reported session now carries owned (null when no inventory has reported it), and recovery records every same-input session until a later successful inventory replaces that evidence. Co-Authored-By: Claude Opus 5.5 * fix(ida): drop a confirmed worker from retained cleanup blockers Recovery inventory now records the requested worker beside any peers, so once idb_close confirmed its release a failing follow-up inventory still reported it as unreleased. Remove the confirmed session from the retained blockers at close confirmation and keep the remaining peers. Co-Authored-By: Claude Opus 5.5 * test(ida): retry cleanup through a detached stdio connection The retry coverage used a recording connection whose detach() only counts calls, so it could not show that detaching keeps a retained connection usable. Run a real stdio supervisor in a long-lived host, fail the first post-close inventory, and require the retry through the same detached connection to confirm release and remove the workspace. The test fails if detach() destroys the pipes instead of unref-ing them. Co-Authored-By: Claude Opus 5.5 * fix(ida): close the stdio supervisor in failed retry tests Merge main and give its new IdaNativeProtocol connection double the required detach() method. The real-stdio retry test now closes its connection in an unconditional teardown, so a failing assertion cannot leave the supervisor waiting on stdin. Co-Authored-By: Claude Opus 5.5 * test(ida): share the cleanup diagnostic expectation in session tests Repeated upstream_sessions assertions and the unrelated live peer literal pushed repository duplication over the jscpd threshold. Build the expected diagnostic with one helper and reuse one peer constant. Co-Authored-By: Claude Opus 5.5 * test(ida): distinguish observed ownership from unreported input bindings * fix(ida): keep requested-ID conflicts in recovery cleanup Recovery inventory kept only same-input entries, unlike the startup and post-close filters. After an explicitly rejected open, an entry under the requested session ID on another input was dropped, so cleanup could close the connection, remove the private workspace and report success while that session stayed registered. Match the requested ID as well so the entry is retained as a blocker and cleanup stays incomplete. Also split the cleanup diagnostics suite under the lint:built function length limit, and make two startup-inventory fixtures pass a defined session id so the test file typechecks. Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Xingchuan Ma Co-authored-by: Claude Opus 5.5 Co-authored-by: morluto --- src/ida/IdaBusyHealth.test.ts | 1 + src/ida/IdaMcpConnection.ts | 17 ++ src/ida/IdaNativeProtocol.test.ts | 1 + src/ida/IdaSessionClient.test.ts | 251 ++++++++++++++++++ src/ida/IdaSessionClient.ts | 127 +++++++-- src/ida/IdaWorkspaceOwnership.test.ts | 6 + .../cli/idaRetainedCleanupCli.test.ts | 92 +++++++ .../ida/retainedConnectionRetry.test.ts | 114 ++++++++ tests/fixtures/idaMcp.ts | 4 + 9 files changed, 596 insertions(+), 17 deletions(-) create mode 100644 tests/boundary/cli/idaRetainedCleanupCli.test.ts create mode 100644 tests/boundary/providers/ida/retainedConnectionRetry.test.ts diff --git a/src/ida/IdaBusyHealth.test.ts b/src/ida/IdaBusyHealth.test.ts index 512cf78a6..b997b2591 100644 --- a/src/ida/IdaBusyHealth.test.ts +++ b/src/ida/IdaBusyHealth.test.ts @@ -18,6 +18,7 @@ const scripted = (answers: readonly JsonValue[]) => { connect: async () => [], serverInfo: () => null, close: async () => undefined, + detach: () => undefined, call: async () => { reads.push(reads.length); const answer = answers[Math.min(reads.length - 1, answers.length - 1)]; diff --git a/src/ida/IdaMcpConnection.ts b/src/ida/IdaMcpConnection.ts index ee7861389..d532a66e8 100644 --- a/src/ida/IdaMcpConnection.ts +++ b/src/ida/IdaMcpConnection.ts @@ -1,3 +1,5 @@ +import { ChildProcess } from "node:child_process"; +import { Socket } from "node:net"; import { snapshotEnvironment } from "../process/snapshotEnvironment.js"; import { Client, @@ -22,6 +24,12 @@ export interface IdaMcpConnection { readResource?(uri: string): Promise; serverInfo(): JsonValue; close(): Promise; + /** + * Keep a connection retained after failed cleanup from holding the host + * process open. A long-lived MCP server can still retry through it; a + * one-shot CLI exits, and the upstream proxy observes end of input. + */ + detach(): void; } const toolResultSchema = z.object({ @@ -205,5 +213,14 @@ export const createIdaMcpConnection = ( }, serverInfo: () => jsonValueSchema.parse(client.getServerVersion() ?? null), close: () => client.close().catch(transportFailure), + detach() { + if (!(transport instanceof StdioClientTransport)) return; + // The SDK exposes only the child's pid, not the handles it holds. + const child: unknown = Reflect.get(transport, "_process"); + if (!(child instanceof ChildProcess)) return; + child.unref(); + for (const stream of [child.stdin, child.stdout, child.stderr]) + if (stream instanceof Socket) stream.unref(); + }, }; }; diff --git a/src/ida/IdaNativeProtocol.test.ts b/src/ida/IdaNativeProtocol.test.ts index 5da9d40f7..dc95727f6 100644 --- a/src/ida/IdaNativeProtocol.test.ts +++ b/src/ida/IdaNativeProtocol.test.ts @@ -90,6 +90,7 @@ const fixture = async () => { }, serverInfo: () => producer.serverInfo(), close: () => producer.close(), + detach: () => producer.detach(), }; const client = new IdaSessionClient( { diff --git a/src/ida/IdaSessionClient.test.ts b/src/ida/IdaSessionClient.test.ts index b41a3c34a..7fa366bce 100644 --- a/src/ida/IdaSessionClient.test.ts +++ b/src/ida/IdaSessionClient.test.ts @@ -16,6 +16,18 @@ afterEach(async () => { for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }); }); +/** A failed close whose diagnostic names exactly these unverified sessions. */ +const reportsSessions = (...sessions: readonly object[]) => ({ + error: { + diagnostics: { upstream_sessions: { state: "unverified", sessions } }, + }, +}); +const LIVE_PEER = { + session_id: "unrelated-live-peer", + backend: "worker", + owned: false, +} as const; + const fixture = async ( mode: "attached" | "headless" = "attached", timeoutMs = 1000, @@ -792,10 +804,19 @@ describe("headless IDA session ownership and cleanup", () => { it("retains the private workspace and reports cleanup failure when worker release is unverified", async () => { const { producer, client, root } = await fixture("headless"); await client.execute("health", {}); + const sessionId = producer.database; producer.failClose = true; const result = await client.close(); expect(result.ok).toBe(false); if (!result.ok) expect(result.error.cleanupIncomplete).toBe(true); + expect(result).toMatchObject( + reportsSessions({ + session_id: sessionId, + backend: "worker", + owned: true, + input_path: producer.inputPath, + }), + ); expect( (await readdir(root)).some((name) => name.startsWith("rea-ida-")), ).toBe(true); @@ -843,10 +864,20 @@ describe("headless IDA reaped worker cleanup", () => { const result = await client.close(); expect(result.ok).toBe(false); if (!result.ok) expect(result.error.cleanupIncomplete).toBe(true); + // The released session must not be named as the remaining worker. + expect(result).toMatchObject(reportsSessions(LIVE_PEER)); expect(producer.closes).toBe(0); expect( (await readdir(root)).some((name) => name.startsWith("rea-ida-")), ).toBe(true); + + // A retry whose inventory is unavailable keeps the observed blocker. + producer.beforeCall = async (name) => { + if (name === "idb_list") throw new Error("inventory unavailable"); + }; + const retried = await client.close(); + expect(retried.ok).toBe(false); + expect(retried).toMatchObject(reportsSessions(LIVE_PEER)); }); it("retries independent inventory verification after an absent-session response", async () => { @@ -899,6 +930,203 @@ describe("headless IDA reaped worker cleanup", () => { }); }); +/** A rejected open response, with helpers for separately inventoried workers. */ +const rejectedOpenFixture = async () => { + const { producer, client } = await fixture("headless"); + producer.overrideCall = (name) => + name === "idb_open" ? { error: "open did not complete" } : undefined; + expect((await client.execute("health", {})).ok).toBe(false); + const open = producer.calls.find(({ name }) => name === "idb_open")?.args; + const entry = (session_id: string, owned: boolean) => ({ + session_id, + input_path: String(open?.input_path), + owned, + backend: "worker", + is_active: true, + }); + return { + producer, + client, + requested: String(open?.preferred_session_id), + entry, + }; +}; + +describe("headless IDA cleanup diagnostics", () => { + it("does not name a worker whose close was confirmed when inventory then fails", async () => { + const { producer, client } = await fixture("headless"); + expect((await client.execute("health", {})).ok).toBe(true); + producer.beforeCall = async (name) => { + if (name === "idb_list") throw new Error("inventory unavailable"); + }; + + const result = await client.close(); + expect(result.ok).toBe(false); + expect(producer.calls.map(({ name }) => name)).toContain("idb_close"); + expect(result).toMatchObject(reportsSessions()); + }); + + it("retains peers and confirmed ownership from successful startup when close fails", async () => { + const { producer, client } = await fixture("headless"); + producer.overrideCall = (name) => { + if (name === "idb_list") + return { + sessions: [ + { + session_id: String(producer.database), + input_path: producer.inputPath, + owned: true, + backend: "worker", + is_active: true, + }, + { + session_id: "startup-peer", + input_path: producer.inputPath, + owned: false, + backend: "worker", + is_active: false, + }, + ], + }; + if (name === "idb_close") return { error: "close rejected" }; + return undefined; + }; + expect((await client.execute("health", {})).ok).toBe(true); + const result = await client.close(); + expect(result).toMatchObject( + reportsSessions( + { + session_id: producer.database, + backend: "worker", + owned: true, + input_path: producer.inputPath, + }, + { + session_id: "startup-peer", + backend: "worker", + owned: false, + input_path: producer.inputPath, + }, + ), + ); + }); + + it("preserves an observed conflicting input binding after close", async () => { + const { producer, client } = await fixture("headless"); + expect((await client.execute("health", {})).ok).toBe(true); + const requested = String(producer.database); + producer.overrideCall = (name) => + name === "idb_list" + ? { + sessions: [ + { + session_id: requested, + input_path: "/different/input", + owned: false, + backend: "worker", + is_active: true, + }, + ], + } + : undefined; + const result = await client.close(); + expect(result).toMatchObject( + reportsSessions({ + session_id: requested, + backend: "worker", + owned: false, + input_path: "/different/input", + }), + ); + }); + + it("keeps peers seen beside the requested worker when its close fails", async () => { + const { producer, client, requested, entry } = await rejectedOpenFixture(); + producer.overrideCall = (name) => { + if (name === "idb_list") + return { + sessions: [ + entry(requested, true), + entry("unrelated-live-peer", false), + ], + }; + if (name === "idb_close") return { error: "close rejected" }; + return undefined; + }; + + const result = await client.close(); + expect(result.ok).toBe(false); + expect(result).toMatchObject( + reportsSessions( + { session_id: requested, backend: "worker", owned: true }, + LIVE_PEER, + ), + ); + }); +}); + +describe("headless IDA cleanup diagnostics for observed peers", () => { + it("drops the requested worker from blockers once its close is confirmed", async () => { + const { producer, client, requested, entry } = await rejectedOpenFixture(); + let inventories = 0; + producer.beforeCall = async (name) => { + if (name === "idb_list" && ++inventories > 1) + throw new Error("inventory unavailable"); + }; + producer.overrideCall = (name) => { + if (name === "idb_list") + return { + sessions: [ + entry(requested, true), + entry("unrelated-live-peer", false), + ], + }; + if (name === "idb_close") + return { + success: true, + session_id: requested, + saved: false, + owned: true, + backend: "worker", + }; + return undefined; + }; + + const result = await client.close(); + expect(result.ok).toBe(false); + expect(result).toMatchObject(reportsSessions(LIVE_PEER)); + }); + + it("reports a blocking GUI session by its backend", async () => { + const { producer, client } = await fixture("headless"); + expect((await client.execute("health", {})).ok).toBe(true); + const sessionId = producer.database; + producer.overrideCall = (name) => { + if (name === "idb_close") + return { error: `Session not found: ${sessionId}` }; + if (name === "idb_list") + return { + sessions: [ + { + session_id: "", + input_path: producer.inputPath, + owned: false, + backend: "gui", + is_active: true, + }, + ], + }; + return undefined; + }; + + const result = await client.close(); + expect(result.ok).toBe(false); + expect(result).toMatchObject( + reportsSessions({ session_id: "", backend: "gui", owned: false }), + ); + }); +}); + /** A completed open response is not proof that its private Worker was ever released. */ describe("headless IDA unresolved successful-open ownership", () => { it.each(["matching", "foreign-input", "malformed"] as const)( @@ -976,6 +1204,29 @@ describe("headless IDA unresolved successful-open ownership", () => { expect(producer.closes).toBe(1); expect(await readdir(root)).toEqual(["target.elf"]); }); + + it("retains cleanup when a rejected open leaves the requested ID on another input", async () => { + const { producer, client, requested } = await rejectedOpenFixture(); + producer.overrideCall = (name) => + name === "idb_list" + ? { + sessions: [ + { + session_id: requested, + input_path: "/different/input", + owned: false, + backend: "worker", + is_active: true, + }, + ], + } + : undefined; + + const result = await client.close(); + expect(result.ok).toBe(false); + if (!result.ok) expect(result.error.cleanupIncomplete).toBe(true); + expect(producer.closes).toBe(0); + }); }); describe("headless IDA cleanup verification", () => { diff --git a/src/ida/IdaSessionClient.ts b/src/ida/IdaSessionClient.ts index 14b20b107..dc0ae5d6f 100644 --- a/src/ida/IdaSessionClient.ts +++ b/src/ida/IdaSessionClient.ts @@ -93,11 +93,25 @@ interface IdaOperationFailureContext { parameters: Readonly>; } +/** Upstream session ids of inventory entries, omitting unreported ones. */ +interface UpstreamSession { + readonly session_id: string; + readonly backend: "worker" | "gui"; + /** Upstream ownership; null when no inventory has reported it. */ + readonly owned: boolean | null; + /** Last inventoried input binding; null until upstream reports it. */ + readonly input_path: string | null; +} + /** A failed cleanup retains its private workspace and reports the remaining resource. */ class IdaCleanupError extends ProviderAdapterError { override readonly cleanupIncomplete = true; override readonly cleanupResources: readonly string[]; - constructor(path: string, cause: unknown) { + constructor( + path: string, + cause: unknown, + sessions?: readonly UpstreamSession[], + ) { super("ida", "close", { cause, diagnostics: { @@ -105,6 +119,23 @@ class IdaCleanupError extends ProviderAdapterError { "IDA cleanup could not be verified; the remaining connection or private workspace is retained for retry.", failure_reason: cause instanceof Error ? cause.message : String(cause), workspace: path, + ...(sessions === undefined + ? {} + : { + upstream_sessions: { + state: "unverified", + sessions: sessions.map( + ({ session_id, backend, owned, input_path }) => ({ + session_id, + backend, + owned, + input_path, + }), + ), + reason: + "Release of upstream sessions matching the requested session ID or private input was not confirmed; input_path preserves the last observed binding and is null when no inventory has reported it. A worker outlives its supervisor and may keep running until it finishes and reaches its idle timeout; exiting REA does not stop it. A gui session belongs to an IDA instance REA does not control. An empty session_id means upstream reported none; owned is null when no inventory has reported ownership; an empty list means the requested worker's close was confirmed but no inventory has identified any remaining session.", + }, + }), }, }); this.cleanupResources = [path]; @@ -123,6 +154,7 @@ export class IdaSessionClient implements AnalysisClient { #databaseCloseCompleted = false; #openAttempted = false; #openCompleted = false; + #blockingSessions: readonly UpstreamSession[] | undefined; #openRejected = false; #callGraphAdvertised = false; #surveyAdvertised = false; @@ -490,6 +522,14 @@ export class IdaSessionClient implements AnalysisClient { await this.connection.call("idb_list", {}), ); this.#lifecycle.push(jsonValueSchema.parse(inventory)); + const workspace = this.#workspace; + this.#recordBlockingSessions( + inventory.sessions.filter( + (entry) => + entry.session_id === workspace.sessionId || + sameIdaHostPath(entry.input_path, workspace.inputPath), + ), + ); const entry = inventory.sessions.find( ({ session_id }) => session_id === opened.session.session_id, ); @@ -589,6 +629,52 @@ export class IdaSessionClient implements AnalysisClient { return shared; } + /** + * Upstream sessions whose release cleanup has not confirmed: the entries + * the latest successful inventory found blocking release; else the + * requested private worker until its close is confirmed, and then none + * known. + */ + #unreleasedSessions(): readonly UpstreamSession[] | undefined { + if (this.#blockingSessions !== undefined) return this.#blockingSessions; + if (!this.#openAttempted || this.#workspace === undefined) return undefined; + return this.#databaseCloseCompleted + ? [] + : [ + { + session_id: this.#workspace.sessionId, + backend: "worker", + owned: null, + input_path: null, + }, + ]; + } + + /** Drop a session whose release upstream confirmed, keeping other blockers. */ + #forgetReleasedSession(sessionId: string): void { + const remaining = this.#blockingSessions?.filter( + ({ session_id }) => session_id !== sessionId, + ); + this.#blockingSessions = + remaining !== undefined && remaining.length > 0 ? remaining : undefined; + } + + /** Replace blocker evidence from a successful inventory; true when blocked. */ + #recordBlockingSessions( + entries: readonly (UpstreamSession & { readonly owned: boolean })[], + ): boolean { + this.#blockingSessions = + entries.length > 0 + ? entries.map(({ session_id, backend, owned, input_path }) => ({ + session_id, + backend, + owned, + input_path, + })) + : undefined; + return entries.length > 0; + } + async #closeDatabase(database: string): Promise { const response = await this.connection.call("idb_close", { database, @@ -629,12 +715,17 @@ export class IdaSessionClient implements AnalysisClient { entry.session_id === workspace.sessionId && sameIdaHostPath(entry.input_path, workspace.inputPath), )?.session_id; - if ( - this.#database === undefined && - inventory.sessions.some((entry) => - sameIdaHostPath(entry.input_path, workspace.inputPath), - ) - ) + // Every same-input session and any entry under the requested ID, + // whatever its input, stays reported until a later successful + // inventory confirms release. + const blocked = this.#recordBlockingSessions( + inventory.sessions.filter( + (entry) => + entry.session_id === workspace.sessionId || + sameIdaHostPath(entry.input_path, workspace.inputPath), + ), + ); + if (this.#database === undefined && blocked) throw new AnalysisProtocolError( "An unowned worker still uses the private workspace; it was retained.", ); @@ -651,20 +742,20 @@ export class IdaSessionClient implements AnalysisClient { if (!this.#databaseCloseCompleted) { await this.#closeDatabase(this.#database); this.#databaseCloseCompleted = true; + this.#forgetReleasedSession(this.#database); } const inventory = databaseListSchema.parse( await this.connection.call("idb_list", {}), ); - if ( - inventory.sessions.some( - (entry) => - entry.session_id === this.#database || - sameIdaHostPath( - entry.input_path, - this.#workspace?.inputPath ?? "", - ), - ) - ) + const remaining = inventory.sessions.filter( + (entry) => + entry.session_id === this.#database || + sameIdaHostPath( + entry.input_path, + this.#workspace?.inputPath ?? "", + ), + ); + if (this.#recordBlockingSessions(remaining)) throw new AnalysisProtocolError( "IDA worker remains registered after close.", ); @@ -681,10 +772,12 @@ export class IdaSessionClient implements AnalysisClient { this.#closed = true; return ok(null); } catch (cause: unknown) { + this.connection.detach(); return err( new IdaCleanupError( this.#workspace?.directory ?? "IDA MCP connection", cause, + this.#unreleasedSessions(), ), ); } diff --git a/src/ida/IdaWorkspaceOwnership.test.ts b/src/ida/IdaWorkspaceOwnership.test.ts index b39c6e916..1f4a10d65 100644 --- a/src/ida/IdaWorkspaceOwnership.test.ts +++ b/src/ida/IdaWorkspaceOwnership.test.ts @@ -98,11 +98,17 @@ describe("IDA workspace ownership before input preparation", () => { expect(closed.error.cleanupResources).toEqual([ expect.stringContaining(workspace ?? "missing workspace"), ]); + // No open was attempted, so there is no worker to report. + expect(closed.error).not.toHaveProperty( + "diagnostics.upstream_sessions", + ); } + expect(producer.detaches).toBe(1); } finally { await chmod(root, 0o700); } expect((await client.close()).ok).toBe(true); + expect(producer.detaches).toBe(1); expect(await readdir(root)).toEqual(["target.elf"]); }, ); diff --git a/tests/boundary/cli/idaRetainedCleanupCli.test.ts b/tests/boundary/cli/idaRetainedCleanupCli.test.ts new file mode 100644 index 000000000..b31903489 --- /dev/null +++ b/tests/boundary/cli/idaRetainedCleanupCli.test.ts @@ -0,0 +1,92 @@ +import { describe, expect } from "vitest"; + +import { workspaceCliTest } from "../../support/cli/workspaceCliFixture.js"; + +import { elf } from "../../../src/domain/binaryTarget.fixture.js"; + +/** + * A headless IDA supervisor that completes the MCP handshake and then leaves + * every tool call unanswered, as an `idb_open` still running auto-analysis + * would. It exits when REA closes its input. + */ +const UNRESPONSIVE_SUPERVISOR = `import { createInterface } from "node:readline"; +const tools = ["idb_open", "idb_list", "idb_close", "server_health", "list_funcs", + "lookup_funcs", "find_regex", "decompile", "disasm", "callees", "xrefs_to"]; +const send = (message) => + process.stdout.write(JSON.stringify({ jsonrpc: "2.0", ...message }) + "\\n"); +for await (const line of createInterface({ input: process.stdin })) { + const request = JSON.parse(line); + if (request.method === "initialize") + send({ id: request.id, result: { + protocolVersion: request.params.protocolVersion, + capabilities: { tools: {} }, + serverInfo: { name: "unresponsive-ida-fixture", version: "1" }, + } }); + else if (request.method === "tools/list") + send({ id: request.id, result: { + tools: tools.map((name) => ({ name, inputSchema: { type: "object" } })), + } }); +} +`; + +describe("headless IDA timeout through the one-shot CLI", () => { + workspaceCliTest( + "exits after a retained-cleanup failure instead of waiting on the supervisor", + async ({ cli, workspace }) => { + const supervisor = await workspace.write( + "supervisor.mjs", + UNRESPONSIVE_SUPERVISOR, + ); + const workspaceRoot = await workspace.mkdir("ida-workspaces"); + const registration = await workspace.write( + "ida-mcp.json", + JSON.stringify({ + command: process.execPath, + args: [supervisor], + mode: "headless", + timeoutMs: 500, + workspaceRoot, + }), + ); + const target = await workspace.write("target.elf", elf(2, 1, 62)); + + const result = await cli.run({ + arguments: ["decompile", target, "main", "--json"], + environment: { + ...workspace.environment, + REA_ANALYSIS_PROVIDER: "ida", + REA_IDA_MCP_CONFIG: registration, + REA_LOG_LEVEL: "silent", + }, + timeoutMs: 30_000, + }); + + expect(result.exitCode).toBe(1); + expect(result.json).toMatchObject({ + code: "cleanup_incomplete", + details: { + provider_id: "ida", + cleanup: "incomplete", + resources: [expect.stringContaining(workspaceRoot)], + cleanup_error: { + details: { + diagnostics: { + upstream_sessions: { + state: "unverified", + sessions: [ + { + session_id: expect.stringMatching(/^rea-/), + backend: "worker", + owned: null, + input_path: null, + }, + ], + }, + }, + }, + }, + }, + }); + }, + ); +}); diff --git a/tests/boundary/providers/ida/retainedConnectionRetry.test.ts b/tests/boundary/providers/ida/retainedConnectionRetry.test.ts new file mode 100644 index 000000000..4f9092842 --- /dev/null +++ b/tests/boundary/providers/ida/retainedConnectionRetry.test.ts @@ -0,0 +1,114 @@ +import { createHash } from "node:crypto"; +import { mkdir, readdir, writeFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { describe, expect, it, onTestFinished } from "vitest"; + +import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js"; + +import { elf } from "../../../../src/domain/binaryTarget.fixture.js"; +import type { BinaryTarget } from "../../../../src/domain/binaryTargetTypes.js"; +import { createIdaMcpConnection } from "../../../../src/ida/IdaMcpConnection.js"; +import { IdaSessionClient } from "../../../../src/ida/IdaSessionClient.js"; + +/** + * A headless IDA supervisor over real stdio that answers the session + * lifecycle, but rejects the first inventory after a close so cleanup fails + * once and must be retried through the retained connection. + */ +const FLAKY_INVENTORY_SUPERVISOR = `import { createInterface } from "node:readline"; +const tools = ["idb_open", "idb_list", "idb_close", "server_health", "list_funcs", + "lookup_funcs", "find_regex", "decompile", "disasm", "callees", "xrefs_to"]; +let session; +let inputPath; +let closed = false; +let inventoryFailed = false; +const send = (message) => + process.stdout.write(JSON.stringify({ jsonrpc: "2.0", ...message }) + "\\n"); +const result = (value) => ({ + content: [{ type: "text", text: JSON.stringify(value) }], + structuredContent: value, +}); +const call = ({ name, arguments: args }) => { + if (name === "idb_open") { + session = args.preferred_session_id; + inputPath = args.input_path; + return result({ success: true, session: { session_id: session, input_path: inputPath } }); + } + if (name === "server_health") + return result({ status: "ok", input_path: inputPath, idb_path: inputPath + ".i64", + module: "target.elf", imagebase: "0x1000", auto_analysis_ready: true }); + if (name === "idb_close") { + closed = true; + return result({ success: true, session_id: session, saved: null, owned: true, backend: "worker" }); + } + if (name === "idb_list") { + if (closed && !inventoryFailed) { + inventoryFailed = true; + return { isError: true, content: [{ type: "text", text: "inventory unavailable" }] }; + } + return result({ sessions: closed ? [] : [{ session_id: session, input_path: inputPath, + owned: true, backend: "worker", is_active: true }] }); + } + return { isError: true, content: [{ type: "text", text: "unsupported " + name }] }; +}; +for await (const line of createInterface({ input: process.stdin })) { + const request = JSON.parse(line); + if (request.method === "initialize") + send({ id: request.id, result: { + protocolVersion: request.params.protocolVersion, + capabilities: { tools: {} }, + serverInfo: { name: "flaky-inventory-ida-fixture", version: "1" }, + } }); + else if (request.method === "tools/list") + send({ id: request.id, result: { + tools: tools.map((name) => ({ name, inputSchema: { type: "object" } })), + } }); + else if (request.method === "tools/call") + send({ id: request.id, result: call(request.params) }); +} +`; + +describe("retained headless IDA connection in a long-lived host", () => { + it("retries cleanup through the detached stdio connection", async () => { + const root = await createTestTempDirectory("rea-ida-retry-"); + const supervisor = join(root, "supervisor.mjs"); + await writeFile(supervisor, FLAKY_INVENTORY_SUPERVISOR); + const workspaceRoot = join(root, "workspaces"); + await mkdir(workspaceRoot); + const bytes = elf(2, 1, 62); + const targetPath = join(root, "target.elf"); + await writeFile(targetPath, bytes); + const target: BinaryTarget = { + path: targetPath, + sha256: createHash("sha256").update(bytes).digest("hex"), + kind: "executable", + format: "elf", + architecture: "x86_64", + availableArchitectures: ["x86_64"], + }; + const config = { + command: process.execPath, + args: [supervisor], + env: {}, + mode: "headless" as const, + timeoutMs: 5000, + workspaceRoot, + }; + const connection = createIdaMcpConnection(config, process.env); + // Close the supervisor even when an assertion fails before cleanup does. + onTestFinished(() => connection.close().catch(() => undefined)); + const client = new IdaSessionClient(config, target, connection); + + expect((await client.execute("health", {})).ok).toBe(true); + const failed = await client.close(); + expect(failed.ok).toBe(false); + if (!failed.ok) expect(failed.error.cleanupIncomplete).toBe(true); + expect(await readdir(workspaceRoot)).toHaveLength(1); + + // The connection was detached when cleanup failed; it must still serve + // the retry that confirms release and removes the private workspace. + expect((await client.close()).ok).toBe(true); + expect(await readdir(workspaceRoot)).toEqual([]); + }); +}); diff --git a/tests/fixtures/idaMcp.ts b/tests/fixtures/idaMcp.ts index 21e5e38f7..6be35a8bf 100644 --- a/tests/fixtures/idaMcp.ts +++ b/tests/fixtures/idaMcp.ts @@ -68,6 +68,7 @@ export class RecordingIdaMcp implements IdaMcpConnection { }[] = []; connects = 0; closes = 0; + detaches = 0; inputSha256: string; inputPath: string; imageBase = "0x1000"; @@ -104,6 +105,9 @@ export class RecordingIdaMcp implements IdaMcpConnection { async close() { this.closes += 1; } + detach() { + this.detaches += 1; + } async call( name: string, args: Readonly>, From c6503edfc4afdc0de6812ba3f7cd0909df5555f7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Llu=C3=ADs=20Ari=C3=B1o?= Date: Tue, 13 Oct 2026 15:38:46 +0200 Subject: [PATCH 24/24] ci: pin GitHub Actions to full commit SHAs (#1901) * ci: pin GitHub Actions to full commit SHAs Replace every third-party `uses: owner/repo@vN` reference in .github/workflows with the 40-character commit SHA that the tag currently resolves to, keeping the exact release as a trailing comment. Each pin was checked with `git ls-remote --tags`: the floating major tag and the commented release dereference to the same commit. Local reusable workflows are unchanged. Re-pin reactivecircus/android-emulator-runner from the SHA of the annotated `v2` tag object (4c44018e) to the commit it points to (a421e438, v2.38.0); Actions resolves both, but only the commit SHA is an immutable reference that tooling can compare with the release. Update the Windows Ghidra workflow trust-boundary test to require SHA-pinned setup-node and upload-artifact references instead of the previous floating tags. Validation: actionlint v1.7.7 (-shellcheck= -pyflakes=, as in CI), YAML parse of all workflows, no remaining unpinned third-party `uses:`, and the workflow boundary/conformance tests. * style(ci): format action identity assertion Retain formatting edits produced before the user stopped validation. Source inspection and git diff --check only. Tests, builds and typechecks skipped by explicit user request. GitHub review writes paused globally. --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> --- .../actions/portable-build-cache/action.yml | 6 +- .github/workflows/ci-timings.yml | 6 +- .github/workflows/ci.yml | 96 +++++++++---------- .github/workflows/nativeaot-fixtures.yml | 8 +- .github/workflows/pages.yml | 4 +- .github/workflows/real-binary-layout.yml | 4 +- .github/workflows/real-browser.yml | 4 +- .github/workflows/real-evm-interface.yml | 4 +- .github/workflows/real-ghidra-windows.yml | 14 +-- .github/workflows/real-go-binary.yml | 6 +- .github/workflows/real-hopper-linux.yml | 8 +- .github/workflows/real-hopper.yml | 4 +- .../workflows/real-javascript-recovery.yml | 4 +- .github/workflows/real-packet-capture.yml | 4 +- .github/workflows/real-recorded-crash.yml | 4 +- .github/workflows/real-signal-recording.yml | 4 +- .github/workflows/real-termux.yml | 6 +- .../workflows/real-web-network-captures.yml | 4 +- .github/workflows/real-web-runtime.yml | 4 +- .github/workflows/real-web-source-map.yml | 4 +- .github/workflows/release.yml | 26 ++--- .github/workflows/sqlite-runtime.yml | 6 +- .github/workflows/website-check.yml | 4 +- .github/workflows/website-pages.yml | 18 ++-- .../filesystem/ciRequiredLanes.test.ts | 4 +- .../ghidra/realGhidraWindowsWorkflow.test.ts | 8 +- 26 files changed, 134 insertions(+), 130 deletions(-) diff --git a/.github/actions/portable-build-cache/action.yml b/.github/actions/portable-build-cache/action.yml index cee775ec5..799b9476b 100644 --- a/.github/actions/portable-build-cache/action.yml +++ b/.github/actions/portable-build-cache/action.yml @@ -12,7 +12,7 @@ runs: steps: - name: Download portable build outputs from this workflow run if: inputs.build_cache_artifact != '' - uses: actions/download-artifact@v4 + uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: name: ${{ inputs.build_cache_artifact }} path: .turbo/cache/ @@ -39,14 +39,14 @@ runs: ' >> "$GITHUB_OUTPUT" - name: Restore portable build outputs if: inputs.build_cache_artifact == '' && inputs.save != 'true' - uses: actions/cache/restore@v4 + uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0 with: path: .turbo/cache/ key: ${{ steps.plan.outputs.key }} restore-keys: ${{ steps.plan.outputs.prefix }} - name: Restore and save portable build outputs if: inputs.build_cache_artifact == '' && inputs.save == 'true' - uses: actions/cache@v4 + uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0 with: path: .turbo/cache/ key: ${{ steps.plan.outputs.key }} diff --git a/.github/workflows/ci-timings.yml b/.github/workflows/ci-timings.yml index 30fa19464..73a9ba4b9 100644 --- a/.github/workflows/ci-timings.yml +++ b/.github/workflows/ci-timings.yml @@ -20,11 +20,11 @@ jobs: timeout-minutes: 5 steps: # Consume only API data and execute batch code from the trusted default branch. - - uses: actions/checkout@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 with: ref: ${{ github.event.repository.default_branch }} persist-credentials: false - - uses: actions/setup-node@v4 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc - name: Record queue, execution, and matched step timing @@ -36,7 +36,7 @@ jobs: - name: Upload CI timing artifacts if: always() && hashFiles('.cache/ci-job-timings/*') != '' continue-on-error: true - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: ci-job-timings-batch path: .cache/ci-job-timings/ diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d2cc59493..142e179bb 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -63,11 +63,11 @@ jobs: matrix: ${{ steps.plan.outputs.matrix }} test_selection: ${{ steps.plan.outputs.test_selection }} steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 with: ref: ${{ inputs.source_sha || github.sha }} fetch-depth: 0 - - uses: actions/setup-node@v4 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc - name: Bind qualification to the caller revision @@ -118,8 +118,8 @@ jobs: env: CHANGES_RESULT: ${{ needs.changes.result }} run: test "${CHANGES_RESULT}" = "success" - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm @@ -138,7 +138,7 @@ jobs: run: npm run format:check - name: Upload complete static diagnostics if: always() && needs.changes.outputs.code == 'true' - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: static-check-reports path: .cache/static-checks/ @@ -157,13 +157,13 @@ jobs: matrix: shard: [1/6, 2/6, 3/6, 4/6, 5/6, 6/6] steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm - run: npm ci - - uses: actions/download-artifact@v4 + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: name: rea-test-runtime path: . @@ -177,7 +177,7 @@ jobs: run: npm run test:run -- --reporter=default --reporter=blob --shard=${{ matrix.shard }} env: REA_CI_TEST_HISTORY: .cache/ci-test-history/test-results.json - - uses: actions/upload-artifact@v4 + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 if: ${{ !cancelled() }} with: name: vitest-report-${{ strategy.job-index }} @@ -208,8 +208,8 @@ jobs: contents: read actions: read steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm @@ -231,13 +231,13 @@ jobs: - name: Download previous timing report if: steps.history.outputs.run-id != '' continue-on-error: true - uses: actions/download-artifact@v4 + uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: name: test-timings run-id: ${{ steps.history.outputs.run-id }} github-token: ${{ github.token }} path: .cache/ci-test-history - - uses: actions/upload-artifact@v4 + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: rea-test-runtime path: | @@ -254,7 +254,7 @@ jobs: # test:prepare caches only portable JS, skill, and catalog outputs. # Native package jobs still run their build and prepack checks; Turbo # restores matching outputs or rebuilds when their inputs differ. - - uses: actions/upload-artifact@v4 + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: rea-build-cache path: .turbo/cache/ @@ -280,16 +280,16 @@ jobs: env: CHANGES_RESULT: ${{ needs.changes.result }} run: test "${CHANGES_RESULT}" = "success" - - uses: actions/checkout@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 if: needs.changes.outputs.code == 'true' && success() - - uses: actions/setup-node@v4 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 if: needs.changes.outputs.code == 'true' && success() with: node-version-file: .nvmrc cache: npm - if: needs.changes.outputs.code == 'true' && success() run: npm ci - - uses: actions/download-artifact@v4 + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 if: needs.changes.outputs.code == 'true' && success() with: path: .vitest/blob @@ -330,7 +330,7 @@ jobs: CI_TEST_SELECTION: ${{ needs.changes.outputs.test_selection }} CI_PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }} run: node scripts/ci/test-report.mjs - - uses: actions/upload-artifact@v4 + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 if: needs.changes.outputs.code == 'true' && !cancelled() && hashFiles('test-results.json') != '' with: name: test-results @@ -345,7 +345,7 @@ jobs: retention-days: 30 - name: Publish successful timing history if: needs.changes.outputs.code == 'true' && needs.changes.outputs.tier == 'full' && success() && needs.test-shard.result == 'success' - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: test-timings path: .cache/ci-test-history/test-results.json @@ -366,7 +366,7 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 10 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 - name: Verify fixtures from a clean checkout run: npm run verify:fixtures @@ -377,8 +377,8 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 10 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc - name: Prepare cross compiler @@ -395,12 +395,12 @@ jobs: - run: npm run build:windows-native - run: npm run verify:windows-native:artifact - run: node scripts/build-windows-native.mjs --inspection-fixture - - uses: actions/upload-artifact@v4 + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: windows-native-inspection-fixture path: .cache/windows-native-inspection-fixture/ if-no-files-found: error - - uses: actions/upload-artifact@v4 + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: windows-native-controls path: native/windows/build/ @@ -423,8 +423,8 @@ jobs: NODE_OPTIONS: "--max-old-space-size=1024" UV_THREADPOOL_SIZE: "2" steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm @@ -438,11 +438,11 @@ jobs: assert.equal(process.arch, process.env.EXPECTED_CI_ARCHITECTURE); console.log(JSON.stringify({ platform: process.platform, architecture: process.arch, node: process.version }));" - run: npm ci - - uses: actions/download-artifact@v4 + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: name: rea-build-cache path: .turbo/cache/ - - uses: actions/download-artifact@v4 + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: name: windows-native-controls path: native/windows/build/ @@ -463,7 +463,7 @@ jobs: run: npm run verify:inspector - name: Retain the exact qualified release package if: needs.changes.outputs.tier == 'full' && matrix.platform == 'linux' && matrix.architecture == 'x64' - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: verified-release-package path: .cache/verified-package/ @@ -479,13 +479,13 @@ jobs: runs-on: macos-15 timeout-minutes: 15 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm - run: npm ci - - uses: actions/download-artifact@v4 + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: name: rea-build-cache path: .turbo/cache/ @@ -527,8 +527,8 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 15 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm @@ -539,7 +539,7 @@ jobs: run: | set -o pipefail npm run verify:readiness | tee readiness-summary.json - - uses: actions/upload-artifact@v4 + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 if: ${{ !cancelled() }} with: name: reconstruction-readiness @@ -556,18 +556,18 @@ jobs: runs-on: windows-latest timeout-minutes: 25 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm - run: npm ci - - uses: actions/download-artifact@v4 + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: name: rea-build-cache path: .turbo/cache/ - run: npm run build:cached - - uses: actions/download-artifact@v4 + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: name: windows-native-controls path: native/windows/build/ @@ -623,7 +623,7 @@ jobs: tests/process-global/updateMaintenance.test.ts tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts - run: npm run verify:windows-native - - uses: actions/download-artifact@v4 + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: name: windows-native-inspection-fixture path: .cache/windows-native-inspection-fixture/ @@ -642,7 +642,7 @@ jobs: tests/boundary/filesystem/directoryArtifactReaderOpen.test.ts tests/boundary/filesystem/javascriptPackagePrecedence.test.ts tests/boundary/filesystem/javascriptArtifactContentIdentity.test.ts - - uses: actions/setup-node@v4 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version: 22.19.0 - name: Verify native path boundaries on Node 22 @@ -652,7 +652,7 @@ jobs: tests/boundary/filesystem/javascriptPackagePrecedence.test.ts tests/boundary/filesystem/javascriptArtifactContentIdentity.test.ts tests/boundary/process/sourceMapUncHosts.test.ts - - uses: actions/upload-artifact@v4 + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 if: always() with: name: windows-capabilities @@ -666,8 +666,8 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 10 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm @@ -675,7 +675,7 @@ jobs: - run: npm run docs:check - name: Generation leaves authored files unchanged run: git diff --exit-code - - uses: actions/upload-artifact@v4 + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: generated-docs path: | @@ -782,8 +782,8 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 5 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc - name: Require every selected lane to pass @@ -794,7 +794,7 @@ jobs: run: node scripts/ci/required.mjs - name: Upload full qualification receipt artifact if: needs.changes.outputs.tier == 'full' && success() - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: ${{ steps.gate.outputs.receipt_name }} path: ${{ steps.gate.outputs.receipt_path }} diff --git a/.github/workflows/nativeaot-fixtures.yml b/.github/workflows/nativeaot-fixtures.yml index ce7dc1ea1..b2cc5801f 100644 --- a/.github/workflows/nativeaot-fixtures.yml +++ b/.github/workflows/nativeaot-fixtures.yml @@ -19,16 +19,16 @@ jobs: DOTNET_PROCESSOR_COUNT: "1" COMPlus_GCHeapHardLimit: "30000000" steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc - - uses: actions/setup-dotnet@v4 + - uses: actions/setup-dotnet@67a3573c9a986a3f9c594539f4ab511d57bb3ce9 # v4.3.1 with: dotnet-version: "8.0.416" - name: Build benign fixture and independent byte oracle run: node scripts/generate-nativeaot-fixtures.mjs - - uses: actions/upload-artifact@v4 + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: nativeaot-win-x64-${{ github.event.pull_request.head.sha || github.sha }} path: | diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml index e276a9f0d..6b8a05b46 100644 --- a/.github/workflows/pages.yml +++ b/.github/workflows/pages.yml @@ -19,8 +19,8 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 10 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm diff --git a/.github/workflows/real-binary-layout.yml b/.github/workflows/real-binary-layout.yml index 5cec952ce..0218543f0 100644 --- a/.github/workflows/real-binary-layout.yml +++ b/.github/workflows/real-binary-layout.yml @@ -28,8 +28,8 @@ jobs: runs-on: ubuntu-24.04 timeout-minutes: 15 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm diff --git a/.github/workflows/real-browser.yml b/.github/workflows/real-browser.yml index 912fdafb1..3c3fe955a 100644 --- a/.github/workflows/real-browser.yml +++ b/.github/workflows/real-browser.yml @@ -21,8 +21,8 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 10 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm diff --git a/.github/workflows/real-evm-interface.yml b/.github/workflows/real-evm-interface.yml index dfdd3970f..6e510c973 100644 --- a/.github/workflows/real-evm-interface.yml +++ b/.github/workflows/real-evm-interface.yml @@ -22,8 +22,8 @@ jobs: runs-on: ubuntu-24.04 timeout-minutes: 15 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm diff --git a/.github/workflows/real-ghidra-windows.yml b/.github/workflows/real-ghidra-windows.yml index d6de7469f..db92cd0f8 100644 --- a/.github/workflows/real-ghidra-windows.yml +++ b/.github/workflows/real-ghidra-windows.yml @@ -16,8 +16,8 @@ jobs: if: github.ref == 'refs/heads/main' runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc - name: Prepare cross compiler @@ -31,7 +31,7 @@ jobs: sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64 - run: npm run build:windows-native - run: npm run verify:windows-native:artifact - - uses: actions/upload-artifact@v4 + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: real-ghidra-windows-native path: native/windows/build/ @@ -45,12 +45,12 @@ jobs: runs-on: [self-hosted, Windows, x64, ghidra-12-1-2] timeout-minutes: 20 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc - run: npm ci - - uses: actions/download-artifact@v4 + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: name: real-ghidra-windows-native path: native/windows/build/ @@ -88,7 +88,7 @@ jobs: env: REA_ANALYSIS_PROVIDER: ghidra run: npm run verify:ghidra:windows:package -- --x86 --dll | Tee-Object -FilePath windows-ghidra-x86-dll-package-proof.log - - uses: actions/upload-artifact@v4 + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 if: always() with: name: windows-ghidra-proof diff --git a/.github/workflows/real-go-binary.yml b/.github/workflows/real-go-binary.yml index b886bd70a..c7d74b001 100644 --- a/.github/workflows/real-go-binary.yml +++ b/.github/workflows/real-go-binary.yml @@ -44,12 +44,12 @@ jobs: runs-on: ubuntu-24.04 timeout-minutes: 20 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm - - uses: actions/setup-go@v5 + - uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5.6.0 with: go-version: "1.26.1" cache: false diff --git a/.github/workflows/real-hopper-linux.yml b/.github/workflows/real-hopper-linux.yml index 2355229de..d5e5f8c33 100644 --- a/.github/workflows/real-hopper-linux.yml +++ b/.github/workflows/real-hopper-linux.yml @@ -15,8 +15,8 @@ jobs: runs-on: ubuntu-24.04 timeout-minutes: 25 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm @@ -36,8 +36,8 @@ jobs: runs-on: [self-hosted, Linux, hopper] timeout-minutes: 20 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm diff --git a/.github/workflows/real-hopper.yml b/.github/workflows/real-hopper.yml index 51135f4b6..b3ceeb7d9 100644 --- a/.github/workflows/real-hopper.yml +++ b/.github/workflows/real-hopper.yml @@ -15,8 +15,8 @@ jobs: runs-on: [self-hosted, macOS, hopper] timeout-minutes: 20 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm diff --git a/.github/workflows/real-javascript-recovery.yml b/.github/workflows/real-javascript-recovery.yml index 817dc4dc0..a5043b62e 100644 --- a/.github/workflows/real-javascript-recovery.yml +++ b/.github/workflows/real-javascript-recovery.yml @@ -23,8 +23,8 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 10 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm diff --git a/.github/workflows/real-packet-capture.yml b/.github/workflows/real-packet-capture.yml index b5e9f9201..8daebfd8a 100644 --- a/.github/workflows/real-packet-capture.yml +++ b/.github/workflows/real-packet-capture.yml @@ -43,8 +43,8 @@ jobs: runs-on: ubuntu-24.04 timeout-minutes: 15 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm diff --git a/.github/workflows/real-recorded-crash.yml b/.github/workflows/real-recorded-crash.yml index 04467a608..0463df27d 100644 --- a/.github/workflows/real-recorded-crash.yml +++ b/.github/workflows/real-recorded-crash.yml @@ -25,8 +25,8 @@ jobs: runs-on: ubuntu-24.04 timeout-minutes: 15 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm diff --git a/.github/workflows/real-signal-recording.yml b/.github/workflows/real-signal-recording.yml index cc902037e..5c1380e2d 100644 --- a/.github/workflows/real-signal-recording.yml +++ b/.github/workflows/real-signal-recording.yml @@ -36,8 +36,8 @@ jobs: runs-on: ubuntu-24.04 timeout-minutes: 15 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm diff --git a/.github/workflows/real-termux.yml b/.github/workflows/real-termux.yml index 204acc1c6..e6f3890e1 100644 --- a/.github/workflows/real-termux.yml +++ b/.github/workflows/real-termux.yml @@ -18,7 +18,7 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 45 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 - name: Download the pinned Termux app and verify its release checksum shell: bash run: | @@ -38,7 +38,7 @@ jobs: sudo udevadm control --reload-rules sudo udevadm trigger --name-match=kvm - name: Build and capture a page inside Termux - uses: reactivecircus/android-emulator-runner@4c44018e59b437e86cdfc41da381398f93ed8808 # v2 + uses: reactivecircus/android-emulator-runner@a421e43855164a8197daf9d8d40fe71c6996bb0d # v2.38.0 with: api-level: 30 arch: x86_64 @@ -50,7 +50,7 @@ jobs: script: bash scripts/verify/termux/emulator.sh - name: Upload Termux and Android diagnostics if: always() - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: termux-browser-diagnostics path: ${{ runner.temp }}/rea-termux/diagnostics diff --git a/.github/workflows/real-web-network-captures.yml b/.github/workflows/real-web-network-captures.yml index d41bb37a5..3ee59d2c8 100644 --- a/.github/workflows/real-web-network-captures.yml +++ b/.github/workflows/real-web-network-captures.yml @@ -22,8 +22,8 @@ jobs: runs-on: ubuntu-24.04 timeout-minutes: 15 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm diff --git a/.github/workflows/real-web-runtime.yml b/.github/workflows/real-web-runtime.yml index 2575c55f9..f5b9f201a 100644 --- a/.github/workflows/real-web-runtime.yml +++ b/.github/workflows/real-web-runtime.yml @@ -17,8 +17,8 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 10 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm diff --git a/.github/workflows/real-web-source-map.yml b/.github/workflows/real-web-source-map.yml index 8e0869eb7..2a1e65dc0 100644 --- a/.github/workflows/real-web-source-map.yml +++ b/.github/workflows/real-web-source-map.yml @@ -17,8 +17,8 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 10 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: npm diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 7cc798cfa..61e39d029 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -48,7 +48,7 @@ jobs: echo "frozen=true" >> "$GITHUB_OUTPUT" fi - - uses: googleapis/release-please-action@v4 + - uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1 if: steps.editorial.outputs.frozen != 'true' with: token: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }} @@ -70,13 +70,13 @@ jobs: sha: ${{ steps.release.outputs.sha }} steps: - name: Check out reviewed release merge - uses: actions/checkout@v6 + uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 with: ref: ${{ github.event.pull_request.merge_commit_sha }} fetch-depth: 0 persist-credentials: false - - uses: actions/setup-node@v6 + - uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0 with: node-version-file: .nvmrc package-manager-cache: false @@ -104,7 +104,7 @@ jobs: node scripts/release-notes.mjs publication --repository "$GH_REPO" \ --source "$SOURCE_SHA" --pr "$RELEASE_PR" - - uses: googleapis/release-please-action@v4 + - uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1 id: release with: token: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }} @@ -191,14 +191,14 @@ jobs: fi - name: Check out release controller and Git history - uses: actions/checkout@v6 + uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 with: ref: ${{ github.sha }} fetch-depth: 0 persist-credentials: false - name: Set up Node.js for checkpoint validation - uses: actions/setup-node@v6 + uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0 with: node-version-file: .nvmrc package-manager-cache: false @@ -226,7 +226,7 @@ jobs: node scripts/release-notes.mjs publication --repository "$GH_REPO" \ --source "$SOURCE_SHA" - - uses: googleapis/release-please-action@v4 + - uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1 id: release with: # A dedicated credential starts PR CI automatically; the built-in @@ -255,7 +255,7 @@ jobs: - name: Check out release pull request if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true' - uses: actions/checkout@v6 + uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 with: ref: ${{ fromJSON(steps.release.outputs.pr).headBranchName }} fetch-depth: 0 @@ -263,7 +263,7 @@ jobs: - name: Set up Node.js for generated documentation if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true' - uses: actions/setup-node@v6 + uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0 with: node-version-file: .nvmrc package-manager-cache: false @@ -314,12 +314,12 @@ jobs: contents: read id-token: write steps: - - uses: actions/checkout@v6 + - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 with: ref: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }} persist-credentials: false - - uses: actions/setup-node@v6 + - uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0 with: node-version-file: .nvmrc package-manager-cache: false @@ -328,7 +328,7 @@ jobs: - run: npm ci - name: Download the exact qualified package - uses: actions/download-artifact@v4 + uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: name: verified-release-package path: .cache/verified-package/ @@ -392,7 +392,7 @@ jobs: contents: read id-token: write steps: - - uses: actions/checkout@v6 + - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 with: ref: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }} persist-credentials: false diff --git a/.github/workflows/sqlite-runtime.yml b/.github/workflows/sqlite-runtime.yml index 6e5bbcd10..fc379d1dc 100644 --- a/.github/workflows/sqlite-runtime.yml +++ b/.github/workflows/sqlite-runtime.yml @@ -22,14 +22,14 @@ jobs: # The repository-pinned Node 24 version also runs the complete test suite. node: ["22.x", "24.15.0", "24.x", "26.x"] steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version: ${{ matrix.node }} check-latest: true cache: npm - run: npm ci - - uses: actions/download-artifact@v4 + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: name: rea-test-runtime path: . diff --git a/.github/workflows/website-check.yml b/.github/workflows/website-check.yml index 2a2c5406f..dc17b4b96 100644 --- a/.github/workflows/website-check.yml +++ b/.github/workflows/website-check.yml @@ -14,9 +14,9 @@ jobs: timeout-minutes: 10 steps: - name: Check out the repository - uses: actions/checkout@v4 + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 - name: Set up Python for website assets - uses: actions/setup-python@v5 + uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0 with: python-version: "3.12" cache: pip diff --git a/.github/workflows/website-pages.yml b/.github/workflows/website-pages.yml index 3c2251658..491b028a6 100644 --- a/.github/workflows/website-pages.yml +++ b/.github/workflows/website-pages.yml @@ -47,9 +47,9 @@ jobs: exit 1 fi - name: Check out the repository - uses: actions/checkout@v4 + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 - name: Set up Python for website assets - uses: actions/setup-python@v5 + uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0 with: python-version: "3.12" cache: pip @@ -74,9 +74,9 @@ jobs: ) PY - name: Configure Pages - uses: actions/configure-pages@v5 + uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0 - name: Upload the website - uses: actions/upload-pages-artifact@v4 + uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0 with: path: website/_site @@ -95,15 +95,15 @@ jobs: steps: - name: Check out publishing configuration if: inputs.target == 'both' - uses: actions/checkout@v4 + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 - name: Set up Node.js for Wrangler if: inputs.target == 'both' - uses: actions/setup-node@v4 + uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc - name: Download the verified website if: inputs.target == 'both' - uses: actions/download-artifact@v4 + uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: name: github-pages path: ${{ runner.temp }}/website-archive @@ -114,7 +114,7 @@ jobs: tar -xf "$RUNNER_TEMP/website-archive/artifact.tar" -C "$RUNNER_TEMP/rea-website-public" - name: Publish the website to Cloudflare if: inputs.target == 'both' - uses: cloudflare/wrangler-action@v4 + uses: cloudflare/wrangler-action@953926a2e2182532811c01a25e53647d93bf07c0 # v4.1.3 with: apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }} accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} @@ -123,7 +123,7 @@ jobs: command: deploy --cwd website --env production --assets "${{ runner.temp }}/rea-website-public" - name: Publish to GitHub Pages id: deployment - uses: actions/deploy-pages@v4 + uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4.0.5 - name: Verify the selected published revisions env: PAGES_URL: ${{ steps.deployment.outputs.page_url }} diff --git a/tests/boundary/filesystem/ciRequiredLanes.test.ts b/tests/boundary/filesystem/ciRequiredLanes.test.ts index b6e02b54a..da6c43186 100644 --- a/tests/boundary/filesystem/ciRequiredLanes.test.ts +++ b/tests/boundary/filesystem/ciRequiredLanes.test.ts @@ -262,8 +262,8 @@ it("binds provider build reuse to the current run while keeping standalone dispa ), ), ); - const download = action.runs.steps.find( - (step) => step.uses === "actions/download-artifact@v4", + const download = action.runs.steps.find((step) => + step.uses?.startsWith("actions/download-artifact@"), ); expect(download?.with).toEqual({ name: "${{ inputs.build_cache_artifact }}", diff --git a/tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts b/tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts index 39eb8ff61..0e936b062 100644 --- a/tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts +++ b/tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts @@ -26,14 +26,18 @@ describe("real Windows Ghidra workflow trust boundary", () => { environment: "real-ghidra-windows", "runs-on": expect.arrayContaining(["self-hosted", "Windows", "x64"]), steps: expect.arrayContaining([ - expect.objectContaining({ uses: "actions/setup-node@v4" }), + expect.objectContaining({ + uses: expect.stringMatching(/^actions\/setup-node@[0-9a-f]{40}$/), + }), expect.objectContaining({ run: "npm ci" }), expect.objectContaining({ env: { REA_ANALYSIS_PROVIDER: "ghidra" }, run: "npm run verify:ghidra:windows | Tee-Object -FilePath windows-ghidra-proof.log", }), expect.objectContaining({ - uses: "actions/upload-artifact@v4", + uses: expect.stringMatching( + /^actions\/upload-artifact@[0-9a-f]{40}$/, + ), with: expect.objectContaining({ path: expect.stringContaining("windows-ghidra-proof.log"), }),