feat(mcp): derive an API surface view from retained web network captures (#1891)
* feat(mcp): derive an API surface view from retained web network captures * fix(mcp): fail closed on sensitive parents and budget api-surface path expansion * fix(mcp): keep recorded api-surface paths and budget only templated segments * fix(mcp): read api-surface paths only from canonical URLs and charge new values once * fix(mcp): reject authority-less api-surface URLs and key groups by rule positions * fix(mcp): label api-surface confidence from all groups and cap transient path memory * fix(mcp): budget api-surface dedup sets and narrow detail outputs Charge retained deduplication sets before allocation, advertise only each analysis operation's summary from its detail output, and document that any templated group makes every API-surface page inferred. * fix(mcp): bound api-surface path text and record limitations Charge retained literal path copies before allocation, publish only valid media-type essences, and deduplicate examined record limitations onto the view. * fix(mcp): keep api-surface refusals and path splits inside the reply budget Sensitive-parent refusals now use the token sensitive_values_unretained so declared literals are not echoed. A later path split is charged against bytes already retained, and the published example includes the HAR record limitation with its matching digest. * fix(mcp): budget api-surface method text before group keys Charge method and origin text against the reply budget before the group key is serialized. Document that samples merge only when method, origin, and exact segment rules match. * fix(mcp): bound api-surface copies and control-character authorities Reject authorities containing a tab or newline so WHATWG cannot move userinfo into the recorded path. Charge four path copies, two method and origin copies, and each distinct content type against bytes already retained. * fix(mcp): preflight combined api-surface group charges Total path, method, origin, and segment charges before retaining a new group. Track content types in a set so distinct MIME essences are not compared quadratically. * fix(mcp): charge api-surface content-type sets on first use Create the per-group content-type set when the first essence is retained, and charge it before allocation. Groups with no content type no longer keep an empty set. * refactor(analysis): centralize API-surface ownership and admission Own grouping, deduplication, and combined retained/temporary resource admission in one accumulator. Charge fixed group storage before allocation and materialize only the requested page. Share recorded URL parsing between capture redaction and projection, preserving source paths and excluding ambiguous credential fields. Cover resource admission invariants and the compiled HAR-to-view workflow. * fix(analysis): admit capture schema clone before parsing * fix(javascript): share Electron mutation proof with wrapped calls * test(mcp): align graph and catalog checks with current contracts * fix(analysis): admit and validate portable capture views --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
This commit is contained in:
Matt Van Horn
GitHub
parent
64b71b180a
commit
880a603fe8
|
|
@ -46,3 +46,9 @@ observed exit signal stays in `exit.signal`, `exit.code` is null for every
|
|||
deadline exit, and the code of a finalizer that exited on its own is in
|
||||
`exit.finalization.exit_code` (null on a signal exit or when no exit was
|
||||
observed).
|
||||
|
||||
Use `inspect_analysis_view` with `view: {kind: "api-surface"}` on retained
|
||||
`inspect_web_network_capture` HAR Evidence to group observed request lines by
|
||||
method, origin and path template. Format matches are inferences, not handler
|
||||
identity or schemas. Group pages cite parent record ordinals; consult record
|
||||
coverage and exclusions before claiming completeness. No requests are replayed.
|
||||
|
|
|
|||
|
|
@ -419,6 +419,13 @@ analysis, so later views and application workflows read it without repeating
|
|||
analysis or moving the complete record through the response. A server without
|
||||
session retention refuses summary detail with `capability_unavailable`.
|
||||
|
||||
`inspect_analysis_view` also accepts `api-surface` for retained HAR capture Evidence,
|
||||
grouping request lines by method, origin and format-rule path template. Templated
|
||||
segments are inferences. Because paging follows the full grouping, every page is
|
||||
inferred Evidence whenever any group is templated, and derived Evidence only when
|
||||
no group was templated.
|
||||
See [derived API surfaces](web-network-captures.md#derived-api-surface).
|
||||
|
||||
`inspect_analysis_view` projects a caller-selected view of already completed
|
||||
`inspect_binary_layout`, `analyze_javascript_application`, or `analyze_function`
|
||||
Evidence. Native views select procedure, pseudocode, assembly, basic blocks,
|
||||
|
|
|
|||
|
|
@ -100,6 +100,11 @@ resources follow the same explicit declarations without changing error types;
|
|||
sensitive issue pointers identify a real ancestor instead of a fabricated
|
||||
coordinate. Header/cookie authentication values and known
|
||||
transport URL userinfo are excluded structurally, including native backups.
|
||||
HAR redaction and API-surface projection share the same recorded-URL parser.
|
||||
Canonical userinfo spans are removed without rewriting the remaining source
|
||||
text. Credential-bearing spellings without a safe raw span, such as slashless
|
||||
or extra-slash HTTP URLs, are excluded as a whole transport field. Ordinary
|
||||
extension text and noncredential path/query values remain unchanged.
|
||||
|
||||
Generated free-text limitations follow the same literal declarations, including
|
||||
global, per-record and Evidence explanations added during projection. Contract
|
||||
|
|
@ -112,6 +117,74 @@ wire fragments. Native dictionary duplicate keys have the unchanged upstream
|
|||
decoder's last-value semantics; this adapter does not claim to recover discarded
|
||||
duplicates or the original numeric spelling of parsed native floats.
|
||||
|
||||
## Derived API surface
|
||||
|
||||
Pass retained HAR Evidence to `inspect_analysis_view` with
|
||||
`view: {"kind":"api-surface","offset":0,"limit":50}`. Omitted offset and limit
|
||||
are 0 and 50. The view groups request lines by method, WHATWG URL origin and
|
||||
path template, sorted by origin, template and method. Query strings, fragments,
|
||||
credentials, headers and bodies are not copied into groups.
|
||||
|
||||
| Rule | Exact segment format | Placeholder |
|
||||
| ---------- | ---------------------------------------------------- | ------------ |
|
||||
| `numeric` | ASCII digits only | `{numeric}` |
|
||||
| `uuid` | RFC 9562 textual UUID, any version, case-insensitive | `{uuid}` |
|
||||
| `hex` | At least 32 hexadecimal digits | `{hex}` |
|
||||
| `iso-date` | Valid calendar date `YYYY-MM-DD` | `{iso-date}` |
|
||||
|
||||
Rules apply in table order to the recorded, still-encoded request path, read
|
||||
before WHATWG dot-segment removal, so `/a/%2e%2e/admin` and `/admin` stay distinct. Segment indexes start
|
||||
at zero after the leading slash. Each matched segment lists its observed literals
|
||||
and `basis: "inferred"`. `templated_groups` counts templated groups across every
|
||||
page. Because paging follows the full grouping, every page is inferred Evidence
|
||||
when that count is nonzero, and derived Evidence only when no group was templated. Version prefixes, short opaque IDs
|
||||
and digit-bearing route names stay literal. Samples merge only when method,
|
||||
origin, and exact segment rules match. The view does not infer further
|
||||
cross-sample patterns.
|
||||
|
||||
Groups cite parent record ordinals and distinct statuses and MIME types (lowercase
|
||||
type/subtype without parameters). A malformed media type is unknown, not a content
|
||||
type. Flags report single samples, mixed statuses or mixed content types.
|
||||
`coverage` pages groups; `records` separately reports examined, total and excluded
|
||||
parent records. Ordinal subsets carry a limitation. Null URLs are excluded as
|
||||
`url-redacted`; unparseable URLs, URLs without an origin, spellings without
|
||||
`scheme://authority` (such as `https:host/path`), and authorities containing a
|
||||
tab or newline as `url-unparseable`.
|
||||
Unavailable request objects or methods are explicitly excluded. Unavailable
|
||||
response fields, including HAR's `status: 0` and an empty `mimeType`, remain
|
||||
unknown, not invented statuses or MIME types.
|
||||
|
||||
The view makes no handler-identity, schema or causality claim. It performs no
|
||||
network activity. Native mitmproxy captures return a typed unsupported error.
|
||||
Parents inspected with `sensitive_values`, or whose Evidence does not record
|
||||
`sensitive_value_count`, return a typed unsupported error whose reason is the
|
||||
canonical token `sensitive_values_unretained`. The view would write path
|
||||
templates and limitation text that it cannot check against declarations the
|
||||
parent does not retain, so the refusal does not repeat declaration text.
|
||||
|
||||
One accumulator owns group identity, observed-value/status/MIME sets, ordinals,
|
||||
exclusions and deduplicated limitations. It scans path spans without splitting
|
||||
literal paths into arrays. Before schema parsing, it reserves the capture clone
|
||||
and validation scratch by walking the parent without cloning it. Inline requests
|
||||
borrow the Evidence value through CLI/MCP envelope validation; the application
|
||||
admits the entire portable envelope before authenticated parsing or hashing.
|
||||
The envelope clone, hash scratch and capture clone stay charged during grouping.
|
||||
Temporary URL/path/MIME work and retained state must
|
||||
fit together within a 96 MiB projection representation budget. A whole group or
|
||||
merge is admitted before allocating its retained sets, map entries or array
|
||||
values. Fixed group storage, distinct text, exclusions and limitations count as
|
||||
well as matched segments. Per-record reservations are released after each
|
||||
record; repeated values do not accumulate another retained charge. Only the
|
||||
requested page is materialized as output groups.
|
||||
|
||||
The representation budget uses conservative storage and worst-case JSON-escaping
|
||||
allowances; it is not an enforced V8 heap or process-memory limit. It is separate
|
||||
from the decoder's actual reply-byte limit and MCP transport admission. Exceeding
|
||||
it returns `resource_constraint` without a partial view. Limitations on the
|
||||
parent envelope, capture and examined records are deduplicated onto the view.
|
||||
Portable captures must retain unique ordinals within their declared total;
|
||||
caller-selected subsets can preserve any requested ordinal order.
|
||||
|
||||
## Resource and lifecycle boundaries
|
||||
|
||||
Complete evidence has a 32 MiB input, 96 MiB decoder reply and 64-level nesting
|
||||
|
|
|
|||
|
|
@ -913,6 +913,30 @@ async function inspect(mode, input, errorCategory) {
|
|||
}
|
||||
assert.notEqual(result.isError, true, mcpTextValue(result));
|
||||
assertSensitiveLimitations(value, input.sensitive_values ?? []);
|
||||
if (input.format === "har") {
|
||||
const surface = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: {
|
||||
source: { kind: "retained-evidence", evidence_id: value.evidence_id },
|
||||
view: { kind: "api-surface" },
|
||||
},
|
||||
});
|
||||
if ((input.sensitive_values ?? []).length > 0) {
|
||||
// Derived view text cannot be checked against undisclosed declarations.
|
||||
assert.equal(surface.isError, true);
|
||||
const refused = JSON.parse(mcpTextValue(surface));
|
||||
for (const literal of input.sensitive_values)
|
||||
assert.ok(!JSON.stringify(refused).includes(literal));
|
||||
} else {
|
||||
assert.notEqual(surface.isError, true, mcpTextValue(surface));
|
||||
const projected = JSON.parse(mcpTextValue(surface));
|
||||
assert.equal(projected.normalized_result.kind, "api-surface");
|
||||
assert.equal(
|
||||
projected.normalized_result.records.examined,
|
||||
value.normalized_result.records.length,
|
||||
);
|
||||
}
|
||||
}
|
||||
return value.normalized_result;
|
||||
}
|
||||
const args = [
|
||||
|
|
|
|||
|
|
@ -1,3 +1,8 @@
|
|||
import {
|
||||
analysisViewCaptureFixture,
|
||||
analysisViewCaptureEvidence,
|
||||
analysisViewObjectDenseCapture,
|
||||
} from "../../../tests/fixtures/analysisView.js";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js";
|
||||
|
|
@ -17,6 +22,9 @@ import {
|
|||
import type { Evidence } from "../../domain/evidence.js";
|
||||
import { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
|
||||
import { err, ok } from "../../domain/result.js";
|
||||
import { EvidenceLedger } from "../investigation/EvidenceLedger.js";
|
||||
import { WEB_NETWORK_CAPTURE_LIMITS } from "../../domain/webNetworkCapture.js";
|
||||
import { inspectAnalysisViewInputSchema } from "../../domain/analysisView/analysisView.js";
|
||||
|
||||
it.each(["/fixtures/unknown.exe", ""])(
|
||||
"preserves recorded empty native identity fields: path=%j",
|
||||
|
|
@ -346,3 +354,142 @@ describe("summarizeRetainedAnalysis", () => {
|
|||
).toEqual(err(conflict));
|
||||
});
|
||||
});
|
||||
|
||||
it("seals inferred and literal HAR pages, including a null artifact digest", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.artifact.sha256 = null;
|
||||
const parent = analysisViewCaptureEvidence(capture);
|
||||
// Any templated group in the full grouping makes every page inferred.
|
||||
for (const [offset, confidence] of [
|
||||
[0, "inferred"],
|
||||
[11, "inferred"],
|
||||
] as const) {
|
||||
const result = inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: parent },
|
||||
view: { kind: "api-surface", offset, limit: 1 },
|
||||
});
|
||||
if (!result.ok) throw result.error;
|
||||
expect(result.value.confidence).toBe(confidence);
|
||||
expect(result.value.subject).toBeNull();
|
||||
expect(parseEvidence(result.value)).toEqual(result.value);
|
||||
}
|
||||
expect(
|
||||
inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: analysisViewLayoutEvidence() },
|
||||
view: { kind: "api-surface" },
|
||||
}),
|
||||
).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisUnsupportedTargetError" },
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects a null projected digest under a present parent subject", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.artifact.sha256 = null;
|
||||
const parent = createEvidence(
|
||||
{ path: capture.artifact.path, sha256: "a".repeat(64), format: "file" },
|
||||
{ id: "fixture", name: "fixture", version: "1" },
|
||||
{
|
||||
operation: "inspect_web_network_capture",
|
||||
parameters: { sensitive_value_count: 0 },
|
||||
result: capture,
|
||||
confidence: "observed",
|
||||
},
|
||||
);
|
||||
expect(
|
||||
inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: parent },
|
||||
view: { kind: "api-surface" },
|
||||
}),
|
||||
).toMatchObject({ ok: false, error: { _tag: "EvidenceIntegrityError" } });
|
||||
});
|
||||
|
||||
it("refuses an object-dense borrowed capture before making a full schema clone", () => {
|
||||
const capture = analysisViewObjectDenseCapture();
|
||||
expect(Buffer.byteLength(JSON.stringify(capture))).toBeLessThan(
|
||||
WEB_NETWORK_CAPTURE_LIMITS.inputBytes,
|
||||
);
|
||||
const parent = analysisViewCaptureEvidence(capture);
|
||||
const ledger = new EvidenceLedger();
|
||||
const recorded = ledger.record(parent);
|
||||
if (!recorded.ok) throw recorded.error;
|
||||
const borrowed = ledger.forAnalysis(parent.evidence_id)!;
|
||||
expect(Object.isFrozen(borrowed.normalized_result)).toBe(true);
|
||||
const result = inspectAnalysisView(
|
||||
{
|
||||
source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
|
||||
view: { kind: "api-surface", limit: 1 },
|
||||
},
|
||||
(id) => ledger.forAnalysis(id),
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
_tag: "AnalysisResourceConstraintError",
|
||||
resource: "memory",
|
||||
},
|
||||
});
|
||||
expect(ledger.forAnalysis(parent.evidence_id)).toBe(borrowed);
|
||||
});
|
||||
|
||||
it("borrows inline Evidence until admission without invoking nested accessors", () => {
|
||||
let accessed = false;
|
||||
const evidence = {
|
||||
get normalized_result() {
|
||||
accessed = true;
|
||||
throw new Error("Evidence must be admitted before parsing");
|
||||
},
|
||||
};
|
||||
const input = {
|
||||
source: { kind: "inline", evidence },
|
||||
view: { kind: "api-surface" },
|
||||
};
|
||||
const parsed = inspectAnalysisViewInputSchema.parse(input);
|
||||
if (parsed.source.kind !== "inline") throw new Error("Expected inline input");
|
||||
expect(parsed.source.evidence).toBe(evidence);
|
||||
expect(inspectAnalysisView(input)).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisInputError" },
|
||||
});
|
||||
expect(accessed).toBe(false);
|
||||
});
|
||||
|
||||
it("keeps inline-envelope admission live alongside capture validation and grouping", () => {
|
||||
// Each clone alone fits; both graphs coexist while the view is projected.
|
||||
const capture = analysisViewObjectDenseCapture(40_000);
|
||||
const parent = analysisViewCaptureEvidence(capture);
|
||||
const retained = inspectAnalysisView(
|
||||
{
|
||||
source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
|
||||
view: { kind: "api-surface" },
|
||||
},
|
||||
() => parent,
|
||||
);
|
||||
if (!retained.ok) throw retained.error;
|
||||
expect(
|
||||
inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: parent },
|
||||
view: { kind: "api-surface" },
|
||||
}),
|
||||
).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisResourceConstraintError", resource: "memory" },
|
||||
});
|
||||
});
|
||||
|
||||
it("admits raw_result and parameters before authenticating a portable capture", () => {
|
||||
const dense = analysisViewObjectDenseCapture();
|
||||
const parent = analysisViewCaptureEvidence();
|
||||
for (const field of ["raw_result", "parameters"] as const) {
|
||||
const result = inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: { ...parent, [field]: dense } },
|
||||
view: { kind: "api-surface" },
|
||||
});
|
||||
// Refuse expansion before hashing can report the deliberately stale ID.
|
||||
expect(result).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisResourceConstraintError", resource: "memory" },
|
||||
});
|
||||
}
|
||||
});
|
||||
|
|
|
|||
|
|
@ -22,6 +22,7 @@ import {
|
|||
} from "../../domain/analysisView/analysisView.js";
|
||||
import { jsonValueSchema } from "../../domain/jsonValue.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import { WebNetworkApiSurfaceAccumulator } from "../../domain/analysisView/WebNetworkApiSurfaceAccumulator.js";
|
||||
|
||||
/** Authenticate a selected-view request and project already completed Evidence. */
|
||||
export const inspectAnalysisView = (
|
||||
|
|
@ -41,11 +42,25 @@ export const inspectAnalysisView = (
|
|||
return inspectAnalysisViewValidated(parsed.data, lookup);
|
||||
};
|
||||
|
||||
/** Project a view from input already parsed by a trusted adapter. */
|
||||
/** Authenticate borrowed inline Evidence after adapter envelope validation. */
|
||||
export const inspectAnalysisViewValidated = (
|
||||
input: InspectAnalysisViewInput,
|
||||
lookup?: EvidenceLookup,
|
||||
): Result<Evidence, AnalysisError> => {
|
||||
const inline = input.source.kind === "inline" ? input.source.evidence : null;
|
||||
const inlineCapture =
|
||||
typeof inline === "object" &&
|
||||
inline !== null &&
|
||||
Object.getOwnPropertyDescriptor(inline, "operation")?.value ===
|
||||
"inspect_web_network_capture";
|
||||
const accumulator =
|
||||
input.view.kind === "api-surface" || inlineCapture
|
||||
? new WebNetworkApiSurfaceAccumulator()
|
||||
: undefined;
|
||||
if (input.source.kind === "inline" && accumulator !== undefined) {
|
||||
const admitted = accumulator.admitInlineEvidence(input.source.evidence);
|
||||
if (!admitted.ok) return admitted;
|
||||
}
|
||||
const resolved = resolveAnalysisViewSource(input.source, lookup);
|
||||
if (!resolved.ok) return resolved;
|
||||
const parent = resolved.value;
|
||||
|
|
@ -55,6 +70,7 @@ export const inspectAnalysisViewValidated = (
|
|||
operation: parent.operation,
|
||||
normalizedResult: parent.normalized_result,
|
||||
limitations: parent.limitations,
|
||||
parameters: parent.parameters,
|
||||
artifact:
|
||||
parent.subject === null
|
||||
? null
|
||||
|
|
@ -64,6 +80,7 @@ export const inspectAnalysisViewValidated = (
|
|||
},
|
||||
},
|
||||
input.view,
|
||||
accumulator,
|
||||
);
|
||||
if (!projected.ok) return projected;
|
||||
if (
|
||||
|
|
@ -176,7 +193,11 @@ const createAnalysisViewEvidence = (
|
|||
},
|
||||
result: jsonValueSchema.parse(result),
|
||||
rawResult: null,
|
||||
confidence: "derived",
|
||||
confidence:
|
||||
// Paging follows the full grouping, so any templated group makes every page inferred.
|
||||
result.kind === "api-surface" && result.templated_groups > 0
|
||||
? "inferred"
|
||||
: "derived",
|
||||
authority: parent.authority,
|
||||
limitations: result.limitations,
|
||||
locations: [
|
||||
|
|
|
|||
|
|
@ -42,6 +42,52 @@ it("preserves the same URL-looking text in an ordinary extension", () => {
|
|||
expect(redactor.redactions).toEqual([]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
"https:user:password@example.test/path",
|
||||
"https:///user:password@example.test/path",
|
||||
"https://\\user:password@example.test/path",
|
||||
"https://\t/user:password@example.test/path",
|
||||
"https://\n/user:password@example.test/path",
|
||||
"https://\r/user:password@example.test/path",
|
||||
"ht\ttps://user:password@example.test/path",
|
||||
"\\\\user:password@example.test/path",
|
||||
"/\\user:password@example.test/path",
|
||||
"\\/user:password@example.test/path",
|
||||
"/\t/user:password@example.test/path",
|
||||
])("excludes credentials whose raw URL span is ambiguous: %s", (value) => {
|
||||
const redactor = new CaptureRedaction([]);
|
||||
expect(redactor.text(value, "/request/url", true)).toBeNull();
|
||||
expect(redactor.redactions).toEqual([
|
||||
{ pointer: "/request/url", reason: "transport-credential" },
|
||||
]);
|
||||
expect(new CaptureRedaction([]).text(value, "/_extension")).toBe(value);
|
||||
});
|
||||
|
||||
it("uses non-special URL authority rules to remove the complete userinfo span", () => {
|
||||
const value = "custom://user:secret@part\\more:password@host/path";
|
||||
const redactor = new CaptureRedaction([]);
|
||||
expect(redactor.text(value, "/response/redirectURL", true)).toBe(
|
||||
"custom://host/path",
|
||||
);
|
||||
expect(redactor.redactions).toEqual([
|
||||
{ pointer: "/response/redirectURL", reason: "transport-credential" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("preserves noncanonical URLs and path/query userinfo lookalikes without parsed credentials", () => {
|
||||
for (const value of [
|
||||
"https:example.test/path?token=ordinary",
|
||||
"https://example.test/a/%2e%2e/user:password@literal?url=user:password@literal",
|
||||
"https://example.test\\user:password@literal",
|
||||
"https://invalid host\\user:password@literal",
|
||||
"//invalid host\\user:password@literal",
|
||||
]) {
|
||||
const redactor = new CaptureRedaction([]);
|
||||
expect(redactor.text(value, "/request/url", true)).toBe(value);
|
||||
expect(redactor.redactions).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["\ud800", "\udfff"])(
|
||||
"does not invent replacement UTF-8 bytes for a non-scalar declaration",
|
||||
(literal) => {
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
import type { WebNetworkCaptureRecord } from "../../domain/webNetworkCapture.js";
|
||||
import { parseRecordedTransportUrl } from "../../domain/recordedTransportUrl.js";
|
||||
|
||||
type Redaction = WebNetworkCaptureRecord["redactions"][number];
|
||||
const CREDENTIAL_HEADERS = new Set([
|
||||
|
|
@ -25,10 +26,10 @@ export class CaptureRedaction {
|
|||
text(value: string, pointer: string, transportUrl = false): string | null {
|
||||
let result = value;
|
||||
if (transportUrl) {
|
||||
const match =
|
||||
/^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?\/\/)([^/?#]*@)/i.exec(result);
|
||||
if (match !== null) {
|
||||
result = `${match[1]}${result.slice(match[0].length)}`;
|
||||
const { credentials } = parseRecordedTransportUrl(value);
|
||||
if (credentials === "ambiguous") return this.credential(pointer);
|
||||
if (credentials !== null) {
|
||||
result = `${value.slice(0, credentials.start)}${value.slice(credentials.end)}`;
|
||||
this.redactions.push({ pointer, reason: "transport-credential" });
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
|
|||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import {
|
||||
WEB_NETWORK_CAPTURE_LIMITS,
|
||||
webNetworkCaptureSchema,
|
||||
webNetworkCapturePayloadSchema,
|
||||
type InspectWebNetworkCaptureInput,
|
||||
type WebNetworkCapture,
|
||||
} from "../../domain/webNetworkCapture.js";
|
||||
|
|
@ -38,14 +38,11 @@ import { PrivateRuntimeRoot } from "../../process/PrivateRuntimeRoot.js";
|
|||
import { redactExplicitFailure } from "../../domain/explicitSensitiveFailure.js";
|
||||
|
||||
const OPERATION = "inspect_web_network_capture";
|
||||
const decodedSchema = webNetworkCaptureSchema.omit({
|
||||
artifact: true,
|
||||
format: true,
|
||||
runtime_attribution: true,
|
||||
limitations: true,
|
||||
});
|
||||
const replySchema = z.discriminatedUnion("ok", [
|
||||
z.strictObject({ ok: z.literal(true), value: decodedSchema }),
|
||||
z.strictObject({
|
||||
ok: z.literal(true),
|
||||
value: webNetworkCapturePayloadSchema,
|
||||
}),
|
||||
z.strictObject({
|
||||
ok: z.literal(false),
|
||||
reason: z.enum([
|
||||
|
|
|
|||
|
|
@ -0,0 +1,61 @@
|
|||
import { expect, it } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import {
|
||||
binaryLayoutSummaryViewResultSchema,
|
||||
javascriptApplicationSummaryViewResultSchema,
|
||||
projectAnalysisView,
|
||||
} from "../domain/analysisView/analysisView.js";
|
||||
import {
|
||||
analysisViewJavaScriptEvidence,
|
||||
analysisViewLayoutEvidence,
|
||||
} from "../../tests/fixtures/analysisView.js";
|
||||
import { toolContract } from "./toolContracts.js";
|
||||
|
||||
const projectSummary = (parent: {
|
||||
readonly evidence_id: string;
|
||||
readonly operation: string;
|
||||
readonly normalized_result: unknown;
|
||||
readonly limitations: readonly string[];
|
||||
}) =>
|
||||
projectAnalysisView(
|
||||
{
|
||||
evidenceId: parent.evidence_id,
|
||||
operation: parent.operation,
|
||||
normalizedResult: parent.normalized_result,
|
||||
limitations: parent.limitations,
|
||||
},
|
||||
{ kind: "summary" },
|
||||
);
|
||||
|
||||
it("advertises only that operation's summary from analysis detail outputs", () => {
|
||||
for (const name of [
|
||||
"inspect_binary_layout",
|
||||
"analyze_javascript_application",
|
||||
] as const) {
|
||||
const schema = JSON.stringify(
|
||||
z.toJSONSchema(toolContract(name).outputSchema),
|
||||
);
|
||||
expect(schema).not.toContain("api-surface");
|
||||
expect(schema).not.toContain("inspect_web_network_capture");
|
||||
}
|
||||
expect(
|
||||
JSON.stringify(
|
||||
z.toJSONSchema(toolContract("inspect_analysis_view").outputSchema),
|
||||
),
|
||||
).toContain("api-surface");
|
||||
|
||||
const layout = projectSummary(analysisViewLayoutEvidence());
|
||||
const javascript = projectSummary(analysisViewJavaScriptEvidence());
|
||||
if (!layout.ok || !javascript.ok) throw new Error("Expected summaries");
|
||||
expect(
|
||||
binaryLayoutSummaryViewResultSchema.safeParse(layout.value).success,
|
||||
).toBe(true);
|
||||
expect(
|
||||
javascriptApplicationSummaryViewResultSchema.safeParse(javascript.value)
|
||||
.success,
|
||||
).toBe(true);
|
||||
expect(
|
||||
binaryLayoutSummaryViewResultSchema.safeParse(javascript.value).success,
|
||||
).toBe(false);
|
||||
});
|
||||
|
|
@ -1,6 +1,5 @@
|
|||
import { z } from "zod";
|
||||
|
||||
import { analysisViewResultSchema } from "../domain/analysisView/analysisView.js";
|
||||
import { evidenceResultOf } from "./toolOutputSchemaPrimitives.js";
|
||||
|
||||
/** Caller-selected delivery of completed analysis Evidence that supports retained views. */
|
||||
|
|
@ -11,7 +10,15 @@ export const analysisDetailSchema = z
|
|||
"complete returns the complete analysis Evidence inline. summary retains that complete Evidence in this session and returns its inspect_analysis_view summary; inspect pages, items or facets through normalized_result.parent_evidence_id.",
|
||||
);
|
||||
|
||||
/** Complete analysis Evidence, or the summary view of its retained record. */
|
||||
export const analysisDetailOutputSchemaOf = <Schema extends z.ZodType>(
|
||||
analysis: Schema,
|
||||
) => evidenceResultOf(z.union([analysis, analysisViewResultSchema]));
|
||||
/**
|
||||
* Complete analysis Evidence, or that operation's summary view of its retained
|
||||
* record. Callers pass the summary variant the handler actually returns so the
|
||||
* advertised output does not accept other view kinds.
|
||||
*/
|
||||
export const analysisDetailOutputSchemaOf = <
|
||||
Analysis extends z.ZodType,
|
||||
Summary extends z.ZodType,
|
||||
>(
|
||||
analysis: Analysis,
|
||||
summary: Summary,
|
||||
) => evidenceResultOf(z.union([analysis, summary]));
|
||||
|
|
|
|||
|
|
@ -0,0 +1,73 @@
|
|||
/** Generated from the two-request apiSurfaceHar fixture through the HAR decoder and view service. */
|
||||
export const API_SURFACE_EXAMPLE = {
|
||||
input: {
|
||||
source: {
|
||||
kind: "retained-evidence",
|
||||
evidence_id:
|
||||
"ev_9e218c18bc6159ec9c8492a9130c286496fb1b6d9f03ec2f4ea3321ce2b15097",
|
||||
},
|
||||
view: {
|
||||
kind: "api-surface",
|
||||
offset: 0,
|
||||
limit: 50,
|
||||
},
|
||||
},
|
||||
output: {
|
||||
parent_evidence_id:
|
||||
"ev_9e218c18bc6159ec9c8492a9130c286496fb1b6d9f03ec2f4ea3321ce2b15097",
|
||||
parent_operation: "inspect_web_network_capture",
|
||||
parent_digest:
|
||||
"9e218c18bc6159ec9c8492a9130c286496fb1b6d9f03ec2f4ea3321ce2b15097",
|
||||
view_digest:
|
||||
"47c1859b7a86dd10770061767300bdb6ec177a9254e8dc7863d5ea8603343da6",
|
||||
artifact: {
|
||||
path: "/fixtures/api-surface.har",
|
||||
sha256:
|
||||
"bf58e2c06e55bfe7e0628157c4ad027ce0aeb81a10e8edef8a7f95b8e5f14225",
|
||||
},
|
||||
coverage: {
|
||||
status: "page",
|
||||
examined: 1,
|
||||
total: 1,
|
||||
next_offset: null,
|
||||
exhausted: true,
|
||||
},
|
||||
limitations: [
|
||||
"Grouping does not establish handler identity, URL-space schemas or causality.",
|
||||
"HAR text is producer-decoded Unicode. Original body bytes remain unknown unless content explicitly uses valid base64; declared sizes are preserved independently of retained bytes. mitmproxy 12.2.3 SaveHar may report null optional postData.text; that null is retained and omitted only from the upstream validation copy.",
|
||||
],
|
||||
unknowns: [],
|
||||
records: {
|
||||
examined: 2,
|
||||
total: 2,
|
||||
excluded: 0,
|
||||
},
|
||||
templated_groups: 1,
|
||||
groups: [
|
||||
{
|
||||
method: "GET",
|
||||
origin: "https://api.example.test",
|
||||
path_template: "/users/{numeric}",
|
||||
templated_segments: [
|
||||
{
|
||||
segment_index: 1,
|
||||
rule: "numeric",
|
||||
basis: "inferred",
|
||||
observed_values: ["8812", "9031"],
|
||||
},
|
||||
],
|
||||
record_ordinals: [0, 1],
|
||||
statuses: [200, 404],
|
||||
content_types: ["application/json", "text/plain"],
|
||||
flags: ["mixed-statuses", "mixed-content-types"],
|
||||
},
|
||||
],
|
||||
excluded_records: [],
|
||||
kind: "api-surface",
|
||||
view: {
|
||||
kind: "api-surface",
|
||||
offset: 0,
|
||||
limit: 50,
|
||||
},
|
||||
},
|
||||
} as const;
|
||||
|
|
@ -1,3 +1,4 @@
|
|||
import { API_SURFACE_EXAMPLE } from "./analysisViewApiSurfaceExample.js";
|
||||
import {
|
||||
analysisViewResultSchema,
|
||||
inspectAnalysisViewInputSchema,
|
||||
|
|
@ -15,10 +16,14 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
|
|||
...toolContractMetadata("inspect_analysis_view"),
|
||||
kind: "application",
|
||||
description:
|
||||
"Inspect selected views of completed binary-layout, JavaScript application or native analyze_function Evidence without repeating analysis. Use an exact same-session retained evidence_id or portable inline Evidence. Select a summary, layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Native views select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.",
|
||||
"Inspect selected views of completed binary-layout, JavaScript application or native analyze_function Evidence without repeating analysis. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. Use an exact same-session retained evidence_id or portable inline Evidence. Select a summary, layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Native views select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.",
|
||||
inputSchema: inspectAnalysisViewInputSchema,
|
||||
outputSchema: evidenceResultOf(analysisViewResultSchema),
|
||||
examples: [
|
||||
{
|
||||
title: "Group requests from retained HAR Evidence",
|
||||
input: API_SURFACE_EXAMPLE.input,
|
||||
},
|
||||
{
|
||||
title: "Summarize retained JavaScript application Evidence",
|
||||
input: {
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ import {
|
|||
electronActiveObservationInputSchema,
|
||||
electronActiveObservationResultSchema,
|
||||
} from "../../domain/javascript/electronActiveObservation.js";
|
||||
import { javascriptApplicationSummaryViewResultSchema } from "../../domain/analysisView/analysisView.js";
|
||||
import {
|
||||
analysisDetailOutputSchemaOf,
|
||||
analysisDetailSchema,
|
||||
|
|
@ -34,6 +35,7 @@ export const analyzeJavaScriptApplicationRequestSchema =
|
|||
});
|
||||
const applicationOutputSchema = analysisDetailOutputSchemaOf(
|
||||
javascriptApplicationAnalysisResultSchema,
|
||||
javascriptApplicationSummaryViewResultSchema,
|
||||
);
|
||||
const reconciliationOutputSchema = evidenceResultOf(
|
||||
javascriptRuntimeReconciliationResultSchema,
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ import {
|
|||
inspectBinaryLayoutInputSchema,
|
||||
binaryLayoutSchema,
|
||||
} from "../../domain/native/binaryLayout.js";
|
||||
import { binaryLayoutSummaryViewResultSchema } from "../../domain/analysisView/analysisView.js";
|
||||
import {
|
||||
analysisDetailOutputSchemaOf,
|
||||
analysisDetailSchema,
|
||||
|
|
@ -46,7 +47,10 @@ export const BINARY_DIAGNOSTICS_TOOL_CONTRACTS = [
|
|||
description:
|
||||
"Inspect an explicit local binary without launching it as a host process. Returns file-backed section/segment ranges, linked addresses, original symbol/REL/RELA table identities, complete encoded RELR tables and upstream-derived offsets with unknown per-word locations/addends, raw name bytes, dependency/interpreter names and static mitigation inferences inline with artifact SHA-256 Evidence. Initial profile: ELF64 x86-64 little-endian EXEC/DYN/REL on Linux x64 via caller-supplied Python with unchanged pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2. Runtime addresses/library paths, overall relocation inventory completeness and complete derived GOT/PLT coverage remain unknown. Zero executable entry values mean absence; relocatable entries are not applicable. Upstream can emulate PLT instructions with Unicorn for derived static maps. Uses bounded complete output and owned temporary files/processes; malformed, unsupported or oversized objects return no partial success. Select detail summary to retain the complete layout Evidence in this session and receive only its summary view, then inspect sections, symbols, mitigations or linkage with inspect_analysis_view.",
|
||||
inputSchema: inspectBinaryLayoutRequestSchema,
|
||||
outputSchema: analysisDetailOutputSchemaOf(binaryLayoutSchema),
|
||||
outputSchema: analysisDetailOutputSchemaOf(
|
||||
binaryLayoutSchema,
|
||||
binaryLayoutSummaryViewResultSchema,
|
||||
),
|
||||
examples: [
|
||||
{
|
||||
title: "Inspect ELF linked and file layout",
|
||||
|
|
|
|||
|
|
@ -6,6 +6,34 @@ import { z } from "zod";
|
|||
import { presentInputJsonSchema } from "./inputSchemaPresentation.js";
|
||||
import { toolContract } from "./toolContracts.js";
|
||||
import { toolInputSchemaWithMetadata } from "./toolSchemaMetadata.js";
|
||||
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "./javascript/javascriptRuntimeReconciliationExample.js";
|
||||
|
||||
it.each(["draft-2020-12", "draft-07"] as const)(
|
||||
"preserves the complete inline Evidence contract before deferred authentication in %s",
|
||||
(target) => {
|
||||
const schema = toolInputSchemaWithMetadata(
|
||||
toolContract("inspect_analysis_view"),
|
||||
)["~standard"].jsonSchema.input({ target });
|
||||
const ajv =
|
||||
target === "draft-2020-12"
|
||||
? new Ajv2020({ strict: false, validateFormats: false })
|
||||
: new Ajv({ strict: false, validateFormats: false });
|
||||
expect(ajv.validateSchema(schema)).toBe(true);
|
||||
const validate = ajv.compile(schema);
|
||||
const input = {
|
||||
source: {
|
||||
kind: "inline",
|
||||
evidence: JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE,
|
||||
},
|
||||
view: { kind: "summary" },
|
||||
};
|
||||
expect(validate(input), JSON.stringify(validate.errors)).toBe(true);
|
||||
expect(
|
||||
validate({ ...input, source: { kind: "inline", evidence: {} } }),
|
||||
).toBe(false);
|
||||
expect(validate({ ...input, source: { kind: "inline" } })).toBe(false);
|
||||
},
|
||||
);
|
||||
|
||||
describe("advertised input references", () => {
|
||||
it.each(["draft-2020-12", "draft-07"] as const)(
|
||||
|
|
|
|||
|
|
@ -0,0 +1,173 @@
|
|||
import { expect, it } from "vitest";
|
||||
import { analysisViewCaptureFixture } from "../../../tests/fixtures/analysisView.js";
|
||||
import type { WebNetworkCaptureRecord } from "../webNetworkCapture.js";
|
||||
import { WebNetworkApiSurfaceAccumulator } from "./WebNetworkApiSurfaceAccumulator.js";
|
||||
|
||||
const record = (
|
||||
ordinal: number,
|
||||
path: string,
|
||||
method = "GET",
|
||||
mimeType = "application/json",
|
||||
): WebNetworkCaptureRecord => ({
|
||||
...analysisViewCaptureFixture().records[0]!,
|
||||
ordinal,
|
||||
limitations: [],
|
||||
reported: {
|
||||
request: { method, url: `https://api.test${path}` },
|
||||
response: { status: 200, content: { mimeType } },
|
||||
},
|
||||
});
|
||||
|
||||
const add = (
|
||||
accumulator: WebNetworkApiSurfaceAccumulator,
|
||||
sample: WebNetworkCaptureRecord,
|
||||
) => {
|
||||
const result = accumulator.add(sample);
|
||||
if (!result.ok) throw result.error;
|
||||
};
|
||||
|
||||
it("admits the capture schema clone and grouping against the same budget", () => {
|
||||
const sample = record(0, `/${"a".repeat(1200)}`);
|
||||
const isolated = new WebNetworkApiSurfaceAccumulator(64 * 1024);
|
||||
add(isolated, sample);
|
||||
const accumulator = new WebNetworkApiSurfaceAccumulator(64 * 1024);
|
||||
expect(
|
||||
accumulator.admitCaptureClone({
|
||||
headers: Array.from({ length: 40 }, () => ({ name: "x", value: "y" })),
|
||||
}).ok,
|
||||
).toBe(true);
|
||||
expect(accumulator.add(sample)).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisResourceConstraintError" },
|
||||
});
|
||||
expect(accumulator.page(0, 1).groups).toEqual([]);
|
||||
});
|
||||
|
||||
it("charges non-enumerable fields that the capture schema would clone", () => {
|
||||
const value = Object.defineProperty({}, "records", {
|
||||
value: Array.from({ length: 100 }, () => ({ reported: {} })),
|
||||
});
|
||||
expect(
|
||||
new WebNetworkApiSurfaceAccumulator(64 * 1024).admitCaptureClone(value),
|
||||
).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisResourceConstraintError" },
|
||||
});
|
||||
});
|
||||
|
||||
it("bounds many short literal groups, including groups with unavailable MIME types", () => {
|
||||
const accumulator = new WebNetworkApiSurfaceAccumulator(64 * 1024);
|
||||
let accepted = 0;
|
||||
for (; accepted < 1000; accepted += 1) {
|
||||
const before = accumulator.page(0, 1000);
|
||||
const result = accumulator.add(
|
||||
record(accepted, `/route${accepted}`, "GET", ""),
|
||||
);
|
||||
if (result.ok) continue;
|
||||
expect(result.error).toMatchObject({
|
||||
_tag: "AnalysisResourceConstraintError",
|
||||
});
|
||||
expect(accumulator.page(0, 1000)).toEqual(before);
|
||||
break;
|
||||
}
|
||||
expect(accepted).toBeGreaterThan(0);
|
||||
expect(accepted).toBeLessThan(1000);
|
||||
expect(accumulator.page(0, 1000).coverage.total).toBe(accepted);
|
||||
});
|
||||
|
||||
it.each([
|
||||
{
|
||||
name: "templated segments",
|
||||
path: "/1".repeat(350),
|
||||
method: "GET",
|
||||
mime: "application/json",
|
||||
},
|
||||
{
|
||||
name: "literal path",
|
||||
path: `/${"a".repeat(30_000)}`,
|
||||
method: "GET",
|
||||
mime: "application/json",
|
||||
},
|
||||
{
|
||||
name: "request identity",
|
||||
path: "/a",
|
||||
method: "M".repeat(60_000),
|
||||
mime: "application/json",
|
||||
},
|
||||
{
|
||||
name: "response text",
|
||||
path: "/a",
|
||||
method: "GET",
|
||||
mime: `text/${"a".repeat(30_000)}`,
|
||||
},
|
||||
])(
|
||||
"admits $name together with previously retained state before publishing a new group",
|
||||
({ path, method, mime }) => {
|
||||
const maximum = 1024 * 1024;
|
||||
const isolated = new WebNetworkApiSurfaceAccumulator(maximum);
|
||||
add(isolated, record(0, path, method, mime));
|
||||
|
||||
const accumulator = new WebNetworkApiSurfaceAccumulator(maximum);
|
||||
for (let ordinal = 0; ordinal < 150; ordinal += 1)
|
||||
add(accumulator, record(ordinal, `/route${ordinal}`));
|
||||
const before = accumulator.page(0, 1000);
|
||||
expect(accumulator.add(record(150, path, method, mime))).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisResourceConstraintError" },
|
||||
});
|
||||
expect(accumulator.page(0, 1000)).toEqual(before);
|
||||
},
|
||||
);
|
||||
|
||||
it("retains repeated segment, status and MIME values once across many samples", () => {
|
||||
const accumulator = new WebNetworkApiSurfaceAccumulator(1024 * 1024);
|
||||
for (let ordinal = 0; ordinal < 1000; ordinal += 1)
|
||||
add(accumulator, record(ordinal, "/1".repeat(250)));
|
||||
const page = accumulator.page(0, 1);
|
||||
expect(page.groups).toHaveLength(1);
|
||||
expect(page.groups[0]?.record_ordinals).toHaveLength(1000);
|
||||
expect(page.groups[0]?.statuses).toEqual([200]);
|
||||
expect(page.groups[0]?.content_types).toEqual(["application/json"]);
|
||||
expect(page.groups[0]?.templated_segments).toHaveLength(250);
|
||||
expect(
|
||||
page.groups[0]?.templated_segments.every(
|
||||
(segment) => segment.observed_values.join() === "1",
|
||||
),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("refuses an entire merge before retaining any novel observed values or its ordinal", () => {
|
||||
const accumulator = new WebNetworkApiSurfaceAccumulator(1024 * 1024);
|
||||
for (let ordinal = 0; ordinal < 100; ordinal += 1)
|
||||
add(accumulator, record(ordinal, `/route${ordinal}`));
|
||||
add(accumulator, record(100, "/1".repeat(200)));
|
||||
const before = accumulator.page(0, 1000);
|
||||
const result = accumulator.add(
|
||||
record(101, `/${"2".repeat(80)}`.repeat(200), "GET", "text/plain"),
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisResourceConstraintError" },
|
||||
});
|
||||
expect(accumulator.page(0, 1000)).toEqual(before);
|
||||
});
|
||||
|
||||
it("bounds excluded-record storage and distinct limitations as well as endpoint groups", () => {
|
||||
const accumulator = new WebNetworkApiSurfaceAccumulator(64 * 1024);
|
||||
let accepted = 0;
|
||||
for (; accepted < 1000; accepted += 1) {
|
||||
const sample = record(accepted, "/a");
|
||||
sample.reported = null;
|
||||
sample.limitations = [`producer caveat ${accepted}`];
|
||||
const result = accumulator.add(sample);
|
||||
if (!result.ok) {
|
||||
expect(result.error).toMatchObject({
|
||||
_tag: "AnalysisResourceConstraintError",
|
||||
});
|
||||
break;
|
||||
}
|
||||
}
|
||||
expect(accepted).toBeGreaterThan(0);
|
||||
expect(accepted).toBeLessThan(1000);
|
||||
expect(accumulator.page(0, 1).excluded_records).toHaveLength(accepted);
|
||||
});
|
||||
|
|
@ -0,0 +1,467 @@
|
|||
import { z } from "zod";
|
||||
import {
|
||||
AnalysisInputError,
|
||||
AnalysisResourceConstraintError,
|
||||
} from "../analysisErrorCore.js";
|
||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||
import { compareUnicodeCodePoints as compare } from "../unicodeCodePointOrder.js";
|
||||
import { parseRecordedTransportUrl } from "../recordedTransportUrl.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import { MAX_JSON_DEPTH } from "../jsonValue.js";
|
||||
import {
|
||||
WEB_NETWORK_CAPTURE_LIMITS,
|
||||
type WebNetworkCaptureRecord,
|
||||
} from "../webNetworkCapture.js";
|
||||
import { pageViewCoverage } from "./analysisViewCoverage.js";
|
||||
import type { UnsignedAnalysisView } from "./analysisView.js";
|
||||
|
||||
type Surface = Extract<UnsignedAnalysisView, { kind: "api-surface" }>;
|
||||
type Group = Surface["groups"][number];
|
||||
type Rule = Group["templated_segments"][number]["rule"];
|
||||
type Exclusion = Surface["excluded_records"][number];
|
||||
interface Segment {
|
||||
readonly segment_index: number;
|
||||
readonly rule: Rule;
|
||||
readonly value: string;
|
||||
}
|
||||
interface Route {
|
||||
readonly template: string;
|
||||
readonly rules: string;
|
||||
readonly segments: readonly Segment[];
|
||||
}
|
||||
interface GroupState {
|
||||
readonly method: string;
|
||||
readonly origin: string;
|
||||
readonly path_template: string;
|
||||
readonly segments: readonly {
|
||||
readonly segment_index: number;
|
||||
readonly rule: Rule;
|
||||
readonly values: Set<string>;
|
||||
}[];
|
||||
readonly ordinals: number[];
|
||||
readonly statuses: Set<number>;
|
||||
readonly contentTypes: Set<string>;
|
||||
}
|
||||
type Buckets = Map<string, Map<string, Map<string, Map<string, GroupState>>>>;
|
||||
|
||||
const requestSchema = z.object({
|
||||
request: z.object({ url: z.unknown(), method: z.unknown() }),
|
||||
});
|
||||
const responseSchema = z.object({
|
||||
response: z.object({
|
||||
status: z.unknown().optional(),
|
||||
content: z.unknown().optional(),
|
||||
}),
|
||||
});
|
||||
const contentSchema = z.object({ mimeType: z.unknown().optional() });
|
||||
const MEDIA_TYPE_ESSENCE =
|
||||
/^[A-Za-z0-9!#$%&'*+.^_`|~-]+\/[A-Za-z0-9!#$%&'*+.^_`|~-]+$/u;
|
||||
|
||||
const mediaTypeEssence = (reported: string): string | null => {
|
||||
const separator = reported.indexOf(";");
|
||||
let start = 0;
|
||||
let end = separator < 0 ? reported.length : separator;
|
||||
// HTTP OWS is SP/HTAB. Scan from both ends so malformed internal runs of
|
||||
// whitespace cannot cause repeated trailing-regex searches.
|
||||
while (start < end && (reported[start] === " " || reported[start] === "\t"))
|
||||
start += 1;
|
||||
while (
|
||||
end > start &&
|
||||
(reported[end - 1] === " " || reported[end - 1] === "\t")
|
||||
)
|
||||
end -= 1;
|
||||
const essence = reported.slice(start, end);
|
||||
// Validate ASCII tokens before case folding can turn Unicode into ASCII.
|
||||
return MEDIA_TYPE_ESSENCE.test(essence) ? essence.toLowerCase() : null;
|
||||
};
|
||||
|
||||
// Conservative representation allowances, not an enforced V8 heap limit.
|
||||
// Fixed charges include the accumulator, bucket/set entries, array growth and
|
||||
// page/schema projection. Text includes UTF-16 storage and two worst-case JSON
|
||||
// encodings (transport text and UTF-8 bytes); no serialization is needed to count.
|
||||
const GROUP_BYTES = 2048;
|
||||
const SEGMENT_BYTES = 1024;
|
||||
const ENTRY_BYTES = 128;
|
||||
const RECORD_SCRATCH_BYTES = 512;
|
||||
const SEGMENT_SCRATCH_BYTES = 512;
|
||||
const CAPTURE_CONTAINER_BYTES = 512;
|
||||
// Covers cloned member slots and per-member validation/entries/map scratch.
|
||||
// Scalar strings remain shared references, so their text is not copied here.
|
||||
const CAPTURE_MEMBER_BYTES = 256;
|
||||
const textBytes = (length: number): number => 32 + length * 14;
|
||||
|
||||
const segmentRule = (value: string): Rule | null => {
|
||||
if (/^[0-9]+$/u.test(value)) return "numeric";
|
||||
if (
|
||||
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/iu.test(
|
||||
value,
|
||||
)
|
||||
)
|
||||
return "uuid";
|
||||
if (/^[0-9a-f]{32,}$/iu.test(value)) return "hex";
|
||||
if (/^[0-9]{4}-[0-9]{2}-[0-9]{2}$/u.test(value)) {
|
||||
const date = new Date(`${value}T00:00:00.000Z`);
|
||||
if (
|
||||
Number.isFinite(date.getTime()) &&
|
||||
date.toISOString().slice(0, 10) === value
|
||||
)
|
||||
return "iso-date";
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
/** Own group identity, deduplication and admission for the entire projection. */
|
||||
export class WebNetworkApiSurfaceAccumulator {
|
||||
readonly #buckets: Buckets = new Map();
|
||||
readonly #groups: GroupState[] = [];
|
||||
readonly #excluded: Exclusion[] = [];
|
||||
readonly #limitations = new Set<string>();
|
||||
#unavailableResponse = false;
|
||||
#retainedBytes = 0;
|
||||
#temporaryBytes = 0;
|
||||
|
||||
constructor(readonly maximumBytes = WEB_NETWORK_CAPTURE_LIMITS.outputBytes) {}
|
||||
|
||||
#retain(bytes: number): boolean {
|
||||
if (this.#retainedBytes + this.#temporaryBytes + bytes > this.maximumBytes)
|
||||
return false;
|
||||
this.#retainedBytes += bytes;
|
||||
return true;
|
||||
}
|
||||
|
||||
#temporary(bytes: number): boolean {
|
||||
if (this.#retainedBytes + this.#temporaryBytes + bytes > this.maximumBytes)
|
||||
return false;
|
||||
this.#temporaryBytes += bytes;
|
||||
return true;
|
||||
}
|
||||
|
||||
#overBudget(): Result<never, AnalysisError> {
|
||||
return err(
|
||||
new AnalysisResourceConstraintError(
|
||||
"inspect_analysis_view",
|
||||
"memory",
|
||||
"API-surface retained state and temporary work exceed the projection representation budget.",
|
||||
{ output_bytes: this.maximumBytes },
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
/** Reserve schema output and validation scratch while the borrowed parent stays live. */
|
||||
admitCaptureClone(value: unknown): Result<void, AnalysisError> {
|
||||
return this.#admitCaptureValue(value, new Set(), 0, false);
|
||||
}
|
||||
|
||||
/** Admit the complete portable envelope before cloning or authenticating it. */
|
||||
admitInlineEvidence(value: unknown): Result<void, AnalysisError> {
|
||||
return this.#admitCaptureValue(value, new Set(), 0, true);
|
||||
}
|
||||
|
||||
#admitCaptureValue(
|
||||
value: unknown,
|
||||
ancestors: Set<object>,
|
||||
depth: number,
|
||||
inline: boolean,
|
||||
): Result<void, AnalysisError> {
|
||||
// Capture records add three levels before reported; inline Evidence adds one.
|
||||
if (
|
||||
depth > MAX_JSON_DEPTH + (inline ? 4 : 3) ||
|
||||
(typeof value === "object" && value !== null && ancestors.has(value))
|
||||
)
|
||||
return this.#invalidCapture();
|
||||
if (!this.#retain(CAPTURE_MEMBER_BYTES)) return this.#overBudget();
|
||||
// Evidence authentication encodes strings and object keys for hashing.
|
||||
// Keep that allowance live alongside the envelope clone and projection.
|
||||
if (
|
||||
inline &&
|
||||
typeof value === "string" &&
|
||||
!this.#retain(textBytes(value.length))
|
||||
)
|
||||
return this.#overBudget();
|
||||
if (typeof value !== "object" || value === null) return ok(undefined);
|
||||
if (!this.#retain(CAPTURE_CONTAINER_BYTES)) return this.#overBudget();
|
||||
ancestors.add(value);
|
||||
// Walk depth-first without building a second capture graph. Member charges
|
||||
// include key enumeration; aliases count each time the schema copies them.
|
||||
if (Array.isArray(value)) {
|
||||
for (let index = 0; index < value.length; index += 1) {
|
||||
const descriptor = Object.getOwnPropertyDescriptor(value, index);
|
||||
if (descriptor === undefined || !Object.hasOwn(descriptor, "value"))
|
||||
return this.#invalidCapture();
|
||||
const admitted = this.#admitCaptureValue(
|
||||
descriptor.value,
|
||||
ancestors,
|
||||
depth + 1,
|
||||
inline,
|
||||
);
|
||||
if (!admitted.ok) return admitted;
|
||||
}
|
||||
} else {
|
||||
const prototype: unknown = Object.getPrototypeOf(value);
|
||||
if (prototype !== Object.prototype && prototype !== null)
|
||||
return this.#invalidCapture();
|
||||
// Zod reads named fields even when they are non-enumerable.
|
||||
for (const key of Object.getOwnPropertyNames(value)) {
|
||||
if (inline && !this.#retain(textBytes(key.length)))
|
||||
return this.#overBudget();
|
||||
const descriptor = Object.getOwnPropertyDescriptor(value, key);
|
||||
if (descriptor === undefined || !Object.hasOwn(descriptor, "value"))
|
||||
return this.#invalidCapture();
|
||||
const admitted = this.#admitCaptureValue(
|
||||
descriptor.value,
|
||||
ancestors,
|
||||
depth + 1,
|
||||
inline,
|
||||
);
|
||||
if (!admitted.ok) return admitted;
|
||||
}
|
||||
}
|
||||
ancestors.delete(value);
|
||||
return ok(undefined);
|
||||
}
|
||||
|
||||
#invalidCapture(): Result<never, AnalysisError> {
|
||||
return err(
|
||||
new AnalysisInputError("inspect_analysis_view", undefined, [
|
||||
{
|
||||
path: ["source"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
"Parent capture must contain bounded ordinary JSON data without cycles or accessors.",
|
||||
},
|
||||
]),
|
||||
);
|
||||
}
|
||||
|
||||
addLimitations(limitations: readonly string[]): Result<void, AnalysisError> {
|
||||
for (const limitation of limitations) {
|
||||
if (this.#limitations.has(limitation)) continue;
|
||||
if (!this.#retain(ENTRY_BYTES + textBytes(limitation.length)))
|
||||
return this.#overBudget();
|
||||
this.#limitations.add(limitation);
|
||||
}
|
||||
return ok(undefined);
|
||||
}
|
||||
|
||||
#exclude(
|
||||
ordinal: number,
|
||||
reason: Exclusion["reason"],
|
||||
): Result<void, AnalysisError> {
|
||||
if (!this.#retain(ENTRY_BYTES + RECORD_SCRATCH_BYTES))
|
||||
return this.#overBudget();
|
||||
this.#excluded.push({ ordinal, reason });
|
||||
return ok(undefined);
|
||||
}
|
||||
|
||||
/** Scan literal spans; only matched segments need descriptors or template parts. */
|
||||
#route(path: string): Result<Route, AnalysisError> {
|
||||
const segments: Segment[] = [];
|
||||
const parts: string[] = [];
|
||||
const rules: string[] = [];
|
||||
let templateLength = path.length;
|
||||
let rulesLength = 0;
|
||||
let copied = 0;
|
||||
let index = 0;
|
||||
let start = 1;
|
||||
for (;;) {
|
||||
const slash = path.indexOf("/", start);
|
||||
const end = slash < 0 ? path.length : slash;
|
||||
const value = path.slice(start, end);
|
||||
const rule = segmentRule(value);
|
||||
if (rule !== null) {
|
||||
if (!this.#temporary(SEGMENT_SCRATCH_BYTES)) return this.#overBudget();
|
||||
const placeholder = `{${rule}}`;
|
||||
const identity = `${index}:${rule};`;
|
||||
segments.push({ segment_index: index, rule, value });
|
||||
parts.push(path.slice(copied, start), placeholder);
|
||||
rules.push(identity);
|
||||
templateLength += placeholder.length - value.length;
|
||||
rulesLength += identity.length;
|
||||
copied = end;
|
||||
}
|
||||
if (slash < 0) break;
|
||||
start = slash + 1;
|
||||
index += 1;
|
||||
}
|
||||
if (segments.length === 0)
|
||||
return ok({ template: path, rules: "", segments });
|
||||
if (!this.#temporary(textBytes(templateLength) + textBytes(rulesLength)))
|
||||
return this.#overBudget();
|
||||
parts.push(path.slice(copied));
|
||||
return ok({ template: parts.join(""), rules: rules.join(""), segments });
|
||||
}
|
||||
|
||||
add(record: WebNetworkCaptureRecord): Result<void, AnalysisError> {
|
||||
const limitations = this.addLimitations(record.limitations);
|
||||
if (!limitations.ok) return limitations;
|
||||
const request = requestSchema.safeParse(record.reported);
|
||||
if (!request.success)
|
||||
return this.#exclude(record.ordinal, "request-unavailable");
|
||||
const { url, method } = request.data.request;
|
||||
if (url === null) return this.#exclude(record.ordinal, "url-redacted");
|
||||
if (typeof url !== "string")
|
||||
return this.#exclude(record.ordinal, "url-unparseable");
|
||||
// Admission precedes URL parsing, text normalization and path materialization.
|
||||
if (!this.#temporary(RECORD_SCRATCH_BYTES + textBytes(url.length)))
|
||||
return this.#overBudget();
|
||||
try {
|
||||
const parsed = parseRecordedTransportUrl(url);
|
||||
if (parsed.origin === null)
|
||||
return this.#exclude(record.ordinal, "url-unparseable");
|
||||
if (typeof method !== "string" || method.length === 0)
|
||||
return this.#exclude(record.ordinal, "method-unavailable");
|
||||
if (parsed.path === null)
|
||||
return this.#exclude(record.ordinal, "url-unparseable");
|
||||
const response = responseSchema.safeParse(record.reported);
|
||||
const status = response.success
|
||||
? response.data.response.status
|
||||
: undefined;
|
||||
const content = contentSchema.safeParse(
|
||||
response.success ? response.data.response.content : undefined,
|
||||
);
|
||||
const mime = content.success ? content.data.mimeType : undefined;
|
||||
const rawMime = typeof mime === "string" ? mime : "";
|
||||
if (!this.#temporary(textBytes(rawMime.length)))
|
||||
return this.#overBudget();
|
||||
const route = this.#route(parsed.path);
|
||||
if (!route.ok) return route;
|
||||
return this.#append(
|
||||
record.ordinal,
|
||||
method,
|
||||
parsed.origin,
|
||||
route.value,
|
||||
typeof status === "number" && status !== 0 ? status : null,
|
||||
mediaTypeEssence(rawMime),
|
||||
);
|
||||
} finally {
|
||||
// Per-record strings and descriptors never become cumulative scratch.
|
||||
this.#temporaryBytes = 0;
|
||||
}
|
||||
}
|
||||
|
||||
#append(
|
||||
ordinal: number,
|
||||
method: string,
|
||||
origin: string,
|
||||
route: Route,
|
||||
status: number | null,
|
||||
mime: string | null,
|
||||
): Result<void, AnalysisError> {
|
||||
let group = this.#buckets
|
||||
.get(method)
|
||||
?.get(origin)
|
||||
?.get(route.template)
|
||||
?.get(route.rules);
|
||||
let bytes = ENTRY_BYTES; // One retained record ordinal, including page output.
|
||||
if (group === undefined) {
|
||||
bytes +=
|
||||
GROUP_BYTES +
|
||||
textBytes(method.length) +
|
||||
textBytes(origin.length) +
|
||||
textBytes(route.template.length) +
|
||||
textBytes(route.rules.length);
|
||||
for (const segment of route.segments)
|
||||
bytes += SEGMENT_BYTES + ENTRY_BYTES + textBytes(segment.value.length);
|
||||
} else {
|
||||
for (const [index, segment] of route.segments.entries()) {
|
||||
if (!group.segments[index]!.values.has(segment.value))
|
||||
bytes += ENTRY_BYTES + textBytes(segment.value.length);
|
||||
}
|
||||
}
|
||||
if (status !== null && !group?.statuses.has(status)) bytes += ENTRY_BYTES;
|
||||
if (mime !== null && !group?.contentTypes.has(mime))
|
||||
bytes += ENTRY_BYTES + textBytes(mime.length);
|
||||
// Reserve the whole mutation while route scratch is still live. No sets,
|
||||
// map entries, arrays or new observed values are retained before admission.
|
||||
if (!this.#retain(bytes)) return this.#overBudget();
|
||||
if (group === undefined) {
|
||||
group = {
|
||||
method,
|
||||
origin,
|
||||
path_template: route.template,
|
||||
segments: route.segments.map((segment) => ({
|
||||
segment_index: segment.segment_index,
|
||||
rule: segment.rule,
|
||||
values: new Set([segment.value]),
|
||||
})),
|
||||
ordinals: [],
|
||||
statuses: new Set(),
|
||||
contentTypes: new Set(),
|
||||
};
|
||||
const byOrigin = this.#buckets.get(method) ?? new Map();
|
||||
this.#buckets.set(method, byOrigin);
|
||||
const byTemplate = byOrigin.get(origin) ?? new Map();
|
||||
byOrigin.set(origin, byTemplate);
|
||||
const byRule = byTemplate.get(route.template) ?? new Map();
|
||||
byTemplate.set(route.template, byRule);
|
||||
byRule.set(route.rules, group);
|
||||
this.#groups.push(group);
|
||||
} else {
|
||||
for (const [index, segment] of route.segments.entries())
|
||||
group.segments[index]!.values.add(segment.value);
|
||||
}
|
||||
group.ordinals.push(ordinal);
|
||||
if (status === null || mime === null) this.#unavailableResponse = true;
|
||||
if (status !== null) group.statuses.add(status);
|
||||
if (mime !== null) group.contentTypes.add(mime);
|
||||
return ok(undefined);
|
||||
}
|
||||
|
||||
page(
|
||||
offset: number,
|
||||
limit: number,
|
||||
): Pick<
|
||||
Surface,
|
||||
| "coverage"
|
||||
| "groups"
|
||||
| "templated_groups"
|
||||
| "excluded_records"
|
||||
| "limitations"
|
||||
| "unknowns"
|
||||
> {
|
||||
this.#groups.sort(
|
||||
(a, b) =>
|
||||
compare(a.origin, b.origin) ||
|
||||
compare(a.path_template, b.path_template) ||
|
||||
compare(a.method, b.method),
|
||||
);
|
||||
const groups = this.#groups
|
||||
.slice(offset, offset + limit)
|
||||
.map((group): Group => ({
|
||||
method: group.method,
|
||||
origin: group.origin,
|
||||
path_template: group.path_template,
|
||||
templated_segments: group.segments.map((segment) => ({
|
||||
segment_index: segment.segment_index,
|
||||
rule: segment.rule,
|
||||
basis: "inferred",
|
||||
observed_values: [...segment.values].sort(compare),
|
||||
})),
|
||||
record_ordinals: [...group.ordinals],
|
||||
statuses: [...group.statuses].sort((a, b) => a - b),
|
||||
content_types: [...group.contentTypes].sort(compare),
|
||||
flags: [
|
||||
...(group.ordinals.length === 1 ? ["single-sample" as const] : []),
|
||||
...(group.statuses.size > 1 ? ["mixed-statuses" as const] : []),
|
||||
...(group.contentTypes.size > 1
|
||||
? ["mixed-content-types" as const]
|
||||
: []),
|
||||
],
|
||||
}));
|
||||
return {
|
||||
coverage: pageViewCoverage(offset, groups.length, this.#groups.length),
|
||||
groups,
|
||||
templated_groups: this.#groups.reduce(
|
||||
(count, group) => count + (group.segments.length > 0 ? 1 : 0),
|
||||
0,
|
||||
),
|
||||
excluded_records: [...this.#excluded],
|
||||
limitations: [...this.#limitations].sort(compare),
|
||||
unknowns: this.#unavailableResponse
|
||||
? [
|
||||
"Some retained records have unavailable status or content type values.",
|
||||
]
|
||||
: [],
|
||||
};
|
||||
}
|
||||
}
|
||||
|
|
@ -21,7 +21,13 @@ import {
|
|||
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
|
||||
import { projectBinaryLayoutView } from "./binaryLayoutView.js";
|
||||
import { projectJavaScriptApplicationView } from "./javascriptApplicationView.js";
|
||||
import {
|
||||
projectWebNetworkApiSurfaceView,
|
||||
apiSurfaceFields,
|
||||
apiSurfaceViewSchema,
|
||||
} from "./webNetworkApiSurfaceView.js";
|
||||
import { projectNativeFunctionView } from "./nativeFunctionView.js";
|
||||
import type { WebNetworkApiSurfaceAccumulator } from "./WebNetworkApiSurfaceAccumulator.js";
|
||||
|
||||
/** Public name of the selected-view workflow. */
|
||||
export const INSPECT_ANALYSIS_VIEW_OPERATION = "inspect_analysis_view" as const;
|
||||
|
|
@ -109,9 +115,17 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
|
|||
itemViewSchema,
|
||||
pageViewSchema,
|
||||
nativeViewSchema,
|
||||
apiSurfaceViewSchema,
|
||||
]);
|
||||
|
||||
/** Inline Evidence or an exact same-session retained reference. */
|
||||
// The application owns admission and authenticated Evidence parsing. SDK and
|
||||
// CLI envelope parsing must borrow this unknown value rather than clone it
|
||||
// before admission. Advertise the owning Evidence contract in both dialects.
|
||||
const inlineEvidenceInputSchema = z.unknown().nonoptional();
|
||||
inlineEvidenceInputSchema._zod.processJSONSchema =
|
||||
evidenceSchema._zod.processJSONSchema;
|
||||
|
||||
/** Unauthenticated inline input or an exact same-session retained reference. */
|
||||
export const analysisViewSourceSchema = z.discriminatedUnion("kind", [
|
||||
z.strictObject({
|
||||
kind: z.literal("retained-evidence"),
|
||||
|
|
@ -119,7 +133,7 @@ export const analysisViewSourceSchema = z.discriminatedUnion("kind", [
|
|||
}),
|
||||
z.strictObject({
|
||||
kind: z.literal("inline"),
|
||||
evidence: evidenceSchema,
|
||||
evidence: inlineEvidenceInputSchema,
|
||||
}),
|
||||
]);
|
||||
|
||||
|
|
@ -178,6 +192,32 @@ const javascriptSummarySchema = z.strictObject({
|
|||
limitation_count: z.number().int().nonnegative(),
|
||||
});
|
||||
|
||||
const summaryViewResultFields = {
|
||||
kind: z.literal("summary"),
|
||||
view: summaryViewSchema,
|
||||
parent_evidence_id: prefixedDigestSchema("ev"),
|
||||
parent_digest: digestSchema,
|
||||
view_digest: digestSchema,
|
||||
artifact: artifactSchema,
|
||||
coverage: coverageSchema,
|
||||
limitations: z.array(z.string()),
|
||||
unknowns: z.array(z.string()),
|
||||
} as const;
|
||||
|
||||
/** Summary `inspect_binary_layout` detail returns; no other view kind. */
|
||||
export const binaryLayoutSummaryViewResultSchema = z.strictObject({
|
||||
...summaryViewResultFields,
|
||||
summary: layoutSummarySchema,
|
||||
parent_operation: z.literal("inspect_binary_layout"),
|
||||
});
|
||||
|
||||
/** Summary `analyze_javascript_application` detail returns; no other view kind. */
|
||||
export const javascriptApplicationSummaryViewResultSchema = z.strictObject({
|
||||
...summaryViewResultFields,
|
||||
summary: javascriptSummarySchema,
|
||||
parent_operation: z.literal("analyze_javascript_application"),
|
||||
});
|
||||
|
||||
const viewResultBase = {
|
||||
parent_evidence_id: prefixedDigestSchema("ev"),
|
||||
parent_operation: z.enum([
|
||||
|
|
@ -195,6 +235,17 @@ const viewResultBase = {
|
|||
|
||||
/** Projected facts for one selected view of retained analysis Evidence. */
|
||||
export const analysisViewResultSchema = z.discriminatedUnion("kind", [
|
||||
z.strictObject({
|
||||
...viewResultBase,
|
||||
...apiSurfaceFields,
|
||||
kind: z.literal("api-surface"),
|
||||
parent_operation: z.literal("inspect_web_network_capture"),
|
||||
view: apiSurfaceViewSchema,
|
||||
artifact: z.strictObject({
|
||||
path: z.string(),
|
||||
sha256: digestSchema.nullable(),
|
||||
}),
|
||||
}),
|
||||
z.strictObject({
|
||||
kind: z.literal("summary"),
|
||||
view: summaryViewSchema,
|
||||
|
|
@ -242,6 +293,8 @@ export interface AnalysisViewParent {
|
|||
readonly normalizedResult: unknown;
|
||||
readonly limitations: readonly string[];
|
||||
readonly artifact?: { readonly path: string; readonly sha256: string } | null;
|
||||
/** Parent Evidence parameters, when the parent operation records them. */
|
||||
readonly parameters?: unknown;
|
||||
}
|
||||
|
||||
/** Construct a typed input failure for one selected-view constraint. */
|
||||
|
|
@ -252,6 +305,10 @@ export const analysisViewInputError = (
|
|||
|
||||
/** View payload before the content digest is attached. */
|
||||
export type UnsignedAnalysisView =
|
||||
| Omit<
|
||||
Extract<AnalysisViewResult, { readonly kind: "api-surface" }>,
|
||||
"view_digest"
|
||||
>
|
||||
| Omit<
|
||||
Extract<AnalysisViewResult, { readonly kind: "summary" }>,
|
||||
"view_digest"
|
||||
|
|
@ -335,6 +392,7 @@ const parentArtifactPath = (parent: AnalysisViewParent): string => {
|
|||
export const projectAnalysisView = (
|
||||
parent: AnalysisViewParent,
|
||||
view: AnalysisViewRequest,
|
||||
apiSurfaceAccumulator?: WebNetworkApiSurfaceAccumulator,
|
||||
): Result<AnalysisViewResult, AnalysisError> => {
|
||||
const digest = parentDigest(parent.evidenceId);
|
||||
if (!digestSchema.safeParse(digest).success)
|
||||
|
|
@ -347,6 +405,17 @@ export const projectAnalysisView = (
|
|||
},
|
||||
]),
|
||||
);
|
||||
if (
|
||||
view.kind === "api-surface" ||
|
||||
parent.operation === "inspect_web_network_capture"
|
||||
) {
|
||||
const projected = projectWebNetworkApiSurfaceView(
|
||||
parent,
|
||||
view,
|
||||
apiSurfaceAccumulator,
|
||||
);
|
||||
return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
|
||||
}
|
||||
if (parent.operation === "inspect_binary_layout") {
|
||||
const layout = binaryLayoutSchema.safeParse(parent.normalizedResult);
|
||||
if (!layout.success)
|
||||
|
|
|
|||
|
|
@ -149,7 +149,7 @@ const layoutSummary = (layout: BinaryLayout) => ({
|
|||
export const projectBinaryLayoutView = (
|
||||
parent: AnalysisViewParent,
|
||||
layout: BinaryLayout,
|
||||
view: AnalysisViewRequest,
|
||||
view: Exclude<AnalysisViewRequest, { kind: "api-surface" }>,
|
||||
): Result<UnsignedAnalysisView, AnalysisError> => {
|
||||
const shared = parentFields(parent, layout);
|
||||
if (view.kind === "native")
|
||||
|
|
|
|||
|
|
@ -239,7 +239,7 @@ const selectModule = (
|
|||
export const projectJavaScriptApplicationView = (
|
||||
parent: AnalysisViewParent,
|
||||
analysis: JavaScriptApplicationAnalysisResult,
|
||||
view: AnalysisViewRequest,
|
||||
view: Exclude<AnalysisViewRequest, { kind: "api-surface" }>,
|
||||
): Result<UnsignedAnalysisView, AnalysisError> => {
|
||||
const shared = parentFields(parent, analysis);
|
||||
const modules = moduleEntries(analysis.graph.nodes);
|
||||
|
|
|
|||
|
|
@ -0,0 +1,460 @@
|
|||
import { expect, it } from "vitest";
|
||||
import {
|
||||
analysisViewCaptureFixture,
|
||||
analysisViewCaptureEvidence,
|
||||
} from "../../../tests/fixtures/analysisView.js";
|
||||
import { projectAnalysisView } from "./analysisView.js";
|
||||
|
||||
const project = (
|
||||
capture = analysisViewCaptureFixture(),
|
||||
offset = 0,
|
||||
limit = 50,
|
||||
parameters: unknown = { sensitive_value_count: 0 },
|
||||
) => {
|
||||
const parent = analysisViewCaptureEvidence(capture);
|
||||
return projectAnalysisView(
|
||||
{
|
||||
evidenceId: parent.evidence_id,
|
||||
operation: parent.operation,
|
||||
normalizedResult: parent.normalized_result,
|
||||
limitations: [],
|
||||
parameters,
|
||||
},
|
||||
{ kind: "api-surface", offset, limit },
|
||||
);
|
||||
};
|
||||
it("groups only exact formats, preserves literal routes, excludes URLs and produces deterministic digests", () => {
|
||||
const result = project();
|
||||
if (!result.ok || result.value.kind !== "api-surface")
|
||||
throw new Error("Expected surface");
|
||||
expect(project()).toEqual(result);
|
||||
const surface = result.value;
|
||||
expect(surface.groups.map((group) => group.path_template)).toEqual([
|
||||
"/api/CreateDocument",
|
||||
"/api/ListDocuments",
|
||||
"/api/Report2",
|
||||
"/api/Report3",
|
||||
"/api/v1/users",
|
||||
"/api/v2/users",
|
||||
"/dates/2023-02-29",
|
||||
"/dates/{iso-date}",
|
||||
"/ids/abc123",
|
||||
"/ids/{hex}",
|
||||
"/ids/{uuid}",
|
||||
"/users/{numeric}",
|
||||
]);
|
||||
expect(surface.groups.at(-1)).toMatchObject({
|
||||
record_ordinals: [0, 1],
|
||||
statuses: [200, 404],
|
||||
content_types: ["application/json", "text/plain"],
|
||||
flags: ["mixed-statuses", "mixed-content-types"],
|
||||
templated_segments: [
|
||||
{
|
||||
segment_index: 1,
|
||||
rule: "numeric",
|
||||
basis: "inferred",
|
||||
observed_values: ["8812", "9031"],
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(surface.excluded_records).toEqual([
|
||||
{ ordinal: 13, reason: "url-redacted" },
|
||||
{ ordinal: 14, reason: "url-unparseable" },
|
||||
]);
|
||||
expect(JSON.stringify(surface)).not.toContain("not-in-view");
|
||||
expect(JSON.stringify(surface)).not.toContain("雪");
|
||||
});
|
||||
it("pages groups independently of record completeness and preserves null digests", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.artifact.sha256 = null;
|
||||
capture.total_records = 30;
|
||||
const result = project(capture, 10, 2);
|
||||
if (!result.ok) throw result.error;
|
||||
expect(result.value).toMatchObject({
|
||||
artifact: { sha256: null },
|
||||
coverage: { examined: 2, total: 12, exhausted: true, next_offset: null },
|
||||
records: { examined: 15, total: 30, excluded: 2 },
|
||||
});
|
||||
expect(result.value.limitations.join(" ")).toContain("ordinal subset");
|
||||
expect(project(capture, 50, 2)).toMatchObject({
|
||||
ok: true,
|
||||
value: {
|
||||
coverage: { status: "empty", exhausted: true, next_offset: null },
|
||||
},
|
||||
});
|
||||
});
|
||||
it("rejects native captures and malformed parents with typed errors", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.format = "mitmproxy";
|
||||
expect(project(capture)).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisUnsupportedTargetError" },
|
||||
});
|
||||
expect(
|
||||
projectAnalysisView(
|
||||
{
|
||||
evidenceId: `ev_${"a".repeat(64)}`,
|
||||
operation: "inspect_web_network_capture",
|
||||
normalizedResult: {},
|
||||
limitations: [],
|
||||
},
|
||||
{ kind: "api-surface", offset: 0, limit: 1 },
|
||||
),
|
||||
).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } });
|
||||
});
|
||||
|
||||
it("preserves partial response facts and explains unavailable requests", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.records = capture.records.slice(0, 5);
|
||||
const values = [
|
||||
{
|
||||
request: {
|
||||
method: "GET",
|
||||
url: "https://user:password@api.example.test/1/2?private=value",
|
||||
},
|
||||
response: { status: 200, content: null },
|
||||
},
|
||||
{ request: { method: null, url: "https://api.example.test/" } },
|
||||
null,
|
||||
{ request: { method: "GET", url: "data:text/plain,hello" } },
|
||||
{
|
||||
request: { method: "GET", url: "https://api.example.test/1/2" },
|
||||
response: { status: null, content: { mimeType: "application/json" } },
|
||||
},
|
||||
];
|
||||
capture.records.forEach((record, index) => {
|
||||
record.reported = values[index] ?? null;
|
||||
});
|
||||
const result = project(capture);
|
||||
if (!result.ok || result.value.kind !== "api-surface")
|
||||
throw new Error("Expected surface");
|
||||
expect(result.value.groups).toMatchObject([
|
||||
{
|
||||
origin: "https://api.example.test",
|
||||
path_template: "/{numeric}/{numeric}",
|
||||
statuses: [200],
|
||||
content_types: ["application/json"],
|
||||
templated_segments: [
|
||||
{ segment_index: 0, observed_values: ["1"] },
|
||||
{ segment_index: 1, observed_values: ["2"] },
|
||||
],
|
||||
},
|
||||
]);
|
||||
expect(result.value.excluded_records).toEqual([
|
||||
{ ordinal: 1, reason: "method-unavailable" },
|
||||
{ ordinal: 2, reason: "request-unavailable" },
|
||||
{ ordinal: 3, reason: "url-unparseable" },
|
||||
]);
|
||||
expect(result.value.unknowns).not.toEqual([]);
|
||||
expect(JSON.stringify(result.value)).not.toContain("password");
|
||||
expect(JSON.stringify(result.value)).not.toContain("private");
|
||||
});
|
||||
|
||||
it("keeps encoded segments literal and distinguishes origin and method", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.records = capture.records.slice(0, 4);
|
||||
const urls = [
|
||||
"https://a.test/%31",
|
||||
"https://a.test/1",
|
||||
"https://b.test/1",
|
||||
"https://a.test/1",
|
||||
];
|
||||
capture.records.forEach((record, index) => {
|
||||
record.reported = {
|
||||
request: { method: index === 3 ? "POST" : "GET", url: urls[index]! },
|
||||
response: {
|
||||
status: 200,
|
||||
content: { mimeType: "application/json; charset=utf-8" },
|
||||
},
|
||||
};
|
||||
});
|
||||
const result = project(capture, 0, 2);
|
||||
if (!result.ok || result.value.kind !== "api-surface")
|
||||
throw new Error("Expected surface");
|
||||
expect(result.value.coverage).toMatchObject({
|
||||
total: 4,
|
||||
next_offset: 2,
|
||||
exhausted: false,
|
||||
});
|
||||
expect(result.value.groups[0]?.path_template).toBe("/%31");
|
||||
});
|
||||
|
||||
it("refuses parents whose sensitive declarations it cannot check", () => {
|
||||
for (const parameters of [{ sensitive_value_count: 1 }, {}, null]) {
|
||||
const result = project(analysisViewCaptureFixture(), 0, 50, parameters);
|
||||
expect(result).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisUnsupportedTargetError" },
|
||||
});
|
||||
if (result.ok) continue;
|
||||
expect(result.error).toMatchObject({
|
||||
reason: "sensitive_values_unretained",
|
||||
});
|
||||
expect(result.error.message).not.toContain("path templates");
|
||||
}
|
||||
});
|
||||
|
||||
it("rejects cyclic captures and accessor members without invoking them", () => {
|
||||
const cyclic: { self?: unknown } = {};
|
||||
cyclic.self = cyclic;
|
||||
let accessed = false;
|
||||
const accessor = {
|
||||
get records() {
|
||||
accessed = true;
|
||||
return [];
|
||||
},
|
||||
};
|
||||
const hiddenAccessor = Object.defineProperty({}, "records", {
|
||||
get() {
|
||||
accessed = true;
|
||||
return [];
|
||||
},
|
||||
});
|
||||
const inherited = Object.create({ records: [] }) as unknown;
|
||||
for (const normalizedResult of [cyclic, accessor, hiddenAccessor, inherited])
|
||||
expect(
|
||||
projectAnalysisView(
|
||||
{
|
||||
evidenceId: `ev_${"a".repeat(64)}`,
|
||||
operation: "inspect_web_network_capture",
|
||||
normalizedResult,
|
||||
limitations: [],
|
||||
},
|
||||
{ kind: "api-surface", offset: 0, limit: 1 },
|
||||
),
|
||||
).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisInputError" },
|
||||
});
|
||||
expect(accessed).toBe(false);
|
||||
});
|
||||
|
||||
it("treats HAR missing-response sentinels as unknown", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.records = capture.records.slice(0, 1);
|
||||
capture.records[0]!.reported = {
|
||||
request: { method: "GET", url: "https://api.example.test/a" },
|
||||
response: { status: 0, content: { mimeType: " ; charset=utf-8" } },
|
||||
};
|
||||
const result = project(capture);
|
||||
if (!result.ok || result.value.kind !== "api-surface")
|
||||
throw new Error("Expected surface");
|
||||
expect(result.value.groups[0]).toMatchObject({
|
||||
statuses: [],
|
||||
content_types: [],
|
||||
});
|
||||
expect(result.value.unknowns).not.toEqual([]);
|
||||
});
|
||||
|
||||
it("refuses a path whose templated representation exceeds projection capacity", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.records = capture.records.slice(0, 1);
|
||||
capture.records[0]!.reported = {
|
||||
request: {
|
||||
method: "GET",
|
||||
url: `https://api.example.test${"/1".repeat(800_000)}`,
|
||||
},
|
||||
};
|
||||
expect(project(capture)).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisResourceConstraintError" },
|
||||
});
|
||||
});
|
||||
|
||||
it("scans highly segmented literal paths and keeps recorded dot segments", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.records = capture.records.slice(0, 3);
|
||||
const urls = [
|
||||
`https://api.example.test${"/a".repeat(90_000)}`,
|
||||
"https://api.example.test/a/%2e%2e/admin",
|
||||
"https://api.example.test/admin",
|
||||
];
|
||||
capture.records.forEach((record, index) => {
|
||||
record.reported = { request: { method: "GET", url: urls[index] ?? null } };
|
||||
});
|
||||
const result = project(capture);
|
||||
if (!result.ok || result.value.kind !== "api-surface")
|
||||
throw new Error("Expected surface");
|
||||
expect(result.value.groups.map((group) => group.path_template)).toEqual(
|
||||
expect.arrayContaining(["/a/%2e%2e/admin", "/admin"]),
|
||||
);
|
||||
expect(result.value.groups).toHaveLength(3);
|
||||
});
|
||||
|
||||
it("excludes slashless URL spellings instead of reading their paths", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.records = capture.records.slice(0, 4);
|
||||
const urls = [
|
||||
"https:user:pass@example.test/path",
|
||||
"https:123/456",
|
||||
"https:///user:pass@example.test/path",
|
||||
"https://\\user:pass@example.test/path",
|
||||
];
|
||||
capture.records.forEach((record, index) => {
|
||||
record.reported = { request: { method: "GET", url: urls[index] ?? null } };
|
||||
});
|
||||
const result = project(capture);
|
||||
if (!result.ok || result.value.kind !== "api-surface")
|
||||
throw new Error("Expected surface");
|
||||
expect(result.value.groups).toEqual([]);
|
||||
expect(result.value.excluded_records).toEqual([
|
||||
{ ordinal: 0, reason: "url-unparseable" },
|
||||
{ ordinal: 1, reason: "url-unparseable" },
|
||||
{ ordinal: 2, reason: "url-unparseable" },
|
||||
{ ordinal: 3, reason: "url-unparseable" },
|
||||
]);
|
||||
expect(JSON.stringify(result.value)).not.toContain("pass");
|
||||
});
|
||||
|
||||
it("excludes control-only authorities instead of publishing their userinfo", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.records = capture.records.slice(0, 3);
|
||||
const urls = [
|
||||
"https://\t/user:pass@example.test/path",
|
||||
"https://\n/user:pass@example.test/path",
|
||||
"https://\r/user:pass@example.test/path",
|
||||
];
|
||||
capture.records.forEach((record, index) => {
|
||||
record.reported = { request: { method: "GET", url: urls[index] ?? null } };
|
||||
});
|
||||
const result = project(capture);
|
||||
if (!result.ok || result.value.kind !== "api-surface")
|
||||
throw new Error("Expected surface");
|
||||
expect(result.value.groups).toEqual([]);
|
||||
expect(result.value.excluded_records).toEqual([
|
||||
{ ordinal: 0, reason: "url-unparseable" },
|
||||
{ ordinal: 1, reason: "url-unparseable" },
|
||||
{ ordinal: 2, reason: "url-unparseable" },
|
||||
]);
|
||||
expect(JSON.stringify(result.value)).not.toContain("pass");
|
||||
});
|
||||
|
||||
it("keeps distinct content types without repeating a retained essence", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
const record = capture.records[0]!;
|
||||
const types = ["text/plain", "application/json", "text/plain"];
|
||||
capture.records = types.map((mimeType, ordinal) => ({
|
||||
...record,
|
||||
ordinal,
|
||||
reported: {
|
||||
request: { method: "GET", url: "https://api.example.test/a" },
|
||||
response: { status: 200, content: { mimeType } },
|
||||
},
|
||||
}));
|
||||
capture.total_records = capture.records.length;
|
||||
const result = project(capture);
|
||||
if (!result.ok || result.value.kind !== "api-surface")
|
||||
throw new Error("Expected surface");
|
||||
expect(result.value.groups[0]?.content_types).toEqual([
|
||||
"application/json",
|
||||
"text/plain",
|
||||
]);
|
||||
});
|
||||
|
||||
it("charges repeated observed values once when merging", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
const record = capture.records[0]!;
|
||||
const path = "/1".repeat(800);
|
||||
capture.records = Array.from({ length: 1_000 }, (_, ordinal) => ({
|
||||
...record,
|
||||
ordinal,
|
||||
reported: {
|
||||
request: { method: "GET", url: `https://api.example.test${path}` },
|
||||
},
|
||||
}));
|
||||
capture.total_records = capture.records.length;
|
||||
const result = project(capture);
|
||||
if (!result.ok || result.value.kind !== "api-surface")
|
||||
throw new Error("Expected surface");
|
||||
expect(result.value.groups).toHaveLength(1);
|
||||
expect(result.value.groups[0]?.templated_segments).toHaveLength(800);
|
||||
});
|
||||
|
||||
it("keeps a recorded literal placeholder apart from a templated segment", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.records = capture.records.slice(0, 2);
|
||||
const urls = [
|
||||
"https://api.example.test/users/{numeric}",
|
||||
"https://api.example.test/users/123",
|
||||
];
|
||||
capture.records.forEach((record, index) => {
|
||||
record.reported = { request: { method: "GET", url: urls[index] ?? null } };
|
||||
});
|
||||
const result = project(capture);
|
||||
if (!result.ok || result.value.kind !== "api-surface")
|
||||
throw new Error("Expected surface");
|
||||
expect(result.value.groups).toHaveLength(2);
|
||||
expect(
|
||||
result.value.groups.map((group) => group.templated_segments.length).sort(),
|
||||
).toEqual([0, 1]);
|
||||
});
|
||||
|
||||
it("treats malformed media types as unknown and keeps source limitations", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
const caveat =
|
||||
"HAR text is producer-decoded Unicode. Original body bytes remain unknown.";
|
||||
const captureCaveat = "Producer omitted exchanges before capture startup.";
|
||||
capture.limitations = [captureCaveat, caveat];
|
||||
const malformed = [
|
||||
"text /plain",
|
||||
"unknown",
|
||||
"\u00a0text/plain\u00a0",
|
||||
"\u000btext/plain\u000b",
|
||||
"\u000ctext/plain\u000c",
|
||||
"text/K",
|
||||
];
|
||||
capture.records = malformed.map((_, ordinal) => ({
|
||||
...capture.records[0]!,
|
||||
ordinal,
|
||||
}));
|
||||
capture.total_records = capture.records.length;
|
||||
capture.records.forEach((record, index) => {
|
||||
record.limitations = [caveat];
|
||||
record.reported = {
|
||||
request: { method: "GET", url: "https://api.example.test/a" },
|
||||
response: {
|
||||
status: 200,
|
||||
content: { mimeType: malformed[index]! },
|
||||
},
|
||||
};
|
||||
});
|
||||
const result = project(capture);
|
||||
if (!result.ok || result.value.kind !== "api-surface")
|
||||
throw new Error("Expected surface");
|
||||
expect(result.value.groups[0]).toMatchObject({
|
||||
content_types: [],
|
||||
flags: [],
|
||||
});
|
||||
expect(result.value.unknowns).not.toEqual([]);
|
||||
expect(result.value.limitations).toContain(caveat);
|
||||
expect(result.value.limitations).toContain(captureCaveat);
|
||||
expect(result.value.limitations.filter((item) => item === caveat)).toEqual([
|
||||
caveat,
|
||||
]);
|
||||
});
|
||||
|
||||
it("compares media types case-insensitively and counts templated groups across pages", () => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.records = capture.records.slice(0, 3);
|
||||
const reported = [
|
||||
["https://api.example.test/a", "Application/JSON"],
|
||||
["https://api.example.test/a", " \tapplication/json\t ; charset=utf-8"],
|
||||
["https://api.example.test/users/7", "application/json"],
|
||||
];
|
||||
capture.records.forEach((record, index) => {
|
||||
const [url, mimeType] = reported[index] ?? [];
|
||||
record.reported = {
|
||||
request: { method: "GET", url: url ?? null },
|
||||
response: { status: 200, content: { mimeType: mimeType ?? null } },
|
||||
};
|
||||
});
|
||||
const result = project(capture, 0, 1);
|
||||
if (!result.ok || result.value.kind !== "api-surface")
|
||||
throw new Error("Expected surface");
|
||||
expect(result.value.groups[0]).toMatchObject({
|
||||
path_template: "/a",
|
||||
content_types: ["application/json"],
|
||||
flags: [],
|
||||
});
|
||||
expect(result.value.templated_groups).toBe(1);
|
||||
});
|
||||
|
|
@ -0,0 +1,140 @@
|
|||
import { z } from "zod";
|
||||
import { AnalysisUnsupportedTargetError } from "../analysisErrorCore.js";
|
||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import { webNetworkCaptureSchema } from "../webNetworkCapture.js";
|
||||
import {
|
||||
analysisViewInputError,
|
||||
type AnalysisViewParent,
|
||||
type AnalysisViewRequest,
|
||||
type UnsignedAnalysisView,
|
||||
} from "./analysisView.js";
|
||||
import { WebNetworkApiSurfaceAccumulator } from "./WebNetworkApiSurfaceAccumulator.js";
|
||||
|
||||
export const apiSurfaceViewSchema = z.strictObject({
|
||||
kind: z.literal("api-surface"),
|
||||
offset: z.number().int().nonnegative().default(0),
|
||||
limit: z.number().int().positive().default(50),
|
||||
});
|
||||
const ruleSchema = z.enum(["numeric", "uuid", "hex", "iso-date"]);
|
||||
const groupSchema = z.strictObject({
|
||||
method: z.string(),
|
||||
origin: z.string(),
|
||||
path_template: z.string(),
|
||||
templated_segments: z.array(
|
||||
z.strictObject({
|
||||
segment_index: z.number().int().nonnegative(),
|
||||
rule: ruleSchema,
|
||||
basis: z.literal("inferred"),
|
||||
observed_values: z.array(z.string()),
|
||||
}),
|
||||
),
|
||||
record_ordinals: z.array(z.number().int().nonnegative()),
|
||||
statuses: z.array(z.number()),
|
||||
content_types: z.array(z.string()),
|
||||
flags: z.array(
|
||||
z.enum(["single-sample", "mixed-statuses", "mixed-content-types"]),
|
||||
),
|
||||
});
|
||||
export const apiSurfaceFields = {
|
||||
records: z.strictObject({
|
||||
examined: z.number().int().nonnegative(),
|
||||
total: z.number().int().nonnegative(),
|
||||
excluded: z.number().int().nonnegative(),
|
||||
}),
|
||||
templated_groups: z
|
||||
.number()
|
||||
.int()
|
||||
.nonnegative()
|
||||
.describe(
|
||||
"Groups across every page that carry at least one templated segment.",
|
||||
),
|
||||
groups: z.array(groupSchema),
|
||||
excluded_records: z.array(
|
||||
z.strictObject({
|
||||
ordinal: z.number().int().nonnegative(),
|
||||
reason: z.enum([
|
||||
"url-redacted",
|
||||
"url-unparseable",
|
||||
"request-unavailable",
|
||||
"method-unavailable",
|
||||
]),
|
||||
}),
|
||||
),
|
||||
};
|
||||
/** Group retained HAR request lines; format matches do not establish handler identity. */
|
||||
export const projectWebNetworkApiSurfaceView = (
|
||||
parent: AnalysisViewParent,
|
||||
view: AnalysisViewRequest,
|
||||
accumulator = new WebNetworkApiSurfaceAccumulator(),
|
||||
): Result<UnsignedAnalysisView, AnalysisError> => {
|
||||
const unsupported = (reason: string) =>
|
||||
err(
|
||||
new AnalysisUnsupportedTargetError(
|
||||
"inspect_analysis_view",
|
||||
parent.evidenceId,
|
||||
reason,
|
||||
),
|
||||
);
|
||||
if (
|
||||
parent.operation !== "inspect_web_network_capture" ||
|
||||
view.kind !== "api-surface"
|
||||
)
|
||||
return unsupported(
|
||||
`View ${view.kind} is unsupported for parent operation ${parent.operation}; api-surface requires inspect_web_network_capture.`,
|
||||
);
|
||||
const admitted = accumulator.admitCaptureClone(parent.normalizedResult);
|
||||
if (!admitted.ok) return admitted;
|
||||
const parsed = webNetworkCaptureSchema.safeParse(parent.normalizedResult);
|
||||
if (!parsed.success)
|
||||
return err(
|
||||
analysisViewInputError([
|
||||
{
|
||||
path: ["source"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
"Parent Evidence result does not match inspect_web_network_capture.",
|
||||
},
|
||||
]),
|
||||
);
|
||||
const capture = parsed.data;
|
||||
if (capture.format !== "har")
|
||||
return unsupported(
|
||||
`api-surface does not support capture format ${capture.format}.`,
|
||||
);
|
||||
const declarations = z
|
||||
.object({ sensitive_value_count: z.number().int().nonnegative() })
|
||||
.safeParse(parent.parameters);
|
||||
if (!declarations.success || declarations.data.sensitive_value_count > 0)
|
||||
return unsupported("sensitive_values_unretained");
|
||||
const limitations = accumulator.addLimitations([
|
||||
...parent.limitations,
|
||||
...capture.limitations,
|
||||
"Grouping does not establish handler identity, URL-space schemas or causality.",
|
||||
...(capture.records.length < capture.total_records
|
||||
? [
|
||||
"Parent contains an ordinal subset; the surface covers only retained records.",
|
||||
]
|
||||
: []),
|
||||
]);
|
||||
if (!limitations.ok) return limitations;
|
||||
for (const record of capture.records) {
|
||||
const added = accumulator.add(record);
|
||||
if (!added.ok) return added;
|
||||
}
|
||||
const page = accumulator.page(view.offset, view.limit);
|
||||
return ok({
|
||||
kind: "api-surface",
|
||||
view,
|
||||
parent_evidence_id: parent.evidenceId,
|
||||
parent_operation: "inspect_web_network_capture",
|
||||
parent_digest: parent.evidenceId.slice(3),
|
||||
artifact: { path: capture.artifact.path, sha256: capture.artifact.sha256 },
|
||||
records: {
|
||||
examined: capture.records.length,
|
||||
total: capture.total_records,
|
||||
excluded: page.excluded_records.length,
|
||||
},
|
||||
...page,
|
||||
});
|
||||
};
|
||||
|
|
@ -520,3 +520,33 @@ it("does not trust helper names or follow returns after a definite throw", () =>
|
|||
expect(result.electron.call_paths ?? []).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
describe("wrapped Electron API identity", () => {
|
||||
it("does not restore rejected Electron identities through wrapper substitution", () => {
|
||||
for (const source of [
|
||||
`const e=require("electron");e.ipcMain={on(){}};e.ipcMain.on("bad",event=>event.sender.copy());`,
|
||||
`function require(){return {ipcMain:{on(){}}}};const e=require("electron");e.ipcMain.on("bad",event=>event.sender.copy());`,
|
||||
`const e=require("electron");const ipc=e.ipcRenderer,alias=ipc;delete alias.send;function send(channel){ipc.send(channel)}send("bad");`,
|
||||
`require("electron/renderer").ipcRenderer={};const e=require("electron");function send(channel){e.ipcRenderer.send(channel)}send("bad");`,
|
||||
`const e=require("electron");[e.ipcRenderer]=replacements;function send(channel){e.ipcRenderer.send(channel)}send("bad");`,
|
||||
`const e=require("electron");for(e.contextBridge of replacements){}function expose(api){e.contextBridge.exposeInMainWorld("api",api)}expose({copy:()=>navigator.clipboard.writeText("bad")});`,
|
||||
`const e=require("electron");const ipc=e.ipcRenderer;Reflect.deleteProperty(ipc,"send");function send(channel){ipc.send(channel)}send("bad");`,
|
||||
]) {
|
||||
const { electron } = analyzeJavaScriptStaticSource(source);
|
||||
expect(electron.call_paths ?? []).toEqual([]);
|
||||
expect(electron.ipc).toEqual([]);
|
||||
expect(electron.context_bridge_apis).toEqual([]);
|
||||
}
|
||||
});
|
||||
it("keeps unaffected Electron methods available after a sibling method write", () => {
|
||||
const { electron } = analyzeJavaScriptStaticSource(`
|
||||
const e=require("electron");
|
||||
delete e.ipcRenderer.send;
|
||||
function invoke(channel){e.ipcRenderer.invoke(channel)}
|
||||
invoke("good");
|
||||
`);
|
||||
expect(electron.ipc).toContainEqual(
|
||||
expect.objectContaining({ channel: "good", operation: "invoke" }),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ import {
|
|||
type SymbolicFunction,
|
||||
} from "./javascriptSymbolicValues.js";
|
||||
import type { JavaScriptSemanticAnalysisState } from "./javascriptSemanticState.js";
|
||||
import type { JavaScriptElectronIdentity } from "./javascriptSemanticAnalysis.js";
|
||||
import type { JavaScriptAnalysisAccumulator } from "./javascriptStaticAnalysisState.js";
|
||||
import type {
|
||||
ElectronCallPath,
|
||||
|
|
@ -48,11 +49,12 @@ export const collectElectronWrappedCallPaths = (
|
|||
state: JavaScriptSemanticAnalysisState,
|
||||
source: string,
|
||||
accumulator: JavaScriptAnalysisAccumulator,
|
||||
identity: JavaScriptElectronIdentity,
|
||||
): readonly ElectronCallPath[] => {
|
||||
// Bounds repeated interprocedural expansion independently of bundle bytes.
|
||||
// Exhaustion is reported as unknown coverage, never as a negative finding.
|
||||
let remainingVisits = 1_000_000;
|
||||
const resolver = new JavaScriptSymbolicBindings(state, program);
|
||||
const resolver = new JavaScriptSymbolicBindings(state, program, identity);
|
||||
const paths: ElectronCallPath[] = [];
|
||||
const seen = new Set<string>();
|
||||
const step = (
|
||||
|
|
@ -390,7 +392,7 @@ export const collectElectronWrappedCallPaths = (
|
|||
} else if (callee.kind === "external") {
|
||||
const path = callee.path;
|
||||
const endpoint = path.join(".");
|
||||
if (/^electron(?:\/(?:main|renderer|common))?$/u.test(path[0] ?? "")) {
|
||||
if (identity.intactExport(path)) {
|
||||
const descriptor = ipcDescriptor(path.slice(1));
|
||||
if (descriptor)
|
||||
ipc(
|
||||
|
|
|
|||
|
|
@ -378,9 +378,24 @@ export const classifyParsedJavaScriptElectronBindings = (
|
|||
export function* classifyParsedJavaScriptElectronBindingsSteps(
|
||||
file: ParsedJavaScriptSource,
|
||||
): Generator<void, ReadonlyMap<number, readonly string[]>> {
|
||||
const facts = new Map<number, readonly string[]>();
|
||||
const state = createState(file.program);
|
||||
yield* collectDefinitionsSteps(file.program, state);
|
||||
return (yield* indexJavaScriptElectronIdentitySteps(file.program, state))
|
||||
.bindings;
|
||||
}
|
||||
|
||||
/** Shared lexical identity and visible-write proof for direct and wrapped calls. */
|
||||
export interface JavaScriptElectronIdentity {
|
||||
readonly bindings: ReadonlyMap<number, readonly string[]>;
|
||||
readonly intactExport: (path: readonly (string | null)[]) => boolean;
|
||||
readonly exportPath: (root: t.Node) => readonly string[] | undefined;
|
||||
}
|
||||
|
||||
export function* indexJavaScriptElectronIdentitySteps(
|
||||
program: t.Program,
|
||||
state: JavaScriptSemanticAnalysisState,
|
||||
): Generator<void, JavaScriptElectronIdentity> {
|
||||
const facts = new Map<number, readonly string[]>();
|
||||
const interop = createSemanticModuleInterop(state);
|
||||
const writes = new SemanticPropertyPathCoverage();
|
||||
let collectedWrites = false;
|
||||
|
|
@ -411,7 +426,7 @@ export function* classifyParsedJavaScriptElectronBindingsSteps(
|
|||
: undefined;
|
||||
};
|
||||
yield* collectJavaScriptIdentityWritesSteps(
|
||||
file.program,
|
||||
program,
|
||||
(root, kind) => {
|
||||
if (kind === "opaque-call") return interop.opaqueMutationPath(root);
|
||||
const exported = kind === "binding" ? undefined : rootExport(root);
|
||||
|
|
@ -431,7 +446,10 @@ export function* classifyParsedJavaScriptElectronBindingsSteps(
|
|||
},
|
||||
);
|
||||
collectedWrites = true;
|
||||
yield* traverseJavaScriptAstSteps(file.program, {
|
||||
const intactExport = (path: readonly (string | null)[]): boolean =>
|
||||
ELECTRON_MODULE.test(path[0] ?? "") &&
|
||||
!writes.covers(["electron", ...path.slice(1)]);
|
||||
yield* traverseJavaScriptAstSteps(program, {
|
||||
enter: (node) => {
|
||||
if (
|
||||
!t.isCallExpression(node) &&
|
||||
|
|
@ -443,12 +461,12 @@ export function* classifyParsedJavaScriptElectronBindingsSteps(
|
|||
const exported = rootExport(root);
|
||||
if (
|
||||
exported !== undefined &&
|
||||
!writes.covers(["electron", ...exported, ...members])
|
||||
intactExport(["electron", ...exported, ...members])
|
||||
)
|
||||
facts.set(root.start ?? -1, exported);
|
||||
},
|
||||
});
|
||||
return facts;
|
||||
return { bindings: facts, intactExport, exportPath: rootExport };
|
||||
}
|
||||
|
||||
// Read import and require origins directly; general value evaluation is
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ import {
|
|||
import type { JavaScriptStaticAnalysis } from "./javascriptStaticAnalysisTypes.js";
|
||||
import type { JavaScriptSemanticAnalysisState } from "./javascriptSemanticState.js";
|
||||
import {
|
||||
classifyParsedJavaScriptElectronBindingsSteps,
|
||||
indexJavaScriptElectronIdentitySteps,
|
||||
indexJavaScriptBindingsSteps,
|
||||
classifyParsedJavaScriptRpcClientRootsSteps,
|
||||
classifyParsedJavaScriptOpenReceivers,
|
||||
|
|
@ -84,17 +84,19 @@ export function* analyzeParsedJavaScriptStaticSourceSteps(
|
|||
openReceiverFacts?: ReadonlyMap<number, JavaScriptOpenReceiverFact>,
|
||||
): Generator<void, JavaScriptStaticAnalysis> {
|
||||
const accumulator = createJavaScriptAnalysisAccumulator(source.length);
|
||||
const electronBindings =
|
||||
yield* classifyParsedJavaScriptElectronBindingsSteps(file);
|
||||
const clientRoots = yield* classifyParsedJavaScriptRpcClientRootsSteps(file);
|
||||
const bindings = yield* indexJavaScriptBindingsSteps(file.program);
|
||||
const electronIdentity = yield* indexJavaScriptElectronIdentitySteps(
|
||||
file.program,
|
||||
bindings,
|
||||
);
|
||||
const clientRoots = yield* classifyParsedJavaScriptRpcClientRootsSteps(file);
|
||||
yield* traverseStaticSourceSteps(
|
||||
file,
|
||||
{
|
||||
source,
|
||||
accumulator,
|
||||
...(openReceiverFacts === undefined ? {} : { openReceiverFacts }),
|
||||
electronBindings,
|
||||
electronBindings: electronIdentity.bindings,
|
||||
rpcScan: createJavaScriptRpcScan(source, clientRoots),
|
||||
},
|
||||
bindings,
|
||||
|
|
@ -104,6 +106,7 @@ export function* analyzeParsedJavaScriptStaticSourceSteps(
|
|||
bindings,
|
||||
source,
|
||||
accumulator,
|
||||
electronIdentity,
|
||||
);
|
||||
addSourceMapDirectives(source, file.comments ?? [], accumulator);
|
||||
return finalizeStaticAnalysis(source, file, accumulator);
|
||||
|
|
|
|||
|
|
@ -4,15 +4,11 @@ import {
|
|||
unwrapJavaScriptExpression,
|
||||
} from "./javascriptAstValues.js";
|
||||
import { javascriptBootstrapAt } from "./javascriptBootstrap.js";
|
||||
import { ELECTRON_MODULE } from "./javascriptElectronMemberWrites.js";
|
||||
import {
|
||||
collectJavaScriptIdentityWritesSteps,
|
||||
ELECTRON_MODULE,
|
||||
electronMemberPath,
|
||||
electronExportPath,
|
||||
} from "./javascriptElectronMemberWrites.js";
|
||||
import { createSemanticModuleInterop } from "./javascriptModuleInterop.js";
|
||||
import { SemanticPropertyPathCoverage } from "./javascriptSemanticPropertyPaths.js";
|
||||
import { semanticRequireOrigin } from "./javascriptSemanticRequireOrigin.js";
|
||||
indexJavaScriptElectronIdentitySteps,
|
||||
type JavaScriptElectronIdentity,
|
||||
} from "./javascriptSemanticAnalysis.js";
|
||||
import { symbolicLogicalRight } from "./javascriptSymbolicControlFlow.js";
|
||||
import {
|
||||
moduleFactory,
|
||||
|
|
@ -71,7 +67,7 @@ export class JavaScriptSymbolicBindings {
|
|||
ReadonlyMap<string, t.ObjectMethod | t.ObjectProperty> | null
|
||||
>();
|
||||
readonly #mutated = new Set<string>();
|
||||
readonly #electronWrites = new SemanticPropertyPathCoverage();
|
||||
readonly #electronIdentity: JavaScriptElectronIdentity;
|
||||
readonly #namespaceOwners = new Map<string, t.Function>();
|
||||
readonly #unsafeExports = new Set<string>();
|
||||
readonly #aliasesByBinding = new Map<string, string[]>();
|
||||
|
|
@ -87,8 +83,11 @@ export class JavaScriptSymbolicBindings {
|
|||
constructor(
|
||||
readonly state: JavaScriptSemanticAnalysisState,
|
||||
program: t.Program,
|
||||
electronIdentity = completeSemanticSteps(
|
||||
indexJavaScriptElectronIdentitySteps(program, state),
|
||||
),
|
||||
) {
|
||||
this.#indexElectronWrites(program);
|
||||
this.#electronIdentity = electronIdentity;
|
||||
this.#indexBindingAliases();
|
||||
traverseJavaScriptAst(program, {
|
||||
enter: (node) => {
|
||||
|
|
@ -202,8 +201,8 @@ export class JavaScriptSymbolicBindings {
|
|||
if (value.kind === "external") {
|
||||
const path = [...value.path, key];
|
||||
if (
|
||||
path[0] === "electron" &&
|
||||
this.#electronWrites.covers(path.slice(1))
|
||||
ELECTRON_MODULE.test(path[0] ?? "") &&
|
||||
!this.#electronIdentity.intactExport(path)
|
||||
) {
|
||||
return symbolicUnknown("overwritten Electron API member");
|
||||
}
|
||||
|
|
@ -499,13 +498,16 @@ export class JavaScriptSymbolicBindings {
|
|||
: symbolicUnknown("unbound identifier");
|
||||
if (binding.definitions.some((d) => d.kind === "assignment"))
|
||||
return symbolicUnknown("reassigned binding");
|
||||
if (this.#mutated.has(binding.bindingId))
|
||||
// Electron member identity is checked at the selected property. A write
|
||||
// to one method must not erase the module origin or unaffected siblings.
|
||||
const electronOrigin = this.#electronIdentity.exportPath(node);
|
||||
if (electronOrigin === undefined && this.#mutated.has(binding.bindingId))
|
||||
return symbolicUnknown("dynamic member writes");
|
||||
for (let e: SymbolicEnvironment | undefined = env; e; e = e.parent) {
|
||||
const value = e.values.get(binding.bindingId);
|
||||
if (value) return value;
|
||||
}
|
||||
if (this.#writes.has(binding.bindingId))
|
||||
if (electronOrigin === undefined && this.#writes.has(binding.bindingId))
|
||||
return symbolicUnknown("overwritten object members");
|
||||
const initializer =
|
||||
binding.initializers.length === 1 ? binding.initializers[0] : undefined;
|
||||
|
|
@ -898,108 +900,6 @@ export class JavaScriptSymbolicBindings {
|
|||
}
|
||||
}
|
||||
|
||||
#indexElectronWrites(program: t.Program): void {
|
||||
const interop = createSemanticModuleInterop(this.state);
|
||||
const pathsByBinding = new Map<string, readonly string[] | null>();
|
||||
const retainPath = (
|
||||
bindingId: string,
|
||||
path: readonly string[] | undefined,
|
||||
): readonly string[] | undefined => {
|
||||
pathsByBinding.set(bindingId, path ?? null);
|
||||
return path;
|
||||
};
|
||||
const bindingPath = (
|
||||
binding: ReturnType<typeof resolveSemanticBindingState>,
|
||||
visited = new Set<string>(),
|
||||
): readonly string[] | undefined => {
|
||||
if (!binding || visited.has(binding.bindingId)) return undefined;
|
||||
const cached = pathsByBinding.get(binding.bindingId);
|
||||
if (cached !== undefined) return cached ?? undefined;
|
||||
visited.add(binding.bindingId);
|
||||
const converted = interop.origin(binding)?.origin;
|
||||
const direct =
|
||||
converted ??
|
||||
(binding.directOrigins.length === 1
|
||||
? binding.directOrigins[0]
|
||||
: undefined);
|
||||
if (direct && ELECTRON_MODULE.test(direct.specifier))
|
||||
return retainPath(binding.bindingId, [
|
||||
"electron",
|
||||
...electronExportPath(direct.importedPath),
|
||||
]);
|
||||
const initializer =
|
||||
binding.initializers.length === 1 ? binding.initializers[0] : undefined;
|
||||
if (
|
||||
!initializer ||
|
||||
!initializer.projection.every(
|
||||
(segment): segment is string => typeof segment === "string",
|
||||
)
|
||||
)
|
||||
return retainPath(binding.bindingId, undefined);
|
||||
const required = semanticRequireOrigin(
|
||||
initializer.node,
|
||||
this.state,
|
||||
)?.origin;
|
||||
if (required && ELECTRON_MODULE.test(required.specifier))
|
||||
return retainPath(binding.bindingId, [
|
||||
"electron",
|
||||
...electronExportPath([
|
||||
...required.importedPath,
|
||||
...initializer.projection.map(String),
|
||||
]),
|
||||
]);
|
||||
const member = electronMemberPath(initializer.node);
|
||||
if (
|
||||
!t.isIdentifier(member.root) ||
|
||||
!member.members.every(
|
||||
(segment): segment is string => typeof segment === "string",
|
||||
)
|
||||
)
|
||||
return retainPath(binding.bindingId, undefined);
|
||||
const source = resolveSemanticBindingState(
|
||||
this.state,
|
||||
member.root,
|
||||
member.root.name,
|
||||
);
|
||||
const sourcePath = bindingPath(source, visited);
|
||||
return retainPath(
|
||||
binding.bindingId,
|
||||
sourcePath === undefined
|
||||
? undefined
|
||||
: [
|
||||
...sourcePath,
|
||||
...member.members,
|
||||
...initializer.projection.map(String),
|
||||
],
|
||||
);
|
||||
};
|
||||
const rootPath = (root: t.Node): readonly string[] | undefined => {
|
||||
const direct = semanticRequireOrigin(root, this.state)?.origin;
|
||||
if (direct && ELECTRON_MODULE.test(direct.specifier))
|
||||
return ["electron", ...electronExportPath(direct.importedPath)];
|
||||
if (!t.isIdentifier(root)) return undefined;
|
||||
const binding = resolveSemanticBindingState(this.state, root, root.name);
|
||||
return bindingPath(binding);
|
||||
};
|
||||
completeSemanticSteps(
|
||||
collectJavaScriptIdentityWritesSteps(
|
||||
program,
|
||||
(root, kind) => {
|
||||
if (kind === "opaque-call") return interop.opaqueMutationPath(root);
|
||||
return rootPath(root) ?? interop.intrinsicPath(root);
|
||||
},
|
||||
interop.intrinsicInvocation,
|
||||
interop.staticString,
|
||||
interop.isFreshObject,
|
||||
(_node, path) => {
|
||||
if (path[0] === "electron") {
|
||||
this.#electronWrites.retain(path.slice(1));
|
||||
}
|
||||
},
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
#invalidateBootstrapTable(
|
||||
tableAliases: ReadonlySet<string>,
|
||||
modules: Map<string, SymbolicValue>,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,75 @@
|
|||
/** Raw coordinates are independent of WHATWG's normalized origin and pathname. */
|
||||
export interface RecordedTransportUrl {
|
||||
readonly origin: string | null;
|
||||
readonly path: string | null;
|
||||
readonly credentials:
|
||||
| { readonly start: number; readonly end: number }
|
||||
| "ambiguous"
|
||||
| null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Interpret transport URL syntax once for credential exclusion and recorded-path
|
||||
* projection. Never use URL.pathname: it removes observed dot segments.
|
||||
* Noncanonical spellings may have parsed credentials but no safe raw span.
|
||||
*/
|
||||
export const parseRecordedTransportUrl = (
|
||||
value: string,
|
||||
): RecordedTransportUrl => {
|
||||
const prefix = /^[\x00-\x20]*(?:([A-Za-z][A-Za-z0-9+.-]*):)?\/\//u.exec(
|
||||
value,
|
||||
);
|
||||
const absolute = URL.parse(value);
|
||||
// Resolve references only to recognize credentials (including WHATWG's
|
||||
// backslash/control-character spellings), never to supply a projected origin.
|
||||
const parsed = absolute ?? URL.parse(value, "https://rea.invalid");
|
||||
const hasCredentials =
|
||||
parsed !== null && (parsed.username !== "" || parsed.password !== "");
|
||||
const origin =
|
||||
absolute !== null && absolute.origin !== "null" ? absolute.origin : null;
|
||||
if (prefix === null)
|
||||
return {
|
||||
origin,
|
||||
path: null,
|
||||
credentials: hasCredentials ? "ambiguous" : null,
|
||||
};
|
||||
|
||||
const authorityStart = prefix[0].length;
|
||||
const protocol =
|
||||
parsed?.protocol ?? `${prefix[1]?.toLowerCase() ?? "https"}:`;
|
||||
const special = ["ftp:", "file:", "http:", "https:", "ws:", "wss:"].includes(
|
||||
protocol,
|
||||
);
|
||||
let authorityEnd = authorityStart;
|
||||
while (
|
||||
authorityEnd < value.length &&
|
||||
!"/?#".includes(value[authorityEnd]!) &&
|
||||
!(special && value[authorityEnd] === "\\")
|
||||
)
|
||||
authorityEnd += 1;
|
||||
const authority = value.slice(authorityStart, authorityEnd);
|
||||
const stableAuthority = authority.length > 0 && !/[\t\n\r]/u.test(authority);
|
||||
const at = authority.lastIndexOf("@");
|
||||
const credentials =
|
||||
!stableAuthority && hasCredentials
|
||||
? "ambiguous"
|
||||
: at >= 0
|
||||
? { start: authorityStart, end: authorityStart + at + 1 }
|
||||
: hasCredentials
|
||||
? "ambiguous"
|
||||
: null;
|
||||
// Only an explicit, unpadded scheme://authority has a recorded absolute path.
|
||||
// Backslashes are authority/path separators for special schemes in WHATWG.
|
||||
if (
|
||||
origin === null ||
|
||||
!/^[A-Za-z][A-Za-z0-9+.-]*:\/\//u.test(value) ||
|
||||
!stableAuthority ||
|
||||
(special && value[authorityEnd] === "\\")
|
||||
)
|
||||
return { origin, path: null, credentials };
|
||||
if (value[authorityEnd] !== "/") return { origin, path: "/", credentials };
|
||||
let pathEnd = authorityEnd;
|
||||
while (pathEnd < value.length && !"?#".includes(value[pathEnd]!))
|
||||
pathEnd += 1;
|
||||
return { origin, path: value.slice(authorityEnd, pathEnd), credentials };
|
||||
};
|
||||
|
|
@ -97,40 +97,69 @@ export const webNetworkCaptureRecordSchema = z.strictObject({
|
|||
limitations: z.array(z.string()),
|
||||
});
|
||||
|
||||
/** Decoded records retain unique producer ordinals within their declared coverage. */
|
||||
export const webNetworkCapturePayloadSchema = z
|
||||
.strictObject({
|
||||
decoder: z.strictObject({
|
||||
id: z.string().min(1),
|
||||
name: z.string().min(1),
|
||||
version: z.string().min(1),
|
||||
}),
|
||||
container: z.strictObject({
|
||||
reported: jsonValueSchema,
|
||||
numeric_literals: z.array(numberSchema),
|
||||
redactions: z.array(redactionSchema),
|
||||
records_pointer: pointerSchema.nullable(),
|
||||
}),
|
||||
total_records: z.number().int().nonnegative(),
|
||||
records: z.array(webNetworkCaptureRecordSchema),
|
||||
})
|
||||
.superRefine((capture, context) => {
|
||||
if (capture.records.length > capture.total_records) {
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
path: ["records"],
|
||||
message: "Retained record count exceeds the declared capture total.",
|
||||
});
|
||||
return;
|
||||
}
|
||||
const seen = new Set<number>();
|
||||
for (const [index, record] of capture.records.entries()) {
|
||||
if (record.ordinal >= capture.total_records || seen.has(record.ordinal)) {
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
path: ["records", index, "ordinal"],
|
||||
message:
|
||||
"Retained ordinals must be unique and within the declared capture total.",
|
||||
});
|
||||
return;
|
||||
}
|
||||
seen.add(record.ordinal);
|
||||
}
|
||||
});
|
||||
|
||||
/** Historical capture inspection has file identity, producer provenance and explicit unknowns. */
|
||||
export const webNetworkCaptureSchema = z.strictObject({
|
||||
artifact: z.strictObject({
|
||||
path: z
|
||||
.string()
|
||||
.describe(
|
||||
"Selected artifact path; empty when explicitly excluded from Evidence.",
|
||||
),
|
||||
sha256: z
|
||||
.string()
|
||||
.regex(/^[0-9a-f]{64}$/)
|
||||
.nullable()
|
||||
.describe(
|
||||
"Observed artifact SHA-256; null only when explicitly excluded by a sensitive declaration.",
|
||||
),
|
||||
bytes: z.number().int().nonnegative(),
|
||||
}),
|
||||
format: z.enum(["har", "mitmproxy"]),
|
||||
decoder: z.strictObject({
|
||||
id: z.string().min(1),
|
||||
name: z.string().min(1),
|
||||
version: z.string().min(1),
|
||||
}),
|
||||
container: z.strictObject({
|
||||
reported: jsonValueSchema,
|
||||
numeric_literals: z.array(numberSchema),
|
||||
redactions: z.array(redactionSchema),
|
||||
records_pointer: pointerSchema.nullable(),
|
||||
}),
|
||||
total_records: z.number().int().nonnegative(),
|
||||
records: z.array(webNetworkCaptureRecordSchema),
|
||||
runtime_attribution: z.literal("unknown"),
|
||||
limitations: z.array(z.string()),
|
||||
});
|
||||
export const webNetworkCaptureSchema =
|
||||
webNetworkCapturePayloadSchema.safeExtend({
|
||||
artifact: z.strictObject({
|
||||
path: z
|
||||
.string()
|
||||
.describe(
|
||||
"Selected artifact path; empty when explicitly excluded from Evidence.",
|
||||
),
|
||||
sha256: z
|
||||
.string()
|
||||
.regex(/^[0-9a-f]{64}$/)
|
||||
.nullable()
|
||||
.describe(
|
||||
"Observed artifact SHA-256; null only when explicitly excluded by a sensitive declaration.",
|
||||
),
|
||||
bytes: z.number().int().nonnegative(),
|
||||
}),
|
||||
format: z.enum(["har", "mitmproxy"]),
|
||||
runtime_attribution: z.literal("unknown"),
|
||||
limitations: z.array(z.string()),
|
||||
});
|
||||
|
||||
export type InspectWebNetworkCaptureInput = z.output<
|
||||
typeof inspectWebNetworkCaptureInputSchema
|
||||
|
|
|
|||
|
|
@ -1,3 +1,6 @@
|
|||
import { apiSurfaceHar } from "../../fixtures/historicalHar.js";
|
||||
import { parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
|
|
@ -6,12 +9,89 @@ import { expect } from "vitest";
|
|||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
|
||||
import { analysisViewLayoutEvidence } from "../../fixtures/analysisView.js";
|
||||
import {
|
||||
analysisViewLayoutEvidence,
|
||||
analysisViewCaptureFixture,
|
||||
analysisViewCaptureEvidence,
|
||||
analysisViewObjectDenseCapture,
|
||||
} from "../../fixtures/analysisView.js";
|
||||
import { createEvidence } from "../../../src/domain/evidence.js";
|
||||
import { ghidraFunctionDossier } from "../../../src/domain/ghidraValues.fixture.js";
|
||||
import { createEvidenceBundle } from "../../../src/domain/evidenceBundle.js";
|
||||
import { inspectEvidenceBundle } from "../../../src/application/investigation/InspectEvidenceBundle.js";
|
||||
|
||||
cliTest(
|
||||
"admits portable captures before parsing and preserves capture caveats",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-inline-capture-cli-");
|
||||
const path = join(root, "view.json");
|
||||
const environment = {
|
||||
HOME: root,
|
||||
XDG_CONFIG_HOME: root,
|
||||
XDG_CACHE_HOME: root,
|
||||
};
|
||||
await writeFile(
|
||||
path,
|
||||
JSON.stringify({
|
||||
source: {
|
||||
kind: "inline",
|
||||
evidence: analysisViewCaptureEvidence(
|
||||
analysisViewObjectDenseCapture(),
|
||||
),
|
||||
},
|
||||
view: { kind: "api-surface" },
|
||||
}),
|
||||
);
|
||||
const refused = await cli.run({
|
||||
arguments: ["inspect-analysis-view", path, "--json"],
|
||||
environment,
|
||||
});
|
||||
expect(refused.exitCode).toBe(1);
|
||||
expect(refused.json).toMatchObject({
|
||||
code: "resource_constraint",
|
||||
details: { resource: "memory" },
|
||||
});
|
||||
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.records = capture.records.slice(0, 1);
|
||||
capture.total_records = 1;
|
||||
const caveat = "Producer omitted exchanges before capture startup.";
|
||||
capture.limitations = [caveat, caveat];
|
||||
capture.records[0]!.reported = {
|
||||
request: { method: "GET", url: "https://api.test/a" },
|
||||
response: {
|
||||
status: 200,
|
||||
content: { mimeType: "\u000btext/plain\u000b" },
|
||||
},
|
||||
};
|
||||
const parent = analysisViewCaptureEvidence(capture);
|
||||
expect(parent.limitations).not.toContain(caveat);
|
||||
await writeFile(
|
||||
path,
|
||||
JSON.stringify({
|
||||
source: { kind: "inline", evidence: parent },
|
||||
view: { kind: "api-surface" },
|
||||
}),
|
||||
);
|
||||
const response = await cli.run({
|
||||
arguments: ["inspect-analysis-view", path, "--json"],
|
||||
environment,
|
||||
});
|
||||
expect(response.exitCode, response.stdout).toBe(0);
|
||||
const result = parseEvidence(response.json);
|
||||
expect(result.limitations.filter((value) => value === caveat)).toEqual([
|
||||
caveat,
|
||||
]);
|
||||
expect(result.normalized_result).toMatchObject({
|
||||
groups: [{ content_types: [], flags: ["single-sample"] }],
|
||||
limitations: expect.arrayContaining([caveat]),
|
||||
unknowns: [
|
||||
"Some retained records have unavailable status or content type values.",
|
||||
],
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"discovers native records from a complete portable bundle using the shared workflow",
|
||||
async ({ cli }) => {
|
||||
|
|
@ -216,3 +296,80 @@ for (const [view, reason] of [
|
|||
},
|
||||
);
|
||||
}
|
||||
|
||||
cliTest(
|
||||
"decodes HAR credentials and projects recorded API paths through the compiled workflows",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-surface-cli-");
|
||||
const fixture = apiSurfaceHar();
|
||||
const entry = fixture.log.entries[0]!;
|
||||
const urls = [
|
||||
"https://user:password@api.example.test/a/%2e%2e/admin?token=ordinary#fragment",
|
||||
"https://api.example.test/admin",
|
||||
"https:user:password@api.example.test/path",
|
||||
"https:///user:password@api.example.test/path",
|
||||
];
|
||||
fixture.log.entries.push(
|
||||
...urls.map((url) => ({
|
||||
...entry,
|
||||
request: { ...entry.request, url },
|
||||
})),
|
||||
);
|
||||
const bytes = JSON.stringify(fixture);
|
||||
const capturePath = join(root, "capture.har");
|
||||
const input = join(root, "view.json");
|
||||
const environment = {
|
||||
HOME: root,
|
||||
XDG_CONFIG_HOME: root,
|
||||
XDG_CACHE_HOME: root,
|
||||
};
|
||||
await writeFile(capturePath, bytes);
|
||||
const capture = await cli.run({
|
||||
arguments: ["inspect-web-network-capture", capturePath, "har", "--json"],
|
||||
environment,
|
||||
});
|
||||
expect(capture.exitCode, capture.stdout).toBe(0);
|
||||
const parent = parseEvidence(capture.json);
|
||||
expect(parent.subject?.digest.sha256).toBe(
|
||||
createHash("sha256").update(bytes).digest("hex"),
|
||||
);
|
||||
expect(JSON.stringify(parent)).not.toContain("password");
|
||||
await writeFile(
|
||||
input,
|
||||
JSON.stringify({
|
||||
source: { kind: "inline", evidence: parent },
|
||||
view: { kind: "api-surface" },
|
||||
}),
|
||||
);
|
||||
const result = await cli.run({
|
||||
arguments: ["inspect-analysis-view", input, "--json"],
|
||||
environment,
|
||||
});
|
||||
expect(result.exitCode, result.stdout).toBe(0);
|
||||
expect(result.json).toMatchObject({
|
||||
confidence: "inferred",
|
||||
evidence_links: [parent.evidence_id],
|
||||
normalized_result: {
|
||||
kind: "api-surface",
|
||||
records: { examined: 17, total: 17, excluded: 2 },
|
||||
coverage: { total: 14, exhausted: true },
|
||||
groups: expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
path_template: "/a/%2e%2e/admin",
|
||||
record_ordinals: [13],
|
||||
}),
|
||||
expect.objectContaining({
|
||||
path_template: "/admin",
|
||||
record_ordinals: [14],
|
||||
}),
|
||||
]),
|
||||
excluded_records: [
|
||||
{ ordinal: 15, reason: "url-redacted" },
|
||||
{ ordinal: 16, reason: "url-redacted" },
|
||||
],
|
||||
},
|
||||
});
|
||||
expect(JSON.stringify(result.json)).not.toContain("password");
|
||||
expect(JSON.stringify(result.json)).not.toContain("ordinary");
|
||||
},
|
||||
);
|
||||
|
|
|
|||
|
|
@ -1,3 +1,11 @@
|
|||
import { API_SURFACE_EXAMPLE } from "../../../src/contracts/analysisViewApiSurfaceExample.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { WebNetworkCaptureService } from "../../../src/application/WebNetworkCaptureService.js";
|
||||
import { decodeHarCapture } from "../../../src/browser/history/HarCapture.js";
|
||||
import { join } from "node:path";
|
||||
import { apiSurfaceHar } from "../../fixtures/historicalHar.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
|
||||
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
|
||||
import { STDIO_DEFAULT_MAX_BUFFER_SIZE } from "@modelcontextprotocol/server";
|
||||
|
|
@ -25,17 +33,26 @@ import {
|
|||
analysisViewBindJavaScriptGraphs,
|
||||
analysisViewLayoutEvidence,
|
||||
analysisViewLayoutFixture,
|
||||
analysisViewCaptureFixture,
|
||||
analysisViewCaptureEvidence,
|
||||
analysisViewObjectDenseCapture,
|
||||
} from "../../fixtures/analysisView.js";
|
||||
import { createTestBinarySession } from "../../fixtures/binarySession.js";
|
||||
import { BINARY_LAYOUT_TEST_PROVIDER } from "../../fixtures/binaryDiagnostics/layout.js";
|
||||
|
||||
const connect = async (binaryLayout?: BinaryLayoutService) => {
|
||||
const connect = async (
|
||||
binaryLayout?: BinaryLayoutService,
|
||||
webNetworkCapture?: WebNetworkCaptureService,
|
||||
) => {
|
||||
const session = createTestBinarySession(() => {
|
||||
throw new Error("selected views must not start a deep provider");
|
||||
});
|
||||
const server = createServer(
|
||||
{ kind: "session", session },
|
||||
binaryLayout === undefined ? {} : { binaryLayout },
|
||||
{
|
||||
...(binaryLayout === undefined ? {} : { binaryLayout }),
|
||||
...(webNetworkCapture === undefined ? {} : { webNetworkCapture }),
|
||||
},
|
||||
);
|
||||
const client = new Client({ name: "analysis-view-mcp", version: "1" });
|
||||
onTestFinished(async () => {
|
||||
|
|
@ -176,6 +193,18 @@ it("advertises exact schemas and projects one layout section from retained Evide
|
|||
const outputSchema: Record<string, unknown> = advertised.outputSchema;
|
||||
expect(ajv.validateSchema(inputSchema)).toBe(true);
|
||||
expect(ajv.validateSchema(outputSchema)).toBe(true);
|
||||
expect(
|
||||
ajv.validate(inputSchema, {
|
||||
source: { kind: "inline", evidence: analysisViewLayoutEvidence() },
|
||||
view: { kind: "summary" },
|
||||
}),
|
||||
).toBe(true);
|
||||
expect(
|
||||
ajv.validate(inputSchema, {
|
||||
source: { kind: "inline", evidence: {} },
|
||||
view: { kind: "summary" },
|
||||
}),
|
||||
).toBe(false);
|
||||
expect(
|
||||
ajv.validate(inputSchema, {
|
||||
source: {
|
||||
|
|
@ -254,6 +283,100 @@ it("reports a tampered inline Evidence ID at its full MCP source path", async ()
|
|||
});
|
||||
});
|
||||
|
||||
it("admits portable captures before SDK Evidence parsing and preserves capture caveats", async () => {
|
||||
const { client } = await connect();
|
||||
const dense = analysisViewCaptureEvidence(analysisViewObjectDenseCapture());
|
||||
const refused = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: {
|
||||
source: { kind: "inline", evidence: dense },
|
||||
view: { kind: "api-surface" },
|
||||
},
|
||||
});
|
||||
expect(parseMcpToolError(refused).error).toMatchObject({
|
||||
code: "resource_constraint",
|
||||
details: { resource: "memory" },
|
||||
});
|
||||
await client.ping();
|
||||
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.records = capture.records.slice(0, 1);
|
||||
capture.total_records = 1;
|
||||
const caveat = "Producer omitted exchanges before capture startup.";
|
||||
capture.limitations = [caveat, caveat];
|
||||
capture.records[0]!.reported = {
|
||||
request: { method: "GET", url: "https://api.test/a" },
|
||||
response: { status: 200, content: { mimeType: "\u00a0text/plain\u00a0" } },
|
||||
};
|
||||
const parent = analysisViewCaptureEvidence(capture);
|
||||
expect(parent.limitations).not.toContain(caveat);
|
||||
const response = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: {
|
||||
source: { kind: "inline", evidence: parent },
|
||||
view: { kind: "api-surface" },
|
||||
},
|
||||
});
|
||||
const result = parseEvidence(response.structuredContent);
|
||||
expect(result.limitations.filter((value) => value === caveat)).toEqual([
|
||||
caveat,
|
||||
]);
|
||||
expect(result.normalized_result).toMatchObject({
|
||||
groups: [{ content_types: [], flags: ["single-sample"] }],
|
||||
limitations: expect.arrayContaining([caveat]),
|
||||
unknowns: [
|
||||
"Some retained records have unavailable status or content type values.",
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects inconsistent portable coverage and accepts caller-ordered ordinal subsets", async () => {
|
||||
const { client } = await connect();
|
||||
for (const [total, ordinals] of [
|
||||
[0, [0]],
|
||||
[2, [0, 0]],
|
||||
[2, [2]],
|
||||
] as const) {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.total_records = total;
|
||||
capture.records = ordinals.map((ordinal) => ({
|
||||
...capture.records[0]!,
|
||||
ordinal,
|
||||
}));
|
||||
const response = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: {
|
||||
source: {
|
||||
kind: "inline",
|
||||
evidence: analysisViewCaptureEvidence(capture),
|
||||
},
|
||||
view: { kind: "api-surface" },
|
||||
},
|
||||
});
|
||||
expect(parseMcpToolError(response).error.code).toBe("invalid_request");
|
||||
}
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.records = [capture.records[8]!, capture.records[1]!];
|
||||
const response = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: {
|
||||
source: {
|
||||
kind: "inline",
|
||||
evidence: analysisViewCaptureEvidence(capture),
|
||||
},
|
||||
view: { kind: "api-surface" },
|
||||
},
|
||||
});
|
||||
expect(
|
||||
parseEvidence(response.structuredContent).normalized_result,
|
||||
).toMatchObject({
|
||||
records: { examined: 2, total: 15 },
|
||||
limitations: expect.arrayContaining([
|
||||
"Parent contains an ordinal subset; the surface covers only retained records.",
|
||||
]),
|
||||
});
|
||||
});
|
||||
|
||||
it("delivers a bounded native dossier view over MCP without a Ghidra provider or lost connection", async () => {
|
||||
const { client, session } = await connect();
|
||||
const original = functionDossierSchema.parse(ghidraFunctionDossier());
|
||||
|
|
@ -624,3 +747,87 @@ it("accepts a transport-constraint retained reference as the view source", async
|
|||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("projects an actual HAR capture retained by the public MCP workflow", async () => {
|
||||
const root = await createTestTempDirectory("rea-surface-mcp-");
|
||||
const path = join(root, "surface.har");
|
||||
await writeFile(path, JSON.stringify(apiSurfaceHar()));
|
||||
const service = new WebNetworkCaptureService({
|
||||
inspect: async (input) => {
|
||||
const bytes = await readFile(input.capture_path);
|
||||
return ok({
|
||||
...decodeHarCapture(bytes.toString("utf8"), input.sensitive_values),
|
||||
artifact: {
|
||||
path: input.capture_path,
|
||||
sha256: createHash("sha256").update(bytes).digest("hex"),
|
||||
bytes: bytes.length,
|
||||
},
|
||||
format: "har",
|
||||
runtime_attribution: "unknown",
|
||||
limitations: [],
|
||||
});
|
||||
},
|
||||
});
|
||||
const { client } = await connect(undefined, service);
|
||||
const captureResponse = await client.callTool({
|
||||
name: "inspect_web_network_capture",
|
||||
arguments: { capture_path: path, format: "har" },
|
||||
});
|
||||
expect(captureResponse.isError).not.toBe(true);
|
||||
const parent = parseEvidence(captureResponse.structuredContent);
|
||||
const response = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: {
|
||||
source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
|
||||
view: { kind: "api-surface" },
|
||||
},
|
||||
});
|
||||
expect(response.isError).not.toBe(true);
|
||||
const evidence = parseEvidence(response.structuredContent);
|
||||
expect(evidence.confidence).toBe("inferred");
|
||||
expect(evidence.normalized_result).toMatchObject({
|
||||
records: { examined: 13, total: 13, excluded: 0 },
|
||||
coverage: { total: 12, exhausted: true },
|
||||
groups: expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
path_template: "/users/{numeric}",
|
||||
record_ordinals: [0, 1],
|
||||
}),
|
||||
]),
|
||||
});
|
||||
});
|
||||
|
||||
it("reproduces the published API surface example from decoded fixture bytes", async () => {
|
||||
const fixture = apiSurfaceHar();
|
||||
fixture.log.entries = fixture.log.entries.slice(0, 2);
|
||||
const bytes = Buffer.from(JSON.stringify(fixture));
|
||||
const service = new WebNetworkCaptureService({
|
||||
inspect: async () =>
|
||||
ok({
|
||||
...decodeHarCapture(bytes.toString("utf8"), []),
|
||||
artifact: {
|
||||
path: "/fixtures/api-surface.har",
|
||||
sha256: createHash("sha256").update(bytes).digest("hex"),
|
||||
bytes: bytes.length,
|
||||
},
|
||||
format: "har",
|
||||
runtime_attribution: "unknown",
|
||||
limitations: [],
|
||||
}),
|
||||
});
|
||||
const { client } = await connect(undefined, service);
|
||||
const capture = await client.callTool({
|
||||
name: "inspect_web_network_capture",
|
||||
arguments: { capture_path: "/fixtures/api-surface.har", format: "har" },
|
||||
});
|
||||
expect(parseEvidence(capture.structuredContent).evidence_id).toBe(
|
||||
API_SURFACE_EXAMPLE.input.source.evidence_id,
|
||||
);
|
||||
const response = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: API_SURFACE_EXAMPLE.input,
|
||||
});
|
||||
expect(parseEvidence(response.structuredContent).normalized_result).toEqual(
|
||||
API_SURFACE_EXAMPLE.output,
|
||||
);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
import { apiSurfaceHar } from "./historicalHar.js";
|
||||
import { webNetworkCaptureSchema } from "../../src/domain/webNetworkCapture.js";
|
||||
import { BINARY_LAYOUT_TEST_PROVIDER } from "./binaryDiagnostics/layout.js";
|
||||
import { binaryLayoutFixture } from "./binaryDiagnostics/layout.js";
|
||||
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../src/contracts/javascript/javascriptRuntimeReconciliationExample.js";
|
||||
|
|
@ -193,3 +195,75 @@ export const analysisViewJavaScriptEvidence = (
|
|||
limitations: analysis.limitations,
|
||||
},
|
||||
);
|
||||
|
||||
/** Capture parent with explicit producer ordinals, including excluded URL values. */
|
||||
export const analysisViewCaptureFixture = () =>
|
||||
webNetworkCaptureSchema.parse({
|
||||
artifact: {
|
||||
path: "/fixtures/surface.har",
|
||||
sha256: "c".repeat(64),
|
||||
bytes: 1024,
|
||||
},
|
||||
format: "har",
|
||||
decoder: { id: "fixture", name: "fixture", version: "1" },
|
||||
container: {
|
||||
reported: {},
|
||||
numeric_literals: [],
|
||||
redactions: [],
|
||||
records_pointer: "/log/entries",
|
||||
},
|
||||
total_records: 15,
|
||||
records: [
|
||||
...apiSurfaceHar().log.entries,
|
||||
{ request: { method: "GET", url: null } },
|
||||
{ request: { method: "GET", url: "invalid URL" } },
|
||||
].map((reported, ordinal) => ({
|
||||
ordinal,
|
||||
location: { kind: "json-pointer", pointer: `/log/entries/${ordinal}` },
|
||||
reported,
|
||||
binary_fields: [],
|
||||
numeric_literals: [],
|
||||
redactions: [],
|
||||
limitations: [],
|
||||
})),
|
||||
runtime_attribution: "unknown",
|
||||
limitations: [],
|
||||
});
|
||||
|
||||
export const analysisViewCaptureEvidence = (
|
||||
capture = analysisViewCaptureFixture(),
|
||||
): Evidence =>
|
||||
createEvidence(
|
||||
capture.artifact.sha256 === null
|
||||
? undefined
|
||||
: {
|
||||
path: capture.artifact.path,
|
||||
sha256: capture.artifact.sha256,
|
||||
format: "file",
|
||||
},
|
||||
{ id: "fixture", name: "fixture", version: "1" },
|
||||
{
|
||||
operation: "inspect_web_network_capture",
|
||||
parameters: { sensitive_value_count: 0 },
|
||||
result: capture,
|
||||
confidence: "observed",
|
||||
},
|
||||
);
|
||||
|
||||
/** A small encoded capture whose many short objects expand during validation. */
|
||||
export const analysisViewObjectDenseCapture = (headerCount = 100_000) => {
|
||||
const capture = analysisViewCaptureFixture();
|
||||
capture.records = capture.records.slice(0, 1);
|
||||
capture.total_records = 1;
|
||||
capture.records[0]!.reported = {
|
||||
request: {
|
||||
method: "GET",
|
||||
url: "https://api.test/items/1",
|
||||
headers: Array.from({ length: headerCount }, () => ({
|
||||
name: "x",
|
||||
value: "y",
|
||||
})),
|
||||
},
|
||||
};
|
||||
return capture;
|
||||
};
|
||||
|
|
|
|||
|
|
@ -34,3 +34,41 @@ export const historicalHar = () => ({
|
|||
],
|
||||
},
|
||||
});
|
||||
|
||||
/** Request paths exercising only the public API surface format rules. */
|
||||
export const apiSurfaceHar = () => {
|
||||
const fixture = historicalHar();
|
||||
const entry = fixture.log.entries[0]!;
|
||||
const paths = [
|
||||
"/users/8812",
|
||||
"/users/9031",
|
||||
"/ids/550e8400-e29b-41d4-a716-446655440000",
|
||||
"/ids/abcdef0123456789abcdef0123456789",
|
||||
"/dates/2024-02-29",
|
||||
"/api/v1/users",
|
||||
"/api/v2/users",
|
||||
"/api/Report2",
|
||||
"/api/Report3",
|
||||
"/api/CreateDocument",
|
||||
"/api/ListDocuments",
|
||||
"/ids/abc123",
|
||||
"/dates/2023-02-29",
|
||||
];
|
||||
fixture.log.entries = paths.map((path, index) => ({
|
||||
...entry,
|
||||
request: {
|
||||
...entry.request,
|
||||
url: `https://api.example.test${path}?not-in-view=value`,
|
||||
},
|
||||
response: {
|
||||
...entry.response,
|
||||
status: index === 1 ? 404 : 200,
|
||||
content: {
|
||||
...entry.response.content,
|
||||
mimeType:
|
||||
index === 1 ? "text/plain" : "application/json; charset=utf-8",
|
||||
},
|
||||
},
|
||||
}));
|
||||
return fixture;
|
||||
};
|
||||
|
|
|
|||
Loading…
Reference in New Issue