feat(browser): capture correlated network content and lifecycle evidence (#797)
* feat(browser): capture correlated network content and lifecycle evidence Keep request identities and redirect relationships across ordered scenario events. Capture selected browser-exposed bytes and duplicate headers through modular readers, settle bounded reads before cleanup, and preserve unavailable states. Keep content-selection mismatches and legacy network coverage unknown during comparison. Verify real CLI/MCP, installed package, and browser lifecycle paths. Refs #774 Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis) Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing) * fix(browser): align capture defaults and redact encoded media metadata Correct the complete documented request and legacy default expectation caught by CI. Cover and redact encoded declared values in captured body media types. Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis) Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing) * test(browser): verify polling capture cutoff completeness Accept incomplete event coverage only when the continuously polling fixture has explicit unfinished requests; retain failures for other missing or truncated sections. Include actual completeness and unfinished transactions in diagnostics. Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis) Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)
This commit is contained in:
parent
50787b2faf
commit
f9eaf056ef
|
|
@ -5,6 +5,9 @@ on:
|
|||
paths:
|
||||
- ".github/workflows/real-browser.yml"
|
||||
- "scripts/verify-real-browser.mjs"
|
||||
- "scripts/verify-browser-network.mjs"
|
||||
- "scripts/lib/browser-*.mjs"
|
||||
- "scripts/fixtures/browser-network-site.mjs"
|
||||
- "scripts/fixtures/browser-verifier-site.mjs"
|
||||
- "scripts/fixtures/page-cdp-proxy.mjs"
|
||||
- "src/browser/**"
|
||||
|
|
|
|||
|
|
@ -194,6 +194,83 @@ external browser without closing it. The request itself defines the operation;
|
|||
there is no additional REA grant step. The browser and host still enforce their
|
||||
own access and process rules.
|
||||
|
||||
## Network evidence
|
||||
|
||||
`capture.events: ["network"]` retains request, response, completion, failure,
|
||||
and unfinished-request metadata. Select content independently:
|
||||
|
||||
```json
|
||||
{
|
||||
"browser": {
|
||||
"mode": "launch",
|
||||
"executable_path": "/absolute/path/to/chromium"
|
||||
},
|
||||
"start_url": { "url": "http://127.0.0.1:3000" },
|
||||
"actions": [
|
||||
{
|
||||
"step_id": "run",
|
||||
"action": "click",
|
||||
"locator": { "kind": "css", "selector": "#start" },
|
||||
"timeout_ms": 10000
|
||||
},
|
||||
{
|
||||
"step_id": "ready",
|
||||
"action": "wait_for",
|
||||
"locator": { "kind": "css", "selector": "#done" },
|
||||
"state": "visible",
|
||||
"timeout_ms": 10000
|
||||
}
|
||||
],
|
||||
"capture": {
|
||||
"network": {
|
||||
"request_body": true,
|
||||
"response_body": true,
|
||||
"header_values": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
All three options default to `false`. Any selected network content also enables
|
||||
network metadata observation; listing `network` in `events` is then optional.
|
||||
Each observed request object receives a capture-scoped `transaction_id`.
|
||||
Requests sharing a URL remain distinct. Redirects link to the preceding
|
||||
transaction only when it was observed; a null predecessor does not establish
|
||||
that no earlier hop existed. Older captures without IDs remain readable and
|
||||
their transaction association is unknown.
|
||||
|
||||
`scenario.network_content` records the resolved content selection. Comparing
|
||||
captures with different selection, or legacy captures without that coverage
|
||||
information, reports network evidence as unknown with a recovery instruction.
|
||||
Capturing more data alone does not establish a website change.
|
||||
|
||||
`network-content` events reference the exact request or response event through
|
||||
`source_event_sequence`. They include independently typed headers and body
|
||||
states. Captured bodies retain base64 bytes, byte count, media type, and SHA-256
|
||||
of the retained bytes. Request bytes are those exposed by Playwright; a
|
||||
`not_exposed` request body does not prove absence or complete multipart file
|
||||
coverage. Response bytes are decoded by the browser, not the compressed wire
|
||||
representation. No later fetch is substituted for the original response.
|
||||
Headers preserve producer order and duplicates. Authorization,
|
||||
Proxy-Authorization, Cookie, and Set-Cookie values are null with
|
||||
`redacted: true`; ordinary values remain intact. Declared UTF-8 secret values
|
||||
and their standard URI/form or JSON-escaped representations are replaced without
|
||||
decoding unrelated binary bytes. Other encodings are not interpreted. Digests describe the
|
||||
retained representation after redaction.
|
||||
|
||||
Only completed responses are read. At the end of the final step, observation
|
||||
stops; requests still in flight emit `request-unfinished`, and their observed
|
||||
responses have explicitly unavailable bodies. This cutoff does not wait for
|
||||
long-lived streaming connections. Already-started content reads settle before
|
||||
browser cleanup, with a provider-owned five-second timeout per read and
|
||||
cancellation support. Failed or unexposed reads retain per-content states and make selected
|
||||
event coverage incomplete. Body events can arrive during later steps or final
|
||||
drain; event bounds describe receipt during the step, while the source event
|
||||
reference establishes the transaction phase. Comparison attributes content to
|
||||
its source event's step and excludes the sequence reference from normalization.
|
||||
Playwright scenario capture does not supply initiator stacks; passive CDP
|
||||
observation retains its own initiator evidence.
|
||||
|
||||
## Scenario comparison
|
||||
|
||||
`compare_web_captures` and `compare-web-captures` also accept two complete
|
||||
|
|
|
|||
|
|
@ -629,9 +629,9 @@
|
|||
"mcp_prompts": 6
|
||||
},
|
||||
"digests": {
|
||||
"tools_sha256": "eec3abb8fa5b25c6e5d6e361b7ebd157e3599f69b68defda85680dbfe061b3d3",
|
||||
"tools_sha256": "24bd870a291b473a8d22fa93fcb0c0595e534f0d69e4275832c6f37d37ab9068",
|
||||
"prompts_sha256": "ef2de8814390b340623c32001610aa2709d90bed51537b6e562ab3abf144ab4d",
|
||||
"combined_sha256": "ed9c36900e9223561d38ba30a44921f4ff101d5eca989920f9edc9dc91df470b",
|
||||
"combined_sha256": "1e812d0cef0fc9a479a62b873c26023d2c9491837434ec59857fb03f17bda5fc",
|
||||
"providers_sha256": "044634314b36b966bd918f99f116f4684108556ac1dabc912e76ccd351d97e3e"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -85,6 +85,14 @@ regressions until a real fixture establishes equivalent coverage.
|
|||
decoding. Its SDK client explicitly permits the larger inline JSON response;
|
||||
this lane does not establish large image-comparison request transport coverage.
|
||||
|
||||
`verify:browser:network` is a focused real-browser lane for transaction identity,
|
||||
selected request/response bytes, binary and compressed responses, duplicate
|
||||
headers, credential and declared-secret redaction, redirects, streaming cutoff,
|
||||
CLI/MCP parity, and owned-profile cleanup. Set `REA_BROWSER_EXECUTABLE` to an
|
||||
installed Chrome-family browser. An optional script argument selects an already
|
||||
installed package's `scripts/rea.mjs` entry point for packaged-artifact checks.
|
||||
The complete `verify:browser` lane includes these same checks.
|
||||
|
||||
The [test suite audit](test-suite-audit.md) records the pruning decisions,
|
||||
replacement evidence and remaining priorities.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
{
|
||||
"manifest_id": "ecm_19cf4bddaf5311c3cda82a950883bbc028940ddae8a5e68eeb28374b88585095",
|
||||
"manifest_id": "ecm_be55eec2622069e55136c644322ea30c22bb7ebc8d7fc45961b536607d947352",
|
||||
"verifier": {
|
||||
"id": "rea-managed-conformance",
|
||||
"version": "1"
|
||||
|
|
@ -13,7 +13,7 @@
|
|||
"skill_digests": [
|
||||
{
|
||||
"skill_id": "reverse-engineer-anything",
|
||||
"sha256": "57f5af72a8f387cbc5249455492297a087a5fff81a986d47f58a7fc879485aef"
|
||||
"sha256": "f329bad02e83305faa35e4fd0e46a698fe5b00e5d849063890227c4bae79ca11"
|
||||
}
|
||||
],
|
||||
"claims": [
|
||||
|
|
|
|||
|
|
@ -132,6 +132,7 @@
|
|||
"verify:ghidra:cross-format": "npm run build:cached && node scripts/verify-real-ghidra.mjs --cross-format",
|
||||
"verify:ghidra:windows": "npm run build:cached && npm run build:ghidra-windows-fixture && node scripts/verify-real-ghidra-windows.mjs",
|
||||
"verify:browser": "npm run build:cached && node scripts/verify-real-browser.mjs",
|
||||
"verify:browser:network": "npm run build:cached && node scripts/verify-browser-network.mjs",
|
||||
"verify:electron": "npm run build:cached && node scripts/verify-real-electron.mjs",
|
||||
"verify:inspector": "npm run build:cached && node scripts/verify-real-javascript-runtime-observation.mjs",
|
||||
"verify:javascript-runtime": "npm run build:cached && node scripts/verify-local-javascript-runtime.mjs",
|
||||
|
|
|
|||
|
|
@ -0,0 +1,109 @@
|
|||
import { once } from "node:events";
|
||||
import { createServer } from "node:http";
|
||||
import { gzipSync } from "node:zlib";
|
||||
|
||||
export const networkFixtureSecret = "私密-network-fixture";
|
||||
export const networkFixtureBinary = Buffer.from([0xff, 0, 0xfe, 1]);
|
||||
export const networkFixtureDecoded = Buffer.from("decoded-response-marker");
|
||||
|
||||
const html = `<!doctype html><meta charset="utf-8"><body><button id="start">Run</button><main id="done" hidden>Done</main>
|
||||
<script>document.querySelector('#start').onclick = async () => {
|
||||
const send = marker => fetch('/same', {method:'POST', headers:{
|
||||
'Content-Type':'application/json', 'Authorization':'Bearer transport-only', 'X-Marker': marker
|
||||
}, body:JSON.stringify({marker, password:'ordinary-selected-value', declared:'${networkFixtureSecret}'})}).then(r=>r.text());
|
||||
await Promise.all([send('slow'),send('fast'),fetch('/redirect').then(r=>r.text()),
|
||||
fetch('/binary').then(r=>r.arrayBuffer()),fetch('/compressed').then(r=>r.text()),
|
||||
fetch('/empty').then(r=>r.text()),
|
||||
fetch('/form', {method:'POST', body:new URLSearchParams({declared:'${networkFixtureSecret}'})}).then(r=>r.text())]);
|
||||
await fetch('/stream');
|
||||
document.querySelector('#done').hidden=false;
|
||||
};</script>`;
|
||||
|
||||
/** Start an isolated fixture with concurrent, redirected, binary, and streaming responses. */
|
||||
export async function startBrowserNetworkSite() {
|
||||
const counts = new Map();
|
||||
const timers = new Set();
|
||||
const server = createServer(async (request, response) => {
|
||||
const path = new URL(request.url, "http://fixture.test").pathname;
|
||||
counts.set(path, (counts.get(path) ?? 0) + 1);
|
||||
if (path === "/same") {
|
||||
const chunks = [];
|
||||
for await (const chunk of request) chunks.push(chunk);
|
||||
const body = JSON.parse(Buffer.concat(chunks).toString());
|
||||
const timer = setTimeout(
|
||||
() => {
|
||||
timers.delete(timer);
|
||||
response.writeHead(200, [
|
||||
"Content-Type",
|
||||
"application/json",
|
||||
"X-Repeat",
|
||||
"first",
|
||||
"X-Repeat",
|
||||
"second",
|
||||
"Set-Cookie",
|
||||
"fixture_session=transport-cookie; HttpOnly",
|
||||
"X-Declared",
|
||||
encodeURIComponent(networkFixtureSecret),
|
||||
]);
|
||||
response.end(
|
||||
JSON.stringify({
|
||||
marker: body.marker,
|
||||
password: body.password,
|
||||
declared: body.declared,
|
||||
}),
|
||||
);
|
||||
},
|
||||
body.marker === "slow" ? 120 : 5,
|
||||
);
|
||||
timers.add(timer);
|
||||
} else if (path === "/redirect") {
|
||||
response.writeHead(302, { Location: "/landing" });
|
||||
response.end();
|
||||
} else if (path === "/landing") {
|
||||
response.end("redirect-target");
|
||||
} else if (path === "/form") {
|
||||
const chunks = [];
|
||||
for await (const chunk of request) chunks.push(chunk);
|
||||
response.end(
|
||||
new URLSearchParams(Buffer.concat(chunks).toString()).get("declared"),
|
||||
);
|
||||
} else if (path === "/binary") {
|
||||
response.writeHead(200, { "Content-Type": "application/octet-stream" });
|
||||
response.end(networkFixtureBinary);
|
||||
} else if (path === "/compressed") {
|
||||
response.writeHead(200, {
|
||||
"Content-Type": "text/plain",
|
||||
"Content-Encoding": "gzip",
|
||||
});
|
||||
response.end(gzipSync(networkFixtureDecoded));
|
||||
} else if (path === "/empty") {
|
||||
response.writeHead(204);
|
||||
response.end();
|
||||
} else if (path === "/stream") {
|
||||
response.writeHead(200, { "Content-Type": "text/event-stream" });
|
||||
response.write("data: still-open\n\n");
|
||||
} else if (path === "/favicon.ico") {
|
||||
response.writeHead(204);
|
||||
response.end();
|
||||
} else {
|
||||
response.writeHead(200, { "Content-Type": "text/html; charset=utf-8" });
|
||||
response.end(html);
|
||||
}
|
||||
});
|
||||
server.listen(0, "127.0.0.1");
|
||||
await once(server, "listening");
|
||||
const address = server.address();
|
||||
if (typeof address !== "object" || address === null)
|
||||
throw new Error("Network fixture has no TCP address");
|
||||
return {
|
||||
origin: `http://127.0.0.1:${address.port}`,
|
||||
counts,
|
||||
async close() {
|
||||
for (const timer of timers) clearTimeout(timer);
|
||||
server.closeAllConnections();
|
||||
await new Promise((resolve, reject) =>
|
||||
server.close((error) => (error ? reject(error) : resolve())),
|
||||
);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
|
@ -0,0 +1,156 @@
|
|||
import assert from "node:assert/strict";
|
||||
import {
|
||||
networkFixtureBinary,
|
||||
networkFixtureDecoded,
|
||||
networkFixtureSecret,
|
||||
} from "../fixtures/browser-network-site.mjs";
|
||||
|
||||
const retained = (event) => {
|
||||
assert.equal(event?.kind, "network-content");
|
||||
assert.equal(event.body.state, "captured", JSON.stringify(event.body));
|
||||
return Buffer.from(event.body.content, "base64");
|
||||
};
|
||||
|
||||
/** Verify association against server-authored markers rather than provider-derived expectations. */
|
||||
export function assertBrowserNetworkEvidence(
|
||||
capture,
|
||||
origin,
|
||||
expectedCleanup = "terminated-owned-process",
|
||||
) {
|
||||
assert.ok(capture.steps.every((step) => step.status === "completed"));
|
||||
assert.equal(capture.browser.cleanup, expectedCleanup);
|
||||
assert.deepEqual(capture.scenario.network_content, {
|
||||
request_body: true,
|
||||
response_body: true,
|
||||
header_values: true,
|
||||
});
|
||||
assert.equal(capture.completeness.equality_eligible, false);
|
||||
assert.ok(!JSON.stringify(capture).includes(networkFixtureSecret));
|
||||
const events = capture.events.items;
|
||||
const content = events.filter((event) => event.kind === "network-content");
|
||||
const requestEvents = events.filter((event) => event.kind === "request");
|
||||
const sameUrlRequests = requestEvents.filter(
|
||||
(event) => event.url.url === `${origin}/same`,
|
||||
);
|
||||
assert.equal(sameUrlRequests.length, 2);
|
||||
assert.equal(
|
||||
new Set(sameUrlRequests.map((event) => event.transaction_id)).size,
|
||||
2,
|
||||
);
|
||||
const markers = [];
|
||||
for (const request of sameUrlRequests) {
|
||||
const requestContent = content.find(
|
||||
(event) =>
|
||||
event.phase === "request" &&
|
||||
event.transaction_id === request.transaction_id,
|
||||
);
|
||||
const responseContent = content.find(
|
||||
(event) =>
|
||||
event.phase === "response" &&
|
||||
event.transaction_id === request.transaction_id,
|
||||
);
|
||||
const input = JSON.parse(retained(requestContent).toString());
|
||||
const output = JSON.parse(retained(responseContent).toString());
|
||||
assert.equal(
|
||||
output.marker,
|
||||
input.marker,
|
||||
"Response was associated with a different same-URL request",
|
||||
);
|
||||
assert.equal(output.password, "ordinary-selected-value");
|
||||
assert.equal(input.declared, "[REDACTED:network_secret]");
|
||||
assert.equal(output.declared, input.declared);
|
||||
assert.equal(requestContent.body.redacted, true);
|
||||
assert.equal(responseContent.headers.state, "captured");
|
||||
assert.ok(
|
||||
responseContent.headers.items.some(
|
||||
(item) =>
|
||||
item.name.toLowerCase() === "x-declared" &&
|
||||
item.value === "[REDACTED:network_secret]" &&
|
||||
item.redacted,
|
||||
),
|
||||
);
|
||||
assert.deepEqual(
|
||||
responseContent.headers.items
|
||||
.filter((item) => item.name.toLowerCase() === "x-repeat")
|
||||
.map((item) => item.value),
|
||||
["first", "second"],
|
||||
);
|
||||
assert.ok(
|
||||
responseContent.headers.items.some(
|
||||
(item) =>
|
||||
item.name.toLowerCase() === "set-cookie" &&
|
||||
item.value === null &&
|
||||
item.redacted,
|
||||
),
|
||||
);
|
||||
assert.ok(
|
||||
requestContent.headers.items.some(
|
||||
(item) =>
|
||||
item.name.toLowerCase() === "authorization" &&
|
||||
item.value === null &&
|
||||
item.redacted,
|
||||
),
|
||||
);
|
||||
assert.ok(
|
||||
events.some(
|
||||
(event) =>
|
||||
event.kind === "request-finished" &&
|
||||
event.transaction_id === request.transaction_id,
|
||||
),
|
||||
);
|
||||
markers.push(input.marker);
|
||||
}
|
||||
assert.deepEqual(markers.sort(), ["fast", "slow"]);
|
||||
const responseFor = (path) => {
|
||||
const response = events.find(
|
||||
(event) =>
|
||||
event.kind === "response" && event.url.url === `${origin}${path}`,
|
||||
);
|
||||
assert.ok(response, `Missing response ${path}`);
|
||||
return content.find(
|
||||
(event) =>
|
||||
event.phase === "response" &&
|
||||
event.source_event_sequence === response.sequence,
|
||||
);
|
||||
};
|
||||
assert.deepEqual(retained(responseFor("/binary")), networkFixtureBinary);
|
||||
assert.deepEqual(retained(responseFor("/compressed")), networkFixtureDecoded);
|
||||
assert.equal(
|
||||
responseFor("/compressed").body.representation,
|
||||
"browser-decoded-response-bytes",
|
||||
);
|
||||
const formRequest = requestEvents.find(
|
||||
(event) => event.url.url === `${origin}/form`,
|
||||
);
|
||||
assert.ok(formRequest);
|
||||
const formContent = content.find(
|
||||
(event) =>
|
||||
event.phase === "request" &&
|
||||
event.transaction_id === formRequest.transaction_id,
|
||||
);
|
||||
assert.equal(
|
||||
retained(formContent).toString(),
|
||||
"declared=[REDACTED:network_secret]",
|
||||
);
|
||||
assert.equal(
|
||||
retained(responseFor("/form")).toString(),
|
||||
"[REDACTED:network_secret]",
|
||||
);
|
||||
assert.equal(responseFor("/stream").body.state, "unavailable");
|
||||
assert.equal(responseFor("/stream").body.reason, "response-unfinished");
|
||||
const first = requestEvents.find(
|
||||
(event) => event.url.url === `${origin}/redirect`,
|
||||
);
|
||||
const next = requestEvents.find(
|
||||
(event) => event.url.url === `${origin}/landing`,
|
||||
);
|
||||
assert.ok(first && next);
|
||||
assert.equal(next.redirected_from_transaction_id, first.transaction_id);
|
||||
return {
|
||||
same_url_markers: markers,
|
||||
binary_bytes: networkFixtureBinary.length,
|
||||
compressed_representation: "browser-decoded-response-bytes",
|
||||
unfinished_stream: "response-unfinished",
|
||||
empty_response_state: responseFor("/empty").body.state,
|
||||
};
|
||||
}
|
||||
|
|
@ -0,0 +1,192 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { promisify } from "node:util";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
|
||||
import { browserScenarioSchema } from "../../dist/domain/browserScenario.js";
|
||||
import { browserScenarioCaptureSchema } from "../../dist/domain/browserScenarioCapture.js";
|
||||
import { parseEvidence } from "../../dist/domain/evidence.js";
|
||||
import {
|
||||
startBrowserNetworkSite,
|
||||
networkFixtureSecret,
|
||||
} from "../fixtures/browser-network-site.mjs";
|
||||
import { assertBrowserNetworkEvidence } from "./browser-network-assertions.mjs";
|
||||
import { scenarioProfiles } from "./browser-scenario-verifier.mjs";
|
||||
|
||||
/** Exercise selected network content through real CLI and stdio MCP, one browser at a time. */
|
||||
export async function verifyBrowserNetworkEvidence(
|
||||
executable,
|
||||
entrypoint = fileURLToPath(new URL("../rea.mjs", import.meta.url)),
|
||||
attachment,
|
||||
) {
|
||||
const site = await startBrowserNetworkSite();
|
||||
const profilesBefore = await scenarioProfiles();
|
||||
try {
|
||||
const scenario = browserScenarioSchema.parse({
|
||||
browser: { mode: "launch", executable_path: executable },
|
||||
start_url: { url: site.origin },
|
||||
actions: [
|
||||
{
|
||||
step_id: "run",
|
||||
action: "click",
|
||||
locator: { kind: "css", selector: "#start" },
|
||||
timeout_ms: 10_000,
|
||||
},
|
||||
{
|
||||
step_id: "ready",
|
||||
action: "wait_for",
|
||||
locator: { kind: "css", selector: "#done" },
|
||||
state: "visible",
|
||||
timeout_ms: 10_000,
|
||||
},
|
||||
],
|
||||
secrets: [
|
||||
{
|
||||
secret_id: "network_secret",
|
||||
environment_variable: "REA_NETWORK_FIXTURE_SECRET",
|
||||
},
|
||||
],
|
||||
capture: {
|
||||
at_end: ["url"],
|
||||
network: {
|
||||
request_body: true,
|
||||
response_body: true,
|
||||
header_values: true,
|
||||
},
|
||||
},
|
||||
});
|
||||
const env = {
|
||||
...process.env,
|
||||
REA_LOG_LEVEL: "silent",
|
||||
REA_NETWORK_FIXTURE_SECRET: networkFixtureSecret,
|
||||
HOPPER_LAUNCHER_PATH: "/rea-unconfigured-provider/hopper",
|
||||
};
|
||||
const { stdout } = await promisify(execFile)(
|
||||
process.execPath,
|
||||
[
|
||||
entrypoint,
|
||||
"capture-browser-scenario",
|
||||
JSON.stringify(scenario),
|
||||
"--json",
|
||||
],
|
||||
{ env, timeout: 60_000, maxBuffer: 16 * 1024 * 1024 },
|
||||
);
|
||||
const cli = browserScenarioCaptureSchema.parse(
|
||||
parseEvidence(JSON.parse(stdout)).normalized_result,
|
||||
);
|
||||
const cliProof = assertBrowserNetworkEvidence(cli, site.origin);
|
||||
// The oracle must reject a deliberately incorrect same-URL association.
|
||||
const swapped = structuredClone(cli);
|
||||
const responses = swapped.events.items.filter(
|
||||
(event) =>
|
||||
event.kind === "network-content" &&
|
||||
event.phase === "response" &&
|
||||
event.body.state === "captured" &&
|
||||
Buffer.from(event.body.content, "base64")
|
||||
.toString()
|
||||
.includes('"marker":'),
|
||||
);
|
||||
assert.equal(responses.length, 2);
|
||||
[responses[0].transaction_id, responses[1].transaction_id] = [
|
||||
responses[1].transaction_id,
|
||||
responses[0].transaction_id,
|
||||
];
|
||||
assert.throws(
|
||||
() => assertBrowserNetworkEvidence(swapped, site.origin),
|
||||
/associated with a different same-URL request/,
|
||||
);
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [entrypoint, "mcp"],
|
||||
env,
|
||||
stderr: "pipe",
|
||||
});
|
||||
const client = new Client({ name: "network-real-e2e", version: "1" });
|
||||
let mcpProof;
|
||||
try {
|
||||
await client.connect(transport);
|
||||
const result = await client.callTool(
|
||||
{ name: "capture_browser_scenario", arguments: scenario },
|
||||
{ timeout: 60_000 },
|
||||
);
|
||||
assert.notEqual(result.isError, true, JSON.stringify(result));
|
||||
const evidence = parseEvidence({
|
||||
...result.structuredContent?.evidence,
|
||||
normalized_result: result.structuredContent?.result,
|
||||
});
|
||||
mcpProof = assertBrowserNetworkEvidence(
|
||||
browserScenarioCaptureSchema.parse(evidence.normalized_result),
|
||||
site.origin,
|
||||
);
|
||||
if (attachment !== undefined) {
|
||||
const connectedScenario = browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
browser: { mode: "connect", ...attachment },
|
||||
});
|
||||
const connected = await client.callTool(
|
||||
{ name: "capture_browser_scenario", arguments: connectedScenario },
|
||||
{ timeout: 60_000 },
|
||||
);
|
||||
assert.notEqual(connected.isError, true, JSON.stringify(connected));
|
||||
const connectedEvidence = parseEvidence({
|
||||
...connected.structuredContent?.evidence,
|
||||
normalized_result: connected.structuredContent?.result,
|
||||
});
|
||||
assert.deepEqual(
|
||||
assertBrowserNetworkEvidence(
|
||||
browserScenarioCaptureSchema.parse(
|
||||
connectedEvidence.normalized_result,
|
||||
),
|
||||
site.origin,
|
||||
"disconnected-external",
|
||||
),
|
||||
cliProof,
|
||||
);
|
||||
assert.ok(
|
||||
(await fetch(`${attachment.cdp_endpoint}/json/version`)).ok,
|
||||
"Network capture closed the external browser",
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
try {
|
||||
await client.close();
|
||||
} finally {
|
||||
await transport.close();
|
||||
}
|
||||
}
|
||||
assert.deepEqual(mcpProof, cliProof);
|
||||
for (const path of [
|
||||
"/redirect",
|
||||
"/landing",
|
||||
"/binary",
|
||||
"/compressed",
|
||||
"/stream",
|
||||
"/empty",
|
||||
"/form",
|
||||
])
|
||||
assert.equal(
|
||||
site.counts.get(path),
|
||||
attachment === undefined ? 2 : 3,
|
||||
`Unexpected implicit refetch of ${path}`,
|
||||
);
|
||||
assert.equal(site.counts.get("/same"), attachment === undefined ? 4 : 6);
|
||||
const profilesAfter = await scenarioProfiles();
|
||||
assert.ok(
|
||||
[...profilesAfter].every((profile) => profilesBefore.has(profile)),
|
||||
"A scenario profile survived cleanup",
|
||||
);
|
||||
return {
|
||||
...cliProof,
|
||||
cli: true,
|
||||
mcp: true,
|
||||
association_fault_rejected: true,
|
||||
implicit_refetches: 0,
|
||||
profile_cleanup: true,
|
||||
connected_network_verified: attachment !== undefined,
|
||||
};
|
||||
} finally {
|
||||
await site.close();
|
||||
}
|
||||
}
|
||||
|
|
@ -204,11 +204,20 @@ export function assertScenarioCapture(capture) {
|
|||
throw new Error(
|
||||
"Scenario event URL did not report declared-secret redaction",
|
||||
);
|
||||
const unfinished = capture.events.items.filter(
|
||||
({ kind }) => kind === "request-unfinished",
|
||||
);
|
||||
// This fixture polls continuously. A request crossing the capture cutoff
|
||||
// makes later activity unknown even when no response body was selected.
|
||||
const expectedMissing = unfinished.length === 0 ? [] : ["events"];
|
||||
if (
|
||||
capture.events.items.length === 0 ||
|
||||
capture.completeness.equality_eligible !== true
|
||||
capture.completeness.equality_eligible !== (unfinished.length === 0) ||
|
||||
JSON.stringify(capture.completeness.missing_sections) !==
|
||||
JSON.stringify(expectedMissing) ||
|
||||
capture.completeness.truncated_sections.length !== 0
|
||||
)
|
||||
throw new Error(
|
||||
"Scenario launch did not produce complete runtime evidence",
|
||||
`Scenario launch completeness disagrees with its capture cutoff: ${JSON.stringify({ completeness: capture.completeness, unfinished })}`,
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,16 @@
|
|||
#!/usr/bin/env node
|
||||
import { access } from "node:fs/promises";
|
||||
import { verifyBrowserNetworkEvidence } from "./lib/browser-network-e2e.mjs";
|
||||
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
|
||||
|
||||
const executable = process.env.REA_BROWSER_EXECUTABLE;
|
||||
if (executable === undefined || executable === "")
|
||||
throw new Error(
|
||||
"verify:browser:network requires REA_BROWSER_EXECUTABLE pointing to an installed Chrome-family browser",
|
||||
);
|
||||
await access(executable);
|
||||
const run = createVerifierRun();
|
||||
const proof = await verifyBrowserNetworkEvidence(executable, process.argv[2]);
|
||||
process.stdout.write(
|
||||
`${JSON.stringify({ ...proof, verifier_run: await completeVerifierRun(run), verified: true })}\n`,
|
||||
);
|
||||
|
|
@ -35,6 +35,7 @@ import {
|
|||
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
|
||||
import { verifyLargeScreenshotE2e } from "./lib/browser-screenshot-e2e.mjs";
|
||||
import { verifyPopupEventCoverage } from "./lib/browser-popup-e2e.mjs";
|
||||
import { verifyBrowserNetworkEvidence } from "./lib/browser-network-e2e.mjs";
|
||||
|
||||
const REAL_BROWSER_STARTUP_TIMEOUT_MS = 60_000;
|
||||
const SCENARIO_SECRET_VALUE = "rea-browser-verifier-secret";
|
||||
|
|
@ -320,6 +321,11 @@ try {
|
|||
|
||||
const largeScreenshot = await verifyLargeScreenshotE2e(endpoint, site.origin);
|
||||
const popupEvents = await verifyPopupEventCoverage(executable);
|
||||
const networkEvidence = await verifyBrowserNetworkEvidence(
|
||||
executable,
|
||||
undefined,
|
||||
{ cdp_endpoint: endpoint, target_id: target },
|
||||
);
|
||||
process.stdout.write(
|
||||
`${JSON.stringify({
|
||||
verifier_run: await completeVerifierRun(verifierRun),
|
||||
|
|
@ -339,6 +345,7 @@ try {
|
|||
screenshotBytes: screenshot.value.artifact.bytes,
|
||||
largeScreenshot,
|
||||
popupEvents,
|
||||
networkEvidence,
|
||||
browserScenarioCli: true,
|
||||
browserScenarioAttachCleanup: "disconnected-external",
|
||||
browserScenarioLaunchCleanup: "terminated-owned-process",
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ description: Reverse engineer native, managed, Electron/JavaScript, packaged, fi
|
|||
metadata:
|
||||
version: "26"
|
||||
tool_count: 125
|
||||
catalog_digest: "ed9c36900e9223561d38ba30a44921f4ff101d5eca989920f9edc9dc91df470b"
|
||||
catalog_digest: "1e812d0cef0fc9a479a62b873c26023d2c9491837434ec59857fb03f17bda5fc"
|
||||
---
|
||||
|
||||
# REA
|
||||
|
|
|
|||
|
|
@ -93,6 +93,11 @@ describe("browserScenarioSchema", () => {
|
|||
service_workers: "block",
|
||||
});
|
||||
expect(parsed.capture).toEqual({
|
||||
network: {
|
||||
request_body: false,
|
||||
response_body: false,
|
||||
header_values: false,
|
||||
},
|
||||
after_each_step: ["screenshot", "url", "accessibility"],
|
||||
at_end: ["dom", "storage"],
|
||||
events: ["console", "page-errors", "network", "websockets"],
|
||||
|
|
|
|||
|
|
@ -13,6 +13,13 @@ import type { BrowserStorageValueFingerprint } from "../domain/browserScenarioCa
|
|||
|
||||
const REDACTION_PREFIX = "[REDACTED:";
|
||||
|
||||
const encodedSecretValues = (secret: string): ReadonlySet<string> =>
|
||||
new Set([
|
||||
secret,
|
||||
encodeURIComponent(secret),
|
||||
new URLSearchParams([["value", secret]]).toString().slice("value=".length),
|
||||
]);
|
||||
|
||||
/** Resolved secret values kept only for one in-memory scenario session. */
|
||||
export class BrowserScenarioSecrets {
|
||||
private constructor(private readonly values: ReadonlyMap<string, string>) {}
|
||||
|
|
@ -62,6 +69,53 @@ export class BrowserScenarioSecrets {
|
|||
return output;
|
||||
}
|
||||
|
||||
/** Replace declared UTF-8, URI/form, and JSON-escaped secrets without decoding binary data. */
|
||||
redactBytes(value: Buffer): Buffer {
|
||||
const candidates = [...this.values]
|
||||
.flatMap(([id, secret]) => {
|
||||
if (secret === "") return [];
|
||||
return [
|
||||
...new Set([
|
||||
...encodedSecretValues(secret),
|
||||
JSON.stringify(secret).slice(1, -1),
|
||||
]),
|
||||
].map((text) => {
|
||||
const needle = Buffer.from(text);
|
||||
return {
|
||||
id,
|
||||
needle,
|
||||
marker: Buffer.from(`${REDACTION_PREFIX}${id}]`),
|
||||
index: value.indexOf(needle),
|
||||
};
|
||||
});
|
||||
})
|
||||
.sort(
|
||||
(left, right) =>
|
||||
right.needle.length - left.needle.length ||
|
||||
(left.id < right.id ? -1 : left.id > right.id ? 1 : 0),
|
||||
);
|
||||
const parts: Buffer[] = [];
|
||||
let offset = 0;
|
||||
while (true) {
|
||||
let next: (typeof candidates)[number] | undefined;
|
||||
for (const candidate of candidates) {
|
||||
if (candidate.index >= 0 && candidate.index < offset)
|
||||
candidate.index = value.indexOf(candidate.needle, offset);
|
||||
if (
|
||||
candidate.index >= 0 &&
|
||||
(next === undefined || candidate.index < next.index)
|
||||
)
|
||||
next = candidate;
|
||||
}
|
||||
if (next === undefined) break;
|
||||
parts.push(value.subarray(offset, next.index), next.marker);
|
||||
offset = next.index + next.needle.length;
|
||||
}
|
||||
if (parts.length === 0) return value;
|
||||
parts.push(value.subarray(offset));
|
||||
return Buffer.concat(parts);
|
||||
}
|
||||
|
||||
/** Remove only declared secret values from one URL and report that action. */
|
||||
sanitizeUrl(value: string): SanitizedBrowserUrl {
|
||||
let output = value;
|
||||
|
|
@ -73,14 +127,7 @@ export class BrowserScenarioSecrets {
|
|||
for (const [id, secret] of replacements) {
|
||||
if (secret === "") continue;
|
||||
const marker = `${REDACTION_PREFIX}${id}]`;
|
||||
const searchParamsValue = new URLSearchParams([["value", secret]])
|
||||
.toString()
|
||||
.slice("value=".length);
|
||||
for (const candidate of new Set([
|
||||
secret,
|
||||
encodeURIComponent(secret),
|
||||
searchParamsValue,
|
||||
]))
|
||||
for (const candidate of encodedSecretValues(secret))
|
||||
output = output.replaceAll(candidate, marker);
|
||||
}
|
||||
const sanitized = sanitizeBrowserUrl(output);
|
||||
|
|
|
|||
|
|
@ -289,6 +289,7 @@ const runScenario = async (
|
|||
start_origin: new URL(scenario.start_url.url).origin,
|
||||
action_count: scenario.actions.length,
|
||||
secret_references: scenario.secrets.map(({ secret_id: id }) => id).sort(),
|
||||
network_content: scenario.capture.network,
|
||||
},
|
||||
duration_ms: Date.now() - startedAt,
|
||||
steps,
|
||||
|
|
@ -296,7 +297,10 @@ const runScenario = async (
|
|||
completeness,
|
||||
limitations: [
|
||||
"Event sequence records provider receipt order; simultaneous browser causality is not inferred.",
|
||||
"Response bodies are not retained.",
|
||||
"Network content is retained only when selected; response bytes are browser-decoded, not compressed wire bytes.",
|
||||
"Request bytes are limited to what Playwright exposes; not_exposed does not establish body absence or multipart file coverage.",
|
||||
"Network content reads settle within 5 seconds each and never wait for unfinished responses or refetch them.",
|
||||
"Playwright scenario events do not expose request initiator stacks; receipt order does not prove causality.",
|
||||
"Storage values are hashed only after declared-secret redaction.",
|
||||
...(session.eventLimitations?.() ?? []),
|
||||
...(session.mode === "connect"
|
||||
|
|
|
|||
|
|
@ -16,6 +16,8 @@ import {
|
|||
type BrowserScenarioEvent,
|
||||
} from "../domain/browserScenarioCapture.js";
|
||||
import type { BrowserScenarioSecrets } from "./BrowserScenarioSecrets.js";
|
||||
import type { BrowserNetworkContentSelection } from "../domain/browserNetworkEvidence.js";
|
||||
import { PlaywrightScenarioNetwork } from "./network/PlaywrightScenarioNetwork.js";
|
||||
|
||||
type EventName =
|
||||
| "console"
|
||||
|
|
@ -40,6 +42,8 @@ interface EventCaptureOptions {
|
|||
readonly ownsContext?: boolean;
|
||||
readonly enabled: ReadonlySet<EventName>;
|
||||
readonly secrets: BrowserScenarioSecrets;
|
||||
readonly network?: BrowserNetworkContentSelection;
|
||||
readonly signal?: AbortSignal;
|
||||
}
|
||||
|
||||
/** Arrival-ordered Playwright event capture with current-step attribution. */
|
||||
|
|
@ -52,9 +56,19 @@ export class PlaywrightScenarioEvents {
|
|||
private readonly ownsContext: boolean;
|
||||
private readonly pages = new Set<Page>();
|
||||
private readonly incompleteFamilies = new Set<EventName>();
|
||||
private readonly network: PlaywrightScenarioNetwork | undefined;
|
||||
private stopNetwork: (() => void) | undefined;
|
||||
private sealed = false;
|
||||
|
||||
constructor(options: EventCaptureOptions) {
|
||||
this.enabled = options.enabled;
|
||||
const network = options.network ?? {
|
||||
request_body: false,
|
||||
response_body: false,
|
||||
header_values: false,
|
||||
};
|
||||
this.enabled = new Set(options.enabled);
|
||||
if (network.request_body || network.response_body || network.header_values)
|
||||
this.enabled = new Set([...this.enabled, "network"]);
|
||||
this.secrets = options.secrets;
|
||||
this.ownsContext = options.ownsContext === true;
|
||||
this.pages.add(options.page);
|
||||
|
|
@ -64,6 +78,13 @@ export class PlaywrightScenarioEvents {
|
|||
);
|
||||
if (this.enabled.has("network")) {
|
||||
const context = options.context ?? options.page.context();
|
||||
const collector = new PlaywrightScenarioNetwork({
|
||||
selection: network,
|
||||
secrets: options.secrets,
|
||||
emit: (event) => this.push(event),
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
});
|
||||
this.network = collector;
|
||||
const inScope = (request: Request): boolean => {
|
||||
if (this.ownsContext) return true;
|
||||
try {
|
||||
|
|
@ -75,15 +96,28 @@ export class PlaywrightScenarioEvents {
|
|||
return false;
|
||||
}
|
||||
};
|
||||
context.on("request", (request) => {
|
||||
if (inScope(request)) this.request("request", request);
|
||||
});
|
||||
context.on("response", (response) => {
|
||||
if (inScope(response.request())) this.response(response);
|
||||
});
|
||||
context.on("requestfailed", (request) => {
|
||||
if (inScope(request)) this.request("request-failed", request);
|
||||
});
|
||||
const onRequest = (request: Request) => {
|
||||
if (inScope(request)) collector.request(request);
|
||||
};
|
||||
const onResponse = (response: Response) => {
|
||||
if (inScope(response.request())) collector.response(response);
|
||||
};
|
||||
const onFailed = (request: Request) => {
|
||||
if (inScope(request)) collector.failed(request);
|
||||
};
|
||||
const onFinished = (request: Request) => {
|
||||
if (inScope(request)) collector.finished(request);
|
||||
};
|
||||
context.on("request", onRequest);
|
||||
context.on("response", onResponse);
|
||||
context.on("requestfailed", onFailed);
|
||||
context.on("requestfinished", onFinished);
|
||||
this.stopNetwork = () => {
|
||||
context.off("request", onRequest);
|
||||
context.off("response", onResponse);
|
||||
context.off("requestfailed", onFailed);
|
||||
context.off("requestfinished", onFinished);
|
||||
};
|
||||
}
|
||||
this.observePage(options.page);
|
||||
}
|
||||
|
|
@ -100,18 +134,30 @@ export class PlaywrightScenarioEvents {
|
|||
return this.sequence;
|
||||
}
|
||||
|
||||
/** Drain selected network content and freeze the result before browser teardown. */
|
||||
async finish(): Promise<void> {
|
||||
this.stopNetwork?.();
|
||||
this.stopNetwork = undefined;
|
||||
try {
|
||||
await this.network?.finish();
|
||||
} finally {
|
||||
this.sealed = true;
|
||||
}
|
||||
}
|
||||
|
||||
/** Report selected popup event families with unrecoverable pre-discovery gaps. */
|
||||
limitations(): readonly string[] {
|
||||
const popupGaps = [...this.incompleteFamilies].map(
|
||||
(family) =>
|
||||
`Popup ${family} events before Page discovery are unavailable; later events are retained.`,
|
||||
);
|
||||
const gaps = [...popupGaps, ...(this.network?.limitations() ?? [])];
|
||||
return this.enabled.has("network") && !this.ownsContext
|
||||
? [
|
||||
"Shared-context network capture excludes requests without a known root or descendant Page frame.",
|
||||
...popupGaps,
|
||||
...gaps,
|
||||
]
|
||||
: popupGaps;
|
||||
: gaps;
|
||||
}
|
||||
|
||||
result(): {
|
||||
|
|
@ -126,7 +172,8 @@ export class PlaywrightScenarioEvents {
|
|||
};
|
||||
}
|
||||
|
||||
private push(event: UnindexedEvent): void {
|
||||
private push(event: UnindexedEvent): number {
|
||||
if (this.sealed) return this.sequence;
|
||||
this.sequence += 1;
|
||||
const parsed = browserScenarioEventSchema.parse({
|
||||
...event,
|
||||
|
|
@ -134,6 +181,7 @@ export class PlaywrightScenarioEvents {
|
|||
step_index: this.stepIndex,
|
||||
});
|
||||
this.items.push(parsed);
|
||||
return this.sequence;
|
||||
}
|
||||
|
||||
private observePage(page: Page): void {
|
||||
|
|
@ -180,47 +228,6 @@ export class PlaywrightScenarioEvents {
|
|||
});
|
||||
}
|
||||
|
||||
private request(kind: "request" | "request-failed", request: Request): void {
|
||||
const observation = {
|
||||
method: this.secrets.redact(request.method()),
|
||||
url: this.safeUrl(request.url()),
|
||||
resource_type: this.secrets.redact(request.resourceType()),
|
||||
header_names: Object.keys(request.headers())
|
||||
.map((name) => this.secrets.redact(name))
|
||||
.sort(),
|
||||
};
|
||||
if (kind === "request-failed") {
|
||||
this.push({
|
||||
...observation,
|
||||
kind,
|
||||
status: null,
|
||||
failure: this.secrets.redact(request.failure()?.errorText ?? "unknown"),
|
||||
});
|
||||
return;
|
||||
}
|
||||
this.push({
|
||||
...observation,
|
||||
kind,
|
||||
status: null,
|
||||
failure: null,
|
||||
});
|
||||
}
|
||||
|
||||
private response(response: Response): void {
|
||||
const request = response.request();
|
||||
this.push({
|
||||
kind: "response",
|
||||
method: this.secrets.redact(request.method()),
|
||||
url: this.safeUrl(response.url()),
|
||||
resource_type: this.secrets.redact(request.resourceType()),
|
||||
status: response.status(),
|
||||
header_names: Object.keys(response.headers())
|
||||
.map((name) => this.secrets.redact(name))
|
||||
.sort(),
|
||||
failure: null,
|
||||
});
|
||||
}
|
||||
|
||||
private webSocket(socket: WebSocket): void {
|
||||
const url = this.safeUrl(socket.url());
|
||||
this.push({ kind: "websocket-opened", url });
|
||||
|
|
|
|||
|
|
@ -170,6 +170,8 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
|||
ownsContext: scenario.browser.mode === "launch",
|
||||
enabled: new Set(scenario.capture.events),
|
||||
secrets,
|
||||
network: scenario.capture.network,
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
});
|
||||
const session = new PlaywrightScenarioSession(opened, {
|
||||
mode: scenario.browser.mode,
|
||||
|
|
@ -263,7 +265,11 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
|||
? ("terminated-owned-process" as const)
|
||||
: ("disconnected-external" as const);
|
||||
this.closed = true;
|
||||
await closePlaywrightScenarioBrowser(this.opened, this.signal);
|
||||
try {
|
||||
await this.eventCapture.finish();
|
||||
} finally {
|
||||
await closePlaywrightScenarioBrowser(this.opened, this.signal);
|
||||
}
|
||||
return this.mode === "launch"
|
||||
? ("terminated-owned-process" as const)
|
||||
: ("disconnected-external" as const);
|
||||
|
|
|
|||
|
|
@ -0,0 +1,120 @@
|
|||
import { createHash } from "node:crypto";
|
||||
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisTimeoutError,
|
||||
} from "../../domain/analysisErrorCore.js";
|
||||
import type {
|
||||
BrowserNetworkBody,
|
||||
BrowserNetworkHeaders,
|
||||
} from "../../domain/browserNetworkEvidence.js";
|
||||
import type { BrowserScenarioSecrets } from "../BrowserScenarioSecrets.js";
|
||||
import { withPlaywrightExecutionBoundary } from "../PlaywrightExecutionBoundary.js";
|
||||
|
||||
/** The producer seam for ordered, duplicate-preserving HTTP headers. */
|
||||
export interface NetworkHeadersSource {
|
||||
headers(): Record<string, string>;
|
||||
headersArray(): Promise<{ name: string; value: string }[]>;
|
||||
}
|
||||
|
||||
/** A bounded read cannot delay browser cleanup indefinitely. */
|
||||
export interface NetworkReadOptions {
|
||||
readonly timeoutMs: number;
|
||||
readonly signal?: AbortSignal;
|
||||
}
|
||||
|
||||
const credentialHeaders = new Set([
|
||||
"authorization",
|
||||
"proxy-authorization",
|
||||
"cookie",
|
||||
"set-cookie",
|
||||
]);
|
||||
|
||||
const errorMessage = (
|
||||
cause: unknown,
|
||||
secrets: BrowserScenarioSecrets,
|
||||
): string =>
|
||||
secrets.redact(
|
||||
cause instanceof Error && cause.message !== ""
|
||||
? cause.message
|
||||
: "Network read failed",
|
||||
);
|
||||
|
||||
/** Read actual producer headers without flattening duplicates or exposing credentials. */
|
||||
export const captureNetworkHeaders = async (
|
||||
source: NetworkHeadersSource,
|
||||
secrets: BrowserScenarioSecrets,
|
||||
options: NetworkReadOptions,
|
||||
): Promise<BrowserNetworkHeaders> => {
|
||||
try {
|
||||
const headers = await withPlaywrightExecutionBoundary(
|
||||
() => source.headersArray(),
|
||||
options.timeoutMs,
|
||||
options.signal,
|
||||
);
|
||||
return {
|
||||
state: "captured",
|
||||
items: headers.map(({ name, value }) => {
|
||||
const safeName = secrets.redact(name);
|
||||
const credential = credentialHeaders.has(name.toLowerCase());
|
||||
const safeValue = credential
|
||||
? null
|
||||
: secrets.redactBytes(Buffer.from(value)).toString("utf8");
|
||||
return {
|
||||
name: safeName,
|
||||
value: safeValue,
|
||||
redacted: credential || safeName !== name || safeValue !== value,
|
||||
};
|
||||
}),
|
||||
};
|
||||
} catch (cause: unknown) {
|
||||
return { state: "unavailable", message: errorMessage(cause, secrets) };
|
||||
}
|
||||
};
|
||||
|
||||
/** Retain only bytes exposed by the browser; no fetch or wire-byte reconstruction occurs. */
|
||||
export const captureNetworkBody = async (
|
||||
read: () => Promise<Buffer | null>,
|
||||
phase: "request" | "response",
|
||||
source: NetworkHeadersSource,
|
||||
secrets: BrowserScenarioSecrets,
|
||||
options: NetworkReadOptions,
|
||||
): Promise<BrowserNetworkBody> => {
|
||||
try {
|
||||
const observed = await withPlaywrightExecutionBoundary(
|
||||
read,
|
||||
options.timeoutMs,
|
||||
options.signal,
|
||||
);
|
||||
if (observed === null) return { state: "not_exposed" };
|
||||
const retained = secrets.redactBytes(observed);
|
||||
const contentType = source.headers()["content-type"];
|
||||
return {
|
||||
state: "captured",
|
||||
representation:
|
||||
phase === "request"
|
||||
? "browser-exposed-request-bytes"
|
||||
: "browser-decoded-response-bytes",
|
||||
encoding: "base64",
|
||||
content: retained.toString("base64"),
|
||||
bytes: retained.length,
|
||||
sha256: createHash("sha256").update(retained).digest("hex"),
|
||||
media_type:
|
||||
contentType === undefined
|
||||
? null
|
||||
: secrets.redactBytes(Buffer.from(contentType)).toString("utf8"),
|
||||
redacted: !retained.equals(observed),
|
||||
};
|
||||
} catch (cause: unknown) {
|
||||
return {
|
||||
state: "unavailable",
|
||||
reason:
|
||||
cause instanceof AnalysisCancelledError
|
||||
? "cancelled"
|
||||
: cause instanceof AnalysisTimeoutError
|
||||
? "body-read-timeout"
|
||||
: "body-read-failed",
|
||||
message: errorMessage(cause, secrets),
|
||||
};
|
||||
}
|
||||
};
|
||||
|
|
@ -0,0 +1,532 @@
|
|||
import { afterEach, expect, it, vi } from "vitest";
|
||||
|
||||
import { browserScenarioSchema } from "../../domain/browserScenario.js";
|
||||
import {
|
||||
browserScenarioEventSchema,
|
||||
type BrowserScenarioEvent,
|
||||
} from "../../domain/browserScenarioCapture.js";
|
||||
import type { BrowserNetworkContentSelection } from "../../domain/browserNetworkEvidence.js";
|
||||
import { BrowserScenarioSecrets } from "../BrowserScenarioSecrets.js";
|
||||
import {
|
||||
PlaywrightScenarioNetwork,
|
||||
type ScenarioNetworkRequest,
|
||||
type ScenarioNetworkResponse,
|
||||
} from "./PlaywrightScenarioNetwork.js";
|
||||
|
||||
class RequestFixture implements ScenarioNetworkRequest {
|
||||
headerReads = 0;
|
||||
bodyReads = 0;
|
||||
predecessor: ScenarioNetworkRequest | null = null;
|
||||
constructor(readonly marker: string) {}
|
||||
method() {
|
||||
return "POST";
|
||||
}
|
||||
url() {
|
||||
return "https://example.test/same-url";
|
||||
}
|
||||
resourceType() {
|
||||
return "fetch";
|
||||
}
|
||||
headers() {
|
||||
return {
|
||||
"content-type": "text/plain",
|
||||
authorization: "Bearer private-transport",
|
||||
};
|
||||
}
|
||||
async headersArray() {
|
||||
this.headerReads += 1;
|
||||
return Object.entries(this.headers()).map(([name, value]) => ({
|
||||
name,
|
||||
value,
|
||||
}));
|
||||
}
|
||||
postDataBuffer(): Buffer | null {
|
||||
this.bodyReads += 1;
|
||||
return Buffer.from(this.marker);
|
||||
}
|
||||
redirectedFrom() {
|
||||
return this.predecessor;
|
||||
}
|
||||
failure() {
|
||||
return { errorText: "connection reset" };
|
||||
}
|
||||
}
|
||||
|
||||
class ResponseFixture implements ScenarioNetworkResponse {
|
||||
bodyReads = 0;
|
||||
headerReads = 0;
|
||||
read: () => Promise<Buffer>;
|
||||
constructor(
|
||||
private readonly source: ScenarioNetworkRequest,
|
||||
marker: string,
|
||||
) {
|
||||
this.read = async () => Buffer.from(marker);
|
||||
}
|
||||
request() {
|
||||
return this.source;
|
||||
}
|
||||
url() {
|
||||
return this.source.url();
|
||||
}
|
||||
status() {
|
||||
return 200;
|
||||
}
|
||||
headers() {
|
||||
return {
|
||||
"content-type": "application/octet-stream",
|
||||
"set-cookie": "session=transport-secret",
|
||||
};
|
||||
}
|
||||
async headersArray() {
|
||||
this.headerReads += 1;
|
||||
return [
|
||||
{ name: "X-Repeat", value: "first" },
|
||||
{ name: "X-Repeat", value: "second" },
|
||||
{ name: "Set-Cookie", value: "session=transport-secret" },
|
||||
];
|
||||
}
|
||||
body() {
|
||||
this.bodyReads += 1;
|
||||
return this.read();
|
||||
}
|
||||
}
|
||||
|
||||
const selection = {
|
||||
request_body: true,
|
||||
response_body: true,
|
||||
header_values: true,
|
||||
};
|
||||
const harness = (
|
||||
selected: BrowserNetworkContentSelection = selection,
|
||||
signal?: AbortSignal,
|
||||
secretValue = "私密",
|
||||
) => {
|
||||
const scenario = browserScenarioSchema.parse({
|
||||
browser: { mode: "launch", executable_path: "/fixture/chrome" },
|
||||
start_url: { url: "https://example.test" },
|
||||
actions: [{ step_id: "done", action: "wait_for_timeout", duration_ms: 1 }],
|
||||
secrets: [
|
||||
{ secret_id: "declared", environment_variable: "FIXTURE_SECRET" },
|
||||
],
|
||||
});
|
||||
const secrets = BrowserScenarioSecrets.resolve(scenario, {
|
||||
FIXTURE_SECRET: secretValue,
|
||||
});
|
||||
if (secrets === undefined) throw new Error("Fixture secret was not resolved");
|
||||
const events: BrowserScenarioEvent[] = [];
|
||||
const collector = new PlaywrightScenarioNetwork({
|
||||
selection: selected,
|
||||
secrets,
|
||||
readTimeoutMs: 50,
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
emit: (event) => {
|
||||
const sequence = events.length + 1;
|
||||
events.push(
|
||||
browserScenarioEventSchema.parse({ ...event, sequence, step_index: 0 }),
|
||||
);
|
||||
return sequence;
|
||||
},
|
||||
});
|
||||
return { collector, events };
|
||||
};
|
||||
|
||||
const bodyText = (event: BrowserScenarioEvent) => {
|
||||
if (event.kind !== "network-content" || event.body.state !== "captured")
|
||||
throw new Error("Expected retained network bytes");
|
||||
return Buffer.from(event.body.content, "base64").toString();
|
||||
};
|
||||
|
||||
afterEach(() => vi.useRealTimers());
|
||||
|
||||
it("redacts declared secrets in browser-produced form and JSON encodings", async () => {
|
||||
for (const secret of ["私密", 'space " and \n']) {
|
||||
const { collector, events } = harness(selection, undefined, secret);
|
||||
const encoded = new RequestFixture(
|
||||
`value=${new URLSearchParams([["value", secret]]).toString().slice(6)}`,
|
||||
);
|
||||
const json = new RequestFixture(JSON.stringify({ value: secret }));
|
||||
collector.request(encoded);
|
||||
collector.request(json);
|
||||
await collector.finish();
|
||||
const content = events.filter(
|
||||
(event) => event.kind === "network-content" && event.phase === "request",
|
||||
);
|
||||
expect(content.map(bodyText)).toEqual([
|
||||
"value=[REDACTED:declared]",
|
||||
'{"value":"[REDACTED:declared]"}',
|
||||
]);
|
||||
}
|
||||
});
|
||||
|
||||
it("associates out-of-order same-URL responses by request identity", async () => {
|
||||
const { collector, events } = harness();
|
||||
const slow = new RequestFixture("slow");
|
||||
const fast = new RequestFixture("fast");
|
||||
collector.request(slow);
|
||||
collector.request(fast);
|
||||
collector.response(new ResponseFixture(fast, "reply:fast"));
|
||||
collector.response(new ResponseFixture(slow, "reply:slow"));
|
||||
collector.finished(fast);
|
||||
collector.finished(slow);
|
||||
await collector.finish();
|
||||
const content = events.filter((event) => event.kind === "network-content");
|
||||
for (const marker of ["slow", "fast"]) {
|
||||
const request = content.find(
|
||||
(event) => event.phase === "request" && bodyText(event) === marker,
|
||||
);
|
||||
const response = content.find(
|
||||
(event) =>
|
||||
event.phase === "response" && bodyText(event) === `reply:${marker}`,
|
||||
);
|
||||
expect(request?.transaction_id).toBe(response?.transaction_id);
|
||||
expect(request?.source_event_sequence).not.toBe(
|
||||
response?.source_event_sequence,
|
||||
);
|
||||
}
|
||||
expect(new Set(content.map((event) => event.transaction_id)).size).toBe(2);
|
||||
});
|
||||
|
||||
it("keeps metadata-only capture free of body and header-value reads", async () => {
|
||||
const { collector, events } = harness({
|
||||
request_body: false,
|
||||
response_body: false,
|
||||
header_values: false,
|
||||
});
|
||||
const request = new RequestFixture("body");
|
||||
const response = new ResponseFixture(request, "reply");
|
||||
collector.request(request);
|
||||
collector.response(response);
|
||||
collector.finished(request);
|
||||
await collector.finish();
|
||||
expect(
|
||||
request.bodyReads +
|
||||
request.headerReads +
|
||||
response.bodyReads +
|
||||
response.headerReads,
|
||||
).toBe(0);
|
||||
expect(events.map(({ kind }) => kind)).toEqual([
|
||||
"request",
|
||||
"response",
|
||||
"request-finished",
|
||||
]);
|
||||
expect(JSON.stringify(events)).not.toContain("private-transport");
|
||||
});
|
||||
|
||||
it("reports unfinished metadata even when content was not selected", async () => {
|
||||
const { collector, events } = harness({
|
||||
request_body: false,
|
||||
response_body: false,
|
||||
header_values: false,
|
||||
});
|
||||
const request = new RequestFixture("body");
|
||||
collector.request(request);
|
||||
await collector.finish();
|
||||
expect(events.map(({ kind }) => kind)).toEqual([
|
||||
"request",
|
||||
"request-unfinished",
|
||||
]);
|
||||
expect(collector.limitations()).toHaveLength(1);
|
||||
expect(request.bodyReads + request.headerReads).toBe(0);
|
||||
});
|
||||
|
||||
it.each([
|
||||
{
|
||||
selection: {
|
||||
request_body: true,
|
||||
response_body: false,
|
||||
header_values: false,
|
||||
},
|
||||
reads: [1, 0, 0, 0],
|
||||
phases: ["request"],
|
||||
},
|
||||
{
|
||||
selection: {
|
||||
request_body: false,
|
||||
response_body: true,
|
||||
header_values: false,
|
||||
},
|
||||
reads: [0, 0, 1, 0],
|
||||
phases: ["response"],
|
||||
},
|
||||
{
|
||||
selection: {
|
||||
request_body: false,
|
||||
response_body: false,
|
||||
header_values: true,
|
||||
},
|
||||
reads: [0, 1, 0, 1],
|
||||
phases: ["request", "response"],
|
||||
},
|
||||
])(
|
||||
"reads only the selected content $selection",
|
||||
async ({ selection: selected, reads, phases }) => {
|
||||
const { collector, events } = harness(selected);
|
||||
const request = new RequestFixture("input");
|
||||
const response = new ResponseFixture(request, "output");
|
||||
collector.request(request);
|
||||
collector.response(response);
|
||||
collector.finished(request);
|
||||
await collector.finish();
|
||||
expect([
|
||||
request.bodyReads,
|
||||
request.headerReads,
|
||||
response.bodyReads,
|
||||
response.headerReads,
|
||||
]).toEqual(reads);
|
||||
expect(
|
||||
events
|
||||
.filter((event) => event.kind === "network-content")
|
||||
.map((event) => event.phase),
|
||||
).toEqual(phases);
|
||||
},
|
||||
);
|
||||
|
||||
it("preserves duplicate headers and binary bytes, redacting credentials and declared UTF-8 secrets", async () => {
|
||||
const { collector, events } = harness();
|
||||
const request = new RequestFixture("input");
|
||||
const response = new ResponseFixture(request, "unused");
|
||||
response.read = async () =>
|
||||
Buffer.concat([
|
||||
Buffer.from([0xff, 0x00]),
|
||||
Buffer.from("私密"),
|
||||
Buffer.from([0xfe]),
|
||||
]);
|
||||
collector.request(request);
|
||||
collector.response(response);
|
||||
expect(response.bodyReads).toBe(0);
|
||||
collector.finished(request);
|
||||
await collector.finish();
|
||||
const content = events.find(
|
||||
(event) => event.kind === "network-content" && event.phase === "response",
|
||||
);
|
||||
if (content?.kind !== "network-content" || content.body.state !== "captured")
|
||||
throw new Error("Missing response");
|
||||
expect(Buffer.from(content.body.content, "base64")).toEqual(
|
||||
Buffer.concat([
|
||||
Buffer.from([0xff, 0x00]),
|
||||
Buffer.from("[REDACTED:declared]"),
|
||||
Buffer.from([0xfe]),
|
||||
]),
|
||||
);
|
||||
expect(content.body.redacted).toBe(true);
|
||||
expect(content.headers).toEqual({
|
||||
state: "captured",
|
||||
items: [
|
||||
{ name: "X-Repeat", value: "first", redacted: false },
|
||||
{ name: "X-Repeat", value: "second", redacted: false },
|
||||
{ name: "Set-Cookie", value: null, redacted: true },
|
||||
],
|
||||
});
|
||||
expect(JSON.stringify(events)).not.toContain("transport-secret");
|
||||
});
|
||||
|
||||
it("does not read unfinished streaming responses and reports the cutoff", async () => {
|
||||
const { collector, events } = harness();
|
||||
const request = new RequestFixture("stream");
|
||||
const response = new ResponseFixture(request, "never-complete");
|
||||
collector.request(request);
|
||||
collector.response(response);
|
||||
await collector.finish();
|
||||
expect(response.bodyReads).toBe(0);
|
||||
expect(events).toContainEqual(
|
||||
expect.objectContaining({
|
||||
kind: "request-unfinished",
|
||||
reason: "capture-ended",
|
||||
}),
|
||||
);
|
||||
expect(events).toContainEqual(
|
||||
expect.objectContaining({
|
||||
kind: "network-content",
|
||||
phase: "response",
|
||||
body: expect.objectContaining({
|
||||
state: "unavailable",
|
||||
reason: "response-unfinished",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
expect(collector.limitations().length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it("bounds hung completed-body reads and rejects late mutation after finalization", async () => {
|
||||
vi.useFakeTimers();
|
||||
const { collector, events } = harness();
|
||||
const request = new RequestFixture("hung");
|
||||
const response = new ResponseFixture(request, "unused");
|
||||
let release: (bytes: Buffer) => void = () => {
|
||||
throw new Error("Read was not started");
|
||||
};
|
||||
response.read = () =>
|
||||
new Promise((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
collector.request(request);
|
||||
collector.response(response);
|
||||
collector.finished(request);
|
||||
const finishing = collector.finish();
|
||||
await vi.advanceTimersByTimeAsync(51);
|
||||
await finishing;
|
||||
const frozen = JSON.stringify(events);
|
||||
expect(events).toContainEqual(
|
||||
expect.objectContaining({
|
||||
kind: "network-content",
|
||||
phase: "response",
|
||||
body: expect.objectContaining({
|
||||
state: "unavailable",
|
||||
reason: "body-read-timeout",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
release(Buffer.from("late"));
|
||||
collector.request(new RequestFixture("after-cutoff"));
|
||||
await Promise.resolve();
|
||||
expect(JSON.stringify(events)).toBe(frozen);
|
||||
});
|
||||
|
||||
it("cancellation settles pending reads and preserves the cancelled reason", async () => {
|
||||
const controller = new AbortController();
|
||||
const { collector, events } = harness(selection, controller.signal);
|
||||
const request = new RequestFixture("cancel");
|
||||
const response = new ResponseFixture(request, "unused");
|
||||
response.read = () => new Promise(() => undefined);
|
||||
collector.request(request);
|
||||
collector.response(response);
|
||||
collector.finished(request);
|
||||
controller.abort();
|
||||
await collector.finish();
|
||||
expect(events).toContainEqual(
|
||||
expect.objectContaining({
|
||||
kind: "network-content",
|
||||
phase: "response",
|
||||
body: expect.objectContaining({
|
||||
state: "unavailable",
|
||||
reason: "cancelled",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("records failures without attempting to read the failed body", async () => {
|
||||
const { collector, events } = harness();
|
||||
const request = new RequestFixture("failed");
|
||||
const response = new ResponseFixture(request, "partial");
|
||||
collector.request(request);
|
||||
collector.response(response);
|
||||
collector.failed(request);
|
||||
await collector.finish();
|
||||
expect(response.bodyReads).toBe(0);
|
||||
expect(events).toContainEqual(
|
||||
expect.objectContaining({
|
||||
kind: "request-failed",
|
||||
failure: "connection reset",
|
||||
}),
|
||||
);
|
||||
expect(events).toContainEqual(
|
||||
expect.objectContaining({
|
||||
kind: "network-content",
|
||||
phase: "response",
|
||||
body: expect.objectContaining({ reason: "request-failed" }),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("links observed redirect hops and leaves unseen predecessors unknown", async () => {
|
||||
const { collector, events } = harness();
|
||||
const first = new RequestFixture("first");
|
||||
const next = new RequestFixture("next");
|
||||
next.predecessor = first;
|
||||
const unseen = new RequestFixture("unseen");
|
||||
unseen.predecessor = new RequestFixture("outside-capture");
|
||||
collector.request(first);
|
||||
collector.request(next);
|
||||
collector.request(unseen);
|
||||
await collector.finish();
|
||||
const requests = events.filter((event) => event.kind === "request");
|
||||
expect(requests[1]?.redirected_from_transaction_id).toBe(
|
||||
requests[0]?.transaction_id,
|
||||
);
|
||||
expect(requests[2]?.redirected_from_transaction_id).toBeNull();
|
||||
});
|
||||
|
||||
it("preserves read errors and does not turn unexposed or empty bytes into the same state", async () => {
|
||||
const { collector, events } = harness();
|
||||
const missing = new RequestFixture("missing");
|
||||
missing.postDataBuffer = () => null;
|
||||
const response = new ResponseFixture(missing, "unused");
|
||||
response.read = async () => {
|
||||
throw new Error("Cannot read 私密 response");
|
||||
};
|
||||
response.headersArray = async () => {
|
||||
throw new Error("Headers unavailable 私密");
|
||||
};
|
||||
const empty = new RequestFixture("");
|
||||
collector.request(missing);
|
||||
collector.response(response);
|
||||
collector.finished(missing);
|
||||
collector.request(empty);
|
||||
collector.response(new ResponseFixture(empty, ""));
|
||||
collector.finished(empty);
|
||||
await collector.finish();
|
||||
expect(events).toContainEqual(
|
||||
expect.objectContaining({
|
||||
kind: "network-content",
|
||||
phase: "request",
|
||||
body: { state: "not_exposed" },
|
||||
}),
|
||||
);
|
||||
expect(events).toContainEqual(
|
||||
expect.objectContaining({
|
||||
kind: "network-content",
|
||||
phase: "response",
|
||||
headers: {
|
||||
state: "unavailable",
|
||||
message: "Headers unavailable [REDACTED:declared]",
|
||||
},
|
||||
body: {
|
||||
state: "unavailable",
|
||||
reason: "body-read-failed",
|
||||
message: "Cannot read [REDACTED:declared] response",
|
||||
},
|
||||
}),
|
||||
);
|
||||
expect(events).toContainEqual(
|
||||
expect.objectContaining({
|
||||
kind: "network-content",
|
||||
phase: "response",
|
||||
body: expect.objectContaining({
|
||||
state: "captured",
|
||||
content: "",
|
||||
bytes: 0,
|
||||
}),
|
||||
}),
|
||||
);
|
||||
expect(collector.limitations().length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it("redacts declared encodings in body media metadata", async () => {
|
||||
const { collector, events } = harness({
|
||||
request_body: false,
|
||||
response_body: true,
|
||||
header_values: false,
|
||||
});
|
||||
const request = new RequestFixture("input");
|
||||
const response = new ResponseFixture(request, "ordinary-bytes");
|
||||
const headers = response.headers();
|
||||
response.headers = () => ({
|
||||
...headers,
|
||||
"content-type": `text/plain; fixture=${encodeURIComponent("私密")}`,
|
||||
});
|
||||
collector.request(request);
|
||||
collector.response(response);
|
||||
collector.finished(request);
|
||||
await collector.finish();
|
||||
expect(events).toContainEqual(
|
||||
expect.objectContaining({
|
||||
kind: "network-content",
|
||||
phase: "response",
|
||||
body: expect.objectContaining({
|
||||
state: "captured",
|
||||
media_type: "text/plain; fixture=[REDACTED:declared]",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
|
@ -0,0 +1,303 @@
|
|||
import type {
|
||||
BrowserNetworkBody,
|
||||
BrowserNetworkContentSelection,
|
||||
} from "../../domain/browserNetworkEvidence.js";
|
||||
import type { BrowserScenarioEvent } from "../../domain/browserScenarioCapture.js";
|
||||
import type { BrowserScenarioSecrets } from "../BrowserScenarioSecrets.js";
|
||||
import {
|
||||
captureNetworkBody,
|
||||
captureNetworkHeaders,
|
||||
type NetworkHeadersSource,
|
||||
type NetworkReadOptions,
|
||||
} from "./PlaywrightNetworkContent.js";
|
||||
|
||||
/** Request object identity is the association key, never URL or receipt adjacency. */
|
||||
export interface ScenarioNetworkRequest extends NetworkHeadersSource {
|
||||
method(): string;
|
||||
url(): string;
|
||||
resourceType(): string;
|
||||
postDataBuffer(): Buffer | null;
|
||||
redirectedFrom(): ScenarioNetworkRequest | null;
|
||||
failure(): { errorText: string } | null;
|
||||
}
|
||||
|
||||
/** A response's body becomes eligible for reading only after requestfinished. */
|
||||
export interface ScenarioNetworkResponse extends NetworkHeadersSource {
|
||||
request(): ScenarioNetworkRequest;
|
||||
url(): string;
|
||||
status(): number;
|
||||
body(): Promise<Buffer>;
|
||||
}
|
||||
|
||||
type UnindexedNetworkEvent = BrowserScenarioEvent extends infer Event
|
||||
? Event extends BrowserScenarioEvent
|
||||
? Event["kind"] extends
|
||||
| "request"
|
||||
| "response"
|
||||
| "request-finished"
|
||||
| "request-failed"
|
||||
| "request-unfinished"
|
||||
| "network-content"
|
||||
? Omit<Event, "sequence" | "step_index">
|
||||
: never
|
||||
: never
|
||||
: never;
|
||||
|
||||
interface Transaction {
|
||||
readonly id: string;
|
||||
readonly redirectedFrom: string | null;
|
||||
response?: {
|
||||
readonly source: ScenarioNetworkResponse;
|
||||
readonly sequence: number;
|
||||
};
|
||||
terminal?: "finished" | "failed";
|
||||
responseScheduled: boolean;
|
||||
}
|
||||
|
||||
interface NetworkCaptureOptions {
|
||||
readonly selection: BrowserNetworkContentSelection;
|
||||
readonly secrets: BrowserScenarioSecrets;
|
||||
readonly emit: (event: UnindexedNetworkEvent) => number;
|
||||
readonly signal?: AbortSignal;
|
||||
/** Production uses 5s; a fixture can supply its own bounded-read clock. */
|
||||
readonly readTimeoutMs?: number;
|
||||
}
|
||||
|
||||
/** Captures correlated network phases and drains selected content before browser cleanup. */
|
||||
export class PlaywrightScenarioNetwork {
|
||||
private readonly transactions = new Map<
|
||||
ScenarioNetworkRequest,
|
||||
Transaction
|
||||
>();
|
||||
private readonly pending: Promise<void>[] = [];
|
||||
private readonly gaps = new Set<string>();
|
||||
private accepting = true;
|
||||
private readonly readOptions: NetworkReadOptions;
|
||||
|
||||
constructor(private readonly options: NetworkCaptureOptions) {
|
||||
this.readOptions = {
|
||||
timeoutMs: options.readTimeoutMs ?? 5_000,
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
};
|
||||
}
|
||||
|
||||
/** Observe a request start, retaining the producer's redirect relationship when known. */
|
||||
request(request: ScenarioNetworkRequest): void {
|
||||
if (!this.accepting) return;
|
||||
const transaction = this.transaction(request);
|
||||
const sequence = this.options.emit({
|
||||
...this.metadata(request, transaction),
|
||||
kind: "request",
|
||||
status: null,
|
||||
failure: null,
|
||||
});
|
||||
if (
|
||||
this.options.selection.request_body ||
|
||||
this.options.selection.header_values
|
||||
)
|
||||
this.collect(
|
||||
transaction,
|
||||
sequence,
|
||||
"request",
|
||||
request,
|
||||
this.options.selection.request_body
|
||||
? captureNetworkBody(
|
||||
async () => request.postDataBuffer(),
|
||||
"request",
|
||||
request,
|
||||
this.options.secrets,
|
||||
this.readOptions,
|
||||
)
|
||||
: Promise.resolve({ state: "not_requested" }),
|
||||
);
|
||||
}
|
||||
|
||||
/** Observe response metadata without starting a potentially unfinished body read. */
|
||||
response(response: ScenarioNetworkResponse): void {
|
||||
if (!this.accepting) return;
|
||||
const request = response.request();
|
||||
const transaction = this.transaction(request);
|
||||
const sequence = this.options.emit({
|
||||
...this.metadata(request, transaction),
|
||||
kind: "response",
|
||||
url: this.options.secrets.sanitizeUrl(response.url()),
|
||||
status: response.status(),
|
||||
header_names: this.headerNames(response),
|
||||
failure: null,
|
||||
});
|
||||
transaction.response = { source: response, sequence };
|
||||
}
|
||||
|
||||
/** Record completion and read the original completed response, never a later refetch. */
|
||||
finished(request: ScenarioNetworkRequest): void {
|
||||
if (!this.accepting) return;
|
||||
const transaction = this.transaction(request);
|
||||
transaction.terminal = "finished";
|
||||
this.options.emit({
|
||||
...this.metadata(request, transaction),
|
||||
kind: "request-finished",
|
||||
status: null,
|
||||
failure: null,
|
||||
});
|
||||
this.responseContent(transaction);
|
||||
}
|
||||
|
||||
/** Preserve request failure separately from missing or unreadable response content. */
|
||||
failed(request: ScenarioNetworkRequest): void {
|
||||
if (!this.accepting) return;
|
||||
const transaction = this.transaction(request);
|
||||
transaction.terminal = "failed";
|
||||
const message = this.options.secrets.redact(
|
||||
request.failure()?.errorText || "Unknown request failure",
|
||||
);
|
||||
this.options.emit({
|
||||
...this.metadata(request, transaction),
|
||||
kind: "request-failed",
|
||||
status: null,
|
||||
failure: message,
|
||||
});
|
||||
this.responseContent(transaction, {
|
||||
state: "unavailable",
|
||||
reason: "request-failed",
|
||||
message,
|
||||
});
|
||||
}
|
||||
|
||||
/** Stop observing, mark unfinished transactions, and settle bounded content reads. */
|
||||
async finish(): Promise<void> {
|
||||
if (!this.accepting) {
|
||||
await Promise.all(this.pending);
|
||||
return;
|
||||
}
|
||||
this.accepting = false;
|
||||
for (const [request, transaction] of this.transactions) {
|
||||
if (transaction.terminal === undefined) {
|
||||
this.gaps.add(
|
||||
"Some network requests had not completed when capture ended; later activity is unknown.",
|
||||
);
|
||||
this.options.emit({
|
||||
...this.metadata(request, transaction),
|
||||
kind: "request-unfinished",
|
||||
status: null,
|
||||
failure: null,
|
||||
reason:
|
||||
this.options.signal?.aborted === true
|
||||
? "cancelled"
|
||||
: "capture-ended",
|
||||
});
|
||||
this.responseContent(transaction, {
|
||||
state: "unavailable",
|
||||
reason:
|
||||
this.options.signal?.aborted === true
|
||||
? "cancelled"
|
||||
: "response-unfinished",
|
||||
message:
|
||||
"The response had not finished at the capture cutoff; no body read or refetch was attempted.",
|
||||
});
|
||||
}
|
||||
}
|
||||
await Promise.all(this.pending);
|
||||
}
|
||||
|
||||
/** Missing selected content contributes to aggregate completeness. */
|
||||
limitations(): readonly string[] {
|
||||
return [...this.gaps];
|
||||
}
|
||||
|
||||
private transaction(request: ScenarioNetworkRequest): Transaction {
|
||||
const known = this.transactions.get(request);
|
||||
if (known !== undefined) return known;
|
||||
const predecessor = request.redirectedFrom();
|
||||
const transaction: Transaction = {
|
||||
id: `request-${this.transactions.size + 1}`,
|
||||
redirectedFrom:
|
||||
predecessor === null
|
||||
? null
|
||||
: (this.transactions.get(predecessor)?.id ?? null),
|
||||
responseScheduled: false,
|
||||
};
|
||||
this.transactions.set(request, transaction);
|
||||
return transaction;
|
||||
}
|
||||
|
||||
private headerNames(source: NetworkHeadersSource): string[] {
|
||||
return Object.keys(source.headers())
|
||||
.map((name) => this.options.secrets.redact(name))
|
||||
.sort();
|
||||
}
|
||||
|
||||
private metadata(request: ScenarioNetworkRequest, transaction: Transaction) {
|
||||
return {
|
||||
transaction_id: transaction.id,
|
||||
redirected_from_transaction_id: transaction.redirectedFrom,
|
||||
method: this.options.secrets.redact(request.method()),
|
||||
url: this.options.secrets.sanitizeUrl(request.url()),
|
||||
resource_type: this.options.secrets.redact(request.resourceType()),
|
||||
header_names: this.headerNames(request),
|
||||
};
|
||||
}
|
||||
|
||||
private responseContent(
|
||||
transaction: Transaction,
|
||||
unavailable?: BrowserNetworkBody,
|
||||
): void {
|
||||
const response = transaction.response;
|
||||
if (
|
||||
response === undefined ||
|
||||
transaction.responseScheduled ||
|
||||
(!this.options.selection.response_body &&
|
||||
!this.options.selection.header_values)
|
||||
)
|
||||
return;
|
||||
transaction.responseScheduled = true;
|
||||
this.collect(
|
||||
transaction,
|
||||
response.sequence,
|
||||
"response",
|
||||
response.source,
|
||||
this.options.selection.response_body
|
||||
? unavailable === undefined
|
||||
? captureNetworkBody(
|
||||
() => response.source.body(),
|
||||
"response",
|
||||
response.source,
|
||||
this.options.secrets,
|
||||
this.readOptions,
|
||||
)
|
||||
: Promise.resolve(unavailable)
|
||||
: Promise.resolve({ state: "not_requested" }),
|
||||
);
|
||||
}
|
||||
|
||||
private collect(
|
||||
transaction: Transaction,
|
||||
sequence: number,
|
||||
phase: "request" | "response",
|
||||
source: NetworkHeadersSource,
|
||||
body: Promise<BrowserNetworkBody>,
|
||||
): void {
|
||||
const headers = this.options.selection.header_values
|
||||
? captureNetworkHeaders(source, this.options.secrets, this.readOptions)
|
||||
: Promise.resolve({ state: "not_requested" } as const);
|
||||
this.pending.push(
|
||||
Promise.all([headers, body]).then(([retainedHeaders, retainedBody]) => {
|
||||
if (
|
||||
retainedHeaders.state === "unavailable" ||
|
||||
retainedBody.state === "unavailable" ||
|
||||
retainedBody.state === "not_exposed"
|
||||
)
|
||||
this.gaps.add(
|
||||
"Some selected network content was unavailable; per-transaction content states retain the reason.",
|
||||
);
|
||||
this.options.emit({
|
||||
kind: "network-content",
|
||||
transaction_id: transaction.id,
|
||||
source_event_sequence: sequence,
|
||||
phase,
|
||||
headers: retainedHeaders,
|
||||
body: retainedBody,
|
||||
});
|
||||
}),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -26,7 +26,7 @@ export const BROWSER_SCENARIO_TOOL_CONTRACTS = [
|
|||
name: "capture_browser_scenario",
|
||||
...toolContractMetadata("capture_browser_scenario"),
|
||||
description:
|
||||
"Run a controlled browser scenario when passive observation cannot exercise the application. Launch the selected executable or connect to a loopback CDP target, provide a start URL and actions, and select storage or artifacts when needed. Environment, empty storage, and a URL-only final capture are defaults; provide initial storage or additional artifacts when needed. Agents can run their own local fixture servers to control application responses. Actions run until completion or cancellation unless timeout_ms is set. Results are returned inline.",
|
||||
"Run a controlled browser scenario when passive observation cannot exercise the application. Launch the selected executable or connect to a loopback CDP target, provide a start URL and actions, and select storage or artifacts when needed. Environment, empty storage, and a URL-only final capture are defaults. capture.network selects request_body, response_body, and header_values independently; selected content is returned inline with transaction and event references, byte digests, and explicit unavailable states. Credentials and declared secrets are redacted. Agents can run their own local fixture servers to control application responses. Actions run until completion or cancellation unless timeout_ms is set. Results are returned inline.",
|
||||
kind: "browser-provider",
|
||||
inputSchema: browserScenarioInputSchema,
|
||||
outputSchema: evidenceResultOf(browserScenarioCaptureSchema),
|
||||
|
|
|
|||
|
|
@ -0,0 +1,54 @@
|
|||
import { createHash } from "node:crypto";
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { browserNetworkBodySchema } from "./browserNetworkEvidence.js";
|
||||
import { browserScenarioSchema } from "./browserScenario.js";
|
||||
|
||||
const bytes = Buffer.from([0xff, 0, 0xfe]);
|
||||
const retained = {
|
||||
state: "captured",
|
||||
representation: "browser-decoded-response-bytes",
|
||||
encoding: "base64",
|
||||
content: bytes.toString("base64"),
|
||||
bytes: bytes.length,
|
||||
sha256: createHash("sha256").update(bytes).digest("hex"),
|
||||
media_type: null,
|
||||
redacted: false,
|
||||
};
|
||||
|
||||
it("validates the actual retained bytes against canonical base64, count, and digest", () => {
|
||||
expect(browserNetworkBodySchema.safeParse(retained).success).toBe(true);
|
||||
for (const mutation of [
|
||||
{ bytes: 4 },
|
||||
{ sha256: "0".repeat(64) },
|
||||
{ content: `${retained.content}\n` },
|
||||
])
|
||||
expect(
|
||||
browserNetworkBodySchema.safeParse({ ...retained, ...mutation }).success,
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("defaults content selection independently and rejects unsupported controls", () => {
|
||||
const input = {
|
||||
browser: { mode: "launch", executable_path: "/fixture/chrome" },
|
||||
start_url: { url: "https://example.test" },
|
||||
actions: [{ step_id: "done", action: "wait_for_timeout", duration_ms: 1 }],
|
||||
};
|
||||
expect(browserScenarioSchema.parse(input).capture.network).toEqual({
|
||||
request_body: false,
|
||||
response_body: false,
|
||||
header_values: false,
|
||||
});
|
||||
expect(
|
||||
browserScenarioSchema.parse({
|
||||
...input,
|
||||
capture: { network: { response_body: true } },
|
||||
}).capture.network,
|
||||
).toEqual({ request_body: false, response_body: true, header_values: false });
|
||||
expect(
|
||||
browserScenarioSchema.safeParse({
|
||||
...input,
|
||||
capture: { network: { refetch: true } },
|
||||
}).success,
|
||||
).toBe(false);
|
||||
});
|
||||
|
|
@ -0,0 +1,95 @@
|
|||
import { createHash } from "node:crypto";
|
||||
import { z } from "zod";
|
||||
|
||||
/** Caller-selected network content; omission retains metadata alone. */
|
||||
export const browserNetworkContentSelectionValuesSchema = z.strictObject({
|
||||
request_body: z.boolean().default(false),
|
||||
response_body: z.boolean().default(false),
|
||||
header_values: z.boolean().default(false),
|
||||
});
|
||||
|
||||
/** Input omission selects metadata without defaulting legacy capture coverage. */
|
||||
export const browserNetworkContentSelectionSchema =
|
||||
browserNetworkContentSelectionValuesSchema.default({
|
||||
request_body: false,
|
||||
response_body: false,
|
||||
header_values: false,
|
||||
});
|
||||
|
||||
/** Self-verifying retained bytes, after any declared-secret redaction. */
|
||||
export const browserNetworkBodySchema = z.union([
|
||||
z.strictObject({ state: z.literal("not_requested") }),
|
||||
z.strictObject({ state: z.literal("not_exposed") }),
|
||||
z.strictObject({
|
||||
state: z.literal("unavailable"),
|
||||
reason: z.enum([
|
||||
"request-failed",
|
||||
"response-unfinished",
|
||||
"body-read-failed",
|
||||
"body-read-timeout",
|
||||
"cancelled",
|
||||
"capture-ended",
|
||||
]),
|
||||
message: z.string().min(1),
|
||||
}),
|
||||
z
|
||||
.strictObject({
|
||||
state: z.literal("captured"),
|
||||
representation: z.enum([
|
||||
"browser-exposed-request-bytes",
|
||||
"browser-decoded-response-bytes",
|
||||
]),
|
||||
encoding: z.literal("base64"),
|
||||
content: z.string(),
|
||||
bytes: z.number().int().min(0),
|
||||
sha256: z.string().regex(/^[a-f0-9]{64}$/u),
|
||||
media_type: z.string().nullable(),
|
||||
redacted: z.boolean(),
|
||||
})
|
||||
.superRefine((artifact, context) => {
|
||||
const bytes = Buffer.from(artifact.content, "base64");
|
||||
if (bytes.toString("base64") !== artifact.content)
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
path: ["content"],
|
||||
message: "Network body must use canonical base64",
|
||||
});
|
||||
if (bytes.length !== artifact.bytes)
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
path: ["bytes"],
|
||||
message: "Network body byte count disagrees with retained bytes",
|
||||
});
|
||||
if (createHash("sha256").update(bytes).digest("hex") !== artifact.sha256)
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
path: ["sha256"],
|
||||
message: "Network body digest disagrees with retained bytes",
|
||||
});
|
||||
}),
|
||||
]);
|
||||
|
||||
/** Headers retain producer order and duplicates, with credential values removed. */
|
||||
export const browserNetworkHeadersSchema = z.union([
|
||||
z.strictObject({ state: z.literal("not_requested") }),
|
||||
z.strictObject({
|
||||
state: z.literal("unavailable"),
|
||||
message: z.string().min(1),
|
||||
}),
|
||||
z.strictObject({
|
||||
state: z.literal("captured"),
|
||||
items: z.array(
|
||||
z.strictObject({
|
||||
name: z.string().min(1),
|
||||
value: z.string().nullable(),
|
||||
redacted: z.boolean(),
|
||||
}),
|
||||
),
|
||||
}),
|
||||
]);
|
||||
|
||||
export type BrowserNetworkContentSelection = z.infer<
|
||||
typeof browserNetworkContentSelectionSchema
|
||||
>;
|
||||
export type BrowserNetworkBody = z.infer<typeof browserNetworkBodySchema>;
|
||||
export type BrowserNetworkHeaders = z.infer<typeof browserNetworkHeadersSchema>;
|
||||
|
|
@ -1,4 +1,5 @@
|
|||
import { z } from "zod";
|
||||
import { browserNetworkContentSelectionValuesSchema } from "./browserNetworkEvidence.js";
|
||||
|
||||
import {
|
||||
browserScenarioCompletenessSchema,
|
||||
|
|
@ -33,6 +34,7 @@ export const browserScenarioCaptureSchema = z
|
|||
start_origin: z.string().min(1),
|
||||
action_count: z.number().int().min(1),
|
||||
secret_references: z.array(z.string().min(1)),
|
||||
network_content: browserNetworkContentSelectionValuesSchema.optional(),
|
||||
}),
|
||||
duration_ms: z.number().int().min(0),
|
||||
steps: z.array(browserScenarioStepSchema).min(1),
|
||||
|
|
@ -45,6 +47,30 @@ export const browserScenarioCaptureSchema = z
|
|||
limitations: z.array(z.string().min(1)),
|
||||
})
|
||||
.superRefine((capture, context) => {
|
||||
const bySequence = new Map(
|
||||
capture.events.items.map((event) => [event.sequence, event]),
|
||||
);
|
||||
if (bySequence.size !== capture.events.items.length)
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
path: ["events", "items"],
|
||||
message: "Event sequence numbers must be unique",
|
||||
});
|
||||
for (const [index, event] of capture.events.items.entries()) {
|
||||
if (event.kind !== "network-content") continue;
|
||||
const source = bySequence.get(event.source_event_sequence);
|
||||
if (
|
||||
source?.kind !== event.phase ||
|
||||
source.transaction_id !== event.transaction_id ||
|
||||
source.sequence >= event.sequence
|
||||
)
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
path: ["events", "items", index, "source_event_sequence"],
|
||||
message:
|
||||
"Network content must reference an earlier matching transaction phase",
|
||||
});
|
||||
}
|
||||
if (capture.events.retained !== capture.events.items.length)
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
|
|
|
|||
|
|
@ -3,6 +3,10 @@ import { createHash } from "node:crypto";
|
|||
import { z } from "zod";
|
||||
|
||||
import { emptyArraySchema } from "./emptyArraySchema.js";
|
||||
import {
|
||||
browserNetworkBodySchema,
|
||||
browserNetworkHeadersSchema,
|
||||
} from "./browserNetworkEvidence.js";
|
||||
import { sanitizedBrowserUrlSchema } from "./browserObservation.js";
|
||||
import { webScreenshotArtifactSchema } from "./webScreenshot.js";
|
||||
import { digestSchema } from "./../domain/digests.js";
|
||||
|
|
@ -123,6 +127,8 @@ const eventBase = {
|
|||
const eventUrl = sanitizedBrowserUrlSchema.nullable();
|
||||
const networkEventShape = {
|
||||
...eventBase,
|
||||
transaction_id: z.string().min(1).optional(),
|
||||
redirected_from_transaction_id: z.string().min(1).nullable().optional(),
|
||||
method: z.string().min(1),
|
||||
url: sanitizedBrowserUrlSchema,
|
||||
resource_type: z.string().min(1),
|
||||
|
|
@ -168,6 +174,28 @@ export const browserScenarioEventSchema = z.union([
|
|||
status: z.null(),
|
||||
failure: z.string().min(1),
|
||||
}),
|
||||
z.strictObject({
|
||||
...networkEventShape,
|
||||
kind: z.literal("request-finished"),
|
||||
status: z.null(),
|
||||
failure: z.null(),
|
||||
}),
|
||||
z.strictObject({
|
||||
...networkEventShape,
|
||||
kind: z.literal("request-unfinished"),
|
||||
status: z.null(),
|
||||
failure: z.null(),
|
||||
reason: z.enum(["capture-ended", "cancelled"]),
|
||||
}),
|
||||
z.strictObject({
|
||||
...eventBase,
|
||||
kind: z.literal("network-content"),
|
||||
transaction_id: z.string().min(1),
|
||||
source_event_sequence: z.number().int().min(1),
|
||||
phase: z.enum(["request", "response"]),
|
||||
headers: browserNetworkHeadersSchema,
|
||||
body: browserNetworkBodySchema,
|
||||
}),
|
||||
z.strictObject({
|
||||
...eventBase,
|
||||
kind: z.enum(["websocket-opened", "websocket-closed"]),
|
||||
|
|
|
|||
|
|
@ -12,6 +12,147 @@ import {
|
|||
type BrowserScenarioCapture,
|
||||
} from "./browserScenarioCapture.js";
|
||||
|
||||
const networkCapture = (sourceSequence: number, receiptStep: number) => {
|
||||
const base = scenarioCapture({});
|
||||
const events = [
|
||||
{
|
||||
sequence: sourceSequence,
|
||||
step_index: 0,
|
||||
kind: "request",
|
||||
method: "POST",
|
||||
url: {
|
||||
url: "https://app.example.test/api",
|
||||
origin: "https://app.example.test",
|
||||
query_parameter_names: [],
|
||||
redacted: false,
|
||||
},
|
||||
resource_type: "fetch",
|
||||
transaction_id: "request-1",
|
||||
header_names: [],
|
||||
status: null,
|
||||
failure: null,
|
||||
},
|
||||
{
|
||||
sequence: sourceSequence + 1,
|
||||
step_index: receiptStep,
|
||||
kind: "network-content",
|
||||
transaction_id: "request-1",
|
||||
phase: "request",
|
||||
source_event_sequence: sourceSequence,
|
||||
headers: { state: "not_requested" },
|
||||
body: {
|
||||
state: "captured",
|
||||
representation: "browser-exposed-request-bytes",
|
||||
encoding: "base64",
|
||||
content: "eA==",
|
||||
bytes: 1,
|
||||
sha256: createHash("sha256").update("x").digest("hex"),
|
||||
media_type: null,
|
||||
redacted: false,
|
||||
},
|
||||
},
|
||||
];
|
||||
return browserScenarioCaptureSchema.parse({
|
||||
...base,
|
||||
scenario: {
|
||||
...base.scenario,
|
||||
network_content: {
|
||||
request_body: true,
|
||||
response_body: false,
|
||||
header_values: false,
|
||||
},
|
||||
},
|
||||
events: { items: events, retained: 2, dropped: 0 },
|
||||
});
|
||||
};
|
||||
|
||||
it("normalizes network content anchors and attributes late content to its source step", () => {
|
||||
const before = networkCapture(3, 0);
|
||||
const after = networkCapture(90, 1);
|
||||
expect(
|
||||
compareBrowserScenarios(
|
||||
compareBrowserScenariosInputSchema.parse({
|
||||
before_scenario: before,
|
||||
after_scenario: after,
|
||||
}),
|
||||
).overall_status,
|
||||
).toBe("unchanged");
|
||||
const content = after.events.items[1];
|
||||
if (content?.kind !== "network-content")
|
||||
throw new Error("Expected content fixture");
|
||||
for (const mutation of [
|
||||
{ transaction_id: "another-request" },
|
||||
{ source_event_sequence: 91 },
|
||||
{ phase: "response" },
|
||||
])
|
||||
expect(
|
||||
browserScenarioCaptureSchema.safeParse({
|
||||
...after,
|
||||
events: {
|
||||
...after.events,
|
||||
items: [after.events.items[0], { ...content, ...mutation }],
|
||||
},
|
||||
}).success,
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("reports content-selection mismatch as unknown instead of a website change", () => {
|
||||
const withBody = networkCapture(1, 0);
|
||||
const metadataOnly = browserScenarioCaptureSchema.parse({
|
||||
...withBody,
|
||||
scenario: {
|
||||
...withBody.scenario,
|
||||
network_content: {
|
||||
request_body: false,
|
||||
response_body: false,
|
||||
header_values: false,
|
||||
},
|
||||
},
|
||||
events: {
|
||||
...withBody.events,
|
||||
items: withBody.events.items.filter(
|
||||
(event) => event.kind !== "network-content",
|
||||
),
|
||||
retained: 1,
|
||||
},
|
||||
});
|
||||
const compared = compareBrowserScenarios(
|
||||
compareBrowserScenariosInputSchema.parse({
|
||||
before_scenario: metadataOnly,
|
||||
after_scenario: withBody,
|
||||
}),
|
||||
);
|
||||
expect(compared.overall_status).toBe("unknown");
|
||||
expect(compared.steps.flatMap((step) => step.artifact_diffs)).toContainEqual(
|
||||
expect.objectContaining({
|
||||
artifact: "events",
|
||||
status: "unknown",
|
||||
reason: expect.stringContaining("content selections differ"),
|
||||
}),
|
||||
);
|
||||
const legacy = browserScenarioCaptureSchema.parse({
|
||||
...metadataOnly,
|
||||
scenario: { ...metadataOnly.scenario, network_content: undefined },
|
||||
});
|
||||
expect(legacy.scenario.network_content).toBeUndefined();
|
||||
const legacyComparison = compareBrowserScenarios(
|
||||
compareBrowserScenariosInputSchema.parse({
|
||||
before_scenario: legacy,
|
||||
after_scenario: metadataOnly,
|
||||
}),
|
||||
);
|
||||
expect(legacyComparison.overall_status).toBe("unknown");
|
||||
expect(
|
||||
legacyComparison.steps.flatMap((step) => step.artifact_diffs),
|
||||
).toContainEqual(
|
||||
expect.objectContaining({
|
||||
artifact: "events",
|
||||
status: "unknown",
|
||||
reason: expect.stringContaining("unknown in a legacy capture"),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
describe("browser scenario comparison", () => {
|
||||
it("records canonical literal normalization and ignores declared volatile fields", () => {
|
||||
const before = scenarioCapture({
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ import {
|
|||
captured,
|
||||
eventEvidenceState,
|
||||
eventsFor,
|
||||
networkContentCoverageReason,
|
||||
screenshotProjection,
|
||||
type CaptureState,
|
||||
} from "./browserScenarioDiffEvidence.js";
|
||||
|
|
@ -340,6 +341,10 @@ const compareStepArtifacts = (
|
|||
),
|
||||
beforeState: eventEvidenceState(input.beforeCapture, input.beforeStep),
|
||||
afterState: eventEvidenceState(input.afterCapture, input.afterStep),
|
||||
incomparableReason: networkContentCoverageReason(
|
||||
input.beforeCapture,
|
||||
input.afterCapture,
|
||||
),
|
||||
}),
|
||||
];
|
||||
};
|
||||
|
|
@ -354,6 +359,7 @@ interface CompareCapturedValuesInput<Value> {
|
|||
readonly project?: (value: Value) => unknown;
|
||||
readonly beforeState?: BrowserScenarioEvidenceState;
|
||||
readonly afterState?: BrowserScenarioEvidenceState;
|
||||
readonly incomparableReason?: string | undefined;
|
||||
}
|
||||
|
||||
const compareCapturedValues = <Value>(
|
||||
|
|
@ -386,7 +392,8 @@ const compareCapturedValues = <Value>(
|
|||
beforeState !== "captured" ||
|
||||
afterState !== "captured" ||
|
||||
before.state !== "captured" ||
|
||||
after.state !== "captured"
|
||||
after.state !== "captured" ||
|
||||
input.incomparableReason !== undefined
|
||||
)
|
||||
return {
|
||||
artifact,
|
||||
|
|
@ -398,7 +405,9 @@ const compareCapturedValues = <Value>(
|
|||
after_state: afterState,
|
||||
before_sha256: beforeSha256,
|
||||
after_sha256: afterSha256,
|
||||
reason: `Artifact coverage is not comparable (${beforeState} before, ${afterState} after).`,
|
||||
reason:
|
||||
input.incomparableReason ??
|
||||
`Artifact coverage is not comparable (${beforeState} before, ${afterState} after).`,
|
||||
},
|
||||
};
|
||||
if (beforeSha256 === null || afterSha256 === null)
|
||||
|
|
|
|||
|
|
@ -98,11 +98,54 @@ export const actionState = (step: BrowserScenarioStep) => ({
|
|||
export const eventsFor = (
|
||||
capture: BrowserScenarioCapture,
|
||||
stepIndex: number,
|
||||
): readonly unknown[] =>
|
||||
capture.events.items.flatMap(
|
||||
({ sequence: _sequence, step_index: eventStepIndex, ...event }) =>
|
||||
eventStepIndex === stepIndex ? [event] : [],
|
||||
): readonly unknown[] => {
|
||||
const sourceSteps = new Map(
|
||||
capture.events.items.map(({ sequence, step_index }) => [
|
||||
sequence,
|
||||
step_index,
|
||||
]),
|
||||
);
|
||||
return capture.events.items.flatMap<unknown>(
|
||||
({ sequence: _sequence, step_index: eventStepIndex, ...event }) => {
|
||||
if (event.kind === "network-content") {
|
||||
const { source_event_sequence: sourceSequence, ...content } = event;
|
||||
return sourceSteps.get(sourceSequence) === stepIndex ? [content] : [];
|
||||
}
|
||||
return eventStepIndex === stepIndex ? [event] : [];
|
||||
},
|
||||
);
|
||||
};
|
||||
|
||||
/** Different content selection is missing comparability, not an observed website change. */
|
||||
export const networkContentCoverageReason = (
|
||||
before: BrowserScenarioCapture,
|
||||
after: BrowserScenarioCapture,
|
||||
): string | undefined => {
|
||||
const left = before.scenario.network_content;
|
||||
const right = after.scenario.network_content;
|
||||
if (left === undefined || right === undefined) {
|
||||
const events = [...before.events.items, ...after.events.items];
|
||||
if (
|
||||
events.some((event) => event.kind === "network-content") ||
|
||||
((left === undefined) !== (right === undefined) &&
|
||||
events.some(
|
||||
(event) =>
|
||||
event.kind === "request" ||
|
||||
event.kind === "response" ||
|
||||
event.kind === "request-failed" ||
|
||||
event.kind === "request-finished" ||
|
||||
event.kind === "request-unfinished",
|
||||
))
|
||||
)
|
||||
return "Network content selection is unknown in a legacy capture; capture both sides with matching content selection.";
|
||||
return undefined;
|
||||
}
|
||||
return left.request_body !== right.request_body ||
|
||||
left.response_body !== right.response_body ||
|
||||
left.header_values !== right.header_values
|
||||
? "Network content selections differ; capture both sides with matching content selection before comparing network evidence."
|
||||
: undefined;
|
||||
};
|
||||
|
||||
/** Wrap one value as captured comparison evidence. */
|
||||
export const captured = <Value>(value: Value): CaptureState<Value> => ({
|
||||
|
|
|
|||
|
|
@ -1,5 +1,7 @@
|
|||
import { z } from "zod";
|
||||
|
||||
import { browserNetworkContentSelectionSchema } from "./browserNetworkEvidence.js";
|
||||
|
||||
import {
|
||||
browserEndpointSchema,
|
||||
browserOriginSchema,
|
||||
|
|
@ -294,8 +296,18 @@ export const browserScenarioCaptureSchema = z
|
|||
]),
|
||||
)
|
||||
.default([]),
|
||||
network: browserNetworkContentSelectionSchema,
|
||||
})
|
||||
.default({
|
||||
after_each_step: [],
|
||||
at_end: ["url"],
|
||||
events: [],
|
||||
network: {
|
||||
request_body: false,
|
||||
response_body: false,
|
||||
header_values: false,
|
||||
},
|
||||
})
|
||||
.default({ after_each_step: [], at_end: ["url"], events: [] })
|
||||
.describe(
|
||||
"Optional retained artifacts and event families. Defaults to only a final sanitized URL; request screenshots, DOM, accessibility, history, storage, and event capture explicitly.",
|
||||
);
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
Loading…
Reference in New Issue