rea/experiments/analysis-efficiency/questions
Ysopwnable c7105d9723
test(perf): preserve negative small-PCAP workflow measurements (#2640)
Preserve reproducible complete versus summary/page PCAP measurements on two official HTTP recordings and a fixed metadata/URI question. Summary/page uses more than twice the SDK JSON and about 51% more total model input in both real-model pairs, so existing complete defaults and production APIs/cache remain unchanged. These results apply to these captures and question; no general speedup is claimed. Refs #2016.

Authenticate experiment SDK envelopes with the pinned runtime's existing Evidence parser and check coverage, target/view/operation ownership and retained-parent identity. Separate factual and exact-workflow grading, retaining diagnostics and partial facts for malformed replies. Register six comparator CLI scenarios in the discovered Vitest evaluation project and remove the manual suite. The final change is limited to nine experiment files and one discovered evaluation test; production behavior is unchanged.

Validation includes eight actual SDK workflows and four actual Codex CLI tasks, independent raw-frame and CLI/MCP checks, authentication of all 18 original receipts, 34 context/grading regressions and six comparator scenarios. All four stored model answers pass factual and workflow regrading without model reruns. Original measured runtime remains 3ae826ceba6f25b08bce50bcb51001209bc16b57; original measurements and private raw artifacts are unchanged. Serial local checks use one CPU with 3-GiB early-stop/4-GiB hard/no-swap limits. Unknown terminal counters remain explicitly unknown.

Integrate upstream shared fixture owners and Windows verification, removing both superseded local differences. All final experiment/comparator files remain byte-identical to the reviewed sources and the final PR scope is 10 files. Full bounded preflight passes in 73.953 seconds with ~2.23 GiB peak; payload cleanup is verified, zero phase OOM/limit events are observed, and terminal counters remain unknown. Current-head automatic CI 38492231165 passes all 35 selected jobs, including every compiled shard, source/static/build/docs, actual Windows/native/package and real application lanes, Portable reports/inventory, reverse-tools coverage/mutation and CI required. Greptile passes and all review threads are resolved. Actual CI checkout is 4eb2262359cb01f2fb1314db0bc18d3dc9092792, integrated with main 41a2e84647. Passing retries do not establish the root cause or a fix of the earlier main worker cleanup failure; its original evidence is preserved. No release.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-14 09:34:02 +08:00
..
android.txt feat(analysis): add opt-in retained inventory and WASM views (#2122) 2026-10-12 21:17:52 +08:00
browser.txt feat(analysis): add opt-in retained inventory and WASM views (#2122) 2026-10-12 21:17:52 +08:00
electron-inventory.txt feat(analysis): add opt-in retained inventory and WASM views (#2122) 2026-10-12 21:17:52 +08:00
evm-uniswap.txt refactor(evm): bind offline analysis through a static descriptor 2026-10-13 01:26:43 +08:00
evm.txt feat(analysis): add opt-in retained inventory and WASM views (#2122) 2026-10-12 21:17:52 +08:00
firmware.txt feat(analysis): add opt-in retained inventory and WASM views (#2122) 2026-10-12 21:17:52 +08:00
flutter-aot.txt perf(flutter): add opt-in retained AOT summary and string pages 2026-10-13 11:39:30 +08:00
javascript-selected.txt feat(analysis): add opt-in retained inventory and WASM views (#2122) 2026-10-12 21:17:52 +08:00
managed-nonempty.txt feat(analysis): add opt-in retained inventory and WASM views (#2122) 2026-10-12 21:17:52 +08:00
managed.txt feat(analysis): add opt-in retained inventory and WASM views (#2122) 2026-10-12 21:17:52 +08:00
native-dos.txt test(native): verify real PE and DOS entry views 2026-10-13 00:04:00 +08:00
native-pe.txt test(native): verify real PE and DOS entry views 2026-10-13 00:04:00 +08:00
native.txt feat(analysis): add opt-in retained inventory and WASM views (#2122) 2026-10-12 21:17:52 +08:00
pcap-http.txt test(perf): preserve negative small-PCAP workflow measurements (#2640) 2026-10-14 09:34:02 +08:00
sqlite-qgis.txt test(perf): verify existing SQLite column selection on QGIS data (#2489) 2026-10-13 12:38:12 +08:00
wasm-interface.txt feat(wasm): add opt-in first-call summary delivery (#2193) 2026-10-12 22:01:47 +08:00
wasm.txt feat(analysis): add opt-in retained inventory and WASM views (#2122) 2026-10-12 21:17:52 +08:00