Commit Graph

35 Commits

Author SHA1 Message Date
morluto 4d4631ac94 test: harden verification fixtures and runtime coverage
Use exact discovered test identities, production MCP sessions and byte-exact setup backups. Add CLI signal cleanup coverage and independent Ghidra, Go and firmware verification assertions; prune redundant test configuration and preserve platform limits.
2026-10-14 08:59:27 +08:00
morluto 0bf01dfa59
perf(ci): share trusted caches and fail fast on PR test failures (#3007)
* fix(ci): restore inherited validation failures

* perf(ci): share trusted caches and fail fast on PR test failures

* fix(ci): bound static cache and hash vitest config for knip

- Bound the rolling .turbo/cache snapshot with 7d/500MB eviction so each
  trusted main save cannot accumulate history toward the GitHub limit.
- Hash vitest.config.ts in knip inputs so removing the CiTestSequencer
  import invalidates the cached passing check.
- Cover both in cache-policy and task-hash regressions.
2026-10-14 04:25:57 +08:00
CindyGest e1897c3c17
fix(verify-package): read Hopper installer applicability from its owner (#2973)
* fix(verify-package): read Hopper installer applicability from its owner

packageSetupSupport derived hopperSetupSupported from the general doctor
host check plus a non-Windows condition, so a Debian-family host REA
supports but the Hopper package never tested reported an automated
installer the packaged setup never offers. verifySetupApply then expected
needs_confirmation for the explicit --install-hopper request and failed
at verify-package-setup.mjs:403.

Applicability now comes from readLinuxDistribution() in the built
application, so the verifier follows the predicate the package ships, and
it stays unknown when no distribution can be read instead of collapsing
into a known-unsupported host. Generic agent configuration and idempotence
assertions stay independent of installer applicability, and the expected
status is still derived from the host rather than from the request being
asserted.

Verified on Kali 2026.3, a Debian-family host with an installer-untested
distribution and the simulated Hopper package environment: before the
change npm run verify:package fails at verify-package-setup.mjs:403 with
the packaged setup correctly returning needs_human; after it the same run
passes every stage. Both applicability branches are covered by
tests/evaluation/package/packageSetupSupport.test.ts, which runs in the
source lane without a build. No real Hopper installation was exercised,
so no engine compatibility claim follows.

Refs #2939

* fix(ci): preserve Hopper doctor readiness checks

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-14 03:19:45 +08:00
CodeAlex 4758ba30b6
fix(package-verifier): reject overview responses without a procedure count (#2936)
* fix(package-verifier): reject overview responses without a procedure count

The analyze and inspect checks compared procedure_count with `< 1`, and an absent
or null normalized_result makes that comparison false, so a binary_overview
response without its required result passed the platform smoke check. Require an
integer procedure count before the positive-count comparison.

* test(package-verifier): cover malformed overview receipts

---------

Co-authored-by: REA Maintenance <76467478+morluto@users.noreply.github.com>
2026-10-14 03:06:53 +08:00
Huamnanx ffd8c9c03d
test(evaluation): grade managed agent fixture facts (#1924)
* test(evaluation): grade managed agent fixture facts

* test(evaluation): cover distinct managed artifact deliveries

* fix(evaluation): require unfiltered managed type inventories

* fix(evaluation): preserve selected entry point grounding

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-13 23:34:17 +08:00
morluto fa0a9b2d32 ci: observe affected-owner test plans against full results 2026-10-13 19:21:09 +08:00
记得吃中饭 2995ec6910
fix(ghidra): resolve file offsets in decompile (#2101)
* fix(native): expose loaded image bases in overviews

* test(ghidra): retain resources when verifier cleanup fails

* docs(ghidra): spell out file-offset address mapping

* fix(ci): separate and retry NativeAOT dependency restoration

* fix(test): resolve CI script file URLs on Windows

* fix(native): retain conflict observations and actionable CLI guidance

* fix(fixtures): keep NativeAOT retry logs credential-safe

* fix(fixtures): redact authenticated URLs at diagnostic creation

* fix(ghidra): restrict source-coordinate observations to decompilation

* fix(cli): identify conflicting file-offset selectors accurately

* fix(ghidra): bind file offsets to admitted target sources

* fix(ghidra): preserve resolved file-offset failure context

* test(ghidra): verify ELF address and file-offset round trips

* test(nativeaot): align diagnostic regressions with fixture host support
2026-10-13 18:23:17 +08:00
morluto 1447fd2f99 fix(evaluation): share completed transcript facts across graders 2026-10-13 13:12:12 +08:00
morluto 2976fac30d test(architecture): consolidate import boundary verification 2026-10-12 01:31:33 +08:00
Ysopwnable 844fa52772
fix(evaluation): authenticate delivered Evidence citations
Authenticate completed successful REA Evidence records for citation accounting and shared fixture grading. Preserve genuine structured and complete text-only delivery, including authenticated inventory substeps.

Closes #1728.
2026-10-11 14:48:46 +08:00
Ysopwnable 2653b194b6
feat(skills)!: improve Codex onboarding and task recovery (#1613)
Guide task-centered recovery from observed client, registration, PATH and
provider blockers. Install the recovery reference, document exact JavaScript
module selectors, and require clarification before searching for an unnamed
app. Validate public setup in isolated Codex accounts and preserve failed
or unrecognized tool outcomes in the existing evaluator.

Refs: #1587

BREAKING CHANGE: Optional verify:agent reports now use schemaVersion 4.
Consumers must accept expectedFirstTool: null and targetClarificationPassed
for missing-target scenarios. Production CLI/MCP contracts are unchanged.
2026-10-11 03:13:56 +08:00
morluto 25711ad259 test: exercise source test selection through Git and Vitest 2026-10-10 00:35:17 +08:00
morluto 63bf91405c refactor!: enforce canonical evidence and provider boundaries
Deliver complete Evidence directly through MCP with required nullable analysis profiles and immutable per-server delivery settings. Bind selected environments at construction and require typed cleanup outcomes with retained retry ownership.

Remove test-only routing, duplicate session and snapshot metadata, producer-field repairs, and ambiguous provider arguments. Separate artifact byte identity from occurrence classification and retain typed source-map and partial native facts across CLI, MCP, and verifier consumers.

BREAKING CHANGE: obsolete Evidence envelopes, session aliases, incomplete snapshots, and missing normalized producer facts are no longer accepted.
2026-10-09 20:18:52 +08:00
morluto abc4f47683 build: separate runtime artifacts and supervise command ownership 2026-10-09 20:11:17 +08:00
YouWish 27a9ceade7
fix!: restore caching, preserve failures, and verify fixture claims (#1211)
Restore profile-bound Ghidra snapshot reuse, preserve direct-analysis results through cleanup failures, and grade configured agent fixture claims against returned evidence.

Register the compiled agent-evaluation fixture entry for static analysis.

Co-authored-by: YouWish <29498139+Y0UW1SH@users.noreply.github.com>
Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com>
2026-10-09 14:49:09 +08:00
morluto 5945915f91 test: prune redundant cases and promote configuration lifecycle coverage 2026-10-09 02:58:03 +08:00
Ysopwnable 6e4535c101
refactor(artifacts): move inventory and extraction into provider owners (#1032) 2026-10-08 05:51:14 +08:00
morluto aafdf8a01b test: prune redundant conformance checks and document test priorities 2026-10-08 04:03:24 +08:00
Ysopwnable 6aa9afb549
refactor(process): separate capture implementation from analyst workflows (#988)
* refactor(process): separate capture implementation from analyst workflows

* fix(ci): generate source catalog before shard tests
2026-10-08 00:46:21 +08:00
morluto 0a4caf4a0a
fix(evaluation): canonicalize Unicode argument keys (#980) 2026-10-07 23:19:49 +08:00
Ysopwnable 33d4761cc7
refactor(binary): move production composition outside application workflows (#965) 2026-10-07 21:56:58 +08:00
ysopwnable ef1a0cc65a
refactor(apple): group artifact producers and verification (#952)
* refactor(apple): group artifact producers and verification

* test(apple): run affected native plist CLI regressions
2026-10-07 21:09:33 +08:00
ysopwnable f0a68dce60
refactor(artifacts): isolate JavaScript file acquisition (#916) 2026-10-07 17:57:33 +08:00
ysopwnable b808e2d6bb
refactor(provider): declare deep capabilities from source (#892) 2026-10-07 15:41:50 +08:00
ysopwnable 021ba690b0
refactor: give Inspector its own capability directory (#869) 2026-10-07 13:53:25 +08:00
N0zoM1z0 3427593daf
build: verify resolved module boundaries (#862) 2026-10-07 13:02:28 +08:00
N0zoM1z0 166dcd21d0
feat(javascript): recover readable modules with Wakaru (#848)
* feat(javascript): recover readable modules with Wakaru

Integrate pinned upstream source behind a typed optional recovery port,
preserve source identity and producer provenance, and publish verified
artifacts for existing JavaScript analysis through CLI and MCP.

Add focused real-tool CI with genuine bundler fixtures and bounded
resource, cancellation, cleanup, schema and packaged-artifact checks.

Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis)
Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)

* fix(javascript): align recovery catalog and browser verification

Share canonical Wakaru provider identity, describe CLI mode options, and
format the new workflow test. Disconnect the real page transport after
its observation readiness progress instead of a fixed delay.

Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis)
Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)

* refactor(javascript): consume the investigation evidence writer

Verify that the SDK recovery handler records complete Evidence through the newly shared investigation owner.

Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis)

* chore(test): account for isolated recovery fixture compilers

* fix(javascript): validate recovery paths before acquiring the engine

Return field-specific invalid_format diagnostics for relative paths and verify compiled CLI/MCP parity even when the optional engine is unavailable.

Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis)

Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)
2026-10-07 12:46:13 +08:00
N0zoM1z0 be3cfbffdf
fix(mcp): isolate optional observation adapter failures (#838) 2026-10-07 11:45:17 +08:00
N0zoM1z0 3ed59b2812
refactor: share Android and firmware provider factories (#826) 2026-10-07 11:20:26 +08:00
N0zoM1z0 75f50d86f6
refactor: add incremental capability migration safeguards (#819) 2026-10-07 10:56:06 +08:00
N0zoM1z0 82957c98bf
chore(test): separate development feedback from full verification (#599) 2026-10-06 00:59:26 +08:00
Kaoru0822 205a977777
fix(package): separate agent and Hopper setup support (#584) 2026-10-06 00:00:15 +08:00
morluto 4fbc501121
feat(native): add inspection primitives, dispatch traces and approved UI observation (#500)
* fix(hopper): normalize string-backed addresses and update verification

* feat(native): add inspection primitives, dispatch traces and approved UI observation

* test: prioritize real artifact E2E and captured golden regressions

* fix(process): queue observed output before reactive rendering actions

* test: prune redundant cases and strengthen boundary assertions

* fix(cli): route native value tracing through shared workflows

* test: replace synthetic success with real workflows and captured data

* fix(catalog): separate portable metadata from native host availability
2026-10-04 16:02:36 +08:00
morluto 2a6eb41147 refactor(tools): return complete analysis results inline 2026-09-29 14:53:50 +08:00
morluto 513d9b1555 test: overhaul deterministic suite architecture 2026-07-29 22:00:38 +00:00