docs: public architecture, contributing, testing and agent guides

Public equivalents of the maintainer-local plan/: docs/architecture.md (layers as enforced by
xtask, Tick time base, data model, command system and undo, render/playback/export pipeline,
automation surfaces), docs/contributing.md (+ root CONTRIBUTING.md), docs/testing.md and
docs/agents.md (MCP/control channel usage, agent work loop, decision rules, Definition of Done).
CLAUDE.md points at the public docs wherever it references plan/; AGENTS.md gains a See also list;
README gains a Documentation section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Thomas 2026-09-30 23:24:13 -04:00
parent 5303af9112
commit 4e2a8077d1
9 changed files with 781 additions and 11 deletions

View File

@ -60,3 +60,12 @@ reproduce an image (for example, point lists traced from someone else's icon).
See `CLAUDE.md`: pure Rust, dependency layering (`cargo xtask layers`), exact `Tick` time, everything is See `CLAUDE.md`: pure Rust, dependency layering (`cargo xtask layers`), exact `Tick` time, everything is
a command, everything is agent-drivable, and the quality gates (`cargo xtask ci`) before every commit. a command, everything is agent-drivable, and the quality gates (`cargo xtask ci`) before every commit.
## See also
- [CONTRIBUTING.md](CONTRIBUTING.md) and [docs/contributing.md](docs/contributing.md): setup, gates, commits, how to add things
- [docs/architecture.md](docs/architecture.md): layers, data model, commands, pipeline
- [docs/testing.md](docs/testing.md): oracle tests, criteria, benchmarks
- [docs/agents.md](docs/agents.md): driving FilmCraft over MCP / the control channel, and the agent work loop
- [docs/control-protocol.md](docs/control-protocol.md): control-channel method reference
- [ATTRIBUTION.md](ATTRIBUTION.md): asset index

View File

@ -3,25 +3,24 @@
FilmCraft is a clean-room, open-source, pure-Rust non-linear video editor targeting Adobe Premiere Pro parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../photocraft` (Photoshop), `../printcraft` (Acrobat) and `../drawcraft` (Illustrator), with the same conventions. FilmCraft is a clean-room, open-source, pure-Rust non-linear video editor targeting Adobe Premiere Pro parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../photocraft` (Photoshop), `../printcraft` (Acrobat) and `../drawcraft` (Illustrator), with the same conventions.
## Start every session here ## Start every session here
1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers). `plan/` is maintainer-local (gitignored, not in the repo). Each step names the public equivalent to use if you don't have it.
2. Read the task in `plan/execution-plan.md` §3, the relevant section of `plan/architecture.md`, and the README/docs of the crate you touch. Visual/behaviour reference: `plan/premiere/`. 1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers). Public: [`ROADMAP.md`](ROADMAP.md).
3. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask unless §7 lists the decision as the user's. 2. Read the task in `plan/execution-plan.md` §3, the relevant section of `plan/architecture.md`, and the README/docs of the crate you touch. Public: [`docs/architecture.md`](docs/architecture.md), [`docs/contributing.md`](docs/contributing.md), [`docs/testing.md`](docs/testing.md). Visual/behaviour reference: `plan/premiere/` (local only, never committed).
3. Follow the autonomous operation protocol (`plan/execution-plan.md` §7; public summary: [`docs/agents.md`](docs/agents.md) §5–7). Don't stop to ask unless it lists the decision as the user's.
`plan/` is gitignored (local-only).
## Non-negotiables ## Non-negotiables
**Read [`AGENTS.md`](AGENTS.md) first; its rules override everything here.** In particular (§1): never use Adobe iconography, images or any other Adobe asset. Every asset must be openly licensed (OSS / public domain / Creative Commons, or original work by a contributor) and must have a `<file>.attribution` sidecar plus an entry in `ATTRIBUTION.md`. `cargo xtask assets` enforces this. **Read [`AGENTS.md`](AGENTS.md) first; its rules override everything here.** In particular (§1): never use Adobe iconography, images or any other Adobe asset. Every asset must be openly licensed (OSS / public domain / Creative Commons, or original work by a contributor) and must have a `<file>.attribution` sidecar plus an entry in `ATTRIBUTION.md`. `cargo xtask assets` enforces this.
- **Clean-room.** Never read/disassemble anything inside Adobe app bundles (names/listings only). Never copy Adobe icons, presets, LUTs, fonts. Never copy GPL/LGPL/AGPL code (FFmpeg, x264, x265, MLT, Kdenlive, Shotcut, Olive, LAME…). ffmpeg/ffprobe run only as external test oracles / fixture generators — never linked or shipped. - **Clean-room.** Never read/disassemble anything inside Adobe app bundles (names/listings only). Never copy Adobe icons, presets, LUTs, fonts. Never copy GPL/LGPL/AGPL code (FFmpeg, x264, x265, MLT, Kdenlive, Shotcut, Olive, LAME…). ffmpeg/ffprobe run only as external test oracles / fixture generators — never linked or shipped.
- **Pure Rust** in the product. OS media APIs only via Rust bindings in `crates/platform`, optional, behind traits. - **Pure Rust** in the product. OS media APIs only via Rust bindings in `crates/platform` (planned; not created yet), optional, behind traits.
- **Layering** (`plan/architecture.md` §3, enforced by `cargo xtask layers`): nothing below L5 depends on egui/eframe/winit/rfd/cpal. L0 codec/container crates depend only on `filmcraft-bitstream`. - **Layering** ([`docs/architecture.md`](docs/architecture.md) §1, enforced by `cargo xtask layers`): nothing below L5 depends on egui/eframe/winit/rfd/cpal. L0 codec/container crates depend only on `filmcraft-bitstream`.
- **Exact time:** all time is `filmcraft_time::Tick` (254 016 000 000/s). Never use f64 seconds for edit math. - **Exact time:** all time is `filmcraft_time::Tick` (254 016 000 000/s). Never use f64 seconds for edit math.
- **Everything is a command** (`crates/engine`): id, label, menu path, shortcut, params, enabled(), run(). UI, CLI, control channel and MCP all dispatch by id. - **Everything is a command** (`crates/engine`): id, label, menu path, shortcut, params, enabled(), run(). UI, CLI, control channel and MCP all dispatch by id.
- **Everything is agent-drivable:** every interactive widget registers an automation id; UI state is serde so the control channel can read/set it. - **Everything is agent-drivable:** every interactive widget registers an automation id; UI state is serde so the control channel can read/set it.
- **Quality gates** before every commit: `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace`, `cargo xtask layers`, `cargo xtask assets`, `cargo xtask wasm` (L0–L5 changes). `cargo xtask ci` runs them all. - **Quality gates** before every commit: `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace`, `cargo xtask layers`, `cargo xtask assets`, `cargo xtask wasm` (checks L0–L4 crates). `cargo xtask ci` runs them all.
- **Commits:** one task id per commit (`M2.4: CABAC residual decoding`). Only green states. End messages with the attribution line required by the environment. - **Commits:** one task id per commit (`M2.4: CABAC residual decoding`). Only green states. End messages with the attribution line required by the environment.
## Running and looking at the app ## Running and looking at the app
- `cargo run -p filmcraft -- --control 9876` opens the desktop app with the JSON-lines control server (see `docs/control-protocol.md`). - `cargo run -p filmcraft -- --control 9876` opens the desktop app with the JSON-lines control server (see `docs/control-protocol.md`).
- For UI work, **look at the result**: drive via the control channel and take `ui.screenshot`, compare with `plan/premiere/screenshots/`. - For UI work, **look at the result**: drive via the control channel and take `ui.screenshot`, compare with `plan/premiere/screenshots/` (maintainer-local; see [`docs/agents.md`](docs/agents.md) §3).
- Parallel agents: separate git worktrees and `CARGO_TARGET_DIR=target/agent-<name>`; keep every `Cargo.toml` valid at all times (the `crates/*` glob means one broken manifest breaks everyone). - Parallel agents: separate git worktrees and `CARGO_TARGET_DIR=target/agent-<name>`; keep every `Cargo.toml` valid at all times (the `crates/*` glob means one broken manifest breaks everyone).
- Test fixtures: generate with ffmpeg into `target/fixtures/` (never commit media). - Test fixtures: generate with ffmpeg into `target/fixtures/` (never commit media). See [`docs/testing.md`](docs/testing.md).

15
CONTRIBUTING.md Normal file
View File

@ -0,0 +1,15 @@
# Contributing to FilmCraft
Thanks for helping. Before you start:
1. Read **[AGENTS.md](AGENTS.md)**. Its rules are absolute. In particular: no Adobe assets of any
kind, every asset openly licensed with an attribution sidecar, no GPL/LGPL code, and ffmpeg only
as an external test oracle.
2. Follow **[docs/contributing.md](docs/contributing.md)**: setup per OS, building and running,
the quality gates (`cargo xtask ci`), commit conventions, and how to add commands, effects,
transitions, codecs, panels and assets.
More reading: [docs/architecture.md](docs/architecture.md), [docs/testing.md](docs/testing.md),
[docs/agents.md](docs/agents.md), [ROADMAP.md](ROADMAP.md).
Contributions are licensed under MIT OR Apache-2.0, the project licence.

View File

@ -182,6 +182,18 @@ cargo run --release -p filmcraft-cli -- mcp # MCP server (headles
The control protocol is documented in [docs/control-protocol.md](docs/control-protocol.md). The control protocol is documented in [docs/control-protocol.md](docs/control-protocol.md).
## Documentation
| | |
|---|---|
| [CONTRIBUTING.md](CONTRIBUTING.md) · [docs/contributing.md](docs/contributing.md) | Setup, quality gates, commit conventions, how to add commands, effects, codecs, panels and assets |
| [AGENTS.md](AGENTS.md) | The rules every contributor must follow: assets, clean room, licences |
| [docs/architecture.md](docs/architecture.md) | Layers, data model, time base, command system, render and export pipeline |
| [docs/testing.md](docs/testing.md) | Unit, property and ffmpeg-oracle tests, accuracy criteria, benchmarks |
| [docs/agents.md](docs/agents.md) | Driving FilmCraft over MCP and the control channel; how agents develop it |
| [docs/control-protocol.md](docs/control-protocol.md) | Control-channel and MCP reference |
| [ROADMAP.md](ROADMAP.md) | Milestones and estimates |
## Status ## Status
FilmCraft is young and moving fast: roughly 30% of the way to Premiere Pro parity, with editing, colour, keyframes, codecs and export working today. [ROADMAP.md](ROADMAP.md) tracks every milestone with estimates. FilmCraft is young and moving fast: roughly 30% of the way to Premiere Pro parity, with editing, colour, keyframes, codecs and export working today. [ROADMAP.md](ROADMAP.md) tracks every milestone with estimates.

167
docs/agents.md Normal file
View File

@ -0,0 +1,167 @@
# Agents
FilmCraft is built to be driven by AI agents, and largely built by them. Part 1 covers using
FilmCraft from an agent. Part 2 covers developing FilmCraft as an agent.
# Part 1: driving FilmCraft
## 1. MCP server
`filmcraft-cli mcp` serves MCP on stdio. It has two modes:
| Mode | Command | What it drives |
|---|---|---|
| Headless | `filmcraft-cli mcp --demo` or `--project p.fcproj` (neither = empty project) | an in-process engine session, no window |
| Bridge | `filmcraft-cli mcp --bridge 127.0.0.1:9876` | the running desktop app started with `filmcraft --control 9876` |
**Claude Code.** The repository's `.mcp.json` registers both servers (`filmcraft` = bridge,
`filmcraft-headless` = demo). Both point at `target/release/filmcraft-cli`, so build it first:
```sh
cargo build --release -p filmcraft-cli -p filmcraft
./target/release/filmcraft --control 9876 & # only for the bridge server
claude # approve the project MCP servers when asked
```
To register the server by hand, or for another MCP client, use the absolute binary path:
```sh
claude mcp add filmcraft-headless -- /abs/path/filmcraft/target/release/filmcraft-cli mcp --demo
```
```json
{ "mcpServers": { "filmcraft": { "command": "/abs/path/filmcraft-cli", "args": ["mcp", "--bridge", "127.0.0.1:9876"] } } }
```
### Tools
| Tool | Modes | Purpose |
|---|---|---|
| `command_list` | both | every command: id, label, menu, shortcut, params, enabled now (`filter`, `enabled_only`) |
| `command_run` | both | run a command `{id, params}`; edits are undoable |
| `project_inspect` | both | bins and items with ids, types, durations; active sequence |
| `sequence_inspect` | both | the active sequence: tracks, clips (ticks and frames), effects, transitions, markers, playhead, selection |
| `media_import` | both | import files by absolute path (`text`, one path per line) |
| `render_frame` | both | PNG of the program frame at `seconds` (headless renders; bridge screenshots the Program monitor) |
| `ui_inspect` | bridge | UI state: tool, workspace, panels, zoom, playback, fps |
| `ui_elements` | bridge | on-screen interactive elements with id, label and rect (`prefix` filter) |
| `ui_click` | bridge | click by `id` or `x`,`y`; `button`, `count`, `modifiers` |
| `ui_drag` | bridge | press–move–release between points or element ids |
| `ui_key` | bridge | key or shortcut: `Space`, `Cmd+K`, `Shift+Delete` |
| `ui_type` | bridge | type text into the focused field |
| `ui_screenshot` | bridge | PNG of the window or one `panel` |
| `ui_control` | bridge | call any control-channel method directly (`ui.set`, `ui.scroll`, `ui.timeline.locate`, …) |
Typical loop: `project_inspect` / `sequence_inspect` → get ids → `command_run` → `render_frame` or
`ui_screenshot` → look at the result → `edit.undo` if it's wrong.
## 2. Control channel
`filmcraft --control 9876` (or `FILMCRAFT_CONTROL_PORT=9876`) listens on `127.0.0.1` only. Send one
JSON request per line and get one JSON reply per line. Full method table:
[control-protocol.md](control-protocol.md).
```jsonc
{"id":1,"method":"engine.commands"}
{"id":2,"method":"engine.execute","params":{"command":"file.openDemoProject"}}
{"id":3,"method":"engine.execute","params":{"command":"playhead.set","params":{"seconds":2}}}
{"id":4,"method":"engine.execute","params":{"command":"sequence.addEdit"}}
{"id":5,"method":"engine.execute","params":{"command":"effects.apply","params":{"effect":"Gaussian Blur"}}}
{"id":6,"method":"ui.elements","params":{"prefix":"tools."}}
{"id":7,"method":"ui.click","params":{"id":"tools.Razor"}}
{"id":8,"method":"ui.key","params":{"key":"Cmd+Z"}}
{"id":9,"method":"ui.screenshot","params":{"path":"/tmp/timeline.png","panel":"Timeline"}}
```
Replies look like `{"id":4,"ok":true,"result":{"cuts":2}}` or `{"id":7,"ok":false,"error":"…"}`.
Minimal client:
```python
import json, socket
s = socket.create_connection(("127.0.0.1", 9876)); f = s.makefile("rw")
def call(method, **params):
f.write(json.dumps({"id": 1, "method": method, "params": params}) + "\n"); f.flush()
return json.loads(f.readline())
print(call("engine.execute", command="sequence.inspect"))
```
Notes:
- Time is in ticks (254 016 000 000 per second). Commands also accept `seconds`, `frame` or
`timecode`.
- `engine.execute` also runs UI-only commands (`tool.razor`, `playback.toggle`,
`window.workspace.color`, `window.panel.<name>`).
- Element ids come from the previous frame. If an element is missing right after a layout change,
the app retries on later frames before giving up.
- Without a window, `filmcraft-cli run script.jsonl` (lines of `{"id":"…","params":{…}}`) runs the
same commands headlessly.
## 3. Verifying UI work
For any UI change, look at the result:
1. `cargo run --release -p filmcraft -- --control 9876`
2. Drive the feature through the control channel or the MCP bridge: commands, then clicks and drags
by automation id.
3. Assert on `ui.inspect`, `ui.elements` and `sequence.inspect`.
4. Take `ui.screenshot` (the whole window and the panel you changed) and open the PNG.
Screenshots you commit must follow [AGENTS.md](../AGENTS.md) §1: FilmCraft only, openly licensed
media, sidecar and `ATTRIBUTION.md` entry.
# Part 2: developing FilmCraft
## 4. Orientation, in this order
1. [AGENTS.md](../AGENTS.md): absolute rules (assets, clean-room, licences).
2. [CLAUDE.md](../CLAUDE.md): working instructions and non-negotiables.
3. [ROADMAP.md](../ROADMAP.md): milestones, what's done, what's running.
4. [architecture.md](architecture.md), then the README and tests of the crate you'll touch.
5. [contributing.md](contributing.md) (how to add things, gates) and [testing.md](testing.md).
Maintainers also keep a local planning folder, `plan/`. It is gitignored and not in the repository,
and holds the task-level plan, the status file and behaviour reference notes. If you don't have it,
everything you need to contribute is in the public docs above. Ask a maintainer for a task id.
## 5. Autonomous work loop
1. **Orient.** Pick the next task: the next unchecked task in the maintainer status file, or an open
ROADMAP item. Read the relevant architecture section and crate README.
2. **Plan tests first.** Write down the acceptance test before writing code.
3. **Implement and test.**
4. **Verify.** Run all gates (`cargo xtask ci`). For UI work, run the app with `--control`, drive it
and look at the screenshots (§3).
5. **Record.** Update the crate README (behaviour decisions, test results, limitations) and
ROADMAP.md when a milestone moves. Commit as `M<n>.<k>: …`, then move on to the next task.
For parallel agents, use one git worktree and one `CARGO_TARGET_DIR` per agent, and keep each crate
with one owner ([contributing.md §5](contributing.md#5-parallel-work-several-agents-or-worktrees)).
## 6. Decision rules
Don't block on choices you can make yourself:
- **Behaviour.** Match the publicly documented and observable behaviour of professional editors
(help pages, published shortcut lists, using the app). Never inspect application internals
([AGENTS.md](../AGENTS.md) §2). If behaviour is undocumented, choose what editors most commonly
expect and record the choice in the crate README.
- **Ask a human only about:**
- licences outside the allowlist;
- spending money;
- publishing or pushing;
- personal data;
- destructive git operations;
- anything AGENTS.md leaves unclear ("when in doubt, leave it out").
## 7. Definition of Done
A feature is done when:
- [ ] its command(s) are registered with label, menu path, shortcut (if any) and a parameter doc;
- [ ] engine tests cover it, including undo/redo and the disabled case;
- [ ] the UI is wired: menu, panel control and shortcut;
- [ ] every new interactive widget has an automation id;
- [ ] you have driven it through the control channel and reviewed a screenshot;
- [ ] all gates pass (`cargo xtask ci`), and new assets have sidecars;
- [ ] the crate README and ROADMAP.md are updated where relevant;
- [ ] it is committed with its task id.

261
docs/architecture.md Normal file
View File

@ -0,0 +1,261 @@
# Architecture
FilmCraft is a Cargo workspace of small crates with strictly enforced layering. The engine is
headless: every feature can be reached without a window, and the egui UI is one client among the
CLI, the JSON control channel and the MCP server.
Design principles:
1. **Engine-first.** Project-changing actions go through `Session::execute(id, params)`.
2. **Everything is a command.** Stable id, label, menu path, shortcut, parameter doc, `enabled()`
predicate with a human-readable reason, and `run()`.
3. **Exact time.** Integer ticks, rational frame rates. No `f64` seconds in edit math.
4. **Copy-on-write snapshots.** The project is an `Arc<Project>`. Undo is a stack of snapshots, and
background readers (playback, export) hold a snapshot without locking.
5. **CPU reference, GPU fast path.** The CPU compositor is the oracle. The GPU path is tested
against it.
6. **Pure Rust, clean-room.** Codecs and containers are written from public specifications
(see [AGENTS.md](../AGENTS.md)).
## 1. Layers
```text
L6 apps/filmcraft · apps/filmcraft-cli
L5 ui-egui · automation
L4 engine
L3 render · gpu · export
L2 edit · codecs · interchange
L1 frame · media · project · audio-dsp
L0 foundation: time · geom · color · bitstream
codecs/containers: isobmff · matroska · h264 · h264enc · hevc · prores · aac
```
Crates are named `filmcraft-<dir>` (`crates/time` is `filmcraft-time`). The apps are `filmcraft`
and `filmcraft-cli`.
| Crate | Layer | Purpose |
|---|---|---|
| `time` | L0 | `Tick`, `FrameRate`, `TimeRange`, timecode parse/format (NDF/DF, frames, feet+frames, samples) |
| `geom` | L0 | `Vec2`, `Rect`, `Affine`, Motion-transform composition |
| `color` | L0 | colour spaces, transfer functions, YUV↔RGB matrices, LUTs |
| `bitstream` | L0 | bit reader/writer, Exp-Golomb, emulation prevention |
| `isobmff` | L0 | MP4/MOV demux and mux |
| `matroska` | L0 | MKV/WebM demux |
| `h264`, `h264enc` | L0 | H.264 decoder; H.264 encoder |
| `hevc` | L0 | H.265 Main/Main 10 decoder |
| `prores` | L0 | ProRes decoder and encoder |
| `aac` | L0 | AAC-LC decoder and encoder |
| `frame` | L1 | `VideoFrame` (planar YUV / RGBA8 / linear RGBA f32, colour metadata), `AudioBuffer` |
| `media` | L1 | `MediaSource` trait, probing/openers, frame cache, generators, stills, WAV |
| `project` | L1 | document model, effect definitions, keyframes |
| `audio-dsp` | L1 | loudness metering (BS.1770 / R128) and audio effects; no dependencies |
| `edit` | L2 | pure edit algebra (insert, overwrite, razor, ripple, roll, slip, slide, rate stretch…) |
| `codecs` | L2 | container + codec hub: MP4/MOV and MKV sources, GOP-aware seeking, decoder registry, audio decoding |
| `interchange` | L2 | EDL, FCP7 XML, FCPXML and OTIO import/export (no file I/O) |
| `render` | L3 | sequence evaluation, CPU compositor, video effects, transitions, audio mix |
| `gpu` | L3 | wgpu compositor (WGSL) |
| `export` | L3 | render → encode → mux pipeline, progress/cancel |
| `engine` | L4 | `Session`, command registry, undo history, media pool, jobs, interchange glue |
| `ui-egui` | L5 | the egui frontend: docking, panels, timeline, monitors, playback, control-channel handlers |
| `automation` | L5 | MCP server (`rmcp`, stdio), headless or bridged to the running app |
| `filmcraft` | L6 | desktop binary: eframe/wgpu window, cpal audio output, file dialogs, native macOS menu, TCP control server |
| `filmcraft-cli` | L6 | headless CLI: `probe`, `commands`, `render`, `run`, `mcp` |
### What `cargo xtask layers` enforces
The table of layers lives in `xtask/src/main.rs` (`LAYERS`). It also reserves names for planned
crates. The check reads `cargo metadata` and looks at normal and build dependencies (dev-dependencies
are exempt):
| Rule | Detail |
|---|---|
| Every crate has a layer | A new crate fails the check until it is added to `LAYERS`. |
| Only downward edges | A crate may not depend on a crate in a higher layer. |
| Same-layer edges are listed | From L1 up, a same-layer edge must be in `SAME_LAYER`: `media→frame`, `project→media`, `project→frame`, `gpu→render`, `export→render`, `cli→filmcraft`, plus a few reserved for planned crates. |
| L0 codecs stay standalone | L0 crates other than `time`, `geom`, `color`, `bitstream` may depend on no workspace crate except `filmcraft-bitstream`. External crates such as `thiserror` and `rayon` are allowed. |
| No UI/OS crates below L5 | `egui`, `eframe`, `egui-wgpu`, `winit`, `rfd`, `cpal`, `muda` are allowed only in L5 and L6. |
`cargo xtask wasm` runs `cargo check --target wasm32-unknown-unknown` on every L0–L4 crate, so
everything up to the engine stays web-portable. `unsafe_code = "deny"` applies workspace-wide.
## 2. Time base
All time is `filmcraft_time::Tick(i64)` at `TICKS_PER_SECOND = 254_016_000_000`.
That number divides evenly into the frame duration of every broadcast rate (23.976, 24, 25, 29.97,
30, 48, 50, 59.94, 60, 120…) and the sample duration of every common audio rate (8 kHz to 192 kHz,
including the 44.1 kHz family). So frame and sample positions are exact integers, edits never drift
at 29.97, and audio and video line up to the sample.
| Type | Use |
|---|---|
| `Tick` | timeline and media positions and durations |
| `FrameRate { num, den }` | `frame_duration()`, `tick_of(frame)`, `frame_at(tick)`, `snap(tick)` |
| `TimeRange` | half-open `start + duration` |
| Timecode | display only (SMPTE NDF/DF, frames, feet+frames, samples); `parse_timecode` and `format_time` |
Commands take time as `time` (ticks), `frame`, `seconds` or `timecode`; the engine converts once at
the boundary.
## 3. Data model (`filmcraft-project`)
```text
Project
├─ root: Bin tree of bins
├─ items: map ItemId → ProjectItem flat
│ kind: Media(MediaClip) | Sequence(Sequence) | Subclip{..} | AdjustmentLayer{..}
└─ next_id
Sequence
├─ settings: frame rate, size, sample rate, …
├─ video_tracks / audio_tracks: Vec<Track>
├─ markers, mark_in / mark_out
Track
├─ locked, sync lock, targeting, mute/solo/visibility
├─ items: Vec<TrackItem> sorted, never overlapping
└─ transitions: Vec<Transition>
TrackItem (a clip instance)
├─ item: ItemId, start (timeline ticks), source_in (media ticks), duration, speed
├─ link group, label, enabled
└─ effects: Vec<EffectInstance> intrinsic Motion/Opacity/Volume… first, then standard effects
EffectInstance
└─ effect id, enabled, params: id → constant value or keyframe track
```
- Everything is plain serde data. `Sequence::check()` validates the invariants (no overlaps, unique
ids), and the engine runs it after every sequence edit.
- Timeline positions are sequence ticks; `source_in` and keyframes are in media time, so trims and
splits never move keyframes.
- **Effect definitions are data.** `project::effect::effect_defs()` lists every video effect,
audio effect and transition with its parameter schema. The Effects panel tree, the Effect
Controls rows and the parameter docs agents see are all generated from it.
- **Project files** (`.fcproj`) are the project serialised as JSON. Saves are atomic: write a
temporary sibling file, then rename it over the target.
## 4. Command system (`filmcraft-engine`)
```rust
pub struct CommandSpec {
pub id: &'static str, // "sequence.addEdit"
pub label: &'static str, // "Add Edit"
pub menu: &'static [&'static str], // ["Sequence"]; empty = not in menus
pub shortcut: Option<&'static str>, // "Cmd+K" (Cmd = ⌘ on macOS, Ctrl elsewhere)
pub params: &'static str, // r#"{"time":ticks?}"#, shown to agents
pub enabled: fn(&Session) -> Result<(), String>, // Err carries the reason
pub run: fn(&mut Session, &Value) -> Result<Value>,
pub journal: bool, // false for read-only queries
}
```
- All commands are in `crates/engine/src/commands.rs` (`cmd!` for actions, `query!` for read-only
queries such as `project.inspect`, `sequence.inspect`, `effects.list`, `jobs.list`). Ids follow
the menu structure: `file.*`, `edit.*`, `clip.*`, `sequence.*`, `markers.*`, `timeline.*`,
`effects.*`…
- `Session::execute(id, params)` finds the spec, checks `enabled`, runs it and appends it to the
journal.
- **Undo.** Edits go through `Session::edit(label, |project, state| …)` or
`Session::edit_sequence(label, |seq, ctx, state| …)`. These clone the project, apply the closure,
and on success push the old `Arc<Project>` with the label onto the undo stack (200 entries).
On error nothing changes. `edit.undo` and `edit.redo` swap snapshots. Thanks to structural sharing
a snapshot costs little.
- **Editor state** (`EditorState`: active sequence, playheads, selection, targeting, edit points…)
is serde, so agents can read it with `state.inspect`.
- **Events** (`ProjectChanged`, `Toast`, `OpenSequence`, `OpenSource`) are drained by frontends
each frame.
- **UI-only commands** (tools, playback, zoom, panels, workspaces) live in
`crates/ui-egui/src/menus.rs` (`UI_COMMANDS`). The menu bar is built from the engine registry plus
this table, and `menus::invoke` is the single entry point for menus, shortcuts and the control
channel.
## 5. Media, render and playback pipeline
```text
file ──► codecs (MP4/MOV, MKV, audio) demux + decode, GOP-aware seek
│ decoder registry: h264, hevc, prores, mjpeg (+ any registered first)
▼
media::MediaSource ──► frame cache (byte-budgeted LRU, shared)
▼
render::render_sequence(project, seq, t, scale) CPU reference
per track bottom→top: map timeline t → media t (speed), fetch frame,
standard effects → Motion → Opacity/blend, transitions, composite
in linear-light premultiplied f32
│
└─ render::plan::plan_frame → gpu::GpuCompositor GPU path
layers = decoded YUV/RGBA frames + matrix + opacity;
anything the shaders don't cover is pre-rendered on the CPU
▼
ui-egui frames.rs worker pool ──► monitors (program/source), thumbnails, prefetch
```
- **Sources.** `media::MediaSource` yields `video_frame(FrameRequest)` and
`audio(start, frames, rate)` in media time. Sources are `Send + Sync` and shared by monitors,
thumbnails, playback and export. The engine's `MediaPool` creates one per project item, lazily,
through registered openers (`codecs::openers()`: MP4/MOV, MKV/WebM, audio files).
- **Seeking.** `codecs::Mp4Source` seeks to the preceding sync sample and decodes forward, caching
every frame of the GOP. Sequential playback reuses the decoder. Decoders implement
`codecs::VideoDecoder`. `register_video_decoder` puts a factory in front of the built-in ones, so a
hardware decoder can take precedence.
- **Compositor.** `render` is the reference for monitors, thumbnails and export. `render::plan`
turns a frame into GPU layers. Non-Normal blend modes, standard effects, adjustment layers, nested
sequences and non-dissolve transitions are rendered on the CPU for that layer or frame and handed to
the GPU as an image, so both paths give the same picture. Setting `FILMCRAFT_CPU_COMPOSITE=1`
forces the CPU path in the desktop app.
- **Frame scheduling.** `crates/ui-egui/src/frames.rs` runs a small pool of worker threads with
prioritised jobs: the frame on screen first, then playback prefetch, then thumbnails. The UI never
decodes. It shows the exact frame when it is ready and holds the nearest cached frame meanwhile.
- **Audio clock.** The desktop app passes a cpal output (`apps/filmcraft/src/audio.rs`) to the UI as
`AudioOut`. While playing, the samples played by the sound card drive the playhead and video follows.
Without an audio device, playback falls back to the wall clock. Dropped frames are counted.
Sequence audio is mixed in `render::audio`, and clip audio effects run on `audio-dsp` via
`render::audio_fx`.
## 6. Export jobs (`filmcraft-export`)
```text
file.exportMedia {path, format, scale, audio, quality}
→ engine creates a Job {id, label, progress, result} and runs it on a background thread
→ export: render frames in parallel batches → encode in order → mux; audio mixed per batch
→ jobs.list shows progress; jobs.cancel sets the shared cancel flag
```
| Format | Encoder | Container |
|---|---|---|
| `h264` | `filmcraft-h264enc` + `filmcraft-aac` | MP4 (`isobmff`) |
| `prores` | `filmcraft-prores` | MOV |
| `mjpeg` | built in | MOV |
| `png`, `gif`, `wav` | built in | image sequence / GIF / WAV |
Video encoders implement `export::VideoEncoder`. Codec crates plug in with `register_encoder` and
`register_audio_encoder`.
Timelines can be exchanged as EDL, FCP7 XML, FCPXML or OTIO. `file.import` detects these formats and
merges the result into the project as one undoable step, and `file.exportInterchange`,
`file.exportEdl`, `file.exportFcpxml` and `file.exportOtio` write them.
## 7. Automation surfaces
All of these dispatch the same command ids.
| Surface | Where | Scope |
|---|---|---|
| UI | `ui-egui` menus, shortcuts, panels | `menus::invoke` → engine or UI command |
| CLI | `filmcraft-cli` | `commands`, `run script.jsonl` (one `{"id","params"}` per line), `render`, `probe` |
| Control channel | `filmcraft --control <port>` | JSON lines on loopback TCP: engine commands plus synthetic input, inspection and screenshots of the live UI |
| MCP | `filmcraft-cli mcp` | stdio MCP server: headless in-process session, or `--bridge` to the control channel |
- **Automation ids.** Every interactive widget calls `app.auto.add(id, rect, label)` each frame
(`crates/ui-egui/src/automation.rs`). Agents click by id, e.g. `tools.Razor`,
`timeline.clip.<id>`, `effects.item.gaussian_blur`, `panel.Timeline`.
- **UI state** that is not project data (tool, workspace, dock layout, zoom, scroll, monitor
settings) is in `crates/ui-egui/src/state.rs` as serde structs, so `ui.inspect` and `ui.set` can
read and write it.
Protocol reference: [control-protocol.md](control-protocol.md). Agent guide: [agents.md](agents.md).
## 8. Not built yet
The layer table reserves names for crates that don't exist yet: `riff`, `mjpeg`, `dnx`,
`keyframe`, `effects`, `text`, `audio`, `captions`, `scopes`, `playback`, `format` and `platform`.
Until they exist, that work lives elsewhere: keyframes and effect definitions in `project`, effects
and the audio mix in `render`, scopes and playback in `ui-egui`, and OS integration (cpal, rfd,
native menus) in `apps/filmcraft`. [ROADMAP.md](../ROADMAP.md) has the milestone status.

211
docs/contributing.md Normal file
View File

@ -0,0 +1,211 @@
# Contributing
Read [AGENTS.md](../AGENTS.md) first. Its rules on assets, clean-room code and licences are
absolute and override everything here. Then read [architecture.md](architecture.md) and the README
of the crate you will change.
## 1. Setup
| | All platforms |
|---|---|
| Rust | stable ≥ 1.90 (edition 2024), via [rustup](https://rustup.rs) |
| wasm target | `rustup target add wasm32-unknown-unknown` (for `cargo xtask wasm`) |
| Components | `rustup component add rustfmt clippy` |
| ffmpeg + ffprobe (optional) | test oracle and fixture generator only; tests skip without it |
| OS | Extra |
|---|---|
| macOS | Xcode Command Line Tools. ffmpeg: `brew install ffmpeg` (includes libx264/libx265). |
| Linux | C toolchain and the system headers cpal and rfd need: ALSA (`libasound2-dev`) and GTK 3 (`libgtk-3-dev`), plus the usual X11/Wayland development packages for winit. ffmpeg from your distribution, built with libx264 and libx265. |
| Windows | MSVC build tools. The oracle tests look for ffmpeg only in `/opt/homebrew/bin`, `/usr/local/bin` and `/usr/bin`, so most of them skip on Windows. |
The oracle tests find ffmpeg at `/opt/homebrew/bin`, `/usr/local/bin` or `/usr/bin`. H.264 and
HEVC fixtures need an ffmpeg built with libx264 and libx265. The VideoToolbox fixtures run only on
macOS.
## 2. Build and run
```sh
cargo run --release -p filmcraft # desktop app with the demo project
cargo run --release -p filmcraft -- --empty # start without the demo project
cargo run --release -p filmcraft -- --control 9876 # plus the JSON-lines control server
cargo run --release -p filmcraft -- a.mp4 b.wav # import media (or open a .fcproj)
cargo run --release -p filmcraft-cli -- commands # list engine commands
cargo run --release -p filmcraft-cli -- render --demo --seconds 3 --out frame.png
cargo run --release -p filmcraft-cli -- mcp --demo # headless MCP server on stdio
```
`FILMCRAFT_CONTROL_PORT=9876` works like `--control 9876`, and `FILMCRAFT_CPU_COMPOSITE=1`
disables the GPU compositor.
Dev builds compile dependencies at `opt-level = 2` and workspace crates at `opt-level = 1`. For
playback and codec speed, use `--release`.
## 3. Quality gates
Every commit must pass all of these:
| Gate | Command |
|---|---|
| Format | `cargo fmt --check` (`rustfmt.toml`: `max_width = 160`) |
| Lints | `cargo clippy --workspace --all-targets -- -D warnings` |
| Tests | `cargo test --workspace` |
| Layering | `cargo xtask layers` |
| Assets | `cargo xtask assets` |
| wasm | `cargo xtask wasm` (checks every L0–L4 crate for `wasm32-unknown-unknown`) |
| All of the above | `cargo xtask ci` (runs clippy and tests with `--release`) |
`cargo xtask` is an alias in `.cargo/config.toml` for `cargo run -p xtask --`.
## 4. Commits
- One task per commit. The subject starts with the task id: `M3.2: trim mode engine — …`,
`M9.9: Matroska/WebM import`. Milestones (`M0`–`M16`) are listed in
[ROADMAP.md](../ROADMAP.md). If your change has no task id, use the area: `docs: …`,
`README: …`, `ROADMAP: …`.
- Commit only green states (all gates pass).
- When a milestone lands, update its row in `ROADMAP.md` (status, what's done, estimates).
- AI agents end commit messages with the attribution line their environment requires.
## 5. Parallel work (several agents or worktrees)
```sh
git worktree add ../filmcraft-hevc -b hevc-work
cd ../filmcraft-hevc
export CARGO_TARGET_DIR=target/agent-hevc # separate build dir, no lock fights
```
- One owner per crate at a time.
- The workspace includes `crates/*` and `apps/*` by glob, so one broken `Cargo.toml` breaks every
build. Keep every manifest valid at all times, including half-finished crates.
- Test fixtures always go to `<repo>/target/fixtures/`, whatever `CARGO_TARGET_DIR` says.
- `.mcp.json` points at `target/release/filmcraft-cli`. With a custom target dir, use your own path.
## 6. How to add…
### A command
1. Add a `cmd!` (action) or `query!` (read-only) entry to `build()` in
`crates/engine/src/commands.rs`:
```rust
cmd!("sequence.addEdit", "Add Edit", ["Sequence"], Some("Cmd+K"), r#"{"time":ticks?}"#, has_seq, |s, p| {
let t = time_p(s, p, "").unwrap_or(s.playhead());
let tg = s.targeting().targeted;
let n = s.edit_sequence("Add Edit", |q, ctx, _| Ok(edit::razor(q, &tg, t, ctx)))?;
Ok(json!({"cuts": n.len()}))
}),
```
- Id: `area.camelCase`, following the menu it appears in.
- `enabled`: reuse a predicate (`always`, `has_seq`, `has_selection`, `has_in_out`, …) or write
one that returns `Err("human reason")`.
- Read time with `time_p` so `time`, `frame`, `seconds` and `timecode` all work.
- Mutate only through `s.edit(…)` or `s.edit_sequence(…)`, so the change is undoable and
validated.
- Put pure timeline logic in `filmcraft-edit` as a function on `&mut Sequence`, with unit or
property tests there.
2. Add tests to `crates/engine/src/tests.rs`: execute, assert, undo, redo, and check the disabled
case.
3. Frontend-only actions (tools, zoom, panels) go in `UI_COMMANDS` in
`crates/ui-egui/src/menus.rs` with a branch in `menus::invoke`.
4. The command now appears in menus (from `menu`), shortcuts, `filmcraft-cli commands`,
`engine.commands` and MCP `command_list` without further work.
### A video effect
1. Define it in `build_effects()` in `crates/project/src/effect.rs` with the `video(id, name,
CATEGORY, params)` helper. Parameter helpers cover floats with slider ranges, choices, bools,
angles, points, colours, curves and wheels. The Effects panel, Effect Controls and agent
parameter docs are generated from this.
2. Implement it as a match arm in `render::effects::apply` (`crates/render/src/effects.rs`). It
works on a linear-light premultiplied f32 `Image`. Scale pixel-size parameters by `cx.px_scale`
so reduced-resolution playback matches full resolution. Unknown ids are a no-op.
3. Add a test in `crates/render/src/tests.rs`.
4. The GPU path needs no change: `render::plan` pre-renders layers with standard effects on the CPU.
Audio effects implement `AudioEffect` in `crates/audio-dsp` and register in its `effects()`
registry. Then add an `audio(…)` definition in `effect.rs` and a `Mapping` from project parameters
to DSP parameters in `crates/render/src/audio_fx.rs`. The DSP must give the same output however the
stream is cut into blocks (there are tests for this).
### A transition
1. Define it with `vtrans(id, name, CATEGORY, params)` in `crates/project/src/effect.rs`. Audio
transitions use `EffectKind::AudioTransition`.
2. Implement it in `render::transitions::apply(e, a, b, p)`: outgoing image `a`, incoming image `b`,
progress `p` in 0..1. Audio crossfade curves go in `transitions::audio_gains`.
3. It can then be applied with `sequence.applyVideoTransition {"effect": "<id>"}`.
### A codec or container crate
1. **Spec first.** Work only from the public specification (ITU-T, ISO/IEC, IETF RFCs, SMPTE,
published container specs). Never copy GPL/LGPL code, and follow AGENTS.md §2 on reading other
implementations. Name the spec edition in the crate README.
2. Create `crates/<name>` as an L0 crate that depends only on `filmcraft-bitstream` (plus
`thiserror` and optional `rayon` behind a `threads` feature). Add `[lints] workspace = true`, add
it to `[workspace.dependencies]`, and add it to `LAYERS` in `xtask/src/main.rs`. It must build
for wasm32.
3. Test it against ffmpeg as an oracle (see [testing.md](testing.md)):
- generate fixtures with ffmpeg on first use into `target/fixtures/<crate>/`;
- skip with a message when ffmpeg is missing;
- never commit media.
4. Wire it in:
- decoders: a `VideoDecoderFactory` in `crates/codecs` (`video.rs`, default list in `lib.rs`);
- containers: an `Opener` in `codecs::openers()`;
- encoders: an `export::VideoEncoder` / audio encoder plus a `Format` in `crates/export`.
5. Write the README: features, API, test matrix, accuracy and speed numbers, limitations. Use
`crates/h264/README.md` as the model.
### A panel or widget
- Panels live in `crates/ui-egui/src/panels/` and are listed in `PanelKind`
(`crates/ui-egui/src/dock.rs`).
- Every interactive element registers an automation id each frame:
```rust
app.auto.add(&format!("effects.item.{}", d.id), rect, d.name);
```
Ids are stable, dot-separated and start with the panel (`timeline.track.V1.lock`, `tools.Razor`).
- UI state that should survive or be scriptable goes in `crates/ui-egui/src/state.rs` (serde), so
`ui.inspect` and `ui.set` can reach it.
- Project changes go through `session.execute(…)`, never by mutating the project directly.
- Icons are drawn in code in `crates/ui-egui/src/icons.rs`, from scratch (AGENTS.md §1).
- Look at the result: run with `--control`, drive it, take `ui.screenshot`
([agents.md](agents.md#3-verifying-ui-work)).
### An asset
The rules are in [AGENTS.md §1](../AGENTS.md#1-assets-no-adobe-artwork-every-asset-licensed-and-attributed).
In short:
- No Adobe iconography, images, fonts, LUTs, presets or other Adobe assets, in any form, including
traced or "inspired-by" copies.
- Open licences only (MIT, Apache-2.0, BSD, ISC, zlib, OFL, CC0, CC BY, CC BY-SA, or your own
work under MIT OR Apache-2.0). No NC or ND licences. If the licence is unclear, leave the asset out.
- Screenshots show only FilmCraft (or other open projects) with media we generated or that is
openly licensed.
For each asset file `X`, commit a sidecar `X.attribution`. This is the real one for
`assets/fonts/Inter-Regular.ttf`:
```text
asset: Inter-Regular.ttf
title: Inter Regular (UI typeface)
author: The Inter Project Authors (Rasmus Andersson et al.)
source: https://github.com/rsms/inter
license: OFL-1.1
license-file: assets/fonts/OFL-Inter.txt
added: 2026-09-30 by Claude (agent) for Brandon Thomas
notes: Unmodified font file, embedded in the UI via include_bytes! (crates/ui-egui/src/theme.rs).
```
For your own work, use `source: original work` and the project licence (`MIT OR Apache-2.0`).
Then add a row to [ATTRIBUTION.md](../ATTRIBUTION.md) with the path in backticks. `cargo xtask
assets` checks every tracked file with a media extension (images, fonts, LUTs, audio, video, PDF,
project files…):
- the sidecar exists;
- `asset`, `title`, `author`, `source`, `license` and `added` are filled in;
- the licence is not NC, ND, Adobe or proprietary;
- the path appears in `ATTRIBUTION.md`;
- the path does not mention Adobe or Premiere;
- no sidecar is orphaned.

96
docs/testing.md Normal file
View File

@ -0,0 +1,96 @@
# Testing
```sh
cargo test --workspace # everything; oracle tests skip without ffmpeg
cargo test -p filmcraft-h264 # one crate
cargo test --release -p filmcraft-hevc # codec tests are much faster in release
cargo xtask ci # all gates (fmt, clippy, tests, layers, assets, wasm)
```
## 1. Kinds of tests
| Kind | Where | Examples |
|---|---|---|
| Unit | `src/` `#[cfg(test)]` modules, `src/tests.rs` | CABAC engines, VLC tables (prefix-freeness, Kraft sums), colour matrices, frame cache |
| Property (`proptest`) | `time`, `bitstream`, `edit`, `isobmff/tests/roundtrip.rs`, `interchange/tests/*` | tick↔frame round trips for every rate, DF timecode, edit invariants (no overlaps, durations conserved), mux→demux round trips, interchange export→import |
| Synthetic streams | `h264/src/synth_tests.rs`, `hevc/src/synth_tests.rs` | hand-built bitstreams for features the reference encoders don't emit; expected output known exactly |
| Engine / command | `crates/engine/src/tests.rs` | run commands on the demo project, assert the sequence, undo/redo, disabled cases |
| Render | `crates/render/src/tests.rs` | compositing, opacity, Motion, cross dissolve midpoint, ½-res vs full, GPU plan vs reference, audio mix, audio-effect continuity |
| Oracle | `crates/*/tests/*oracle*.rs`, `conformance.rs` | compare with ffmpeg/ffprobe (§2) |
| Robustness / fuzz | `*/tests/robustness.rs`, `*/tests/fuzz.rs` | seeded mutation and truncation of real and synthetic files; nothing may panic |
| Performance | `*/tests/perf.rs` (`#[ignore]`) | §5 |
Commit `*.proptest-regressions` files so failing cases are re-run.
## 2. ffmpeg oracle tests
ffmpeg and ffprobe are **external processes** used to generate fixtures and to check results. They
are never linked or shipped ([AGENTS.md](../AGENTS.md) §2).
- Each crate's `tests/common/mod.rs` finds the tools in `/opt/homebrew/bin`, `/usr/local/bin` or
`/usr/bin`. If they are missing, the test prints a message and returns (it passes).
- Fixtures are generated on first use into `<repo>/target/fixtures/<crate>/` and reused afterwards.
Delete that directory to regenerate them. Never commit media.
- Fixture sources are synthetic: `testsrc2`, `mandelbrot`, SMPTE bars, noise and fades, sine tones.
H.264 and HEVC fixtures need ffmpeg built with libx264 and libx265. VideoToolbox fixtures are
generated only on macOS.
### Pass criteria per codec
| Crate | Oracle check | Criterion |
|---|---|---|
| `h264` | decode every fixture single-threaded and frame-threaded; compare with `ffmpeg -f rawvideo -pix_fmt yuv420p` | **bit-exact**, every frame |
| `hevc` | same, `yuv420p` / `yuv420p10le` | **bit-exact** (8- and 10-bit) |
| `h264enc` | `ffmpeg -ec 0` decodes our stream | no errors, exact frame count, **bit-identical to the encoder's reconstruction**; B-frame order checked with ffprobe; rate-control targets |
| `prores` decode | ffmpeg `prores_ks` / `prores_aw` fixtures, compared at native depth | within **±1 LSB** (different integer IDCT); alpha bit-exact |
| `prores` encode | ffmpeg decodes with `-xerror` | no errors; agrees with our decoder within ±1 |
| `aac` decode | ffmpeg's decode of the same stream | max abs error ~1e-7 (float) |
| `aac` encode | ffmpeg decodes our stream | no errors, no clipping, CBR within ±5% of target; SNR reported |
| `isobmff` | `ffprobe -show_packets` on ffmpeg-made MP4/MOV | packet offsets, sizes, pts/dts, durations, key flags and stream parameters equal; remuxed files decode with `ffmpeg -v error` silent |
| `matroska` | `ffprobe -show_packets` | every packet (stream, size, key flag, pts, duration) equal; seeks land on the latest keyframe ≤ target |
| `export` | our own demuxer/decoder reads the file back; ffprobe counts frames when present | expected size, duration, colour, audio level; exact frame count |
Each codec README has the full fixture matrix and the measured results.
## 3. Rendering: CPU reference and GPU parity
- The CPU compositor in `filmcraft-render` is the reference. Its tests check computed values
(opacity blends, dissolve midpoints, Motion placement, ½-resolution against downsampled full
resolution) and that `render::plan` matches `render_sequence`.
- `crates/gpu/src/tests.rs` composites the same frame plan (a YUV layer plus a transformed,
semi-transparent RGBA layer) on the GPU and on the CPU. It requires a 99th-percentile channel
difference of ≤ 6 and a mean of < 1.5 (8-bit levels). It skips when no GPU adapter is available.
- Effects implemented only on the CPU need no GPU test: the plan pre-renders those layers on the
CPU.
- Golden-image files (rendered PNGs compared against committed references) are not used yet. If you
add them, generate the references from FilmCraft itself, give each one an attribution sidecar,
and document the tolerance (PSNR in dB, or max/percentile error).
## 4. UI: control channel and screenshots
There is no automated UI test suite yet. UI work is checked by driving the real app:
```sh
cargo run --release -p filmcraft -- --control 9876
```
Then script it over the control channel or MCP: run commands, click by automation id,
`ui.inspect` / `ui.elements` to assert state, and `ui.screenshot {"path": "…"}` to capture the window
or a single panel (`{"panel": "Timeline"}`). Look at every screenshot. Examples are in
[agents.md](agents.md). The headless path (`filmcraft-cli run script.jsonl`, MCP `--demo`) covers
engine behaviour without a window.
## 5. Performance
| Benchmark | Command |
|---|---|
| H.264 decode, 1080p, threads sweep | `cargo test --release -p filmcraft-h264 --test perf -- --ignored --nocapture` |
| HEVC decode, 1080p / 2160p | `cargo test --release -p filmcraft-hevc --test perf -- --ignored --nocapture` |
| ProRes decode/encode | `cargo test --release -p filmcraft-prores --test perf -- --ignored --nocapture` |
| Encoder speed / PSNR / bitrate | `cargo run --release -p filmcraft-h264enc --example h264enc_synth -- …` |
| Decode a real file through the media stack | `cargo run --release -p filmcraft-cli -- bench-decode file.mp4 --frames 120` |
| Coding-tool coverage of the fixtures | `cargo test --release -p filmcraft-h264 --test conformance coverage_report -- --ignored --nocapture` (same for `hevc`) |
The perf tests check bit-exactness before they time anything. Results go in the crate README's
performance table, with machine and thread count. Headline numbers go in
[ROADMAP.md](../ROADMAP.md). Measure on an idle machine: parallel agent builds distort timings.

View File

@ -1,6 +1,6 @@
//! Workspace automation: `cargo xtask <layers|wasm|ci>`. //! Workspace automation: `cargo xtask <layers|wasm|ci>`.
//! //!
//! - `layers`: enforces the dependency layering of `plan/architecture.md` §3 (downward-only edges, //! - `layers`: enforces the dependency layering of `docs/architecture.md` §1 (downward-only edges,
//! listed same-layer edges, L0 codec crates depend on `bitstream` only, no UI/OS crates below L5). //! listed same-layer edges, L0 codec crates depend on `bitstream` only, no UI/OS crates below L5).
//! - `wasm`: `cargo check --target wasm32-unknown-unknown` for every crate in L0–L4. //! - `wasm`: `cargo check --target wasm32-unknown-unknown` for every crate in L0–L4.
//! - `assets`: every asset file (image, icon, font, LUT, audio, video…) has a complete //! - `assets`: every asset file (image, icon, font, LUT, audio, video…) has a complete