docs: public architecture, contributing, testing and agent guides
Public equivalents of the maintainer-local plan/: docs/architecture.md (layers as enforced by xtask, Tick time base, data model, command system and undo, render/playback/export pipeline, automation surfaces), docs/contributing.md (+ root CONTRIBUTING.md), docs/testing.md and docs/agents.md (MCP/control channel usage, agent work loop, decision rules, Definition of Done). CLAUDE.md points at the public docs wherever it references plan/; AGENTS.md gains a See also list; README gains a Documentation section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
5303af9112
commit
4e2a8077d1
|
|
@ -60,3 +60,12 @@ reproduce an image (for example, point lists traced from someone else's icon).
|
||||||
|
|
||||||
See `CLAUDE.md`: pure Rust, dependency layering (`cargo xtask layers`), exact `Tick` time, everything is
|
See `CLAUDE.md`: pure Rust, dependency layering (`cargo xtask layers`), exact `Tick` time, everything is
|
||||||
a command, everything is agent-drivable, and the quality gates (`cargo xtask ci`) before every commit.
|
a command, everything is agent-drivable, and the quality gates (`cargo xtask ci`) before every commit.
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [CONTRIBUTING.md](CONTRIBUTING.md) and [docs/contributing.md](docs/contributing.md): setup, gates, commits, how to add things
|
||||||
|
- [docs/architecture.md](docs/architecture.md): layers, data model, commands, pipeline
|
||||||
|
- [docs/testing.md](docs/testing.md): oracle tests, criteria, benchmarks
|
||||||
|
- [docs/agents.md](docs/agents.md): driving FilmCraft over MCP / the control channel, and the agent work loop
|
||||||
|
- [docs/control-protocol.md](docs/control-protocol.md): control-channel method reference
|
||||||
|
- [ATTRIBUTION.md](ATTRIBUTION.md): asset index
|
||||||
|
|
|
||||||
19
CLAUDE.md
19
CLAUDE.md
|
|
@ -3,25 +3,24 @@
|
||||||
FilmCraft is a clean-room, open-source, pure-Rust non-linear video editor targeting Adobe Premiere Pro parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../photocraft` (Photoshop), `../printcraft` (Acrobat) and `../drawcraft` (Illustrator), with the same conventions.
|
FilmCraft is a clean-room, open-source, pure-Rust non-linear video editor targeting Adobe Premiere Pro parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../photocraft` (Photoshop), `../printcraft` (Acrobat) and `../drawcraft` (Illustrator), with the same conventions.
|
||||||
|
|
||||||
## Start every session here
|
## Start every session here
|
||||||
1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers).
|
`plan/` is maintainer-local (gitignored, not in the repo). Each step names the public equivalent to use if you don't have it.
|
||||||
2. Read the task in `plan/execution-plan.md` §3, the relevant section of `plan/architecture.md`, and the README/docs of the crate you touch. Visual/behaviour reference: `plan/premiere/`.
|
1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers). Public: [`ROADMAP.md`](ROADMAP.md).
|
||||||
3. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask unless §7 lists the decision as the user's.
|
2. Read the task in `plan/execution-plan.md` §3, the relevant section of `plan/architecture.md`, and the README/docs of the crate you touch. Public: [`docs/architecture.md`](docs/architecture.md), [`docs/contributing.md`](docs/contributing.md), [`docs/testing.md`](docs/testing.md). Visual/behaviour reference: `plan/premiere/` (local only, never committed).
|
||||||
|
3. Follow the autonomous operation protocol (`plan/execution-plan.md` §7; public summary: [`docs/agents.md`](docs/agents.md) §5–7). Don't stop to ask unless it lists the decision as the user's.
|
||||||
`plan/` is gitignored (local-only).
|
|
||||||
|
|
||||||
## Non-negotiables
|
## Non-negotiables
|
||||||
**Read [`AGENTS.md`](AGENTS.md) first; its rules override everything here.** In particular (§1): never use Adobe iconography, images or any other Adobe asset. Every asset must be openly licensed (OSS / public domain / Creative Commons, or original work by a contributor) and must have a `<file>.attribution` sidecar plus an entry in `ATTRIBUTION.md`. `cargo xtask assets` enforces this.
|
**Read [`AGENTS.md`](AGENTS.md) first; its rules override everything here.** In particular (§1): never use Adobe iconography, images or any other Adobe asset. Every asset must be openly licensed (OSS / public domain / Creative Commons, or original work by a contributor) and must have a `<file>.attribution` sidecar plus an entry in `ATTRIBUTION.md`. `cargo xtask assets` enforces this.
|
||||||
- **Clean-room.** Never read/disassemble anything inside Adobe app bundles (names/listings only). Never copy Adobe icons, presets, LUTs, fonts. Never copy GPL/LGPL/AGPL code (FFmpeg, x264, x265, MLT, Kdenlive, Shotcut, Olive, LAME…). ffmpeg/ffprobe run only as external test oracles / fixture generators — never linked or shipped.
|
- **Clean-room.** Never read/disassemble anything inside Adobe app bundles (names/listings only). Never copy Adobe icons, presets, LUTs, fonts. Never copy GPL/LGPL/AGPL code (FFmpeg, x264, x265, MLT, Kdenlive, Shotcut, Olive, LAME…). ffmpeg/ffprobe run only as external test oracles / fixture generators — never linked or shipped.
|
||||||
- **Pure Rust** in the product. OS media APIs only via Rust bindings in `crates/platform`, optional, behind traits.
|
- **Pure Rust** in the product. OS media APIs only via Rust bindings in `crates/platform` (planned; not created yet), optional, behind traits.
|
||||||
- **Layering** (`plan/architecture.md` §3, enforced by `cargo xtask layers`): nothing below L5 depends on egui/eframe/winit/rfd/cpal. L0 codec/container crates depend only on `filmcraft-bitstream`.
|
- **Layering** ([`docs/architecture.md`](docs/architecture.md) §1, enforced by `cargo xtask layers`): nothing below L5 depends on egui/eframe/winit/rfd/cpal. L0 codec/container crates depend only on `filmcraft-bitstream`.
|
||||||
- **Exact time:** all time is `filmcraft_time::Tick` (254 016 000 000/s). Never use f64 seconds for edit math.
|
- **Exact time:** all time is `filmcraft_time::Tick` (254 016 000 000/s). Never use f64 seconds for edit math.
|
||||||
- **Everything is a command** (`crates/engine`): id, label, menu path, shortcut, params, enabled(), run(). UI, CLI, control channel and MCP all dispatch by id.
|
- **Everything is a command** (`crates/engine`): id, label, menu path, shortcut, params, enabled(), run(). UI, CLI, control channel and MCP all dispatch by id.
|
||||||
- **Everything is agent-drivable:** every interactive widget registers an automation id; UI state is serde so the control channel can read/set it.
|
- **Everything is agent-drivable:** every interactive widget registers an automation id; UI state is serde so the control channel can read/set it.
|
||||||
- **Quality gates** before every commit: `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace`, `cargo xtask layers`, `cargo xtask assets`, `cargo xtask wasm` (L0–L5 changes). `cargo xtask ci` runs them all.
|
- **Quality gates** before every commit: `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace`, `cargo xtask layers`, `cargo xtask assets`, `cargo xtask wasm` (checks L0–L4 crates). `cargo xtask ci` runs them all.
|
||||||
- **Commits:** one task id per commit (`M2.4: CABAC residual decoding`). Only green states. End messages with the attribution line required by the environment.
|
- **Commits:** one task id per commit (`M2.4: CABAC residual decoding`). Only green states. End messages with the attribution line required by the environment.
|
||||||
|
|
||||||
## Running and looking at the app
|
## Running and looking at the app
|
||||||
- `cargo run -p filmcraft -- --control 9876` opens the desktop app with the JSON-lines control server (see `docs/control-protocol.md`).
|
- `cargo run -p filmcraft -- --control 9876` opens the desktop app with the JSON-lines control server (see `docs/control-protocol.md`).
|
||||||
- For UI work, **look at the result**: drive via the control channel and take `ui.screenshot`, compare with `plan/premiere/screenshots/`.
|
- For UI work, **look at the result**: drive via the control channel and take `ui.screenshot`, compare with `plan/premiere/screenshots/` (maintainer-local; see [`docs/agents.md`](docs/agents.md) §3).
|
||||||
- Parallel agents: separate git worktrees and `CARGO_TARGET_DIR=target/agent-<name>`; keep every `Cargo.toml` valid at all times (the `crates/*` glob means one broken manifest breaks everyone).
|
- Parallel agents: separate git worktrees and `CARGO_TARGET_DIR=target/agent-<name>`; keep every `Cargo.toml` valid at all times (the `crates/*` glob means one broken manifest breaks everyone).
|
||||||
- Test fixtures: generate with ffmpeg into `target/fixtures/` (never commit media).
|
- Test fixtures: generate with ffmpeg into `target/fixtures/` (never commit media). See [`docs/testing.md`](docs/testing.md).
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,15 @@
|
||||||
|
# Contributing to FilmCraft
|
||||||
|
|
||||||
|
Thanks for helping. Before you start:
|
||||||
|
|
||||||
|
1. Read **[AGENTS.md](AGENTS.md)**. Its rules are absolute. In particular: no Adobe assets of any
|
||||||
|
kind, every asset openly licensed with an attribution sidecar, no GPL/LGPL code, and ffmpeg only
|
||||||
|
as an external test oracle.
|
||||||
|
2. Follow **[docs/contributing.md](docs/contributing.md)**: setup per OS, building and running,
|
||||||
|
the quality gates (`cargo xtask ci`), commit conventions, and how to add commands, effects,
|
||||||
|
transitions, codecs, panels and assets.
|
||||||
|
|
||||||
|
More reading: [docs/architecture.md](docs/architecture.md), [docs/testing.md](docs/testing.md),
|
||||||
|
[docs/agents.md](docs/agents.md), [ROADMAP.md](ROADMAP.md).
|
||||||
|
|
||||||
|
Contributions are licensed under MIT OR Apache-2.0, the project licence.
|
||||||
12
README.md
12
README.md
|
|
@ -182,6 +182,18 @@ cargo run --release -p filmcraft-cli -- mcp # MCP server (headles
|
||||||
|
|
||||||
The control protocol is documented in [docs/control-protocol.md](docs/control-protocol.md).
|
The control protocol is documented in [docs/control-protocol.md](docs/control-protocol.md).
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| [CONTRIBUTING.md](CONTRIBUTING.md) · [docs/contributing.md](docs/contributing.md) | Setup, quality gates, commit conventions, how to add commands, effects, codecs, panels and assets |
|
||||||
|
| [AGENTS.md](AGENTS.md) | The rules every contributor must follow: assets, clean room, licences |
|
||||||
|
| [docs/architecture.md](docs/architecture.md) | Layers, data model, time base, command system, render and export pipeline |
|
||||||
|
| [docs/testing.md](docs/testing.md) | Unit, property and ffmpeg-oracle tests, accuracy criteria, benchmarks |
|
||||||
|
| [docs/agents.md](docs/agents.md) | Driving FilmCraft over MCP and the control channel; how agents develop it |
|
||||||
|
| [docs/control-protocol.md](docs/control-protocol.md) | Control-channel and MCP reference |
|
||||||
|
| [ROADMAP.md](ROADMAP.md) | Milestones and estimates |
|
||||||
|
|
||||||
## Status
|
## Status
|
||||||
|
|
||||||
FilmCraft is young and moving fast: roughly 30% of the way to Premiere Pro parity, with editing, colour, keyframes, codecs and export working today. [ROADMAP.md](ROADMAP.md) tracks every milestone with estimates.
|
FilmCraft is young and moving fast: roughly 30% of the way to Premiere Pro parity, with editing, colour, keyframes, codecs and export working today. [ROADMAP.md](ROADMAP.md) tracks every milestone with estimates.
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,167 @@
|
||||||
|
# Agents
|
||||||
|
|
||||||
|
FilmCraft is built to be driven by AI agents, and largely built by them. Part 1 covers using
|
||||||
|
FilmCraft from an agent. Part 2 covers developing FilmCraft as an agent.
|
||||||
|
|
||||||
|
# Part 1: driving FilmCraft
|
||||||
|
|
||||||
|
## 1. MCP server
|
||||||
|
|
||||||
|
`filmcraft-cli mcp` serves MCP on stdio. It has two modes:
|
||||||
|
|
||||||
|
| Mode | Command | What it drives |
|
||||||
|
|---|---|---|
|
||||||
|
| Headless | `filmcraft-cli mcp --demo` or `--project p.fcproj` (neither = empty project) | an in-process engine session, no window |
|
||||||
|
| Bridge | `filmcraft-cli mcp --bridge 127.0.0.1:9876` | the running desktop app started with `filmcraft --control 9876` |
|
||||||
|
|
||||||
|
**Claude Code.** The repository's `.mcp.json` registers both servers (`filmcraft` = bridge,
|
||||||
|
`filmcraft-headless` = demo). Both point at `target/release/filmcraft-cli`, so build it first:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo build --release -p filmcraft-cli -p filmcraft
|
||||||
|
./target/release/filmcraft --control 9876 & # only for the bridge server
|
||||||
|
claude # approve the project MCP servers when asked
|
||||||
|
```
|
||||||
|
|
||||||
|
To register the server by hand, or for another MCP client, use the absolute binary path:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
claude mcp add filmcraft-headless -- /abs/path/filmcraft/target/release/filmcraft-cli mcp --demo
|
||||||
|
```
|
||||||
|
|
||||||
|
```json
|
||||||
|
{ "mcpServers": { "filmcraft": { "command": "/abs/path/filmcraft-cli", "args": ["mcp", "--bridge", "127.0.0.1:9876"] } } }
|
||||||
|
```
|
||||||
|
|
||||||
|
### Tools
|
||||||
|
|
||||||
|
| Tool | Modes | Purpose |
|
||||||
|
|---|---|---|
|
||||||
|
| `command_list` | both | every command: id, label, menu, shortcut, params, enabled now (`filter`, `enabled_only`) |
|
||||||
|
| `command_run` | both | run a command `{id, params}`; edits are undoable |
|
||||||
|
| `project_inspect` | both | bins and items with ids, types, durations; active sequence |
|
||||||
|
| `sequence_inspect` | both | the active sequence: tracks, clips (ticks and frames), effects, transitions, markers, playhead, selection |
|
||||||
|
| `media_import` | both | import files by absolute path (`text`, one path per line) |
|
||||||
|
| `render_frame` | both | PNG of the program frame at `seconds` (headless renders; bridge screenshots the Program monitor) |
|
||||||
|
| `ui_inspect` | bridge | UI state: tool, workspace, panels, zoom, playback, fps |
|
||||||
|
| `ui_elements` | bridge | on-screen interactive elements with id, label and rect (`prefix` filter) |
|
||||||
|
| `ui_click` | bridge | click by `id` or `x`,`y`; `button`, `count`, `modifiers` |
|
||||||
|
| `ui_drag` | bridge | press–move–release between points or element ids |
|
||||||
|
| `ui_key` | bridge | key or shortcut: `Space`, `Cmd+K`, `Shift+Delete` |
|
||||||
|
| `ui_type` | bridge | type text into the focused field |
|
||||||
|
| `ui_screenshot` | bridge | PNG of the window or one `panel` |
|
||||||
|
| `ui_control` | bridge | call any control-channel method directly (`ui.set`, `ui.scroll`, `ui.timeline.locate`, …) |
|
||||||
|
|
||||||
|
Typical loop: `project_inspect` / `sequence_inspect` → get ids → `command_run` → `render_frame` or
|
||||||
|
`ui_screenshot` → look at the result → `edit.undo` if it's wrong.
|
||||||
|
|
||||||
|
## 2. Control channel
|
||||||
|
|
||||||
|
`filmcraft --control 9876` (or `FILMCRAFT_CONTROL_PORT=9876`) listens on `127.0.0.1` only. Send one
|
||||||
|
JSON request per line and get one JSON reply per line. Full method table:
|
||||||
|
[control-protocol.md](control-protocol.md).
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
{"id":1,"method":"engine.commands"}
|
||||||
|
{"id":2,"method":"engine.execute","params":{"command":"file.openDemoProject"}}
|
||||||
|
{"id":3,"method":"engine.execute","params":{"command":"playhead.set","params":{"seconds":2}}}
|
||||||
|
{"id":4,"method":"engine.execute","params":{"command":"sequence.addEdit"}}
|
||||||
|
{"id":5,"method":"engine.execute","params":{"command":"effects.apply","params":{"effect":"Gaussian Blur"}}}
|
||||||
|
{"id":6,"method":"ui.elements","params":{"prefix":"tools."}}
|
||||||
|
{"id":7,"method":"ui.click","params":{"id":"tools.Razor"}}
|
||||||
|
{"id":8,"method":"ui.key","params":{"key":"Cmd+Z"}}
|
||||||
|
{"id":9,"method":"ui.screenshot","params":{"path":"/tmp/timeline.png","panel":"Timeline"}}
|
||||||
|
```
|
||||||
|
|
||||||
|
Replies look like `{"id":4,"ok":true,"result":{"cuts":2}}` or `{"id":7,"ok":false,"error":"…"}`.
|
||||||
|
Minimal client:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import json, socket
|
||||||
|
s = socket.create_connection(("127.0.0.1", 9876)); f = s.makefile("rw")
|
||||||
|
def call(method, **params):
|
||||||
|
f.write(json.dumps({"id": 1, "method": method, "params": params}) + "\n"); f.flush()
|
||||||
|
return json.loads(f.readline())
|
||||||
|
print(call("engine.execute", command="sequence.inspect"))
|
||||||
|
```
|
||||||
|
|
||||||
|
Notes:
|
||||||
|
|
||||||
|
- Time is in ticks (254 016 000 000 per second). Commands also accept `seconds`, `frame` or
|
||||||
|
`timecode`.
|
||||||
|
- `engine.execute` also runs UI-only commands (`tool.razor`, `playback.toggle`,
|
||||||
|
`window.workspace.color`, `window.panel.<name>`).
|
||||||
|
- Element ids come from the previous frame. If an element is missing right after a layout change,
|
||||||
|
the app retries on later frames before giving up.
|
||||||
|
- Without a window, `filmcraft-cli run script.jsonl` (lines of `{"id":"…","params":{…}}`) runs the
|
||||||
|
same commands headlessly.
|
||||||
|
|
||||||
|
## 3. Verifying UI work
|
||||||
|
|
||||||
|
For any UI change, look at the result:
|
||||||
|
|
||||||
|
1. `cargo run --release -p filmcraft -- --control 9876`
|
||||||
|
2. Drive the feature through the control channel or the MCP bridge: commands, then clicks and drags
|
||||||
|
by automation id.
|
||||||
|
3. Assert on `ui.inspect`, `ui.elements` and `sequence.inspect`.
|
||||||
|
4. Take `ui.screenshot` (the whole window and the panel you changed) and open the PNG.
|
||||||
|
|
||||||
|
Screenshots you commit must follow [AGENTS.md](../AGENTS.md) §1: FilmCraft only, openly licensed
|
||||||
|
media, sidecar and `ATTRIBUTION.md` entry.
|
||||||
|
|
||||||
|
# Part 2: developing FilmCraft
|
||||||
|
|
||||||
|
## 4. Orientation, in this order
|
||||||
|
|
||||||
|
1. [AGENTS.md](../AGENTS.md): absolute rules (assets, clean-room, licences).
|
||||||
|
2. [CLAUDE.md](../CLAUDE.md): working instructions and non-negotiables.
|
||||||
|
3. [ROADMAP.md](../ROADMAP.md): milestones, what's done, what's running.
|
||||||
|
4. [architecture.md](architecture.md), then the README and tests of the crate you'll touch.
|
||||||
|
5. [contributing.md](contributing.md) (how to add things, gates) and [testing.md](testing.md).
|
||||||
|
|
||||||
|
Maintainers also keep a local planning folder, `plan/`. It is gitignored and not in the repository,
|
||||||
|
and holds the task-level plan, the status file and behaviour reference notes. If you don't have it,
|
||||||
|
everything you need to contribute is in the public docs above. Ask a maintainer for a task id.
|
||||||
|
|
||||||
|
## 5. Autonomous work loop
|
||||||
|
|
||||||
|
1. **Orient.** Pick the next task: the next unchecked task in the maintainer status file, or an open
|
||||||
|
ROADMAP item. Read the relevant architecture section and crate README.
|
||||||
|
2. **Plan tests first.** Write down the acceptance test before writing code.
|
||||||
|
3. **Implement and test.**
|
||||||
|
4. **Verify.** Run all gates (`cargo xtask ci`). For UI work, run the app with `--control`, drive it
|
||||||
|
and look at the screenshots (§3).
|
||||||
|
5. **Record.** Update the crate README (behaviour decisions, test results, limitations) and
|
||||||
|
ROADMAP.md when a milestone moves. Commit as `M<n>.<k>: …`, then move on to the next task.
|
||||||
|
|
||||||
|
For parallel agents, use one git worktree and one `CARGO_TARGET_DIR` per agent, and keep each crate
|
||||||
|
with one owner ([contributing.md §5](contributing.md#5-parallel-work-several-agents-or-worktrees)).
|
||||||
|
|
||||||
|
## 6. Decision rules
|
||||||
|
|
||||||
|
Don't block on choices you can make yourself:
|
||||||
|
|
||||||
|
- **Behaviour.** Match the publicly documented and observable behaviour of professional editors
|
||||||
|
(help pages, published shortcut lists, using the app). Never inspect application internals
|
||||||
|
([AGENTS.md](../AGENTS.md) §2). If behaviour is undocumented, choose what editors most commonly
|
||||||
|
expect and record the choice in the crate README.
|
||||||
|
- **Ask a human only about:**
|
||||||
|
- licences outside the allowlist;
|
||||||
|
- spending money;
|
||||||
|
- publishing or pushing;
|
||||||
|
- personal data;
|
||||||
|
- destructive git operations;
|
||||||
|
- anything AGENTS.md leaves unclear ("when in doubt, leave it out").
|
||||||
|
|
||||||
|
## 7. Definition of Done
|
||||||
|
|
||||||
|
A feature is done when:
|
||||||
|
|
||||||
|
- [ ] its command(s) are registered with label, menu path, shortcut (if any) and a parameter doc;
|
||||||
|
- [ ] engine tests cover it, including undo/redo and the disabled case;
|
||||||
|
- [ ] the UI is wired: menu, panel control and shortcut;
|
||||||
|
- [ ] every new interactive widget has an automation id;
|
||||||
|
- [ ] you have driven it through the control channel and reviewed a screenshot;
|
||||||
|
- [ ] all gates pass (`cargo xtask ci`), and new assets have sidecars;
|
||||||
|
- [ ] the crate README and ROADMAP.md are updated where relevant;
|
||||||
|
- [ ] it is committed with its task id.
|
||||||
|
|
@ -0,0 +1,261 @@
|
||||||
|
# Architecture
|
||||||
|
|
||||||
|
FilmCraft is a Cargo workspace of small crates with strictly enforced layering. The engine is
|
||||||
|
headless: every feature can be reached without a window, and the egui UI is one client among the
|
||||||
|
CLI, the JSON control channel and the MCP server.
|
||||||
|
|
||||||
|
Design principles:
|
||||||
|
|
||||||
|
1. **Engine-first.** Project-changing actions go through `Session::execute(id, params)`.
|
||||||
|
2. **Everything is a command.** Stable id, label, menu path, shortcut, parameter doc, `enabled()`
|
||||||
|
predicate with a human-readable reason, and `run()`.
|
||||||
|
3. **Exact time.** Integer ticks, rational frame rates. No `f64` seconds in edit math.
|
||||||
|
4. **Copy-on-write snapshots.** The project is an `Arc<Project>`. Undo is a stack of snapshots, and
|
||||||
|
background readers (playback, export) hold a snapshot without locking.
|
||||||
|
5. **CPU reference, GPU fast path.** The CPU compositor is the oracle. The GPU path is tested
|
||||||
|
against it.
|
||||||
|
6. **Pure Rust, clean-room.** Codecs and containers are written from public specifications
|
||||||
|
(see [AGENTS.md](../AGENTS.md)).
|
||||||
|
|
||||||
|
## 1. Layers
|
||||||
|
|
||||||
|
```text
|
||||||
|
L6 apps/filmcraft · apps/filmcraft-cli
|
||||||
|
L5 ui-egui · automation
|
||||||
|
L4 engine
|
||||||
|
L3 render · gpu · export
|
||||||
|
L2 edit · codecs · interchange
|
||||||
|
L1 frame · media · project · audio-dsp
|
||||||
|
L0 foundation: time · geom · color · bitstream
|
||||||
|
codecs/containers: isobmff · matroska · h264 · h264enc · hevc · prores · aac
|
||||||
|
```
|
||||||
|
|
||||||
|
Crates are named `filmcraft-<dir>` (`crates/time` is `filmcraft-time`). The apps are `filmcraft`
|
||||||
|
and `filmcraft-cli`.
|
||||||
|
|
||||||
|
| Crate | Layer | Purpose |
|
||||||
|
|---|---|---|
|
||||||
|
| `time` | L0 | `Tick`, `FrameRate`, `TimeRange`, timecode parse/format (NDF/DF, frames, feet+frames, samples) |
|
||||||
|
| `geom` | L0 | `Vec2`, `Rect`, `Affine`, Motion-transform composition |
|
||||||
|
| `color` | L0 | colour spaces, transfer functions, YUV↔RGB matrices, LUTs |
|
||||||
|
| `bitstream` | L0 | bit reader/writer, Exp-Golomb, emulation prevention |
|
||||||
|
| `isobmff` | L0 | MP4/MOV demux and mux |
|
||||||
|
| `matroska` | L0 | MKV/WebM demux |
|
||||||
|
| `h264`, `h264enc` | L0 | H.264 decoder; H.264 encoder |
|
||||||
|
| `hevc` | L0 | H.265 Main/Main 10 decoder |
|
||||||
|
| `prores` | L0 | ProRes decoder and encoder |
|
||||||
|
| `aac` | L0 | AAC-LC decoder and encoder |
|
||||||
|
| `frame` | L1 | `VideoFrame` (planar YUV / RGBA8 / linear RGBA f32, colour metadata), `AudioBuffer` |
|
||||||
|
| `media` | L1 | `MediaSource` trait, probing/openers, frame cache, generators, stills, WAV |
|
||||||
|
| `project` | L1 | document model, effect definitions, keyframes |
|
||||||
|
| `audio-dsp` | L1 | loudness metering (BS.1770 / R128) and audio effects; no dependencies |
|
||||||
|
| `edit` | L2 | pure edit algebra (insert, overwrite, razor, ripple, roll, slip, slide, rate stretch…) |
|
||||||
|
| `codecs` | L2 | container + codec hub: MP4/MOV and MKV sources, GOP-aware seeking, decoder registry, audio decoding |
|
||||||
|
| `interchange` | L2 | EDL, FCP7 XML, FCPXML and OTIO import/export (no file I/O) |
|
||||||
|
| `render` | L3 | sequence evaluation, CPU compositor, video effects, transitions, audio mix |
|
||||||
|
| `gpu` | L3 | wgpu compositor (WGSL) |
|
||||||
|
| `export` | L3 | render → encode → mux pipeline, progress/cancel |
|
||||||
|
| `engine` | L4 | `Session`, command registry, undo history, media pool, jobs, interchange glue |
|
||||||
|
| `ui-egui` | L5 | the egui frontend: docking, panels, timeline, monitors, playback, control-channel handlers |
|
||||||
|
| `automation` | L5 | MCP server (`rmcp`, stdio), headless or bridged to the running app |
|
||||||
|
| `filmcraft` | L6 | desktop binary: eframe/wgpu window, cpal audio output, file dialogs, native macOS menu, TCP control server |
|
||||||
|
| `filmcraft-cli` | L6 | headless CLI: `probe`, `commands`, `render`, `run`, `mcp` |
|
||||||
|
|
||||||
|
### What `cargo xtask layers` enforces
|
||||||
|
|
||||||
|
The table of layers lives in `xtask/src/main.rs` (`LAYERS`). It also reserves names for planned
|
||||||
|
crates. The check reads `cargo metadata` and looks at normal and build dependencies (dev-dependencies
|
||||||
|
are exempt):
|
||||||
|
|
||||||
|
| Rule | Detail |
|
||||||
|
|---|---|
|
||||||
|
| Every crate has a layer | A new crate fails the check until it is added to `LAYERS`. |
|
||||||
|
| Only downward edges | A crate may not depend on a crate in a higher layer. |
|
||||||
|
| Same-layer edges are listed | From L1 up, a same-layer edge must be in `SAME_LAYER`: `media→frame`, `project→media`, `project→frame`, `gpu→render`, `export→render`, `cli→filmcraft`, plus a few reserved for planned crates. |
|
||||||
|
| L0 codecs stay standalone | L0 crates other than `time`, `geom`, `color`, `bitstream` may depend on no workspace crate except `filmcraft-bitstream`. External crates such as `thiserror` and `rayon` are allowed. |
|
||||||
|
| No UI/OS crates below L5 | `egui`, `eframe`, `egui-wgpu`, `winit`, `rfd`, `cpal`, `muda` are allowed only in L5 and L6. |
|
||||||
|
|
||||||
|
`cargo xtask wasm` runs `cargo check --target wasm32-unknown-unknown` on every L0–L4 crate, so
|
||||||
|
everything up to the engine stays web-portable. `unsafe_code = "deny"` applies workspace-wide.
|
||||||
|
|
||||||
|
## 2. Time base
|
||||||
|
|
||||||
|
All time is `filmcraft_time::Tick(i64)` at `TICKS_PER_SECOND = 254_016_000_000`.
|
||||||
|
|
||||||
|
That number divides evenly into the frame duration of every broadcast rate (23.976, 24, 25, 29.97,
|
||||||
|
30, 48, 50, 59.94, 60, 120…) and the sample duration of every common audio rate (8 kHz to 192 kHz,
|
||||||
|
including the 44.1 kHz family). So frame and sample positions are exact integers, edits never drift
|
||||||
|
at 29.97, and audio and video line up to the sample.
|
||||||
|
|
||||||
|
| Type | Use |
|
||||||
|
|---|---|
|
||||||
|
| `Tick` | timeline and media positions and durations |
|
||||||
|
| `FrameRate { num, den }` | `frame_duration()`, `tick_of(frame)`, `frame_at(tick)`, `snap(tick)` |
|
||||||
|
| `TimeRange` | half-open `start + duration` |
|
||||||
|
| Timecode | display only (SMPTE NDF/DF, frames, feet+frames, samples); `parse_timecode` and `format_time` |
|
||||||
|
|
||||||
|
Commands take time as `time` (ticks), `frame`, `seconds` or `timecode`; the engine converts once at
|
||||||
|
the boundary.
|
||||||
|
|
||||||
|
## 3. Data model (`filmcraft-project`)
|
||||||
|
|
||||||
|
```text
|
||||||
|
Project
|
||||||
|
├─ root: Bin tree of bins
|
||||||
|
├─ items: map ItemId → ProjectItem flat
|
||||||
|
│ kind: Media(MediaClip) | Sequence(Sequence) | Subclip{..} | AdjustmentLayer{..}
|
||||||
|
└─ next_id
|
||||||
|
|
||||||
|
Sequence
|
||||||
|
├─ settings: frame rate, size, sample rate, …
|
||||||
|
├─ video_tracks / audio_tracks: Vec<Track>
|
||||||
|
├─ markers, mark_in / mark_out
|
||||||
|
Track
|
||||||
|
├─ locked, sync lock, targeting, mute/solo/visibility
|
||||||
|
├─ items: Vec<TrackItem> sorted, never overlapping
|
||||||
|
└─ transitions: Vec<Transition>
|
||||||
|
TrackItem (a clip instance)
|
||||||
|
├─ item: ItemId, start (timeline ticks), source_in (media ticks), duration, speed
|
||||||
|
├─ link group, label, enabled
|
||||||
|
└─ effects: Vec<EffectInstance> intrinsic Motion/Opacity/Volume… first, then standard effects
|
||||||
|
EffectInstance
|
||||||
|
└─ effect id, enabled, params: id → constant value or keyframe track
|
||||||
|
```
|
||||||
|
|
||||||
|
- Everything is plain serde data. `Sequence::check()` validates the invariants (no overlaps, unique
|
||||||
|
ids), and the engine runs it after every sequence edit.
|
||||||
|
- Timeline positions are sequence ticks; `source_in` and keyframes are in media time, so trims and
|
||||||
|
splits never move keyframes.
|
||||||
|
- **Effect definitions are data.** `project::effect::effect_defs()` lists every video effect,
|
||||||
|
audio effect and transition with its parameter schema. The Effects panel tree, the Effect
|
||||||
|
Controls rows and the parameter docs agents see are all generated from it.
|
||||||
|
- **Project files** (`.fcproj`) are the project serialised as JSON. Saves are atomic: write a
|
||||||
|
temporary sibling file, then rename it over the target.
|
||||||
|
|
||||||
|
## 4. Command system (`filmcraft-engine`)
|
||||||
|
|
||||||
|
```rust
|
||||||
|
pub struct CommandSpec {
|
||||||
|
pub id: &'static str, // "sequence.addEdit"
|
||||||
|
pub label: &'static str, // "Add Edit"
|
||||||
|
pub menu: &'static [&'static str], // ["Sequence"]; empty = not in menus
|
||||||
|
pub shortcut: Option<&'static str>, // "Cmd+K" (Cmd = ⌘ on macOS, Ctrl elsewhere)
|
||||||
|
pub params: &'static str, // r#"{"time":ticks?}"#, shown to agents
|
||||||
|
pub enabled: fn(&Session) -> Result<(), String>, // Err carries the reason
|
||||||
|
pub run: fn(&mut Session, &Value) -> Result<Value>,
|
||||||
|
pub journal: bool, // false for read-only queries
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- All commands are in `crates/engine/src/commands.rs` (`cmd!` for actions, `query!` for read-only
|
||||||
|
queries such as `project.inspect`, `sequence.inspect`, `effects.list`, `jobs.list`). Ids follow
|
||||||
|
the menu structure: `file.*`, `edit.*`, `clip.*`, `sequence.*`, `markers.*`, `timeline.*`,
|
||||||
|
`effects.*`…
|
||||||
|
- `Session::execute(id, params)` finds the spec, checks `enabled`, runs it and appends it to the
|
||||||
|
journal.
|
||||||
|
- **Undo.** Edits go through `Session::edit(label, |project, state| …)` or
|
||||||
|
`Session::edit_sequence(label, |seq, ctx, state| …)`. These clone the project, apply the closure,
|
||||||
|
and on success push the old `Arc<Project>` with the label onto the undo stack (200 entries).
|
||||||
|
On error nothing changes. `edit.undo` and `edit.redo` swap snapshots. Thanks to structural sharing
|
||||||
|
a snapshot costs little.
|
||||||
|
- **Editor state** (`EditorState`: active sequence, playheads, selection, targeting, edit points…)
|
||||||
|
is serde, so agents can read it with `state.inspect`.
|
||||||
|
- **Events** (`ProjectChanged`, `Toast`, `OpenSequence`, `OpenSource`) are drained by frontends
|
||||||
|
each frame.
|
||||||
|
- **UI-only commands** (tools, playback, zoom, panels, workspaces) live in
|
||||||
|
`crates/ui-egui/src/menus.rs` (`UI_COMMANDS`). The menu bar is built from the engine registry plus
|
||||||
|
this table, and `menus::invoke` is the single entry point for menus, shortcuts and the control
|
||||||
|
channel.
|
||||||
|
|
||||||
|
## 5. Media, render and playback pipeline
|
||||||
|
|
||||||
|
```text
|
||||||
|
file ──► codecs (MP4/MOV, MKV, audio) demux + decode, GOP-aware seek
|
||||||
|
│ decoder registry: h264, hevc, prores, mjpeg (+ any registered first)
|
||||||
|
▼
|
||||||
|
media::MediaSource ──► frame cache (byte-budgeted LRU, shared)
|
||||||
|
▼
|
||||||
|
render::render_sequence(project, seq, t, scale) CPU reference
|
||||||
|
per track bottom→top: map timeline t → media t (speed), fetch frame,
|
||||||
|
standard effects → Motion → Opacity/blend, transitions, composite
|
||||||
|
in linear-light premultiplied f32
|
||||||
|
│
|
||||||
|
└─ render::plan::plan_frame → gpu::GpuCompositor GPU path
|
||||||
|
layers = decoded YUV/RGBA frames + matrix + opacity;
|
||||||
|
anything the shaders don't cover is pre-rendered on the CPU
|
||||||
|
▼
|
||||||
|
ui-egui frames.rs worker pool ──► monitors (program/source), thumbnails, prefetch
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Sources.** `media::MediaSource` yields `video_frame(FrameRequest)` and
|
||||||
|
`audio(start, frames, rate)` in media time. Sources are `Send + Sync` and shared by monitors,
|
||||||
|
thumbnails, playback and export. The engine's `MediaPool` creates one per project item, lazily,
|
||||||
|
through registered openers (`codecs::openers()`: MP4/MOV, MKV/WebM, audio files).
|
||||||
|
- **Seeking.** `codecs::Mp4Source` seeks to the preceding sync sample and decodes forward, caching
|
||||||
|
every frame of the GOP. Sequential playback reuses the decoder. Decoders implement
|
||||||
|
`codecs::VideoDecoder`. `register_video_decoder` puts a factory in front of the built-in ones, so a
|
||||||
|
hardware decoder can take precedence.
|
||||||
|
- **Compositor.** `render` is the reference for monitors, thumbnails and export. `render::plan`
|
||||||
|
turns a frame into GPU layers. Non-Normal blend modes, standard effects, adjustment layers, nested
|
||||||
|
sequences and non-dissolve transitions are rendered on the CPU for that layer or frame and handed to
|
||||||
|
the GPU as an image, so both paths give the same picture. Setting `FILMCRAFT_CPU_COMPOSITE=1`
|
||||||
|
forces the CPU path in the desktop app.
|
||||||
|
- **Frame scheduling.** `crates/ui-egui/src/frames.rs` runs a small pool of worker threads with
|
||||||
|
prioritised jobs: the frame on screen first, then playback prefetch, then thumbnails. The UI never
|
||||||
|
decodes. It shows the exact frame when it is ready and holds the nearest cached frame meanwhile.
|
||||||
|
- **Audio clock.** The desktop app passes a cpal output (`apps/filmcraft/src/audio.rs`) to the UI as
|
||||||
|
`AudioOut`. While playing, the samples played by the sound card drive the playhead and video follows.
|
||||||
|
Without an audio device, playback falls back to the wall clock. Dropped frames are counted.
|
||||||
|
Sequence audio is mixed in `render::audio`, and clip audio effects run on `audio-dsp` via
|
||||||
|
`render::audio_fx`.
|
||||||
|
|
||||||
|
## 6. Export jobs (`filmcraft-export`)
|
||||||
|
|
||||||
|
```text
|
||||||
|
file.exportMedia {path, format, scale, audio, quality}
|
||||||
|
→ engine creates a Job {id, label, progress, result} and runs it on a background thread
|
||||||
|
→ export: render frames in parallel batches → encode in order → mux; audio mixed per batch
|
||||||
|
→ jobs.list shows progress; jobs.cancel sets the shared cancel flag
|
||||||
|
```
|
||||||
|
|
||||||
|
| Format | Encoder | Container |
|
||||||
|
|---|---|---|
|
||||||
|
| `h264` | `filmcraft-h264enc` + `filmcraft-aac` | MP4 (`isobmff`) |
|
||||||
|
| `prores` | `filmcraft-prores` | MOV |
|
||||||
|
| `mjpeg` | built in | MOV |
|
||||||
|
| `png`, `gif`, `wav` | built in | image sequence / GIF / WAV |
|
||||||
|
|
||||||
|
Video encoders implement `export::VideoEncoder`. Codec crates plug in with `register_encoder` and
|
||||||
|
`register_audio_encoder`.
|
||||||
|
|
||||||
|
Timelines can be exchanged as EDL, FCP7 XML, FCPXML or OTIO. `file.import` detects these formats and
|
||||||
|
merges the result into the project as one undoable step, and `file.exportInterchange`,
|
||||||
|
`file.exportEdl`, `file.exportFcpxml` and `file.exportOtio` write them.
|
||||||
|
|
||||||
|
## 7. Automation surfaces
|
||||||
|
|
||||||
|
All of these dispatch the same command ids.
|
||||||
|
|
||||||
|
| Surface | Where | Scope |
|
||||||
|
|---|---|---|
|
||||||
|
| UI | `ui-egui` menus, shortcuts, panels | `menus::invoke` → engine or UI command |
|
||||||
|
| CLI | `filmcraft-cli` | `commands`, `run script.jsonl` (one `{"id","params"}` per line), `render`, `probe` |
|
||||||
|
| Control channel | `filmcraft --control <port>` | JSON lines on loopback TCP: engine commands plus synthetic input, inspection and screenshots of the live UI |
|
||||||
|
| MCP | `filmcraft-cli mcp` | stdio MCP server: headless in-process session, or `--bridge` to the control channel |
|
||||||
|
|
||||||
|
- **Automation ids.** Every interactive widget calls `app.auto.add(id, rect, label)` each frame
|
||||||
|
(`crates/ui-egui/src/automation.rs`). Agents click by id, e.g. `tools.Razor`,
|
||||||
|
`timeline.clip.<id>`, `effects.item.gaussian_blur`, `panel.Timeline`.
|
||||||
|
- **UI state** that is not project data (tool, workspace, dock layout, zoom, scroll, monitor
|
||||||
|
settings) is in `crates/ui-egui/src/state.rs` as serde structs, so `ui.inspect` and `ui.set` can
|
||||||
|
read and write it.
|
||||||
|
|
||||||
|
Protocol reference: [control-protocol.md](control-protocol.md). Agent guide: [agents.md](agents.md).
|
||||||
|
|
||||||
|
## 8. Not built yet
|
||||||
|
|
||||||
|
The layer table reserves names for crates that don't exist yet: `riff`, `mjpeg`, `dnx`,
|
||||||
|
`keyframe`, `effects`, `text`, `audio`, `captions`, `scopes`, `playback`, `format` and `platform`.
|
||||||
|
Until they exist, that work lives elsewhere: keyframes and effect definitions in `project`, effects
|
||||||
|
and the audio mix in `render`, scopes and playback in `ui-egui`, and OS integration (cpal, rfd,
|
||||||
|
native menus) in `apps/filmcraft`. [ROADMAP.md](../ROADMAP.md) has the milestone status.
|
||||||
|
|
@ -0,0 +1,211 @@
|
||||||
|
# Contributing
|
||||||
|
|
||||||
|
Read [AGENTS.md](../AGENTS.md) first. Its rules on assets, clean-room code and licences are
|
||||||
|
absolute and override everything here. Then read [architecture.md](architecture.md) and the README
|
||||||
|
of the crate you will change.
|
||||||
|
|
||||||
|
## 1. Setup
|
||||||
|
|
||||||
|
| | All platforms |
|
||||||
|
|---|---|
|
||||||
|
| Rust | stable ≥ 1.90 (edition 2024), via [rustup](https://rustup.rs) |
|
||||||
|
| wasm target | `rustup target add wasm32-unknown-unknown` (for `cargo xtask wasm`) |
|
||||||
|
| Components | `rustup component add rustfmt clippy` |
|
||||||
|
| ffmpeg + ffprobe (optional) | test oracle and fixture generator only; tests skip without it |
|
||||||
|
|
||||||
|
| OS | Extra |
|
||||||
|
|---|---|
|
||||||
|
| macOS | Xcode Command Line Tools. ffmpeg: `brew install ffmpeg` (includes libx264/libx265). |
|
||||||
|
| Linux | C toolchain and the system headers cpal and rfd need: ALSA (`libasound2-dev`) and GTK 3 (`libgtk-3-dev`), plus the usual X11/Wayland development packages for winit. ffmpeg from your distribution, built with libx264 and libx265. |
|
||||||
|
| Windows | MSVC build tools. The oracle tests look for ffmpeg only in `/opt/homebrew/bin`, `/usr/local/bin` and `/usr/bin`, so most of them skip on Windows. |
|
||||||
|
|
||||||
|
The oracle tests find ffmpeg at `/opt/homebrew/bin`, `/usr/local/bin` or `/usr/bin`. H.264 and
|
||||||
|
HEVC fixtures need an ffmpeg built with libx264 and libx265. The VideoToolbox fixtures run only on
|
||||||
|
macOS.
|
||||||
|
|
||||||
|
## 2. Build and run
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo run --release -p filmcraft # desktop app with the demo project
|
||||||
|
cargo run --release -p filmcraft -- --empty # start without the demo project
|
||||||
|
cargo run --release -p filmcraft -- --control 9876 # plus the JSON-lines control server
|
||||||
|
cargo run --release -p filmcraft -- a.mp4 b.wav # import media (or open a .fcproj)
|
||||||
|
cargo run --release -p filmcraft-cli -- commands # list engine commands
|
||||||
|
cargo run --release -p filmcraft-cli -- render --demo --seconds 3 --out frame.png
|
||||||
|
cargo run --release -p filmcraft-cli -- mcp --demo # headless MCP server on stdio
|
||||||
|
```
|
||||||
|
|
||||||
|
`FILMCRAFT_CONTROL_PORT=9876` works like `--control 9876`, and `FILMCRAFT_CPU_COMPOSITE=1`
|
||||||
|
disables the GPU compositor.
|
||||||
|
|
||||||
|
Dev builds compile dependencies at `opt-level = 2` and workspace crates at `opt-level = 1`. For
|
||||||
|
playback and codec speed, use `--release`.
|
||||||
|
|
||||||
|
## 3. Quality gates
|
||||||
|
|
||||||
|
Every commit must pass all of these:
|
||||||
|
|
||||||
|
| Gate | Command |
|
||||||
|
|---|---|
|
||||||
|
| Format | `cargo fmt --check` (`rustfmt.toml`: `max_width = 160`) |
|
||||||
|
| Lints | `cargo clippy --workspace --all-targets -- -D warnings` |
|
||||||
|
| Tests | `cargo test --workspace` |
|
||||||
|
| Layering | `cargo xtask layers` |
|
||||||
|
| Assets | `cargo xtask assets` |
|
||||||
|
| wasm | `cargo xtask wasm` (checks every L0–L4 crate for `wasm32-unknown-unknown`) |
|
||||||
|
| All of the above | `cargo xtask ci` (runs clippy and tests with `--release`) |
|
||||||
|
|
||||||
|
`cargo xtask` is an alias in `.cargo/config.toml` for `cargo run -p xtask --`.
|
||||||
|
|
||||||
|
## 4. Commits
|
||||||
|
|
||||||
|
- One task per commit. The subject starts with the task id: `M3.2: trim mode engine — …`,
|
||||||
|
`M9.9: Matroska/WebM import`. Milestones (`M0`–`M16`) are listed in
|
||||||
|
[ROADMAP.md](../ROADMAP.md). If your change has no task id, use the area: `docs: …`,
|
||||||
|
`README: …`, `ROADMAP: …`.
|
||||||
|
- Commit only green states (all gates pass).
|
||||||
|
- When a milestone lands, update its row in `ROADMAP.md` (status, what's done, estimates).
|
||||||
|
- AI agents end commit messages with the attribution line their environment requires.
|
||||||
|
|
||||||
|
## 5. Parallel work (several agents or worktrees)
|
||||||
|
|
||||||
|
```sh
|
||||||
|
git worktree add ../filmcraft-hevc -b hevc-work
|
||||||
|
cd ../filmcraft-hevc
|
||||||
|
export CARGO_TARGET_DIR=target/agent-hevc # separate build dir, no lock fights
|
||||||
|
```
|
||||||
|
|
||||||
|
- One owner per crate at a time.
|
||||||
|
- The workspace includes `crates/*` and `apps/*` by glob, so one broken `Cargo.toml` breaks every
|
||||||
|
build. Keep every manifest valid at all times, including half-finished crates.
|
||||||
|
- Test fixtures always go to `<repo>/target/fixtures/`, whatever `CARGO_TARGET_DIR` says.
|
||||||
|
- `.mcp.json` points at `target/release/filmcraft-cli`. With a custom target dir, use your own path.
|
||||||
|
|
||||||
|
## 6. How to add…
|
||||||
|
|
||||||
|
### A command
|
||||||
|
|
||||||
|
1. Add a `cmd!` (action) or `query!` (read-only) entry to `build()` in
|
||||||
|
`crates/engine/src/commands.rs`:
|
||||||
|
```rust
|
||||||
|
cmd!("sequence.addEdit", "Add Edit", ["Sequence"], Some("Cmd+K"), r#"{"time":ticks?}"#, has_seq, |s, p| {
|
||||||
|
let t = time_p(s, p, "").unwrap_or(s.playhead());
|
||||||
|
let tg = s.targeting().targeted;
|
||||||
|
let n = s.edit_sequence("Add Edit", |q, ctx, _| Ok(edit::razor(q, &tg, t, ctx)))?;
|
||||||
|
Ok(json!({"cuts": n.len()}))
|
||||||
|
}),
|
||||||
|
```
|
||||||
|
- Id: `area.camelCase`, following the menu it appears in.
|
||||||
|
- `enabled`: reuse a predicate (`always`, `has_seq`, `has_selection`, `has_in_out`, …) or write
|
||||||
|
one that returns `Err("human reason")`.
|
||||||
|
- Read time with `time_p` so `time`, `frame`, `seconds` and `timecode` all work.
|
||||||
|
- Mutate only through `s.edit(…)` or `s.edit_sequence(…)`, so the change is undoable and
|
||||||
|
validated.
|
||||||
|
- Put pure timeline logic in `filmcraft-edit` as a function on `&mut Sequence`, with unit or
|
||||||
|
property tests there.
|
||||||
|
2. Add tests to `crates/engine/src/tests.rs`: execute, assert, undo, redo, and check the disabled
|
||||||
|
case.
|
||||||
|
3. Frontend-only actions (tools, zoom, panels) go in `UI_COMMANDS` in
|
||||||
|
`crates/ui-egui/src/menus.rs` with a branch in `menus::invoke`.
|
||||||
|
4. The command now appears in menus (from `menu`), shortcuts, `filmcraft-cli commands`,
|
||||||
|
`engine.commands` and MCP `command_list` without further work.
|
||||||
|
|
||||||
|
### A video effect
|
||||||
|
|
||||||
|
1. Define it in `build_effects()` in `crates/project/src/effect.rs` with the `video(id, name,
|
||||||
|
CATEGORY, params)` helper. Parameter helpers cover floats with slider ranges, choices, bools,
|
||||||
|
angles, points, colours, curves and wheels. The Effects panel, Effect Controls and agent
|
||||||
|
parameter docs are generated from this.
|
||||||
|
2. Implement it as a match arm in `render::effects::apply` (`crates/render/src/effects.rs`). It
|
||||||
|
works on a linear-light premultiplied f32 `Image`. Scale pixel-size parameters by `cx.px_scale`
|
||||||
|
so reduced-resolution playback matches full resolution. Unknown ids are a no-op.
|
||||||
|
3. Add a test in `crates/render/src/tests.rs`.
|
||||||
|
4. The GPU path needs no change: `render::plan` pre-renders layers with standard effects on the CPU.
|
||||||
|
|
||||||
|
Audio effects implement `AudioEffect` in `crates/audio-dsp` and register in its `effects()`
|
||||||
|
registry. Then add an `audio(…)` definition in `effect.rs` and a `Mapping` from project parameters
|
||||||
|
to DSP parameters in `crates/render/src/audio_fx.rs`. The DSP must give the same output however the
|
||||||
|
stream is cut into blocks (there are tests for this).
|
||||||
|
|
||||||
|
### A transition
|
||||||
|
|
||||||
|
1. Define it with `vtrans(id, name, CATEGORY, params)` in `crates/project/src/effect.rs`. Audio
|
||||||
|
transitions use `EffectKind::AudioTransition`.
|
||||||
|
2. Implement it in `render::transitions::apply(e, a, b, p)`: outgoing image `a`, incoming image `b`,
|
||||||
|
progress `p` in 0..1. Audio crossfade curves go in `transitions::audio_gains`.
|
||||||
|
3. It can then be applied with `sequence.applyVideoTransition {"effect": "<id>"}`.
|
||||||
|
|
||||||
|
### A codec or container crate
|
||||||
|
|
||||||
|
1. **Spec first.** Work only from the public specification (ITU-T, ISO/IEC, IETF RFCs, SMPTE,
|
||||||
|
published container specs). Never copy GPL/LGPL code, and follow AGENTS.md §2 on reading other
|
||||||
|
implementations. Name the spec edition in the crate README.
|
||||||
|
2. Create `crates/<name>` as an L0 crate that depends only on `filmcraft-bitstream` (plus
|
||||||
|
`thiserror` and optional `rayon` behind a `threads` feature). Add `[lints] workspace = true`, add
|
||||||
|
it to `[workspace.dependencies]`, and add it to `LAYERS` in `xtask/src/main.rs`. It must build
|
||||||
|
for wasm32.
|
||||||
|
3. Test it against ffmpeg as an oracle (see [testing.md](testing.md)):
|
||||||
|
- generate fixtures with ffmpeg on first use into `target/fixtures/<crate>/`;
|
||||||
|
- skip with a message when ffmpeg is missing;
|
||||||
|
- never commit media.
|
||||||
|
4. Wire it in:
|
||||||
|
- decoders: a `VideoDecoderFactory` in `crates/codecs` (`video.rs`, default list in `lib.rs`);
|
||||||
|
- containers: an `Opener` in `codecs::openers()`;
|
||||||
|
- encoders: an `export::VideoEncoder` / audio encoder plus a `Format` in `crates/export`.
|
||||||
|
5. Write the README: features, API, test matrix, accuracy and speed numbers, limitations. Use
|
||||||
|
`crates/h264/README.md` as the model.
|
||||||
|
|
||||||
|
### A panel or widget
|
||||||
|
|
||||||
|
- Panels live in `crates/ui-egui/src/panels/` and are listed in `PanelKind`
|
||||||
|
(`crates/ui-egui/src/dock.rs`).
|
||||||
|
- Every interactive element registers an automation id each frame:
|
||||||
|
```rust
|
||||||
|
app.auto.add(&format!("effects.item.{}", d.id), rect, d.name);
|
||||||
|
```
|
||||||
|
Ids are stable, dot-separated and start with the panel (`timeline.track.V1.lock`, `tools.Razor`).
|
||||||
|
- UI state that should survive or be scriptable goes in `crates/ui-egui/src/state.rs` (serde), so
|
||||||
|
`ui.inspect` and `ui.set` can reach it.
|
||||||
|
- Project changes go through `session.execute(…)`, never by mutating the project directly.
|
||||||
|
- Icons are drawn in code in `crates/ui-egui/src/icons.rs`, from scratch (AGENTS.md §1).
|
||||||
|
- Look at the result: run with `--control`, drive it, take `ui.screenshot`
|
||||||
|
([agents.md](agents.md#3-verifying-ui-work)).
|
||||||
|
|
||||||
|
### An asset
|
||||||
|
|
||||||
|
The rules are in [AGENTS.md §1](../AGENTS.md#1-assets-no-adobe-artwork-every-asset-licensed-and-attributed).
|
||||||
|
In short:
|
||||||
|
|
||||||
|
- No Adobe iconography, images, fonts, LUTs, presets or other Adobe assets, in any form, including
|
||||||
|
traced or "inspired-by" copies.
|
||||||
|
- Open licences only (MIT, Apache-2.0, BSD, ISC, zlib, OFL, CC0, CC BY, CC BY-SA, or your own
|
||||||
|
work under MIT OR Apache-2.0). No NC or ND licences. If the licence is unclear, leave the asset out.
|
||||||
|
- Screenshots show only FilmCraft (or other open projects) with media we generated or that is
|
||||||
|
openly licensed.
|
||||||
|
|
||||||
|
For each asset file `X`, commit a sidecar `X.attribution`. This is the real one for
|
||||||
|
`assets/fonts/Inter-Regular.ttf`:
|
||||||
|
|
||||||
|
```text
|
||||||
|
asset: Inter-Regular.ttf
|
||||||
|
title: Inter Regular (UI typeface)
|
||||||
|
author: The Inter Project Authors (Rasmus Andersson et al.)
|
||||||
|
source: https://github.com/rsms/inter
|
||||||
|
license: OFL-1.1
|
||||||
|
license-file: assets/fonts/OFL-Inter.txt
|
||||||
|
added: 2026-09-30 by Claude (agent) for Brandon Thomas
|
||||||
|
notes: Unmodified font file, embedded in the UI via include_bytes! (crates/ui-egui/src/theme.rs).
|
||||||
|
```
|
||||||
|
|
||||||
|
For your own work, use `source: original work` and the project licence (`MIT OR Apache-2.0`).
|
||||||
|
|
||||||
|
Then add a row to [ATTRIBUTION.md](../ATTRIBUTION.md) with the path in backticks. `cargo xtask
|
||||||
|
assets` checks every tracked file with a media extension (images, fonts, LUTs, audio, video, PDF,
|
||||||
|
project files…):
|
||||||
|
|
||||||
|
- the sidecar exists;
|
||||||
|
- `asset`, `title`, `author`, `source`, `license` and `added` are filled in;
|
||||||
|
- the licence is not NC, ND, Adobe or proprietary;
|
||||||
|
- the path appears in `ATTRIBUTION.md`;
|
||||||
|
- the path does not mention Adobe or Premiere;
|
||||||
|
- no sidecar is orphaned.
|
||||||
|
|
@ -0,0 +1,96 @@
|
||||||
|
# Testing
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo test --workspace # everything; oracle tests skip without ffmpeg
|
||||||
|
cargo test -p filmcraft-h264 # one crate
|
||||||
|
cargo test --release -p filmcraft-hevc # codec tests are much faster in release
|
||||||
|
cargo xtask ci # all gates (fmt, clippy, tests, layers, assets, wasm)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 1. Kinds of tests
|
||||||
|
|
||||||
|
| Kind | Where | Examples |
|
||||||
|
|---|---|---|
|
||||||
|
| Unit | `src/` `#[cfg(test)]` modules, `src/tests.rs` | CABAC engines, VLC tables (prefix-freeness, Kraft sums), colour matrices, frame cache |
|
||||||
|
| Property (`proptest`) | `time`, `bitstream`, `edit`, `isobmff/tests/roundtrip.rs`, `interchange/tests/*` | tick↔frame round trips for every rate, DF timecode, edit invariants (no overlaps, durations conserved), mux→demux round trips, interchange export→import |
|
||||||
|
| Synthetic streams | `h264/src/synth_tests.rs`, `hevc/src/synth_tests.rs` | hand-built bitstreams for features the reference encoders don't emit; expected output known exactly |
|
||||||
|
| Engine / command | `crates/engine/src/tests.rs` | run commands on the demo project, assert the sequence, undo/redo, disabled cases |
|
||||||
|
| Render | `crates/render/src/tests.rs` | compositing, opacity, Motion, cross dissolve midpoint, ½-res vs full, GPU plan vs reference, audio mix, audio-effect continuity |
|
||||||
|
| Oracle | `crates/*/tests/*oracle*.rs`, `conformance.rs` | compare with ffmpeg/ffprobe (§2) |
|
||||||
|
| Robustness / fuzz | `*/tests/robustness.rs`, `*/tests/fuzz.rs` | seeded mutation and truncation of real and synthetic files; nothing may panic |
|
||||||
|
| Performance | `*/tests/perf.rs` (`#[ignore]`) | §5 |
|
||||||
|
|
||||||
|
Commit `*.proptest-regressions` files so failing cases are re-run.
|
||||||
|
|
||||||
|
## 2. ffmpeg oracle tests
|
||||||
|
|
||||||
|
ffmpeg and ffprobe are **external processes** used to generate fixtures and to check results. They
|
||||||
|
are never linked or shipped ([AGENTS.md](../AGENTS.md) §2).
|
||||||
|
|
||||||
|
- Each crate's `tests/common/mod.rs` finds the tools in `/opt/homebrew/bin`, `/usr/local/bin` or
|
||||||
|
`/usr/bin`. If they are missing, the test prints a message and returns (it passes).
|
||||||
|
- Fixtures are generated on first use into `<repo>/target/fixtures/<crate>/` and reused afterwards.
|
||||||
|
Delete that directory to regenerate them. Never commit media.
|
||||||
|
- Fixture sources are synthetic: `testsrc2`, `mandelbrot`, SMPTE bars, noise and fades, sine tones.
|
||||||
|
H.264 and HEVC fixtures need ffmpeg built with libx264 and libx265. VideoToolbox fixtures are
|
||||||
|
generated only on macOS.
|
||||||
|
|
||||||
|
### Pass criteria per codec
|
||||||
|
|
||||||
|
| Crate | Oracle check | Criterion |
|
||||||
|
|---|---|---|
|
||||||
|
| `h264` | decode every fixture single-threaded and frame-threaded; compare with `ffmpeg -f rawvideo -pix_fmt yuv420p` | **bit-exact**, every frame |
|
||||||
|
| `hevc` | same, `yuv420p` / `yuv420p10le` | **bit-exact** (8- and 10-bit) |
|
||||||
|
| `h264enc` | `ffmpeg -ec 0` decodes our stream | no errors, exact frame count, **bit-identical to the encoder's reconstruction**; B-frame order checked with ffprobe; rate-control targets |
|
||||||
|
| `prores` decode | ffmpeg `prores_ks` / `prores_aw` fixtures, compared at native depth | within **±1 LSB** (different integer IDCT); alpha bit-exact |
|
||||||
|
| `prores` encode | ffmpeg decodes with `-xerror` | no errors; agrees with our decoder within ±1 |
|
||||||
|
| `aac` decode | ffmpeg's decode of the same stream | max abs error ~1e-7 (float) |
|
||||||
|
| `aac` encode | ffmpeg decodes our stream | no errors, no clipping, CBR within ±5% of target; SNR reported |
|
||||||
|
| `isobmff` | `ffprobe -show_packets` on ffmpeg-made MP4/MOV | packet offsets, sizes, pts/dts, durations, key flags and stream parameters equal; remuxed files decode with `ffmpeg -v error` silent |
|
||||||
|
| `matroska` | `ffprobe -show_packets` | every packet (stream, size, key flag, pts, duration) equal; seeks land on the latest keyframe ≤ target |
|
||||||
|
| `export` | our own demuxer/decoder reads the file back; ffprobe counts frames when present | expected size, duration, colour, audio level; exact frame count |
|
||||||
|
|
||||||
|
Each codec README has the full fixture matrix and the measured results.
|
||||||
|
|
||||||
|
## 3. Rendering: CPU reference and GPU parity
|
||||||
|
|
||||||
|
- The CPU compositor in `filmcraft-render` is the reference. Its tests check computed values
|
||||||
|
(opacity blends, dissolve midpoints, Motion placement, ½-resolution against downsampled full
|
||||||
|
resolution) and that `render::plan` matches `render_sequence`.
|
||||||
|
- `crates/gpu/src/tests.rs` composites the same frame plan (a YUV layer plus a transformed,
|
||||||
|
semi-transparent RGBA layer) on the GPU and on the CPU. It requires a 99th-percentile channel
|
||||||
|
difference of ≤ 6 and a mean of < 1.5 (8-bit levels). It skips when no GPU adapter is available.
|
||||||
|
- Effects implemented only on the CPU need no GPU test: the plan pre-renders those layers on the
|
||||||
|
CPU.
|
||||||
|
- Golden-image files (rendered PNGs compared against committed references) are not used yet. If you
|
||||||
|
add them, generate the references from FilmCraft itself, give each one an attribution sidecar,
|
||||||
|
and document the tolerance (PSNR in dB, or max/percentile error).
|
||||||
|
|
||||||
|
## 4. UI: control channel and screenshots
|
||||||
|
|
||||||
|
There is no automated UI test suite yet. UI work is checked by driving the real app:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo run --release -p filmcraft -- --control 9876
|
||||||
|
```
|
||||||
|
|
||||||
|
Then script it over the control channel or MCP: run commands, click by automation id,
|
||||||
|
`ui.inspect` / `ui.elements` to assert state, and `ui.screenshot {"path": "…"}` to capture the window
|
||||||
|
or a single panel (`{"panel": "Timeline"}`). Look at every screenshot. Examples are in
|
||||||
|
[agents.md](agents.md). The headless path (`filmcraft-cli run script.jsonl`, MCP `--demo`) covers
|
||||||
|
engine behaviour without a window.
|
||||||
|
|
||||||
|
## 5. Performance
|
||||||
|
|
||||||
|
| Benchmark | Command |
|
||||||
|
|---|---|
|
||||||
|
| H.264 decode, 1080p, threads sweep | `cargo test --release -p filmcraft-h264 --test perf -- --ignored --nocapture` |
|
||||||
|
| HEVC decode, 1080p / 2160p | `cargo test --release -p filmcraft-hevc --test perf -- --ignored --nocapture` |
|
||||||
|
| ProRes decode/encode | `cargo test --release -p filmcraft-prores --test perf -- --ignored --nocapture` |
|
||||||
|
| Encoder speed / PSNR / bitrate | `cargo run --release -p filmcraft-h264enc --example h264enc_synth -- …` |
|
||||||
|
| Decode a real file through the media stack | `cargo run --release -p filmcraft-cli -- bench-decode file.mp4 --frames 120` |
|
||||||
|
| Coding-tool coverage of the fixtures | `cargo test --release -p filmcraft-h264 --test conformance coverage_report -- --ignored --nocapture` (same for `hevc`) |
|
||||||
|
|
||||||
|
The perf tests check bit-exactness before they time anything. Results go in the crate README's
|
||||||
|
performance table, with machine and thread count. Headline numbers go in
|
||||||
|
[ROADMAP.md](../ROADMAP.md). Measure on an idle machine: parallel agent builds distort timings.
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
//! Workspace automation: `cargo xtask <layers|wasm|ci>`.
|
//! Workspace automation: `cargo xtask <layers|wasm|ci>`.
|
||||||
//!
|
//!
|
||||||
//! - `layers`: enforces the dependency layering of `plan/architecture.md` §3 (downward-only edges,
|
//! - `layers`: enforces the dependency layering of `docs/architecture.md` §1 (downward-only edges,
|
||||||
//! listed same-layer edges, L0 codec crates depend on `bitstream` only, no UI/OS crates below L5).
|
//! listed same-layer edges, L0 codec crates depend on `bitstream` only, no UI/OS crates below L5).
|
||||||
//! - `wasm`: `cargo check --target wasm32-unknown-unknown` for every crate in L0–L4.
|
//! - `wasm`: `cargo check --target wasm32-unknown-unknown` for every crate in L0–L4.
|
||||||
//! - `assets`: every asset file (image, icon, font, LUT, audio, video…) has a complete
|
//! - `assets`: every asset file (image, icon, font, LUT, audio, video…) has a complete
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue