docs: public architecture, contributing, testing and agent guides
Public equivalents of the maintainer-local plan/: docs/architecture.md (layers as enforced by xtask, Tick time base, data model, command system and undo, render/playback/export pipeline, automation surfaces), docs/contributing.md (+ root CONTRIBUTING.md), docs/testing.md and docs/agents.md (MCP/control channel usage, agent work loop, decision rules, Definition of Done). CLAUDE.md points at the public docs wherever it references plan/; AGENTS.md gains a See also list; README gains a Documentation section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
5303af9112
commit
4e2a8077d1
|
|
@ -60,3 +60,12 @@ reproduce an image (for example, point lists traced from someone else's icon).
|
|||
|
||||
See `CLAUDE.md`: pure Rust, dependency layering (`cargo xtask layers`), exact `Tick` time, everything is
|
||||
a command, everything is agent-drivable, and the quality gates (`cargo xtask ci`) before every commit.
|
||||
|
||||
## See also
|
||||
|
||||
- [CONTRIBUTING.md](CONTRIBUTING.md) and [docs/contributing.md](docs/contributing.md): setup, gates, commits, how to add things
|
||||
- [docs/architecture.md](docs/architecture.md): layers, data model, commands, pipeline
|
||||
- [docs/testing.md](docs/testing.md): oracle tests, criteria, benchmarks
|
||||
- [docs/agents.md](docs/agents.md): driving FilmCraft over MCP / the control channel, and the agent work loop
|
||||
- [docs/control-protocol.md](docs/control-protocol.md): control-channel method reference
|
||||
- [ATTRIBUTION.md](ATTRIBUTION.md): asset index
|
||||
|
|
|
|||
19
CLAUDE.md
19
CLAUDE.md
|
|
@ -3,25 +3,24 @@
|
|||
FilmCraft is a clean-room, open-source, pure-Rust non-linear video editor targeting Adobe Premiere Pro parity (and beyond). Native on macOS, Windows, Linux; web via WASM. Sibling of `../photocraft` (Photoshop), `../printcraft` (Acrobat) and `../drawcraft` (Illustrator), with the same conventions.
|
||||
|
||||
## Start every session here
|
||||
1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers).
|
||||
2. Read the task in `plan/execution-plan.md` §3, the relevant section of `plan/architecture.md`, and the README/docs of the crate you touch. Visual/behaviour reference: `plan/premiere/`.
|
||||
3. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask unless §7 lists the decision as the user's.
|
||||
|
||||
`plan/` is gitignored (local-only).
|
||||
`plan/` is maintainer-local (gitignored, not in the repo). Each step names the public equivalent to use if you don't have it.
|
||||
1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers). Public: [`ROADMAP.md`](ROADMAP.md).
|
||||
2. Read the task in `plan/execution-plan.md` §3, the relevant section of `plan/architecture.md`, and the README/docs of the crate you touch. Public: [`docs/architecture.md`](docs/architecture.md), [`docs/contributing.md`](docs/contributing.md), [`docs/testing.md`](docs/testing.md). Visual/behaviour reference: `plan/premiere/` (local only, never committed).
|
||||
3. Follow the autonomous operation protocol (`plan/execution-plan.md` §7; public summary: [`docs/agents.md`](docs/agents.md) §5–7). Don't stop to ask unless it lists the decision as the user's.
|
||||
|
||||
## Non-negotiables
|
||||
**Read [`AGENTS.md`](AGENTS.md) first; its rules override everything here.** In particular (§1): never use Adobe iconography, images or any other Adobe asset. Every asset must be openly licensed (OSS / public domain / Creative Commons, or original work by a contributor) and must have a `<file>.attribution` sidecar plus an entry in `ATTRIBUTION.md`. `cargo xtask assets` enforces this.
|
||||
- **Clean-room.** Never read/disassemble anything inside Adobe app bundles (names/listings only). Never copy Adobe icons, presets, LUTs, fonts. Never copy GPL/LGPL/AGPL code (FFmpeg, x264, x265, MLT, Kdenlive, Shotcut, Olive, LAME…). ffmpeg/ffprobe run only as external test oracles / fixture generators — never linked or shipped.
|
||||
- **Pure Rust** in the product. OS media APIs only via Rust bindings in `crates/platform`, optional, behind traits.
|
||||
- **Layering** (`plan/architecture.md` §3, enforced by `cargo xtask layers`): nothing below L5 depends on egui/eframe/winit/rfd/cpal. L0 codec/container crates depend only on `filmcraft-bitstream`.
|
||||
- **Pure Rust** in the product. OS media APIs only via Rust bindings in `crates/platform` (planned; not created yet), optional, behind traits.
|
||||
- **Layering** ([`docs/architecture.md`](docs/architecture.md) §1, enforced by `cargo xtask layers`): nothing below L5 depends on egui/eframe/winit/rfd/cpal. L0 codec/container crates depend only on `filmcraft-bitstream`.
|
||||
- **Exact time:** all time is `filmcraft_time::Tick` (254 016 000 000/s). Never use f64 seconds for edit math.
|
||||
- **Everything is a command** (`crates/engine`): id, label, menu path, shortcut, params, enabled(), run(). UI, CLI, control channel and MCP all dispatch by id.
|
||||
- **Everything is agent-drivable:** every interactive widget registers an automation id; UI state is serde so the control channel can read/set it.
|
||||
- **Quality gates** before every commit: `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace`, `cargo xtask layers`, `cargo xtask assets`, `cargo xtask wasm` (L0–L5 changes). `cargo xtask ci` runs them all.
|
||||
- **Quality gates** before every commit: `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace`, `cargo xtask layers`, `cargo xtask assets`, `cargo xtask wasm` (checks L0–L4 crates). `cargo xtask ci` runs them all.
|
||||
- **Commits:** one task id per commit (`M2.4: CABAC residual decoding`). Only green states. End messages with the attribution line required by the environment.
|
||||
|
||||
## Running and looking at the app
|
||||
- `cargo run -p filmcraft -- --control 9876` opens the desktop app with the JSON-lines control server (see `docs/control-protocol.md`).
|
||||
- For UI work, **look at the result**: drive via the control channel and take `ui.screenshot`, compare with `plan/premiere/screenshots/`.
|
||||
- For UI work, **look at the result**: drive via the control channel and take `ui.screenshot`, compare with `plan/premiere/screenshots/` (maintainer-local; see [`docs/agents.md`](docs/agents.md) §3).
|
||||
- Parallel agents: separate git worktrees and `CARGO_TARGET_DIR=target/agent-<name>`; keep every `Cargo.toml` valid at all times (the `crates/*` glob means one broken manifest breaks everyone).
|
||||
- Test fixtures: generate with ffmpeg into `target/fixtures/` (never commit media).
|
||||
- Test fixtures: generate with ffmpeg into `target/fixtures/` (never commit media). See [`docs/testing.md`](docs/testing.md).
|
||||
|
|
|
|||
|
|
@ -0,0 +1,15 @@
|
|||
# Contributing to FilmCraft
|
||||
|
||||
Thanks for helping. Before you start:
|
||||
|
||||
1. Read **[AGENTS.md](AGENTS.md)**. Its rules are absolute. In particular: no Adobe assets of any
|
||||
kind, every asset openly licensed with an attribution sidecar, no GPL/LGPL code, and ffmpeg only
|
||||
as an external test oracle.
|
||||
2. Follow **[docs/contributing.md](docs/contributing.md)**: setup per OS, building and running,
|
||||
the quality gates (`cargo xtask ci`), commit conventions, and how to add commands, effects,
|
||||
transitions, codecs, panels and assets.
|
||||
|
||||
More reading: [docs/architecture.md](docs/architecture.md), [docs/testing.md](docs/testing.md),
|
||||
[docs/agents.md](docs/agents.md), [ROADMAP.md](ROADMAP.md).
|
||||
|
||||
Contributions are licensed under MIT OR Apache-2.0, the project licence.
|
||||
12
README.md
12
README.md
|
|
@ -182,6 +182,18 @@ cargo run --release -p filmcraft-cli -- mcp # MCP server (headles
|
|||
|
||||
The control protocol is documented in [docs/control-protocol.md](docs/control-protocol.md).
|
||||
|
||||
## Documentation
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| [CONTRIBUTING.md](CONTRIBUTING.md) · [docs/contributing.md](docs/contributing.md) | Setup, quality gates, commit conventions, how to add commands, effects, codecs, panels and assets |
|
||||
| [AGENTS.md](AGENTS.md) | The rules every contributor must follow: assets, clean room, licences |
|
||||
| [docs/architecture.md](docs/architecture.md) | Layers, data model, time base, command system, render and export pipeline |
|
||||
| [docs/testing.md](docs/testing.md) | Unit, property and ffmpeg-oracle tests, accuracy criteria, benchmarks |
|
||||
| [docs/agents.md](docs/agents.md) | Driving FilmCraft over MCP and the control channel; how agents develop it |
|
||||
| [docs/control-protocol.md](docs/control-protocol.md) | Control-channel and MCP reference |
|
||||
| [ROADMAP.md](ROADMAP.md) | Milestones and estimates |
|
||||
|
||||
## Status
|
||||
|
||||
FilmCraft is young and moving fast: roughly 30% of the way to Premiere Pro parity, with editing, colour, keyframes, codecs and export working today. [ROADMAP.md](ROADMAP.md) tracks every milestone with estimates.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,167 @@
|
|||
# Agents
|
||||
|
||||
FilmCraft is built to be driven by AI agents, and largely built by them. Part 1 covers using
|
||||
FilmCraft from an agent. Part 2 covers developing FilmCraft as an agent.
|
||||
|
||||
# Part 1: driving FilmCraft
|
||||
|
||||
## 1. MCP server
|
||||
|
||||
`filmcraft-cli mcp` serves MCP on stdio. It has two modes:
|
||||
|
||||
| Mode | Command | What it drives |
|
||||
|---|---|---|
|
||||
| Headless | `filmcraft-cli mcp --demo` or `--project p.fcproj` (neither = empty project) | an in-process engine session, no window |
|
||||
| Bridge | `filmcraft-cli mcp --bridge 127.0.0.1:9876` | the running desktop app started with `filmcraft --control 9876` |
|
||||
|
||||
**Claude Code.** The repository's `.mcp.json` registers both servers (`filmcraft` = bridge,
|
||||
`filmcraft-headless` = demo). Both point at `target/release/filmcraft-cli`, so build it first:
|
||||
|
||||
```sh
|
||||
cargo build --release -p filmcraft-cli -p filmcraft
|
||||
./target/release/filmcraft --control 9876 & # only for the bridge server
|
||||
claude # approve the project MCP servers when asked
|
||||
```
|
||||
|
||||
To register the server by hand, or for another MCP client, use the absolute binary path:
|
||||
|
||||
```sh
|
||||
claude mcp add filmcraft-headless -- /abs/path/filmcraft/target/release/filmcraft-cli mcp --demo
|
||||
```
|
||||
|
||||
```json
|
||||
{ "mcpServers": { "filmcraft": { "command": "/abs/path/filmcraft-cli", "args": ["mcp", "--bridge", "127.0.0.1:9876"] } } }
|
||||
```
|
||||
|
||||
### Tools
|
||||
|
||||
| Tool | Modes | Purpose |
|
||||
|---|---|---|
|
||||
| `command_list` | both | every command: id, label, menu, shortcut, params, enabled now (`filter`, `enabled_only`) |
|
||||
| `command_run` | both | run a command `{id, params}`; edits are undoable |
|
||||
| `project_inspect` | both | bins and items with ids, types, durations; active sequence |
|
||||
| `sequence_inspect` | both | the active sequence: tracks, clips (ticks and frames), effects, transitions, markers, playhead, selection |
|
||||
| `media_import` | both | import files by absolute path (`text`, one path per line) |
|
||||
| `render_frame` | both | PNG of the program frame at `seconds` (headless renders; bridge screenshots the Program monitor) |
|
||||
| `ui_inspect` | bridge | UI state: tool, workspace, panels, zoom, playback, fps |
|
||||
| `ui_elements` | bridge | on-screen interactive elements with id, label and rect (`prefix` filter) |
|
||||
| `ui_click` | bridge | click by `id` or `x`,`y`; `button`, `count`, `modifiers` |
|
||||
| `ui_drag` | bridge | press–move–release between points or element ids |
|
||||
| `ui_key` | bridge | key or shortcut: `Space`, `Cmd+K`, `Shift+Delete` |
|
||||
| `ui_type` | bridge | type text into the focused field |
|
||||
| `ui_screenshot` | bridge | PNG of the window or one `panel` |
|
||||
| `ui_control` | bridge | call any control-channel method directly (`ui.set`, `ui.scroll`, `ui.timeline.locate`, …) |
|
||||
|
||||
Typical loop: `project_inspect` / `sequence_inspect` → get ids → `command_run` → `render_frame` or
|
||||
`ui_screenshot` → look at the result → `edit.undo` if it's wrong.
|
||||
|
||||
## 2. Control channel
|
||||
|
||||
`filmcraft --control 9876` (or `FILMCRAFT_CONTROL_PORT=9876`) listens on `127.0.0.1` only. Send one
|
||||
JSON request per line and get one JSON reply per line. Full method table:
|
||||
[control-protocol.md](control-protocol.md).
|
||||
|
||||
```jsonc
|
||||
{"id":1,"method":"engine.commands"}
|
||||
{"id":2,"method":"engine.execute","params":{"command":"file.openDemoProject"}}
|
||||
{"id":3,"method":"engine.execute","params":{"command":"playhead.set","params":{"seconds":2}}}
|
||||
{"id":4,"method":"engine.execute","params":{"command":"sequence.addEdit"}}
|
||||
{"id":5,"method":"engine.execute","params":{"command":"effects.apply","params":{"effect":"Gaussian Blur"}}}
|
||||
{"id":6,"method":"ui.elements","params":{"prefix":"tools."}}
|
||||
{"id":7,"method":"ui.click","params":{"id":"tools.Razor"}}
|
||||
{"id":8,"method":"ui.key","params":{"key":"Cmd+Z"}}
|
||||
{"id":9,"method":"ui.screenshot","params":{"path":"/tmp/timeline.png","panel":"Timeline"}}
|
||||
```
|
||||
|
||||
Replies look like `{"id":4,"ok":true,"result":{"cuts":2}}` or `{"id":7,"ok":false,"error":"…"}`.
|
||||
Minimal client:
|
||||
|
||||
```python
|
||||
import json, socket
|
||||
s = socket.create_connection(("127.0.0.1", 9876)); f = s.makefile("rw")
|
||||
def call(method, **params):
|
||||
f.write(json.dumps({"id": 1, "method": method, "params": params}) + "\n"); f.flush()
|
||||
return json.loads(f.readline())
|
||||
print(call("engine.execute", command="sequence.inspect"))
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- Time is in ticks (254 016 000 000 per second). Commands also accept `seconds`, `frame` or
|
||||
`timecode`.
|
||||
- `engine.execute` also runs UI-only commands (`tool.razor`, `playback.toggle`,
|
||||
`window.workspace.color`, `window.panel.<name>`).
|
||||
- Element ids come from the previous frame. If an element is missing right after a layout change,
|
||||
the app retries on later frames before giving up.
|
||||
- Without a window, `filmcraft-cli run script.jsonl` (lines of `{"id":"…","params":{…}}`) runs the
|
||||
same commands headlessly.
|
||||
|
||||
## 3. Verifying UI work
|
||||
|
||||
For any UI change, look at the result:
|
||||
|
||||
1. `cargo run --release -p filmcraft -- --control 9876`
|
||||
2. Drive the feature through the control channel or the MCP bridge: commands, then clicks and drags
|
||||
by automation id.
|
||||
3. Assert on `ui.inspect`, `ui.elements` and `sequence.inspect`.
|
||||
4. Take `ui.screenshot` (the whole window and the panel you changed) and open the PNG.
|
||||
|
||||
Screenshots you commit must follow [AGENTS.md](../AGENTS.md) §1: FilmCraft only, openly licensed
|
||||
media, sidecar and `ATTRIBUTION.md` entry.
|
||||
|
||||
# Part 2: developing FilmCraft
|
||||
|
||||
## 4. Orientation, in this order
|
||||
|
||||
1. [AGENTS.md](../AGENTS.md): absolute rules (assets, clean-room, licences).
|
||||
2. [CLAUDE.md](../CLAUDE.md): working instructions and non-negotiables.
|
||||
3. [ROADMAP.md](../ROADMAP.md): milestones, what's done, what's running.
|
||||
4. [architecture.md](architecture.md), then the README and tests of the crate you'll touch.
|
||||
5. [contributing.md](contributing.md) (how to add things, gates) and [testing.md](testing.md).
|
||||
|
||||
Maintainers also keep a local planning folder, `plan/`. It is gitignored and not in the repository,
|
||||
and holds the task-level plan, the status file and behaviour reference notes. If you don't have it,
|
||||
everything you need to contribute is in the public docs above. Ask a maintainer for a task id.
|
||||
|
||||
## 5. Autonomous work loop
|
||||
|
||||
1. **Orient.** Pick the next task: the next unchecked task in the maintainer status file, or an open
|
||||
ROADMAP item. Read the relevant architecture section and crate README.
|
||||
2. **Plan tests first.** Write down the acceptance test before writing code.
|
||||
3. **Implement and test.**
|
||||
4. **Verify.** Run all gates (`cargo xtask ci`). For UI work, run the app with `--control`, drive it
|
||||
and look at the screenshots (§3).
|
||||
5. **Record.** Update the crate README (behaviour decisions, test results, limitations) and
|
||||
ROADMAP.md when a milestone moves. Commit as `M<n>.<k>: …`, then move on to the next task.
|
||||
|
||||
For parallel agents, use one git worktree and one `CARGO_TARGET_DIR` per agent, and keep each crate
|
||||
with one owner ([contributing.md §5](contributing.md#5-parallel-work-several-agents-or-worktrees)).
|
||||
|
||||
## 6. Decision rules
|
||||
|
||||
Don't block on choices you can make yourself:
|
||||
|
||||
- **Behaviour.** Match the publicly documented and observable behaviour of professional editors
|
||||
(help pages, published shortcut lists, using the app). Never inspect application internals
|
||||
([AGENTS.md](../AGENTS.md) §2). If behaviour is undocumented, choose what editors most commonly
|
||||
expect and record the choice in the crate README.
|
||||
- **Ask a human only about:**
|
||||
- licences outside the allowlist;
|
||||
- spending money;
|
||||
- publishing or pushing;
|
||||
- personal data;
|
||||
- destructive git operations;
|
||||
- anything AGENTS.md leaves unclear ("when in doubt, leave it out").
|
||||
|
||||
## 7. Definition of Done
|
||||
|
||||
A feature is done when:
|
||||
|
||||
- [ ] its command(s) are registered with label, menu path, shortcut (if any) and a parameter doc;
|
||||
- [ ] engine tests cover it, including undo/redo and the disabled case;
|
||||
- [ ] the UI is wired: menu, panel control and shortcut;
|
||||
- [ ] every new interactive widget has an automation id;
|
||||
- [ ] you have driven it through the control channel and reviewed a screenshot;
|
||||
- [ ] all gates pass (`cargo xtask ci`), and new assets have sidecars;
|
||||
- [ ] the crate README and ROADMAP.md are updated where relevant;
|
||||
- [ ] it is committed with its task id.
|
||||
|
|
@ -0,0 +1,261 @@
|
|||
# Architecture
|
||||
|
||||
FilmCraft is a Cargo workspace of small crates with strictly enforced layering. The engine is
|
||||
headless: every feature can be reached without a window, and the egui UI is one client among the
|
||||
CLI, the JSON control channel and the MCP server.
|
||||
|
||||
Design principles:
|
||||
|
||||
1. **Engine-first.** Project-changing actions go through `Session::execute(id, params)`.
|
||||
2. **Everything is a command.** Stable id, label, menu path, shortcut, parameter doc, `enabled()`
|
||||
predicate with a human-readable reason, and `run()`.
|
||||
3. **Exact time.** Integer ticks, rational frame rates. No `f64` seconds in edit math.
|
||||
4. **Copy-on-write snapshots.** The project is an `Arc<Project>`. Undo is a stack of snapshots, and
|
||||
background readers (playback, export) hold a snapshot without locking.
|
||||
5. **CPU reference, GPU fast path.** The CPU compositor is the oracle. The GPU path is tested
|
||||
against it.
|
||||
6. **Pure Rust, clean-room.** Codecs and containers are written from public specifications
|
||||
(see [AGENTS.md](../AGENTS.md)).
|
||||
|
||||
## 1. Layers
|
||||
|
||||
```text
|
||||
L6 apps/filmcraft · apps/filmcraft-cli
|
||||
L5 ui-egui · automation
|
||||
L4 engine
|
||||
L3 render · gpu · export
|
||||
L2 edit · codecs · interchange
|
||||
L1 frame · media · project · audio-dsp
|
||||
L0 foundation: time · geom · color · bitstream
|
||||
codecs/containers: isobmff · matroska · h264 · h264enc · hevc · prores · aac
|
||||
```
|
||||
|
||||
Crates are named `filmcraft-<dir>` (`crates/time` is `filmcraft-time`). The apps are `filmcraft`
|
||||
and `filmcraft-cli`.
|
||||
|
||||
| Crate | Layer | Purpose |
|
||||
|---|---|---|
|
||||
| `time` | L0 | `Tick`, `FrameRate`, `TimeRange`, timecode parse/format (NDF/DF, frames, feet+frames, samples) |
|
||||
| `geom` | L0 | `Vec2`, `Rect`, `Affine`, Motion-transform composition |
|
||||
| `color` | L0 | colour spaces, transfer functions, YUV↔RGB matrices, LUTs |
|
||||
| `bitstream` | L0 | bit reader/writer, Exp-Golomb, emulation prevention |
|
||||
| `isobmff` | L0 | MP4/MOV demux and mux |
|
||||
| `matroska` | L0 | MKV/WebM demux |
|
||||
| `h264`, `h264enc` | L0 | H.264 decoder; H.264 encoder |
|
||||
| `hevc` | L0 | H.265 Main/Main 10 decoder |
|
||||
| `prores` | L0 | ProRes decoder and encoder |
|
||||
| `aac` | L0 | AAC-LC decoder and encoder |
|
||||
| `frame` | L1 | `VideoFrame` (planar YUV / RGBA8 / linear RGBA f32, colour metadata), `AudioBuffer` |
|
||||
| `media` | L1 | `MediaSource` trait, probing/openers, frame cache, generators, stills, WAV |
|
||||
| `project` | L1 | document model, effect definitions, keyframes |
|
||||
| `audio-dsp` | L1 | loudness metering (BS.1770 / R128) and audio effects; no dependencies |
|
||||
| `edit` | L2 | pure edit algebra (insert, overwrite, razor, ripple, roll, slip, slide, rate stretch…) |
|
||||
| `codecs` | L2 | container + codec hub: MP4/MOV and MKV sources, GOP-aware seeking, decoder registry, audio decoding |
|
||||
| `interchange` | L2 | EDL, FCP7 XML, FCPXML and OTIO import/export (no file I/O) |
|
||||
| `render` | L3 | sequence evaluation, CPU compositor, video effects, transitions, audio mix |
|
||||
| `gpu` | L3 | wgpu compositor (WGSL) |
|
||||
| `export` | L3 | render → encode → mux pipeline, progress/cancel |
|
||||
| `engine` | L4 | `Session`, command registry, undo history, media pool, jobs, interchange glue |
|
||||
| `ui-egui` | L5 | the egui frontend: docking, panels, timeline, monitors, playback, control-channel handlers |
|
||||
| `automation` | L5 | MCP server (`rmcp`, stdio), headless or bridged to the running app |
|
||||
| `filmcraft` | L6 | desktop binary: eframe/wgpu window, cpal audio output, file dialogs, native macOS menu, TCP control server |
|
||||
| `filmcraft-cli` | L6 | headless CLI: `probe`, `commands`, `render`, `run`, `mcp` |
|
||||
|
||||
### What `cargo xtask layers` enforces
|
||||
|
||||
The table of layers lives in `xtask/src/main.rs` (`LAYERS`). It also reserves names for planned
|
||||
crates. The check reads `cargo metadata` and looks at normal and build dependencies (dev-dependencies
|
||||
are exempt):
|
||||
|
||||
| Rule | Detail |
|
||||
|---|---|
|
||||
| Every crate has a layer | A new crate fails the check until it is added to `LAYERS`. |
|
||||
| Only downward edges | A crate may not depend on a crate in a higher layer. |
|
||||
| Same-layer edges are listed | From L1 up, a same-layer edge must be in `SAME_LAYER`: `media→frame`, `project→media`, `project→frame`, `gpu→render`, `export→render`, `cli→filmcraft`, plus a few reserved for planned crates. |
|
||||
| L0 codecs stay standalone | L0 crates other than `time`, `geom`, `color`, `bitstream` may depend on no workspace crate except `filmcraft-bitstream`. External crates such as `thiserror` and `rayon` are allowed. |
|
||||
| No UI/OS crates below L5 | `egui`, `eframe`, `egui-wgpu`, `winit`, `rfd`, `cpal`, `muda` are allowed only in L5 and L6. |
|
||||
|
||||
`cargo xtask wasm` runs `cargo check --target wasm32-unknown-unknown` on every L0–L4 crate, so
|
||||
everything up to the engine stays web-portable. `unsafe_code = "deny"` applies workspace-wide.
|
||||
|
||||
## 2. Time base
|
||||
|
||||
All time is `filmcraft_time::Tick(i64)` at `TICKS_PER_SECOND = 254_016_000_000`.
|
||||
|
||||
That number divides evenly into the frame duration of every broadcast rate (23.976, 24, 25, 29.97,
|
||||
30, 48, 50, 59.94, 60, 120…) and the sample duration of every common audio rate (8 kHz to 192 kHz,
|
||||
including the 44.1 kHz family). So frame and sample positions are exact integers, edits never drift
|
||||
at 29.97, and audio and video line up to the sample.
|
||||
|
||||
| Type | Use |
|
||||
|---|---|
|
||||
| `Tick` | timeline and media positions and durations |
|
||||
| `FrameRate { num, den }` | `frame_duration()`, `tick_of(frame)`, `frame_at(tick)`, `snap(tick)` |
|
||||
| `TimeRange` | half-open `start + duration` |
|
||||
| Timecode | display only (SMPTE NDF/DF, frames, feet+frames, samples); `parse_timecode` and `format_time` |
|
||||
|
||||
Commands take time as `time` (ticks), `frame`, `seconds` or `timecode`; the engine converts once at
|
||||
the boundary.
|
||||
|
||||
## 3. Data model (`filmcraft-project`)
|
||||
|
||||
```text
|
||||
Project
|
||||
├─ root: Bin tree of bins
|
||||
├─ items: map ItemId → ProjectItem flat
|
||||
│ kind: Media(MediaClip) | Sequence(Sequence) | Subclip{..} | AdjustmentLayer{..}
|
||||
└─ next_id
|
||||
|
||||
Sequence
|
||||
├─ settings: frame rate, size, sample rate, …
|
||||
├─ video_tracks / audio_tracks: Vec<Track>
|
||||
├─ markers, mark_in / mark_out
|
||||
Track
|
||||
├─ locked, sync lock, targeting, mute/solo/visibility
|
||||
├─ items: Vec<TrackItem> sorted, never overlapping
|
||||
└─ transitions: Vec<Transition>
|
||||
TrackItem (a clip instance)
|
||||
├─ item: ItemId, start (timeline ticks), source_in (media ticks), duration, speed
|
||||
├─ link group, label, enabled
|
||||
└─ effects: Vec<EffectInstance> intrinsic Motion/Opacity/Volume… first, then standard effects
|
||||
EffectInstance
|
||||
└─ effect id, enabled, params: id → constant value or keyframe track
|
||||
```
|
||||
|
||||
- Everything is plain serde data. `Sequence::check()` validates the invariants (no overlaps, unique
|
||||
ids), and the engine runs it after every sequence edit.
|
||||
- Timeline positions are sequence ticks; `source_in` and keyframes are in media time, so trims and
|
||||
splits never move keyframes.
|
||||
- **Effect definitions are data.** `project::effect::effect_defs()` lists every video effect,
|
||||
audio effect and transition with its parameter schema. The Effects panel tree, the Effect
|
||||
Controls rows and the parameter docs agents see are all generated from it.
|
||||
- **Project files** (`.fcproj`) are the project serialised as JSON. Saves are atomic: write a
|
||||
temporary sibling file, then rename it over the target.
|
||||
|
||||
## 4. Command system (`filmcraft-engine`)
|
||||
|
||||
```rust
|
||||
pub struct CommandSpec {
|
||||
pub id: &'static str, // "sequence.addEdit"
|
||||
pub label: &'static str, // "Add Edit"
|
||||
pub menu: &'static [&'static str], // ["Sequence"]; empty = not in menus
|
||||
pub shortcut: Option<&'static str>, // "Cmd+K" (Cmd = ⌘ on macOS, Ctrl elsewhere)
|
||||
pub params: &'static str, // r#"{"time":ticks?}"#, shown to agents
|
||||
pub enabled: fn(&Session) -> Result<(), String>, // Err carries the reason
|
||||
pub run: fn(&mut Session, &Value) -> Result<Value>,
|
||||
pub journal: bool, // false for read-only queries
|
||||
}
|
||||
```
|
||||
|
||||
- All commands are in `crates/engine/src/commands.rs` (`cmd!` for actions, `query!` for read-only
|
||||
queries such as `project.inspect`, `sequence.inspect`, `effects.list`, `jobs.list`). Ids follow
|
||||
the menu structure: `file.*`, `edit.*`, `clip.*`, `sequence.*`, `markers.*`, `timeline.*`,
|
||||
`effects.*`…
|
||||
- `Session::execute(id, params)` finds the spec, checks `enabled`, runs it and appends it to the
|
||||
journal.
|
||||
- **Undo.** Edits go through `Session::edit(label, |project, state| …)` or
|
||||
`Session::edit_sequence(label, |seq, ctx, state| …)`. These clone the project, apply the closure,
|
||||
and on success push the old `Arc<Project>` with the label onto the undo stack (200 entries).
|
||||
On error nothing changes. `edit.undo` and `edit.redo` swap snapshots. Thanks to structural sharing
|
||||
a snapshot costs little.
|
||||
- **Editor state** (`EditorState`: active sequence, playheads, selection, targeting, edit points…)
|
||||
is serde, so agents can read it with `state.inspect`.
|
||||
- **Events** (`ProjectChanged`, `Toast`, `OpenSequence`, `OpenSource`) are drained by frontends
|
||||
each frame.
|
||||
- **UI-only commands** (tools, playback, zoom, panels, workspaces) live in
|
||||
`crates/ui-egui/src/menus.rs` (`UI_COMMANDS`). The menu bar is built from the engine registry plus
|
||||
this table, and `menus::invoke` is the single entry point for menus, shortcuts and the control
|
||||
channel.
|
||||
|
||||
## 5. Media, render and playback pipeline
|
||||
|
||||
```text
|
||||
file ──► codecs (MP4/MOV, MKV, audio) demux + decode, GOP-aware seek
|
||||
│ decoder registry: h264, hevc, prores, mjpeg (+ any registered first)
|
||||
▼
|
||||
media::MediaSource ──► frame cache (byte-budgeted LRU, shared)
|
||||
▼
|
||||
render::render_sequence(project, seq, t, scale) CPU reference
|
||||
per track bottom→top: map timeline t → media t (speed), fetch frame,
|
||||
standard effects → Motion → Opacity/blend, transitions, composite
|
||||
in linear-light premultiplied f32
|
||||
│
|
||||
└─ render::plan::plan_frame → gpu::GpuCompositor GPU path
|
||||
layers = decoded YUV/RGBA frames + matrix + opacity;
|
||||
anything the shaders don't cover is pre-rendered on the CPU
|
||||
▼
|
||||
ui-egui frames.rs worker pool ──► monitors (program/source), thumbnails, prefetch
|
||||
```
|
||||
|
||||
- **Sources.** `media::MediaSource` yields `video_frame(FrameRequest)` and
|
||||
`audio(start, frames, rate)` in media time. Sources are `Send + Sync` and shared by monitors,
|
||||
thumbnails, playback and export. The engine's `MediaPool` creates one per project item, lazily,
|
||||
through registered openers (`codecs::openers()`: MP4/MOV, MKV/WebM, audio files).
|
||||
- **Seeking.** `codecs::Mp4Source` seeks to the preceding sync sample and decodes forward, caching
|
||||
every frame of the GOP. Sequential playback reuses the decoder. Decoders implement
|
||||
`codecs::VideoDecoder`. `register_video_decoder` puts a factory in front of the built-in ones, so a
|
||||
hardware decoder can take precedence.
|
||||
- **Compositor.** `render` is the reference for monitors, thumbnails and export. `render::plan`
|
||||
turns a frame into GPU layers. Non-Normal blend modes, standard effects, adjustment layers, nested
|
||||
sequences and non-dissolve transitions are rendered on the CPU for that layer or frame and handed to
|
||||
the GPU as an image, so both paths give the same picture. Setting `FILMCRAFT_CPU_COMPOSITE=1`
|
||||
forces the CPU path in the desktop app.
|
||||
- **Frame scheduling.** `crates/ui-egui/src/frames.rs` runs a small pool of worker threads with
|
||||
prioritised jobs: the frame on screen first, then playback prefetch, then thumbnails. The UI never
|
||||
decodes. It shows the exact frame when it is ready and holds the nearest cached frame meanwhile.
|
||||
- **Audio clock.** The desktop app passes a cpal output (`apps/filmcraft/src/audio.rs`) to the UI as
|
||||
`AudioOut`. While playing, the samples played by the sound card drive the playhead and video follows.
|
||||
Without an audio device, playback falls back to the wall clock. Dropped frames are counted.
|
||||
Sequence audio is mixed in `render::audio`, and clip audio effects run on `audio-dsp` via
|
||||
`render::audio_fx`.
|
||||
|
||||
## 6. Export jobs (`filmcraft-export`)
|
||||
|
||||
```text
|
||||
file.exportMedia {path, format, scale, audio, quality}
|
||||
→ engine creates a Job {id, label, progress, result} and runs it on a background thread
|
||||
→ export: render frames in parallel batches → encode in order → mux; audio mixed per batch
|
||||
→ jobs.list shows progress; jobs.cancel sets the shared cancel flag
|
||||
```
|
||||
|
||||
| Format | Encoder | Container |
|
||||
|---|---|---|
|
||||
| `h264` | `filmcraft-h264enc` + `filmcraft-aac` | MP4 (`isobmff`) |
|
||||
| `prores` | `filmcraft-prores` | MOV |
|
||||
| `mjpeg` | built in | MOV |
|
||||
| `png`, `gif`, `wav` | built in | image sequence / GIF / WAV |
|
||||
|
||||
Video encoders implement `export::VideoEncoder`. Codec crates plug in with `register_encoder` and
|
||||
`register_audio_encoder`.
|
||||
|
||||
Timelines can be exchanged as EDL, FCP7 XML, FCPXML or OTIO. `file.import` detects these formats and
|
||||
merges the result into the project as one undoable step, and `file.exportInterchange`,
|
||||
`file.exportEdl`, `file.exportFcpxml` and `file.exportOtio` write them.
|
||||
|
||||
## 7. Automation surfaces
|
||||
|
||||
All of these dispatch the same command ids.
|
||||
|
||||
| Surface | Where | Scope |
|
||||
|---|---|---|
|
||||
| UI | `ui-egui` menus, shortcuts, panels | `menus::invoke` → engine or UI command |
|
||||
| CLI | `filmcraft-cli` | `commands`, `run script.jsonl` (one `{"id","params"}` per line), `render`, `probe` |
|
||||
| Control channel | `filmcraft --control <port>` | JSON lines on loopback TCP: engine commands plus synthetic input, inspection and screenshots of the live UI |
|
||||
| MCP | `filmcraft-cli mcp` | stdio MCP server: headless in-process session, or `--bridge` to the control channel |
|
||||
|
||||
- **Automation ids.** Every interactive widget calls `app.auto.add(id, rect, label)` each frame
|
||||
(`crates/ui-egui/src/automation.rs`). Agents click by id, e.g. `tools.Razor`,
|
||||
`timeline.clip.<id>`, `effects.item.gaussian_blur`, `panel.Timeline`.
|
||||
- **UI state** that is not project data (tool, workspace, dock layout, zoom, scroll, monitor
|
||||
settings) is in `crates/ui-egui/src/state.rs` as serde structs, so `ui.inspect` and `ui.set` can
|
||||
read and write it.
|
||||
|
||||
Protocol reference: [control-protocol.md](control-protocol.md). Agent guide: [agents.md](agents.md).
|
||||
|
||||
## 8. Not built yet
|
||||
|
||||
The layer table reserves names for crates that don't exist yet: `riff`, `mjpeg`, `dnx`,
|
||||
`keyframe`, `effects`, `text`, `audio`, `captions`, `scopes`, `playback`, `format` and `platform`.
|
||||
Until they exist, that work lives elsewhere: keyframes and effect definitions in `project`, effects
|
||||
and the audio mix in `render`, scopes and playback in `ui-egui`, and OS integration (cpal, rfd,
|
||||
native menus) in `apps/filmcraft`. [ROADMAP.md](../ROADMAP.md) has the milestone status.
|
||||
|
|
@ -0,0 +1,211 @@
|
|||
# Contributing
|
||||
|
||||
Read [AGENTS.md](../AGENTS.md) first. Its rules on assets, clean-room code and licences are
|
||||
absolute and override everything here. Then read [architecture.md](architecture.md) and the README
|
||||
of the crate you will change.
|
||||
|
||||
## 1. Setup
|
||||
|
||||
| | All platforms |
|
||||
|---|---|
|
||||
| Rust | stable ≥ 1.90 (edition 2024), via [rustup](https://rustup.rs) |
|
||||
| wasm target | `rustup target add wasm32-unknown-unknown` (for `cargo xtask wasm`) |
|
||||
| Components | `rustup component add rustfmt clippy` |
|
||||
| ffmpeg + ffprobe (optional) | test oracle and fixture generator only; tests skip without it |
|
||||
|
||||
| OS | Extra |
|
||||
|---|---|
|
||||
| macOS | Xcode Command Line Tools. ffmpeg: `brew install ffmpeg` (includes libx264/libx265). |
|
||||
| Linux | C toolchain and the system headers cpal and rfd need: ALSA (`libasound2-dev`) and GTK 3 (`libgtk-3-dev`), plus the usual X11/Wayland development packages for winit. ffmpeg from your distribution, built with libx264 and libx265. |
|
||||
| Windows | MSVC build tools. The oracle tests look for ffmpeg only in `/opt/homebrew/bin`, `/usr/local/bin` and `/usr/bin`, so most of them skip on Windows. |
|
||||
|
||||
The oracle tests find ffmpeg at `/opt/homebrew/bin`, `/usr/local/bin` or `/usr/bin`. H.264 and
|
||||
HEVC fixtures need an ffmpeg built with libx264 and libx265. The VideoToolbox fixtures run only on
|
||||
macOS.
|
||||
|
||||
## 2. Build and run
|
||||
|
||||
```sh
|
||||
cargo run --release -p filmcraft # desktop app with the demo project
|
||||
cargo run --release -p filmcraft -- --empty # start without the demo project
|
||||
cargo run --release -p filmcraft -- --control 9876 # plus the JSON-lines control server
|
||||
cargo run --release -p filmcraft -- a.mp4 b.wav # import media (or open a .fcproj)
|
||||
cargo run --release -p filmcraft-cli -- commands # list engine commands
|
||||
cargo run --release -p filmcraft-cli -- render --demo --seconds 3 --out frame.png
|
||||
cargo run --release -p filmcraft-cli -- mcp --demo # headless MCP server on stdio
|
||||
```
|
||||
|
||||
`FILMCRAFT_CONTROL_PORT=9876` works like `--control 9876`, and `FILMCRAFT_CPU_COMPOSITE=1`
|
||||
disables the GPU compositor.
|
||||
|
||||
Dev builds compile dependencies at `opt-level = 2` and workspace crates at `opt-level = 1`. For
|
||||
playback and codec speed, use `--release`.
|
||||
|
||||
## 3. Quality gates
|
||||
|
||||
Every commit must pass all of these:
|
||||
|
||||
| Gate | Command |
|
||||
|---|---|
|
||||
| Format | `cargo fmt --check` (`rustfmt.toml`: `max_width = 160`) |
|
||||
| Lints | `cargo clippy --workspace --all-targets -- -D warnings` |
|
||||
| Tests | `cargo test --workspace` |
|
||||
| Layering | `cargo xtask layers` |
|
||||
| Assets | `cargo xtask assets` |
|
||||
| wasm | `cargo xtask wasm` (checks every L0–L4 crate for `wasm32-unknown-unknown`) |
|
||||
| All of the above | `cargo xtask ci` (runs clippy and tests with `--release`) |
|
||||
|
||||
`cargo xtask` is an alias in `.cargo/config.toml` for `cargo run -p xtask --`.
|
||||
|
||||
## 4. Commits
|
||||
|
||||
- One task per commit. The subject starts with the task id: `M3.2: trim mode engine — …`,
|
||||
`M9.9: Matroska/WebM import`. Milestones (`M0`–`M16`) are listed in
|
||||
[ROADMAP.md](../ROADMAP.md). If your change has no task id, use the area: `docs: …`,
|
||||
`README: …`, `ROADMAP: …`.
|
||||
- Commit only green states (all gates pass).
|
||||
- When a milestone lands, update its row in `ROADMAP.md` (status, what's done, estimates).
|
||||
- AI agents end commit messages with the attribution line their environment requires.
|
||||
|
||||
## 5. Parallel work (several agents or worktrees)
|
||||
|
||||
```sh
|
||||
git worktree add ../filmcraft-hevc -b hevc-work
|
||||
cd ../filmcraft-hevc
|
||||
export CARGO_TARGET_DIR=target/agent-hevc # separate build dir, no lock fights
|
||||
```
|
||||
|
||||
- One owner per crate at a time.
|
||||
- The workspace includes `crates/*` and `apps/*` by glob, so one broken `Cargo.toml` breaks every
|
||||
build. Keep every manifest valid at all times, including half-finished crates.
|
||||
- Test fixtures always go to `<repo>/target/fixtures/`, whatever `CARGO_TARGET_DIR` says.
|
||||
- `.mcp.json` points at `target/release/filmcraft-cli`. With a custom target dir, use your own path.
|
||||
|
||||
## 6. How to add…
|
||||
|
||||
### A command
|
||||
|
||||
1. Add a `cmd!` (action) or `query!` (read-only) entry to `build()` in
|
||||
`crates/engine/src/commands.rs`:
|
||||
```rust
|
||||
cmd!("sequence.addEdit", "Add Edit", ["Sequence"], Some("Cmd+K"), r#"{"time":ticks?}"#, has_seq, |s, p| {
|
||||
let t = time_p(s, p, "").unwrap_or(s.playhead());
|
||||
let tg = s.targeting().targeted;
|
||||
let n = s.edit_sequence("Add Edit", |q, ctx, _| Ok(edit::razor(q, &tg, t, ctx)))?;
|
||||
Ok(json!({"cuts": n.len()}))
|
||||
}),
|
||||
```
|
||||
- Id: `area.camelCase`, following the menu it appears in.
|
||||
- `enabled`: reuse a predicate (`always`, `has_seq`, `has_selection`, `has_in_out`, …) or write
|
||||
one that returns `Err("human reason")`.
|
||||
- Read time with `time_p` so `time`, `frame`, `seconds` and `timecode` all work.
|
||||
- Mutate only through `s.edit(…)` or `s.edit_sequence(…)`, so the change is undoable and
|
||||
validated.
|
||||
- Put pure timeline logic in `filmcraft-edit` as a function on `&mut Sequence`, with unit or
|
||||
property tests there.
|
||||
2. Add tests to `crates/engine/src/tests.rs`: execute, assert, undo, redo, and check the disabled
|
||||
case.
|
||||
3. Frontend-only actions (tools, zoom, panels) go in `UI_COMMANDS` in
|
||||
`crates/ui-egui/src/menus.rs` with a branch in `menus::invoke`.
|
||||
4. The command now appears in menus (from `menu`), shortcuts, `filmcraft-cli commands`,
|
||||
`engine.commands` and MCP `command_list` without further work.
|
||||
|
||||
### A video effect
|
||||
|
||||
1. Define it in `build_effects()` in `crates/project/src/effect.rs` with the `video(id, name,
|
||||
CATEGORY, params)` helper. Parameter helpers cover floats with slider ranges, choices, bools,
|
||||
angles, points, colours, curves and wheels. The Effects panel, Effect Controls and agent
|
||||
parameter docs are generated from this.
|
||||
2. Implement it as a match arm in `render::effects::apply` (`crates/render/src/effects.rs`). It
|
||||
works on a linear-light premultiplied f32 `Image`. Scale pixel-size parameters by `cx.px_scale`
|
||||
so reduced-resolution playback matches full resolution. Unknown ids are a no-op.
|
||||
3. Add a test in `crates/render/src/tests.rs`.
|
||||
4. The GPU path needs no change: `render::plan` pre-renders layers with standard effects on the CPU.
|
||||
|
||||
Audio effects implement `AudioEffect` in `crates/audio-dsp` and register in its `effects()`
|
||||
registry. Then add an `audio(…)` definition in `effect.rs` and a `Mapping` from project parameters
|
||||
to DSP parameters in `crates/render/src/audio_fx.rs`. The DSP must give the same output however the
|
||||
stream is cut into blocks (there are tests for this).
|
||||
|
||||
### A transition
|
||||
|
||||
1. Define it with `vtrans(id, name, CATEGORY, params)` in `crates/project/src/effect.rs`. Audio
|
||||
transitions use `EffectKind::AudioTransition`.
|
||||
2. Implement it in `render::transitions::apply(e, a, b, p)`: outgoing image `a`, incoming image `b`,
|
||||
progress `p` in 0..1. Audio crossfade curves go in `transitions::audio_gains`.
|
||||
3. It can then be applied with `sequence.applyVideoTransition {"effect": "<id>"}`.
|
||||
|
||||
### A codec or container crate
|
||||
|
||||
1. **Spec first.** Work only from the public specification (ITU-T, ISO/IEC, IETF RFCs, SMPTE,
|
||||
published container specs). Never copy GPL/LGPL code, and follow AGENTS.md §2 on reading other
|
||||
implementations. Name the spec edition in the crate README.
|
||||
2. Create `crates/<name>` as an L0 crate that depends only on `filmcraft-bitstream` (plus
|
||||
`thiserror` and optional `rayon` behind a `threads` feature). Add `[lints] workspace = true`, add
|
||||
it to `[workspace.dependencies]`, and add it to `LAYERS` in `xtask/src/main.rs`. It must build
|
||||
for wasm32.
|
||||
3. Test it against ffmpeg as an oracle (see [testing.md](testing.md)):
|
||||
- generate fixtures with ffmpeg on first use into `target/fixtures/<crate>/`;
|
||||
- skip with a message when ffmpeg is missing;
|
||||
- never commit media.
|
||||
4. Wire it in:
|
||||
- decoders: a `VideoDecoderFactory` in `crates/codecs` (`video.rs`, default list in `lib.rs`);
|
||||
- containers: an `Opener` in `codecs::openers()`;
|
||||
- encoders: an `export::VideoEncoder` / audio encoder plus a `Format` in `crates/export`.
|
||||
5. Write the README: features, API, test matrix, accuracy and speed numbers, limitations. Use
|
||||
`crates/h264/README.md` as the model.
|
||||
|
||||
### A panel or widget
|
||||
|
||||
- Panels live in `crates/ui-egui/src/panels/` and are listed in `PanelKind`
|
||||
(`crates/ui-egui/src/dock.rs`).
|
||||
- Every interactive element registers an automation id each frame:
|
||||
```rust
|
||||
app.auto.add(&format!("effects.item.{}", d.id), rect, d.name);
|
||||
```
|
||||
Ids are stable, dot-separated and start with the panel (`timeline.track.V1.lock`, `tools.Razor`).
|
||||
- UI state that should survive or be scriptable goes in `crates/ui-egui/src/state.rs` (serde), so
|
||||
`ui.inspect` and `ui.set` can reach it.
|
||||
- Project changes go through `session.execute(…)`, never by mutating the project directly.
|
||||
- Icons are drawn in code in `crates/ui-egui/src/icons.rs`, from scratch (AGENTS.md §1).
|
||||
- Look at the result: run with `--control`, drive it, take `ui.screenshot`
|
||||
([agents.md](agents.md#3-verifying-ui-work)).
|
||||
|
||||
### An asset
|
||||
|
||||
The rules are in [AGENTS.md §1](../AGENTS.md#1-assets-no-adobe-artwork-every-asset-licensed-and-attributed).
|
||||
In short:
|
||||
|
||||
- No Adobe iconography, images, fonts, LUTs, presets or other Adobe assets, in any form, including
|
||||
traced or "inspired-by" copies.
|
||||
- Open licences only (MIT, Apache-2.0, BSD, ISC, zlib, OFL, CC0, CC BY, CC BY-SA, or your own
|
||||
work under MIT OR Apache-2.0). No NC or ND licences. If the licence is unclear, leave the asset out.
|
||||
- Screenshots show only FilmCraft (or other open projects) with media we generated or that is
|
||||
openly licensed.
|
||||
|
||||
For each asset file `X`, commit a sidecar `X.attribution`. This is the real one for
|
||||
`assets/fonts/Inter-Regular.ttf`:
|
||||
|
||||
```text
|
||||
asset: Inter-Regular.ttf
|
||||
title: Inter Regular (UI typeface)
|
||||
author: The Inter Project Authors (Rasmus Andersson et al.)
|
||||
source: https://github.com/rsms/inter
|
||||
license: OFL-1.1
|
||||
license-file: assets/fonts/OFL-Inter.txt
|
||||
added: 2026-09-30 by Claude (agent) for Brandon Thomas
|
||||
notes: Unmodified font file, embedded in the UI via include_bytes! (crates/ui-egui/src/theme.rs).
|
||||
```
|
||||
|
||||
For your own work, use `source: original work` and the project licence (`MIT OR Apache-2.0`).
|
||||
|
||||
Then add a row to [ATTRIBUTION.md](../ATTRIBUTION.md) with the path in backticks. `cargo xtask
|
||||
assets` checks every tracked file with a media extension (images, fonts, LUTs, audio, video, PDF,
|
||||
project files…):
|
||||
|
||||
- the sidecar exists;
|
||||
- `asset`, `title`, `author`, `source`, `license` and `added` are filled in;
|
||||
- the licence is not NC, ND, Adobe or proprietary;
|
||||
- the path appears in `ATTRIBUTION.md`;
|
||||
- the path does not mention Adobe or Premiere;
|
||||
- no sidecar is orphaned.
|
||||
|
|
@ -0,0 +1,96 @@
|
|||
# Testing
|
||||
|
||||
```sh
|
||||
cargo test --workspace # everything; oracle tests skip without ffmpeg
|
||||
cargo test -p filmcraft-h264 # one crate
|
||||
cargo test --release -p filmcraft-hevc # codec tests are much faster in release
|
||||
cargo xtask ci # all gates (fmt, clippy, tests, layers, assets, wasm)
|
||||
```
|
||||
|
||||
## 1. Kinds of tests
|
||||
|
||||
| Kind | Where | Examples |
|
||||
|---|---|---|
|
||||
| Unit | `src/` `#[cfg(test)]` modules, `src/tests.rs` | CABAC engines, VLC tables (prefix-freeness, Kraft sums), colour matrices, frame cache |
|
||||
| Property (`proptest`) | `time`, `bitstream`, `edit`, `isobmff/tests/roundtrip.rs`, `interchange/tests/*` | tick↔frame round trips for every rate, DF timecode, edit invariants (no overlaps, durations conserved), mux→demux round trips, interchange export→import |
|
||||
| Synthetic streams | `h264/src/synth_tests.rs`, `hevc/src/synth_tests.rs` | hand-built bitstreams for features the reference encoders don't emit; expected output known exactly |
|
||||
| Engine / command | `crates/engine/src/tests.rs` | run commands on the demo project, assert the sequence, undo/redo, disabled cases |
|
||||
| Render | `crates/render/src/tests.rs` | compositing, opacity, Motion, cross dissolve midpoint, ½-res vs full, GPU plan vs reference, audio mix, audio-effect continuity |
|
||||
| Oracle | `crates/*/tests/*oracle*.rs`, `conformance.rs` | compare with ffmpeg/ffprobe (§2) |
|
||||
| Robustness / fuzz | `*/tests/robustness.rs`, `*/tests/fuzz.rs` | seeded mutation and truncation of real and synthetic files; nothing may panic |
|
||||
| Performance | `*/tests/perf.rs` (`#[ignore]`) | §5 |
|
||||
|
||||
Commit `*.proptest-regressions` files so failing cases are re-run.
|
||||
|
||||
## 2. ffmpeg oracle tests
|
||||
|
||||
ffmpeg and ffprobe are **external processes** used to generate fixtures and to check results. They
|
||||
are never linked or shipped ([AGENTS.md](../AGENTS.md) §2).
|
||||
|
||||
- Each crate's `tests/common/mod.rs` finds the tools in `/opt/homebrew/bin`, `/usr/local/bin` or
|
||||
`/usr/bin`. If they are missing, the test prints a message and returns (it passes).
|
||||
- Fixtures are generated on first use into `<repo>/target/fixtures/<crate>/` and reused afterwards.
|
||||
Delete that directory to regenerate them. Never commit media.
|
||||
- Fixture sources are synthetic: `testsrc2`, `mandelbrot`, SMPTE bars, noise and fades, sine tones.
|
||||
H.264 and HEVC fixtures need ffmpeg built with libx264 and libx265. VideoToolbox fixtures are
|
||||
generated only on macOS.
|
||||
|
||||
### Pass criteria per codec
|
||||
|
||||
| Crate | Oracle check | Criterion |
|
||||
|---|---|---|
|
||||
| `h264` | decode every fixture single-threaded and frame-threaded; compare with `ffmpeg -f rawvideo -pix_fmt yuv420p` | **bit-exact**, every frame |
|
||||
| `hevc` | same, `yuv420p` / `yuv420p10le` | **bit-exact** (8- and 10-bit) |
|
||||
| `h264enc` | `ffmpeg -ec 0` decodes our stream | no errors, exact frame count, **bit-identical to the encoder's reconstruction**; B-frame order checked with ffprobe; rate-control targets |
|
||||
| `prores` decode | ffmpeg `prores_ks` / `prores_aw` fixtures, compared at native depth | within **±1 LSB** (different integer IDCT); alpha bit-exact |
|
||||
| `prores` encode | ffmpeg decodes with `-xerror` | no errors; agrees with our decoder within ±1 |
|
||||
| `aac` decode | ffmpeg's decode of the same stream | max abs error ~1e-7 (float) |
|
||||
| `aac` encode | ffmpeg decodes our stream | no errors, no clipping, CBR within ±5% of target; SNR reported |
|
||||
| `isobmff` | `ffprobe -show_packets` on ffmpeg-made MP4/MOV | packet offsets, sizes, pts/dts, durations, key flags and stream parameters equal; remuxed files decode with `ffmpeg -v error` silent |
|
||||
| `matroska` | `ffprobe -show_packets` | every packet (stream, size, key flag, pts, duration) equal; seeks land on the latest keyframe ≤ target |
|
||||
| `export` | our own demuxer/decoder reads the file back; ffprobe counts frames when present | expected size, duration, colour, audio level; exact frame count |
|
||||
|
||||
Each codec README has the full fixture matrix and the measured results.
|
||||
|
||||
## 3. Rendering: CPU reference and GPU parity
|
||||
|
||||
- The CPU compositor in `filmcraft-render` is the reference. Its tests check computed values
|
||||
(opacity blends, dissolve midpoints, Motion placement, ½-resolution against downsampled full
|
||||
resolution) and that `render::plan` matches `render_sequence`.
|
||||
- `crates/gpu/src/tests.rs` composites the same frame plan (a YUV layer plus a transformed,
|
||||
semi-transparent RGBA layer) on the GPU and on the CPU. It requires a 99th-percentile channel
|
||||
difference of ≤ 6 and a mean of < 1.5 (8-bit levels). It skips when no GPU adapter is available.
|
||||
- Effects implemented only on the CPU need no GPU test: the plan pre-renders those layers on the
|
||||
CPU.
|
||||
- Golden-image files (rendered PNGs compared against committed references) are not used yet. If you
|
||||
add them, generate the references from FilmCraft itself, give each one an attribution sidecar,
|
||||
and document the tolerance (PSNR in dB, or max/percentile error).
|
||||
|
||||
## 4. UI: control channel and screenshots
|
||||
|
||||
There is no automated UI test suite yet. UI work is checked by driving the real app:
|
||||
|
||||
```sh
|
||||
cargo run --release -p filmcraft -- --control 9876
|
||||
```
|
||||
|
||||
Then script it over the control channel or MCP: run commands, click by automation id,
|
||||
`ui.inspect` / `ui.elements` to assert state, and `ui.screenshot {"path": "…"}` to capture the window
|
||||
or a single panel (`{"panel": "Timeline"}`). Look at every screenshot. Examples are in
|
||||
[agents.md](agents.md). The headless path (`filmcraft-cli run script.jsonl`, MCP `--demo`) covers
|
||||
engine behaviour without a window.
|
||||
|
||||
## 5. Performance
|
||||
|
||||
| Benchmark | Command |
|
||||
|---|---|
|
||||
| H.264 decode, 1080p, threads sweep | `cargo test --release -p filmcraft-h264 --test perf -- --ignored --nocapture` |
|
||||
| HEVC decode, 1080p / 2160p | `cargo test --release -p filmcraft-hevc --test perf -- --ignored --nocapture` |
|
||||
| ProRes decode/encode | `cargo test --release -p filmcraft-prores --test perf -- --ignored --nocapture` |
|
||||
| Encoder speed / PSNR / bitrate | `cargo run --release -p filmcraft-h264enc --example h264enc_synth -- …` |
|
||||
| Decode a real file through the media stack | `cargo run --release -p filmcraft-cli -- bench-decode file.mp4 --frames 120` |
|
||||
| Coding-tool coverage of the fixtures | `cargo test --release -p filmcraft-h264 --test conformance coverage_report -- --ignored --nocapture` (same for `hevc`) |
|
||||
|
||||
The perf tests check bit-exactness before they time anything. Results go in the crate README's
|
||||
performance table, with machine and thread count. Headline numbers go in
|
||||
[ROADMAP.md](../ROADMAP.md). Measure on an idle machine: parallel agent builds distort timings.
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
//! Workspace automation: `cargo xtask <layers|wasm|ci>`.
|
||||
//!
|
||||
//! - `layers`: enforces the dependency layering of `plan/architecture.md` §3 (downward-only edges,
|
||||
//! - `layers`: enforces the dependency layering of `docs/architecture.md` §1 (downward-only edges,
|
||||
//! listed same-layer edges, L0 codec crates depend on `bitstream` only, no UI/OS crates below L5).
|
||||
//! - `wasm`: `cargo check --target wasm32-unknown-unknown` for every crate in L0–L4.
|
||||
//! - `assets`: every asset file (image, icon, font, LUT, audio, video…) has a complete
|
||||
|
|
|
|||
Loading…
Reference in New Issue