Keep FLAC beside main's new AAC Audio format (21 formats, both audio-only
presets), use is_mp4_with / is_mp4_video in audio_codec() and the Audio
Format picker so MP4 (incl. AV1) offers AAC or FLAC, and add flacLevel to
main's export.media params.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: `cargo xtask parity` (#781, #782) measured only the menu bar; the
Effects panel's effects and transitions had no measured parity figure
(docs/gaps.md G7).
Fix: `--area effects|transitions` compares the catalogue the engine's
effect definitions list in the Effects panel (new `effect_catalog`
example in filmcraft-project, the panel's own top-folder filter) with a
reference of folder paths or bare names (default the local, never
committed plan/premiere/<area>.json). Same report, caps and `--min` as
menus; placement by folder; Presets / Lumetri Presets and the other
area's items are out of scope, so one panel snapshot serves both areas.
`--area menus` (default) is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: symphonia rejects every Layer II frame whose bit rate / mode
combination ISO/IEC 11172-3 does not allow (mono above 192 kbit/s, stereo
below 64 kbit/s). ffmpeg writes mono MP2 at 384 kbit/s by default, so such
tracks played as silence in MPEG-TS / PS, Matroska and raw .mp2 files.
Fix: new L0 crate filmcraft-mpa, a clean-room Layer I / II decoder written
from ISO/IEC 11172-3 and 13818-3 (all modes incl. joint stereo, MPEG-1/2/2.5
rates, the disallowed combinations decoded with the allocation table their
bit rate per channel selects). PacketDecoder::mpeg_audio uses it for layers
I and II (packets of several frames too); raw .mp2 files decode with it,
resyncing after damage. Oracle tests vs ffmpeg: 73-84 dB SNR on ffmpeg /
twolame streams and synthetic Layer I / joint stereo frames; fuzzed damaged
frames never panic.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: the menu parity figure in docs/target-app-parity.md came from an
ad hoc label-matching script over string literals; nothing repeatable
measured the real menu bar (docs/gaps.md G7).
Fix: `cargo xtask parity` dumps the shipped menu bar (new
`menus::default_menu_items`, deterministic: English, default prefs,
built-in workspaces; printed by `examples/menu_tree.rs`) and compares it
with a reference list of menu paths, by default the local, never-committed
`plan/premiere/menus.json` (any JSON of paths via --reference; parsing is
depth- and size-capped). Presence and placement are reported separately
(same path, elsewhere, missing); app/Help/recent/account/macOS items are
out of scope. Writes a markdown checklist (target/parity/ or --out) and
--min fails below a presence threshold. Without a reference it prints our
inventory only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: the macOS bundle had no NSMicrophoneUsageDescription and empty
hardened-runtime entitlements, so macOS never asked for microphone access
and voice-over recording got no input.
Fix: declare NSMicrophoneUsageDescription in Info.plist.in and grant
com.apple.security.device.audio-input in entitlements.plist; an xtask test
guards both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No conflicts. main's new Chinese catalog requires every tl! string:
added zh-CN for the FLAC labels (format, preset, Audio Format entry,
Compression Level and its hint).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New L0 crate written from the AVI RIFF file reference and OpenDML 1.02:
avih and per-stream strh / strf (BITMAPINFOHEADER, WAVEFORMATEX(TENSIBLE))
/ strn, and chunk tables from the best index a file has: the OpenDML indx
super index and ix## standard indexes (field indexes too), else idx1
(offsets relative to movi or absolute) plus a scan of RIFF AVIX
extensions it cannot see, else a scan of the movi lists (key frames then
unknown). Frames are read on demand through a ByteSource.
Sizes are clamped to the file, so a cut-off recording opens with what
is there; rec lists nest at most 3 deep and tables are bounded by what
the file could hold. Tests: hand-built files for every index path,
truncation, and every single-byte mutation and truncation (no panic);
ffmpeg-made files against ffprobe; an ignored test on a 1.16 GB OpenDML
file with AVIX extensions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New L0 crate filmcraft-flac, a FLAC encoder written from RFC 9639:
constant / verbatim / fixed / LPC subframes (Levinson-Durbin on Tukey-
windowed autocorrelation, up to order 12), partitioned Rice residuals,
left/side, side/right and mid/side stereo, CRC-8/CRC-16, STREAMINFO with
MD5 (RFC 1321, own implementation), levels 0-8, all lossless.
Export:
- Format::Flac, an audio-only .flac (16/24-bit), streamed one second at
a time like WAV/AIFF; STREAMINFO is rewritten at the end with frame
sizes, total samples and MD5. Preset "FLAC 48 kHz 24-bit".
- AudioCodec::Flac in MP4 and QuickTime (fLaC + dfLa), one frame per
sample with the short last frame timed exactly; the dfLa STREAMINFO
is replaced before the moov is written (Mp4Writer::set_sample_entry).
MXF stays PCM.
- AudioSettings::flac_level (default 5); Export panel: FLAC (lossless)
audio format and Compression Level; file.exportMedia `format: "flac"`,
`audioCodec`, `flacLevel`. Spanish and Japanese strings.
ffmpeg oracle: every stream decodes bit-exactly with the STREAMINFO MD5
equal to ffmpeg's hash; .flac and FLAC-in-MP4/MOV exports decode to the
same samples as a WAV export of the range.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- filmcraft-tts (feature kokoro): Kokoro-82M on candle, implemented from
the StyleTTS 2 / iSTFTNet / ALBERT papers, config.json and the
checkpoint's tensor shapes; official .pth read with candle's restricted
pickle reader; voice packs from their zip. Neural Voice: tts-text
phonemize -> Kokoro per chunk -> pauses -> pitch via audio-dsp.
- catalog: 9 US English voices (model card grade C+ or better) + CMUdict,
every file pinned to a revision and SHA-256 (336 MB).
- speech: generic download_files (same verified downloader).
- engine: tts.render (background synthesis into Session::tts_cache),
tts.downloadVoices (background job, cancellable), voices list with
package state; feature neural-voices (default on in the desktop app).
- ui: natural/basic voice groups, consent box (size, sources, licences),
download progress + Cancel, synthesis never blocks the window;
ui.set {tts:{...}} for agents.
Measured: Whisper-base WER on 50 original sentences 2.1-3.6 % for all 9
voices (built-in robotic voice 96.6 %); 2.2-2.3x real time on CPU.
Gates: fmt, clippy, layers, assets, wasm pass; tests pass except
shortcuts_tests::assign_reassigns_conflicts_and_undoes (fails on main)
and the frame pool test (flaky under load, passes alone).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New L2 crate filmcraft-tts-text: normalize(script, lang) -> Vec<Token> for the
Text to Speech text front end (P2a). English only (EnUs/EnGb), per the amended
brief. Handles cardinals to 999 999 999 999, negatives, decimals, ordinals,
money ($, €, £), percent, years (1100-2099), clock times, numeric and
abbreviated dates, phone-like digit groups, Mr./Mrs./Dr./St./Mt./vs./etc./e.g./
i.e./a.m./p.m., spelled acronyms, &/+@#/, and [pause Ns|ms] markers capped at
10 s. Pure Rust, deps serde + thiserror only, wasm-clean.
Never crashes (AGENTS.md §0): >1 MiB is NormalizeError::TooLong, oversized and
leading-zero numbers are read digit by digit, malformed pause markers become
plain text, all slicing is get()/boundary-safe, and a catch_unwind suite covers
empty/whitespace/oversized/10k-digit/$$$$/mixed-script/emoji/RTL/combining/NUL
inputs plus determinism.
Fix round (Claude review): a '.' after a number now ends the sentence (year
rule and money/decimal follow-through), so "in 1999." reads as a year with a
SentenceEnd and "$1,234,567." gets its final SentenceEnd; new table cases for
sentence-final numbers and the year rule under punctuation (., ,, !, ()); the
remaining direct indexes in non-test code converted to get()/first()/last().
Gates: fmt, clippy -p filmcraft-tts-text --all-targets -D warnings, tests
(25 suites, ~140 table cases), cargo xtask layers, cargo xtask wasm all green.
cargo xtask ci not run this round (Claude runs it when merging); it stops on a
pre-existing filmcraft-engine shortcuts test failure unrelated to this crate.
🤖 Written by Merlin (Hermes Agent, profile merlin)
Write what a narrator says, pick a voice, pitch and pace, and add it to
the timeline. Selecting a narration clip loads it back into the panel;
Save regenerates it in place as one undo step, keeping the clip's start
and length (longer speech kept whole in the file, shorter speech padded
with silence).
- filmcraft-tts (L2): Voice trait, voice catalogue, pause markers
([pause 1s]), and an original formant synthesizer (Basic Female/Male;
robotic placeholder until the neural voices land).
- project: Narration + Project::narrations (schema v13, no-op migration).
- engine: tts.voices/preview/create/edit/inspect; narrations never
overwrite clips (free targeted track, free track, else a new track).
- ui: Text to Speech panel beside Properties, preview playback,
click-to-edit, automation ids.
Gates: fmt, clippy -D warnings, layers, assets and wasm (rustup
toolchain) pass. Tests: all pass except three that fail identically on
main (61238bf) on this machine: shortcuts_tests::assign_reassigns_
conflicts_and_undoes, gpu fx_tests::gpu_effects_match_cpu_exactly, and
the frame pool test, which passes when run alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Desktop/shared:
- Frame workers catch panics per job: a failing decode/render no longer kills the worker
or leaves its frame in flight forever (blank monitors). Counted as perf.stats failedJobs.
- UI pass catches panics and shows a "FilmCraft hit an error" window (Save Project /
Continue) instead of closing the app.
- Panic hook writes <data dir>/Logs/crash-<day>.log with thread, location and backtrace;
a window that can't start shows an error dialog instead of exiting silently.
- GPU compositor only on adapters that render/blend Rgba16Float with >= 4096 textures;
GPU validation errors and oversize frames switch to the CPU compositor instead of
wgpu's default panic.
- Dock layout and Media Browser splitter no longer panic on tiny panels/windows.
Web:
- Export mode / previews / voice-over / log reveal no longer call std::env::temp_dir
(panics on wasm32): new filmcraft_engine::temp_dir(); default export dir /exports.
- Fatal-error overlay with Reload; WebGPU device failure retries on WebGL2 (?webgl);
cache-busted wasm/glue URLs; webcodecs entries[0] guard.
Tests: robustness_ui (worker survives injected panics, UI panic recovery, tiny windows),
crash hook, dock split, GPU plan size, default export dir; web smoke test extended.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- assets/app-icon/: canonical SVG + light SVG, 1024 PNG, macOS runtime PNG, .icns, .ico,
hicolor theme, LICENSE.txt and README (palette, geometry, provenance, regeneration).
- packaging/icons.sh regenerates everything with resvg, iconutil and the new `cargo xtask ico`.
- Native app sets the window icon (with_icon) and the Wayland app id; build.rs embeds the .ico
and VERSIONINFO into filmcraft.exe on Windows (winresource, Windows-host build-dep only).
- packaging/linux/ai.storyteller.filmcraft.desktop with Icon=ai.storyteller.filmcraft.
- Web app: favicon.svg replaced by favicon.png from the same artwork; sidecars and
ATTRIBUTION.md updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decode fps per codec (H.264, HEVC, VP9, AV1, ProRes at 1080p/2160p),
Program-monitor playback and scrubbing through the real frame scheduler,
timeline UI frame time with a 1000-clip / 20-track sequence under
egui_kittest (+ wgpu render), export throughput (H.264 + AAC, ProRes),
project save/open of 5000 clips, and peak RSS per section (each section
runs in its own process under /usr/bin/time). Writes JSON and Markdown
to target/bench. bench_playback now shares its code (examples/bench/).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- crates/speech (L2): Transcriber trait, FixedTranscriber, Whisper model catalogue (pinned,
SHA-256 verified) with optional downloader (feature `download`: ureq + rustls + RustCrypto),
pure-Rust Whisper inference on candle with word timestamps (feature `whisper`), log-mel, VAD
word tightening, MFCC speaker clustering. Both features off by default and for wasm.
- project: per-media-item transcripts (Project::transcripts); schema v9 with a no-op v8->v9
migration and a v8 fixture.
- edit::transcript: sequence transcript derived from clip transcripts, search, paragraphs,
word ranges, pause/filler detection, ripple delete of many ranges, caption layout.
- engine: transcript.* commands (generate, set, delete, inspect, search, select, extract, lift,
renameSpeaker, removeFillers, removePauses, createCaptions, models, downloadModel);
Session::transcriber for hosts/tests; features `whisper` / `speech-download` (also on the
desktop and CLI apps). Without whisper, transcript.generate fails with a clear error.
- ui: Text panel Transcript tab (Transcribe button, speaker paragraphs of clickable words,
Shift+click selection -> In/Out, extract/lift/remove fillers/create captions, search) with
text.transcript.* automation ids; headless UI test.
- docs/transcripts.md, crates/speech/README.md, architecture and project-files updates.
whisper-tiny measures 12.2% WER over 797 words on local samples.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- eframe web runner on WebGPU (WebGL2 fallback; GPU compositor on WebGPU, CPU on WebGL2).
- fs: virtual file table of File/Blob handles, 1 MiB chunked async range reads with an
LRU cache and read-ahead (pending + retry instead of blocking), downloads for written
files, /opfs/ paths written to OPFS.
- import: File System Access picker / <input type=file>, drag-and-drop anywhere,
container index prefetch (moov wherever it is), retried imports; project open/save.
- recovery: OPFS auto-save snapshot while dirty, media copies, reopened on next visit.
- audio: AudioWorklet output fed ahead from the UI thread; played-frame audio clock.
- webcodecs: H.264/HEVC/VP9/AV1 MP4/MOV decoded by the browser VideoDecoder via a
reader opener (fallback to our decoders on any error), probed with isConfigSupported.
- api: window.filmcraft (execute/request/commands/screenshot/importFiles/importUrl/
openProject/files/readFile/info).
- cargo xtask web [--dev] [--serve PORT] (wasm-bindgen CLI); xtask wasm also checks
filmcraft-ui-egui and filmcraft-web; layer L6 for the web app.
- Original web icon (favicon.svg) with attribution sidecar; docs/web.md; headless-Chrome
smoke test (tests/smoke.mjs).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New L0 crate filmcraft-dnx, written from SMPTE ST 2019-1:2016 + Amd 1:2023
(tables extracted from the standard's text by tools/extract_tables.py).
Decoder: every compression ID (DNxHD 1235-1260 incl. field-coded and
macroblock-adaptive interlace, thin rasters, RGB 4:4:4; DNxHR 1270-1274
LB/SQ/HQ/HQX/444), 8/10/12-bit, 4:2:0/4:2:2/4:4:4, DCT and lossless alpha,
macroblock rows in parallel. Matches ffmpeg within IDCT precision (max 2 LSB,
<=50 per million beyond 1, 87-99% identical) on 22 ffmpeg fixtures; follows
ffmpeg where it departs from the standard's text (reconstruction offset,
CID 1271 weights, 4:4:4 signalling) - documented in the README.
Encoder: DNxHR LB/SQ/HQ/HQX(10/12-bit)/444, CBR at the eq. 7.1 frame sizes,
uniform-qsf binary search + per-macroblock refinement; ffmpeg decodes every
profile with -xerror (HQ luma PSNR 58.8-66.3 dB).
Wiring: MOV AVdn/AVdh import (codec label from the CID), export
Format::DnxHr (MOV AVdh, dnxProfile lb|sq|hq|hqx) in the export dialog and
file.exportMedia.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New L1 crate filmcraft-text: fonts (bundled Inter Regular/Medium/SemiBold/Bold/Italic,
JetBrains Mono, Noto Serif; DirectorySource scans system font folders reading only name/OS2
tables; family/style resolution with synthetic bold/italic; per-char fallback), layout
(harfrust kerning/ligatures, unicode-bidi, unicode-linebreak, alignment incl. justify,
leading, tracking, baseline shift, all/small caps, underline, carets/hit-test/selection,
cached), raster (moved from captions; active-edge list, shapes, emboldening), render
(outline + sub-pixel glyph-mask caches; transformed-outline path for rotation/skew),
mask (distance-transform strokes, blur).
- Captions burn-in now shapes and kerns via the engine (font.rs/raster.rs removed).
- Timecode / Clip Name effects draw real text (full clip names, not seven-segment digits);
text_burnin golden re-blessed.
- UI takes its font bytes from the text crate (no duplicate copies in the binary).
- New fonts: Inter Bold, Inter Italic (OFL-1.1, rsms/inter v4.1), Noto Serif Regular
(OFL-1.1, notofonts) with sidecars, ATTRIBUTION.md rows and licence text.
- Perf: 3-line 96 px title at 1080p ~1.6 ms cold / 0.15 ms warm (release).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- crates/ui-egui/examples/bench_playback.rs drives the Program monitor's own FrameServer,
prefetch (now FrameServer::schedule_playback, shared with the monitor) and PlaybackMeter at a
simulated 60 Hz display, optionally compositing on a headless wgpu device. Scenarios:
h264-1080, stack3 (3 × 1080p, scaled/rotated/opacity), h264-2160, demo, after-preview
(live play → Render Effects In to Out → 2 plays), seek-storm. Reports latency percentiles,
on-time ratio, worker/decode wall and thread-CPU time, UI-thread present time, process CPU per
frame, decoder seeks/samples (codecs::gop_stats) and wasted jobs; --json for machine output.
Fixtures are generated with ffmpeg into target/fixtures/playback.
- FrameServer records per-job timings (thread CPU via rustix clocks) while profiling is on.
- Dropped/shown were counted per UI refresh: "shown" counted every refresh that had the frame
(322 "shown" in an 8 s play of 192 frames) and "dropped" counted a frame that arrived a few ms
after its first refresh. PlaybackMeter counts timeline frames: shown when the exact picture was
on screen while due, dropped otherwise, including frames passed over without a refresh.
Baseline (GPU path, 8 s, load avg 104–168 on 14 cores; CPU columns are load-robust):
h264-1080 Full 191/1 shown/dropped, 46 ms CPU/frame; Half 188/4
stack3 Full 189/3, 102 ms CPU/frame; Half 157/35 (3 extra decoder seeks)
h264-2160 Full 8/184, 313 ms CPU/frame, 10 seeks; demo Full 114/78, Half 191/1
after-preview: live 0/192; 1st play after render 124/67, 2nd 119/73 (UI-thread present
p50 21 ms / p95 45 ms); seek-storm: 0 of 40 seeks shown within 150 ms
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
crates/golden (test-only, L3): nine procedural 320x180 scenes (transform
and opacity, blend modes, Gaussian Blur, Lumetri basic, Crop, Cross
Dissolve / Dip to Black / Wipe mid-transition, timecode/clip-name text)
rendered on the CPU and compared with committed goldens (PSNR >= 45 dB,
max <= 12, p99 <= 2); FILMCRAFT_BLESS=1 regenerates them with
attribution sidecars. testkit::golden adds PNG I/O, diff metrics and
failure images. The same scenes are composited on the GPU and compared
with the CPU.
GPU parity exposed a real bug: GpuCompositor cached uploads by
pixel-buffer address without owning the buffer, so a new frame
allocated at a freed frame's address was drawn with a stale texture
(whole wrong frames, PSNR ~12 dB). The cache now keeps the buffer
alive; regression test upload_cache_keeps_buffers_alive.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Runs each crate's ignored generate_fixtures test (h264, hevc, isobmff,
matroska, prores) - the same generators the oracle tests use - and
prints made/cached/skipped per fixture plus a summary. Optional crate
filter: cargo xtask fixtures hevc prores.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New dev-dependency-only crate crates/testkit (L0, no FilmCraft deps):
FILMCRAFT_FFMPEG/FILMCRAFT_FFPROBE, then PATH (incl. .exe), then known
dirs; SKIPPED lines and FILMCRAFT_REQUIRE_ORACLES=1 to make skips fail;
require_ffmpeg!/require_ffprobe!/require_oracles!; fixtures_dir() under
<workspace>/target/fixtures with per-thread temp names and atomic
generation. Oracle helpers in aac, opus, h264, hevc, h264enc, isobmff,
matroska, prores and export migrated (codecs left on its local lookup).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Public equivalents of the maintainer-local plan/: docs/architecture.md (layers as enforced by
xtask, Tick time base, data model, command system and undo, render/playback/export pipeline,
automation surfaces), docs/contributing.md (+ root CONTRIBUTING.md), docs/testing.md and
docs/agents.md (MCP/control channel usage, agent work loop, decision rules, Definition of Done).
CLAUDE.md points at the public docs wherever it references plan/; AGENTS.md gains a See also list;
README gains a Documentation section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brandon Thomas
vangel76
JordyUSA
jahvari
skylar taylor-barrick
Manuel Camacho
mweinbach