All 27 blend modes now composite on the GPU instead of sending the
whole frame to the CPU.
- render::plan: PlanLayer carries its Blend; plan_frame no longer falls
back to a CPU frame for non-Normal modes. Media, graphic clips,
multicam angles and CPU-rendered layer images (standard effects) keep
their clip's mode; inside a transition the mode is Normal, as in the
CPU reference. execute_cpu composites with each layer's mode.
- gpu: Normal keeps the single fixed-function pass. Dissolve also uses
fixed-function "over": the CPU's 64-bit pixel hash is reproduced in
WGSL on u32 pairs (pattern bit-exact). The other 25 modes copy the
accumulator region under the layer's quad into a backdrop texture and
composite in fs_blend with the CPU math (sRGB-encoded straight colour,
W3C formulas, back to linear, same premultiplied mix and clamping).
- Tests: per-mode GPU-vs-CPU parity (gpu crate: overlapping transformed
layers, partial opacity/alpha, empty/partial backdrop; golden: every
mode through plan_frame incl. an effect layer, blended clips in a
cross dissolve), exact-value edge cases (black/white, Divide by 0,
Dodge/Burn at 0 and 1, zero alpha) within 4.9e-4, Dissolve pattern
exact, off-output layers, quad bounds.
- Bench: new stack3-blend playback scenario (V2 Screen, V3 Overlay);
CPU ms/frame 151-214 -> 86-89, Full 19/173 and 138/54 -> 191/1, 192/0.
- Docs: architecture §5, gpu crate docs, testing, performance.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Lumetri grades PQ / HLG working spaces in their own signal normalised to
HDR White (Basic Correction) / HDR Range (Curves) in cd/m²
(filmcraft_color::GradeSpace): exposure in stops of light, highlights above
HDR White kept, HDR Specular scales them; Rec. 709 unchanged.
- HDR static metadata (mdcv/clli, Matroska MasteringMetadata/MaxCLL/MaxFALL)
on VideoStreamInfo::hdr; PQ media use MaxCLL / mastering peak as the
BT.2390 source peak; media.colorInfo reports it.
- HSL Secondary Refine: Denoise (median of the key) and Blur (Gaussian).
- Lumetri Presets (18 original looks in 4 folders) with thumbnails rendered by
our Lumetri on a procedural preview picture; Effects panel rows and a
thumbnail grid when wide; lumetri.presets / applyPreset / presetThumbnails.
- Color Match: Comparison View toggle; Apply Match uses its reference frame.
- Goldens lumetri_hdr_pq and lumetri_presets_grid; D-Log M stays unsupported
(no published formula).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
filmcraft-mxf gains a streaming writer (ST 377-1 partitions, primer, header
metadata, VBR index segments with temporal offsets, RIP; ST 378 OP1a
frame wrapping, ST 390 OP-Atom clip wrapping) for VC-3, ProRes, AVC byte
stream and PCM with start timecode and caller-supplied package UMIDs, plus
write_opatom_pcm. filmcraft-export adds mxf-op1a / mxf-opatom formats
(DNxHR, ProRes or H.264 + PCM; OP-Atom writes one mono file per channel),
presets, the engine mxfVideoCodec parameter and CLI .mxf inference.
Tests read our files back and check them against ffprobe / ffmpeg.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
List / Icon / Freeform views, Metadata Display, view presets, font size,
Preview Area, bin tabs and windows, hover-scrub In/Out; Media Browser tree,
Favorites, history, file types, thumbnails, Edit Columns, import and drag.
Engine: project_panel and media_browser modules, Services::list_entries.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The column pass and the add/clip of idct4/idct8 work on whole rows so they
vectorise (same integer steps in the spec's row-then-column order; checked
against the per-column formulation on random blocks). Docs: draft decoding in
architecture.md and perf.stats fields in control-protocol.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5.1 / multichannel:
- audio-dsp::channels: layouts (L R C LFE Ls Rs), ITU-R BS.775 up/downmix, 5.1 Mixdown Types,
equal-power 5.1 panner (puck, Center %, LFE).
- Mixer graph buses are 2 or 6 channels wide; 5.1 tracks/submixes/Mix; strips feeding a 5.1 bus
use the 5.1 panner (lanes pan51.x/y/center/lfe stored in MixerStrip::lanes, no schema change);
5.1 strips fold into stereo buses with BS.775; per-channel meters; clip effects on 6 channels.
- file.newSequence {mix, trackType}, sequence.settings {mix}; playback honours the 5.1 Mixdown
Type preference on stereo devices and plays 6 channels on 5.1 devices.
- Export Audio Channels 5.1: WAV WAVE_FORMAT_EXTENSIBLE (mask 0x3F), QuickTime PCM with a chan
atom, AAC channel configuration 6 (reordered to C L R Ls Rs LFE).
- Track Mixer: 5.1 panner puck with Center/LFE, N-channel meters, Show/Hide Tracks, Meter
Input(s) Only; transport record button runs voice-over.
Voice-over: engine AudioInput trait + SyntheticInput, cpal input in the app, audio.voiceover.*
commands (punch-in, pre/post-roll, countdown cues, WAV next to project / scratch disk, one undo
step), track header mic button and Voice-Over Record Settings dialog.
Remix: beat tracking + chroma/cepstra self-similarity planner in audio-dsp, hidden `remix` clip
effect holding the plan, render::remix playback with equal-power crossfades, clip.remix.*
commands, Remix tool and Remix Properties dialog.
Clip Mixer automation: per-track Latch/Touch/Write (clipMixer.setMode/touch/release) heard live
and written as thinned clip Volume/Panner keyframes at the end of the pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- HEVC inverse transform: sum basis rows of non-zero coefficients with a
const-generic size (bit-exact, tested against the direct product);
SAO edge offset gets a check-free interior loop. HEVC CPU/frame -60%.
- H.264/HEVC/VP9 planes are moved into VideoFrames, not copied.
- Catch-up decoding: frame jobs tell sources which frames are late
(filmcraft_media::cancel::with_catch_up); the GOP cache skips
non-reference samples of late frames (H.264 nal_ref_idc 0, HEVC
sub-layer non-reference pictures). Scrubs keep the last 2 s decoded.
- perf.stats: engine query + UI/control method (decode counters,
playback shown/dropped, frame-worker decode/render ms, cache use).
- Timeline waveform source peak cached; bench seek section and --only.
- docs/performance.md with before/after results.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- crates/speech (L2): Transcriber trait, FixedTranscriber, Whisper model catalogue (pinned,
SHA-256 verified) with optional downloader (feature `download`: ureq + rustls + RustCrypto),
pure-Rust Whisper inference on candle with word timestamps (feature `whisper`), log-mel, VAD
word tightening, MFCC speaker clustering. Both features off by default and for wasm.
- project: per-media-item transcripts (Project::transcripts); schema v9 with a no-op v8->v9
migration and a v8 fixture.
- edit::transcript: sequence transcript derived from clip transcripts, search, paragraphs,
word ranges, pause/filler detection, ripple delete of many ranges, caption layout.
- engine: transcript.* commands (generate, set, delete, inspect, search, select, extract, lift,
renameSpeaker, removeFillers, removePauses, createCaptions, models, downloadModel);
Session::transcriber for hosts/tests; features `whisper` / `speech-download` (also on the
desktop and CLI apps). Without whisper, transcript.generate fails with a clear error.
- ui: Text panel Transcript tab (Transcribe button, speaker paragraphs of clickable words,
Shift+click selection -> In/Out, extract/lift/remove fillers/create captions, search) with
text.transcript.* automation ids; headless UI test.
- docs/transcripts.md, crates/speech/README.md, architecture and project-files updates.
whisper-tiny measures 12.2% WER over 797 words on local samples.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- eframe web runner on WebGPU (WebGL2 fallback; GPU compositor on WebGPU, CPU on WebGL2).
- fs: virtual file table of File/Blob handles, 1 MiB chunked async range reads with an
LRU cache and read-ahead (pending + retry instead of blocking), downloads for written
files, /opfs/ paths written to OPFS.
- import: File System Access picker / <input type=file>, drag-and-drop anywhere,
container index prefetch (moov wherever it is), retried imports; project open/save.
- recovery: OPFS auto-save snapshot while dirty, media copies, reopened on next visit.
- audio: AudioWorklet output fed ahead from the UI thread; played-frame audio clock.
- webcodecs: H.264/HEVC/VP9/AV1 MP4/MOV decoded by the browser VideoDecoder via a
reader opener (fallback to our decoders on any error), probed with isConfigSupported.
- api: window.filmcraft (execute/request/commands/screenshot/importFiles/importUrl/
openProject/files/readFile/info).
- cargo xtask web [--dev] [--serve PORT] (wasm-bindgen CLI); xtask wasm also checks
filmcraft-ui-egui and filmcraft-web; layer L6 for the web app.
- Original web icon (favicon.svg) with attribution sidecar; docs/web.md; headless-Chrome
smoke test (tests/smoke.mjs).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Av1Decoder in filmcraft-codecs: feeds the av1C configuration OBUs before the
first sample (and after every reset), converts pictures to 8/16-bit planar
YUV with the stream's colour description, and reports random-access points
from the first frame header of each temporal unit (shown key frame) so GOP
seeking ignores bogus container sync flags. Matroska V_AV1 tracks map to an
av01 sample entry; MP4 labels show the AV1 profile.
All-intra 8-bit MP4 and 10-bit WebM files seek sample-exactly against
libdav1d. Inter frames still return "unsupported" until inter prediction
lands.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
masks.track {direction, frames?, method?, wait?}: progress + cancel via jobs.*, keyframes written live as one undo step per run. Synthetic known-motion tests: <0.75 px drift over 10 frames (render), 0.29 px max vertex error over -8..+12 frames end to end (engine).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New L0 crate filmcraft-dnx, written from SMPTE ST 2019-1:2016 + Amd 1:2023
(tables extracted from the standard's text by tools/extract_tables.py).
Decoder: every compression ID (DNxHD 1235-1260 incl. field-coded and
macroblock-adaptive interlace, thin rasters, RGB 4:4:4; DNxHR 1270-1274
LB/SQ/HQ/HQX/444), 8/10/12-bit, 4:2:0/4:2:2/4:4:4, DCT and lossless alpha,
macroblock rows in parallel. Matches ffmpeg within IDCT precision (max 2 LSB,
<=50 per million beyond 1, 87-99% identical) on 22 ffmpeg fixtures; follows
ffmpeg where it departs from the standard's text (reconstruction offset,
CID 1271 weights, 4:4:4 signalling) - documented in the README.
Encoder: DNxHR LB/SQ/HQ/HQX(10/12-bit)/444, CBR at the eq. 7.1 frame sizes,
uniform-qsf binary search + per-macroblock refinement; ffmpeg decodes every
profile with -xerror (HQ luma PSNR 58.8-66.3 dB).
Wiring: MOV AVdn/AVdh import (codec label from the CID), export
Format::DnxHr (MOV AVdh, dnxProfile lb|sq|hq|hqx) in the export dialog and
file.exportMedia.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- media.createProxies: background job (jobs.list progress) transcoding with our own
encoders — ProRes 422 Proxy ¼/½, ProRes LT ½, H.264 ¼/½ (audio included) — into
<media dir>/Proxies or a chosen folder; attached when the job finishes
(Session::poll_persistence applies finished media jobs).
- media.attachProxies (duration ±1 frame and frame rate must match; size/aspect may
differ), media.detachProxies, media.reconnectFullRes, media.toggleProxies
(Preferences ▸ Media ▸ enableProxies; bumps the view revision without dirtying the
project), media.proxyPresets.
- Proxy frames are smaller; ProxySource reports the original size and the compositor
derives its pixel scale from the frame, so Motion and effects render the same
picture (44 dB PSNR vs full res with Motion + Gaussian Blur, 48 dB at ½).
Export always renders full resolution.
- ExportSettings::prores_profile (proxy / lt / standard / hq).
- project.ingestSettings + file.import: copy (fingerprint-verified), transcode,
create proxies, copy and create proxies.
- UI: Toggle Proxies button on both monitors (lit when on), Create Proxies dialog,
Project panel Proxy submenu and P badges.
- Measured (one core, 4K H.264 testsrc2): CPU composite 201 → 35 ms/frame at ¼ with
ProRes Proxy ¼, 277 → 131 ms at ½ with H.264 ¼; decode 9.2 → 0.7 ms with H.264 ¼.
- Tests: create+attach+toggle same composite modulo scale, attach checks, export
ignores proxies, ingest copy + proxies, ignored 4K benchmark, headless UI test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Missing media no longer fails: opening a project scans for missing files
(`media.findMissing`, `file.open` → missingMedia). Missing clips render our own
offline slate (render::offline: red striped field, warning triangle, "MEDIA NOT
FOUND", file name, hint) with silent audio, and the Link Media dialog opens.
- Media identity: imports record the size and a fast content fingerprint (hash of
size + first/last MiB) in MediaClip::identity (optional field, older builds
ignore it, no schema bump).
- media.relink checks file name / extension / fingerprint ("Clip ID") / duration /
media start / frame size+rate+channels, refuses a mismatched fingerprint unless
forced, supports Align Timecode (Project::shift_media_time), and relinks the
others automatically by the derived folder remap (Windows/macOS separators).
- media.autoRelink {from,to} | {folder}, media.search, media.replaceFootage,
media.makeOffline {deleteFiles?}, media.offlineAll, media.status, media.linkMedia.
- Media pool caches sources per reference (path, offline flag), so relink, Make
Offline and undo take effect at once; proxy-aware resolution and a full-res
provider that export uses.
- codecs::gop no longer holds its lock while decoding: rayon decoders inside a
parallel export could steal another frame of the same source and deadlock.
- UI: Link Media dialog (rows, match options, folder search with preview, Locate…,
Link, Offline, Offline All, Cancel), Make Offline dialog, Project panel offline
markers and context menu, media thumbnails keyed by media reference.
- Tests: moved media opens offline (slate rendered) and relinks others by folder,
prefix remap and folder search, fingerprint mismatch refused, Make Offline,
headless UI test of the dialog.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- model: SequenceSettings::color (ColorPipeline: working space, wide gamut, Auto Tone Map Media;
serde default Rec. 709) and Interpretation::color_space (override; None = metadata)
- render::colorman: source colour space from the override or VUI/colr/MKV metadata, decode tables
(camera log with super-whites, PQ, HLG + OOTF), gamut matrix, tone mapping into SDR working
spaces, gamut mapping; working → SDR monitor transform at the top level
(RenderOptions::working_output keeps working values for HDR export and scopes). Plain Rec. 709
media in a Rec. 709 sequence keeps the old fast path and stays on the GPU; managed layers and
HDR/wide-gamut sequences composite on the CPU
- export: H.264 and ProRes of a PQ/HLG sequence encode BT.2020 PQ/HLG and signal it: VUI / ProRes
header, colr (nclx/nclc), and for PQ mdcv + clli boxes and H.264 SEI (payloadTypes 137/144);
`sdr: true` (and render previews) export tone-mapped SDR. isobmff reads/writes mdcv and clli
- commands: sequence.colorSettings (Sequence ▸ Color Management…), clip.interpretFootage
(Clip ▸ Modify ▸ Interpret Footage…, Project panel context menu), color.spaces,
media.colorInfo; dialogs with automation ids colorDialog.*
- scopes: HDR sequences show a cd/m² waveform on a PQ scale (203 marked) and a BT.2020
vectorscope from the working-space render (scopes.hdrWaveform)
- display-referred wide-gamut input is desaturated into gamut keeping in-gamut colours exactly;
the soft compression is reserved for camera gamuts
- tests: colorman unit tests, engine (interpret/undo, PQ working space values and display),
goldens log_to_rec709 and hdr_tone_map (own renders + sidecars), export PQ/HLG × ProRes/H.264
read back by our demuxer and ffprobe (color_transfer/primaries/space, mastering display and
content light side data), headless UI (dialogs, HDR scopes)
- docs: architecture §5.2 (pipeline, HDR export, display colour management: no macOS EDR yet)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- crates/ui-egui/src/panels/essential_sound.rs: per-type sections (Loudness with Auto-Match,
Repair, Clarity, Creative, Ducking with Generate Keyframes, Duration note, Pan), preset dropdown
with save/delete, code-drawn type glyphs, automation ids for every control; every control runs an
essentialSound.* command and a slider drag is one undo step
- headless kittest test (essential_sound_ui.rs) with optional wgpu snapshots
- docs: architecture §5.2, control-protocol ids, testing, ROADMAP
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Play waits for the first PREROLL_FRAMES (6) frames, at most 0.5 s, before starting the clock
and audio: the first frames after Play were due before any decoder could produce them.
- A new on-screen request for a view drops the queued and cancels the running request it
replaces (scrubbing used to decode every visited position oldest-first).
- playback_plan: when per-frame CPU work (effects; decode excluded, it is sequential anyway)
exceeds what the workers can do in real time, start only frames that can still be on time and
space them evenly, instead of every frame arriving late. Cost estimate updated from finished
and (as a lower bound) cancelled prefetch jobs. No change while the workers keep up.
- PlaybackMeter::resync: frames while the window is hidden (occluded/minimized) are not drops.
- Scopes render after the Program prefetch while playing (prio 40, was 2).
- ui.inspect playback.preroll; docs/architecture.md frame scheduling.
bench, same-session A/B, median of 3 (GPU path, Full, 8 s, load avg 22–64):
h264-1080 190/2 → 192/0 shown/dropped (3/3 runs 0 dropped)
stack3 189/3 → 192/0; h264-2160 173/19 → 192/0; demo 186/6 → 189/3
seek-storm: 10 → 15 of 40 seeks shown within 150 ms, 1581 → 1300 ms CPU per seek
after-preview live effects play (median of 2): 118/74 → 134/58, cancelled jobs 74 → 48;
plays from previews 192/0 both
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brandon Thomas