filmcraft-mxf gains a streaming writer (ST 377-1 partitions, primer, header
metadata, VBR index segments with temporal offsets, RIP; ST 378 OP1a
frame wrapping, ST 390 OP-Atom clip wrapping) for VC-3, ProRes, AVC byte
stream and PCM with start timecode and caller-supplied package UMIDs, plus
write_opatom_pcm. filmcraft-export adds mxf-op1a / mxf-opatom formats
(DNxHR, ProRes or H.264 + PCM; OP-Atom writes one mono file per channel),
presets, the engine mxfVideoCodec parameter and CLI .mxf inference.
Tests read our files back and check them against ffprobe / ffmpeg.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Shared mob-style composition model; AAF objects in structured storage over
filmcraft-cfb; OMF objects in a Bento container; embedded / consolidated audio
supplied by the caller (essence::audio_needs, MediaOptions).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- HEVC inverse transform: sum basis rows of non-zero coefficients with a
const-generic size (bit-exact, tested against the direct product);
SAO edge offset gets a check-free interior loop. HEVC CPU/frame -60%.
- H.264/HEVC/VP9 planes are moved into VideoFrames, not copied.
- Catch-up decoding: frame jobs tell sources which frames are late
(filmcraft_media::cancel::with_catch_up); the GOP cache skips
non-reference samples of late frames (H.264 nal_ref_idc 0, HEVC
sub-layer non-reference pictures). Scrubs keep the last 2 s decoded.
- perf.stats: engine query + UI/control method (decode counters,
playback shown/dropped, frame-worker decode/render ms, cache use).
- Timeline waveform source peak cached; bench seek section and --only.
- docs/performance.md with before/after results.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- assets/app-icon/: canonical SVG + light SVG, 1024 PNG, macOS runtime PNG, .icns, .ico,
hicolor theme, LICENSE.txt and README (palette, geometry, provenance, regeneration).
- packaging/icons.sh regenerates everything with resvg, iconutil and the new `cargo xtask ico`.
- Native app sets the window icon (with_icon) and the Wayland app id; build.rs embeds the .ico
and VERSIONINFO into filmcraft.exe on Windows (winresource, Windows-host build-dep only).
- packaging/linux/ai.storyteller.filmcraft.desktop with Icon=ai.storyteller.filmcraft.
- Web app: favicon.svg replaced by favicon.png from the same artwork; sidecars and
ATTRIBUTION.md updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- crates/speech (L2): Transcriber trait, FixedTranscriber, Whisper model catalogue (pinned,
SHA-256 verified) with optional downloader (feature `download`: ureq + rustls + RustCrypto),
pure-Rust Whisper inference on candle with word timestamps (feature `whisper`), log-mel, VAD
word tightening, MFCC speaker clustering. Both features off by default and for wasm.
- project: per-media-item transcripts (Project::transcripts); schema v9 with a no-op v8->v9
migration and a v8 fixture.
- edit::transcript: sequence transcript derived from clip transcripts, search, paragraphs,
word ranges, pause/filler detection, ripple delete of many ranges, caption layout.
- engine: transcript.* commands (generate, set, delete, inspect, search, select, extract, lift,
renameSpeaker, removeFillers, removePauses, createCaptions, models, downloadModel);
Session::transcriber for hosts/tests; features `whisper` / `speech-download` (also on the
desktop and CLI apps). Without whisper, transcript.generate fails with a clear error.
- ui: Text panel Transcript tab (Transcribe button, speaker paragraphs of clickable words,
Shift+click selection -> In/Out, extract/lift/remove fillers/create captions, search) with
text.transcript.* automation ids; headless UI test.
- docs/transcripts.md, crates/speech/README.md, architecture and project-files updates.
whisper-tiny measures 12.2% WER over 797 words on local samples.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- filmcraft-media: ByteReader/ReaderOpener (open containers without reading the whole
file) and the pending flag for asynchronous (web Blob) reads.
- filmcraft-codecs: MP4/MOV and Matroska open from a ByteReader; registry of reader
openers (platform sources first).
- filmcraft-engine: Services::reader / export_in_memory; streamed imports; media pool
never caches a still-loading file as offline; Session::pump_jobs advances stepped
exports on hosts without threads.
- filmcraft-export: Exporter (one batch per step, re-renders on pending reads) used by
export(); OutputSink for in-memory output.
- filmcraft-ui-egui: FrameServer::pump (time-budgeted cooperative frame jobs on wasm,
retries jobs whose media is loading); screenshots as base64 on wasm; web-time.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decoder::stats() reports frames, tiles and the busy time of setup, tile decoding,
deblocking, CDEF, superres, loop restoration and output. The ignored perf test
measures single-threaded throughput with the thread CPU clock (robust to machine
load), prints ms/frame per stage, and the all-cores wall-clock rate next to the
load average. Adds high-quality and 4x2-tile 1080p fixtures. Decoder::with_threads
and flush() are added as the API the threaded decoder will use.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agents' ui.* requests used ViewportCommand::Focus, so the user's typing in another app landed in FilmCraft as shortcuts (stray Extract, Mark In/Out, markers…). Verified live: the frontmost app stays frontmost through launch, ui.click, ui.key and ui.screenshot, and the requests still take effect.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Av1Decoder in filmcraft-codecs: feeds the av1C configuration OBUs before the
first sample (and after every reset), converts pictures to 8/16-bit planar
YUV with the stream's colour description, and reports random-access points
from the first frame header of each temporal unit (shown key frame) so GOP
seeking ignores bogus container sync flags. Matroska V_AV1 tracks map to an
av01 sample entry; MP4 labels show the AV1 profile.
All-intra 8-bit MP4 and 10-bit WebM files seek sample-exactly against
libdav1d. Inter frames still return "unsupported" until inter prediction
lands.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New L0 crate filmcraft-dnx, written from SMPTE ST 2019-1:2016 + Amd 1:2023
(tables extracted from the standard's text by tools/extract_tables.py).
Decoder: every compression ID (DNxHD 1235-1260 incl. field-coded and
macroblock-adaptive interlace, thin rasters, RGB 4:4:4; DNxHR 1270-1274
LB/SQ/HQ/HQX/444), 8/10/12-bit, 4:2:0/4:2:2/4:4:4, DCT and lossless alpha,
macroblock rows in parallel. Matches ffmpeg within IDCT precision (max 2 LSB,
<=50 per million beyond 1, 87-99% identical) on 22 ffmpeg fixtures; follows
ffmpeg where it departs from the standard's text (reconstruction offset,
CID 1271 weights, 4:4:4 signalling) - documented in the README.
Encoder: DNxHR LB/SQ/HQ/HQX(10/12-bit)/444, CBR at the eq. 7.1 frame sizes,
uniform-qsf binary search + per-macroblock refinement; ffmpeg decodes every
profile with -xerror (HQ luma PSNR 58.8-66.3 dB).
Wiring: MOV AVdn/AVdh import (codec label from the CID), export
Format::DnxHr (MOV AVdh, dnxProfile lb|sq|hq|hqx) in the export dialog and
file.exportMedia.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- media.createProxies: background job (jobs.list progress) transcoding with our own
encoders — ProRes 422 Proxy ¼/½, ProRes LT ½, H.264 ¼/½ (audio included) — into
<media dir>/Proxies or a chosen folder; attached when the job finishes
(Session::poll_persistence applies finished media jobs).
- media.attachProxies (duration ±1 frame and frame rate must match; size/aspect may
differ), media.detachProxies, media.reconnectFullRes, media.toggleProxies
(Preferences ▸ Media ▸ enableProxies; bumps the view revision without dirtying the
project), media.proxyPresets.
- Proxy frames are smaller; ProxySource reports the original size and the compositor
derives its pixel scale from the frame, so Motion and effects render the same
picture (44 dB PSNR vs full res with Motion + Gaussian Blur, 48 dB at ½).
Export always renders full resolution.
- ExportSettings::prores_profile (proxy / lt / standard / hq).
- project.ingestSettings + file.import: copy (fingerprint-verified), transcode,
create proxies, copy and create proxies.
- UI: Toggle Proxies button on both monitors (lit when on), Create Proxies dialog,
Project panel Proxy submenu and P badges.
- Measured (one core, 4K H.264 testsrc2): CPU composite 201 → 35 ms/frame at ¼ with
ProRes Proxy ¼, 277 → 131 ms at ½ with H.264 ¼; decode 9.2 → 0.7 ms with H.264 ¼.
- Tests: create+attach+toggle same composite modulo scale, attach checks, export
ignores proxies, ingest copy + proxies, ignored 4K benchmark, headless UI test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- model: SequenceSettings::color (ColorPipeline: working space, wide gamut, Auto Tone Map Media;
serde default Rec. 709) and Interpretation::color_space (override; None = metadata)
- render::colorman: source colour space from the override or VUI/colr/MKV metadata, decode tables
(camera log with super-whites, PQ, HLG + OOTF), gamut matrix, tone mapping into SDR working
spaces, gamut mapping; working → SDR monitor transform at the top level
(RenderOptions::working_output keeps working values for HDR export and scopes). Plain Rec. 709
media in a Rec. 709 sequence keeps the old fast path and stays on the GPU; managed layers and
HDR/wide-gamut sequences composite on the CPU
- export: H.264 and ProRes of a PQ/HLG sequence encode BT.2020 PQ/HLG and signal it: VUI / ProRes
header, colr (nclx/nclc), and for PQ mdcv + clli boxes and H.264 SEI (payloadTypes 137/144);
`sdr: true` (and render previews) export tone-mapped SDR. isobmff reads/writes mdcv and clli
- commands: sequence.colorSettings (Sequence ▸ Color Management…), clip.interpretFootage
(Clip ▸ Modify ▸ Interpret Footage…, Project panel context menu), color.spaces,
media.colorInfo; dialogs with automation ids colorDialog.*
- scopes: HDR sequences show a cd/m² waveform on a PQ scale (203 marked) and a BT.2020
vectorscope from the working-space render (scopes.hdrWaveform)
- display-referred wide-gamut input is desaturated into gamut keeping in-gamut colours exactly;
the soft compression is reserved for camera gamuts
- tests: colorman unit tests, engine (interpret/undo, PQ working space values and display),
goldens log_to_rec709 and hdr_tone_map (own renders + sidecars), export PQ/HLG × ProRes/H.264
read back by our demuxer and ffprobe (color_transfer/primaries/space, mastering display and
content light side data), headless UI (dialogs, HDR scopes)
- docs: architecture §5.2 (pipeline, HDR export, display colour management: no macOS EDR yet)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- project::essential: Dialogue/Music/SFX/Ambience settings (Loudness, Repair, Clarity, Creative,
Ducking, Pan, Clip Volume, Mute), our own EQ/reverb/type presets, and apply(): settings become
ordinary clip effects (EffectInstance::essential) in canonical order; only changed parameters
are written, so keyframes survive; auto-match gain → clip gain, Clip Volume → Volume offset,
Pan → Panner
- effect defs: DeEsser, DeReverb (params), Enhance Speech, Stereo Width; render mappings for them,
and DeNoise amount / DeHummer gain now reach the DSP; Mute; render::audio::clip_signal
- engine essentialSound.* (inspect, setType, clearType, set, applyPreset, savePreset,
deletePreset, autoMatch, generateDucking), undoable; per-type loudness targets and user presets
in preferences
- tests: auto-match within 0.001 LU of target (−23 / −16, with and without a chain), ducking
keyframes within 20 ms on a synthetic dialogue+music project and −15.00 dB in the mix, repair
metrics (hum −39 dB, rumble −22 dB, noise floor −13 dB, sibilance −19 dB, reverb tail −10 dB),
render chunking bit-identical and WAV export = render; full Dialogue chain 22× realtime (release)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Audio Track Mixer: strips for audio tracks, submixes and the Mix with
effect/send slots (disclosure), input mapping, output assignment, pan
knob, automation mode, M/S/R (solo safe on submixes), fader with dB
scale and our own taper, stereo meters fed by the playing mix, value
hot text, footer transport; horizontal/vertical scrolling. Fader and
knob drags are mixer.touch while held and mixer.release on let go, so
they are heard live and recorded in Latch/Touch/Write.
- Audio Clip Mixer: per-track strips for the clip under the playhead
(clip Volume / Panner, write-keyframes toggle), clipMixer.set with one
undo step per drag (Session::edit_merged).
- Timeline: per-track "Show Keyframes" button picks clip keyframes or a
track lane (volume, panner, mute, sends, insert params); the lane is
drawn across the track, Pen / Cmd-click adds, diamonds drag in time and
value, right-click deletes.
- Playback starts an automation pass and stopping writes it; the audio
callback reads the newest project snapshot (LiveMix), so mixer moves
are heard while playing. Audio Meters read the graph's live meters
instead of re-mixing at the playhead.
- Automation ids on every control; headless kittest tests drive the
panels by id and by multi-frame drags (FILMCRAFT_UI_SNAPSHOT_DIR renders
offscreen PNGs through wgpu).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GpuCompositor::upload converted f32 RGBA layers (effects rendered on the CPU, captions,
whole-frame fallbacks) and 16-bit YUV frames (ProRes render previews) to half floats on the UI
thread every time a frame was shown: 8.3 M values per 1080p RGBA layer, 4 M per ProRes frame.
filmcraft_gpu::prepare(plan) now does those conversions on the frame worker that built the plan
(parallel chunks; 16-bit codes through a cached per-bit-depth table) and the plan cache stores
them with the plan (frames::GpuPlan), bounded by count and bytes (1 GiB). composite_prepared()
only copies bytes into textures. The texels are bit-identical (new test compares the read-back
output of both paths for f32, 10-bit 4:2:2 and fallback-image plans).
bench, same-session A/B (GPU path, after-preview Full, 2 rounds, load avg 50–95):
UI-thread present time p50/p95: plays from previews 11–31 / 34–77 ms → 3.6–5.8 / 4–14 ms;
live effects play 17–28 / 31–56 ms → 4.1 / 12 ms (first run)
plays from previews: 184/8, 183/9, 189/3, 169/23 → 192/0, 192/0, 191/1, 192/0 shown/dropped
live effects play (CPU-bound, ~6 cores at this load): too noisy to compare (0–117 shown either way)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Program monitor overlay for graphic clips: Type tool click creates a text layer (in the
selected graphic under the playhead or a new clip) and edits it in place with a caret,
selection highlight, arrows/word/line navigation, Shift-select, Cmd+A/C/X/V, Return,
Backspace/Delete, Esc; typing is one undo step. Selection tool selects, moves and scales
layers (bounding box, handles, anchor); double-click edits text. Rectangle/Ellipse tools
drag out shapes; Pen tool places a closed path. Automation ids program.layer.*.
- Properties (when a graphic is selected) and Essential Graphics: Layers list, Align and
Transform (align / distribute), Text (source text, font family + style incl. system fonts,
size, paragraph alignment, tracking, leading, baseline shift, box width, faux bold/italic,
caps, underline, kerning/ligatures), Shape, Appearance (fill, 2 strokes, background,
shadow). Automation ids graphics.*.
- Headless UI test (egui_kittest) drives the whole flow through the control channel; with
FILMCRAFT_UI_SHOTS it renders PNGs via wgpu (works with a locked screen).
- docs/graphics.md; architecture, ROADMAP, ATTRIBUTION (code-drawn glyphs) updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- project: ItemKind::Graphic (transparent canvas source, hidden from the Project panel,
unlimited duration); graphic layers are hidden effect instances graphic_text /
graphic_shape on the clip (serde for free, keyframes on every animatable property,
Source Text holds), with appearance (fill, two strokes outer/centre/inner, background
box with radius, drop shadow) and transform (position, anchor, scale, rotation,
opacity); eval_layer gives a plain LayerSpec. No schema migration: all additive.
- render: graphic_clip rasterises each layer straight to the output transform (crisp at
any Motion scale/rotation), caches layer rasters, composes a tight image for the GPU
plan; standard effects on a graphic run at source resolution first.
- engine: graphics.newText (Cmd+T) / newShape (new clip above the footage at the playhead
or a layer in an existing clip), setText (typing coalesces into one undo step), set,
selectLayer, deleteLayer, arrangeLayer, align (frame / selection), distribute,
graphics.list, fonts.list. EditorState.graphic_layers holds the layer selection.
- goldens: graphic_title, graphic_shapes (CPU and GPU parity).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New L1 crate filmcraft-text: fonts (bundled Inter Regular/Medium/SemiBold/Bold/Italic,
JetBrains Mono, Noto Serif; DirectorySource scans system font folders reading only name/OS2
tables; family/style resolution with synthetic bold/italic; per-char fallback), layout
(harfrust kerning/ligatures, unicode-bidi, unicode-linebreak, alignment incl. justify,
leading, tracking, baseline shift, all/small caps, underline, carets/hit-test/selection,
cached), raster (moved from captions; active-edge list, shapes, emboldening), render
(outline + sub-pixel glyph-mask caches; transformed-outline path for rotation/skew),
mask (distance-transform strokes, blur).
- Captions burn-in now shapes and kerns via the engine (font.rs/raster.rs removed).
- Timecode / Clip Name effects draw real text (full clip names, not seven-segment digits);
text_burnin golden re-blessed.
- UI takes its font bytes from the text crate (no duplicate copies in the binary).
- New fonts: Inter Bold, Inter Italic (OFL-1.1, rsms/inter v4.1), Noto Serif Regular
(OFL-1.1, notofonts) with sidecars, ATTRIBUTION.md rows and licence text.
- Perf: 3-line 96 px title at 1080p ~1.6 ms cold / 0.15 ms warm (release).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brandon Thomas