Progress docs per craftrules standards/progress-docs.md: target-app parity, gaps, roadmap with alpha gate, readiness by audience (full / mainstream / essentials) with Opus 5.5 hours, and feature, format, hardware, UI and localization parity docs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- GpuRendering defaults to Off (opt-in per export) until measured on Windows and macOS;
docs and ROADMAP say so.
- The shared export device installs an uncaptured-error handler and a device-lost callback
that log once and retire GPU export for the session (frames go to the CPU) instead of
wgpu's default panic.
- RendererPool drops a renderer that panicked and retires the pool, so the rest of the
export renders on the CPU (regression test in gpu_hook_scenarios); the duplicate
hostile-size test that raced on the global registry is folded into it.
- Rebased onto main's render_plan/transitions: render_export_prepared reads the linear
accumulator after render_plan; transition (Composite) plans fall back to the CPU.
- es/ja rows for the new Export strings; web_time::Instant in the export pipeline.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Narrations take schema v14 (main's v13 is audio streams): v13_to_v14, docs updated.
- neural-voices is off by default in the desktop app, like whisper; the model download stays
an explicit, confirmed job.
- Removed the debug probe2 test module from tts-text.
- project_subset prunes narrations of items it drops.
- unique_wav_path is bounded (99 999 names) and returns an error instead of looping.
- narration.rs uses web_time::Instant (wasm clock test); tts.* commands get the
"Text to Speech" shortcut category, with es/ja rows for it and Edit Narration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A 1440x1080 clip with 4:3 pixels showed squeezed: the demuxers read its
pixel aspect ratio, but nothing after import used it. New Sequence From
Clip made a 1440x1080 sequence with square pixels, the Motion transform
sized clips from storage pixels (pillarboxed at 1440 wide in a 1920x1080
sequence), and the monitors drew the frame at its storage aspect.
- project: checked_par / sane_par / par_ratio / conformed_size treat
file and project ratios as hostile (a zero term, or more than 8:1
either way, is square). MediaClip::pixel_aspect (Interpret Footage
wins over the file), Project::source_par (subclips follow the parent,
nested sequences have their own, graphics and adjustment layers have
none) and Project::conformed_source_size.
- render: motion_matrix takes the source's ratio and works in square
display units (anchor in source pixels, position in sequence pixels,
scale and rotation at the display aspect); Scale to Frame Size fits
the display size. CPU, GPU plan, adjustment regions and the mask /
graphics overlays share it.
- engine: New Sequence From Clip, New Sequence from an item and
multicam sequences copy the clip's ratio; default media scaling and
Fit / Fill frame use the display size; generators made for a
non-square sequence get its pixels; transcodes render pixel for
pixel. clip.interpretFootage takes pixelAspect ([num, den] or "file").
- ui: the Source and Program monitors draw the picture at its display
aspect; overlays map both axes; Sequence Settings shows the display
aspect (16:9 for 1440x1080 at 4:3).
Tests: render par_tests (fill, GPU plan parity, Scale to Frame Size,
non-square sequences, rotation, Interpret Footage, hostile ratios),
engine par_tests (New Sequence From Clip, overwrite into 1920x1080,
default scaling, Fit / Fill, Interpret Footage with undo and refused
values, hostile file ratios, generators, and ffmpeg-made MP4 and MKV
files), project unit tests, and the headless par_ui monitor test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main gained NVENC H.265 Main / Main 10 (#328, #360), export stage timings
(#370) and VA-API decoding (#358). Resolution: main's NVENC session, FFI and
export factory, with this PR's changes ported onto them:
- ffi: the portable Guid (now also for the HEVC GUIDs), NV_ENC_DEVICE_TYPE_CUDA;
- session: device.rs (Direct3D 11 / retained CUDA primary context), the
Linux libnvidia-encode.so.1 loader, Device::KIND;
- export: factory_with and its tests, the even-size check, the saturating
max bitrate; H.265 through NVENC stays Windows-only (the factory declines
it on Linux, where nothing probed or tested it);
- Cargo.toml: main's Linux libloading dependency covers both users;
- docs: README / ROADMAP / architecture now say NVENC H.264 runs on Windows
and Linux; ADR and platform README keep both addenda / test rows;
- abi_tests: note why the MSVC-generated expectations hold on LP64 Linux.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pool::trim() from Pipeline::drop flushed the whole process-wide pool after every
render-preview segment and proxy, and threw away playback's U8/U16 planes too.
Now Pipeline::drop calls pool::trim_f32() only when !settings.part_of_batch.
PoolStats gains f32_idle_bytes so the tests can check the float shelf alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move Video Limiter from CPU (crates/render/src/vfx/color.rs) to the GPU
compositor (filmcraft_render::gpufx::FxOp, crates/gpu/src/fx.rs, crates/gpu/src/fx.wgsl).
Parity matches CPU reference within 9.28e-6 across all axis modes, clip levels,
compressions, and gamut warning color.
🤖 Written by agy (Antigravity Agent, Gemini 3.8 Flash)
Add Vignette effect to the GPU effect stage (render::gpufx, gpu::fx,
fx.wgsl). Evaluated parameters are shared between the CPU reference and
GPU compute passes. Replaces old CPU vignette body in vfx::color with
gpufx::FxOp::eval and gpufx::FxOp::apply.
Gates: fmt, clippy, test -p filmcraft-render, test -p filmcraft-gpu,
layers, wasm all verified.
🤖 Written by agy (Antigravity Agent, Gemini 3.8 Flash)
Runtime-load the NVIDIA encode and CUDA driver libraries, retaining the primary context through session destruction. Preserve the Windows Direct3D path and default-off selection, and test driver rejection and deterministic software fallback.
Co-Authored-By: OpenAI Codex <noreply@openai.com>
HDR sequences export as real 10-bit HEVC Main 10 through NVENC where the GPU
reports 10-bit HEVC: float BT.2020 R'G'B' -> rgbf_to_yuv420_10 -> P010 input;
VUI BT.2020 + PQ/HLG; HDR10 mastering-display / content-light as SEI on every
IDR (PQ); colr/mdcv/clli in the sample entry. New register_hdr_probe /
hdr_available in the export crate; macOS and GPUs without 10-bit keep
tone-mapping HDR sequences to 8-bit SDR HEVC. ABI asserts extended and
re-verified against nvEncodeAPI.h 12.1 (0 drift). 8-bit SDR HEVC unchanged;
the two #328 tests that asserted HDR is declined now expect Main 10.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HEVC through VA-API the same way as H.264: the software HEVC decoder's
parsers and DPB (now generic over the picture type in filmcraft-hevc:
POC, reference picture sets, reference lists, output order, RASL
handling) drive the stateless hardware. Main, Main 10 and Main Still
Picture, 4:2:0 8/10-bit (P010 read-back); scaling lists, tiles, WPP,
dependent slices, weighted prediction. Missing references and range
extensions go to the software decoder.
Tests: hardware parity on HEVC Main / Main 10 incl. open GOP, scaling
lists, slices+WPP, weighted prediction, tskip+AMP, 1080p/2160p (streamed
comparison to stay within memory); HEVC front end against a recording
stand-in on every OS; ABI checks extended to va_dec_hevc.h. Benchmark on
Iris Xe: 4K HEVC 203 -> 10 ms CPU per frame, 4K Main 10 212 -> 18 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stateless VA-API decoding with FilmCraft's own H.264 host side: the
software decoder's SPS/PPS/slice-header parsers and its DPB (now generic
over the picture type in filmcraft-h264) drive the GPU, so reference
marking, reference lists and output order are shared and the pictures are
bit-exact with the software decoder. libva is loaded at run time; no libva
or driver means software decoding as before. frame_num gaps (open-GOP
seeks) get the software decoder's stand-in frames. HybridDecoder keeps the
transparent software fallback.
Tests: hardware parity on 7 H.264 streams incl. 1080p/2160p, seeks,
flushes, forced failures, fuzzing, concurrency; front end against a
recording stand-in on every OS; ABI checks against va.h. Docs: platform
README, ADR 0001 addendum, architecture, performance (H.264 2160p CPU per
frame 417 -> 11 ms on Iris Xe), roadmap.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- register() asks the HEVC probe on a thread of its own (the first probe is
about half a second: D3D11 device + NVENC session), so the first draw of
Export's format list does not wait for it; tests/nvenc_hevc_warm.rs.
- A machine with no HEVC encoder is not a declined hardware attempt: the
factory returns None without counting `declined`.
- HEVC tests skip only when the probe says there is no HEVC encoder; on a
machine that has one, a configuration NVENC refuses fails the test.
- Wording: NVENC is FilmCraft's only HEVC encoder on Windows (Windows itself
ships others).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One NVENC session, ring and NV12 path now serve H.264 and HEVC (Codec enum,
Profile::HevcMain). HEVC Main 8-bit 4:2:0 SDR BT.709, MP4/MOV hvc1, parameter
sets only in an hvcC built from the SPS the encoder wrote. The H.265 format is
available through a cached NVENC probe; choosing it is the opt-in, declines are
counted errors. ABI asserts regenerated from nvEncodeAPI.h 12.1 (no drift) and
extended for HEVC; tests, docs and roadmap updated.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Loudness Auto-Match metered every clip as the stereo pair clip_signal
plays, so a mono source (or a mono pick from Modify > Audio Channels /
Breakout to Mono), copied onto both sides, read 3.01 LU louder than the
file and got 3 dB too little gain. BS.1770 and ffmpeg's ebur128 measure
a mono programme as one channel; now so does clip_loudness, through a new
render::audio::clip_is_mono. Stereo and wider sources are unchanged, as
are the export loudness pass and the Meters panel (they measure the real
output mix).
Regression test: the same speech as a 1-channel WAV and as dual-mono
stereo measure 3.01 LU apart, the mono clip agrees with a one-channel
meter over the file within 0.1 LU, and a mono pick counts as mono.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The export pipeline hands the float image of every frame it converts back to
`filmcraft_frame::pool` (#184), which keeps up to 320 MiB of them, and the decoders' plane shelves
hold up to 192 MiB of each type. An export's images are the size of its frames, which nothing else
asks for, so a finished export left them idle until the next one.
`pool::trim()` frees every idle buffer. `Pipeline` calls it when it is dropped, one place for every
way an export ends (done, failed, cancelled, dropped half way by a host that steps it, as the web
build does) and for every caller of `export` / `Exporter` (previews, proxies and AAF/OMF mixdowns
share the pipeline). A job still rendering at that moment allocates a few images again: the pool is
only a cache.
Measured with a synthetic 3 s 1080p24 matte exported as PNG frames three times in one process
(Apple M1, `footprint`, 10 s after the last export): 575 MB without the trim, the pool holding
332 MB idle, and 258 MB with it. macOS returns the freed pages over a few seconds, not at once.
Tests: `crates/frame/tests/pool_trim.rs` (every shelf emptied, the pool works afterwards) and
`crates/export/tests/pool_trim.rs` (an image sequence, a finished and a cancelled video, an export
that could not start and a stepped export dropped half way each leave nothing idle; without the
trim they fail).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- filmcraft-tts (feature kokoro): Kokoro-82M on candle, implemented from
the StyleTTS 2 / iSTFTNet / ALBERT papers, config.json and the
checkpoint's tensor shapes; official .pth read with candle's restricted
pickle reader; voice packs from their zip. Neural Voice: tts-text
phonemize -> Kokoro per chunk -> pauses -> pitch via audio-dsp.
- catalog: 9 US English voices (model card grade C+ or better) + CMUdict,
every file pinned to a revision and SHA-256 (336 MB).
- speech: generic download_files (same verified downloader).
- engine: tts.render (background synthesis into Session::tts_cache),
tts.downloadVoices (background job, cancellable), voices list with
package state; feature neural-voices (default on in the desktop app).
- ui: natural/basic voice groups, consent box (size, sources, licences),
download progress + Cancel, synthesis never blocks the window;
ui.set {tts:{...}} for agents.
Measured: Whisper-base WER on 50 original sentences 2.1-3.6 % for all 9
voices (built-in robotic voice 96.6 %); 2.2-2.3x real time on CPU.
Gates: fmt, clippy, layers, assets, wasm pass; tests pass except
shortcuts_tests::assign_reassigns_conflicts_and_undoes (fails on main)
and the frame pool test (flaky under load, passes alone).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Write what a narrator says, pick a voice, pitch and pace, and add it to
the timeline. Selecting a narration clip loads it back into the panel;
Save regenerates it in place as one undo step, keeping the clip's start
and length (longer speech kept whole in the file, shorter speech padded
with silence).
- filmcraft-tts (L2): Voice trait, voice catalogue, pause markers
([pause 1s]), and an original formant synthesizer (Basic Female/Male;
robotic placeholder until the neural voices land).
- project: Narration + Project::narrations (schema v13, no-op migration).
- engine: tts.voices/preview/create/edit/inspect; narrations never
overwrite clips (free targeted track, free track, else a new track).
- ui: Text to Speech panel beside Properties, preview playback,
click-to-edit, automation ids.
Gates: fmt, clippy -D warnings, layers, assets and wasm (rustup
toolchain) pass. Tests: all pass except three that fail identically on
main (61238bf) on this machine: shortcuts_tests::assign_reassigns_
conflicts_and_undoes, gpu fx_tests::gpu_effects_match_cpu_exactly, and
the frame pool test, which passes when run alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merge origin/main into the NVENC branch. #183 already provides the
HardwareEncoding enum, ExportSettings.hardware_encoding, the
hardwareEncoding export param and the Export > Video > Hardware Encoding
checkbox, so the duplicates from this branch are dropped and NVENC uses
them. filmcraft-export is now an all-platform dependency of
filmcraft-platform (VideoToolbox and NVENC both register encoders).
Never-crash fixes in the NVENC backend:
- submit() rejects a driver input pitch narrower than a row (and unlocks
the buffer); fill_nv12 uses get_mut instead of slicing rows.
- composition offsets use saturating math and i32::try_from; frame-rate,
size and timestamp casts are checked; sizes above u16::MAX decline.
- Annex B scanning and NAL type checks use get()/first().
- A failed nvEncOpenEncodeSessionEx that returns a handle destroys it.
ffi.rs carries NVIDIA's MIT notice for nvEncodeAPI.h, listed in
ATTRIBUTION.md under a new "Third-party source code" section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds `Format::Hevc` (Export > Format > H.265 (HEVC); `"format": "hevc"` in
`file.exportMedia`): HEVC Main, 8-bit 4:2:0, in MP4 (`hvc1`) or QuickTime, with AAC audio,
encoded by the macOS hardware encoder. There is no built-in H.265 encoder (one in pure Rust
is a project of its own, and x265 is GPL), so the format exists only where the OS has one.
- `videotoolbox_encode`: the session wrapper creates HEVC sessions too (`VtProfile::HevcMain`;
the profile also says which codec). Keyframes are the HEVC random access points (NAL types
16 to 21); the parameter sets are VPS / SPS / PPS; the `hvcC` record in the MP4 sample
entry is the one VideoToolbox writes (sample description extension atoms), so profile,
level and flags are the encoder's own. `VtEncoder::uses_hardware()` asks VideoToolbox
whether the session runs on a hardware encoder (a test checks H.264 and HEVC), so a
requirement dropped by accident could never become a silent software encode.
- `hardware_encode`: one adapter for both codecs. Choosing the format is the opt-in (the
`hardware_encoding` setting is for H.264). `hevc_available()` (one small hardware session,
created once) is registered with the new `filmcraft_export::register_format_probe`, so
`available(Format::Hevc)`, the format list of Export mode and `exportFormats` follow the
machine. `Format::has_builtin_encoder` says which formats need such a probe.
- `export`: `Format::is_h26x` where H.264 and H.265 behave alike (MP4 or QuickTime, AAC, size
estimate, summary line). Two-pass VBR is refused up front ("H.265 export has no two-pass
mode") and HDR sequences export as SDR, because there is no encoder to fall back to.
- Export mode: H.265 shows H.264's multiplexer, audio, metadata and bitrate controls, without
profile, level, two-pass or the hardware checkbox, and says "Encoder: Hardware".
Apple M1 (8 cores), 1080p25, a 9:29 timeline (camera clip, ProRes 4444 overlays, AAC,
loudness): hardware H.265 at 10 Mb/s 195 s, hardware H.264 at 20 Mb/s 204 s (CPU 1029 s and
1042 s; the same H.264 export took 193 s and 200 s on other runs). It is not faster: the codec
that compresses further is the more expensive one (raw encoders: 166 and 189 frames/s), and
the export is bound by the CPU compositor, not by the media engine.
Quality per bit, 30 s of the same timeline against a ProRes 422 HQ export (average PSNR over
the three planes, SSIM): 20 Mb/s 46.88 dB / 0.9902 (H.264) and 46.91 dB / 0.9903 (H.265);
10 Mb/s 44.96 / 0.9863 and 45.09 / 0.9863; 6 Mb/s 43.72 / 0.9833 and 44.31 / 0.9843; 4 Mb/s
42.91 / 0.9807 and 43.64 / 0.9825; 3 Mb/s 42.37 / 0.9786 and 43.05 / 0.9807. The same picture
at 10 Mb/s and above; at 3 to 6 Mb/s H.265 needs 22 to 31 % less bitrate for the same PSNR.
Both formats therefore keep the same default bitrate. Details in docs/performance.md (HW3).
Tests: tests/hardware_encode.rs gets the HEVC counterparts of the H.264 ones (what is taken
and declined, a round trip through our HEVC decoder with luma PSNR above 30 dB, keyframes, MP4
and QuickTime exports with AAC, ffmpeg / ffprobe reading `hevc` / `Main` / `hvc1` /
`yuv420p` / BT.709, exact sizes, hostile configurations, encoders dropped at any point, the
format list agreeing with the probe, sessions on the hardware encoder); settings_tests.rs
checks names, ids, defaults, the summary, two-pass refusal and the probe; export_ui.rs the
controls H.265 shows and hides.
Stacked on the hardware H.264 encoder (the first commit of this branch).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The CPU compositor, which is what export runs, works on premultiplied linear f32 images,
33 MB each at 1080p, and every layer was one of them. A profile of a 9:29 export with a
hardware H.264 encoder (so the encoder is no longer the bottleneck) put 61-64 % of the busy
CPU in compositing, 12 % in zeroing and copying memory, 8-11 % each in the float -> 8-bit
and RGBA -> Y'CbCr conversions and 4-7 % in decoding the ProRes overlays: a banner that is
transparent except for its bottom 100 rows still cost a whole conversion and a whole mix,
and the bottom layer paid for a zero-filled canvas and a pass that mixes it over that canvas.
Three changes, each of which gives the bits the general path gives:
- The canvas is allocated when a layer needs it, and an opaque first layer (a Y'CbCr frame
without an alpha plane, opacity 100 %, Normal blend, the size of the output) becomes the
canvas instead of being mixed over zeros.
- A clip of plain media whose frame has an alpha plane (a ProRes 4444 banner) is converted
and mixed only inside the rectangle where its alpha is not zero: `VideoFrame::alpha_region`,
`to_linear_f32_region` and `blend::composite_at` (`composite` is `composite_at` at the
origin; Dissolve's pattern follows the pixel's place in the output). A clip with effects,
opacity masks, frame blending, colour management or a picture that is not placed one to
one on the output takes the general path, as before.
- `filmcraft_frame::pool` keeps idle f32 images (320 MiB at most). `take_f32_overwritten`
hands one back with its old contents, which the conversions overwrite whole (so no
zero-fill), and the export pipeline gives each frame's image back after the 8-bit
conversion.
Apple M1 (8 cores), a machine that was not idle, so the two builds alternate and the
median is reported:
- Synthetic 1080p frame (`cargo run --release -p filmcraft-render --example frame_bench`,
5 rounds): camera only 8.9 -> 6.5 ms, camera + banner 16.2 -> 7.8 ms, same picture checksum.
- A real project (1080p25 camera clip, ProRes 4444 banner and subscribe bar), 30 s with the
banner exported with the built-in encoder, which is deterministic: 46.4 -> 36.7 s, and
both exports are the same file (same MD5), so every frame of the real pipeline matches.
- The same project with a hardware encoder, 3 rounds: 30 s with the banner 17.4 -> 11.2 s
(CPU 99 -> 56 s), 30 s of the camera alone 11.4 -> 8.8 s, 20 s with the subscribe bar
11.1 -> 7.0 s.
Tests: `render/src/region_tests.rs` compares each shortcut with the general path bit for bit
(an opaque bottom layer, banners over and under a camera, an 8-bit sticker, every blend mode,
several opacities, moved, scaled and effect clips, a bigger source, reduced-resolution
rendering), each render twice so the second starts from recycled buffers, and checks which
layers take a shortcut; the frame tests check that a region is the crop of the whole
conversion, that `alpha_region` is the box around what is not transparent, and that a buffer
full of NaN never leaks into a conversion, for every pixel format. Breaking a shortcut on
purpose (a region one pixel off, an opaque layer taken as the canvas at half opacity, a
conversion that skips the alpha channel) makes them fail.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Export encodes H.264 on the CPU with the built-in encoder, which is most of the time
of an export (about two thirds of the busy CPU in a profile of a 20 s export). On
macOS the media engine can do it: a VideoToolbox compression session behind
`filmcraft_export::VideoEncoder`, registered by `filmcraft_platform::register()`.
- `ExportSettings::hardware_encoding` (`HardwareEncoding::Off` | `Auto`; Export > Video >
Hardware Encoding, `"hardwareEncoding": "off|auto"` in `file.exportMedia`, a boolean is
understood too), off by default: the built-in encoder's output is byte-identical on
every machine, a hardware encoder's is not.
- `videotoolbox_encode` (FFI, same containment as the decoder: SAFETY comments, no
panic across the callback, safe API) and `hardware_encode` (safe adapter and factory).
- The factory declines, so the built-in encoder is used, for other formats, MXF, two-pass
VBR, HDR, odd sizes, non-square pixels and any configuration VideoToolbox cannot create a
hardware session for. An encoder that fails mid-export stops it with an error: unlike a
decoder, it cannot hand a half-written stream to another one.
- No B-frames: on real 1080p footage the quality is the same with and without them
(+-0.3 dB at equal bitrate) and the bitrate lands closer to the target, and without
them every frame is presented at its decode time (no composition offsets, no edit list).
- `export::register_encoder` ignores a factory registered twice; `encoder_registered`.
Apple M1, 8 cores, single runs: a 9:29 timeline (camera clip, ProRes 4444 overlays with
alpha, AAC, loudness) exports in 311 s instead of 793 s (2.5x faster); its densest 134 s
in 84 s instead of 395 s (4.7x). The outputs compare at SSIM 0.990 / PSNR 46.5 dB. The encoder
alone runs 200 fps at 1080p (docs/performance.md).
Tests (tests/hardware_encode.rs): what is taken and declined, a round trip through our
software decoder, an export that decodes in our decoder and in ffmpeg, the fallback for
everything hardware declines, exact output sizes, an empty range, hostile configurations
and encoders dropped at any point; the engine's `hardwareEncoding` parameter.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Extends the Windows backend (#141) to VP9 (profiles 0 and 2) and AV1 (main, 8/10-bit), 4:2:0.
- codecs::hw: FrameStreamInfo (vpcC / av1C): profile, bit depth, chroma, picture size and colour as
the software decoders report them (vp9_color / av1_color are now shared with them), random-access
samples, in-band format changes; StreamInfo lets HybridDecoder handle NAL and frame-coded streams.
- platform: media_foundation::stream (what each codec needs: DXVA profile, MFT request, input
format), MfDecoder generalised; VP9 frames / AV1 temporal units go in as they are; the MFT's
declared picture size (display aperture) is checked against the bitstream's.
- Declined up front: VP9 profiles 1/3, AV1 profiles 1/2, 4:2:2/4:4:4, mono, 12-bit, no codec
extension, no DXVA profile. A key frame of another size or a different sequence header falls back
to the software decoder.
- Tests: bit-exact parity (360p / 1080p / 2160p, seeks, resets, flush, hidden frames), forced
failure, damaged samples and av1C, format change fallback, WebM / Matroska through the media stack.
- Bench: AV1 fixtures with libaom when ffmpeg lacks SVT-AV1.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMorBHwct72udS2LiC3ZHp
docs/architecture.md still listed every nested sequence among the things
rendered on the CPU and handed to the GPU as one image. Since NEST11 that
holds only for nests that are not plain.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A nested sequence was left out of both formats as a gap, with a report entry
telling the user to "render and replace" it first (a command FilmCraft does
not have). An AAF with nested compositions was read with gaps in their place.
What Premiere Pro 26.5.2 does, read off files it exported for a sequence with
a nest:
- AAF: the nested sequence is a composition mob of its own, and its clips are
source clips that point at it.
- OMF: the nested sequence's sound is rendered into the document as one clip
named after the sequence.
Export
- AAF: a nested sequence is written as a composition mob (no usage code; the
exported sequence stays the top-level one). A clip of it is a source clip
at that mob's first slot of the clip's kind, its start in that slot's edit
units. Nests in nests are followed; a nest used twice is written once. With
embedded or consolidated audio, the media of the clips inside a nest is
prepared with the rest (`NestNeeds::Inside`).
- OMF: `audio_needs` lists the range of the nested sequence each nest clip
plays (`NestNeeds::Render`); the engine mixes it through the export audio
pipeline with what is on the clip, and it is written as media. Without that
essence the nest is a gap and the report names the sequence.
Import (AAF)
- A source clip that points at a composition becomes a clip of a nested
sequence; the composition is read once.
- Premiere Pro's layout is read: all video tracks in one slot (a nested scope
with a segment per track; before, only the last segment was read, so a
Premiere AAF came in with one video track), and "top-level" tagged on nested
compositions too (top-level are those no other composition uses).
- A composition without a frame size takes that of the first picture it
shows, not of the first picture in the file.
A sequence that claims to be inside itself is a gap with a report entry in
both directions; the walks are bounded (16 levels).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: `file.exportInterchange {"format": "<anything unknown>"}` wrote
FCP7 XML and returned ok, so a typo or an unsupported format produced a
file of the wrong kind under the name that was asked for.
Solution: only a missing `format` means FCP7 XML. A given format must be
one of the six documented names (edl, xml, fcpxml, otio, aaf, omf: each
format's extension, in either case). Anything else, including a value
that is not a string, is a parameter error that lists exactly those
names, and nothing is written. No other spelling is accepted (".xml",
"fcpxmld", "omfi"): none was accepted before either, they silently
became XML. The parameter doc now lists aaf and omf, which the command
already took, and the default; docs/architecture.md says the same.
Test: `export_interchange_refuses_an_unknown_format`, next to the
interchange round-trip test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A media clip whose enabled standard effects all have a GPU implementation
plans as a GPU layer carrying them (render::plan::LayerFx): parameters are
evaluated on the CPU at the frame's time (render::gpufx::FxOp::eval, shared
with the CPU path so both clamp hostile values identically), the source is
drawn into an Rgba32Float working image at the size the CPU decodes it, the
effects run as WGSL compute passes (gpu::fx, fx.wgsl) with the CPU
reference's math, and the result is placed with Motion, opacity and blend
mode as before. Layers without effects keep the single draw.
GPU effects: Brightness & Contrast, ProcAmp, Tint, Black & White, Color
Balance, Leave Color, Change to Color, Color Pass, Color Replace, Channel
Mix, ASC CDL, Gamma Correction, Levels, Extract, Invert, Posterize, Alpha
Adjust, Gaussian / Camera / Directional Blur (+ legacy aliases), Sharpen,
Unsharp Mask, Crop, Edge Feather, Transform, H/V Flip, Mirror, Offset.
Masked effects, opacity masks, non-finite parameters and minifying
Transform/Crop zooms stay on the CPU. Devices without compute shaders run
the effects on the CPU inside the compositor.
Also fixes a debug-build overflow in Image::transformed for non-finite maps.
Parity tests (exact f32 checks and composited plans), a golden test that
plans effect chains as GPU layers, and the stack3-fx playback benchmark.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New crates/platform (filmcraft-platform, L5): OS media FFI only. The one
crate allowed `unsafe` (own [lints] with unsafe_code = "deny",
#[allow(unsafe_code)] only on the videotoolbox module, SAFETY comment on
every block, safe Result-returning public API). AGENTS.md §0.3 names the
exception; docs/adr/0001-platform-ffi.md records why and the rules.
- VideoToolbox decoder for avcC / hvcC, 8/10-bit 4:2:0 and 4:2:2: session from
the sample entry's parameter sets with hardware required, asynchronous
decode (2 in flight), biplanar CVPixelBuffer copied to planar Yuv8/Yuv16,
reorder buffer of the stream's own depth, RASL skipped after a starting CRA.
Output callback under catch_unwind.
- HybridDecoder: transparent mid-stream fallback to the software decoder
(replays samples since the last restart point, drops already-returned
pictures). New codecs::software_video_decoder (built-in factories only) and
codecs::hw (switch, counters, NalStreamInfo shared with software decoders;
vui_color / sar_par helpers now used by both).
- Settings ▸ Playback ▸ Hardware decoding: Auto / Off (engine applies it to
the codecs switch); desktop app, CLI/MCP and bench call register().
- perf.stats decode.framesDecoded and decode.hardware counters.
- Tests: bit-exact parity with the software decoders (H.264 High, HEVC Main
open GOP, Main 10; resets, reseeks, flushes), forced-failure fallback,
mutation fuzz of samples and parameter sets, cross-platform fallback with a
stand-in decoder, setting Off, HEVC 4:2:2 vs ffmpeg.
- Bench --hw auto|off; results in docs/performance.md (4K CPU/frame
~86-119 ms -> ~4 ms).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All 27 blend modes now composite on the GPU instead of sending the
whole frame to the CPU.
- render::plan: PlanLayer carries its Blend; plan_frame no longer falls
back to a CPU frame for non-Normal modes. Media, graphic clips,
multicam angles and CPU-rendered layer images (standard effects) keep
their clip's mode; inside a transition the mode is Normal, as in the
CPU reference. execute_cpu composites with each layer's mode.
- gpu: Normal keeps the single fixed-function pass. Dissolve also uses
fixed-function "over": the CPU's 64-bit pixel hash is reproduced in
WGSL on u32 pairs (pattern bit-exact). The other 25 modes copy the
accumulator region under the layer's quad into a backdrop texture and
composite in fs_blend with the CPU math (sRGB-encoded straight colour,
W3C formulas, back to linear, same premultiplied mix and clamping).
- Tests: per-mode GPU-vs-CPU parity (gpu crate: overlapping transformed
layers, partial opacity/alpha, empty/partial backdrop; golden: every
mode through plan_frame incl. an effect layer, blended clips in a
cross dissolve), exact-value edge cases (black/white, Divide by 0,
Dodge/Burn at 0 and 1, zero alpha) within 4.9e-4, Dissolve pattern
exact, off-output layers, quad bounds.
- Bench: new stack3-blend playback scenario (V2 Screen, V3 Overlay);
CPU ms/frame 151-214 -> 86-89, Full 19/173 and 138/54 -> 191/1, 192/0.
- Docs: architecture §5, gpu crate docs, testing, performance.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Exporting the same project twice could give different files, because two
export parameters followed the number of cores the process sees:
- The muxer got one chunk per render batch, and a batch is one frame per
rayon thread. The chunk layout (stsc/stco, PCM packet sizes) changed with
the core count; the samples themselves did not.
- The H.264 encoder used one slice per core (capped at one per four MB rows),
so the bitstream and every decoded picture changed with the core count.
On a host whose CPU quota changes (containers, power limits), two exports in a
row differed. Now packets are muxed in fixed groups of 16 output frames
(INTERLEAVE), each followed by its audio, whatever the batch size, and H.264
exports use one slice per four macroblock rows. Hosts with 17 or more threads
already used these values, so their 1080p output does not change.
Tests: determinism_tests (fail on main): same bytes for ProRes, H.264 and
Motion-JPEG at 2/3/5/16 cores and 1/7 frames per step; 64 slices for a
4096-line H.264 export.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brandon Thomas
lloyd
Manuel Camacho
storybored379
Davi Andrei
JordyUSA
Pathos0925
Dario Carnelutti
justjohn12345
mweinbach
punkhop