diff --git a/AGENTS.md b/AGENTS.md index f616a9f8..0683cb87 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -142,6 +142,7 @@ as PhotoCraft does with `cargo xtask corpus`. - [docs/agents.md](docs/agents.md): driving FilmCraft over MCP / the control channel, and the agent work loop - [docs/control-protocol.md](docs/control-protocol.md): control-channel method reference - [ATTRIBUTION.md](ATTRIBUTION.md): asset index +- [ROADMAP.md](ROADMAP.md), [docs/target-app-parity.md](docs/target-app-parity.md), [docs/gaps.md](docs/gaps.md): stage, parity with Premiere Pro and the ranked work list (craftrules `standards/progress-docs.md`) ## Contributor credits (About window) diff --git a/CLAUDE.md b/CLAUDE.md index 9779b45c..2ecdacc2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ FilmCraft is a clean-room, open-source, pure-Rust non-linear video editor target ## Start every session here `plan/` is maintainer-local (gitignored, not in the repo). Each step names the public equivalent to use if you don't have it. -1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers). Public: [`ROADMAP.md`](ROADMAP.md). +1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers). Public: [`ROADMAP.md`](ROADMAP.md), then [`docs/gaps.md`](docs/gaps.md) (ranked shortfalls) and [`docs/roadmap.md`](docs/roadmap.md) (current focus). 2. Read the task in `plan/execution-plan.md` §3, the relevant section of `plan/architecture.md`, and the README/docs of the crate you touch. Public: [`docs/architecture.md`](docs/architecture.md), [`docs/contributing.md`](docs/contributing.md), [`docs/testing.md`](docs/testing.md). Visual/behaviour reference: `plan/premiere/` (local only, never committed). 3. Follow the autonomous operation protocol (`plan/execution-plan.md` §7; public summary: [`docs/agents.md`](docs/agents.md) §5–7). Don't stop to ask unless it lists the decision as the user's. diff --git a/Cargo.lock b/Cargo.lock index d94a4457..68dc628b 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1667,6 +1667,7 @@ dependencies = [ name = "filmcraft" version = "0.5.0" dependencies = [ + "arboard", "chrono", "cpal", "eframe", @@ -1828,6 +1829,7 @@ dependencies = [ "filmcraft-vp9", "image", "log", + "rayon", "serde_json", "symphonia", "thiserror 2.0.21", diff --git a/README.md b/README.md index fc1acfeb..e74820ab 100644 --- a/README.md +++ b/README.md @@ -274,20 +274,23 @@ The control protocol is documented in [docs/control-protocol.md](docs/control-pr | [docs/control-protocol.md](docs/control-protocol.md) | Control-channel and MCP reference | | [docs/project-files.md](docs/project-files.md) | `.fcproj` format, schema migrations, auto-save and crash recovery | | [docs/graphics.md](docs/graphics.md) · [docs/captions.md](docs/captions.md) | Text engine, graphic clips and tools; caption tracks and formats | -| [ROADMAP.md](ROADMAP.md) | Honest assessment, what's missing, milestones and estimates | +| [ROADMAP.md](ROADMAP.md) | Stage, parity numbers, languages, what's next and the progress log | +| [docs/target-app-parity.md](docs/target-app-parity.md) · [docs/gaps.md](docs/gaps.md) · [docs/roadmap.md](docs/roadmap.md) | Parity assessment against Premiere Pro, the ranked list of shortfalls, milestones and current focus | +| [docs/codec-parity.md](docs/codec-parity.md) · [docs/file-format-parity.md](docs/file-format-parity.md) · [docs/hardware-parity.md](docs/hardware-parity.md) · [docs/ui-parity.md](docs/ui-parity.md) · [docs/localization-parity.md](docs/localization-parity.md) | Parity checklists by dimension | ## Status FilmCraft is young and moving fast. Editing, trimming, multicam, colour, keyframes, effects, titles, captions, mixing, codecs and export work today. -We track two numbers ([ROADMAP.md](ROADMAP.md#honest-assessment-2026-10-05)): +FilmCraft is in **alpha**. We track two numbers ([docs/target-app-parity.md](docs/target-app-parity.md), re-measured 2026-10-10 against Premiere Pro 26.5.2): -- **Feature checklist: ~87%.** Premiere Pro's menu items, effects, transitions, panels and formats that exist in FilmCraft. -- **Ready for real work: ~50–60%.** Our honest estimate of how close FilmCraft is to replacing Premiere on real projects. +- **Feature breadth: ~86%.** Premiere Pro's menu items, effects, transitions, panels and formats that exist in FilmCraft (menus 92% and effects 100% measured). +- **Ready for real work: ~56%.** Our honest estimate of how close FilmCraft is to replacing Premiere on real projects. The biggest gaps today: - **Speed on big footage.** Hardware decoding works on macOS and Windows, and for H.264 and HEVC on Linux (VA-API, or NVDEC with NVIDIA's driver; VP9 and AV1 decode in software there). Blend modes and the most common effects run on the GPU, but Lumetri, keys and export rendering still run on the CPU; H.264 encoding can use the hardware encoder on macOS and on Windows and Linux with an NVIDIA GPU (opt-in, Export ▸ Hardware encoding), and H.265 export is hardware-only on both (HDR sequences export Main 10 PQ / HLG with NVENC; VideoToolbox writes 8-bit SDR) ([#30](https://github.com/storytold/filmcraft/issues/30)). +- **Premiere projects.** `.prproj` files can't be opened yet; bring sequences over as FCP7 XML, AAF or OTIO. - **No plugins.** No VST3 / Audio Units or OpenFX hosting. - **Delivery codecs.** H.264 is our only software delivery-codec export; H.265 exports only through a hardware encoder (NVIDIA on Windows: Main 8-bit SDR and Main 10 HDR PQ / HLG; VideoToolbox on macOS: 8-bit Main, SDR); no AV1 export yet. - **Real-world media and platforms.** Our decoders are bit-exact on conformance streams, but camera and phone files in the wild are less tested. Windows and Linux get far less testing than macOS. diff --git a/ROADMAP.md b/ROADMAP.md index e22211db..e5d94acf 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,164 +1,111 @@ # FilmCraft Roadmap -Progress toward parity with Adobe Premiere Pro, with estimates. Updated as milestones land. +**Stage: alpha** · next: beta, ~19 points of ready-for-real-work and ~400–700 h away, plus opening Premiere's `.prproj` ([why](docs/target-app-parity.md#stage); passes the [core-workflow gate](docs/roadmap.md#alpha-gate)) -**Last updated:** 2026-10-05 · **Feature checklist:** ~87% · **Ready for real work:** ~50–60% · **Code:** 35 crates, 1605+ tests +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (full re-measure against Premiere Pro 2026 26.5.2; scorecard, gaps, milestones and localization moved into docs/ per the craftrules progress-docs standard) · **Target:** Adobe Premiere Pro 2026 (26.5.2) -Two numbers, because they answer different questions. The **feature checklist** counts whether -Premiere's menu items, effects, panels and formats exist in FilmCraft. **Ready for real work** is -our honest estimate of how close FilmCraft is to replacing Premiere for an editor on real -projects: depth, correctness, robustness, speed and the pro ecosystem. The gap between the two -is the work that matters most now. Agents: read [Where we are lacking](#where-we-are-lacking) -before choosing work. +Progress toward parity with Adobe Premiere Pro. This page is the summary; the detail is in +[docs/target-app-parity.md](docs/target-app-parity.md) (assessment, methodology, calibration), +[docs/gaps.md](docs/gaps.md) (the ranked work list: **agents, read it before choosing work**) and +[docs/roadmap.md](docs/roadmap.md) (milestones and current focus). -## Honest assessment (2026-10-05) +## Headline numbers -### How the checklist number is made, and what it doesn't tell you - -- **Partly measured.** The menu figure comes from diffing a dump of Premiere 26.5's menus against - `filmcraft-cli commands`. The effects, transitions and audio-effects counts are checked by unit - tests against Premiere's Effects panel tree (`premiere_26_video_effects_catalogue`, - `premiere_audio_set_is_complete_and_foldered`, `premiere_26_transition_tree`). -- **Partly agent-reported.** The other areas' percentages are written by the agents doing the work. - No automated parity check runs in CI (PhotoCraft has `cargo xtask parity`; we don't yet), so - nothing stops a number drifting upward. -- **Presence, not quality.** An item counts once it exists. An effect described as an - "approximation" counts the same as an exact one. Nothing compares our rendering with - Premiere's. -- **Weights understate what users hit first.** Performance carries 5%. Plugins and AI features - aren't in the scorecard at all. - -### By dimension - -| Dimension | Checklist says | Honest estimate | Evidence | -|---|---|---|---| -| **Feature breadth** (menus, commands, panels) | ~86–95% | **~85%** | Menu diff against Premiere 26.5; effect, transition and audio-effect catalogues complete by test. The long tail is mostly there. | -| **Correctness in depth** | not tracked | **~60–70%** | A first-time contributor found four real bugs in core paths within hours (#5–#8: AIFF import failed, Slide left linked audio behind, transitions didn't follow ripple trims, export ignored start/end times). User reports: Color Matte always grey with no picker (#29); ALSA audio error on Linux (#23). Settings are "most wired", not all. | -| **Codecs and media** | ~90% | **~75%** | Our decoders are bit-exact on conformance streams. Real camera and phone media (variable frame rate, damaged files, unusual containers) is far less tested. No camera RAW (RED, BRAW, ARRIRAW), no E-AC-3. | -| **Export** | ~92% | **~70%** | Delivery: H.264, plus H.265 through a hardware encoder only (NVENC on Windows: Main 8-bit SDR and Main 10 HDR PQ / HLG; VideoToolbox on macOS: Main 8-bit SDR); ProRes, DNxHR and image sequences cover mastering. No AV1 export, no 10-bit / HDR HEVC on macOS. AAF and OMF have never been validated in Avid Media Composer or Pro Tools. Export renders and encodes on the CPU only. | -| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS (VideoToolbox, #33; active in the desktop app only since #41, so releases up to 0.2.1 decoded in software) and Windows (Media Foundation / Direct3D 11 DXVA, H.264, HEVC Main / Main 10, VP9 and AV1 4:2:0 8/10-bit, #30), and for H.264 and HEVC on Linux (VA-API: 4K H.264 at 11 ms of CPU per frame instead of 417 ms, 4K HEVC at 10 ms instead of 203 ms). The GPU does compositing, blend modes (#32) and 33 common effects (GPU2, GPU4, GPU5): colour adjustments, crop/flip/transform, blurs, sharpen, vignette, video limiter. Lumetri, keys, masks and the remaining effects are CPU, and so is export rendering unless Export ▸ GPU Rendering is turned on (opt-in); H.264 encoding can use VideoToolbox on macOS (opt-in per export, no B-frames) and NVENC on Windows and Linux with an NVIDIA GPU (opt-in per export), and H.265 (HEVC) export exists on macOS and Windows only, through the same hardware encoders. Linux decodes H.264 and HEVC in hardware (VP9 and AV1 there are software); its only hardware encoder is NVENC H.264. 8K isn't real-time (AV1 is on Windows with a hardware decoder). The effect list shows Premiere's "GPU accelerated" badge, but only 33 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. | -| **Stability** | not tracked | **improving, unproven** | The never-crash pass (no panics in product code, last-resort guards) landed on 2026-10-04. There's no field record from real users' projects yet. | -| **Plugins and pro ecosystem** | not tracked | **~0–10%** | No audio plugin hosting (VST3 / Audio Units) and no third-party video effects (OpenFX). For many professional editors this alone rules FilmCraft out. Team Projects and Productions are out of scope by design. | -| **AI features** | partial | **~25–35%** | Speech to text exists but is off in default builds (the `whisper` feature). Enhance Speech is a DSP chain, not a model. Auto Reframe is approximate. No Generative Extend, and no media-intelligence search or auto colour. | -| **Platforms** | not tracked | **macOS solid; Windows and Linux thin** | Development and testing happen on macOS. Windows and Linux are built for releases but barely exercised at runtime. The web build is a compile target; the browser app shell isn't finished. | -| **UI fidelity** | ~85% | **~80%** | Agents compare with Premiere reference screenshots; this hasn't been checked independently. | - -### Where we are lacking - -In priority order. Agents should prefer this work over adding more checklist items. - -1. **Hardware acceleration** (#30), the most visible gap to users: - - hardware decode on Linux (VA-API): H.264 and HEVC are in; VP9 and AV1 remain; - - zero-copy decoded frames into wgpu; - - the remaining effects on the GPU: Lumetri, keys, masks (33 common effects are done); - - GPU export, then hardware encode: H.264 and H.265 on macOS are in (H.264 opt-in; H.265 has no other encoder), H.264 and H.265 (Main 8-bit, and Main 10 HDR PQ / HLG) on NVENC (Windows); Main 10 / HDR HEVC on VideoToolbox, Linux and other Windows vendors remain. -2. **A measured parity number.** Add `cargo xtask parity`, run in CI. It should cover menus, - commands, effects, transitions, panels, preferences and formats against the Premiere snapshot in - `plan/premiere/`. It should report presence and fidelity separately (exact, approximate, stub). -3. **Real-world media corpus.** Phone, camera and screen recordings, variable frame rate, damaged - files, long GOPs, odd containers. Each gets an import → edit → export round-trip test with an - ffprobe / ffmpeg oracle. -4. **Workflow acceptance tests.** Scripted MCP sessions that do real jobs end to end: assemble and - trim a scene, colour it, mix and loudness-normalise it, caption it, deliver it, round-trip - through XML / AAF. -5. **Windows and Linux at runtime.** CI that runs the test suite and a headless playback/export - smoke test on both, not just release builds. Fix the Linux audio path (#23). -6. **Delivery codecs.** HEVC export is in on macOS (VideoToolbox, 8-bit, no software encoder) and on Windows (NVENC, 8-bit and Main 10 HDR); still to do: HEVC elsewhere, 10-bit / HDR HEVC on macOS, and AV1 (`rav1e`, BSD-2). -7. **Plugin hosting** (VST3 / Audio Units, then OpenFX). Needs FFI, so it falls under the isolated - `unsafe` crate rule (AGENTS.md §0.3). **Owner decision needed** before starting. -8. **AI features** with openly licensed local models: transcription on by default, text-based - editing, speech enhancement, reframing. -9. **The web app shell**: file access, WebCodecs, audio. - -### Where we're going - -**Next:** close items 1–5. Hardware acceleration fixes the gap users notice first (CPU-bound 4K playback and export). The -parity tool, media corpus and workflow tests make the progress number trustworthy. Windows and -Linux CI makes "runs everywhere" true. - -**After that:** delivery codecs, plugin hosting (pending the owner's decision) and AI features, -measured with the same tools. - -**Goal:** "ready for real work" at 85%+, measured rather than estimated, on all three desktop -platforms. - -## Feature checklist scorecard - -Measured against Premiere Pro 26.5 on this machine. Menu items: the native menu bar dump, minus -Adobe-cloud-only items (Team Projects, Productions, Firefly, Stock, Dynamic Link, account/help pages), -matched against `filmcraft-cli commands` and the UI command table. Effects: the Effects panel tree. - -| Area | Weight | Measured | Coverage | -|---|---|---|---| -| Editing, timeline, trimming, multicam | 20% | core edit algebra, trim modes, sync, multicam, menu long tail, Scene Edit Detection, Automate to Sequence, Premiere default keyboard (143 new commands; 39 cloud/absent-panel keys skipped) | ~86% | -| Menus / commands | (cross-check) | ~325 of 344 in-scope menu items | ~95% | -| Effects and transitions | 12% | video effects 93/93 (+Legacy, Obsolete), audio effects 53/53, video transitions 84/84 (+21 Legacy), audio transitions 3/3; some approximations (Warp Stabilizer 2-D, Morph Cut, Auto Reframe) | ~92% | -| Media I/O and codecs | 12% | H.264, HEVC, VP9, AV1, ProRes, DNx, MJPEG, MPEG-2/MPEG-1 (4:2:2 incl. IMX/XDCAM), Opus, AAC, AC-3, MP2; MP4/MOV/MKV/WebM, MXF (incl. MPEG-2), MPEG TS/PS (AVCHD, broadcast, DVD), Ogg, image sequences, BWF; no E-AC-3, camera raw, HW decode | ~90% | -| Panels and UI fidelity | 12% | all main panels incl. Metadata, Timecode, Events, Progress, Reference Monitor, full Lumetri Scopes, audio effect editors; Project panel List/Icon/Freeform with view presets and hover scrub; Media Browser with Favorites/navigation/ingest | ~85% | -| Audio | 10% | mixer, automation (track + clip), Essential Sound, meters, all 53 effects, 5.1 tracks/buses/panner/export, voice-over record, Remix | ~88% | -| Colour | 8% | Lumetri complete, LUTs, colour management, HDR, all Lumetri Scopes (+ `scopes.read` for agents) | ~88% | -| Graphics and captions | 8% | text engine, shapes, per-character styles, our own graphics templates (.fcgt, 8 built-ins, export/install/edit), rolls/crawls, responsive pins + time, captions SRT/VTT/SCC/MCC/STL/TTML/DFXP, transcripts | ~85% | -| Export | 8% | own H.264/AAC, ProRes, DNxHR, MXF OP1a/OP-Atom, image sequences, GIF, WAV, 5.1; preset library + user presets, full Export-mode settings, queue, Quick Export; AAF (Edit Protocol) import/export, OMF 2.0 export; no AAF/OMF validation against Avid/Pro Tools yet | ~92% | -| Preferences and project management | 5% | Settings dialog with 16 categories (most settings wired), project settings, scratch disks, search bins, templates | ~80% | -| Performance | 5% | 1080p real-time, 3×1080p; 4K H.264 real-time at load ≤140 (−23% decoder cycles, opt-in draft decoding at 1/2–1/4); HEVC 2.5× cheaper, catch-up decoding, `cargo xtask bench` + `perf.stats`; blend modes on the GPU (#32); 8K and AV1 not yet real-time on a loaded machine; H.264 and HEVC decode in hardware on Linux (VA-API), and on Windows only decode and opt-in NVENC H.264 export (see the honest assessment: ~35–40% against Premiere) | ~62% | -| **Weighted total** | | | **~87%** | - -## Estimate to parity - -This estimate is for the **feature checklist**. Closing the honest-assessment gaps (above) is the -"Robust on real-world material" row, plus plugins and AI features, which it doesn't include. - -Measured throughput in the last work block (2026-10-01 night → 10-02): five Opus 5.5 agents in -parallel for ~4.5 wall-clock hours (~18 agent-hours including integration) moved parity by ~4 points -(≈4–5 agent-hours per point), on a machine that was heavily overloaded (load 150–300 on 14 cores) and -once ran out of disk. The remaining points are a long tail (each effect, dialog and preference page is -small, but there are many), so the estimate applies a 1.3–1.5× tail factor. - -| | Opus 5.5 agent-hours | Wall-clock (4–5 parallel agents + integrator) | +| | Value | How | |---|---|---| -| Feature parity by checklist (~19 points left) | ~90–125 | **~20–32 h** | -| Robust on real-world material (codec edge cases, 4K/8K performance, pro workflows) | +120–200 | **+30–50 h** | -| **Total to "100% and better"** | **~210–325** | **~50–82 h (≈2–3.5 days, 24/7)** | +| **Feature breadth** | **~86%** | Does each Premiere feature exist? Menus 92% (measured: label diff against Premiere 26.5.2's menu dump), effects / transitions / audio effects 100% (measured by tests), other areas estimated | +| **Ready for real work** | **~56%** (51–61%) | How close FilmCraft is to replacing Premiere for an editor on real projects: depth, correctness, file compatibility, speed, stability, ecosystem. Estimated depth per area, combined by a [written formula](docs/target-app-parity.md#three-readiness-numbers) | +| **Mainstream practitioner** | **~55%** | A typical professional editor's weekly work only, discounted ×0.90 interaction, ×0.92 stability, ×0.88 file exchange with Premiere users ([method](docs/target-app-parity.md#mainstream-practitioner)) | +| **Essentials user** | **~60%** | Import, cut, title, music, export at default settings, discounted ×0.88 launch / stability, ×0.90 discoverability, ×0.95 opening files people send ([method](docs/target-app-parity.md#essentials-user)) | +| **To beta** | **~400–700 Opus 5.5 agent-hours** | Estimated, calibrated from this repo's PR history | +| **To full parity** | **~1,550–2,750 agent-hours** | ~70% parallelizes; plugins, AI models, native-speaker review and some hardware need a human | +| Code (measured) | 45 crates, ~343k lines of Rust, 2,520 tests, 268 merged PRs, 69 authors | `git`, `gh`, `grep` | -The 2026-10-02 block moved parity ~62% → ~72% with six agents in ~6 wall-clock hours, despite the -disk filling twice; the remaining work is mostly panels (scopes, Metadata, Media Browser), export -presets/queue, codecs (MXF, image sequences, hardware decode), performance and graphics templates. +Two numbers, because they answer different questions. The gap between them is the work that +matters most now: breadth is nearly done, depth is not. -Limits on speed: CPU and disk on one machine (more than ~5 agents slows everyone down: each worktree -build is 6–7 GB and a full `cargo xtask ci` takes 20–60 min under load), and a single integrator -merging, resolving conflicts (e.g. two agents bumping the project schema) and re-running CI. With one -agent and no parallelism, multiply wall-clock by ~3–4. +### Readiness by audience -Not reachable clean-room and locally: **Generative Extend** (needs a large video-generation model). -**Enhance Speech** and **Auto Reframe** are feasible only with openly licensed models we can ship. +| Audience | Ready % | Opus 5.5 agent wall-clock hours to ~95% | Work that dominates | +|---|---|---|---| +| Full target (ready for real work) | ~56% | ~1,450–2,600 (~70% parallelizes) | Plugin hosting, AI models, localization incl. RTL / Devanagari, camera RAW and delivery codecs, `.prproj`, hardware acceleration, UI fidelity | +| Mainstream practitioner | ~55% | ~500–850 (~65% parallelizes) | `.prproj` import (60–120), UI fidelity (120–200), stability and PR CI (40–80), 10-bit / 4:2:2 camera decode (30–50), playback / hardware (60–100), import and export gaps (60–100) | +| Essentials user | ~60% | ~180–350 (~60% parallelizes) | Launch crashes on Windows / Intel / older macOS (40–80), discoverability and timeline friction (60–110), playback lag (30–60), common import gaps (20–40), audio devices (15–30) | -## Milestones +Method and calibration: [docs/target-app-parity.md](docs/target-app-parity.md#three-readiness-numbers). -Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining agent-hours. +## By dimension -| # | Milestone | Status | Done | Remaining | Est. | -|---|---|---|---|---|---| -| M0 | Skeleton + visual shell | ✅ | Workspace, 20 crates, dock/workspaces, Premiere 26 look, native menus, control channel, MCP, xtask gates (layers, wasm) | — | — | -| M1 | Media I/O | ✅ | MP4/MOV demux+mux, WAV, stills, MJPEG, symphonia audio (MP3/FLAC/ALAC/Vorbis), GOP seek + frame cache, import | Media Browser polish | 2 | -| M2 | H.264 decoder | ✅ | Own decoder, bit-exact on 37+ streams, 500–600 fps 1080p | — | — | -| M3 | Editing core | 🟡 | Edit algebra (insert/overwrite/razor/lift/extract/ripple/roll/slip/slide/rate-stretch/nest/paste), tools, markers, trim mode + Trim Monitor + dynamic J/K/L trimming, Keyboard Shortcuts editor with FilmCraft/Premiere/FCP/Avid presets; Edit/Clip/File menu commands (Label colours and Select Label Group, Paste/Remove Attributes, Select All Matching, Remove Unused, Consolidate Duplicates, Sequence From Clip, Bin From Selection, Offline File, Close Project, Make/Edit Subclip, Modify Audio Channels/Timecode, Frame Hold Options/Add Frame Hold/Insert Frame Hold Segment, Time Interpolation with frame blending, Fit/Fill frame, Breakout to Mono, Extract Audio, Replace With Clip; M3.11: Scene Edit Detection (pure-Rust cut detection, background job), Normalize Mix Track, Simplify Sequence, Transcribe Sequence, Find/Find Next and search bins, Automate to Sequence, Edit Original, Edit Offline, Source Settings, Update Metadata (XMP), Generate Audio Waveform, Project Settings General/Scratch Disks, Get Media File Properties, Save as Template, Selection as FilmCraft Project, Avid Log Exchange export, Flash Cue markers, Dynamic Audio Waveforms, Reveal Log Files, System Compatibility Report); M3.12 keyboard parity: Premiere's default keyboard (~115 keyboard-only commands: edit-point navigation on targeted/any track, select clip at playhead/next/previous, extend edit to playhead, nudge/slip/slide selection, target and source-patch toggles, clip volume ±1 dB/many, frame maximize/full screen/panel cycling, monitor zoom, track heights, Project and Text panel keyboard navigation, text size/leading/alignment, Export Frame, poster frames; see docs/keyboard.md); multicam (Create Multi-Camera Source Sequence, Multi-Camera view with live switching on 1–9, angle switching, Enable/Flatten, Edit Cameras) and sync (Synchronize, Merge Clips; In/Out/timecode/marker/audio — GCC-PHAT, sample-accurate) | Multicam paging >16 angles and grid thumbnails, optical flow (renders as frame blending), 39 Premiere default shortcuts skipped with reasons in docs/keyboard.md (Productions, AI/cloud tools, work area bar, Production/Search panels) | 6–10 | -| M4 | Playback | 🟡 | Audio-clock master, prefetch with cancellation, J/K/L, correct dropped-frame stats, playback resolution, render bar + content-hashed render previews, App Nap opt-out; 1080p H.264 and 3 stacked 1080p streams play with 0 dropped frames | 4K under load, 8K, frame-threaded AV1 decode, reduced-resolution decode for multicam grids | 6–10 | -| M5 | Effects, keyframes, GPU | 🟡 | All 93 Premiere 26 video effects and 84 transitions (CPU), blend modes and 33 common effects on the GPU (#32, GPU2, GPU4, GPU5), keyframes + value/velocity graphs, wgpu compositor, effect + opacity masks (ellipse/polygon/pen, feather, expansion, modes; CPU/WGSL parity), mask tracking (Lucas–Kanade + RANSAC), adjustment layers, effect presets (built-in + user, JSON import/export) | Effects on the GPU (in progress, #30), GPU masks in the live path, GPU export, exact Warp Stabilizer (3-D) and Morph Cut | 20–30 | -| M6 | Export | ✅ | Own H.264 encoder (High/Main/Baseline, B-frames, VBR/CBR/2-pass) → MP4 or QuickTime + own AAC; ProRes, DNxHR, MJPEG, PNG/TIFF/BMP sequences (Premiere-style numbering), GIF, WAV, AIFF; background jobs; M6.5: Export mode parity (frame size / rate / scaling / pixel aspect, profile / level, CBR / VBR 1- and 2-pass, keyframe distance, audio codec / rate / channels / bitrate / sample size, multiplexer, burn-in or sidecar captions, image / name / timecode overlays, video limiter, loudness normalization with true-peak limiter, metadata, ranges, estimated size, summary), 24 built-in presets + user presets with favourites and import/export (Preset Manager), export queue (reorder, cancel, retry, several sequences / ranges), Quick Export; `export.*` commands and `filmcraft-cli export --preset` | Interlaced encoding, 5.1 audio, smart render, publishing destinations | 6–8 | -| M7 | Audio | 🟡 | Mixer graph (tracks → submixes → Mix, pre/post-fader inserts and sends, latency-compensated, sample-accurate, ~6× realtime for 24 tracks × 3 effects on one core), Audio Track Mixer + Audio Clip Mixer panels, track automation (Off/Read/Latch/Touch/Write, recorded live while playing, thinned to keyframes, timeline lanes with pen editing), solo/solo-safe, channel mapping basics, peak + BS.1770 loudness meters (match ffmpeg), DSP crate with 16 clip/track effects, Audio Gain (set/adjust/normalize), Constant Power / Constant Gain / Exponential Fade, Essential Sound (types, Loudness auto-match, Repair incl. DeEss/DeReverb, Clarity, Creative, Ducking keyframes, presets; full Dialogue chain 22× realtime) | Music duration remix, ML speech enhancement, 5.1 panner and multichannel buses, voice-over record, effect editor windows (EQ curve), remaining effects (multiband, convolution reverb), clip-mixer automation recording | 6–9 | -| M8 | Colour | 🟡 | Lumetri: basic, creative + looks, RGB & hue curves, wheels, HSL secondary, vignette, section bypass; Input LUT / Look LUT (.cube 1D/3D/shaper, .3dl; tetrahedral CPU + WGSL; project LUT library; built-in camera conversions); Colour Match (Oklab tonal-range statistics, skin protection, solved in Lumetri wheels); colour management: Rec. 709 / Rec. 2100 PQ / HLG working spaces + wide gamut, Interpret Footage colour space (S-Log3, V-Log, Canon Log 2/3, LogC3/4, Apple Log, D-Log from published specs), metadata auto-detect, BT.2390 tone mapping, gamut mapping, HDR export signalling (VUI/colr/mdcv/clli/SEI, ffprobe-verified); Lumetri Scopes: Vectorscope YUV (75/100 % targets, skin-tone line) and HLS, Histogram, Parade (RGB/YUV/RGB-White), Waveform (RGB/Luma/YC/YC no Chroma), presets, Rec. 601/709/2100, 8-bit/float/HDR (cd/m²) scales, multi-scope grid, `scopes.read` | HDR-aware Lumetri maths, macOS EDR monitors, mastering metadata → tone-map peak, D-Log M (no published formula), HSL Secondary refine | 3–5 | -| M9 | More codecs | 🟡 | ProRes decode+encode, AAC decode+encode, HEVC Main/Main 10 decoder (bit-exact on 41 fixtures, ~225 fps 1080p), VP9 decoder (profiles 0–3, 8/10/12-bit, bit-exact on 50+ fixtures; WebM/MKV `V_VP9` and MP4 `vp09` import with key-frame-checked seeking), Matroska/WebM import (H.264/HEVC/VP9/ProRes/MJPEG + AAC/Opus/FLAC/MP3/Vorbis/PCM), Opus decoder (SILK/CELT/hybrid, 5.1/7.1 multistream; all RFC 8251 vectors range-exact; WebM/MKV/MP4), DNxHD/DNxHR decoder (all SMPTE ST 2019-1 CIDs, 8/10/12-bit, 4:2:2/4:4:4, interlaced, alpha; within IDCT precision of ffmpeg on 22 fixtures) + DNxHR LB/SQ/HQ/HQX/444 encoder and MOV `AVdh` export, AV1 decoder (Main profile, 8/10-bit, all intra/inter tools, loop filters, superres, film grain, intra BC, spatial layers; bit-exact vs libdav1d on 19 SVT fixtures + 22 libaom vectors; MP4 `av01` / WebM `V_AV1` import), MXF import (OP1a/OP-Atom; AVC with POC-ordered B pictures bit-exact, DNxHD/DNxHR, ProRes, PCM/AES3; MPEG-2 reported unsupported; index-table seeking, material package timecode), Ogg Opus/Vorbis (granule seeking, pre-skip, end trimming), image sequences, Broadcast WAV timecode | VP9 frame threading, AV1 threading/SIMD (~4 fps 1080p today), AV1 High/Professional profiles, MPEG-2 decoder, hardware decode | 12–20 | -| M10 | Graphics & captions | 🟡 | Caption tracks (Subtitle/CEA-608/708/Teletext formats, track style), SRT/WebVTT/SCC import+export (frame-exact, property-tested), caption editing (add/split/merge/trim/move, sync-locked insert/extract), Text panel Captions tab, burn-in in Program monitor and export; text engine (`crates/text`: bundled + system fonts, harfrust shaping, bidi, line breaking, paragraph layout, glyph cache; 3-line 1080p title ≈ 0.15 ms warm); graphic clips with text + shape layers (fill, 2 strokes, background, shadow, keyframable transform), Type tool with on-monitor editing, shape/pen tools, Properties/Essential Graphics editor, align/distribute; graphics templates (own `.fcgt` format, 8 original built-ins, Browse tab with engine-rendered thumbnails, export / install / apply / editable properties; never reads .mogrt), rolls/crawls with ease and pre/postroll, responsive pins and intro/outro, per-character styles, Upgrade Caption to Graphic, source graphics, Replace Fonts; MCC (608+708), EBU STL, TTML/IMSC1, DFXP (frame-exact, property-tested) | 608/708 embedding in video streams, speech-to-text | 12–18 | -| M11 | Interchange & project management | ✅ | `.fcproj` schema versions + migrations, atomic saves, Save a Copy/Revert, auto-save ring + crash-recovery journal (Preferences ▸ Auto Save, recovery prompt), FCP7 XML, FCPXML, EDL, OTIO; offline media (own slate) + Link Media (fingerprint-checked relink, folder remap, search, Align Timecode, Make Offline); proxies (ProRes Proxy/LT, H.264 ¼/½ background jobs, attach/detach/reconnect, monitor toggle, export full-res) + ingest (copy/transcode/proxies); Project Manager (collect, consolidate + transcode with handles, size estimate) | Rename media to clip names, image-sequence conversion, Media Browser-driven relink, smart (cross-drive) path tracking | 1–2 | -| M12–M16 | Multicam, web (WASM), platform, long tail | 🟡 | Multicam: audio sync (GCC-PHAT, sub-sample), Merge Clips, multi-camera source sequences, Multi-Camera view with live switching (1–9); L0–L4 crates compile to wasm32 | Web app shell (file access, WebCodecs, audio), scene detection, auto reframe, ~800 remaining commands and dialogs, performance hardening | 35–55 | +| Dimension | Ready | Hours | Doc | +|---|---|---|---| +| Features (editing, effects, colour, audio, graphics, export settings) | ~66% | 350–620 | [target-app-parity.md](docs/target-app-parity.md#by-feature-area) | +| UI / UX fidelity | ~60% | 120–200 | [ui-parity.md](docs/ui-parity.md) | +| File formats (containers, `.prproj`, interchange, captions, stills) | ~60% | 140–240 | [file-format-parity.md](docs/file-format-parity.md) | +| Codecs | ~60% | 150–250 | [codec-parity.md](docs/codec-parity.md) | +| Hardware acceleration | ~45% | 120–200 | [hardware-parity.md](docs/hardware-parity.md) | +| Localization | ~35% | 150–230 | [localization-parity.md](docs/localization-parity.md) | +| Performance | ~55% | 60–120 | [performance.md](docs/performance.md) | +| Stability | ~45% | 60–120 | [gaps.md G4](docs/gaps.md#g4-crashes-at-start-up-and-no-test-ci-on-pull-requests) | +| Platforms (macOS, Windows, Linux, FreeBSD, web) | ~60% | 50–90 | [gaps.md G9](docs/gaps.md#g9-windows-and-linux-at-run-time) | +| Plugins and ecosystem | ~5% | 200–400 | [gaps.md G11](docs/gaps.md#g11-no-plugin-hosting) | +| AI features | ~25% | 150–300 | [gaps.md G12](docs/gaps.md#g12-ai-features) | -## Running now +## Features -- **Hardware acceleration (#30):** landed: blend modes on the GPU (#32); VideoToolbox hardware decode + the `platform` FFI crate (#33); 33 common effects on the GPU (GPU2, GPU4, GPU5). Windows: Media Foundation / Direct3D 11 H.264 + HEVC decode (HW2) and VP9 + AV1 (HW3), NVENC H.264 export (HW4, opt-in) and H.265 export. Linux: VA-API H.264 + HEVC decode (HW5). Next: VA-API VP9 / AV1, zero-copy upload, Lumetri on the GPU. -- **Next:** [Where we are lacking](#where-we-are-lacking) items 1–5. GPU export waits for the export-crate work (#19 and the never-crash follow-up) to land. +| Area | Breadth | Ready | Hours | +|---|---|---|---| +| Editing, timeline, trimming, multicam | ~90% | ~70% | 80–140 | +| Effects and transitions | 100% (measured) | ~60% | 80–140 | +| Media import and codecs | ~75% | ~60% | 150–250 | +| Audio | ~88% | ~65% | 50–90 | +| Panels and UI fidelity | ~88% | ~60% | 120–200 | +| Colour | ~90% | ~70% | 40–70 | +| Graphics, titles, captions | ~85% | ~65% | 50–90 | +| Export and delivery | ~80% | ~65% | 60–110 | +| Project files and interchange | ~70% | ~45% | 90–160 | +| Preferences and project management | ~80% | ~65% | 20–40 | -## Log +Evidence for every row: [docs/target-app-parity.md](docs/target-app-parity.md#by-feature-area). +## Languages + +Premiere ships 10 interface languages. Measured from our catalogs (3,498 strings): + +| Language | Status | Strings | +|---|---|---| +| English | full | 100% | +| Simplified Chinese | partial (whole interface; engine errors English) | 99.6% | +| Spanish | partial (whole interface) | 99.9% | +| Hindi | none (no Devanagari shaping in the interface) | 0% | +| Arabic | none (no RTL interface) | 0% | +| French | none | 0% | +| Portuguese (Brazil) | menus only | 9.9% | +| Indonesian | none | 0% | +| Japanese | partial (whole interface) | 99.7% | +| German | none | 0% | +| Korean | none | 0% | +| Vietnamese | none | 0% | + +Also shipped: Ukrainian (menus only, 10%). Detail: [docs/localization-parity.md](docs/localization-parity.md). + +## Upcoming + +Ranked; detail and "done when" in [docs/roadmap.md](docs/roadmap.md) and [docs/gaps.md](docs/gaps.md). + +1. **No launch crashes, and CI on pull requests** (macOS 12, Intel UHD, Windows 11 reports; no workflow runs the tests today). 40–80 h. +2. **Open Premiere's `.prproj`**, the beta blocker. 60–120 h. +3. **10-bit / 4:2:2 camera media without hardware** (H.264 High 10 / 4:2:2, HEVC 4:2:2; #626). 30–50 h. +4. **Measure fidelity against Premiere** and add `cargo xtask parity`. 50–90 h. +5. **Real-world media corpus and workflow acceptance tests.** 40–70 h. +6. **Hardware acceleration, continued** (#30): zero-copy upload, more effects on the GPU, Quick Sync / AMD encoders, VideoToolbox Main 10. 120–200 h. +7. **UI fidelity:** Program monitor transform handles, Effect Controls, docking. 120–200 h. +8. **Plugin hosting** (VST3 / AU, then OpenFX). Owner decision needed. 200–400 h. + +## Progress log + +- **2026-10-10 (re-measure):** full parity re-measure against Premiere Pro 26.5.2: menu label diff 338 / 366 (92%), installed bundle listings (10 UI languages, exporter and codec frameworks), 188 open issues. Breadth ~86% (file formats now counted), ready ~55%, stage **alpha**; progress docs reorganized per the craftrules standard ([target-app-parity](docs/target-app-parity.md), [gaps](docs/gaps.md), [roadmap](docs/roadmap.md), [codec](docs/codec-parity.md), [file-format](docs/file-format-parity.md), [hardware](docs/hardware-parity.md), [UI](docs/ui-parity.md), [localization](docs/localization-parity.md)). +- **2026-10-10:** ~75 community PRs merged: GPU export rendering (Auto) with Vignette, Video Limiter and Lumetri basic / creative / vignette on the GPU (#421); Linux NVDEC H.264 / HEVC decode (#549); text-to-speech narrations with Kokoro-82M (#278); Japanese for the whole interface (#449) and Ukrainian menus (#407); ALSA default-device fallback (#254, fixes #23); drop import into the timeline (#517); system appearance (#505); timecode scrub editing (#576); NVENC RGBA input (#486); marker CSV export (#454); many fixes (#526–#575). +- **2026-10-09:** release 0.5.0. NVENC H.265 export on Windows, Main and Main 10 HDR (#328, #360); NVENC H.264 export (#386); VA-API H.264 / HEVC decode (#358); Source monitor playback (#423); multi-stream container audio, schema v13 (#361); New Sequence and Sequence Settings dialogs (#269, #355); Spanish for the panels (#238); background decoders and memory stress tests (#385, #381). - **2026-10-08 (Source range dragging):** implicit full-clip In/Out, video-only/audio-only/linked both controls, picture/waveform drag gestures and one-command undoable timeline placement. Shared Source/Program draggable frame-bounded handles and range translation, cancellation, Source marker/navigation routing and marked-range playback/looping. - **2026-10-08 (Source playback):** normal forward playback for video/audio media and subclips in the Source monitor, an independent Source clock, Play/Pause and focused Space, frame stepping, device-clock audio with wall-clock fallback, seeking, and end-of-clip stopping. Windows desktop review verified a real video file and standalone WAV, including In/Out marking. Reverse/shuttle Source playback and source sequences remain unsupported; no parity percentage changed. - **2026-10-08 (night):** HDR H.265 (HEVC Main 10, PQ and HLG, BT.2020, limited range) export on Windows through NVENC (#30): an HDR sequence now exports as real 10-bit HDR (`rgbf_to_yuv420_10` into P010 input buffers; VUI BT.2020 + PQ / HLG; the HDR10 mastering-display and content-light messages as SEI on every IDR for PQ, through the generic SEI payload array because NVENC 12.1 has no mastering-display field; `colr` / `mdcv` / `clli` in the sample entry), where the GPU reports 10-bit HEVC (new `register_hdr_probe` / `hdr_available` in the export crate; elsewhere, and on macOS, HDR sequences still tone-map to 8-bit SDR HEVC, and `settings.sdr` forces that). ffprobe reads `hevc` / `Main 10` / `hvc1` / `yuv420p10le` / `bt2020` / `smpte2084` or `arib-std-b67` / `bt2020nc` / `tv` with the mastering-display and content-light side data for PQ; ffmpeg's decode equals ours code for code; the in-process round trip is at 71.1 dB luma PSNR on the 10-bit scale with 803 distinct levels on a ramp and the peak code 940 intact (RTX 5060). 8-bit SDR H.265 is unchanged. @@ -209,6 +156,13 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining - **2026-09-30 (late morning):** keyframe value/velocity graphs; xtask gates; H.264 MP4 export; Lumetri curves, wheels, looks, HSL secondary. - **2026-09-30 (early morning):** H.264 decoder, ProRes, AAC; GPU compositor; MCP; Premiere 26 visual fidelity pass. -### Interface localization +## Revision history -Edit > Language switches English/Japanese/Spanish/Portuguese (Brazilian)/Ukrainian/Simplified Chinese and persists across restarts through `general.interfaceLanguage` (also available in Settings > General). Until a language is chosen, System Language (the default) follows the operating system's or browser's preferred languages and falls back to English. Commands `app.language.english`, `app.language.japanese`, `app.language.spanish`, `app.language.portuguese`, `app.language.ukrainian` and `app.language.chinese` are reachable through the control channel. Translations live in one catalog per language (`crates/ui-egui/src/i18n/.tsv`); UI code wraps its strings in `tl!("…")` / `tlf!("…{name}", name)`, and names that come from registries (effects and their parameters, settings, commands, panels, workspaces, presets) are translated where they are drawn. **Spanish, Japanese and Simplified Chinese cover the whole interface**: menus, panels, dialogs, the Settings pages, the Effects/Effect Controls registry, the Keyboard Shortcuts dialog and status messages; tests fail when a `tl!` literal, a menu label or a registry name has no Spanish entry (`ja.tsv` has the same entries, but the tests do not require them yet; `zh-cn.tsv` must cover every `tl!` literal and menu label too). Searches match the displayed labels as well as English sources. Brazilian Portuguese and Ukrainian cover the core menus; untranslated labels use English. Ukrainian uses the bundled Cyrillic interface fonts. Engine error messages, the control channel, the CLI and MCP stay in English. Japanese text uses the Japanese fonts from [craft-fonts](https://github.com/storytold/craft-fonts) when FilmCraft is built with them (`CRAFT_FONTS_DIR`; all official releases, including the web build), otherwise a Japanese font already installed on the system; with neither, switching to Japanese is refused with a message. Simplified Chinese (`zh-cn`; System Language picks it for `zh`, `zh-CN`, `zh-SG` and `zh-Hans`, not for Traditional locales) uses the Chinese fallback font the interface always installs (craft-fonts, otherwise an installed system face); with none, switching to it is refused the same way. Existing vertical text support is preserved. +| Date | Change | Summary | +|---|---|---| +| 2026-10-10 | minor | Readiness-by-audience table with hours to ~95% per audience | +| 2026-10-10 | minor | Headline adds mainstream practitioner (~55%) and essentials user (~60%); ready for real work ~55% -> ~56% (formula written out, same inputs); beta distance ~19 points | +| 2026-10-10 | minor | Stage banner links the core-workflow gate (all six workflows pass on macOS; stays alpha) | +| 2026-10-10 | major | Restructured to the craftrules progress-docs shape: stage banner, headline numbers, dimension / feature / language tables, upcoming; re-measured against Premiere 26.5.2 (breadth ~87 → ~86%, ready ~50–60% → ~55%); honest assessment and scorecard moved to docs/target-app-parity.md, "Where we are lacking" to docs/gaps.md, milestones to docs/roadmap.md, interface localization to docs/localization-parity.md | +| 2026-10-08 | minor | Log entries for HEVC export, Source playback and range dragging | +| 2026-10-05 | major | Honest assessment added: checklist ~87% vs ready for real work ~50–60% | diff --git a/apps/filmcraft-cli/src/args.rs b/apps/filmcraft-cli/src/args.rs index bac68feb..49cb2acd 100644 --- a/apps/filmcraft-cli/src/args.rs +++ b/apps/filmcraft-cli/src/args.rs @@ -19,6 +19,7 @@ const VALUED: &[&str] = &[ "--end", "--data-dir", "--settings", + "--gpu-rendering", ]; #[derive(Debug, Default)] @@ -60,6 +61,11 @@ impl Args { self.opts.iter().any(|(n, _)| n == k) } + /// The first option that is not in `allowed`, if any. + pub fn unknown_opt(&self, allowed: &[&str]) -> Option<&str> { + self.opts.iter().map(|(n, _)| n.as_str()).find(|n| !allowed.contains(n)) + } + /// Command params from positionals `from..`: one JSON object, or `key=value` pairs /// (dotted keys nest: `color.r=1`). pub fn params_from(&self, from: usize) -> Result { @@ -135,9 +141,27 @@ mod tests { assert_eq!(a.params_from(2).unwrap(), json!({"a": {"b": [1]}})); } + #[test] + fn unknown_opt_reports_first_unlisted_option() { + let allowed = ["--bridge", "--project"]; + assert_eq!(args("mcp --automation-read-root X").unknown_opt(&allowed), Some("--automation-read-root")); + assert_eq!(args("mcp --bridge=a --typo=1").unknown_opt(&allowed), Some("--typo")); + assert_eq!(args("mcp --project p --bridge a").unknown_opt(&allowed), None); + } + #[test] fn double_dash_ends_options() { let a = args("import -- --weird-name.mov a.mov"); assert_eq!(a.positionals, ["import", "--weird-name.mov", "a.mov"]); } + + #[test] + fn gpu_rendering_takes_a_space_separated_value() { + let a = args("export --gpu-rendering off out.wav --start 0 --end 0.1"); + assert_eq!(a.positionals, ["export", "out.wav"]); + assert_eq!(a.opt("--gpu-rendering"), Some("off")); + let b = args("export out.wav --gpu-rendering auto"); + assert_eq!(b.positionals, ["export", "out.wav"]); + assert_eq!(b.opt("--gpu-rendering"), Some("auto")); + } } diff --git a/apps/filmcraft-cli/src/main.rs b/apps/filmcraft-cli/src/main.rs index b051d6cb..66645777 100644 --- a/apps/filmcraft-cli/src/main.rs +++ b/apps/filmcraft-cli/src/main.rs @@ -462,6 +462,9 @@ async fn cli() { diag(format_args!("rendered {}x{} in {:.1} ms → {out}", img.w, img.h, dt.as_secs_f64() * 1000.0)); } "mcp" => { + if let Some(opt) = a.unknown_opt(&["--bridge", "--project", "--demo", "--data-dir"]) { + usage(format!("unknown mcp option `{opt}`")); + } let server = match a.opt("--bridge") { Some(addr) => filmcraft_automation::FilmcraftMcp::bridge(addr).unwrap_or_else(|e| fail(e)), None => match Backend::open(&a) { diff --git a/apps/filmcraft-web/src/download_pacing.rs b/apps/filmcraft-web/src/download_pacing.rs new file mode 100644 index 00000000..01f9f421 --- /dev/null +++ b/apps/filmcraft-web/src/download_pacing.rs @@ -0,0 +1,24 @@ +//! Spacing of browser downloads (pure, so it is tested natively in `tests/download_pacing.rs`; the +//! anchor clicks live in [`crate::fs::download`]). +//! +//! Chrome drops anchor-click downloads that start in one tight burst, so an image-sequence export +//! (one file per frame) delivered only about half of its frames (#378). Each download is given a +//! slot [`GAP_MS`] after the previous one. + +/// Milliseconds between two download clicks. +pub const GAP_MS: f64 = 125.0; + +/// Hands out click times. +#[derive(Clone, Copy, Debug, Default, PartialEq)] +pub struct Pacer { + next_free: f64, +} + +impl Pacer { + /// Reserve the next slot at time `now` (ms); returns how long to wait before clicking. + pub fn reserve(&mut self, now: f64) -> f64 { + let start = now.max(self.next_free); + self.next_free = start + GAP_MS; + start - now + } +} diff --git a/apps/filmcraft-web/src/fs.rs b/apps/filmcraft-web/src/fs.rs index adf417ca..416fbf9d 100644 --- a/apps/filmcraft-web/src/fs.rs +++ b/apps/filmcraft-web/src/fs.rs @@ -54,6 +54,7 @@ struct Fs { thread_local! { static FS: RefCell = RefCell::new(Fs::default()); + static DOWNLOADS: RefCell = RefCell::new(crate::download_pacing::Pacer::default()); } /// Wake-up hook for arriving data. @@ -405,7 +406,8 @@ impl Services for WebServices { } } -/// Offer bytes as a browser download named `name`. +/// Offer bytes as a browser download named `name`. Downloads are clicked at least +/// [`crate::download_pacing::GAP_MS`] apart: Chrome drops a burst of them (#378). pub fn download(name: &str, data: &[u8]) -> Result<(), wasm_bindgen::JsValue> { let doc = web_sys::window().and_then(|w| w.document()).ok_or("no document")?; let blob = bytes_to_blob(data, "application/octet-stream")?; @@ -414,10 +416,20 @@ pub fn download(name: &str, data: &[u8]) -> Result<(), wasm_bindgen::JsValue> { a.set_href(&url); a.set_download(file_name(name)); a.style().set_property("display", "none")?; - doc.body().ok_or("no body")?.append_child(&a)?; - a.click(); - a.remove(); + let delay = DOWNLOADS.with(|d| d.borrow_mut().reserve(js_sys::Date::now())); + let name = name.to_owned(); wasm_bindgen_futures::spawn_local(async move { + if delay > 0.0 { + sleep(delay as i32).await; + } + let clicked = match doc.body() { + Some(body) => body.append_child(&a).map(|_| a.click()), + None => Err("no body".into()), + }; + a.remove(); + if let Err(e) = clicked { + log::warn!("download {name}: {e:?}"); + } sleep(60_000).await; let _ = web_sys::Url::revoke_object_url(&url); }); diff --git a/apps/filmcraft-web/src/lib.rs b/apps/filmcraft-web/src/lib.rs index d769ea4b..21512ccc 100644 --- a/apps/filmcraft-web/src/lib.rs +++ b/apps/filmcraft-web/src/lib.rs @@ -20,6 +20,7 @@ pub mod api; pub mod audio; +pub mod download_pacing; pub mod fs; pub mod import; pub mod opfs; diff --git a/apps/filmcraft-web/tests/download_pacing.rs b/apps/filmcraft-web/tests/download_pacing.rs new file mode 100644 index 00000000..893f40d3 --- /dev/null +++ b/apps/filmcraft-web/tests/download_pacing.rs @@ -0,0 +1,26 @@ +//! Download spacing. It is pure, so it runs natively even though the rest of the crate is +//! wasm32-only. + +#[path = "../src/download_pacing.rs"] +mod download_pacing; + +use download_pacing::{GAP_MS, Pacer}; + +/// #378: 20 downloads requested in the same instant are spread out, not clicked in one burst. +#[test] +fn burst_of_downloads_is_spaced_out() { + let mut p = Pacer::default(); + let delays: Vec = (0..20).map(|_| p.reserve(1000.0)).collect(); + assert_eq!(delays[0], 0.0); + for (i, d) in delays.iter().enumerate() { + assert_eq!(*d, i as f64 * GAP_MS); + } +} + +/// A download after a quiet period is not delayed. +#[test] +fn download_after_a_pause_is_immediate() { + let mut p = Pacer::default(); + assert_eq!(p.reserve(0.0), 0.0); + assert_eq!(p.reserve(10_000.0), 0.0); +} diff --git a/apps/filmcraft/Cargo.toml b/apps/filmcraft/Cargo.toml index cf1cfff5..dc9b1c47 100644 --- a/apps/filmcraft/Cargo.toml +++ b/apps/filmcraft/Cargo.toml @@ -32,6 +32,8 @@ serde_json = { workspace = true } log = { workspace = true } rfd = { workspace = true } cpal = "0.16" +# Files and images on the system clipboard for Paste (#611); egui already uses it for text. +arboard = "3" chrono = { version = "0.4", default-features = false, features = ["clock"] } sys-locale = "0.3" diff --git a/apps/filmcraft/src/main.rs b/apps/filmcraft/src/main.rs index 8e9668d6..646c22ba 100644 --- a/apps/filmcraft/src/main.rs +++ b/apps/filmcraft/src/main.rs @@ -237,6 +237,7 @@ fn main() -> eframe::Result { app.hooks.pick_folder_at = Some(Box::new(|dir: &str| rfd::FileDialog::new().set_directory(dir).pick_folder().map(|p| p.to_string_lossy().into_owned()))); app.hooks.open_path = Some(Box::new(open_path)); + app.hooks.clipboard_media = Some(Box::new(clipboard_media)); // Settings ▸ General ▸ Interface Language ▸ System Language (#218). app.hooks.cursor_screen_position = Some(Box::new(filmcraft_platform::cursor::cursor_screen_position)); app.hooks.system_languages = Some(Box::new(|| sys_locale::get_locales().collect())); @@ -283,6 +284,18 @@ fn main() -> eframe::Result { started } +/// Files copied in the file manager, else an image, on the system clipboard (Paste, #611). +fn clipboard_media() -> Option { + let mut clipboard = arboard::Clipboard::new().ok()?; + if let Ok(files) = clipboard.get().file_list() + && !files.is_empty() + { + return Some(filmcraft_ui_egui::ClipboardMedia::Files(files.iter().map(|p| p.to_string_lossy().into_owned()).collect())); + } + let image = clipboard.get_image().ok()?; + Some(filmcraft_ui_egui::ClipboardMedia::Image { width: image.width, height: image.height, rgba: image.bytes.into_owned() }) +} + /// Open a file in its default application, or reveal it in the file manager (Edit Original, /// Reveal Log Files). fn open_path(path: &str, reveal: bool) -> Result<(), String> { diff --git a/contributors/people.toml b/contributors/people.toml index 50788355..4af53d71 100644 --- a/contributors/people.toml +++ b/contributors/people.toml @@ -56,3 +56,8 @@ sync_github_name = false [people.liangforstudy] sync_github_name = true + +[people.maxm11] +real_name = "Max Murphy" +display_name = "Max Murphy" +sync_github_name = false diff --git a/crates/audio-dsp/src/channels.rs b/crates/audio-dsp/src/channels.rs index cda0f0c2..df0c534b 100644 --- a/crates/audio-dsp/src/channels.rs +++ b/crates/audio-dsp/src/channels.rs @@ -2,7 +2,7 @@ //! //! **Channel order.** 5.1 audio is always in the SMPTE / WAVE order **L, R, C, LFE, Ls, Rs** //! ([`L`], [`R`], [`C`], [`LFE`], [`LS`], [`RS`]); stereo is L, R; mono is one channel. Codecs with a -//! different native order (AAC: C, L, R, Ls, Rs, LFE) reorder at the encoder boundary. +//! different native order (AAC: C, L, R, Ls, Rs, LFE) reorder at the codec boundary. //! //! **Downmix** follows ITU-R BS.775 (Table 2, "3/2 source" rows), with `k = 1/√2` (−3 dB): //! diff --git a/crates/audio-dsp/src/effects/dynamics.rs b/crates/audio-dsp/src/effects/dynamics.rs index 5807d1ba..40dec64c 100644 --- a/crates/audio-dsp/src/effects/dynamics.rs +++ b/crates/audio-dsp/src/effects/dynamics.rs @@ -263,6 +263,8 @@ pub struct Limiter { true_peak: bool, /// Look-ahead (samples) and detector alignment delay. look: usize, + /// Delay-line capacity (samples of look-ahead) allocated at construction for 0…30 ms. + max_look: usize, det_delay: usize, bank: PolyphaseBank, interp: Vec, @@ -285,17 +287,21 @@ impl Limiter { ParamSpec::new("input_gain", "Input Boost", 0.0, 24.0, 0.0, Unit::Decibels), ParamSpec::new("ceiling", "Ceiling", -24.0, 0.0, -1.0, Unit::Decibels), ParamSpec::log("release", "Release", 1.0, 1000.0, 60.0, Unit::Milliseconds), + ParamSpec::new("lookahead", "Look-Ahead", 0.0, 30.0, 5.0, Unit::Milliseconds), ParamSpec::toggle("true_peak", "True Peak", true), ]; - /// Fixed look-ahead time. + /// Default look-ahead when the parameter is left at its DSP default. pub const LOOKAHEAD_MS: f32 = 5.0; + /// Longest look-ahead the delay line is sized for (Hard Limiter's 0…30 ms range). + pub const MAX_LOOKAHEAD_MS: f32 = 30.0; pub fn new(sample_rate: f32, channels: usize) -> Self { let channels = channels.max(1); let bank = PolyphaseBank::new(true_peak_factor(sample_rate as f64).max(2)); - let look = ((Self::LOOKAHEAD_MS * 0.001 * sample_rate).round() as usize).max(1); + let max_look = ((Self::MAX_LOOKAHEAD_MS * 0.001 * sample_rate).round() as usize).max(1); + let look = ((Self::LOOKAHEAD_MS * 0.001 * sample_rate).round() as usize).max(1).min(max_look); let det_delay = bank.delay(); - let dlen = look + det_delay + 1; + let dlen = max_look + det_delay + 1; let mut s = Limiter { pv: ParamValues::new(Self::PARAMS), sr: sample_rate, @@ -304,6 +310,7 @@ impl Limiter { rel: 0.0, true_peak: true, look, + max_look, det_delay, bank, interp: vec![Interpolator::default(); channels], @@ -326,6 +333,15 @@ impl Limiter { self.ceiling = db_to_gain(self.pv.v("ceiling")); self.rel = coef(self.pv.v("release"), self.sr); self.true_peak = self.pv.on("true_peak"); + let look = ((self.pv.v("lookahead") * 0.001 * self.sr).round() as usize).max(1).min(self.max_look); + if look != self.look { + self.look = look; + self.minq.clear(); + self.box_ring = vec![1.0; look + 1]; + self.box_pos = 0; + self.box_sum = (look + 1) as f64; + self.rel_state = 1.0; + } if snap { self.in_gain.snap(); } @@ -363,7 +379,7 @@ impl AudioEffect for Limiter { for i in 0..n { let g_in = self.in_gain.tick(); let det_pos = (self.wpos + dlen - self.det_delay) % dlen; - let out_pos = (self.wpos + 1) % dlen; // oldest = look + det_delay samples ago + let out_pos = (self.wpos + dlen - (self.look + self.det_delay)) % dlen; let mut det = 0.0f32; for ch in 0..nch { let x = channels[ch][i] * g_in; @@ -555,4 +571,24 @@ mod tests { let peak = tail.iter().fold(0.0f32, |a, v| a.max(v.abs())); assert!((lin_to_db(peak) + 6.0).abs() < 0.3, "{}", lin_to_db(peak)); } + + #[test] + fn limiter_lookahead_changes_latency_and_transient_shape() { + let mut a = Limiter::new(SR, 1); + let mut b = Limiter::new(SR, 1); + a.set_param("lookahead", 0.0); + b.set_param("lookahead", 30.0); + a.set_param("ceiling", -6.0); + b.set_param("ceiling", -6.0); + assert!(b.latency() > a.latency(), "{} vs {}", b.latency(), a.latency()); + let mut xa: Vec = (0..4800).map(|i| 0.08 * (2.0 * std::f32::consts::PI * 997.0 * i as f32 / SR).sin()).collect(); + for s in &mut xa[1200..1248] { + *s = 0.9; + } + let mut xb = xa.clone(); + a.process(&mut [&mut xa]); + b.process(&mut [&mut xb]); + let max = xa.iter().zip(&xb).map(|(x, y)| (x - y).abs()).fold(0.0f32, f32::max); + assert!(max > 1e-4, "look-ahead of 0 ms and 30 ms produced the same waveform (max Δ {max})"); + } } diff --git a/crates/captions/src/burn.rs b/crates/captions/src/burn.rs index 5c43e525..77c9bf84 100644 --- a/crates/captions/src/burn.rs +++ b/crates/captions/src/burn.rs @@ -49,13 +49,10 @@ impl Overlay { } } -/// The caption face: Inter SemiBold at `px`. -fn caption_style(px: f32) -> TextStyle { - TextStyle { style: "SemiBold".into(), size: px, ..Default::default() } -} - -fn measure(text: &str, px: f32) -> f32 { - filmcraft_text::measure(text, &caption_style(px)) +/// The caption face: the track's font family (Inter when empty or not installed) in SemiBold at `px`. +fn caption_style(font: &str, px: f32) -> TextStyle { + let family = if font.trim().is_empty() { filmcraft_text::fonts::DEFAULT_FAMILY } else { font.trim() }; + TextStyle { family: family.into(), style: "SemiBold".into(), size: px, ..Default::default() } } fn lin(c: [u8; 4]) -> [f32; 4] { @@ -64,12 +61,12 @@ fn lin(c: [u8; 4]) -> [f32; 4] { } /// Word-wrap one line to `max_w` pixels. -fn wrap(line: &str, px: f32, max_w: f32) -> Vec { +fn wrap(line: &str, ts: &TextStyle, max_w: f32) -> Vec { let mut out = Vec::new(); let mut cur = String::new(); for word in line.split(' ') { let cand = if cur.is_empty() { word.to_string() } else { format!("{cur} {word}") }; - if !cur.is_empty() && measure(&cand, px) > max_w { + if !cur.is_empty() && filmcraft_text::measure(&cand, ts) > max_w { out.push(std::mem::take(&mut cur)); cur = word.to_string(); } else { @@ -115,16 +112,17 @@ pub fn render_caption(c: &Caption, style: &CaptionStyle, w: usize, h: usize) -> let scale = h as f32 / 1080.0; let px = (style.size * scale).max(4.0); let max_w = w as f32 * 0.9; + let ts = caption_style(&style.font, px); let mut lines: Vec = Vec::new(); // (speaker names are metadata; like Premiere, they are not burned in) for l in c.plain_lines() { - lines.extend(wrap(l.trim(), px, max_w)); + lines.extend(wrap(l.trim(), &ts, max_w)); } lines.retain(|l| !l.is_empty()); if lines.is_empty() { return None; } - let vm = filmcraft_text::fonts::face(filmcraft_text::resolve("Inter", "SemiBold").face).metrics(px); + let vm = filmcraft_text::fonts::face(filmcraft_text::resolve(&ts.family, &ts.style).face).metrics(px); let (asc, desc) = (vm.ascent, vm.descent); let lh = px * style.line_spacing.max(0.8); let pad_x = (px * 0.3).round(); @@ -141,7 +139,7 @@ pub fn render_caption(c: &Caption, style: &CaptionStyle, w: usize, h: usize) -> .max(0.0) .round(); let side = w as f32 * 0.05; - let widths: Vec = lines.iter().map(|l| measure(l, px)).collect(); + let widths: Vec = lines.iter().map(|l| filmcraft_text::measure(l, &ts)).collect(); let xs: Vec = widths .iter() .map(|&lw| match align { @@ -176,7 +174,7 @@ pub fn render_caption(c: &Caption, style: &CaptionStyle, w: usize, h: usize) -> } } let baseline = lt + (lh - (asc + desc)) / 2.0 + asc; - let l = layout(line, &caption_style(px), &ParagraphStyle::default()); + let l = layout(line, &ts, &ParagraphStyle::default()); let mut m = filmcraft_text::Mask { w: ow, h: oh, a: std::mem::take(&mut cover) }; render::draw(&l, &render::at(xs[i], baseline.round()), &mut m, (ox, oy)); cover = m.a; @@ -305,6 +303,27 @@ mod tests { assert!(o.px.chunks(4).any(|p| p[0] > 0.9 && p[1] < 0.05 && p[3] > 0.9), "red outline pixels"); } + #[test] + fn track_font_is_used() { + // the style's font used to be ignored (always Inter) + let inter = render_caption(&cap("Hello world"), &CaptionStyle::default(), 1920, 1080).unwrap(); + let serif = CaptionStyle { font: "Noto Serif".into(), ..Default::default() }; + let serif = render_caption(&cap("Hello world"), &serif, 1920, 1080).unwrap(); + assert_ne!((inter.w, &inter.px), (serif.w, &serif.px), "Noto Serif renders like Inter"); + let mono = CaptionStyle { font: "jetbrains mono".into(), ..Default::default() }; + let mono = render_caption(&cap("Hello world"), &mono, 1920, 1080).unwrap(); + assert_ne!((inter.w, &inter.px), (mono.w, &mono.px), "JetBrains Mono renders like Inter"); + } + + #[test] + fn unknown_or_empty_font_falls_back_to_inter() { + let inter = render_caption(&cap("Hello"), &CaptionStyle::default(), 1280, 720).unwrap(); + for font in ["", " ", "No Such Font 123"] { + let st = CaptionStyle { font: font.into(), ..Default::default() }; + assert_eq!(render_caption(&cap("Hello"), &st, 1280, 720), Some(inter.clone()), "font {font:?}"); + } + } + #[test] fn hidden_track_draws_nothing() { let mut t = CaptionTrack::new(TrackId(1), "S".into(), CaptionFormat::Subtitle); diff --git a/crates/captions/src/lib.rs b/crates/captions/src/lib.rs index 87ef1e56..b4d4758b 100644 --- a/crates/captions/src/lib.rs +++ b/crates/captions/src/lib.rs @@ -307,7 +307,8 @@ pub fn parse_clock(s: &str) -> Option { /// Format ticks as `HH:MM:SSmmm` (rounded to the nearest millisecond). pub fn format_clock(t: Tick, sep: char) -> String { - let ms = (t.0.max(0) + TICKS_PER_MS / 2) / TICKS_PER_MS; + // The rounding addition must not overflow for an input-derived Tick near i64::MAX. + let ms = (i128::from(t.0.max(0)) + i128::from(TICKS_PER_MS) / 2) / i128::from(TICKS_PER_MS); let (h, m, s, f) = (ms / 3_600_000, (ms / 60_000) % 60, (ms / 1000) % 60, ms % 1000); format!("{h:02}:{m:02}:{s:02}{sep}{f:03}") } @@ -368,6 +369,8 @@ mod tests { assert_eq!(parse_clock("00:61:00,000"), None); assert_eq!(parse_clock("aa:00:00,000"), None); assert_eq!(format_clock(Tick(TICKS_PER_SECOND * 3723 + TICKS_PER_MS * 45), ','), "01:02:03,045"); + assert!(format_clock(Tick(i64::MAX), ',').ends_with(",077")); + assert_eq!(format_clock(Tick(i64::MIN), '.'), "00:00:00.000"); } #[test] diff --git a/crates/captions/src/stl.rs b/crates/captions/src/stl.rs index ad012395..da49f3b1 100644 --- a/crates/captions/src/stl.rs +++ b/crates/captions/src/stl.rs @@ -215,15 +215,22 @@ fn encode_text(text: &str) -> Vec { if ch == '<' && let Some(end) = rest.find('>') { - match rest[1..end].trim().to_ascii_lowercase().as_str() { - "i" => out.push(0x80), - "/i" => out.push(0x81), - "u" => out.push(0x82), - "/u" => out.push(0x83), - _ => {} + let tag = &rest[1..end]; + // Only a syntactically plausible tag is markup. Comparisons such as + // "2 < 3 > 1" and "x < y > z" must not lose their middle text. + let name = tag.strip_prefix('/').unwrap_or(tag); + let is_tag = name.as_bytes().first().is_some_and(u8::is_ascii_alphabetic); + if is_tag { + match tag.trim().to_ascii_lowercase().as_str() { + "i" => out.push(0x80), + "/i" => out.push(0x81), + "u" => out.push(0x82), + "/u" => out.push(0x83), + _ => {} + } + rest = &rest[end + 1..]; + continue; } - rest = &rest[end + 1..]; - continue; } if ch == '\n' { out.push(0x8a); @@ -451,6 +458,22 @@ mod tests { assert!(parse(&b[..500]).is_err()); } + #[test] + fn literal_angle_brackets_are_not_discarded_as_markup() { + let literal = "2 < 3 > 1, x < y > z, and 5 <7> 6"; + let doc = Document { + cues: vec![Cue { start: FrameRate::FPS_25.tick_of(25), end: FrameRate::FPS_25.tick_of(75), text: literal.into(), ..Default::default() }], + ..Default::default() + }; + let encoded = write(&doc, Some(FrameRate::FPS_25)); + let decoded = parse(&encoded).expect("valid EBU STL"); + assert_eq!(decoded.cues.len(), 1); + assert_eq!(decoded.cues[0].text, literal); + // Recognized styling still becomes STL control bytes. + assert_eq!(encode_text("Hi"), vec![0x80, b'H', b'i', 0x81]); + assert_eq!(encode_text("Hi"), vec![0x80, b'H', b'i', 0x81]); + } + #[test] fn programme_start_is_subtracted() { let doc = Document { diff --git a/crates/captions/src/ttml.rs b/crates/captions/src/ttml.rs index 1109890e..1987ba32 100644 --- a/crates/captions/src/ttml.rs +++ b/crates/captions/src/ttml.rs @@ -252,6 +252,10 @@ fn parse_time(s: &str, tm: Timing) -> Option { let (h, m) = (num(parts.first()?)?, num(parts.get(1)?)?); let sec_part = parts.get(2)?; let (si, sf) = sec_part.split_once('.').unwrap_or((sec_part, "")); + let seconds = num(si)?; + if m >= 60 || seconds >= 60 { + return None; + } let mut t = (h * 3600 + m * 60) * tps + decimal_ticks(si, sf, tps)?; if let Some(fp) = parts.get(3) { if parts.len() > 4 { @@ -259,13 +263,15 @@ fn parse_time(s: &str, tm: Timing) -> Option { } let (fi, fs) = fp.split_once('.').unwrap_or((fp, "")); let frames = num(fi)?; - let _ = tm.base; let sub: i128 = if fs.is_empty() { 0 } else { num(fs)? }; + if frames >= i128::from(tm.base) || sub >= i128::from(tm.sub_frame_rate) { + return None; + } // frames at the effective rate; sub-frames divide a frame let ft = tm.rate.tick_of(1).0 as i128; t += frames * ft + sub * ft / tm.sub_frame_rate.max(1) as i128; } - return Some(Tick(t as i64)); + return i64::try_from(t).ok().map(Tick); } let unit_start = s.find(|c: char| c.is_ascii_alphabetic())?; let (v, unit) = s.split_at(unit_start); @@ -290,7 +296,7 @@ fn parse_time(s: &str, tm: Timing) -> Option { } _ => return None, }; - Some(Tick(t as i64)) + i64::try_from(t).ok().map(Tick) } #[derive(Clone, Copy, Default, PartialEq)] @@ -552,6 +558,34 @@ mod tests { assert_eq!(parse_time("1:2", t), None); } + #[test] + fn invalid_and_overflowed_time_expressions_are_ignored() { + let normal = tm(FrameRate::FPS_25); + for expr in [ + "999999999999999h", + "999999999999999m", + "999999999999999s", + "999999999999999ms", + "999999999999999f", + "999999999999999t", + "00:60:00", + "00:00:60", + "00:00:01:25", + ] { + assert_eq!(parse_time(expr, normal), None, "{expr}"); + } + let subframes = Timing { sub_frame_rate: 2, ..normal }; + assert!(parse_time("00:00:01:12.1", subframes).is_some()); + assert_eq!(parse_time("00:00:01:12.2", subframes), None); + // Bad cues must not wrap to a plausible but unrelated timeline position. + let xml = + r#"

Bad

Good

"#; + let doc = parse(xml).expect("well-formed TTML"); + assert_eq!(doc.cues.len(), 1); + assert_eq!(doc.cues[0].text, "Good"); + assert!(!doc.warnings.is_empty()); + } + #[test] fn reads_styles_spans_and_nesting() { let x = r#" diff --git a/crates/cfb/src/read.rs b/crates/cfb/src/read.rs index 58c300b5..5ba3ed68 100644 --- a/crates/cfb/src/read.rs +++ b/crates/cfb/src/read.rs @@ -84,7 +84,6 @@ impl<'a> CompoundFile<'a> { let mini_cutoff = u32_at(data, 56) as u64; let first_minifat = u32_at(data, 60); let first_difat = u32_at(data, 68); - let n_difat = u32_at(data, 72) as usize; let mut header_clsid = [0u8; 16]; header_clsid.copy_from_slice(&data[8..24]); let max_sectors = data.len() / sector_size + 1; @@ -108,8 +107,10 @@ impl<'a> CompoundFile<'a> { let mut seen = 0; while d <= MAXREGSECT && fat_sectors.len() < n_fat { seen += 1; - if seen > n_difat.max(1) + max_sectors { - return Err(invalid("DIFAT chain loops")); + // a DIFAT chain cannot have more sectors than the file holds; the header's DIFAT sector + // count (up to 2^32 - 1) is not trusted, or a self-pointing sector loops that often + if seen > max_sectors { + return Err(invalid(format!("DIFAT chain loops: more than {max_sectors} sectors in a {}-byte file", data.len()))); } let s = cf.sector(d)?; let per = sector_size / 4 - 1; @@ -169,13 +170,17 @@ impl<'a> CompoundFile<'a> { let mut out = Vec::new(); let mut s = start; let mut steps = 0usize; + // A chain cannot visit more sectors than the file holds. Not `self.fat.len()`: the FAT can + // be far longer (one FAT sector named by many header / DIFAT slots), and a looping chain + // would then append a whole sector per FAT entry (hundreds of MB, or an abort). + let max_steps = self.data.len() / self.sector_size + 1; while s != ENDOFCHAIN { if s > MAXREGSECT { return Err(invalid(format!("bad sector {s:#x} in a chain"))); } steps += 1; - if steps > self.fat.len().max(1) { - return Err(invalid("sector chain loops")); + if steps > max_steps { + return Err(invalid(format!("sector chain loops: more than {max_steps} sectors in a {}-byte file", self.data.len()))); } let b = self.sector(s)?; out.extend_from_slice(b); diff --git a/crates/cfb/src/tests.rs b/crates/cfb/src/tests.rs index cdcb62db..7e9854fe 100644 --- a/crates/cfb/src/tests.rs +++ b/crates/cfb/src/tests.rs @@ -199,3 +199,95 @@ fn mutation_never_panics_or_hangs() { } } } + +fn put_u32(b: &mut [u8], at: usize, v: u32) { + b[at..at + 4].copy_from_slice(&v.to_le_bytes()); +} + +/// A 512-byte header (§2.2) with sector shift `shift`: no directory, mini FAT or DIFAT chain, and +/// all 109 FAT slots free. Tests fill in the fields they attack. +fn bare_header(shift: u16) -> Vec { + let mut h = vec![0u8; 512]; + h[..8].copy_from_slice(&SIGNATURE); + h[24..26].copy_from_slice(&0x3Eu16.to_le_bytes()); + h[26..28].copy_from_slice(&(if shift == 9 { 3u16 } else { 4 }).to_le_bytes()); + h[28..30].copy_from_slice(&0xFFFEu16.to_le_bytes()); + h[30..32].copy_from_slice(&shift.to_le_bytes()); + h[32..34].copy_from_slice(&6u16.to_le_bytes()); + for at in [48, 60, 68] { + put_u32(&mut h, at, ENDOFCHAIN); + } + put_u32(&mut h, 56, MINI_STREAM_CUTOFF); + h[76..].fill(0xFF); + h +} + +/// A DIFAT sector that names no FAT sector and points to itself. The loop guard used to be the +/// header's DIFAT sector count (up to 2^32 - 1), so this 1 KB file kept `open` (and so AAF +/// sniffing) busy for 14-16 s here (minutes with `u32::MAX`), reading a sector each pass. A DIFAT +/// chain cannot be longer than the file, so the walk now stops after as many passes as the file +/// has sectors (1024 / 512 + 1 = 3 here) and fails on the 4th. +#[test] +fn difat_self_loop_fails_fast() { + let mut b = bare_header(9); + put_u32(&mut b, 44, 1); // one FAT sector, to be found through the DIFAT + put_u32(&mut b, 68, 0); // first DIFAT sector + put_u32(&mut b, 72, 100_000_000); // claimed DIFAT sector count (inflates the old loop bound) + let mut difat = vec![0xFF; 512]; // no FAT sector numbers... + put_u32(&mut difat, 508, 0); // ...and the next DIFAT sector is this one + b.extend_from_slice(&difat); + // The message names the bound the walk stopped at: the file's 3 sectors, not the header's + // 100,000,000 (the old guard allowed 100,000,003 passes). + let expect = format!("DIFAT chain loops: more than 3 sectors in a {}-byte file", b.len()); + assert_eq!(CompoundFile::open(&b).unwrap_err(), Error::Invalid(expect)); +} + +/// All 109 header slots name the same FAT sector, so the FAT is 109 copies of it, and the +/// directory chain loops (sector 0 → 0). The loop guard used to be the FAT length, so this 450 KB +/// file appended a sector per FAT entry before failing: a 457 MB buffer per open (117 GB with +/// 64 KB sectors in a 7 MB file). A chain cannot visit more sectors than the file has. +#[test] +fn duplicated_fat_sector_chain_loop_fails_fast() { + let ss = 4096; + let mut b = bare_header(12); + put_u32(&mut b, 44, 109); // FAT sectors: the 109 header slots... + for i in 0..109 { + put_u32(&mut b, 76 + i * 4, 0); // ...all naming sector 0 + } + put_u32(&mut b, 48, 0); // the directory starts at sector 0 + b.resize(ss, 0); + let mut fat = vec![0xFF; ss]; + put_u32(&mut fat, 0, 0); // FAT[0] = 0: sector 0 is followed by itself + b.extend_from_slice(&fat); + b.resize(110 * ss, 0); // big enough for 109 FAT sectors to pass the size check + // The message names the bound the walk stopped at: 111, this file's 110 sectors + 1, not the + // 109 * 1024 = 111,616 entries of the inflated FAT that the old guard allowed (a 457 MB buffer). + let expect = format!("sector chain loops: more than 111 sectors in a {}-byte file", b.len()); + assert_eq!(CompoundFile::open(&b).unwrap_err(), Error::Invalid(expect)); +} + +/// Positive control for the chain bound: the longest chains a file can hold still read back, in +/// both versions: a directory (read without a size limit) and a stream, each filling every sector +/// but the header and the FAT (and the directory, for the stream). +#[test] +fn chains_filling_the_file_still_read() { + for v in [Version::V3, Version::V4] { + let ss = v_sector(v); + let mut w = Writer::new(v); + for i in 0..1000 { + w.stream(Writer::ROOT, &format!("e{i}"), Vec::new()).unwrap(); + } + let b = w.finish(); + assert!(b.len() / ss <= (1001 * 128usize).div_ceil(ss) + 3, "header, directory, FAT"); + let cf = CompoundFile::open(&b).unwrap(); + assert_eq!(cf.root().children.len(), 1000); + assert_eq!(cf.read_path("e999").unwrap(), Vec::::new()); + + let mut w = Writer::new(v); + let data = pattern(64 * ss, 3); + w.stream(Writer::ROOT, "big", data.clone()).unwrap(); + let b = w.finish(); + assert_eq!(b.len(), (1 + 64 + 2) * ss, "header, stream, directory, FAT"); + assert_eq!(CompoundFile::open(&b).unwrap().read_path("big").unwrap(), data); + } +} diff --git a/crates/codecs/Cargo.toml b/crates/codecs/Cargo.toml index 3a490e2c..7caf5eb9 100644 --- a/crates/codecs/Cargo.toml +++ b/crates/codecs/Cargo.toml @@ -31,6 +31,7 @@ filmcraft-av1 = { workspace = true } filmcraft-apv = { workspace = true } filmcraft-aac = { workspace = true } filmcraft-opus = { workspace = true } +rayon = { workspace = true } thiserror = { workspace = true } log = { workspace = true } # Decode timing counters (`gop_stats`); std's Instant panics on wasm32. diff --git a/crates/codecs/src/audio.rs b/crates/codecs/src/audio.rs index 5f7c02f4..e364402a 100644 --- a/crates/codecs/src/audio.rs +++ b/crates/codecs/src/audio.rs @@ -16,6 +16,7 @@ use crate::{CodecError, Result}; enum Inner { /// Our own AAC-LC decoder. `up` interpolates HE-AAC's core to the output rate (see [`Upsample2x`]). + /// Surround output is reordered to WAV/SMPTE order (see [`aac_to_wav_order`]). Aac { dec: Box, asc: Vec, up: Option> }, /// AAC whose configuration arrives with the first frame (ADTS / LATM). AacPending, @@ -52,6 +53,29 @@ fn vorbis_to_wav_order(channels: usize) -> Option<&'static [usize]> { }) } +/// Channel order for AAC surround (ISO/IEC 14496-3 Table 1.19, our decoder's element order: C, L R, +/// surrounds, LFE): output channel → decoded channel, giving the WAV/SMPTE order (L R C LFE Ls Rs) +/// used elsewhere, as the AC-3 decoder does and the AAC encoder's caller undoes on export. Only the +/// unambiguous channel configurations 3–6; 1, 2, 7 and PCE layouts keep the decoded order. +fn aac_to_wav_order(channel_config: u8) -> Option<&'static [usize]> { + Some(match channel_config { + 3 => &[1, 2, 0], + 4 => &[1, 2, 0, 3], + 5 => &[1, 2, 0, 3, 4], + 6 => &[1, 2, 0, 5, 3, 4], + _ => return None, + }) +} + +/// `channels` reordered by `order` (output channel → decoded channel). A decode whose channel count +/// does not match the order is returned unchanged. +fn reorder(mut channels: Vec>, order: Option<&[usize]>) -> Vec> { + match order { + Some(order) if order.len() == channels.len() => order.iter().map(|&c| channels.get_mut(c).map(std::mem::take).unwrap_or_default()).collect(), + _ => channels, + } +} + /// Duration of an Opus packet in 48 kHz samples from its TOC and frame count (RFC 6716 §3.1), /// without parsing frame lengths. For a multistream packet this is the first stream's TOC, which /// every stream shares. `None` for an empty or invalid (> 120 ms) packet. @@ -152,18 +176,17 @@ impl PacketDecoder { if let Some(up) = up { out = up.process(out); } + let out = reorder(out, aac_to_wav_order(dec.config().channel_config)); self.channels = out.len(); return Ok(out); } Inner::Opus { dec, order } => { // A corrupt packet is concealed like a lost one (keeps timing and decoder state). - let mut out = match dec.decode(Some(data)) { + let out = match dec.decode(Some(data)) { Ok(o) => o, Err(_) => dec.decode(None).map_err(|e| CodecError::Decode(e.to_string()))?, }; - if let Some(order) = order { - out = order.iter().map(|&c| std::mem::take(&mut out[c])).collect(); - } + let out = reorder(out, *order); self.channels = out.len(); return Ok(out); } @@ -811,6 +834,63 @@ mod he_aac_tests { } } +#[cfg(test)] +mod surround_order_tests { + use super::*; + + /// Power of `x` at `hz` (Goertzel), normalised by length. + fn power(x: &[f32], hz: f32, rate: f32) -> f32 { + let k = 2.0 * (std::f32::consts::TAU * hz / rate).cos(); + let (mut s1, mut s2) = (0.0f32, 0.0f32); + for &v in x { + let s0 = v + k * s1 - s2; + s2 = s1; + s1 = s0; + } + (s1 * s1 + s2 * s2 - k * s1 * s2) / x.len() as f32 + } + + /// #721: 5.1 AAC (configuration 6, coded C L R Ls Rs LFE) decodes to WAV/SMPTE order + /// L R C LFE Ls Rs, as the rest of FilmCraft and the AAC export path expect. + #[test] + fn aac_51_decodes_in_wav_order() { + let rate = 48_000u32; + // tones per speaker, in WAV order: L, R, C, LFE, Ls, Rs + let hz = [300.0f32, 400.0, 500.0, 60.0, 700.0, 800.0]; + let tone = |f: f32| (0..rate as usize).map(|i| 0.25 * (std::f32::consts::TAU * f * i as f32 / rate as f32).sin()).collect::>(); + let wav: Vec> = hz.iter().map(|&f| tone(f)).collect(); + // the encoder takes AAC element order (as the exporter feeds it): C L R Ls Rs LFE + let coded: Vec<&[f32]> = [2usize, 0, 1, 4, 5, 3].iter().map(|&c| wav[c].as_slice()).collect(); + let mut enc = filmcraft_aac::Encoder::new(filmcraft_aac::EncoderConfig::cbr(rate, 6, 384_000)).unwrap(); + let mut units = enc.encode(&coded); + units.extend(enc.flush()); + + let mut dec = PacketDecoder::aac(&enc.audio_specific_config(), rate).unwrap(); + let mut out = vec![Vec::new(); 6]; + for u in &units { + for (o, c) in out.iter_mut().zip(dec.decode(u, 0).unwrap()) { + o.extend(c); + } + } + assert_eq!(dec.channels, 6); + for (ch, samples) in out.iter().enumerate() { + let window = &samples[4800..38_400]; + let own = power(window, hz[ch], rate as f32); + for (other, &f) in hz.iter().enumerate().filter(|&(o, _)| o != ch) { + let p = power(window, f, rate as f32); + assert!(own > 100.0 * p, "channel {ch}: {} Hz {own} vs {f} Hz (channel {other}) {p}", hz[ch]); + } + } + } + + #[test] + fn reorder_leaves_mismatched_channel_counts_alone() { + let chans = vec![vec![1.0], vec![2.0]]; + assert_eq!(reorder(chans.clone(), aac_to_wav_order(6)), chans); + assert_eq!(reorder(vec![vec![0.0], vec![1.0], vec![2.0]], aac_to_wav_order(3)), vec![vec![1.0], vec![2.0], vec![0.0]]); + } +} + #[cfg(test)] mod packet_time_tests { use super::*; diff --git a/crates/codecs/src/gop.rs b/crates/codecs/src/gop.rs index b23b130e..4f442887 100644 --- a/crates/codecs/src/gop.rs +++ b/crates/codecs/src/gop.rs @@ -109,8 +109,9 @@ impl std::ops::Sub for GopStats { } } -/// A container's video sample table, in decode (file) order. -pub trait VideoSamples { +/// A container's video sample table, in decode (file) order. `Sync`: a cache decodes on its own +/// threads ([`isolated`]). +pub trait VideoSamples: Sync { fn count(&self) -> usize; /// Presentation timestamp of sample `i` (track units). fn pts(&self, i: usize) -> i64; @@ -425,6 +426,57 @@ const MIN_FRAMES: usize = 64; /// rather than restarting at the sync sample. const CONTINUE_THROUGH: usize = 48; +/// Threads that run the caches' decoding (see [`isolated`]); `None` where none can be made. +#[cfg(not(target_arch = "wasm32"))] +fn decode_pool() -> Option<&'static rayon::ThreadPool> { + static POOL: std::sync::OnceLock> = std::sync::OnceLock::new(); + POOL.get_or_init(|| { + let n = std::thread::available_parallelism().map(|n| n.get()).unwrap_or(1); + rayon::ThreadPoolBuilder::new().num_threads(n).thread_name(|i| format!("decode-{i}")).build().ok() + }) + .as_ref() +} + +/// Run a cache's decoding, its lock held by the caller, on [`decode_pool`], the caller waiting +/// without doing other rayon work meanwhile (#480). +/// +/// That decoding starts rayon work (a decoder's slices, cropping and rotating the frames). On the +/// pool of the render jobs that ask the cache for frames, a worker that took part of it could, +/// waiting inside it, pick up another of those jobs and block on the lock: the lock holder then +/// waits for its work on the blocked worker's stack, and that worker for the lock, for good (an +/// export of rotated iPhone clips stopped making frames). On the decode pool that work only ever +/// runs beside other caches' decoding, which never waits for a cache's lock. A caller that is a +/// rayon worker would also run other jobs of its pool while it waits for the decode pool, some +/// of them asking for a cache's lock while it holds this one: it waits on a plain thread instead. +fn isolated(f: impl FnOnce() -> crate::Result + Send) -> crate::Result { + let ctx = filmcraft_media::cancel::Context::current(); + let job = move || ctx.run(f); + #[cfg(not(target_arch = "wasm32"))] + if let Some(pool) = decode_pool() { + if rayon::current_thread_index().is_none() { + return pool.install(job); + } + let slot = Mutex::new(Some(job)); + let waited = std::thread::scope(|scope| { + let spawned = std::thread::Builder::new().name("decode-wait".into()).spawn_scoped(scope, || { + let job = slot.lock().unwrap_or_else(PoisonError::into_inner).take(); + job.map(|job| pool.install(job)) + }); + spawned.map(|h| h.join()) + }); + return match waited { + Ok(Ok(Some(r))) => r, + Ok(Err(_)) => Err(CodecError::Decode("internal error while decoding".into())), + // no thread to wait on: decode here, as before + Ok(Ok(None)) | Err(_) => match slot.into_inner().unwrap_or_else(PoisonError::into_inner) { + Some(job) => job(), + None => Err(CodecError::Decode("internal error while decoding".into())), + }, + }; + } + job() +} + impl GopCache { pub fn new(explicit_color: Option) -> Self { // unit tests count decoder restarts: each of their caches gets a pool of its own @@ -482,7 +534,11 @@ impl GopCache { } fn store(&self, st: &mut State, pts: i64, f: VideoFrame, draft: bool) { - let f = self.finish(f); + self.store_finished(st, pts, self.finish(f), draft); + } + + /// Cache a frame [`Self::finish`] has been applied to. + fn store_finished(&self, st: &mut State, pts: i64, f: VideoFrame, draft: bool) { st.last_used = Instant::now(); if draft { st.drafts.insert(pts); @@ -604,6 +660,22 @@ impl GopCache { if st.intra { return self.intra_frame(st, s, i, want_pts); } + // with the lock held: see `isolated` + let st = &mut *st; + isolated(|| self.decode_until(st, s, i, want_pts, late_before, draft)) + } + + /// Decode (the lock held) until the frame with `want_pts` (sample `i`) is cached; return it. + fn decode_until( + &self, + st: &mut State, + s: &dyn VideoSamples, + i: usize, + want_pts: i64, + late_before: Option, + draft: bool, + ) -> crate::Result> { + let n = s.count(); let mut key = s.sync_before(i); // Continue the running decoder when it has passed the wanted sample's sync sample and // either has not reached the sample yet or has been fed it without outputting it yet @@ -656,14 +728,14 @@ impl GopCache { let out = timed(|| decoder.decode(&data, s.pts(k)))?; st.next += 1; DECODED.fetch_add(1, Ordering::Relaxed); - self.store_output(&mut st, out); + self.store_output(st, out); if st.cached(want_pts, draft).is_some() { break; } } if st.cached(want_pts, draft).is_none() { let out = st.decoder.as_mut().map(|d| timed(|| d.flush())).unwrap_or_default(); - self.store_output(&mut st, out); + self.store_output(st, out); st.next = usize::MAX; } // nearest decoded frame at or before the wanted pts (robust to decoder pts quirks) @@ -748,14 +820,17 @@ impl GopCache { DECODED.fetch_add(1, Ordering::Relaxed); timed(|| dec.decode(&data, want_pts)) }); + // crop and rotate before taking the lock: that work runs on rayon (see `isolated`) + let res = res.map(|out| { + count_frames(&out); + out.into_iter().map(|d| (d.pts, self.finish(d.frame), d.draft)).collect::>() + }); let mut st = self.shared.state.lock().unwrap_or_else(|e| e.into_inner()); if st.spare.len() < 16 { st.spare.push(dec); } - let res = res?; - count_frames(&res); - for d in res { - self.store(&mut st, d.pts, d.frame, d.draft); + for (pts, frame, draft) in res? { + self.store_finished(&mut st, pts, frame, draft); } st.at_or_before(want_pts, true).ok_or_else(|| CodecError::Decode("frame not produced".into())) } @@ -887,6 +962,74 @@ mod tests { assert!(s.decodes.load(Ordering::Relaxed) <= 60 + 24 + 1); } + /// A decoder that, like a slice-threaded one, splits each picture's work over rayon. + struct ParDec(Box); + + impl VideoDecoder for ParDec { + fn decode(&mut self, sample: &[u8], pts: i64) -> crate::Result> { + use rayon::prelude::*; + let rows: u64 = + (0..64u32).into_par_iter().with_max_len(1).map(|r| (0..20_000u64).fold(u64::from(r), |a, x| a.wrapping_mul(31).wrapping_add(x))).sum(); + std::hint::black_box(rows); + self.0.decode(sample, pts) + } + fn flush(&mut self) -> Vec { + self.0.flush() + } + fn reset(&mut self) { + self.0.reset() + } + fn name(&self) -> &str { + "par" + } + } + + struct ParSamples(Samples); + + impl VideoSamples for ParSamples { + fn count(&self) -> usize { + self.0.count() + } + fn pts(&self, i: usize) -> i64 { + self.0.pts(i) + } + fn sync_before(&self, i: usize) -> usize { + self.0.sync_before(i) + } + fn sample_at(&self, t: i64) -> Option { + self.0.sample_at(t) + } + fn read(&self, i: usize) -> crate::Result> { + self.0.read(i) + } + fn make_decoder(&self) -> crate::Result> { + Ok(Box::new(ParDec(self.0.make_decoder()?))) + } + } + + /// #480: render jobs on rayon asking one source for frames while its decoding (a decoder's + /// slices, rotating the frames) runs on rayon too. A worker that took part of that work and, + /// waiting inside it, picked up another frame request blocked on the cache's lock for good: + /// an export of rotated iPhone clips stopped making frames. + #[test] + fn rayon_frame_requests_while_decoding_on_rayon_never_deadlock() { + use rayon::prelude::*; + let s = Arc::new(ParSamples(samples(240, 30, 4, false))); + let (tx, rx) = std::sync::mpsc::channel(); + std::thread::spawn(move || { + for round in 0..40usize { + // export-like: many frame workers, frames of several GOPs of a fresh source at once + let c = GopCache::new(None).with_rotation(1); + (0..96usize).into_par_iter().with_max_len(1).for_each(|k| { + let i = (k * 37 + round * 53) % 240; + assert_eq!(index_of(&c.frame(&*s, i as i64 * 1000).expect("frame")), i); + }); + } + let _ = tx.send(()); + }); + assert!(rx.recv_timeout(Duration::from_secs(120)).is_ok(), "frame requests deadlocked"); + } + #[test] fn running_decoder_continues_through_a_nearby_sync_sample() { let s = samples(200, 30, 0, false); diff --git a/crates/codecs/src/lib.rs b/crates/codecs/src/lib.rs index a31da3a7..f7de9e3d 100644 --- a/crates/codecs/src/lib.rs +++ b/crates/codecs/src/lib.rs @@ -30,6 +30,7 @@ pub mod mxf; pub mod ogg; mod stream_color; pub mod video; +mod webm_alpha; use std::sync::{Arc, RwLock}; diff --git a/crates/codecs/src/mkv.rs b/crates/codecs/src/mkv.rs index cef204d9..2efb5b53 100644 --- a/crates/codecs/src/mkv.rs +++ b/crates/codecs/src/mkv.rs @@ -58,6 +58,8 @@ pub struct MkvSource { /// Video codec as an ISO-BMFF sample entry (for the decoder factories). ventry: Option, video: GopCache, + /// The alpha layer of a VP9 track with `AlphaMode` 1 (WebM transparency, #402). + alpha: Option>, /// The playable audio tracks in file order: `info.audio_streams[k]` describes `audios[k]`. audios: Vec, } @@ -380,7 +382,12 @@ impl MkvSource { Some(MkvAudio { track: i, state, starts, preroll }) }) .collect(); - Ok(Self { info, bytes, file, vtrack, ventry, video: GopCache::new(explicit_color).with_rotation(rotation), audios }) + // WebM transparency: VP9 alpha in each block's BlockAdditional (ID 1) + let alpha = vtrack.and_then(|i| file.tracks.get(i)).filter(|t| { + matches!(t.codec, Codec::Vp9 { .. }) && t.video.as_ref().is_some_and(|v| v.alpha_mode != 0) && t.samples.iter().any(|s| s.addition.is_some()) + }); + let alpha = alpha.map(|t| Arc::new(crate::webm_alpha::AlphaLayer::new(bytes.clone(), t.samples.iter().map(|s| (s.pts, s.addition))))); + Ok(Self { info, bytes, file, vtrack, ventry, video: GopCache::new(explicit_color).with_rotation(rotation), alpha, audios }) } fn read(&self, track: usize, i: usize) -> crate::Result> { @@ -486,9 +493,15 @@ impl VideoSamples for MkvVideo<'_> { self.src.read(self.track, i) } fn make_decoder(&self) -> crate::Result> { - match &self.src.ventry { - Some(e) => make_video_decoder(e), - None => Err(CodecError::Unsupported(format!("no decoder for {} video", codec_label(&self.src.file.tracks[self.track].codec)))), + let Some(e) = &self.src.ventry else { + return Err(CodecError::Unsupported(format!("no decoder for {} video", codec_label(&self.src.file.tracks[self.track].codec)))); + }; + let color = make_video_decoder(e)?; + let Some(layer) = &self.src.alpha else { return Ok(color) }; + // the alpha layer always goes through our own decoder (it is luma only) + match crate::software_video_decoder(e) { + Ok(alpha) => Ok(Box::new(crate::webm_alpha::AlphaDecoder::new(color, alpha, layer.clone()))), + Err(_) => Ok(color), } } } diff --git a/crates/codecs/src/mp4.rs b/crates/codecs/src/mp4.rs index 62f4f547..dffa0dd6 100644 --- a/crates/codecs/src/mp4.rs +++ b/crates/codecs/src/mp4.rs @@ -9,7 +9,7 @@ use std::sync::{Arc, Mutex}; use filmcraft_color::ColorInfo; use filmcraft_frame::{AudioBuffer, Region, VideoFrame}; -use filmcraft_isobmff::{CleanAperture, CodecConfig, Mp4File, TrackKind}; +use filmcraft_isobmff::{ByteSource, CleanAperture, CodecConfig, Mp4File, TrackKind}; use filmcraft_media::{AudioStreamInfo, FrameRequest, MediaError, MediaInfo, MediaKind, MediaSource, SharedSource, VideoStreamInfo}; use filmcraft_time::{FrameRate, Tick}; @@ -162,6 +162,48 @@ fn color_from(entry: &filmcraft_isobmff::SampleEntry, w: u32, h: u32) -> ColorIn crate::stream_color::resolve(w, h, &sources) } +/// First bytes of a Sony `rtmd` sample needed for the start-timecode block (`frames` is last). +const RTMD_HEADER_LEN: usize = 0x12; + +/// Sony XAVC `rtmd` (real-time metadata) start timecode fields, read from the first sample: hours, +/// minutes, seconds, a drop-frame flag and frames. Sony stores them as raw decimal bytes in the +/// sample's fixed header. The layout is proprietary and undocumented, so it is reproduced here from +/// real footage; it is the same block ffmpeg reports as the `timecode` tag of an `rtmd` stream. +/// +/// Fields outside SMPTE bounds — including the labels drop-frame counting skips — are rejected +/// (no fabricated start time from damaged metadata). +fn rtmd_fields(sample: &[u8], rate: FrameRate) -> Option<(i64, i64, i64, i64, bool)> { + let f = sample.get(0x0d..0x12)?; + let (h, m, s) = (f[0] as i64, f[1] as i64, f[2] as i64); + let frames = f[4] as i64; + let drop = f[3] != 0; + let base = rate.timecode_base(); + if h > 23 || m > 59 || s > 59 || frames >= base { + return None; + } + // Drop-frame counting skips the first `base / 15` frame labels of every minute except each + // tenth minute, so those labels never occur in valid footage (SMPTE ST 12-1). + if drop && rate.supports_drop_frame() && s == 0 && m % 10 != 0 && frames < base / 15 { + return None; + } + Some((h, m, s, frames, drop)) +} + +/// Start timecode from a Sony `rtmd` track as a frame count at `rate` (issue #460), used when the +/// file has no `tmcd` track. `None` if there is no `rtmd` track or it has no usable first sample. +/// Only the fixed-size header is read (a damaged `stsz` size can't drive a huge allocation). +fn rtmd_start_timecode(file: &Mp4File, bytes: &crate::Src, rate: FrameRate) -> Option { + let t = file.tracks.iter().find(|t| t.entries.first().is_some_and(|e| e.format.0 == *b"rtmd"))?; + let s = t.samples.first()?; + if s.size < RTMD_HEADER_LEN as u32 { + return None; + } + let mut buf = [0u8; RTMD_HEADER_LEN]; + bytes.read_at(s.offset, &mut buf).ok()?; + let (h, m, s, f, drop) = rtmd_fields(&buf, rate)?; + Some(filmcraft_time::fields_to_frames(h, m, s, f, rate, drop)) +} + impl Mp4Source { pub fn open(name: &str, bytes: Arc<[u8]>) -> crate::Result { Self::open_reader(name, Arc::new(filmcraft_media::reader::MemReader(bytes))) @@ -170,7 +212,7 @@ impl Mp4Source { /// Open from a random-access reader: only the index is read now, samples on demand. pub fn open_reader(name: &str, reader: filmcraft_media::SharedReader) -> crate::Result { let bytes = crate::Src(reader); - let file = filmcraft_isobmff::open(&bytes).map_err(|e| CodecError::Container(e.to_string()))?; + let mut file = filmcraft_isobmff::open(&bytes).map_err(|e| CodecError::Container(e.to_string()))?; // A track needs samples, a sample description and a timescale to be playable (damaged // files can lack them: indexing `entries[0]` or dividing by the timescale used to panic). let playable = |t: &&filmcraft_isobmff::Track, kind| t.kind == kind && !t.samples.is_empty() && !t.entries.is_empty() && t.timescale > 0; @@ -271,6 +313,9 @@ impl Mp4Source { Some(CodecConfig::Timecode(tc)) => tc.start_frame.map(|f| f as i64), _ => None, }); + // Sony XAVC files with no `tmcd` track carry the start timecode in an `rtmd` metadata + // track; convert its fields to a frame count using the video frame rate. + let start_timecode = start_timecode.or_else(|| rtmd_start_timecode(&file, &bytes, video.as_ref()?.frame_rate)); let info = MediaInfo { name: name.to_string(), kind: if video.is_some() { MediaKind::Movie } else { MediaKind::AudioOnly }, @@ -281,7 +326,7 @@ impl Mp4Source { start_timecode, file_size: Some(bytes.0.len()), }; - let audios: Vec = atracks + let mut audios: Vec = atracks .iter() .zip(&info.audio_streams) .filter_map(|(&i, ainfo)| { @@ -300,6 +345,7 @@ impl Mp4Source { Some(Mp4Audio { track: i, state, starts, offset, preroll }) }) .collect(); + rebase_unedited_video(&mut file, vtrack, &mut audios); Ok(Self { info, bytes, file, vtrack, video: GopCache::new(explicit_color).with_rotation(rotation).with_crop(aperture), audios }) } @@ -605,6 +651,35 @@ pub fn opener(name: &str, bytes: Arc<[u8]>) -> Option, audios: &mut [Mp4Audio]) { + let Some(vi) = vtrack else { return }; + let Some(t) = file.tracks.get(vi).filter(|t| t.edits.is_empty() && t.timescale > 0) else { return }; + let Some(first) = t.samples.iter().map(|s| s.pts).min().map(|p| p.saturating_add(t.edit_offset)).filter(|&p| p > 0) else { return }; + let vts = i64::from(t.timescale); + let video_start = Tick::from_rational(first, 1, vts); + let audio_ts = |a: &Mp4Audio| file.tracks.get(a.track).map_or(1, |t| i64::from(t.timescale.max(1))); + // an audio track starts at its offset (a negative one only hides priming before zero) + let shift = audios.iter().fold(video_start, |m, a| m.min(Tick::from_rational(a.offset.max(0), 1, audio_ts(a)))); + if shift <= Tick::ZERO { + return; + } + for a in audios.iter_mut() { + let d = shift.to_rational_round(1, audio_ts(a)); + a.offset = a.offset.saturating_sub(d); + } + let v = if shift == video_start { first } else { shift.to_rational_round(1, vts) }; + if let Some(t) = file.tracks.get_mut(vi) { + t.edit_offset = t.edit_offset.saturating_sub(v); + } +} + /// How long a track plays. With an edit list that is the sum of its edits (ISO/IEC 14496-12 /// §8.6.6, movie timescale), as `tkhd` records it. The media duration (`mdhd`) runs on the decode /// timeline: it is longer than the presentation by the B-frame delay an edit skips (`media_time`, one @@ -700,6 +775,30 @@ mod tests { assert_eq!(frames(&Mp4Source::open("zero.mov", mov_with_edits(zero)).expect("open")), 4); } + /// #714: a video track without an edit list whose pictures carry a composition delay (B-frame + /// style, as x264 writes them with no `elst`) starts at its first picture. Media time zero + /// repeated picture 0 for the length of the delay and the last pictures never showed. + #[test] + fn unedited_composition_delay_starts_at_the_first_picture() { + let mut mux = Mp4Writer::new(std::io::Cursor::new(Vec::new()), WriterOptions::new(Brand::Mov)).expect("writer"); + let t = mux.add_track(TrackConfig::new(SampleEntry::prores(FourCc(*b"apch"), 64, 32), 25)).expect("track"); + for k in 0..4u16 { + let mut fr = filmcraft_prores::Frame::new(64, 32, filmcraft_prores::ChromaFormat::Yuv422, 10, false); + fr.y.fill(100 + 200 * k); + let data = filmcraft_prores::Encoder::new(filmcraft_prores::Profile::Hq, 64, 32).encode(&fr).expect("encode"); + mux.write_sample(t, WriteSample { data: &data, duration: 1, composition_offset: 2, is_sync: true }).expect("sample"); + } + let src = Mp4Source::open("delay.mov", mux.finish().expect("finish").into_inner().into()).expect("open"); + assert_eq!(src.info().duration, Tick::from_rational(4, 1, 25)); + let luma = |k: i64| { + let f = src.video_frame(FrameRequest::full(Tick::from_rational(k, 1, 25))).expect("frame"); + let l = f.luma8(); + l.iter().map(|&v| u32::from(v)).sum::() / l.len().max(1) as u32 + }; + let seen: Vec = (0..4).map(luma).collect(); + assert!(seen.windows(2).all(|w| w[0] < w[1]), "one picture per frame, in order: {seen:?}"); + } + /// Overwrite the big-endian u32 `skip` bytes after the first `fourcc` box type. fn patch_u32(b: &mut [u8], fourcc: &[u8; 4], skip: usize, v: u32) { let k = b.windows(4).position(|x| x == fourcc).expect("box") + 4 + skip; @@ -914,6 +1013,48 @@ mod tests { assert_eq!((v.pixel_format.as_str(), v.has_alpha), (want, alpha), "{}", String::from_utf8_lossy(fourcc)); } } + + #[test] + fn rtmd_fields_parse_and_reject_damaged_metadata() { + let rate = FrameRate::FPS_59_94; + // 03:37:12:34, non-drop (a Sony FX3/a6400 XAVC header). + let mut s = [0u8; 24]; + s[0x0d..0x12].copy_from_slice(&[3, 37, 12, 0, 34]); + assert_eq!(rtmd_fields(&s, rate), Some((3, 37, 12, 34, false))); + s[0x10] = 1; + assert_eq!(rtmd_fields(&s, rate), Some((3, 37, 12, 34, true))); + // Too short to hold the block: no fields, no panic. + assert_eq!(rtmd_fields(&[0u8; 6], rate), None); + // Damaged fields are rejected rather than normalized into a plausible time. + for bad in [[3, 37, 12, 0, 60], [3, 60, 0, 0, 0], [24, 0, 0, 0, 0], [0, 0, 60, 0, 0]] { + let mut b = [0u8; 24]; + b[0x0d..0x12].copy_from_slice(&bad); + assert_eq!(rtmd_fields(&b, rate), None, "{bad:?}"); + } + } + + #[test] + fn rtmd_fields_reject_drop_frame_skipped_labels() { + let df = |h, m, s, f| { + let mut b = [0u8; 24]; + b[0x0d..0x12].copy_from_slice(&[h, m, s, 1, f]); + rtmd_fields(&b, FrameRate::FPS_59_94) + }; + // At 59.94 DF, frames 00-03 of a non-tenth minute are skipped, so they never occur. + assert_eq!(df(0, 1, 0, 0), None); + assert_eq!(df(0, 1, 0, 3), None); + assert_eq!(df(0, 1, 0, 4), Some((0, 1, 0, 4, true))); + assert_eq!(df(0, 10, 0, 0), Some((0, 10, 0, 0, true))); // tenth minute: label exists + assert_eq!(df(0, 1, 1, 0), Some((0, 1, 1, 0, true))); // seconds > 0: label exists + // At 29.97 DF only frames 00-01 are skipped. + let df30 = |m, s, f| { + let mut b = [0u8; 24]; + b[0x0d..0x12].copy_from_slice(&[0, m, s, 1, f]); + rtmd_fields(&b, FrameRate::FPS_29_97) + }; + assert_eq!(df30(1, 0, 0), None); + assert_eq!(df30(1, 0, 2), Some((0, 1, 0, 2, true))); + } } #[cfg(test)] diff --git a/crates/codecs/src/tests.rs b/crates/codecs/src/tests.rs index 3f1f5047..751b18fd 100644 --- a/crates/codecs/src/tests.rs +++ b/crates/codecs/src/tests.rs @@ -387,6 +387,68 @@ fn mkv_vp9_422_seeks_bit_exact() { check_vp9_seeks("noise_vp9_422.mkv", "Matroska", "yuv422p", &["-profile:v", "1", "-deadline", "realtime", "-speed", "8"], 30); } +/// #402: a WebM whose VP9 track carries an alpha layer (`AlphaMode` 1, a second VP9 stream in +/// each block's `BlockAdditional`) decodes with that alpha. Frames requested out of order (seeks +/// across key frames, then forward) match libvpx's decode exactly, colour and alpha. +#[test] +fn vp9_webm_alpha_layer_matches_libvpx() { + let (w, h, frames) = (176usize, 144usize, 30usize); + // the alpha changes every frame, so a picture paired with the wrong alpha frame shows + let Some(path) = fixture_path( + "alpha_vp9.webm", + &[ + "-f", + "lavfi", + "-i", + "testsrc2=s=176x144:r=25,format=yuva420p,geq=lum='lum(X,Y)':cb='cb(X,Y)':cr='cr(X,Y)':a='mod(X*2+N*7\\,256)'", + "-frames:v", + "30", + "-c:v", + "libvpx-vp9", + "-pix_fmt", + "yuva420p", + "-g", + "10", + "-deadline", + "realtime", + "-speed", + "8", + "-b:v", + "600k", + ], + ) else { + return; + }; + // the reference: ffmpeg's libvpx decoder (its native VP9 decoder ignores the alpha layer) + let src_path = path.to_string_lossy().into_owned(); + let Some(reference) = + fixture("alpha_vp9.webm.yuva420p.yuv", &["-c:v", "libvpx-vp9", "-i", &src_path, "-fps_mode", "passthrough", "-f", "rawvideo", "-pix_fmt", "yuva420p"]) + else { + return; + }; + let (luma, chroma) = (w * h, (w / 2) * (h / 2)); + let frame_len = luma * 2 + chroma * 2; + if reference.len() != frame_len * frames { + eprintln!("libvpx reference has {} bytes, not {}; skipping", reference.len(), frame_len * frames); + return; + } + let bytes: Arc<[u8]> = std::fs::read(&path).unwrap().into(); + let src = crate::open_bytes("alpha_vp9.webm", bytes).unwrap(); + let info = src.info().clone(); + assert_eq!(info.container, "WebM"); + assert!(info.video.as_ref().unwrap().has_alpha); + let rate = info.frame_rate(); + let order: Vec = [29, 3, 17, 12, 0, 25, 21, 9].into_iter().chain(10..frames).collect(); + for k in order { + let f = src.video_frame(FrameRequest::full(rate.tick_of(k as i64))).unwrap(); + let r = &reference[k * frame_len..(k + 1) * frame_len]; + let filmcraft_frame::PixelData::Yuv8 { planes, alpha, .. } = &f.data else { panic!("frame {k}: not 8-bit YUV: {}", f.format_label()) }; + assert_eq!(planes[0].as_slice(), &r[..luma], "frame {k}: luma"); + let alpha = alpha.as_ref().unwrap_or_else(|| panic!("frame {k}: no alpha plane")); + assert_eq!(alpha.as_slice(), &r[luma + 2 * chroma..], "frame {k}: alpha"); + } +} + /// Every sample flagged as sync (as in an MP4 without `stss`): the GOP cache must still start /// decoding at a real VP9 key frame. #[test] diff --git a/crates/codecs/src/webm_alpha.rs b/crates/codecs/src/webm_alpha.rs new file mode 100644 index 00000000..e02c905d --- /dev/null +++ b/crates/codecs/src/webm_alpha.rs @@ -0,0 +1,290 @@ +//! WebM alpha (#402): a VP9 track with `AlphaMode` 1 carries, in each block's `BlockAdditional` +//! with `BlockAddID` 1, a second VP9 stream coded independently of the colour one. Its luma plane, +//! full range, is the frame's alpha (the WebM convention, as libvpx-based decoders read it). +//! +//! [`AlphaDecoder`] wraps the colour decoder: every sample it decodes, it also decodes that +//! sample's alpha frame with a second (software) VP9 decoder and attaches the luma plane to the +//! colour picture with the same timestamp, holding a colour picture back until its alpha is out. The alpha stream has key frames of its own, which need +//! not coincide with the colour ones: after a seek (or any other break in the order samples are +//! fed), the alpha decoder restarts from the nearest alpha key frame at or before the sample. +//! +//! Alpha is best effort: a missing, damaged or undecodable alpha frame leaves that picture opaque, +//! never fails the colour decode. + +use std::collections::{HashMap, VecDeque}; +use std::sync::Arc; + +use filmcraft_frame::{PixelData, VideoFrame}; + +use crate::video::{DecodedFrame, VideoDecoder}; + +/// How far back the alpha decoder looks for a key frame when it resynchronises. +const MAX_RESYNC: usize = 600; +/// Decoded alpha pictures waiting for their colour picture, and colour pictures waiting for their +/// alpha (VP9 pictures come out in order, so a few are plenty; past that a picture goes out opaque +/// rather than stall). +const MAX_PENDING: usize = 8; + +/// Where each sample's alpha frame is stored. +pub(crate) struct AlphaLayer { + src: crate::Src, + /// Per sample, in file (decode) order. + additions: Vec>, + pts: Vec, + /// First sample with each pts. + by_pts: HashMap, +} + +impl AlphaLayer { + /// `samples`: (pts, addition) of every sample of the track, in file order. + pub(crate) fn new(src: crate::Src, samples: impl Iterator)>) -> Self { + let (mut additions, mut all_pts) = (Vec::new(), Vec::new()); + let mut by_pts = HashMap::new(); + for (i, (pts, add)) in samples.enumerate() { + additions.push(add); + all_pts.push(pts); + by_pts.entry(pts).or_insert(i); + } + Self { src, additions, pts: all_pts, by_pts } + } + + fn read(&self, i: usize) -> Option> { + let (offset, size) = (*self.additions.get(i)?)?; + let end = offset.checked_add(size as u64)?; + if end > self.src.0.len() { + return None; + } + let mut data = vec![0; size as usize]; + self.src.0.read_at(offset, &mut data).ok()?; + Some(data) + } +} + +/// A colour decoder plus the decoder of its alpha layer. +/// +/// Both decoders may hand pictures back later than the sample that made them (frame threading), +/// and not necessarily after the same call. A colour picture is therefore held until its alpha +/// picture has come out, or until it is certain that it won't: the alpha decoder has moved past +/// it (pictures come out in order), it failed, or the sample has no alpha frame. +pub(crate) struct AlphaDecoder { + color: Box, + alpha: Box, + layer: Arc, + /// Sample the alpha decoder expects next; `None` after a reset or a failure. + next: Option, + /// Decoded alpha pictures not yet paired, by pts. + pending: VecDeque<(i64, VideoFrame)>, + /// Latest pts the alpha decoder has produced since the last reset. + alpha_max: Option, + /// Colour pictures waiting for their alpha, in output order. + held: VecDeque, +} + +impl AlphaDecoder { + pub(crate) fn new(color: Box, alpha: Box, layer: Arc) -> Self { + Self { color, alpha, layer, next: None, pending: VecDeque::new(), alpha_max: None, held: VecDeque::new() } + } + + fn keep(&mut self, frames: Vec) { + for f in frames { + self.alpha_max = Some(self.alpha_max.map_or(f.pts, |m| m.max(f.pts))); + self.pending.push_back((f.pts, f.frame)); + } + while self.pending.len() > MAX_PENDING { + self.pending.pop_front(); + } + } + + /// Decode the alpha frame of the sample with `pts`, first catching up from an alpha key frame + /// when the sample doesn't follow the previous one. + fn feed_alpha(&mut self, pts: i64) { + let Some(&i) = self.layer.by_pts.get(&pts) else { + self.next = None; + return; + }; + if self.next != Some(i) { + self.alpha.reset(); + self.pending.clear(); + self.alpha_max = None; + let is_key = |k: usize| self.layer.read(k).is_some_and(|d| filmcraft_vp9::is_keyframe(&d)); + let start = (i.saturating_sub(MAX_RESYNC)..=i).rev().find(|&k| is_key(k)); + if let Some(start) = start { + for k in start..i { + let (Some(d), Some(&p)) = (self.layer.read(k), self.layer.pts.get(k)) else { continue }; + match self.alpha.decode(&d, p) { + Ok(frames) => self.keep(frames), + Err(_) => break, + } + } + } + } + self.next = Some(i + 1); + let Some(d) = self.layer.read(i) else { return }; + match self.alpha.decode(&d, pts) { + Ok(frames) => self.keep(frames), + Err(e) => { + log::debug!("WebM alpha frame at pts {pts}: {e}"); + self.next = None; + } + } + } + + /// Whether the alpha picture for `pts` can still come out of the alpha decoder. + fn alpha_may_come(&self, pts: i64) -> bool { + let has_frame = self.layer.by_pts.get(&pts).is_some_and(|&i| self.layer.additions.get(i).is_some_and(|a| a.is_some())); + has_frame && self.next.is_some() && self.alpha_max.is_none_or(|m| m < pts) + } + + /// Release held colour pictures, in order, as far as their alpha is settled. `all`: release + /// every one (flush), with or without alpha. + fn release(&mut self, all: bool) -> Vec { + let mut out = Vec::new(); + while let Some(f) = self.held.front() { + let pts = f.pts; + let found = self.pending.iter().position(|(p, _)| *p == pts); + if found.is_none() && !all && self.held.len() <= MAX_PENDING && self.alpha_may_come(pts) { + break; + } + let Some(mut f) = self.held.pop_front() else { break }; + if let Some(k) = found + && let Some((_, a)) = self.pending.remove(k) + { + set_alpha(&mut f.frame, &a); + } + out.push(f); + } + out + } +} + +impl VideoDecoder for AlphaDecoder { + fn decode(&mut self, sample: &[u8], pts: i64) -> crate::Result> { + let out = self.color.decode(sample, pts)?; + self.held.extend(out); + self.feed_alpha(pts); + Ok(self.release(false)) + } + fn flush(&mut self) -> Vec { + let out = self.color.flush(); + self.held.extend(out); + let rest = self.alpha.flush(); + self.keep(rest); + self.release(true) + } + fn reset(&mut self) { + self.color.reset(); + self.alpha.reset(); + self.next = None; + self.pending.clear(); + self.alpha_max = None; + self.held.clear(); + } + fn name(&self) -> &str { + self.color.name() + } + fn intra_only(&self) -> bool { + self.color.intra_only() + } + fn is_random_access(&self, sample: &[u8]) -> Option { + self.color.is_random_access(sample) + } + fn is_disposable(&self, sample: &[u8]) -> bool { + self.color.is_disposable(sample) + } + fn set_draft(&mut self, on: bool) { + self.color.set_draft(on); + self.alpha.set_draft(on); + } + fn thread_limited(&self) -> bool { + self.color.thread_limited() + } +} + +/// The luma plane of `a` (an alpha picture) as `frame`'s alpha, converted to `frame`'s bit depth. +/// Nothing happens when the sizes differ or either picture has no usable planes. +pub(crate) fn set_alpha(frame: &mut VideoFrame, a: &VideoFrame) { + if (frame.width, frame.height) != (a.width, a.height) { + return; + } + let n = frame.width as usize * frame.height as usize; + // alpha codes as 16-bit with their depth, so every pairing converts through one path + let (codes, abits): (Vec, u32) = match &a.data { + PixelData::Yuv8 { planes, .. } => (planes[0].iter().map(|&v| v as u16).collect(), 8), + PixelData::Yuv16 { planes, bits, .. } => (planes[0].to_vec(), (*bits).clamp(1, 16)), + _ => return, + }; + if codes.len() != n { + return; + } + let rescale = |v: u16, to: u32| -> u32 { + let (from_max, to_max) = ((1u32 << abits) - 1, (1u32 << to.clamp(1, 16)) - 1); + (v as u32 * to_max + from_max / 2) / from_max + }; + match &mut frame.data { + PixelData::Yuv8 { alpha, .. } => *alpha = Some(Arc::new(codes.iter().map(|&v| rescale(v, 8) as u8).collect())), + PixelData::Yuv16 { alpha, bits, .. } => { + let to = *bits; + *alpha = Some(Arc::new(codes.iter().map(|&v| rescale(v, to) as u16).collect())); + } + PixelData::Rgba8(rgba) => { + let px = Arc::make_mut(rgba); + for (p, &v) in px.as_chunks_mut::<4>().0.iter_mut().zip(&codes) { + p[3] = rescale(v, 8) as u8; + } + } + PixelData::RgbaF32(_) => {} + } +} + +#[cfg(test)] +mod tests { + use super::*; + use filmcraft_frame::Chroma; + + fn yuv8(w: u32, h: u32, luma: Vec) -> VideoFrame { + let n = (w * h) as usize; + let c = n / 4; + let data = PixelData::Yuv8 { planes: [Arc::new(luma), Arc::new(vec![128; c]), Arc::new(vec![128; c])], chroma: Chroma::C420, alpha: None }; + VideoFrame { width: w, height: h, data, color: Default::default(), par: (1, 1), pts: filmcraft_time::Tick::ZERO } + } + + fn yuv16(w: u32, h: u32, bits: u32) -> VideoFrame { + let n = (w * h) as usize; + let data = + PixelData::Yuv16 { planes: [Arc::new(vec![0; n]), Arc::new(vec![0; n / 4]), Arc::new(vec![0; n / 4])], chroma: Chroma::C420, bits, alpha: None }; + VideoFrame { width: w, height: h, data, color: Default::default(), par: (1, 1), pts: filmcraft_time::Tick::ZERO } + } + + #[test] + fn alpha_luma_becomes_the_alpha_plane_at_the_frame_depth() { + let a = yuv8(2, 2, vec![0, 128, 255, 64]); + // 8-bit picture: the luma as is + let mut f = yuv8(2, 2, vec![9; 4]); + set_alpha(&mut f, &a); + let PixelData::Yuv8 { alpha, .. } = &f.data else { panic!() }; + assert_eq!(alpha.as_deref(), Some(&vec![0, 128, 255, 64])); + // 10-bit picture: full range kept (255 → 1023) + let mut f = yuv16(2, 2, 10); + set_alpha(&mut f, &a); + let PixelData::Yuv16 { alpha, .. } = &f.data else { panic!() }; + assert_eq!(alpha.as_deref(), Some(&vec![0, 514, 1023, 257])); + // RGBA picture (VP9 RGB profiles): the A channel + let mut f = VideoFrame::rgba8(2, 2, vec![7; 16]); + set_alpha(&mut f, &a); + let PixelData::Rgba8(px) = &f.data else { panic!() }; + assert_eq!(px.chunks(4).map(|p| p[3]).collect::>(), [0, 128, 255, 64]); + } + + #[test] + fn an_alpha_picture_of_another_size_is_ignored() { + let mut f = yuv8(4, 2, vec![9; 8]); + set_alpha(&mut f, &yuv8(2, 2, vec![0; 4])); + let PixelData::Yuv8 { alpha, .. } = &f.data else { panic!() }; + assert!(alpha.is_none()); + // a luma plane shorter than the picture (damaged) is ignored too + let mut f = yuv8(2, 2, vec![9; 4]); + set_alpha(&mut f, &yuv8(2, 2, vec![0; 3])); + let PixelData::Yuv8 { alpha, .. } = &f.data else { panic!() }; + assert!(alpha.is_none()); + } +} diff --git a/crates/codecs/tests/mp4_rtmd.rs b/crates/codecs/tests/mp4_rtmd.rs new file mode 100644 index 00000000..349dbb07 --- /dev/null +++ b/crates/codecs/tests/mp4_rtmd.rs @@ -0,0 +1,261 @@ +//! Sony XAVC MP4 with no `tmcd` track: the start timecode lives in an `rtmd` metadata track +//! (issue #460). A hand-built file exercises the whole probe path; the byte layout of the `rtmd` +//! sample is the one real XAVC footage uses (checked against ffprobe's `timecode` tag). + +use std::sync::Arc; + +use filmcraft_time::FrameRate; + +/// Minimal ISO-BMFF box writer (the demuxer only needs headers and sample tables here). +#[derive(Default)] +struct B(Vec); + +impl B { + fn u16(&mut self, v: u16) { + self.0.extend_from_slice(&v.to_be_bytes()); + } + fn u32(&mut self, v: u32) { + self.0.extend_from_slice(&v.to_be_bytes()); + } + fn bytes(&mut self, b: &[u8]) { + self.0.extend_from_slice(b); + } + fn zeros(&mut self, n: usize) { + self.0.resize(self.0.len() + n, 0); + } + /// A container box (`size` + `kind` + children). + fn container(&mut self, kind: &[u8; 4], f: impl FnOnce(&mut B)) { + let at = self.0.len(); + self.u32(0); + self.bytes(kind); + f(self); + let len = (self.0.len() - at) as u32; + self.0[at..at + 4].copy_from_slice(&len.to_be_bytes()); + } + /// A full box (`size` + `kind` + version/flags + body). + fn full(&mut self, kind: &[u8; 4], f: impl FnOnce(&mut B)) { + self.container(kind, |b| { + b.zeros(4); + f(b); + }); + } + fn leaf(&mut self, kind: &[u8; 4], payload: &[u8]) { + self.container(kind, |b| b.bytes(payload)); + } +} + +/// A `stsd` visual sample entry (JPEG keeps the video track decodable enough to open). +fn video_entry(b: &mut B) { + b.bytes(&[0; 6]); + b.u16(1); + b.u16(0); + b.u16(0); + b.bytes(&[0; 4]); + b.u32(0); + b.u32(0); + b.u16(1920); + b.u16(1080); + b.u32(0x0048_0000); + b.u32(0x0048_0000); + b.u32(0); + b.u16(1); + b.bytes(&[0; 32]); + b.u16(24); + b.u16(0xFFFF); +} + +/// A video track at 59.94 fps, plus a Sony `rtmd` track whose first sample is `rtmd_sample`. +fn file_with_rtmd(rtmd_sample: &[u8]) -> Vec { + file_with_rtmd_claimed(rtmd_sample, rtmd_sample.len() as u32) +} + +/// As [`file_with_rtmd`] but the `rtmd` sample's `stsz` size is `claimed` (which may exceed the +/// bytes actually present, as a corrupt file can). +fn file_with_rtmd_claimed(rtmd_sample: &[u8], claimed: u32) -> Vec { + let mut b = B::default(); + b.leaf(b"ftyp", b"isom\x00\x00\x00\x00isom"); + let mdat_start = b.0.len(); + let mdat: Vec = [&[1, 2, 3, 4][..], rtmd_sample].concat(); + b.leaf(b"mdat", &mdat); + let video_off = (mdat_start + 8) as u32; + let rtmd_off = video_off + 4; + b.container(b"moov", |b| { + b.full(b"mvhd", |b| { + b.zeros(8); + b.u32(1000); + b.u32(1001); + b.zeros(80); + }); + // Video track (id 1): one sample of 1001/60000 s → 59.94 fps. + b.container(b"trak", |b| { + b.full(b"tkhd", |b| { + b.zeros(8); + b.u32(1); + b.zeros(4); + b.u32(1001); + b.zeros(52); + b.u32(1920 << 16); + b.u32(1080 << 16); + }); + b.container(b"mdia", |b| { + b.full(b"mdhd", |b| { + b.zeros(8); + b.u32(60000); + b.u32(1001); + b.u16(0x55C4); + b.u16(0); + }); + b.full(b"hdlr", |b| { + b.zeros(4); + b.bytes(b"vide"); + b.zeros(12); + b.bytes(b"Video\0"); + }); + b.container(b"minf", |b| { + b.container(b"stbl", |b| { + b.full(b"stsd", |b| { + b.u32(1); + b.container(b"jpeg", video_entry); + }); + b.full(b"stts", |b| { + b.u32(1); + b.u32(1); + b.u32(1001); + }); + b.full(b"stsc", |b| { + b.u32(1); + b.u32(1); + b.u32(1); + b.u32(1); + }); + b.full(b"stsz", |b| { + b.u32(0); + b.u32(1); + b.u32(4); + }); + b.full(b"stco", |b| { + b.u32(1); + b.u32(video_off); + }); + }); + }); + }); + }); + // Sony rtmd metadata track (id 2). + b.container(b"trak", |b| { + b.full(b"tkhd", |b| { + b.zeros(8); + b.u32(2); + b.zeros(4); + b.u32(1); + b.zeros(52); + b.zeros(8); + }); + b.container(b"mdia", |b| { + b.full(b"mdhd", |b| { + b.zeros(8); + b.u32(1); + b.u32(1); + b.u16(0x55C4); + b.u16(0); + }); + b.full(b"hdlr", |b| { + b.zeros(4); + b.bytes(b"meta"); + b.zeros(12); + b.bytes(b"Timed Metadata\0"); + }); + b.container(b"minf", |b| { + b.container(b"stbl", |b| { + b.full(b"stsd", |b| { + b.u32(1); + b.leaf(b"rtmd", &[0, 0, 0, 0, 0, 0, 0, 1]); + }); + b.full(b"stts", |b| { + b.u32(1); + b.u32(1); + b.u32(1); + }); + b.full(b"stsc", |b| { + b.u32(1); + b.u32(1); + b.u32(1); + b.u32(1); + }); + b.full(b"stsz", |b| { + b.u32(0); + b.u32(1); + b.u32(claimed); + }); + b.full(b"stco", |b| { + b.u32(1); + b.u32(rtmd_off); + }); + }); + }); + }); + }); + }); + b.0 +} + +/// A Sony `rtmd` sample: hours, minutes, seconds at 0x0d–0x0f, a drop-frame flag at 0x10 and the +/// frame number at 0x11, then the start of the SMPTE KLV key. +fn rtmd_sample(h: u8, m: u8, s: u8, f: u8, drop: u8) -> Vec { + let mut v = vec![0u8; 24]; + v[0x0d] = h; + v[0x0e] = m; + v[0x0f] = s; + v[0x10] = drop; + v[0x11] = f; + v[0x12..0x16].copy_from_slice(&[0x06, 0x0e, 0x2b, 0x34]); + v +} + +#[test] +fn rtmd_start_timecode_is_read() { + let file = file_with_rtmd(&rtmd_sample(3, 37, 12, 34, 0)); + let src = filmcraft_codecs::open_bytes("fx3.mp4", Arc::from(file.into_boxed_slice())).unwrap(); + let rate = FrameRate::new(60000, 1001); + let tc = src.info().start_timecode.expect("rtmd start timecode"); + assert_eq!(tc, 781_954); + assert_eq!(filmcraft_time::format_timecode_frames(tc, rate, false), "03:37:12:34"); +} + +#[test] +fn rtmd_without_usable_sample_is_ignored() { + // Too short to carry the timecode block: no start timecode, and no panic. + let file = file_with_rtmd(&[0u8; 6]); + let src = filmcraft_codecs::open_bytes("short.mp4", Arc::from(file.into_boxed_slice())).unwrap(); + assert_eq!(src.info().start_timecode, None); +} + +#[test] +fn huge_claimed_rtmd_sample_does_not_exhaust_memory() { + // The stsz size claims 16 MiB but only the header bytes exist: only the fixed-size header is + // read, so a corrupt size cannot drive a huge allocation during open (issue #460 review). + let file = file_with_rtmd_claimed(&rtmd_sample(1, 2, 3, 4, 0), 16 * 1024 * 1024); + let src = filmcraft_codecs::open_bytes("big.mp4", Arc::from(file.into_boxed_slice())).unwrap(); + assert_eq!(src.info().start_timecode, Some(223_384)); // 01:02:03:04 at 59.94 +} + +#[test] +fn damaged_timecode_fields_are_not_published() { + // Frame 60 at 59.94 fps (valid range is 0–59): unusable metadata, not a fabricated time. + let file = file_with_rtmd(&rtmd_sample(3, 37, 12, 60, 0)); + let src = filmcraft_codecs::open_bytes("bad.mp4", Arc::from(file.into_boxed_slice())).unwrap(); + assert_eq!(src.info().start_timecode, None); +} + +#[test] +fn drop_frame_skipped_labels_are_not_published() { + // 00:01:00;00 at 59.94 fps: frames 00-03 of a non-tenth minute don't exist in drop-frame + // counting, so this metadata is unusable rather than a start time. + let file = file_with_rtmd(&rtmd_sample(0, 1, 0, 0, 1)); + let src = filmcraft_codecs::open_bytes("df_bad.mp4", Arc::from(file.into_boxed_slice())).unwrap(); + assert_eq!(src.info().start_timecode, None); + // The first real label of that minute (frame 04) is accepted. + let file = file_with_rtmd(&rtmd_sample(0, 1, 0, 4, 1)); + let src = filmcraft_codecs::open_bytes("df_ok.mp4", Arc::from(file.into_boxed_slice())).unwrap(); + assert_eq!(src.info().start_timecode, Some(3600)); +} diff --git a/crates/engine/src/clip_ops_tests.rs b/crates/engine/src/clip_ops_tests.rs index 8e90933a..11707e25 100644 --- a/crates/engine/src/clip_ops_tests.rs +++ b/crates/engine/src/clip_ops_tests.rs @@ -957,3 +957,30 @@ fn reset_keeps_keyframes_and_resets_one_parameter() { s.execute("edit.undo", json!({})).unwrap(); assert_eq!(f64_param(&clip(&s, a.id.0), "motion", "rotation"), 45.0); } + +#[test] +fn keyboard_slip_keeps_linked_clips_in_sync_past_the_shorter_handle() { + let mut s = Session::default(); + s.execute("file.newProject", json!({"name": "Slip"})).unwrap(); + s.execute("file.newSequence", json!({"name": "Sequence", "fps": 25, "video": 1, "audio": 1, "width": 16, "height": 16})).unwrap(); + let r = s.execute("file.newOfflineFile", json!({"name": "Media", "seconds": 4, "fps": 25, "video": true, "audio": true})).unwrap(); + let item = r["item"].as_u64().unwrap(); + let fr = s.sequence_rate().tick_of(1); + let r = + s.execute("timeline.place", json!({"item": item, "track": "V1", "audioTrack": "A1", "time": 0, "sourceIn": fr.0 * 20, "duration": fr.0 * 60})).unwrap(); + let (v, a) = (r["clips"][0].as_u64().unwrap(), r["clips"][1].as_u64().unwrap()); + // lengthen only the picture: it keeps 10 frames of tail handle, the sound keeps 20 + s.execute("sequence.linkedSelection", json!({"on": false})).unwrap(); + s.execute("timeline.trim", json!({"clip": v, "edge": "out", "deltaFrames": 10})).unwrap(); + s.execute("sequence.linkedSelection", json!({"on": true})).unwrap(); + s.execute("timeline.select", json!({"clips": [v, a]})).unwrap(); + for _ in 0..3 { + s.execute("timeline.slipLeft5", json!({})).unwrap(); + } + assert_eq!((clip(&s, v).source_in, clip(&s, a).source_in), (Tick(fr.0 * 30), Tick(fr.0 * 30))); + // with linked selection off each selected clip still slips on its own + s.execute("sequence.linkedSelection", json!({"on": false})).unwrap(); + s.execute("timeline.select", json!({"clips": [v, a]})).unwrap(); + s.execute("timeline.slipLeft5", json!({})).unwrap(); + assert_eq!((clip(&s, v).source_in, clip(&s, a).source_in), (Tick(fr.0 * 30), Tick(fr.0 * 35))); +} diff --git a/crates/engine/src/commands.rs b/crates/engine/src/commands.rs index 928c25d1..d1312fb2 100644 --- a/crates/engine/src/commands.rs +++ b/crates/engine/src/commands.rs @@ -349,6 +349,24 @@ fn move_clips(s: &mut Session, p: &Value) -> Result { Ok(json!({"moved": moves.iter().map(|m| m.0.0).collect::>(), "overwritten": overwritten})) } +/// Slip clips together: the media delta of each clip is scaled down by the same factor so that the +/// most constrained clip (media end or start) still moves, and no linked partner slips further. +pub(crate) fn slip_together(q: &mut filmcraft_project::Sequence, ctx: &mut edit::EditCtx<'_>, deltas: &[(ClipId, Tick)]) -> Result<()> { + let mut scale = 1.0_f64; + for (c, d) in deltas { + if d.0 == 0 { + continue; + } + let x = edit::slip(&mut q.clone(), *c, *d, ctx)?; + scale = scale.min(x.0.abs() as f64 / d.0.abs() as f64); + } + for (c, d) in deltas { + let dd = if scale < 1.0 { Tick((d.0 as f64 * scale).round() as i64) } else { *d }; + edit::slip(q, *c, dd, ctx)?; + } + Ok(()) +} + /// Expand a clip selection with linked partners (when linked selection is on). pub fn with_links(s: &Session, clips: &[ClipId]) -> Vec { let Some(seq) = s.active_sequence() else { return clips.to_vec() }; @@ -688,6 +706,20 @@ pub(crate) fn place_item( }) } +/// How long an item runs on the Timeline without In/Out marks: its duration, or for a still (and a +/// generator without a length) Settings ▸ Timeline ▸ Still Image Default Duration. +pub(crate) fn full_duration(s: &Session, pi: &filmcraft_project::ProjectItem) -> Tick { + match &pi.kind { + ItemKind::Media(m) + if matches!(m.info.kind, filmcraft_media::MediaKind::Still) + || (matches!(m.info.kind, filmcraft_media::MediaKind::Synthetic) && m.info.duration.0 <= 0) => + { + s.prefs.timeline.still_duration(s.sequence_rate()) + } + _ => pi.duration(), + } +} + pub(crate) fn source_range(s: &Session) -> Option<(ItemId, TimeRange)> { let item = s.state.source_item?; let pi = s.project.item(item)?; @@ -913,7 +945,7 @@ fn build() -> Vec { Ok(json!({"items": ids.iter().map(|i| i.0).collect::>()})) } ), - cmd!("file.import", "Import…", ["File"], Some("Cmd+I"), r#"{"paths":[str],"bin":binId?,"imageSequence":bool?}"#, always, |s, p| { + cmd!("file.import", "Import…", ["File"], Some("Cmd+I"), r#"{"paths":[str],"bin":binId?,"imageSequence":bool?,"replace":bool?}"#, always, |s, p| { let bin = u64_p(p, "bin").map(filmcraft_project::BinId); let paths: Vec = match p.get("paths").and_then(Value::as_array) { Some(a) => a.iter().filter_map(|v| v.as_str().map(str::to_string)).collect(), @@ -922,6 +954,10 @@ fn build() -> Vec { if paths.is_empty() { return Err(bad("file.import", "need `paths`")); } + if bool_p(p, "replace").unwrap_or(false) { + let name = s.project.name.clone(); + s.execute("file.newProject", json!({ "name": name }))?; + } let mut ids = Vec::new(); let mut errors = Vec::new(); let mut sequences = Vec::new(); @@ -1180,7 +1216,7 @@ fn build() -> Vec { "Media…", ["File", "Export"], None, - r#"{"path":str,"preset":str?,"settings":ExportSettings?,"format":"h264|hevc|prores|dnxhr|apv|mjpeg|mxf-op1a|mxf-opatom|png|tiff|bmp|gif|wav|aiff"?,"width":u32?,"height":u32?,"fps":f64?,"bitrateKbps":u32?,"bitrateMode":"cbr|vbr1Pass|vbr2Pass"?,"hardwareEncoding":"off|auto"?,"scale":f32=1,"audio":bool=true,"quality":0..100,"burnCaptions":bool=false,"alpha":bool=false,"captionSidecar":"srt|vtt"?,"loudnessLufs":f64?,"proresProfile":"proxy|lt|standard|hq"?,"dnxProfile":"lb|sq|hq|hqx"?,"apvProfile":"422-10|422-12|444-10|444-12"?,"mxfVideoCodec":"dnxhr|proRes|h264"?,"sequence":id?,"range":"entire|inOut|workArea|custom"?,"startSeconds":f64?,"endSeconds":f64?,"wait":bool=false}"#, + r#"{"path":str,"preset":str?,"settings":ExportSettings?,"format":"h264|hevc|prores|dnxhr|apv|mjpeg|mxf-op1a|mxf-opatom|png|tiff|bmp|gif|wav|aiff"?,"width":u32?,"height":u32?,"fps":f64?,"bitrateKbps":u32?,"bitrateMode":"cbr|vbr1Pass|vbr2Pass|crf"?,"crf":0..51=23,"hardwareEncoding":"off|auto"?,"scale":f32=1,"audio":bool=true,"quality":0..100,"burnCaptions":bool=false,"alpha":bool=false,"captionSidecar":"srt|vtt"?,"loudnessLufs":f64?,"proresProfile":"proxy|lt|standard|hq"?,"dnxProfile":"lb|sq|hq|hqx"?,"apvProfile":"422-10|422-12|444-10|444-12"?,"mxfVideoCodec":"dnxhr|proRes|h264"?,"sequence":id?,"range":"entire|inOut|workArea|custom"?,"startSeconds":f64?,"endSeconds":f64?,"wait":bool=false}"#, has_seq, crate::export_tools::export_media ), @@ -1221,7 +1257,7 @@ fn build() -> Vec { let caps = s.state.caption_selection.clone(); return crate::captions::delete(s, &caps, false); } - let sel = with_links(s, &clips_p(s, p)); + let sel = if p.get("clips").is_some() || p.get("clip").is_some() { with_links(s, &clips_p(s, p)) } else { s.state.selection.clone() }; s.edit_sequence("Clear", |q, _, st| { edit::delete_items(q, &sel); st.selection.clear(); @@ -1234,7 +1270,7 @@ fn build() -> Vec { let caps = s.state.caption_selection.clone(); return crate::captions::delete(s, &caps, true); } - let sel = with_links(s, &clips_p(s, p)); + let sel = if p.get("clips").is_some() || p.get("clip").is_some() { with_links(s, &clips_p(s, p)) } else { s.state.selection.clone() }; s.edit_sequence("Ripple Delete", |q, _, st| { let spans = edit::ripple_delete_items(q, &sel)?; if st.ripple_sequence_markers { @@ -2115,16 +2151,7 @@ fn build() -> Vec { if p.get("audio").is_some() && audio && pi.has_audio() && a.is_none() { return Err(bad("timeline.place", "no audio destination: add or enable an audio track")); } - let full = match &pi.kind { - // Settings ▸ Timeline ▸ Still Image Default Duration - ItemKind::Media(m) - if matches!(m.info.kind, filmcraft_media::MediaKind::Still) - || (matches!(m.info.kind, filmcraft_media::MediaKind::Synthetic) && m.info.duration.0 <= 0) => - { - s.prefs.timeline.still_duration(s.sequence_rate()) - } - _ => pi.duration(), - }; + let full = full_duration(s, pi); let (mi, mo) = match &pi.kind { ItemKind::Media(m) => (m.mark_in, m.mark_out.map(|o| o + pi.frame_rate().frame_duration())), ItemKind::Subclip { range, .. } => (Some(range.start), Some(range.end())), @@ -2136,13 +2163,13 @@ fn build() -> Vec { Ok(json!({"clips": ids.iter().map(|c| c.0).collect::>()})) } ), - cmd!("timeline.select", "Select Clips", [], None, r#"{"clips":[id],"transitions":[id]?,"add":bool,"toggle":bool}"#, has_seq, |s, p| { + cmd!("timeline.select", "Select Clips", [], None, r#"{"clips":[id],"transitions":[id]?,"add":bool,"toggle":bool,"linked":bool?}"#, has_seq, |s, p| { if let Some(ids) = p.get("transitions") { return select_transitions(s, ids, bool_p(p, "add").unwrap_or(false), bool_p(p, "toggle").unwrap_or(false)); } let clips: Vec = p.get("clips").and_then(Value::as_array).map(|a| a.iter().filter_map(|v| v.as_u64().map(ClipId)).collect()).unwrap_or_default(); - let clips = with_links(s, &clips); + let clips = if bool_p(p, "linked").unwrap_or(true) { with_links(s, &clips) } else { clips }; // selecting clips leaves trim mode (Premiere: clip and edit point selections are exclusive) s.state.edit_points.clear(); s.state.trim_shift = Default::default(); @@ -2326,17 +2353,8 @@ fn build() -> Vec { let clips = with_links(s, &[c]); s.edit_sequence("Slip", |q, ctx, _| { // clamp across all linked partners, then slip them by the common delta - let mut dd = d; - for c in &clips { - let x = edit::slip(&mut q.clone(), *c, dd, ctx)?; - if x.abs() < dd.abs() { - dd = x; - } - } - for c in &clips { - edit::slip(q, *c, dd, ctx)?; - } - Ok(()) + let deltas: Vec<(ClipId, Tick)> = clips.iter().map(|c| (*c, d)).collect(); + slip_together(q, ctx, &deltas) })?; Ok(Value::Null) }), @@ -2571,15 +2589,34 @@ fn build() -> Vec { { e.params.insert(pid.clone(), filmcraft_project::Param::new(d.default.clone())); } - let prm = crate::masks::target_param(e, &pq, &pid).ok_or_else(|| bad("effects.setParam", format!("no param `{pid}`")))?; - let v = json_to_param(&prm.value, &val).ok_or_else(|| bad("effects.setParam", "value has the wrong type"))?; - if keyframe { - // the stopwatch and the value in one step - prm.put_keyframe(mt, v); - } else { - prm.set_at(mt, v); + let (keyframes, v) = { + let prm = crate::masks::target_param(e, &pq, &pid).ok_or_else(|| bad("effects.setParam", format!("no param `{pid}`")))?; + let v = json_to_param(&prm.value, &val).ok_or_else(|| bad("effects.setParam", "value has the wrong type"))?; + let keyframes = if keyframe { + // the stopwatch and the value in one step + prm.put_keyframe(mt, v.clone()); + prm.keyframes.len() + } else { + prm.set_at(mt, v.clone()); + prm.keyframes.len() + }; + (keyframes, v) + }; + if e.effect == "ultra_key" + && pid == "setting" + && let ParamValue::Choice(setting) = v + && let Some(rows) = filmcraft_project::effect::ultra_key_setting(setting) + { + for (id, value) in rows { + if let Some(p) = e.param_mut(id) { + p.value = ParamValue::Float(*value); + p.keyframes.clear(); + } else { + e.params.insert((*id).to_string(), filmcraft_project::Param::new(ParamValue::Float(*value))); + } + } } - Ok(prm.keyframes.len()) + Ok(keyframes) })?; // 0 keyframes: the value is static (`time` was not used) Ok(json!({"keyframes": keyframes})) @@ -2854,6 +2891,7 @@ fn build() -> Vec { v.extend(crate::keyboard::commands()); v.extend(crate::project_panel::commands()); v.extend(crate::media_browser::commands()); + v.extend(crate::paste_media::commands()); // Edit ▸ Label ▸ , Paste Attributes, subclips, Video / Audio Options, Replace With Clip… // and their menu order crate::clip_ops::apply_layout(&mut v); diff --git a/crates/engine/src/export_tests.rs b/crates/engine/src/export_tests.rs index 2757ebba..dec8a8fa 100644 --- a/crates/engine/src/export_tests.rs +++ b/crates/engine/src/export_tests.rs @@ -64,6 +64,21 @@ fn export_integer_parameters_cannot_wrap_or_overflow() { assert!(s.execute("export.resolve", json!({"bitrateKbps":8000.0, "keyframeDistance":48.0})).is_ok(), "integer-valued floats are integers"); } +#[test] +fn wav_resolve_honours_explicit_sample_rates() { + let mut s = demo(); + for rate in [4000, 192_000, 192_001, 200_000, 352_800, 384_000] { + let result = s.execute("export.resolve", json!({"format":"wav", "settings":{"audio":{"sample_rate":rate}}})).unwrap(); + assert_eq!(result["settings"]["audio"]["sample_rate"], rate); + assert_eq!(result["output"]["sampleRate"], rate); + } + let result = s.execute("export.resolve", json!({"format":"wav", "settings":{"audio":{"sample_rate":null}}})).unwrap(); + assert_eq!(result["output"]["sampleRate"], 48_000); + for rate in [0, 384_001] { + assert!(s.execute("export.resolve", json!({"format":"wav", "settings":{"audio":{"sample_rate":rate}}})).is_err()); + } +} + #[test] fn nested_camel_case_settings_are_honoured() { let mut s = demo(); @@ -481,6 +496,20 @@ fn hardware_encoding_is_off_unless_asked_for() { assert_eq!(old.hardware_encoding, Off); } +#[test] +fn crf_param_picks_constant_quality() { + use filmcraft_export::BitrateMode::{Cbr, Crf, Vbr1Pass}; + let s = demo(); + let setting = |p: Value| crate::export_tools::settings_from_params(&s, &p, "file.exportMedia").map(|(_, st)| (st.bitrate_mode, st.crf)); + assert_eq!(setting(json!({"path": "x.mp4"})).unwrap(), (Vbr1Pass, 23.0)); + // a factor alone means CRF mode; the mode can also be named, or another one kept explicitly + assert_eq!(setting(json!({"path": "x.mp4", "crf": 18})).unwrap(), (Crf, 18.0)); + assert_eq!(setting(json!({"path": "x.mp4", "bitrateMode": "crf"})).unwrap(), (Crf, 23.0)); + assert_eq!(setting(json!({"path": "x.mp4", "bitrateMode": "cbr", "crf": 18})).unwrap(), (Cbr, 18.0)); + let e = setting(json!({"path": "x.mp4", "bitrateMode": "abr"})).unwrap_err().to_string(); + assert!(e.contains("crf"), "{e}"); +} + #[test] fn queue_exports_several_sequences_and_ranges() { let mut s = demo(); diff --git a/crates/engine/src/export_tools.rs b/crates/engine/src/export_tools.rs index 12e629e8..5c4d875b 100644 --- a/crates/engine/src/export_tools.rs +++ b/crates/engine/src/export_tools.rs @@ -26,7 +26,7 @@ use std::sync::Arc; use std::sync::atomic::Ordering; use filmcraft_export::presets::{DEFAULT_PRESET, preset_key}; -use filmcraft_export::{ExportPreset, ExportSettings, Format, GpuRendering, HardwareEncoding}; +use filmcraft_export::{BitrateMode, ExportPreset, ExportSettings, Format, GpuRendering, HardwareEncoding}; use filmcraft_project::{ItemId, Project}; use filmcraft_time::{FrameRate, Tick, TimeRange}; use serde::{Deserialize, Serialize}; @@ -213,7 +213,13 @@ pub fn settings_from_params(s: &Session, p: &Value, cmd: &str) -> Result<(Option settings.max_bitrate_kbps = Some(v); } if let Some(v) = str_p(p, "bitrateMode") { - settings.bitrate_mode = serde_json::from_value(json!(v)).map_err(|_| bad(cmd, "bitrateMode: cbr | vbr1Pass | vbr2Pass"))?; + settings.bitrate_mode = serde_json::from_value(json!(v)).map_err(|_| bad(cmd, "bitrateMode: cbr | vbr1Pass | vbr2Pass | crf"))?; + } + if let Some(v) = f64_p(p, "crf") { + settings.crf = v as f32; + if p.get("bitrateMode").is_none() { + settings.bitrate_mode = BitrateMode::Crf; + } } if let Some(v) = crate::commands::checked_u32_p(p, "keyframeDistance", cmd)? { settings.keyframe_distance = Some(v); diff --git a/crates/engine/src/image_sequence_tests.rs b/crates/engine/src/image_sequence_tests.rs index ca36d9cc..8d028fe7 100644 --- a/crates/engine/src/image_sequence_tests.rs +++ b/crates/engine/src/image_sequence_tests.rs @@ -11,6 +11,24 @@ use serde_json::{Value, json}; use crate::Session; use crate::media_test_util::tmp_dir; +#[test] +fn import_replace_starts_from_an_empty_project() { + let dir = tmp_dir("import-replace"); + let wav = dir.join("tone.wav"); + std::fs::write(&wav, filmcraft_media::wav::write_wav16(&[0.1; 4800], 2, 48_000)).unwrap(); + let mut s = demo(); + let demo_items = s.project.items.len(); + assert!(demo_items > 1); + assert!(!s.state.open_sequences.is_empty()); + s.execute("file.import", json!({"paths": [wav.to_string_lossy()], "replace": true})).unwrap(); + assert!(s.state.open_sequences.is_empty()); + assert_eq!(s.project.items.len(), 1); + assert!(s.project.items.values().any(|it| it.name == "tone.wav")); + let mut merge = demo(); + merge.execute("file.import", json!({"paths": [wav.to_string_lossy()]})).unwrap(); + assert!(merge.project.items.len() > demo_items); +} + fn demo() -> Session { let mut s = Session::default(); s.execute("file.openDemoProject", json!({})).unwrap(); diff --git a/crates/engine/src/interchange.rs b/crates/engine/src/interchange.rs index 84acdf5e..bd37f6bb 100644 --- a/crates/engine/src/interchange.rs +++ b/crates/engine/src/interchange.rs @@ -59,7 +59,8 @@ pub fn import(s: &mut Session, path: &str, bytes: &[u8], format: Format) -> Resu proj.resolve_placed_auto_points(|seq| !before.contains(&seq), |source| !is_file_media(source)); Ok(seqs) })?; - // Link media: probe each new file-backed item that exists. + // Link media: probe each new file-backed item that exists, through the host's reader as a + // direct import does (only the container index is read now, not the whole file: #461). let new_media: Vec<(ItemId, String)> = s .project .items @@ -76,11 +77,7 @@ pub fn import(s: &mut Session, path: &str, bytes: &[u8], format: Format) -> Resu let mut linked = 0; let mut offline = Vec::new(); for (id, mpath) in new_media { - let opened = s.services.read_file(&mpath).ok().and_then(|b| { - let fname = std::path::Path::new(&mpath).file_name().map(|n| n.to_string_lossy().to_string()).unwrap_or_default(); - s.media.open_bytes(&fname, b.into()).ok() - }); - match opened { + match s.media.open_file(&mpath, &*s.services).ok() { Some(src) => { let info = src.info().clone(); let identity = crate::relink::identity_of(&*s.services, &mpath).ok(); diff --git a/crates/engine/src/interchange_auto_points_tests.rs b/crates/engine/src/interchange_auto_points_tests.rs index d5d5e625..b5a94ab0 100644 --- a/crates/engine/src/interchange_auto_points_tests.rs +++ b/crates/engine/src/interchange_auto_points_tests.rs @@ -1,6 +1,6 @@ //! Interchange imports and the "auto" (NaN) point defaults of the effects they add: the points //! are resolved as placing a clip resolves them, the anchor once the media's real size is known, -//! and a project saved after an import opens again. +//! and a project saved after an import opens again. Also how the media they name are linked. use filmcraft_geom::Vec2; use filmcraft_media::DemoScene; @@ -216,3 +216,43 @@ fn relinking_later_resolves_the_anchor_that_waited_for_the_real_size() { assert_eq!(motion_point(&first_clip(&t, seq), "anchor"), Vec2::new(160.0, 90.0)); let _ = std::fs::remove_dir_all(&d); } + +/// Linking the media an interchange document names opens each file through the host's reader, as +/// a direct import does, instead of reading it whole into memory (#461: minutes for 4K files). +#[cfg(any(unix, windows))] +#[test] +fn an_interchange_import_links_media_without_reading_the_files_whole() { + /// The desktop filesystem, except that reading a media file whole fails. + struct NoWholeMediaReads; + impl crate::Services for NoWholeMediaReads { + fn read_file(&self, path: &str) -> std::io::Result> { + if path.ends_with(".mov") { + return Err(std::io::Error::other(format!("{path}: read whole"))); + } + crate::FsServices.read_file(path) + } + fn write_file(&self, path: &str, data: &[u8]) -> std::io::Result<()> { + crate::FsServices.write_file(path, data) + } + fn file_size(&self, path: &str) -> std::io::Result { + crate::FsServices.file_size(path) + } + fn read_range(&self, path: &str, offset: u64, len: usize) -> std::io::Result> { + crate::FsServices.read_range(path, offset, len) + } + fn reader(&self, path: &str) -> Option> { + crate::FsServices.reader(path) + } + } + let d = tmp_dir("ix-streamed-link"); + std::fs::create_dir_all(d.join("media")).unwrap(); + make_movie(&d.join("media/shot.mov"), DemoScene::OceanSunset, 320, 180, 24); + let doc = d.join("cut.xml"); + std::fs::write(&doc, xml_with_a_scale_only_motion_filter()).unwrap(); + let mut s = Session::new(std::sync::Arc::new(NoWholeMediaReads)); + let (r, seq) = import(&mut s, &doc); + assert_eq!(r["linkedMedia"], 1, "{r}"); + let clip = first_clip(&s, seq); + assert_eq!(s.project.source_size(clip.item), Some((320, 180)), "the file's real size, from its index"); + let _ = std::fs::remove_dir_all(&d); +} diff --git a/crates/engine/src/keyboard.rs b/crates/engine/src/keyboard.rs index d8487251..ad5f5d2a 100644 --- a/crates/engine/src/keyboard.rs +++ b/crates/engine/src/keyboard.rs @@ -40,7 +40,7 @@ use filmcraft_project::{ClipId, ItemKind, ParamValue, Sequence, TrackId, TrackKi use filmcraft_time::Tick; use serde_json::{Value, json}; -use crate::commands::{CommandSpec, always, bad, has_selection, has_seq, item_p, time_p, with_links}; +use crate::commands::{CommandSpec, always, bad, has_selection, has_seq, item_p, slip_together, time_p, with_links}; use crate::{EngineError, Result, Session}; type Run = fn(&mut Session, &Value) -> Result; @@ -248,14 +248,21 @@ fn slip_selection(s: &mut Session, frames: i64) -> Result { let rate = s.sequence_rate(); let clips = with_links(s, &s.state.selection.clone()); let seq = s.active_sequence().ok_or(EngineError::NoSequence)?; - // media delta per clip (source frames run at the clip's speed) - let deltas: Vec<(ClipId, Tick)> = clips - .iter() - .filter_map(|c| seq.find_item(*c).map(|(_, i)| (*c, Tick((rate.tick_of(frames).0 as f64 * i.speed.abs().max(1e-9)).round() as i64)))) - .collect(); + let linked = s.state.linked_selection; + // media delta per clip (source frames run at the clip's speed); linked partners slip as one group + let mut groups: Vec<(Option, Vec<(ClipId, Tick)>)> = Vec::new(); + for c in &clips { + let Some((_, i)) = seq.find_item(*c) else { continue }; + let d = Tick((rate.tick_of(frames).0 as f64 * i.speed.abs().max(1e-9)).round() as i64); + let link = i.link.filter(|_| linked); + match groups.iter_mut().find(|(l, _)| link.is_some() && *l == link) { + Some((_, g)) => g.push((*c, d)), + None => groups.push((link, vec![(*c, d)])), + } + } s.edit_sequence("Slip", |q, ctx, _| { - for (c, d) in &deltas { - edit::slip(q, *c, *d, ctx)?; + for (_, g) in &groups { + slip_together(q, ctx, g)?; } Ok(()) })?; diff --git a/crates/engine/src/lib.rs b/crates/engine/src/lib.rs index eea55b32..fe6f94be 100644 --- a/crates/engine/src/lib.rs +++ b/crates/engine/src/lib.rs @@ -31,6 +31,7 @@ pub mod mixer; pub mod multicam; pub mod narration; pub mod panels; +pub mod paste_media; pub mod perf; pub mod presets; pub mod previews; @@ -1088,6 +1089,8 @@ mod panels_tests; #[cfg(test)] mod par_tests; #[cfg(test)] +mod paste_media_tests; +#[cfg(test)] mod presets_tests; #[cfg(test)] mod previews_tests; diff --git a/crates/engine/src/media_browser.rs b/crates/engine/src/media_browser.rs index 515a10d4..b5a4d5da 100644 --- a/crates/engine/src/media_browser.rs +++ b/crates/engine/src/media_browser.rs @@ -12,7 +12,7 @@ //! | `mediaBrowser.favorite` | Add to / Remove from Favorites | //! | `mediaBrowser.clearRecent` | Clear Recent Directories | //! | `mediaBrowser.settings` | file types, view (list / thumbnails), columns (Edit Columns…), Import as Image Sequence, Hover Scrub, thumbnail size | -//! | `mediaBrowser.import` | Import (the selection or `paths`; Project ▸ Ingest settings apply) | +//! | `mediaBrowser.import` | Import (the selection or `paths`; a folder becomes a bin, its sub-folders bins inside it; Project ▸ Ingest settings apply) | //! | `mediaBrowser.openInSource` | Open In Source Monitor (Shift+O) | //! | `mediaBrowser.probe` | a file's media properties (frame rate, duration, video / audio info), cached | //! @@ -421,32 +421,134 @@ fn paths_p(s: &Session, p: &Value) -> Vec { } } -/// `mediaBrowser.import {paths?, bin?, imageSequence?}`: imports files (directories import their -/// supported files, like dragging a folder in Premiere). +/// Most folders one import walks, and how deep it goes: a folder import mirrors the tree as bins, +/// and symlinked folders can loop. +const FOLDER_IMPORT_MAX_FOLDERS: usize = 4096; +const FOLDER_IMPORT_MAX_DEPTH: usize = 32; + +/// A folder to import as a bin: its name, its files and its sub-folders that hold files. +struct Folder { + name: String, + files: Vec, + subs: Vec, +} + +/// The files the browser shows in `dir` and its sub-folders, as a tree; `None` when the tree +/// holds none (or `budget` folders were walked already). +fn walk_folder(services: &dyn Services, dir: &str, depth: usize, budget: &mut usize) -> Option { + *budget = budget.checked_sub(1)?; + let entries = list(services, dir, "all").ok()?; + let mut folder = Folder { name: base_name(dir), files: Vec::new(), subs: Vec::new() }; + for e in entries { + if !e.is_dir { + folder.files.push(e.path); + } else if depth < FOLDER_IMPORT_MAX_DEPTH + && let Some(sub) = walk_folder(services, &e.path, depth + 1, budget) + { + folder.subs.push(sub); + } + } + (!folder.files.is_empty() || !folder.subs.is_empty()).then_some(folder) +} + +/// Add a `file.import` result to the merged one: arrays add up, other values (the Ingest result) +/// are listed. +fn merge_import(out: &mut serde_json::Map, r: Value) { + let Value::Object(r) = r else { return }; + for (k, v) in r { + let slot = out.entry(k).or_insert_with(|| json!([])); + let Value::Array(all) = slot else { continue }; + match v { + Value::Null => {} + Value::Array(a) => all.extend(a), + other => all.push(other), + } + } +} + +/// Run `file.import` and merge its result; a failure becomes an error of the merged result. +fn import_into(s: &mut Session, files: &[String], bin: Option, out: &mut serde_json::Map) { + let mut q = json!({"paths": files, "imageSequence": false}); + if let Some(b) = bin { + q["bin"] = json!(b); + } + match s.execute("file.import", q) { + Ok(r) => merge_import(out, r), + Err(e) => merge_import(out, json!({"errors": [e.to_string()]})), + } +} + +/// Import a folder as a bin named after it, in `parent`: its files go into the bin, each +/// sub-folder becomes a bin inside it (Premiere's Media Browser keeps the folder structure). +fn import_folder(s: &mut Session, folder: &Folder, parent: Option, out: &mut serde_json::Map) -> Result<()> { + let name = folder.name.clone(); + let bin = s.edit("New Bin", |pr, _| Ok(pr.add_bin(&name, parent.map(filmcraft_project::BinId))))?; + merge_import(out, json!({"bins": [bin.0]})); + if !folder.files.is_empty() { + import_into(s, &folder.files, Some(bin.0), out); + } + // recursion as deep as the walk went, which is bounded + for sub in &folder.subs { + import_folder(s, sub, Some(bin.0), out)?; + } + Ok(()) +} + +/// `mediaBrowser.import {paths?, bin?, imageSequence?}`: imports files. A folder becomes a bin +/// named after it (in `bin`) holding its files, and each sub-folder a bin inside that, as in +/// Premiere; the result then also lists the new `bins`. Import as Image Sequence takes the first +/// file instead. fn import(s: &mut Session, p: &Value) -> Result { let paths = paths_p(s, p); if paths.is_empty() { return Err(bad("mediaBrowser.import", "select files in the Media Browser")); } let seq = bool_p(p, "imageSequence").unwrap_or(s.prefs.media_browser.import_as_image_sequence); + let bin = u64_p(p, "bin"); let mut files = Vec::new(); + let mut folders = Vec::new(); + let mut empty = Vec::new(); + let mut budget = FOLDER_IMPORT_MAX_FOLDERS; for path in paths { - // a directory imports its media files (not recursive) let is_dir = matches!(s.services.list_entries(&path), Some(Ok(_))) && kind_of(&path).is_none(); - match list(&*s.services, &path, "all") { - Ok(entries) if is_dir => files.extend(entries.into_iter().filter(|e| !e.is_dir).map(|e| e.path)), - _ => files.push(path), + if !is_dir { + files.push(path); + } else if seq { + // an image sequence: the folder's frames + match list(&*s.services, &path, "all") { + Ok(entries) => files.extend(entries.into_iter().filter(|e| !e.is_dir).map(|e| e.path)), + Err(_) => files.push(path), + } + } else { + match walk_folder(&*s.services, &path, 0, &mut budget) { + Some(f) => folders.push(f), + None => empty.push(path), + } } } // an image sequence imports from its first selected frame only if seq { files.truncate(1); } - let mut q = json!({"paths": files, "imageSequence": seq}); - if let Some(b) = u64_p(p, "bin") { - q["bin"] = json!(b); + if folders.is_empty() { + if files.is_empty() && !empty.is_empty() { + return Err(bad("mediaBrowser.import", format!("no media files in {}", empty.join(", ")))); + } + let mut q = json!({"paths": files, "imageSequence": seq}); + if let Some(b) = bin { + q["bin"] = json!(b); + } + return s.execute("file.import", q); } - s.execute("file.import", q) + let mut out = serde_json::Map::new(); + merge_import(&mut out, json!({"items": [], "errors": empty.iter().map(|e| format!("{e}: no media files")).collect::>(), "bins": []})); + if !files.is_empty() { + import_into(s, &files, bin, &mut out); + } + for f in &folders { + import_folder(s, f, bin, &mut out)?; + } + Ok(Value::Object(out)) } /// The project item that already references a file. diff --git a/crates/engine/src/media_browser_tests.rs b/crates/engine/src/media_browser_tests.rs index 972220b3..aa1b5938 100644 --- a/crates/engine/src/media_browser_tests.rs +++ b/crates/engine/src/media_browser_tests.rs @@ -210,6 +210,47 @@ fn import_the_selection_and_open_in_source() { assert_eq!(r["items"].as_array().unwrap().len(), 1); } +/// Importing a folder keeps its structure (#411): the folder becomes a bin named after it, each +/// sub-folder with media a bin inside that, and every file lands in its folder's bin. +#[test] +fn importing_a_folder_mirrors_its_sub_folders_as_bins() { + let mut fs = FakeFs::new(); + let wav = filmcraft_media::wav::write_wav16(&[0.0; 4800], 2, 48_000); + fs.dirs.extend(["/Shoot", "/Shoot/Day 1", "/Shoot/Day 1/Cam A", "/Shoot/Empty"].map(String::from)); + for f in ["/Shoot/slate.wav", "/Shoot/Day 1/room.wav", "/Shoot/Day 1/Cam A/take1.wav", "/Shoot/Day 1/Cam A/take2.wav"] { + fs.files.get_mut().unwrap().insert(f.into(), wav.clone()); + } + let mut s = Session::new(Arc::new(fs)); + let r = s.execute("mediaBrowser.import", json!({"paths": ["/Shoot"]})).unwrap(); + assert_eq!(r["items"].as_array().unwrap().len(), 4, "{r}"); + assert_eq!(r["bins"].as_array().unwrap().len(), 3, "{r}"); + let tree = |b: &filmcraft_project::Bin| -> Vec { + b.children + .iter() + .map(|c| match c { + filmcraft_project::BinEntry::Bin(b) => format!("bin {}", b.name), + filmcraft_project::BinEntry::Item(i) => s.project.item(*i).unwrap().name.clone(), + }) + .collect() + }; + let sub = |b: &filmcraft_project::Bin, name: &str| -> filmcraft_project::Bin { + b.children + .iter() + .find_map(|c| match c { + filmcraft_project::BinEntry::Bin(b) if b.name == name => Some(b.clone()), + _ => None, + }) + .unwrap() + }; + assert_eq!(tree(&s.project.root), ["bin Shoot"]); + let shoot = sub(&s.project.root, "Shoot"); + // a folder without media gets no bin + assert_eq!(tree(&shoot), ["slate.wav", "bin Day 1"]); + let day = sub(&shoot, "Day 1"); + assert_eq!(tree(&day), ["room.wav", "bin Cam A"]); + assert_eq!(tree(&sub(&day, "Cam A")), ["take1.wav", "take2.wav"]); +} + #[test] fn import_with_ingest_settings() { let mut s = session(); diff --git a/crates/engine/src/paste_media.rs b/crates/engine/src/paste_media.rs new file mode 100644 index 00000000..10ff7eed --- /dev/null +++ b/crates/engine/src/paste_media.rs @@ -0,0 +1,123 @@ +//! Paste media from the system clipboard (#611). The desktop app reads the clipboard: copied files +//! paste by path, and an image is first saved by [`save_pasted_image`] as `Pasted Image .png` +//! where voice-over takes and narrations go (Scratch Disks ▸ Captured Audio and Video, else next +//! to the project). +//! +//! `edit.pasteMedia` imports the files and puts them on the Timeline at the playhead, end to end, +//! without overwriting anything: each clip goes on the first free track from the targeted one +//! outward (up for video, down for audio), and a track is added when every track is busy there. +//! With `insert` they are inserted on the targeted tracks instead, as Paste Insert does. + +use filmcraft_project::{BinId, ClipId, ItemId, TrackId, TrackKind}; +use filmcraft_time::Tick; +use serde_json::{Value, json}; + +use crate::commands::{CommandSpec, always, bad, bool_p, u64_p}; +use crate::{EngineError, Result, Session}; + +/// Write a pasted image (PNG bytes) to a new `Pasted Image .png`; returns its path. +pub fn save_pasted_image(s: &mut Session, png: &[u8]) -> Result { + let dir = crate::voiceover::media_dir(s, &Value::Null, "Pasted Images"); + let names: std::collections::HashSet = s.project.items.values().map(|i| i.name.clone()).collect(); + // bounded like voiceover::unique_wav_path: `file_size` is a filesystem call per try + let path = (1..=99_999u32) + .map(|k| format!("Pasted Image {k}.png")) + .map(|file| (std::path::Path::new(&dir).join(&file).to_string_lossy().into_owned(), file)) + .find(|(path, file)| !names.contains(file) && s.services.file_size(path).is_err()) + .map(|(path, _)| path) + .ok_or_else(|| EngineError::Other(format!("{dir}: no free file name for \"Pasted Image .png\"")))?; + if !cfg!(target_arch = "wasm32") { + std::fs::create_dir_all(&dir).map_err(|e| EngineError::Other(format!("{dir}: {e}")))?; + } + s.services.write_file(&path, png).map_err(|e| EngineError::Other(format!("{path}: {e}")))?; + Ok(path) +} + +fn paste_media(s: &mut Session, p: &Value) -> Result { + let cmd = "edit.pasteMedia"; + let paths: Vec = + p.get("paths").and_then(Value::as_array).map(|a| a.iter().filter_map(|v| v.as_str().map(str::to_string)).collect()).unwrap_or_default(); + if paths.is_empty() { + return Err(bad(cmd, "need `paths`")); + } + let place = bool_p(p, "place").unwrap_or(true); + if place && s.active_sequence().is_none() { + return Err(EngineError::NoSequence); + } + let insert = bool_p(p, "insert").unwrap_or(false); + let n0 = s.history.undo.len(); + let r = import_and_place(s, &paths, u64_p(p, "bin").map(BinId), place, insert); + crate::clip_ops::collapse_history(s, n0, if insert { "Paste Insert" } else { "Paste" }); + r +} + +fn import_and_place(s: &mut Session, paths: &[String], bin: Option, place: bool, insert: bool) -> Result { + let mut import = json!({"paths": paths}); + if let Some(b) = bin { + import["bin"] = json!(b.0); + } + let imported = s.execute("file.import", import)?; + let items: Vec = imported["items"].as_array().map(|a| a.iter().filter_map(Value::as_u64).map(ItemId).collect()).unwrap_or_default(); + if !place { + return Ok(json!({"items": items.iter().map(|i| i.0).collect::>(), "clips": []})); + } + let mut at = s.playhead(); + let mut clips: Vec = Vec::new(); + for item in &items { + let Some(pi) = s.project.item(*item) else { continue }; + let (dur, video, audio) = (crate::commands::full_duration(s, pi), pi.has_video(), pi.has_audio()); + let mut params = json!({"item": item.0, "time": at.0, "insert": insert}); + if !insert { + if video { + params["track"] = json!(free_track(s, TrackKind::Video, at, at + dur)?.0); + } + if audio { + params["audioTrack"] = json!(free_track(s, TrackKind::Audio, at, at + dur)?.0); + } + } + let placed = s.execute("timeline.place", params)?; + let ids: Vec = placed["clips"].as_array().map(|a| a.iter().filter_map(Value::as_u64).map(ClipId).collect()).unwrap_or_default(); + let seq = s.active_sequence().ok_or(EngineError::NoSequence)?; + at = ids.iter().filter_map(|c| seq.find_item(*c)).map(|(_, it)| it.end()).max().unwrap_or(at + dur); + clips.extend(ids); + } + // the playhead goes to the end of what was pasted, as after Paste + s.set_playhead(at); + s.state.selection = clips.clone(); + Ok(json!({"items": items.iter().map(|i| i.0).collect::>(), "clips": clips.iter().map(|c| c.0).collect::>()})) +} + +/// The first unlocked track of `kind` from the targeted one outward with nothing in `start..end`. +/// When every track is busy there, a new one is added (on top for video, at the bottom for audio). +fn free_track(s: &mut Session, kind: TrackKind, start: Tick, end: Tick) -> Result { + let seq = s.active_sequence().ok_or(EngineError::NoSequence)?; + let tg = s.targeting(); + let (tracks, dest) = match kind { + TrackKind::Video => (&seq.video_tracks, tg.video_dest), + TrackKind::Audio => (&seq.audio_tracks, tg.audio_dest), + }; + let from = dest.and_then(|d| tracks.iter().position(|t| t.id == d)).unwrap_or(0); + let free = tracks.iter().skip(from).find(|t| !t.locked && !t.items.iter().any(|i| i.start < end && i.end() > start)).map(|t| t.id); + if let Some(t) = free { + return Ok(t); + } + let (params, key) = match kind { + TrackKind::Video => (json!({"video": 1}), "video"), + TrackKind::Audio => (json!({"video": 0, "audio": 1}), "audio"), + }; + let added = s.execute("sequence.addTracks", params)?; + added[key][0].as_u64().map(TrackId).ok_or_else(|| EngineError::Other("no track was added".into())) +} + +pub(crate) fn commands() -> Vec { + vec![CommandSpec { + id: "edit.pasteMedia", + label: "Paste Media", + menu: &[], + shortcut: None, + params: r#"{"paths":[str],"insert":bool?,"place":bool=true,"bin":binId?}"#, + enabled: always, + run: paste_media, + journal: true, + }] +} diff --git a/crates/engine/src/paste_media_tests.rs b/crates/engine/src/paste_media_tests.rs new file mode 100644 index 00000000..6ca2dc2b --- /dev/null +++ b/crates/engine/src/paste_media_tests.rs @@ -0,0 +1,173 @@ +//! Pasting files and images from the system clipboard (#611). + +use std::path::Path; + +use filmcraft_project::{BinEntry, BinId, ClipId, ItemId, TrackKind}; +use filmcraft_time::Tick; +use serde_json::{Value, json}; + +use crate::Session; +use crate::media_test_util::tmp_dir; + +fn demo() -> Session { + let mut s = Session::default(); + s.execute("file.openDemoProject", json!({})).unwrap(); + s +} + +fn write_png(path: &Path) -> String { + image::RgbaImage::from_pixel(64, 36, image::Rgba([40, 120, 200, 255])).save(path).unwrap(); + path.to_string_lossy().into_owned() +} + +fn write_wav(path: &Path, seconds: f32) -> String { + std::fs::write(path, crate::voiceover::write_wav_f32(&vec![0.0; (48_000.0 * seconds) as usize], 48_000)).unwrap(); + path.to_string_lossy().into_owned() +} + +/// Every clip of the active sequence: (track kind, track index, clip id, start, end). +fn clips(s: &Session) -> Vec<(TrackKind, usize, u64, Tick, Tick)> { + let q = s.active_sequence().unwrap(); + let mut v = Vec::new(); + for (kind, tracks) in [(TrackKind::Video, &q.video_tracks), (TrackKind::Audio, &q.audio_tracks)] { + for (i, t) in tracks.iter().enumerate() { + v.extend(t.items.iter().map(|c| (kind, i, c.id.0, c.start, c.end()))); + } + } + v +} + +fn pasted(r: &Value) -> Vec { + r["clips"].as_array().unwrap().iter().map(|c| c.as_u64().unwrap()).collect() +} + +/// An image pasted over footage goes on the first video track that is free there, from the +/// targeted one up; every clip already on the Timeline stays as it was. One undo step takes the +/// clip and the imported item back; the playhead moves to its end, as after Paste. +#[test] +fn a_pasted_image_sits_above_the_footage_without_overwriting_it() { + let dir = tmp_dir("paste-media-image"); + let mut s = demo(); + let png = write_png(&dir.join("shot.png")); + let at = s.playhead(); + let (before, items, undo) = (clips(&s), s.project.items.len(), s.history.undo.len()); + assert!(before.iter().any(|c| c.0 == TrackKind::Video && c.1 == 0 && c.3 <= at && c.4 > at), "the playhead is over a V1 clip"); + let r = s.execute("edit.pasteMedia", json!({"paths": [png]})).unwrap(); + let ids = pasted(&r); + assert_eq!(ids.len(), 1, "a still has picture only"); + let after = clips(&s); + for c in &before { + assert!(after.contains(c), "{c:?} is untouched"); + } + let new = *after.iter().find(|c| c.2 == ids[0]).unwrap(); + assert_eq!((new.0, new.3), (TrackKind::Video, at)); + assert!(new.1 > 0, "not on V1, where the footage is"); + assert!(!before.iter().any(|c| c.0 == TrackKind::Video && c.1 == new.1 && c.3 < new.4 && c.4 > new.3), "on a track that was free there"); + assert!(before.iter().any(|c| c.0 == TrackKind::Video && c.1 < new.1 && c.3 < new.4 && c.4 > new.3), "so it sits above a clip"); + assert_eq!(new.4 - new.3, s.prefs.timeline.still_duration(s.sequence_rate()), "for the still image default duration"); + assert_eq!(s.playhead(), new.4); + assert_eq!(s.state.selection, vec![ClipId(ids[0])]); + assert_eq!(s.history.undo.len(), undo + 1); + assert_eq!(s.undo().as_deref(), Some("Paste")); + assert_eq!((clips(&s), s.project.items.len()), (before, items), "undo takes back the clip and the import"); + let _ = std::fs::remove_dir_all(&dir); +} + +/// With every video track busy at the playhead a new one is added on top; undo removes it too. +/// Audio goes on the first free audio track from the targeted one down, or a new one at the bottom. +#[test] +fn a_track_is_added_when_every_track_is_busy() { + let dir = tmp_dir("paste-media-tracks"); + let mut s = demo(); + let png = write_png(&dir.join("a.png")); + let wav = write_wav(&dir.join("b.wav"), 2.0); + let at = s.playhead(); + // fill every track at the playhead + for _ in 0..8 { + let q = s.active_sequence().unwrap(); + let busy = |ts: &[filmcraft_project::Track]| ts.iter().all(|t| t.items.iter().any(|c| c.start <= at && c.end() > at + Tick::from_seconds_f64(5.0))); + if busy(&q.video_tracks) && busy(&q.audio_tracks) { + break; + } + s.set_playhead(at); + s.execute("edit.pasteMedia", json!({"paths": [png.clone(), wav.clone()]})).unwrap(); + } + let (nv, na) = { + let q = s.active_sequence().unwrap(); + (q.video_tracks.len(), q.audio_tracks.len()) + }; + let before = clips(&s); + s.set_playhead(at); + let r = s.execute("edit.pasteMedia", json!({"paths": [png, wav]})).unwrap(); + let ids = pasted(&r); + let q = s.active_sequence().unwrap(); + assert_eq!((q.video_tracks.len(), q.audio_tracks.len()), (nv + 1, na + 1), "one new track of each kind"); + let after = clips(&s); + let image = after.iter().find(|c| c.2 == ids[0]).unwrap(); + assert_eq!((image.0, image.1), (TrackKind::Video, nv), "the image on the new top track"); + let sound = after.iter().find(|c| c.2 == ids[1]).unwrap(); + assert_eq!((sound.0, sound.1, sound.3), (TrackKind::Audio, na, image.4), "the sound on the new bottom track, after the image"); + for c in &before { + assert!(after.contains(c), "{c:?} is untouched"); + } + s.undo(); + let q = s.active_sequence().unwrap(); + assert_eq!((q.video_tracks.len(), q.audio_tracks.len()), (nv, na), "undo removes the tracks it added"); + let _ = std::fs::remove_dir_all(&dir); +} + +/// `insert` is Paste Insert: on the targeted video track, pushing later clips right. +#[test] +fn paste_insert_inserts_on_the_targeted_track() { + let dir = tmp_dir("paste-media-insert"); + let mut s = demo(); + for t in ["V2", "V3", "A2", "A3"] { + let _ = s.execute("timeline.setTrack", json!({"track": t, "syncLock": false})); + } + let png = write_png(&dir.join("i.png")); + let at = s.playhead(); + let last = s.active_sequence().unwrap().video_tracks[0].items.iter().map(|c| c.end()).max().unwrap(); + let r = s.execute("edit.pasteMedia", json!({"paths": [png], "insert": true})).unwrap(); + let new = *clips(&s).iter().find(|c| c.2 == pasted(&r)[0]).unwrap(); + let still = s.prefs.timeline.still_duration(s.sequence_rate()); + assert_eq!((new.0, new.1, new.3), (TrackKind::Video, 0, at), "on the targeted track, V1"); + assert_eq!(s.active_sequence().unwrap().video_tracks[0].items.iter().map(|c| c.end()).max().unwrap(), last + still, "later clips moved right"); + assert_eq!(s.undo().as_deref(), Some("Paste Insert")); + let _ = std::fs::remove_dir_all(&dir); +} + +/// Without `place` (the Project panel) the files are only imported, into `bin`. +#[test] +fn without_place_files_are_only_imported() { + let dir = tmp_dir("paste-media-import"); + let mut s = demo(); + let png = write_png(&dir.join("p.png")); + let bin = s.execute("file.newBin", json!({"name": "Pasted"})).unwrap()["bin"].as_u64().unwrap(); + let before = clips(&s); + let r = s.execute("edit.pasteMedia", json!({"paths": [png], "place": false, "bin": bin})).unwrap(); + assert_eq!(clips(&s), before); + let item = ItemId(r["items"][0].as_u64().unwrap()); + let pasted_bin = s.project.root.find_bin(BinId(bin)).unwrap(); + assert!(pasted_bin.children.iter().any(|e| matches!(e, BinEntry::Item(i) if *i == item)), "imported into the bin"); + assert!(s.execute("edit.pasteMedia", json!({"paths": []})).is_err(), "nothing to paste"); + let _ = std::fs::remove_dir_all(&dir); +} + +/// A pasted image is written as `Pasted Image .png` next to the saved project, with the first +/// number no file or project item uses. +#[test] +fn pasted_images_are_saved_next_to_the_project() { + let dir = tmp_dir("paste-media-save"); + let mut s = demo(); + s.path = Some(dir.join("film.fcproj").to_string_lossy().into_owned()); + let mut png = Vec::new(); + image::RgbaImage::from_pixel(8, 8, image::Rgba([0, 0, 0, 255])).write_to(&mut std::io::Cursor::new(&mut png), image::ImageFormat::Png).unwrap(); + let first = crate::paste_media::save_pasted_image(&mut s, &png).unwrap(); + assert_eq!(Path::new(&first), dir.join("Pasted Image 1.png")); + assert_eq!(std::fs::read(&first).unwrap(), png); + let second = crate::paste_media::save_pasted_image(&mut s, &png).unwrap(); + assert_eq!(Path::new(&second), dir.join("Pasted Image 2.png")); + s.execute("edit.pasteMedia", json!({"paths": [second]})).unwrap(); + assert!(s.project.items.values().any(|i| i.name == "Pasted Image 2.png")); + let _ = std::fs::remove_dir_all(&dir); +} diff --git a/crates/engine/src/presets.rs b/crates/engine/src/presets.rs index 75f71144..bd1c14e8 100644 --- a/crates/engine/src/presets.rs +++ b/crates/engine/src/presets.rs @@ -365,12 +365,24 @@ pub fn instantiate(p: &EffectPreset, target: &TrackItem, size: (u32, u32)) -> Ve out } -/// Put preset effects on a clip: intrinsic ones replace the clip's instance, others are inserted -/// before the intrinsic effects. +/// Keep `old`'s keyframes that lie outside the span `new` animates, so presets keyframing the same +/// parameter at different times (Fade In, then Fade Out) add up instead of replacing each other. +fn merge_keyframes(new: &mut EffectInstance, old: &EffectInstance) { + for (id, p) in new.params.iter_mut() { + let (Some(first), Some(last)) = (p.keyframes.first().map(|k| k.time), p.keyframes.last().map(|k| k.time)) else { continue }; + let Some(o) = old.params.get(id) else { continue }; + p.keyframes.extend(o.keyframes.iter().filter(|k| k.time < first || k.time > last).cloned()); + p.keyframes.sort_by_key(|k| k.time); + } +} + +/// Put preset effects on a clip: intrinsic ones replace the clip's instance (keeping its keyframes +/// outside the span the preset animates), others are inserted before the intrinsic effects. pub fn put_effects(it: &mut TrackItem, effects: Vec) { - for e in effects { + for mut e in effects { let intrinsic = e.def().is_some_and(|d| d.intrinsic); if intrinsic && let Some(slot) = it.effects.iter_mut().find(|x| x.effect == e.effect) { + merge_keyframes(&mut e, slot); *slot = e; continue; } diff --git a/crates/engine/src/presets_tests.rs b/crates/engine/src/presets_tests.rs index db8065b3..7a2ab0e1 100644 --- a/crates/engine/src/presets_tests.rs +++ b/crates/engine/src/presets_tests.rs @@ -194,6 +194,25 @@ fn builtins_apply_and_user_presets_persist() { let _ = std::fs::remove_dir_all(&dir); } +/// #333: Fade In then Fade Out on one clip keeps both ramps (they keyframe the same Opacity). +#[test] +fn fade_in_and_fade_out_add_up_on_one_clip() { + let (mut s, clips) = demo(); + let c = clips[0]; + s.execute("presets.apply", json!({"preset": "Fade In", "clips": [c.0]})).unwrap(); + s.execute("presets.apply", json!({"preset": "Fade Out", "clips": [c.0]})).unwrap(); + let it = item(&s, c); + let op = it.effect("opacity").unwrap().param("opacity").unwrap(); + let keys: Vec<(Tick, ParamValue)> = op.keyframes.iter().map(|k| (k.time, k.value.clone())).collect(); + let (tin, tout) = (it.source_in, it.source_in + media_len(&it)); + let f = ParamValue::Float; + assert_eq!(keys, vec![(tin, f(0.0)), (tin + secs(1.0), f(100.0)), (tout - secs(1.0), f(100.0)), (tout, f(0.0))], "both fades survive"); + // re-applying Fade In replaces its own ramp, not the Fade Out one + s.execute("presets.apply", json!({"preset": "Fade In", "clips": [c.0]})).unwrap(); + let it = item(&s, c); + assert_eq!(it.effect("opacity").unwrap().param("opacity").unwrap().keyframes.len(), 4); +} + #[test] fn mask_geometry_scales_to_the_target_frame_size() { let (s, clips) = demo(); diff --git a/crates/engine/src/project_panel.rs b/crates/engine/src/project_panel.rs index 84659296..735f5b28 100644 --- a/crates/engine/src/project_panel.rs +++ b/crates/engine/src/project_panel.rs @@ -17,9 +17,9 @@ //! | `project.freeform.saveArrangement` / `.restoreArrangement` / `.arrangements` / `.deleteArrangement` | saved arrangements | //! | `project.freeform.options` | Freeform View Options… (grid, snap, names, durations) | //! | `project.renameBin` | rename a bin (inline rename, undoable) | -//! | `project.selectAll` / `project.deselectAll` | select every item in the project / clear the Project panel selection (Cmd+A / Cmd+Shift+A with the panel focused) | +//! | `project.selectAll` / `project.deselectAll` | select the items a bin shows (`bin`, plus the sub-bins open in List view in `expanded`; without `bin` every item in the project) / clear the Project panel selection (Cmd+A / Cmd+Shift+A with the panel focused) | -use std::collections::BTreeMap; +use std::collections::{BTreeMap, BTreeSet}; use filmcraft_project::{Bin, BinEntry, BinId, ItemId, ItemKind, MediaRef, Project, ProjectItem, TrackKind}; use filmcraft_time::{Tick, TimeDisplay, format_time}; @@ -1171,6 +1171,43 @@ fn rename_bin(s: &mut Session, p: &Value) -> Result { Ok(json!({"bin": bin.0, "name": name})) } +/// Select All in the Project panel. With `bin`, only what a panel showing that bin lists: its +/// items, plus the items of the sub-bins named in `expanded` (bins twirled open in List view), +/// never those of a closed bin. Without `bin`, every item in the project. +fn select_all(s: &mut Session, p: &Value) -> Result { + const CMD: &str = "project.selectAll"; + // strict: a negative or fractional `bin` is an error, not bin 0 + let id = match p.get("bin") { + None | Some(Value::Null) => { + s.state.project_selection = s.project.items.keys().copied().collect(); + return Ok(json!({"selected": s.state.project_selection.len()})); + } + Some(b) => BinId(b.as_u64().ok_or_else(|| bad(CMD, "`bin` must be a bin id"))?), + }; + let bin = s.project.root.find_bin(id).ok_or_else(|| bad(CMD, format!("no bin {}", id.0)))?; + let expanded: BTreeSet = match p.get("expanded") { + None | Some(Value::Null) => Default::default(), + Some(Value::Array(a)) => { + a.iter().map(|v| v.as_u64().map(BinId).ok_or_else(|| bad(CMD, "`expanded` must be a list of bin ids"))).collect::>()? + } + Some(_) => return Err(bad(CMD, "`expanded` must be a list of bin ids")), + }; + // iterative walk: a hostile project file cannot overflow the stack with deeply nested bins + let mut sel = Vec::new(); + let mut stack = vec![bin]; + while let Some(b) = stack.pop() { + for e in &b.children { + match e { + BinEntry::Item(i) if s.project.item(*i).is_some_and(|it| listed(&s.project, it)) => sel.push(*i), + BinEntry::Bin(sub) if expanded.contains(&sub.id) => stack.push(sub), + _ => {} + } + } + } + s.state.project_selection = sel; + Ok(json!({"bin": id.0, "selected": s.state.project_selection.len()})) +} + // ------------------------------------------------------------------------------------ registry pub(crate) fn commands() -> Vec { @@ -1223,10 +1260,7 @@ pub(crate) fn commands() -> Vec { freeform_options, ), c("project.renameBin", "Rename Bin", r#"{"bin":binId,"name":str}"#, always, rename_bin), - c("project.selectAll", "Select All Project Items", "{}", always, |s, _| { - s.state.project_selection = s.project.items.keys().copied().collect(); - Ok(json!({"selected": s.state.project_selection.len()})) - }), + c("project.selectAll", "Select All Project Items", r#"{"bin":binId?,"expanded":[binId]?}"#, always, select_all), c("project.deselectAll", "Deselect All Project Items", "{}", always, |s, _| { s.state.project_selection.clear(); Ok(Value::Null) diff --git a/crates/engine/src/project_panel_tests.rs b/crates/engine/src/project_panel_tests.rs index 42d3c79e..b094a54f 100644 --- a/crates/engine/src/project_panel_tests.rs +++ b/crates/engine/src/project_panel_tests.rs @@ -2,7 +2,7 @@ //! sorting, view presets, Freeform positions / stacks / arrangements (undoable, saved with the //! project) and bin renames. -use filmcraft_project::{ItemId, ItemKind}; +use filmcraft_project::{BinEntry, ItemId, ItemKind}; use serde_json::{Value, json}; use crate::Session; @@ -305,3 +305,75 @@ fn select_all_selects_every_project_item() { assert!(!s.state.selection.is_empty()); assert_eq!(s.state.project_selection, before); } + +#[test] +fn select_all_in_a_bin_skips_closed_bins_and_other_bins() { + // #456: Cmd+A in the Project panel selected every item in the project, closed bins included + let mut s = demo(); + let outer = s.execute("file.newBin", json!({"name": "Footage A"})).unwrap()["bin"].as_u64().unwrap(); + let inner = s.execute("file.newBin", json!({"name": "Day 1", "parent": outer})).unwrap()["bin"].as_u64().unwrap(); + let ids = media(&s); + assert!(ids.len() >= 2); + s.execute("project.moveToBin", json!({"items": [ids[0].0], "bin": outer})).unwrap(); + s.execute("project.moveToBin", json!({"items": [ids[1].0], "bin": inner})).unwrap(); + s.state.project_selection.clear(); + + // only the bin's own items: the closed sub-bin's item stays unselected + let r = s.execute("project.selectAll", json!({"bin": outer})).unwrap(); + assert_eq!(r["selected"].as_u64(), Some(1)); + assert_eq!(s.state.project_selection, vec![ids[0]]); + + // a sub-bin twirled open in List view counts + s.execute("project.selectAll", json!({"bin": outer, "expanded": [inner]})).unwrap(); + let mut sel = s.state.project_selection.clone(); + sel.sort(); + assert_eq!(sel, vec![ids[0], ids[1]]); + + // the root: its direct items only, nothing from the (closed) bins + let root = s.project.root.id; + let mut want: Vec = s + .project + .root + .children + .iter() + .filter_map(|e| if let BinEntry::Item(i) = e { Some(*i) } else { None }) + .filter(|i| s.project.item(*i).is_some_and(|it| project_panel::listed(&s.project, it))) + .collect(); + want.sort(); + s.execute("project.selectAll", json!({"bin": root.0})).unwrap(); + let mut sel = s.state.project_selection.clone(); + sel.sort(); + assert_eq!(sel, want); + assert!(!sel.contains(&ids[0]) && !sel.contains(&ids[1])); + assert!(sel.len() < s.project.items.len()); + + // an open bin inside a closed one is not shown, so it does not count either + s.execute("project.selectAll", json!({"bin": root.0, "expanded": [inner]})).unwrap(); + assert!(!s.state.project_selection.contains(&ids[1])); + // ... but open all the way down, it does + s.execute("project.selectAll", json!({"bin": root.0, "expanded": [outer, inner]})).unwrap(); + assert!(s.state.project_selection.contains(&ids[0]) && s.state.project_selection.contains(&ids[1])); +} + +#[test] +fn select_all_rejects_bad_bins_without_touching_the_selection() { + let mut s = demo(); + let ids = media(&s); + let root = s.project.root.id.0; + s.execute("project.select", json!({"items": [ids[0].0]})).unwrap(); + for p in [ + json!({"bin": 987_654_321u64}), + json!({"bin": "footage"}), + json!({"bin": -1}), + json!({"bin": root, "expanded": "all"}), + json!({"bin": root, "expanded": ["x"]}), + ] { + assert!(s.execute("project.selectAll", p.clone()).is_err(), "{p}"); + assert_eq!(s.state.project_selection, vec![ids[0]], "{p}"); + } + // unknown ids in `expanded` are simply not open bins + assert!(s.execute("project.selectAll", json!({"bin": root, "expanded": [987_654_321u64]})).is_ok()); + // `bin: null` is the same as no bin: the whole project + let r = s.execute("project.selectAll", json!({"bin": null})).unwrap(); + assert_eq!(r["selected"].as_u64(), Some(s.project.items.len() as u64)); +} diff --git a/crates/engine/src/project_tools.rs b/crates/engine/src/project_tools.rs index a23e7530..a0e2d181 100644 --- a/crates/engine/src/project_tools.rs +++ b/crates/engine/src/project_tools.rs @@ -22,9 +22,9 @@ //! | `clip.automateToSequence` | Clip ▸ Automate to Sequence… | //! | `help.systemReport` | query behind Help ▸ System Compatibility Report… | -use std::collections::BTreeSet; +use std::collections::{BTreeMap, BTreeSet}; -use filmcraft_project::{BinEntry, BinId, FindOp, FindQuery, FindRow, ItemId, ItemKind, MarkerKind, MediaRef, Project, SearchBin, TrackKind}; +use filmcraft_project::{BinEntry, FindOp, FindQuery, FindRow, ItemId, ItemKind, MarkerKind, MediaRef, Project, SearchBin, TrackKind}; use filmcraft_time::{Tick, TimeDisplay, TimeRange}; use serde::{Deserialize, Serialize}; use serde_json::{Value, json}; @@ -704,18 +704,42 @@ fn new_from_template(s: &mut Session, p: &Value) -> Result { /// everything those items need), as `project.delete` does. The project's own top bin does not, /// since no command acts on it (#244), and only ids that name a real item end up in the result. pub fn closure(p: &Project, roots: &[ItemId]) -> BTreeSet { + // Index the bin tree once. Told apart from items with a set membership test (rather than a + // full `find_bin` per queued id, which is O(n²) when exporting many items from a flat + // project) and expanded from the indexed children (rather than a second walk per bin). + let mut bins: BTreeSet = BTreeSet::new(); + let mut children: BTreeMap, Vec)> = BTreeMap::new(); + let mut bin_stack = vec![&p.root]; + while let Some(b) = bin_stack.pop() { + bins.insert(b.id.0); + let mut direct = (Vec::new(), Vec::new()); + for c in &b.children { + match c { + BinEntry::Item(i) => direct.0.push(*i), + BinEntry::Bin(x) => { + direct.1.push(x.id.0); + bin_stack.push(x); + } + } + } + children.insert(b.id.0, direct); + } + let mut keep: BTreeSet = BTreeSet::new(); let mut todo: Vec = roots.to_vec(); let mut seen_bins: BTreeSet = BTreeSet::new(); + let mut bin_todo: Vec = Vec::new(); while let Some(id) = todo.pop() { - let bin = BinId(id.0); - if bin != p.root.id && p.root.find_bin(bin).is_some() { - if seen_bins.insert(id.0) - && let Some(b) = p.root.find_bin(bin) - { - let mut items = Vec::new(); - b.all_items(&mut items); - todo.extend(items); + if bins.contains(&id.0) && id.0 != p.root.id.0 { + bin_todo.push(id.0); + while let Some(bid) = bin_todo.pop() { + if !seen_bins.insert(bid) { + continue; + } + if let Some((items, subs)) = children.get(&bid) { + todo.extend(items.iter().copied()); + bin_todo.extend(subs.iter().copied()); + } } continue; } diff --git a/crates/engine/src/sequence_tools.rs b/crates/engine/src/sequence_tools.rs index 50409099..a0027913 100644 --- a/crates/engine/src/sequence_tools.rs +++ b/crates/engine/src/sequence_tools.rs @@ -131,7 +131,7 @@ fn groups() -> Vec<(&'static str, Vec)> { "Delete Tracks…", &["Sequence"], None, - r#"{"video":"empty"|"V2"|id?,"audio":"empty"|"A2"|id?}"#, + r#"{"video":"empty"|"V2"|id?,"audio":"empty"|"A2"|id?,"captions":"empty"|"C2"|id?}"#, has_seq, delete_tracks, )], @@ -748,6 +748,18 @@ fn delete_tracks(s: &mut Session, p: &Value) -> Result { } doomed.extend(mine); } + // caption tracks: unlike video/audio, a sequence may have none, so all of them can go + match p.get("captions") { + None | Some(Value::Null) => {} + Some(v) if v.as_str().is_some_and(|x| matches!(x, "empty" | "allEmpty" | "all-empty")) => { + doomed.extend(q.caption_tracks.iter().filter(|t| t.captions.is_empty()).map(|t| t.id)); + } + Some(v @ (Value::String(_) | Value::Number(_))) => { + let id = crate::captions::track_param(s, &json!({"track": v})).ok_or_else(|| bad("sequence.deleteTracks", "unknown caption track"))?; + doomed.push(id); + } + Some(_) => return Err(bad("sequence.deleteTracks", "`captions` must be \"empty\", a name like \"C2\" or a track id")), + } if doomed.is_empty() { return Err(EngineError::Other("no tracks to delete".into())); } @@ -755,7 +767,9 @@ fn delete_tracks(s: &mut Session, p: &Value) -> Result { s.edit_sequence("Delete Tracks", |q, _, st| { q.video_tracks.retain(|t| !doomed.contains(&t.id)); q.audio_tracks.retain(|t| !doomed.contains(&t.id)); + q.caption_tracks.retain(|t| !doomed.contains(&t.id)); st.selection.retain(|c| q.find_item(*c).is_some()); + st.caption_selection.retain(|c| q.find_caption(*c).is_some()); if let Some(tg) = st.targeting.get_mut(&seq_id) { tg.targeted.retain(|t| !doomed.contains(t)); if tg.video_dest.is_some_and(|t| doomed.contains(&t)) { diff --git a/crates/engine/src/sequence_tools_tests.rs b/crates/engine/src/sequence_tools_tests.rs index 01b55b43..9e164ce2 100644 --- a/crates/engine/src/sequence_tools_tests.rs +++ b/crates/engine/src/sequence_tools_tests.rs @@ -350,6 +350,42 @@ fn delete_tracks_empty_and_specific() { assert_eq!(s.active_sequence().unwrap(), &q); } +#[test] +fn delete_tracks_removes_caption_tracks() { + let mut s = demo(); + let before = s.active_sequence().unwrap().clone(); + let n0 = before.caption_tracks.len(); + for _ in 0..3 { + s.execute("captions.newTrack", json!({"format": "Subtitle"})).unwrap(); + } + // new tracks are inserted on top: C1 gets a caption, C2 and C3 stay empty + let cap = s.execute("captions.add", json!({"track": "C1", "text": "Hi", "seconds": 1.0})).unwrap()["caption"].as_u64().unwrap(); + s.execute("captions.select", json!({"captions": [cap]})).unwrap(); + let empty = s.active_sequence().unwrap().caption_tracks.iter().filter(|t| t.captions.is_empty()).count(); + assert!(empty >= 2); + let r = s.execute("sequence.deleteTracks", json!({"captions": "empty"})).unwrap(); + assert_eq!(r["deleted"], json!(empty)); + let q = s.active_sequence().unwrap(); + assert!(q.caption_tracks.iter().all(|t| !t.captions.is_empty())); + assert_eq!(q.video_tracks.len(), before.video_tracks.len(), "video tracks untouched"); + assert!(s.execute("sequence.deleteTracks", json!({"captions": "empty"})).is_err(), "nothing left to delete"); + // unknown names, ids and value types are errors, not panics + for bad in [json!("C99"), json!("C0"), json!("Cx"), json!(999_999), json!(true), json!(["C1"])] { + assert!(s.execute("sequence.deleteTracks", json!({"captions": bad})).is_err(), "{bad}"); + } + // a specific track (with its captions); the last caption track may go too + let left = s.active_sequence().unwrap().caption_tracks.len(); + for _ in 0..left { + s.execute("sequence.deleteTracks", json!({"captions": "C1"})).unwrap(); + } + assert!(s.active_sequence().unwrap().caption_tracks.is_empty()); + assert!(s.state.caption_selection.is_empty(), "selection of deleted captions cleared"); + for _ in 0..left + 1 { + s.execute("edit.undo", json!({})).unwrap(); + } + assert_eq!(s.active_sequence().unwrap().caption_tracks.len(), n0 + 3); +} + #[test] fn split_points_on_the_sequence() { let (mut s, bars) = fresh(); diff --git a/crates/engine/src/settings.rs b/crates/engine/src/settings.rs index 2f020835..7b0c11dd 100644 --- a/crates/engine/src/settings.rs +++ b/crates/engine/src/settings.rs @@ -104,6 +104,9 @@ pub struct AppearancePrefs { pub accessible_contrast: bool, /// Highlight (accent) colour of selections, focus and primary buttons (`#rrggbb`). pub highlight_color: String, + /// "UI Scale": `auto` (default: the scale the operating system reports) or a fixed + /// percentage of [`UI_SCALES`] (`100` = one point per physical pixel). + pub ui_scale: String, } impl Default for AppearancePrefs { @@ -115,11 +118,19 @@ impl Default for AppearancePrefs { light_theme: "light".into(), accessible_contrast: false, highlight_color: DEFAULT_HIGHLIGHT.into(), + ui_scale: "auto".into(), } } } impl AppearancePrefs { + /// The fixed UI scale (physical pixels per point) "UI Scale" asks for, or `None` for `auto` + /// (and for a value outside [`UI_SCALES`]), which keeps the operating system's scale. + pub fn ui_scale_factor(&self) -> Option { + // `auto` doesn't parse; every other listed value is a positive whole percentage + UI_SCALES.iter().any(|(k, _)| *k == self.ui_scale).then(|| self.ui_scale.parse::().ok()).flatten().map(|pct| pct / 100.0) + } + /// The theme to show (`darkest` | `dark` | `light`): the light or dark theme of the mode. Auto /// follows `system_light` (the operating system's appearance) and is dark without an answer. pub fn shown_theme(&self, system_light: Option) -> &str { @@ -669,6 +680,17 @@ const BIN_OPEN: &[(&str, &str)] = &[("openInPlace", "Open in place"), ("openNewT const PROJECT_OPEN: &[(&str, &str)] = &[("openNewTab", "Open new tab"), ("openNewWindow", "Open in new window")]; const THEMES: &[(&str, &str)] = &[("darkest", "Darkest"), ("dark", "Dark"), ("light", "Light")]; pub const APPEARANCE_MODES: &[(&str, &str)] = &[("auto", "Sync with system"), ("dark", "Dark"), ("light", "Light")]; +pub const UI_SCALES: &[(&str, &str)] = &[ + ("auto", "Auto (system)"), + ("75", "75%"), + ("100", "100%"), + ("125", "125%"), + ("150", "150%"), + ("175", "175%"), + ("200", "200%"), + ("250", "250%"), + ("300", "300%"), +]; pub const DARK_THEMES: &[(&str, &str)] = &[("darkest", "Darkest"), ("dark", "Dark")]; pub const LIGHT_THEMES: &[(&str, &str)] = &[("light", "Light")]; const MIXDOWN: &[(&str, &str)] = &[("front", "Front Only"), ("frontRear", "Front + Rear"), ("frontLfe", "Front + LFE"), ("frontRearLfe", "Front + Rear + LFE")]; @@ -795,6 +817,7 @@ static CATEGORIES: &[Category] = &[ f("appearance.colorTheme", "Color Theme", Kind::Choice(THEMES), true), b("appearance.accessibleContrast", "Accessible color contrast", true), f("appearance.highlightColor", "Highlight Color", Kind::Color, true), + f("appearance.uiScale", "UI Scale", Kind::Choice(UI_SCALES), true), ], }, Category { diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index 9852734d..f2b50dd8 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -139,6 +139,45 @@ fn trim_linked_and_ripple_delete() { assert_eq!(q.video_tracks[0].items[0].start, s.sequence_rate().tick_of(12), "the trim gap remains"); } +#[test] +fn clear_respects_individually_selected_linked_clips_and_undo() { + for audio in [false, true] { + let mut s = demo(); + let video = s.active_sequence().unwrap().video_tracks[0].items[0].clone(); + let sound = s.active_sequence().unwrap().audio_tracks[0].items.iter().find(|i| i.link == video.link).unwrap().clone(); + let (selected, partner) = if audio { (sound.id, video.id) } else { (video.id, sound.id) }; + s.execute("timeline.select", json!({"clips": [selected.0], "linked": false})).unwrap(); + assert_eq!(s.state.selection, vec![selected]); + s.execute("edit.clear", json!({})).unwrap(); + assert!(s.active_sequence().unwrap().find_item(selected).is_none()); + assert!(s.active_sequence().unwrap().find_item(partner).is_some()); + s.execute("edit.undo", json!({})).unwrap(); + assert!(s.active_sequence().unwrap().find_item(selected).is_some()); + s.execute("timeline.select", json!({"clips": [selected.0]})).unwrap(); + assert_eq!(s.state.selection.len(), 2); + s.execute("edit.clear", json!({})).unwrap(); + assert!(s.active_sequence().unwrap().find_item(partner).is_none()); + } +} + +#[test] +fn ripple_delete_respects_individual_selection_and_explicit_clear_keeps_link_behavior() { + let mut s = demo(); + let video = s.active_sequence().unwrap().video_tracks[0].items[0].clone(); + let sound = s.active_sequence().unwrap().audio_tracks[0].items.iter().find(|i| i.link == video.link).unwrap().id; + let tracks: Vec<_> = s.active_sequence().unwrap().all_tracks().map(|t| t.id.0).collect(); + for track in tracks { + s.execute("timeline.setTrack", json!({"track": track, "syncLock": false})).unwrap(); + } + s.execute("timeline.select", json!({"clips": [video.id.0], "linked": false})).unwrap(); + s.execute("edit.rippleDelete", json!({})).unwrap(); + assert!(s.active_sequence().unwrap().find_item(video.id).is_none()); + assert!(s.active_sequence().unwrap().find_item(sound).is_some()); + s.execute("edit.undo", json!({})).unwrap(); + s.execute("edit.clear", json!({"clips": [video.id.0]})).unwrap(); + assert!(s.active_sequence().unwrap().find_item(sound).is_none()); +} + #[test] fn slide_moves_linked_audio_with_the_video() { let mut s = demo(); diff --git a/crates/engine/src/vfx_tests.rs b/crates/engine/src/vfx_tests.rs index cb2df958..f1aeb598 100644 --- a/crates/engine/src/vfx_tests.rs +++ b/crates/engine/src/vfx_tests.rs @@ -115,3 +115,16 @@ fn new_effect_params_are_settable_and_keyframable() { let img = s.render_program(0.125).unwrap(); assert!(img.px.iter().all(|v| v.is_finite())); } + +#[test] +fn ultra_key_setting_writes_aggressive_parameters() { + let mut s = demo(); + let clip = first_v1_clip(&s); + s.execute("effects.apply", json!({"clips": [clip], "effect": "ultra_key"})).unwrap(); + s.execute("effects.setParam", json!({"clip": clip, "effect": "ultra_key", "param": "setting", "value": 2})).unwrap(); + let e = s.active_sequence().unwrap().find_item(filmcraft_project::ClipId(clip)).unwrap().1.effect("ultra_key").unwrap(); + assert_eq!(e.param("tolerance").unwrap().value.as_f64(), Some(90.0)); + assert_eq!(e.param("pedestal").unwrap().value.as_f64(), Some(50.0)); + assert_eq!(e.param("choke").unwrap().value.as_f64(), Some(10.0)); + assert_eq!(e.param("contrast").unwrap().value.as_f64(), Some(10.0)); +} diff --git a/crates/export/src/audio_out.rs b/crates/export/src/audio_out.rs index b92b6054..5fd2b90c 100644 --- a/crates/export/src/audio_out.rs +++ b/crates/export/src/audio_out.rs @@ -145,6 +145,11 @@ impl AudioOut { Some(buf) } + /// Output samples still to come (per channel). + pub fn remaining(&self) -> u64 { + u64::try_from(self.end.saturating_sub(self.out)).unwrap_or(0) + } + /// Everything still to come (to the end of the range). pub fn rest(&mut self, sources: &dyn SourceProvider) -> Option>> { self.pull(self.end, sources) diff --git a/crates/export/src/lib.rs b/crates/export/src/lib.rs index 312d0301..43966c05 100644 --- a/crates/export/src/lib.rs +++ b/crates/export/src/lib.rs @@ -274,6 +274,10 @@ pub struct ExportSettings { /// stream needs is raised. pub h264_level: Option, pub bitrate_mode: BitrateMode, + /// Constant rate factor of [`BitrateMode::Crf`], 0 (best) to 51 (smallest); 23 by default, + /// 18 is visually near-lossless. + #[serde(default = "default_crf")] + pub crf: f32, /// May H.264 be encoded by the system's hardware encoder (VideoToolbox on macOS, NVENC on /// Windows)? Off unless asked for: hardware output depends on the machine, so it is not /// byte-reproducible like the built-in encoder's (`determinism_tests`). @@ -471,6 +475,7 @@ impl Default for ExportSettings { h264_profile: H264Profile::High, h264_level: None, bitrate_mode: BitrateMode::default(), + crf: DEFAULT_CRF, hardware_encoding: HardwareEncoding::default(), gpu_rendering: GpuRendering::Off, max_bitrate_kbps: None, @@ -490,6 +495,13 @@ impl Default for ExportSettings { } } +/// [`ExportSettings::crf`] of new settings and of settings saved before it existed. +pub const DEFAULT_CRF: f32 = 23.0; + +fn default_crf() -> f32 { + DEFAULT_CRF +} + impl ExportSettings { /// Reject settings the encoders cannot honour. pub fn validate(&self) -> Result<()> { @@ -524,6 +536,14 @@ impl ExportSettings { if self.format == Format::Hevc && self.bitrate_mode == BitrateMode::Vbr2Pass { return Err(ExportError::Unsupported("H.265 export has no two-pass mode: choose CBR or VBR, 1 pass".into())); } + if self.bitrate_mode == BitrateMode::Crf { + if self.format == Format::Hevc { + return Err(ExportError::Unsupported("H.265 export has no CRF mode: choose CBR or VBR, 1 pass".into())); + } + if !self.crf.is_finite() || !(0.0..=51.0).contains(&self.crf) { + return Err(ExportError::Unsupported("CRF must be between 0 and 51".into())); + } + } Ok(()) } } @@ -1442,6 +1462,8 @@ fn h264_factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSetti let max = s.max_bitrate_kbps.filter(|m| *m >= kbps).unwrap_or_else(|| (u64::from(kbps) * 3 / 2).min(u64::from(u32::MAX)) as u32); cfg.rate = match s.bitrate_mode { BitrateMode::Cbr => filmcraft_h264enc::RateControl::Cbr { kbps }, + // the encoder clamps to 0..=51; a NaN from a hand-edited preset becomes the default + BitrateMode::Crf => filmcraft_h264enc::RateControl::Crf(if s.crf.is_finite() { s.crf } else { DEFAULT_CRF }), _ => filmcraft_h264enc::RateControl::Vbr { target_kbps: kbps, max_kbps: max }, }; cfg.pass = match &s.h264_pass { @@ -1572,20 +1594,41 @@ pub fn export(project: &Arc, seq: ItemId, settings: &ExportSettings, so let batch = rayon::current_num_threads().clamp(2, 16) as i64; let (bytes, nframes) = match settings.format { Format::Wav | Format::Aiff => { + let mut a = audio_out::AudioOut::new(project.clone(), seq, settings, range)?; + let (ch, sr, bits) = (a.channels as u16, a.sr, settings.audio.bits); + let container = if settings.format == Format::Wav { pcm::PcmContainer::Wav } else { pcm::PcmContainer::Aiff }; + let frames = a.remaining(); + // the sizes are known up front: refuse an AIFF that cannot hold them before measuring + let head = pcm::header(container, ch, sr, bits, frames).map_err(ExportError::Unsupported)?; + // one second per chunk: memory stays flat however long the range is + let chunk = i64::from(sr.max(1)); if !settings.part_of_batch { - progress.total.store(1, Ordering::Relaxed); + progress.total.store(frames.div_ceil(chunk as u64).max(1), Ordering::Relaxed); progress.set_status(format!("Exporting audio ({})", settings.format.label())); } - let mut a = audio_out::AudioOut::new(project.clone(), seq, settings, range)?; a.measure(settings, sources, &cancelled)?; *progress.loudness.lock().unwrap_or_else(|e| e.into_inner()) = a.loudness; - let planar = a.rest(sources).unwrap_or_else(|| vec![Vec::new(); a.channels]); - let inter = audio_out::interleave(&planar); - let (ch, sr, bits) = (a.channels as u16, a.sr, settings.audio.bits); - let data = if settings.format == Format::Wav { pcm::write_wav(&inter, ch, sr, bits) } else { pcm::write_aiff(&inter, ch, sr, bits) }; - let n = write_output(settings, &settings.path, data)?; - progress.done.store(1, Ordering::Relaxed); - (n, planar.first().map_or(0, Vec::len) as u64) + let mut out = Out::create(settings)?; + let io = |e: std::io::Error| ExportError::Io(e.to_string()); + out.write_all(&head).map_err(io)?; + let mut buf = Vec::new(); + while let Some(planar) = a.pull(a.out.saturating_add(chunk), sources) { + if cancelled() { + return Err(ExportError::Cancelled); + } + buf.clear(); + pcm::encode(container, &audio_out::interleave(&planar), bits, &mut buf); + out.write_all(&buf).map_err(io)?; + if !settings.part_of_batch { + progress.done.fetch_add(1, Ordering::Relaxed); + } + } + out.write_all(pcm::pad(frames, ch, bits)).map_err(io)?; + let n = out.finish(settings)?; + if settings.part_of_batch { + progress.done.store(1, Ordering::Relaxed); + } + (n, frames) } Format::PngSequence | Format::TiffSequence | Format::BmpSequence | Format::Gif => { let pipe = pipeline::Pipeline::new(project.clone(), seq, settings, false)?; diff --git a/crates/export/src/pcm.rs b/crates/export/src/pcm.rs index c479a017..03b88a74 100644 --- a/crates/export/src/pcm.rs +++ b/crates/export/src/pcm.rs @@ -1,5 +1,16 @@ -//! Audio-only files: WAV (RIFF, little-endian PCM) and AIFF (big-endian PCM), 16 or 24 bit, plus -//! image-sequence file naming. +//! Audio-only files: WAV (RIFF, little-endian PCM; RF64 past 4 GiB) and AIFF (big-endian PCM), +//! 16 or 24 bit, plus image-sequence file naming. +//! +//! The header is written first from the known length ([`header`]), then the samples in chunks +//! ([`encode`]) and the pad byte ([`pad`]), so an export streams to disk instead of building the +//! file in memory. + +/// The container of an audio-only export. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum PcmContainer { + Wav, + Aiff, +} /// Quantise one sample to a signed integer of `bits` (16 or 24). fn quantise(s: f32, bits: u16) -> i32 { @@ -7,41 +18,128 @@ fn quantise(s: f32, bits: u16) -> i32 { (s.clamp(-1.0, 1.0) * max).round() as i32 } +/// The sample size written for a requested `bits`: 24, else 16. +fn sample_bits(bits: u16) -> u16 { + if bits >= 24 { 24 } else { 16 } +} + +/// Bytes of sample data for `frames` frames, or None when that does not fit 64 bits. +fn data_len(frames: u64, channels: u16, bits: u16) -> Option { + frames.checked_mul(u64::from(channels))?.checked_mul(u64::from(sample_bits(bits) / 8)) +} + +/// The pad byte after an odd-sized data chunk (RIFF and AIFF chunks are word-aligned). +pub fn pad(frames: u64, channels: u16, bits: u16) -> &'static [u8] { + if data_len(frames, channels, bits).is_some_and(|n| n & 1 == 1) { &[0] } else { &[] } +} + +/// Everything before the samples of a file of `frames` frames. A WAV whose RIFF size would not +/// fit 32 bits (4 GiB: 24-bit 5.1 at 96 kHz after about 41 minutes) is written as RF64 (EBU Tech +/// 3306), with the 64-bit sizes in a `ds64` chunk. AIFF has no 64-bit form, so a longer AIFF is +/// an error naming WAV instead. +pub fn header(container: PcmContainer, channels: u16, sample_rate: u32, bits: u16, frames: u64) -> Result, String> { + let too_long = || "the audio is too long to write".to_string(); + let bits = sample_bits(bits); + let bps = bits / 8; + let data = data_len(frames, channels, bits).ok_or_else(too_long)?; + let block_align = channels.checked_mul(bps).ok_or("too many audio channels")?; + match container { + PcmContainer::Wav => { + let byte_rate = u32::try_from(u64::from(sample_rate) * u64::from(block_align)).map_err(|_| "the audio sample rate is too high".to_string())?; + let ext = channels > 2; + let fmt_len: u32 = if ext { 40 } else { 16 }; + let padded = data.checked_add(data & 1).ok_or_else(too_long)?; + // "WAVE" + fmt chunk + data chunk header + samples + let riff = (4 + 8 + u64::from(fmt_len) + 8).checked_add(padded).ok_or_else(too_long)?; + let rf64 = riff > u64::from(u32::MAX); + let mut v = Vec::with_capacity(80); + if rf64 { + const DS64: u64 = 8 + 28; + v.extend_from_slice(b"RF64"); + v.extend_from_slice(&u32::MAX.to_le_bytes()); + v.extend_from_slice(b"WAVEds64"); + v.extend_from_slice(&28u32.to_le_bytes()); + v.extend_from_slice(&riff.checked_add(DS64).ok_or_else(too_long)?.to_le_bytes()); + v.extend_from_slice(&data.to_le_bytes()); + v.extend_from_slice(&frames.to_le_bytes()); + v.extend_from_slice(&0u32.to_le_bytes()); // no table entries + } else { + v.extend_from_slice(b"RIFF"); + v.extend_from_slice(&(riff as u32).to_le_bytes()); + v.extend_from_slice(b"WAVE"); + } + v.extend_from_slice(b"fmt "); + v.extend_from_slice(&fmt_len.to_le_bytes()); + v.extend_from_slice(&(if ext { 0xFFFEu16 } else { 1 }).to_le_bytes()); + v.extend_from_slice(&channels.to_le_bytes()); + v.extend_from_slice(&sample_rate.to_le_bytes()); + v.extend_from_slice(&byte_rate.to_le_bytes()); + v.extend_from_slice(&block_align.to_le_bytes()); + v.extend_from_slice(&bits.to_le_bytes()); + if ext { + // cbSize, valid bits, channel mask, KSDATAFORMAT_SUBTYPE_PCM + v.extend_from_slice(&22u16.to_le_bytes()); + v.extend_from_slice(&bits.to_le_bytes()); + let mask: u32 = if channels == 6 { 0x3F } else { 0 }; + v.extend_from_slice(&mask.to_le_bytes()); + v.extend_from_slice(&[0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, 0x80, 0x00, 0x00, 0xAA, 0x00, 0x38, 0x9B, 0x71]); + } + v.extend_from_slice(b"data"); + v.extend_from_slice(&(if rf64 { u32::MAX } else { data as u32 }).to_le_bytes()); + Ok(v) + } + PcmContainer::Aiff => { + let aiff_too_long = || "AIFF holds at most 4 GB of audio: export WAV, which has no such limit".to_string(); + let frames32 = u32::try_from(frames).map_err(|_| aiff_too_long())?; + // SSND: offset + block size + samples + let ssnd = u32::try_from(data.checked_add(8).ok_or_else(aiff_too_long)?).map_err(|_| aiff_too_long())?; + let form = (4u32 + (8 + 18)).checked_add(8).and_then(|n| n.checked_add(ssnd)).and_then(|n| n.checked_add(ssnd & 1)).ok_or_else(aiff_too_long)?; + let mut v = Vec::with_capacity(54); + v.extend_from_slice(b"FORM"); + v.extend_from_slice(&form.to_be_bytes()); + v.extend_from_slice(b"AIFFCOMM"); + v.extend_from_slice(&18u32.to_be_bytes()); + v.extend_from_slice(&channels.to_be_bytes()); + v.extend_from_slice(&frames32.to_be_bytes()); + v.extend_from_slice(&bits.to_be_bytes()); + v.extend_from_slice(&extended(sample_rate)); + v.extend_from_slice(b"SSND"); + v.extend_from_slice(&ssnd.to_be_bytes()); + v.extend_from_slice(&0u32.to_be_bytes()); // offset + v.extend_from_slice(&0u32.to_be_bytes()); // block size + Ok(v) + } + } +} + +/// Append interleaved samples as `bits`-bit PCM (little-endian for WAV, big-endian for AIFF). +pub fn encode(container: PcmContainer, interleaved: &[f32], bits: u16, out: &mut Vec) { + let bits = sample_bits(bits); + let bps = usize::from(bits / 8); + out.reserve(interleaved.len().saturating_mul(bps)); + for &s in interleaved { + let q = quantise(s, bits); + match container { + PcmContainer::Wav => out.extend_from_slice(q.to_le_bytes().get(..bps).unwrap_or_default()), + PcmContainer::Aiff => out.extend_from_slice(q.to_be_bytes().get(4 - bps..).unwrap_or_default()), + } + } +} + +/// A whole file in memory (short audio; exports stream with [`header`] and [`encode`]). +fn write_whole(container: PcmContainer, interleaved: &[f32], channels: u16, sample_rate: u32, bits: u16) -> Result, String> { + let frames = (interleaved.len() / usize::from(channels.max(1))) as u64; + let mut v = header(container, channels, sample_rate, bits, frames)?; + encode(container, interleaved, bits, &mut v); + v.extend_from_slice(pad(frames, channels, bits)); + Ok(v) +} + /// A RIFF/WAVE file of interleaved samples. More than two channels are written as /// `WAVE_FORMAT_EXTENSIBLE` with the speaker mask of the layout (6 channels: L, R, C, LFE, Ls, Rs /// = `0x3F`). -pub fn write_wav(interleaved: &[f32], channels: u16, sample_rate: u32, bits: u16) -> Vec { - let bits: u16 = if bits >= 24 { 24 } else { 16 }; - let bps = (bits / 8) as u32; - let data_len = interleaved.len() as u32 * bps; - let ext = channels > 2; - let fmt_len: u32 = if ext { 40 } else { 16 }; - let mut v = Vec::with_capacity(28 + fmt_len as usize + data_len as usize); - v.extend_from_slice(b"RIFF"); - v.extend_from_slice(&(20 + fmt_len + data_len).to_le_bytes()); - v.extend_from_slice(b"WAVEfmt "); - v.extend_from_slice(&fmt_len.to_le_bytes()); - v.extend_from_slice(&(if ext { 0xFFFEu16 } else { 1 }).to_le_bytes()); - v.extend_from_slice(&channels.to_le_bytes()); - v.extend_from_slice(&sample_rate.to_le_bytes()); - v.extend_from_slice(&(sample_rate * channels as u32 * bps).to_le_bytes()); - v.extend_from_slice(&(channels * bps as u16).to_le_bytes()); - v.extend_from_slice(&bits.to_le_bytes()); - if ext { - // cbSize, valid bits, channel mask, KSDATAFORMAT_SUBTYPE_PCM - v.extend_from_slice(&22u16.to_le_bytes()); - v.extend_from_slice(&bits.to_le_bytes()); - let mask: u32 = if channels == 6 { 0x3F } else { 0 }; - v.extend_from_slice(&mask.to_le_bytes()); - v.extend_from_slice(&[0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, 0x80, 0x00, 0x00, 0xAA, 0x00, 0x38, 0x9B, 0x71]); - } - v.extend_from_slice(b"data"); - v.extend_from_slice(&data_len.to_le_bytes()); - for &s in interleaved { - let q = quantise(s, bits).to_le_bytes(); - v.extend_from_slice(&q[..bps as usize]); - } - v +pub fn write_wav(interleaved: &[f32], channels: u16, sample_rate: u32, bits: u16) -> Result, String> { + write_whole(PcmContainer::Wav, interleaved, channels, sample_rate, bits) } /// IEEE 754 80-bit extended encoding of a sample rate (AIFF `COMM`). @@ -59,34 +157,8 @@ fn extended(rate: u32) -> [u8; 10] { } /// An AIFF file (`FORM`/`AIFF`, `COMM` + `SSND`) of interleaved samples. -pub fn write_aiff(interleaved: &[f32], channels: u16, sample_rate: u32, bits: u16) -> Vec { - let bits: u16 = if bits >= 24 { 24 } else { 16 }; - let bps = (bits / 8) as usize; - let frames = interleaved.len() / channels.max(1) as usize; - let data_len = (interleaved.len() * bps) as u32; - let ssnd_len = 8 + data_len; - let form_len = 4 + (8 + 18) + (8 + ssnd_len) + (ssnd_len & 1); - let mut v = Vec::with_capacity(form_len as usize + 8); - v.extend_from_slice(b"FORM"); - v.extend_from_slice(&form_len.to_be_bytes()); - v.extend_from_slice(b"AIFFCOMM"); - v.extend_from_slice(&18u32.to_be_bytes()); - v.extend_from_slice(&channels.to_be_bytes()); - v.extend_from_slice(&(frames as u32).to_be_bytes()); - v.extend_from_slice(&bits.to_be_bytes()); - v.extend_from_slice(&extended(sample_rate)); - v.extend_from_slice(b"SSND"); - v.extend_from_slice(&ssnd_len.to_be_bytes()); - v.extend_from_slice(&0u32.to_be_bytes()); // offset - v.extend_from_slice(&0u32.to_be_bytes()); // block size - for &s in interleaved { - let q = quantise(s, bits).to_be_bytes(); - v.extend_from_slice(&q[4 - bps..]); - } - if ssnd_len & 1 == 1 { - v.push(0); - } - v +pub fn write_aiff(interleaved: &[f32], channels: u16, sample_rate: u32, bits: u16) -> Result, String> { + write_whole(PcmContainer::Aiff, interleaved, channels, sample_rate, bits) } /// File name of frame `index` of an image sequence of `count` frames exported to `path`: @@ -126,12 +198,98 @@ mod tests { #[test] fn wav_and_aiff_layout() { let s = [0.5f32, -0.5, 1.0, -1.0]; - let w = write_wav(&s, 2, 48_000, 24); + let w = write_wav(&s, 2, 48_000, 24).unwrap(); assert_eq!(w.len(), 44 + 12); assert_eq!(&w[34..36], &24u16.to_le_bytes()); - let a = write_aiff(&s, 2, 48_000, 16); + let a = write_aiff(&s, 2, 48_000, 16).unwrap(); assert_eq!(&a[..4], b"FORM"); assert_eq!(u32::from_be_bytes(a[4..8].try_into().unwrap()) as usize, a.len() - 8); assert_eq!(&a[a.len() - 2..], &(-32767i16).to_be_bytes()); } + + fn le32(b: &[u8], at: usize) -> u32 { + u32::from_le_bytes(b[at..at + 4].try_into().unwrap()) + } + + fn le64(b: &[u8], at: usize) -> u64 { + u64::from_le_bytes(b[at..at + 8].try_into().unwrap()) + } + + /// 24-bit 5.1 at 96 kHz passes 4 GiB after about 41 minutes; the sizes overflowed `u32` + /// (a panic in debug builds, a corrupt header in release). + #[test] + fn wav_past_4_gib_is_rf64() { + let frames = 96_000u64 * 60 * 60; // an hour: 6.2 GB + let h = header(PcmContainer::Wav, 6, 96_000, 24, frames).unwrap(); + let data = frames * 6 * 3; + assert_eq!(&h[..4], b"RF64"); + assert_eq!(le32(&h, 4), u32::MAX); + assert_eq!(&h[8..16], b"WAVEds64"); + assert_eq!(le32(&h, 16), 28); + // the 64-bit RIFF size covers everything after the first 8 bytes + assert_eq!(le64(&h, 20), h.len() as u64 - 8 + data); + assert_eq!((le64(&h, 28), le64(&h, 36), le32(&h, 44)), (data, frames, 0)); + assert_eq!(&h[48..52], b"fmt "); + assert_eq!(&h[h.len() - 8..h.len() - 4], b"data"); + assert_eq!(le32(&h, h.len() - 4), u32::MAX); + } + + #[test] + fn wav_up_to_4_gib_stays_riff() { + // the longest stereo 16-bit RIFF: 36 bytes of header after the size field + let max = (u64::from(u32::MAX) - 36) / 4; + let h = header(PcmContainer::Wav, 2, 48_000, 16, max).unwrap(); + assert_eq!(&h[..4], b"RIFF"); + assert_eq!(u64::from(le32(&h, 4)), 36 + max * 4); + assert_eq!(u64::from(le32(&h, 40)), max * 4); + assert_eq!(&header(PcmContainer::Wav, 2, 48_000, 16, max + 1).unwrap()[..4], b"RF64"); + } + + #[test] + fn aiff_past_4_gib_is_refused_not_wrapped() { + let e = header(PcmContainer::Aiff, 2, 48_000, 24, 48_000 * 60 * 60 * 8).unwrap_err(); + assert!(e.contains("export WAV"), "{e}"); + assert!(header(PcmContainer::Aiff, 2, 48_000, 24, 48_000 * 60 * 60).is_ok()); + } + + #[test] + fn hostile_sizes_are_errors() { + for c in [PcmContainer::Wav, PcmContainer::Aiff] { + assert!(header(c, 2, 48_000, 24, u64::MAX).is_err()); + assert!(header(c, u16::MAX, 48_000, 24, 1).is_err()); + } + assert!(header(PcmContainer::Wav, 6, u32::MAX, 24, 1).is_err()); + } + + #[test] + fn odd_data_chunks_are_padded() { + // one 24-bit mono frame: 3 bytes of data, one pad byte, counted in the RIFF size + let w = write_wav(&[0.25], 1, 48_000, 24).unwrap(); + assert_eq!((w.len(), le32(&w, 4) as usize, le32(&w, 40)), (48, 40, 3)); + let a = write_aiff(&[0.25], 1, 48_000, 24).unwrap(); + assert_eq!(u32::from_be_bytes(a[4..8].try_into().unwrap()) as usize, a.len() - 8); + assert!(pad(2, 1, 24).is_empty()); + } + + /// A reader takes the RF64 sizes: ffprobe reports the duration from `ds64` (skipped without it). + #[test] + fn ffprobe_reads_rf64_duration() { + let dir = std::env::temp_dir().join(format!("filmcraft-rf64-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let path = dir.join("long.wav"); + let frames = 48_000u64 * 60 * 60 * 5; // 5 hours of 24-bit stereo: 5.2 GB + let mut f = header(PcmContainer::Wav, 2, 48_000, 24, frames).unwrap(); + encode(PcmContainer::Wav, &[0.0; 9_600], 24, &mut f); + std::fs::write(&path, &f).unwrap(); + let out = std::process::Command::new("ffprobe") + .args(["-v", "error", "-show_entries", "format=format_name,duration", "-of", "default=nw=1"]) + .arg(&path) + .output(); + let _ = std::fs::remove_dir_all(&dir); + let Ok(out) = out else { return }; + let text = String::from_utf8_lossy(&out.stdout); + assert!(text.contains("format_name=wav"), "{text}"); + let secs: f64 = text.lines().find_map(|l| l.strip_prefix("duration=")).and_then(|d| d.parse().ok()).unwrap_or(0.0); + assert!((secs - 18_000.0).abs() < 1.0, "{text}"); + } } diff --git a/crates/export/src/settings.rs b/crates/export/src/settings.rs index 8e8fc7ef..e56b0921 100644 --- a/crates/export/src/settings.rs +++ b/crates/export/src/settings.rs @@ -102,6 +102,9 @@ pub enum BitrateMode { Vbr1Pass, /// Variable bitrate, two passes (the picture is rendered and analysed first). Vbr2Pass, + /// Constant quality: every frame is coded at [`ExportSettings::crf`] and the bitrate follows the + /// picture (H.264, built-in encoder only). + Crf, } impl BitrateMode { @@ -110,6 +113,7 @@ impl BitrateMode { BitrateMode::Cbr => "CBR", BitrateMode::Vbr1Pass => "VBR, 1 pass", BitrateMode::Vbr2Pass => "VBR, 2 pass", + BitrateMode::Crf => "CRF (constant quality)", } } } @@ -406,16 +410,7 @@ impl ExportSettings { 6.. => 6, _ => 2, }; - Resolved { - width: w, - height: h, - rate, - sample_rate: self.audio.sample_rate.filter(|r| (8000..=192_000).contains(r)).unwrap_or(seq_sr), - channels, - target_kbps: target, - max_kbps: max, - keyint, - } + Resolved { width: w, height: h, rate, sample_rate: self.audio.sample_rate.unwrap_or(seq_sr), channels, target_kbps: target, max_kbps: max, keyint } } /// The audio codec actually used. @@ -520,6 +515,7 @@ impl ExportSettings { ); v += &match self.bitrate_mode { BitrateMode::Cbr => format!(", {}", mbps(r.target_kbps)), + BitrateMode::Crf => format!(" {}", self.crf), _ => format!(", Target {}, Max {}", mbps(r.target_kbps), mbps(r.max_kbps)), }; v += &format!(", keyframe every {} frames", r.keyint); diff --git a/crates/export/src/settings_tests.rs b/crates/export/src/settings_tests.rs index b494226b..6e8ed180 100644 --- a/crates/export/src/settings_tests.rs +++ b/crates/export/src/settings_tests.rs @@ -167,6 +167,50 @@ fn wav_and_aiff_audio_only() { } } +#[test] +fn wav_export_honours_explicit_sample_rates() { + let (project, sequence, sources) = matte([0.0, 0.0, 0.0, 1.0], 64, 36, Some(-6.0)); + let dir = Scratch::new("wav-sample-rates"); + for rate in [1, 4000, 7350, 8000, 48_000, 192_000, 192_001, 200_000, 352_800, 384_000] { + let settings = ExportSettings { + format: Format::Wav, + path: dir.path(&format!("{rate}.wav")), + audio: AudioSettings { sample_rate: Some(rate), ..Default::default() }, + range: Some(TimeRange::new(Tick::ZERO, Tick(TICKS_PER_SECOND / 4))), + ..Default::default() + }; + settings.validate().unwrap(); + let resolved = settings.resolve(64, 36, FrameRate::FPS_24, 48_000); + assert_eq!(resolved.sample_rate, rate, "requested {rate} Hz"); + export(&project, sequence, &settings, &sources, &Progress::default()).unwrap(); + let bytes = std::fs::read(&settings.path).unwrap(); + assert_eq!(u32::from_le_bytes(bytes[24..28].try_into().unwrap()), rate); + assert_eq!(u32::from_le_bytes(bytes[40..44].try_into().unwrap()), rate / 4 * 4, "quarter-second stereo 16-bit PCM"); + if let Some(probe) = ffprobe_json(&["-show_streams"], &settings.path) { + assert_eq!(probe["streams"][0]["sample_rate"], rate.to_string()); + } + } + let settings = ExportSettings { format: Format::Wav, ..Default::default() }; + assert_eq!(settings.resolve(64, 36, FrameRate::FPS_24, 44_100).sample_rate, 44_100); +} + +#[test] +fn aac_export_rejects_unsupported_explicit_sample_rates() { + let (project, sequence, sources) = matte([0.0, 0.0, 0.0, 1.0], 64, 36, Some(-6.0)); + let dir = Scratch::new("aac-sample-rates"); + for rate in [4000, 192_001, 200_000, 384_000] { + let settings = ExportSettings { + path: dir.path(&format!("{rate}.mp4")), + audio: AudioSettings { sample_rate: Some(rate), ..Default::default() }, + range: Some(TimeRange::new(Tick::ZERO, Tick(TICKS_PER_SECOND / 4))), + ..Default::default() + }; + let error = export(&project, sequence, &settings, &sources, &Progress::default()).unwrap_err(); + assert!(error.to_string().contains("sample rate"), "{error}"); + assert!(!Path::new(&settings.path).exists(), "an unsupported rate must not produce a substitute file"); + } +} + #[test] fn invalid_export_allocations_are_rejected_before_rendering() { let (project, sequence, _) = matte([1.0, 0.0, 0.0, 1.0], 64, 36, None); @@ -324,6 +368,40 @@ fn two_pass_and_cbr_h264() { } } +#[test] +fn crf_h264() { + // settings saved before CRF existed read as the default; the range is checked + let old: ExportSettings = serde_json::from_value(serde_json::json!({"format": "h264", "bitrateMode": "vbr1Pass"})).unwrap(); + assert_eq!(old.crf, DEFAULT_CRF); + let c: ExportSettings = serde_json::from_value(serde_json::json!({"bitrateMode": "crf", "crf": 18.0})).unwrap(); + assert_eq!((c.bitrate_mode, c.crf), (BitrateMode::Crf, 18.0)); + for bad in [-1.0, 51.5, f32::NAN, f32::INFINITY] { + assert!(ExportSettings { bitrate_mode: BitrateMode::Crf, crf: bad, ..Default::default() }.validate().is_err(), "{bad}"); + } + // the value only matters in CRF mode + assert!(ExportSettings { crf: f32::NAN, ..Default::default() }.validate().is_ok()); + let summary = c.summary(1920, 1080, FrameRate::FPS_25, 48_000, Tick(10 * TICKS_PER_SECOND)); + assert!(summary.video.contains("CRF (constant quality) 18"), "{}", summary.video); + + // one pass at constant quality: a lower factor spends more bits, and both decode to the picture + let (p, seq, m) = matte([0.2, 0.5, 0.8, 1.0], 320, 180, None); + let dir = Scratch::new("crf"); + let mut sizes = Vec::new(); + for crf in [10.0, 40.0] { + let path = dir.path(&format!("crf{crf}.mp4")); + let s = ExportSettings { format: Format::H264, path: path.clone(), bitrate_mode: BitrateMode::Crf, crf, include_audio: false, ..Default::default() }; + let prog = Progress::default(); + let r = export(&p, seq, &s, &m, &prog).unwrap(); + assert_eq!((r.frames, prog.total.load(Ordering::Relaxed)), (24, 24)); + let bytes: Arc<[u8]> = std::fs::read(&path).unwrap().into(); + sizes.push(bytes.len()); + let src = filmcraft_codecs::open_bytes("x.mp4", bytes).unwrap(); + let f = src.video_frame(FrameRequest::full(Tick(TICKS_PER_SECOND / 2))).unwrap().to_rgba8(); + assert!((f[2] as i32 - 204).abs() < 12 && (f[1] as i32 - 127).abs() < 12, "CRF {crf}: {:?}", &f[..4]); + } + assert!(sizes[0] > sizes[1], "CRF 10 {} bytes, CRF 40 {} bytes", sizes[0], sizes[1]); +} + #[test] fn h264_profile_level_and_keyframes() { let (p, seq, m) = matte([0.5, 0.5, 0.5, 1.0], 320, 180, None); @@ -516,6 +594,8 @@ fn h265_is_a_format_that_needs_a_registered_encoder() { let e = ExportSettings { bitrate_mode: BitrateMode::Vbr2Pass, ..h.clone() }.validate().unwrap_err(); assert!(e.to_string().contains("two-pass"), "{e}"); assert!(ExportSettings { bitrate_mode: BitrateMode::Cbr, ..h.clone() }.validate().is_ok()); + let e = ExportSettings { bitrate_mode: BitrateMode::Crf, ..h.clone() }.validate().unwrap_err(); + assert!(e.to_string().contains("CRF"), "{e}"); // formats with a built-in encoder are always available; this one only when a registered probe says so assert!(Format::ALL.iter().filter(|f| f.has_builtin_encoder()).all(|f| available(*f))); diff --git a/crates/h264enc/src/lib.rs b/crates/h264enc/src/lib.rs index c09c8cc0..e20a91d1 100644 --- a/crates/h264enc/src/lib.rs +++ b/crates/h264enc/src/lib.rs @@ -297,7 +297,60 @@ pub struct Encoder { const LOG2_MAX_FRAME_NUM: u32 = 8; const LOG2_MAX_POC_LSB: u32 = 8; +/// The bitrate a level must allow for constant quality `crf`, which has no bitrate cap. It +/// assumes the worst footage: about 3 bits per pixel at CRF 0 (FilmCraft's CRF 0 exports of real +/// camera footage measured 1.7 on average and 2.4 in their busiest second), halving every 6 steps +/// (one doubling of the quantiser step). The default CRF 23 keeps 1080p30 at level 4. +fn crf_level_kbps(crf: f32, mbw: usize, mbh: usize, fps: f64) -> u32 { + let crf = if crf.is_finite() { f64::from(crf.clamp(0.0, 51.0)) } else { 23.0 }; + let bpp = 3.0 * (-crf / 6.0).exp2(); + let pixels = (mbw as f64) * (mbh as f64) * 256.0; + // a float-to-int `as` saturates, and NaN (a hostile fps) becomes 0 + (pixels * fps * bpp / 1000.0).ceil() as u32 +} + impl Encoder { + /// The settings that shaped the stream, x264-style (`key=value` separated by spaces), for the + /// user_data_unregistered SEI: a file says how it was made (`strings`, `ffprobe -show_data`). + fn options(&self) -> String { + let c = &self.cfg; + let rate = match c.rate { + RateControl::Qp(q) => format!("rc=cqp qp={q}"), + RateControl::Crf(f) => format!("rc=crf crf={:.1}", f.clamp(0.0, 51.0)), + RateControl::Cbr { kbps } => format!("rc=cbr bitrate={kbps}"), + RateControl::Vbr { target_kbps, max_kbps } => format!("rc=vbr bitrate={target_kbps} vbv_maxrate={max_kbps}"), + }; + let pass = match c.pass { + Pass::Single => "", + Pass::First => " pass=1", + Pass::Second(_) => " pass=2", + }; + let profile = match c.profile { + Profile::Baseline => "baseline", + Profile::Main => "main", + Profile::High => "high", + }; + let preset = match c.preset { + Preset::Speed => "speed", + Preset::Balanced => "balanced", + Preset::Quality => "quality", + }; + let level = self.sps.level_idc; + format!( + "{rate}{pass} profile={profile} level={}.{} preset={preset} keyint={} bframes={} refs={} cabac={} 8x8dct={} slices={} aq={:.2} scenecut={}", + level / 10, + level % 10, + c.keyint, + c.bframes, + self.max_refs, + u8::from(c.profile != Profile::Baseline), + u8::from(c.profile == Profile::High), + self.slices, + c.aq_strength, + u8::from(c.scenecut), + ) + } + pub fn new(mut cfg: EncoderConfig) -> Result { if cfg.width == 0 || cfg.height == 0 || !cfg.width.is_multiple_of(2) || !cfg.height.is_multiple_of(2) || cfg.width > 16384 || cfg.height > 16384 { return Err(Error::InvalidDimensions(cfg.width, cfg.height)); @@ -376,7 +429,7 @@ impl Encoder { let max_refs = if cfg.bframes > 0 { 2 } else { 1 }; let (kbps_for_level, rc_kind, vbv) = match (&cfg.rate, &cfg.pass) { (RateControl::Qp(q), _) => (None, RcKind::Qp((*q).min(51)), (None, None)), - (RateControl::Crf(c), _) => (None, RcKind::Crf(c.clamp(0.0, 51.0)), (None, None)), + (RateControl::Crf(c), _) => (Some(crf_level_kbps(*c, mbw, mbh, fps)), RcKind::Crf(c.clamp(0.0, 51.0)), (None, None)), (RateControl::Cbr { kbps }, Pass::Second(st)) => (Some(*kbps), ratecontrol::RateControl::plan_two_pass(st, *kbps), (Some(*kbps), Some(*kbps))), (RateControl::Cbr { kbps }, Pass::First) => (Some(*kbps), RcKind::Abr { kbps: *kbps }, (None, None)), (RateControl::Cbr { kbps }, Pass::Single) => (Some(*kbps), RcKind::Abr { kbps: *kbps }, (Some(*kbps), Some(*kbps))), @@ -832,7 +885,7 @@ impl Encoder { } if !self.sei_sent { self.sei_sent = true; - let text = format!("filmcraft-h264enc {} - clean-room H.264 encoder", env!("CARGO_PKG_VERSION")); + let text = format!("filmcraft-h264enc {} - clean-room H.264 encoder - options: {}", env!("CARGO_PKG_VERSION"), self.options()); nals.push(nal::nal(0, NAL_SEI, &nal::sei_user_data_rbsp(&text))); if self.cfg.mastering_display.is_some() || self.cfg.content_light.is_some() { nals.push(nal::nal(0, NAL_SEI, &nal::sei_hdr_rbsp(self.cfg.mastering_display.as_ref(), self.cfg.content_light))); diff --git a/crates/h264enc/tests/crf_level_and_note.rs b/crates/h264enc/tests/crf_level_and_note.rs new file mode 100644 index 00000000..b93ebf66 --- /dev/null +++ b/crates/h264enc/tests/crf_level_and_note.rs @@ -0,0 +1,45 @@ +//! Constant quality has no bitrate cap, so the level must allow what low factors spend; and every +//! stream says which settings made it. + +mod common; + +use common::synth; +use filmcraft_h264enc::*; + +fn crf(w: u32, h: u32, fps: u32, f: f32) -> Encoder { + let mut c = EncoderConfig::new(w, h, fps, 1); + c.rate = RateControl::Crf(f); + c.format = PacketFormat::LengthPrefixed; + Encoder::new(c).unwrap() +} + +#[test] +fn crf_level_allows_the_bitrate_of_the_factor() { + let level = |w, h, fps, f| crf(w, h, fps, f).avcc()[3]; + // 1080p30: the default and visually lossless factors stay at 4; CRF 0 (measured at 105 Mb/s + // average, 147 Mb/s peak on camera footage) needs more than 4.2's 62.5 Mb/s + assert_eq!(level(1920, 1080, 30, 23.0), 40); + assert_eq!(level(1920, 1080, 30, 18.0), 40); + assert_eq!(level(1920, 1080, 30, 0.0), 51); + // never below what the picture size and rate need + assert_eq!(level(3840, 2160, 30, 51.0), 51); + // a hostile factor is treated as in range + assert_eq!(level(1920, 1080, 30, f32::NAN), 40); + assert_eq!(level(1920, 1080, 30, -100.0), 51); +} + +#[test] +fn first_access_unit_carries_the_settings() { + let (w, h) = (64, 48); + let mut enc = crf(w as u32, h as u32, 30, 18.0); + let f = synth(w, h, 0); + let mut packets = enc.encode(&f.frame(), 0).unwrap(); + packets.extend(enc.flush()); + let text = |p: &Packet| String::from_utf8_lossy(&p.data).into_owned(); + let first = text(&packets[0]); + for want in ["filmcraft-h264enc", "rc=crf crf=18.0", "profile=high", "level=", "bframes=2", "cabac=1"] { + assert!(first.contains(want), "{want} missing from {first:?}"); + } + // once per stream + assert!(packets.iter().skip(1).all(|p| !text(p).contains("filmcraft-h264enc"))); +} diff --git a/crates/interchange/src/aaf/hostile_tests.rs b/crates/interchange/src/aaf/hostile_tests.rs new file mode 100644 index 00000000..bd6dd6d6 --- /dev/null +++ b/crates/interchange/src/aaf/hostile_tests.rs @@ -0,0 +1,69 @@ +//! Hostile object trees: a crafted file must give an error, never a crash, a hang or an +//! allocation sized by a count in the file (AGENTS.md §0). + +use filmcraft_cfb::{Version, Writer}; + +use super::store::{self, SF_WEAK_SET, SF_WEAK_VECTOR, Value, utf16z}; + +const PID: u16 = 0x0007; + +/// A root object whose one property `PID` is a weak reference vector or set (stored form `sf`) +/// named `Keys`, with `index` as its `Keys index` stream. +fn weak_collection_file(sf: u16, index: Vec) -> Vec { + let name = utf16z("Keys"); + // byte order 'L', format version, 1 entry { pid, stored form, value length }, the value + let mut props = vec![0x4C, 0x20]; + for x in [1, PID, sf, name.len() as u16] { + props.extend_from_slice(&x.to_le_bytes()); + } + props.extend_from_slice(&name); + let mut w = Writer::new(Version::V3); + w.stream(Writer::ROOT, "properties", props).unwrap(); + w.stream(Writer::ROOT, "Keys index", index).unwrap(); + w.finish() +} + +/// A weak collection index: u32 count, u16 tag, u16 key pid, u8 key size, then the keys. +fn index(count: u32, size: u8, keys: &[u8]) -> Vec { + let mut v = count.to_le_bytes().to_vec(); + v.extend_from_slice(&0u16.to_le_bytes()); + v.extend_from_slice(&0x1B01u16.to_le_bytes()); + v.push(size); + v.extend_from_slice(keys); + v +} + +fn read_keys(sf: u16, index: Vec) -> Result>, String> { + match store::read(&weak_collection_file(sf, index))?.get(PID) { + Some(Value::WeakVec(_, _, keys)) => Ok(keys.clone()), + other => panic!("not a weak collection: {other:?}"), + } +} + +/// With a key size of 0 every key fits, so the count alone drove the loop: up to 4.29 billion +/// empty keys (about 100 GB) from a 9-byte index. A million is enough to show it, cheaply. +#[test] +fn zero_size_weak_collection_keys_are_an_error() { + for sf in [SF_WEAK_VECTOR, SF_WEAK_SET] { + match read_keys(sf, index(1_000_000, 0, &[])) { + Err(e) => assert!(e.contains("zero-size"), "{e}"), + Ok(keys) => panic!("read {} empty keys instead of an error", keys.len()), + } + } +} + +#[test] +fn weak_collection_count_beyond_the_index_is_an_error() { + for sf in [SF_WEAK_VECTOR, SF_WEAK_SET] { + let e = read_keys(sf, index(u32::MAX, 4, &[1, 2, 3, 4, 5, 6, 7, 8])).unwrap_err(); + assert!(e.contains("short weak collection index"), "{e}"); + } +} + +#[test] +fn weak_collections_within_the_index_are_read() { + for sf in [SF_WEAK_VECTOR, SF_WEAK_SET] { + assert_eq!(read_keys(sf, index(0, 0, &[])).unwrap(), Vec::>::new()); + assert_eq!(read_keys(sf, index(2, 4, &[1, 2, 3, 4, 5, 6, 7, 8])).unwrap(), [vec![1, 2, 3, 4], vec![5, 6, 7, 8]]); + } +} diff --git a/crates/interchange/src/aaf/mod.rs b/crates/interchange/src/aaf/mod.rs index 707c6a43..d8896b3d 100644 --- a/crates/interchange/src/aaf/mod.rs +++ b/crates/interchange/src/aaf/mod.rs @@ -86,5 +86,7 @@ pub fn sniff(bytes: &[u8]) -> bool { } } +#[cfg(test)] +mod hostile_tests; #[cfg(test)] mod tests; diff --git a/crates/interchange/src/aaf/store.rs b/crates/interchange/src/aaf/store.rs index 69b17e88..02234c46 100644 --- a/crates/interchange/src/aaf/store.rs +++ b/crates/interchange/src/aaf/store.rs @@ -480,6 +480,13 @@ impl R<'_> { return Err(bad("short weak collection index")); }; let size = size as usize; + if count > 0 && size == 0 { + return Err(bad("zero-size weak collection key")); + } + // every key must lie in the index, so its length (not the count) bounds the loop + if (count as usize).checked_mul(size).and_then(|n| n.checked_add(9)).is_none_or(|end| end > idx.len()) { + return Err(bad("short weak collection index")); + } let mut keys = Vec::new(); for k in 0..count as usize { keys.push(idx.get(9 + k * size..9 + (k + 1) * size).ok_or_else(|| bad("short weak collection index"))?.to_vec()); diff --git a/crates/interchange/src/edl.rs b/crates/interchange/src/edl.rs index 9774c006..ecee0391 100644 --- a/crates/interchange/src/edl.rs +++ b/crates/interchange/src/edl.rs @@ -467,7 +467,10 @@ fn import_event(ctx: &mut Ctx, ev: &Event, report: &mut Report) { let t0 = ctx.rec(l2.tcs[2], df); let t1 = ctx.rec(l2.tcs[3], df); let d = ctx.rate.tick_of(l2.dur); - let effect = transition_for(l2.trans, ev.effect.as_deref(), report); + // Resolve separately for each track kind: named audio dissolves have + // different definitions from their video counterparts. + let mut video_effect = None; + let mut audio_effect = None; let to_black = l2.reel.eq_ignore_ascii_case("BL"); let from_black = l1.reel.eq_ignore_ascii_case("BL"); // Incoming clip. @@ -505,10 +508,8 @@ fn import_event(ctx: &mut Ctx, ev: &Event, report: &mut Report) { continue; } let audio = kind == TrackKind::Audio; - let mut eff = effect.clone(); - if audio && eff.def().is_some_and(|d| d.kind != filmcraft_project::EffectKind::AudioTransition) { - eff = find_effect("constant_power").map(|d| d.instance()).unwrap_or(eff); - } + let cache = if audio { &mut audio_effect } else { &mut video_effect }; + let eff = cache.get_or_insert_with(|| transition_for(l2.trans, ev.effect.as_deref(), audio, report)).clone(); let id = TransitionId(ctx.b.alloc()); let track = &mut ctx.seq.tracks_mut(kind)[idx]; track.transitions.push(Transition { id, effect: eff, start: t0, duration: d, from, to, align, reverse: false }); @@ -564,9 +565,18 @@ fn lay_cut(ctx: &mut Ctx, ev: &Event, l: &Line, video_base: usize, df: bool, m2: ctx.lay(l, item, &name, t0, t1 - t0, src, *m2, video_base); } -fn transition_for(t: Trans, effect_name: Option<&str>, report: &mut Report) -> filmcraft_project::EffectInstance { +fn transition_for(t: Trans, effect_name: Option<&str>, audio: bool, report: &mut Report) -> filmcraft_project::EffectInstance { if let Some(n) = effect_name { - return transition_effect(n, false, report); + let effect = transition_effect(n, audio, report); + if audio && effect.def().is_some_and(|d| d.kind != filmcraft_project::EffectKind::AudioTransition) { + report.warn(format!("transition \"{n}\" is not an audio transition; imported as Constant Power")); + return find_effect("constant_power").map(|d| d.instance()).unwrap_or(effect); + } + return effect; + } + if audio { + // Unnamed EDL audio dissolves default to constant-power fading. + return transition_effect("Constant Power", true, report); } match t { Trans::Wipe(code) => { diff --git a/crates/interchange/tests/edl.rs b/crates/interchange/tests/edl.rs index c1c21475..14100d14 100644 --- a/crates/interchange/tests/edl.rs +++ b/crates/interchange/tests/edl.rs @@ -99,6 +99,51 @@ fn golden_drop_frame_dissolves() { assert!(gm.as_media().unwrap().offline); } +#[test] +fn named_edl_audio_dissolves_keep_their_effects() { + let fixture = "TITLE: AudioDissolve +FCM: NON-DROP FRAME + +001 AX A C 00:00:00:00 00:00:02:00 01:00:00:00 01:00:02:00 +* FROM CLIP NAME: A + +002 AX A C 00:00:02:00 00:00:02:00 01:00:02:00 01:00:02:00 +002 AX A D 024 00:00:02:00 00:00:04:00 01:00:02:00 01:00:04:00 +* TO CLIP NAME: B +"; + let opts = ImportOptions { edl_frame_rate: Some(FrameRate::FPS_24), ..Default::default() }; + for (name, expected, warning) in [ + (Some("CONSTANT GAIN"), "constant_gain", false), + (Some("EXPONENTIAL FADE"), "exponential_fade", false), + (Some("CONSTANT POWER"), "constant_power", false), + (Some("UNRECOGNIZED TRANSITION"), "constant_power", true), + (None, "constant_power", false), + ] { + let text = format!("{fixture}{}", name.map(|name| format!("* EFFECT NAME: {name}\n")).unwrap_or_default()); + let (imp, rep) = import_with(text.as_bytes(), Format::Edl, &opts).unwrap(); + let seq = imp.project.sequence(only_seq(&imp)).unwrap(); + let track = &seq.audio_tracks[0]; + assert_eq!(track.transitions.len(), 1, "{name:?}"); + let tr = &track.transitions[0]; + assert_eq!(tr.effect.effect, expected, "{name:?}: {rep}"); + assert_eq!(tr.start, f(FrameRate::FPS_24, 48), "{name:?}"); + assert_eq!(tr.duration, f(FrameRate::FPS_24, 24), "{name:?}"); + assert_eq!(tr.align, TransitionAlign::StartAtCut, "{name:?}"); + assert!(tr.from.is_some() && tr.to.is_some(), "{name:?}"); + assert_eq!(rep.mentions("not supported"), warning, "{name:?}: {rep}"); + if warning { + assert!(rep.mentions("Constant Power"), "{rep}"); + } + } + + // The same event with video channels retains the video-only Cross Dissolve. + let video = fixture.replace("AX A", "AX V") + "* EFFECT NAME: CROSS DISSOLVE\n"; + let (imp, rep) = import_with(video.as_bytes(), Format::Edl, &opts).unwrap(); + let seq = imp.project.sequence(only_seq(&imp)).unwrap(); + assert_eq!(seq.video_tracks[0].transitions[0].effect.effect, "cross_dissolve"); + assert!(!rep.mentions("not supported"), "{rep}"); +} + #[test] fn explicit_rate_and_wipes_and_keys() { let edl = "TITLE: Keys diff --git a/crates/matroska/src/demux.rs b/crates/matroska/src/demux.rs index 9344059c..4278a1b8 100644 --- a/crates/matroska/src/demux.rs +++ b/crates/matroska/src/demux.rs @@ -301,6 +301,9 @@ struct BlockRef { discardable: bool, invisible: bool, discard_padding: Option, + /// (offset, size) of the `BlockAdditional` with `BlockAddID` 1 (WebM: the VP8/VP9 alpha + /// layer), kept for unlaced blocks only. + addition: Option<(u64, u32)>, } #[derive(Clone, Copy, Debug)] @@ -318,6 +321,7 @@ struct FrameRef { discard_padding: Option, block_offset: u64, lace: u16, + addition: Option<(u64, u32)>, } fn track_lookup(tracks: &[Track]) -> impl Fn(u64) -> Option + '_ { @@ -354,6 +358,36 @@ fn parse_block( Ok(Some((track, rel, flags, frames))) } +/// The `BlockAdditional` with `BlockAddID` 1 (the default ID) in a `BlockAdditions` element: +/// (offset, size) of its data. Other IDs, empty additions and damaged children are skipped. +fn block_addition(io: &mut Io<'_, S>, start: u64, end: u64) -> Result> { + let mut pos = start; + while pos < end { + let Some(more) = io.header(pos, end)? else { break }; + let me = more.end(end); + if more.id == BLOCK_MORE { + let (mut id, mut data) = (1u64, None); + let mut p = more.data_start(); + while p < me { + let Some(c) = io.header(p, me)? else { break }; + let ce = c.end(me); + let len = ce.saturating_sub(c.data_start()); + match c.id { + BLOCK_ADD_ID => id = ebml::uint(io.peek(c.data_start(), len.min(8) as usize)?), + BLOCK_ADDITIONAL => data = u32::try_from(len).ok().filter(|&n| n > 0).map(|n| (c.data_start(), n)), + _ => {} + } + p = ce; + } + if id == 1 && data.is_some() { + return Ok(data); + } + } + pos = me; + } + Ok(None) +} + /// Parse one SimpleBlock or BlockGroup element. fn parse_block_elem( io: &mut Io<'_, S>, @@ -376,17 +410,19 @@ fn parse_block_elem( discardable: flags & 0x01 != 0, invisible: flags & 0x08 != 0, discard_padding: None, + addition: None, })) } BLOCK_GROUP => { let mut pos = ds; let mut block = None; - let (mut dur, mut has_ref, mut pad) = (None, false, None); + let (mut dur, mut has_ref, mut pad, mut addition) = (None, false, None, None); while pos < end { let Some(c) = io.header(pos, end)? else { break }; let ce = c.end(end); match c.id { BLOCK => block = parse_block(io, c.data_start(), ce - c.data_start(), lookup)?, + BLOCK_ADDITIONS => addition = block_addition(io, c.data_start(), ce)?, BLOCK_DURATION => dur = Some(ebml::uint(io.peek(c.data_start(), (ce - c.data_start()) as usize)?)), REFERENCE_BLOCK => has_ref = true, DISCARD_PADDING => pad = Some(ebml::int(io.peek(c.data_start(), (ce - c.data_start()) as usize)?)), @@ -395,6 +431,8 @@ fn parse_block_elem( pos = ce; } let Some((track, rel, flags, frames)) = block else { return Ok(None) }; + // with lacing, an addition belongs to the block, not to one of its frames + let addition = addition.filter(|_| frames.len() == 1); Ok(Some(BlockRef { track, block_offset: h.pos, @@ -405,6 +443,7 @@ fn parse_block_elem( discardable: false, invisible: flags & 0x08 != 0, discard_padding: pad, + addition, })) } _ => Ok(None), @@ -450,6 +489,7 @@ fn expand(b: &BlockRef, t: &Track, scale: u64, skip: u16, out: &mut impl Extend< discard_padding: b.discard_padding, block_offset: b.block_offset, lace: i as u16, + addition: b.addition, } })); } @@ -780,6 +820,7 @@ fn add_blocks(file: &mut MkvFile, cluster: u32, blocks: &[BlockRef]) { cluster, block_offset: f.block_offset, lace: f.lace, + addition: f.addition, }); } } @@ -838,6 +879,20 @@ impl MkvFile { Ok(data) } + /// The `BlockAdditional` (ID 1) of sample `index` of `track`, `None` when the block has none + /// (see [`Sample::addition`]). Stored as is: header stripping applies to frames only. + pub fn read_sample_addition(&self, src: &S, track: usize, index: usize) -> Result>> { + let t = self.tracks.get(track).ok_or(Error::NoSuchTrack(track))?; + let s = t.samples.get(index).ok_or(Error::NoSuchSample(index))?; + let Some((offset, size)) = s.addition else { return Ok(None) }; + if offset.checked_add(size as u64).is_none_or(|end| end > src.len()) { + return Err(Error::Invalid("block addition past end of file".into())); + } + let mut data = vec![0; size as usize]; + src.read_at(offset, &mut data)?; + Ok(Some(data)) + } + /// Keyframe sample of `track` with the greatest pts ≤ `time_ns` (indexed files only). pub fn keyframe_before(&self, track: usize, time_ns: i64) -> Option { let t = self.tracks.get(track)?; diff --git a/crates/matroska/src/ids.rs b/crates/matroska/src/ids.rs index 6334f451..3b7e025c 100644 --- a/crates/matroska/src/ids.rs +++ b/crates/matroska/src/ids.rs @@ -55,6 +55,9 @@ pub const ENCRYPTED_BLOCK: u32 = 0xAF; pub const SILENT_TRACKS: u32 = 0x5854; pub const BLOCK: u32 = 0xA1; pub const BLOCK_ADDITIONS: u32 = 0x75A1; +pub const BLOCK_MORE: u32 = 0xA6; +pub const BLOCK_ADD_ID: u32 = 0xEE; +pub const BLOCK_ADDITIONAL: u32 = 0xA5; pub const BLOCK_DURATION: u32 = 0x9B; pub const REFERENCE_PRIORITY: u32 = 0xFA; pub const REFERENCE_BLOCK: u32 = 0xFB; diff --git a/crates/matroska/src/track.rs b/crates/matroska/src/track.rs index af8af356..1288895f 100644 --- a/crates/matroska/src/track.rs +++ b/crates/matroska/src/track.rs @@ -191,6 +191,10 @@ pub struct Sample { pub block_offset: u64, /// Index of this frame within a laced block. pub lace: u16, + /// (offset, size) of the block's `BlockAdditional` with `BlockAddID` 1, when it has one and + /// is not laced. In WebM this is the alpha layer of a VP8/VP9 track with `AlphaMode` 1: a + /// second, independently coded stream whose luma plane is the frame's alpha. + pub addition: Option<(u64, u32)>, } /// A Matroska track (`TrackEntry`) plus, once indexed, its sample table. diff --git a/crates/media/src/cancel.rs b/crates/media/src/cancel.rs index bfb391dd..5c2b5f28 100644 --- a/crates/media/src/cancel.rs +++ b/crates/media/src/cancel.rs @@ -96,10 +96,64 @@ pub fn cancelled() -> bool { CURRENT.with(|c| c.borrow().as_ref().is_some_and(|f| f.load(Ordering::Relaxed))) } +/// The hints of the work running on this thread ([`with_cancel`], [`with_catch_up`], +/// [`with_draft`], [`with_background`]), to carry them to another thread doing part of that work. +#[derive(Clone, Default)] +pub struct Context { + cancel: Option>, + catch_up: Option, + draft: bool, + background: bool, +} + +impl Context { + /// This thread's hints. + pub fn current() -> Self { + Context { cancel: CURRENT.with(|c| c.borrow().clone()), catch_up: catch_up(), draft: draft(), background: background() } + } + + /// Run `f` with these hints, restoring this thread's however `f` ends (a pool thread must + /// not keep another job's hints after a panic). + pub fn run(self, f: impl FnOnce() -> R) -> R { + struct Restore(Context); + impl Drop for Restore { + fn drop(&mut self) { + let prev = std::mem::take(&mut self.0); + CURRENT.with(|c| *c.borrow_mut() = prev.cancel); + CATCH_UP.with(|c| c.set(prev.catch_up)); + DRAFT.with(|c| c.set(prev.draft)); + BACKGROUND.with(|c| c.set(prev.background)); + } + } + let _restore = Restore(Context::current()); + CURRENT.with(|c| *c.borrow_mut() = self.cancel); + CATCH_UP.with(|c| c.set(self.catch_up)); + DRAFT.with(|c| c.set(self.draft)); + BACKGROUND.with(|c| c.set(self.background)); + f() + } +} + #[cfg(test)] mod tests { use super::*; + #[test] + fn context_carries_the_hints_to_another_thread_and_restores_its_own() { + let flag = Arc::new(AtomicBool::new(true)); + let ctx = with_cancel(&flag, || with_draft(true, || with_background(true, || with_catch_up(Some(Tick(5)), Context::current)))); + std::thread::scope(|s| { + s.spawn(|| { + let seen = with_draft(false, || ctx.clone().run(|| (cancelled(), draft(), background(), catch_up()))); + assert_eq!(seen, (true, true, true, Some(Tick(5)))); + // a panic inside leaves this thread's own hints in place + let r = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| ctx.clone().run(|| panic!("job")))); + assert!(r.is_err()); + assert_eq!((cancelled(), draft(), background(), catch_up()), (false, false, false, None)); + }); + }); + } + #[test] fn flag_is_scoped_to_the_closure() { let flag = Arc::new(AtomicBool::new(false)); diff --git a/crates/media/src/reader.rs b/crates/media/src/reader.rs index 2aeb47bf..c3852edb 100644 --- a/crates/media/src/reader.rs +++ b/crates/media/src/reader.rs @@ -2,8 +2,9 @@ //! //! A [`ByteReader`] serves reads at any offset (a native file, a web `Blob` read in chunks, an //! in-memory buffer). Container openers that only need the index and then single samples (MP4/MOV, -//! Matroska) open from a reader ([`ReaderOpener`]); formats decoded in one go (stills, WAV, -//! standalone compressed audio) fall back to reading the whole file through it. +//! Matroska) open from a reader ([`ReaderOpener`]), as does WAV (header now, sample ranges on +//! demand); formats decoded in one go (stills, standalone compressed audio) fall back to reading +//! the whole file through it. use std::io; use std::sync::Arc; @@ -43,24 +44,131 @@ impl ByteReader for MemReader { } } -/// A file on disk, read in place: the file stays open and only the requested ranges are read, so -/// opening a clip costs its index rather than its size. +/// Files [`FileReader`]s keep open at most, together (#67). Each open media source used to hold +/// its own descriptor, so a project with hundreds of sources hit the soft descriptor limit (256 on +/// macOS, 1024 on Linux) and the rest showed as failed or offline. Past this many, the least +/// recently read file is closed and its reader reopens it on demand. +#[cfg(any(unix, windows))] +pub const MAX_OPEN_FILES: usize = 128; + +/// The open files of a set of [`FileReader`]s, least recently used first. +#[cfg(any(unix, windows))] +#[derive(Debug)] +pub(crate) struct FilePool { + cap: usize, + state: std::sync::Mutex, +} + +#[cfg(any(unix, windows))] +#[derive(Debug, Default)] +struct PoolState { + next_id: u64, + open: std::collections::VecDeque<(u64, Arc)>, +} + +#[cfg(any(unix, windows))] +impl FilePool { + pub(crate) fn new(cap: usize) -> Self { + Self { cap: cap.max(1), state: Default::default() } + } + + fn state(&self) -> std::sync::MutexGuard<'_, PoolState> { + self.state.lock().unwrap_or_else(std::sync::PoisonError::into_inner) + } + + fn next_id(&self) -> u64 { + let mut s = self.state(); + s.next_id = s.next_id.wrapping_add(1); + s.next_id + } + + /// The open file of reader `id`, now the most recently used. + fn get(&self, id: u64) -> Option> { + let mut s = self.state(); + let at = s.open.iter().position(|(i, _)| *i == id)?; + let entry = s.open.remove(at)?; + let file = entry.1.clone(); + s.open.push_back(entry); + Some(file) + } + + /// Keep `file` open for reader `id`, closing the least recently used files past the cap. A + /// file another thread is reading from stays open until that read ends. + fn insert(&self, id: u64, file: std::fs::File) -> Arc { + let mut s = self.state(); + if let Some((_, open)) = s.open.iter().find(|(i, _)| *i == id) { + return open.clone(); // another thread reopened it first + } + let file = Arc::new(file); + s.open.push_back((id, file.clone())); + while s.open.len() > self.cap { + s.open.pop_front(); + } + file + } + + fn remove(&self, id: u64) { + self.state().open.retain(|(i, _)| *i != id); + } + + #[cfg(test)] + fn open_count(&self) -> usize { + self.state().open.len() + } +} + +#[cfg(any(unix, windows))] +static FILE_POOL: std::sync::LazyLock> = std::sync::LazyLock::new(|| Arc::new(FilePool::new(MAX_OPEN_FILES))); + +/// A file on disk, read in place: only the requested ranges are read, so opening a clip costs its +/// index rather than its size. The file is kept open in a shared pool of at most +/// [`MAX_OPEN_FILES`] and reopened by path when the pool closed it. #[cfg(any(unix, windows))] #[derive(Debug)] pub struct FileReader { - file: std::fs::File, + path: std::path::PathBuf, len: u64, + id: u64, + pool: Arc, } #[cfg(any(unix, windows))] impl FileReader { pub fn open(path: &std::path::Path) -> io::Result { + Self::open_in(path, &FILE_POOL) + } + + pub(crate) fn open_in(path: &std::path::Path, pool: &Arc) -> io::Result { let file = std::fs::File::open(path)?; let m = file.metadata()?; if !m.is_file() { return Err(io::Error::new(io::ErrorKind::NotFound, format!("{} is not a file", path.display()))); } - Ok(Self { file, len: m.len() }) + let id = pool.next_id(); + pool.insert(id, file); + Ok(Self { path: path.to_path_buf(), len: m.len(), id, pool: pool.clone() }) + } + + /// The open file, reopened when the pool closed it. A file now shorter than when it was opened + /// (replaced or truncated) is refused rather than read as the old one; a longer one (still + /// being written) is read up to the opened length, as before. + fn file(&self) -> io::Result> { + if let Some(file) = self.pool.get(self.id) { + return Ok(file); + } + let file = std::fs::File::open(&self.path)?; + let m = file.metadata()?; + if !m.is_file() || m.len() < self.len { + return Err(io::Error::other(format!("{} changed since it was opened", self.path.display()))); + } + Ok(self.pool.insert(self.id, file)) + } +} + +#[cfg(any(unix, windows))] +impl Drop for FileReader { + fn drop(&mut self) { + self.pool.remove(self.id); } } @@ -71,15 +179,16 @@ impl ByteReader for FileReader { } /// Positional reads: threads decoding different parts of the file do not share a cursor. fn read_at(&self, offset: u64, buf: &mut [u8]) -> io::Result<()> { + let file = self.file()?; #[cfg(unix)] { - std::os::unix::fs::FileExt::read_exact_at(&self.file, buf, offset) + std::os::unix::fs::FileExt::read_exact_at(&*file, buf, offset) } #[cfg(windows)] { let (mut done, mut at) = (0usize, offset); while let Some(rest) = buf.get_mut(done..).filter(|r| !r.is_empty()) { - match std::os::windows::fs::FileExt::seek_read(&self.file, rest, at) { + match std::os::windows::fs::FileExt::seek_read(&*file, rest, at) { Ok(0) => return Err(eof()), Ok(n) => { done = done.saturating_add(n); @@ -109,15 +218,15 @@ pub type ReaderOpener = fn(name: &str, head: &[u8], reader: &SharedReader) -> Op /// Bytes sniffed by [`open_reader`]. pub const HEAD_LEN: usize = 64 * 1024; -/// Open media from a reader: reader openers first; otherwise read the whole file and use the -/// byte openers (`extra`, then stills / WAV). +/// Open media from a reader: reader openers first, then WAV (read in place); otherwise read the +/// whole file and use the byte openers (`extra`, then stills). pub fn open_reader(name: &str, reader: SharedReader, reader_openers: &[ReaderOpener], extra: &[Opener]) -> Result { open_reader_within(name, reader, reader_openers, extra, u64::MAX) } /// Like [`open_reader`], but a file no reader opener takes is read whole only up to `max_whole` /// bytes; a larger one is refused instead. For looking at files rather than importing them (the -/// Media Browser's properties and thumbnails, #157): reading a multi-gigabyte AVI or WAV into +/// Media Browser's properties and thumbnails, #157): reading a multi-gigabyte AVI into /// memory to show its duration, or to find out it isn't supported, thrashes the disk. pub fn open_reader_within(name: &str, reader: SharedReader, reader_openers: &[ReaderOpener], extra: &[Opener], max_whole: u64) -> Result { let head = read_range(&*reader, 0, HEAD_LEN).map_err(|e| MediaError::Io(format!("{name}: {e}")))?; @@ -126,6 +235,9 @@ pub fn open_reader_within(name: &str, reader: SharedReader, reader_openers: &[Re return r; } } + if crate::wav::sniff(&head) { + return Ok(Arc::new(crate::wav::WavSource::open(name, reader)?)); + } if reader.len() > max_whole { return Err(MediaError::Unsupported(format!("{name}: no streaming reader for this format, and it is too large to read whole here"))); } @@ -169,6 +281,37 @@ mod tests { std::fs::remove_dir_all(&dir).unwrap(); } + /// #67: more readers than the pool keeps open stay readable; past the cap the least recently + /// read file is closed and reopened on demand, and a file truncated meanwhile is refused. + #[cfg(any(unix, windows))] + #[test] + fn more_file_readers_than_the_pool_holds_stay_readable() { + let dir = std::env::temp_dir().join(format!("filmcraft-file-pool-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let pool = Arc::new(FilePool::new(2)); + let paths: Vec<_> = (0..5).map(|i| dir.join(format!("{i}.bin"))).collect(); + for (i, p) in paths.iter().enumerate() { + std::fs::write(p, format!("file{i}-0123456789")).unwrap(); + } + let readers: Vec<_> = paths.iter().map(|p| FileReader::open_in(p, &pool).unwrap()).collect(); + assert_eq!(pool.open_count(), 2); + for _ in 0..2 { + for (i, r) in readers.iter().enumerate() { + let mut b = [0u8; 5]; + r.read_at(0, &mut b).unwrap(); + assert_eq!(b, *format!("file{i}").as_bytes()); + assert!(pool.open_count() <= 2); + } + } + // reader 0 was closed by the later reads; its file is now shorter than when opened + std::fs::write(&paths[0], b"short").unwrap(); + let mut b = [0u8; 5]; + assert!(readers[0].read_at(0, &mut b).is_err()); + drop(readers); + assert_eq!(pool.open_count(), 0, "dropped readers close their files"); + std::fs::remove_dir_all(&dir).unwrap(); + } + /// A reader that counts the bytes read from it. struct Counting(MemReader, std::sync::atomic::AtomicU64); impl ByteReader for Counting { @@ -181,35 +324,58 @@ mod tests { } } - /// #157: looking at a file no streaming reader takes (an AVI, a large WAV) read all of it. - /// Within a limit it reads the head only and refuses; a small file still opens. + /// #157: looking at a file no streaming reader takes (an AVI) read all of it. Within a limit + /// it reads the head only and refuses. #[test] fn open_within_reads_only_the_head_of_large_unstreamable_files() { - let mut wav = crate::wav::write_wav16(&vec![0.25; 48_000 * 2], 2, 48_000); - let small = wav.len() as u64; - let r = Arc::new(Counting(MemReader(Arc::from(wav.clone())), Default::default())); - let src = open_reader_within("a.wav", r.clone(), &[], &[], small).unwrap(); - assert_eq!(src.info().audio().unwrap().channels, 2); - // the same file over the limit: refused after reading the sniffing head only - let r = Arc::new(Counting(MemReader(Arc::from(wav.clone())), Default::default())); - let e = open_reader_within("a.wav", r.clone(), &[], &[], small - 1).err().unwrap(); + let mut avi = vec![0u8; 4 * HEAD_LEN]; + avi[..4].copy_from_slice(b"RIFF"); + avi[8..12].copy_from_slice(b"AVI "); + let r = Arc::new(Counting(MemReader(Arc::from(avi)), Default::default())); + let e = open_reader_within("a.avi", r.clone(), &[], &[], HEAD_LEN as u64).err().unwrap(); assert!(matches!(e, MediaError::Unsupported(_)), "{e:?}"); assert!(r.1.load(std::sync::atomic::Ordering::SeqCst) <= HEAD_LEN as u64); - // an unsupported format is refused without being read whole either - wav.resize(4 * HEAD_LEN, 0); - wav[..4].copy_from_slice(b"RIFF"); - wav[8..12].copy_from_slice(b"AVI "); + } + + /// #279: a WAV is read in place, never whole: opening it reads the sniffing head and its chunk + /// headers, and audio reads only the requested samples, however large the file (and over the + /// Media Browser's whole-file limit too). + #[test] + fn wav_opens_in_place_and_reads_only_requested_samples() { + let rate = 48_000usize; + let samples: Vec = (0..rate * 2 * 20).map(|i| ((i / 2) % 100) as f32 / 200.0).collect(); + let wav = crate::wav::write_wav16(&samples, 2, rate as u32); + let size = wav.len() as u64; + assert!(size > 3 * HEAD_LEN as u64); let r = Arc::new(Counting(MemReader(Arc::from(wav)), Default::default())); - assert!(open_reader_within("a.avi", r.clone(), &[], &[], HEAD_LEN as u64).is_err()); - assert!(r.1.load(std::sync::atomic::Ordering::SeqCst) <= HEAD_LEN as u64); + let read = || r.1.load(std::sync::atomic::Ordering::SeqCst); + let src = open_reader_within("a.wav", r.clone(), &[], &[], HEAD_LEN as u64).unwrap(); + assert_eq!(src.info().audio().unwrap().channels, 2); + assert_eq!(src.info().duration, filmcraft_time::Tick::from_units(20, 1)); + assert_eq!(src.info().file_size, Some(size)); + let after_open = read(); + assert!(after_open <= HEAD_LEN as u64 + 64, "{after_open}"); + // 10 ms from the middle: about that much is read, and the samples are right + let a = src.audio(rate as i64 * 10 + 7, 480, rate as u32).unwrap(); + assert!(read() - after_open <= 481 * 4, "{}", read() - after_open); + let frame = rate * 10 + 7 + 3; + assert!((a.channels[0][3] - samples[frame * 2]).abs() < 1e-4); + assert!((a.channels[1][3] - samples[frame * 2 + 1]).abs() < 1e-4); + // past the end: silence, nothing read + let before = read(); + let a = src.audio(rate as i64 * 30, 16, rate as u32).unwrap(); + assert!(a.channels[0].iter().all(|&s| s == 0.0)); + assert_eq!(read(), before); } #[test] fn falls_back_to_byte_openers() { - // a WAV is opened through the whole-file path - let wav = crate::wav::write_wav16(&[0.0, 0.5, -0.5, 0.25], 2, 48_000); - let r: SharedReader = Arc::new(MemReader(wav.into())); - let s = open_reader("a.wav", r, &[], &[]).unwrap(); - assert!(s.info().has_audio()); + // a format without a reader opener is read whole and handed to the byte openers + fn whole(_: &str, bytes: Arc<[u8]>) -> Option> { + Some(Err(MediaError::Unsupported(format!("got {} bytes", bytes.len())))) + } + let r: SharedReader = Arc::new(MemReader(Arc::from(&b"bytes"[..]))); + let e = open_reader("a.xyz", r, &[], &[whole]).err().unwrap(); + assert!(matches!(&e, MediaError::Unsupported(m) if m == "got 5 bytes"), "{e:?}"); } } diff --git a/crates/media/src/wav.rs b/crates/media/src/wav.rs index 6b63bf4a..4a0df3d2 100644 --- a/crates/media/src/wav.rs +++ b/crates/media/src/wav.rs @@ -7,17 +7,32 @@ use std::sync::Arc; use filmcraft_frame::{AudioBuffer, VideoFrame}; use filmcraft_time::Tick; +use crate::reader::{MemReader, SharedReader, read_range}; use crate::{AudioStreamInfo, FrameRequest, MediaError, MediaInfo, MediaKind, MediaSource, Result}; pub fn sniff(b: &[u8]) -> bool { b.len() >= 12 && (&b[0..4] == b"RIFF" || &b[0..4] == b"RF64") && &b[8..12] == b"WAVE" } +/// Chunks walked looking for `fmt `/`data`/`bext`/`iXML` (real files have a handful). +const MAX_CHUNKS: usize = 1 << 16; +/// Bytes of an iXML chunk searched for the time reference. +const IXML_MAX: u64 = 1 << 20; +/// Output frames converted per read of the data chunk. +const BLOCK_FRAMES: usize = 4096; +/// Largest single read of the data chunk; a wider span (an extreme resampling ratio) reads the +/// two frames each output frame needs instead. +const MAX_SPAN_BYTES: u64 = 4 << 20; + +/// A WAV file read in place: opening it reads its chunk headers (not its samples), and +/// [`MediaSource::audio`] reads only the sample range it is asked for, so a 90-minute recording +/// costs its header rather than its size (#279). pub struct WavSource { info: MediaInfo, - bytes: Arc<[u8]>, - data_off: usize, - data_len: usize, + reader: SharedReader, + data_off: u64, + /// Whole frames only. + frames: u64, channels: usize, bits: u16, float: bool, @@ -26,63 +41,80 @@ pub struct WavSource { } fn le16(b: &[u8], o: usize) -> u16 { - u16::from_le_bytes([b[o], b[o + 1]]) + b.get(o..o.saturating_add(2)).and_then(|s| s.try_into().ok()).map_or(0, u16::from_le_bytes) } fn le32(b: &[u8], o: usize) -> u32 { - u32::from_le_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]]) + b.get(o..o.saturating_add(4)).and_then(|s| s.try_into().ok()).map_or(0, u32::from_le_bytes) } impl WavSource { + /// Parse a WAV file held in memory. pub fn parse(name: &str, bytes: Arc<[u8]>) -> Result { - let b = &bytes[..]; - let mut pos = 12; + Self::open(name, Arc::new(MemReader(bytes))) + } + + /// Open a WAV file through a random-access reader. + pub fn open(name: &str, reader: SharedReader) -> Result { + let io = |e: std::io::Error| MediaError::Io(format!("{name}: {e}")); + let len = reader.len(); + let mut pos: u64 = 12; let mut fmt = None; let mut data = None; let mut bext_tr = None; let mut ixml_tr = None; - while pos + 8 <= b.len() { - let id = &b[pos..pos + 4]; - let mut len = le32(b, pos + 4) as usize; - let body = pos + 8; - if id == b"data" && (len == 0xFFFF_FFFF || body + len > b.len()) { - len = b.len() - body; + for _ in 0..MAX_CHUNKS { + let Some(body) = pos.checked_add(8).filter(|&b| b <= len) else { break }; + let mut hdr = [0u8; 8]; + reader.read_at(pos, &mut hdr).map_err(io)?; + let id = &hdr[0..4]; + let mut clen = u64::from(le32(&hdr, 4)); + let rest = len - body; + if id == b"data" && (clen == 0xFFFF_FFFF || clen > rest) { + clen = rest; } - if body + len > b.len() { + if clen > rest { break; } match id { - b"fmt " if len >= 16 => { - let mut tag = le16(b, body); - let ch = le16(b, body + 2); - let rate = le32(b, body + 4); - let bits = le16(b, body + 14); - if tag == 0xFFFE && len >= 40 { - tag = le16(b, body + 24); + b"fmt " if clen >= 16 => { + let b = read_range(&*reader, body, clen.min(40) as usize).map_err(io)?; + let mut tag = le16(&b, 0); + let ch = le16(&b, 2); + let rate = le32(&b, 4); + let bits = le16(&b, 14); + if tag == 0xFFFE && clen >= 40 { + tag = le16(&b, 24); } fmt = Some((tag, ch, rate, bits)); } - b"data" => data = Some((body, len)), + b"data" => data = Some((body, clen)), // bext: Description 256, Originator 32, OriginatorReference 32, OriginationDate 10, // OriginationTime 8, then TimeReference (low u32, high u32) - b"bext" if len >= 346 => bext_tr = Some(le32(b, body + 338) as u64 | (le32(b, body + 342) as u64) << 32), - b"iXML" => ixml_tr = ixml_time_reference(&b[body..body + len]), + b"bext" if clen >= 346 => { + let b = read_range(&*reader, body + 338, 8).map_err(io)?; + bext_tr = Some(u64::from(le32(&b, 0)) | u64::from(le32(&b, 4)) << 32); + } + b"iXML" => { + let b = read_range(&*reader, body, clen.min(IXML_MAX) as usize).map_err(io)?; + ixml_tr = ixml_time_reference(&b); + } _ => {} } - pos = body + len + (len & 1); + pos = body + clen + (clen & 1); } let (tag, ch, rate, bits) = fmt.ok_or_else(|| MediaError::Decode(format!("{name}: missing fmt chunk")))?; - let (off, len) = data.ok_or_else(|| MediaError::Decode(format!("{name}: missing data chunk")))?; + let (off, dlen) = data.ok_or_else(|| MediaError::Decode(format!("{name}: missing data chunk")))?; let float = tag == 3; if !(tag == 1 || float) || ch == 0 || !matches!(bits, 8 | 16 | 24 | 32 | 64) { return Err(MediaError::Unsupported(format!("{name}: WAV format tag {tag}, {bits} bits"))); } - let frame_bytes = ch as usize * bits as usize / 8; - let frames = len / frame_bytes; + let frame_bytes = u64::from(ch) * u64::from(bits / 8); + let frames = dlen / frame_bytes; let time_reference = bext_tr.or(ixml_tr); let mut info = MediaInfo { name: name.into(), kind: MediaKind::AudioOnly, - duration: Tick::from_units(frames as i64, rate as i64), + duration: Tick::from_units(i64::try_from(frames).unwrap_or(i64::MAX), rate as i64), video: None, audio_streams: vec![AudioStreamInfo { sample_rate: rate, @@ -92,16 +124,16 @@ impl WavSource { }], container: if time_reference.is_some() { "Broadcast WAV".into() } else { "WAV".into() }, start_timecode: None, - file_size: Some(b.len() as u64), + file_size: Some(len), }; // start timecode: frames at the media's frame rate (the default rate for audio-only media) if let Some(tr) = time_reference.filter(|_| rate > 0) { let r = info.frame_rate(); let n = tr as i128 * r.num as i128; let d = rate as i128 * r.den as i128; - info.start_timecode = Some(((n + d / 2) / d) as i64); + info.start_timecode = (n + d / 2).checked_div(d).and_then(|t| i64::try_from(t).ok()); } - Ok(Self { info, data_off: off, data_len: frames * frame_bytes, channels: ch as usize, bits, float, rate, bytes, time_reference }) + Ok(Self { info, reader, data_off: off, frames, channels: ch as usize, bits, float, rate, time_reference }) } /// Broadcast Wave start time: samples since midnight. @@ -109,17 +141,32 @@ impl WavSource { self.time_reference } - fn sample(&self, frame: usize, ch: usize) -> f32 { + fn frame_bytes(&self) -> usize { + self.channels * (self.bits as usize / 8) + } + + /// Read whole frames `first..=last` of the data chunk. + fn read_frames(&self, first: u64, last: u64) -> Result> { + let fb = self.frame_bytes() as u64; + let n = last.saturating_sub(first).saturating_add(1).saturating_mul(fb); + let off = self.data_off.saturating_add(first.saturating_mul(fb)); + let mut buf = vec![0u8; usize::try_from(n).map_err(|_| MediaError::Decode(format!("{}: audio read too large", self.info.name)))?]; + self.reader.read_at(off, &mut buf).map_err(|e| MediaError::Io(format!("{}: {e}", self.info.name)))?; + Ok(buf) + } + + /// Sample `ch` of frame `frame` of `block`, a run of whole frames. + fn sample(&self, block: &[u8], frame: usize, ch: usize) -> f32 { let bps = self.bits as usize / 8; - let o = self.data_off + (frame * self.channels + ch) * bps; - let b = &self.bytes; - match (self.bits, self.float) { - (8, _) => (b[o] as f32 - 128.0) / 128.0, - (16, _) => i16::from_le_bytes([b[o], b[o + 1]]) as f32 / 32768.0, - (24, _) => ((i32::from_le_bytes([0, b[o], b[o + 1], b[o + 2]]) >> 8) as f32) / 8_388_608.0, - (32, false) => i32::from_le_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]]) as f32 / 2_147_483_648.0, - (32, true) => f32::from_le_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]]), - (64, true) => f64::from_le_bytes(b[o..o + 8].try_into().unwrap_or([0; 8])) as f32, + let o = (frame * self.channels + ch) * bps; + let Some(b) = block.get(o..o.saturating_add(bps)) else { return 0.0 }; + match (self.bits, self.float, b) { + (8, _, &[a]) => (a as f32 - 128.0) / 128.0, + (16, _, &[a, b]) => i16::from_le_bytes([a, b]) as f32 / 32768.0, + (24, _, &[a, b, c]) => ((i32::from_le_bytes([0, a, b, c]) >> 8) as f32) / 8_388_608.0, + (32, false, &[a, b, c, d]) => i32::from_le_bytes([a, b, c, d]) as f32 / 2_147_483_648.0, + (32, true, &[a, b, c, d]) => f32::from_le_bytes([a, b, c, d]), + (64, true, b) => f64::from_le_bytes(b.try_into().unwrap_or([0; 8])) as f32, _ => 0.0, } } @@ -133,25 +180,54 @@ impl MediaSource for WavSource { Err(MediaError::NoStream("video")) } fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> Result { - let total = self.data_len / (self.channels * self.bits as usize / 8); + let total = self.frames; let mut out = AudioBuffer::silence(sample_rate, self.channels, frames); + if total == 0 || sample_rate == 0 { + return Ok(out); + } // Linear-interpolating resample when rates differ (the audio crate provides sinc resampling). let ratio = self.rate as f64 / sample_rate as f64; - for i in 0..frames { - let src = (start + i as i64) as f64 * ratio; - if src < 0.0 { + let src_of = |i: usize| start.saturating_add(i as i64) as f64 * ratio; + let mut i = 0; + while i < frames { + let end = frames.min(i + BLOCK_FRAMES); + let (lo, hi) = (src_of(i), src_of(end - 1)); + if hi < 0.0 { + i = end; continue; } - let i0 = src.floor() as usize; - if i0 >= total { + let first = lo.max(0.0).floor() as u64; + if first >= total { break; } - let fr = (src - i0 as f64) as f32; - for c in 0..self.channels { - let a = self.sample(i0, c); - let b = if i0 + 1 < total { self.sample(i0 + 1, c) } else { a }; - out.channels[c][i] = a + (b - a) * fr; + let last = (hi.floor() as u64).saturating_add(1).min(total - 1); + let span = last.saturating_sub(first).saturating_add(1).saturating_mul(self.frame_bytes() as u64); + let block = if span <= MAX_SPAN_BYTES { Some(self.read_frames(first, last)?) } else { None }; + for slot in i..end { + let src = src_of(slot); + if src < 0.0 { + continue; + } + let i0 = src.floor() as u64; + if i0 >= total { + break; + } + let i1 = (i0 + 1).min(total - 1); + let fr = (src - i0 as f64) as f32; + let (buf, base) = match &block { + Some(b) => (std::borrow::Cow::Borrowed(b.as_slice()), first), + None => (std::borrow::Cow::Owned(self.read_frames(i0, i1)?), i0), + }; + let (k0, k1) = ((i0 - base) as usize, (i1 - base) as usize); + for c in 0..self.channels { + let a = self.sample(&buf, k0, c); + let b = self.sample(&buf, k1, c); + if let Some(s) = out.channels.get_mut(c).and_then(|ch| ch.get_mut(slot)) { + *s = a + (b - a) * fr; + } + } } + i = end; } Ok(out) } diff --git a/crates/platform/src/hardware_encode.rs b/crates/platform/src/hardware_encode.rs index 4af31781..c715e927 100644 --- a/crates/platform/src/hardware_encode.rs +++ b/crates/platform/src/hardware_encode.rs @@ -33,6 +33,9 @@ pub fn config_for(w: u32, h: u32, rate: FrameRate, s: &ExportSettings) -> Result if s.bitrate_mode == BitrateMode::Vbr2Pass || !matches!(s.h264_pass, H264Pass::Single) { return Err("two-pass encoding".into()); } + if s.bitrate_mode == BitrateMode::Crf { + return Err("CRF (constant quality)".into()); + } if s.signal.is_hdr() { return Err("HDR".into()); } diff --git a/crates/platform/src/nvenc/export.rs b/crates/platform/src/nvenc/export.rs index a63b6a0e..4344cb52 100644 --- a/crates/platform/src/nvenc/export.rs +++ b/crates/platform/src/nvenc/export.rs @@ -162,6 +162,9 @@ fn config(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) - if !matches!(s.h264_pass, H264Pass::Single) || (hevc && s.bitrate_mode == BitrateMode::Vbr2Pass) { return Err("two-pass VBR".into()); } + if s.bitrate_mode == BitrateMode::Crf { + return Err("CRF (constant quality)".into()); + } // the hardware path writes the aspect ratio of H.264 only if hevc && s.pixel_aspect.is_some_and(|(n, d)| n != d) { return Err("non-square pixels".into()); @@ -357,6 +360,7 @@ mod tests { ("an unwritable colour signal", ExportSettings { signal: ColorSignal { primaries: 9, transfer: 1, matrix: 1 }, ..ok.clone() }), ("HDR with BT.709 primaries", ExportSettings { signal: ColorSignal { primaries: 1, transfer: 16, matrix: 9 }, ..ok.clone() }), ("two-pass", ExportSettings { bitrate_mode: BitrateMode::Vbr2Pass, ..ok.clone() }), + ("CRF", ExportSettings { bitrate_mode: BitrateMode::Crf, ..ok.clone() }), ("analysis pass", ExportSettings { h264_pass: H264Pass::First, ..ok.clone() }), ("interlaced", ExportSettings { field_order: FieldOrder::UpperFirst, ..ok.clone() }), ("non-square pixels", ExportSettings { pixel_aspect: Some((4, 3)), ..ok.clone() }), diff --git a/crates/platform/src/videotoolbox_encode.rs b/crates/platform/src/videotoolbox_encode.rs index 4e9130dd..5edf4767 100644 --- a/crates/platform/src/videotoolbox_encode.rs +++ b/crates/platform/src/videotoolbox_encode.rs @@ -19,7 +19,7 @@ //! for H.264, VPS / SPS / PPS and VideoToolbox's own `hvcC` record for HEVC) come with the first //! compressed frame, and the muxer asks for them after the first group of frames. -use std::ffi::{c_int, c_void}; +use std::ffi::{CStr, c_char, c_int, c_void}; use std::panic::AssertUnwindSafe; use std::ptr::NonNull; use std::sync::{Mutex, PoisonError}; @@ -39,10 +39,10 @@ use objc2_core_video::{ use objc2_video_toolbox::{ VTCompressionSession, VTEncodeInfoFlags, VTSessionCopyProperty, VTSessionSetProperty, kVTCompressionPropertyKey_AllowFrameReordering, kVTCompressionPropertyKey_AllowOpenGOP, kVTCompressionPropertyKey_AverageBitRate, kVTCompressionPropertyKey_ColorPrimaries, - kVTCompressionPropertyKey_ConstantBitRate, kVTCompressionPropertyKey_DataRateLimits, kVTCompressionPropertyKey_ExpectedFrameRate, - kVTCompressionPropertyKey_MaxKeyFrameInterval, kVTCompressionPropertyKey_ProfileLevel, kVTCompressionPropertyKey_RealTime, - kVTCompressionPropertyKey_TransferFunction, kVTCompressionPropertyKey_UsingHardwareAcceleratedVideoEncoder, kVTCompressionPropertyKey_YCbCrMatrix, - kVTProfileLevel_H264_Baseline_AutoLevel, kVTProfileLevel_H264_High_AutoLevel, kVTProfileLevel_H264_Main_AutoLevel, kVTProfileLevel_HEVC_Main_AutoLevel, + kVTCompressionPropertyKey_DataRateLimits, kVTCompressionPropertyKey_ExpectedFrameRate, kVTCompressionPropertyKey_MaxKeyFrameInterval, + kVTCompressionPropertyKey_ProfileLevel, kVTCompressionPropertyKey_RealTime, kVTCompressionPropertyKey_TransferFunction, + kVTCompressionPropertyKey_UsingHardwareAcceleratedVideoEncoder, kVTCompressionPropertyKey_YCbCrMatrix, kVTProfileLevel_H264_Baseline_AutoLevel, + kVTProfileLevel_H264_High_AutoLevel, kVTProfileLevel_H264_Main_AutoLevel, kVTProfileLevel_HEVC_Main_AutoLevel, kVTVideoEncoderSpecification_RequireHardwareAcceleratedVideoEncoder, }; @@ -421,6 +421,28 @@ fn set_bool(session: &VTCompressionSession, key: &CFString, on: bool) -> i32 { set_value(session, key, CFBoolean::new(on).as_ref()) } +unsafe extern "C" { + fn dlsym(handle: *mut c_void, symbol: *const c_char) -> *mut c_void; +} + +/// `kVTCompressionPropertyKey_ConstantBitRate`, or `None` on a macOS that does not have it. +/// +/// The key exists from macOS 13. Naming it as a linked symbol makes dyld refuse to start the whole +/// app on macOS 11 and 12 ("Symbol not found"), although the encoder already falls back to an +/// average bitrate when the key is unavailable. So it is looked up at run time instead. +fn constant_bitrate_key() -> Option<&'static CFString> { + // `RTLD_DEFAULT` on macOS: search every image loaded into the process, VideoToolbox included. + const RTLD_DEFAULT: *mut c_void = -2isize as *mut c_void; + const NAME: &CStr = c"kVTCompressionPropertyKey_ConstantBitRate"; + // SAFETY: `NAME` is a valid NUL-terminated string; `dlsym` only reads it. + let symbol = unsafe { dlsym(RTLD_DEFAULT, NAME.as_ptr()) }; + let slot = NonNull::new(symbol)?.cast::<*const CFString>(); + // SAFETY: the symbol is a `CFStringRef` constant exported by VideoToolbox, so `slot` points at a + // pointer to an immutable CFString that lives as long as the framework stays loaded (the whole + // process); it is read once and never written. + unsafe { slot.read().as_ref() } +} + fn set_value(session: &VTCompressionSession, key: &CFString, value: &CFType) -> i32 { // SAFETY: the session is valid, `key` is a framework constant and `value` a live CF object. unsafe { VTSessionSetProperty(session.as_ref(), key, Some(value)) } @@ -527,14 +549,14 @@ impl VtEncoder { let _ = set_value(session, fps_key, CFNumber::new_f64(fps).as_ref()); // SAFETY: reading immutable framework constants. - let (avg_key, limits_key, cbr_key) = - unsafe { (kVTCompressionPropertyKey_AverageBitRate, kVTCompressionPropertyKey_DataRateLimits, kVTCompressionPropertyKey_ConstantBitRate) }; + let (avg_key, limits_key) = unsafe { (kVTCompressionPropertyKey_AverageBitRate, kVTCompressionPropertyKey_DataRateLimits) }; + let cbr_key = constant_bitrate_key(); let bps = |kbps: u32| i32::try_from(u64::from(kbps).saturating_mul(1000)).unwrap_or(i32::MAX); match c.rate { VtRate::Cbr { kbps } => { // true constant bitrate where the OS offers it (macOS 13+); otherwise an average // with a one-second ceiling at the same rate - if set_value(session, cbr_key, CFNumber::new_i32(bps(kbps)).as_ref()) != 0 { + if !cbr_key.is_some_and(|key| set_value(session, key, CFNumber::new_i32(bps(kbps)).as_ref()) == 0) { require(set_value(session, avg_key, CFNumber::new_i32(bps(kbps)).as_ref()), "the bitrate")?; let limits = CFArray::::from_objects(&[CFNumber::new_i32(bps(kbps) / 8).as_ref(), CFNumber::new_f64(1.0).as_ref()]); let _ = set_value(session, limits_key, limits.as_ref()); diff --git a/crates/platform/tests/hardware_encode.rs b/crates/platform/tests/hardware_encode.rs index 2962f670..94b1b928 100644 --- a/crates/platform/tests/hardware_encode.rs +++ b/crates/platform/tests/hardware_encode.rs @@ -106,6 +106,7 @@ fn declines_what_the_hardware_path_does_not_take() { let declined = |s: &ExportSettings, what: &str| assert!(config_for(W, H, FPS, s).is_err(), "{what} must not use the hardware encoder"); declined(&ExportSettings { hardware_encoding: HardwareEncoding::Off, ..ok.clone() }, "hardware encoding off"); declined(&ExportSettings { bitrate_mode: BitrateMode::Vbr2Pass, ..ok.clone() }, "two-pass VBR"); + declined(&ExportSettings { bitrate_mode: BitrateMode::Crf, ..ok.clone() }, "CRF"); declined(&ExportSettings { h264_pass: H264Pass::First, ..ok.clone() }, "an analysis pass"); declined(&ExportSettings { signal: filmcraft_export::ColorSignal::PQ, ..ok.clone() }, "HDR"); declined(&ExportSettings { pixel_aspect: Some((4, 3)), ..ok.clone() }, "non-square pixels"); diff --git a/crates/project/src/caption.rs b/crates/project/src/caption.rs index 013c6efc..d8236e8b 100644 --- a/crates/project/src/caption.rs +++ b/crates/project/src/caption.rs @@ -74,7 +74,7 @@ pub enum CaptionAnchor { #[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] #[serde(default)] pub struct CaptionStyle { - /// Font family name. Only the bundled Inter is rendered today; other names fall back to it. + /// Font family name (bundled, craft-fonts or system); unknown or empty names fall back to Inter. pub font: String, /// Font size in pixels for a 1080-line frame (scaled with the frame height). pub size: f32, diff --git a/crates/project/src/effect.rs b/crates/project/src/effect.rs index 1abd4e66..031bceb1 100644 --- a/crates/project/src/effect.rs +++ b/crates/project/src/effect.rs @@ -17,7 +17,7 @@ use crate::mask::Mask; pub use audio::{GEQ10_LABELS, GEQ20_LABELS, GEQ30_LABELS, PREMIERE_AUDIO_EFFECTS}; mod vfx; -pub use vfx::{EASINGS, ECHO_OPERATORS, FRAME_LAYOUTS, LIGHT_IDS, SIMPLE_BLEND, TRACK_CHOICES, auto_point}; +pub use vfx::{EASINGS, ECHO_OPERATORS, FRAME_LAYOUTS, LIGHT_IDS, SIMPLE_BLEND, TRACK_CHOICES, auto_point, ultra_key_setting}; #[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] pub enum EffectKind { @@ -619,7 +619,8 @@ fn build_effects() -> Vec { ang("skew_axis", "Skew Axis", 0.0), ang("rotation", "Rotation", 0.0), f("opacity", "Opacity", 100.0, 0.0, 100.0, ""), - f("shutter_angle", "Shutter Angle", 0.0, 0.0, 360.0, ""), + b("shutter_override", "Use Effect Shutter Angle", false), + f("shutter_angle", "Shutter Angle", 180.0, 0.0, 360.0, ""), ], ), video("mirror", "Mirror", DISTORT, vec![pt("center", "Reflection Center", f64::NAN, f64::NAN), ang("angle", "Reflection Angle", 0.0)]), diff --git a/crates/project/src/effect/vfx.rs b/crates/project/src/effect/vfx.rs index 7833c51b..70d9dae1 100644 --- a/crates/project/src/effect/vfx.rs +++ b/crates/project/src/effect/vfx.rs @@ -1014,6 +1014,60 @@ pub(super) fn defs() -> Vec { v } +/// Parameter values written when Ultra Key's Setting is Default, Relaxed or Aggressive. +/// Custom (index 3) leaves the current values. Aggressive matches the matte that cleans +/// light and dark clothing on a typical green screen. +pub fn ultra_key_setting(setting: u32) -> Option<&'static [(&'static str, f64)]> { + Some(match setting { + 0 => &[ + ("transparency", 45.0), + ("highlight", 10.0), + ("shadow", 50.0), + ("tolerance", 50.0), + ("pedestal", 10.0), + ("choke", 0.0), + ("soften", 0.0), + ("contrast", 0.0), + ("mid_point", 50.0), + ("desaturate", 25.0), + ("range", 50.0), + ("spill", 50.0), + ("spill_luma", 50.0), + ], + 1 => &[ + ("transparency", 30.0), + ("highlight", 15.0), + ("shadow", 35.0), + ("tolerance", 70.0), + ("pedestal", 5.0), + ("choke", 0.0), + ("soften", 8.0), + ("contrast", 0.0), + ("mid_point", 50.0), + ("desaturate", 15.0), + ("range", 50.0), + ("spill", 40.0), + ("spill_luma", 50.0), + ], + 2 => &[ + ("transparency", 40.0), + ("highlight", 10.0), + ("shadow", 55.0), + ("tolerance", 90.0), + ("pedestal", 50.0), + ("choke", 10.0), + ("soften", 10.0), + ("contrast", 10.0), + ("mid_point", 50.0), + ("desaturate", 50.0), + ("range", 50.0), + ("spill", 50.0), + ("spill_luma", 50.0), + ], + _ => return None, + }) +} + /// Extra parameters added to core effects by the 26.x rebuilds (old instances fall back to the /// defaults, so projects round-trip unchanged). pub(super) fn extend_core(v: &mut [EffectDef]) { diff --git a/crates/render/src/audio_fx.rs b/crates/render/src/audio_fx.rs index db84a1f6..adf9d35e 100644 --- a/crates/render/src/audio_fx.rs +++ b/crates/render/src/audio_fx.rs @@ -44,11 +44,12 @@ pub(crate) fn mapping(id: &str) -> Option { }, "hard_limiter" => Mapping { dsp: "limiter", - preroll: |e, t| (f(e, "release", t) as f64 / 1000.0 * 5.0).clamp(0.05, 2.0), + preroll: |e, t| (f(e, "lookahead", t) as f64 / 1000.0 + f(e, "release", t) as f64 / 1000.0 * 5.0).clamp(0.05, 2.0), apply: |d, e, t| { d.set_param("ceiling", f(e, "max", t)); d.set_param("input_gain", f(e, "boost", t).max(0.0)); d.set_param("release", f(e, "release", t)); + d.set_param("lookahead", f(e, "lookahead", t)); }, }, "delay" => Mapping { diff --git a/crates/render/src/tests.rs b/crates/render/src/tests.rs index 31f5edfc..ecb02d9d 100644 --- a/crates/render/src/tests.rs +++ b/crates/render/src/tests.rs @@ -301,6 +301,16 @@ fn audio_fx_limiter_holds_ceiling() { assert!(peak <= ceiling * 1.01 && peak > ceiling * 0.7, "peak {peak} vs ceiling {ceiling}"); } +#[test] +fn audio_fx_limiter_lookahead_changes_the_mix() { + let (p0, seq0, map0) = tone_with(&[("amplify", &[("gain", 30.0)]), ("hard_limiter", &[("max", -6.0), ("lookahead", 0.0)])]); + let (p1, seq1, map1) = tone_with(&[("amplify", &[("gain", 30.0)]), ("hard_limiter", &[("max", -6.0), ("lookahead", 30.0)])]); + let a = mix(&p0, seq0, &map0, 0, 48_000); + let b = mix(&p1, seq1, &map1, 0, 48_000); + let max = a.iter().zip(&b).map(|(x, y)| (x - y).abs()).fold(0f32, f32::max); + assert!(max > 1e-4, "look-ahead of 0 ms and 30 ms mixed the same clip"); +} + #[test] fn audio_fx_chain_is_continuous_and_random_access_matches() { let (p, seq, map) = tone_with(&[("delay", &[("delay", 0.05), ("feedback", 50.0), ("mix", 50.0)])]); diff --git a/crates/render/src/vfx/color.rs b/crates/render/src/vfx/color.rs index 35f4e464..309e6d72 100644 --- a/crates/render/src/vfx/color.rs +++ b/crates/render/src/vfx/color.rs @@ -161,17 +161,14 @@ pub fn ultra_key(img: &mut Image, e: &EffectInstance, cx: &FxCtx) { let k = rgb_to_ycbcr(kc[0], kc[1], kc[2], Matrix::Bt709); let kmag2 = (k[1] * k[1] + k[2] * k[2]).max(1e-6); let setting = chv(e, "setting"); - let (g_mul, ped_mul) = match setting { - 1 => (0.85, 0.7), - 2 => (1.2, 1.5), - _ => (1.0, 1.0), - }; - let transparency = fv(e, "transparency", cx) / 100.0; - let highlight = fv(e, "highlight", cx) / 100.0; - let shadow = fv(e, "shadow", cx) / 100.0; - let tol = 0.08 + fv(e, "tolerance", cx) / 100.0 * 0.5; - let pedestal = (fv(e, "pedestal", cx) / 100.0 * 0.3 * ped_mul).min(0.9); - let gain = (1.0 + 2.0 * transparency) * g_mul; + let preset = filmcraft_project::effect::ultra_key_setting(setting); + let pf = |id: &str, fallback: f32| preset.and_then(|rows| rows.iter().find(|(k, _)| *k == id).map(|(_, v)| *v as f32)).unwrap_or(fallback); + let transparency = pf("transparency", fv(e, "transparency", cx)) / 100.0; + let highlight = pf("highlight", fv(e, "highlight", cx)) / 100.0; + let shadow = pf("shadow", fv(e, "shadow", cx)) / 100.0; + let tol = 0.08 + pf("tolerance", fv(e, "tolerance", cx)) / 100.0 * 0.5; + let pedestal = (pf("pedestal", fv(e, "pedestal", cx)) / 100.0 * 0.3).min(0.9); + let gain = 1.0 + 2.0 * transparency; let output = chv(e, "output"); let dom = if kc[1] >= kc[0] && kc[1] >= kc[2] { 1 @@ -199,10 +196,10 @@ pub fn ultra_key(img: &mut Image, e: &EffectInstance, cx: &FxCtx) { *a = ((m.clamp(0.0, 1.0) - pedestal) / (1.0 - pedestal)).clamp(0.0, 1.0); }); // cleanup - let choke = fv(e, "choke", cx) / 100.0; - let soften = fv(e, "soften", cx) / 100.0; - let contrast = fv(e, "contrast", cx) / 100.0; - let mid = fv(e, "mid_point", cx) / 100.0; + let choke = pf("choke", fv(e, "choke", cx)) / 100.0; + let soften = pf("soften", fv(e, "soften", cx)) / 100.0; + let contrast = pf("contrast", fv(e, "contrast", cx)) / 100.0; + let mid = pf("mid_point", fv(e, "mid_point", cx)) / 100.0; if choke > 0.0 { let r = (choke * 4.0 * cx.px_scale).max(0.5); let b = blur_plane(&alpha, img.w, img.h, r); @@ -217,10 +214,10 @@ pub fn ultra_key(img: &mut Image, e: &EffectInstance, cx: &FxCtx) { alpha.par_iter_mut().for_each(|a| *a = ((*a - mid) * g + mid).clamp(0.0, 1.0)); } // spill suppression + colour correction on the foreground - let spill = fv(e, "spill", cx) / 100.0; - let desat = fv(e, "desaturate", cx) / 100.0; - let range = fv(e, "range", cx) / 100.0; - let sluma = fv(e, "spill_luma", cx) / 100.0; + let spill = pf("spill", fv(e, "spill", cx)) / 100.0; + let desat = pf("desaturate", fv(e, "desaturate", cx)) / 100.0; + let range = pf("range", fv(e, "range", cx)) / 100.0; + let sluma = pf("spill_luma", fv(e, "spill_luma", cx)) / 100.0; let (cc_s, cc_h, cc_l) = (fv(e, "cc_saturation", cx) / 100.0, fv(e, "cc_hue", cx) / 360.0, fv(e, "cc_luminance", cx) / 100.0); let cc = (cc_s - 1.0).abs() > 1e-4 || cc_h.abs() > 1e-6 || (cc_l - 1.0).abs() > 1e-4; img.px.par_chunks_mut(4).zip(alpha.par_iter()).for_each(|(p, &a)| { diff --git a/crates/render/src/vfx/distort.rs b/crates/render/src/vfx/distort.rs index 6dd33993..db22ed10 100644 --- a/crates/render/src/vfx/distort.rs +++ b/crates/render/src/vfx/distort.rs @@ -231,6 +231,50 @@ pub fn rotate_3d(img: &mut Image, e: &EffectInstance, cx: &FxCtx) { }); } +/// Transform: affine place, with optional shutter-angle motion blur when the override is on. +pub fn transform_fx(img: &mut Image, e: &EffectInstance, cx: &FxCtx) { + let fps = cx.env.map_or(24.0, |env| env.frame_rate()).max(1.0); + let angle = fv(e, "shutter_angle", cx); + let n = if bv(e, "shutter_override") && angle > 0.5 { 7 } else { 1 }; + let span = (angle as f64 / 360.0) / fps; + motion_blurred(img, n, |im, k| { + let t = cx.t + Tick::from_seconds_f64(k * span); + let cx2 = FxCtx { + t, + px_scale: cx.px_scale, + seconds: cx.seconds + k * span, + timecode: cx.timecode, + clip_name: cx.clip_name, + project: cx.project, + env: cx.env, + working: cx.working, + }; + let anchor = crate::effects::point(e, "anchor", &cx2, im); + let pos = crate::effects::point(e, "position", &cx2, im); + let sh = crate::effects::f(e, "scale_height", &cx2) as f64 / 100.0; + let sw = if crate::effects::b(e, "uniform_scale") { sh } else { crate::effects::f(e, "scale_width", &cx2) as f64 / 100.0 }; + let rot = crate::effects::f(e, "rotation", &cx2) as f64; + let skew = (crate::effects::f(e, "skew", &cx2) as f64).to_radians().tan(); + let skew_axis = crate::effects::f(e, "skew_axis", &cx2) as f64; + let op = crate::effects::f(e, "opacity", &cx2) / 100.0; + let sk = + Affine::rotate_deg(skew_axis).then_apply(&Affine { a: 1.0, b: 0.0, c: skew, d: 1.0, e: 0.0, f: 0.0 }).then_apply(&Affine::rotate_deg(-skew_axis)); + let m = Affine::translate(pos.x, pos.y) + .then_apply(&Affine::rotate_deg(rot)) + .then_apply(&sk) + .then_apply(&Affine::scale(sw, sh)) + .then_apply(&Affine::translate(-anchor.x, -anchor.y)); + affine_warp(im, &m); + if op < 1.0 - 1e-5 { + im.px.par_chunks_mut(4).for_each(|p| { + for v in p.iter_mut() { + *v *= op; + } + }); + } + }); +} + /// Grow / Shrink: scale animated across the clip from Scale From to Scale To. pub fn grow(img: &mut Image, e: &EffectInstance, cx: &FxCtx) { let p = ease(chv(e, "easing"), clip_progress(cx)); diff --git a/crates/render/src/vfx/mod.rs b/crates/render/src/vfx/mod.rs index bdca8e52..9c7c966a 100644 --- a/crates/render/src/vfx/mod.rs +++ b/crates/render/src/vfx/mod.rs @@ -484,6 +484,7 @@ pub fn apply(img: &mut Image, e: &EffectInstance, cx: &FxCtx) -> bool { "spin" => distort::spin(img, e, cx), "wiggle" => distort::wiggle(img, e, cx), "camera_shake" => distort::camera_shake(img, e, cx), + "transform" if bv(e, "shutter_override") && fv(e, "shutter_angle", cx) > 0.5 => distort::transform_fx(img, e, cx), "spacer" => distort::spacer(img, e, cx), "clone" => distort::clone_fx(img, e, cx), "auto_align" => distort::auto_align(img, e, cx), diff --git a/crates/render/src/vfx/tests.rs b/crates/render/src/vfx/tests.rs index 6a00196d..6cd0e01b 100644 --- a/crates/render/src/vfx/tests.rs +++ b/crates/render/src/vfx/tests.rs @@ -4,7 +4,7 @@ use std::sync::Arc; -use filmcraft_project::{EffectKind, ParamKind, find_effect}; +use filmcraft_project::{EffectKind, Interpolation, Keyframe, Param, ParamKind, find_effect}; use super::*; @@ -504,6 +504,12 @@ fn keyers_known_values() { set(&mut e, "output", ParamValue::Choice(1)); let m = run("ultra_key", &e, &img, &cx()); assert!(m.get(1, 0)[0] < 0.05 && m.get(6, 0)[0] > 0.9 && m.get(1, 0)[3] == 1.0); + let mut aggressive = inst("ultra_key"); + set(&mut aggressive, "setting", ParamValue::Choice(2)); + let ag = run("ultra_key", &aggressive, &img, &cx()); + let def = run("ultra_key", &inst("ultra_key"), &img, &cx()); + let delta = (0..img.w * img.h).map(|i| (ag.px[i * 4 + 3] - def.px[i * 4 + 3]).abs()).fold(0.0f32, f32::max); + assert!(delta > 1e-4, "Aggressive Setting left the Default matte unchanged"); // Alpha Adjust: invert and opacity let mut a = Image::filled(2, 2, [0.1, 0.1, 0.1, 0.25]); let mut e = inst("alpha_adjust"); @@ -759,6 +765,27 @@ fn transform_presets_animate_over_the_clip() { assert_eq!(text::burnin_text(&e2, &mid), "FILE 24"); } +#[test] +fn transform_shutter_override_blurs_moving_position() { + let mut e = inst("transform"); + set(&mut e, "shutter_override", ParamValue::Bool(true)); + setf(&mut e, "shutter_angle", 180.0); + let a = Keyframe::new(Tick::ZERO, ParamValue::Vec2(filmcraft_geom::Vec2::new(8.0, 16.0))); + let mut b = Keyframe::new(Tick::from_seconds_f64(1.0), ParamValue::Vec2(filmcraft_geom::Vec2::new(40.0, 16.0))); + b.interp = Interpolation::Linear; + e.params.insert("position".into(), Param { value: ParamValue::Vec2(filmcraft_geom::Vec2::new(8.0, 16.0)), keyframes: vec![a, b] }); + let img = picture(32, 32); + let cx = FxCtx { t: Tick::from_seconds_f64(0.5), seconds: 0.5, ..cx() }; + let sharp = { + let mut off = e.clone(); + set(&mut off, "shutter_override", ParamValue::Bool(false)); + run("transform", &off, &img, &cx) + }; + let blurred = run("transform", &e, &img, &cx); + let delta = max_diff(&sharp, &blurred); + assert!(delta > 1e-3, "shutter override left the Transform picture unchanged ({delta})"); +} + #[test] fn clone_auto_align_and_rounded_crop() { let img = picture(40, 20); diff --git a/crates/tts-text/src/normalize.rs b/crates/tts-text/src/normalize.rs index 7b5ee2c1..235d8423 100644 --- a/crates/tts-text/src/normalize.rs +++ b/crates/tts-text/src/normalize.rs @@ -347,7 +347,14 @@ fn parse_amount(rest: &str, _lang: Lang) -> Option<(u64, usize, usize, u32, bool } let major = int_digits.parse::().ok()?; let had_minor = after_decimal && !frac_digits.is_empty(); - let minor = if had_minor { frac_digits.parse::().ok()? } else { 0 }; + // One fractional digit denotes tenths of a major unit (e.g. $1.5 = $1.50), + // not five minor units. Preserve the existing two-digit and longer readings. + let minor = if had_minor { + let value = frac_digits.parse::().ok()?; + if frac_digits.len() == 1 { value * 10 } else { value } + } else { + 0 + }; Some((major, int_digits.len(), used, minor, had_minor)) } @@ -600,14 +607,20 @@ fn has_leading_zero_str(digits: &str) -> bool { /// `%`, an ordinal suffix (`st`/`nd`/`rd`/`th`), a decimal fraction or `-`-joined /// digit groups. +/// Percentage sign attached to a number, or separated from it by whitespace. +fn percent_suffix_len(rest: &str) -> Option { + let suffix = rest.trim_start_matches(char::is_whitespace); + suffix.strip_prefix('%').map(|_| rest.len() - suffix.len() + 1) +} + fn parse_continuation(rest: &str, used: usize, value: u64) -> Option<(Vec, usize)> { let after = rest.get(used..)?; + if let Some(suffix_len) = percent_suffix_len(after) { + let mut words = english_cardinal(value); + words.push("percent".into()); + return Some((words.into_iter().map(Token::Word).collect(), used + suffix_len)); + } match peek(after, 0) { - Some('%') => { - let mut words = english_cardinal(value); - words.push("percent".into()); - Some((words.into_iter().map(Token::Word).collect(), used + 1)) - } Some('s' | 'S' | 't' | 'T' | 'n' | 'N' | 'd' | 'D' | 'r' | 'R') => { let suffix = after.get(..2)?; if is_ordinal_suffix(suffix) && !peek(after, 2).is_some_and(|c| c.is_alphabetic()) { @@ -623,7 +636,12 @@ fn parse_continuation(rest: &str, used: usize, value: u64) -> Option<(Vec return None; } words.extend(frac); - Some((words.into_iter().map(Token::Word).collect(), used + 1 + frac_used)) + let number_end = used + 1 + frac_used; + if let Some(suffix_len) = rest.get(number_end..).and_then(percent_suffix_len) { + words.push("percent".into()); + return Some((words.into_iter().map(Token::Word).collect(), number_end + suffix_len)); + } + Some((words.into_iter().map(Token::Word).collect(), number_end)) } Some('-') => parse_digit_groups(rest, used), _ => None, @@ -700,13 +718,17 @@ fn parse_digit_groups(rest: &str, used: usize) -> Option<(Vec, usize)> { } return Some((words, end)); } - if groups.len() == 2 && groups.iter().all(|g| g.len() <= 3) { + // For shorter non-phone-like groups, retain every group in order. Previously + // three original groups ("5-10-15") dropped the middle one by reading only + // the first and last endpoints. + if groups.iter().all(|g| g.len() <= 3) { let a = rest.get(..used)?.parse::().ok()?; - let b_str = groups.last()?; - let b = b_str.parse::().ok()?; let mut words = english_cardinal(a); - words.push("to".into()); - words.extend(english_cardinal(b)); + for group in &groups { + let n = group.parse::().ok()?; + words.push("to".into()); + words.extend(english_cardinal(n)); + } return Some((words.into_iter().map(Token::Word).collect(), end)); } None @@ -1165,6 +1187,13 @@ mod tests { assert_eq!(words("$3.50", Lang::EnUs), "three dollars and fifty cents"); assert_eq!(words("$1", Lang::EnUs), "one dollar"); assert_eq!(words("$1.00", Lang::EnUs), "one dollar"); + assert_eq!(words("$1.5", Lang::EnUs), "one dollar and fifty cents"); + assert_eq!(words("$2.5", Lang::EnUs), "two dollars and fifty cents"); + assert_eq!(words("$0.5", Lang::EnUs), "zero dollars and fifty cents"); + assert_eq!(words("$1.05", Lang::EnUs), "one dollar and five cents"); + assert_eq!(words("The price is $1.5.", Lang::EnUs), words("The price is $1.50.", Lang::EnUs)); + assert_eq!(words("£1.5", Lang::EnGb), "one pound and fifty pence"); + assert_eq!(words("€1.5", Lang::EnUs), "one euro and fifty cents"); assert_eq!(words("$5.05", Lang::EnUs), "five dollars and five cents"); assert_eq!(words("$0.75", Lang::EnUs), "zero dollars and seventy-five cents"); assert_eq!(words("$1,234.56", Lang::EnUs), "one thousand two hundred thirty-four dollars and fifty-six cents"); @@ -1175,11 +1204,37 @@ mod tests { assert_eq!(words("$0.50", Lang::EnUs), "zero dollars and fifty cents"); } + #[test] + fn short_hyphenated_groups_keep_every_number() { + for lang in [Lang::EnUs, Lang::EnGb] { + assert_eq!(words("1-2-3", lang), "one to two to three"); + assert_eq!(words("5-10-15", lang), "five to ten to fifteen"); + assert_eq!(words("10-20-30", lang), "ten to twenty to thirty"); + assert_eq!(words("1-2-3-4", lang), "one to two to three to four"); + assert_eq!(words("1-2-3-4-5", lang), "one to two to three to four to five"); + assert_eq!(words("1-2-3-4-5-6", lang), "one to two to three to four to five to six"); + assert_eq!(words("2-3", lang), "two to three"); + assert_eq!(words("10-12", lang), "ten to twelve"); + assert_eq!(words("555-0134", lang), "five five five zero one three four"); + assert_eq!(words("005-010-015", lang), "zero zero five zero one zero zero one five"); + } + } + #[test] fn percent() { assert_eq!(words("20%", Lang::EnUs), "twenty percent"); assert_eq!(words("100%", Lang::EnUs), "one hundred percent"); assert_eq!(words("7%", Lang::EnUs), "seven percent"); + for lang in [Lang::EnUs, Lang::EnGb] { + assert_eq!(words("20.5%", lang), "twenty point five percent"); + assert_eq!(words("0.5%", lang), "zero point five percent"); + assert_eq!(words("50 %", lang), "fifty percent"); + assert_eq!(words("50\t%", lang), "fifty percent"); + assert_eq!(words("50\u{a0}%", lang), "fifty percent"); + assert_eq!(words("100 %", lang), "one hundred percent"); + assert_eq!(words("20.5 %", lang), "twenty point five percent"); + assert_eq!(words("20.5%", lang), words("twenty point five percent", lang)); + } } #[test] diff --git a/crates/ui-egui/src/control.rs b/crates/ui-egui/src/control.rs index 16260258..3cd86c35 100644 --- a/crates/ui-egui/src/control.rs +++ b/crates/ui-egui/src/control.rs @@ -27,7 +27,7 @@ use serde_json::{Value, json}; use crate::FilmcraftApp; use crate::dock::PanelKind; -use crate::state::{Mode, PlaybackRes, Tool}; +use crate::state::{Mode, PlaybackRes, ThumbnailMode, Tool}; /// Prefix marking errors that may resolve after another frame. pub const RETRY: &str = "\u{1}"; @@ -189,10 +189,30 @@ pub fn handle(app: &mut FilmcraftApp, ctx: &egui::Context, req: &ControlRequest) if let Some(v) = tl.get("audioTrackHeight").and_then(Value::as_f64) { app.ui.timeline.audio_track_h = v as f32; } + // Show Video Thumbnails: "off" or a mode ("head", "headAndTail", "continuous") + if let Some(m) = tl.get("thumbnails").and_then(Value::as_str) { + if m.eq_ignore_ascii_case("off") { + app.ui.timeline.show_thumbnails = false; + } else { + let Some(mode) = ThumbnailMode::from_name(m) else { + return err(format!("unknown thumbnail mode `{m}` (off, head, headAndTail, continuous)")); + }; + app.ui.timeline.show_thumbnails = true; + app.ui.timeline.thumbnail_mode = mode; + } + } } if let Some(q) = s("effectsSearch") { app.ui.effects_search = q.to_string(); } + // Effect Controls keyframe selection (`[{clip, effect, param, mask?, time}]`), replaced; + // references to missing keyframes are dropped by the panel + if let Some(v) = p.get("keyframeSelection") { + match serde_json::from_value(v.clone()) { + Ok(sel) => app.ui.keyframe_selection = sel, + Err(e) => return err(format!("`keyframeSelection`: {e}")), + } + } if let Some(v) = p.get("safeMargins").and_then(Value::as_bool) { app.ui.program.safe_margins = v; } diff --git a/crates/ui-egui/src/i18n/es.tsv b/crates/ui-egui/src/i18n/es.tsv index 980ef537..85a8a090 100644 --- a/crates/ui-egui/src/i18n/es.tsv +++ b/crates/ui-egui/src/i18n/es.tsv @@ -407,6 +407,10 @@ Delete Video Tracks Eliminar pistas de video Delete Audio Tracks Eliminar pistas de audio All Empty Tracks Todas las pistas vacías + Caption Tracks Pistas de subtítulos + Delete Caption Tracks Eliminar pistas de subtítulos + Delete Caption Track Eliminar pista de subtítulos + Delete Empty Caption Tracks Eliminar pistas de subtítulos vacías Version {version} Versión {version} A clean-room, pure-Rust non-linear video editor. Part of the ArtCraft family. Un editor de video no lineal en Rust puro, desarrollado en sala limpia. Parte de la familia ArtCraft. Join the ArtCraft Discord Únete al Discord de ArtCraft @@ -1216,6 +1220,9 @@ Caption track options Opciones de pista de subtítulos Timeline Display Settings Ajustes de visualización de la línea de tiempo Show Video Thumbnails Mostrar miniaturas de video + Head Cabeza + Head and Tail Cabeza y cola + Continuous Continuas Show Audio Waveform Mostrar forma de onda de audio Expand All Tracks Expandir todas las pistas Minimize All Tracks Minimizar todas las pistas @@ -2896,6 +2903,7 @@ Target Bitrate Velocidad de bits objetivo Adaptive ({bpp} bits per pixel) Adaptable ({bpp} bits por píxel) Maximum Bitrate Velocidad de bits máxima + Quality (CRF) Calidad (CRF) Key Frame Distance Distancia entre fotogramas clave Quality Calidad Render at Maximum Depth Procesar a la máxima profundidad @@ -2941,6 +2949,7 @@ File Name & Location Nombre y ubicación del archivo Quick Export: {path} Exportación rápida: {path} Graphics problem: switched to software compositing Problema de gráficos: se ha cambiado a la composición por software + GPU acceleration is unavailable; preview renders on the CPU La aceleración por GPU no está disponible; la vista previa se renderiza en la CPU Exporting Exportando Click to select, or click in empty space and drag to marquee select. Use Shift, Opt, and Cmd for other options. Haz clic para seleccionar, o haz clic en un espacio vacío y arrastra para seleccionar con un marco. Usa Mayús, Opción y Cmd para otras opciones. Click to select all clips to the right in all tracks. Shift-click for a single track. Haz clic para seleccionar todos los clips a la derecha en todas las pistas. Mayús+clic para una sola pista. @@ -3300,6 +3309,7 @@ Upper First Superior primero VBR, 1 pass VBR, 1 pasada VBR, 2 pass VBR, 2 pasadas + CRF (constant quality) CRF (calidad constante) Vimeo 1080p Full HD Vimeo 1080p Full HD Vimeo 2160p 4K Ultra HD Vimeo 2160p 4K Ultra HD Voice-over Record Grabación de voz en off @@ -3502,6 +3512,8 @@ Light Theme Tema claro Next Appearance Mode Siguiente modo de apariencia Sync with system Sincronizar con el sistema + UI Scale Escala de la interfaz + Auto (system) Automática (sistema) In Entrada Out Salida Export Markers as CSV… Exportar marcadores como CSV… @@ -3513,3 +3525,4 @@ Composite on the GPU when available Componer en la GPU cuando esté disponible Text to Speech Texto a voz Edit Narration Editar narración + Use Effect Shutter Angle Usar el ángulo de obturación del efecto diff --git a/crates/ui-egui/src/i18n/ja.tsv b/crates/ui-egui/src/i18n/ja.tsv index f9ec82d0..8a7e9c20 100644 --- a/crates/ui-egui/src/i18n/ja.tsv +++ b/crates/ui-egui/src/i18n/ja.tsv @@ -1216,6 +1216,9 @@ Caption track options キャプショントラックのオプション Timeline Display Settings タイムライン表示設定 Show Video Thumbnails ビデオのサムネイルを表示 + Head 先頭 + Head and Tail 先頭と末尾 + Continuous 連続 Show Audio Waveform オーディオ波形を表示 Expand All Tracks すべてのトラックを展開 Minimize All Tracks すべてのトラックを最小化 @@ -2891,6 +2894,7 @@ Target Bitrate ターゲットビットレート Adaptive ({bpp} bits per pixel) アダプティブ (1 ピクセルあたり {bpp} ビット) Maximum Bitrate 最大ビットレート + Quality (CRF) 品質 (CRF) Key Frame Distance キーフレーム間隔 Quality 画質 Render at Maximum Depth 最大深度でレンダリングする @@ -3295,6 +3299,7 @@ Upper First 上位フィールド優先 VBR, 1 pass VBR、1 パス VBR, 2 pass VBR、2 パス + CRF (constant quality) CRF (品質一定) Vimeo 1080p Full HD Vimeo 1080p Full HD Vimeo 2160p 4K Ultra HD Vimeo 2160p 4K Ultra HD Voice-over Record ナレーションを録音 @@ -3447,6 +3452,8 @@ Light Theme ライトテーマ Next Appearance Mode 次の外観モード Sync with system システムと同期 + UI Scale UI のスケール + Auto (system) 自動 (システム) Go to previous keyframe 前のキーフレームへ移動 Go to next keyframe 次のキーフレームへ移動 Add keyframe キーフレームを追加 @@ -3506,3 +3513,4 @@ Edit Narration ナレーションを編集 Clear Transition トランジションを消去 Set Transition Duration トランジションのデュレーションを設定 + GPU acceleration is unavailable; preview renders on the CPU GPU アクセラレーションを利用できません。プレビューは CPU でレンダリングされます diff --git a/crates/ui-egui/src/i18n/mod.rs b/crates/ui-egui/src/i18n/mod.rs index fb88bec6..205fb5c2 100644 --- a/crates/ui-egui/src/i18n/mod.rs +++ b/crates/ui-egui/src/i18n/mod.rs @@ -663,7 +663,7 @@ mod tests { ex::MxfVideoCodec::ALL.iter().for_each(|c| push(&mut out, c.label())); ex::Placement::ALL.iter().for_each(|p| push(&mut out, p.label())); [ex::FieldOrder::Progressive, ex::FieldOrder::UpperFirst, ex::FieldOrder::LowerFirst].iter().for_each(|f| push(&mut out, f.label())); - [ex::BitrateMode::Cbr, ex::BitrateMode::Vbr1Pass, ex::BitrateMode::Vbr2Pass].iter().for_each(|m| push(&mut out, m.label())); + [ex::BitrateMode::Cbr, ex::BitrateMode::Vbr1Pass, ex::BitrateMode::Vbr2Pass, ex::BitrateMode::Crf].iter().for_each(|m| push(&mut out, m.label())); [ex::H264Profile::Baseline, ex::H264Profile::Main, ex::H264Profile::High].iter().for_each(|p| push(&mut out, p.label())); crate::panels::export_mode::RANGES.iter().for_each(|(_, l)| push(&mut out, l)); crate::panels::export_mode::PARS.iter().for_each(|(l, _)| push(&mut out, l)); diff --git a/crates/ui-egui/src/i18n/zh-cn.tsv b/crates/ui-egui/src/i18n/zh-cn.tsv index 9e46602c..ceca0bd5 100644 --- a/crates/ui-egui/src/i18n/zh-cn.tsv +++ b/crates/ui-egui/src/i18n/zh-cn.tsv @@ -1209,6 +1209,9 @@ Caption track options 字幕轨道选项 Timeline Display Settings 时间轴显示设置 Show Video Thumbnails 显示视频缩略图 + Head 首帧 + Head and Tail 首尾帧 + Continuous 连续 Show Audio Waveform 显示音频波形 Expand All Tracks 展开所有轨道 Minimize All Tracks 最小化所有轨道 @@ -2884,6 +2887,7 @@ Target Bitrate 目标码率 Adaptive ({bpp} bits per pixel) 自适应（每像素 {bpp} 位） Maximum Bitrate 最大码率 + Quality (CRF) 质量 (CRF) Key Frame Distance 关键帧间隔 Quality 质量 Render at Maximum Depth 以最大深度渲染 @@ -3288,6 +3292,7 @@ Upper First 上方优先 VBR, 1 pass VBR，1 遍 VBR, 2 pass VBR，2 遍 + CRF (constant quality) CRF（恒定质量） Vimeo 1080p Full HD Vimeo 1080p 全高清 Vimeo 2160p 4K Ultra HD Vimeo 2160p 4K 超高清 Voice-over Record 画外音录制 @@ -3489,9 +3494,16 @@ Appearance Mode 外观模式 Next Appearance Mode 下一个外观模式 Sync with system 跟随系统 + UI Scale 界面缩放 + Auto (system) 自动（系统） GPU Rendering GPU 渲染 Composite on the GPU when available 可用时在 GPU 上合成 Text to Speech 文字转语音 Edit Narration 编辑旁白 Clear Transition 清除过渡 Set Transition Duration 设置过渡持续时间 + GPU acceleration is unavailable; preview renders on the CPU GPU 加速不可用；预览在 CPU 上渲染 + Caption Tracks 字幕轨道 + Delete Caption Track 删除字幕轨道 + Delete Caption Tracks 删除多个字幕轨道 + Delete Empty Caption Tracks 删除空字幕轨道 diff --git a/crates/ui-egui/src/lib.rs b/crates/ui-egui/src/lib.rs index 857f42a5..0faa82e1 100644 --- a/crates/ui-egui/src/lib.rs +++ b/crates/ui-egui/src/lib.rs @@ -143,6 +143,17 @@ pub struct HostHooks { /// whose Wayland compositor sends no theme to winit). Without it, or without an answer, Auto /// uses `egui::Context::system_theme`. pub system_theme: Option, + /// Files or an image on the system clipboard, for Paste in the Timeline and the Project panel + /// (#611). Without it Paste only pastes clips copied in FilmCraft. + pub clipboard_media: Option Option>>, +} + +/// What [`HostHooks::clipboard_media`] found on the system clipboard. +pub enum ClipboardMedia { + /// Files copied in the file manager. + Files(Vec), + /// An image (a screenshot, an image copied in a browser): 8-bit RGBA, row by row. + Image { width: usize, height: usize, rgba: Vec }, } /// Reads the system appearance (Settings ▸ Appearance ▸ Appearance Mode ▸ Sync with system). @@ -302,12 +313,16 @@ pub struct FilmcraftApp { pub tl: panels::timeline::TlState, /// Commands from outside the UI (native menu bar), invoked on the UI thread. pub command_inbox: Option>, + /// A V key press (or a Ctrl+V paste) reached the app and its release hasn't yet. + v_down: bool, /// GPU compositor (when running on wgpu): device state + compositor + the egui texture it feeds. pub gpu: Option, /// Preview render job being watched (job id, where to start playing when it completes). watched_render: Option<(u64, Tick)>, /// Settings last applied to the UI (theme, tooltips, frame cache, audio device). applied_prefs: Option, + /// Settings ▸ Appearance ▸ UI Scale last applied, with the system scale it was applied for. + applied_ui_scale: Option<(Option, f32)>, workspace_restored: bool, /// Window ▸ Workspaces: saved layouts ([`dock::WORKSPACES_FILE`] in the data directory). pub workspaces: dock::WorkspacePrefs, @@ -335,12 +350,31 @@ fn plan_side(plan: &filmcraft_render::plan::FramePlan) -> usize { plan.max_side() } +/// A software rasterizer (llvmpipe, SwiftShader, softpipe — VMs and containers without GPU device +/// access, missing or broken drivers) runs wgpu entirely on the CPU. Compositing through one is +/// slower than the native CPU compositor and hides the cause: GPU usage stays at 0 % while the +/// CPU saturates. Returns a description when the adapter is one. +pub fn software_rasterizer(info: &eframe::wgpu::AdapterInfo) -> Option { + rasterizer_reason(&info.name, info.device_type) +} + +fn rasterizer_reason(name: &str, device_type: eframe::wgpu::DeviceType) -> Option { + let lower = name.to_ascii_lowercase(); + if device_type == eframe::wgpu::DeviceType::Cpu || ["llvmpipe", "swiftshader", "softpipe"].iter().any(|s| lower.contains(s)) { + return Some(format!("software rasterizer ({name}, {device_type:?})")); + } + None +} + /// Whether the GPU compositor can run on this adapter: it renders and blends Rgba16Float /// targets and uploads whole frames as textures. Older or OpenGL-backed GPUs (some Intel Macs, /// VMs, Linux without Vulkan, WebGL) can't; those use the CPU compositor, which renders the same /// frames. Returns the reason when unsupported. pub fn gpu_compositor_unsupported(adapter: &eframe::wgpu::Adapter) -> Option { use eframe::wgpu::{TextureFormat, TextureFormatFeatureFlags as F, TextureUsages as U}; + if let Some(why) = software_rasterizer(&adapter.get_info()) { + return Some(why); + } let f = adapter.get_texture_format_features(TextureFormat::Rgba16Float); if !f.allowed_usages.contains(U::RENDER_ATTACHMENT | U::TEXTURE_BINDING) { return Some("Rgba16Float is not renderable".into()); @@ -360,6 +394,9 @@ impl FilmcraftApp { pub fn set_wgpu(&mut self, rs: eframe::egui_wgpu::RenderState) { if let Some(why) = gpu_compositor_unsupported(&rs.adapter) { log::warn!("GPU compositor disabled ({why}); compositing on the CPU"); + // Say so instead of letting the preview quietly crawl: "GPU 0 %, CPU 100 %" with no + // explanation read like a renderer bug when the machine simply has no GPU access. + self.ui.status = format!("{} ({why})", tl!("GPU acceleration is unavailable; preview renders on the CPU")); return; } // wgpu's default reaction to a validation error or lost device is to panic, which closed @@ -489,9 +526,11 @@ impl FilmcraftApp { toast: None, tl: Default::default(), command_inbox: None, + v_down: false, gpu: None, watched_render: None, applied_prefs: None, + applied_ui_scale: None, workspace_restored: false, workspaces, menu_workspaces: Default::default(), @@ -551,9 +590,28 @@ impl FilmcraftApp { ctx.global_style_mut(|s| s.interaction.tooltip_delay = delay); } + /// Settings ▸ Appearance ▸ UI Scale: a fixed scale makes a point that many physical pixels, + /// whatever scale the system reports. It is applied only when the setting or the system scale + /// changes, so Ctrl+- / Ctrl++ still zoom the interface for the session; going back to Auto + /// restores the system's scale. + fn apply_ui_scale(&mut self, ctx: &egui::Context) { + let want = self.session.prefs.appearance.ui_scale_factor(); + let native = ctx.native_pixels_per_point().filter(|v| v.is_finite() && *v > 0.0).unwrap_or(1.0); + if self.applied_ui_scale == Some((want, native)) { + return; + } + let prev = self.applied_ui_scale.replace((want, native)); + match want { + Some(scale) => ctx.set_zoom_factor(scale / native), + None if prev.is_some_and(|(p, _)| p.is_some()) => ctx.set_zoom_factor(1.0), + None => {} + } + } + /// Make the UI follow the settings after they change (theme, tooltips, frame cache budget, /// play after rendering, audio device). pub fn apply_prefs(&mut self, ctx: &egui::Context) { + self.apply_ui_scale(ctx); // Appearance Mode ▸ Sync with system: follow the system while running. The host reports // changes as they happen (no polling) and wakes the UI only when the value changes. if self.applied_prefs.is_some() { @@ -1285,15 +1343,44 @@ impl FilmcraftApp { if self.session.trim_play.dynamic.is_some() && ctx.input_mut(|i| i.consume_key(egui::Modifiers::NONE, egui::Key::Escape)) { let _ = self.session.execute("trim.cancelDynamic", json!({})); } - // Panel shortcuts of the focused panel first: they override application shortcuts. + // Panel shortcuts of the focused panel override application shortcuts on the same chord. + // egui ignores extra Shift and Alt when matching, so the most specific chords go first + // across both scopes (the sort is stable, so the panel still wins a tie): otherwise the + // Timeline's `Delete` (Clear) swallows the app-wide `Shift+Delete` (Ripple Delete) (#680). let focused = self.ui.focused.title(); let mut fire = Vec::new(); ctx.input_mut(|i| { let modifiers = i.modifiers; + // With no text on the system clipboard, egui-winit drops the Ctrl+V press but still + // passes its release: put the press back, so Paste can paste files or an image (#611). + let mut swallowed = None; + for e in &i.events { + match e { + egui::Event::Key { key: egui::Key::V, pressed, modifiers: m, .. } => { + if !pressed && !self.v_down { + swallowed = Some(*m); + } + self.v_down = *pressed; + } + egui::Event::Paste(_) if modifiers.command => self.v_down = true, + _ => {} + } + } + if let Some(m) = swallowed { + i.events.push(egui::Event::Key { + key: egui::Key::V, + physical_key: Some(egui::Key::V), + pressed: true, + repeat: false, + modifiers: m | egui::Modifiers::COMMAND, + }); + } clipboard_events_as_keys(&mut i.events, modifiers); let panel = self.bindings.iter().filter(|b| b.3.as_deref() == Some(focused)); let app_wide = self.bindings.iter().filter(|b| b.3.is_none()); - for (m, k, id, _) in panel.chain(app_wide) { + let mut candidates: Vec<&menus::KeyBinding> = panel.chain(app_wide).collect(); + candidates.sort_by_key(|b| std::cmp::Reverse(menus::specificity(b.0))); + for (m, k, id, _) in candidates { if i.consume_key(*m, *k) { fire.push(id.clone()); } @@ -1301,11 +1388,27 @@ impl FilmcraftApp { }); for mut id in fire { // Select All / Deselect All act on the Project panel's items when it has focus (#168). - if self.ui.focused == PanelKind::Project && matches!(id.as_str(), "edit.selectAll" | "edit.deselectAll") { - id = id.replacen("edit.", "project.", 1); + // Select All takes only what the shown bin lists, not the whole project (#456). + if self.ui.focused == PanelKind::Project && id == "edit.selectAll" { + let (cmd, params) = panels::project::select_all_command(self); + if let Err(e) = menus::invoke(self, ctx, &cmd, params) { + self.ui.status = e; + } + continue; } + if self.ui.focused == PanelKind::Project && id == "edit.deselectAll" { + id = "project.deselectAll".into(); + } + // Clear in the Project panel removes the selected bin when no item is selected (#587). + let clear = if self.ui.focused == PanelKind::Project && id == "project.delete" { + crate::panels::project_views::clear_params(&self.session.state.project_selection, self.ui.project_panel.selected_bin) + } else { + None + }; // Mark In/Out in the Source monitor when it has focus. - let params = if self.ui.focused == PanelKind::Source + let params = if let Some(p) = clear { + p + } else if self.ui.focused == PanelKind::Source && (matches!(id.as_str(), "markers.markIn" | "markers.markOut" | "markers.clearInOut") || id.starts_with("markers.markSplit") || id.starts_with("markers.goToSplit")) @@ -1735,7 +1838,21 @@ impl eframe::App for FilmcraftApp { .position(|e| pointer(e) || matches!(e, egui::Event::Key { pressed: false, .. })) .map_or(self.synthetic.len(), |i| if pointer(&self.synthetic[i]) { i.max(1) } else { i + 1 }) }; - raw_input.events.extend(self.synthetic.drain(..n)); + // A press's modifiers go down before it and up after its release, as real keys would, + // so a drag that egui recognises frames after the press still sees Shift / Ctrl. + for e in self.synthetic.drain(..n) { + match e { + egui::Event::PointerButton { pressed: true, modifiers, .. } if !modifiers.is_none() => { + raw_input.events.push(egui::Event::ModifiersChanged(modifiers)); + raw_input.events.push(e); + } + egui::Event::PointerButton { pressed: false, modifiers, .. } if !modifiers.is_none() => { + raw_input.events.push(e); + raw_input.events.push(egui::Event::ModifiersChanged(egui::Modifiers::NONE)); + } + _ => raw_input.events.push(e), + } + } } } @@ -1954,6 +2071,20 @@ mod clipboard_key_tests { mod gpu_fallback_tests { use filmcraft_render::plan::{FramePlan, PlanLayer}; + /// Software rasterizers must not be treated as a GPU compositor: the preview crawled while + /// "GPU" usage read 0 % because wgpu had silently fallen back to llvmpipe. + #[test] + fn software_rasterizers_are_detected() { + use eframe::wgpu::DeviceType as D; + let r = super::rasterizer_reason; + assert!(r("llvmpipe (LLVM 23.1.1, 256 bits)", D::Cpu).is_some()); + assert!(r("NVIDIA GeForce RTX 4080", D::Cpu).is_some()); + assert!(r("SwiftShader Device", D::Other).is_some()); + assert!(r("softpipe", D::Cpu).is_some()); + assert!(r("AMD Radeon RX 5700 XT", D::DiscreteGpu).is_none()); + assert!(r("Apple M3 Pro", D::IntegratedGpu).is_none()); + } + #[test] fn plan_side_covers_output_and_every_layer() { let frame = |w: u32, h: u32| std::sync::Arc::new(filmcraft_frame::VideoFrame::rgba_f32(w, h, vec![0.0; (w * h * 4) as usize])); diff --git a/crates/ui-egui/src/menus.rs b/crates/ui-egui/src/menus.rs index 9dff12d3..86314ed2 100644 --- a/crates/ui-egui/src/menus.rs +++ b/crates/ui-egui/src/menus.rs @@ -287,7 +287,8 @@ pub fn invoke(app: &mut FilmcraftApp, ctx: &egui::Context, id: &str, mut params: } match id { "playback.slowForward" | "playback.slowReverse" if targets_source(app, ¶ms) => { - return Err("Source playback currently supports normal forward speed".into()); + app.shuttle_source(if id == "playback.slowForward" { 0.25 } else { -0.25 })?; + return Ok(json!({"playing": app.source_playback.clock.playing, "speed": app.source_playback.clock.speed})); } "playback.slowForward" | "playback.slowReverse" => { app.play(if id == "playback.slowForward" { 0.25 } else { -0.25 }); @@ -315,12 +316,12 @@ pub fn invoke(app: &mut FilmcraftApp, ctx: &egui::Context, id: &str, mut params: app.toggle_play(1.0); return Ok(json!({"playing": app.playback.playing})); } - "playback.forward" if targets_source(app, ¶ms) => { - app.play_source()?; - return Ok(json!({"playing": app.source_playback.clock.playing, "speed": 1.0})); - } - "playback.reverse" if targets_source(app, ¶ms) => { - return Err("Source playback currently supports normal forward speed; use Play/Space or frame stepping".into()); + // L / J: pressing again in the same direction doubles the speed, up to 8× + "playback.forward" | "playback.reverse" if targets_source(app, ¶ms) => { + let (clock, dir) = (&app.source_playback.clock, if id == "playback.forward" { 1.0 } else { -1.0 }); + let speed = if clock.playing && clock.speed * dir > 0.0 { (clock.speed * 2.0).clamp(-8.0, 8.0) } else { dir }; + app.shuttle_source(speed)?; + return Ok(json!({"playing": app.source_playback.clock.playing, "speed": app.source_playback.clock.speed})); } "playhead.stepBack" | "playhead.stepForward" | "playhead.stepBack5" | "playhead.stepForward5" if targets_source(app, ¶ms) => { app.stop_source(); @@ -490,6 +491,16 @@ pub fn invoke(app: &mut FilmcraftApp, ctx: &egui::Context, id: &str, mut params: app.dialog = Some(crate::Dialog::RevertConfirm); return Ok(json!({"dialog": "revert"})); } + // Files or an image on the system clipboard paste into the Timeline, or import into the + // Project panel (#611). Copying clips puts their names there as text, so media on the + // clipboard was copied after them; without media, Paste pastes the copied clips. + "edit.paste" | "edit.pasteInsert" + if params.as_object().is_none_or(|m| m.is_empty()) && matches!(app.ui.focused, PanelKind::Timeline | PanelKind::Project) => + { + if let Some(r) = paste_clipboard_media(app, id == "edit.pasteInsert") { + return r; + } + } "file.recover" if params.as_object().is_none_or(|m| m.is_empty()) => { if app.session.recovery_candidates().is_empty() { return Err("there are no unsaved changes to recover".into()); @@ -545,6 +556,32 @@ fn put_clip_names_on_system_clipboard(app: &FilmcraftApp, ctx: &egui::Context) { ctx.copy_text(if text.trim().is_empty() { format!("{} clips", names.len()) } else { text }); } +/// Paste the files or the image on the system clipboard: placed on the Timeline when it has focus, +/// imported into the shown bin in the Project panel. None when the clipboard holds neither. +fn paste_clipboard_media(app: &mut FilmcraftApp, insert: bool) -> Option> { + let media = (app.hooks.clipboard_media.as_mut()?)()?; + let paths = match media { + crate::ClipboardMedia::Files(paths) => paths, + crate::ClipboardMedia::Image { width, height, rgba } => match encode_png(width, height, rgba) + .and_then(|png| filmcraft_engine::paste_media::save_pasted_image(&mut app.session, &png).map_err(|e| e.to_string())) + { + Ok(path) => vec![path], + Err(e) => return Some(Err(e)), + }, + }; + let params = json!({"paths": paths, "insert": insert, "place": app.ui.focused == PanelKind::Timeline, "bin": app.import_bin().0}); + Some(app.session.execute("edit.pasteMedia", params).map_err(|e| e.to_string())) +} + +/// A compressed PNG of a clipboard image; an error when its size and pixels disagree. +fn encode_png(width: usize, height: usize, rgba: Vec) -> Result, String> { + let (w, h) = (u32::try_from(width).map_err(|e| e.to_string())?, u32::try_from(height).map_err(|e| e.to_string())?); + let img = image::RgbaImage::from_raw(w, h, rgba).ok_or("the image on the clipboard is malformed")?; + let mut png = Vec::new(); + img.write_to(&mut std::io::Cursor::new(&mut png), image::ImageFormat::Png).map_err(|e| e.to_string())?; + Ok(png) +} + /// A menu tree entry for display / `ui.menu.list`. #[derive(Clone, Debug, serde::Serialize)] pub struct MenuItem { @@ -683,10 +720,16 @@ pub fn bindings(app: &FilmcraftApp) -> Vec { let shifted: Vec = v.iter().filter(|b| b.0.shift).filter_map(|(m, k, id, p)| shifted_key(*k).map(|k2| (*m, k2, id.clone(), p.clone()))).collect(); v.extend(shifted); - v.sort_by_key(|(m, ..)| std::cmp::Reverse(m.command as u8 + m.shift as u8 + m.alt as u8 + m.ctrl as u8)); + v.sort_by_key(|(m, ..)| std::cmp::Reverse(specificity(*m))); v } +/// How many modifiers a chord names: chords naming more are matched first, because egui ignores +/// extra Shift and Alt when matching a key press. +pub fn specificity(m: egui::Modifiers) -> u8 { + m.command as u8 + m.shift as u8 + m.alt as u8 + m.ctrl as u8 +} + /// The key a US layout reports for `k` with Shift held, when it differs. fn shifted_key(k: egui::Key) -> Option { use egui::Key; diff --git a/crates/ui-egui/src/panels/dialogs.rs b/crates/ui-egui/src/panels/dialogs.rs index eed8fd28..8ee12750 100644 --- a/crates/ui-egui/src/panels/dialogs.rs +++ b/crates/ui-egui/src/panels/dialogs.rs @@ -306,18 +306,19 @@ fn add_tracks(app: &mut FilmcraftApp, ctx: &egui::Context) -> bool { keep } -/// Sequence ▸ Delete Tracks…: "Delete Video Tracks" / "Delete Audio Tracks" checkboxes, each with a -/// track choice (All Empty Tracks or one track). Returns whether the dialog stays open. +/// Sequence ▸ Delete Tracks…: "Delete Video Tracks" / "Delete Audio Tracks" (and, when the sequence +/// has caption tracks, "Delete Caption Tracks") checkboxes, each with a track choice (All Empty +/// Tracks or one track). Returns whether the dialog stays open. /// -/// Automation ids: `deleteTracks.video`, `deleteTracks.audio` (checkboxes), -/// `deleteTracks.video.target`, `deleteTracks.audio.target` (track menus), -/// `deleteTracks..option.` (menu entries while open), `deleteTracks.ok`, -/// `deleteTracks.cancel`. +/// Automation ids: `deleteTracks.video`, `deleteTracks.audio`, `deleteTracks.captions` (checkboxes), +/// `deleteTracks..target` (track menus), `deleteTracks..option.` +/// (menu entries while open), `deleteTracks.ok`, `deleteTracks.cancel`. fn delete_tracks(app: &mut FilmcraftApp, ctx: &egui::Context) -> bool { let Some(seq) = app.session.active_sequence() else { return false }; let names = |n: usize, p: &str| (1..=n).map(|i| format!("{p}{i}")).collect::>(); let vnames = names(seq.video_tracks.len(), "V"); let anames = names(seq.audio_tracks.len(), "A"); + let cnames = names(seq.caption_tracks.len(), "C"); let mut draft = app.ui.delete_tracks.clone(); let mut keep = true; let mut apply = false; @@ -332,7 +333,11 @@ fn delete_tracks(app: &mut FilmcraftApp, ctx: &egui::Context) -> bool { for (kind, title, delete, on, target, list) in [ ("video", tl!("Video Tracks"), tl!("Delete Video Tracks"), &mut draft.video, &mut draft.video_target, &vnames), ("audio", tl!("Audio Tracks"), tl!("Delete Audio Tracks"), &mut draft.audio, &mut draft.audio_target, &anames), + ("captions", tl!("Caption Tracks"), tl!("Delete Caption Tracks"), &mut draft.captions, &mut draft.captions_target, &cnames), ] { + if list.is_empty() { + continue; + } ui.add_space(4.0); ui.label(egui::RichText::new(title).strong()); let label = delete.to_string(); @@ -362,7 +367,7 @@ fn delete_tracks(app: &mut FilmcraftApp, ctx: &egui::Context) -> bool { keep = false; } let o = ui.add_enabled( - draft.video || draft.audio, + draft.video || draft.audio || (draft.captions && !cnames.is_empty()), egui::Button::new(egui::RichText::new(tl!("OK")).color(egui::Color32::WHITE)).fill(app.tokens.accent), ); elems.push(("deleteTracks.ok".into(), o.rect, "OK".into())); @@ -385,6 +390,9 @@ fn delete_tracks(app: &mut FilmcraftApp, ctx: &egui::Context) -> bool { if draft.audio { p.insert("audio".into(), serde_json::json!(draft.audio_target)); } + if draft.captions && !cnames.is_empty() { + p.insert("captions".into(), serde_json::json!(draft.captions_target)); + } if let Err(e) = app.session.execute("sequence.deleteTracks", serde_json::Value::Object(p)) { app.ui.status = e.to_string(); } diff --git a/crates/ui-egui/src/panels/effect_controls.rs b/crates/ui-egui/src/panels/effect_controls.rs index e1f3aebb..cfe7b6e1 100644 --- a/crates/ui-egui/src/panels/effect_controls.rs +++ b/crates/ui-egui/src/panels/effect_controls.rs @@ -9,6 +9,7 @@ use serde_json::{Value, json}; use crate::FilmcraftApp; use crate::icons::{self, Icon}; +use crate::state::KeyframeRef; use crate::theme::Tokens; const ROW_H: f32 = 22.0; @@ -17,6 +18,8 @@ const RULER_H: f32 = 24.0; const CLIP_BAR_H: f32 = 18.0; /// Distance between the keyframe navigator's buttons (◀ ◆ ▶). const NAV_STEP: f32 = 16.0; +/// The Y curve of a point parameter's value graph (X uses the accent colour). +const Y_CURVE: Color32 = Color32::from_rgb(0x60, 0xc0, 0x80); /// Properties panel: the keyframe diamonds' column, from a row's right edge (the arrows sit one /// [`NAV_STEP`] to either side of it, and a section's reset button above it). const PROPS_NAV_X: f32 = 26.0; @@ -41,6 +44,16 @@ fn graph_drag_offset(ui: &egui::Ui, response: &egui::Response, previous: egui::V } } +/// Whether `k` names a keyframe of the clip `it` (its effect, parameter and media time). +fn keyframe_exists(it: &TrackItem, k: &KeyframeRef) -> bool { + let Some(e) = it.effects.get(k.effect) else { return false }; + let param = match k.mask { + Some(m) => e.masks.get(m).and_then(|m| m.param(&k.param)), + None => e.params.get(&k.param), + }; + param.is_some_and(|p| p.keyframes.iter().any(|kf| kf.time == k.time)) +} + fn selected_clip(app: &FilmcraftApp) -> Option<(ClipId, TrackItem, TrackKind)> { selected_clips(app).into_iter().next() } @@ -71,6 +84,8 @@ fn selected_clips(app: &FilmcraftApp) -> Vec<(ClipId, TrackItem, TrackKind)> { pub fn show(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect) { let t = app.tokens; let clips = selected_clips(app); + // keep only selected keyframes that still exist on a clip shown here + app.ui.keyframe_selection.retain(|k| clips.iter().any(|(c, it, _)| c.0 == k.clip && keyframe_exists(it, k))); let Some((clip, it, _)) = clips.first().cloned() else { // a transition clicked in the Timeline (#430) if let Some(id) = crate::panels::transition_controls::selected(app) { @@ -207,7 +222,7 @@ pub fn show(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect) { if app.ui.expanded_fx.contains(&graph_key(clip, idx, pd.id)) && let Some(param) = e.params.get(pd.id) && param.is_animated() - && matches!(param.value, ParamValue::Float(_)) + && graphable(¶m.value) { graph_rows(app, bui, body, clip, idx, None, pd, param, &lane, it, &mut actions); } @@ -422,8 +437,8 @@ pub(crate) fn param_row( }; let (r, _) = ui.allocate_exact_size(vec2(body.width(), ROW_H), Sense::hover()); let mut x = r.min.x + 26.0; - // twirl-down for the value/velocity graphs (animated scalar params) - if param.is_animated() && matches!(param.value, ParamValue::Float(_)) { + // twirl-down for the value/velocity graphs (animated scalar and point params) + if param.is_animated() && graphable(¶m.value) { let key = graph_key(clip, idx, pkey); let open = app.ui.expanded_fx.contains(&key); let tw = Rect::from_center_size(pos2(r.min.x + 12.0, r.center().y), vec2(12.0, 12.0)); @@ -572,7 +587,13 @@ pub(crate) fn param_row( let kr = Rect::from_center_size(pos2(kx, y), vec2(11.0, 11.0)); let resp = ui.interact(kr.expand(2.0), id, Sense::click_and_drag()); app.auto.add(&format!("effectControls.{}.{}.keyframe.{}", e.effect, pkey, k.time.0), kr, "keyframe"); - let sel = k.time == mt || resp.dragged(); + // highlighted when selected (not merely under the playhead: that lit up every + // parameter's keyframe at the same time, #412) + let is_this = |s: &KeyframeRef| s.clip == clip.0 && s.effect == idx && s.mask == mask && s.param == pd.id && s.time == k.time; + if resp.clicked() || resp.drag_started() { + app.ui.keyframe_selection = vec![KeyframeRef { clip: clip.0, effect: idx, param: pd.id.to_string(), mask, time: k.time }]; + } + let sel = resp.dragged() || app.ui.keyframe_selection.iter().any(is_this); let col = if sel { t.hot_text } else { Color32::from_rgb(0xb0, 0xb0, 0xb0) }; match k.interp { filmcraft_project::Interpolation::Hold => { @@ -594,6 +615,10 @@ pub(crate) fn param_row( rate.snap_nearest(it.start + Tick(((f + off / lane.width()) as f64 * dur) as i64)).clamp(it.start, it.end() - rate.frame_duration()); let new_media = it.source_in + Tick(((new_tl - it.start).0 as f64 * it.speed.abs()) as i64); if new_media != k.time { + // the moved keyframe stays selected + for s in app.ui.keyframe_selection.iter_mut().filter(|s| is_this(s)) { + s.time = new_media; + } actions.push(( "effects.moveKeyframe".into(), with_mask(json!({"clip": clip.0, "effect": eff_json, "param": pd.id, "mediaTime": k.time.0, "to": new_media.0})), @@ -898,7 +923,22 @@ pub(crate) fn graph_key(clip: ClipId, idx: usize, pid: &str) -> String { format!("graph:{}:{}:{}", clip.0, idx, pid) } -/// Value and velocity graphs of an animated scalar parameter, drawn across the keyframe lane. +/// Parameters with value/velocity graphs: scalars, and points such as Position (#239). +pub(crate) fn graphable(v: &ParamValue) -> bool { + matches!(v, ParamValue::Float(_) | ParamValue::Vec2(_)) +} + +/// The curves a graphed value draws: a scalar's one, or a point's X and Y. +fn graph_components(v: &ParamValue) -> Vec { + match v { + ParamValue::Float(x) => vec![*x], + ParamValue::Vec2(p) => vec![p.x, p.y], + _ => Vec::new(), + } +} + +/// Value and velocity graphs of an animated scalar or point parameter, drawn across the keyframe +/// lane. A point draws an X and a Y curve, and its velocity graph is the speed along its path. /// Keyframes drag vertically (value); Bezier influence handles drag horizontally. #[allow(clippy::too_many_arguments)] pub(crate) fn graph_rows( @@ -928,15 +968,14 @@ pub(crate) fn graph_rows( let to_media = |f: f64| it.source_in + Tick((f * dur * speed) as i64); let to_f = |m: Tick| ((m - it.source_in).0 as f64 / speed / dur) as f32; let x_of = |f: f32| lane.min.x + f * lane.width(); - // samples + // samples: one row of curve values per sample (a point's NaN "auto" coordinate is skipped) let n = (lane.width() / 2.0).max(8.0) as usize; - let vals: Vec = (0..=n).map(|i| param.f64_at(to_media(i as f64 / n as f64))).collect(); - let (mut lo, mut hi) = vals.iter().fold((f64::MAX, f64::MIN), |(a, b), v| (a.min(*v), b.max(*v))); - for k in ¶m.keyframes { - if let ParamValue::Float(v) = k.value { - lo = lo.min(v); - hi = hi.max(v); - } + let vals: Vec> = (0..=n).map(|i| graph_components(¶m.value_at(to_media(i as f64 / n as f64)))).collect(); + let curves = graph_components(¶m.value).len(); + let at_keys: Vec> = param.keyframes.iter().map(|k| graph_components(&k.value)).collect(); + let (mut lo, mut hi) = vals.iter().chain(&at_keys).flatten().filter(|v| v.is_finite()).fold((f64::MAX, f64::MIN), |(a, b), v| (a.min(*v), b.max(*v))); + if lo > hi { + (lo, hi) = (0.0, 0.0); } if hi - lo < 1e-6 { lo -= 1.0; @@ -962,17 +1001,35 @@ pub(crate) fn graph_rows( p.text(pos2(vr.max.x - 30.0, area.min.y + 6.0), Align2::RIGHT_CENTER, format!("{hi:.dec$}"), Tokens::ui(10.0), t.text_dim); p.text(pos2(vr.max.x - 30.0, area.max.y - 6.0), Align2::RIGHT_CENTER, format!("{lo:.dec$}"), Tokens::ui(10.0), t.text_dim); p.text(pos2(vr.min.x + 44.0, area.center().y), Align2::LEFT_CENTER, tl!("Value"), Tokens::ui(11.0), t.text_dim); - let line: Vec = vals.iter().enumerate().map(|(i, v)| pos2(x_of(i as f32 / n as f32), y_of(*v))).collect(); - p.add(egui::Shape::line(line, Stroke::new(1.5, t.accent))); - // velocity (units per second, derivative of the sampled value) - let secs = dur * speed / filmcraft_time::TICKS_PER_SECOND as f64 / n as f64; - let vel: Vec = (0..n).map(|i| (vals[i + 1] - vals[i]) / secs.max(1e-9)).collect(); - let vmax = vel.iter().fold(1e-6f64, |a, v| a.max(v.abs())) * 1.15; + let curve_color = |c: usize| if c == 0 { t.accent } else { Y_CURVE }; + if curves > 1 { + p.text(pos2(vr.min.x + 44.0, area.center().y + 14.0), Align2::LEFT_CENTER, "X", Tokens::ui(10.5), curve_color(0)); + p.text(pos2(vr.min.x + 58.0, area.center().y + 14.0), Align2::LEFT_CENTER, "Y", Tokens::ui(10.5), curve_color(1)); + } + for c in 0..curves { + let line: Vec = + vals.iter().enumerate().filter_map(|(i, v)| v.get(c).filter(|v| v.is_finite()).map(|v| pos2(x_of(i as f32 / n as f32), y_of(*v)))).collect(); + p.add(egui::Shape::line(line, Stroke::new(1.5, curve_color(c)))); + } + // velocity (units per second, derivative of the sampled value; a point's speed along its path) + let secs = (dur * speed / filmcraft_time::TICKS_PER_SECOND as f64 / n as f64).max(1e-9); + let vel: Vec = vals + .windows(2) + .map(|w| { + let [a, b] = w else { return 0.0 }; + let d: Vec = a.iter().zip(b).map(|(a, b)| (b - a) / secs).collect(); + match d.as_slice() { + [v] => *v, + d => d.iter().map(|v| v * v).sum::().sqrt(), + } + }) + .collect(); + let vmax = vel.iter().filter(|v| v.is_finite()).fold(1e-6f64, |a, v| a.max(v.abs())) * 1.15; let varea = Rect::from_min_max(pos2(lane.min.x, velr.min.y + 2.0), pos2(lane.max.x, velr.max.y - 4.0)); p.rect_filled(varea, 0.0, t.keyframe_plot_bg); let vy = |v: f64| varea.center().y - (v / vmax) as f32 * varea.height() / 2.0; p.line_segment([pos2(varea.min.x, varea.center().y), pos2(varea.max.x, varea.center().y)], Stroke::new(1.0, t.keyframe_plot_axis)); - let vline: Vec = vel.iter().enumerate().map(|(i, v)| pos2(x_of((i as f32 + 0.5) / n as f32), vy(*v))).collect(); + let vline: Vec = vel.iter().enumerate().filter(|(_, v)| v.is_finite()).map(|(i, v)| pos2(x_of((i as f32 + 0.5) / n as f32), vy(*v))).collect(); p.add(egui::Shape::line(vline, Stroke::new(1.2, Color32::from_rgb(0xd0, 0xa0, 0x40)))); p.text(pos2(velr.min.x + 44.0, varea.center().y), Align2::LEFT_CENTER, tl!("Velocity"), Tokens::ui(11.0), t.text_dim); p.text(pos2(velr.max.x - 30.0, varea.min.y + 6.0), Align2::RIGHT_CENTER, format!("{vmax:.1}/s"), Tokens::ui(10.0), t.text_dim); @@ -980,55 +1037,83 @@ pub(crate) fn graph_rows( // keyframes + handles let ks = ¶m.keyframes; for (i, k) in ks.iter().enumerate() { - let ParamValue::Float(v) = k.value else { continue }; let f = to_f(k.time); if !(-0.01..=1.01).contains(&f) { continue; } - let id = egui::Id::new(("kfg", clip.0, idx, mask, pd.id, k.time.0)); - let dy: f32 = ui.data(|d| d.get_temp(id)).unwrap_or(0.0); - let c = pos2(x_of(f), y_of(v) + dy); - let r = Rect::from_center_size(c, vec2(10.0, 10.0)); - let resp = ui.interact(r.expand(2.0), id, Sense::drag()); - app.auto.add(&format!("effectControls.{}.graph.keyframe.{}", pd.id, k.time.0), r, "keyframe value"); - // influence handles: flat (ease) tangents with length ∝ influence × neighbouring segment let eases_out = !matches!(k.interp, filmcraft_project::Interpolation::Linear | filmcraft_project::Interpolation::Hold | filmcraft_project::Interpolation::EaseIn); let eases_in = !matches!(k.interp, filmcraft_project::Interpolation::Linear | filmcraft_project::Interpolation::Hold | filmcraft_project::Interpolation::EaseOut); - for (side, on, nb) in [(1.0f32, eases_out, ks.get(i + 1)), (-1.0f32, eases_in && i > 0, if i > 0 { ks.get(i - 1) } else { None })] { - let (true, Some(nb)) = (on, nb) else { continue }; - let seg = (x_of(to_f(nb.time)) - c.x).abs(); - let infl = if side > 0.0 { k.out_influence } else { k.in_influence } as f32; - let hid = id.with(if side > 0.0 { "out" } else { "in" }); - let hdx: f32 = ui.data(|d| d.get_temp(hid)).unwrap_or(0.0); - let hx = c.x + side * (infl * seg + hdx * side).clamp(seg * 0.01, seg); - let hp = pos2(hx, c.y); - p.line_segment([c, hp], Stroke::new(1.0, t.plot_handle_dim)); - p.circle_filled(hp, 3.5, t.keyframe_handle); - let hr = ui.interact(Rect::from_center_size(hp, vec2(10.0, 10.0)), hid.with("h"), Sense::drag()); - if hr.dragged() { - let nx = graph_drag_offset(ui, &hr, vec2(hdx, 0.0)).x; - ui.data_mut(|d| d.insert_temp(hid, nx)); + let comps = graph_components(&k.value); + for (ci, &v) in comps.iter().enumerate() { + if !v.is_finite() { + continue; } - if hr.drag_stopped() { - ui.data_mut(|d| d.remove::(hid)); - let ni = ((infl * seg + hdx * side) / seg.max(1.0)).clamp(0.01, 1.0); - let key = if side > 0.0 { "outInfluence" } else { "inInfluence" }; - actions.push(("effects.setKeyframe".into(), with_mask(json!({"clip": clip.0, "effect": idx, "param": pd.id, "mediaTime": k.time.0, key: ni})))); + let id = egui::Id::new(("kfg", clip.0, idx, mask, pd.id, k.time.0)); + let id = if ci == 0 { id } else { id.with(ci) }; + let dy: f32 = ui.data(|d| d.get_temp(id)).unwrap_or(0.0); + let c = pos2(x_of(f), y_of(v) + dy); + let r = Rect::from_center_size(c, vec2(10.0, 10.0)); + let resp = ui.interact(r.expand(2.0), id, Sense::drag()); + let auto_id = match (comps.len(), ci) { + (1, _) => format!("effectControls.{}.graph.keyframe.{}", pd.id, k.time.0), + (_, 0) => format!("effectControls.{}.graph.keyframe.{}.x", pd.id, k.time.0), + _ => format!("effectControls.{}.graph.keyframe.{}.y", pd.id, k.time.0), + }; + app.auto.add(&auto_id, r, "keyframe value"); + // influence handles: flat (ease) tangents with length ∝ influence × neighbouring segment; + // a point's sit on its X curve only, as both coordinates share the influence + let (eases_out, eases_in) = (ci == 0 && eases_out, ci == 0 && eases_in); + for (side, on, nb) in [(1.0f32, eases_out, ks.get(i + 1)), (-1.0f32, eases_in && i > 0, if i > 0 { ks.get(i - 1) } else { None })] { + let (true, Some(nb)) = (on, nb) else { continue }; + let seg = (x_of(to_f(nb.time)) - c.x).abs(); + let infl = if side > 0.0 { k.out_influence } else { k.in_influence } as f32; + let hid = id.with(if side > 0.0 { "out" } else { "in" }); + let hdx: f32 = ui.data(|d| d.get_temp(hid)).unwrap_or(0.0); + let hx = c.x + side * (infl * seg + hdx * side).clamp(seg * 0.01, seg); + let hp = pos2(hx, c.y); + p.line_segment([c, hp], Stroke::new(1.0, t.plot_handle_dim)); + p.circle_filled(hp, 3.5, t.keyframe_handle); + let hr = ui.interact(Rect::from_center_size(hp, vec2(10.0, 10.0)), hid.with("h"), Sense::drag()); + if hr.dragged() { + let nx = graph_drag_offset(ui, &hr, vec2(hdx, 0.0)).x; + ui.data_mut(|d| d.insert_temp(hid, nx)); + } + if hr.drag_stopped() { + ui.data_mut(|d| d.remove::(hid)); + let ni = ((infl * seg + hdx * side) / seg.max(1.0)).clamp(0.01, 1.0); + let key = if side > 0.0 { "outInfluence" } else { "inInfluence" }; + actions.push(( + "effects.setKeyframe".into(), + with_mask(json!({"clip": clip.0, "effect": idx, "param": pd.id, "mediaTime": k.time.0, key: ni})), + )); + } + } + p.circle_filled(c, 4.5, if resp.dragged() { t.hot_text } else { t.plot_handle }); + if resp.dragged() { + let ny = graph_drag_offset(ui, &resp, vec2(0.0, dy)).y; + ui.data_mut(|d| d.insert_temp(id, ny)); + p.text(c + vec2(8.0, -10.0), Align2::LEFT_BOTTOM, format!("{:.dec$}", v_of(c.y)), Tokens::ui(10.5), t.hot_text); + } + if resp.drag_stopped() { + ui.data_mut(|d| d.remove::(id)); + let nv = v_of(c.y); + let nv = if let ParamKind::Float { min, max, .. } = pd.kind { nv.clamp(min, max) } else { nv }; + let value = if comps.len() == 1 { + json!(nv) + } else { + let mut moved = comps.clone(); + if let Some(slot) = moved.get_mut(ci) { + *slot = nv; + } + json!(moved) + }; + actions.push(( + "effects.setKeyframe".into(), + with_mask(json!({"clip": clip.0, "effect": idx, "param": pd.id, "mediaTime": k.time.0, "value": value})), + )); } - } - p.circle_filled(c, 4.5, if resp.dragged() { t.hot_text } else { t.plot_handle }); - if resp.dragged() { - let ny = graph_drag_offset(ui, &resp, vec2(0.0, dy)).y; - ui.data_mut(|d| d.insert_temp(id, ny)); - p.text(c + vec2(8.0, -10.0), Align2::LEFT_BOTTOM, format!("{:.dec$}", v_of(c.y)), Tokens::ui(10.5), t.hot_text); - } - if resp.drag_stopped() { - ui.data_mut(|d| d.remove::(id)); - let nv = v_of(c.y); - let nv = if let ParamKind::Float { min, max, .. } = pd.kind { nv.clamp(min, max) } else { nv }; - actions.push(("effects.setKeyframe".into(), with_mask(json!({"clip": clip.0, "effect": idx, "param": pd.id, "mediaTime": k.time.0, "value": nv})))); } } } @@ -1212,33 +1297,33 @@ mod graph_drag_tests { ctx: egui::Context, clip: ClipId, time: f64, + param: &'static str, } impl Driver { fn new() -> Self { + Self::with("scale", ParamValue::Float(40.0), ParamValue::Float(80.0)) + } + + fn with(param: &'static str, from: ParamValue, to: ParamValue) -> Self { let mut session = filmcraft_engine::Session::default(); session.execute("file.openDemoProject", json!({})).unwrap(); let seq = session.state.active_sequence.unwrap(); let clip = session.active_sequence().unwrap().video_tracks[0].items[0].id; let it = std::sync::Arc::make_mut(&mut session.project).sequence_mut(seq).unwrap().find_item_mut(clip).unwrap().1; - let mut a = Keyframe::new(it.source_in + Tick(it.duration.0 / 4), ParamValue::Float(40.0)); - let mut b = Keyframe::new(it.source_in + Tick(it.duration.0 * 3 / 4), ParamValue::Float(80.0)); + let mut a = Keyframe::new(it.source_in + Tick(it.duration.0 / 4), from.clone()); + let mut b = Keyframe::new(it.source_in + Tick(it.duration.0 * 3 / 4), to); a.interp = Interpolation::Bezier; b.interp = Interpolation::Bezier; - it.effects - .iter_mut() - .find(|e| e.effect == "motion") - .unwrap() - .params - .insert("scale".into(), Param { value: ParamValue::Float(40.0), keyframes: vec![a, b] }); + it.effects.iter_mut().find(|e| e.effect == "motion").unwrap().params.insert(param.into(), Param { value: from, keyframes: vec![a, b] }); let app = FilmcraftApp::new(session); let ctx = egui::Context::default(); crate::theme::install(&ctx, &app.tokens); - Self { app, ctx, clip, time: 0.0 } + Self { app, ctx, clip, time: 0.0, param } } fn param(&self) -> Param { - self.app.session.active_sequence().unwrap().find_item(self.clip).unwrap().1.effect("motion").unwrap().params["scale"].clone() + self.app.session.active_sequence().unwrap().find_item(self.clip).unwrap().1.effect("motion").unwrap().params[self.param].clone() } fn undo_len(&mut self) -> usize { @@ -1254,11 +1339,11 @@ mod graph_drag_tests { let it = self.app.session.active_sequence().unwrap().find_item(self.clip).unwrap().1.clone(); let index = it.effects.iter().position(|e| e.effect == "motion").unwrap(); let effect = &it.effects[index]; - let pd = effect.def().unwrap().param("scale").unwrap(); + let pd = effect.def().unwrap().param(self.param).unwrap(); let body = Rect::from_min_size(pos2(8.0, 8.0), vec2(320.0, 200.0)); let lane = Rect::from_min_max(pos2(360.0, 8.0), pos2(760.0, 220.0)); let mut actions = Vec::new(); - graph_rows(&mut self.app, ui, body, self.clip, index, None, pd, &effect.params["scale"], &lane, &it, &mut actions); + graph_rows(&mut self.app, ui, body, self.clip, index, None, pd, &effect.params[self.param], &lane, &it, &mut actions); run(&mut self.app, ui.ctx(), actions); }); out.textures_delta.clear(); @@ -1331,4 +1416,26 @@ mod graph_drag_tests { } } } + + /// #239: Position (a point) has graphs too; dragging its X keyframe moves only X, in one undo step. + #[test] + fn position_graph_drags_one_coordinate_of_a_point_keyframe() { + use filmcraft_geom::Vec2; + let mut d = Driver::with("position", ParamValue::Vec2(Vec2::new(400.0, 300.0)), ParamValue::Vec2(Vec2::new(800.0, 500.0))); + assert!(graphable(&d.param().value)); + let before = d.param(); + let undo = d.undo_len(); + let start = d.frame(vec![])[0]; + assert!(d.app.auto.find(&format!("effectControls.position.graph.keyframe.{}.y", before.keyframes[0].time.0)).is_some()); + d.frame(vec![egui::Event::PointerMoved(start), button(start, true)]); + d.frame(vec![egui::Event::PointerMoved(start - vec2(0.0, 20.0))]); + d.frame(vec![button(start - vec2(0.0, 20.0), false)]); + d.frame(vec![]); + let changed = d.param(); + assert_eq!(d.undo_len(), undo + 1); + let (old, new) = (before.keyframes[0].value.as_vec2().unwrap(), changed.keyframes[0].value.as_vec2().unwrap()); + assert!(new.x > old.x, "dragging the X keyframe up raises X: {old:?} -> {new:?}"); + assert_eq!(new.y, old.y); + assert_eq!(changed.keyframes[1], before.keyframes[1]); + } } diff --git a/crates/ui-egui/src/panels/export_mode.rs b/crates/ui-egui/src/panels/export_mode.rs index b66ab552..e2298ec9 100644 --- a/crates/ui-egui/src/panels/export_mode.rs +++ b/crates/ui-egui/src/panels/export_mode.rs @@ -642,14 +642,22 @@ fn video_section(ui: &mut egui::Ui, reg: &mut Reg, s: &mut ExportSettings, t: &T }); } row(ui, t, tl!("Bitrate Encoding"), |ui| { - // a hardware encoder has one pass: H.265 offers no two-pass mode - let o = [BitrateMode::Cbr, BitrateMode::Vbr1Pass, BitrateMode::Vbr2Pass]; - let labels: Vec<(String, bool)> = o.iter().map(|m| (m.label().to_string(), !(hevc && *m == BitrateMode::Vbr2Pass))).collect(); + // a hardware encoder has one pass and no constant-quality mode: H.265 offers neither + let o = [BitrateMode::Cbr, BitrateMode::Vbr1Pass, BitrateMode::Vbr2Pass, BitrateMode::Crf]; + let labels: Vec<(String, bool)> = + o.iter().map(|m| (m.label().to_string(), !(hevc && matches!(m, BitrateMode::Vbr2Pass | BitrateMode::Crf)))).collect(); if let Some(i) = combo(ui, reg, "export.video.bitrateMode", s.bitrate_mode.label(), &labels, 140.0) { s.bitrate_mode = o[i]; } }); - if let Some(bpp) = s.adaptive_bitrate { + if s.bitrate_mode == BitrateMode::Crf { + row(ui, t, tl!("Quality (CRF)"), |ui| { + let mut crf = if s.crf.is_finite() { f64::from(s.crf) } else { f64::from(filmcraft_engine::export::DEFAULT_CRF) }; + if drag(ui, reg, "export.video.crf", &mut crf, 0.0..=51.0, 0.1, "", 0) { + s.crf = crf.round() as f32; + } + }); + } else if let Some(bpp) = s.adaptive_bitrate { row(ui, t, tl!("Target Bitrate"), |ui| { ui.label(egui::RichText::new(tlf!("Adaptive ({bpp} bits per pixel)", bpp)).size(12.0)); let r = ui.small_button(tl!("Set")); diff --git a/crates/ui-egui/src/panels/graphics.rs b/crates/ui-egui/src/panels/graphics.rs index 09aead4f..3060810a 100644 --- a/crates/ui-egui/src/panels/graphics.rs +++ b/crates/ui-egui/src/panels/graphics.rs @@ -186,7 +186,8 @@ fn handle_points(quad: &[Pos2; 4]) -> [Pos2; 8] { } /// Dragging handle `n` of `quad` from `start` to `cur`: the box grows towards the pointer while -/// the opposite corner / edge stays put. A corner scales both axes alike; an edge only its own. +/// the opposite corner / edge stays put. A corner scales each axis by the pointer's travel along +/// it (both alike with `keep_ratio`, Shift); an edge scales only its own. struct HandleScale { /// The handle opposite the dragged one, which does not move. pin: Pos2, @@ -195,10 +196,12 @@ struct HandleScale { uy: egui::Vec2, fx: f32, fy: f32, + /// Width and height scale apart (an edge, or a corner without Shift). + apart: bool, } impl HandleScale { - fn new(quad: &[Pos2; 4], n: usize, start: Pos2, cur: Pos2) -> Self { + fn new(quad: &[Pos2; 4], n: usize, start: Pos2, cur: Pos2, keep_ratio: bool) -> Self { let pts = handle_points(quad); let n = n.min(7); let pin = if n < 4 { pts[(n + 2) % 4] } else { pts[4 + (n - 4 + 2) % 4] }; @@ -209,14 +212,15 @@ impl HandleScale { if from.abs() < 1.0 { 1.0 } else { ((cur - pin).dot(dir) / from).max(0.01) } }; let (fx, fy) = match n { - 0..=3 => { + 0..=3 if keep_ratio => { let f = along((start - pin).normalized()); (f, f) } + 0..=3 => (along(ux), along(uy)), 4 | 6 => (1.0, along(uy)), _ => (along(ux), 1.0), }; - Self { pin, ux, uy, fx, fy } + Self { pin, ux, uy, fx, fy, apart: n >= 4 || !keep_ratio } } /// Where a screen point of the layer ends up. @@ -231,9 +235,29 @@ impl HandleScale { /// Premiere Pro 26 with the Program monitor at Fit. const NEAR_ANCHOR: f32 = 60.0; +/// Box `from` (`[x0, y0, x1, y1]`) with handle `n` moved by (`dx`, `dy`): that side or corner +/// follows, the opposite one stays, and the box keeps at least `min` across instead of flipping. +fn drag_sides(from: [f32; 4], n: usize, dx: f32, dy: f32, min: f32) -> [f32; 4] { + let [mut x0, mut y0, mut x1, mut y1] = from; + if matches!(n, 0 | 3 | 7) { + x0 = (x0 + dx).min(x1 - min); + } + if matches!(n, 1 | 2 | 5) { + x1 = (x1 + dx).max(x0 + min); + } + if matches!(n, 0 | 1 | 4) { + y0 = (y0 + dy).min(y1 - min); + } + if matches!(n, 2 | 3 | 6) { + y1 = (y1 + dy).max(y0 + min); + } + [x0, y0, x1, y1] +} + /// What dragging handle `n` of a layer from `start` to `cur` does, as Premiere Pro does it: point /// text scales about its anchor point, a paragraph-text box is resized (its text re-wraps at the -/// same size), and a shape stretches away from its opposite side. +/// same size), a shape's Size follows the handle (Scale untouched), and a path stretches away from +/// its opposite side. enum HandleDrag { Stretch(HandleScale), /// Scale both axes by `f` about the anchor point, which is at `anchor` on screen. @@ -248,36 +272,54 @@ enum HandleDrag { from: [f32; 4], tall: bool, }, + /// The shape's box becomes `rect` (in the layer's pixels; it was `from`, with Size `size`). + Size { + rect: [f32; 4], + from: [f32; 4], + size: (f32, f32), + }, } impl HandleDrag { - fn new(v: &LayerView, n: usize, start: Pos2, cur: Pos2) -> Self { + fn new(v: &LayerView, n: usize, start: Pos2, cur: Pos2, keep_ratio: bool) -> Self { let n = n.min(7); let quad = v.quad(); - let Some(t) = text_of(&v.spec) else { return Self::Stretch(HandleScale::new(&quad, n, start, cur)) }; + let (dx, dy) = match (v.to_local(start), v.to_local(cur)) { + (Some(a), Some(b)) => (b.0 - a.0, b.1 - a.1), + _ => (0.0, 0.0), + }; + let Some(t) = text_of(&v.spec) else { + let LayerContent::Shape(s) = &v.spec.content else { return Self::Stretch(HandleScale::new(&quad, n, start, cur, keep_ratio)) }; + // a path is its points, so it stretches; the other shapes are drawn from their Size + if s.shape == 3 { + return Self::Stretch(HandleScale::new(&quad, n, start, cur, keep_ratio)); + } + let from = v.local; + let mut rect = drag_sides(from, n, dx, dy, 1.0); + let (w, h) = (from[2] - from[0], from[3] - from[1]); + if keep_ratio && n < 4 && w > 0.0 && h > 0.0 { + // Shift: the side that moved further sets the scale of both; the opposite corner stays + let (fw, fh) = ((rect[2] - rect[0]) / w, (rect[3] - rect[1]) / h); + let f = if (fw - 1.0).abs() >= (fh - 1.0).abs() { fw } else { fh }; + let f = f.max(1.0 / w).max(1.0 / h); + if matches!(n, 0 | 3) { + rect[0] = rect[2] - w * f; + } else { + rect[2] = rect[0] + w * f; + } + if matches!(n, 0 | 1) { + rect[1] = rect[3] - h * f; + } else { + rect[3] = rect[1] + h * f; + } + } + return Self::Size { rect, from, size: s.size }; + }; if is_paragraph(t) { // a box without a height is as tall as its text let from = if t.box_height > 0.0 { [0.0, 0.0, t.box_width, t.box_height] } else { [0.0, v.local[1], t.box_width, v.local[3]] }; - let (dx, dy) = match (v.to_local(start), v.to_local(cur)) { - (Some(a), Some(b)) => (b.0 - a.0, b.1 - a.1), - _ => (0.0, 0.0), - }; - // the handle follows the pointer and the opposite side stays; the box does not flip - let min = (t.size * 0.5).max(1.0); - let [mut x0, mut y0, mut x1, mut y1] = from; - if matches!(n, 0 | 3 | 7) { - x0 = (x0 + dx).min(x1 - min); - } - if matches!(n, 1 | 2 | 5) { - x1 = (x1 + dx).max(x0 + min); - } - if matches!(n, 0 | 1 | 4) { - y0 = (y0 + dy).min(y1 - min); - } - if matches!(n, 2 | 3 | 6) { - y1 = (y1 + dy).max(y0 + min); - } - return Self::Resize { rect: [x0, y0, x1, y1], from, tall: t.box_height > 0.0 || !matches!(n, 5 | 7) }; + let rect = drag_sides(from, n, dx, dy, (t.size * 0.5).max(1.0)); + return Self::Resize { rect, from, tall: t.box_height > 0.0 || !matches!(n, 5 | 7) }; } // Every handle scales both axes alike. Only the pointer's travel along one side of the box // counts: along its width for the two side handles, along its height for all the others. @@ -305,12 +347,14 @@ impl HandleDrag { match self { Self::Stretch(h) => v.quad().map(|q| h.apply(q)), Self::Scale { anchor, f } => v.quad().map(|q| *anchor + (q - *anchor) * *f), - Self::Resize { rect: b, .. } => [(b[0], b[1]), (b[2], b[1]), (b[2], b[3]), (b[0], b[3])].map(|(x, y)| sp(&v.to_screen, x, y)), + Self::Resize { rect: b, .. } | Self::Size { rect: b, .. } => { + [(b[0], b[1]), (b[2], b[1]), (b[2], b[3]), (b[0], b[3])].map(|(x, y)| sp(&v.to_screen, x, y)) + } } } /// The `graphics.set` properties that carry out the drag. - fn props(&self, v: &LayerView, n: usize) -> Value { + fn props(&self, v: &LayerView) -> Value { let tr = &v.spec.transform; match self { Self::Stretch(h) => { @@ -322,8 +366,7 @@ impl HandleDrag { "scale_width": tr.scale.x * 100.0 * h.fx as f64, "position": [tr.position.x + delta.x, tr.position.y + delta.y], }); - if n >= 4 { - // an edge stretches one axis only + if h.apart { props["uniform_scale"] = json!(false); } props @@ -341,6 +384,18 @@ impl HandleDrag { } props } + Self::Size { rect, from, size } => { + // The box scales with Size about the layer's origin, so the origin moves to `o` (in + // the old layer pixels) to put the box at `rect`. The anchor point is renumbered so + // that it, the Position and the side that was not dragged stay where they are. + let f = |a: f32, b: f32| if b > 0.0 { a / b } else { 1.0 }; + let (fx, fy) = (f(rect[2] - rect[0], from[2] - from[0]), f(rect[3] - rect[1], from[3] - from[1])); + let o = (rect[0] - from[0] * fx, rect[1] - from[1] * fy); + json!({ + "size": [size.0 * fx, size.1 * fy], + "anchor": [tr.anchor.x - o.0 as f64, tr.anchor.y - o.1 as f64], + }) + } } } } @@ -386,6 +441,19 @@ const TEXT_EDIT_ID: &str = "gfx-text-edit"; fn end_edit(app: &mut FilmcraftApp, ui: &egui::Ui) { app.ui.gfx_edit = None; ui.memory_mut(|m| m.surrender_focus(egui::Id::new(TEXT_EDIT_ID))); + ui.data_mut(|d| d.remove::(egui::Id::new(TEXT_EDIT_ID).with("panel"))); +} + +/// A click on the Program monitor's empty space (the picture's, or around it): end the text edit +/// and let go of the selected graphic layers and mask (#683). +pub(crate) fn deselect_on_monitor(app: &mut FilmcraftApp, ui: &egui::Ui) { + end_edit(app, ui); + app.session.state.graphic_layers.clear(); + if app.session.state.selected_mask.is_some() + && let Err(e) = app.session.execute("masks.select", json!({"none": true})) + { + app.ui.status = e.to_string(); + } } /// Draw the overlay and handle graphics tools on the Program monitor picture `pic`. @@ -561,7 +629,7 @@ pub fn monitor_overlay(app: &mut FilmcraftApp, ui: &mut egui::Ui, pic: Rect, fra } DragKind::Handle(n) => { if let Some(v) = v { - let h = HandleDrag::new(v, n, d.start, cur); + let h = HandleDrag::new(v, n, d.start, cur, ui.input(|i| i.modifiers.shift)); painter.add(egui::Shape::closed_line(h.quad(v).to_vec(), Stroke::new(1.0, Color32::WHITE))); } } @@ -614,7 +682,7 @@ pub fn monitor_overlay(app: &mut FilmcraftApp, ui: &mut egui::Ui, pic: Rect, fra } DragKind::Handle(n) if moved => { if let Some(v) = v { - let props = HandleDrag::new(v, n, d.start, cur).props(v, n); + let props = HandleDrag::new(v, n, d.start, cur, ui.input(|i| i.modifiers.shift)).props(v); actions.push(("graphics.set".into(), json!({"clip": v.clip.0, "layer": v.layer, "props": props}))); } } @@ -678,10 +746,8 @@ pub fn monitor_overlay(app: &mut FilmcraftApp, ui: &mut egui::Ui, pic: Rect, fra end_edit(app, ui); } actions.push(("graphics.selectLayer".into(), json!({"clip": v.clip.0, "layers": [v.layer]}))); - } else if app.ui.gfx_edit.is_some() { - end_edit(app, ui); - } else if sel_clip.is_some() && !sel_layers.is_empty() { - app.session.state.graphic_layers.clear(); + } else { + deselect_on_monitor(app, ui); } } Tool::Pen => { @@ -806,10 +872,26 @@ fn start_edit_at(app: &mut FilmcraftApp, ui: &egui::Ui, v: &LayerView, p: Pos2, /// Keyboard handling while a text layer is being edited. fn text_edit(app: &mut FilmcraftApp, ui: &mut egui::Ui, v: &LayerView, mut ed: GfxEdit, resp: &egui::Response, actions: &mut Vec<(String, Value)>) { let _ = resp; + // moving to another panel ends the edit; Properties and Essential Graphics style the text + // being edited, so working there keeps it (#683). Only a move counts: an edit started with + // another panel focused (an agent's `ui.set`) goes on until the focus moves. + use crate::dock::PanelKind; + let panel_id = egui::Id::new(TEXT_EDIT_ID).with("panel"); + let before = ui.data(|d| d.get_temp::(panel_id)); + ui.data_mut(|d| d.insert_temp(panel_id, app.ui.focused)); + if before.is_some_and(|b| b != app.ui.focused) && !matches!(app.ui.focused, PanelKind::Program | PanelKind::Properties | PanelKind::EssentialGraphics) { + end_edit(app, ui); + return; + } let eid = egui::Id::new(TEXT_EDIT_ID); // a real (non-clickable) widget holds keyboard focus so shortcuts pause while typing let fr = ui.interact(Rect::from_points(&v.quad()), eid, Sense::focusable_noninteractive()); fr.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::TextEdit, true, "Text layer")); + // while another field has the keyboard (a value typed in Properties) the keys are its own; + // the text takes the keyboard back once nothing has it (#683) + if ui.memory(|m| m.focused().is_some_and(|f| f != eid)) { + return; + } ui.memory_mut(|m| { m.request_focus(eid); m.set_focus_lock_filter(eid, egui::EventFilter { tab: true, horizontal_arrows: true, vertical_arrows: true, escape: true }); diff --git a/crates/ui-egui/src/panels/media_dialogs.rs b/crates/ui-egui/src/panels/media_dialogs.rs index de3370d2..cef30cd8 100644 --- a/crates/ui-egui/src/panels/media_dialogs.rs +++ b/crates/ui-egui/src/panels/media_dialogs.rs @@ -162,120 +162,125 @@ fn link_media(app: &mut FilmcraftApp, ctx: &egui::Context) { let sel_path = d.candidate.and_then(|c| d.candidates.get(c)).map(|c| c.0.clone()); let preview = sel_path.as_deref().and_then(|p| preview_texture(app, ctx, p)); let rows_max_h = (ctx.content_rect().height() * 0.3).clamp(90.0, 320.0); + let width = (ctx.content_rect().width() - 40.0).clamp(1.0, 760.0); egui::Window::new(tl!("Link Media")) .id(egui::Id::new("Link Media")) .collapsible(false) .resizable(false) - .default_width(760.0) + .default_width(width) + .max_width(width) .anchor(egui::Align2::CENTER_CENTER, [0.0, 0.0]) .show(ctx, |ui| { - ui.label(tlf!("{n} clip(s) can't find their media. Locate them, search a folder, or leave them offline.", n = rows.len())); - ui.add_space(6.0); - egui::ScrollArea::vertical().id_salt("link-media-rows-scroll").max_height(rows_max_h).auto_shrink([false, true]).show(ui, |ui| { - egui::Grid::new("link-media-rows").num_columns(4).striped(true).spacing([14.0, 4.0]).show(ui, |ui| { - for h in [tl!("Clip Name"), tl!("File Name"), tl!("File Path"), tl!("Status")] { - ui.label(RichText::new(h).strong()); - } - ui.end_row(); - for (k, (_, name, fname, path, status)) in rows.iter().enumerate() { - let r = ui.selectable_label(d.row == k, name); - push(&mut elems, format!("linkMedia.row.{k}"), &r, name.clone()); - if r.clicked() && d.row != k { - d.row = k; - d.candidates.clear(); - d.candidate = None; + egui::ScrollArea::vertical().id_salt("link-media-body-scroll").max_height((ctx.content_rect().height() - 140.0).max(1.0)).show(ui, |ui| { + ui.label(tlf!("{n} clip(s) can't find their media. Locate them, search a folder, or leave them offline.", n = rows.len())); + ui.add_space(6.0); + egui::ScrollArea::both().id_salt("link-media-rows-scroll").max_height(rows_max_h).auto_shrink([false, true]).show(ui, |ui| { + egui::Grid::new("link-media-rows").num_columns(4).striped(true).spacing([14.0, 4.0]).show(ui, |ui| { + for h in [tl!("Clip Name"), tl!("File Name"), tl!("File Path"), tl!("Status")] { + ui.label(RichText::new(h).strong()); } - ui.label(fname); - ui.label(RichText::new(path).small()); - ui.label(RichText::new(*status).color(Color32::from_rgb(0xe0, 0x5a, 0x5a))); ui.end_row(); - } - }); - }); - ui.separator(); - ui.horizontal(|ui| { - ui.label(tl!("Match file properties:")); - for (id, label, v) in [ - ("fileName", tl!("File Name"), &mut d.file_name), - ("extension", tl!("File Extension"), &mut d.extension), - ("clipId", tl!("Clip ID (fingerprint)"), &mut d.clip_id), - ("duration", tl!("Duration"), &mut d.duration), - ("mediaStart", tl!("Media Start"), &mut d.media_start), - ("metadata", tl!("Frame Size / Rate"), &mut d.metadata), - ] { - let r = ui.checkbox(v, label); - push(&mut elems, format!("linkMedia.match.{id}"), &r, label); - } - }); - ui.horizontal(|ui| { - let r = ui.checkbox(&mut d.align_timecode, tl!("Align Timecode")); - push(&mut elems, "linkMedia.alignTimecode", &r, "Align Timecode"); - let r = ui.checkbox(&mut d.relink_others, tl!("Relink others automatically")); - push(&mut elems, "linkMedia.relinkOthers", &r, "Relink others automatically"); - }); - ui.separator(); - ui.horizontal(|ui| { - ui.label(tl!("Search in:")); - let r = ui.add(egui::TextEdit::singleline(&mut d.folder).desired_width(360.0).hint_text(tl!("Folder"))); - push(&mut elems, "linkMedia.folder", &r, "folder"); - let r = ui.button(tl!("Browse…")); - push(&mut elems, "linkMedia.browse", &r, "Browse…"); - if r.clicked() { - action = Some("browse"); - } - let r = ui.checkbox(&mut d.exact_name, tl!("Exact name matches only")); - push(&mut elems, "linkMedia.exactName", &r, "Exact name matches only"); - let r = ui.button(tl!("Search")); - push(&mut elems, "linkMedia.search", &r, "Search"); - if r.clicked() { - action = Some("search"); - } - }); - ui.horizontal(|ui| { - ui.vertical(|ui| { - ui.set_width(520.0); - if d.candidates.is_empty() { - ui.label(RichText::new(tl!("No candidates yet — Search a folder or Locate the file.")).weak()); - } - egui::ScrollArea::vertical().id_salt("link-media-candidates-scroll").max_height(108.0).auto_shrink([false, true]).show(ui, |ui| { - for (k, (path, ok, idm, problems)) in d.candidates.iter().enumerate() { - let mark = match (ok, idm) { - (true, Some(true)) => tl!("✔ same file"), - (true, _) => tl!("✔ matches"), - (false, Some(false)) => tl!("✖ different file"), - _ => "✖", - }; - let text = format!("{mark} {path}"); - let r = ui.selectable_label( - d.candidate == Some(k), - RichText::new(&text).color(if *ok { Color32::LIGHT_GREEN } else { Color32::from_rgb(0xe0, 0x8a, 0x6a) }), - ); - let r = if problems.is_empty() { r } else { r.on_hover_text(problems) }; - push(&mut elems, format!("linkMedia.candidate.{k}"), &r, text); - if r.clicked() { - d.candidate = Some(k); + for (k, (_, name, fname, path, status)) in rows.iter().enumerate() { + let r = ui.selectable_label(d.row == k, name); + push(&mut elems, format!("linkMedia.row.{k}"), &r, name.clone()); + if r.clicked() && d.row != k { + d.row = k; + d.candidates.clear(); + d.candidate = None; } + ui.label(fname); + ui.label(RichText::new(path).small()); + ui.label(RichText::new(*status).color(Color32::from_rgb(0xe0, 0x5a, 0x5a))); + ui.end_row(); } }); }); - let (rect, _) = ui.allocate_exact_size(egui::vec2(192.0, 108.0), egui::Sense::hover()); - ui.painter().rect_filled(rect, 3.0, Color32::from_rgb(12, 12, 12)); - match &preview { - Some(t) => { - let fitted = crate::panels::monitor::fit(rect, t.size()[0] as f32, t.size()[1] as f32); - ui.painter().image(t.id(), fitted, egui::Rect::from_min_max(egui::pos2(0.0, 0.0), egui::pos2(1.0, 1.0)), Color32::WHITE); + ui.separator(); + ui.horizontal_wrapped(|ui| { + ui.label(tl!("Match file properties:")); + for (id, label, v) in [ + ("fileName", tl!("File Name"), &mut d.file_name), + ("extension", tl!("File Extension"), &mut d.extension), + ("clipId", tl!("Clip ID (fingerprint)"), &mut d.clip_id), + ("duration", tl!("Duration"), &mut d.duration), + ("mediaStart", tl!("Media Start"), &mut d.media_start), + ("metadata", tl!("Frame Size / Rate"), &mut d.metadata), + ] { + let r = ui.checkbox(v, label); + push(&mut elems, format!("linkMedia.match.{id}"), &r, label); } - None => { - ui.painter().text(rect.center(), egui::Align2::CENTER_CENTER, tl!("Preview"), crate::theme::Tokens::ui(11.0), Color32::GRAY); + }); + ui.horizontal_wrapped(|ui| { + let r = ui.checkbox(&mut d.align_timecode, tl!("Align Timecode")); + push(&mut elems, "linkMedia.alignTimecode", &r, "Align Timecode"); + let r = ui.checkbox(&mut d.relink_others, tl!("Relink others automatically")); + push(&mut elems, "linkMedia.relinkOthers", &r, "Relink others automatically"); + }); + ui.separator(); + ui.horizontal_wrapped(|ui| { + ui.label(tl!("Search in:")); + let r = ui.add(egui::TextEdit::singleline(&mut d.folder).desired_width((ui.available_width() * 0.5).min(360.0)).hint_text(tl!("Folder"))); + push(&mut elems, "linkMedia.folder", &r, "folder"); + let r = ui.button(tl!("Browse…")); + push(&mut elems, "linkMedia.browse", &r, "Browse…"); + if r.clicked() { + action = Some("browse"); } + let r = ui.checkbox(&mut d.exact_name, tl!("Exact name matches only")); + push(&mut elems, "linkMedia.exactName", &r, "Exact name matches only"); + let r = ui.button(tl!("Search")); + push(&mut elems, "linkMedia.search", &r, "Search"); + if r.clicked() { + action = Some("search"); + } + }); + ui.horizontal(|ui| { + let candidates_width = (ui.available_width() - 192.0 - ui.spacing().item_spacing.x).max(1.0); + ui.vertical(|ui| { + ui.set_width(candidates_width); + if d.candidates.is_empty() { + ui.label(RichText::new(tl!("No candidates yet — Search a folder or Locate the file.")).weak()); + } + egui::ScrollArea::vertical().id_salt("link-media-candidates-scroll").max_height(108.0).auto_shrink([false, true]).show(ui, |ui| { + for (k, (path, ok, idm, problems)) in d.candidates.iter().enumerate() { + let mark = match (ok, idm) { + (true, Some(true)) => tl!("✔ same file"), + (true, _) => tl!("✔ matches"), + (false, Some(false)) => tl!("✖ different file"), + _ => "✖", + }; + let text = format!("{mark} {path}"); + let r = ui.selectable_label( + d.candidate == Some(k), + RichText::new(&text).color(if *ok { Color32::LIGHT_GREEN } else { Color32::from_rgb(0xe0, 0x8a, 0x6a) }), + ); + let r = if problems.is_empty() { r } else { r.on_hover_text(problems) }; + push(&mut elems, format!("linkMedia.candidate.{k}"), &r, text); + if r.clicked() { + d.candidate = Some(k); + } + } + }); + }); + let (rect, _) = ui.allocate_exact_size(egui::vec2(192.0, 108.0), egui::Sense::hover()); + ui.painter().rect_filled(rect, 3.0, Color32::from_rgb(12, 12, 12)); + match &preview { + Some(t) => { + let fitted = crate::panels::monitor::fit(rect, t.size()[0] as f32, t.size()[1] as f32); + ui.painter().image(t.id(), fitted, egui::Rect::from_min_max(egui::pos2(0.0, 0.0), egui::pos2(1.0, 1.0)), Color32::WHITE); + } + None => { + ui.painter().text(rect.center(), egui::Align2::CENTER_CENTER, tl!("Preview"), crate::theme::Tokens::ui(11.0), Color32::GRAY); + } + } + elems.push(("linkMedia.preview".into(), rect, "preview".into())); + }); + if !d.message.is_empty() { + ui.colored_label(Color32::from_rgb(0xe0, 0x8a, 0x6a), &d.message); } - elems.push(("linkMedia.preview".into(), rect, "preview".into())); }); - if !d.message.is_empty() { - ui.colored_label(Color32::from_rgb(0xe0, 0x8a, 0x6a), &d.message); - } ui.add_space(6.0); - ui.horizontal(|ui| { + ui.horizontal_wrapped(|ui| { for (id, label) in [("offlineAll", tl!("Offline All")), ("offline", tl!("Offline")), ("cancel", tl!("Cancel")), ("locate", tl!("Locate…"))] { let r = ui.button(label); push(&mut elems, format!("linkMedia.{id}"), &r, label); diff --git a/crates/ui-egui/src/panels/mixer.rs b/crates/ui-egui/src/panels/mixer.rs index db2c92d7..bcc1b42a 100644 --- a/crates/ui-egui/src/panels/mixer.rs +++ b/crates/ui-egui/src/panels/mixer.rs @@ -1010,8 +1010,23 @@ pub fn clip_mixer(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect) { let m = meters.get(&tr.id.0).cloned().unwrap_or_default(); draw_meters(ui, mrect, &m, if m.is_empty() { filmcraft_render::mixer::width_of(tr.channels) } else { m.len() }, &t); } - small_text(ui, pos2(sr.min.x + 8.0, sr.max.y - 34.0), Align2::LEFT_CENTER, &db_text(vol), 11.5, dimc(t.hot_text)); - app.auto.add(&format!("{ap}.value"), Rect::from_min_size(pos2(sr.min.x + 4.0, sr.max.y - 42.0), vec2(50.0, 16.0)), &db_text(vol)); + // dB readout: drag to scrub, click to type a value (clamped to the fader range), like the track mixer + let vr = Rect::from_min_size(pos2(sr.min.x + 4.0, sr.max.y - 42.0), vec2(50.0, 16.0)); + let mut vui = ui.new_child(egui::UiBuilder::new().max_rect(vr)); + if !active { + vui.set_opacity(0.4); + } + let (vresp, nv) = crate::widgets::hot_number(&mut vui, egui::Id::new((&ap, "value")), vol.max(-96.0), 0.1, (FADER_MIN_DB, FADER_MAX_DB), 1, "", &t); + app.auto.add(&format!("{ap}.value"), vresp.rect, &db_text(vol)); + if let (Some(v), true) = (nv, active) { + set(&mut acts, "volume", "level", v, !vresp.dragged() || vresp.drag_started()); + if !vresp.dragged() { + release(&mut acts, "volume"); + } + } + if vresp.drag_stopped() { + release(&mut acts, "volume"); + } small_text(ui, pos2(sr.min.x + 6.0, sr.max.y - 10.0), Align2::LEFT_CENTER, &label, 12.0, t.text); small_text(ui, pos2(sr.min.x + 30.0, sr.max.y - 10.0), Align2::LEFT_CENTER, &tr.name, 12.0, t.text); } diff --git a/crates/ui-egui/src/panels/monitor.rs b/crates/ui-egui/src/panels/monitor.rs index 11f4bc2d..d94867f3 100644 --- a/crates/ui-egui/src/panels/monitor.rs +++ b/crates/ui-egui/src/panels/monitor.rs @@ -279,6 +279,11 @@ pub fn show(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect, which: Which) let c = if app.playback.meter.counts().1 > 0 { t.render_yellow } else { t.render_green }; ui.painter().circle_filled(pos2(video_area.min.x + 10.0, video_area.min.y + 10.0), 4.0, c); } + // A click on the Program monitor around the picture lets go of what is selected on it (#683; + // in the picture the overlays do that). Under the picture and the overlays, so they come first. + if which == Which::Program && show_picture && ui.interact(video_area, egui::Id::new((prefix, "background")), Sense::click()).clicked() { + crate::panels::graphics::deselect_on_monitor(app, ui); + } // Click/drag in the picture: the Hand tool pans a magnified picture, otherwise focus. let pic_resp = ui.interact(pic, egui::Id::new((prefix, "pic")), if show_picture { Sense::click_and_drag() } else { Sense::hover() }); app.auto.add(&format!("{prefix}.picture"), pic.intersect(video_area), "picture"); @@ -692,6 +697,19 @@ fn mini_timeline( } } +// A hidden transport action is still available through its regular shortcut/menu. +fn transport_visible(hidden: &[String], command: &str) -> bool { + !hidden.iter().any(|id| id == command) +} + +fn set_transport_visible(hidden: &mut Vec, command: &str, visible: bool) { + if visible { + hidden.retain(|id| id != command); + } else if transport_visible(hidden, command) { + hidden.push(command.to_owned()); + } +} + fn transport(app: &mut FilmcraftApp, ui: &mut egui::Ui, row: Rect, which: Which) { let t = app.tokens; let src = which == Which::Source; @@ -726,12 +744,21 @@ fn transport(app: &mut FilmcraftApp, ui: &mut egui::Ui, row: Rect, which: Which) (Icon::Proxy, "media.toggleProxies", tl!("Toggle Proxies")), ] }; + let mut buttons = buttons; + if !src { + // Optional control, discoverable through the editor without changing existing layouts. + buttons.push((Icon::Loop, "playback.loop", tl!("Loop"))); + } + let hidden = monitor_view::view(app, which).transport_hidden.clone(); let bw = 30.0; - let total = buttons.len() as f32 * bw; + let total = buttons.iter().filter(|(_, cmd, _)| transport_visible(&hidden, cmd)).count() as f32 * bw; let mut x = row.center().x - total / 2.0; let ctx = ui.ctx().clone(); let prefix = if src { "source" } else { "program" }; - for (icon, cmd, tip) in buttons { + for &(icon, cmd, tip) in &buttons { + if !transport_visible(&hidden, cmd) { + continue; + } let r = Rect::from_min_size(pos2(x, row.min.y + 2.0), vec2(bw - 2.0, 26.0)); let resp = ui.interact(r, egui::Id::new((prefix, cmd)), Sense::click()).on_hover_text(tip); app.auto.add(&format!("{prefix}.transport.{cmd}"), r, tip); @@ -767,10 +794,32 @@ fn transport(app: &mut FilmcraftApp, ui: &mut egui::Ui, row: Rect, which: Which) } x += bw; } - // button editor "+" + // Customize this monitor's transport separately from the other monitor. let r = Rect::from_min_size(pos2(row.max.x - 30.0, row.min.y + 4.0), vec2(22.0, 22.0)); let resp = ui.interact(r, egui::Id::new((prefix, "btn-editor")), Sense::click()).on_hover_text(tl!("Button Editor")); + app.auto.add(&format!("{prefix}.transport.editor"), r, "Button Editor"); icons::paint(ui.painter(), r.shrink(5.0), Icon::Plus, if resp.hovered() { t.tab_text_active } else { t.text_dim }); + let mut edited_hidden = hidden.clone(); + egui::Popup::menu(&resp).close_behavior(egui::PopupCloseBehavior::CloseOnClickOutside).show(|ui| { + ui.set_min_width(225.0); + for &(_, cmd, tip) in &buttons { + let mut visible = transport_visible(&edited_hidden, cmd); + let checkbox = ui.checkbox(&mut visible, tip); + app.auto.add(&format!("{prefix}.transport.editor.{cmd}"), checkbox.rect, tip); + if checkbox.changed() { + set_transport_visible(&mut edited_hidden, cmd, visible); + } + } + ui.separator(); + let reset = ui.button(tl!("Reset")); + app.auto.add(&format!("{prefix}.transport.editor.reset"), reset.rect, "Reset"); + if reset.clicked() { + edited_hidden = crate::state::MonitorView::default().transport_hidden; + } + }); + if edited_hidden != hidden { + monitor_view::view_mut(app, which).transport_hidden = edited_hidden; + } } fn source_nav(app: &mut FilmcraftApp, cmd: &str) { @@ -797,6 +846,21 @@ mod tests { Rgba { w, h, px: rgba.iter().copied().cycle().take(w * h * 4).collect() } } + #[test] + fn button_editor_updates_visibility_without_affecting_other_monitor() { + let mut program = crate::state::MonitorView::default().transport_hidden; + let source = program.clone(); + assert!(!transport_visible(&program, "playback.loop")); + set_transport_visible(&mut program, "playback.loop", true); + set_transport_visible(&mut program, "playback.loop", true); + assert!(transport_visible(&program, "playback.loop")); + assert!(program.is_empty(), "enabling twice must not leave hidden entries"); + set_transport_visible(&mut program, "playhead.stepForward", false); + set_transport_visible(&mut program, "playhead.stepForward", false); + assert_eq!(program, vec!["playhead.stepForward"]); + assert!(transport_visible(&source, "playhead.stepForward")); + } + #[test] fn eyedropper_averages_the_pixels_around_the_point() { let mut img = flat(8, 8, [0, 200, 0, 255]); diff --git a/crates/ui-egui/src/panels/project.rs b/crates/ui-egui/src/panels/project.rs index 09b04069..551f5cce 100644 --- a/crates/ui-egui/src/panels/project.rs +++ b/crates/ui-egui/src/panels/project.rs @@ -254,6 +254,21 @@ pub fn visible_items(app: &FilmcraftApp, inst: Inst) -> Vec { crate::panels::project_views::items_in_view(app, &v, &app.ui.project_search.to_ascii_lowercase()) } +/// The command Select All (Cmd+A) runs with the Project panel focused: the items of the shown +/// bin, plus those of sub-bins twirled open in List view (`project.selectAll {bin, expanded}`). +/// Items in closed bins and in bins of other tabs stay unselected, as in Premiere (#456). While a +/// search filters the panel, exactly the rows it shows (`project.select {items}`). +pub fn select_all_command(app: &FilmcraftApp) -> (String, Value) { + let inst = shown_inst(app); + if !app.ui.project_search.is_empty() { + let items: Vec = visible_items(app, inst).into_iter().map(|i| i.0).collect(); + return ("project.select".into(), json!({"items": items})); + } + let v = view_of(app, inst); + let expanded: &[u64] = if v.mode == ViewMode::List { &app.ui.expanded_bins } else { &[] }; + ("project.selectAll".into(), json!({"bin": v.bin.0, "expanded": expanded})) +} + // ------------------------------------------------------------------------------------ panel pub fn show(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect) { @@ -609,7 +624,10 @@ fn footer(app: &mut FilmcraftApp, ui: &mut egui::Ui, bar: Rect, v: &View, action let parent = (v.bin != app.session.project.root.id).then_some(v.bin.0); actions.push(("file.newBin".into(), json!({"name": "New Bin", "parent": parent}))); } - "project.delete" if app.session.state.project_selection.is_empty() => app.ui.status = tl!("Select items to clear").into(), + "project.delete" => match crate::panels::project_views::clear_params(&app.session.state.project_selection, app.ui.project_panel.selected_bin) { + Some(p) => actions.push((id.into(), p)), + None => app.ui.status = tl!("Select items to clear").into(), + }, _ => actions.push((id.into(), json!({}))), } } @@ -673,6 +691,7 @@ pub fn panel_menu(app: &mut FilmcraftApp, ui: &mut egui::Ui) -> bool { let inst = shown_inst(app); let v = view_of(app, inst); let has_sel = !app.session.state.project_selection.is_empty() || app.ui.project_panel.selected_bin.is_some(); + let clear = crate::panels::project_views::clear_params(&app.session.state.project_selection, app.ui.project_panel.selected_bin); let has_project_path = app.session.path.is_some(); let sc = |id: &str| app.session.shortcuts.primary(id); let (s_close, s_save, s_bin, s_find) = (sc("file.closeProject"), sc("file.save"), sc("file.newBin"), sc("edit.find")); @@ -712,8 +731,8 @@ pub fn panel_menu(app: &mut FilmcraftApp, ui: &mut egui::Ui) -> bool { if item(ui, "rename", tl!("Rename"), has_sel, None) { actions.push(("projectPanel.rename".into(), json!({}))); } - if item(ui, "delete", tl!("Delete"), !app.session.state.project_selection.is_empty(), Some("Backspace")) { - actions.push(("project.delete".into(), json!({}))); + if item(ui, "delete", tl!("Delete"), clear.is_some(), Some("Backspace")) { + actions.push(("project.delete".into(), clear.unwrap_or_default())); } ui.separator(); if item(ui, "automateToSequence", tl!("Automate to Sequence…"), !app.session.state.project_selection.is_empty(), None) { diff --git a/crates/ui-egui/src/panels/project_views.rs b/crates/ui-egui/src/panels/project_views.rs index 7a54afae..46d80023 100644 --- a/crates/ui-egui/src/panels/project_views.rs +++ b/crates/ui-egui/src/panels/project_views.rs @@ -314,6 +314,20 @@ fn bin_menu(app: &mut FilmcraftApp, ui: &mut egui::Ui, v: &View, bin: u64, actio actions.push(("file.newBin".into(), json!({"name": "New Bin", "parent": bin}))); ui.close(); } + ui.separator(); + if ui.button(tl!("Clear")).clicked() { + actions.push(("project.delete".into(), json!({"items": [bin]}))); + ui.close(); + } +} + +/// Parameters for `project.delete`: the selected items, else the selected bin by id (the engine +/// deletes a named bin with everything in it). `None` when nothing is selected. +pub fn clear_params(selection: &[filmcraft_project::ItemId], selected_bin: Option) -> Option { + if !selection.is_empty() { + return Some(json!({})); + } + selected_bin.map(|b| json!({"items": [b]})) } /// Drop project items dragged onto a bin: move them (the selection when the dragged item is in it). @@ -422,6 +436,7 @@ pub fn list_view(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect, v: &View let root = app.session.project.root.find_bin(v.bin).cloned().unwrap_or_default(); let out = egui::ScrollArea::both().id_salt((&v.prefix, "list-scroll")).auto_shrink([false, false]).show(&mut bui, |ui| { ui.set_min_width(lc.width); + let bg = background(ui, v); let mut row = 0usize; list_bin(app, ui, &root, 0, filter, &mut row, actions, v, &lc); if v.inst == Inst::Main && v.bin == app.session.project.root.id { @@ -430,7 +445,7 @@ pub fn list_view(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect, v: &View }; crate::panels::menu_dialogs::search_bin_rows(app, ui, &mut row, actions, &mut draw); } - empty_space(app, ui, v, actions); + empty_space(app, ui, v, &bg, actions); }); // pinned header, scrolled horizontally with the rows let hr = Rect::from_min_size(rect.min, vec2(rect.width(), header_h)); @@ -491,21 +506,88 @@ pub fn list_view(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect, v: &View } } -/// Empty space under the rows / cards: click deselects, double-click imports, right-click menu. -fn empty_space(app: &mut FilmcraftApp, ui: &mut egui::Ui, v: &View, actions: &mut Actions) { +/// The List / Icon view background, under the rows and cards (call it before drawing them, so they +/// sit on top). Starts the frame's [`note_hit`] list. +fn background(ui: &egui::Ui, v: &View) -> egui::Response { + ui.data_mut(|d| d.remove::>(hits_id(v))); + ui.interact(ui.clip_rect(), egui::Id::new((&v.prefix, "background")), Sense::click_and_drag()) +} + +/// Empty space under the rows / cards, and the gaps between them (`bg`): click deselects, +/// double-click imports, right-click menu, drag selects with a marquee. +fn empty_space(app: &mut FilmcraftApp, ui: &mut egui::Ui, v: &View, bg: &egui::Response, actions: &mut Actions) { let rest = ui.available_rect_before_wrap(); let r = Rect::from_min_size(rest.min, vec2(rest.width(), rest.height().max(40.0))); - let resp = ui.allocate_rect(r, Sense::click()); + ui.allocate_rect(r, Sense::hover()); app.auto.add(&format!("{}.empty", v.prefix), r.intersect(ui.clip_rect()), "Empty area"); - if resp.clicked() { + marquee(app, ui, v, bg, ui.max_rect().min, actions); + if bg.clicked() { app.ui.project_panel.selected_bin = None; actions.push(("project.select".into(), json!({"items": []}))); } - if resp.double_clicked() { + if bg.double_clicked() { actions.push(("file.import".into(), json!({}))); } let bin = (v.bin != app.session.project.root.id).then_some(v.bin.0); - resp.context_menu(|ui| background_menu(app, ui, v, bin, actions)); + bg.context_menu(|ui| background_menu(app, ui, v, bin, actions)); +} + +fn hits_id(v: &View) -> egui::Id { + egui::Id::new((&v.prefix, "marquee-hits")) +} + +/// Remember where an item's row or card was drawn this frame, for [`marquee`]. +fn note_hit(ui: &egui::Ui, v: &View, id: ItemId, r: Rect) { + ui.data_mut(|d| d.get_temp_mut_or_default::>(hits_id(v)).push((id.0, r))); +} + +/// Marquee selection: dragging `bg` (empty space) selects every item drawn this frame whose rect +/// the rectangle touches, live. Shift/Ctrl add to the selection the drag started with; Escape +/// cancels and restores it. `origin` is the content's top-left, so scrolling keeps the anchor. +fn marquee(app: &mut FilmcraftApp, ui: &egui::Ui, v: &View, bg: &egui::Response, origin: egui::Pos2, actions: &mut Actions) { + // (press point relative to `origin`, selection when the drag began, adds to it) + type Drag = (egui::Vec2, Vec, bool); + let key = egui::Id::new((&v.prefix, "marquee")); + let hits: Vec<(u64, Rect)> = ui.data_mut(|d| d.remove_temp(hits_id(v))).unwrap_or_default(); + let current: Vec = app.session.state.project_selection.iter().map(|i| i.0).collect(); + // the left button only: right-click opens the menu, the middle button is not a selection + let primary = egui::PointerButton::Primary; + if bg.drag_started_by(primary) { + let press = ui.input(|i| i.pointer.press_origin()).unwrap_or(origin); + let m = ui.input(|i| i.modifiers); + let add = m.shift || m.command; + if !add { + app.ui.project_panel.selected_bin = None; + } + ui.data_mut(|d| d.insert_temp::(key, (press - origin, current.clone(), add))); + } + let Some((anchor, before, add)) = ui.data(|d| d.get_temp::(key)) else { return }; + let cancel = ui.input(|i| i.key_pressed(egui::Key::Escape)); + if cancel || !(bg.dragged_by(primary) || bg.drag_stopped_by(primary)) { + ui.data_mut(|d| d.remove::(key)); + if cancel && before != current { + actions.push(("project.select".into(), json!({"items": before}))); + } + return; + } + let Some(now) = ui.ctx().pointer_latest_pos() else { return }; + let r = Rect::from_two_pos(origin + anchor, now); + let mut sel = if add { before } else { Vec::new() }; + for (id, hr) in &hits { + if hr.intersects(r) && !sel.contains(id) { + sel.push(*id); + } + } + if sel != current { + actions.push(("project.select".into(), json!({"items": sel}))); + } + let t = app.tokens; + ui.painter().rect_filled(r, 0.0, t.accent.gamma_multiply(0.18)); + ui.painter().rect_stroke(r, 0.0, Stroke::new(1.0, t.accent), StrokeKind::Inside); + app.auto.add(&format!("{}.marquee", v.prefix), r, "Marquee"); + if bg.drag_stopped_by(primary) { + ui.data_mut(|d| d.remove::(key)); + } } fn background_menu(app: &mut FilmcraftApp, ui: &mut egui::Ui, v: &View, bin: Option, actions: &mut Actions) { @@ -580,6 +662,7 @@ fn list_bin(app: &mut FilmcraftApp, ui: &mut egui::Ui, bin: &Bin, depth: usize, } let Some(it) = app.session.project.item(id).cloned() else { continue }; let (r, resp) = ui.allocate_exact_size(vec2(lc.width, lc.row_h), Sense::click_and_drag()); + note_hit(ui, v, id, r); let selected = app.session.state.project_selection.contains(&id); if selected { ui.painter().rect_filled(r, 0.0, t.row_selected); @@ -806,6 +889,7 @@ pub fn icon_view(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect, v: &View }; let n = bins.len() + ids.len(); egui::ScrollArea::vertical().id_salt((&v.prefix, "icon-scroll")).auto_shrink([false, false]).show(ui, |ui| { + let bg = background(ui, v); let per_row = ((rect.width() - 12.0) / cell.x).floor().max(1.0) as usize; if v.inst == Inst::Main { app.ui.keys.icon_columns = per_row; @@ -827,13 +911,14 @@ pub fn icon_view(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect, v: &View let Some(kind) = app.session.project.item(id).map(|x| x.kind.clone()) else { continue }; let was_selected = app.session.state.project_selection.contains(&id); if let Some(resp) = card(app, ui, r, id, &v.prefix, true, true, Sense::click_and_drag()) { + note_hit(ui, v, id, r); let name_r = Rect::from_min_max(pos2(r.min.x, r.min.y + th_h), r.max); slow_click_rename(app, ui, &resp, name_r, id, was_selected); item_interactions(app, ui, &resp, id, &kind, actions, true, true); } } } - empty_space(app, ui, v, actions); + empty_space(app, ui, v, &bg, actions); }); } @@ -856,7 +941,8 @@ pub fn freeform_view(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect, v: & }; let extent = cards.iter().fold(vec2(rect.width(), rect.height()), |e, c| vec2(e.x.max(c.x + c.size + 40.0), e.y.max(c.y + c.size * 9.0 / 16.0 + 60.0))); egui::ScrollArea::both().id_salt((&v.prefix, "ff-scroll")).auto_shrink([false, false]).show(ui, |ui| { - let (canvas, bg) = ui.allocate_exact_size(extent, Sense::click()); + ui.data_mut(|d| d.remove::>(hits_id(v))); + let (canvas, bg) = ui.allocate_exact_size(extent, Sense::click_and_drag()); app.auto.add(&format!("{}.freeform", v.prefix), canvas, "Freeform canvas"); if opts.snap { let g = opts.grid.max(4.0); @@ -888,6 +974,7 @@ pub fn freeform_view(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect, v: & let r = Rect::from_min_size(canvas.min + vec2(c.x, c.y) + off, vec2(c.size, c.size * 9.0 / 16.0 + 20.0)); let Some(kind) = app.session.project.item(ItemId(c.item)).map(|x| x.kind.clone()) else { continue }; let Some(resp) = card(app, ui, r, ItemId(c.item), &v.prefix, opts.show_names, opts.show_durations, Sense::click_and_drag()) else { continue }; + note_hit(ui, v, ItemId(c.item), r); if let Some(s) = c.stack && s == c.item && let Some(n) = stack_n.get(&s) @@ -914,6 +1001,7 @@ pub fn freeform_view(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect, v: & } resp.context_menu(|ui| card_menu(app, ui, v, c.item, actions)); } + marquee(app, ui, v, &bg, canvas.min, actions); if let Some((item, off)) = released { ui.ctx().data_mut(|d| d.remove::<(u64, egui::Vec2)>(drag_key)); let inside = ui.ctx().pointer_latest_pos().is_some_and(|p| rect.contains(p)); @@ -1033,3 +1121,17 @@ fn canvas_menu(app: &mut FilmcraftApp, ui: &mut egui::Ui, v: &View, bin: Option< ui.separator(); background_menu(app, ui, v, bin.filter(|b| *b != app.session.project.root.id.0), actions); } + +#[cfg(test)] +mod clear_tests { + use super::clear_params; + use filmcraft_project::ItemId; + use serde_json::json; + + #[test] + fn clear_falls_back_to_the_selected_bin() { + assert_eq!(clear_params(&[], Some(7)), Some(json!({"items": [7]}))); + assert_eq!(clear_params(&[ItemId(3)], Some(7)), Some(json!({}))); + assert_eq!(clear_params(&[], None), None); + } +} diff --git a/crates/ui-egui/src/panels/settings.rs b/crates/ui-egui/src/panels/settings.rs index 41395d86..021b8473 100644 --- a/crates/ui-egui/src/panels/settings.rs +++ b/crates/ui-egui/src/panels/settings.rs @@ -534,11 +534,21 @@ fn color_field(app: &mut FilmcraftApp, ui: &mut Ui, d: &mut SettingsDraft, key: if r.changed() { put(&mut d.values, key, json!(settings::hex(rgb))); } - let mut text = choice_text(get(&d.values, key)); + // While the field has focus it shows what is typed, even when that is not a colour yet (one + // character into `#2f6bdf`); each valid colour is kept as it comes. Leaving shows the kept one. + let draft_id = egui::Id::new(("settings-hex", key)); + let mut text = ui.data(|m| m.get_temp::(draft_id)).unwrap_or_else(|| choice_text(get(&d.values, key))); let t = ui.add(egui::TextEdit::singleline(&mut text).desired_width(70.0).font(egui::TextStyle::Monospace)); app.auto.add(&format!("settings.{key}.hex"), t.rect, label); - if t.changed() && settings::parse_hex(&text).is_some() { - put(&mut d.values, key, json!(settings::hex(settings::parse_hex(&text).unwrap_or(rgb)))); + if t.has_focus() { + ui.data_mut(|m| m.insert_temp(draft_id, text.clone())); + } else { + ui.data_mut(|m| m.remove::(draft_id)); + } + if t.changed() + && let Some(c) = settings::parse_hex(&text) + { + put(&mut d.values, key, json!(settings::hex(c))); } } @@ -785,6 +795,30 @@ mod tests { use super::*; use crate::theme::ThemeKind; + /// #457: on a desktop that reports 150%, UI Scale 100% makes a point one physical pixel. It is + /// applied once, so Ctrl+- still zooms for the session, and Auto gives the system's scale back. + #[test] + fn ui_scale_overrides_the_system_scale_and_auto_restores_it() { + let ctx = egui::Context::default(); + let mut app = FilmcraftApp::new(filmcraft_engine::Session::default()); + // a frame applying the settings, then the next one, where a new zoom takes effect + let pass = |app: &mut FilmcraftApp| { + let mut input = egui::RawInput::default(); + input.viewports.entry(egui::ViewportId::ROOT).or_default().native_pixels_per_point = Some(1.5); + ctx.run_ui(input.clone(), |ui| app.apply_prefs(ui.ctx())).textures_delta.clear(); + ctx.run_ui(input, |_| {}).textures_delta.clear(); + ctx.zoom_factor() + }; + assert_eq!(pass(&mut app), 1.0, "Auto keeps the system's scale"); + app.session.execute("prefs.set", json!({"key": "appearance.uiScale", "value": "100"})).unwrap(); + assert_eq!(pass(&mut app), 1.0 / 1.5, "100% is one physical pixel per point"); + ctx.set_zoom_factor(0.9 / 1.5); // Ctrl+- + assert_eq!(pass(&mut app), 0.9 / 1.5, "a session zoom isn't undone every frame"); + assert!(app.session.execute("prefs.set", json!({"key": "appearance.uiScale", "value": "130"})).is_err()); + app.session.execute("prefs.set", json!({"key": "appearance.uiScale", "value": "auto"})).unwrap(); + assert_eq!(pass(&mut app), 1.0); + } + #[test] fn appearance_button_cycles_modes_without_losing_theme_choices() { let ctx = egui::Context::default(); diff --git a/crates/ui-egui/src/panels/text.rs b/crates/ui-egui/src/panels/text.rs index ba59f9a7..84200095 100644 --- a/crates/ui-egui/src/panels/text.rs +++ b/crates/ui-egui/src/panels/text.rs @@ -395,6 +395,22 @@ fn style_strip(app: &mut FilmcraftApp, ui: &mut egui::Ui, r: Rect, track_idx: us let mut child = ui.new_child(egui::UiBuilder::new().max_rect(r).id_salt("caption-style")); child.horizontal_centered(|ui| { ui.spacing_mut().item_spacing.x = 6.0; + let r = egui::ComboBox::from_id_salt(("caption-font", tr.id.0)).selected_text(&st.font).width(130.0).height(400.0).show_ui(ui, |ui| { + if !filmcraft_text::fonts::system_scanned() { + filmcraft_text::fonts::scan_system(); + } + // names starting with '.' are macOS-private system faces (UI/fallback fonts, some without a space glyph) + for (f, _) in filmcraft_text::families().into_iter().filter(|(f, _)| !f.starts_with('.')) { + let resp = ui.selectable_label(f.eq_ignore_ascii_case(&st.font), &f); + if ui.is_rect_visible(resp.rect) { + app.auto.add(&format!("text.captions.style.font.option.{f}"), resp.rect, &f); + } + if resp.clicked() { + actions.push(("captions.setStyle".into(), json!({"track": tr.id.0, "font": f}))); + } + } + }); + app.auto.add("text.captions.style.font", r.response.rect, "Caption font"); let mut size = st.size; let resp = ui.add(egui::DragValue::new(&mut size).range(8.0..=200.0).speed(0.5).suffix(" px")); app.auto.add("text.captions.style.size", resp.rect, "Caption size"); diff --git a/crates/ui-egui/src/panels/timeline.rs b/crates/ui-egui/src/panels/timeline.rs index d5997839..17a5eef9 100644 --- a/crates/ui-egui/src/panels/timeline.rs +++ b/crates/ui-egui/src/panels/timeline.rs @@ -19,7 +19,7 @@ use serde_json::{Value, json}; use crate::FilmcraftApp; use crate::icons::{self, Icon}; -use crate::state::{TimelineView, Tool}; +use crate::state::{ThumbnailMode, TimelineView, Tool}; use crate::theme::Tokens; use super::timeline_hit::{EdgeKind, Grab, edge_geometry, grab_at}; @@ -355,7 +355,11 @@ pub fn show(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect) { } let bg = if r.index % 2 == 0 { t.tl_track_bg } else { t.tl_track_bg_alt }; painter.rect_filled(row, 0.0, bg); - painter.line_segment([pos2(row.min.x, r.rect.max.y - 0.5), pos2(row.max.x, r.rect.max.y - 0.5)], Stroke::new(1.0, t.tl_bg)); + painter.line_segment([pos2(row.min.x, r.rect.max.y - 0.5), pos2(row.max.x, r.rect.max.y - 0.5)], Stroke::new(1.0, t.separator)); + // the top video track also gets a line above it + if r.kind == TrackKind::Video && r.index + 1 == nv { + painter.line_segment([pos2(row.min.x, r.rect.min.y + 0.5), pos2(row.max.x, r.rect.min.y + 0.5)], Stroke::new(1.0, t.separator)); + } } // in/out shading across tracks if seq.mark_in.is_some() || seq.mark_out.is_some() { @@ -491,15 +495,11 @@ pub fn show(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect) { let ph = app.session.playhead(); let px = layout.x_of(ph); if px >= content.min.x - 1.0 && px <= content.max.x + 1.0 { - let head = [ - pos2(px - 6.0, ruler.min.y + 2.0), - pos2(px + 6.0, ruler.min.y + 2.0), - pos2(px + 6.0, ruler.max.y - 10.0), - pos2(px, ruler.max.y - 4.0), - pos2(px - 6.0, ruler.max.y - 10.0), - ]; + // a short head on the tick row, as in Effect Controls, so it leaves the markers and timecodes clear + let (top, tip) = (ruler.max.y - 13.0, ruler.max.y); + let head = [pos2(px - 5.0, top), pos2(px + 5.0, top), pos2(px + 5.0, tip - 5.0), pos2(px, tip), pos2(px - 5.0, tip - 5.0)]; painter.add(egui::Shape::convex_polygon(head.to_vec(), t.playhead, Stroke::NONE)); - painter.line_segment([pos2(px, ruler.max.y - 4.0), pos2(px, content.max.y)], Stroke::new(1.0, t.playhead)); + painter.line_segment([pos2(px, tip), pos2(px, content.max.y)], Stroke::new(1.0, t.playhead)); } // snap indicator if let Some(sx) = app.tl.snap_x.take() { @@ -604,6 +604,36 @@ fn in_range(app: &FilmcraftApp, it: &TrackItem) -> bool { it.start < b && it.end() > a } +/// Most thumbnails one clip draws: a long clip far zoomed in only shows the visible ones anyway. +const MAX_THUMBNAIL_TILES: usize = 256; + +/// The video thumbnails of a clip whose picture area spans `x0..x1` (points), for frames `tile_w` +/// wide: the left edge of each tile and the offset into the clip (`dur` long, frames of `frame`) +/// whose picture it shows. Only tiles that meet `visible` (the panel's x range) are listed. +/// Continuous tiles sit on a grid anchored at the clip's head, so scrolling only reveals new tiles +/// and the others keep their frames. A clip too narrow for more than one tile shows its head. +pub fn thumbnail_tiles(mode: ThumbnailMode, x0: f32, x1: f32, tile_w: f32, visible: (f32, f32), dur: Tick, frame: Tick) -> Vec<(f32, Tick)> { + let width = x1 - x0; + if !(x0.is_finite() && width.is_finite() && width > 0.0 && tile_w.is_finite() && tile_w >= 1.0) { + return Vec::new(); + } + let last = (dur - frame).max(Tick::ZERO); + let offset_at = |x: f32| Tick(((x - x0) as f64 / width as f64 * dur.0 as f64).round() as i64).clamp(Tick::ZERO, last); + let shown = |x: f32| x < visible.1 && x + tile_w > visible.0; + let head = (x0, Tick::ZERO); + let tiles = match mode { + ThumbnailMode::Head => vec![head], + ThumbnailMode::HeadAndTail if width >= 2.0 * tile_w => vec![head, (x1 - tile_w, last)], + ThumbnailMode::HeadAndTail => vec![head], + ThumbnailMode::Continuous => { + let first = ((visible.0 - x0).max(0.0) / tile_w).floor() as usize; + let end = ((visible.1.min(x1) - x0) / tile_w).ceil().max(1.0) as usize; + (first..end).take(MAX_THUMBNAIL_TILES).map(|k| x0 + k as f32 * tile_w).map(|x| (x, offset_at(x))).collect() + } + }; + tiles.into_iter().filter(|(x, _)| shown(*x)).collect() +} + #[allow(clippy::too_many_arguments)] fn draw_clip( app: &mut FilmcraftApp, @@ -632,7 +662,8 @@ fn draw_clip( p.line_segment([pos2(body.min.x, body.min.y + 0.5), pos2(body.max.x, body.min.y + 0.5)], Stroke::new(1.0, lighten(fill, 0.15))); let name_h = 16.0; let w = body.width(); - // head thumbnail (video): left-aligned, aspect-correct, below the name band + // video thumbnails, aspect-correct, below the name band: the head frame, head and tail, or + // frames side by side across the clip (Show Video Thumbnails in the wrench menu) if kind == TrackKind::Video && app.ui.timeline.show_thumbnails && body.height() > 28.0 && w > 20.0 && it.enabled { let th = Rect::from_min_max(pos2(body.min.x + 1.0, body.min.y + name_h), pos2(body.max.x - 1.0, body.max.y - 1.0)); let aspect = app.session.project.item(it.item).and_then(|pi| match &pi.kind { @@ -640,24 +671,28 @@ fn draw_clip( filmcraft_project::ItemKind::Sequence(s) => Some(s.settings.width as f32 / s.settings.height as f32), _ => None, }); - if let Some(aspect) = aspect { - let tw = (th.height() * aspect).min(th.width()); - let mt = rate.snap(it.source_in); - // a multi-camera clip shows its angle's clip - let (thumb_item, mt) = app - .session - .project - .sequence(it.item) - .and_then(|q| { - let ti = q.angle_video_track_index(it.multicam_angle(q)?)?; - let inner = q.video_tracks[ti].item_at(mt)?; - Some((inner.item, inner.source_time_at(mt))) - }) - .unwrap_or((it.item, mt)); - if let Some((tex, _)) = app.thumbnail(ctx, thumb_item, mt, 160) { - let r = Rect::from_min_size(th.min, vec2(tw, th.height())); - let cp = p.with_clip_rect(th.intersect(p.clip_rect())); - cp.image(tex, r, Rect::from_min_max(pos2(0.0, 0.0), pos2((tw / (th.height() * aspect)).min(1.0), 1.0)), Color32::WHITE); + if let Some(aspect) = aspect.filter(|a| a.is_finite() && *a > 0.0) { + let tw = th.height() * aspect; + let cp = p.with_clip_rect(th.intersect(p.clip_rect())); + let visible = (cp.clip_rect().min.x, cp.clip_rect().max.x); + let mode = app.ui.timeline.thumbnail_mode; + for (x, offset) in thumbnail_tiles(mode, th.min.x, th.max.x, tw, visible, it.duration, rate.frame_duration()) { + let mt = rate.snap(it.source_time_at(it.start + offset)); + // a multi-camera clip shows its angle's clip + let (thumb_item, mt) = app + .session + .project + .sequence(it.item) + .and_then(|q| { + let ti = q.angle_video_track_index(it.multicam_angle(q)?)?; + let inner = q.video_tracks.get(ti)?.item_at(mt)?; + Some((inner.item, inner.source_time_at(mt))) + }) + .unwrap_or((it.item, mt)); + if let Some((tex, _)) = app.thumbnail(ctx, thumb_item, mt, 160) { + let r = Rect::from_min_size(pos2(x, th.min.y), vec2(tw, th.height())); + cp.image(tex, r, Rect::from_min_max(pos2(0.0, 0.0), pos2(1.0, 1.0)), Color32::WHITE); + } } } } @@ -989,6 +1024,9 @@ fn draw_headers(app: &mut FilmcraftApp, ui: &mut egui::Ui, seq: &Sequence, rows: let p = ui.painter().with_clip_rect(visible); p.rect_filled(hrect, 0.0, t.tl_header_bg); p.line_segment([pos2(hrect.min.x, hrect.max.y - 0.5), pos2(hrect.max.x, hrect.max.y - 0.5)], Stroke::new(1.0, t.separator)); + if r.kind == TrackKind::Video && r.index + 1 == seq.video_tracks.len() { + p.line_segment([pos2(hrect.min.x, hrect.min.y + 0.5), pos2(hrect.max.x, hrect.min.y + 0.5)], Stroke::new(1.0, t.separator)); + } let label = format!("{}{}", if r.kind == TrackKind::Video { "V" } else { "A" }, r.index + 1); let btn_rect = |x0: f32| Rect::from_min_max(pos2(hrect.min.x + x0, hrect.min.y + 1.0), pos2(hrect.min.x + x0 + 24.0, hrect.max.y - 2.0)); // 1. source patch (absent when unpatched) @@ -1109,6 +1147,38 @@ fn draw_headers(app: &mut FilmcraftApp, ui: &mut egui::Ui, seq: &Sequence, rows: egui::Id::new(("hdr", r.track.0)), Sense::click(), ); + app.auto.add(&format!("timeline.track.{label}.name"), resp.rect, "Track name (right-click to rename)"); + let rename_id = egui::Id::new(("timeline-track-rename", r.track.0)); + if resp.secondary_clicked() { + // Start each menu opening from the committed name, not a cancelled draft. + ui.data_mut(|d| d.insert_temp(rename_id, tr.name.clone())); + } + egui::Popup::context_menu(&resp).close_behavior(egui::PopupCloseBehavior::CloseOnClickOutside).show(|ui| { + ui.set_min_width(220.0); + ui.label(tl!("Rename")); + let mut name: String = ui.data(|d| d.get_temp(rename_id)).unwrap_or_else(|| tr.name.clone()); + let input = ui.add(egui::TextEdit::singleline(&mut name).desired_width(210.0)); + app.auto.add(&format!("timeline.track.{label}.renameField"), input.rect, "New track name"); + if resp.secondary_clicked() { + input.request_focus(); + } + let save_on_enter = input.has_focus() && ui.input(|i| i.key_pressed(egui::Key::Enter)); + let valid = !name.trim().is_empty(); + ui.horizontal(|ui| { + let apply = ui.add_enabled(valid, egui::Button::new(tl!("Apply"))); + app.auto.add(&format!("timeline.track.{label}.renameApply"), apply.rect, "Apply track name"); + if (apply.clicked() || (valid && save_on_enter)) && name.trim() != tr.name { + actions.push(("timeline.setTrack".into(), json!({"track": r.track.0, "name": name.trim()}))); + ui.close(); + } else if apply.clicked() || (valid && save_on_enter) { + ui.close(); + } + if ui.button(tl!("Cancel")).clicked() { + ui.close(); + } + }); + ui.data_mut(|d| d.insert_temp(rename_id, name)); + }); if resp.double_clicked() { let h = if r.kind == TrackKind::Video { &mut app.ui.timeline.video_track_h } else { &mut app.ui.timeline.audio_track_h }; *h = if *h < 50.0 { 64.0 } else { 30.0 }; @@ -1179,7 +1249,20 @@ fn draw_top(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect, seq: &Sequenc } if key == "settings" { egui::Popup::menu(&resp).show(|ui| { - ui.checkbox(&mut app.ui.timeline.show_thumbnails, tl!("Show Video Thumbnails")); + let c = ui.checkbox(&mut app.ui.timeline.show_thumbnails, tl!("Show Video Thumbnails")); + app.auto.add("timeline.settings.showThumbnails", c.rect, "Show Video Thumbnails"); + let on = app.ui.timeline.show_thumbnails; + ui.indent("timeline.settings.thumbnails", |ui| { + for (mode, label) in + [(ThumbnailMode::Head, tl!("Head")), (ThumbnailMode::HeadAndTail, tl!("Head and Tail")), (ThumbnailMode::Continuous, tl!("Continuous"))] + { + let r = ui.add_enabled(on, egui::RadioButton::new(app.ui.timeline.thumbnail_mode == mode, label)); + app.auto.add(&format!("timeline.settings.thumbnails.{}", mode.name()), r.rect, label); + if r.clicked() { + app.ui.timeline.thumbnail_mode = mode; + } + } + }); ui.checkbox(&mut app.ui.timeline.show_waveforms, tl!("Show Audio Waveform")); ui.separator(); let mut te = app.session.state.show_through_edits; @@ -1963,14 +2046,9 @@ fn interact(app: &mut FilmcraftApp, ui: &mut egui::Ui, seq: &Sequence, layout: & // select (shift toggles; alt selects one side of a link) let sel = &app.session.state.selection; if mods.shift { - let _ = app.session.execute("timeline.select", json!({"clips": [clip.0], "toggle": true})); - } else if !sel.contains(&clip) { - if mods.alt { - app.session.state.selection = vec![clip]; - app.session.state.transition_selection.clear(); - } else { - let _ = app.session.execute("timeline.select", json!({"clips": [clip.0]})); - } + let _ = app.session.execute("timeline.select", json!({"clips": [clip.0], "toggle": true, "linked": !mods.alt})); + } else if mods.alt || !sel.contains(&clip) { + let _ = app.session.execute("timeline.select", json!({"clips": [clip.0], "linked": !mods.alt})); } if resp.drag_started() { let clips = app.session.state.selection.clone(); @@ -1996,7 +2074,9 @@ fn interact(app: &mut FilmcraftApp, ui: &mut egui::Ui, seq: &Sequence, layout: & } } (_, Grab::Other(Hit::Transition { .. })) => None, - (_, Grab::Other(Hit::Empty { .. })) => { + // empty space: an empty stretch of a track, or no track at all (below the tracks, between + // the video and audio tracks, #683) + (_, Grab::Other(Hit::Empty { .. } | Hit::None)) if layout.content.contains(p) => { if resp.drag_started() { Some(Drag::Marquee { start: p }) } else { @@ -2006,7 +2086,7 @@ fn interact(app: &mut FilmcraftApp, ui: &mut egui::Ui, seq: &Sequence, layout: & None } } - (_, Grab::Other(Hit::None)) => None, + (_, Grab::Other(Hit::Empty { .. } | Hit::None)) => None, }; if let Some(d) = started { if matches!(d, Drag::Scrub) { @@ -2524,3 +2604,47 @@ mod chain_starts_tests { assert_eq!(chain_starts(Tick(5), &[Tick(-3), Tick(4)]), vec![Tick(5), Tick(5)]); } } + +#[cfg(test)] +mod thumbnail_tiles_tests { + use super::*; + + const S: Tick = Tick(TICKS_PER_SECOND); + const FRAME: Tick = Tick(TICKS_PER_SECOND / 25); + + #[test] + fn continuous_thumbnails_tile_the_visible_part_of_the_clip() { + // a 10 s clip 1000 pt wide, 80 pt tiles, the panel showing x 250..650 + let tiles = thumbnail_tiles(ThumbnailMode::Continuous, 0.0, 1000.0, 80.0, (250.0, 650.0), Tick(10 * S.0), FRAME); + let xs: Vec = tiles.iter().map(|t| t.0).collect(); + assert_eq!(xs, [240.0, 320.0, 400.0, 480.0, 560.0, 640.0]); + // each tile shows the clip time under its left edge (100 pt per second) + assert_eq!(tiles[0].1, Tick(24 * S.0 / 10)); + assert_eq!(tiles[5].1, Tick(64 * S.0 / 10)); + // scrolling 30 pt moves the tiles with the clip: the ones still shown keep their frames + let scrolled = thumbnail_tiles(ThumbnailMode::Continuous, -30.0, 970.0, 80.0, (250.0, 650.0), Tick(10 * S.0), FRAME); + assert_eq!(scrolled.first(), Some(&(210.0, tiles[0].1))); + assert!(scrolled.iter().all(|(x, t)| tiles.iter().any(|(x0, t0)| *x0 - 30.0 == *x && t0 == t))); + // the last tile never shows a time past the clip's last frame + let end = thumbnail_tiles(ThumbnailMode::Continuous, 0.0, 1000.0, 80.0, (0.0, 2000.0), Tick(10 * S.0), FRAME); + assert_eq!(end.len(), 13); + assert!(end.iter().all(|(_, t)| *t <= Tick(10 * S.0) - FRAME)); + // a clip far zoomed in asks for a bounded number of frames + let huge = thumbnail_tiles(ThumbnailMode::Continuous, -1.0e9, 1.0e9, 1.0, (-2.0e9, 2.0e9), Tick(10 * S.0), FRAME); + assert_eq!(huge.len(), MAX_THUMBNAIL_TILES); + } + + #[test] + fn head_and_tail_thumbnails_need_room_for_two_frames() { + let dur = Tick(4 * S.0); + assert_eq!(thumbnail_tiles(ThumbnailMode::Head, 100.0, 500.0, 80.0, (0.0, 800.0), dur, FRAME), [(100.0, Tick::ZERO)]); + assert_eq!(thumbnail_tiles(ThumbnailMode::HeadAndTail, 100.0, 500.0, 80.0, (0.0, 800.0), dur, FRAME), [(100.0, Tick::ZERO), (420.0, dur - FRAME)]); + // narrower than two tiles: the head only, as in Continuous narrower than one tile + assert_eq!(thumbnail_tiles(ThumbnailMode::HeadAndTail, 100.0, 250.0, 80.0, (0.0, 800.0), dur, FRAME), [(100.0, Tick::ZERO)]); + assert_eq!(thumbnail_tiles(ThumbnailMode::Continuous, 100.0, 150.0, 80.0, (0.0, 800.0), dur, FRAME), [(100.0, Tick::ZERO)]); + // off screen or degenerate: nothing + assert!(thumbnail_tiles(ThumbnailMode::Head, 900.0, 1300.0, 80.0, (0.0, 800.0), dur, FRAME).is_empty()); + assert!(thumbnail_tiles(ThumbnailMode::Continuous, 0.0, 400.0, f32::NAN, (0.0, 800.0), dur, FRAME).is_empty()); + assert!(thumbnail_tiles(ThumbnailMode::Continuous, 0.0, 0.0, 80.0, (0.0, 800.0), dur, FRAME).is_empty()); + } +} diff --git a/crates/ui-egui/src/panels/timeline_captions.rs b/crates/ui-egui/src/panels/timeline_captions.rs index bc5b6898..781c7ade 100644 --- a/crates/ui-egui/src/panels/timeline_captions.rs +++ b/crates/ui-egui/src/panels/timeline_captions.rs @@ -83,6 +83,23 @@ pub fn paint(app: &mut FilmcraftApp, ui: &mut egui::Ui, seq: &Sequence, area: Re painter.rect_filled(badge, 3.0, if tr.enabled { FILL } else { t.field_bg }); painter.text(badge.center(), Align2::CENTER_CENTER, &label, Tokens::semibold(10.5), Color32::WHITE); app.auto.add(&format!("timeline.captionTrack.{label}"), badge, &tr.name); + // right-click the header: delete this track or every empty caption track (registered + // before the lock / eye toggles so those stay on top) + let hresp = ui.interact(hrect, egui::Id::new(("cap-header", tr.id.0)), Sense::click()); + hresp.context_menu(|ui| { + for (key, text, cmd, p) in [ + ("delete", tl!("Delete Caption Track"), "captions.deleteTrack", json!({"track": tr.id.0})), + ("deleteEmpty", tl!("Delete Empty Caption Tracks"), "sequence.deleteTracks", json!({"captions": "empty"})), + ] { + let enabled = key != "deleteEmpty" || seq.caption_tracks.iter().any(|t| t.captions.is_empty()); + let r = ui.add_enabled(enabled, egui::Button::new(text)); + app.auto.add(&format!("timeline.captionTrack.{label}.menu.{key}"), r.rect, text); + if r.clicked() { + actions.push((cmd.into(), p)); + ui.close(); + } + } + }); let lock_r = Rect::from_center_size(pos2(hrect.min.x + 49.0, row.center().y), vec2(18.0, 18.0)); let lresp = crate::widgets::icon_toggle( ui, diff --git a/crates/ui-egui/src/panels/trim_monitor.rs b/crates/ui-egui/src/panels/trim_monitor.rs index 6634320c..2cb4838e 100644 --- a/crates/ui-egui/src/panels/trim_monitor.rs +++ b/crates/ui-egui/src/panels/trim_monitor.rs @@ -244,9 +244,12 @@ pub fn show(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect) { ui.ctx().set_cursor_icon(egui::CursorIcon::ResizeColumn); } if resp.dragged() { + // read the delta before locking: drag_delta() takes the context's write lock, + // the same one data_mut() holds, and that lock is not reentrant (deadlock) + let dx = resp.drag_delta().x; let total = ui.ctx().data_mut(|d| { let v = d.get_temp_mut_or_default::(egui::Id::new("trim-monitor-dx")); - *v += resp.drag_delta().x; + *v += dx; *v }); let n = (total * fpp).round() as i64; diff --git a/crates/ui-egui/src/source_playback.rs b/crates/ui-egui/src/source_playback.rs index 494f9d08..7cdb7c21 100644 --- a/crates/ui-egui/src/source_playback.rs +++ b/crates/ui-egui/src/source_playback.rs @@ -30,7 +30,16 @@ impl FilmcraftApp { } pub fn play_source(&mut self) -> Result<(), String> { - self.start_source_playback(None, None) + self.start_source_playback(None, None, 1.0) + } + + /// J / K / L in the Source monitor: play at `speed` × real time (negative = backward) from the + /// Source playhead. Sound plays at normal forward speed only, as in the Program monitor. + pub fn shuttle_source(&mut self, speed: f64) -> Result<(), String> { + if !speed.is_finite() || speed == 0.0 { + return Err("Source shuttle speed must be a non-zero number".into()); + } + self.start_source_playback(None, None, speed.clamp(-8.0, 8.0)) } pub fn play_source_range(&mut self, from_playhead: bool, preroll: bool) -> Result<(), String> { @@ -53,10 +62,10 @@ impl FilmcraftApp { if start >= stop { return Err("Source playhead is at or beyond the Out point".into()); } - self.start_source_playback(Some(start), Some(stop)) + self.start_source_playback(Some(start), Some(stop), 1.0) } - fn start_source_playback(&mut self, start: Option, stop: Option) -> Result<(), String> { + fn start_source_playback(&mut self, start: Option, stop: Option, speed: f64) -> Result<(), String> { let item = self.session.state.source_item.ok_or("Open a media file in the Source monitor first")?; let view = source_view(&self.session, item).ok_or("Source item is unavailable")?; let media = self.session.project.item(view.media).and_then(|i| i.as_media()).ok_or("Source playback currently supports media files and subclips")?; @@ -76,7 +85,8 @@ impl FilmcraftApp { self.stop(); self.stop_source(); let mut time = start.unwrap_or(self.session.state.source_playhead); - if start.is_none() && (time < first || time >= view.rate.snap(Tick(end.0.saturating_sub(view.rate.frame_duration().0))).max(first)) { + // forward from the end (or before a looped range) starts over; backward runs from where it is + if start.is_none() && speed > 0.0 && (time < first || time >= view.rate.snap(Tick(end.0.saturating_sub(view.rate.frame_duration().0))).max(first)) { time = first; } self.session.execute("source.setPlayhead", json!({"time": time.0})).map_err(|e| e.to_string())?; @@ -84,8 +94,8 @@ impl FilmcraftApp { self.source_playback.range_start = start; self.source_playback.item = Some(item); self.source_playback.shown = time; - self.source_playback.clock = Playback { playing: true, speed: 1.0, anchor_time: -1.0, anchor_tick: time, looping, stop_at: stop, ..Default::default() }; - self.source_playback.clock.meter.start(1.0); + self.source_playback.clock = Playback { playing: true, speed, anchor_time: -1.0, anchor_tick: time, looping, stop_at: stop, ..Default::default() }; + self.source_playback.clock.meter.start(speed); self.start_source_audio(); Ok(()) } @@ -105,6 +115,12 @@ impl FilmcraftApp { fn start_source_audio(&mut self) { self.source_playback.clock.audio_clock = false; + if (self.source_playback.clock.speed - 1.0).abs() > 1e-9 { + if let Some(audio) = self.audio.as_mut() { + audio.stop(); + } + return; + } let Some(item) = self.source_playback.item else { return }; let Some(view) = source_view(&self.session, item) else { return }; let Some(media) = self.session.project.item(view.media).and_then(|i| i.as_media()) else { return }; @@ -221,7 +237,24 @@ impl FilmcraftApp { Some((frames, sr)) if clock.audio_clock && sr > 0 => frames as f64 / f64::from(sr), _ => (now - clock.anchor_time).max(0.0), }; - let next = Tick(clock.anchor_tick.0.saturating_add(Tick::from_seconds_f64(elapsed).0)); + let next = Tick(clock.anchor_tick.0.saturating_add(Tick::from_seconds_f64(elapsed * clock.speed).0)); + if clock.speed < 0.0 { + // backward: stops on the first frame (as the Program monitor stops at the start) + let ended = next <= view.start; + match self.session.execute("source.setPlayhead", json!({"time": next.max(view.start).0})) { + Ok(_) => self.source_playback.shown = self.session.state.source_playhead, + Err(e) => { + self.stop_source(); + self.ui.status = e.to_string(); + return; + } + } + if ended { + self.stop_source(); + } + ctx.request_repaint(); + return; + } let range = view.selected_range(); let end = clock.stop_at.unwrap_or_else(|| if clock.looping { range.end() } else { view.end }).min(view.end); let ended = next >= end; diff --git a/crates/ui-egui/src/state.rs b/crates/ui-egui/src/state.rs index aacea877..aa89ca02 100644 --- a/crates/ui-egui/src/state.rs +++ b/crates/ui-egui/src/state.rs @@ -190,6 +190,9 @@ pub struct TimelineView { pub audio_track_h: f32, pub header_w: f32, pub show_thumbnails: bool, + /// Which frames a video clip shows while `show_thumbnails` is on (the wrench menu). + #[serde(default)] + pub thumbnail_mode: ThumbnailMode, pub show_waveforms: bool, /// Follow playhead during playback (page scroll). pub follow: bool, @@ -225,6 +228,7 @@ impl Default for TimelineView { audio_track_h: 56.0, header_w: 204.0, show_thumbnails: true, + thumbnail_mode: ThumbnailMode::Head, show_waveforms: true, follow: true, fit_pending: true, @@ -234,6 +238,35 @@ impl Default for TimelineView { } } +/// Video thumbnails on timeline clips (Timeline wrench menu ▸ Show Video Thumbnails). +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] +pub enum ThumbnailMode { + /// The clip's first frame at its head. + #[default] + Head, + /// The first frame at the head and the last one at the tail. + HeadAndTail, + /// Frames side by side across the whole clip. + Continuous, +} + +impl ThumbnailMode { + pub const ALL: [ThumbnailMode; 3] = [ThumbnailMode::Head, ThumbnailMode::HeadAndTail, ThumbnailMode::Continuous]; + + /// The name used by automation ids and `ui.set` (`timeline.thumbnails`). + pub fn name(self) -> &'static str { + match self { + ThumbnailMode::Head => "head", + ThumbnailMode::HeadAndTail => "headAndTail", + ThumbnailMode::Continuous => "continuous", + } + } + + pub fn from_name(name: &str) -> Option { + Self::ALL.into_iter().find(|m| m.name().eq_ignore_ascii_case(name)) + } +} + /// Monitor display mode (View ▸ Display Mode / the monitor's wrench menu). Multi-Camera is the /// separate `MonitorView::multicam` switch. #[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] @@ -277,6 +310,8 @@ pub struct MonitorView { pub pan: [f32; 2], pub safe_margins: bool, pub show_transport: bool, + /// Transport buttons hidden through the monitor's Button Editor. Stored per monitor. + pub transport_hidden: Vec, /// Program monitor display mode Multi-Camera (angle grid + program). pub multicam: bool, pub display: DisplayMode, @@ -300,6 +335,8 @@ impl Default for MonitorView { pan: [0.0, 0.0], safe_margins: false, show_transport: true, + // The optional Loop button is available from the editor without altering default layouts. + transport_hidden: vec!["playback.loop".into()], multicam: false, display: DisplayMode::Composite, compare_ref: None, @@ -368,6 +405,19 @@ pub struct Eyedropper { pub mask: Option, } +/// A keyframe selected in the Effect Controls keyframe lane: clip, effect index, parameter (of an +/// effect mask when `mask` is set) and the keyframe's media time. The panel drops references that +/// no longer match a keyframe (deleted, moved, another clip selected). +#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] +pub struct KeyframeRef { + pub clip: u64, + pub effect: usize, + pub param: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub mask: Option, + pub time: filmcraft_time::Tick, +} + #[derive(Clone, Debug, Serialize, Deserialize)] pub struct UiState { #[serde(default)] @@ -408,6 +458,9 @@ pub struct UiState { /// monitor (Esc or a click elsewhere disarms it). Never saved. #[serde(skip)] pub eyedropper: Option, + /// Effect Controls: the selected keyframes, highlighted in the keyframe lane (a click selects one). + #[serde(default)] + pub keyframe_selection: Vec, /// Essential Sound sub-tab: "Edit" or "Browse". #[serde(default)] pub essential_sound_tab: String, @@ -827,7 +880,7 @@ impl Default for SequenceSettingsDraft { } /// The Delete Tracks dialog (Sequence ▸ Delete Tracks…): per kind, whether to delete and which -/// track (`"empty"` = All Empty Tracks, or a track name such as `"V2"`). +/// track (`"empty"` = All Empty Tracks, or a track name such as `"V2"` / `"C2"`). #[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] #[serde(default)] pub struct DeleteTracksDraft { @@ -835,11 +888,13 @@ pub struct DeleteTracksDraft { pub video_target: String, pub audio: bool, pub audio_target: String, + pub captions: bool, + pub captions_target: String, } impl Default for DeleteTracksDraft { fn default() -> Self { - Self { video: false, video_target: "empty".into(), audio: false, audio_target: "empty".into() } + Self { video: false, video_target: "empty".into(), audio: false, audio_target: "empty".into(), captions: false, captions_target: "empty".into() } } } @@ -904,6 +959,7 @@ impl Default for UiState { show_scopes: false, status: String::new(), eyedropper: None, + keyframe_selection: Vec::new(), essential_sound_tab: "Edit".into(), tts: Default::default(), export: Default::default(), diff --git a/crates/ui-egui/tests/captions_font_ui.rs b/crates/ui-egui/tests/captions_font_ui.rs new file mode 100644 index 00000000..bd6822d5 --- /dev/null +++ b/crates/ui-egui/tests/captions_font_ui.rs @@ -0,0 +1,95 @@ +//! Headless UI test of the caption track font picker in the Text panel's Captions tab: picking a +//! family sets the track style's font, and macOS-private (`.`-prefixed) faces are not offered. + +use std::sync::mpsc::{Sender, channel}; + +use egui_kittest::Harness; +use filmcraft_engine::Session; +use filmcraft_ui_egui::FilmcraftApp; +use filmcraft_ui_egui::control::ControlRequest; +use serde_json::{Value, json}; + +struct Driver { + harness: Harness<'static, FilmcraftApp>, + tx: Sender, +} + +impl Driver { + fn demo() -> Self { + let mut session = Session::default(); + session.execute("file.openDemoProject", json!({})).expect("demo project"); + let (tx, rx) = channel(); + let app = FilmcraftApp::new(session).with_control(rx); + let harness = Harness::builder().with_size(egui::vec2(1600.0, 980.0)).with_max_steps(10_000).build_eframe(move |_cc| app); + let mut d = Driver { harness, tx }; + d.frames(4); + d + } + + fn frames(&mut self, n: usize) { + for _ in 0..n { + let ctx = self.harness.ctx.clone(); + let mut raw = std::mem::take(self.harness.input_mut()); + eframe::App::raw_input_hook(self.harness.state_mut(), &ctx, &mut raw); + *self.harness.input_mut() = raw; + self.harness.step(); + } + } + + fn ok(&mut self, method: &str, params: Value) -> Value { + let (req, reply) = ControlRequest::new(method, params.clone()); + self.tx.send(req).unwrap(); + for _ in 0..600 { + self.frames(1); + if let Ok(v) = reply.try_recv() { + assert_eq!(v["ok"], json!(true), "{method} {params} failed: {v}"); + return v["result"].clone(); + } + } + panic!("no reply to {method} {params}"); + } + + fn ids(&mut self, prefix: &str) -> Vec { + let v = self.ok("ui.elements", json!({"prefix": prefix})); + v.as_array().unwrap().iter().filter_map(|e| e["id"].as_str().map(str::to_string)).collect() + } + + fn font(&mut self, track: u64) -> String { + let r = self.ok("engine.execute", json!({"command": "captions.list", "params": {}})); + let t = r["tracks"].as_array().unwrap().iter().find(|t| t["id"] == json!(track)).expect("track").clone(); + t["style"]["font"].as_str().unwrap().to_string() + } +} + +#[test] +fn caption_font_picker_sets_the_track_font() { + let mut d = Driver::demo(); + d.ok("ui.set", json!({"workspace": "Captions and Graphics"})); + let track = d.ok("engine.execute", json!({"command": "captions.newTrack", "params": {"format": "Subtitle"}}))["track"].as_u64().unwrap(); + d.ok("engine.execute", json!({"command": "captions.add", "params": {"track": track, "text": "Hello", "seconds": 1.0}})); + d.frames(2); + d.ok("ui.click", json!({"id": "text.tab.Captions"})); + d.frames(3); + assert_eq!(d.font(track), "Inter"); + + d.ok("ui.click", json!({"id": "text.captions.style.font"})); + d.frames(3); + let options = d.ids("text.captions.style.font.option."); + assert!(!options.is_empty(), "the font list is open"); + assert!(!options.iter().any(|o| o.starts_with("text.captions.style.font.option..")), "private faces offered: {options:?}"); + // with system fonts the list is long: scroll until the bundled serif is on screen + let serif = "text.captions.style.font.option.Noto Serif"; + for _ in 0..200 { + let shown = d.ids("text.captions.style.font.option."); + if shown.iter().any(|o| o == serif) { + break; + } + // wheel over a row in the middle (the edge rows are clipped by the popup) + let Some(mid) = shown.get(shown.len() / 2) else { break }; + d.ok("ui.scroll", json!({"id": mid, "dy": -120.0})); + d.frames(2); + } + d.ok("ui.click", json!({"id": serif})); + d.frames(3); + assert_eq!(d.font(track), "Noto Serif"); +} diff --git a/crates/ui-egui/tests/clipboard_paste_ui.rs b/crates/ui-egui/tests/clipboard_paste_ui.rs new file mode 100644 index 00000000..86beeb73 --- /dev/null +++ b/crates/ui-egui/tests/clipboard_paste_ui.rs @@ -0,0 +1,206 @@ +//! Paste files and images from the system clipboard (#611): the real `FilmcraftApp` under +//! `egui_kittest`, with a stand-in for the desktop app's clipboard hook. + +use std::cell::Cell; +use std::rc::Rc; +use std::sync::mpsc::{Sender, channel}; + +use egui_kittest::Harness; +use filmcraft_engine::Session; +use filmcraft_ui_egui::control::ControlRequest; +use filmcraft_ui_egui::{ClipboardMedia, FilmcraftApp}; +use serde_json::{Value, json}; + +struct Driver { + harness: Harness<'static, FilmcraftApp>, + tx: Sender, + dir: std::path::PathBuf, +} + +impl Driver { + /// The demo project saved (in name) in a fresh folder, so pasted images are written there. + fn demo(name: &str) -> Self { + let dir = std::env::temp_dir().join(format!("filmcraft-clipboard-{name}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let mut session = Session::default(); + session.execute("file.openDemoProject", json!({})).expect("demo project"); + session.path = Some(dir.join("film.fcproj").to_string_lossy().into_owned()); + let (tx, rx) = channel(); + let app = FilmcraftApp::new(session).with_control(rx); + let harness = Harness::builder().with_size(egui::vec2(1600.0, 980.0)).with_max_steps(10_000).build_eframe(move |_cc| app); + let mut d = Driver { harness, tx, dir }; + d.frames(4); + d + } + + fn frames(&mut self, n: usize) { + for _ in 0..n { + let ctx = self.harness.ctx.clone(); + let mut raw = std::mem::take(self.harness.input_mut()); + eframe::App::raw_input_hook(self.harness.state_mut(), &ctx, &mut raw); + *self.harness.input_mut() = raw; + self.harness.step(); + } + } + + fn ok(&mut self, method: &str, params: Value) -> Value { + let (req, reply) = ControlRequest::new(method, params.clone()); + self.tx.send(req).unwrap(); + for _ in 0..600 { + self.frames(1); + if let Ok(v) = reply.try_recv() { + assert_eq!(v["ok"], json!(true), "{method} {params} failed: {v}"); + return v["result"].clone(); + } + } + panic!("no reply to {method} {params}"); + } + + fn app(&mut self) -> &mut FilmcraftApp { + self.harness.state_mut() + } + + /// The clipboard hook hands out `media` (a copy each time); returns how often it was asked. + fn clipboard(&mut self, media: Option ClipboardMedia>) -> Rc> { + let asked = Rc::new(Cell::new(0)); + let count = asked.clone(); + self.app().hooks.clipboard_media = Some(Box::new(move || { + count.set(count.get() + 1); + media.map(|m| m()) + })); + asked + } + + /// Ctrl+V as egui-winit delivers it when the system clipboard holds no text: the press is + /// dropped, only the release arrives. + fn ctrl_v_without_text(&mut self, modifiers: egui::Modifiers) { + let key = |pressed| egui::Event::Key { key: egui::Key::V, physical_key: Some(egui::Key::V), pressed, repeat: false, modifiers }; + self.harness.input_mut().events.extend([egui::Event::ModifiersChanged(modifiers), key(false)]); + self.frames(1); + self.harness.input_mut().events.push(egui::Event::ModifiersChanged(egui::Modifiers::NONE)); + self.frames(2); + } + + /// Ctrl+V with text on the clipboard: egui-winit sends `Event::Paste`, then the release. + fn ctrl_v_with_text(&mut self) { + let release = + egui::Event::Key { key: egui::Key::V, physical_key: Some(egui::Key::V), pressed: false, repeat: false, modifiers: egui::Modifiers::COMMAND }; + self.harness.input_mut().events.extend([egui::Event::ModifiersChanged(egui::Modifiers::COMMAND), egui::Event::Paste("text".into()), release]); + self.frames(1); + self.harness.input_mut().events.push(egui::Event::ModifiersChanged(egui::Modifiers::NONE)); + self.frames(2); + } + + /// (track kind, track index, clip id, item name, start, end) of every clip, video first. + fn clips(&mut self) -> Vec<(char, usize, u64, String, i64, i64)> { + let q = self.app().session.active_sequence().unwrap().clone(); + let mut v = Vec::new(); + for (kind, tracks) in [('V', &q.video_tracks), ('A', &q.audio_tracks)] { + for (i, t) in tracks.iter().enumerate() { + v.extend(t.items.iter().map(|c| (kind, i, c.id.0, c.name.clone(), c.start.0, c.end().0))); + } + } + v + } +} + +impl Drop for Driver { + fn drop(&mut self) { + let _ = std::fs::remove_dir_all(&self.dir); + } +} + +fn red_image() -> ClipboardMedia { + ClipboardMedia::Image { width: 32, height: 18, rgba: [220, 30, 30, 255].repeat(32 * 18) } +} + +/// Ctrl+V in the Timeline with an image on the clipboard (no text, so only the key release +/// arrives) saves it next to the project and places it at the playhead above the footage, +/// overwriting nothing. One undo takes it back. +#[test] +fn ctrl_v_pastes_an_image_above_the_footage() { + let mut d = Driver::demo("image"); + d.ok("ui.set", json!({"focused": "Timeline"})); + d.clipboard(Some(red_image)); + let before = d.clips(); + let at = d.app().session.playhead().0; + d.ctrl_v_without_text(egui::Modifiers::COMMAND); + let after = d.clips(); + for c in &before { + assert!(after.contains(c), "{c:?} is untouched"); + } + let new: Vec<_> = after.iter().filter(|c| !before.contains(c)).cloned().collect(); + assert_eq!(new.len(), 1, "one clip pasted: {new:?}"); + let (kind, track, _, name, start, end) = new[0].clone(); + assert_eq!((kind, name.as_str(), start), ('V', "Pasted Image 1.png", at)); + assert!(track > 0 && before.iter().any(|c| c.0 == 'V' && c.1 < track && c.4 < end && c.5 > start), "it sits above a clip"); + assert!(d.dir.join("Pasted Image 1.png").is_file(), "saved next to the project"); + d.ok("ui.menu.invoke", json!({"id": "edit.undo"})); + assert_eq!(d.clips(), before); +} + +/// Files copied in the file manager paste the same way, even when the clipboard also holds text +/// (Ctrl+V then arrives as a paste event); Ctrl+Shift+V inserts them on the targeted track. +#[test] +fn copied_files_paste_and_paste_insert_inserts() { + let mut d = Driver::demo("files"); + d.ok("ui.set", json!({"focused": "Timeline"})); + let png = d.dir.join("copied.png"); + image::RgbaImage::from_pixel(16, 9, image::Rgba([0, 200, 0, 255])).save(&png).unwrap(); + let path = png.to_string_lossy().into_owned(); + d.app().hooks.clipboard_media = Some(Box::new(move || Some(ClipboardMedia::Files(vec![path.clone()])))); + let before = d.clips(); + d.ctrl_v_with_text(); + let new: Vec<_> = d.clips().into_iter().filter(|c| !before.contains(c)).collect(); + assert_eq!(new.iter().map(|c| c.3.as_str()).collect::>(), ["copied.png"]); + assert!(new[0].1 > 0, "not over the footage on V1"); + // Paste Insert: on V1, pushing the clips after the playhead right + let v1_end = |d: &mut Driver| d.clips().iter().filter(|c| c.0 == 'V' && c.1 == 0).map(|c| c.5).max().unwrap(); + let end = v1_end(&mut d); + d.ctrl_v_without_text(egui::Modifiers::COMMAND | egui::Modifiers::SHIFT); + assert!(v1_end(&mut d) > end, "Ctrl+Shift+V inserts on V1"); +} + +/// In the Project panel Ctrl+V imports into the shown bin and places nothing. +#[test] +fn ctrl_v_in_the_project_panel_imports() { + let mut d = Driver::demo("project"); + d.ok("ui.set", json!({"focused": "Project"})); + d.clipboard(Some(red_image)); + let (before, items) = (d.clips(), d.app().session.project.items.len()); + d.ctrl_v_without_text(egui::Modifiers::COMMAND); + assert_eq!(d.clips(), before, "nothing on the Timeline"); + assert_eq!(d.app().session.project.items.len(), items + 1); + assert!(d.app().session.project.items.values().any(|i| i.name == "Pasted Image 1.png")); +} + +/// With nothing but clips copied in FilmCraft, Ctrl+V pastes the clips as before, also when the +/// system clipboard holds no text at all. A plain V press and release (the Selection tool) pastes +/// nothing and doesn't read the clipboard. +#[test] +fn without_media_ctrl_v_pastes_copied_clips() { + let mut d = Driver::demo("clips"); + d.ok("ui.set", json!({"focused": "Timeline"})); + let asked = d.clipboard(None); + let first = d.clips().into_iter().find(|c| c.0 == 'V' && c.1 == 0).unwrap(); + d.ok("engine.execute", json!({"command": "timeline.select", "params": {"clips": [first.2]}})); + d.ok("engine.execute", json!({"command": "edit.copy", "params": {}})); + let copies = |d: &mut Driver| d.clips().iter().filter(|c| c.0 == 'V' && c.3 == first.3).count(); + let n = copies(&mut d); + d.ctrl_v_without_text(egui::Modifiers::COMMAND); + assert_eq!(copies(&mut d), n + 1, "the copied clip is pasted"); + assert_eq!(asked.get(), 1); + for pressed in [true, false] { + d.harness.input_mut().events.push(egui::Event::Key { + key: egui::Key::V, + physical_key: Some(egui::Key::V), + pressed, + repeat: false, + modifiers: egui::Modifiers::NONE, + }); + d.frames(1); + } + d.frames(2); + assert_eq!((copies(&mut d), asked.get()), (n + 1, 1), "V alone pastes nothing"); +} diff --git a/crates/ui-egui/tests/delete_caption_tracks_ui.rs b/crates/ui-egui/tests/delete_caption_tracks_ui.rs new file mode 100644 index 00000000..cd65b134 --- /dev/null +++ b/crates/ui-egui/tests/delete_caption_tracks_ui.rs @@ -0,0 +1,131 @@ +//! Headless UI test of deleting caption tracks: the caption track header's right-click menu +//! (Delete Caption Track / Delete Empty Caption Tracks) and the Caption Tracks section of +//! Sequence ▸ Delete Tracks…. + +use std::sync::mpsc::{Sender, channel}; + +use egui_kittest::Harness; +use filmcraft_engine::Session; +use filmcraft_ui_egui::FilmcraftApp; +use filmcraft_ui_egui::control::ControlRequest; +use serde_json::{Value, json}; + +struct Driver { + harness: Harness<'static, FilmcraftApp>, + tx: Sender, +} + +impl Driver { + fn demo() -> Self { + let mut session = Session::default(); + session.execute("file.openDemoProject", json!({})).expect("demo project"); + let (tx, rx) = channel(); + let app = FilmcraftApp::new(session).with_control(rx); + let harness = Harness::builder().with_size(egui::vec2(1600.0, 980.0)).with_max_steps(10_000).build_eframe(move |_cc| app); + let mut d = Driver { harness, tx }; + d.frames(4); + d + } + + fn frames(&mut self, n: usize) { + for _ in 0..n { + let ctx = self.harness.ctx.clone(); + let mut raw = std::mem::take(self.harness.input_mut()); + eframe::App::raw_input_hook(self.harness.state_mut(), &ctx, &mut raw); + *self.harness.input_mut() = raw; + self.harness.step(); + } + } + + fn ok(&mut self, method: &str, params: Value) -> Value { + let (req, reply) = ControlRequest::new(method, params.clone()); + self.tx.send(req).unwrap(); + for _ in 0..600 { + self.frames(1); + if let Ok(v) = reply.try_recv() { + assert_eq!(v["ok"], json!(true), "{method} {params} failed: {v}"); + return v["result"].clone(); + } + } + panic!("no reply to {method} {params}"); + } + + fn ids(&mut self, prefix: &str) -> Vec { + let v = self.ok("ui.elements", json!({"prefix": prefix})); + v.as_array().unwrap().iter().filter_map(|e| e["id"].as_str().map(str::to_string)).collect() + } + + fn exec(&mut self, cmd: &str, params: Value) -> Value { + self.ok("engine.execute", json!({"command": cmd, "params": params})) + } + + /// Caption track names top first, with their caption counts. + fn caption_tracks(&mut self) -> Vec<(String, usize)> { + let r = self.exec("captions.list", json!({})); + r["tracks"].as_array().unwrap().iter().map(|t| (t["name"].as_str().unwrap().to_string(), t["captions"].as_array().unwrap().len())).collect() + } + + fn right_click_menu(&mut self, track: &str, item: &str) { + self.ok("ui.click", json!({"id": format!("timeline.captionTrack.{track}"), "button": "right"})); + self.frames(3); + self.ok("ui.click", json!({"id": format!("timeline.captionTrack.{track}.menu.{item}")})); + self.frames(3); + } +} + +fn setup() -> Driver { + let mut d = Driver::demo(); + // start from no caption tracks, then add four (each goes on top): Keep (with a caption), E1–E3 empty + while !d.caption_tracks().is_empty() { + d.exec("captions.deleteTrack", json!({"track": "C1"})); + } + for name in ["E3", "E2", "Keep", "E1"] { + d.exec("captions.newTrack", json!({"format": "Subtitle", "name": name})); + } + d.exec("captions.add", json!({"track": "C2", "text": "Hello", "seconds": 1.0})); + d.frames(3); + assert_eq!(d.caption_tracks(), [("E1".into(), 0), ("Keep".into(), 1), ("E2".into(), 0), ("E3".into(), 0)]); + d +} + +#[test] +fn header_menu_deletes_one_or_all_empty_caption_tracks() { + let mut d = setup(); + d.right_click_menu("C1", "delete"); + assert_eq!(d.caption_tracks(), [("Keep".into(), 1), ("E2".into(), 0), ("E3".into(), 0)]); + // from the non-empty track's menu: only the empty ones go + d.right_click_menu("C1", "deleteEmpty"); + assert_eq!(d.caption_tracks(), [("Keep".into(), 1)]); + // the last caption track can be deleted too, captions and all; undo brings it back + d.right_click_menu("C1", "delete"); + assert!(d.caption_tracks().is_empty()); + assert!(d.ids("timeline.captionTrack.").is_empty(), "no caption rows left"); + d.exec("edit.undo", json!({})); + d.frames(3); + assert_eq!(d.caption_tracks(), [("Keep".into(), 1)]); +} + +#[test] +fn delete_tracks_dialog_has_a_caption_section() { + let mut d = setup(); + d.ok("ui.menu.invoke", json!({"command": "sequence.deleteTracks", "params": {}})); + d.frames(3); + d.ok("ui.click", json!({"id": "deleteTracks.captions"})); + d.frames(2); + // one specific track: C3 (E2) + d.ok("ui.click", json!({"id": "deleteTracks.captions.target"})); + d.frames(3); + d.ok("ui.click", json!({"id": "deleteTracks.captions.option.C3"})); + d.frames(2); + d.ok("ui.click", json!({"id": "deleteTracks.ok"})); + d.frames(3); + assert_eq!(d.caption_tracks(), [("E1".into(), 0), ("Keep".into(), 1), ("E3".into(), 0)]); + // All Empty Tracks (the default choice) + d.ok("ui.menu.invoke", json!({"command": "sequence.deleteTracks", "params": {}})); + d.frames(3); + d.ok("ui.click", json!({"id": "deleteTracks.captions"})); + d.frames(2); + d.ok("ui.click", json!({"id": "deleteTracks.ok"})); + d.frames(3); + assert_eq!(d.caption_tracks(), [("Keep".into(), 1)]); +} diff --git a/crates/ui-egui/tests/deselect_ui.rs b/crates/ui-egui/tests/deselect_ui.rs new file mode 100644 index 00000000..66095e65 --- /dev/null +++ b/crates/ui-egui/tests/deselect_ui.rs @@ -0,0 +1,154 @@ +//! Clicking empty space deselects, consistently (#683): the Timeline's empty space below and +//! between the tracks, a text layer being edited when another panel is clicked, and the Program +//! monitor's empty space around the picture (graphic layers, masks). + +use std::sync::mpsc::{Sender, channel}; + +use egui_kittest::Harness; +use filmcraft_engine::Session; +use filmcraft_ui_egui::FilmcraftApp; +use filmcraft_ui_egui::control::ControlRequest; +use filmcraft_ui_egui::state::GfxEdit; +use serde_json::{Value, json}; + +struct Driver { + harness: Harness<'static, FilmcraftApp>, + tx: Sender, +} + +impl Driver { + fn demo() -> Self { + let mut session = Session::default(); + session.execute("file.openDemoProject", json!({})).expect("demo project"); + let (tx, rx) = channel(); + let app = FilmcraftApp::new(session).with_control(rx); + let harness = Harness::builder().with_size(egui::vec2(1600.0, 980.0)).with_max_steps(10_000).build_eframe(move |_cc| app); + let mut d = Driver { harness, tx }; + d.frames(4); + d + } + fn frames(&mut self, n: usize) { + for _ in 0..n { + let ctx = self.harness.ctx.clone(); + let mut raw = std::mem::take(self.harness.input_mut()); + eframe::App::raw_input_hook(self.harness.state_mut(), &ctx, &mut raw); + *self.harness.input_mut() = raw; + self.harness.step(); + } + } + fn ok(&mut self, method: &str, params: Value) -> Value { + let (req, reply) = ControlRequest::new(method, params.clone()); + self.tx.send(req).unwrap(); + for _ in 0..600 { + self.frames(1); + if let Ok(v) = reply.try_recv() { + assert_eq!(v["ok"], json!(true), "{method} {params} failed: {v}"); + return v["result"].clone(); + } + } + panic!("no reply to {method} {params}"); + } + fn exec(&mut self, command: &str, params: Value) -> Value { + self.ok("engine.execute", json!({"command": command, "params": params})) + } + fn rect(&mut self, id: &str) -> [f64; 4] { + let v = self.ok("ui.elements", json!({"prefix": id})); + let e = v.as_array().unwrap().iter().find(|e| e["id"] == id).unwrap_or_else(|| panic!("no element {id}")).clone(); + let r: Vec = e["rect"].as_array().unwrap().iter().map(|x| x.as_f64().unwrap()).collect(); + [r[0], r[1], r[2], r[3]] + } + fn click_at(&mut self, x: f64, y: f64) { + self.ok("ui.click", json!({"x": x, "y": y})); + self.frames(3); + } + fn app(&mut self) -> &mut FilmcraftApp { + self.harness.state_mut() + } +} + +/// A click on the Timeline below the tracks deselects, as one on an empty stretch of a track does; +/// a drag from there draws a box selection. +#[test] +fn empty_timeline_space_below_the_tracks_deselects_and_starts_a_marquee() { + let mut d = Driver::demo(); + d.ok("ui.set", json!({"focused": "Timeline"})); + let seq = d.exec("sequence.inspect", json!({})); + let clip = seq["video"][0]["items"][0]["clip"].as_u64().unwrap(); + let area = d.rect("timeline.tracks"); + // a point in the track area that is on no track at all: going up from the area's bottom + let x = area[0] + area[2] * 0.5; + let y = (0..60) + .map(|k| area[1] + area[3] - 3.0 - 4.0 * k as f64) + .find(|y| d.ok("ui.timeline.hit", json!({"x": x, "y": y}))["hit"] == "none") + .expect("the track area has space on no track"); + let below = (x, y); + d.exec("timeline.select", json!({"clips": [clip]})); + d.frames(2); + d.click_at(below.0, below.1); + assert!(d.app().session.state.selection.is_empty(), "a click below the tracks deselects"); + // a drag from below the tracks up over the clips selects them + let top = d.rect("timeline.track.V1.locked"); + d.ok("ui.drag", json!({"from": {"x": area[0] + 4.0, "y": below.1}, "to": {"x": area[0] + area[2] * 0.6, "y": top[1] + top[3] / 2.0}, "steps": 8})); + d.frames(3); + assert!(!d.app().session.state.selection.is_empty(), "the box from below the tracks selected clips"); +} + +/// A text layer being edited stops being edited when another panel is clicked (its keys went into +/// the text before, #227); working in Properties, which styles the text, keeps the edit. +#[test] +fn clicking_another_panel_ends_text_editing() { + let mut d = Driver::demo(); + let seq = d.exec("sequence.inspect", json!({})); + let at = seq["playhead"].as_i64().unwrap(); + let clip = d.exec("graphics.newText", json!({"text": "Title", "time": at}))["clip"].as_u64().unwrap(); + d.ok("ui.panel.show", json!({"panel": "Properties"})); + let edit = |d: &mut Driver| { + d.app().ui.gfx_edit = Some(GfxEdit { clip, layer: 0, caret: 0, anchor: 0 }); + d.app().ui.focused = filmcraft_ui_egui::dock::PanelKind::Program; + d.frames(3); + }; + edit(&mut d); + assert!(d.app().ui.gfx_edit.is_some()); + let props = d.rect("panel.Properties"); + d.click_at(props[0] + props[2] / 2.0, props[1] + props[3] - 10.0); + assert!(d.app().ui.gfx_edit.is_some(), "a click in Properties keeps the edit"); + let tl = d.rect("timeline.tracks"); + d.click_at(tl[0] + tl[2] / 2.0, tl[1] + tl[3] - 6.0); + assert!(d.app().ui.gfx_edit.is_none(), "a click in the Timeline ends it"); +} + +/// A click on the Program monitor around the picture lets go of the selected graphic layer and +/// mask; so does a click on the picture's empty space. +#[test] +fn empty_monitor_space_deselects_layers_and_masks() { + let mut d = Driver::demo(); + let pic = d.rect("program.picture"); + let panel = d.rect("panel.Program"); + // the empty space beside or above the picture, a few points away from it + let outside = if pic[0] - panel[0] > 20.0 { (pic[0] - 8.0, pic[1] + pic[3] / 2.0) } else { (pic[0] + pic[2] / 2.0, pic[1] + pic[3] + 8.0) }; + let seq = d.exec("sequence.inspect", json!({})); + let clip = seq["video"][0]["items"][0]["clip"].as_u64().unwrap(); + // a graphic layer + let at = seq["playhead"].as_i64().unwrap(); + let gclip = d.exec("graphics.newText", json!({"text": "Title", "time": at}))["clip"].as_u64().unwrap(); + d.exec("graphics.selectLayer", json!({"clip": gclip, "layers": [0]})); + d.frames(2); + assert!(!d.app().session.state.graphic_layers.is_empty()); + d.click_at(outside.0, outside.1); + assert!(d.app().session.state.graphic_layers.is_empty(), "a click around the picture deselects the layer"); + // a mask: around the picture, and on the picture's empty space + d.exec("timeline.select", json!({"clips": [clip]})); + d.exec("effects.apply", json!({"clips": [clip], "effect": "gaussian_blur"})); + let fx = { + let s = &d.harness.state().session; + let it = s.active_sequence().unwrap().find_item(filmcraft_project::ClipId(clip)).unwrap().1; + it.effects.iter().position(|e| e.effect == "gaussian_blur").unwrap() + }; + d.exec("masks.add", json!({"clip": clip, "effect": fx, "shape": "ellipse"})); + for (x, y) in [outside, (pic[0] + 6.0, pic[1] + 6.0)] { + d.exec("masks.select", json!({"clip": clip, "effect": fx, "mask": 0})); + d.frames(2); + d.click_at(x, y); + assert!(d.app().session.state.selected_mask.is_none(), "a click at ({x}, {y}) deselects the mask"); + } +} diff --git a/crates/ui-egui/tests/graphics_handles_ui.rs b/crates/ui-egui/tests/graphics_handles_ui.rs index 53879852..b7553926 100644 --- a/crates/ui-egui/tests/graphics_handles_ui.rs +++ b/crates/ui-egui/tests/graphics_handles_ui.rs @@ -1,6 +1,6 @@ //! Headless UI tests of the handles of graphic layers in the Program monitor: point text scales -//! about its anchor point, a paragraph-text box is resized, a shape stretches away from its -//! opposite side (docs/graphics.md). +//! about its anchor point, a paragraph-text box is resized, a shape's Size follows the handle +//! while its opposite side and anchor point stay (docs/graphics.md). //! //! With `FILMCRAFT_UI_SHOTS=` the tests also render the UI with wgpu and save PNGs there. @@ -130,6 +130,14 @@ impl Driver { self.frames(4); } + /// Press or let go of Shift, as a keyboard reports it (the control channel's drags carry + /// modifiers only on their press and release). + fn hold_shift(&mut self, on: bool) { + let m = if on { egui::Modifiers::SHIFT } else { egui::Modifiers::NONE }; + self.harness.input_mut().events.push(egui::Event::ModifiersChanged(m)); + self.frames(1); + } + fn drag_handle(&mut self, pre: &str, n: usize, dx: f64, dy: f64) { let h = self.handle(pre, n); self.drag(h, (dx, dy)); @@ -372,6 +380,58 @@ fn shape_corner_handles_scale_away_from_the_opposite_corner() { assert!(bl2.0 < bl.0 - 20.0 && bl2.1 > bl.1 + 5.0, "bottom-left follows the pointer: {bl:?} -> {bl2:?}"); } +/// Width and height of a layer's box in its own pixels. +fn box_size(l: &Value) -> (f64, f64) { + let b = &l["localBounds"]; + (num(&b[2]) - num(&b[0]), num(&b[3]) - num(&b[1])) +} + +#[test] +fn shape_handles_change_its_size_and_shift_keeps_the_ratio() { + let mut d = Driver::new(); + let pre = d.shape(); + let clip: u64 = pre.split('.').nth(2).unwrap().parse().unwrap(); + // a corner straight right: only wider, as in Premiere; the Size changes, not the Scale + let (tl, br, anchor) = (d.handle(&pre, 0), d.handle(&pre, 2), d.at(&pre, "anchor")); + let before = d.layer(clip); + d.drag_handle(&pre, 2, 60.0, 0.0); + let l = d.layer(clip); + assert_eq!((num(&l["scale"]), num(&l["scaleWidth"])), (100.0, 100.0), "{l}"); + let (w, h) = box_size(&l); + assert!(w > 510.0 && (h - 300.0).abs() < 1e-3, "wider, as tall: {w} x {h}"); + let (tl2, br2) = (d.handle(&pre, 0), d.handle(&pre, 2)); + assert!(near(tl, tl2), "top-left stays put: {tl:?} -> {tl2:?}"); + assert!(near(br2, (br.0 + 60.0, br.1)), "bottom-right follows the pointer: {br:?} -> {br2:?}"); + assert!(near(d.at(&pre, "anchor"), anchor), "the anchor point stays put"); + assert_eq!(l["position"], before["position"]); + + // with Shift both sides grow alike from the opposite corner + d.hold_shift(true); + d.drag_handle(&pre, 2, 40.0, 0.0); + d.hold_shift(false); + let l2 = d.layer(clip); + let (w2, h2) = box_size(&l2); + assert!(h2 > 310.0, "taller too: {w2} x {h2}"); + assert!((w2 / h2 - w / h).abs() < 1e-3, "same proportions: {w}x{h} -> {w2}x{h2}"); + assert!(near(d.handle(&pre, 0), tl), "top-left still stays put"); + assert_eq!(num(&l2["scale"]), 100.0); + // one undo step per drag + d.exec("edit.undo", json!({})); + assert_eq!(box_size(&d.layer(clip)), (w, h)); +} + +#[test] +fn polygon_handles_keep_the_opposite_corner() { + let mut d = Driver::new(); + let r = d.exec("graphics.newPolygon", json!({"sides": 5, "position": [800, 500], "size": [300, 300]})); + let pre = d.select(r["clip"].as_u64().unwrap()); + // a pentagon's box is not centred on the shape's origin; the top-left corner stays anyway + let (tl, br) = (d.handle(&pre, 0), d.handle(&pre, 2)); + d.drag_handle(&pre, 2, 50.0, 40.0); + assert!(near(d.handle(&pre, 0), tl), "top-left stays put"); + assert!(near(d.handle(&pre, 2), (br.0 + 50.0, br.1 + 40.0)), "bottom-right follows the pointer"); +} + #[test] fn shape_edge_handles_stretch_one_axis() { let mut d = Driver::new(); diff --git a/crates/ui-egui/tests/keyboard_ui.rs b/crates/ui-egui/tests/keyboard_ui.rs index 03660632..930ae086 100644 --- a/crates/ui-egui/tests/keyboard_ui.rs +++ b/crates/ui-egui/tests/keyboard_ui.rs @@ -71,6 +71,31 @@ impl Driver { } } +#[test] +fn alt_click_isolates_already_selected_linked_clip_before_delete() { + for audio in [false, true] { + let mut d = Driver::new(); + let video = d.app().session.active_sequence().unwrap().video_tracks[0].items[0].clone(); + let sound = d.app().session.active_sequence().unwrap().audio_tracks[0].items.iter().find(|i| i.link == video.link).unwrap().clone(); + let (selected, partner) = if audio { (sound.id, video.id) } else { (video.id, sound.id) }; + d.exec("timeline.select", json!({"clips": [selected.0]})); + assert_eq!(d.app().session.state.selection.len(), 2); + let r = d.app().auto.find(&format!("timeline.clip.{}", selected.0)).unwrap().rect; + d.harness.input_mut().events.push(egui::Event::ModifiersChanged(egui::Modifiers::ALT)); + d.frames(1); + d.ok("ui.click", json!({"x": r[0] + r[2] * 0.5, "y": r[1] + r[3] * 0.3, "modifiers": {"alt": true}})); + d.frames(3); + assert_eq!(d.app().session.state.selection, vec![selected]); + d.harness.input_mut().events.push(egui::Event::ModifiersChanged(egui::Modifiers::NONE)); + d.frames(1); + d.key("Delete"); + assert!(d.app().session.active_sequence().unwrap().find_item(selected).is_none()); + assert!(d.app().session.active_sequence().unwrap().find_item(partner).is_some()); + d.exec("edit.undo", json!({})); + assert!(d.app().session.active_sequence().unwrap().find_item(selected).is_some()); + } +} + /// Ids added for keyboard parity (engine + UI). fn new_ids() -> Vec { let mut v: Vec = filmcraft_engine::keyboard::commands().iter().map(|c| c.id.to_string()).collect(); @@ -238,3 +263,27 @@ fn delete_key_clears_the_selected_timeline_clip() { d.key("Cmd+Z"); assert!(items(&mut d).iter().any(|i| i["clip"].as_u64() == Some(clip)), "and undo brings it back"); } + +#[test] +fn shift_delete_ripple_deletes_with_the_timeline_focused() { + // #680: the Timeline's own `Delete` (Clear) was tried before the app-wide `Shift+Delete` + // (Ripple Delete), and egui ignores the extra Shift, so Shift+Delete left a gap. + let mut d = Driver::new(); + // the demo's other tracks have clips in the way of a sync-locked ripple + for t in ["V2", "V3", "A2", "A3"] { + d.exec("timeline.setTrack", json!({"track": t, "syncLock": false})); + } + let items = |d: &mut Driver| d.exec("sequence.inspect", json!({}))["video"][0]["items"].as_array().unwrap().clone(); + let v1 = items(&mut d); + let (clip, start) = (v1[1]["clip"].as_u64().unwrap(), v1[1]["start"].as_i64().unwrap()); + let next = v1[2]["clip"].as_u64().unwrap(); + d.exec("playhead.set", json!({"time": start + 1_000_000_000})); + d.focus("Timeline"); + d.key("D"); + assert!(d.app().session.state.selection.iter().any(|c| c.0 == clip), "D selects the clip under the playhead"); + d.key("Shift+Delete"); + let after = items(&mut d); + assert!(!after.iter().any(|i| i["clip"].as_u64() == Some(clip)), "Shift+Delete removed the selected clip"); + let moved = after.iter().find(|i| i["clip"].as_u64() == Some(next)).map(|i| i["start"].as_i64().unwrap()); + assert_eq!(moved, Some(start), "and closed the gap"); +} diff --git a/crates/ui-egui/tests/keyframes_ui.rs b/crates/ui-egui/tests/keyframes_ui.rs index 87986f54..39f9c35b 100644 --- a/crates/ui-egui/tests/keyframes_ui.rs +++ b/crates/ui-egui/tests/keyframes_ui.rs @@ -241,3 +241,36 @@ fn effect_controls_ruler_moves_the_playhead() { assert!(d.find("effectControls.playhead").is_none(), "no handle while the playhead is past the clip"); d.rect("effectControls.ruler"); } + +/// #412: Scale and Position both have a keyframe at the same time. A click on Scale's keyframe in +/// the Effect Controls lane selects that keyframe only (every keyframe under the playhead, which +/// the click moves there, used to light up); a keyframe that is gone drops out of the selection. +#[test] +fn clicking_a_lane_keyframe_selects_only_that_keyframe() { + let (mut d, clip) = Driver::demo(); + let t0 = d.seek(1.0); + for param in ["scale", "position"] { + d.exec("effects.addKeyframe", json!({"clip": clip.id, "effect": "motion", "param": param})); + } + let time = d.scale(&clip).0[0]; + let motion = { + let s = &d.harness.state().session; + let it = s.active_sequence().unwrap().find_item(ClipId(clip.id)).unwrap().1; + it.effects.iter().position(|e| e.effect == "motion").unwrap() + }; + let selection = |d: &mut Driver| d.ok("ui.inspect", json!({}))["ui"]["keyframe_selection"].clone(); + d.seek(2.0); + assert_eq!(selection(&mut d), json!([]), "nothing selected yet"); + + d.click(&format!("effectControls.motion.scale.keyframe.{time}")); + assert_eq!(selection(&mut d), json!([{"clip": clip.id, "effect": motion, "param": "scale", "time": time}]), "only Scale's keyframe"); + assert_eq!(d.playhead(), t0, "the click still moves the playhead to the keyframe"); + d.shot("keyframes-select-one"); + + d.click(&format!("effectControls.motion.position.keyframe.{time}")); + assert_eq!(selection(&mut d), json!([{"clip": clip.id, "effect": motion, "param": "position", "time": time}]), "a click replaces the selection"); + + // the diamond removes the keyframe at the playhead, the selected one: the selection empties + d.click("effectControls.motion.position.addKeyframe"); + assert_eq!(selection(&mut d), json!([]), "a deleted keyframe is not selected"); +} diff --git a/crates/ui-egui/tests/media_ui.rs b/crates/ui-egui/tests/media_ui.rs index 80246ef0..c36dc8ca 100644 --- a/crates/ui-egui/tests/media_ui.rs +++ b/crates/ui-egui/tests/media_ui.rs @@ -351,6 +351,43 @@ fn source_menu_steps_preserve_zero_tick_frame_duration_cursor() { } } +#[test] +fn link_media_large_lists_and_long_paths_keep_actions_on_screen() { + for (width, height) in [(1600.0, 980.0), (1024.0, 768.0), (800.0, 600.0)] { + let mut s = Session::default(); + s.execute("file.openDemoProject", json!({})).unwrap(); + let mut p = (*s.project).clone(); + let template = p.items.values().find_map(|i| i.as_media()).unwrap().clone(); + for n in 0..100 { + let mut media = template.clone(); + media.media = MediaRef::File { path: format!("/missing/{}/clip-{n}.mov", "long-project-directory/".repeat(20)) }; + media.offline = true; + let id = p.add_item(&format!("Missing clip {n}"), Label::Iris, ItemKind::Media(media), None); + s.offline.missing.push(id); + } + s.project = Arc::new(p); + let mut app = FilmcraftApp::new(s); + app.ui.link_media = Some(Default::default()); + let mut builder = Harness::builder().with_size(egui::vec2(width, height)).with_max_steps(10_000); + let snapshots = std::env::var_os("FILMCRAFT_UI_SNAPSHOT_DIR").map(PathBuf::from); + if snapshots.is_some() { + builder = builder.wgpu(); + } + let mut h = builder.build_eframe(move |_| app); + for _ in 0..8 { + h.step(); + } + for id in ["linkMedia.locate", "linkMedia.cancel", "linkMedia.offlineAll", "linkMedia.link"] { + let r = h.state().auto.find(id).unwrap().rect; + assert!(r[0] >= 0.0 && r[1] >= 0.0 && r[0] + r[2] <= width && r[1] + r[3] <= height, "{id} at {r:?} outside {width}x{height}"); + } + if let Some(dir) = snapshots { + std::fs::create_dir_all(&dir).unwrap(); + h.render().unwrap().save(dir.join(format!("media-long-paths-{width}x{height}.png"))).unwrap(); + } + } +} + #[test] fn link_media_dialog_relinks_moved_media() { let root = tmp_dir("link"); diff --git a/crates/ui-egui/tests/mixer_ui.rs b/crates/ui-egui/tests/mixer_ui.rs index 0d4b6f28..e698d2bf 100644 --- a/crates/ui-egui/tests/mixer_ui.rs +++ b/crates/ui-egui/tests/mixer_ui.rs @@ -269,6 +269,27 @@ fn clip_mixer_strips_follow_the_playhead() { d.snapshot("mixer-clip-mixer", Some("clipMixer.")); } +#[test] +fn clip_mixer_db_value_can_be_typed() { + let mut d = Driver::demo(); + d.ok("ui.panel.show", json!({"panel": "AudioClipMixer"})); + d.frames(3); + d.exec("playhead.set", json!({"seconds": 1.0})); + d.frames(2); + d.click("clipMixer.A1.value"); + d.ok("ui.key", json!({"key": "Cmd+A"})); + d.frames(1); + d.ok("ui.type", json!({"text": "-20"})); + d.frames(2); + d.ok("ui.key", json!({"key": "Enter"})); + d.frames(3); + let seq = d.exec("sequence.inspect", json!({})); + let fx = &seq["audio"][0]["items"][0]["effects"]; + let vol = fx.as_array().unwrap().iter().find(|e| e["effect"] == "volume").unwrap(); + let level: f64 = vol["params"]["level"]["value"].as_str().and_then(|v| v.strip_prefix("Float(")?.strip_suffix(')')?.parse().ok()).unwrap(); + assert!((level + 20.0).abs() < 1e-6, "typed -20 dB into the clip mixer readout: {level}"); +} + #[test] fn audio_gain_dialog_normalizes_the_selection() { let mut d = Driver::demo(); diff --git a/crates/ui-egui/tests/project_panel_ui.rs b/crates/ui-egui/tests/project_panel_ui.rs index bb504f32..3fac337e 100644 --- a/crates/ui-egui/tests/project_panel_ui.rs +++ b/crates/ui-egui/tests/project_panel_ui.rs @@ -447,6 +447,44 @@ fn bins_open_in_place_in_a_tab_or_in_a_window() { assert_eq!(name, "Camera A"); } +#[test] +fn select_all_takes_only_the_shown_bin() { + // #456: Cmd+A selected every item in the project, closed bins and other tabs included + let mut d = project_driver(); + d.exec("project.view.set", json!({"view": "list"})); + let (bin, items) = d.footage(); + assert!(!items.is_empty()); + let sorted = |v: &[ItemId]| { + let mut v: Vec = v.iter().map(|i| i.0).collect(); + v.sort(); + v + }; + let mut footage = items.clone(); + footage.sort(); + // the root with the Footage bin closed: none of its items + d.app().ui.expanded_bins.clear(); + d.frames(2); + d.key("Cmd+A"); + let sel = d.app().session.state.project_selection.clone(); + assert!(sel.iter().all(|i| !items.contains(&i.0)), "{sel:?}"); + assert!(sel.iter().all(|i| d.app().session.project.root.children.contains(&BinEntry::Item(*i))), "{sel:?}"); + // twirled open in List view, its items are shown and count + d.app().ui.expanded_bins.push(bin); + d.frames(2); + d.key("Cmd+A"); + let sel = d.app().session.state.project_selection.clone(); + assert!(items.iter().all(|i| sel.contains(&ItemId(*i))), "{sel:?}"); + d.key("Cmd+Shift+A"); + assert!(d.app().session.state.project_selection.is_empty()); + // the bin opened in its own tab: exactly its items + d.app().ui.expanded_bins.clear(); + d.app().session.prefs.general.bins_double_click = "openNewTab".into(); + double_click_bin(&mut d, bin, false); + assert_eq!(d.app().ui.project_panel.active_tab, Some(0)); + d.key("Cmd+A"); + assert_eq!(sorted(&d.app().session.state.project_selection), footage); +} + #[test] fn footer_buttons_are_wired() { let mut d = project_driver(); @@ -626,3 +664,129 @@ fn media_browser_drag_imports_into_the_project_panel() { assert_eq!(d.app().session.project.items.len(), n0 + 1, "kept after a drop on the Timeline"); let _ = std::fs::remove_dir_all(&dir); } + +// ---- marquee selection (#578) + +/// The Footage bin open in place in `view`, with its items' rects (row by row, left to right). +fn footage_in(d: &mut Driver, view: &str) -> Vec<(u64, [f32; 4])> { + let (bin, items) = d.footage(); + d.exec("project.view.set", json!({"view": view})); + d.ok("ui.menu.invoke", json!({"id": "projectPanel.openBin", "params": {"bin": bin, "how": "inPlace"}})); + d.wait_for(&format!("project.item.{}", items[0])); + let mut rects: Vec<(u64, [f32; 4])> = items.iter().map(|i| (*i, d.rect(&format!("project.item.{i}")))).collect(); + rects.sort_by(|a, b| (a.1[1], a.1[0]).partial_cmp(&(b.1[1], b.1[0])).unwrap()); + rects +} + +fn selection(d: &mut Driver) -> Vec { + let mut s: Vec = d.app().session.state.project_selection.iter().map(|i| i.0).collect(); + s.sort_unstable(); + s +} + +fn sorted(mut v: Vec) -> Vec { + v.sort_unstable(); + v +} + +fn centre(r: [f32; 4]) -> (f32, f32) { + (r[0] + r[2] / 2.0, r[1] + r[3] / 2.0) +} + +fn marquee(d: &mut Driver, from: (f32, f32), to: (f32, f32), shift: bool) { + d.ok("ui.drag", json!({"from": {"x": from.0, "y": from.1}, "to": {"x": to.0, "y": to.1}, "steps": 12, "modifiers": {"shift": shift}})); + d.frames(3); +} + +#[test] +fn marquee_selects_cards_in_icon_view() { + let mut d = project_driver(); + let cards = footage_in(&mut d, "icon"); + let [(i0, a), (i1, b), (i2, c)] = [cards[0], cards[1], cards[2]]; + assert!((a[1] - c[1]).abs() < 1.0, "the first three cards share a row"); + // from the gap above-left of the first card to the middle of the second + marquee(&mut d, (a[0] - 4.0, a[1] - 4.0), centre(b), false); + assert_eq!(selection(&mut d), sorted(vec![i0, i1])); + // Shift adds + marquee(&mut d, (c[0] + 4.0, c[1] - 4.0), centre(c), true); + assert_eq!(selection(&mut d), sorted(vec![i0, i1, i2])); + // a plain drag replaces + marquee(&mut d, (a[0] - 4.0, a[1] - 4.0), centre(a), false); + assert_eq!(selection(&mut d), vec![i0]); + // the gaps between cards are empty space: a click there deselects + d.ok("ui.click", json!({"x": a[0] - 4.0, "y": a[1] - 4.0})); + d.frames(2); + assert!(selection(&mut d).is_empty()); +} + +#[test] +fn escape_cancels_a_marquee_and_keeps_the_selection() { + let mut d = project_driver(); + let cards = footage_in(&mut d, "icon"); + let [(i0, a), (i1, b), (i2, c)] = [cards[0], cards[1], cards[2]]; + d.exec("project.select", json!({"items": [i2]})); + let send = |d: &mut Driver, e: egui::Event| { + d.harness.input_mut().events.push(e); + d.frames(1); + }; + let (from, to) = (egui::pos2(a[0] - 4.0, a[1] - 4.0), egui::pos2(centre(b).0, centre(b).1)); + send(&mut d, egui::Event::PointerMoved(from)); + send(&mut d, egui::Event::PointerButton { pos: from, button: egui::PointerButton::Primary, pressed: true, modifiers: Default::default() }); + for k in 1..=10 { + send(&mut d, egui::Event::PointerMoved(from.lerp(to, k as f32 / 10.0))); + } + assert_eq!(selection(&mut d), sorted(vec![i0, i1]), "the selection follows the rectangle"); + send(&mut d, egui::Event::Key { key: egui::Key::Escape, physical_key: None, pressed: true, repeat: false, modifiers: Default::default() }); + let end = egui::pos2(centre(c).0, centre(c).1); + send(&mut d, egui::Event::PointerMoved(end)); + send(&mut d, egui::Event::PointerButton { pos: end, button: egui::PointerButton::Primary, pressed: false, modifiers: Default::default() }); + d.frames(2); + assert_eq!(selection(&mut d), vec![i2], "Escape restored the selection the drag began with"); +} + +#[test] +fn only_the_left_button_draws_a_marquee() { + let mut d = project_driver(); + let cards = footage_in(&mut d, "icon"); + let [(i0, a), (_, b)] = [cards[0], cards[1]]; + d.exec("project.select", json!({"items": [i0]})); + let (from, to) = (egui::pos2(a[0] - 4.0, a[1] - 4.0), egui::pos2(centre(b).0, centre(b).1)); + for button in [egui::PointerButton::Middle, egui::PointerButton::Secondary] { + let mut send = |e: egui::Event| { + d.harness.input_mut().events.push(e); + d.frames(1); + }; + send(egui::Event::PointerMoved(from)); + send(egui::Event::PointerButton { pos: from, button, pressed: true, modifiers: Default::default() }); + for k in 1..=10 { + send(egui::Event::PointerMoved(from.lerp(to, k as f32 / 10.0))); + } + send(egui::Event::PointerButton { pos: to, button, pressed: false, modifiers: Default::default() }); + d.key("Escape"); + assert_eq!(selection(&mut d), vec![i0], "{button:?} drag left the selection alone"); + assert!(!d.has("project.marquee")); + } +} + +#[test] +fn marquee_selects_rows_in_list_view() { + let mut d = project_driver(); + let rows = footage_in(&mut d, "list"); + let n = rows.len(); + let (last, before_last) = (rows[n - 1], rows[n - 2]); + let empty = d.rect("project.empty"); + // from the empty space under the rows up into the second-to-last row + marquee(&mut d, (empty[0] + 60.0, empty[1] + 20.0), (empty[0] + 60.0, centre(before_last.1).1), false); + assert_eq!(selection(&mut d), sorted(vec![last.0, before_last.0])); +} + +#[test] +fn marquee_selects_cards_in_freeform_view() { + let mut d = project_driver(); + let cards = footage_in(&mut d, "freeform"); + let [(i0, a), (i1, b)] = [cards[0], cards[1]]; + assert!((a[1] - b[1]).abs() < 1.0, "the first two cards share a row"); + marquee(&mut d, (a[0] - 6.0, a[1] - 6.0), centre(b), false); + assert_eq!(selection(&mut d), sorted(vec![i0, i1])); + assert!(!d.app().session.project.item(ItemId(i0)).unwrap().metadata.contains_key(FREEFORM_POS), "the marquee moved no card"); +} diff --git a/crates/ui-egui/tests/settings_ui.rs b/crates/ui-egui/tests/settings_ui.rs index 3d19aaaf..eca13d50 100644 --- a/crates/ui-egui/tests/settings_ui.rs +++ b/crates/ui-egui/tests/settings_ui.rs @@ -333,6 +333,31 @@ fn appearance_labels_and_tooltips_take_effect() { assert!(d.harness.ctx.global_style().interaction.tooltip_delay < 1.0); } +#[test] +fn hex_field_takes_typing_a_character_at_a_time() { + let mut d = Driver::demo(); + d.menu("app.settings.appearance"); + let hex = |d: &mut Driver| d.draft()["values"]["appearance"]["highlightColor"].clone(); + let before = hex(&mut d); + d.ok("ui.click", json!({"id": "settings.appearance.highlightColor.hex"})); + d.frames(2); + d.ok("ui.key", json!({"key": "Cmd+A"})); + // not a colour yet: the field keeps it and the setting stays + d.ok("ui.type", json!({"text": "#12ab"})); + d.frames(2); + assert_eq!(hex(&mut d), before); + d.ok("ui.type", json!({"text": "34"})); + d.frames(2); + assert_eq!(hex(&mut d), "#12ab34"); + // deleting a character leaves the last colour + d.ok("ui.key", json!({"key": "Backspace"})); + d.frames(2); + assert_eq!(hex(&mut d), "#12ab34"); + d.ok("ui.type", json!({"text": "5"})); + d.frames(2); + assert_eq!(hex(&mut d), "#12ab35"); +} + #[test] fn memory_and_media_cache_pages() { let mut d = Driver::demo(); diff --git a/crates/ui-egui/tests/source_playback_ui.rs b/crates/ui-egui/tests/source_playback_ui.rs index 7ae95546..c2028303 100644 --- a/crates/ui-egui/tests/source_playback_ui.rs +++ b/crates/ui-egui/tests/source_playback_ui.rs @@ -196,3 +196,37 @@ fn a_single_frame_source_range_starts_at_the_marked_last_frame() { assert!(!h.state().source_playback.clock.playing); assert_eq!(h.state().session.state.source_playhead, last); } + +#[test] +fn j_and_l_shuttle_the_source_both_ways_and_double_the_speed() { + let mut h = Harness::builder().with_step_dt(0.25).with_size(egui::vec2(1280.0, 800.0)).build_eframe(move |_cc| app(5)); + h.step(); + h.state_mut().ui.focused = PanelKind::Source; + let before = h.state().session.project.to_json(); + let program = h.state().session.playhead(); + let ctx = h.ctx.clone(); + for speed in [1.0, 2.0, 4.0, 8.0, 8.0] { + let r = menus::invoke(h.state_mut(), &ctx, "playback.forward", json!({})).unwrap(); + assert_eq!(r["speed"], speed); + } + // J while running forward turns around at normal speed, then runs backward to the first frame + h.state_mut().session.execute("source.setPlayhead", json!({"seconds":2})).unwrap(); + assert_eq!(menus::invoke(h.state_mut(), &ctx, "playback.reverse", json!({})).unwrap()["speed"], -1.0); + let at = h.state().session.state.source_playhead; + h.step(); + h.step(); + assert!(h.state().session.state.source_playhead < at, "J plays the Source backward"); + assert!(!h.state().source_playback.clock.audio_clock, "shuttle speeds play without sound"); + assert_eq!(menus::invoke(h.state_mut(), &ctx, "playback.reverse", json!({})).unwrap()["speed"], -2.0); + for _ in 0..8 { + h.step(); + } + assert!(!h.state().source_playback.clock.playing); + assert_eq!(h.state().session.state.source_playhead, Tick::ZERO); + assert_eq!(menus::invoke(h.state_mut(), &ctx, "playback.slowForward", json!({})).unwrap()["speed"], 0.25); + menus::invoke(h.state_mut(), &ctx, "playback.stop", json!({})).unwrap(); + assert!(!h.state().source_playback.clock.playing); + assert_eq!(h.state().session.playhead(), program); + assert!(!h.state().playback.playing); + assert_eq!(h.state().session.project.to_json(), before); +} diff --git a/crates/ui-egui/tests/timeline_rename_ui.rs b/crates/ui-egui/tests/timeline_rename_ui.rs new file mode 100644 index 00000000..aa1ef4c9 --- /dev/null +++ b/crates/ui-egui/tests/timeline_rename_ui.rs @@ -0,0 +1,46 @@ +//! Right-click renaming a Timeline track uses timeline.setTrack, which is undoable. + +use egui_kittest::Harness; +use filmcraft_engine::Session; +use filmcraft_ui_egui::FilmcraftApp; +use serde_json::json; + +fn step(h: &mut Harness<'static, FilmcraftApp>) { + let ctx = h.ctx.clone(); + let mut raw = std::mem::take(h.input_mut()); + eframe::App::raw_input_hook(h.state_mut(), &ctx, &mut raw); + *h.input_mut() = raw; + ctx.request_repaint(); + h.step(); +} + +fn click(h: &mut Harness<'static, FilmcraftApp>, id: &str, button: egui::PointerButton) { + let [x, y, w, height] = h.state().auto.find(id).expect("visible control").rect; + let at = egui::pos2(x + w * 0.5, y + height * 0.5); + h.input_mut().events.push(egui::Event::PointerMoved(at)); + step(h); + h.input_mut().events.push(egui::Event::PointerButton { pos: at, button, pressed: true, modifiers: Default::default() }); + step(h); + h.input_mut().events.push(egui::Event::PointerButton { pos: at, button, pressed: false, modifiers: Default::default() }); + step(h); +} + +#[test] +fn rename_from_video_track_header_is_undoable() { + let mut s = Session::default(); + s.execute("file.openDemoProject", json!({})).expect("demo project"); + let mut h = Harness::builder().with_size(egui::vec2(1600.0, 980.0)).with_step_dt(1.0 / 60.0).build_eframe(move |_cc| FilmcraftApp::new(s)); + for _ in 0..5 { + step(&mut h); + } + let original = h.state().session.active_sequence().expect("active sequence").video_tracks[0].name.clone(); + let id = h.state().session.active_sequence().expect("active sequence").video_tracks[0].id.0; + click(&mut h, "timeline.track.V1.name", egui::PointerButton::Secondary); + assert!(h.state().auto.find("timeline.track.V1.renameField").is_some(), "track rename popup should open"); + h.ctx.data_mut(|d| d.insert_temp(egui::Id::new(("timeline-track-rename", id)), "B-roll".to_string())); + step(&mut h); + click(&mut h, "timeline.track.V1.renameApply", egui::PointerButton::Primary); + assert_eq!(h.state().session.active_sequence().expect("active sequence").video_tracks[0].name, "B-roll"); + h.state_mut().session.execute("edit.undo", json!({})).expect("undo rename"); + assert_eq!(h.state().session.active_sequence().expect("active sequence").video_tracks[0].name, original); +} diff --git a/crates/ui-egui/tests/timeline_scroll_ui.rs b/crates/ui-egui/tests/timeline_scroll_ui.rs index ef23d50f..af20a815 100644 --- a/crates/ui-egui/tests/timeline_scroll_ui.rs +++ b/crates/ui-egui/tests/timeline_scroll_ui.rs @@ -263,3 +263,25 @@ fn the_scroll_bar_thumb_follows_the_pointer_and_reaches_both_ends() { let (scroll, pps, ..) = d.view(); assert!(pps < 50.0 && (scroll - at).abs() < 1e-6, "{pps} {scroll}"); } + +/// Show Video Thumbnails ▸ Continuous asks for frames across each video clip, not only its head; +/// `ui.set` switches the mode (and Off) and refuses an unknown one. +#[test] +fn continuous_video_thumbnails_ask_for_frames_across_each_clip() { + let mut d = Driver::demo(); + d.ok("ui.set", json!({"timeline": {"thumbnails": "head", "pps": 50.0, "scroll": 0.0}})); + d.view(); + d.frames(10); + let head = d.app().frames.stats().request_misses; + d.ok("ui.set", json!({"timeline": {"thumbnails": "continuous"}})); + d.frames(3); + let continuous = d.app().frames.stats().request_misses; + assert!(continuous >= head + 5, "continuous thumbnails queued {} more frames", continuous - head); + let tl = d.ok("ui.inspect", none())["ui"]["timeline"].clone(); + assert_eq!((tl["show_thumbnails"].clone(), tl["thumbnail_mode"].clone()), (json!(true), json!("Continuous"))); + d.ok("ui.set", json!({"timeline": {"thumbnails": "off"}})); + assert!(!d.app().ui.timeline.show_thumbnails); + let v = d.call("ui.set", json!({"timeline": {"thumbnails": "filmstrip"}})); + assert_eq!(v["ok"], json!(false), "{v}"); + assert!(!d.app().ui.timeline.show_thumbnails); +} diff --git a/crates/ui-egui/tests/timeline_trim_ui.rs b/crates/ui-egui/tests/timeline_trim_ui.rs index 8ea07cd2..bac77555 100644 --- a/crates/ui-egui/tests/timeline_trim_ui.rs +++ b/crates/ui-egui/tests/timeline_trim_ui.rs @@ -298,3 +298,29 @@ fn dragging_a_caption_edge_inward_trims_it() { assert_eq!(s1, s0, "the caption's start stayed: trimmed, not moved"); assert!(e1 < e0, "the caption's end moved in: {e0} → {e1}"); } + +/// Dragging a picture in the Trim Monitor must not deadlock: the shift bookkeeping used to call +/// `drag_delta()` inside `data_mut()`, and both take egui's non-reentrant context write-lock, so +/// the drag froze the whole app on its first recognised frame (the delta is read before locking +/// now). The drag commits one normal trim on release. +#[test] +fn dragging_in_the_trim_monitor_trims_without_hanging() { + let mut d = Driver::demo(); + let city = d.v1_clip(CITY); + d.exec("trim.selectEditPoint", json!({"clip": city, "edge": "out", "kind": "trim"})); + d.frames(2); + // the Program panel shows the two-up Trim Monitor once an edit point is selected + let els = d.ok("ui.elements", json!({"prefix": "trimMonitor."})); + let out = + els.as_array().expect("elements").iter().find(|e| e["id"] == "trimMonitor.outgoing").unwrap_or_else(|| panic!("trim monitor not shown: {els}")).clone(); + let r: Vec = out["rect"].as_array().expect("rect").iter().map(|v| v.as_f64().expect("number") as f32).collect(); + let (cx, cy) = (r[0] + r[2] / 2.0, r[1] + r[3] / 2.0); + // about 66 px of drag = the 6 pt threshold + 60 px of motion, at 6 px per frame; + // inward (left) trims into the clip's media, outward would clamp at the seamless cut + let (_, e0) = d.span(city); + d.drag(pos2(cx, cy), pos2(cx - 66.0, cy), 33); + let (_, e1) = d.span(city); + let frames = (e1 - e0) / d.frame(); + assert!(e1 < e0, "the Out edge moved in by {} frames", frames.abs()); + assert!((6..=14).contains(&-frames), "about 11 frames expected for 60 px at 6 px/frame, moved {}", frames.abs()); +} diff --git a/docs/agents.md b/docs/agents.md index 2e303cbb..50b4e34e 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -14,6 +14,10 @@ FilmCraft from an agent. Part 2 covers developing FilmCraft as an agent. | Headless | `filmcraft-cli mcp --demo` or `--project p.fcproj` (neither = empty project) | an in-process engine session, no window | | Bridge | `filmcraft-cli mcp --bridge 127.0.0.1:9876` | the running desktop app started with `filmcraft --control 9876` | +Headless mode also takes `--data-dir DIR` (user presets). Any other option is rejected (exit 2): +there are no folder-confinement flags such as `--automation-read-root`, so the server can read +and write wherever the user can. + **Claude Code.** The repository's `.mcp.json` registers both servers (`filmcraft` = bridge, `filmcraft-headless` = demo). Both point at `target/release/filmcraft-cli`, so build it first: @@ -244,8 +248,8 @@ media, sidecar and `ATTRIBUTION.md` entry. 1. [AGENTS.md](../AGENTS.md): absolute rules (assets, clean-room, licences). 2. [CLAUDE.md](../CLAUDE.md): working instructions and non-negotiables. -3. [ROADMAP.md](../ROADMAP.md): read the **honest assessment** and **Where we are lacking** first, then - milestones, what's done and what's running. +3. [ROADMAP.md](../ROADMAP.md) for the summary, then [gaps.md](gaps.md) (the ranked shortfalls) first, + [target-app-parity.md](target-app-parity.md) and [roadmap.md](roadmap.md) (milestones, current focus). 4. [architecture.md](architecture.md), then the README and tests of the crate you'll touch. 5. [contributing.md](contributing.md) (how to add things, gates) and [testing.md](testing.md). @@ -256,15 +260,16 @@ everything you need to contribute is in the public docs above. Ask a maintainer ## 5. Autonomous work loop 1. **Orient.** Pick the next task: the next unchecked task in the maintainer status file, or an open - ROADMAP item. Prefer the gaps in ROADMAP's *Where we are lacking* (speed, correctness on real - media, measurement, Windows/Linux) over adding more checklist items. Read the relevant + [gaps.md](gaps.md) entry. Prefer those gaps (stability, `.prproj`, camera media, measurement, + hardware, Windows/Linux) over adding more checklist items. Read the relevant architecture section and crate README. 2. **Plan tests first.** Write down the acceptance test before writing code. 3. **Implement and test.** 4. **Verify.** Run all gates (`cargo xtask ci`). For UI work, run the app with `--control`, drive it and look at the screenshots (§3). 5. **Record.** Update the crate README (behaviour decisions, test results, limitations) and - ROADMAP.md when a milestone moves. Commit as `M.: …`, then move on to the next task. + the progress docs when a milestone moves (ROADMAP.md, docs/roadmap.md, the relevant parity doc + and gaps.md, with their timestamps). Commit as `M.: …`, then move on to the next task. For parallel agents, use one git worktree and one `CARGO_TARGET_DIR` per agent, and keep each crate with one owner ([contributing.md §5](contributing.md#5-parallel-work-several-agents-or-worktrees)). @@ -295,17 +300,17 @@ A feature is done when: - [ ] every new interactive widget has an automation id; - [ ] you have driven it through the control channel and reviewed a screenshot; - [ ] all gates pass (`cargo xtask ci`), and new assets have sidecars; -- [ ] the crate README and ROADMAP.md are updated where relevant; +- [ ] the crate README and the progress docs (ROADMAP.md, docs/gaps.md, the parity docs) are updated where relevant; - [ ] it is committed with its task id. ### Reporting progress honestly -ROADMAP.md keeps two numbers: the **feature checklist** (does it exist?) and **ready for real -work** (does it hold up?). When you update either: +ROADMAP.md and [target-app-parity.md](target-app-parity.md) keep two numbers: **feature breadth** +(does it exist?) and **ready for real work** (does it hold up?). When you update either: - say how a number was obtained: *measured* (a test, a diff, a benchmark you ran) or *estimated*; - count an approximation, a stub or a partly wired setting as such, never as done; -- record known bugs and limitations in the crate README and the honest assessment, not only +- record known bugs and limitations in the crate README and [gaps.md](gaps.md), not only what works; - benchmarks state the machine, the load average and before/after on the same build; - don't raise a percentage without evidence a reviewer can rerun. diff --git a/docs/architecture.md b/docs/architecture.md index 9481641f..0c7908bb 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,5 +1,7 @@ # Architecture +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** trivial (status line, revision history and roadmap link added; content checked against the crate list) · **Target:** Adobe Premiere Pro 2026 (26.5.2) + FilmCraft is a Cargo workspace of small crates with strictly enforced layering. The engine is headless: every feature can be reached without a window, and the egui UI is one client among the CLI, the JSON control channel and the MCP server. @@ -694,7 +696,9 @@ Video encoders implement `export::VideoEncoder`. Codec crates plug in with `regi Timelines can be exchanged as EDL, FCP7 XML, FCPXML, OTIO, AAF or OMF. `file.import` detects these formats and merges the result into the project as one undoable step (AAF / OMF embedded audio is -written next to the document first), and `file.exportInterchange`, `file.exportEdl`, +written next to the document first). `file.import {replace: true}` starts from an empty project +with the same name (`file.newProject`) so a headless import→export round-trip contains only the +imported timeline. `file.exportInterchange`, `file.exportEdl`, `file.exportFcpxml`, `file.exportOtio`, `file.exportAaf` and `file.exportOmf` write them. `file.exportInterchange {format}` takes `edl`, `xml` (the default), `fcpxml`, `otio`, `aaf` or `omf`; any other value is a parameter error and nothing is written. For AAF and @@ -752,4 +756,10 @@ The layer table reserves names for crates that don't exist yet: `riff`, `mjpeg`, other OS integration stays in `apps/filmcraft`.) Until they exist, that work lives elsewhere: keyframes and effect definitions in `project`, effects and the audio mix in `render`, playback in `ui-egui`, and OS integration (cpal, rfd, -native menus) in `apps/filmcraft`. [ROADMAP.md](../ROADMAP.md) has the milestone status. +native menus) in `apps/filmcraft`. [roadmap.md](roadmap.md) has the milestone status and [gaps.md](gaps.md) what is missing. + +## Revision history + +| Date | Change | Summary | +|---|---|---| +| 2026-10-10 | trivial | Status line and revision history added (craftrules progress-docs standard); milestone link points at docs/roadmap.md | diff --git a/docs/codec-parity.md b/docs/codec-parity.md new file mode 100644 index 00000000..83416bf2 --- /dev/null +++ b/docs/codec-parity.md @@ -0,0 +1,63 @@ +# Codec parity + +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (new file: every codec Premiere Pro 26.5.2 decodes or encodes, against ours) · **Target:** Adobe Premiere Pro 2026 (26.5.2) and Media Encoder 2026 + +Video and audio codecs, decode and encode, with fidelity and how each is tested. Containers, +project and interchange files are in [file-format-parity.md](file-format-parity.md); hardware +paths per platform in [hardware-parity.md](hardware-parity.md). All our codecs are clean-room +pure Rust from public specifications ([AGENTS.md](../AGENTS.md) §2); ffmpeg is a test oracle only. + +Premiere's list comes from Adobe's published supported-formats pages and the names of the +installed bundle's `MediaIO/systempresets` and `Frameworks` entries (listings only, never their +contents). Percent is ready-for-real-work for that codec: depth, profiles, speed. + +## Video + +| Codec | Premiere | Our decode | Our encode | Fidelity and tests | % | Est. | +|---|---|---|---|---|---|---| +| H.264 / AVC | decode all common profiles incl. High 10, 4:2:2 (XAVC S-I, AVC-Intra); encode 8/10-bit | own decoder, **8-bit 4:2:0 only** (Baseline / Main / High, CABAC / CAVLC); no High 10 / 4:2:2 / 4:4:4, interlaced field / MBAFF, lossless | own encoder (High / Main / Baseline, B-frames, CBR / VBR / 2-pass); 8-bit 4:2:0 only; hardware VideoToolbox / NVENC | decode bit-exact on 37+ conformance streams, 500–600 fps 1080p; encode ffprobe-checked, byte-identical across machines | 65% | 20–35 h | +| HEVC / H.265 | decode Main, Main 10, 4:2:2 10 (XAVC HS); encode 8/10-bit, HDR | own decoder Main / Main 10 / Main Still Picture, bit-exact on 41 fixtures, ~225 fps 1080p; **no 4:2:2 / RExt** | **hardware only** (VideoToolbox 8-bit; NVENC 8-bit and Main 10 HDR on Windows); no software encoder | NVENC round trip 48.9 dB (8-bit), 71.1 dB (10-bit); ffprobe | 60% | 30–60 h | +| ProRes 422 family / 4444 / XQ | decode + encode all | own decode + encode, all six profiles, alpha | yes (export UI exposes 422 flavours; 4444 / XQ request #342) | ±1 LSB, 318 fps | 85% | 3–6 h | +| ProRes RAW | decode | none (#345) | — | — | 0% | 30–50 h (spec availability uncertain) | +| DNxHD / DNxHR | decode + encode, MXF / MOV | own decode (all SMPTE ST 2019-1 CIDs, 8/10/12-bit, 4:2:2 / 4:4:4, interlaced, alpha) and encode LB / SQ / HQ / HQX / 444 | yes, MOV + MXF | within IDCT precision of ffmpeg on 22 fixtures | 90% | 2–4 h | +| AV1 | decode (hardware on Windows); export through Media Encoder plug-ins / recent versions | own decoder Main 8/10-bit, all tools, film grain, bit-exact vs libdav1d on 41 vectors; hardware on Windows | none | slow in software (~4 fps 1080p single-thread) | 55% | 30–50 h (encoder + speed) | +| VP9 | not a Premiere import format on all versions | own decoder profiles 0–3, 8/10/12-bit, bit-exact on 50+ fixtures, frame threading | none | conformance | ahead | — | +| APV (RFC 9924) | — (Samsung's pro codec; not in Premiere) | own decode | own encode | spec tests | ahead | — | +| MPEG-2 / MPEG-1 | decode incl. 4:2:2 (IMX, XDCAM); encode DVD / Blu-ray / MXF | own decoder (4:2:2, interlaced, field pictures, IEEE 1180 IDCT) | **none** | conformance fixtures | 60% | 20–35 h (encoder) | +| Motion JPEG | decode + encode | yes | yes (MOV) | — | 90% | — | +| JPEG 2000 (MXF, DCP) | decode + encode (Kakadu) | none | none | — | 0% | 40–70 h | +| JPEG XS | decode (`libjpegxs`) | none | none | — | 0% | 20–40 h | +| DV / DVCPRO / DV100 | decode + encode | none | none | — | 0% | 15–25 h | +| MPEG-4 Part 2 | decode | none | none | — | 0% | 10–20 h | +| WMV / VC-1 | decode, encode on Windows | none | none | — | 0% | 20–35 h | +| Camera RAW: R3D (incl. R3D NE), ARRIRAW, Sony RAW / X-OCN, Canon RAW, BRAW (plug-in), Cinema DNG | decode (vendor SDKs) | none (#383) | — | — | 0% | 60–100 h for the ones with public specs; vendor-SDK-only formats need an owner decision | +| GIF (animated) | encode | decode as still | encode | — | 80% | — | +| Uncompressed / v210 / RGB | decode + encode (AVI, MOV) | partial (MOV PCM-style video not verified) | none | — | 30% | 8–15 h | + +## Audio + +| Codec | Premiere | Our decode | Our encode | Fidelity and tests | % | Est. | +|---|---|---|---|---|---|---| +| PCM (WAV, BWF, AIFF, MOV, MXF, Blu-ray / DVD LPCM) | yes | yes, BWF timecode | WAV, AIFF, MOV, MXF | sample-exact | 95% | — | +| AAC (LC, HE, LATM) | decode + encode | own decoder (LC, HE core), LATM | own LC encoder | loudness oracle vs ffmpeg | 90% | 3–5 h | +| AC-3 | decode (licensed) | own decoder (ATSC A/52) | none | conformance | 80% | — | +| **E-AC-3 (Dolby Digital Plus)** | decode | **refused** (bsid 11–16, #647) | none | — | 0% | 15–25 h | +| MP3 / MP2 | decode + MP3 encode | decode (symphonia, MPL-2.0; own MP2) | **none** | — | 60% | 10–20 h (MP3 encoder) | +| Opus | — (not a Premiere import format) | own decoder, RFC 8251 range-exact | none | conformance vectors | ahead | — | +| FLAC, ALAC, Vorbis | partial | decode (symphonia); FLAC in MP4 silent (#603) | none | — | 70% | 3–6 h | +| WMA | decode on Windows | none | none | — | 0% | 10–20 h | + +## Summary + +- **Ahead of Premiere:** VP9, Opus, APV decode / encode; bit-exact software decoders everywhere, + including Linux and FreeBSD, where Premiere does not run. +- **Behind, by user impact:** 10-bit / 4:2:2 H.264 and HEVC in software ([G3](gaps.md#g3-10-bit-and-422-camera-media-needs-hardware)), + E-AC-3, camera RAW, HEVC and AV1 software encoding, MPEG-2 / JPEG 2000 encoding for broadcast + and DCP, DV, WMV. +- **Codec dimension: ~60%, 150–250 h** ([target-app-parity.md](target-app-parity.md#by-dimension)). + +## Revision history + +| Date | Change | Summary | +|---|---|---| +| 2026-10-10 | major | Created: codec-by-codec comparison with Premiere 26.5.2 from the codec crates' READMEs, the export format registry and the installed bundle's listings | diff --git a/docs/contributing.md b/docs/contributing.md index c38791fc..245334b0 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -136,7 +136,7 @@ comes with a regression test; parsers, decoders and commands get fuzz / hostile- [ROADMAP.md](../ROADMAP.md). If your change has no task id, use the area: `docs: …`, `README: …`, `ROADMAP: …`. - Commit only green states (all gates pass). -- When a milestone lands, update its row in `ROADMAP.md` (status, what's done, estimates). +- When a milestone lands, update its row in [roadmap.md](roadmap.md) (status, what's done, estimates), the summary in `ROADMAP.md`, and the parity doc and [gaps.md](gaps.md) entry it moves, with their "Last updated" lines and revision history. - AI agents end commit messages with the attribution line their environment requires. ## 5. Parallel work (several agents or worktrees) diff --git a/docs/control-protocol.md b/docs/control-protocol.md index 9cf58cf6..28b86c30 100644 --- a/docs/control-protocol.md +++ b/docs/control-protocol.md @@ -29,7 +29,7 @@ Methods (handlers in `crates/ui-egui/src/control.rs`): | `engine.commands` / `ui.menu.list` | – | command registry / menu tree | | `ui.inspect` | – | UI state (tool, workspace, dock, timeline view, playback, fps…) | | `ui.elements` | `{prefix?}` | every on-screen interactive element: id, label, rect | -| `ui.set` | `{tool, workspace, mode, theme, focused, playbackRes, timeline:{pps,scroll,fit}, program:{…}, source:{…}, clipDialog:{param: value}, menuDialog:{param: value}, panels:{scopes, timecode, reference, events, progress}, export:{…}}` | `program` / `source` merge fields into the monitor view state (`res`, `paused_res`, `high_quality`, `display`, `zoom`, `pan`, `show_rulers`, `show_guides`, `lock_guides`, `snap`, `guides`, `compare_ref`, …; see [monitors.md](monitors.md)); `clipDialog` sets fields of the open Edit / Clip / File dialog (Paste Attributes, Make Subclip, Frame Hold Options, …); `menuDialog` those of the M3.11 dialogs (Find, Create Search Bin, Project Settings, Scene Edit Detection, Simplify Sequence, Automate to Sequence…; see `panels::menu_dialogs`); `panels` merges into the panel settings (`ui.inspect` → `ui.panels`; see `panels::panel_state`): Lumetri Scopes `{shown: ["vectorscopeYuv"|"vectorscopeHls"|"histogram"|"parade"|"waveform"], waveformType, paradeType, colorSpace, brightness, scale, clamp, targets}`, Timecode `{rows: [{source, mode, display}], showName}`, Reference Monitor `{ganged, time, display: "composite"|"scopes", scopes}`, Events `{level, selected}`, Progress `{showFinished}` `export` deep-merges into Export mode state (`preset` — applied first —, `settings` (ExportSettings JSON), `fileName`, `location`, `range`, `customStart`, `customEnd`, `openSections`, `manager` (Preset Manager: `query`, `favoritesOnly`, `selected`, `saveName`), `quickOpen`, `quickPath`, `quickPreset`); `program` / `source` merge fields into the monitor view state (`res`, `paused_res`, `high_quality`, `display`, `zoom`, `pan`, `show_rulers`, `show_guides`, `lock_guides`, `snap`, `guides`, `compare_ref`, …; see [monitors.md](monitors.md)); `clipDialog` sets fields of the open Edit / Clip / File dialog (Paste Attributes, Make Subclip, Frame Hold Options, …); `menuDialog` those of the M3.11 dialogs (Find, Create Search Bin, Project Settings, Scene Edit Detection, Simplify Sequence, Automate to Sequence…; see `panels::menu_dialogs`); `projectPanel` merges into the Project panel's view state (`bin` shown in place, `tabs` (bins in a tab or window: `{bin, floating, view, iconSize, nav}`), `activeTab`, `selectedBin`, `rename: {item|bin, text}`, `dialog` (`{"metadataDisplay": {columns, filter}}`, `{"savePresetAs": {name}}`, `{"managePresets": {selected, name}}`, `{"freeformOptions": {options}}`, `{"saveArrangement": {name, bin}}`)); `mediaBrowser` into the Media Browser's (`expanded`, `pathEdit`, `editColumns`, `treeWidth`). View settings, columns and presets are engine commands (`project.view.set`, `project.columns.set`, `project.viewPreset.*`, `mediaBrowser.settings`) | +| `ui.set` | `{tool, workspace, mode, theme, focused, playbackRes, timeline:{pps,scroll,fit}, program:{…}, source:{…}, clipDialog:{param: value}, menuDialog:{param: value}, panels:{scopes, timecode, reference, events, progress}, export:{…}, keyframeSelection:[…]}` | `keyframeSelection` replaces the Effect Controls keyframe selection (`[{clip, effect, param, mask?, time}]`, `time` in media ticks; read back as `ui.inspect` → `ui.keyframe_selection`; a click on a lane keyframe selects just that one); `program` / `source` merge fields into the monitor view state (`res`, `paused_res`, `high_quality`, `display`, `zoom`, `pan`, `show_rulers`, `show_guides`, `lock_guides`, `snap`, `guides`, `compare_ref`, …; see [monitors.md](monitors.md)); `clipDialog` sets fields of the open Edit / Clip / File dialog (Paste Attributes, Make Subclip, Frame Hold Options, …); `menuDialog` those of the M3.11 dialogs (Find, Create Search Bin, Project Settings, Scene Edit Detection, Simplify Sequence, Automate to Sequence…; see `panels::menu_dialogs`); `panels` merges into the panel settings (`ui.inspect` → `ui.panels`; see `panels::panel_state`): Lumetri Scopes `{shown: ["vectorscopeYuv"|"vectorscopeHls"|"histogram"|"parade"|"waveform"], waveformType, paradeType, colorSpace, brightness, scale, clamp, targets}`, Timecode `{rows: [{source, mode, display}], showName}`, Reference Monitor `{ganged, time, display: "composite"|"scopes", scopes}`, Events `{level, selected}`, Progress `{showFinished}` `export` deep-merges into Export mode state (`preset` — applied first —, `settings` (ExportSettings JSON), `fileName`, `location`, `range`, `customStart`, `customEnd`, `openSections`, `manager` (Preset Manager: `query`, `favoritesOnly`, `selected`, `saveName`), `quickOpen`, `quickPath`, `quickPreset`); `program` / `source` merge fields into the monitor view state (`res`, `paused_res`, `high_quality`, `display`, `zoom`, `pan`, `show_rulers`, `show_guides`, `lock_guides`, `snap`, `guides`, `compare_ref`, …; see [monitors.md](monitors.md)); `clipDialog` sets fields of the open Edit / Clip / File dialog (Paste Attributes, Make Subclip, Frame Hold Options, …); `menuDialog` those of the M3.11 dialogs (Find, Create Search Bin, Project Settings, Scene Edit Detection, Simplify Sequence, Automate to Sequence…; see `panels::menu_dialogs`); `projectPanel` merges into the Project panel's view state (`bin` shown in place, `tabs` (bins in a tab or window: `{bin, floating, view, iconSize, nav}`), `activeTab`, `selectedBin`, `rename: {item|bin, text}`, `dialog` (`{"metadataDisplay": {columns, filter}}`, `{"savePresetAs": {name}}`, `{"managePresets": {selected, name}}`, `{"freeformOptions": {options}}`, `{"saveArrangement": {name, bin}}`)); `mediaBrowser` into the Media Browser's (`expanded`, `pathEdit`, `editColumns`, `treeWidth`). View settings, columns and presets are engine commands (`project.view.set`, `project.columns.set`, `project.viewPreset.*`, `mediaBrowser.settings`) | | `ui.panel.show` / `ui.panel.close` | `{panel}` | | | `ui.click` / `ui.move` | `{id}` or `{x,y}`, `button`, `count`, `modifiers` | synthetic pointer input | | `ui.drag` | `{from, to, steps, modifiers}` | press–move–release | @@ -41,6 +41,9 @@ Methods (handlers in `crates/ui-egui/src/control.rs`): | `ui.resize`, `ui.focus`, `app.quit` | | `ui.focus` is the only request that activates the app and takes keyboard focus | | `perf.stats` | – | performance counters (also the command `perf.stats`, so `engine.execute` and MCP `command_run` reach it; headless sessions return the engine part). `decode`: GOP-cache requests / hits / `cacheHitRate`, decoder seeks, samples decoded and skipped while catching up, `draftFrames` (decoded in draft mode), `h264Threads` (frame-threading workers per H.264 decoder), `liveDecoders` (sources holding a decoder now), `cacheMB` / `cacheBudgetMB` (decoded frames all sources hold, and the shared budget), `planePoolMB` / `planesReused` (idle buffers of the frame pool, the planes of evicted frames and the compositor's float images, which a standalone export frees when it ends; and buffers taken from it instead of allocated), decoder ms (total and per sample), `framesDecoded`, `hardware` (Settings ▸ Playback ▸ Hardware decoding: `enabled`, pictures from hardware decoders `frames` vs `softwareFrames`, `sessions`, `declined` streams, mid-stream `fallbacks`); `playback`: playing, shown / dropped frames and `dropRate` of the current or last play, resolution, `draftDecode` (Settings ▸ Playback ▸ Draft decoding), `audio` (desktop: sound is mixed 200 ms ahead of the device; device `callbacks`, `underruns` and `missingFrames` played as silence because the mixer fell behind, `minLeadMs` the least sound left buffered); `frames`: frame-worker jobs, cancelled, `requestHitRate`, `decodeMs` / `renderMs` (mean, p50, p95 of the last 256 jobs), queue, render-cost estimate, cache use; `monitor`: the `frame` and `revision` of the picture the Program monitor shows against `playheadFrame` and `projectRevision` (how far a drag or a scrub is ahead of the picture); `ui`: fps and frame ms; `process`: CPU seconds; `media`, `jobs`. Counters are cumulative: diff two readings to measure an interval ([performance.md](performance.md)) | +**Timeline thumbnails.** `ui.set {timeline: {thumbnails}}` sets the Timeline wrench menu's Show Video +Thumbnails: `off`, `head`, `headAndTail` or `continuous`. + **Selection or explicit targets.** A command that acts on the selection is "not available right now" when nothing is selected. When `params` name the targets under a key the command documents (`clips` / `clip`: timeline clips of the active sequence, `items` / `item`: project items), enablement is diff --git a/docs/file-format-parity.md b/docs/file-format-parity.md new file mode 100644 index 00000000..b19262c8 --- /dev/null +++ b/docs/file-format-parity.md @@ -0,0 +1,94 @@ +# File format parity + +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (new file: every file type Premiere Pro 26.5.2 reads or writes, against ours) · **Target:** Adobe Premiere Pro 2026 (26.5.2) and Media Encoder 2026 + +Containers, stills, audio files, project and interchange files, captions and presets. Codecs inside +the containers are in [codec-parity.md](codec-parity.md). Our import list is +`filmcraft_media::{VIDEO,AUDIO,STILL}_EXTENSIONS` plus the openers in `filmcraft_codecs::openers()`; +our export list is `filmcraft_export::Format`. Premiere's list is from Adobe's supported-formats +documentation and the exporter folders in the installed bundle's `MediaIO/systempresets` (names +only). "Tested" names the evidence. + +## Project and interchange + +| Format | Premiere | Read | Write | Tested | % | +|---|---|---|---|---|---| +| **Premiere project `.prproj`** | native | **no** | **no** | — | **0%** ([G2](gaps.md#g2-premieres-prproj-cannot-be-opened)) | +| Our project `.fcproj` | — | yes, schema migrations | yes, atomic, auto-save, recovery journal | migration tests | — | +| FCP7 XML (`xmeml`) | read + write | yes | yes | round-trip tests; Premiere-exported XML slow (#461) | 75% | +| FCPXML | read (via XtoCC) | yes | yes | round-trip tests | 80% | +| AAF (Edit Protocol) | read + write | yes | yes (embedded / linked / consolidated) | own CFB; **never opened in Media Composer / Pro Tools** | 55% | +| OMF 2.0 | write | no | yes | never opened in Pro Tools | 50% | +| EDL (CMX 3600) | read + write | yes | yes | tests | 85% | +| OpenTimelineIO | read + write (26.0) | yes | yes | tests | 80% | +| Avid Log Exchange (ALE) | write | no | yes | — | 70% | +| Marker CSV / text | write | — | yes (#454) | — | 80% | +| Selection as project | Premiere project | — | `.fcproj` only | — | 50% | +| Templates (`.prproj` templates) | yes | — | our templates | — | 70% | +| Motion Graphics templates `.mogrt` | read + write | **refused by design** (clean-room, AGENTS.md §1); our `.fcgt` instead | `.fcgt` | — | n/a | +| Effect presets `.prfpset` | read + write | no | our JSON presets | — | 40% | +| Keyboard shortcuts `.kys` | read + write | no | our presets incl. a Premiere layout | — | 60% | +| Export presets `.epr` | read + write | no | our presets (import / export) | — | 50% | +| LUTs `.cube`, `.3dl` | read | yes (1D / 3D / shaper) | — | tetrahedral CPU = WGSL | 95% | + +## Video containers + +| Container | Premiere | Read | Write | Tested | % | +|---|---|---|---|---|---| +| MP4 / M4V / 3GP | read + write | yes | yes (H.264, HEVC + AAC) | ffprobe, round trips | 90% | +| QuickTime MOV | read + write | yes | yes (H.264, HEVC, ProRes, DNxHR, APV, MJPEG, PCM) | ffprobe | 90% | +| MXF OP1a / OP-Atom | read + write (XDCAM, AVC-Intra, P2, AS-10 / AS-11, JPEG 2000, DNx) | yes (AVC, DNx, ProRes, MPEG-2, PCM / AES3, timecode) | OP1a (DNxHR, ProRes, H.264), OP-Atom | index-table seeking tests | 70% | +| MPEG TS / PS (AVCHD `.mts`, `.m2ts`, `.ts`, `.mpg`, `.vob`, `.mod`) | read; write MPEG-2 / Blu-ray / DVD | yes | **no** | fixtures | 65% | +| Matroska / WebM | read (limited) | yes (H.264, HEVC, VP9, AV1, ProRes, MJPEG + AAC, Opus, FLAC, MP3, Vorbis, PCM); WebM black report (#432), MKV multi-audio (#601) | no | fixtures | 75% | +| **AVI** | read + write | **no** (extension listed, no demuxer, #598) | no | — | 0% | +| WMV / ASF | read; write on Windows | no | no | — | 0% | +| FLV / F4V | read + write | no | no | — | 0% | +| DCP | write (Wraptor) | no | no | — | 0% | +| APV raw / Y4M | — | yes | APV in MOV | — | ahead | +| Ogg (Opus / Vorbis) | — | yes | no | granule-seek tests | ahead | +| Camera folder structures (XDCAM, P2, AVCHD, Canon XF, RED) | Media Browser reads them | AVCHD files yes; spanned clips and metadata no | — | — | 30% | + +## Stills and image sequences + +| Format | Premiere | Read | Write (frame / sequence) | % | +|---|---|---|---|---| +| PNG, JPEG, TIFF, BMP, GIF | read + write | yes | PNG / TIFF / BMP sequences, GIF; **no JPEG sequence** | 80% | +| WebP | — | yes | no | ahead | +| **PSD** (layers as sequence) | read (layered) | no | — | 0% | +| **OpenEXR** | read + write | no | no | 0% | +| **DPX** | read + write | no | no | 0% | +| Targa | read + write | no | no | 0% | +| HEIF / HEIC | read | no | — | 0% | +| Radiance HDR, AI / EPS, ICO, PTL | read | no | — | 0% | + +## Audio files + +| Format | Premiere | Read | Write | % | +|---|---|---|---|---| +| WAV / BWF (timecode, iXML partial) | read + write | yes | yes | 90% | +| AIFF / AIFC | read + write | yes | yes | 95% | +| MP3 | read + write | read | **no** | 60% | +| AAC / M4A | read + write | yes | inside MP4 / MOV only, **no AAC-only file** | 70% | +| FLAC, Ogg, Opus | partial | yes | no | ahead on read | +| WMA | read (Windows) | no | no | 0% | + +## Captions and subtitles + +| Format | Premiere | Read | Write | Tested | % | +|---|---|---|---|---|---| +| SRT, WebVTT, SCC | read + write | yes | yes | frame-exact, property-tested | 95% | +| MCC (608 + 708), EBU STL, TTML / IMSC1, DFXP | read + write | yes | yes | frame-exact | 90% | +| 608 / 708 embedded in the video stream | write (MXF, H.264 SEI) | no | **no** | — | 0% | +| Burn-in | yes | — | yes | golden tests | 95% | + +## Summary + +**File formats dimension: ~60%, 140–240 h** (codecs counted separately). The beta blocker is +`.prproj`. After it, by user impact: AVI and WMV import, EXR / DPX / PSD stills, AAF / OMF +validation in Avid and Pro Tools, DVD / Blu-ray / DCP / broadcast MXF delivery. + +## Revision history + +| Date | Change | Summary | +|---|---|---| +| 2026-10-10 | major | Created: format-by-format comparison with Premiere 26.5.2 from the import / export registries, crate READMEs, the installed bundle's exporter listings and open issues | diff --git a/docs/gaps.md b/docs/gaps.md new file mode 100644 index 00000000..3532ae7b --- /dev/null +++ b/docs/gaps.md @@ -0,0 +1,240 @@ +# Where FilmCraft falls short of Premiere Pro + +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (new file: ROADMAP's "Where we are lacking" re-measured against Premiere Pro 26.5.2 and the open issue list) · **Target:** Adobe Premiere Pro 2026 (26.5.2) + +Every known shortfall, one entry each, ranked by what most stops an editor using FilmCraft for real +work. This is the work list: agents should prefer it over adding checklist items. Numbers are in +[target-app-parity.md](target-app-parity.md); estimates are Opus 5.5 agent-hours, calibrated there. +When you close or shrink a gap, update its entry, its parity doc and the revision history. + +Summary by kind: **feature gaps** G1, G2, G12, G13, G18, G19; **UI / UX** G8; **file formats** +G2, G10, G13, G15; **codecs** G3, G10; **hardware** G5; **localization** G14; **stability and +platforms** G4, G9, G17; **performance** G16; **ecosystem** G11; **measurement** G1, G6, G7; +**web** G20. + +| # | Gap | Kind | Blocks beta | Estimate | +|---|---|---|---|---| +| G1 | Nothing compares our output with Premiere's | measurement | no, but every ready number depends on it | 40–70 h | +| G2 | Premiere's `.prproj` cannot be opened | file format | **yes** | 60–120 h | +| G3 | 10-bit and 4:2:2 camera media needs hardware | codec | yes | 30–50 h | +| G4 | Crashes at start-up, and no test CI on pull requests | stability | yes | 40–80 h | +| G5 | Hardware acceleration is partial | hardware | partly | 120–200 h | +| G6 | No real-world media corpus or workflow acceptance tests | measurement | yes | 40–70 h | +| G7 | No measured parity tool in CI | measurement | no | 10–20 h | +| G8 | UI fidelity: monitor handles, Effect Controls, docking | UI / UX | partly | 120–200 h | +| G9 | Windows and Linux at run time | platforms | partly | 50–90 h | +| G10 | Container and codec gaps on import | file format, codec | partly | 120–200 h | +| G11 | No plugin hosting | ecosystem | no | 200–400 h | +| G12 | AI features | AI | no | 150–300 h | +| G13 | Delivery formats | file format | no | 60–110 h | +| G14 | Localization: 8 of the 12 key languages missing | localization | no | 150–230 h | +| G15 | Interchange never validated in Avid / Pro Tools; slow XML import | file format | partly | 20–40 h | +| G16 | Performance reports on 0.5.0 | performance | partly | 30–60 h | +| G17 | Audio device and playback reliability | stability | partly | 20–40 h | +| G18 | Effects depth: approximations and missing behaviour | feature | no | 60–110 h | +| G19 | Speech to text off in release builds | feature, AI | no | 5–10 h | +| G20 | The web app shell | platforms | no | 40–80 h | + +## G1. Nothing compares our output with Premiere's + +- **Missing:** a fidelity harness. Every effect, transition, Lumetri control, blend mode, keyframe + interpolation and audio effect counts once it exists; nothing renders the same project in + Premiere and FilmCraft and compares frames or samples. Effects described as approximations + count the same as exact ones. +- **Evidence:** catalogue tests check names and folders only. EffectCraft's live After Effects + comparison (its gaps.md G1) found real bugs in features marked done on its first run. +- **Impact:** "ready for real work" stays an estimate; colour and effect differences surface only + when a user notices. +- **Done when:** a corpus of original test projects (our own media), opened in Premiere through + FCP7 XML / OTIO, rendered by both, with per-effect PSNR / ΔE and audio-sample scores reproducible + from one command; Premiere output stays local in `plan/premiere/` (gitignored). Owner runs + Premiere (installed). **40–70 h.** Doc: [target-app-parity.md](target-app-parity.md). + +## G2. Premiere's `.prproj` cannot be opened + +- **Missing:** import of Premiere project files. `grep -ri prproj crates apps` finds nothing. + Today an editor must export FCP7 XML, AAF or OTIO from Premiere first, losing bins, effects and + most metadata; there is no "Selection as Premiere Project" export either. +- **Impact:** the beta bar requires opening the target's main format; every switching editor's + existing work is behind this. +- **Approach:** `.prproj` is gzip-compressed XML. Clean-room from projects we author ourselves in + Premiere (the owner has `plan/premiere/fixtures/filmcraft_ref.prproj`): sequences, tracks, clips, + in/out, speed, transitions, markers, bins, labels, then the common effect parameters (Motion, + Opacity, Lumetri basic, Volume). Report what was not imported. Export later. **60–120 h.** + Doc: [file-format-parity.md](file-format-parity.md). + +## G3. 10-bit and 4:2:2 camera media needs hardware + +- **Missing:** our H.264 decoder is 8-bit 4:2:0 only (`crates/h264/README.md`: no High 10, High + 4:2:2, 4:4:4, lossless, interlaced field / MBAFF); HEVC has no 4:2:2 / RExt. Most mirrorless and + cinema cameras record exactly these (Sony XAVC S-I / XAVC HS 4:2:2 10-bit, Panasonic, Canon + XF-AVC, Fujifilm). +- **Evidence:** #626 (Panasonic Lumix 4K 10-bit 4:2:2 MOV). On macOS VideoToolbox decodes 10-bit + 4:2:2 H.264 / HEVC on chips that support it; Windows Media Foundation and Linux decline them, and + there is no software fallback, so the clip fails. Premiere decodes them everywhere (and in + hardware on Apple silicon, Intel and NVIDIA Blackwell). +- **Impact:** camera originals from most current cameras do not open on Windows or Linux. +- **Estimate:** H.264 High 10 + High 4:2:2 (chroma geometry, deblocking, transforms) 15–25 h; + HEVC Main 4:2:2 10 (RExt subset) 15–25 h. Doc: [codec-parity.md](codec-parity.md). + +## G4. Crashes at start-up, and no test CI on pull requests + +- **Missing:** a launch that never fails, and CI that runs the gates. +- **Evidence:** open issues: macOS 12 Monterey aborts on a missing VideoToolbox symbol (#468, + #661, #655), Intel UHD graphics (`igvk64.dll`, #512), Windows 11 does not run or fails after setup + (#380, #582, #413, #477), crash on new project + import (#439), "it just keeps crashing" (#687). + `.github/workflows/` holds release, packaging lint, FreeBSD and Windows ARM64 builds; **none runs + `cargo test` or `cargo xtask ci` on a pull request**, so 243 PRs merged since 10-05 were gated by + contributors' local runs (main needed "green again after the batch merges" on 10-10). +- **Impact:** a user who cannot start the app gets zero value; regressions land unnoticed. +- **Done when:** weak-linked or runtime-loaded OS symbols on old macOS, a software / safe-mode + start when the GPU adapter fails, a PR workflow running `cargo xtask ci` on macOS, Windows and + Linux, and a crash-report triage pass. **40–80 h.** + +## G5. Hardware acceleration is partial + +- **Have:** hardware decode on macOS (VideoToolbox H.264 / HEVC), Windows (Media Foundation / + D3D11: H.264, HEVC, VP9, AV1) and Linux (VA-API and NVDEC: H.264, HEVC); hardware encode + VideoToolbox H.264 / HEVC 8-bit, NVENC H.264 (Windows, Linux) and HEVC incl. Main 10 HDR + (Windows); GPU compositing, blend modes, 34 effect ids incl. Lumetri basic / creative / vignette, + GPU export rendering (#421). +- **Missing:** zero-copy decoded frames into wgpu (every frame is read back to the CPU and + re-uploaded); VideoToolbox Main 10 / HDR HEVC encode and B-frames; ProRes hardware decode / + encode on Apple silicon; Intel Quick Sync and AMD AMF encoders (and their 4:2:2 decode); + VA-API VP9 / AV1 and VA-API encode; HEVC encode on Linux; AV1 hardware encode; ~48 of Premiere's + ~82 accelerated effects (keys, masks, Lumetri curves / wheels / HSL, distortions, transitions); + external video I/O (DeckLink / AJA, Mercury Transmit), control surfaces (EUCON / Mackie); + a CPU fallback adapter renders everything on the CPU (#632). +- **Estimate:** 120–200 h (hardware we don't own needs a human). Doc: + [hardware-parity.md](hardware-parity.md). + +## G6. No real-world media corpus or workflow acceptance tests + +- **Missing:** a pinned corpus of phone, camera and screen recordings (variable frame rate, + damaged files, long GOP, odd containers, multi-track audio), each with an import → edit → export + round trip checked against an ffprobe / ffmpeg oracle; scripted MCP sessions that do real jobs + end to end. +- **Evidence:** export stopped producing frames with iPhone HEVC clips on Linux (#480); WebM shows + black (#432); only the first audio track imported (#397, #416, #601; multi-stream support landed + in #361, reporters not yet confirmed); bitrate target produced 6.1 Mb/s for 40 (#371). +- **Done when:** corpus fetched pinned and sha256-verified (as PhotoCraft's `cargo xtask corpus`, + craftrules `standards/test-corpora.md`), runs nightly, ≥ 50 files. **40–70 h.** + +## G7. No measured parity tool in CI + +- **Missing:** `cargo xtask parity` (menus, commands, effects, transitions, panels, preferences, + formats against `plan/premiere/` snapshots), presence and fidelity reported separately (exact, + approximate, stub), generating `docs/parity-checklist.md`. +- **Evidence:** the 92% menu figure in this pass came from an ad hoc label-matching script. +- **Estimate:** 10–20 h (the menu snapshot is local-only, so the tool needs a committed, Adobe-free + list of menu paths or runs only where `plan/` exists). + +## G8. UI fidelity: monitor handles, Effect Controls, docking + +- **Missing:** on-screen transform handles for Motion / Transform in the Program monitor (#639); + Effect Controls keyframe zoom bar, draggable divider, row lines (#640–#645); panel docking + rearrangement (#493); clicking empty space to deselect (#683); filmstrip thumbnails (#614); + marquee selection in the Project panel (#578); Freeform bins and middle-mouse pan (#579); + Program monitor Button Editor (+) (#431); keyframe curves for Position (#448); transition + editing controls (#577); track renaming (#654); window controls on macOS (#637); UI scale on + X11 (#457). +- **Estimate:** 120–200 h. Doc: [ui-parity.md](ui-parity.md). + +## G9. Windows and Linux at run time + +- **Missing:** routine runtime testing off macOS. Linux playback stutter (#617), Wayland title + bar (#433), case-sensitive extensions (#593), "Report an issue" link (#642), Windows bins cannot + be left (#587), Sequence Settings won't open on Windows (#586, #351), old hardware (#491). +- **Estimate:** 50–90 h (overlaps G4's CI work). + +## G10. Container and codec gaps on import + +- **Missing:** E-AC-3 (#647; our AC-3 decoder refuses bsid 11–16), AVI (#598; listed as an + extension but no demuxer), WMV / ASF, DV / DVCPRO / DV100, MPEG-4 Part 2, FLAC in MP4 (#603), + camera RAW (R3D incl. R3D NE, ARRIRAW, Sony RAW / X-OCN, Canon RAW, BRAW, ProRes RAW; #345, + #383), JPEG 2000 / JPEG XS MXF, Cinema DNG; stills: PSD, EXR, DPX, Targa, HEIF / HEIC, Radiance + HDR, AI / EPS; transparent video (#402); Sony start timecode in `rtmd` (#460); stereo 3D flag + (#500); MKV multi-audio (#601). +- **Estimate:** 120–200 h (camera RAW alone 60–100 h, and some SDKs have licences we cannot use: + clean-room from public specs only where specs exist). Docs: [codec-parity.md](codec-parity.md), + [file-format-parity.md](file-format-parity.md). + +## G11. No plugin hosting + +- **Missing:** VST3 / Audio Units, OpenFX, Premiere / After Effects plug-in APIs, UXP panels + (#325). +- **Impact:** many professional editors depend on third-party effects and audio plug-ins. +- **Needs:** an owner decision (FFI under the `unsafe` rule, AGENTS.md §0.3). **200–400 h.** + +## G12. AI features + +- **Missing:** Object Mask (Premiere 26.0, #406, #694), Generative Extend (needs a large video + model; not reachable locally), media-intelligence search, caption translation, model-based + Enhance Speech, Auto Reframe beyond heuristics, auto colour, Auto-Tag Audio Types. +- **Have:** local Whisper transcription (off in releases, G19), Kokoro-82M text-to-speech + narration (#278), scene edit detection, colour match. +- **Needs:** openly licensed models we can ship. **150–300 h.** + +## G13. Delivery formats + +- **Missing:** AV1 export (software or hardware), MPEG-2 / DVD / Blu-ray, DCP, JPEG 2000 MXF, + AS-10 / AS-11, XDCAM HD MXF, P2, EXR / DPX / Targa / JPEG sequences, MP3 and AAC-only audio, + ProRes 4444 / XQ in the export UI (#342; the encoder has the profiles), interlaced encoding, + smart render, HEVC software encoder (HEVC exists only where a hardware encoder does). +- **Estimate:** 60–110 h. Doc: [file-format-parity.md](file-format-parity.md). + +## G14. Localization: 8 of the 12 key languages missing + +- **Have:** Spanish, Japanese and Simplified Chinese catalogs cover ~99.6–99.9% of the ~3,500 + cataloged strings; Portuguese (Brazil) and Ukrainian ~10% (menus). +- **Missing:** Hindi, Arabic (no RTL interface layout; Arabic titles reported broken, #395), + French (#498), Indonesian, German (#490), Korean (#508; needs Hangul fonts in craft-fonts), + Vietnamese, Portuguese beyond menus; Premiere also ships Italian and Russian. Engine errors, + CLI and MCP stay English. No language has a recorded native-speaker review. +- **Estimate:** 150–230 h. Doc: [localization-parity.md](localization-parity.md). + +## G15. Interchange never validated in Avid / Pro Tools; slow XML import + +- **Missing:** AAF and OMF exports have never been opened in Media Composer or Pro Tools; FCP7 XML + / OTIO import of a Premiere sequence took 97 s / 66 s for three media files (#461); `file.import` + merges into existing project content in headless round trips (#592). +- **Estimate:** 20–40 h (needs a human with Avid / Pro Tools, both installed on the owner's Mac). + +## G16. Performance reports on 0.5.0 + +- **Missing:** steady UI under load. Constant lag after 0.4 → 0.5 (#525), ~100% CPU (#523), + "Render Effects In to Out" (#424). AV1 software decode is ~4 fps at 1080p single-threaded; 8K + is not real time. +- **Estimate:** 30–60 h. Doc: [performance.md](performance.md). + +## G17. Audio device and playback reliability + +- **Evidence:** Asus Xonar DX output broken (#605), no audio (#602), audio cuts out with + overlapping clips (#336), buggy audio (#410), Linux stutter (#617). No ASIO; cpal only. +- **Estimate:** 20–40 h. + +## G18. Effects depth: approximations and missing behaviour + +- **Missing:** Warp Stabilizer is 2-D (Premiere's is 3-D subspace), Morph Cut approximate, optical + flow renders as frame blending, Transform shutter angle adds no motion blur (#415), Ultra Key + Setting scales only two factors (#458), Lumetri HDR-aware maths, multicam paging beyond 16 + angles, redesigned 26.0 mask tools (rounded corners, constrained lines). +- **Estimate:** 60–110 h. + +## G19. Speech to text off in release builds + +- **Missing:** the `whisper` feature is off by default and in releases, so Transcribe Sequence, + text-based editing and auto captions need a source build. +- **Estimate:** 5–10 h (model download UX, licence check). + +## G20. The web app shell + +- **Missing:** file access, WebCodecs decode, audio in the browser build beyond what exists; + image-sequence export downloads drop frames in Chrome (#378). +- **Estimate:** 40–80 h. Doc: [web.md](web.md). + +## Revision history + +| Date | Change | Summary | +|---|---|---| +| 2026-10-10 | major | Created from ROADMAP.md's "Where we are lacking" (2026-10-05); re-ranked against Premiere 26.5.2 and the 188 open issues; added `.prproj`, 10-bit / 4:2:2 camera decode, launch crashes and missing PR CI, localization, delivery formats | diff --git a/docs/graphics.md b/docs/graphics.md index 433d491b..fd04667d 100644 --- a/docs/graphics.md +++ b/docs/graphics.md @@ -116,7 +116,7 @@ duration (a track is added if needed). | Tool | On the Program monitor | |---|---| | Type (T) | Click empty picture: new point-text layer with a caret. Drag on empty picture: new paragraph-text layer with that box. Click a text layer: caret there. The box of the text being typed into is red. Type; ←/→ (⌥ word, ⌘ line), ↑/↓, Home/End, Shift to select, ⌘A, ⌘C/⌘X/⌘V, Return = new line, Backspace/Delete, Esc = stop editing. Drag inside the edited text to select. | -| Selection (V) | Click a layer to select it (box with handles and anchor point; only selected layers have a box, and a layer whose visibility is off has none and cannot be clicked); drag to move; drag the anchor point to move it alone; drag a handle to scale point text about its anchor, resize a paragraph-text box, or stretch a shape away from its opposite side (see *Point text and paragraph text*); double-click a text layer to edit it. | +| Selection (V) | Click a layer to select it (box with handles and anchor point; only selected layers have a box, and a layer whose visibility is off has none and cannot be clicked); drag to move; drag the anchor point to move it alone; drag a handle to scale point text about its anchor, resize a paragraph-text box (see *Point text and paragraph text*), or change a shape's Size: the dragged side or corner follows the pointer while the opposite one and the anchor point stay, Scale is untouched, and Shift on a corner keeps the proportions (a path, which has no Size, stretches instead); double-click a text layer to edit it. | | Rectangle / Ellipse | Drag to draw a shape layer. | | Pen (P) | Click to place points; click the first point (or Return) to close the path; Esc cancels. | diff --git a/docs/hardware-parity.md b/docs/hardware-parity.md new file mode 100644 index 00000000..d39f4b9f --- /dev/null +++ b/docs/hardware-parity.md @@ -0,0 +1,87 @@ +# Hardware parity + +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (new file: hardware decode / encode per platform and codec, GPU effects, devices; re-measured after NVDEC #549 and GPU export #421) · **Target:** Adobe Premiere Pro 2026 (26.5.2) + +Each hardware feature Premiere uses, per platform, against ours. Implementation detail lives in +[`crates/platform/README.md`](../crates/platform/README.md) (decode / encode backends, the one +crate allowed `unsafe`, [ADR 0001](adr/0001-platform-ffi.md)), the GPU compositor in +[architecture.md](architecture.md), and timings in [performance.md](performance.md). +Hardware paths are always optional: our software codecs are the tested fallback, and a hardware +decoder that fails mid-stream hands over to software (`HybridDecoder`). + +Premiere's side is from Adobe's published GPU and hardware-acceleration documentation and +release notes (Premiere does not run on Linux). + +## Hardware decode + +| Codec / profile | Premiere macOS | **Ours macOS** (VideoToolbox) | Premiere Windows | **Ours Windows** (Media Foundation / D3D11) | **Ours Linux** (NVDEC, VA-API) | +|---|---|---|---|---|---| +| H.264 8-bit 4:2:0 | yes | **yes** | Intel, NVIDIA, AMD | **yes** (Baseline / Main / High) | **yes** (NVDEC; VA-API Intel / AMD / Mesa) | +| H.264 10-bit / 4:2:2 | Apple silicon | **yes** where the chip decodes it | Intel; NVIDIA Blackwell (MXF too) | no (declined; no software fallback, [G3](gaps.md#g3-10-bit-and-422-camera-media-needs-hardware)) | no | +| HEVC Main / Main 10 4:2:0 | yes | **yes** | yes | **yes** | **yes** (NVDEC 8/10; VA-API Main / Main 10) | +| HEVC 4:2:2 10-bit | Apple silicon | **yes** where supported | Intel; NVIDIA Blackwell | no | no | +| VP9 profile 0 / 2 | — | software | — | **yes** | software (VA-API VP9 not yet) | +| AV1 main 8 / 10 | — | software | yes | **yes** | software | +| ProRes | Apple silicon (M1 Pro / Max and later) | software (fast: 318 fps) | software | software | software | +| Field-coded H.264 | yes | no | yes | no | no | + +Measured (`cargo xtask bench --hw off|auto`): M4 Pro, CPU per 2160p frame H.264 119 → 4.2 ms, +HEVC 86 → 3.7 ms, 4K plays with no dropped frames at Full; Intel Iris Xe VA-API H.264 2160p +417 → 11.4 ms, HEVC Main 10 2160p 212 → 18.4 ms. Every decoded picture is read back to the CPU +and re-uploaded to wgpu (**no zero-copy** yet), which Premiere avoids. + +## Hardware encode + +| Codec | Premiere macOS | **Ours macOS** | Premiere Windows | **Ours Windows** | **Ours Linux** | +|---|---|---|---|---|---| +| H.264 8-bit | yes | **yes** (VideoToolbox, opt-in, no B-frames) | Intel QSV, NVENC, AMD | **NVENC only** (opt-in) | **NVENC only** | +| H.264 10-bit / 4:2:2 | yes (Apple silicon) | no | Intel | no | no | +| HEVC Main 8-bit | yes | **yes** (hardware-only format) | QSV, NVENC, AMD | **NVENC only** | no | +| HEVC Main 10 / HDR (PQ, HLG) | yes | **no** (tone-mapped to 8-bit SDR) | yes | **NVENC yes** (HDR10 SEI, `mdcv` / `clli`) | no | +| ProRes | Apple silicon | software only | software | software | software | +| AV1 | — | no | via plug-ins / newer GPUs | no | no | + +Measured: VideoToolbox H.264 1080p25 encode 200 fps on M1; a 9:29 timeline exported in 311 s +against 793 s with our software encoder (SSIM 0.990). NVENC HEVC round trip 48.9 dB luma PSNR +(8-bit), 71.1 dB (10-bit) on an RTX 5060. + +## GPU rendering + +| Feature | Premiere (Mercury Playback Engine: Metal, CUDA, OpenCL) | Ours (wgpu: Metal, D3D12, Vulkan) | % | +|---|---|---|---| +| Compositing, blend modes, transforms | GPU | **GPU** (#32), CPU compositor is the oracle | 90% | +| Accelerated video effects | ~82 of ~92 effects marked Accelerated (reference list `plan/premiere/10-effects-list.md`) | **34 effect ids** (`render::gpufx::GPU_EFFECTS`): colour, levels, blurs, sharpen, crop, transform, flips, vignette, video limiter, ASC CDL, channel mix, Lumetri basic / creative / vignette | ~40% | +| Lumetri curves, wheels, HSL secondary | GPU | CPU | 0% | +| Keys (Ultra Key), masks, distortions, transitions | GPU | CPU (masks have WGSL parity tests, not in the live path) | 10% | +| Export rendering on the GPU | yes | **yes** (#421; Export ▸ GPU Rendering, Auto): 2.4× (1080p) to 3.2× (4K) faster on effects-heavy timelines | 80% | +| Thumbnails on the GPU (26.0) | yes | CPU | 0% | +| Software adapter fallback | warns | silently renders everything on the CPU (#632) | — | +| Intel integrated graphics | supported | start-up crash on Intel UHD (#512) | — | + +## Devices and I/O + +| Hardware | Premiere | Ours | % | +|---|---|---|---| +| Audio interfaces | CoreAudio, ASIO, WASAPI; multichannel mapping | cpal (CoreAudio, WASAPI, ALSA); no ASIO; device reports #605, #602 | 60% | +| Voice-over microphone | yes | yes (punch-in); macOS privacy declaration missing (#659) | 80% | +| External video monitoring (Mercury Transmit, Blackmagic DeckLink, AJA) | yes | no | 0% | +| Second computer monitor full-screen playback | yes | full screen inside the app window; no separate-display output or floating windows on other screens (#466) | 40% | +| HDR / EDR display | macOS EDR, Windows HDR | no (SDR display of HDR sequences) | 0% | +| Control surfaces (EUCON, Mackie HUI / MCU) | yes | no | 0% | +| LTC timecode input | yes | no | 0% | +| Tape / deck control | legacy | no | n/a | + +## Summary + +**Hardware dimension: ~45%, 120–200 h.** Ahead of Premiere: hardware decode on Linux (VA-API, +NVDEC) and FreeBSD builds. Behind, by user impact: zero-copy upload; GPU effects beyond 34 +(Lumetri curves / wheels / HSL, keys, masks, transitions); Quick Sync and AMD encoders on Windows; +VideoToolbox Main 10 HDR encode; 10-bit / 4:2:2 decode off macOS; HDR display; external video +I/O and control surfaces (need hardware we don't own). Calibration: one decode backend ~3–5 h, +one encoder ~4–8 h ([target-app-parity.md](target-app-parity.md#calibration)). + +## Revision history + +| Date | Change | Summary | +|---|---|---| +| 2026-10-10 | major | Created from ROADMAP.md's performance-and-hardware row and `crates/platform/README.md`; added Linux NVDEC (#549), GPU export and Lumetri on the GPU (#421), Premiere's per-platform hardware list, devices and I/O | diff --git a/docs/localization-parity.md b/docs/localization-parity.md new file mode 100644 index 00000000..a4a18386 --- /dev/null +++ b/docs/localization-parity.md @@ -0,0 +1,65 @@ +# Localization parity + +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (new file: per-language status measured from the catalogs; replaces ROADMAP.md's "Interface localization" section) · **Target:** Adobe Premiere Pro 2026 (26.5.2) + +Per-language status of FilmCraft's interface. Premiere 26.5.2 ships 10 interface languages (the +installed bundle's `.lproj` folders): German, English, Spanish, French, Italian, Japanese, Korean, +Portuguese (Brazil), Russian, Simplified Chinese. + +## How it works + +Edit ▸ Language (also Settings ▸ General, `general.interfaceLanguage`) switches the interface and +persists across restarts; System Language (the default) follows the OS's or browser's preferred +languages and falls back to English. Commands `app.language.` +are reachable over the control channel. One catalog per language, +`crates/ui-egui/src/i18n/.tsv` (format in the header of `es.tsv`); UI code wraps strings in +`tl!("…")` / `tlf!("…{name}", name)`, and registry names (effects and their parameters, settings, +commands, panels, workspaces, presets) are translated where they are drawn. Searches match the +displayed labels as well as English sources. Tests fail when a `tl!` literal, a menu label or a +registry name has no Spanish entry; `zh-cn.tsv` must also cover every `tl!` literal and menu +label; `ja.tsv` has the same entries but is not yet required by tests. Engine error messages, the +control channel, the CLI and MCP stay English. + +Fonts come from [craft-fonts](https://github.com/storytold/craft-fonts) (`CRAFT_FONTS_DIR`, all +official releases) or an installed system face; without one, switching to Japanese or Chinese is +refused with a message. Simplified Chinese is picked for `zh`, `zh-CN`, `zh-SG`, `zh-Hans`, not +Traditional locales. Ukrainian uses the bundled Cyrillic interface fonts. Vertical text in titles +is supported. The interface (egui) does **not** shape complex scripts or lay out right to left; +the title engine (`crates/text`, harfrust + bidi) does, but Arabic titles are reported broken (#395). + +## Status + +Measured 2026-10-10 by counting entries in each catalog. The denominator is the union of all +catalogs' sources (**3,498 strings**); strings not yet wrapped in `tl!` and engine messages are +not in it, so real coverage is somewhat lower than shown. Estimates calibrated from #449 (Japanese, +whole interface, one PR) at ~4–8 h per language for the catalog plus tests. + +| Language | Code | UI strings | Dialogs / tooltips / help | Script support | Native review | Status | To `full` | +|---|---|---|---|---|---|---|---| +| English | en | 3,498 (100%, source) | yes; no help docs in-app | Latin | n/a | **full** | — | +| Simplified Chinese | zh-cn | 3,485 (99.6%) | dialogs, Settings, effects, shortcuts, status; engine errors English | CJK font fallback; system IME via winit | not recorded | partial | 6–10 h + review | +| Spanish | es | 3,496 (99.9%) | as above | Latin | not recorded | partial | 4–8 h + review | +| Hindi | hi | 0 (0%) | — | **no Devanagari shaping in the interface** | — | none | 30–50 h (shaping in UI + catalog + fonts) | +| Arabic | ar | 0 (0%) | — | **no RTL layout, no shaping in the interface**; titles broken (#395) | — | none | 40–70 h | +| French | fr | 0 (0%) | — | Latin | — | none (requested #498) | 6–10 h + review | +| Portuguese (Brazil) | pt-br | 348 (9.9%) | menus only | Latin | not recorded | menus only | 5–9 h + review | +| Indonesian | id | 0 (0%) | — | Latin | — | none | 6–10 h + review | +| Japanese | ja | 3,487 (99.7%) | as zh-cn; system-font kanji may pick the Chinese fallback (#547) | CJK fonts, vertical title text | not recorded | partial | 4–8 h + review | +| German | de | 0 (0%) | — | Latin | — | none (requested #490) | 6–10 h + review | +| Korean | ko | 0 (0%) | — | needs Hangul fonts in craft-fonts and IME testing | — | none (requested #508) | 10–20 h + review | +| Vietnamese | vi | 0 (0%) | — | Latin with stacked diacritics (font coverage to check) | — | none | 6–12 h + review | +| *Other shipped:* Ukrainian | uk | 351 (10.0%) | menus only | Cyrillic (bundled) | not recorded | menus only | 5–9 h | + +Premiere also ships Italian and Russian, which we don't (6–10 h each). + +**Localization dimension:** 4 of the 12 key languages near-complete (English, Spanish, Japanese, +Simplified Chinese), 1 menus-only, 7 none; against Premiere's 10 languages we cover 4 well and 1 +partly. **~35%, 150–230 h** for all twelve to `full` plus Italian and Russian, of which the script +work (RTL and complex shaping in the egui interface) is the largest and least parallel part. +Every language needs native-speaker review (human). + +## Revision history + +| Date | Change | Summary | +|---|---|---| +| 2026-10-10 | major | Created from ROADMAP.md's "Interface localization" section; catalog coverage measured (es 3,496, ja 3,487, zh-cn 3,485, pt-br 348, uk 351 of 3,498); the 12-language table and script-support gaps added | diff --git a/docs/monitors.md b/docs/monitors.md index d60ff85b..3610c9ea 100644 --- a/docs/monitors.md +++ b/docs/monitors.md @@ -43,14 +43,17 @@ Play auditions the full source span by default. Play In to Out and Play to Out address the marked Source range when Source has focus. The Source wrench menu can loop the marked range; Program keeps its separate loop setting. -Normal forward playback has sound. L starts normal forward Source playback, K stops, and the -left/right frame-step keys address Source when it has focus. Reverse and shuttle-speed Source -playback and source sequences are not yet supported and report an explanatory error. A failed -audio device leaves the picture advancing with a status message; a decoding error stops Source. +Normal forward playback has sound. With Source focused, L plays forward and J backward; pressing +the same key again doubles the speed up to 8×, K stops, Shuttle Slow Right/Left play at ¼ speed, +and the left/right frame-step keys step the Source playhead. As in Program, shuttle speeds other +than normal forward play without sound, and backward playback stops on the first frame. Source +sequences are not yet supported and report an explanatory error. A failed audio device leaves +the picture advancing with a status message; a decoding error stops Source. Commands: `source.playback.play`, `source.playback.stop`, `source.playback.toggle` (no parameters), -available through the UI command registry, CLI bridge and MCP. `playback.toggle` also accepts -`monitor: "source"|"program"` to override focus. `ui.inspect.sourcePlayback` reports playing, +available through the UI command registry, CLI bridge and MCP. `playback.toggle`, +`playback.forward`, `playback.reverse`, `playback.slowForward` and `playback.slowReverse` also +accept `monitor: "source"|"program"` to override focus. `ui.inspect.sourcePlayback` reports playing, audioClock and playhead. Test: `cargo test --release -p filmcraft-ui-egui --test source_playback_ui`. ## Drag a marked Source range to the timeline diff --git a/docs/roadmap.md b/docs/roadmap.md new file mode 100644 index 00000000..3484e30a --- /dev/null +++ b/docs/roadmap.md @@ -0,0 +1,91 @@ +# Roadmap detail + +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (new file: milestones moved from ROADMAP.md, Current focus rewritten from the 2026-10-10 re-measure, beta milestones added) · **Target:** Adobe Premiere Pro 2026 (26.5.2) + +Forward-looking plan: milestones with status, the current focus and what's next, with estimates in +Opus 5.5 agent-hours (calibration in [target-app-parity.md](target-app-parity.md#calibration)). +The one-page summary is [ROADMAP.md](../ROADMAP.md); the ranked work list is [gaps.md](gaps.md). +There is no `cargo xtask scorecard` or `cargo xtask parity` yet ([G7](gaps.md#g7-no-measured-parity-tool-in-ci)). + +## Current focus + +In order. Each item links to its gap entry, which says what "done" means. + +1. **Stop launch crashes and add pull-request CI** ([G4](gaps.md#g4-crashes-at-start-up-and-no-test-ci-on-pull-requests)): `cargo xtask ci` on macOS, Windows and Linux for every PR; old-macOS and Intel-GPU start-up. 40–80 h. +2. **Open `.prproj`** ([G2](gaps.md#g2-premieres-prproj-cannot-be-opened)): the beta blocker. 60–120 h. +3. **10-bit / 4:2:2 camera media in software** ([G3](gaps.md#g3-10-bit-and-422-camera-media-needs-hardware)). 30–50 h. +4. **Fidelity harness against Premiere** ([G1](gaps.md#g1-nothing-compares-our-output-with-premieres)) and **`cargo xtask parity`** ([G7](gaps.md#g7-no-measured-parity-tool-in-ci)): turns estimates into measurements. 50–90 h. +5. **Real-world media corpus and workflow tests** ([G6](gaps.md#g6-no-real-world-media-corpus-or-workflow-acceptance-tests)). 40–70 h. +6. **Hardware acceleration, continued** (#30, [G5](gaps.md#g5-hardware-acceleration-is-partial)): zero-copy upload, Lumetri curves / wheels / HSL and keys on the GPU, VideoToolbox Main 10, Quick Sync / AMF encoders. 120–200 h. +7. **UI fidelity issues** ([G8](gaps.md#g8-ui-fidelity-monitor-handles-effect-controls-docking)): Program monitor transform handles, Effect Controls, docking. 120–200 h. + +## Alpha gate + +Premiere's core daily workflows, checked end to end on the main platform (macOS, Apple silicon), +including saving the work as `.fcproj` and reopening it (atomic saves, schema migrations, +auto-save and crash recovery; migration tests). Strict reading: a "partial" that stops the +workflow would make FilmCraft pre-alpha. + +| Workflow | Works end to end? | Evidence | Hours to pass | +|---|---|---|---| +| Ingest and organize media (import camera / phone / screen files, bins, metadata, proxies) | partial, not blocking | H.264 / HEVC / ProRes / DNx / MXF / AVCHD / MKV import with bit-exact decoders, Media Browser, bins, proxies, Link Media. Gaps: H.264 10-bit / 4:2:2 (Sony XAVC S-I, Canon XF-AVC) opens nowhere and HEVC 4:2:2 only through VideoToolbox (#626, [G3](gaps.md#g3-10-bit-and-422-camera-media-needs-hardware)); E-AC-3 (#647), AVI (#598), camera RAW | 0 (G3 is a beta item, 30–50 h) | +| Assemble and trim a sequence (insert / overwrite, ripple / roll / slip / slide, razor, J/K/L, multicam) | yes | Edit algebra with undo tests, Trim Monitor, Premiere keyboard preset, multicam with audio sync; UX issues (#683, #648, #668) slow it down but don't stop it | 0 | +| Colour correct and grade (Lumetri, LUTs, scopes, colour management) | yes | Lumetri complete with CPU / WGSL parity tests, all Lumetri Scopes, LUT import, HDR export signalling checked with ffprobe | 0 | +| Mix audio (levels, automation, Essential Sound, loudness) | yes on macOS | Mixer graph, automation, 53 effects, BS.1770 loudness matching ffmpeg; device reports are Windows / Linux (#605, #617) | 0 | +| Titles and captions | yes | Text engine, graphic clips, templates, SRT / VTT / SCC / MCC / STL / TTML import-export, burn-in; Arabic titles broken (#395) | 0 | +| Deliver (export H.264 / HEVC / ProRes / DNxHR / MXF with presets, queue) | yes | Own H.264 + AAC encoder, ProRes, DNxHR, MXF, hardware HEVC on macOS, ffprobe-checked exports; bitrate target miss (#371) | 0 | + +**Passes:** every core workflow runs end to end on macOS and the project saves and reopens, so +FilmCraft stays **alpha**; the one partial (10-bit / 4:2:2 camera codecs) blocks some cameras' +originals, not the workflow, and is on the beta list. Interchange with Premiere's own `.prproj` +is a beta requirement, not an alpha one (sequences come over as FCP7 XML, AAF or OTIO). + +## Beta milestones + +Beta = ready for real work ≥ ~75% and `.prproj` opens reliably ([target-app-parity.md](target-app-parity.md#stage)). + +| # | Milestone | Status | Est. | +|---|---|---|---| +| B1 | PR CI on three OSs + launch-crash fixes | ⬜ | 40–80 | +| B2 | `.prproj` import (sequences, clips, transitions, markers, bins, common effect parameters) | ⬜ | 60–120 | +| B3 | Camera media: H.264 High 10 / 4:2:2, HEVC 4:2:2, E-AC-3, AVI, multi-track audio confirmed | 🟡 (multi-stream audio #361) | 50–90 | +| B4 | Fidelity harness + `cargo xtask parity` + media corpus + workflow tests | ⬜ | 90–160 | +| B5 | Hardware: zero-copy, more GPU effects, Windows vendor encoders, VideoToolbox Main 10 | 🟡 | 60–100 | +| B6 | UI fidelity top issues and Windows / Linux runtime | 🟡 | 90–160 | +| B7 | AAF / OMF validated in Avid and Pro Tools; XML import speed | ⬜ | 20–40 | + +## Milestones (history and remaining) + +Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining agent-hours. + +| # | Milestone | Status | Done | Remaining | Est. | +|---|---|---|---|---|---| +| M0 | Skeleton + visual shell | ✅ | Workspace, 20 crates, dock/workspaces, Premiere 26 look, native menus, control channel, MCP, xtask gates (layers, wasm) | — | — | +| M1 | Media I/O | ✅ | MP4/MOV demux+mux, WAV, stills, MJPEG, symphonia audio (MP3/FLAC/ALAC/Vorbis), GOP seek + frame cache, import | Media Browser polish | 2 | +| M2 | H.264 decoder | ✅ | Own decoder, bit-exact on 37+ streams, 500–600 fps 1080p | — | — | +| M3 | Editing core | 🟡 | Edit algebra (insert/overwrite/razor/lift/extract/ripple/roll/slip/slide/rate-stretch/nest/paste), tools, markers, trim mode + Trim Monitor + dynamic J/K/L trimming, Keyboard Shortcuts editor with FilmCraft/Premiere/FCP/Avid presets; Edit/Clip/File menu commands (Label colours and Select Label Group, Paste/Remove Attributes, Select All Matching, Remove Unused, Consolidate Duplicates, Sequence From Clip, Bin From Selection, Offline File, Close Project, Make/Edit Subclip, Modify Audio Channels/Timecode, Frame Hold Options/Add Frame Hold/Insert Frame Hold Segment, Time Interpolation with frame blending, Fit/Fill frame, Breakout to Mono, Extract Audio, Replace With Clip; M3.11: Scene Edit Detection (pure-Rust cut detection, background job), Normalize Mix Track, Simplify Sequence, Transcribe Sequence, Find/Find Next and search bins, Automate to Sequence, Edit Original, Edit Offline, Source Settings, Update Metadata (XMP), Generate Audio Waveform, Project Settings General/Scratch Disks, Get Media File Properties, Save as Template, Selection as FilmCraft Project, Avid Log Exchange export, Flash Cue markers, Dynamic Audio Waveforms, Reveal Log Files, System Compatibility Report); M3.12 keyboard parity: Premiere's default keyboard (~115 keyboard-only commands: edit-point navigation on targeted/any track, select clip at playhead/next/previous, extend edit to playhead, nudge/slip/slide selection, target and source-patch toggles, clip volume ±1 dB/many, frame maximize/full screen/panel cycling, monitor zoom, track heights, Project and Text panel keyboard navigation, text size/leading/alignment, Export Frame, poster frames; see docs/keyboard.md); multicam (Create Multi-Camera Source Sequence, Multi-Camera view with live switching on 1–9, angle switching, Enable/Flatten, Edit Cameras) and sync (Synchronize, Merge Clips; In/Out/timecode/marker/audio — GCC-PHAT, sample-accurate) | Multicam paging >16 angles and grid thumbnails, optical flow (renders as frame blending), 39 Premiere default shortcuts skipped with reasons in docs/keyboard.md (Productions, AI/cloud tools, work area bar, Production/Search panels) | 6–10 | +| M4 | Playback | 🟡 | Audio-clock master, prefetch with cancellation, J/K/L, correct dropped-frame stats, playback resolution, render bar + content-hashed render previews, App Nap opt-out; 1080p H.264 and 3 stacked 1080p streams play with 0 dropped frames | 4K under load, 8K, frame-threaded AV1 decode, reduced-resolution decode for multicam grids | 6–10 | +| M5 | Effects, keyframes, GPU | 🟡 | All 93 Premiere 26 video effects and 84 transitions (CPU), blend modes and 33 common effects on the GPU (#32, GPU2, GPU4, GPU5), keyframes + value/velocity graphs, wgpu compositor, effect + opacity masks (ellipse/polygon/pen, feather, expansion, modes; CPU/WGSL parity), mask tracking (Lucas–Kanade + RANSAC), adjustment layers, effect presets (built-in + user, JSON import/export) | More effects on the GPU (34 ids done incl. Lumetri basic / creative / vignette; GPU export landed #421), GPU masks in the live path, exact Warp Stabilizer (3-D) and Morph Cut | 40–70 | +| M6 | Export | ✅ | Own H.264 encoder (High/Main/Baseline, B-frames, VBR/CBR/2-pass) → MP4 or QuickTime + own AAC; ProRes, DNxHR, MJPEG, PNG/TIFF/BMP sequences (Premiere-style numbering), GIF, WAV, AIFF; background jobs; M6.5: Export mode parity (frame size / rate / scaling / pixel aspect, profile / level, CBR / VBR 1- and 2-pass, keyframe distance, audio codec / rate / channels / bitrate / sample size, multiplexer, burn-in or sidecar captions, image / name / timecode overlays, video limiter, loudness normalization with true-peak limiter, metadata, ranges, estimated size, summary), 24 built-in presets + user presets with favourites and import/export (Preset Manager), export queue (reorder, cancel, retry, several sequences / ranges), Quick Export; `export.*` commands and `filmcraft-cli export --preset` | Interlaced encoding, 5.1 audio, smart render, publishing destinations | 6–8 | +| M7 | Audio | 🟡 | Mixer graph (tracks → submixes → Mix, pre/post-fader inserts and sends, latency-compensated, sample-accurate, ~6× realtime for 24 tracks × 3 effects on one core), Audio Track Mixer + Audio Clip Mixer panels, track automation (Off/Read/Latch/Touch/Write, recorded live while playing, thinned to keyframes, timeline lanes with pen editing), solo/solo-safe, channel mapping basics, peak + BS.1770 loudness meters (match ffmpeg), DSP crate with 16 clip/track effects, Audio Gain (set/adjust/normalize), Constant Power / Constant Gain / Exponential Fade, Essential Sound (types, Loudness auto-match, Repair incl. DeEss/DeReverb, Clarity, Creative, Ducking keyframes, presets; full Dialogue chain 22× realtime) | Music duration remix, ML speech enhancement, 5.1 panner and multichannel buses, voice-over record, effect editor windows (EQ curve), remaining effects (multiband, convolution reverb), clip-mixer automation recording | 6–9 | +| M8 | Colour | 🟡 | Lumetri: basic, creative + looks, RGB & hue curves, wheels, HSL secondary, vignette, section bypass; Input LUT / Look LUT (.cube 1D/3D/shaper, .3dl; tetrahedral CPU + WGSL; project LUT library; built-in camera conversions); Colour Match (Oklab tonal-range statistics, skin protection, solved in Lumetri wheels); colour management: Rec. 709 / Rec. 2100 PQ / HLG working spaces + wide gamut, Interpret Footage colour space (S-Log3, V-Log, Canon Log 2/3, LogC3/4, Apple Log, D-Log from published specs), metadata auto-detect, BT.2390 tone mapping, gamut mapping, HDR export signalling (VUI/colr/mdcv/clli/SEI, ffprobe-verified); Lumetri Scopes: Vectorscope YUV (75/100 % targets, skin-tone line) and HLS, Histogram, Parade (RGB/YUV/RGB-White), Waveform (RGB/Luma/YC/YC no Chroma), presets, Rec. 601/709/2100, 8-bit/float/HDR (cd/m²) scales, multi-scope grid, `scopes.read` | HDR-aware Lumetri maths, macOS EDR monitors, mastering metadata → tone-map peak, D-Log M (no published formula), HSL Secondary refine | 3–5 | +| M9 | More codecs | 🟡 | ProRes decode+encode, AAC decode+encode, HEVC Main/Main 10 decoder (bit-exact on 41 fixtures, ~225 fps 1080p), VP9 decoder (profiles 0–3, 8/10/12-bit, bit-exact on 50+ fixtures; WebM/MKV `V_VP9` and MP4 `vp09` import with key-frame-checked seeking), Matroska/WebM import (H.264/HEVC/VP9/ProRes/MJPEG + AAC/Opus/FLAC/MP3/Vorbis/PCM), Opus decoder (SILK/CELT/hybrid, 5.1/7.1 multistream; all RFC 8251 vectors range-exact; WebM/MKV/MP4), DNxHD/DNxHR decoder (all SMPTE ST 2019-1 CIDs, 8/10/12-bit, 4:2:2/4:4:4, interlaced, alpha; within IDCT precision of ffmpeg on 22 fixtures) + DNxHR LB/SQ/HQ/HQX/444 encoder and MOV `AVdh` export, AV1 decoder (Main profile, 8/10-bit, all intra/inter tools, loop filters, superres, film grain, intra BC, spatial layers; bit-exact vs libdav1d on 19 SVT fixtures + 22 libaom vectors; MP4 `av01` / WebM `V_AV1` import), MXF import (OP1a/OP-Atom; AVC with POC-ordered B pictures bit-exact, DNxHD/DNxHR, ProRes, PCM/AES3; MPEG-2 reported unsupported; index-table seeking, material package timecode), Ogg Opus/Vorbis (granule seeking, pre-skip, end trimming), image sequences, Broadcast WAV timecode | VP9 frame threading, AV1 threading/SIMD (~4 fps 1080p today), AV1 High/Professional profiles, H.264 High 10 / 4:2:2 and HEVC 4:2:2 in software (G3), E-AC-3, AVI (MPEG-2 decoder and hardware decode have landed) | 30–50 | +| M10 | Graphics & captions | 🟡 | Caption tracks (Subtitle/CEA-608/708/Teletext formats, track style), SRT/WebVTT/SCC import+export (frame-exact, property-tested), caption editing (add/split/merge/trim/move, sync-locked insert/extract), Text panel Captions tab, burn-in in Program monitor and export; text engine (`crates/text`: bundled + system fonts, harfrust shaping, bidi, line breaking, paragraph layout, glyph cache; 3-line 1080p title ≈ 0.15 ms warm); graphic clips with text + shape layers (fill, 2 strokes, background, shadow, keyframable transform), Type tool with on-monitor editing, shape/pen tools, Properties/Essential Graphics editor, align/distribute; graphics templates (own `.fcgt` format, 8 original built-ins, Browse tab with engine-rendered thumbnails, export / install / apply / editable properties; never reads .mogrt), rolls/crawls with ease and pre/postroll, responsive pins and intro/outro, per-character styles, Upgrade Caption to Graphic, source graphics, Replace Fonts; MCC (608+708), EBU STL, TTML/IMSC1, DFXP (frame-exact, property-tested) | 608/708 embedding in video streams, speech-to-text | 12–18 | +| M11 | Interchange & project management | ✅ | `.fcproj` schema versions + migrations, atomic saves, Save a Copy/Revert, auto-save ring + crash-recovery journal (Preferences ▸ Auto Save, recovery prompt), FCP7 XML, FCPXML, EDL, OTIO; offline media (own slate) + Link Media (fingerprint-checked relink, folder remap, search, Align Timecode, Make Offline); proxies (ProRes Proxy/LT, H.264 ¼/½ background jobs, attach/detach/reconnect, monitor toggle, export full-res) + ingest (copy/transcode/proxies); Project Manager (collect, consolidate + transcode with handles, size estimate) | Rename media to clip names, image-sequence conversion, Media Browser-driven relink, smart (cross-drive) path tracking | 1–2 | +| M12–M16 | Multicam, web (WASM), platform, long tail | 🟡 | Multicam: audio sync (GCC-PHAT, sub-sample), Merge Clips, multi-camera source sequences, Multi-Camera view with live switching (1–9); L0–L4 crates compile to wasm32 | Web app shell (file access, WebCodecs, audio), scene detection, auto reframe, ~800 remaining commands and dialogs, performance hardening | 35–55 | +| HW | Hardware acceleration (#30) | 🟡 | Blend modes and 34 effect ids on the GPU incl. Lumetri basic / creative / vignette (#32, GPU2–GPU7); GPU export rendering (#421); decode: VideoToolbox (#33, #41), Media Foundation / D3D11 H.264 / HEVC / VP9 / AV1 (HW2, HW3), VA-API H.264 / HEVC (#358), NVDEC H.264 / HEVC (#549); encode: VideoToolbox H.264 / HEVC, NVENC H.264 (Windows, Linux) and HEVC incl. Main 10 HDR (#328, #360) | Zero-copy, remaining GPU effects, VideoToolbox Main 10, Quick Sync / AMF, VA-API VP9 / AV1 and encode | 120–200 | +| L10N | Localization | 🟡 | Spanish, Japanese, Simplified Chinese whole interface; Portuguese (Brazil), Ukrainian menus | Hindi, Arabic (RTL), French, Indonesian, German, Korean, Vietnamese, Italian, Russian; native review | 150–230 | + +## After beta + +Plugin hosting (VST3 / AU, then OpenFX; owner decision, [G11](gaps.md#g11-no-plugin-hosting)), +AI features with openly licensed models ([G12](gaps.md#g12-ai-features)), delivery formats +([G13](gaps.md#g13-delivery-formats)), localization ([G14](gaps.md#g14-localization-8-of-the-12-key-languages-missing)), +the web app shell ([G20](gaps.md#g20-the-web-app-shell)). + +## Revision history + +| Date | Change | Summary | +|---|---|---| +| 2026-10-10 | minor | Alpha gate table added (craftrules core-workflow gate): six core workflows, all pass on macOS; stage stays alpha | +| 2026-10-10 | major | Created: milestone table moved from ROADMAP.md (M5 and M9 remaining work updated), HW and L10N rows, beta milestones B1–B7, Current focus from the re-measure | diff --git a/docs/target-app-parity.md b/docs/target-app-parity.md new file mode 100644 index 00000000..29b2f706 --- /dev/null +++ b/docs/target-app-parity.md @@ -0,0 +1,358 @@ +# Parity with Adobe Premiere Pro + +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (full re-measure against Premiere Pro 2026 26.5.2; replaces the scorecard and estimate that lived in ROADMAP.md) · **Target:** Adobe Premiere Pro 2026 (26.5.2, macOS) + +The authoritative assessment of how close FilmCraft is to Premiere Pro, and how much work is left. +[ROADMAP.md](../ROADMAP.md) summarizes it; [gaps.md](gaps.md) lists every shortfall one at a time. +The deep checklists are [file-format-parity.md](file-format-parity.md), +[codec-parity.md](codec-parity.md), [hardware-parity.md](hardware-parity.md), +[ui-parity.md](ui-parity.md) and [localization-parity.md](localization-parity.md). + +## Headline + +| | Value | Kind | +|---|---|---| +| **Feature breadth** | **~86%** | Partly measured: menus 92% (label diff, below), effects / transitions / audio effects 100% (unit tests); the other areas are estimated | +| **Ready for real work** | **~56%** (range 51–61%) | Estimated depth per area, combined with written weights ([formula](#three-readiness-numbers)) | +| **Mainstream practitioner** | **~55%** | Weekly editing areas only, with written discounts ([method](#mainstream-practitioner)) | +| **Essentials user** | **~60%** | Core features at default settings, with written discounts ([method](#essentials-user)) | +| **Stage** | **alpha** | ~19 points and ~400–700 h from beta, and beta also needs `.prproj` import ([why](#stage)) | +| **Remaining to beta** | **~400–700 Opus 5.5 agent-hours** | Estimated, calibrated below | +| **Remaining to full parity** | **~1,550–2,750 Opus 5.5 agent-hours** (to ~95%: ~1,450–2,600) | Estimated; ~70% parallelizes | +| Codebase (measured 2026-10-10) | 45 crates + 3 apps, ~343,000 lines of Rust, 2,520 `#[test]` functions, 1,098 commits, 268 merged PRs, 69 authors | `git`, `gh`, `grep` | + +The previous estimate (2026-10-05) was breadth ~87%, ready ~50–60%, and "~210–325 agent-hours to +100%". The breadth figure is one point lower now because this pass counts the file formats Premiere +lists and we lack (AVI, WMV, camera RAW, EXR / DPX / PSD stills, `.prproj`, DVD / Blu-ray / DCP +export) rather than only the formats we have. The hours went up because the old figure covered the +checklist only and left out plugins, AI, localization, `.prproj` and the depth work that 188 open +issues now document. That is new evidence, not drift: see [Calibration](#calibration). + +## What we measured against, and how + +- **Target:** Adobe Premiere Pro 2026, version **26.5.2** (Spotlight `kMDItemVersion` of the + installed `/Applications/Adobe Premiere Pro 2026`), plus Adobe Media Encoder 2026 for export. +- **Menus (measured).** The maintainers' native menu-bar dump of Premiere 26.5.2 + (`plan/premiere/menus.json`, taken 2026-09-30 through System Events; local only, never + committed) has 433 leaf items. Removing Adobe-cloud, account, help, macOS system and + recent-file entries leaves 375. A label match against the string literals of `crates/engine`, + `crates/ui-egui`, `crates/edit` and `crates/project` finds **338 of 375 (90.1%)**; of the 37 + misses, 9 are dynamic or system items (About, AutoFill, Dictation, Emoji, window names), so + in-scope coverage is **338 / 366 ≈ 92%**. Real misses: Spelling (2), Render and Replace / + Restore Unrendered, Enhance Speech toggle, Auto-Tag Audio Types, Auto Reframe Sequence, Generate + / Reassociate Source Clips, Project Shortcut, Refresh All Projects, Export Selection as Premiere + Project, 5 workspaces (Essentials, Metalogging, Prelude, Starter, Text-Based Editing), + Audio Clip / Track Effect Editor, Sequence Index, AI Models and Collaboration settings, UXP / + Exchange plug-in entries. A label match is presence only, and can over-count (a label that + exists in a different place); it is a lower bound on misses, not proof of behaviour. The script + is ad hoc; turning it into `cargo xtask parity` is gap [G7](gaps.md#g7-no-measured-parity-tool-in-ci). +- **Effects (measured by tests).** `premiere_26_video_effects_catalogue`, + `premiere_audio_set_is_complete_and_foldered` and `premiere_26_transition_tree` check the + catalogue against Premiere's Effects panel: video effects 93/93 (+Legacy, Obsolete), audio + effects 53/53, video transitions 84/84 (+21 Legacy), audio transitions 3/3. +- **The installed app (file names and listings only, per [AGENTS.md](../AGENTS.md) §2; nothing + inside the bundle was opened or read).** `Contents/Resources/*.lproj`: de, en, es, fr, it, ja, + ko, pt, ru, zh_CN (10 UI languages). `Contents/MediaIO/systempresets/` folder names are + four-character exporter / codec codes: JPEG, QuickTime, MP3, PNG, Windows Media, AIFF, AVI, + BMP, DPX, TIFF, Targa, WAV, HEVC, MPEG-2 / DVD / Blu-ray, MXF (JPEG 2000, DNx, P2, XDCAM HD, + AS-10, AS-11), MP4 / AAC / H.264 / H.264 Blu-ray, FLV, uncompressed AVI, DCP, GIF, raw PCM, + OpenEXR. `Contents/Frameworks` names show what it decodes natively: ARRIRAW (ARRI Image SDK), + RED R3D, Sony RAW, Canon RAW (`CrmSdk`), Codex HDE, DNx SDK, JPEG XS, JPEG 2000 (Kakadu), + DV100, MPEG-4 Part 2, OMF (`Pro4OMF`), AAF, LTC timecode, C2PA content credentials, and an ONNX + runtime for its AI features. `PlugIns` lists OpenEXR, Radiance and RED / MXF XMP handlers. +- **Our side:** this repository on `origin/main` at `98ba9ac` (2026-10-10): code, crate READMEs, + tests, `docs/performance.md` benchmarks, and the 188 open / 118 closed GitHub issues (user + reports from releases 0.2–0.5). +- **Public documentation:** Adobe's "What's new" and release notes for 26.0–26.3 (Object Mask, + redesigned mask tools, R3D NE, OpenTimelineIO, GPU thumbnails, 4:2:2 10-bit hardware decode on + NVIDIA Blackwell), and Adobe's published lists of supported file formats and GPU / hardware + acceleration. +- **Not done in this pass:** running Premiere side by side on shared projects. No rendering or + behaviour of ours is compared with Premiere's output automatically yet ([G1](gaps.md#g1-nothing-compares-our-output-with-premieres)). + +## Weights + +Feature areas carry 70% of "ready for real work", cross-cutting qualities 30%. Weights reflect +what a working editor uses daily: the timeline first, then media, effects and panels. + +| Group | Weight | +|---|---| +| Editing, timeline, trimming, multicam | 20% of features | +| Effects and transitions | 12% | +| Media import and codecs | 12% | +| Audio | 10% | +| Panels and UI fidelity | 10% | +| Colour | 8% | +| Graphics, titles, captions | 8% | +| Export and delivery | 8% | +| Project files and interchange | 8% | +| Preferences and project management | 4% | +| *Cross-cutting:* hardware and performance | 10% of overall | +| Stability | 8% | +| Platforms | 5% | +| Plugins and ecosystem | 5% | +| AI features | 2% | + +Feature breadth weights the same ten feature areas only (cross-cutting qualities have no +"exists / doesn't" count). + +## By feature area + +Breadth = does it exist. Ready = depth, correctness, fidelity, robustness. Hours = Opus 5.5 +agent-hours to close the area to Premiere, sequential. + +| Area | Weight | Breadth | Ready | Hours | Evidence | Detail | +|---|---|---|---|---|---|---| +| Editing, timeline, trimming, multicam | 20% | ~90% | ~70% | 80–140 | Edit algebra, trim modes, Trim Monitor, multicam with audio sync, Premiere default keyboard (143 commands), Source playback and range dragging (10-08). Open: ripple-delete of a gap by column (#668), Backspace on gaps (#648), clip stretching (#653), deselect on empty space (#683), new tracks by dragging media (#483), Escape cancelling drags (#580), time remapping (#510), effects on a whole track (#623) | [ui-parity.md](ui-parity.md) | +| Effects and transitions | 12% | 100% (measured) | ~60% | 80–140 | Every effect exists; Warp Stabilizer is 2-D, Morph Cut and Auto Reframe approximate, optical flow renders as frame blending. 34 effect ids run on the GPU against ~82 of Premiere's ~92 marked Accelerated. No Object Mask (26.0, AI). Transform shutter angle adds no motion blur (#415); Ultra Key setting (#458) | [hardware-parity.md](hardware-parity.md) | +| Media import and codecs | 12% | ~75% | ~60% | 150–250 | Own bit-exact decoders for H.264, HEVC, VP9, AV1, ProRes, DNx, APV, MPEG-2, AAC, AC-3, Opus. But H.264 only 8-bit 4:2:0 and HEVC no 4:2:2 in software (most 10-bit cameras, #626); no camera RAW (R3D, ARRIRAW, BRAW, Sony RAW, Canon RAW, ProRes RAW, #345, #383); no E-AC-3 (#647), AVI (#598), WMV, DV, XAVC / P2 metadata; Sony start timecode (#460); WebM black (#432) | [codec-parity.md](codec-parity.md), [file-format-parity.md](file-format-parity.md) | +| Audio | 10% | ~88% | ~65% | 50–90 | Mixer graph, automation, all 53 effects, Essential Sound, 5.1, voice-over, Remix, multi-stream container audio (#361). Device and playback reports: crackle / cut-outs (#336, #410, #602, #605, #617); Enhance Speech is DSP, not a model; ALSA fallback fixed (#254) | | +| Panels and UI fidelity | 10% | ~88% | ~60% | 120–200 | All main panels. Missing: on-screen transform handles in the Program monitor (#639), Effect Controls keyframe zoom / divider / row lines (#640–#645), docking rearrangement (#493), multi-monitor windows (#466), filmstrip thumbnails (#614), marquee in Project panel (#578) | [ui-parity.md](ui-parity.md) | +| Colour | 8% | ~90% | ~70% | 40–70 | Lumetri complete incl. scopes, LUTs, colour management, HDR export. Lumetri on the GPU covers basic, creative and vignette only; HDR-aware Lumetri maths, macOS EDR display, D-Log M; a Lumetri report (#628) | | +| Graphics, titles, captions | 8% | ~85% | ~65% | 50–90 | Text engine with shaping and bidi, shapes, gradients (#514), our `.fcgt` templates (`.mogrt` refused by design), rolls / crawls, all caption formats. Arabic titles reported broken (#395); no 608/708 embedding in the video stream; speech-to-text off in release builds | | +| Export and delivery | 8% | ~80% | ~65% | 60–110 | H.264 (own encoder), HEVC through hardware only, ProRes, DNxHR, APV, MXF OP1a / OP-Atom, image sequences, GIF, WAV / AIFF, presets, queue, GPU export rendering (#421). Missing: AV1, MPEG-2 / DVD / Blu-ray, DCP, JPEG 2000 / XS MXF, AS-10 / AS-11, XDCAM HD, EXR / DPX / Targa sequences, MP3, AAC-only audio, ProRes 4444 in the UI (#342), interlaced, smart render; bitrate target missed (#371) | [file-format-parity.md](file-format-parity.md) | +| Project files and interchange | 8% | ~70% | ~45% | 90–160 | `.fcproj` with migrations, auto-save, recovery; FCP7 XML, FCPXML, AAF, OMF, EDL, OTIO, ALE. **Premiere's own `.prproj` cannot be opened** (the main beta blocker); AAF / OMF never validated in Media Composer / Pro Tools; XML import of a Premiere sequence took 97 s (#461) | [file-format-parity.md](file-format-parity.md) | +| Preferences and project management | 4% | ~80% | ~65% | 20–40 | 16 Settings categories, most wired; Project Manager, proxies, Link Media; Media Manager limits (#411); images as proxies (#451) | | +| **Features, weighted** | 100% | **~86%** | **~64%** | **740–1,290** | | | + +## By dimension + +| Dimension | Ready | Hours | Doc | +|---|---|---|---| +| Features (editing, effects, colour, audio, graphics, export settings, preferences) | ~66% | 350–620 | this file | +| UI / UX fidelity | ~60% | 120–200 | [ui-parity.md](ui-parity.md) | +| File formats (containers, project, interchange, captions, stills) | ~60% | 140–240 | [file-format-parity.md](file-format-parity.md) | +| Codecs (decode and encode, fidelity, camera media) | ~60% | 150–250 | [codec-parity.md](codec-parity.md) | +| Hardware acceleration (decode, encode, GPU effects, I/O devices) | ~45% | 120–200 | [hardware-parity.md](hardware-parity.md) | +| Localization (12 key languages) | ~35% of Premiere's set; 4 of 12 key languages near-complete | 150–230 | [localization-parity.md](localization-parity.md) | +| Performance (beyond hardware) | ~55% | 60–120 | [performance.md](performance.md) | +| Stability | ~45% | 60–120 | [gaps.md](gaps.md#g4-crashes-at-start-up-and-no-test-ci-on-pull-requests) | +| Platforms | ~60% | 50–90 | [gaps.md](gaps.md#g9-windows-and-linux-at-run-time) | +| Plugins and ecosystem | ~5% | 200–400 | [gaps.md](gaps.md#g11-no-plugin-hosting) | +| AI features | ~25% | 150–300 | [gaps.md](gaps.md#g12-ai-features) | +| **Total** | **~56%** | **~1,550–2,750** | | + +Dimension hours are disjoint (the feature row excludes UI, formats and codecs), so they add up. +Evidence for the cross-cutting rows: + +- **Hardware ~45%.** Hardware decode exists on all three desktop OSs (VideoToolbox; Media + Foundation / D3D11 incl. VP9 and AV1; VA-API and NVDEC for H.264 / HEVC on Linux). Hardware + encode: VideoToolbox H.264 / HEVC 8-bit; NVENC H.264 (Windows, Linux) and HEVC incl. Main 10 HDR + (Windows). Missing against Premiere: Intel Quick Sync and AMD encoders, 10-bit HEVC on macOS, + ProRes hardware, zero-copy upload, ~48 of Premiere's accelerated effects, external video I/O, + control surfaces. Detail: [hardware-parity.md](hardware-parity.md). +- **Performance ~55%.** 4K H.264 / HEVC play with no dropped frames with hardware decoding (M4 + Pro); GPU export is 2.4–3.2× faster on effects-heavy timelines; AV1 software decode is still + slow and 8K is not real time. Users on 0.5.0 report constant lag and ~100% CPU (#523, #525) and + a CPU fallback adapter rendering everything in software (#632). +- **Stability ~45%.** Never-crash rules enforced by lints; hostile-input fuzzing. But open + launch crashes on macOS 12 (missing VideoToolbox symbol, #468 / #661, #655), Intel UHD graphics + (#512), Windows 11 (#380, #582), a generic "keeps crashing" (#687), and **no workflow runs the + test suite on pull requests** (`.github/workflows` has release, packaging lint, FreeBSD and + Windows ARM64 builds only); `cargo xtask ci` runs locally only. +- **Platforms ~60%.** Releases for macOS, Windows (x64, ARM64), Linux (AppImage, Flatpak, deb, + tar) and FreeBSD; Linux is something Premiere doesn't run on at all. Runtime testing is still + mostly macOS; Windows and Linux reports dominate the issue list. The web build runs but its shell + is incomplete. No mobile. +- **Plugins ~5%.** No VST3 / Audio Units, no OpenFX, no Premiere / After Effects plug-in API, no + UXP panels (#325). Needs an owner decision (FFI). MCP, CLI and the control channel are ahead of + Premiere's scripting. +- **AI ~25%.** Speech to text exists (local Whisper) but is off in release builds; text-to-speech + narration with Kokoro-82M landed 10-10 (#278, beyond Premiere); scene edit detection; Enhance + Speech and Auto Reframe are DSP / heuristics. Missing: Object Mask (#406, #694), Generative + Extend, media-intelligence search, caption translation, auto colour. + +## Three readiness numbers + +| Audience | Ready % | Opus 5.5 agent wall-clock hours to ~95% | Work that dominates | +|---|---|---|---| +| Full target (ready for real work) | ~56% | ~1,450–2,600 (~70% parallelizes) | Plugin hosting, AI models, localization incl. RTL / Devanagari, camera RAW and delivery codecs, `.prproj`, hardware acceleration, UI fidelity | +| Mainstream practitioner | ~55% | ~500–850 (~65% parallelizes) | `.prproj` import (60–120), UI fidelity (120–200), stability and PR CI (40–80), 10-bit / 4:2:2 camera decode (30–50), playback / hardware (60–100), import and export gaps (60–100) | +| Essentials user | ~60% | ~180–350 (~60% parallelizes) | Launch crashes on Windows / Intel / older macOS (40–80), discoverability and timeline friction (60–110), playback lag (30–60), common import gaps (20–40), audio devices (15–30) | + +Hours per audience are calibrated like the rest (one hardware backend ~3–5 h, one encoder ~4–8 h, a +whole-interface language ~4–8 h, ~4–5 h per breadth point, depth work ~1.5–2× that). The +audiences nest, so essentials ≤ mainstream ≤ full: the essentials work is a subset of the +mainstream list, which is a subset of the full list. Needs a human: the plugin decision and AI +models (full only), native-speaker review (full only), Avid / Pro Tools validation (mainstream +and full), Windows / Intel test machines (all three). + + +All three use the depth ("Ready") estimates of the [feature areas](#by-feature-area); they differ in +which areas count and how they are discounted. The stage follows the first one and the +[core-workflow gate](roadmap.md#alpha-gate). + +### Ready for real work (full target) + +Written formula: 0.70 × (feature areas, weighted as in [Weights](#weights)) + the cross-cutting +rows, each weight × its percent. + +- Feature areas: 0.20×70 + 0.12×60 + 0.12×60 + 0.10×65 + 0.10×60 + 0.08×70 + 0.08×65 + 0.08×65 + + 0.08×45 + 0.04×65 = **63.1%**, × 0.70 = 44.2 points. +- Cross-cutting: hardware and performance 10% × 45, stability 8% × 45, platforms 5% × 60, + plugins 5% × 5, AI 2% × 25 = 4.5 + 3.6 + 3.0 + 0.25 + 0.5 = **11.9 points**. +- Total **56.0% ≈ ~56%** (range 51–61%). Earlier in this pass the + headline said ~55% (range 50–60%): the same inputs, rounded down by judgement instead of + computed. Nothing in the evidence changed; the number moved only because the arithmetic is now + written out. The stage is unaffected (still alpha; beta needs ~75% and `.prproj`). + +### Mainstream practitioner + +A typical professional editor's week: cut, trim, colour, mix, title, caption and deliver, on +their own machine, exchanging projects with Premiere users. Excluded: plugins, generative and cloud +AI, Team Projects / Productions, control surfaces and external I/O, other interface languages. +Depth here is judged on what that editor actually uses inside each area, so it is higher than the +full-area depth (e.g. the ~20 common effects rather than all 93 and their approximations). + +| Area | Weight | Depth | Why | +|---|---|---|---| +| Editing and trimming | 28% | 80% | Insert / overwrite, ripple / roll / slip / slide, razor, J/K/L, markers, multicam all work; gap deletion (#648, #668) and deselect (#683) are friction | +| Importing everyday footage | 14% | 65% | Phone and 8-bit camera H.264 / HEVC, ProRes, DNx, MXF open; 10-bit / 4:2:2 H.264 (Sony XAVC S-I, Canon) does not (#626); E-AC-3, AVI missing | +| Common effects and transitions | 9% | 75% | Lumetri, Transform, Crop, blurs, Cross Dissolve, keys exist; Warp Stabilizer is 2-D, Object Mask missing | +| Colour (Lumetri, scopes, LUTs) | 9% | 80% | Complete, parity-tested CPU / GPU; curves / wheels on CPU | +| Audio (levels, Essential Sound, loudness, ducking) | 11% | 75% | Complete; audio-device reports off macOS | +| Titles and captions | 7% | 75% | Text, shapes, templates, SRT / SCC / VTT; `.mogrt` not read by design | +| Export with presets | 9% | 75% | H.264, HEVC (hardware), ProRes, DNxHR, presets, queue; bitrate miss (#371) | +| Save, reopen, auto-save, relink, proxies | 4% | 80% | Atomic saves, migrations, recovery, Link Media | +| Playback speed on their machine | 9% | 65% | 4K H.264 / HEVC real time with hardware decode on macOS; lag / CPU reports (#523, #525) | +| **Weighted depth** | 100% | **74.8%** | | + +Discounts for what still stops real work: + +- **Interaction fidelity ×0.90:** about 40 open UI issues; no transform handles in the Program + monitor (#639); Effect Controls keyframe zoom / divider (#640–#645); docking (#493). +- **Stability on real machines ×0.92:** current macOS on Apple silicon is solid, but launch + crashes on macOS 12, Intel UHD and Windows 11 (#468, #512, #380, #582, #687), and no CI runs the + tests on pull requests. +- **File exchange with Premiere users ×0.88:** `.prproj` does not open (colleagues send + projects, not XML); AAF / OMF never validated in Avid / Pro Tools; FCP7 XML / OTIO do work. + +74.8 × 0.90 × 0.92 × 0.88 = **54.5% ≈ ~55%.** It is not higher than the full number, unlike most +Craft apps: the parts the mainstream number leaves out (plugins, AI, localization) weigh only +~12% in the full number, while the three discounts hit exactly what a working editor meets every +week. Raising it means fixing [G2](gaps.md#g2-premieres-prproj-cannot-be-opened), +[G3](gaps.md#g3-10-bit-and-422-camera-media-needs-hardware), +[G4](gaps.md#g4-crashes-at-start-up-and-no-test-ci-on-pull-requests) and +[G8](gaps.md#g8-ui-fidelity-monitor-handles-effect-controls-docking). + +### Essentials user + +Someone who imports phone or camera clips, puts them on a timeline, trims, adds a title, music and +a dissolve, and exports an MP4 with the default preset. Excluded: everything above that the +mainstream number excludes, plus advanced options, multicam, colour grading beyond presets, +interchange and pro formats. + +| Feature | Weight | Depth | Why | +|---|---|---|---| +| New project, import clips | 15% | 80% | Drag-and-drop and Import work for phone MP4 / MOV (H.264, HEVC); MKV multi-audio (#601), AVI (#598) fail | +| Put clips on the timeline, arrange | 15% | 85% | Drop to timeline (#517), snapping, tracks; new tracks by dropping missing (#483) | +| Trim, split, delete | 15% | 80% | Razor, trim, ripple delete; Backspace on gaps (#648) and deselect (#683) confuse | +| Undo / redo | 5% | 95% | Snapshot undo for every command | +| Titles | 10% | 75% | Type tool, templates; Arabic text broken (#395) | +| Transitions (cross dissolve) | 5% | 85% | Drag on, resize (#503) | +| Music and volume | 10% | 75% | Clip volume, fades, meters; device issues off macOS | +| Playback | 10% | 70% | Smooth on current Macs; lag / CPU reports on 0.5.0 (#523, #525) | +| Export MP4 with the default preset | 10% | 80% | Own H.264 + AAC, Quick Export | +| Save and reopen | 5% | 85% | `.fcproj`, auto-save, recovery prompt | +| **Weighted depth** | 100% | **80.0%** | | + +Discounts a beginner actually hits: + +- **Launch and stability ×0.88:** the beginner's machine is often a Windows laptop with Intel + graphics or an older Mac; open launch crashes there (#380, #413, #477, #512, #582, #655, #661, + #687). +- **Discoverability and UI clarity ×0.90:** a dense Premiere-style interface with no guided start; + reports about deselecting, deleting gaps and transitions controls (#683, #648, #577). +- **Opening files people send them ×0.95:** phone and web video mostly opens; AVI, E-AC-3 audio + (#647), FLAC-in-MP4 (#603) and some WebM (#432) do not. + +80.0 × 0.88 × 0.90 × 0.95 = **60.2% ≈ ~60%.** + +### User evidence + +Counted 2026-10-10 from all 323 GitHub issues and their 270 comments, excluding the four +repository collaborators (script in a private scratch folder; regex for praise and for "switched +from" phrases, then read by hand): + +- **Praise:** 31 distinct non-maintainer people across 37 issues used praise words. About a dozen + are explicit praise, e.g. "Great job making this go!" (#313), "I personally love everything done + on it" (#227), "impressive otherwise" (#208), "Great project guys" (#451). Several of them pair + the praise with a bug ("great project although it obviously has a lot of bugs", #455). +- **Switched from Premiere:** **0** reports of having switched. Two people describe testing it as + an alternative or hoping to leave Adobe (#411 "testing FilmCraft as an alternative workflow to + Adobe Premiere", #325 "finally moving away from adobe would be a god send"). +- **Open issues:** 202 open from non-maintainers; by title, ~93 are bug reports, of which **~64 + are core-path** (launch, import, playback, timeline, audio, export) and ~29 niche; ~109 are + feature requests or questions (Object Mask, plugin compatibility, languages, Android, + collaboration, node-based colour). + +This supports the discounts: people like the direction and use it, but core-path bugs outnumber +niche requests among the bug reports about two to one, and nobody reports having switched yet. + +## Stage + +**Alpha.** It passes the core-workflow gate ([roadmap.md](roadmap.md#alpha-gate): all six of Premiere's core daily workflows work end to end on macOS and the project saves and reopens). Core workflows run end to end (import, edit, trim, colour, mix, caption, deliver, +round-trip through XML / AAF), but ready-for-real-work is ~56%, below the ~75% beta bar, and the +standard's beta condition "opens and saves the target's main format reliably" fails outright: +FilmCraft cannot open a `.prproj` (it reads Premiere's FCP7 XML / AAF / OTIO exports instead). +Distance to beta: ~19 points of ready-for-real-work plus `.prproj` import, ~400–700 agent-hours: + +| To reach beta | Hours | +|---|---| +| `.prproj` import of mainstream projects (sequences, clips, effects' common parameters, markers) | 60–120 | +| Stability: launch crashes, PR CI running `cargo xtask ci` on all three OSs, crash triage | 40–80 | +| H.264 High 10 / 4:2:2 and HEVC 4:2:2 software decoding (camera media) | 30–50 | +| Real-world media corpus and scripted workflow acceptance tests | 40–70 | +| UI fidelity top issues (Program monitor handles, Effect Controls, docking, deselect) | 60–100 | +| Hardware: zero-copy, VideoToolbox Main 10, Quick Sync / AMF encode, more GPU effects | 60–100 | +| Windows / Linux runtime fixes, audio device issues | 30–60 | +| Container gaps users hit: E-AC-3, AVI, multi-track audio, still formats | 30–50 | +| Interchange validation (AAF / OMF in Avid and Pro Tools, XML speed) | 20–40 | +| Performance: lag / CPU reports, AV1 speed | 30–60 | +| **Total** | **~400–730** | + +## Calibration + +Hours are Opus 5.5 agent wall-clock hours, calibrated against this repo's history (`git log`, +`gh pr view` timestamps): + +- **Whole history.** First commit 2026-09-30; by 2026-10-10, 1,098 commits and 268 merged PRs + (243 since 10-05, most from community contributors' agents) took breadth from 0 to ~86%. The + 2026-10-01/02 blocks measured **~4–5 agent-hours per checklist point** with five parallel + agents (old ROADMAP). +- **A hardware decode backend** (one OS API, H.264 + HEVC, with parity tests): VideoToolbox #33 + (first commit 07:25, merged 11:26 the same day), VA-API #358 (+3,942 lines), NVDEC #549 (06:17 → + 11:04): **~3–5 h each**. +- **A hardware encoder per codec and OS:** NVENC HEVC #328 (+1,635 lines, 19:28 → 13:46 next day + including review), Main 10 HDR #360 (02:38 → 14:05): **~4–8 h each**. +- **GPU export rendering plus four GPU effects** (#421, 15 commits over ~23 h): **~8–12 h**. +- **A whole-interface language** (~3,500 strings, tests): Japanese #449 in one commit: + **~4–8 h**, plus native review (human). A menus-only language (Ukrainian #407, ~350 strings): + ~1 h. +- **A clean-room decoder from spec, bit-exact**: AV1 (09-30 → 10-01 night, several agents): ~15–30 h. +- **Depth work costs more per point than breadth.** Each remaining point of ready-for-real-work + is a bug class, a format edge or a fidelity comparison rather than a missing menu item; the + estimates apply ~1.5–2× the breadth rate. + +- **Wall-clock limits** (from the 2026-10-02 blocks): more than ~5 local agents slows everyone + (each worktree build is 6–7 GB; a full `cargo xtask ci` takes 20–60 min under load; the disk + filled twice), and one integrator merging and re-running CI is a bottleneck. With 4–5 parallel + agents plus an integrator, wall-clock is ~¼–⅓ of the agent-hours; community PRs now add + capacity beyond that. + +What parallelizes: almost everything by area (~70%). Serial: `.prproj` import before beta, +fidelity harness before fidelity numbers. **Needs a human:** the plugin-hosting decision (FFI, +licences), native-speaker review of every language, licensed or trained models for AI features, +hardware we don't own (Intel Quick Sync, AMD encoders, Blackwell 4:2:2, DeckLink / AJA I/O, +control surfaces), and validation in Avid Media Composer / Pro Tools. + +## Revision history + +| Date | Change | Summary | +|---|---|---| +| 2026-10-10 | minor | Readiness table: hours to ~95% for each audience (full ~1,450–2,600, mainstream ~500–850, essentials ~180–350); full number confirmed as an additive weighted sum (no change) | +| 2026-10-10 | minor | Added the three readiness numbers: full ready-for-real-work formula written out (~55% -> ~56%, same inputs, now computed rather than rounded down), mainstream practitioner ~55% and essentials user ~60% with written weights and discounts, and counted user evidence (31 people praising, 0 switch reports, ~64 core-path vs ~29 niche open bugs) | +| 2026-10-10 | minor | Stage checked against the new core-workflow gate (docs/roadmap.md#alpha-gate): passes, stays alpha | +| 2026-10-10 | major | Created from ROADMAP.md's scorecard and estimate; full re-measure against Premiere Pro 26.5.2 (menu label diff, installed-bundle listings, 188 open issues); breadth ~87 → ~86% (file formats counted), ready ~50–60% → ~55%, hours re-estimated to include plugins, AI, localization and `.prproj`; stage alpha | +| 2026-10-05 | major | Honest assessment added in ROADMAP.md (checklist ~87% vs ready ~50–60%) | +| 2026-10-02 | minor | Scorecard ~62% → ~72% → ~87% as the menu long tail, effects and codecs landed | diff --git a/docs/ui-parity.md b/docs/ui-parity.md new file mode 100644 index 00000000..611bb827 --- /dev/null +++ b/docs/ui-parity.md @@ -0,0 +1,38 @@ +# UI and interaction parity + +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (new file: tools, handles, panels, keyboard and feel against Premiere Pro 26.5.2, with the open UI issues) · **Target:** Adobe Premiere Pro 2026 (26.5.2, macOS) + +How FilmCraft looks, feels and responds compared with Premiere: tools, on-screen handles, snapping, +nudging, modifier keys, shortcuts, drag behaviour, numeric entry, panels, context menus. Keyboard +detail is in [keyboard.md](keyboard.md), monitor view options in [monitors.md](monitors.md). +Reference screenshots and behaviour notes are maintainer-local (`plan/premiere/`, never committed). + +**UI / UX dimension: ~60% ready, breadth ~88%, 120–200 h** ([target-app-parity.md](target-app-parity.md#by-dimension)). +UI fidelity was compared with Premiere reference screenshots by the agents that built it; that +comparison has not been repeated independently. The ready figure is pulled down by the volume of +open interaction reports (about 40 of the 188 open issues). + +## Areas + +| Area | Premiere | Ours | % | Open issues | Est. | +|---|---|---|---|---|---| +| Menus | 366 in-scope menu items | ~338 present (label diff, 92%) | 92% | spelling, Render and Replace, 5 workspaces | 10–20 h | +| Keyboard | Premiere default keyboard + editor | Premiere-compatible preset (143 keyboard-only commands; 39 cloud / absent-panel keys skipped with reasons), FilmCraft / FCP / Avid presets, editor | 90% | Ctrl chords (#256 fixed), modifier hints on Windows (#270 fixed) | 5–10 h | +| Timeline editing tools | Selection, Track Select, Ripple / Rolling / Rate Stretch, Razor, Slip / Slide, Pen, Hand, Zoom, Type | all present, trim modes, Trim Monitor, dynamic trimming | 80% | deselect in empty space (#683), gap delete (#648, #668), clip stretch (#653), new tracks by dropping (#483), Escape cancels drag (#580), delete in timeline (#680, #404) | 30–50 h | +| Program monitor on-screen controls | Motion / Transform handles, mask handles, safe margins, guides | guides, rulers, safe margins, snapping, mask drawing; **no transform handles** (#639); Button Editor (+) does nothing (#431) | 55% | #639, #431 | 15–25 h | +| Effect Controls | property rows, keyframe area with zoom, divider, graphs, eyedroppers | rows, keyframes, value / velocity graphs, eyedropper (#518); missing keyframe zoom (#641, #645), draggable divider (#643), row lines (#640), Position curves (#448) | 60% | #412, #640–#645, #448 | 15–25 h | +| Panels and docking | 17 workspaces, dock / undock / float, drop zones | 11 workspaces, docking; rearranging windows reported missing (#493); multi-monitor floating windows (#466) | 60% | #493, #466 | 20–35 h | +| Project panel | List / Icon / Freeform, hover scrub, marquee | all three views, hover scrub, view presets; no marquee (#578), Freeform bins and pan (#579), bins can't be left on Windows (#587), duplicate imports allowed (#356) | 75% | #578, #579, #587, #456 | 10–20 h | +| Timeline display | filmstrip thumbnails, waveforms, track heights, renaming | clip thumbnails, waveforms, heights; no continuous filmstrip (#614), no track rename (#654), wheel scroll in audio area (#400) | 70% | #614, #654, #400 | 8–15 h | +| Transitions | drag, resize, alignment, edit in Effect Controls | drag / select / resize (#503); transition controls (#577) | 70% | #577 | 5–10 h | +| Numeric entry and scrubbing | scrubbable hot text, typed values | scrubbable values, timecode scrub-edit (#576); audio clip mixer values cannot be typed (#481) | 80% | #481 | 3–6 h | +| Drag and drop | OS files into Project / Timeline, clipboard paste | OS drops into the timeline (#517); no clipboard paste of images / files (#611) | 70% | #519, #611 | 5–10 h | +| Theme and appearance | dark / light, brightness | dark / light / system (#505, #555) | 85% | macOS window controls (#637), Wayland title bar (#433), X11 scale (#457) | 5–10 h | +| Context menus | per panel and per item | most panels; check against Premiere not done | 70% | — | 5–10 h | +| Accessibility | screen reader partial, keyboard navigation | keyboard navigation; every widget has an automation id; no screen-reader support | 30% | — | 15–30 h | + +## Revision history + +| Date | Change | Summary | +|---|---|---| +| 2026-10-10 | major | Created: UI areas against Premiere 26.5.2, menu label diff, keyboard parity from keyboard.md, open UI issues itemized | diff --git a/packaging/macos/Info.plist.in b/packaging/macos/Info.plist.in index 78958249..e9d9dc3b 100644 --- a/packaging/macos/Info.plist.in +++ b/packaging/macos/Info.plist.in @@ -42,6 +42,10 @@ NSRequiresAquaSystemAppearance + + NSMicrophoneUsageDescription + FilmCraft uses the microphone to record voice-over onto the timeline. NSHumanReadableCopyright Copyright © the FilmCraft authors. MIT OR Apache-2.0. diff --git a/packaging/macos/entitlements.plist b/packaging/macos/entitlements.plist index 79fa2237..74540670 100644 --- a/packaging/macos/entitlements.plist +++ b/packaging/macos/entitlements.plist @@ -1,9 +1,13 @@ - + + com.apple.security.device.audio-input + diff --git a/xtask/src/main.rs b/xtask/src/main.rs index 631678d8..648f217b 100644 --- a/xtask/src/main.rs +++ b/xtask/src/main.rs @@ -504,3 +504,24 @@ fn main() -> ExitCode { } } } + +#[cfg(test)] +mod tests { + use std::path::Path; + + fn macos_packaging(file: &str) -> String { + let path = Path::new(env!("CARGO_MANIFEST_DIR")).join("../packaging/macos").join(file); + std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{}: {e}", path.display())) + } + + /// Voice-over recording needs both halves on macOS: the usage string (else the permission + /// prompt never appears) and the hardened-runtime audio-input entitlement (else the input is + /// blocked). Regression test for #659. + #[test] + fn macos_bundle_may_use_the_microphone() { + let plist = macos_packaging("Info.plist.in"); + assert!(plist.contains("NSMicrophoneUsageDescription")); + let entitlements = macos_packaging("entitlements.plist"); + assert!(entitlements.contains("com.apple.security.device.audio-input\n ")); + } +}