shifathosn
/
filmcraft
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

M10.7: graphics templates (.fcgt), rolls/crawls, responsive design, per-character styles, MCC/STL/TTML/DFXP captions 

- Text engine: per-character style runs (layout_rich), per-run underline/colour index.
- Project: graphic_design (style runs, pins, roll/crawl maths, responsive time, source graphics),
  gtemplate (.fcgt format, validation, base64 resources, 8 original built-in templates; .mogrt refused).
- Render: item-level graphic evaluation (responsive time, pins, rolls), per-run fills.
- Engine: graphics.template.* (list/apply/export/install/remove/set/controls/thumbnail),
  graphics.setRoll/setResponsiveTime/pin/setCharStyle, Upgrade Caption to Graphic,
  Upgrade to Source Graphic (shared edits), Replace Fonts in Projects, fonts used.
- Captions: MacCaption MCC (CDP, CEA-608 + CEA-708), EBU STL (Tech 3264), TTML/IMSC1, DFXP.
- Schema v12 (no-op migration + fixture), docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Thomas 2026-10-02 22:52:15 -04:00
parent 2323218446
commit ee9fd53079
43 changed files with 5746 additions and 137 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
ATTRIBUTION.md
View File

@ -97,6 +97,7 @@ is traced or derived from Adobe artwork.
| UI icons (tools, transport, panels, header) | `crates/ui-egui/src/icons.rs`: vector paths on a 16×16 grid |
| Align / distribute / paragraph-alignment glyphs in the graphics panel | `crates/ui-egui/src/panels/graphics.rs` (`align_glyph`): bars and lines drawn with the egui painter |
| Demo footage (ocean sunset, aurora, city night, dunes, forest, plasma), bars and tone, counting leader, colour matte | `crates/media/src/generators.rs` |
| Built-in graphics templates (Lower Third – Slab / Rule, Ticker – Crawl, Title – Centered / Boxed, End Credits – Roll, Callout – Pointer / Tag) and their placeholder text | `crates/project/src/gtemplate.rs` (`builtin_templates`): original designs defined in code with the bundled Inter font; thumbnails are rendered at runtime, no template, image or font files |
| Lumetri "Look" presets (Teal & Orange, Warm Film, …) | `crates/render/src/effects.rs` (`apply_look`): procedural colour transforms, not LUT files |
| Colour palette and layout metrics | `crates/ui-egui/src/theme.rs`: colour values and sizes only; no artwork |

1
Cargo.lock generated
View File

@ -1734,6 +1734,7 @@ dependencies = [
"filmcraft-text",
"filmcraft-time",
"proptest",
"roxmltree",
"thiserror 2.0.21",
]

2
ROADMAP.md
View File

@ -67,7 +67,7 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining
| M7 | Audio | 🟡 | Mixer graph (tracks → submixes → Mix, pre/post-fader inserts and sends, latency-compensated, sample-accurate, ~6× realtime for 24 tracks × 3 effects on one core), Audio Track Mixer + Audio Clip Mixer panels, track automation (Off/Read/Latch/Touch/Write, recorded live while playing, thinned to keyframes, timeline lanes with pen editing), solo/solo-safe, channel mapping basics, peak + BS.1770 loudness meters (match ffmpeg), DSP crate with 16 clip/track effects, Audio Gain (set/adjust/normalize), Constant Power / Constant Gain / Exponential Fade, Essential Sound (types, Loudness auto-match, Repair incl. DeEss/DeReverb, Clarity, Creative, Ducking keyframes, presets; full Dialogue chain 22× realtime) | Music duration remix, ML speech enhancement, 5.1 panner and multichannel buses, voice-over record, effect editor windows (EQ curve), remaining effects (multiband, convolution reverb), clip-mixer automation recording | 6–9 |
| M8 | Colour | 🟡 | Lumetri: basic, creative + looks, RGB & hue curves, wheels, HSL secondary, vignette, section bypass; Input LUT / Look LUT (.cube 1D/3D/shaper, .3dl; tetrahedral CPU + WGSL; project LUT library; built-in camera conversions); Colour Match (Oklab tonal-range statistics, skin protection, solved in Lumetri wheels); colour management: Rec. 709 / Rec. 2100 PQ / HLG working spaces + wide gamut, Interpret Footage colour space (S-Log3, V-Log, Canon Log 2/3, LogC3/4, Apple Log, D-Log from published specs), metadata auto-detect, BT.2390 tone mapping, gamut mapping, HDR export signalling (VUI/colr/mdcv/clli/SEI, ffprobe-verified); Lumetri Scopes: Vectorscope YUV (75/100 % targets, skin-tone line) and HLS, Histogram, Parade (RGB/YUV/RGB-White), Waveform (RGB/Luma/YC/YC no Chroma), presets, Rec. 601/709/2100, 8-bit/float/HDR (cd/m²) scales, multi-scope grid, `scopes.read` | HDR-aware Lumetri maths, macOS EDR monitors, mastering metadata → tone-map peak, D-Log M (no published formula), HSL Secondary refine | 3–5 |
| M9 | More codecs | 🟡 | ProRes decode+encode, AAC decode+encode, HEVC Main/Main 10 decoder (bit-exact on 41 fixtures, ~225 fps 1080p), VP9 decoder (profiles 0–3, 8/10/12-bit, bit-exact on 50+ fixtures; WebM/MKV `V_VP9` and MP4 `vp09` import with key-frame-checked seeking), Matroska/WebM import (H.264/HEVC/VP9/ProRes/MJPEG + AAC/Opus/FLAC/MP3/Vorbis/PCM), Opus decoder (SILK/CELT/hybrid, 5.1/7.1 multistream; all RFC 8251 vectors range-exact; WebM/MKV/MP4), DNxHD/DNxHR decoder (all SMPTE ST 2019-1 CIDs, 8/10/12-bit, 4:2:2/4:4:4, interlaced, alpha; within IDCT precision of ffmpeg on 22 fixtures) + DNxHR LB/SQ/HQ/HQX/444 encoder and MOV `AVdh` export, AV1 decoder (Main profile, 8/10-bit, all intra/inter tools, loop filters, superres, film grain, intra BC, spatial layers; bit-exact vs libdav1d on 19 SVT fixtures + 22 libaom vectors; MP4 `av01` / WebM `V_AV1` import), MXF import (OP1a/OP-Atom; AVC with POC-ordered B pictures bit-exact, DNxHD/DNxHR, ProRes, PCM/AES3; MPEG-2 reported unsupported; index-table seeking, material package timecode), Ogg Opus/Vorbis (granule seeking, pre-skip, end trimming), image sequences, Broadcast WAV timecode | VP9 frame threading, AV1 threading/SIMD (~4 fps 1080p today), AV1 High/Professional profiles, MPEG-2 decoder, hardware decode | 12–20 |
| M10 | Graphics & captions | 🟡 | Caption tracks (Subtitle/CEA-608/708/Teletext formats, track style), SRT/WebVTT/SCC import+export (frame-exact, property-tested), caption editing (add/split/merge/trim/move, sync-locked insert/extract), Text panel Captions tab, burn-in in Program monitor and export; text engine (`crates/text`: bundled + system fonts, harfrust shaping, bidi, line breaking, paragraph layout, glyph cache; 3-line 1080p title ≈ 0.15 ms warm); graphic clips with text + shape layers (fill, 2 strokes, background, shadow, keyframable transform), Type tool with on-monitor editing, shape/pen tools, Properties/Essential Graphics editor, align/distribute | Responsive design pins, rolls/crawls, per-character styles, motion graphics templates, MCC/STL/TTML, 608/708 embedding, speech-to-text | 12–18 |
| M10 | Graphics & captions | 🟡 | Caption tracks (Subtitle/CEA-608/708/Teletext formats, track style), SRT/WebVTT/SCC import+export (frame-exact, property-tested), caption editing (add/split/merge/trim/move, sync-locked insert/extract), Text panel Captions tab, burn-in in Program monitor and export; text engine (`crates/text`: bundled + system fonts, harfrust shaping, bidi, line breaking, paragraph layout, glyph cache; 3-line 1080p title ≈ 0.15 ms warm); graphic clips with text + shape layers (fill, 2 strokes, background, shadow, keyframable transform), Type tool with on-monitor editing, shape/pen tools, Properties/Essential Graphics editor, align/distribute; graphics templates (own `.fcgt` format, 8 original built-ins, Browse tab with engine-rendered thumbnails, export / install / apply / editable properties; never reads .mogrt), rolls/crawls with ease and pre/postroll, responsive pins and intro/outro, per-character styles, Upgrade Caption to Graphic, source graphics, Replace Fonts; MCC (608+708), EBU STL, TTML/IMSC1, DFXP (frame-exact, property-tested) | 608/708 embedding in video streams, speech-to-text | 12–18 |
| M11 | Interchange & project management | ✅ | `.fcproj` schema versions + migrations, atomic saves, Save a Copy/Revert, auto-save ring + crash-recovery journal (Preferences ▸ Auto Save, recovery prompt), FCP7 XML, FCPXML, EDL, OTIO; offline media (own slate) + Link Media (fingerprint-checked relink, folder remap, search, Align Timecode, Make Offline); proxies (ProRes Proxy/LT, H.264 ¼/½ background jobs, attach/detach/reconnect, monitor toggle, export full-res) + ingest (copy/transcode/proxies); Project Manager (collect, consolidate + transcode with handles, size estimate) | Rename media to clip names, image-sequence conversion, Media Browser-driven relink, smart (cross-drive) path tracking | 1–2 |
| M12–M16 | Multicam, web (WASM), platform, long tail | 🟡 | Multicam: audio sync (GCC-PHAT, sub-sample), Merge Clips, multi-camera source sequences, Multi-Camera view with live switching (1–9); L0–L4 crates compile to wasm32 | Web app shell (file access, WebCodecs, audio), scene detection, auto reframe, ~800 remaining commands and dialogs, performance hardening | 35–55 |

3
crates/captions/Cargo.toml
View File

@ -1,6 +1,6 @@
[package]
name = "filmcraft-captions"
description = "FilmCraft captions: SRT, WebVTT and SCC (CEA-608 pop-on) import/export, caption layout and burn-in rasterisation"
description = "FilmCraft captions: SRT, WebVTT, SCC, MCC, EBU STL and TTML/IMSC1/DFXP import/export, caption layout and burn-in rasterisation"
version.workspace = true
edition.workspace = true
license.workspace = true
@ -15,6 +15,7 @@ filmcraft-color = { workspace = true }
filmcraft-project = { workspace = true }
filmcraft-text = { workspace = true }
thiserror = { workspace = true }
roxmltree = { workspace = true }
[dev-dependencies]
proptest = { workspace = true }

18
crates/captions/README.md
View File

@ -1,7 +1,7 @@
# filmcraft-captions
Caption files and caption burn-in for FilmCraft (layer L2: depends on `filmcraft-time`,
`filmcraft-color`, `filmcraft-project` and `filmcraft-text`).
`filmcraft-color`, `filmcraft-project`, `filmcraft-text` and `roxmltree` for TTML).
The caption *model* (caption tracks in a sequence, captions, track style) lives in
`filmcraft-project` (`caption.rs`) because it is part of the `.fcproj` document. This crate reads
@ -14,6 +14,10 @@ and writes caption files, converts them to and from caption tracks, and draws ca
| SubRip `.srt` | yes | yes | BOM, UTF-16, Windows-1252 fallback, CRLF/CR, missing or wrong indexes, `,` or `.` milliseconds, 1–9 fraction digits, missing hours, SSA coordinates, cues not separated by blank lines. Inline tags are kept as text. |
| WebVTT `.vtt` | yes | yes | Cue identifiers, cue settings (verbatim), a leading `<v Speaker>` voice span (as the caption's speaker) and STYLE / REGION / NOTE blocks are kept. A missing `WEBVTT` signature is tolerated. |
| Scenarist SCC `.scc` | yes | yes | CEA-608 at 29.97 fps, drop-frame or not. Reads pop-on, roll-up and paint-on (channel 1); writes pop-on. |
| MacCaption MCC `.mcc` | yes | yes | One SMPTE ST 334-2 caption distribution packet (CDP) per frame in MacCaption's hex text with letter abbreviations. Writes 29.97 fps (`30DF` / `30`) with CEA-608 field 1 (the SCC schedule) **and** CEA-708 service 1 (two-window pop-on emulation). Reads CEA-608 field 1 when present, else CEA-708 service 1; time code rates 30DF, 30, 60DF, 60, 25, 24, 50. |
| EBU STL `.stl` | yes | yes | EBU Tech 3264-E: GSI block + 128-byte TTI blocks, `STL25.01` (25 fps) or `STL30.01` (read/written as 29.97 fps, non-drop labels), ISO/IEC 6937 Latin text (accents as diacritic + letter), italics / underline codes as `<i>` / `<u>`, extension blocks for long subtitles, start-of-programme (TCP) subtracted. |
| TTML `.ttml` (IMSC1) | yes | yes | Writes the IMSC1 Text profile (TTML2 namespaces): `<p>` per cue, `<br/>`, styled `<span>`s for `<i>` / `<b>` / `<u>`, `ttm:agent` speakers, exact frame times (`123f`) at the sequence rate. Reads any TTML (TTML1, TTML2, the 2006 `ttaf1` drafts): clock / frame / offset / tick times, `dur`, nested time containers, timed spans, referenced styles. |
| DFXP `.dfxp` | yes | yes | TTML1 with the DFXP presentation profile; clock times in milliseconds (frame-exact after `snap_to_frames`). |
All cue times are exact `Tick`s. Milliseconds convert exactly (1 ms = 254 016 000 ticks); SCC
frames map through `FrameRate::FPS_29_97`. `Document::snap_to_frames` snaps cues to a sequence's
@ -43,10 +47,18 @@ Text is set in **Inter SemiBold** (`assets/fonts/Inter-SemiBold.ttf`, SIL OFL 1.
`ATTRIBUTION.md`) by the `filmcraft-text` engine: shaped with kerning and ligatures, bidi-ordered,
and drawn from its sub-pixel positioned glyph cache.
Specifications used: SMPTE ST 334-2 (CDP structure), CEA-608 / CEA-708 (public descriptions of
the caption commands), the MacCaption MCC format description (line syntax and abbreviations),
EBU Tech 3264-E (1991), ISO/IEC 6937, W3C TTML1 (2nd ed.), TTML2 and IMSC 1.1. No other
implementation's code was consulted.
## Tests
- Unit tests per module (timestamps, decoding, sloppy SRT, WebVTT blocks and voices, CEA-608
tables, PAC round trips, known SCC pop-on and roll-up streams, layout).
- `tests/roundtrip.rs`: property tests — SRT (also with BOM + CRLF), WebVTT (ids, speakers,
settings, STYLE blocks) and SCC (drop-frame and non-drop, gaps and back-to-back captions)
round-trip exactly; readers never panic on random bytes; SRT → VTT → SCC keeps text and frames.
settings, STYLE blocks), SCC and MCC (drop-frame and non-drop, gaps and back-to-back captions;
MCC also through CEA-708 alone), EBU STL (25 and 29.97 fps, accents, extension blocks), TTML
(23.976 / 25 / 29.97 / 59.94, ids, speakers) round-trip frame-exactly; DFXP round-trips after
snapping to frames; readers never panic on random bytes (also a valid STL header followed by
random blocks, and random MCC lines); SRT → VTT → SCC keeps text and frames.

80
crates/captions/src/lib.rs
View File

@ -1,8 +1,10 @@
//! FilmCraft captions.
//!
//! - **Files:** [SubRip](srt) (`.srt`), [WebVTT](vtt) (`.vtt`) and [Scenarist SCC](scc) (`.scc`,
//! CEA-608 pop-on, roll-up and paint-on on read; pop-on on write) parse into a [`Document`] of
//! [`Cue`]s and write back. Readers are forgiving (BOM, UTF-16, Windows-1252 fallback, CRLF/CR,
//! - **Files:** [SubRip](srt) (`.srt`), [WebVTT](vtt) (`.vtt`), [Scenarist SCC](scc) (`.scc`,
//! CEA-608 pop-on, roll-up and paint-on on read; pop-on on write), [MacCaption MCC](mcc)
//! (`.mcc`, SMPTE ST 334-2 packets with CEA-608 and CEA-708), [EBU STL](stl) (`.stl`, EBU
//! Tech 3264) and [TTML](ttml) (`.ttml` IMSC1 Text profile, `.dfxp` DFXP) parse into a
//! [`Document`] of [`Cue`]s and write back. Readers are forgiving (BOM, UTF-16, Windows-1252 fallback, CRLF/CR,
//! missing SRT indexes, `,` or `.` milliseconds, missing hours, no blank line between cues);
//! writers are strict. WebVTT cue identifiers, cue settings, `<v Speaker>` voices and
//! STYLE/REGION/NOTE blocks are kept.
@ -15,8 +17,11 @@
pub mod burn;
pub mod cea608;
pub mod mcc;
pub mod scc;
pub mod srt;
pub mod stl;
pub mod ttml;
pub mod vtt;
use filmcraft_project::{Caption, CaptionFormat, CaptionTrack, ClipId, TrackId};
@ -28,15 +33,27 @@ pub enum Format {
Srt,
WebVtt,
Scc,
/// MacCaption MCC (CEA-608 + CEA-708 in SMPTE ST 334-2 packets).
Mcc,
/// EBU STL (EBU Tech 3264).
Stl,
/// TTML, IMSC1 Text profile.
Ttml,
/// DFXP (TTML1).
Dfxp,
}
impl Format {
pub const ALL: [Format; 3] = [Format::Srt, Format::WebVtt, Format::Scc];
pub const ALL: [Format; 7] = [Format::Srt, Format::WebVtt, Format::Scc, Format::Mcc, Format::Stl, Format::Ttml, Format::Dfxp];
pub fn name(self) -> &'static str {
match self {
Format::Srt => "SubRip (SRT)",
Format::WebVtt => "WebVTT",
Format::Scc => "Scenarist SCC (CEA-608)",
Format::Mcc => "MacCaption MCC (CEA-608/708)",
Format::Stl => "EBU STL",
Format::Ttml => "TTML (IMSC1)",
Format::Dfxp => "DFXP (TTML1)",
}
}
pub fn extension(self) -> &'static str {
@ -44,6 +61,10 @@ impl Format {
Format::Srt => "srt",
Format::WebVtt => "vtt",
Format::Scc => "scc",
Format::Mcc => "mcc",
Format::Stl => "stl",
Format::Ttml => "ttml",
Format::Dfxp => "dfxp",
}
}
pub fn from_name(s: &str) -> Option<Format> {
@ -51,6 +72,10 @@ impl Format {
"srt" | "subrip" => Some(Format::Srt),
"vtt" | "webvtt" => Some(Format::WebVtt),
"scc" | "cea608" | "cea-608" => Some(Format::Scc),
"mcc" | "maccaption" | "cea708" | "cea-708" => Some(Format::Mcc),
"stl" | "ebu-stl" | "ebustl" | "ebu" => Some(Format::Stl),
"ttml" | "imsc" | "imsc1" | "xml" => Some(Format::Ttml),
"dfxp" => Some(Format::Dfxp),
_ => None,
}
}
@ -58,9 +83,15 @@ impl Format {
pub fn track_format(self) -> CaptionFormat {
match self {
Format::Scc => CaptionFormat::Cea608,
Format::Mcc => CaptionFormat::Cea708,
Format::Stl => CaptionFormat::Teletext,
_ => CaptionFormat::Subtitle,
}
}
/// Whether the format is a binary file (EBU STL).
pub fn is_binary(self) -> bool {
self == Format::Stl
}
}
#[derive(Debug, Clone, PartialEq, thiserror::Error)]
@ -138,6 +169,9 @@ impl Document {
/// Guess a caption format from content (and the file extension as a hint).
pub fn detect(bytes: &[u8], ext: Option<&str>) -> Option<Format> {
if stl::sniff(bytes) {
return Some(Format::Stl);
}
let text = decode_text(&bytes[..bytes.len().min(4096)]);
let head = text.trim_start_matches('\u{feff}').trim_start();
if head.starts_with("WEBVTT") {
@ -146,12 +180,21 @@ pub fn detect(bytes: &[u8], ext: Option<&str>) -> Option<Format> {
if head.starts_with("Scenarist_SCC") {
return Some(Format::Scc);
}
if head.starts_with("File Format=MacCaption_MCC") {
return Some(Format::Mcc);
}
let ext = ext.map(|e| e.trim_start_matches('.').to_ascii_lowercase());
if (head.starts_with('<') && (head.contains("<tt ") || head.contains("<tt>") || head.contains(":tt ")))
&& (head.contains("ns/ttml") || head.contains("ttaf1"))
{
let legacy = head.contains("ttaf1") || head.contains("dfxp");
return Some(if ext.as_deref() == Some("dfxp") || (legacy && ext.as_deref() != Some("ttml")) { Format::Dfxp } else { Format::Ttml });
}
let has_arrow_timing = head.lines().take(40).any(|l| srt::parse_timing_line(l).is_some());
match ext.as_deref() {
Some("srt") if has_arrow_timing || head.is_empty() => Some(Format::Srt),
Some("vtt") if has_arrow_timing => Some(Format::WebVtt),
Some("srt" | "vtt" | "scc") => None,
Some("srt" | "vtt" | "scc" | "mcc" | "stl" | "ttml" | "dfxp" | "xml") => None,
_ if has_arrow_timing => Some(Format::Srt),
_ => None,
}
@ -159,24 +202,33 @@ pub fn detect(bytes: &[u8], ext: Option<&str>) -> Option<Format> {
/// Parse caption bytes.
pub fn parse(bytes: &[u8], format: Format) -> Result<Document> {
if format == Format::Stl {
return stl::parse(bytes);
}
let text = decode_text(bytes);
match format {
Format::Srt => srt::parse(&text),
Format::WebVtt => vtt::parse(&text),
Format::Scc => scc::parse(&text),
Format::Mcc => mcc::parse(&text),
Format::Ttml | Format::Dfxp => ttml::parse(&text),
Format::Stl => unreachable!("handled above"),
}
}
/// Options for [`write`].
#[derive(Clone, Copy, Debug)]
pub struct WriteOptions {
/// SCC: drop-frame timecode (the usual choice for 29.97 fps SCC).
/// SCC / MCC: drop-frame timecode (the usual choice for 29.97 fps).
pub drop_frame: bool,
/// The sequence frame rate: TTML (IMSC1) writes exact frame times at it; EBU STL picks
/// `STL25.01` or `STL30.01` from it (None = 25 fps). Other formats ignore it.
pub rate: Option<FrameRate>,
}
impl Default for WriteOptions {
fn default() -> Self {
Self { drop_frame: true }
Self { drop_frame: true, rate: None }
}
}
@ -186,6 +238,10 @@ pub fn write(doc: &Document, format: Format, opts: WriteOptions) -> Vec<u8> {
Format::Srt => srt::write(doc).into_bytes(),
Format::WebVtt => vtt::write(doc).into_bytes(),
Format::Scc => scc::write(doc, opts.drop_frame).into_bytes(),
Format::Mcc => mcc::write(doc, opts.drop_frame).into_bytes(),
Format::Stl => stl::write(doc, opts.rate),
Format::Ttml => ttml::write(doc, ttml::Flavor::Imsc1, opts.rate, "en").into_bytes(),
Format::Dfxp => ttml::write(doc, ttml::Flavor::Dfxp, opts.rate, "en").into_bytes(),
}
}
@ -326,6 +382,16 @@ mod tests {
assert_eq!(detect(b"Scenarist_SCC V1.0\n\n00:00:00;00\t9420\n", Some("scc")), Some(Format::Scc));
assert_eq!(detect(b"hello", Some("srt")), None);
assert_eq!(detect(b"<xml/>", None), None);
assert_eq!(detect(b"File Format=MacCaption_MCC V1.0\n", Some("mcc")), Some(Format::Mcc));
let ttml = b"<?xml version=\"1.0\"?><tt xmlns=\"http://www.w3.org/ns/ttml\"><body/></tt>";
assert_eq!(detect(ttml, Some("ttml")), Some(Format::Ttml));
assert_eq!(detect(ttml, Some("dfxp")), Some(Format::Dfxp));
assert_eq!(detect(b"<tt xmlns=\"http://www.w3.org/2006/10/ttaf1\"></tt>", Some("xml")), Some(Format::Dfxp));
let stl = stl::write(&Document::default(), None);
assert_eq!(detect(&stl, Some("stl")), Some(Format::Stl));
for f in Format::ALL {
assert_eq!(Format::from_name(f.extension()), Some(f));
}
}
#[test]

694
crates/captions/src/mcc.rs Normal file
View File

@ -0,0 +1,694 @@
//! MacCaption MCC (`.mcc`): closed-caption data as text, one line per video frame, each line a
//! SMPTE ST 334-2 caption distribution packet (CDP) carrying CEA-608 byte pairs and CEA-708
//! (DTVCC) packets, written in hexadecimal with MacCaption's letter abbreviations.
//!
//! ```text
//! File Format=MacCaption_MCC V1.0
//! …
//! Time Code Rate=30DF
//!
//! 00:00:01;00 T49S494F43ZZ72F4FC9420RFE...G7400Z4C
//! ```
//!
//! - **Line:** `timecode<TAB>data`; data is ancillary DID/SDID `61 01`, a byte count, then the
//! CDP. Letters stand for byte runs: `G`–`O` = 1–9 × `FA 00 00`, `P` = `FB 80 80`,
//! `Q` = `FC 80 80`, `R` = `FD 80 80`, `S` = `96 69`, `T` = `61 01`, `U` = `E1 00 00 00`,
//! `Z` = `00`.
//! - **CDP (SMPTE ST 334-2):** identifier `96 69`, length, frame-rate code, flags, sequence
//! counter, a `cc_data` section (`72`, `E0 | cc_count`, cc_count × (marker/valid/type, two
//! bytes)), optional time code / service info sections, footer `74`, the counter again and a
//! checksum that makes the packet's byte sum 0 mod 256.
//! - **Writing** (29.97 fps, `30DF` or `30` timecode labels): every frame from the first caption
//! data to one frame after the last carries a CDP with 20 cc_data triplets: field 1 CEA-608
//! pop-on data (the same schedule as [SCC](crate::scc)), a null field-2 pair, CEA-708
//! service 1 data and padding. The CEA-708 captions emulate pop-on with two windows: each
//! caption is defined (hidden) and loaded into one window while the other shows, then a
//! DisplayWindows / HideWindows pair switches them on the caption's first frame; HideWindows
//! clears it at its end.
//! - **Reading:** every CDP's `cc_data`; channel 1 CEA-608 field-1 data is decoded with the SCC
//! decoder when present, otherwise CEA-708 service 1 (windows, text, CR, clear / display /
//! hide / toggle / delete windows, DefineWindow visibility); a cue starts whenever the visible
//! text changes. `Time Code Rate` 30DF / 30 → 29.97 fps, 60DF / 60 → 59.94, 24, 25, 50.
use std::collections::BTreeMap;
use filmcraft_time::{FrameRate, fields_to_frames, frames_to_fields};
use crate::cea608::with_parity;
use crate::{Cue, Document, Error, Result, scc};
/// MCC files written by FilmCraft are 29.97 fps.
pub const RATE: FrameRate = FrameRate::FPS_29_97;
const CC_COUNT: usize = 20;
/// cc_data triplets per frame left for CEA-708 (after the two CEA-608 fields).
const DTVCC_PAIRS: usize = CC_COUNT - 2;
// ---------------------------------------------------------------------------------------------
// Hex with MacCaption abbreviations
// ---------------------------------------------------------------------------------------------
const ABBREV: &[(char, &[u8])] = &[
('P', &[0xfb, 0x80, 0x80]),
('Q', &[0xfc, 0x80, 0x80]),
('R', &[0xfd, 0x80, 0x80]),
('S', &[0x96, 0x69]),
('T', &[0x61, 0x01]),
('U', &[0xe1, 0x00, 0x00, 0x00]),
('Z', &[0x00]),
];
/// Bytes → MCC text (uppercase hex with abbreviations).
pub fn compress(b: &[u8]) -> String {
let mut out = String::new();
let mut i = 0;
'outer: while i < b.len() {
let mut n = 0;
while n < 9 && b[i + 3 * n..].starts_with(&[0xfa, 0x00, 0x00]) {
n += 1;
}
if n > 0 {
out.push((b'G' + n as u8 - 1) as char);
i += 3 * n;
continue;
}
for (c, seq) in ABBREV {
if b[i..].starts_with(seq) {
out.push(*c);
i += seq.len();
continue 'outer;
}
}
out.push_str(&format!("{:02X}", b[i]));
i += 1;
}
out
}
/// MCC text → bytes (None if malformed).
pub fn expand(s: &str) -> Option<Vec<u8>> {
let mut out = Vec::new();
let mut hex: Option<u8> = None;
for c in s.chars().filter(|c| !c.is_whitespace()) {
if let Some(d) = c.to_digit(16) {
match hex.take() {
Some(h) => out.push(h << 4 | d as u8),
None => hex = Some(d as u8),
}
continue;
}
if hex.is_some() {
return None;
}
let c = c.to_ascii_uppercase();
match c {
'G'..='O' => {
for _ in 0..(c as u8 - b'G' + 1) {
out.extend_from_slice(&[0xfa, 0x00, 0x00]);
}
}
_ => out.extend_from_slice(ABBREV.iter().find(|a| a.0 == c)?.1),
}
}
if hex.is_some() { None } else { Some(out) }
}
// ---------------------------------------------------------------------------------------------
// CDP
// ---------------------------------------------------------------------------------------------
/// One frame's caption data: a field-1 CEA-608 pair (without parity) and DTVCC pairs
/// `(start_of_packet, b1, b2)`.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct FrameData {
pub f1: Option<(u8, u8)>,
pub dtvcc: Vec<(bool, u8, u8)>,
}
fn rate_code(r: FrameRate) -> u8 {
match r {
FrameRate::FPS_23_976 => 1,
FrameRate::FPS_24 => 2,
FrameRate::FPS_25 => 3,
FrameRate::FPS_29_97 => 4,
FrameRate::FPS_30 => 5,
FrameRate::FPS_50 => 6,
FrameRate::FPS_59_94 => 7,
FrameRate::FPS_60 => 8,
_ => 4,
}
}
/// A CDP for one 29.97 fps frame, prefixed with the ancillary DID/SDID (`61 01`) and count.
pub fn build_cdp(seq: u16, data: &FrameData) -> Vec<u8> {
let mut cc: Vec<u8> = Vec::with_capacity(CC_COUNT * 3);
match data.f1 {
Some((a, b)) => cc.extend_from_slice(&[0xfc, with_parity(a), with_parity(b)]),
None => cc.extend_from_slice(&[0xfc, 0x80, 0x80]),
}
cc.extend_from_slice(&[0xfd, 0x80, 0x80]);
for &(start, a, b) in data.dtvcc.iter().take(DTVCC_PAIRS) {
cc.extend_from_slice(&[if start { 0xff } else { 0xfe }, a, b]);
}
while cc.len() < CC_COUNT * 3 {
cc.extend_from_slice(&[0xfa, 0x00, 0x00]);
}
let len = 7 + 2 + cc.len() + 4;
let [s1, s0] = seq.to_be_bytes();
let mut cdp = vec![0x96, 0x69, len as u8, rate_code(RATE) << 4 | 0x0f, 0x43, s1, s0, 0x72, 0xe0 | CC_COUNT as u8];
cdp.extend_from_slice(&cc);
cdp.extend_from_slice(&[0x74, s1, s0]);
let sum: u32 = cdp.iter().map(|&b| b as u32).sum();
cdp.push(((256 - sum % 256) % 256) as u8);
let mut out = vec![0x61, 0x01, cdp.len() as u8];
out.extend(cdp);
out
}
/// The cc_data triplets of one line's bytes (an ancillary packet with a CDP, or a bare CDP).
pub fn cc_triplets(b: &[u8]) -> Option<Vec<[u8; 3]>> {
let cdp = if b.starts_with(&[0x61, 0x01]) { b.get(3..)? } else { b };
if !cdp.starts_with(&[0x96, 0x69]) {
return None;
}
let len = (*cdp.get(2)? as usize).min(cdp.len());
let cdp = &cdp[..len];
let mut i = 7;
let mut out = Vec::new();
while i < cdp.len() {
match cdp[i] {
0x71 => i += 5,
0x72 => {
let n = (*cdp.get(i + 1)? & 0x1f) as usize;
for k in 0..n {
let at = i + 2 + 3 * k;
out.push([*cdp.get(at)?, *cdp.get(at + 1)?, *cdp.get(at + 2)?]);
}
i += 2 + 3 * n;
}
0x73 => i += 2 + 7 * (*cdp.get(i + 1)? & 0x0f) as usize,
0x74 => break,
0x75..=0xef => i += 2 + *cdp.get(i + 1)? as usize,
_ => return None,
}
}
Some(out)
}
// ---------------------------------------------------------------------------------------------
// CEA-708 service data
// ---------------------------------------------------------------------------------------------
/// G2 characters we write / read (after EXT1).
const G2: &[(u8, char)] = &[
(0x25, '…'),
(0x2a, 'Š'),
(0x2c, 'Œ'),
(0x31, '‘'),
(0x32, '’'),
(0x33, '“'),
(0x34, '”'),
(0x35, '•'),
(0x39, '™'),
(0x3a, 'š'),
(0x3c, 'œ'),
(0x3d, '℠'),
(0x3f, 'Ÿ'),
(0x76, '⅛'),
(0x77, '⅜'),
(0x78, '⅝'),
(0x79, '⅞'),
];
/// Service-data atoms (a command with its parameters, or one character) for a character.
fn char_atom(c: char) -> Vec<u8> {
match c {
'♪' => vec![0x7f],
' '..='~' => vec![c as u8],
'\u{a0}'..='\u{ff}' => vec![c as u32 as u8],
_ => match G2.iter().find(|g| g.1 == c) {
Some((b, _)) => vec![0x10, *b],
None => vec![b'?'],
},
}
}
/// Atoms that define window `win` hidden and load `rows` into it.
fn load_atoms(win: u8, rows: &[String]) -> Vec<Vec<u8>> {
let rc = rows.len().clamp(1, 12) as u8 - 1;
let cols = rows.iter().map(|r| r.chars().count()).max().unwrap_or(1).clamp(1, 42) as u8 - 1;
let mut v = vec![
// DefineWindow: hidden, row/column lock, relative position 90 % down / 50 % across,
// anchored at its bottom centre, rows / columns, window style 1, pen style 1
vec![0x98 + win, 0x18, 0x80 | 90, 50, 7 << 4 | rc, cols, 1 << 3 | 1],
vec![0x88, 1 << win], // ClearWindows
];
for (i, r) in rows.iter().enumerate() {
if i > 0 {
v.push(vec![0x0d]);
}
v.extend(r.chars().map(char_atom));
}
v
}
/// Pack atoms into DTVCC packets for service 1 (each ≤ 34 bytes, so one fits in a frame).
fn packets(atoms: &[Vec<u8>], seq: &mut u8) -> Vec<Vec<u8>> {
let mut blocks: Vec<Vec<u8>> = Vec::new();
let mut cur: Vec<u8> = Vec::new();
for a in atoms {
if cur.len() + a.len() > 31 {
blocks.push(std::mem::take(&mut cur));
}
cur.extend_from_slice(a);
}
if !cur.is_empty() {
blocks.push(cur);
}
blocks
.into_iter()
.map(|b| {
let mut p = vec![0, 1 << 5 | b.len() as u8];
p.extend(b);
if p.len() % 2 == 1 {
p.push(0);
}
p[0] = (*seq & 3) << 6 | (p.len() / 2) as u8;
*seq = seq.wrapping_add(1);
p
})
.collect()
}
fn pairs_of(packet: &[u8]) -> Vec<(bool, u8, u8)> {
packet.chunks(2).enumerate().map(|(i, c)| (i == 0, c[0], *c.get(1).unwrap_or(&0))).collect()
}
/// Frames → DTVCC pairs, with per-frame capacity.
struct Dtv {
frames: BTreeMap<i64, Vec<(bool, u8, u8)>>,
}
impl Dtv {
fn room(&self, f: i64) -> usize {
DTVCC_PAIRS - self.frames.get(&f).map_or(0, Vec::len)
}
/// Put a packet in the first frame ≥ `f` with room; returns that frame.
fn place(&mut self, mut f: i64, packet: &[u8]) -> i64 {
let pairs = pairs_of(packet);
while self.room(f) < pairs.len() {
f += 1;
}
self.frames.entry(f).or_default().extend(pairs);
f
}
}
/// Schedule CEA-708 service 1 pop-on captions (see the module docs).
fn schedule_708(doc: &Document) -> BTreeMap<i64, Vec<(bool, u8, u8)>> {
let mut d = Dtv { frames: BTreeMap::new() };
let mut seq = 0u8;
let mut cursor = 0i64;
let mut cues: Vec<&Cue> = doc.cues.iter().filter(|c| c.end > c.start).collect();
cues.sort_by_key(|c| c.start);
for (i, c) in cues.iter().enumerate() {
let win = (i % 2) as u8;
let s = scc::frame_of(c.start).max(cursor);
let e = scc::frame_of(c.end);
let rows = scc::wrap_rows(&filmcraft_project::plain_text(&c.text));
let load = packets(&load_atoms(win, &rows), &mut seq);
let mut f = (s - load.len() as i64).max(cursor);
let mut last = f;
for p in &load {
last = d.place(f, p);
f = last;
}
let show = packets(&[vec![0x8a, 1 << (1 - win)], vec![0x89, 1 << win]], &mut seq);
let shown = d.place(s.max(last), &show[0]);
cursor = shown + 1;
let next_start = cues.get(i + 1).map(|n| scc::frame_of(n.start));
if next_start != Some(e) {
let hide = packets(&[vec![0x8a, 1 << win]], &mut seq);
d.place(e.max(cursor), &hide[0]);
}
}
d.frames
}
/// CEA-708 decoder state for service 1.
#[derive(Clone, Default)]
struct Win {
defined: bool,
visible: bool,
rows: Vec<String>,
}
struct Dec708 {
win: [Win; 8],
cur: usize,
packet: Vec<u8>,
want: usize,
open: Option<(i64, String)>,
cues: Vec<(i64, i64, String)>,
}
impl Dec708 {
fn new() -> Self {
Dec708 { win: Default::default(), cur: 0, packet: Vec::new(), want: 0, open: None, cues: Vec::new() }
}
fn visible_text(&self) -> String {
let mut lines = Vec::new();
for w in self.win.iter().filter(|w| w.defined && w.visible) {
lines.extend(w.rows.iter().map(|r| r.trim().to_string()).filter(|r| !r.is_empty()));
}
lines.join("\n")
}
fn sync(&mut self, f: i64) {
let now = self.visible_text();
if self.open.as_ref().map(|o| o.1.as_str()) == Some(now.as_str()) || (self.open.is_none() && now.is_empty()) {
return;
}
if let Some((s, t)) = self.open.take()
&& f > s
{
self.cues.push((s, f, t));
}
if !now.is_empty() {
self.open = Some((f, now));
}
}
fn pair(&mut self, f: i64, start: bool, a: u8, b: u8) {
if start {
self.packet.clear();
let size = (a & 0x3f) as usize;
self.want = if size == 0 { 128 } else { size * 2 };
} else if self.want == 0 {
return;
}
self.packet.extend_from_slice(&[a, b]);
if self.packet.len() >= self.want {
let p = std::mem::take(&mut self.packet);
self.want = 0;
self.service_blocks(&p[1..]);
self.sync(f);
}
}
fn service_blocks(&mut self, mut b: &[u8]) {
while let Some(&h) = b.first() {
let (svc, len) = (h >> 5, (h & 0x1f) as usize);
if h == 0 {
break;
}
let (svc, skip) = if svc == 7 { (b.get(1).map_or(0, |x| x & 0x3f), 2) } else { (svc, 1) };
let data = b.get(skip..skip + len).unwrap_or(&[]);
if svc == 1 {
self.service_data(data);
}
b = b.get(skip + len..).unwrap_or(&[]);
}
}
fn w(&mut self) -> &mut Win {
&mut self.win[self.cur]
}
fn put(&mut self, c: char) {
let w = self.w();
if w.rows.is_empty() {
w.rows.push(String::new());
}
w.rows.last_mut().expect("row").push(c);
}
fn bitmap(&mut self, m: u8, f: impl Fn(&mut Win)) {
for (i, w) in self.win.iter_mut().enumerate() {
if m & (1 << i) != 0 {
f(w);
}
}
}
fn service_data(&mut self, d: &[u8]) {
let mut i = 0;
let arg = |k: usize| d.get(k).copied().unwrap_or(0);
while i < d.len() {
let c = d[i];
i += 1;
match c {
0x03 | 0x00 => {}
0x08 => {
if let Some(r) = self.w().rows.last_mut() {
r.pop();
}
}
0x0c => self.w().rows = vec![String::new()],
0x0d => self.w().rows.push(String::new()),
0x0e => {
if let Some(r) = self.w().rows.last_mut() {
r.clear();
}
}
0x10 => {
let x = arg(i);
i += 1;
match x {
0x00..=0x07 => {}
0x08..=0x0f => i += 1,
0x10..=0x17 => i += 2,
0x18..=0x1f => i += 3,
0x20 | 0x21 => self.put(' '),
0x20..=0x7f => {
if let Some((_, ch)) = G2.iter().find(|g| g.0 == x) {
self.put(*ch);
}
}
0x80..=0x87 => i += 4,
0x88..=0x8f => i += 5,
0x90..=0x9f => i += 1 + (arg(i) & 0x1f) as usize,
_ => {}
}
}
0x11..=0x17 => i += 1,
0x18..=0x1f => i += 2,
0x20..=0x7e => self.put(c as char),
0x7f => self.put('♪'),
0x80..=0x87 => self.cur = (c - 0x80) as usize,
0x88 => {
self.bitmap(arg(i), |w| w.rows = vec![String::new()]);
i += 1;
}
0x89 => {
self.bitmap(arg(i), |w| w.visible = true);
i += 1;
}
0x8a => {
self.bitmap(arg(i), |w| w.visible = false);
i += 1;
}
0x8b => {
self.bitmap(arg(i), |w| w.visible = !w.visible);
i += 1;
}
0x8c => {
self.bitmap(arg(i), |w| *w = Win::default());
i += 1;
}
0x8d => i += 1,
0x8e | 0x8f | 0x93..=0x96 => {}
0x90 | 0x92 => i += 2,
0x91 => i += 3,
0x97 => i += 4,
0x98..=0x9f => {
let n = (c - 0x98) as usize;
let vis = arg(i) & 0x20 != 0;
let w = &mut self.win[n];
if !w.defined {
*w = Win { defined: true, visible: vis, rows: vec![String::new()] };
} else {
w.visible = vis;
}
self.cur = n;
i += 6;
}
0xa0..=0xff => self.put(c as char),
_ => {}
}
}
}
}
// ---------------------------------------------------------------------------------------------
// File
// ---------------------------------------------------------------------------------------------
/// Write `doc` as MCC: CEA-608 field 1 and CEA-708 service 1 (see the module docs).
pub fn write(doc: &Document, drop_frame: bool) -> String {
write_with(doc, drop_frame, true, true)
}
/// [`write`] choosing which caption streams to include.
pub fn write_with(doc: &Document, drop_frame: bool, cea608: bool, cea708: bool) -> String {
let f1 = if cea608 { scc::schedule(doc) } else { BTreeMap::new() };
let dtv = if cea708 { schedule_708(doc) } else { BTreeMap::new() };
let mut out = String::from("File Format=MacCaption_MCC V1.0\n\n");
out.push_str("///////////////////////////////////////////////////////////////////////////////////\n");
out.push_str("// Closed captions written by FilmCraft: SMPTE ST 334-2 caption distribution packets\n");
out.push_str("// carrying CEA-608 field 1 and CEA-708 service 1 data, one packet per frame.\n");
out.push_str("///////////////////////////////////////////////////////////////////////////////////\n\n");
let n = doc.cues.len() as u64;
let h = doc.cues.iter().fold(0xcbf2_9ce4_8422_2325u64 ^ n, |h, c| (h ^ c.start.0 as u64 ^ (c.end.0 as u64).rotate_left(17)).wrapping_mul(0x100_0000_01b3));
out.push_str(&format!(
"UUID={:08X}-{:04X}-4{:03X}-8{:03X}-{:012X}\n",
h >> 32,
(h >> 16) & 0xffff,
h & 0xfff,
(h >> 20) & 0xfff,
h.rotate_left(29) & 0xffff_ffff_ffff
));
out.push_str("Creation Program=FilmCraft\n");
out.push_str(&format!("Time Code Rate={}\n\n", if drop_frame { "30DF" } else { "30" }));
let first = f1.keys().next().copied().into_iter().chain(dtv.keys().next().copied()).min();
let last = f1.keys().next_back().copied().into_iter().chain(dtv.keys().next_back().copied()).max();
if let (Some(a), Some(b)) = (first, last) {
for (seq, f) in (a..=b + 1).enumerate() {
let data = FrameData { f1: f1.get(&f).copied(), dtvcc: dtv.get(&f).cloned().unwrap_or_default() };
let (_, hh, mm, ss, ff) = frames_to_fields(f, RATE, drop_frame);
let sep = if drop_frame { ';' } else { ':' };
out.push_str(&format!("{hh:02}:{mm:02}:{ss:02}{sep}{ff:02}\t{}\n", compress(&build_cdp(seq as u16, &data))));
}
}
out
}
fn tc_rate(v: &str) -> Option<(FrameRate, bool)> {
Some(match v.trim().to_ascii_uppercase().as_str() {
"30DF" => (FrameRate::FPS_29_97, true),
"30" => (FrameRate::FPS_29_97, false),
"60DF" => (FrameRate::FPS_59_94, true),
"60" => (FrameRate::FPS_59_94, false),
"24" => (FrameRate::FPS_24, false),
"25" => (FrameRate::FPS_25, false),
"50" => (FrameRate::FPS_50, false),
_ => return None,
})
}
pub fn parse(text: &str) -> Result<Document> {
let mut lines = text.lines().enumerate();
match lines.by_ref().find(|(_, l)| !l.trim().is_empty()) {
Some((_, l)) if l.trim().starts_with("File Format=MacCaption_MCC") => {}
_ => return Err(Error::NotFormat("MacCaption MCC")),
}
let mut rate = (RATE, true);
let mut doc = Document::default();
let mut f1: Vec<(i64, u8, u8)> = Vec::new();
let mut dtv: Vec<(i64, bool, u8, u8)> = Vec::new();
let mut any608 = false;
for (n, l) in lines {
let l = l.trim_end();
if l.trim().is_empty() || l.starts_with("//") {
continue;
}
if let Some(v) = l.strip_prefix("Time Code Rate=") {
rate = tc_rate(v).ok_or_else(|| Error::Syntax { line: n + 1, msg: format!("unsupported time code rate \"{v}\"") })?;
continue;
}
if l.contains('=') {
continue; // other key=value header lines
}
let Some((tc, data)) = l.trim_start().split_once(['\t', ' ']) else {
continue;
};
let tc = tc.split('.').next().unwrap_or(tc);
let parts: Vec<i64> = tc.split([':', ';']).filter_map(|p| p.parse().ok()).collect();
let [h, m, s, f] = parts[..] else {
return Err(Error::Syntax { line: n + 1, msg: format!("bad timecode \"{tc}\"") });
};
let frame = fields_to_frames(h, m, s, f, rate.0, rate.1 || tc.contains(';'));
let Some(bytes) = expand(data) else {
doc.warnings.push(format!("line {}: bad caption data", n + 1));
continue;
};
let Some(trips) = cc_triplets(&bytes) else {
doc.warnings.push(format!("line {}: not a caption distribution packet", n + 1));
continue;
};
for [m0, a, b] in trips {
if m0 & 0x04 == 0 {
continue;
}
match m0 & 0x03 {
0 => {
any608 |= a & 0x7f != 0 || b & 0x7f != 0;
f1.push((frame, a, b));
}
1 => {}
t => dtv.push((frame, t == 3, a, b)),
}
}
}
if any608 {
doc.cues = scc::decode_pairs(f1, rate.0);
} else {
let mut d = Dec708::new();
let mut last = 0;
for (f, start, a, b) in dtv {
d.pair(f, start, a, b);
last = f + 1;
}
if let Some((s, t)) = d.open.take() {
d.cues.push((s, last.max(s + 1), t));
}
doc.cues = d.cues.into_iter().map(|(s, e, text)| Cue { start: rate.0.tick_of(s), end: rate.0.tick_of(e), text, ..Default::default() }).collect();
}
Ok(doc)
}
#[cfg(test)]
mod tests {
use super::*;
fn cue(s: i64, e: i64, text: &str) -> Cue {
Cue { start: RATE.tick_of(s), end: RATE.tick_of(e), text: text.into(), ..Default::default() }
}
#[test]
fn abbreviations_round_trip() {
let b = [0x61, 0x01, 0x49, 0x96, 0x69, 0xfa, 0, 0, 0xfa, 0, 0, 0xfc, 0x80, 0x80, 0x00, 0x12, 0xe1, 0, 0, 0, 0xfb, 0x80, 0x80, 0xfd, 0x80, 0x80];
let s = compress(&b);
assert_eq!(s, "T49SHQZ12UPR");
assert_eq!(expand(&s).unwrap(), b);
assert_eq!(expand("T52S5"), None);
}
#[test]
fn cdp_structure_and_checksum() {
let d = FrameData { f1: Some((0x14, 0x20)), dtvcc: vec![(true, 0x02, 0x21), (false, 0x89, 0x01)] };
let b = build_cdp(7, &d);
assert_eq!(&b[..3], &[0x61, 0x01, 73]);
let cdp = &b[3..];
assert_eq!(cdp.len(), 73);
assert_eq!(&cdp[..9], &[0x96, 0x69, 73, 0x4f, 0x43, 0, 7, 0x72, 0xf4]);
assert_eq!(cdp.iter().map(|&x| x as u32).sum::<u32>() % 256, 0, "checksum");
let t = cc_triplets(&b).unwrap();
assert_eq!(t.len(), 20);
assert_eq!(t[0], [0xfc, 0x94, 0x20]);
assert_eq!(t[2], [0xff, 0x02, 0x21]);
assert_eq!(t[19], [0xfa, 0, 0]);
}
#[test]
fn writes_and_reads_608_and_708() {
let doc = Document {
cues: vec![cue(60, 120, "Hello, world!"), cue(120, 200, "Second caption\nwith two lines"), cue(260, 300, "Café “q” ♪")],
..Default::default()
};
let s = write(&doc, true);
assert!(s.starts_with("File Format=MacCaption_MCC V1.0\n"));
assert!(s.contains("Time Code Rate=30DF"));
let back = parse(&s).unwrap();
let got: Vec<(i64, i64, &str)> = back.cues.iter().map(|c| (RATE.frame_at(c.start), RATE.frame_at(c.end), c.text.as_str())).collect();
assert_eq!(got, vec![(60, 120, "Hello, world!"), (120, 200, "Second caption\nwith two lines"), (260, 300, "Café “q” ♪")]);
// 708 only
let s708 = write_with(&doc, false, false, true);
let back = parse(&s708).unwrap();
let got: Vec<(i64, i64, &str)> = back.cues.iter().map(|c| (RATE.frame_at(c.start), RATE.frame_at(c.end), c.text.as_str())).collect();
assert_eq!(got, vec![(60, 120, "Hello, world!"), (120, 200, "Second caption\nwith two lines"), (260, 300, "Café “q” ♪")]);
assert!(parse("garbage").is_err());
}
}

32
crates/captions/src/scc.rs
View File

@ -242,6 +242,24 @@ impl Decoder {
}
}
/// Decode channel-1 CEA-608 byte pairs `(frame, b1, b2)` (with or without parity; frames
/// ascending) into cues at `rate`. A caption still showing after the last pair stays up for one
/// second more.
pub fn decode_pairs(pairs: impl IntoIterator<Item = (i64, u8, u8)>, rate: FrameRate) -> Vec<Cue> {
let mut dec = Decoder::new();
let mut last_frame = 0;
for (f, b1, b2) in pairs {
dec.pair(f, b1, b2);
last_frame = last_frame.max(f + 1);
}
let one_second = rate.frame_at(Tick(filmcraft_time::TICKS_PER_SECOND)).max(1);
if let Some((s, text)) = dec.open.take() {
let end = last_frame.max(s + 1) + one_second;
dec.cues.push((s, end, text));
}
dec.cues.into_iter().map(|(s, e, text)| Cue { start: rate.tick_of(s), end: rate.tick_of(e), text, ..Default::default() }).collect()
}
pub fn parse(text: &str) -> Result<Document> {
let mut lines = text.lines().enumerate().filter(|(_, l)| !l.trim().is_empty());
match lines.next() {
@ -369,7 +387,9 @@ fn word((a, b): (u8, u8)) -> u16 {
(cea608::with_parity(a) as u16) << 8 | cea608::with_parity(b) as u16
}
pub fn write(doc: &Document, drop_frame: bool) -> String {
/// Schedule pop-on CEA-608 byte pairs (without parity) for `doc`, one pair per 29.97 fps frame,
/// so each caption's End of Caption lands on its start frame (see the module docs).
pub fn schedule(doc: &Document) -> BTreeMap<i64, (u8, u8)> {
let mut occ: BTreeMap<i64, (u8, u8)> = BTreeMap::new();
let mut cursor = 0i64; // first frame this cue's load may use
let mut prev_end: Option<i64> = None;
@ -422,6 +442,16 @@ pub fn write(doc: &Document, drop_frame: bool) -> String {
let f = place_free_from(&mut occ, pe, edm);
occ.insert(f + 1, edm);
}
occ
}
/// The parity-coded SCC code word of a byte pair.
pub fn code_word(p: (u8, u8)) -> u16 {
word(p)
}
pub fn write(doc: &Document, drop_frame: bool) -> String {
let occ = schedule(doc);
let mut out = String::from(HEADER);
out.push_str("\n\n");
let mut line: Option<(i64, Vec<String>)> = None;

466
crates/captions/src/stl.rs Normal file
View File

@ -0,0 +1,466 @@
//! EBU STL (`.stl`): the EBU subtitling data exchange format, EBU Tech 3264-E (1991).
//!
//! A file is a 1024-byte General Subtitle Information (GSI) block of ASCII fields followed by
//! 128-byte Text and Timing Information (TTI) blocks. Each TTI block carries a subtitle number,
//! an extension block number (`0xFF` = last block of the subtitle), in / out timecodes (hours,
//! minutes, seconds, frames as binary bytes), vertical position, justification, a comment flag
//! and a 112-byte text field (`0x8A` = new line, `0x8F` = unused; `0x80`/`0x81` italics on/off,
//! `0x82`/`0x83` underline on/off; `0x00`–`0x1F` teletext control codes are skipped on read).
//!
//! - **Frame rates:** `STL25.01` is 25 fps; `STL30.01` is read and written as 29.97 fps with
//! non-drop-frame timecode labels (how it is used in practice). Timecodes are made relative to
//! the start-of-programme timecode (TCP).
//! - **Text:** character code table `00` (Latin, ISO/IEC 6937): ASCII, the 0xA0–0xFF specials and
//! accented letters as a non-spacing diacritic (0xC1–0xCF) followed by the base letter. Other
//! tables are read as Latin. Characters with no 6937 code are written as `?`.
//! - Long subtitles continue in extension blocks; user-data (`EBN 0xFE`) and comment blocks are
//! skipped.
use filmcraft_time::{FrameRate, Tick, fields_to_frames, frames_to_fields};
use crate::{Cue, Document, Error, Result};
const GSI: usize = 1024;
const TTI: usize = 128;
const TF: usize = 112;
/// Read the frame rate from the Disk Format Code.
fn rate_of(dfc: &[u8]) -> Option<FrameRate> {
match dfc {
b"STL25.01" => Some(FrameRate::FPS_25),
b"STL30.01" => Some(FrameRate::FPS_29_97),
_ => None,
}
}
/// Whether bytes look like an EBU STL file.
pub fn sniff(bytes: &[u8]) -> bool {
bytes.len() >= GSI && bytes[3..11].starts_with(b"STL") && rate_of(&bytes[3..11]).is_some()
}
// ---------------------------------------------------------------------------------------------
// ISO/IEC 6937
// ---------------------------------------------------------------------------------------------
const SPECIALS: &[(u8, char)] = &[
(0xa0, '\u{a0}'),
(0xa1, '¡'),
(0xa2, '¢'),
(0xa3, '£'),
(0xa4, '$'),
(0xa5, '¥'),
(0xa6, '#'),
(0xa7, '§'),
(0xa8, '¤'),
(0xa9, '‘'),
(0xaa, '“'),
(0xab, '«'),
(0xac, '←'),
(0xad, '↑'),
(0xae, '→'),
(0xaf, '↓'),
(0xb0, '°'),
(0xb1, '±'),
(0xb2, '²'),
(0xb3, '³'),
(0xb4, '×'),
(0xb5, 'µ'),
(0xb6, '¶'),
(0xb7, '·'),
(0xb8, '÷'),
(0xb9, '’'),
(0xba, '”'),
(0xbb, '»'),
(0xbc, '¼'),
(0xbd, '½'),
(0xbe, '¾'),
(0xbf, '¿'),
(0xd0, '―'),
(0xd1, '¹'),
(0xd2, '®'),
(0xd3, '©'),
(0xd4, '™'),
(0xd5, '♪'),
(0xd6, '¬'),
(0xd7, '¦'),
(0xdc, '⅛'),
(0xdd, '⅜'),
(0xde, '⅝'),
(0xdf, '⅞'),
(0xe0, 'Ω'),
(0xe1, 'Æ'),
(0xe2, 'Đ'),
(0xe3, 'ª'),
(0xe4, 'Ħ'),
(0xe6, 'IJ'),
(0xe7, 'Ŀ'),
(0xe8, 'Ł'),
(0xe9, 'Ø'),
(0xea, 'Œ'),
(0xeb, 'º'),
(0xec, 'Þ'),
(0xed, 'Ŧ'),
(0xee, 'Ŋ'),
(0xef, 'ʼn'),
(0xf0, 'ĸ'),
(0xf1, 'æ'),
(0xf2, 'đ'),
(0xf3, 'ð'),
(0xf4, 'ħ'),
(0xf5, 'ı'),
(0xf6, 'ij'),
(0xf7, 'ŀ'),
(0xf8, 'ł'),
(0xf9, 'ø'),
(0xfa, 'œ'),
(0xfb, 'ß'),
(0xfc, 'þ'),
(0xfd, 'ŧ'),
(0xfe, 'ŋ'),
];
/// Non-spacing diacritics 0xC1–0xCF: (code, base letters, composed letters).
const DIACRITICS: &[(u8, &str, &str)] = &[
(0xc1, "AEIOUaeiou", "ÀÈÌÒÙàèìòù"),
(0xc2, "ACEILNORSUYZacegilnorsuyz", "ÁĆÉÍĹŃÓŔŚÚÝŹáćéģíĺńóŕśúýź"),
(0xc3, "ACEGHIJOSUWYaceghijosuwy", "ÂĈÊĜĤÎĴÔŜÛŴŶâĉêĝĥîĵôŝûŵŷ"),
(0xc4, "AINOUainou", "ÃĨÑÕŨãĩñõũ"),
(0xc5, "AEIOUaeiou", "ĀĒĪŌŪāēīōū"),
(0xc6, "AGUagu", "ĂĞŬăğŭ"),
(0xc7, "CEGIZcegz", "ĊĖĠİŻċėġż"),
(0xc8, "AEIOUYaeiouy", "ÄËÏÖÜŸäëïöüÿ"),
(0xca, "AUau", "ÅŮåů"),
(0xcb, "CGKLNRSTcklnrst", "ÇĢĶĻŅŖŞŢçķļņŗşţ"),
(0xcd, "OUou", "ŐŰőű"),
(0xce, "AEIUaeiu", "ĄĘĮŲąęįų"),
(0xcf, "CDELNRSTZcdelnrstz", "ČĎĚĽŇŘŠŤŽčďěľňřšťž"),
];
fn compose(d: u8, base: u8) -> Option<char> {
let (_, bases, comp) = DIACRITICS.iter().find(|x| x.0 == d)?;
let i = bases.bytes().position(|b| b == base)?;
comp.chars().nth(i)
}
/// ISO 6937 bytes for a character (None = not representable).
pub fn encode_6937(c: char) -> Option<Vec<u8>> {
if (' '..='~').contains(&c) {
return Some(vec![c as u8]);
}
if let Some((b, _)) = SPECIALS.iter().find(|x| x.1 == c && x.0 != 0xa4 && x.0 != 0xa6) {
return Some(vec![*b]);
}
for (d, bases, comp) in DIACRITICS {
if let Some(i) = comp.chars().position(|x| x == c) {
return Some(vec![*d, bases.as_bytes()[i]]);
}
}
None
}
/// Decode a text field (up to the first unused byte) into caption text with `<i>` / `<u>` tags.
fn decode_tf(tf: &[u8], out: &mut String, italic: &mut bool, underline: &mut bool) {
let mut i = 0;
while i < tf.len() {
let b = tf[i];
i += 1;
match b {
0x8f => {}
0x8a => {
// a line break; repeated 0x8A (double-height rows) count once
if !out.ends_with('\n') || out.is_empty() {
out.push('\n');
}
}
0x80 if !*italic => {
out.push_str("<i>");
*italic = true;
}
0x81 if *italic => {
out.push_str("</i>");
*italic = false;
}
0x82 if !*underline => {
out.push_str("<u>");
*underline = true;
}
0x83 if *underline => {
out.push_str("</u>");
*underline = false;
}
0x00..=0x1f | 0x7f..=0x9f => {}
0x20..=0x7e => out.push(b as char),
0xc1..=0xcf => {
if let Some(&base) = tf.get(i)
&& let Some(c) = compose(b, base)
{
out.push(c);
i += 1;
}
}
_ => {
if let Some((_, c)) = SPECIALS.iter().find(|x| x.0 == b) {
out.push(*c);
}
}
}
}
}
/// Caption text → text-field bytes (`<i>`, `<u>` become control codes; other tags dropped).
fn encode_text(text: &str) -> Vec<u8> {
let mut out = Vec::new();
let mut rest = text;
while let Some(ch) = rest.chars().next() {
if ch == '<'
&& let Some(end) = rest.find('>')
{
match rest[1..end].trim().to_ascii_lowercase().as_str() {
"i" => out.push(0x80),
"/i" => out.push(0x81),
"u" => out.push(0x82),
"/u" => out.push(0x83),
_ => {}
}
rest = &rest[end + 1..];
continue;
}
if ch == '\n' {
out.push(0x8a);
} else {
out.extend(encode_6937(ch).unwrap_or_else(|| vec![b'?']));
}
rest = &rest[ch.len_utf8()..];
}
out
}
// ---------------------------------------------------------------------------------------------
// Reader
// ---------------------------------------------------------------------------------------------
fn ascii_field(b: &[u8]) -> String {
String::from_utf8_lossy(b).trim().to_string()
}
fn tc_frames(b: &[u8], rate: FrameRate) -> i64 {
fields_to_frames(b[0] as i64, b[1] as i64, b[2] as i64, b[3] as i64, rate, false)
}
pub fn parse(bytes: &[u8]) -> Result<Document> {
if !sniff(bytes) {
return Err(Error::NotFormat("EBU STL"));
}
let rate = rate_of(&bytes[3..11]).expect("sniffed");
let gsi = &bytes[..GSI];
// start of programme (HHMMSSFF, ASCII)
let tcp = ascii_field(&gsi[256..264]);
let start = if tcp.len() == 8 && tcp.bytes().all(|b| b.is_ascii_digit()) {
let n = |i: usize| tcp[i..i + 2].parse::<i64>().unwrap_or(0);
fields_to_frames(n(0), n(2), n(4), n(6), rate, false)
} else {
0
};
let mut doc = Document::default();
let mut cur: Option<(u16, i64, i64, String, bool, bool)> = None;
let finish = |c: Option<(u16, i64, i64, String, bool, bool)>, doc: &mut Document| {
if let Some((_, tin, tout, mut text, italic, underline)) = c {
if italic {
text.push_str("</i>");
}
if underline {
text.push_str("</u>");
}
let text = text.lines().map(str::trim).filter(|l| !l.is_empty()).collect::<Vec<_>>().join("\n");
if tout > tin && !text.is_empty() {
doc.cues.push(Cue { start: rate.tick_of((tin - start).max(0)), end: rate.tick_of((tout - start).max(0)), text, ..Default::default() });
}
}
};
for blk in bytes[GSI..].chunks_exact(TTI) {
let sn = u16::from_le_bytes([blk[1], blk[2]]);
let ebn = blk[3];
let cf = blk[15];
if ebn == 0xfe || cf != 0 {
continue; // user data / comment
}
let tin = tc_frames(&blk[5..9], rate);
let tout = tc_frames(&blk[9..13], rate);
let same = cur.as_ref().is_some_and(|c| c.0 == sn);
if !same {
finish(cur.take(), &mut doc);
cur = Some((sn, tin, tout, String::new(), false, false));
}
if let Some(c) = cur.as_mut() {
let (mut it, mut ul) = (c.4, c.5);
decode_tf(&blk[16..16 + TF], &mut c.3, &mut it, &mut ul);
c.4 = it;
c.5 = ul;
}
if ebn == 0xff {
finish(cur.take(), &mut doc);
}
}
finish(cur.take(), &mut doc);
if bytes.len() > GSI && !(bytes.len() - GSI).is_multiple_of(TTI) {
doc.warnings.push("trailing bytes after the last TTI block".into());
}
Ok(doc)
}
// ---------------------------------------------------------------------------------------------
// Writer
// ---------------------------------------------------------------------------------------------
fn put(buf: &mut [u8], at: usize, len: usize, s: &str) {
let b = s.as_bytes();
for i in 0..len {
buf[at + i] = *b.get(i).unwrap_or(&b' ');
}
}
/// Write `doc` at `rate` (25 fps → `STL25.01`; 29.97 / 30 → `STL30.01`; others are written at 25).
pub fn write(doc: &Document, rate: Option<FrameRate>) -> Vec<u8> {
let (rate, dfc) = match rate {
Some(r) if r == FrameRate::FPS_29_97 || r == FrameRate::FPS_30 => (FrameRate::FPS_29_97, "STL30.01"),
_ => (FrameRate::FPS_25, "STL25.01"),
};
// TTI blocks first (the GSI counts them)
let mut ttis: Vec<[u8; TTI]> = Vec::new();
let mut n_subs = 0u16;
let mut first_in = None;
for c in doc.cues.iter().filter(|c| c.end > c.start) {
let sn = n_subs;
n_subs = n_subs.wrapping_add(1);
let tin = rate.frame_at(rate.snap_nearest(c.start));
let tout = rate.frame_at(rate.snap_nearest(c.end)).max(tin + 1);
first_in.get_or_insert(tin);
let text = encode_text(&c.text);
let chunks: Vec<&[u8]> = if text.is_empty() { vec![&[][..]] } else { split_tf(&text) };
let lines = c.text.lines().count().clamp(1, 11);
for (k, chunk) in chunks.iter().enumerate() {
let mut b = [0u8; TTI];
b[0] = 0;
b[1..3].copy_from_slice(&sn.to_le_bytes());
b[3] = if k + 1 == chunks.len() { 0xff } else { k.min(0xfd) as u8 };
b[4] = 0;
for (at, f) in [(5, tin), (9, tout)] {
let (_, h, m, s, fr) = frames_to_fields(f, rate, false);
b[at..at + 4].copy_from_slice(&[h.min(255) as u8, m as u8, s as u8, fr as u8]);
}
b[13] = (22 - 2 * (lines - 1)) as u8;
b[14] = 2; // centred
b[15] = 0;
b[16..].fill(0x8f);
b[16..16 + chunk.len()].copy_from_slice(chunk);
ttis.push(b);
}
}
let mut gsi = [b' '; GSI];
put(&mut gsi, 0, 3, "850");
put(&mut gsi, 3, 8, dfc);
put(&mut gsi, 11, 1, "0");
put(&mut gsi, 12, 2, "00");
put(&mut gsi, 14, 2, "09");
put(&mut gsi, 16, 32, "FilmCraft");
put(&mut gsi, 224, 6, "000101");
put(&mut gsi, 230, 6, "000101");
put(&mut gsi, 236, 2, "00");
put(&mut gsi, 238, 5, &format!("{:05}", ttis.len().min(99_999)));
put(&mut gsi, 243, 5, &format!("{:05}", n_subs));
put(&mut gsi, 248, 3, "001");
put(&mut gsi, 251, 2, "40");
put(&mut gsi, 253, 2, "23");
put(&mut gsi, 255, 1, "1");
put(&mut gsi, 256, 8, "00000000");
let (_, h, m, s, f) = frames_to_fields(first_in.unwrap_or(0), rate, false);
put(&mut gsi, 264, 8, &format!("{h:02}{m:02}{s:02}{f:02}"));
put(&mut gsi, 272, 1, "1");
put(&mut gsi, 273, 1, "1");
let mut out = gsi.to_vec();
for t in ttis {
out.extend_from_slice(&t);
}
out
}
/// Split text-field bytes into ≤ 112-byte chunks without splitting a diacritic from its letter.
fn split_tf(text: &[u8]) -> Vec<&[u8]> {
let mut out = Vec::new();
let mut i = 0;
while i < text.len() {
let mut end = (i + TF).min(text.len());
if end < text.len() && (0xc1..=0xcf).contains(&text[end - 1]) {
end -= 1;
}
out.push(&text[i..end]);
i = end;
}
out
}
/// Frame of `t` on the STL grid of `rate` (see [`write`]).
pub fn frame_of(t: Tick, rate: FrameRate) -> i64 {
rate.frame_at(rate.snap_nearest(t))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn iso6937_round_trip() {
for c in "Ñandú café Škoda Łódź Ærø ß £ ½ ♪ « » ‘q’ “q”".chars() {
let b = encode_6937(c).unwrap_or_else(|| panic!("{c}"));
let mut s = String::new();
decode_tf(&b, &mut s, &mut false, &mut false);
assert_eq!(s, c.to_string());
}
assert_eq!(encode_6937('€'), None);
}
#[test]
fn writes_gsi_and_extension_blocks() {
let long = "x".repeat(150);
let doc = Document {
cues: vec![
Cue { start: FrameRate::FPS_25.tick_of(25), end: FrameRate::FPS_25.tick_of(75), text: "<i>Hello</i>\nWörld".into(), ..Default::default() },
Cue { start: FrameRate::FPS_25.tick_of(100), end: FrameRate::FPS_25.tick_of(150), text: long.clone(), ..Default::default() },
],
..Default::default()
};
let b = write(&doc, Some(FrameRate::FPS_25));
assert_eq!(&b[3..11], b"STL25.01");
assert_eq!(b.len(), GSI + 3 * TTI, "the long subtitle takes two blocks");
assert_eq!(&b[238..243], b"00003");
assert_eq!(&b[243..248], b"00002");
let tti0 = &b[GSI..GSI + TTI];
assert_eq!(&tti0[5..9], &[0, 0, 1, 0]);
assert_eq!(&tti0[9..13], &[0, 0, 3, 0]);
assert_eq!(tti0[3], 0xff);
assert_eq!(&tti0[16..24], &[0x80, b'H', b'e', b'l', b'l', b'o', 0x81, 0x8a]);
assert_eq!(b[GSI + TTI + 3], 0);
assert_eq!(b[GSI + 2 * TTI + 3], 0xff);
let back = parse(&b).unwrap();
assert_eq!(back.cues.len(), 2);
assert_eq!(back.cues[0].text, "<i>Hello</i>\nWörld");
assert_eq!(back.cues[1].text, long);
assert_eq!(back.cues[1].start, FrameRate::FPS_25.tick_of(100));
assert!(parse(&b[..500]).is_err());
}
#[test]
fn programme_start_is_subtracted() {
let doc = Document {
cues: vec![Cue { start: FrameRate::FPS_25.tick_of(250), end: FrameRate::FPS_25.tick_of(300), text: "A".into(), ..Default::default() }],
..Default::default()
};
let mut b = write(&doc, None);
// re-stamp as a programme starting at 10:00:00:00 with absolute in-cues
b[256..264].copy_from_slice(b"10000000");
b[GSI + 5] = 10;
b[GSI + 9] = 10;
let back = parse(&b).unwrap();
assert_eq!(back.cues[0].start, FrameRate::FPS_25.tick_of(250));
}
}

547
crates/captions/src/ttml.rs Normal file
View File

@ -0,0 +1,547 @@
//! TTML (W3C Timed Text Markup Language): IMSC1 Text profile (`.ttml`, written per TTML2 /
//! IMSC 1.1 Text) and DFXP (`.dfxp`, TTML1's Distribution Format Exchange Profile).
//!
//! **Writing.** One `<p>` per cue inside `body/div`, lines separated by `<br/>`; `<i>`, `<b>` and
//! `<u>` become `<span>`s with `tts:fontStyle="italic"`, `tts:fontWeight="bold"` and
//! `tts:textDecoration="underline"`. IMSC1 files signal `ttp:profile` (the IMSC1 Text profile
//! designator), a bottom region and a default style. With a frame rate, IMSC1 times are frame
//! offsets (`123f`) at `ttp:frameRate` / `ttp:frameRateMultiplier`, which are exact; without
//! one, and always for DFXP (for older players), times are clock times with milliseconds.
//!
//! **Reading** is namespace-agnostic (TTML1 2010, the 2006 `ttaf1` drafts and TTML2 all read):
//! clock times (`HH:MM:SS`, `.fraction`, `:FF` frames and `.subframes`), offset times (`h`, `m`,
//! `s`, `ms`, `f`, `t` with `ttp:tickRate`), `begin` / `end` / `dur`, nested time containers
//! (`body` / `div` begins add up), timed `<span>`s inside an untimed `<p>`, `<br/>`, whitespace
//! collapsing, styles referenced by id (italic / bold / underline) and `ttm:agent` / `xml:id`.
use filmcraft_time::{FrameRate, TICKS_PER_SECOND, Tick};
use crate::{Cue, Document, Error, Result, TICKS_PER_MS};
/// Which TTML flavour to write.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Flavor {
/// IMSC1 Text profile (TTML2 namespaces).
Imsc1,
/// DFXP (TTML1).
Dfxp,
}
const NS_TT: &str = "http://www.w3.org/ns/ttml";
const NS_TTP: &str = "http://www.w3.org/ns/ttml#parameter";
const NS_TTS: &str = "http://www.w3.org/ns/ttml#styling";
const NS_TTM: &str = "http://www.w3.org/ns/ttml#metadata";
const IMSC1_TEXT: &str = "http://www.w3.org/ns/ttml/profile/imsc1/text";
const DFXP_PROFILE: &str = "http://www.w3.org/ns/ttml/profile/dfxp-presentation";
fn esc(s: &str) -> String {
let mut o = String::with_capacity(s.len());
for c in s.chars() {
match c {
'&' => o.push_str("&amp;"),
'<' => o.push_str("&lt;"),
'>' => o.push_str("&gt;"),
'"' => o.push_str("&quot;"),
'\'' => o.push_str("&apos;"),
c => o.push(c),
}
}
o
}
fn clock(t: Tick) -> String {
let ms = (t.0.max(0) + TICKS_PER_MS / 2) / TICKS_PER_MS;
format!("{:02}:{:02}:{:02}.{:03}", ms / 3_600_000, (ms / 60_000) % 60, (ms / 1000) % 60, ms % 1000)
}
/// Time expression for `t`: frames at `rate` when it is on a frame boundary, else clock time.
fn time_expr(t: Tick, rate: Option<FrameRate>) -> String {
match rate {
Some(r) if r.tick_of(r.frame_at(t)) == t => format!("{}f", r.frame_at(t)),
_ => clock(t),
}
}
/// Caption text (`\n` lines, `<i>`/`<b>`/`<u>` tags) → TTML inline content.
fn inline(text: &str) -> String {
let mut out = String::new();
let mut open: Vec<&str> = Vec::new();
let mut rest = text;
while !rest.is_empty() {
if let Some(after) = rest.strip_prefix('<')
&& let Some(end) = after.find('>')
{
let tag = after[..end].trim().to_ascii_lowercase();
let (closing, name) = match tag.strip_prefix('/') {
Some(n) => (true, n.trim().to_string()),
None => (false, tag.split_whitespace().next().unwrap_or("").to_string()),
};
let attr = match name.as_str() {
"i" => Some("tts:fontStyle=\"italic\""),
"b" => Some("tts:fontWeight=\"bold\""),
"u" => Some("tts:textDecoration=\"underline\""),
_ => None,
};
if let Some(a) = attr {
if closing {
if let Some(pos) = open.iter().rposition(|x| *x == a) {
// close spans opened after it, then it, then reopen the inner ones
let inner: Vec<&str> = open.drain(pos..).skip(1).collect();
for _ in 0..=inner.len() {
out.push_str("</span>");
}
for x in inner {
out.push_str(&format!("<span {x}>"));
open.push(x);
}
}
} else {
out.push_str(&format!("<span {a}>"));
open.push(a);
}
}
rest = &after[end + 1..];
continue;
}
let ch = rest.chars().next().expect("non-empty");
if ch == '\n' {
out.push_str("<br/>");
} else {
out.push_str(&esc(&ch.to_string()));
}
rest = &rest[ch.len_utf8()..];
}
for _ in open {
out.push_str("</span>");
}
out
}
fn is_ncname(s: &str) -> bool {
let mut c = s.chars();
c.next().is_some_and(|f| f.is_alphabetic() || f == '_') && c.all(|x| x.is_alphanumeric() || matches!(x, '_' | '-' | '.'))
}
/// Write `doc` as TTML. `rate` (IMSC1 only) makes times exact frame offsets; `lang` is the
/// document language (`xml:lang`).
pub fn write(doc: &Document, flavor: Flavor, rate: Option<FrameRate>, lang: &str) -> String {
let mut o = String::from("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n");
let rate = if flavor == Flavor::Imsc1 { rate } else { None };
o.push_str(&format!("<tt xmlns=\"{NS_TT}\" xmlns:ttp=\"{NS_TTP}\" xmlns:tts=\"{NS_TTS}\" xmlns:ttm=\"{NS_TTM}\""));
o.push_str(&format!(" ttp:profile=\"{}\" ttp:timeBase=\"media\"", if flavor == Flavor::Imsc1 { IMSC1_TEXT } else { DFXP_PROFILE }));
if let Some(r) = rate {
let base = r.timecode_base();
o.push_str(&format!(" ttp:frameRate=\"{base}\""));
// effective rate = base × multiplier
let (mn, md) = (r.num, r.den * base);
let g = gcd(mn, md);
if (mn / g, md / g) != (1, 1) {
o.push_str(&format!(" ttp:frameRateMultiplier=\"{} {}\"", mn / g, md / g));
}
}
o.push_str(&format!(" xml:lang=\"{}\">\n", esc(if lang.is_empty() { "en" } else { lang })));
o.push_str(" <head>\n");
o.push_str(" <styling>\n <style xml:id=\"s1\" tts:color=\"white\" tts:fontFamily=\"proportionalSansSerif\" tts:fontSize=\"100%\" tts:textAlign=\"center\"/>\n </styling>\n");
o.push_str(" <layout>\n <region xml:id=\"r1\" tts:origin=\"10% 10%\" tts:extent=\"80% 80%\" tts:displayAlign=\"after\"/>\n </layout>\n");
o.push_str(" </head>\n <body region=\"r1\" style=\"s1\">\n <div>\n");
for (i, c) in doc.cues.iter().enumerate() {
let id = c.id.as_deref().filter(|x| is_ncname(x)).map(str::to_string).unwrap_or_else(|| format!("c{}", i + 1));
let agent = c.speaker.as_deref().map(|s| format!(" ttm:role=\"x-speaker\" ttm:agent=\"{}\"", esc(&agent_id(s)))).unwrap_or_default();
o.push_str(&format!(
" <p xml:id=\"{}\" begin=\"{}\" end=\"{}\"{agent}>{}</p>\n",
esc(&id),
time_expr(c.start, rate),
time_expr(c.end, rate),
inline(&c.text)
));
}
o.push_str(" </div>\n </body>\n</tt>\n");
o
}
/// `ttm:agent` refers to an agent id; we write the speaker name itself (made an NCName).
fn agent_id(s: &str) -> String {
let v: String = s.chars().map(|c| if c.is_alphanumeric() || c == '-' || c == '.' { c } else { '_' }).collect();
if v.chars().next().is_some_and(|c| c.is_alphabetic() || c == '_') { v } else { format!("_{v}") }
}
fn gcd(a: i64, b: i64) -> i64 {
if b == 0 { a.abs().max(1) } else { gcd(b, a % b) }
}
// ------------------------------------------------------------------------------------------------
// Reader
// ------------------------------------------------------------------------------------------------
/// Timing parameters from the root element.
#[derive(Clone, Copy)]
struct Timing {
/// Effective frame rate (frameRate × multiplier).
rate: FrameRate,
/// Nominal frames per second for `HH:MM:SS:FF`.
base: i64,
sub_frame_rate: i64,
tick_rate: i64,
}
fn parse_ratio(s: &str) -> Option<(i64, i64)> {
let mut it = s.split_whitespace().map(|x| x.parse::<i64>().ok());
let a = it.next()??;
let b = it.next().flatten().unwrap_or(1);
(a > 0 && b > 0).then_some((a, b))
}
fn attr<'a>(n: roxmltree::Node<'a, '_>, local: &str) -> Option<&'a str> {
n.attributes().find(|a| a.name() == local).map(|a| a.value())
}
fn timing(root: roxmltree::Node) -> Timing {
let base = attr(root, "frameRate").and_then(|v| v.trim().parse::<i64>().ok()).filter(|v| *v > 0).unwrap_or(30);
let (mn, md) = attr(root, "frameRateMultiplier").and_then(parse_ratio).unwrap_or((1, 1));
let rate = FrameRate::new(base * mn, md);
let sub = attr(root, "subFrameRate").and_then(|v| v.trim().parse::<i64>().ok()).filter(|v| *v > 0).unwrap_or(1);
let tick_rate = attr(root, "tickRate").and_then(|v| v.trim().parse::<i64>().ok()).filter(|v| *v > 0).unwrap_or(if attr(root, "frameRate").is_some() {
base * sub
} else {
1
});
Timing { rate, base, sub_frame_rate: sub, tick_rate }
}
/// Seconds as an exact decimal string → ticks.
fn decimal_ticks(int: &str, frac: &str, unit_ticks: i128) -> Option<i128> {
let i: i128 = if int.is_empty() { 0 } else { int.parse().ok()? };
let mut t = i * unit_ticks;
if !frac.is_empty() {
if frac.len() > 12 || !frac.bytes().all(|b| b.is_ascii_digit()) {
return None;
}
let f: i128 = frac.parse().ok()?;
t += f * unit_ticks / 10i128.pow(frac.len() as u32);
}
Some(t)
}
/// Parse a TTML time expression into ticks.
fn parse_time(s: &str, tm: Timing) -> Option<Tick> {
let s = s.trim();
if s.is_empty() {
return None;
}
let tps = TICKS_PER_SECOND as i128;
if s.contains(':') {
let parts: Vec<&str> = s.split(':').collect();
let num = |p: &str| -> Option<i128> { (!p.is_empty() && p.len() <= 6 && p.bytes().all(|b| b.is_ascii_digit())).then(|| p.parse().ok()).flatten() };
let (h, m) = (num(parts.first()?)?, num(parts.get(1)?)?);
let sec_part = parts.get(2)?;
let (si, sf) = sec_part.split_once('.').unwrap_or((sec_part, ""));
let mut t = (h * 3600 + m * 60) * tps + decimal_ticks(si, sf, tps)?;
if let Some(fp) = parts.get(3) {
if parts.len() > 4 {
return None;
}
let (fi, fs) = fp.split_once('.').unwrap_or((fp, ""));
let frames = num(fi)?;
let _ = tm.base;
let sub: i128 = if fs.is_empty() { 0 } else { num(fs)? };
// frames at the effective rate; sub-frames divide a frame
let ft = tm.rate.tick_of(1).0 as i128;
t += frames * ft + sub * ft / tm.sub_frame_rate.max(1) as i128;
}
return Some(Tick(t as i64));
}
let unit_start = s.find(|c: char| c.is_ascii_alphabetic())?;
let (v, unit) = s.split_at(unit_start);
let (vi, vf) = v.split_once('.').unwrap_or((v, ""));
if !vi.bytes().all(|b| b.is_ascii_digit()) || vi.len() > 15 {
return None;
}
let t = match unit {
"h" => decimal_ticks(vi, vf, tps * 3600)?,
"m" => decimal_ticks(vi, vf, tps * 60)?,
"s" => decimal_ticks(vi, vf, tps)?,
"ms" => decimal_ticks(vi, vf, tps / 1000)?,
"f" => {
let n: i128 = vi.parse().ok()?;
let fd = tm.rate.tick_of(1).0 as i128;
let whole = tm.rate.tick_of(n as i64).0 as i128;
whole + decimal_ticks("0", vf, fd)?
}
"t" => {
let n: i128 = vi.parse().ok()?;
n * tps / tm.tick_rate.max(1) as i128
}
_ => return None,
};
Some(Tick(t as i64))
}
#[derive(Clone, Copy, Default, PartialEq)]
struct Fmt {
italic: bool,
bold: bool,
underline: bool,
}
fn style_of(n: roxmltree::Node, styles: &std::collections::HashMap<String, Fmt>, parent: Fmt) -> Fmt {
let mut f = parent;
if let Some(refs) = attr(n, "style") {
for r in refs.split_whitespace() {
if let Some(s) = styles.get(r) {
f.italic |= s.italic;
f.bold |= s.bold;
f.underline |= s.underline;
}
}
}
own_fmt(n, &mut f);
f
}
fn own_fmt(n: roxmltree::Node, f: &mut Fmt) {
if let Some(v) = attr(n, "fontStyle") {
f.italic = v.trim() == "italic" || v.trim() == "oblique";
}
if let Some(v) = attr(n, "fontWeight") {
f.bold = v.trim() == "bold";
}
if let Some(v) = attr(n, "textDecoration") {
let v = v.trim();
if v.contains("noUnderline") || v == "none" {
f.underline = false;
} else if v.contains("underline") {
f.underline = true;
}
}
}
/// Text being collected from a `<p>`: whitespace collapses to one space, which is written only
/// before the next visible character or opening tag (never at a line start).
struct Out {
s: String,
pending: bool,
}
impl Out {
fn flush(&mut self) {
if self.pending && !self.s.is_empty() && !self.s.ends_with('\n') {
self.s.push(' ');
}
self.pending = false;
}
}
/// Collect a `<p>`'s (or `<span>`'s) text with `<i>`/`<b>`/`<u>` tags, collapsing whitespace.
fn collect(n: roxmltree::Node, styles: &std::collections::HashMap<String, Fmt>, fmt: Fmt, out: &mut Out) {
for c in n.children() {
if c.is_text() {
for ch in c.text().unwrap_or("").chars() {
if ch.is_whitespace() {
out.pending = true;
} else {
out.flush();
out.s.push(ch);
}
}
} else if c.is_element() {
match c.tag_name().name() {
"br" => {
out.pending = false;
out.s.push('\n');
}
"span" => {
let f = style_of(c, styles, fmt);
let tags: Vec<&str> = [(f.italic && !fmt.italic, "i"), (f.bold && !fmt.bold, "b"), (f.underline && !fmt.underline, "u")]
.into_iter()
.filter(|x| x.0)
.map(|x| x.1)
.collect();
if !tags.is_empty() {
out.flush();
}
for t in &tags {
out.s.push_str(&format!("<{t}>"));
}
collect(c, styles, f, out);
for t in tags.iter().rev() {
out.s.push_str(&format!("</{t}>"));
}
}
_ => collect(c, styles, fmt, out),
}
}
}
}
fn clean(s: &str) -> String {
s.lines().map(str::trim).collect::<Vec<_>>().join("\n").trim().to_string()
}
pub fn parse(text: &str) -> Result<Document> {
let xml = roxmltree::Document::parse(text).map_err(|_| Error::NotFormat("TTML"))?;
let root = xml.root_element();
if root.tag_name().name() != "tt" {
return Err(Error::NotFormat("TTML"));
}
let tm = timing(root);
let mut styles = std::collections::HashMap::new();
for s in root.descendants().filter(|n| n.is_element() && n.tag_name().name() == "style") {
if let Some(id) = s.attributes().find(|a| a.name() == "id").map(|a| a.value()) {
let mut f = Fmt::default();
own_fmt(s, &mut f);
styles.insert(id.to_string(), f);
}
}
let mut doc = Document::default();
let Some(body) = root.children().find(|n| n.is_element() && n.tag_name().name() == "body") else {
return Ok(doc);
};
walk(body, Tick::ZERO, &tm, &styles, Fmt::default(), &mut doc);
doc.cues.sort_by_key(|c| (c.start, c.end));
Ok(doc)
}
/// Begin and end of a timed element relative to `parent` (None when it has no timing).
fn times(n: roxmltree::Node, parent: Tick, tm: &Timing, doc: &mut Document) -> (Option<Tick>, Option<Tick>) {
let p = |k: &str, doc: &mut Document| -> Option<Tick> {
let v = attr(n, k)?;
let t = parse_time(v, *tm);
if t.is_none() {
doc.warnings.push(format!("bad time expression \"{v}\""));
}
t
};
let begin = p("begin", doc).map(|b| parent + b);
let end = match (p("end", doc), p("dur", doc)) {
(Some(e), _) => Some(parent + e),
(None, Some(d)) => Some(begin.unwrap_or(parent) + d),
_ => None,
};
(begin, end)
}
fn walk(n: roxmltree::Node, parent: Tick, tm: &Timing, styles: &std::collections::HashMap<String, Fmt>, fmt: Fmt, doc: &mut Document) {
for c in n.children().filter(|c| c.is_element()) {
let f = style_of(c, styles, fmt);
match c.tag_name().name() {
"div" => {
let (b, _) = times(c, parent, tm, doc);
walk(c, b.unwrap_or(parent), tm, styles, f, doc);
}
"p" => {
let (b, e) = times(c, parent, tm, doc);
let speaker = attr(c, "agent").map(|a| a.trim().trim_start_matches('_').replace('_', " "));
let id = c.attributes().find(|a| a.name() == "id").map(|a| a.value().to_string());
match (b, e) {
(Some(b), Some(e)) => {
let mut o = Out { s: String::new(), pending: false };
let tags: Vec<&str> = [(f.italic, "i"), (f.bold, "b"), (f.underline, "u")].into_iter().filter(|x| x.0).map(|x| x.1).collect();
for t in &tags {
o.s.push_str(&format!("<{t}>"));
}
collect(c, styles, f, &mut o);
for t in tags.iter().rev() {
o.s.push_str(&format!("</{t}>"));
}
let text = clean(&o.s);
if e > b && !text.is_empty() {
doc.cues.push(Cue { start: b, end: e, text, speaker, id, ..Default::default() });
}
}
_ => {
// untimed paragraph: its timed spans are the cues
let base = b.unwrap_or(parent);
for sp in c.children().filter(|x| x.is_element() && x.tag_name().name() == "span") {
let (sb, se) = times(sp, base, tm, doc);
if let (Some(sb), Some(se)) = (sb, se) {
let mut o = Out { s: String::new(), pending: false };
collect(sp, styles, style_of(sp, styles, f), &mut o);
let text = clean(&o.s);
if se > sb && !text.is_empty() {
doc.cues.push(Cue { start: sb, end: se, text, speaker: speaker.clone(), ..Default::default() });
}
}
}
}
}
}
_ => {}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn tm(rate: FrameRate) -> Timing {
Timing { rate, base: rate.timecode_base(), sub_frame_rate: 1, tick_rate: 1 }
}
#[test]
fn time_expressions() {
let t = tm(FrameRate::FPS_25);
let s = |v: f64| Tick((v * TICKS_PER_SECOND as f64).round() as i64);
assert_eq!(parse_time("00:00:01.500", t), Some(s(1.5)));
assert_eq!(parse_time("01:00:00", t), Some(s(3600.0)));
assert_eq!(parse_time("00:00:02:12", t), Some(s(2.48)));
assert_eq!(parse_time("2.5s", t), Some(s(2.5)));
assert_eq!(parse_time("1500ms", t), Some(s(1.5)));
assert_eq!(parse_time("0.5m", t), Some(s(30.0)));
assert_eq!(parse_time("50f", t), Some(s(2.0)));
let ntsc = tm(FrameRate::FPS_29_97);
assert_eq!(parse_time("30f", ntsc), Some(FrameRate::FPS_29_97.tick_of(30)));
let ticks = Timing { tick_rate: 10_000_000, ..t };
assert_eq!(parse_time("15000000t", ticks), Some(s(1.5)));
assert_eq!(parse_time("bogus", t), None);
assert_eq!(parse_time("1:2", t), None);
}
#[test]
fn reads_styles_spans_and_nesting() {
let x = r#"<?xml version="1.0"?>
<tt xmlns="http://www.w3.org/2006/10/ttaf1" xmlns:tts="http://www.w3.org/2006/10/ttaf1#styling" xmlns:ttp="http://www.w3.org/2006/10/ttaf1#parameter" ttp:frameRate="25">
<head><styling><style xml:id="it" tts:fontStyle="italic"/></styling></head>
<body>
<div begin="10s">
<p begin="00:00:01.000" end="00:00:02.000">Hello
<span style="it">big</span> world<br/>
second line</p>
<p begin="3s" dur="1s"><span tts:fontWeight="bold">Bold</span> &amp; plain</p>
</div>
<div><p><span begin="20s" end="21s">span timed</span></p></div>
</body>
</tt>"#;
let d = parse(x).unwrap();
let got: Vec<(f64, f64, &str)> = d.cues.iter().map(|c| (c.start.seconds(), c.end.seconds(), c.text.as_str())).collect();
assert_eq!(got, vec![(11.0, 12.0, "Hello <i>big</i> world\nsecond line"), (13.0, 14.0, "<b>Bold</b> & plain"), (20.0, 21.0, "span timed")]);
assert!(parse("<html/>").is_err());
assert!(parse("not xml").is_err());
}
#[test]
fn writes_imsc1_with_frames_and_spans() {
let r = FrameRate::FPS_29_97;
let doc = Document {
cues: vec![Cue {
start: r.tick_of(30),
end: r.tick_of(75),
text: "A <i>tilted</i> line\nand <b>more</b> & \"quotes\"".into(),
speaker: Some("Ann Lee".into()),
..Default::default()
}],
..Default::default()
};
let x = write(&doc, Flavor::Imsc1, Some(r), "en");
assert!(x.contains(IMSC1_TEXT) && x.contains("ttp:frameRate=\"30\"") && x.contains("ttp:frameRateMultiplier=\"1000 1001\""), "{x}");
assert!(x.contains("begin=\"30f\" end=\"75f\""), "{x}");
assert!(x.contains("<span tts:fontStyle=\"italic\">tilted</span>"), "{x}");
let back = parse(&x).unwrap();
assert_eq!(back.cues.len(), 1);
assert_eq!((back.cues[0].start, back.cues[0].end), (r.tick_of(30), r.tick_of(75)));
assert_eq!(back.cues[0].text, doc.cues[0].text);
assert_eq!(back.cues[0].speaker.as_deref(), Some("Ann Lee"));
let d = write(&doc, Flavor::Dfxp, Some(r), "en");
assert!(d.contains(DFXP_PROFILE) && d.contains("begin=\"00:00:01.001\""), "{d}");
}
}

125
crates/captions/tests/roundtrip.rs
View File

@ -2,7 +2,7 @@
//! documents each format can represent exactly.
use filmcraft_captions::{Cue, Document, Format, WriteOptions, parse, scc, write};
use filmcraft_time::{TICKS_PER_SECOND, Tick};
use filmcraft_time::{FrameRate, TICKS_PER_SECOND, Tick};
use proptest::prelude::*;
const MS: i64 = TICKS_PER_SECOND / 1000;
@ -93,10 +93,10 @@ proptest! {
})
.collect();
let doc = Document { cues, ..Default::default() };
let bytes = write(&doc, Format::Scc, WriteOptions { drop_frame });
let bytes = write(&doc, Format::Scc, WriteOptions { drop_frame, ..Default::default() });
let back = parse(&bytes, Format::Scc).unwrap();
prop_assert_eq!(&back.cues, &doc.cues);
prop_assert_eq!(write(&back, Format::Scc, WriteOptions { drop_frame }), bytes);
prop_assert_eq!(write(&back, Format::Scc, WriteOptions { drop_frame, ..Default::default() }), bytes);
}
/// Back-to-back SCC captions (no gap) still come back frame-exact when each lasts long enough
@ -117,6 +117,114 @@ proptest! {
prop_assert_eq!(back.cues, doc.cues);
}
/// MCC: frame-exact at 29.97 fps (CEA-608 path), like SCC.
#[test]
fn mcc_roundtrip(
v in proptest::collection::vec((60i64..400, 3i64..300, proptest::collection::vec("[A-Za-z0-9,.!?'()é-]{1,8}", 1..4)), 1..6),
drop_frame in any::<bool>(),
) {
let mut f = 0i64;
let cues: Vec<Cue> = v
.into_iter()
.map(|(gap, dur, words)| {
let start = f + gap;
f = start + dur;
Cue { start: scc::RATE.tick_of(start), end: scc::RATE.tick_of(f), text: words.join(" "), ..Default::default() }
})
.collect();
let doc = Document { cues, ..Default::default() };
let bytes = write(&doc, Format::Mcc, WriteOptions { drop_frame, ..Default::default() });
let back = parse(&bytes, Format::Mcc).unwrap();
prop_assert_eq!(&back.cues, &doc.cues);
prop_assert_eq!(write(&back, Format::Mcc, WriteOptions { drop_frame, ..Default::default() }), bytes);
}
/// MCC CEA-708 path alone is frame-exact too.
#[test]
fn mcc_708_roundtrip(v in proptest::collection::vec((60i64..400, 3i64..300, proptest::collection::vec("[A-Za-z0-9,.!?'()é-]{1,8}", 1..4)), 1..6)) {
let mut f = 0i64;
let cues: Vec<Cue> = v
.into_iter()
.map(|(gap, dur, words)| {
let start = f + gap;
f = start + dur;
Cue { start: scc::RATE.tick_of(start), end: scc::RATE.tick_of(f), text: words.join(" "), ..Default::default() }
})
.collect();
let doc = Document { cues, ..Default::default() };
let text = filmcraft_captions::mcc::write_with(&doc, true, false, true);
let back = parse(text.as_bytes(), Format::Mcc).unwrap();
prop_assert_eq!(&back.cues, &doc.cues);
}
/// EBU STL: frame-exact at 25 and 29.97 fps, multi-line text with accents, long subtitles in
/// extension blocks.
#[test]
fn stl_roundtrip(v in proptest::collection::vec((0i64..200, 1i64..300, text()), 0..10), ntsc in any::<bool>()) {
let rate = if ntsc { FrameRate::FPS_29_97 } else { FrameRate::FPS_25 };
let mut f = 0i64;
let cues: Vec<Cue> = v
.into_iter()
.map(|(gap, dur, text)| {
let start = f + gap;
f = start + dur;
Cue { start: rate.tick_of(start), end: rate.tick_of(f), text, ..Default::default() }
})
.collect();
let doc = Document { cues, ..Default::default() };
let opts = WriteOptions { rate: Some(rate), ..Default::default() };
let bytes = write(&doc, Format::Stl, opts);
let back = parse(&bytes, Format::Stl).unwrap();
prop_assert_eq!(&back.cues, &doc.cues);
prop_assert_eq!(write(&back, Format::Stl, opts), bytes);
}
/// TTML (IMSC1): frame-exact at the sequence rate; ids and speakers kept.
#[test]
fn ttml_roundtrip(
v in proptest::collection::vec((0i64..200, 1i64..300, text()), 0..10),
rate in prop_oneof![Just(FrameRate::FPS_23_976), Just(FrameRate::FPS_25), Just(FrameRate::FPS_29_97), Just(FrameRate::FPS_59_94)],
speakers in proptest::collection::vec(proptest::option::of("[A-Z][a-z]{1,6}( [A-Z][a-z]{1,6})?"), 10),
) {
let mut f = 0i64;
let cues: Vec<Cue> = v
.into_iter()
.enumerate()
.map(|(i, (gap, dur, text))| {
let start = f + gap;
f = start + dur;
Cue { start: rate.tick_of(start), end: rate.tick_of(f), text, speaker: speakers[i].clone(), id: Some(format!("c{}", i + 1)), ..Default::default() }
})
.collect();
let doc = Document { cues, ..Default::default() };
let opts = WriteOptions { rate: Some(rate), ..Default::default() };
let bytes = write(&doc, Format::Ttml, opts);
let back = parse(&bytes, Format::Ttml).unwrap();
prop_assert_eq!(&back.cues, &doc.cues);
prop_assert_eq!(write(&back, Format::Ttml, opts), bytes);
}
/// DFXP: clock times in milliseconds; frame-exact after snapping to the sequence rate.
#[test]
fn dfxp_roundtrip(v in proptest::collection::vec((0i64..200, 1i64..300, text()), 0..10), ntsc in any::<bool>()) {
let rate = if ntsc { FrameRate::FPS_29_97 } else { FrameRate::FPS_25 };
let mut f = 0i64;
let cues: Vec<Cue> = v
.into_iter()
.enumerate()
.map(|(i, (gap, dur, text))| {
let start = f + gap;
f = start + dur;
Cue { start: rate.tick_of(start), end: rate.tick_of(f), text, id: Some(format!("c{}", i + 1)), ..Default::default() }
})
.collect();
let doc = Document { cues, ..Default::default() };
let bytes = write(&doc, Format::Dfxp, WriteOptions { rate: Some(rate), ..Default::default() });
let mut back = parse(&bytes, Format::Dfxp).unwrap();
back.snap_to_frames(rate);
prop_assert_eq!(&back.cues, &doc.cues);
}
/// Readers never panic on arbitrary input.
#[test]
fn readers_are_total(bytes in proptest::collection::vec(any::<u8>(), 0..400)) {
@ -124,6 +232,17 @@ proptest! {
let _ = parse(&bytes, f);
}
}
/// The binary STL reader never panics on a valid header followed by arbitrary blocks, and
/// the MCC reader on arbitrary data lines.
#[test]
fn stl_and_mcc_readers_are_total(tail in proptest::collection::vec(any::<u8>(), 0..1200), line in "[0-9A-Za-z:;\t ]{0,200}") {
let mut stl = write(&Document::default(), Format::Stl, WriteOptions::default());
stl.extend(tail);
let _ = parse(&stl, Format::Stl);
let mcc = format!("File Format=MacCaption_MCC V1.0\nTime Code Rate=30DF\n\n00:00:00;00\t{line}\n");
let _ = parse(mcc.as_bytes(), Format::Mcc);
}
}
#[test]

1
crates/edit/src/tests.rs
View File

@ -60,6 +60,7 @@ impl Fx {
hold_filters: false,
field_options: None,
source_channels: Vec::new(),
graphic: None,
}
}
fn put(&mut self, track: TrackId, start: i64, dur: i64, src_in: i64) -> ClipId {

1
crates/edit/src/through.rs
View File

@ -180,6 +180,7 @@ mod tests {
hold_filters: false,
field_options: None,
source_channels: Vec::new(),
graphic: None,
}
}

1
crates/edit/src/transcript/tests.rs
View File

@ -57,6 +57,7 @@ fn clip(id: u64, start: f64, dur: f64, src_in: f64) -> TrackItem {
hold_filters: false,
field_options: None,
source_channels: Vec::new(),
graphic: None,
}
}

10
crates/engine/src/captions.rs
View File

@ -95,11 +95,11 @@ fn hex(c: [u8; 4]) -> String {
format!("#{:02x}{:02x}{:02x}{:02x}", c[0], c[1], c[2], c[3])
}
/// The caption format of a file, if it is one (extension `.srt` / `.vtt` / `.scc`, confirmed by
/// content).
/// The caption format of a file, if it is one (extension `.srt` / `.vtt` / `.scc` / `.mcc` /
/// `.stl` / `.ttml` / `.dfxp` / `.xml`, confirmed by content).
pub fn detect(path: &str, bytes: &[u8]) -> Option<Format> {
let ext = std::path::Path::new(path).extension().and_then(|e| e.to_str()).map(str::to_ascii_lowercase);
if !matches!(ext.as_deref(), Some("srt" | "vtt" | "scc" | "webvtt")) {
if !matches!(ext.as_deref(), Some("srt" | "vtt" | "scc" | "webvtt" | "mcc" | "stl" | "ttml" | "dfxp" | "xml")) {
return None;
}
filmcraft_captions::detect(bytes, ext.as_deref())
@ -206,7 +206,7 @@ fn export(s: &mut Session, p: &Value) -> Result<Value> {
let track = seq.caption_track(tid).ok_or_else(|| bad("captions.export", "no such caption track"))?;
let doc: Document = filmcraft_captions::document_from_track(track, Tick::ZERO);
let df = bool_p(p, "dropFrame").unwrap_or(true);
let bytes = filmcraft_captions::write(&doc, format, WriteOptions { drop_frame: df });
let bytes = filmcraft_captions::write(&doc, format, WriteOptions { drop_frame: df, rate: Some(seq.settings.frame_rate) });
let n = doc.cues.len();
s.services.write_file(&path, &bytes).map_err(|e| EngineError::Other(e.to_string()))?;
Ok(json!({"path": path, "format": format.name(), "captions": n, "bytes": bytes.len()}))
@ -500,7 +500,7 @@ pub fn commands() -> Vec<CommandSpec> {
"Captions…",
&["File", "Export"],
None,
r#"{"path":str,"format":"srt|vtt|scc"?,"track":id|"C1"?,"dropFrame":bool=true}"#,
r#"{"path":str,"format":"srt|vtt|scc|mcc|stl|ttml|dfxp"? (default: from the extension),"track":id|"C1"?,"dropFrame":bool=true}"#,
has_caption_track,
export,
),

1
crates/engine/src/commands.rs
View File

@ -2123,6 +2123,7 @@ fn build() -> Vec<CommandSpec> {
v.extend(crate::essential_sound::commands());
v.extend(crate::color::commands());
v.extend(crate::graphics::commands());
v.extend(crate::graphic_templates::commands());
v.extend(crate::shortcuts::commands());
v.extend(crate::relink::commands());
v.extend(crate::proxies::commands());

3
crates/engine/src/export_tools.rs
View File

@ -335,7 +335,8 @@ fn write_sidecar(s: &Session, project: &Project, seq: ItemId, settings: &ExportS
let format = filmcraft_captions::Format::from_name(kind).ok_or_else(|| bad("export", format!("caption sidecar `{kind}`: srt | vtt")))?;
let range = filmcraft_export::export_range(project, seq, settings).map_err(|e| EngineError::Other(e.to_string()))?;
let doc = filmcraft_captions::document_from_track(track, range.start);
let bytes = filmcraft_captions::write(&doc, format, filmcraft_captions::WriteOptions { drop_frame: q.settings.drop_frame });
let bytes =
filmcraft_captions::write(&doc, format, filmcraft_captions::WriteOptions { drop_frame: q.settings.drop_frame, rate: Some(q.settings.frame_rate) });
let ext = if format == filmcraft_captions::Format::WebVtt { "vtt" } else { kind };
let path = format!("{}.{ext}", settings.path.rsplit_once('.').map_or(settings.path.as_str(), |x| x.0));
s.services.write_file(&path, &bytes).map_err(|e| EngineError::Other(format!("{path}: {e}")))?;

1247
crates/engine/src/graphic_templates.rs Normal file
View File

File diff suppressed because it is too large Load Diff

365
crates/engine/src/graphic_templates_tests.rs Normal file
View File

@ -0,0 +1,365 @@
use serde_json::{Value, json};
use crate::Session;
use crate::media_test_util::tmp_dir;
use filmcraft_project::graphic::LayerContent;
use filmcraft_project::{ClipId, ParamValue, TrackItem};
use filmcraft_render::graphic_clip::{item_layer_specs, layer_bounds};
use filmcraft_time::Tick;
fn demo(dir: &str) -> Session {
let mut s = Session::default();
s.execute("file.openDemoProject", json!({})).unwrap();
s.prefs_path = Some(tmp_dir(dir).join("prefs.json"));
// far after the demo clips, so graphics have free tracks around them
s.execute("playhead.set", json!({"seconds": 600})).unwrap();
s
}
fn item(s: &Session, clip: ClipId) -> TrackItem {
s.active_sequence().unwrap().find_item(clip).unwrap().1.clone()
}
fn canvas(s: &Session) -> (u32, u32) {
let q = s.active_sequence().unwrap();
(q.settings.width, q.settings.height)
}
fn texts(s: &Session, clip: ClipId) -> Vec<String> {
let it = item(s, clip);
item_layer_specs(&it, it.source_in, canvas(s))
.into_iter()
.filter_map(|(_, sp)| match sp.content {
LayerContent::Text(t) => Some(t.text),
_ => None,
})
.collect()
}
fn clip_of(v: &Value) -> ClipId {
ClipId(v["clip"].as_u64().unwrap())
}
#[test]
fn template_export_install_apply_round_trip() {
let mut s = demo("gt-export");
let c = clip_of(&s.execute("graphics.newText", json!({"text": "Headline", "position": [200, 900]})).unwrap());
s.execute("graphics.newText", json!({"text": "Subhead", "clip": c.0, "position": [200, 960], "size": 40})).unwrap();
s.execute("graphics.newShape", json!({"shape": "rectangle", "clip": c.0, "position": [400, 920], "size": [500, 150]})).unwrap();
s.execute("graphics.arrangeLayer", json!({"clip": c.0, "layer": 2, "to": "back"})).unwrap();
s.execute("graphics.setRoll", json!({"clip": c.0, "mode": "crawlLeft", "startOffScreen": true})).unwrap();
let r = s
.execute(
"graphics.template.export",
json!({"clip": c.0, "name": "Show Lower Third", "category": "Lower Thirds", "controls": [
{"layer": 1, "param": "text", "name": "Headline"},
{"layer": "Subhead", "param": "text", "name": "Subhead"},
{"layer": 0, "param": "fill_color", "name": "Box Color"},
{"layer": 1, "param": "size", "name": "Headline Size", "min": 20, "max": 200},
{"layer": 2, "param": "enabled", "name": "Show Subhead"},
]}),
)
.unwrap();
let path = r["path"].as_str().unwrap().to_string();
assert!(path.ends_with("show-lower-third.fcgt"), "{path}");
assert_eq!(r["controls"].as_array().unwrap().len(), 5);
assert_eq!(r["controls"][3]["kind"], "slider");
assert_eq!(r["controls"][4]["kind"], "checkbox");
// the file is our documented JSON format
let file: Value = serde_json::from_slice(&std::fs::read(&path).unwrap()).unwrap();
assert_eq!(file["format"], "filmcraft.graphicsTemplate");
assert_eq!(file["version"], 1);
assert_eq!(file["graphic"]["roll"]["mode"], "crawlLeft");
// install into another user's library
let mut s2 = demo("gt-install");
let ins = s2.execute("graphics.template.install", json!({"path": path})).unwrap();
assert_eq!(ins["id"], "user:show-lower-third");
let list = s2.execute("graphics.template.list", json!({"query": "lower third", "source": "user"})).unwrap();
assert_eq!(list.as_array().unwrap().len(), 1, "{list}");
let all = s2.execute("graphics.template.list", json!({})).unwrap();
assert!(all.as_array().unwrap().len() >= 9, "built-ins and the user template");
let undo_before = s2.history.undo.len();
let a = s2
.execute(
"graphics.template.apply",
json!({"template": "Show Lower Third", "values": {"Headline": "Breaking News", "Box Color": "#ff0000", "headline_size": 500}}),
)
.unwrap();
assert_eq!(s2.history.undo.len(), undo_before + 1, "one undo step");
let c2 = clip_of(&a);
assert_eq!(texts(&s2, c2), vec!["Breaking News", "Subhead"]);
let it = item(&s2, c2);
assert_eq!(it.graphic.as_ref().unwrap().roll.mode, filmcraft_project::RollMode::CrawlLeft);
let ctl = s2.execute("graphics.template.controls", json!({"clip": c2.0})).unwrap();
assert_eq!(ctl["controls"][0]["value"], "Breaking News");
assert_eq!(ctl["controls"][2]["value"], "#ff0000");
assert_eq!(ctl["controls"][3]["value"], 200.0, "slider clamped to its range");
s2.execute("graphics.template.set", json!({"clip": c2.0, "control": "Show Subhead", "value": false})).unwrap();
let ctl = s2.execute("graphics.template.controls", json!({"clip": c2.0})).unwrap();
assert_eq!(ctl["controls"][4]["value"], false);
s2.execute("edit.undo", json!({})).unwrap();
s2.execute("edit.undo", json!({})).unwrap();
assert!(s2.active_sequence().unwrap().find_item(c2).is_none(), "undo removes the applied template");
// unknown property
assert!(s2.execute("graphics.template.apply", json!({"template": "Show Lower Third", "values": {"Nope": 1}})).is_err());
assert!(s2.active_sequence().unwrap().video_tracks.iter().all(|t| t.items.iter().all(|i| i.name != "Show Lower Third")));
// remove
s2.execute("graphics.template.remove", json!({"template": "user:show-lower-third"})).unwrap();
assert!(s2.execute("graphics.template.list", json!({"source": "user"})).unwrap().as_array().unwrap().is_empty());
}
#[test]
fn mogrt_files_are_never_read() {
let mut s = demo("gt-mogrt");
let d = tmp_dir("gt-mogrt-files");
let m = d.join("fancy.mogrt");
std::fs::write(&m, b"PK\x03\x04whatever").unwrap();
let e = s.execute("graphics.template.install", json!({"path": m.to_string_lossy()})).unwrap_err().to_string();
assert!(e.contains(".mogrt"), "{e}");
// a ZIP renamed to .fcgt is refused too
let z = d.join("renamed.fcgt");
std::fs::write(&z, b"PK\x03\x04whatever").unwrap();
let e = s.execute("graphics.template.install", json!({"path": z.to_string_lossy()})).unwrap_err().to_string();
assert!(e.contains("does not read"), "{e}");
}
#[test]
fn builtin_templates_apply_and_thumbnail() {
let mut s = demo("gt-builtin");
let list = s.execute("graphics.template.list", json!({"category": "Callouts"})).unwrap();
assert_eq!(list.as_array().unwrap().len(), 2);
for e in s.execute("graphics.template.list", json!({"source": "builtin"})).unwrap().as_array().unwrap().clone() {
let id = e["id"].as_str().unwrap();
let r = s.execute("graphics.template.apply", json!({"template": id})).unwrap();
assert!(!texts(&s, clip_of(&r)).is_empty(), "{id}");
let th = s.execute("graphics.template.thumbnail", json!({"template": id, "width": 160})).unwrap();
assert_eq!((th["width"].as_u64(), th["height"].as_u64()), (Some(160), Some(90)));
let png = filmcraft_project::gtemplate::base64_decode(th["pngBase64"].as_str().unwrap()).unwrap();
assert!(png.starts_with(b"\x89PNG"));
let (_, _, rgba) = super::thumbnail(&super::find_template(&s, id).unwrap().template, 160);
assert!(rgba.chunks(4).filter(|p| p[3] > 128).count() > 50, "{id}: the thumbnail shows something");
s.execute("playhead.set", json!({"seconds": 600 + 20 * s.history.undo.len()})).unwrap();
}
}
#[test]
fn responsive_time_protects_intro_and_outro_when_trimmed() {
let mut s = demo("gt-time");
let c = clip_of(&s.execute("graphics.newText", json!({"text": "Fade", "seconds": 5})).unwrap());
let sec = |x: f64| Tick::from_seconds_f64(x);
// opacity: 0 → 100 over the first second, 100 → 0 over the last
s.edit_sequence("kf", |q, _, _| {
let (_, it) = q.find_item_mut(c).unwrap();
let (si, d) = (it.source_in, it.duration);
let e = it.effects.iter_mut().find(|e| e.effect == "graphic_text").unwrap();
let p = e.params.get_mut("opacity").unwrap();
p.toggle_animation(si);
p.set_at(si, ParamValue::Float(0.0));
for (t, v) in [(si + sec(1.0), 100.0), (si + d - sec(1.0), 100.0), (si + d, 0.0)] {
p.set_at(t, ParamValue::Float(v));
}
Ok(())
})
.unwrap();
s.execute("graphics.setResponsiveTime", json!({"clip": c.0, "introSeconds": 1, "outroSeconds": 1})).unwrap();
// opacity (0..100) at clip-relative `u` seconds, or `u` seconds before the clip end
let op = |s: &Session, u: f64| {
let it = item(s, c);
let sp = item_layer_specs(&it, it.source_in + sec(u), canvas(s));
(sp[0].1.transform.opacity * 100.0) as f64
};
let op_end = |s: &Session, before: f64| {
let it = item(s, c);
let d = it.duration;
let sp = item_layer_specs(&it, it.source_in + d - sec(before), canvas(s));
(sp[0].1.transform.opacity * 100.0) as f64
};
let near = |a: f64, b: f64| (a - b).abs() < 1e-3;
let d0 = item(&s, c).duration;
assert!(near(op(&s, 0.5), 50.0));
assert!(near(op_end(&s, 0.5), 50.0));
// extend by 150 frames: the intro and outro keep their timing, the middle holds
let rate = s.sequence_rate();
s.execute("timeline.trim", json!({"clip": c.0, "edge": "out", "deltaFrames": 150})).unwrap();
assert_eq!(item(&s, c).duration, d0 + rate.tick_of(150));
assert!(near(op(&s, 0.5), 50.0));
assert!(near(op(&s, 5.0), 100.0));
assert!(near(op_end(&s, 0.5), 50.0));
assert!(near(op_end(&s, 0.25), 25.0));
// trim the head by 60 frames: the intro plays from the new start
s.execute("timeline.trim", json!({"clip": c.0, "edge": "in", "deltaFrames": 60})).unwrap();
assert!(near(op(&s, 0.0), 0.0));
assert!(near(op(&s, 0.5), 50.0));
assert!(near(op_end(&s, 0.5), 50.0), "the outro still ends at the clip end");
// without responsive time the head trim would have cut the fade-in
assert!(s.execute("graphics.setResponsiveTime", json!({"clip": c.0, "introSeconds": 30})).is_err());
}
#[test]
fn rolls_at_exact_times() {
let mut s = demo("gt-roll");
let c = clip_of(&s.execute("graphics.newShape", json!({"shape": "rectangle", "position": [960, 540], "size": [200, 100], "seconds": 4})).unwrap());
s.execute("graphics.setRoll", json!({"clip": c.0, "mode": "roll", "startOffScreen": true, "endOffScreen": true, "prerollFrames": 0})).unwrap();
let (w, h) = canvas(&s);
let top = |s: &Session, u: Tick| {
let it = item(s, c);
layer_bounds(&item_layer_specs(&it, it.source_in + u, (w, h))[0].1)[1]
};
let d = item(&s, c).duration;
// starts with its top on the frame bottom, ends with its bottom on the frame top: travel = h + 100
assert!((top(&s, Tick::ZERO) - h as f64).abs() < 1e-6);
assert!((top(&s, d) - (-100.0)).abs() < 1e-6);
assert!((top(&s, Tick(d.0 / 4)) - (h as f64 - (h as f64 + 100.0) / 4.0)).abs() < 1e-6);
// ease in / out
s.execute("graphics.setRoll", json!({"clip": c.0, "easeInSeconds": 1, "easeOutSeconds": 1})).unwrap();
let u = Tick::from_seconds_f64(0.5);
let v = 1.0 / (d.seconds() - 1.0);
let want = h as f64 - (h as f64 + 100.0) * v * 0.25 / 2.0;
assert!((top(&s, u) - want).abs() < 1e-6, "{} vs {want}", top(&s, u));
s.execute("graphics.setRoll", json!({"clip": c.0, "mode": "off"})).unwrap();
assert!((top(&s, u) - 490.0).abs() < 1e-6);
assert!(s.execute("graphics.setRoll", json!({"clip": c.0, "mode": "sideways"})).is_err());
}
#[test]
fn pins_follow_text_growth() {
let mut s = demo("gt-pins");
let c = clip_of(&s.execute("graphics.newText", json!({"text": "Hi", "position": [300, 500], "size": 80})).unwrap());
s.execute("graphics.newShape", json!({"shape": "rectangle", "clip": c.0, "position": [330, 470], "size": [200, 120]})).unwrap();
s.execute("graphics.arrangeLayer", json!({"clip": c.0, "layer": 1, "to": "back"})).unwrap();
// box (layer 0) pinned to the text (layer 1) on all edges
s.execute("graphics.pin", json!({"clip": c.0, "layer": 0, "to": 1})).unwrap();
let boxes = |s: &mut Session| {
let l = s.execute("graphics.list", json!({"clip": c.0})).unwrap();
let q = |i: usize| {
let v: Vec<f64> = l["layers"][i]["quad"].as_array().unwrap().iter().flat_map(|p| [p[0].as_f64().unwrap(), p[1].as_f64().unwrap()]).collect();
[v[0], v[1], v[4], v[5]]
};
(q(0), q(1))
};
let (b0, t0) = boxes(&mut s);
let gap = [b0[0] - t0[0], b0[1] - t0[1], b0[2] - t0[2], b0[3] - t0[3]];
s.execute("graphics.setText", json!({"clip": c.0, "layer": 1, "text": "Hello, responsive world"})).unwrap();
let (b1, t1) = boxes(&mut s);
assert!(t1[2] > t0[2] + 300.0, "the text grew");
for k in 0..4 {
assert!((b1[k] - t1[k] - gap[k]).abs() < 1e-3, "edge {k}: {b1:?} {t1:?} {gap:?}");
}
// move the text: the box follows; unpin
s.execute("graphics.set", json!({"clip": c.0, "layer": 1, "props": {"position": [100, 300]}})).unwrap();
let (b2, t2) = boxes(&mut s);
assert!((b2[0] - t2[0] - gap[0]).abs() < 1e-3);
s.execute("graphics.pin", json!({"clip": c.0, "layer": 0, "to": "none"})).unwrap();
let list = s.execute("graphics.list", json!({"clip": c.0})).unwrap();
assert!(list["layers"][0]["pin"].is_null());
assert!(s.execute("graphics.pin", json!({"clip": c.0, "layer": 0, "to": 0})).is_err(), "not to itself");
}
#[test]
fn character_styles_follow_edits() {
let mut s = demo("gt-chars");
let c = clip_of(&s.execute("graphics.newText", json!({"text": "Hello World"})).unwrap());
s.execute("graphics.setCharStyle", json!({"clip": c.0, "start": 6, "end": 11, "style": {"bold": true, "color": "#ff0000", "size": 140}})).unwrap();
let st = s.execute("graphics.list", json!({"clip": c.0})).unwrap()["layers"][0]["styles"].clone();
assert_eq!(st, json!([{"start": 6, "end": 11, "fauxBold": true, "fill": [1.0, 0.0, 0.0, 1.0], "size": 140.0}]));
s.execute("graphics.setText", json!({"clip": c.0, "text": "Oh, Hello World"})).unwrap();
let st = s.execute("graphics.list", json!({"clip": c.0})).unwrap()["layers"][0]["styles"].clone();
assert_eq!((st[0]["start"].as_u64(), st[0]["end"].as_u64()), (Some(10), Some(15)));
// the layout uses the run: the styled word is taller
let it = item(&s, c);
let sp = &item_layer_specs(&it, it.source_in, canvas(&s))[0].1;
let LayerContent::Text(t) = &sp.content else { panic!() };
let l = filmcraft_render::graphic_clip::text_layout(t);
assert!(l.glyphs.iter().filter(|g| g.run == 1).all(|g| g.size == 140.0 && g.synth_bold));
s.execute("graphics.setCharStyle", json!({"clip": c.0, "clear": true})).unwrap();
assert_eq!(s.execute("graphics.list", json!({"clip": c.0})).unwrap()["layers"][0]["styles"], json!([]));
assert!(s.execute("graphics.setCharStyle", json!({"clip": c.0, "start": 3, "end": 3, "style": {"bold": true}})).is_err());
assert!(s.execute("graphics.setCharStyle", json!({"clip": c.0, "style": {"sparkle": true}})).is_err());
}
#[test]
fn upgrade_caption_to_graphic() {
let mut s = demo("gt-caption");
s.execute("captions.add", json!({"text": "Hello <i>there</i>", "seconds": 600, "durationSeconds": 2})).unwrap();
let caps = s.execute("captions.list", json!({})).unwrap()["tracks"][0]["captions"].clone();
let before = caps.as_array().unwrap().len();
let cap = caps.as_array().unwrap().iter().find(|c| c["text"] == "Hello <i>there</i>").unwrap().clone();
let r = s.execute("graphics.upgradeCaption", json!({})).unwrap();
assert_eq!(r["upgraded"], 1);
let c = ClipId(r["clips"][0].as_u64().unwrap());
assert_eq!(texts(&s, c), vec!["Hello there"]);
let it = item(&s, c);
assert_eq!(it.start.0, cap["start"].as_i64().unwrap());
assert_eq!(it.end().0, cap["end"].as_i64().unwrap());
let ex = it.effects.iter().find(|e| e.effect == "graphic_text").unwrap().layer.as_ref().unwrap();
assert_eq!((ex.runs[0].start, ex.runs[0].end, ex.runs[0].style.faux_italic), (6, 11, Some(true)));
let after = s.execute("captions.list", json!({})).unwrap()["tracks"][0]["captions"].as_array().unwrap().len();
assert_eq!(after, before - 1, "the caption became a graphic");
s.execute("edit.undo", json!({})).unwrap();
assert!(s.active_sequence().unwrap().find_item(c).is_none());
assert_eq!(s.execute("captions.list", json!({})).unwrap()["tracks"][0]["captions"].as_array().unwrap().len(), before);
}
#[test]
fn source_graphics_share_edits() {
let mut s = demo("gt-source");
let c = clip_of(&s.execute("graphics.newText", json!({"text": "Shared"})).unwrap());
let r = s.execute("graphics.upgradeToSourceGraphic", json!({"clip": c.0})).unwrap();
let item_id = r["item"].as_u64().unwrap();
assert!(s.execute("graphics.upgradeToSourceGraphic", json!({"clip": c.0})).is_err());
// a second instance from the project item
let v1 = s.active_sequence().unwrap().find_item(c).unwrap().0;
s.execute("timeline.place", json!({"item": item_id, "track": v1.0, "time": Tick::from_seconds_f64(700.0).0})).unwrap();
let c2 = s.active_sequence().unwrap().video_tracks.iter().flat_map(|t| &t.items).find(|i| i.item.0 == item_id && i.id != c).unwrap().id;
assert_eq!(texts(&s, c2), vec!["Shared"]);
s.execute("graphics.setText", json!({"clip": c2.0, "text": "Edited once"})).unwrap();
assert_eq!(texts(&s, c), vec!["Edited once"], "the other instance follows");
s.execute("graphics.set", json!({"clip": c.0, "props": {"size": 60}})).unwrap();
let size = |s: &Session, cl: ClipId| item(s, cl).effects.iter().find(|e| e.effect == "graphic_text").unwrap().params["size"].value.clone();
assert_eq!(size(&s, c2), ParamValue::Float(60.0));
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(size(&s, c2), size(&s, c), "undo restores both");
assert_ne!(size(&s, c2), ParamValue::Float(60.0));
// new plain graphics do not reuse the source graphic's item
let c3 = clip_of(&s.execute("graphics.newText", json!({"text": "Plain", "seconds": 1, "time": Tick::from_seconds_f64(800.0).0})).unwrap());
assert_ne!(item(&s, c3).item.0, item_id);
}
#[test]
fn replace_fonts_in_project() {
let mut s = demo("gt-fonts");
let c = clip_of(&s.execute("graphics.newText", json!({"text": "Serif", "font": "Noto Serif"})).unwrap());
s.execute("graphics.setCharStyle", json!({"clip": c.0, "start": 0, "end": 2, "style": {"font": "Noto Serif"}})).unwrap();
let used = s.execute("graphics.fonts.used", json!({})).unwrap();
assert!(used.as_array().unwrap().iter().any(|f| f["family"] == "Noto Serif" && f["missing"] == false), "{used}");
let r = s.execute("file.replaceFonts", json!({"from": "Noto Serif", "to": "JetBrains Mono"})).unwrap();
assert_eq!(r["replaced"], 2);
let used = s.execute("graphics.fonts.used", json!({})).unwrap();
assert!(!used.as_array().unwrap().iter().any(|f| f["family"] == "Noto Serif"));
s.execute("edit.undo", json!({})).unwrap();
let used = s.execute("graphics.fonts.used", json!({})).unwrap();
assert!(used.as_array().unwrap().iter().any(|f| f["family"] == "Noto Serif"));
}
#[test]
fn caption_formats_through_the_engine() {
let mut s = demo("gt-capfmt");
s.execute("captions.add", json!({"text": "One", "seconds": 1, "durationSeconds": 2})).unwrap();
s.execute("captions.add", json!({"text": "Two\nlines", "seconds": 4, "durationSeconds": 1})).unwrap();
let d = tmp_dir("gt-capfmt-files");
let want = s.execute("captions.list", json!({})).unwrap()["tracks"][0]["captions"].clone();
for ext in ["mcc", "stl", "ttml", "dfxp"] {
let path = d.join(format!("caps.{ext}")).to_string_lossy().to_string();
let r = s.execute("captions.export", json!({"path": path})).unwrap();
assert_eq!(r["captions"], 2, "{ext}");
let imp = s.execute("captions.import", json!({"path": path})).unwrap();
assert_eq!(imp["captions"], 2, "{ext}: {imp}");
let got = s.execute("captions.list", json!({"track": "C1"})).unwrap()["tracks"][0]["captions"].clone();
for i in 0..2 {
assert_eq!(got[i]["text"], want[i]["text"], "{ext}");
assert_eq!(got[i]["in"], want[i]["in"], "{ext}");
assert_eq!(got[i]["out"], want[i]["out"], "{ext}");
}
}
}

70
crates/engine/src/graphics.rs
View File

@ -10,7 +10,7 @@
use filmcraft_geom::Vec2;
use filmcraft_project::graphic::{self, LayerContent, SHAPE_OPTS, eval_layer, layer_display_name, layer_indices, new_shape_layer, new_text_layer};
use filmcraft_project::{ClipId, ItemId, ItemKind, Label, ParamValue, Sequence, TrackKind};
use filmcraft_render::graphic_clip::{layer_local_bounds, layer_quad};
use filmcraft_render::graphic_clip::{item_layer_specs, layer_local_bounds, layer_quad};
use filmcraft_time::{Tick, TimeRange};
use serde_json::{Value, json};
@ -60,7 +60,7 @@ fn has_graphic(s: &Session) -> std::result::Result<(), String> {
}
/// Effect index of graphic layer `layer` (or the selected / frontmost layer) of `clip`.
fn layer_effect_index(s: &Session, clip: ClipId, p: &Value) -> Result<(usize, usize)> {
pub(crate) fn layer_effect_index(s: &Session, clip: ClipId, p: &Value) -> Result<(usize, usize)> {
let seq = s.active_sequence().ok_or(EngineError::NoSequence)?;
let (_, it) = seq.find_item(clip).ok_or_else(|| bad("graphics", "no such clip"))?;
let idx = layer_indices(&it.effects);
@ -78,16 +78,17 @@ fn layer_effect_index(s: &Session, clip: ClipId, p: &Value) -> Result<(usize, us
Ok((l, e))
}
fn vec2_p(p: &Value, k: &str) -> Option<Vec2> {
pub(crate) fn vec2_p(p: &Value, k: &str) -> Option<Vec2> {
let a = p.get(k)?.as_array()?;
Some(Vec2::new(a.first()?.as_f64()?, a.get(1)?.as_f64()?))
}
/// Find or create the graphic source item for the active sequence's frame size.
fn graphic_source(p: &mut filmcraft_project::Project, w: u32, h: u32, rate: filmcraft_time::FrameRate) -> ItemId {
if let Some((id, _)) =
p.items.iter().find(|(_, i)| matches!(i.kind, ItemKind::Graphic { width, height, rate: r } if width == w && height == h && r == rate))
{
/// Find or create the graphic source item for the active sequence's frame size (source graphics
/// have their own items and are never reused here).
pub(crate) fn graphic_source(p: &mut filmcraft_project::Project, w: u32, h: u32, rate: filmcraft_time::FrameRate) -> ItemId {
if let Some((id, _)) = p.items.iter().find(|(id, i)| {
!p.source_graphics.contains_key(id) && matches!(i.kind, ItemKind::Graphic { width, height, rate: r } if width == w && height == h && r == rate)
}) {
return *id;
}
p.add_item("Graphic", Label::Rose, ItemKind::Graphic { width: w, height: h, rate }, None)
@ -105,7 +106,7 @@ fn new_graphic_clip(s: &mut Session, layer: filmcraft_project::EffectInstance, n
/// Place a video clip of the item `source` returns (with its duration) at the playhead (or
/// `time`), above the clips there, as one undo step; selects it.
fn place_video_clip(
pub(crate) fn place_video_clip(
s: &mut Session,
name: &str,
p: &Value,
@ -197,7 +198,7 @@ fn param_id(k: &str) -> &str {
}
/// JSON value for a property, accepting names for choices ("center", "small caps", "ellipse"…).
fn to_param(template: &ParamValue, id: &str, v: &Value) -> Option<ParamValue> {
pub(crate) fn to_param(template: &ParamValue, id: &str, v: &Value) -> Option<ParamValue> {
if let (ParamValue::Choice(_), Value::String(name)) = (template, v) {
let opts: &[&str] = match id {
"align" => graphic::ALIGN_OPTS,
@ -213,8 +214,9 @@ fn to_param(template: &ParamValue, id: &str, v: &Value) -> Option<ParamValue> {
crate::commands::json_to_param(template, v)
}
/// Set properties `props` on a layer (keyframe-aware at time `tl`).
fn set_props(s: &mut Session, clip: ClipId, eidx: usize, props: &serde_json::Map<String, Value>, tl: Tick, label: &str) -> Result<()> {
/// Set properties `props` on a layer (keyframe-aware at time `tl`). Changing the text keeps
/// per-character styles on their characters.
pub(crate) fn set_props(s: &mut Session, clip: ClipId, eidx: usize, props: &serde_json::Map<String, Value>, tl: Tick, label: &str) -> Result<()> {
let props = props.clone();
s.edit_sequence(label, |q, _, _| {
let (_, it) = q.find_item_mut(clip).ok_or(filmcraft_edit::EditError::NoItem(clip))?;
@ -228,7 +230,13 @@ fn set_props(s: &mut Session, clip: ClipId, eidx: usize, props: &serde_json::Map
let id = param_id(k);
let prm = e.params.get_mut(id).ok_or_else(|| bad("graphics.set", format!("no property `{k}`")))?;
let pv = to_param(&prm.value, id, v).ok_or_else(|| bad("graphics.set", format!("`{k}`: value has the wrong type")))?;
prm.set_at(mt, pv);
let old = prm.value_at(mt);
prm.set_at(mt, pv.clone());
if let (ParamValue::Text(a), ParamValue::Text(b), "text") = (&old, &pv, id)
&& let Some(x) = e.layer.as_mut()
{
x.runs = filmcraft_project::graphic_design::adjust_runs(a, b, &x.runs);
}
}
Ok(())
})
@ -244,13 +252,15 @@ fn list_layers(s: &Session, clip: ClipId) -> Result<Value> {
};
let ph = s.playhead();
let mt = it.source_time_at(ph.clamp(it.start, it.end() - Tick(1)));
let shown = item_layer_specs(it, mt, size);
let layers: Vec<Value> = layer_indices(&it.effects)
.iter()
.enumerate()
.filter_map(|(i, &ei)| {
let e = &it.effects[ei];
let sp = eval_layer(e, mt, size)?;
let sp = shown.iter().find(|(x, _)| *x == ei).map(|(_, s)| s.clone()).or_else(|| eval_layer(e, mt, size))?;
let q = layer_quad(&sp);
let extra = e.layer.as_deref();
let (kind, text) = match &sp.content {
LayerContent::Text(t) => ("text", Some(t.text.clone())),
LayerContent::Shape(sh) => (SHAPE_OPTS.get(sh.shape as usize).copied().unwrap_or("Shape"), None),
@ -263,12 +273,19 @@ fn list_layers(s: &Session, clip: ClipId) -> Result<Value> {
"text": text,
"enabled": e.enabled,
"position": [sp.transform.position.x, sp.transform.position.y],
"uid": extra.map_or(0, |x| x.uid),
"pin": extra.and_then(|x| x.pin.as_ref()).map(|p| serde_json::to_value(p).unwrap_or_default()),
"styles": extra.map(|x| serde_json::to_value(&x.runs).unwrap_or_default()).unwrap_or(json!([])),
"localBounds": layer_local_bounds(&sp),
"quad": q.iter().map(|p| [p.x, p.y]).collect::<Vec<_>>(),
}))
})
.collect();
Ok(json!({"clip": clip.0, "name": it.name, "start": it.start.0, "duration": it.duration.0, "canvas": [size.0, size.1], "layers": layers}))
let meta = it.graphic.as_deref().map(|m| serde_json::to_value(m).unwrap_or_default());
let source = s.project.source_graphics.contains_key(&it.item).then_some(it.item.0);
Ok(
json!({"clip": clip.0, "name": it.name, "start": it.start.0, "duration": it.duration.0, "canvas": [size.0, size.1], "layers": layers, "graphic": meta, "sourceGraphic": source}),
)
}
fn quad_bounds(q: &[Vec2; 4]) -> [f64; 4] {
@ -289,11 +306,15 @@ fn layer_boxes(s: &Session, clip: ClipId, layers: &[usize]) -> Result<(Vec<(usiz
};
let mt = it.source_time_at(s.playhead().clamp(it.start, it.end() - Tick(1)));
let idx = layer_indices(&it.effects);
let shown = item_layer_specs(it, mt, size);
let mut out = Vec::new();
for &l in layers {
let ei = *idx.get(l).ok_or_else(|| bad("graphics.align", format!("no layer {l}")))?;
let sp = eval_layer(&it.effects[ei], mt, size).ok_or_else(|| bad("graphics.align", "bad layer"))?;
out.push((ei, sp.transform.position, quad_bounds(&layer_quad(&sp))));
let sp = shown.iter().find(|(x, _)| *x == ei).map(|(_, s)| s.clone()).ok_or_else(|| bad("graphics.align", "bad layer"))?;
// the position stored in the layer (what moving it changes)
let own = eval_layer(&it.effects[ei], mt, size).map_or(sp.transform.position, |o| o.transform.position);
// bounds where it is shown (after pins / roll); moving `own` by d moves those by d
out.push((ei, own, quad_bounds(&layer_quad(&sp))));
}
Ok((out, size))
}
@ -309,14 +330,23 @@ fn layers_p(s: &Session, clip: ClipId, p: &Value) -> Result<Vec<usize>> {
Ok(vec![l])
}
fn move_layers(s: &mut Session, clip: ClipId, moves: Vec<(usize, Vec2)>, label: &str) -> Result<()> {
pub(crate) fn move_layers(s: &mut Session, clip: ClipId, moves: Vec<(usize, Vec2)>, label: &str) -> Result<()> {
let tl = s.playhead();
s.edit_sequence(label, |q, _, _| {
let (_, it) = q.find_item_mut(clip).ok_or(filmcraft_edit::EditError::NoItem(clip))?;
let mt = it.source_time_at(tl.clamp(it.start, it.end() - Tick(1)));
for (ei, pos) in &moves {
if let Some(prm) = it.effects.get_mut(*ei).and_then(|e| e.params.get_mut("position")) {
prm.set_at(mt, ParamValue::Vec2(*pos));
let Some(e) = it.effects.get_mut(*ei) else { continue };
let Some(prm) = e.params.get_mut("position") else { continue };
let old = prm.value_at(mt).as_vec2().unwrap_or(*pos);
prm.set_at(mt, ParamValue::Vec2(*pos));
// a pinned layer keeps following its target from its new place
if let Some(pin) = e.layer.as_mut().and_then(|x| x.pin.as_mut()) {
let (dx, dy) = (pos.x - old.x, pos.y - old.y);
pin.offsets[0] += dx;
pin.offsets[2] += dx;
pin.offsets[1] += dy;
pin.offsets[3] += dy;
}
}
Ok(())

5
crates/engine/src/lib.rs
View File

@ -16,6 +16,7 @@ pub mod commands;
pub mod demo;
pub mod essential_sound;
pub mod export_tools;
pub mod graphic_templates;
pub mod graphics;
pub mod interchange;
pub mod masks;
@ -552,6 +553,10 @@ impl Session {
self.exec_depth += 1;
let r = (spec.run)(self, &params);
self.exec_depth -= 1;
// source graphics: an edited instance updates the shared layers and the other instances
if r.is_ok() && spec.journal && id.starts_with("graphics.") && !self.project.source_graphics.is_empty() {
graphic_templates::sync_source_graphics(self);
}
if self.exec_depth == 0
&& let Err(e) = &r
{

5
crates/export/src/lib.rs
View File

@ -951,6 +951,11 @@ impl Write for SharedBuf {
}
}
/// Encode straight sRGB RGBA8 pixels as a PNG file (thumbnails, previews).
pub fn encode_png(rgba: Vec<u8>, w: u32, h: u32) -> Result<Vec<u8>> {
encode_still(Format::PngSequence, rgba, w, h)
}
/// Encode one still of an image sequence.
fn encode_still(format: Format, rgba: Vec<u8>, w: u32, h: u32) -> Result<Vec<u8>> {
let enc = |e: image::ImageError| ExportError::Encode(e.to_string());

13
crates/format/src/lib.rs
View File

@ -30,6 +30,9 @@
//! - **v7** (M5.5): effect masks (keyframable Bézier mask paths on effect instances). No-op step.
//! - **v8** (M12): multi-camera source sequences, multi-camera clips and merged clips. No-op step;
//! older builds would drop the camera data when saving.
//! - **v12** (M10.7): graphics templates, rolls / crawls, responsive design (pins, intro / outro),
//! per-character text styles (`TrackItem::graphic`, `EffectInstance::layer`) and source graphics
//! (`Project::source_graphics`). No-op step; older builds would drop these fields when saving.
pub mod atomic;
pub mod autosave;
@ -48,7 +51,7 @@ pub type Migration = fn(Value) -> Result<Value, String>;
/// `MIGRATIONS[i]` upgrades schema `i + 1` to `i + 2`. Append one function per schema bump; never
/// edit a shipped one.
pub const MIGRATIONS: &[Migration] = &[v1_to_v2, v2_to_v3, v3_to_v4, v4_to_v5, v5_to_v6, v6_to_v7, v7_to_v8, v8_to_v9, v9_to_v10, v10_to_v11];
pub const MIGRATIONS: &[Migration] = &[v1_to_v2, v2_to_v3, v3_to_v4, v4_to_v5, v5_to_v6, v6_to_v7, v7_to_v8, v8_to_v9, v9_to_v10, v10_to_v11, v11_to_v12];
/// The schema version this build writes (and the newest it reads).
pub const SCHEMA_VERSION: u32 = MIGRATIONS.len() as u32 + 1;
@ -270,13 +273,19 @@ fn v10_to_v11(doc: Value) -> Result<Value, String> {
Ok(doc)
}
/// v11 → v12: graphics templates, rolls / crawls, responsive design and per-character text styles
/// on graphic clips, and source graphics. Existing data needs no change.
fn v11_to_v12(doc: Value) -> Result<Value, String> {
Ok(doc)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn schema_version_matches_table() {
assert_eq!(SCHEMA_VERSION, 11);
assert_eq!(SCHEMA_VERSION, 12);
}
#[test]

17
crates/format/tests/fixtures.rs
View File

@ -192,6 +192,23 @@ fn v8_loads_without_transcripts_and_v9_roundtrips_them() {
assert_eq!(again.project.transcripts[&ItemId(7)].speakers[0].name, "Speaker 1");
}
/// Schema 11 (before M10.7): no source graphics; graphic clips without template / roll /
/// responsive data and layers without style runs or pins load with those empty.
#[test]
fn v11_minimal_loads_without_graphics_design_data() {
let l = decode(&fixture("v11-minimal.fcproj")).unwrap();
assert_eq!(l.schema_version, 11);
assert!(l.migrated());
let p = &l.project;
assert_eq!(p.name, "Before Graphics Templates");
assert!(p.source_graphics.is_empty());
assert!(p.root.find_bin(filmcraft_project::BinId(1)).is_some());
let again = decode(&encode(p, true)).unwrap();
assert_eq!(again.schema_version, SCHEMA_VERSION);
const { assert!(SCHEMA_VERSION >= 12) };
assert_eq!(&again.project, p);
}
/// Schema 10 (before M3.11): no search bins, no safe areas / capture format / scratch disks in
/// the project settings.
#[test]

1
crates/format/tests/fixtures/v11-minimal.fcproj vendored Normal file
View File

@ -0,0 +1 @@
{"schema_version":11,"generator":"FilmCraft 0.1.0","project":{"name":"Before Graphics Templates","settings":{"renderer":"FilmCraft GPU Acceleration (wgpu)","video_display":"Timecode","audio_display_samples":true,"default_still_duration":1270080000000,"default_transition_duration_frames":24,"default_audio_transition_duration":254016000000},"root":{"id":0,"name":"Before Graphics Templates","children":[{"Bin":{"id":1,"name":"Titles","children":[]}}]},"items":{},"next_id":2}}

2
crates/interchange/src/common.rs
View File

@ -430,6 +430,7 @@ impl Builder {
hold_filters: false,
field_options: None,
source_channels: Vec::new(),
graphic: None,
}
}
@ -617,6 +618,7 @@ pub(crate) fn transition_effect(name: &str, audio: bool, report: &mut Report) ->
masks: vec![],
post_fader: false,
essential: false,
layer: None,
})
}

5
crates/project/src/effect.rs
View File

@ -96,6 +96,7 @@ impl EffectDef {
masks: Vec::new(),
post_fader: false,
essential: false,
layer: None,
}
}
}
@ -113,6 +114,10 @@ pub struct EffectInstance {
/// Managed by the Essential Sound panel (see [`crate::essential`]).
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub essential: bool,
/// Graphic layers only: stable uid, per-character styles and responsive pin
/// ([`crate::graphic_design::LayerExtra`]). Schema v12.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub layer: Option<Box<crate::graphic_design::LayerExtra>>,
}
impl EffectInstance {

1
crates/project/src/essential.rs
View File

@ -783,6 +783,7 @@ mod tests {
hold_filters: false,
field_options: None,
source_channels: Vec::new(),
graphic: None,
}
}

62
crates/project/src/graphic.rs
View File

@ -196,6 +196,8 @@ pub struct TextProps {
pub box_width: f32,
/// Vertical text (characters stacked top to bottom; columns right to left).
pub vertical: bool,
/// Per-character style overrides as *byte* ranges of `text` (sorted, non-overlapping).
pub runs: Vec<(std::ops::Range<usize>, crate::graphic_design::CharStyle)>,
}
#[derive(Clone, Debug, PartialEq)]
@ -220,6 +222,10 @@ pub enum LayerContent {
pub struct LayerSpec {
pub name: String,
pub enabled: bool,
/// Stable layer id within the clip (0 = none).
pub uid: u64,
/// Responsive Design – Position.
pub pin: Option<crate::graphic_design::Pin>,
pub content: LayerContent,
pub appearance: Appearance,
pub transform: LayerTransform,
@ -260,23 +266,28 @@ fn vv(e: &EffectInstance, id: &str, t: Tick) -> Vec2 {
/// means the canvas centre.
pub fn eval_layer(e: &EffectInstance, t: Tick, frame: (u32, u32)) -> Option<LayerSpec> {
let content = match e.effect.as_str() {
TEXT_LAYER => LayerContent::Text(TextProps {
text: ss(e, "text", t),
font: ss(e, "font", t),
style: ss(e, "font_style", t),
size: ff(e, "size", t).max(0.1),
align: ch(e, "align", t),
tracking: ff(e, "tracking", t),
kerning: bb(e, "kerning", t),
ligatures: bb(e, "ligatures", t),
leading: ff(e, "leading", t),
baseline_shift: ff(e, "baseline_shift", t),
faux_bold: bb(e, "faux_bold", t),
faux_italic: bb(e, "faux_italic", t),
caps: ch(e, "caps", t),
underline: bb(e, "underline", t),
box_width: ff(e, "box_width", t).max(0.0),
vertical: bb(e, "vertical", t),
TEXT_LAYER => LayerContent::Text({
let text = ss(e, "text", t);
let runs = byte_runs(&text, e.layer.as_ref().map_or(&[][..], |x| &x.runs[..]));
TextProps {
text,
runs,
font: ss(e, "font", t),
style: ss(e, "font_style", t),
size: ff(e, "size", t).max(0.1),
align: ch(e, "align", t),
tracking: ff(e, "tracking", t),
kerning: bb(e, "kerning", t),
ligatures: bb(e, "ligatures", t),
leading: ff(e, "leading", t),
baseline_shift: ff(e, "baseline_shift", t),
faux_bold: bb(e, "faux_bold", t),
faux_italic: bb(e, "faux_italic", t),
caps: ch(e, "caps", t),
underline: bb(e, "underline", t),
box_width: ff(e, "box_width", t).max(0.0),
vertical: bb(e, "vertical", t),
}
}),
SHAPE_LAYER => {
let sz = vv(e, "size", t);
@ -316,6 +327,8 @@ pub fn eval_layer(e: &EffectInstance, t: Tick, frame: (u32, u32)) -> Option<Laye
Some(LayerSpec {
name: ss(e, "name", t),
enabled: e.enabled,
uid: e.layer.as_ref().map_or(0, |x| x.uid),
pin: e.layer.as_ref().and_then(|x| x.pin.clone()).filter(|p| p.any()),
content,
appearance: Appearance { fill: bb(e, "fill", t).then(|| cc(e, "fill_color", t)), strokes, background, shadow },
transform: LayerTransform {
@ -328,6 +341,21 @@ pub fn eval_layer(e: &EffectInstance, t: Tick, frame: (u32, u32)) -> Option<Laye
})
}
/// Character-offset style runs → byte ranges of `text` (clamped; empty runs dropped).
pub fn byte_runs(text: &str, runs: &[crate::graphic_design::StyleRun]) -> Vec<(std::ops::Range<usize>, crate::graphic_design::CharStyle)> {
if runs.is_empty() {
return Vec::new();
}
let offs: Vec<usize> = text.char_indices().map(|(b, _)| b).chain(std::iter::once(text.len())).collect();
let at = |c: usize| offs[c.min(offs.len() - 1)];
runs.iter().map(|r| (at(r.start)..at(r.end), r.style.clone())).filter(|(r, s)| r.start < r.end && !s.is_empty()).collect()
}
/// Character count of a text layer's source text at time `t` (for style-run commands).
pub fn text_chars(e: &EffectInstance, t: Tick) -> usize {
ss(e, "text", t).chars().count()
}
fn set(e: &mut EffectInstance, id: &str, v: ParamValue) {
e.params.insert(id.to_string(), Param::new(v));
}

593
crates/project/src/graphic_design.rs Normal file
View File

@ -0,0 +1,593 @@
//! Responsive design and per-character styles of graphic clips (M10.7).
//!
//! - **Per-character styles** ([`StyleRun`], [`CharStyle`]): a text layer's runs override font,
//! style, size, fill, faux bold / italic, underline, tracking, baseline shift and caps for a
//! range of characters. Ranges are in *characters* (not bytes), sorted and non-overlapping;
//! [`apply_char_style`] merges a new style into a range and [`adjust_runs`] keeps runs on the
//! right characters when the text is edited (typed characters take the style of the character
//! before them).
//! - **Responsive Design – Position** ([`Pin`]): a layer's edges pinned to another layer (by its
//! [`LayerExtra::uid`]) or to the video frame keep their distance to the same edges of the
//! target, so a box pinned to a text layer grows with the text. [`resolve_pins`] works on plain
//! bounding boxes; the renderer turns the result into position / size changes.
//! - **Responsive Design – Time** ([`GraphicMeta`]): the intro and outro of a graphic keep their
//! timing when the clip is trimmed or extended; the middle stretches ([`remap_time`]).
//! - **Rolls and crawls** ([`Roll`]): the whole graphic moves up (Roll) or sideways (Crawl Left /
//! Right), optionally starting and ending off screen, with preroll, ease-in, ease-out and
//! postroll ([`roll_offset`], [`roll_progress`]).
use std::collections::BTreeMap;
use filmcraft_time::Tick;
use serde::{Deserialize, Serialize};
use crate::effect::EffectInstance;
fn is_zero(v: &u64) -> bool {
*v == 0
}
/// Extra data of one graphic layer, stored on its effect instance (`EffectInstance::layer`).
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
#[serde(default)]
pub struct LayerExtra {
/// Stable id of the layer within its clip (0 = not assigned yet). Pins and template controls
/// refer to layers by it, so they survive reordering.
#[serde(skip_serializing_if = "is_zero")]
pub uid: u64,
/// Per-character style overrides of a text layer.
#[serde(skip_serializing_if = "Vec::is_empty")]
pub runs: Vec<StyleRun>,
/// Responsive Design – Position.
#[serde(skip_serializing_if = "Option::is_none")]
pub pin: Option<Pin>,
}
impl LayerExtra {
pub fn is_empty(&self) -> bool {
self.uid == 0 && self.runs.is_empty() && self.pin.is_none()
}
}
/// Character formatting overrides (None = inherit the layer's value).
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct CharStyle {
#[serde(skip_serializing_if = "Option::is_none")]
pub font: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub font_style: Option<String>,
/// Font size in px.
#[serde(skip_serializing_if = "Option::is_none")]
pub size: Option<f32>,
/// Fill colour (sRGB, straight alpha).
#[serde(skip_serializing_if = "Option::is_none")]
pub fill: Option<[f32; 4]>,
#[serde(skip_serializing_if = "Option::is_none")]
pub faux_bold: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub faux_italic: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub underline: Option<bool>,
/// Tracking in 1/1000 em.
#[serde(skip_serializing_if = "Option::is_none")]
pub tracking: Option<f32>,
/// Baseline shift in px (positive = up).
#[serde(skip_serializing_if = "Option::is_none")]
pub baseline_shift: Option<f32>,
/// 0 normal, 1 all caps, 2 small caps.
#[serde(skip_serializing_if = "Option::is_none")]
pub caps: Option<u32>,
}
impl CharStyle {
pub fn is_empty(&self) -> bool {
*self == CharStyle::default()
}
/// `self` with every field `over` sets replaced.
pub fn merged(&self, over: &CharStyle) -> CharStyle {
CharStyle {
font: over.font.clone().or_else(|| self.font.clone()),
font_style: over.font_style.clone().or_else(|| self.font_style.clone()),
size: over.size.or(self.size),
fill: over.fill.or(self.fill),
faux_bold: over.faux_bold.or(self.faux_bold),
faux_italic: over.faux_italic.or(self.faux_italic),
underline: over.underline.or(self.underline),
tracking: over.tracking.or(self.tracking),
baseline_shift: over.baseline_shift.or(self.baseline_shift),
caps: over.caps.or(self.caps),
}
}
}
/// A range of characters `start..end` with its own formatting.
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
pub struct StyleRun {
pub start: usize,
pub end: usize,
#[serde(flatten)]
pub style: CharStyle,
}
/// Per-character styles of a text of `n` characters.
fn expand(runs: &[StyleRun], n: usize) -> Vec<CharStyle> {
let mut v = vec![CharStyle::default(); n];
for r in runs {
for c in v.iter_mut().take(r.end.min(n)).skip(r.start.min(n)) {
*c = c.merged(&r.style);
}
}
v
}
/// Runs from per-character styles (adjacent equal styles merged, unstyled characters omitted).
fn compress(v: &[CharStyle]) -> Vec<StyleRun> {
let mut out: Vec<StyleRun> = Vec::new();
for (i, s) in v.iter().enumerate() {
if s.is_empty() {
continue;
}
match out.last_mut() {
Some(r) if r.end == i && r.style == *s => r.end = i + 1,
_ => out.push(StyleRun { start: i, end: i + 1, style: s.clone() }),
}
}
out
}
/// Normalise runs for a text of `n` characters: clamped, sorted, non-overlapping, merged.
pub fn normalize_runs(runs: &[StyleRun], n: usize) -> Vec<StyleRun> {
compress(&expand(runs, n))
}
/// Merge `style` into characters `start..end` of a text of `n` characters.
pub fn apply_char_style(runs: &[StyleRun], n: usize, start: usize, end: usize, style: &CharStyle) -> Vec<StyleRun> {
let mut v = expand(runs, n);
for c in v.iter_mut().take(end.min(n)).skip(start.min(n)) {
*c = c.merged(style);
}
compress(&v)
}
/// Remove every override from characters `start..end`.
pub fn clear_char_style(runs: &[StyleRun], n: usize, start: usize, end: usize) -> Vec<StyleRun> {
let mut v = expand(runs, n);
for c in v.iter_mut().take(end.min(n)).skip(start.min(n)) {
*c = CharStyle::default();
}
compress(&v)
}
/// Keep runs on their characters when `old` text becomes `new` (one contiguous replacement:
/// common prefix and suffix are kept; inserted characters take the style of the character before
/// the insertion point, or after it at the very start).
pub fn adjust_runs(old: &str, new: &str, runs: &[StyleRun]) -> Vec<StyleRun> {
if runs.is_empty() || old == new {
return runs.to_vec();
}
let a: Vec<char> = old.chars().collect();
let b: Vec<char> = new.chars().collect();
let pre = a.iter().zip(&b).take_while(|(x, y)| x == y).count();
let max_suf = a.len().min(b.len()) - pre;
let suf = a.iter().rev().zip(b.iter().rev()).take(max_suf).take_while(|(x, y)| x == y).count();
let v = expand(runs, a.len());
let inserted = b.len() - pre - suf;
let fill = if pre > 0 { v[pre - 1].clone() } else { v.get(a.len() - suf).cloned().unwrap_or_default() };
let mut out: Vec<CharStyle> = v[..pre].to_vec();
out.extend(std::iter::repeat_n(fill, inserted));
out.extend_from_slice(&v[a.len() - suf..]);
compress(&out)
}
/// What a pinned layer follows.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum PinTarget {
/// The video frame (the graphic's canvas).
#[default]
Frame,
/// Another layer of the same clip, by [`LayerExtra::uid`].
Layer(u64),
}
/// Responsive Design – Position: which edges of this layer follow the same edges of the target,
/// and their distances (this layer's edge minus the target's edge, canvas pixels) captured when
/// the pin was made.
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
#[serde(default)]
pub struct Pin {
pub to: PinTarget,
pub left: bool,
pub top: bool,
pub right: bool,
pub bottom: bool,
/// Distances `[left, top, right, bottom]`.
pub offsets: [f64; 4],
}
impl Pin {
pub fn any(&self) -> bool {
self.left || self.top || self.right || self.bottom
}
/// A pin of `edges` (left, top, right, bottom) from layer bounds `own` to target bounds `target`.
pub fn new(to: PinTarget, edges: [bool; 4], own: [f64; 4], target: [f64; 4]) -> Pin {
Pin {
to,
left: edges[0],
top: edges[1],
right: edges[2],
bottom: edges[3],
offsets: [own[0] - target[0], own[1] - target[1], own[2] - target[2], own[3] - target[3]],
}
}
}
/// One layer for [`resolve_pins`]: its uid, pin, unpinned bounds `[x0, y0, x1, y1]` (canvas px)
/// and whether pinning opposite edges may resize it (shapes) or only move it (text).
#[derive(Clone, Debug)]
pub struct PinNode {
pub uid: u64,
pub pin: Option<Pin>,
pub bounds: [f64; 4],
pub resizable: bool,
}
/// New bounds of every layer after pinning (same order as `nodes`). Targets are resolved before
/// the layers pinned to them; pins in a cycle, or to a missing layer, are ignored.
pub fn resolve_pins(nodes: &[PinNode], frame: [f64; 4]) -> Vec<[f64; 4]> {
let mut out: Vec<Option<[f64; 4]>> = vec![None; nodes.len()];
fn solve(i: usize, nodes: &[PinNode], frame: [f64; 4], out: &mut Vec<Option<[f64; 4]>>, visiting: &mut Vec<usize>) -> [f64; 4] {
if let Some(b) = out[i] {
return b;
}
let n = &nodes[i];
let own = n.bounds;
let Some(pin) = n.pin.as_ref().filter(|p| p.any()) else {
out[i] = Some(own);
return own;
};
if visiting.contains(&i) {
return own; // cycle: this pin is ignored
}
visiting.push(i);
let target = match pin.to {
PinTarget::Frame => Some(frame),
PinTarget::Layer(uid) => nodes.iter().position(|m| m.uid == uid && uid != 0).filter(|&j| j != i).map(|j| solve(j, nodes, frame, out, visiting)),
};
visiting.pop();
let Some(t) = target else {
out[i] = Some(own);
return own;
};
let axis = |lo_on: bool, hi_on: bool, lo: usize, hi: usize| -> (f64, f64) {
let size = own[hi] - own[lo];
match (lo_on, hi_on) {
(true, true) if n.resizable => (t[lo] + pin.offsets[lo], t[hi] + pin.offsets[hi]),
(true, _) => (t[lo] + pin.offsets[lo], t[lo] + pin.offsets[lo] + size),
(false, true) => (t[hi] + pin.offsets[hi] - size, t[hi] + pin.offsets[hi]),
(false, false) => (own[lo], own[hi]),
}
};
let (x0, x1) = axis(pin.left, pin.right, 0, 2);
let (y0, y1) = axis(pin.top, pin.bottom, 1, 3);
let b = [x0, y0, x1, y1];
out[i] = Some(b);
b
}
let mut visiting = Vec::new();
(0..nodes.len()).map(|i| solve(i, nodes, frame, &mut out, &mut visiting)).collect()
}
/// Roll / crawl direction (Responsive Design – Time ▸ Roll).
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum RollMode {
#[default]
Off,
/// Moves up (credits).
Roll,
/// Moves right to left (tickers).
CrawlLeft,
/// Moves left to right.
CrawlRight,
}
impl RollMode {
pub const ALL: [RollMode; 4] = [RollMode::Off, RollMode::Roll, RollMode::CrawlLeft, RollMode::CrawlRight];
pub fn label(self) -> &'static str {
match self {
RollMode::Off => "Off",
RollMode::Roll => "Roll",
RollMode::CrawlLeft => "Crawl Left",
RollMode::CrawlRight => "Crawl Right",
}
}
pub fn from_name(s: &str) -> Option<RollMode> {
let n = s.to_ascii_lowercase().replace([' ', '_', '-'], "");
RollMode::ALL.into_iter().find(|m| m.label().to_ascii_lowercase().replace(' ', "") == n || (n == "none" && *m == RollMode::Off))
}
}
/// Roll / crawl options. Times are clip-relative durations.
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct Roll {
pub mode: RollMode,
/// The content starts just outside the frame (below it for a roll, right of it for a left
/// crawl, left of it for a right crawl); otherwise it starts where it was laid out.
pub start_off_screen: bool,
/// The content leaves the frame completely; otherwise it stops when its last edge reaches the
/// frame edge (content that fits stays where it was laid out).
pub end_off_screen: bool,
/// Hold before moving.
pub preroll: Tick,
/// Accelerate over this long.
pub ease_in: Tick,
/// Decelerate over this long.
pub ease_out: Tick,
/// Hold after moving.
pub postroll: Tick,
}
/// Fraction (0..=1) of the roll's travel at clip time `u` of a clip `d` long: still during the
/// preroll, then constant speed with linear acceleration over `ease_in` and deceleration over
/// `ease_out` (a trapezoidal speed profile), still during the postroll.
pub fn roll_progress(r: &Roll, u: Tick, d: Tick) -> f64 {
let ta = r.preroll.0.max(0) as f64;
let tb = (d.0 - r.postroll.0.max(0)) as f64;
let t_total = tb - ta;
let u = u.0 as f64;
if t_total <= 0.0 {
return if u <= ta { 0.0 } else { 1.0 };
}
let tau = (u - ta).clamp(0.0, t_total);
let ei = (r.ease_in.0.max(0) as f64).min(t_total);
let eo = (r.ease_out.0.max(0) as f64).min(t_total - ei);
let v = 1.0 / (t_total - ei / 2.0 - eo / 2.0);
let s = if tau < ei {
v * tau * tau / (2.0 * ei)
} else if tau <= t_total - eo {
v * (ei / 2.0 + (tau - ei))
} else {
let rest = t_total - tau;
1.0 - v * rest * rest / (2.0 * eo)
};
s.clamp(0.0, 1.0)
}
/// Offset (canvas px) of the whole graphic for a roll / crawl at clip time `u` of a clip `d`
/// long. `canvas` is the frame size and `content` the union of the layers' bounds
/// `[x0, y0, x1, y1]` where they were laid out.
pub fn roll_offset(r: &Roll, u: Tick, d: Tick, canvas: (u32, u32), content: [f64; 4]) -> (f64, f64) {
let (w, h) = (canvas.0 as f64, canvas.1 as f64);
let [x0, y0, x1, y1] = content;
let lerp = |a: f64, b: f64| a + (b - a) * roll_progress(r, u, d);
match r.mode {
RollMode::Off => (0.0, 0.0),
RollMode::Roll => {
let from = if r.start_off_screen { h - y0 } else { 0.0 };
let to = if r.end_off_screen { -y1 } else { (h - y1).min(0.0) };
(0.0, lerp(from, to))
}
RollMode::CrawlLeft => {
let from = if r.start_off_screen { w - x0 } else { 0.0 };
let to = if r.end_off_screen { -x1 } else { (w - x1).min(0.0) };
(lerp(from, to), 0.0)
}
RollMode::CrawlRight => {
let from = if r.start_off_screen { -x1 } else { 0.0 };
let to = if r.end_off_screen { w - x0 } else { (-x0).max(0.0) };
(lerp(from, to), 0.0)
}
}
}
/// Graphic-level settings of a graphic clip (`TrackItem::graphic`).
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct GraphicMeta {
#[serde(skip_serializing_if = "is_roll_off")]
pub roll: Roll,
/// Responsive Design – Time: protected intro duration (from the clip start).
pub intro: Tick,
/// Protected outro duration (up to the clip end).
pub outro: Tick,
/// Media time of the clip start when the intro / outro were set; keyframes are read relative
/// to it, so trimming the head keeps the intro playing from the new start.
pub design_in: Tick,
/// Clip length (media time) when the intro / outro were set.
pub design_duration: Tick,
/// The graphics template this graphic was made from, with its editable properties.
#[serde(skip_serializing_if = "Option::is_none")]
pub template: Option<crate::gtemplate::TemplateLink>,
}
fn is_roll_off(r: &Roll) -> bool {
*r == Roll::default()
}
impl GraphicMeta {
pub fn is_empty(&self) -> bool {
*self == GraphicMeta::default()
}
pub fn has_responsive_time(&self) -> bool {
(self.intro.0 > 0 || self.outro.0 > 0) && self.design_duration.0 > 0
}
}
/// Responsive Design – Time: the media time at which to read the graphic's keyframes at
/// clip-relative time `u` of a clip `d` long. Inside the intro, time runs from `design_in`;
/// inside the outro, it runs up to the designed end; the middle is stretched linearly between
/// them. Without an intro or outro this is `design_in + u` (the plain clip time).
pub fn remap_time(m: &GraphicMeta, u: Tick, d: Tick) -> Tick {
if !m.has_responsive_time() {
return m.design_in + u;
}
let (intro, outro, dd) = (m.intro.0.max(0), m.outro.0.max(0), m.design_duration.0);
let (u, d) = (u.0, d.0);
let local = if u < intro {
u
} else if d - u <= outro {
dd - (d - u)
} else {
let (a, b) = (intro as i128, (d - outro) as i128);
let (aa, bb) = (intro as i128, (dd - outro) as i128);
if b <= a { aa as i64 } else { (aa + (u as i128 - a) * (bb - aa) / (b - a)) as i64 }
};
m.design_in + Tick(local)
}
/// A source graphic (Graphics and Titles ▸ Upgrade to Source Graphic): the layers and settings
/// shared by every clip of one project item. Editing one instance updates the others.
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
#[serde(default)]
pub struct SourceGraphic {
pub layers: Vec<EffectInstance>,
#[serde(skip_serializing_if = "Option::is_none")]
pub meta: Option<GraphicMeta>,
}
/// Source graphics by project item.
pub type SourceGraphics = BTreeMap<crate::ItemId, SourceGraphic>;
#[cfg(test)]
mod tests {
use super::*;
use filmcraft_time::TICKS_PER_SECOND;
fn sec(s: f64) -> Tick {
Tick((s * TICKS_PER_SECOND as f64).round() as i64)
}
fn bold() -> CharStyle {
CharStyle { faux_bold: Some(true), ..Default::default() }
}
#[test]
fn char_styles_merge_split_and_follow_edits() {
let runs = apply_char_style(&[], 10, 2, 6, &bold());
assert_eq!(runs, vec![StyleRun { start: 2, end: 6, style: bold() }]);
let red = CharStyle { fill: Some([1.0, 0.0, 0.0, 1.0]), ..Default::default() };
let runs = apply_char_style(&runs, 10, 4, 8, &red);
assert_eq!(runs.len(), 3);
assert_eq!((runs[1].start, runs[1].end), (4, 6));
assert_eq!(runs[1].style, bold().merged(&red));
assert_eq!((runs[2].start, runs[2].end), (6, 8));
// clear the middle
let c = clear_char_style(&runs, 10, 3, 7);
assert_eq!(c.iter().map(|r| (r.start, r.end)).collect::<Vec<_>>(), vec![(2, 3), (7, 8)]);
// typing before the run shifts it; typing inside extends it
let r = vec![StyleRun { start: 6, end: 11, style: bold() }];
let a = adjust_runs("Hello World", "Oh, Hello World", &r);
assert_eq!((a[0].start, a[0].end), (10, 15));
let b = adjust_runs("Hello World", "Hello Wonderful World", &r);
assert_eq!((b[0].start, b[0].end), (6, 21));
// deleting the styled word removes the run
assert!(adjust_runs("Hello World", "Hello ", &r).is_empty());
// runs normalise and clamp
let n = normalize_runs(&[StyleRun { start: 5, end: 50, style: bold() }, StyleRun { start: 0, end: 5, style: bold() }], 8);
assert_eq!(n, vec![StyleRun { start: 0, end: 8, style: bold() }]);
}
#[test]
fn pins_follow_targets_and_resize_shapes() {
// text at x 100..300, box pinned on all edges with 10 px padding
let text = [100.0, 50.0, 300.0, 100.0];
let boxb = [90.0, 40.0, 310.0, 110.0];
let pin = Pin::new(PinTarget::Layer(1), [true; 4], boxb, text);
assert_eq!(pin.offsets, [-10.0, -10.0, 10.0, 10.0]);
// the text grows to 500 px wide: the box follows
let nodes = vec![
PinNode { uid: 2, pin: Some(pin.clone()), bounds: boxb, resizable: true },
PinNode { uid: 1, pin: None, bounds: [100.0, 50.0, 600.0, 100.0], resizable: false },
];
let out = resolve_pins(&nodes, [0.0, 0.0, 1920.0, 1080.0]);
assert_eq!(out[0], [90.0, 40.0, 610.0, 110.0]);
// text layers only move: right edge pinned keeps the distance to the target's right edge
let p2 = Pin::new(PinTarget::Layer(1), [false, false, true, false], [320.0, 50.0, 400.0, 90.0], text);
let nodes = vec![
PinNode { uid: 3, pin: Some(p2), bounds: [320.0, 50.0, 400.0, 90.0], resizable: false },
PinNode { uid: 1, pin: None, bounds: [100.0, 50.0, 600.0, 100.0], resizable: false },
];
assert_eq!(resolve_pins(&nodes, [0.0; 4])[0], [620.0, 50.0, 700.0, 90.0]);
// chains resolve in order; cycles are ignored
let a =
PinNode { uid: 1, pin: Some(Pin::new(PinTarget::Layer(2), [true, false, false, false], [0.0; 4], [0.0; 4])), bounds: [0.0; 4], resizable: false };
let b =
PinNode { uid: 2, pin: Some(Pin::new(PinTarget::Layer(1), [true, false, false, false], [0.0; 4], [0.0; 4])), bounds: [0.0; 4], resizable: false };
let _ = resolve_pins(&[a, b], [0.0; 4]);
// pinned to the frame: a different frame size moves it
let f = Pin::new(PinTarget::Frame, [false, false, true, true], [1700.0, 900.0, 1820.0, 1000.0], [0.0, 0.0, 1920.0, 1080.0]);
let n = [PinNode { uid: 1, pin: Some(f), bounds: [1700.0, 900.0, 1820.0, 1000.0], resizable: false }];
assert_eq!(resolve_pins(&n, [0.0, 0.0, 1280.0, 720.0])[0], [1060.0, 540.0, 1180.0, 640.0]);
}
#[test]
fn roll_progress_is_exact() {
let d = sec(10.0);
let mut r = Roll { mode: RollMode::Roll, ..Default::default() };
// linear without easing
assert_eq!(roll_progress(&r, Tick::ZERO, d), 0.0);
assert!((roll_progress(&r, sec(2.5), d) - 0.25).abs() < 1e-12);
assert_eq!(roll_progress(&r, d, d), 1.0);
// preroll 2 s, postroll 2 s: still, then 6 s of travel
r.preroll = sec(2.0);
r.postroll = sec(2.0);
assert_eq!(roll_progress(&r, sec(1.0), d), 0.0);
assert!((roll_progress(&r, sec(5.0), d) - 0.5).abs() < 1e-12);
assert_eq!(roll_progress(&r, sec(9.0), d), 1.0);
// ease in 2 s, ease out 2 s over 6 s: v = 1/(6-1-1) = 0.25/s
r.ease_in = sec(2.0);
r.ease_out = sec(2.0);
assert!((roll_progress(&r, sec(3.0), d) - 0.25 * 1.0 / (2.0 * 2.0)).abs() < 1e-12);
assert!((roll_progress(&r, sec(4.0), d) - 0.25).abs() < 1e-12);
assert!((roll_progress(&r, sec(5.0), d) - 0.5).abs() < 1e-12);
assert!((roll_progress(&r, sec(7.0), d) - (1.0 - 0.25 / 4.0)).abs() < 1e-12);
}
#[test]
fn roll_and_crawl_offsets() {
let d = sec(4.0);
let canvas = (1920, 1080);
let content = [200.0, 100.0, 1700.0, 2500.0];
let r = Roll { mode: RollMode::Roll, start_off_screen: true, end_off_screen: true, ..Default::default() };
// starts with the top at the frame bottom, ends with the bottom at the frame top
assert_eq!(roll_offset(&r, Tick::ZERO, d, canvas, content), (0.0, 980.0));
assert_eq!(roll_offset(&r, d, d, canvas, content), (0.0, -2500.0));
assert_eq!(roll_offset(&r, sec(2.0), d, canvas, content), (0.0, (980.0 - 2500.0) / 2.0));
// not ending off screen: stops when the bottom reaches the frame bottom
let r2 = Roll { end_off_screen: false, ..r.clone() };
assert_eq!(roll_offset(&r2, d, d, canvas, content), (0.0, 1080.0 - 2500.0));
// short content without off-screen options does not move
let r3 = Roll { mode: RollMode::Roll, ..Default::default() };
assert_eq!(roll_offset(&r3, d, d, canvas, [0.0, 100.0, 10.0, 200.0]), (0.0, 0.0));
let cl = Roll { mode: RollMode::CrawlLeft, start_off_screen: true, end_off_screen: true, ..Default::default() };
let line = [50.0, 900.0, 4050.0, 980.0];
assert_eq!(roll_offset(&cl, Tick::ZERO, d, canvas, line), (1870.0, 0.0));
assert_eq!(roll_offset(&cl, d, d, canvas, line), (-4050.0, 0.0));
let cr = Roll { mode: RollMode::CrawlRight, ..cl.clone() };
assert_eq!(roll_offset(&cr, Tick::ZERO, d, canvas, line), (-4050.0, 0.0));
assert_eq!(roll_offset(&cr, d, d, canvas, line), (1870.0, 0.0));
assert_eq!(RollMode::from_name("crawl left"), Some(RollMode::CrawlLeft));
}
#[test]
fn responsive_time_protects_intro_and_outro() {
let m = GraphicMeta { intro: sec(1.0), outro: sec(1.0), design_in: Tick::ZERO, design_duration: sec(5.0), ..Default::default() };
// design length: identity
for s in [0.0, 0.5, 1.0, 2.5, 4.0, 4.9] {
assert_eq!(remap_time(&m, sec(s), sec(5.0)), sec(s));
}
// extended to 9 s: intro unchanged, outro aligned to the end, middle stretched 3 s → 7 s
assert_eq!(remap_time(&m, sec(0.5), sec(9.0)), sec(0.5));
assert_eq!(remap_time(&m, sec(8.5), sec(9.0)), sec(4.5));
assert_eq!(remap_time(&m, sec(8.0), sec(9.0)), sec(4.0));
assert_eq!(remap_time(&m, sec(4.5), sec(9.0)), sec(2.5));
// trimmed to 3 s: intro and outro still play in full, the middle shrinks
assert_eq!(remap_time(&m, sec(0.75), sec(3.0)), sec(0.75));
assert_eq!(remap_time(&m, sec(2.25), sec(3.0)), sec(4.25));
// no responsive time: plain clip time from design_in
let plain = GraphicMeta { design_in: sec(1.0), ..Default::default() };
assert_eq!(remap_time(&plain, sec(2.0), sec(3.0)), sec(3.0));
}
}

764
crates/project/src/gtemplate.rs Normal file
View File

@ -0,0 +1,764 @@
//! Graphics templates (`.fcgt`): FilmCraft's own format for reusable graphics with editable
//! properties, the counterpart of motion graphics templates in other editors.
//!
//! A template file is UTF-8 JSON ([`GraphicsTemplate`]):
//!
//! ```json
//! {
//! "format": "filmcraft.graphicsTemplate",
//! "version": 1,
//! "id": "user:my-lower-third",
//! "name": "My Lower Third",
//! "category": "Lower Thirds",
//! "description": "", "author": "", "license": "", "tags": [],
//! "canvas": [1920, 1080],
//! "duration": 1270080000000,
//! "layers": [ /* graphic_text / graphic_shape effect instances, as in a project file */ ],
//! "graphic": { /* roll, intro / outro (GraphicMeta) */ },
//! "controls": [ { "id": "name", "name": "Name", "kind": "text", "layer": 1, "param": "text" } ],
//! "resources": { "fonts": [ { "family": "…", "style": "…", "license": "OFL-1.1", "data": "<base64>" } ] }
//! }
//! ```
//!
//! - `version` is the format version; readers refuse newer versions.
//! - Layers are referenced by their [`LayerExtra::uid`](crate::graphic_design::LayerExtra); every
//! layer of a template has one.
//! - `controls` are the properties a user of the template edits (Essential Graphics ▸ Edit):
//! text, colour, slider (with `min` / `max`), checkbox, font and position, each bound to one
//! layer parameter (`enabled` = the layer's visibility).
//! - `resources.fonts` optionally embeds font files (base64) that the template needs; installing
//! the template registers them. Only embed fonts whose licence allows redistribution.
//!
//! FilmCraft never reads Adobe `.mogrt` files (or any other application's template format);
//! [`GraphicsTemplate::from_bytes`] rejects ZIP archives with a message saying so.
//!
//! The built-in templates ([`builtin_templates`]) are original designs made for FilmCraft in code
//! (no image, font or template files), under the project licence.
use filmcraft_geom::Vec2;
use filmcraft_time::{TICKS_PER_SECOND, Tick};
use serde::{Deserialize, Serialize};
use crate::effect::EffectInstance;
use crate::graphic::{self, new_shape_layer, new_text_layer};
use crate::graphic_design::{GraphicMeta, LayerExtra, Pin, PinTarget, Roll, RollMode};
use crate::keyframe::{Param, ParamValue};
pub const TEMPLATE_FORMAT: &str = "filmcraft.graphicsTemplate";
pub const TEMPLATE_VERSION: u32 = 1;
pub const TEMPLATE_EXTENSION: &str = "fcgt";
/// Kind of an editable template property.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum ControlKind {
#[default]
Text,
Color,
Slider,
Checkbox,
Font,
Position,
}
impl ControlKind {
pub const ALL: [ControlKind; 6] =
[ControlKind::Text, ControlKind::Color, ControlKind::Slider, ControlKind::Checkbox, ControlKind::Font, ControlKind::Position];
pub fn name(self) -> &'static str {
match self {
ControlKind::Text => "text",
ControlKind::Color => "color",
ControlKind::Slider => "slider",
ControlKind::Checkbox => "checkbox",
ControlKind::Font => "font",
ControlKind::Position => "position",
}
}
pub fn from_name(s: &str) -> Option<ControlKind> {
let s = s.to_ascii_lowercase();
let s = if s == "colour" { "color".to_string() } else { s };
ControlKind::ALL.into_iter().find(|k| k.name() == s)
}
/// The kind that edits parameter `param` of a layer.
pub fn for_param(param: &str) -> ControlKind {
match param {
"text" | "name" => ControlKind::Text,
"font" | "font_style" => ControlKind::Font,
"position" | "anchor" => ControlKind::Position,
"enabled" => ControlKind::Checkbox,
p if p.ends_with("_color") => ControlKind::Color,
p => match crate::find_effect(graphic::TEXT_LAYER)
.and_then(|d| d.param(p))
.or_else(|| crate::find_effect(graphic::SHAPE_LAYER).and_then(|d| d.param(p)))
{
Some(d) => match &d.default {
ParamValue::Bool(_) => ControlKind::Checkbox,
ParamValue::Color(_) => ControlKind::Color,
ParamValue::Vec2(_) => ControlKind::Position,
ParamValue::Text(_) => ControlKind::Text,
_ => ControlKind::Slider,
},
None => ControlKind::Slider,
},
}
}
}
/// One editable property of a template (Essential Graphics ▸ Edit).
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
#[serde(default)]
pub struct TemplateControl {
/// Stable id (`[a-z0-9_-]`), unique in the template.
pub id: String,
/// Label shown to the user.
pub name: String,
pub kind: ControlKind,
/// The layer (by uid) and parameter id it edits (`enabled` = layer visibility).
pub layer: u64,
pub param: String,
/// Slider range.
#[serde(skip_serializing_if = "Option::is_none")]
pub min: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub max: Option<f64>,
}
/// The template a placed graphic came from (kept on the clip, `GraphicMeta::template`).
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
#[serde(default)]
pub struct TemplateLink {
pub id: String,
pub name: String,
pub controls: Vec<TemplateControl>,
}
/// A font embedded in a template.
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
#[serde(default)]
pub struct FontResource {
pub family: String,
pub style: String,
/// SPDX id of the font's licence (must allow redistribution).
pub license: String,
/// Original file name (informational).
pub file: String,
/// The font file, base64.
pub data: String,
}
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
#[serde(default)]
pub struct Resources {
#[serde(skip_serializing_if = "Vec::is_empty")]
pub fonts: Vec<FontResource>,
}
impl Resources {
pub fn is_empty(&self) -> bool {
self.fonts.is_empty()
}
}
/// A graphics template (`.fcgt`).
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(default)]
pub struct GraphicsTemplate {
pub format: String,
pub version: u32,
pub id: String,
pub name: String,
pub category: String,
pub description: String,
pub author: String,
pub license: String,
pub tags: Vec<String>,
/// Frame size the template was designed for.
pub canvas: [u32; 2],
/// Default clip duration.
pub duration: Tick,
pub layers: Vec<EffectInstance>,
/// Roll / responsive time settings (`template` is not stored here).
pub graphic: GraphicMeta,
pub controls: Vec<TemplateControl>,
#[serde(skip_serializing_if = "Resources::is_empty")]
pub resources: Resources,
}
impl Default for GraphicsTemplate {
fn default() -> Self {
GraphicsTemplate {
format: TEMPLATE_FORMAT.into(),
version: TEMPLATE_VERSION,
id: String::new(),
name: String::new(),
category: "My Templates".into(),
description: String::new(),
author: String::new(),
license: String::new(),
tags: Vec::new(),
canvas: [1920, 1080],
duration: Tick(5 * TICKS_PER_SECOND),
layers: Vec::new(),
graphic: GraphicMeta::default(),
controls: Vec::new(),
resources: Resources::default(),
}
}
}
/// Problems reading or validating a template.
#[derive(Debug, Clone, PartialEq, thiserror::Error)]
pub enum TemplateError {
#[error(
"this file is a ZIP archive (an Adobe .mogrt or another application's package?): FilmCraft does not read motion graphics templates from other applications, only its own .fcgt graphics templates"
)]
ForeignPackage,
#[error("not a FilmCraft graphics template: {0}")]
NotTemplate(String),
#[error("this graphics template was made by a newer FilmCraft (format version {0}; this build reads up to {TEMPLATE_VERSION})")]
TooNew(u32),
#[error("invalid graphics template: {0}")]
Invalid(String),
}
/// Lower-case slug of a name (`My Lower Third` → `my-lower-third`).
pub fn slug(name: &str) -> String {
let mut s = String::new();
for c in name.chars() {
if c.is_ascii_alphanumeric() {
s.push(c.to_ascii_lowercase());
} else if !s.ends_with('-') && !s.is_empty() {
s.push('-');
}
}
let s = s.trim_end_matches('-').to_string();
if s.is_empty() { "template".into() } else { s }
}
impl GraphicsTemplate {
/// Serialize (pretty JSON).
pub fn to_json(&self) -> String {
serde_json::to_string_pretty(self).unwrap_or_default()
}
/// Read a template file. ZIP archives (such as `.mogrt` packages) are refused without being
/// inspected.
pub fn from_bytes(bytes: &[u8]) -> Result<GraphicsTemplate, TemplateError> {
if bytes.starts_with(b"PK\x03\x04") || bytes.starts_with(b"PK\x05\x06") {
return Err(TemplateError::ForeignPackage);
}
let text =
std::str::from_utf8(bytes.strip_prefix(b"\xef\xbb\xbf").unwrap_or(bytes)).map_err(|_| TemplateError::NotTemplate("not UTF-8 text".into()))?;
let v: serde_json::Value = serde_json::from_str(text).map_err(|e| TemplateError::NotTemplate(e.to_string()))?;
if v.get("format").and_then(|f| f.as_str()) != Some(TEMPLATE_FORMAT) {
return Err(TemplateError::NotTemplate(format!("`format` is not \"{TEMPLATE_FORMAT}\"")));
}
let ver = v.get("version").and_then(|x| x.as_u64()).unwrap_or(0) as u32;
if ver > TEMPLATE_VERSION {
return Err(TemplateError::TooNew(ver));
}
if ver == 0 {
return Err(TemplateError::Invalid("missing `version`".into()));
}
let t: GraphicsTemplate = serde_json::from_value(v).map_err(|e| TemplateError::Invalid(e.to_string()))?;
t.validate()?;
Ok(t)
}
/// Check the template's structure: layers are graphic layers with unique uids, controls
/// point at existing layers and parameters, ids are unique.
pub fn validate(&self) -> Result<(), TemplateError> {
let bad = |m: String| Err(TemplateError::Invalid(m));
if self.name.trim().is_empty() {
return bad("the template has no name".into());
}
if self.layers.is_empty() {
return bad("the template has no layers".into());
}
if self.canvas[0] == 0 || self.canvas[1] == 0 || self.canvas[0] > 16384 || self.canvas[1] > 16384 {
return bad(format!("bad canvas size {:?}", self.canvas));
}
let mut uids = Vec::new();
for (i, l) in self.layers.iter().enumerate() {
if !graphic::is_layer(l) {
return bad(format!("layer {i} is `{}`, not a graphic layer", l.effect));
}
let uid = layer_uid(l);
if uid == 0 || uids.contains(&uid) {
return bad(format!("layer {i} has no unique uid"));
}
uids.push(uid);
}
let mut ids = Vec::new();
for c in &self.controls {
if c.id.is_empty() || ids.contains(&c.id) {
return bad(format!("control `{}` has no unique id", c.name));
}
ids.push(c.id.clone());
let Some(l) = self.layers.iter().find(|l| layer_uid(l) == c.layer) else {
return bad(format!("control `{}` points at a missing layer", c.name));
};
if c.param != "enabled" && !l.params.contains_key(&c.param) {
return bad(format!("control `{}`: the layer has no parameter `{}`", c.name, c.param));
}
}
for f in &self.resources.fonts {
if base64_decode(&f.data).is_none() {
return bad(format!("font `{}` is not valid base64", f.family));
}
}
Ok(())
}
/// The layers and settings for a graphic clip on a `canvas`-sized frame. When the frame size
/// differs from the template's, the layers are scaled uniformly (fit) and centred, and pin
/// distances scale with them. The result links back to the template.
pub fn instantiate(&self, canvas: (u32, u32)) -> (Vec<EffectInstance>, GraphicMeta) {
let (w, h) = (canvas.0 as f64, canvas.1 as f64);
let (tw, th) = (self.canvas[0] as f64, self.canvas[1] as f64);
let s = (w / tw).min(h / th);
let off = Vec2::new((w - tw * s) / 2.0, (h - th * s) / 2.0);
let mut layers = self.layers.clone();
if (s - 1.0).abs() > 1e-9 || off.x.abs() > 1e-9 || off.y.abs() > 1e-9 {
for l in &mut layers {
if let Some(p) = l.params.get_mut("position") {
p.map_values(|v| match v {
ParamValue::Vec2(q) if !q.x.is_nan() => ParamValue::Vec2(Vec2::new(q.x * s + off.x, q.y * s + off.y)),
o => o,
});
}
if let Some(p) = l.params.get_mut("scale") {
p.map_values(|v| match v {
ParamValue::Float(x) => ParamValue::Float(x * s),
o => o,
});
}
if let Some(pin) = l.layer.as_mut().and_then(|x| x.pin.as_mut()) {
pin.offsets.iter_mut().for_each(|o| *o *= s);
}
}
}
let mut meta = self.graphic.clone();
meta.template = Some(TemplateLink { id: self.id.clone(), name: self.name.clone(), controls: self.controls.clone() });
(layers, meta)
}
}
/// A layer's uid (0 = none).
pub fn layer_uid(e: &EffectInstance) -> u64 {
e.layer.as_ref().map_or(0, |x| x.uid)
}
/// Give every layer without a uid a fresh one (unique among `layers`).
pub fn ensure_uids(layers: &mut [EffectInstance]) {
let mut next = layers.iter().map(layer_uid).max().unwrap_or(0) + 1;
for l in layers.iter_mut() {
if layer_uid(l) == 0 {
l.layer.get_or_insert_with(Default::default).uid = next;
next += 1;
}
}
}
// ------------------------------------------------------------------------------------------------
// base64 (RFC 4648, standard alphabet, padded)
// ------------------------------------------------------------------------------------------------
const B64: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
pub fn base64_encode(data: &[u8]) -> String {
let mut out = String::with_capacity(data.len().div_ceil(3) * 4);
for c in data.chunks(3) {
let n = (c[0] as u32) << 16 | (*c.get(1).unwrap_or(&0) as u32) << 8 | *c.get(2).unwrap_or(&0) as u32;
for k in 0..4 {
if k <= c.len() {
out.push(B64[(n >> (18 - 6 * k) & 63) as usize] as char);
} else {
out.push('=');
}
}
}
out
}
pub fn base64_decode(s: &str) -> Option<Vec<u8>> {
let mut out = Vec::with_capacity(s.len() / 4 * 3);
let mut acc = 0u32;
let mut bits = 0;
for b in s.bytes().filter(|b| !b.is_ascii_whitespace()) {
if b == b'=' {
break;
}
let v = B64.iter().position(|&x| x == b)? as u32;
acc = acc << 6 | v;
bits += 6;
if bits >= 8 {
bits -= 8;
out.push((acc >> bits) as u8);
acc &= (1 << bits) - 1;
}
}
Some(out)
}
// ------------------------------------------------------------------------------------------------
// Built-in templates (original designs)
// ------------------------------------------------------------------------------------------------
fn set(e: &mut EffectInstance, id: &str, v: ParamValue) {
e.params.insert(id.to_string(), Param::new(v));
}
fn rgb(hex: u32) -> ParamValue {
let c = |s: u32| ((hex >> s) & 0xff) as f32 / 255.0;
ParamValue::Color([c(16), c(8), c(0), 1.0])
}
fn text(uid: u64, name: &str, s: &str, pos: (f64, f64), size: f64, style: &str) -> EffectInstance {
let mut e = new_text_layer(s, Vec2::new(pos.0, pos.1), size);
set(&mut e, "name", ParamValue::Text(name.into()));
set(&mut e, "font", ParamValue::Text("Inter".into()));
set(&mut e, "font_style", ParamValue::Text(style.into()));
e.layer = Some(Box::new(LayerExtra { uid, ..Default::default() }));
e
}
fn shape(uid: u64, name: &str, kind: u32, pos: (f64, f64), size: (f64, f64), color: u32) -> EffectInstance {
let mut e = new_shape_layer(kind, Vec2::new(pos.0, pos.1), Vec2::new(size.0, size.1), vec![]);
set(&mut e, "name", ParamValue::Text(name.into()));
set(&mut e, "fill_color", rgb(color));
e.layer = Some(Box::new(LayerExtra { uid, ..Default::default() }));
e
}
fn pin(e: &mut EffectInstance, to: u64, edges: [bool; 4], offsets: [f64; 4]) {
let p =
Pin { to: if to == 0 { PinTarget::Frame } else { PinTarget::Layer(to) }, left: edges[0], top: edges[1], right: edges[2], bottom: edges[3], offsets };
e.layer.get_or_insert_with(Default::default).pin = Some(p);
}
fn ctl(id: &str, name: &str, layer: u64, param: &str) -> TemplateControl {
TemplateControl { id: id.into(), name: name.into(), kind: ControlKind::for_param(param), layer, param: param.into(), min: None, max: None }
}
fn slider(id: &str, name: &str, layer: u64, param: &str, min: f64, max: f64) -> TemplateControl {
TemplateControl { min: Some(min), max: Some(max), ..ctl(id, name, layer, param) }
}
fn builtin(
id: &str,
name: &str,
category: &str,
description: &str,
seconds: i64,
layers: Vec<EffectInstance>,
controls: Vec<TemplateControl>,
) -> GraphicsTemplate {
GraphicsTemplate {
id: format!("builtin:{id}"),
name: name.into(),
category: category.into(),
description: description.into(),
author: "FilmCraft contributors".into(),
license: "MIT OR Apache-2.0".into(),
tags: vec![category.to_ascii_lowercase()],
duration: Tick(seconds * TICKS_PER_SECOND),
layers,
controls,
..Default::default()
}
}
/// The built-in templates: original FilmCraft designs (lower thirds, titles, end credits,
/// callouts, a ticker), defined here in code. Designed at 1920×1080 with the bundled Inter font.
pub fn builtin_templates() -> Vec<GraphicsTemplate> {
let mut v = Vec::new();
// Lower third: a slab that grows with the name, a small accent square and a role line.
{
let mut slab = shape(1, "Slab", 0, (400.0, 900.0), (560.0, 150.0), 0x1d4ed8);
set(&mut slab, "opacity", ParamValue::Float(92.0));
pin(&mut slab, 2, [true, false, true, false], [-40.0, 0.0, 48.0, 0.0]);
let mut accent = shape(4, "Accent", 0, (130.0, 900.0), (12.0, 150.0), 0xfacc15);
pin(&mut accent, 1, [true, false, false, false], [-12.0, 0.0, 0.0, 0.0]);
let mut name = text(2, "Name", "Alex Morgan", (160.0, 895.0), 62.0, "SemiBold");
set(&mut name, "fill_color", rgb(0xffffff));
let mut role = text(3, "Role", "Field Producer", (162.0, 950.0), 36.0, "Regular");
set(&mut role, "fill_color", rgb(0xdbeafe));
v.push(builtin(
"lower-third-slab",
"Lower Third – Slab",
"Lower Thirds",
"Name and role on a coloured slab that widens with the name.",
6,
vec![slab, accent, name, role],
vec![
ctl("name", "Name", 2, "text"),
ctl("role", "Role", 3, "text"),
ctl("slab_color", "Slab Color", 1, "fill_color"),
ctl("accent_color", "Accent Color", 4, "fill_color"),
ctl("text_color", "Name Color", 2, "fill_color"),
ctl("show_role", "Show Role", 3, "enabled"),
ctl("font", "Font", 2, "font"),
],
));
}
// Lower third: name with a rule underneath that stretches to the name's width.
{
let mut name = text(1, "Name", "Sam Rivera", (140.0, 900.0), 58.0, "Bold");
let shadow = |e: &mut EffectInstance| {
set(e, "shadow", ParamValue::Bool(true));
set(e, "shadow_distance", ParamValue::Float(3.0));
set(e, "shadow_blur", ParamValue::Float(12.0));
set(e, "shadow_opacity", ParamValue::Float(60.0));
};
shadow(&mut name);
let mut rule = shape(2, "Rule", 0, (300.0, 920.0), (320.0, 5.0), 0x22d3ee);
pin(&mut rule, 1, [true, false, true, false], [0.0, 0.0, 0.0, 0.0]);
let mut role = text(3, "Role", "Lead Engineer", (140.0, 966.0), 34.0, "Medium");
set(&mut role, "tracking", ParamValue::Float(40.0));
shadow(&mut role);
v.push(builtin(
"lower-third-rule",
"Lower Third – Rule",
"Lower Thirds",
"Name over a thin rule that always matches the name's width.",
6,
vec![name, rule, role],
vec![
ctl("name", "Name", 1, "text"),
ctl("role", "Role", 3, "text"),
ctl("rule_color", "Rule Color", 2, "fill_color"),
ctl("font", "Font", 1, "font"),
slider("name_size", "Name Size", 1, "size", 24.0, 140.0),
],
));
}
// Centred title with a subtitle and a short rule between them.
{
let mut title = text(1, "Title", "THE LONG WAY HOME", (960.0, 520.0), 112.0, "Bold");
set(&mut title, "align", ParamValue::Choice(1));
set(&mut title, "tracking", ParamValue::Float(60.0));
let mut rule = shape(2, "Rule", 0, (960.0, 556.0), (220.0, 4.0), 0xf97316);
set(&mut rule, "opacity", ParamValue::Float(90.0));
let mut sub = text(3, "Subtitle", "a story in three parts", (960.0, 628.0), 44.0, "Italic");
set(&mut sub, "align", ParamValue::Choice(1));
set(&mut sub, "fill_color", rgb(0xe5e7eb));
v.push(builtin(
"title-centered",
"Title – Centered",
"Titles",
"Large centred title, accent rule and subtitle.",
5,
vec![title, rule, sub],
vec![
ctl("title", "Title", 1, "text"),
ctl("subtitle", "Subtitle", 3, "text"),
slider("title_size", "Title Size", 1, "size", 40.0, 240.0),
ctl("title_color", "Title Color", 1, "fill_color"),
ctl("accent", "Accent Color", 2, "fill_color"),
ctl("font", "Font", 1, "font"),
],
));
}
// Title in a box (text background) with a soft shadow.
{
let mut title = text(1, "Title", "Chapter One", (960.0, 560.0), 96.0, "SemiBold");
set(&mut title, "align", ParamValue::Choice(1));
set(&mut title, "fill_color", rgb(0x111827));
set(&mut title, "background", ParamValue::Bool(true));
set(&mut title, "background_color", rgb(0xfef3c7));
set(&mut title, "background_opacity", ParamValue::Float(100.0));
set(&mut title, "background_size", ParamValue::Float(36.0));
set(&mut title, "background_radius", ParamValue::Float(10.0));
set(&mut title, "shadow", ParamValue::Bool(true));
set(&mut title, "shadow_distance", ParamValue::Float(8.0));
set(&mut title, "shadow_blur", ParamValue::Float(30.0));
set(&mut title, "shadow_opacity", ParamValue::Float(45.0));
v.push(builtin(
"title-boxed",
"Title – Boxed",
"Titles",
"A title on a rounded card with a soft shadow.",
5,
vec![title],
vec![
ctl("title", "Title", 1, "text"),
ctl("text_color", "Text Color", 1, "fill_color"),
ctl("box_color", "Box Color", 1, "background_color"),
slider("box_opacity", "Box Opacity", 1, "background_opacity", 0.0, 100.0),
ctl("position", "Position", 1, "position"),
],
));
}
// End credits that roll from below the frame to above it.
{
let mut heading = text(1, "Heading", "CREDITS", (960.0, 160.0), 72.0, "Bold");
set(&mut heading, "align", ParamValue::Choice(1));
set(&mut heading, "tracking", ParamValue::Float(120.0));
let body = "Directed by\nJordan Lee\n\nWritten by\nPriya Natarajan\n\nEdited by\nMarco Bianchi\n\nMusic by\nHana Kobayashi\n\nMade with FilmCraft";
let mut credits = text(2, "Credits", body, (960.0, 300.0), 46.0, "Regular");
set(&mut credits, "align", ParamValue::Choice(1));
set(&mut credits, "leading", ParamValue::Float(10.0));
let mut t = builtin(
"end-credits-roll",
"End Credits – Roll",
"End Credits",
"Centred credits that roll up from below the frame and leave at the top.",
12,
vec![heading, credits],
vec![
ctl("heading", "Heading", 1, "text"),
ctl("credits", "Credits", 2, "text"),
ctl("font", "Font", 2, "font"),
ctl("color", "Text Color", 2, "fill_color"),
],
);
t.graphic.roll = Roll { mode: RollMode::Roll, start_off_screen: true, end_off_screen: true, ..Default::default() };
v.push(t);
}
// Callout: a label on a box with a pointer that follows it.
{
let mut label = text(1, "Label", "Look here", (1260.0, 380.0), 48.0, "SemiBold");
set(&mut label, "fill_color", rgb(0xffffff));
set(&mut label, "background", ParamValue::Bool(true));
set(&mut label, "background_color", rgb(0xdc2626));
set(&mut label, "background_opacity", ParamValue::Float(100.0));
set(&mut label, "background_size", ParamValue::Float(18.0));
set(&mut label, "background_radius", ParamValue::Float(6.0));
let mut dot = shape(2, "Pointer", 1, (1200.0, 470.0), (34.0, 34.0), 0xdc2626);
set(&mut dot, "stroke", ParamValue::Bool(true));
set(&mut dot, "stroke_color", rgb(0xffffff));
set(&mut dot, "stroke_width", ParamValue::Float(5.0));
pin(&mut dot, 1, [true, true, false, false], [-60.0, 55.0, 0.0, 0.0]);
v.push(builtin(
"callout-pointer",
"Callout – Pointer",
"Callouts",
"A boxed label with a round pointer; move the label and the pointer follows.",
4,
vec![dot, label],
vec![
ctl("label", "Label", 1, "text"),
ctl("color", "Color", 1, "background_color"),
ctl("pointer", "Pointer Color", 2, "fill_color"),
ctl("position", "Position", 1, "position"),
],
));
}
// Callout: a pill-shaped tag around its text, with a status dot.
{
let mut tag = shape(1, "Tag", 0, (300.0, 140.0), (300.0, 70.0), 0x0f172a);
set(&mut tag, "corner_radius", ParamValue::Float(35.0));
set(&mut tag, "opacity", ParamValue::Float(88.0));
pin(&mut tag, 2, [true; 4], [-70.0, -18.0, 30.0, 18.0]);
let mut dot = shape(3, "Dot", 1, (190.0, 140.0), (22.0, 22.0), 0x22c55e);
pin(&mut dot, 2, [true, false, false, false], [-44.0, 0.0, 0.0, 0.0]);
let mut label = text(2, "Text", "LIVE", (220.0, 155.0), 40.0, "Bold");
set(&mut label, "tracking", ParamValue::Float(80.0));
v.push(builtin(
"callout-tag",
"Callout – Tag",
"Callouts",
"A rounded tag that fits its text, with a status dot.",
4,
vec![tag, dot, label],
vec![
ctl("text", "Text", 2, "text"),
ctl("tag_color", "Tag Color", 1, "fill_color"),
ctl("dot_color", "Dot Color", 3, "fill_color"),
ctl("show_dot", "Show Dot", 3, "enabled"),
],
));
}
// Ticker: a crawl from right to left.
{
let mut line = text(
1,
"Ticker",
"Breaking: the river festival moves to Saturday • Roads close at 6 pm • Shuttle buses run every 15 minutes",
(40.0, 1010.0),
44.0,
"Medium",
);
set(&mut line, "background", ParamValue::Bool(true));
set(&mut line, "background_color", rgb(0x111827));
set(&mut line, "background_opacity", ParamValue::Float(85.0));
set(&mut line, "background_size", ParamValue::Float(14.0));
let mut t = builtin(
"ticker-crawl",
"Ticker – Crawl",
"Lower Thirds",
"A news-style line that crawls in from the right and leaves at the left.",
15,
vec![line],
vec![ctl("text", "Text", 1, "text"), ctl("text_color", "Text Color", 1, "fill_color"), ctl("bar_color", "Bar Color", 1, "background_color")],
);
t.graphic.roll = Roll { mode: RollMode::CrawlLeft, start_off_screen: true, end_off_screen: true, ..Default::default() };
v.push(t);
}
v
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn base64_round_trip() {
for n in 0..20 {
let data: Vec<u8> = (0..n).map(|i| (i * 37 + 11) as u8).collect();
assert_eq!(base64_decode(&base64_encode(&data)).unwrap(), data);
}
assert_eq!(base64_encode(b"Man"), "TWFu");
assert_eq!(base64_encode(b"Ma"), "TWE=");
assert!(base64_decode("@@@").is_none());
}
#[test]
fn builtins_are_valid_and_round_trip() {
let all = builtin_templates();
assert!(all.len() >= 8);
let mut ids: Vec<&str> = all.iter().map(|t| t.id.as_str()).collect();
ids.dedup();
assert_eq!(ids.len(), all.len());
for cat in ["Lower Thirds", "Titles", "End Credits", "Callouts"] {
assert!(all.iter().any(|t| t.category == cat), "{cat}");
}
for t in all {
t.validate().unwrap_or_else(|e| panic!("{}: {e}", t.name));
let back = GraphicsTemplate::from_bytes(t.to_json().as_bytes()).unwrap();
assert_eq!(back, t);
}
}
#[test]
fn refuses_foreign_and_newer_files() {
assert_eq!(GraphicsTemplate::from_bytes(b"PK\x03\x04rest"), Err(TemplateError::ForeignPackage));
assert!(matches!(GraphicsTemplate::from_bytes(b"{\"format\":\"x\"}"), Err(TemplateError::NotTemplate(_))));
let mut t = builtin_templates().remove(0);
t.version = 99;
assert_eq!(GraphicsTemplate::from_bytes(t.to_json().as_bytes()), Err(TemplateError::TooNew(99)));
let mut t = builtin_templates().remove(0);
t.controls[0].layer = 999;
assert!(matches!(t.validate(), Err(TemplateError::Invalid(_))));
}
#[test]
fn instantiates_on_other_frame_sizes() {
let t = builtin_templates().into_iter().find(|t| t.id == "builtin:title-centered").unwrap();
let (same, meta) = t.instantiate((1920, 1080));
assert_eq!(same, t.layers);
assert_eq!(meta.template.as_ref().unwrap().id, t.id);
let (half, _) = t.instantiate((960, 540));
assert_eq!(half[0].params["position"].value, ParamValue::Vec2(Vec2::new(480.0, 260.0)));
assert_eq!(half[0].params["scale"].value, ParamValue::Float(50.0));
// a vertical frame: fit to width, centred vertically
let (tall, _) = t.instantiate((1080, 1920));
let ParamValue::Vec2(p) = tall[0].params["position"].value else { panic!() };
assert!((p.x - 540.0).abs() < 1e-9 && (p.y - (520.0 * 0.5625 + (1920.0 - 1080.0 * 0.5625) / 2.0)).abs() < 1e-9);
assert_eq!(slug("My Lower Third!"), "my-lower-third");
assert_eq!(ControlKind::for_param("background_opacity"), ControlKind::Slider);
assert_eq!(ControlKind::for_param("shadow"), ControlKind::Checkbox);
}
}

7
crates/project/src/keyframe.rs
View File

@ -160,6 +160,13 @@ impl Param {
pub fn is_animated(&self) -> bool {
!self.keyframes.is_empty()
}
/// Apply `f` to the static value and every keyframe value.
pub fn map_values(&mut self, f: impl Fn(ParamValue) -> ParamValue) {
self.value = f(std::mem::replace(&mut self.value, ParamValue::Float(0.0)));
for k in &mut self.keyframes {
k.value = f(std::mem::replace(&mut k.value, ParamValue::Float(0.0)));
}
}
/// Set a value at `t`: updates the static value, or adds/replaces a keyframe when animated.
pub fn set_at(&mut self, t: Tick, value: ParamValue) {

20
crates/project/src/lib.rs
View File

@ -12,6 +12,8 @@ pub mod effect;
pub mod essential;
pub mod find;
pub mod graphic;
pub mod graphic_design;
pub mod gtemplate;
pub mod keyframe;
pub mod mask;
pub mod mixer;
@ -30,6 +32,7 @@ pub use caption::{Caption, CaptionAlign, CaptionAnchor, CaptionFormat, CaptionSt
pub use effect::{EffectDef, EffectInstance, EffectKind, ParamDef, ParamKind, effect_defs, find_effect};
pub use essential::{AudioType, EssentialSound};
pub use find::{FindOp, FindQuery, FindRow, SearchBin};
pub use graphic_design::{CharStyle, GraphicMeta, LayerExtra, Pin, PinTarget, Roll, RollMode, SourceGraphic, StyleRun};
pub use keyframe::{Interpolation, Keyframe, Param, ParamValue};
pub use mask::{Mask, MaskMode, MaskPath, MaskVertex, TrackMethod};
pub use mixer::{AutomationMode, InputMap, MixerStrip, TrackSend};
@ -533,6 +536,10 @@ pub struct TrackItem {
/// left/right). Empty = the first two channels (mono sources on both sides).
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub source_channels: Vec<u16>,
/// Graphic clips: roll / crawl, responsive time and the template the graphic came from
/// ([`GraphicMeta`]). Schema v12.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub graphic: Option<Box<GraphicMeta>>,
}
/// Time interpolation for speed-changed clips (Clip ▸ Video Options ▸ Time Interpolation).
@ -1085,6 +1092,10 @@ pub struct Project {
/// Schema v11.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub search_bins: Vec<SearchBin>,
/// Source graphics (Graphics and Titles ▸ Upgrade to Source Graphic): shared layers by
/// project item. Schema v12.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub source_graphics: BTreeMap<ItemId, SourceGraphic>,
}
/// A LUT imported into the project (`lut.import`). Lumetri refers to it as `lib:<id>`.
@ -1118,6 +1129,7 @@ impl Project {
luts: Vec::new(),
transcripts: BTreeMap::new(),
search_bins: Vec::new(),
source_graphics: BTreeMap::new(),
}
}
@ -1261,10 +1273,15 @@ impl Project {
let it = self.items.get(&item)?;
let name = it.name.clone();
let label = it.label;
let effects = match kind {
let mut effects = match kind {
TrackKind::Video => effect::intrinsic_video(),
TrackKind::Audio => effect::intrinsic_audio(),
};
// a source graphic edits in with its shared layers and settings
let source_graphic = self.source_graphics.get(&item).filter(|_| kind == TrackKind::Video).cloned();
if let Some(sg) = &source_graphic {
effects.extend(sg.layers.iter().cloned());
}
// a multi-camera source sequence edits in as a multi-camera clip showing its first angle
let multicam =
it.as_sequence().and_then(|q| q.multicam.as_ref()).map(|m| MulticamSel { enabled: true, angle: m.first_video_angle().unwrap_or(0) as u32 });
@ -1298,6 +1315,7 @@ impl Project {
hold_filters: false,
field_options: None,
source_channels,
graphic: source_graphic.and_then(|sg| sg.meta.map(Box::new)),
})
}

342
crates/render/src/graphic_clip.rs
View File

@ -2,16 +2,21 @@
//! with fill, up to two strokes (outer / centre / inner), a background box and a drop shadow.
//! Each layer is rasterised into a tight premultiplied linear-light image that is cached while
//! the layer and its transform stay the same.
//!
//! [`item_layer_specs`] evaluates a graphic clip's layers the way they are shown: keyframes read
//! through Responsive Design – Time, per-character style runs, responsive pins resolved (pinned
//! layers move, pinned shapes resize), and the roll / crawl offset applied to every layer.
use std::collections::HashMap;
use std::hash::{Hash, Hasher};
use std::sync::{Arc, Mutex, OnceLock};
use filmcraft_geom::{Affine, Vec2};
use filmcraft_project::EffectInstance;
use filmcraft_project::graphic::{LayerContent, LayerSpec, ShapeProps, TextProps, eval_layer};
use filmcraft_project::graphic_design::{GraphicMeta, PinNode, remap_time, resolve_pins, roll_offset};
use filmcraft_project::{EffectInstance, TrackItem};
use filmcraft_text::raster::{Xform, apply, fill_into, xform_scale};
use filmcraft_text::{Align, Caps, Layout, Mask, ParagraphStyle, StrokeKind, TextStyle, layout, mask, render};
use filmcraft_text::{Align, Caps, Layout, Mask, ParagraphStyle, StrokeKind, StyleRun, TextStyle, layout_rich, mask, render};
use filmcraft_time::Tick;
use rayon::prelude::*;
@ -42,11 +47,7 @@ pub fn text_styles(t: &TextProps) -> (TextStyle, ParagraphStyle) {
baseline_shift: t.baseline_shift,
faux_bold: t.faux_bold,
faux_italic: t.faux_italic,
caps: match t.caps {
1 => Caps::All,
2 => Caps::Small,
_ => Caps::Normal,
},
caps: caps_of(t.caps),
underline: t.underline,
},
ParagraphStyle {
@ -64,10 +65,42 @@ pub fn text_styles(t: &TextProps) -> (TextStyle, ParagraphStyle) {
)
}
fn caps_of(c: u32) -> Caps {
match c {
1 => Caps::All,
2 => Caps::Small,
_ => Caps::Normal,
}
}
/// The per-character style runs of a text layer as text-engine runs (overrides applied to the
/// layer's style).
pub fn text_runs(t: &TextProps, base: &TextStyle) -> Vec<StyleRun> {
t.runs
.iter()
.map(|(r, c)| StyleRun {
range: r.clone(),
style: TextStyle {
family: c.font.clone().filter(|f| !f.is_empty()).unwrap_or_else(|| base.family.clone()),
style: c.font_style.clone().filter(|f| !f.is_empty()).unwrap_or_else(|| base.style.clone()),
size: c.size.map_or(base.size, |s| s.max(0.1)),
tracking: c.tracking.unwrap_or(base.tracking),
baseline_shift: c.baseline_shift.unwrap_or(base.baseline_shift),
faux_bold: c.faux_bold.unwrap_or(base.faux_bold),
faux_italic: c.faux_italic.unwrap_or(base.faux_italic),
caps: c.caps.map_or(base.caps, caps_of),
underline: c.underline.unwrap_or(base.underline),
..base.clone()
},
})
.collect()
}
/// The laid-out text of a text layer (cached by the text engine).
pub fn text_layout(t: &TextProps) -> Arc<Layout> {
let (st, ps) = text_styles(t);
layout(&t.text, &st, &ps)
let runs = text_runs(t, &st);
layout_rich(&t.text, &st, &runs, &ps)
}
/// A shape layer's outline in layer pixels (centred on the layer origin).
@ -261,7 +294,31 @@ pub fn raster_layer(spec: &LayerSpec, m: &Xform, cw: usize, ch: usize) -> Option
}
}
if let Some(c) = ap.fill {
over_mask(&mut px, &cover, premul_linear(c));
// per-character fills: each style run with its own colour is drawn on its own
let run_fills: Vec<Option<[f32; 4]>> = match &spec.content {
LayerContent::Text(t) => t.runs.iter().map(|(_, s)| s.fill).collect(),
_ => Vec::new(),
};
match &text {
Some(l) if run_fills.iter().any(Option::is_some) => {
let mut groups: Vec<([f32; 4], Vec<u16>)> = Vec::new();
for r in 0..=run_fills.len() {
let col = if r == 0 { c } else { run_fills[r - 1].unwrap_or(c) };
match groups.iter_mut().find(|g| g.0 == col) {
Some(g) => g.1.push(r as u16),
None => groups.push((col, vec![r as u16])),
}
}
for (col, runs) in groups {
let mut m_run = Mask::new(w, h);
for r in runs {
render::draw_run(l, m, &mut m_run, (x0, y0), Some(r));
}
over_mask(&mut px, &m_run, premul_linear(col));
}
}
_ => over_mask(&mut px, &cover, premul_linear(c)),
}
}
for ((sm, c), (_, _, k)) in strokes.iter().zip(&ap.strokes).rev() {
if !outer(*k) {
@ -333,25 +390,143 @@ pub fn composite_raster(img: &mut Image, r: &LayerRaster, dx: i32, dy: i32) {
});
}
/// Evaluated layers of a graphic clip's effect list at clip time `t`, in paint order.
pub fn graphic_specs(effects: &[EffectInstance], t: Tick, canvas: (u32, u32)) -> Vec<LayerSpec> {
effects.iter().filter(|e| filmcraft_project::graphic::is_layer(e)).filter_map(|e| eval_layer(e, t, canvas)).collect()
/// Axis-aligned bounds `[x0, y0, x1, y1]` of a layer's content box in canvas pixels.
pub fn layer_bounds(spec: &LayerSpec) -> [f64; 4] {
let q = layer_quad(spec);
let mut b = [f64::MAX, f64::MAX, f64::MIN, f64::MIN];
for p in q {
b = [b[0].min(p.x), b[1].min(p.y), b[2].max(p.x), b[3].max(p.y)];
}
b
}
/// Draw all layers of a graphic clip onto `out`; `m` maps graphic canvas pixels to `out` pixels.
pub fn render_graphic(effects: &[EffectInstance], t: Tick, canvas: (u32, u32), m: &Affine, out: &mut Image) {
for spec in graphic_specs(effects, t, canvas) {
let lm = xform(&m.then_apply(&layer_matrix(&spec)));
if let Some(r) = raster_layer_cached(&spec, &lm, out.w, out.h) {
/// Resolve responsive pins of evaluated layers in place: pinned layers move, shapes pinned on
/// opposite edges resize.
pub fn apply_pins(specs: &mut [LayerSpec], canvas: (u32, u32)) {
if !specs.iter().any(|s| s.pin.is_some()) {
return;
}
let nodes: Vec<PinNode> = specs
.iter()
.map(|s| PinNode { uid: s.uid, pin: s.pin.clone(), bounds: layer_bounds(s), resizable: matches!(s.content, LayerContent::Shape(_)) })
.collect();
let out = resolve_pins(&nodes, [0.0, 0.0, canvas.0 as f64, canvas.1 as f64]);
for ((s, n), b) in specs.iter_mut().zip(&nodes).zip(out) {
if b == n.bounds {
continue;
}
let (ow, oh) = (n.bounds[2] - n.bounds[0], n.bounds[3] - n.bounds[1]);
let (nw, nh) = (b[2] - b[0], b[3] - b[1]);
if let LayerContent::Shape(sh) = &mut s.content {
let kx = if ow > 1e-9 { nw / ow } else { 1.0 };
let ky = if oh > 1e-9 { nh / oh } else { 1.0 };
if (kx - 1.0).abs() > 1e-12 || (ky - 1.0).abs() > 1e-12 {
sh.size = ((sh.size.0 as f64 * kx) as f32, (sh.size.1 as f64 * ky) as f32);
sh.points.iter_mut().for_each(|p| *p = [(p[0] as f64 * kx) as f32, (p[1] as f64 * ky) as f32]);
}
}
let now = layer_bounds(s);
s.transform.position = Vec2::new(s.transform.position.x + b[0] - now[0], s.transform.position.y + b[1] - now[1]);
}
}
/// Evaluated layers of a graphic clip's effect list at clip time `t`, in paint order, with pins
/// resolved.
pub fn graphic_specs(effects: &[EffectInstance], t: Tick, canvas: (u32, u32)) -> Vec<LayerSpec> {
let mut v: Vec<LayerSpec> = effects.iter().filter(|e| filmcraft_project::graphic::is_layer(e)).filter_map(|e| eval_layer(e, t, canvas)).collect();
apply_pins(&mut v, canvas);
v
}
/// The media time at which a graphic clip's layers are read at media time `mt` (Responsive Design
/// – Time), and the clip-relative time and clip length (media time) for rolls.
pub fn graphic_time(item: &TrackItem, mt: Tick) -> (Tick, Tick, Tick) {
let u = mt - item.source_in;
let d = Tick((item.duration.0 as f64 * item.speed.abs().max(1e-9)).round() as i64);
let t = match item.graphic.as_deref() {
Some(m) if m.has_responsive_time() => remap_time(m, u, d),
_ => mt,
};
(t, u, d)
}
/// Union of the layers' bounds (with background padding) for rolls and crawls.
fn content_bounds(specs: &[LayerSpec]) -> Option<[f64; 4]> {
let mut b: Option<[f64; 4]> = None;
for s in specs.iter().filter(|s| s.enabled) {
let mut lb = layer_bounds(s);
if let Some((_, pad, _)) = s.appearance.background {
let k = s.transform.scale.x.abs().max(s.transform.scale.y.abs());
let p = pad as f64 * k;
lb = [lb[0] - p, lb[1] - p, lb[2] + p, lb[3] + p];
}
b = Some(match b {
Some(a) => [a[0].min(lb[0]), a[1].min(lb[1]), a[2].max(lb[2]), a[3].max(lb[3])],
None => lb,
});
}
b
}
/// A graphic clip's layers at media time `mt` as shown: `(effect index, layer)` in paint order,
/// with Responsive Design – Time, pins and the roll / crawl offset applied. Includes hidden
/// layers (their `enabled` is false).
pub fn item_layer_specs(item: &TrackItem, mt: Tick, canvas: (u32, u32)) -> Vec<(usize, LayerSpec)> {
let (t, u, d) = graphic_time(item, mt);
let mut idx = Vec::new();
let mut specs = Vec::new();
for (i, e) in item.effects.iter().enumerate() {
if filmcraft_project::graphic::is_layer(e)
&& let Some(s) = eval_layer(e, t, canvas)
{
idx.push(i);
specs.push(s);
}
}
apply_pins(&mut specs, canvas);
if let Some(meta) = item.graphic.as_deref() {
apply_roll(meta, &mut specs, u, d, canvas);
}
idx.into_iter().zip(specs).collect()
}
/// Move every layer by the roll / crawl offset at clip time `u` of a clip `d` long.
pub fn apply_roll(meta: &GraphicMeta, specs: &mut [LayerSpec], u: Tick, d: Tick, canvas: (u32, u32)) {
if meta.roll.mode == filmcraft_project::RollMode::Off {
return;
}
let Some(cb) = content_bounds(specs) else { return };
let (dx, dy) = roll_offset(&meta.roll, u, d, canvas, cb);
for s in specs.iter_mut() {
s.transform.position = Vec2::new(s.transform.position.x + dx, s.transform.position.y + dy);
}
}
/// Draw all layers of a graphic clip at media time `mt` onto `out`; `m` maps graphic canvas
/// pixels to `out` pixels.
pub fn render_graphic(item: &TrackItem, mt: Tick, canvas: (u32, u32), m: &Affine, out: &mut Image) {
let specs: Vec<LayerSpec> = item_layer_specs(item, mt, canvas).into_iter().map(|(_, s)| s).collect();
draw_specs(&specs, m, out);
}
/// Draw a graphic's layers (an effect list, without clip settings) at time `t` onto `out`.
pub fn render_graphic_layers(effects: &[EffectInstance], t: Tick, canvas: (u32, u32), m: &Affine, out: &mut Image) {
draw_specs(&graphic_specs(effects, t, canvas), m, out);
}
fn draw_specs(specs: &[LayerSpec], m: &Affine, out: &mut Image) {
for spec in specs {
let lm = xform(&m.then_apply(&layer_matrix(spec)));
if let Some(r) = raster_layer_cached(spec, &lm, out.w, out.h) {
composite_raster(out, &r, 0, 0);
}
}
}
/// All layers composed into one tight image (for the GPU plan): `(image, x, y)` in output pixels.
pub fn render_graphic_tight(effects: &[EffectInstance], t: Tick, canvas: (u32, u32), m: &Affine, w: usize, h: usize) -> Option<(Image, i32, i32)> {
pub fn render_graphic_tight(item: &TrackItem, mt: Tick, canvas: (u32, u32), m: &Affine, w: usize, h: usize) -> Option<(Image, i32, i32)> {
let rasters: Vec<Arc<LayerRaster>> =
graphic_specs(effects, t, canvas).iter().filter_map(|spec| raster_layer_cached(spec, &xform(&m.then_apply(&layer_matrix(spec))), w, h)).collect();
item_layer_specs(item, mt, canvas).iter().filter_map(|(_, spec)| raster_layer_cached(spec, &xform(&m.then_apply(&layer_matrix(spec))), w, h)).collect();
let x0 = rasters.iter().map(|r| r.x).min()?.max(0);
let y0 = rasters.iter().map(|r| r.y).min()?.max(0);
let x1 = rasters.iter().map(|r| r.x + r.img.w as i32).max()?.min(w as i32);
@ -376,11 +551,125 @@ mod tests {
e.params.get_mut(k).unwrap_or_else(|| panic!("{k}")).value = v;
}
/// A 5 s graphic clip holding `layers` (canvas 400×300 at 25 fps).
fn graphic_item(layers: Vec<EffectInstance>) -> TrackItem {
let mut p = filmcraft_project::Project::new("t");
let rate = filmcraft_time::FrameRate::FPS_25;
let g = p.add_item("G", filmcraft_project::Label::Rose, filmcraft_project::ItemKind::Graphic { width: 400, height: 300, rate }, None);
let mut it = p
.make_track_item(g, filmcraft_project::TrackKind::Video, Tick::ZERO, filmcraft_time::TimeRange::new(Tick::ZERO, Tick::from_seconds_f64(5.0)), rate)
.unwrap();
it.effects.extend(layers);
it
}
fn ink_box(img: &Image) -> Option<[usize; 4]> {
let mut b: Option<[usize; 4]> = None;
for y in 0..img.h {
for x in 0..img.w {
if img.get(x, y)[3] > 0.5 {
b = Some(match b {
Some(a) => [a[0].min(x), a[1].min(y), a[2].max(x), a[3].max(y)],
None => [x, y, x, y],
});
}
}
}
b
}
#[test]
fn per_character_fill_and_size() {
use filmcraft_project::graphic_design::{CharStyle, LayerExtra, StyleRun};
let mut e = new_text_layer("AAAA", Vec2::new(20.0, 100.0), 60.0);
set(&mut e, "fill_color", ParamValue::Color([1.0, 1.0, 1.0, 1.0]));
let red = CharStyle { fill: Some([1.0, 0.0, 0.0, 1.0]), ..Default::default() };
e.layer = Some(Box::new(LayerExtra { runs: vec![StyleRun { start: 2, end: 4, style: red }], ..Default::default() }));
let mut img = Image::new(400, 200);
render_graphic_layers(&[e.clone()], Tick::ZERO, (400, 200), &Affine::IDENTITY, &mut img);
let l = text_layout(match &eval_layer(&e, Tick::ZERO, (400, 200)).unwrap().content {
LayerContent::Text(t) => t,
_ => unreachable!(),
});
let split = 20.0 + l.caret(2).0;
let (mut white, mut red_px) = (0, 0);
for y in 0..200 {
for x in 0..400 {
let p = img.get(x, y);
if p[3] > 0.9 {
if p[1] > 0.9 {
white += 1;
assert!((x as f32) < split + 1.0, "white only in the first half at {x}");
} else if p[0] > 0.9 && p[1] < 0.05 {
red_px += 1;
assert!((x as f32) > split - 1.0, "red only in the second half at {x}");
}
}
}
}
assert!(white > 200 && red_px > 200, "{white} {red_px}");
// a bigger run is taller and sits on the same baseline
let big = CharStyle { size: Some(120.0), ..Default::default() };
e.layer = Some(Box::new(LayerExtra { runs: vec![StyleRun { start: 3, end: 4, style: big }], ..Default::default() }));
let mut img2 = Image::new(400, 200);
render_graphic_layers(&[e], Tick::ZERO, (400, 200), &Affine::IDENTITY, &mut img2);
let (b1, b2) = (ink_box(&img).unwrap(), ink_box(&img2).unwrap());
assert!(b2[1] + 30 < b1[1], "the big letter rises higher: {b1:?} {b2:?}");
assert!(b2[3].abs_diff(b1[3]) <= 1, "same baseline: {b1:?} {b2:?}");
}
#[test]
fn pinned_box_follows_text_growth() {
use filmcraft_project::graphic_design::{LayerExtra, Pin, PinTarget};
let mut t = new_text_layer("Hi", Vec2::new(50.0, 100.0), 40.0);
t.layer = Some(Box::new(LayerExtra { uid: 1, ..Default::default() }));
let mut bx = new_shape_layer(0, Vec2::new(60.0, 90.0), Vec2::new(40.0, 40.0), vec![]);
let tb = layer_bounds(&eval_layer(&t, Tick::ZERO, (400, 300)).unwrap());
let padded = [tb[0] - 10.0, tb[1] - 10.0, tb[2] + 10.0, tb[3] + 10.0];
let pin = Pin::new(PinTarget::Layer(1), [true; 4], padded, tb);
bx.layer = Some(Box::new(LayerExtra { uid: 2, pin: Some(pin), ..Default::default() }));
for text in ["Hi", "Hello there, world"] {
set(&mut t, "text", ParamValue::Text(text.into()));
let specs = graphic_specs(&[bx.clone(), t.clone()], Tick::ZERO, (400, 300));
let (b, tt) = (layer_bounds(&specs[0]), layer_bounds(&specs[1]));
for k in 0..4 {
let want = tt[k] + if k < 2 { -10.0 } else { 10.0 };
assert!((b[k] - want).abs() < 1e-3, "{text}: edge {k}: {b:?} vs {tt:?}");
}
}
}
#[test]
fn roll_moves_the_whole_graphic() {
use filmcraft_project::graphic_design::{GraphicMeta, Roll, RollMode};
let e = new_shape_layer(0, Vec2::new(200.0, 150.0), Vec2::new(100.0, 100.0), vec![]);
let mut item = graphic_item(vec![e]);
item.graphic = Some(Box::new(GraphicMeta {
roll: Roll { mode: RollMode::Roll, start_off_screen: true, end_off_screen: true, ..Default::default() },
..Default::default()
}));
let at = |item: &TrackItem, s: f64| {
let sp = item_layer_specs(item, Tick::from_seconds_f64(s), (400, 300));
layer_bounds(&sp[0].1)
};
// the box (y 100..200) starts with its top at the frame bottom and ends above the frame
assert!((at(&item, 0.0)[1] - 300.0).abs() < 1e-6);
assert!(at(&item, 5.0)[3].abs() < 1e-6);
// half way: offset (200 + (-200)) / 2 = 0 → back where it was laid out
assert!((at(&item, 2.5)[1] - 100.0).abs() < 1e-6, "{:?}", at(&item, 2.5));
let mut img = Image::new(400, 300);
render_graphic(&item, Tick::from_seconds_f64(2.5), (400, 300), &Affine::IDENTITY, &mut img);
assert!(img.get(200, 150)[3] > 0.9, "centre at mid-roll");
let mut img0 = Image::new(400, 300);
render_graphic(&item, Tick::ZERO, (400, 300), &Affine::IDENTITY, &mut img0);
assert!(ink_box(&img0).is_none(), "off screen at the start");
}
#[test]
fn text_layer_renders_white_text_at_position() {
let e = new_text_layer("HELLO", Vec2::new(100.0, 150.0), 60.0);
let mut img = Image::new(400, 200);
render_graphic(&[e], Tick::ZERO, (400, 200), &Affine::IDENTITY, &mut img);
render_graphic_layers(&[e], Tick::ZERO, (400, 200), &Affine::IDENTITY, &mut img);
let ink: Vec<(usize, usize)> = (0..200).flat_map(|y| (0..400).map(move |x| (x, y))).filter(|&(x, y)| img.get(x, y)[3] > 0.5).collect();
assert!(ink.len() > 500);
let minx = ink.iter().map(|p| p.0).min().unwrap();
@ -401,7 +690,7 @@ mod tests {
set(&mut e, "shadow_distance", ParamValue::Float(20.0));
set(&mut e, "shadow_blur", ParamValue::Float(0.0));
let mut img = Image::new(200, 200);
render_graphic(&[e], Tick::ZERO, (200, 200), &Affine::IDENTITY, &mut img);
render_graphic_layers(&[e], Tick::ZERO, (200, 200), &Affine::IDENTITY, &mut img);
let p = img.get(100, 100);
assert!(p[2] > 0.9 && p[0] < 0.05, "blue fill {p:?}");
let s = img.get(100, 78);
@ -422,7 +711,7 @@ mod tests {
set(&mut e, "stroke_type", ParamValue::Choice(kind));
set(&mut e, "stroke_color", ParamValue::Color([0.0, 1.0, 0.0, 1.0]));
let mut img = Image::new(200, 200);
render_graphic(&[e], Tick::ZERO, (200, 200), &Affine::IDENTITY, &mut img);
render_graphic_layers(&[e], Tick::ZERO, (200, 200), &Affine::IDENTITY, &mut img);
// diamond: top vertex at y = 40; probe straight below/above the right vertex (160, 100)
let green = |x: usize| {
let p = img.get(x, 100);
@ -440,12 +729,12 @@ mod tests {
set(&mut e, "rotation", ParamValue::Float(90.0));
set(&mut e, "opacity", ParamValue::Float(50.0));
let mut img = Image::new(200, 200);
render_graphic(&[e.clone()], Tick::ZERO, (200, 200), &Affine::IDENTITY, &mut img);
render_graphic_layers(&[e.clone()], Tick::ZERO, (200, 200), &Affine::IDENTITY, &mut img);
assert!((img.get(100, 60)[3] - 0.5).abs() < 0.02, "vertical after 90°");
assert_eq!(img.get(60, 100)[3], 0.0);
// a 2× output transform (e.g. Motion scale) draws vectors crisply at the larger size
let mut big = Image::new(400, 400);
render_graphic(&[e], Tick::ZERO, (200, 200), &Affine::scale(2.0, 2.0), &mut big);
render_graphic_layers(&[e], Tick::ZERO, (200, 200), &Affine::scale(2.0, 2.0), &mut big);
assert!((big.get(200, 110)[3] - 0.5).abs() < 0.02);
let q = layer_quad(&eval_layer(&find_effect("graphic_shape").unwrap().instance(), Tick::ZERO, (200, 200)).unwrap());
assert!((q[0].x - (100.0 - 200.0)).abs() < 1e-6 && (q[2].y - (100.0 + 100.0)).abs() < 1e-6);
@ -457,8 +746,9 @@ mod tests {
let b = new_shape_layer(1, Vec2::new(150.0, 120.0), Vec2::new(60.0, 60.0), vec![]);
let fx = vec![b, a];
let mut full = Image::new(220, 160);
render_graphic(&fx, Tick::ZERO, (220, 160), &Affine::IDENTITY, &mut full);
let (tight, x, y) = render_graphic_tight(&fx, Tick::ZERO, (220, 160), &Affine::IDENTITY, 220, 160).unwrap();
let item = graphic_item(fx);
render_graphic(&item, Tick::ZERO, (220, 160), &Affine::IDENTITY, &mut full);
let (tight, x, y) = render_graphic_tight(&item, Tick::ZERO, (220, 160), &Affine::IDENTITY, 220, 160).unwrap();
assert!(tight.w < 220 && tight.h < 160);
for yy in 0..tight.h {
for xx in 0..tight.w {

4
crates/render/src/lib.rs
View File

@ -295,7 +295,7 @@ pub(crate) fn item_layer(
// vectors straight to the output: no resampling, crisp at any Motion scale
let m = Affine::scale(opts.scale as f64, opts.scale as f64).then_apply(&motion);
let mut canvas = Image::new(w, h);
graphic_clip::render_graphic(&item.effects, mt, src_size, &m, &mut canvas);
graphic_clip::render_graphic(item, mt, src_size, &m, &mut canvas);
let (op, bl) = opacity_blend(item, mt);
return Some((canvas, op, bl));
}
@ -391,7 +391,7 @@ pub(crate) fn base_layer(
// standard effects work on the graphic at source resolution, then Motion places it
let (gw, gh) = (((src_size.0 as f32 * want).ceil() as usize).max(1), ((src_size.1 as f32 * want).ceil() as usize).max(1));
let mut img = Image::new(gw, gh);
graphic_clip::render_graphic(&item.effects, mt, src_size, &Affine::scale(want as f64, want as f64), &mut img);
graphic_clip::render_graphic(item, mt, src_size, &Affine::scale(want as f64, want as f64), &mut img);
img
}
})

2
crates/render/src/plan.rs
View File

@ -171,7 +171,7 @@ fn push_item(
let Some(size) = crate::source_size(project, item.item) else { return };
let (w, h) = output_size(seq, opts.scale);
let m = Affine::scale(opts.scale as f64, opts.scale as f64).then_apply(&motion_matrix(seq, item, size, mt));
if let Some((img, x, y)) = crate::graphic_clip::render_graphic_tight(&item.effects, mt, size, &m, w, h) {
if let Some((img, x, y)) = crate::graphic_clip::render_graphic_tight(item, mt, size, &m, w, h) {
out.push(PlanLayer { frame: cpu_frame(img), matrix: Affine::translate(x as f64, y as f64), opacity: op * extra_opacity });
}
return;

3
crates/text/README.md
View File

@ -8,6 +8,7 @@ and the Timecode / Clip Name effects. Pure Rust; it builds for `wasm32-unknown-u
|---|---|
| `fonts` | Font database. Bundled faces (always present, also on the web): Inter Regular / Medium / SemiBold / Bold / Italic, JetBrains Mono Regular, Noto Serif Regular — all SIL OFL 1.1, unmodified, in `assets/fonts/` with `.attribution` sidecars. More faces come from `FontSource`s: `DirectorySource::system()` scans the platform font folders on first need (only the `name` and `OS/2` tables are read, via `sfnt`); on wasm the scan finds nothing and hosts can call `add_font_data`. `resolve(family, style)` picks a face, synthesises bold/italic when the family lacks them, and falls back to Inter (flagged `missing`). `fallback_for` finds a face for characters the chosen face lacks. |
| `layout` | Paragraph layout: per-character font fallback → bidi levels (UAX #9) → items (runs of one face, direction and size) shaped with harfrust (OpenType kerning, ligatures, marks, complex scripts) → tracking → line breaking at UAX #14 opportunities (area text) → visual reordering per line → alignment (left / centre / right / justify), leading (added to the natural line height), baseline shift, all caps / synthesised small caps (78 %), underline. Every character boundary gets a caret stop (ligatures are split evenly), so editors can place carets, hit-test and draw selections. Layouts are cached by text + style (512 entries, LRU). |
| `layout_rich` | Per-character style runs (`StyleRun`: a byte range with its own `TextStyle`): runs split shaping items (font, size, tracking, caps, faux styles), baseline shift and underline apply per run, a line holding a larger run is pushed down from the line above (baseline distance = max(base line height, previous descent + this line's ascent + leading)), and every glyph / underline carries its style index so renderers can colour runs (`render::draw_run`). |
| `raster` | Our anti-aliased rasteriser (non-zero winding, 5 sub-scanlines, exact horizontal coverage, active-edge list), contour paths with transforms, rounded rectangles, ellipses, polygons and outline emboldening. Coverage is linear, so compositing in linear light is correct. |
| `render` | Glyph outlines (cached per face + glyph) and glyph masks (cached per size in ⅛ px, ¼-px horizontal phase, synthetic style). Axis-aligned uniform transforms use the mask cache with sub-pixel x positioning; rotated / skewed / non-uniform transforms fill the transformed outlines of the whole run in one pass. Synthetic bold grows outlines by 2.5 % of the size; synthetic italic slants by 12°. |
| `mask` | Outer / centre / inner strokes from an exact Euclidean distance transform (Felzenszwalb–Huttenlocher) with sub-pixel refinement, a three-pass box blur (≈ Gaussian) for shadows, offsets. |
@ -25,6 +26,6 @@ on an M-series Mac: about 1.6 ms cold (shaping, outline extraction, glyph raster
## Not yet
Per-character styling within one text layer (the whole layer has one style), vertical text, OpenType
OpenType
feature toggles beyond kerning/ligatures, variable-font axes, hinting, colour (emoji) glyphs, and
hyphenation. Words longer than an area-text line overflow instead of breaking mid-word.

4
crates/text/src/fonts.rs
View File

@ -171,6 +171,10 @@ impl Face {
.as_ref()
.map(FaceBytes::get)
}
/// A copy of the font file's bytes (None if it is missing or unreadable).
pub fn data(&self) -> Option<Vec<u8>> {
self.bytes().map(<[u8]>::to_vec)
}
/// The parsed font (None if the file is missing or unreadable).
pub fn font(&self) -> Option<FontRef<'_>> {
FontRef::from_index(self.bytes()?, self.info.index).ok()

231
crates/text/src/layout.rs
View File

@ -2,6 +2,12 @@
//! ligatures, mark positioning, complex scripts), line breaking (UAX #14), alignment, leading,
//! tracking, baseline shift, all caps / small caps, underline. Results are cached.
//!
//! **Per-character styles** ([`layout_rich`]): [`StyleRun`]s give byte ranges their own
//! [`TextStyle`] (font, size, tracking, baseline shift, faux bold / italic, caps, underline).
//! Runs split shaping items, lines grow to fit the largest run, and every [`Glyph`] carries the
//! index of the style it was set in (`run`: 0 = the base style, `i + 1` = `runs[i]`), so renderers
//! can colour runs.
//!
//! Coordinates are pixels, y down. For **point text** (`ParagraphStyle::width == None`) the
//! origin is the alignment point on the first baseline: x = 0 is the left edge (left / justify),
//! the centre (centre) or the right edge (right). For **area text** (`width == Some(w)`) the
@ -125,6 +131,56 @@ pub struct Glyph {
pub cluster: usize,
pub synth_bold: bool,
pub synth_italic: bool,
/// Style the glyph was set in: 0 = the base style, `i + 1` = `runs[i]` of [`layout_rich`].
pub run: u16,
}
/// A byte range of the text set in its own style ([`layout_rich`]). Later runs win where runs
/// overlap; ranges are clamped to the text.
#[derive(Clone, Debug, PartialEq)]
pub struct StyleRun {
pub range: Range<usize>,
pub style: TextStyle,
}
impl Hash for StyleRun {
fn hash<H: Hasher>(&self, h: &mut H) {
self.range.hash(h);
self.style.hash(h);
}
}
/// The styles of one layout: `list[0]` is the base style, `list[i + 1]` is run `i`.
struct Styles<'a> {
list: Vec<&'a TextStyle>,
prim: Vec<Resolved>,
/// Style index of every byte (None = all base style).
per_byte: Option<Vec<u16>>,
}
impl<'a> Styles<'a> {
fn new(text: &str, base: &'a TextStyle, runs: &'a [StyleRun]) -> Self {
let mut list = vec![base];
list.extend(runs.iter().map(|r| &r.style));
let prim = list.iter().map(|s| fonts::resolve(&s.family, &s.style)).collect();
let per_byte = (!runs.is_empty()).then(|| {
let mut v = vec![0u16; text.len() + 1];
for (i, r) in runs.iter().enumerate() {
let (a, b) = (r.range.start.min(text.len()), r.range.end.min(text.len()));
for x in v.iter_mut().take(b).skip(a) {
*x = (i + 1).min(u16::MAX as usize) as u16;
}
}
v
});
Styles { list, prim, per_byte }
}
fn at(&self, byte: usize) -> usize {
self.per_byte.as_ref().and_then(|v| v.get(byte)).map_or(0, |&i| i as usize)
}
fn rich(&self) -> bool {
self.per_byte.is_some()
}
}
/// One laid-out line.
@ -149,6 +205,8 @@ pub struct Layout {
pub lines: Vec<Line>,
/// Underline rectangles `[x0, y0, x1, y1]`.
pub underlines: Vec<[f32; 4]>,
/// Style index of each underline (parallel to `underlines`; see [`Glyph::run`]).
pub underline_runs: Vec<u16>,
/// Logical bounds `[x0, y0, x1, y1]` (line boxes; area text spans the box width).
pub bounds: [f32; 4],
/// The face requested was missing (substituted).
@ -241,6 +299,8 @@ struct Item {
range: Range<usize>,
rtl: bool,
glyphs: Vec<ShapedGlyph>,
/// Style index ([`Styles`]).
style: usize,
}
fn upper_single(c: char) -> char {
@ -297,10 +357,14 @@ struct ParaLine {
left_trim: f32,
ascent: f32,
descent: f32,
/// Underlined segments `(x0, x1, style)` in line coordinates (rich layouts only).
underlines: Vec<(f32, f32, u16)>,
}
/// Lay out one paragraph (no newlines) whose text starts at byte `base` of the whole string.
fn paragraph(text: &str, base: usize, style: &TextStyle, para: &ParagraphStyle, primary: Resolved) -> Vec<ParaLine> {
fn paragraph(text: &str, base: usize, sty: &Styles, para: &ParagraphStyle) -> Vec<ParaLine> {
let style = sty.list[0];
let primary = sty.prim[0];
let pm = fonts::face(primary.face).metrics(style.size);
if text.is_empty() {
return vec![ParaLine {
@ -311,41 +375,45 @@ fn paragraph(text: &str, base: usize, style: &TextStyle, para: &ParagraphStyle,
left_trim: 0.0,
ascent: pm.ascent,
descent: pm.descent,
underlines: Vec::new(),
}];
}
let default_level = para.rtl.map(|r| if r { Level::rtl() } else { Level::ltr() });
let bidi = BidiInfo::new(text, default_level);
let pinfo = &bidi.paragraphs[0];
let base_rtl = pinfo.level.is_rtl();
// per char: (byte, char, shaped char), face, level, small
let chars: Vec<(usize, char, char, FaceId, bool, bool)> = text
// per char: (byte, char, shaped char), face, level, small, style
let chars: Vec<(usize, char, char, FaceId, bool, bool, usize)> = text
.char_indices()
.map(|(b, c)| {
let (sc, small) = match style.caps {
let si = sty.at(base + b);
let (sc, small) = match sty.list[si].caps {
Caps::Normal => (c, false),
Caps::All => (upper_single(c), false),
Caps::Small if c.is_lowercase() => (upper_single(c), true),
Caps::Small => (c, false),
};
let face = if sc.is_whitespace() { primary.face } else { fonts::fallback_for(sc, primary.face) };
(b, c, sc, face, bidi.levels[b].is_rtl(), small)
let pf = sty.prim[si].face;
let face = if sc.is_whitespace() { pf } else { fonts::fallback_for(sc, pf) };
(b, c, sc, face, bidi.levels[b].is_rtl(), small, si)
})
.collect();
// items: runs of equal (face, rtl, small)
// items: runs of equal (face, rtl, small, style)
let mut items: Vec<Item> = Vec::new();
let mut i = 0;
while i < chars.len() {
let (face, rtl, small) = (chars[i].3, chars[i].4, chars[i].5);
let (face, rtl, small, si) = (chars[i].3, chars[i].4, chars[i].5, chars[i].6);
let mut j = i + 1;
while j < chars.len() && chars[j].3 == face && chars[j].4 == rtl && chars[j].5 == small {
while j < chars.len() && chars[j].3 == face && chars[j].4 == rtl && chars[j].5 == small && chars[j].6 == si {
j += 1;
}
let st = sty.list[si];
let sub: Vec<(usize, char, char)> = chars[i..j].iter().map(|c| (c.0, c.1, c.2)).collect();
let size = if small { style.size * SMALL_CAPS_SCALE } else { style.size };
let mut glyphs = shape_item(text, &sub, rtl, face, size, style);
let size = if small { st.size * SMALL_CAPS_SCALE } else { st.size };
let mut glyphs = shape_item(text, &sub, rtl, face, size, st);
// tracking after each cluster
if style.tracking != 0.0 {
let t = style.tracking * style.size / 1000.0;
if st.tracking != 0.0 {
let t = st.tracking * st.size / 1000.0;
for k in 0..glyphs.len() {
if k + 1 == glyphs.len() || glyphs[k + 1].cluster != glyphs[k].cluster {
glyphs[k].adv += t;
@ -353,7 +421,7 @@ fn paragraph(text: &str, base: usize, style: &TextStyle, para: &ParagraphStyle,
}
}
let end = if j < chars.len() { chars[j].0 } else { text.len() };
items.push(Item { range: chars[i].0..end, rtl, glyphs });
items.push(Item { range: chars[i].0..end, rtl, glyphs, style: si });
i = j;
}
// advance per char byte (cluster start)
@ -431,17 +499,20 @@ fn paragraph(text: &str, base: usize, style: &TextStyle, para: &ParagraphStyle,
idx.reverse();
}
for k in idx {
let si = items[k].style;
let (st, pr) = (sty.list[si], sty.prim[si]);
for g in items[k].glyphs.iter().filter(|g| run.contains(&g.cluster)) {
let x0 = pen;
glyphs.push(Glyph {
face: g.face,
id: g.id,
x: pen + g.dx,
y: -g.dy - style.baseline_shift,
y: -g.dy - st.baseline_shift,
size: g.size,
cluster: base + g.cluster,
synth_bold: primary.synth_bold || style.faux_bold,
synth_italic: primary.synth_italic || style.faux_italic,
synth_bold: pr.synth_bold || st.faux_bold,
synth_italic: pr.synth_italic || st.faux_italic,
run: si as u16,
});
pen += g.adv;
if space_extra > 0.0 && text[g.cluster..].starts_with(' ') && g.cluster < ce {
@ -469,6 +540,25 @@ fn paragraph(text: &str, base: usize, style: &TextStyle, para: &ParagraphStyle,
carets.push((base + b, x));
}
}
// underlined segments of rich layouts: clusters set in an underlined style, merged
let mut underlines: Vec<(f32, f32, u16)> = Vec::new();
if sty.rich() {
let mut segs: Vec<(f32, f32, u16)> = cluster_starts
.iter()
.filter(|&&cs| cs < ce)
.filter_map(|&cs| {
let si = sty.at(base + cs);
sty.list[si].underline.then(|| (extents[&cs].0, extents[&cs].1, si as u16))
})
.collect();
segs.sort_by(|a, b| a.0.total_cmp(&b.0));
for sg in segs {
match underlines.last_mut() {
Some(l) if l.2 == sg.2 && (sg.0 - l.1).abs() < 0.01 => l.1 = l.1.max(sg.1),
_ => underlines.push(sg),
}
}
}
// end of line caret
let end_x = match chars.iter().rev().find(|c| c.0 < r.end && c.0 >= r.start) {
Some(c) => {
@ -492,7 +582,7 @@ fn paragraph(text: &str, base: usize, style: &TextStyle, para: &ParagraphStyle,
let mut asc = pm.ascent;
let mut desc = pm.descent;
for g in &glyphs {
if g.face != primary.face {
if g.face != primary.face || (sty.rich() && g.size != style.size) {
let m = fonts::face(g.face).metrics(g.size);
asc = asc.max(m.ascent);
desc = desc.max(m.descent);
@ -507,6 +597,7 @@ fn paragraph(text: &str, base: usize, style: &TextStyle, para: &ParagraphStyle,
left_trim: if base_rtl { tw } else { 0.0 },
ascent: asc,
descent: desc,
underlines,
});
}
out
@ -514,23 +605,39 @@ fn paragraph(text: &str, base: usize, style: &TextStyle, para: &ParagraphStyle,
/// Lay out `text` (paragraphs separated by `\n`).
pub fn layout_uncached(text: &str, style: &TextStyle, para: &ParagraphStyle) -> Layout {
let primary = fonts::resolve(&style.family, &style.style);
let pm = fonts::face(primary.face).metrics(style.size);
layout_rich_uncached(text, style, &[], para)
}
/// Lay out `text` with per-character style runs (see [`StyleRun`]); `style` is the base style.
pub fn layout_rich_uncached(text: &str, style: &TextStyle, runs: &[StyleRun], para: &ParagraphStyle) -> Layout {
let sty = Styles::new(text, style, runs);
let pm = fonts::face(sty.prim[0].face).metrics(style.size);
let line_h = (pm.ascent + pm.descent + pm.line_gap).max(style.size * 0.5) + para.leading;
let mut lay = Layout { missing_font: primary.missing, text_len: text.len(), ..Default::default() };
let mut lay = Layout { missing_font: sty.prim.iter().any(|p| p.missing), text_len: text.len(), ..Default::default() };
if para.vertical {
return layout_vertical(text, style, para, primary, line_h, lay);
return layout_vertical(text, &sty, para, line_h, lay);
}
let mut base = 0usize;
let mut plines = Vec::new();
for p in text.split('\n') {
let p_clean = p.strip_suffix('\r').unwrap_or(p);
plines.extend(paragraph(p_clean, base, style, para, primary));
plines.extend(paragraph(p_clean, base, &sty, para));
base += p.len() + 1;
}
let first_baseline = if para.width.is_some() { pm.ascent } else { 0.0 };
let rich = sty.rich();
let first_baseline = match (para.width.is_some(), rich) {
(false, _) => 0.0,
(true, false) => pm.ascent,
(true, true) => plines.first().map_or(pm.ascent, |l| l.ascent),
};
let mut baseline = first_baseline;
let mut prev_desc = 0.0f32;
for (i, pl) in plines.into_iter().enumerate() {
let baseline = first_baseline + line_h * i as f32;
if i > 0 {
// rich text: a line holding a larger run pushes its baseline down
baseline += if rich { line_h.max(prev_desc + pl.ascent + para.leading) } else { line_h };
}
prev_desc = pl.descent;
let shift = match (para.width, para.align) {
(None, Align::Left | Align::Justify) => 0.0,
(None, Align::Center) => -pl.content / 2.0,
@ -546,8 +653,17 @@ pub fn layout_uncached(text: &str, style: &TextStyle, para: &ParagraphStyle) ->
g
}));
let x = shift + pl.left_trim;
if style.underline && pl.content > 0.0 {
if rich {
for (x0, x1, si) in &pl.underlines {
let st = sty.list[*si as usize];
let um = fonts::face(sty.prim[*si as usize].face).metrics(st.size);
let y = baseline - st.baseline_shift + um.underline_pos;
lay.underlines.push([x0 + shift, y, x1 + shift, y + um.underline_thickness]);
lay.underline_runs.push(*si);
}
} else if style.underline && pl.content > 0.0 {
lay.underlines.push([x, baseline + pm.underline_pos, x + pl.content, baseline + pm.underline_pos + pm.underline_thickness]);
lay.underline_runs.push(0);
}
lay.lines.push(Line {
range: pl.range,
@ -577,8 +693,9 @@ pub fn layout_uncached(text: &str, style: &TextStyle, para: &ParagraphStyle) ->
/// Vertical layout: each character is its own line (so carets, hit testing and selection work
/// unchanged), stacked top to bottom and centred on its column; columns run right to left.
fn layout_vertical(text: &str, style: &TextStyle, para: &ParagraphStyle, primary: Resolved, col_w: f32, mut lay: Layout) -> Layout {
let pm = fonts::face(primary.face).metrics(style.size);
fn layout_vertical(text: &str, sty: &Styles, para: &ParagraphStyle, col_w: f32, mut lay: Layout) -> Layout {
let style = sty.list[0];
let pm = fonts::face(sty.prim[0].face).metrics(style.size);
let row = (pm.ascent + pm.descent).max(style.size * 0.5) + style.tracking * style.size / 1000.0;
let flat = ParagraphStyle::default();
lay.vertical = true;
@ -589,10 +706,10 @@ fn layout_vertical(text: &str, style: &TextStyle, para: &ParagraphStyle, primary
let p_clean = p.strip_suffix('\r').unwrap_or(p);
let mut cells: Vec<ParaLine> = Vec::new();
if p_clean.is_empty() {
cells.extend(paragraph("", base, style, &flat, primary));
cells.extend(paragraph("", base, sty, &flat));
}
for (off, ch) in p_clean.char_indices() {
cells.extend(paragraph(&p_clean[off..off + ch.len_utf8()], base + off, style, &flat, primary));
cells.extend(paragraph(&p_clean[off..off + ch.len_utf8()], base + off, sty, &flat));
}
let h = row * cells.len() as f32;
let y0 = match para.align {
@ -639,10 +756,16 @@ fn cache() -> &'static Mutex<HashMap<u64, (u64, Arc<Layout>)>> {
/// Lay out `text` (cached by text and styles).
pub fn layout(text: &str, style: &TextStyle, para: &ParagraphStyle) -> Arc<Layout> {
layout_rich(text, style, &[], para)
}
/// Lay out `text` with per-character style runs (cached by text, styles and runs).
pub fn layout_rich(text: &str, style: &TextStyle, runs: &[StyleRun], para: &ParagraphStyle) -> Arc<Layout> {
let mut h = std::collections::hash_map::DefaultHasher::new();
text.hash(&mut h);
style.hash(&mut h);
para.hash(&mut h);
runs.hash(&mut h);
let key = h.finish();
static CLOCK: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
let now = CLOCK.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
@ -650,7 +773,7 @@ pub fn layout(text: &str, style: &TextStyle, para: &ParagraphStyle) -> Arc<Layou
e.0 = now;
return e.1.clone();
}
let l = Arc::new(layout_uncached(text, style, para));
let l = Arc::new(layout_rich_uncached(text, style, runs, para));
let mut c = cache().lock().unwrap_or_else(|e| e.into_inner());
if c.len() >= 512 {
// evict the oldest quarter
@ -810,6 +933,52 @@ mod tests {
assert!(l.glyphs.iter().all(|g| g.id != 0));
}
#[test]
fn style_runs_split_items_and_grow_lines() {
let base = st(40.0);
let big = TextStyle { size: 80.0, ..base.clone() };
let text = "small BIG small";
let plain = layout(text, &base, &ParagraphStyle::default());
let runs = [StyleRun { range: 6..9, style: big.clone() }];
let rich = layout_rich(text, &base, &runs, &ParagraphStyle::default());
// glyphs carry their style index; the run is wider than the plain text
let tagged: Vec<u16> = rich.glyphs.iter().map(|g| g.run).collect();
assert_eq!(tagged.iter().filter(|r| **r == 1).count(), 3, "{tagged:?}");
assert!(rich.glyphs.iter().filter(|g| g.run == 1).all(|g| g.size == 80.0));
assert!(rich.lines[0].width > plain.lines[0].width + 30.0);
assert!(rich.lines[0].ascent > plain.lines[0].ascent * 1.5, "the line grows to the run");
// carets still cover every character boundary
assert_eq!(rich.lines[0].carets.len(), text.len() + 1);
// a line holding a big run moves down from the line above it
let two = layout_rich("next\nBIG", &base, &[StyleRun { range: 5..8, style: big }], &ParagraphStyle::default());
let two_plain = layout("next\nBIG", &base, &ParagraphStyle::default());
assert!(two.lines[1].baseline > two_plain.lines[1].baseline + 10.0);
// no runs: identical to the plain layout
assert_eq!(*layout_rich(text, &base, &[], &ParagraphStyle::default()), *plain);
}
#[test]
fn style_runs_shift_underline_and_track() {
let base = st(40.0);
let text = "ab cd ef";
let runs = [
StyleRun { range: 3..5, style: TextStyle { underline: true, baseline_shift: 10.0, ..base.clone() } },
StyleRun { range: 6..8, style: TextStyle { tracking: 500.0, faux_bold: true, ..base.clone() } },
];
let l = layout_rich(text, &base, &runs, &ParagraphStyle::default());
assert_eq!(l.underlines.len(), 1, "only the underlined run");
assert_eq!(l.underline_runs, vec![1]);
let c = l.glyphs.iter().find(|g| g.cluster == 3).unwrap();
let a = l.glyphs.iter().find(|g| g.cluster == 0).unwrap();
assert!((c.y - (a.y - 10.0)).abs() < 1e-3, "baseline shift per run");
let u = l.underlines[0];
assert!(u[0] >= l.caret(3).0 - 0.01 && u[2] <= l.caret(5).0 + 0.01, "{u:?}");
assert!(l.glyphs.iter().filter(|g| g.run == 2).all(|g| g.synth_bold));
// tracking widens only the tracked run: 2 clusters × 0.5 em
let plain = layout(text, &base, &ParagraphStyle::default());
assert!((l.lines[0].width - plain.lines[0].width - 2.0 * 20.0).abs() < 0.5, "{} {}", l.lines[0].width, plain.lines[0].width);
}
#[test]
fn empty_text_has_a_caret() {
let l = layout("", &st(40.0), &ParagraphStyle::default());

4
crates/text/src/lib.rs
View File

@ -5,7 +5,7 @@
//! web unless the host registers font data), family/style resolution with synthetic bold/italic
//! and per-character fallback.
//! - [`layout`]: shaping (harfrust: kerning, ligatures, complex scripts), bidi, line breaking,
//! paragraph layout, carets and hit testing; cached.
//! paragraph layout, per-character style runs, carets and hit testing; cached.
//! - [`raster`]: our own anti-aliased path rasteriser (linear coverage, so compositing in linear
//! light is correct) and vector shapes.
//! - [`render`]: glyph outlines and the glyph-mask cache (sub-pixel positioned), drawing layouts
@ -20,7 +20,7 @@ pub mod render;
pub mod sfnt;
pub use fonts::{FaceId, Resolved, families, resolve};
pub use layout::{Align, Caps, Glyph, Layout, Line, ParagraphStyle, TextStyle, layout, measure};
pub use layout::{Align, Caps, Glyph, Layout, Line, ParagraphStyle, StyleRun, TextStyle, layout, layout_rich, measure};
pub use mask::StrokeKind;
pub use raster::{Mask, Path, Xform};

34
crates/text/src/render.rs
View File

@ -170,13 +170,20 @@ pub fn is_axis_uniform(m: &Xform) -> bool {
/// The whole layout (glyphs + underlines) as one device-space path.
pub fn layout_path(l: &Layout, m: &Xform) -> Path {
layout_path_run(l, m, None)
}
/// The glyphs and underlines set in style `run` (all with None) as one device-space path.
pub fn layout_path_run(l: &Layout, m: &Xform, run: Option<u16>) -> Path {
let mut p = Path::new();
for g in &l.glyphs {
for g in l.glyphs.iter().filter(|g| run.is_none_or(|r| g.run == r)) {
let gm = compose(m, &[1.0, 0.0, 0.0, 1.0, g.x, g.y]);
p.extend(&glyph_path(g, &gm));
}
for u in &l.underlines {
p.extend(&Path::rect(u[0], u[1], u[2], u[3]).transformed(m));
for (i, u) in l.underlines.iter().enumerate() {
if run.is_none_or(|r| l.underline_runs.get(i).copied().unwrap_or(0) == r) {
p.extend(&Path::rect(u[0], u[1], u[2], u[3]).transformed(m));
}
}
p
}
@ -197,10 +204,17 @@ pub fn device_bounds(l: &Layout, m: &Xform) -> [f32; 4] {
/// Add the coverage of layout `l` under transform `m` into `out` (whose pixel (0,0) is device
/// pixel `origin`).
pub fn draw(l: &Layout, m: &Xform, out: &mut Mask, origin: (i32, i32)) {
draw_run(l, m, out, origin, None)
}
/// [`draw`] limited to the glyphs and underlines set in style `run` (see
/// [`Glyph::run`](crate::Glyph::run)); None draws everything.
pub fn draw_run(l: &Layout, m: &Xform, out: &mut Mask, origin: (i32, i32), run: Option<u16>) {
let (ox, oy) = origin;
let keep = |r: u16| run.is_none_or(|x| x == r);
if is_axis_uniform(m) {
let s = m[0];
for g in &l.glyphs {
for g in l.glyphs.iter().filter(|g| keep(g.run)) {
let (x, y) = apply(m, g.x, g.y);
let px = g.size * s;
if px < 0.5 {
@ -213,13 +227,17 @@ pub fn draw(l: &Layout, m: &Xform, out: &mut Mask, origin: (i32, i32)) {
}
if !l.underlines.is_empty() {
let mut p = Path::new();
for u in &l.underlines {
p.extend(&Path::rect(u[0], u[1], u[2], u[3]).transformed(m));
for (i, u) in l.underlines.iter().enumerate() {
if keep(l.underline_runs.get(i).copied().unwrap_or(0)) {
p.extend(&Path::rect(u[0], u[1], u[2], u[3]).transformed(m));
}
}
if !p.is_empty() {
fill_into(&p, out, -ox as f32, -oy as f32);
}
fill_into(&p, out, -ox as f32, -oy as f32);
}
} else {
let p = layout_path(l, m);
let p = layout_path_run(l, m, run);
fill_into(&p, out, -ox as f32, -oy as f32);
}
}

93
docs/graphics.md
View File

@ -101,8 +101,95 @@ shadow). Automation ids: `graphics.layers.<n>`, `graphics.prop.<parameter id>`,
`graphics.align.<how>`, `graphics.distribute.<axis>`, `graphics.sourceText`, `graphics.NewTextLayer`,
`graphics.NewRectangle`, `graphics.NewEllipse`, `graphics.deleteLayer`, `graphics.section.<name>`.
## Per-character styles
Select characters with the Type tool and change font, style, size, colour, faux bold / italic,
underline, tracking, baseline shift or caps in the Properties panel: the change applies to the
selection only (`graphics.setCharStyle`). Styles are stored on the layer as runs of characters
(`EffectInstance::layer.runs`, character offsets, sorted and non-overlapping) and follow the text
when it is edited (typed characters take the style of the character before them). The text engine
lays the runs out together (`filmcraft_text::layout_rich`): a run can change the font and size,
lines grow to fit the largest run, and each run can have its own fill colour.
| Command | Params |
|---|---|
| `graphics.setCharStyle` | `clip`, `layer`, `start`, `end` (characters; default the whole text), `style` {`font`, `fontStyle`, `size`, `color`, `bold`, `italic`, `underline`, `tracking`, `baselineShift`, `caps`}, `clear` (remove styles from the range) |
## Responsive design
**Position (pins).** A layer's edges can be pinned to another layer of the same graphic or to the
video frame (`graphics.pin {layer, to: "frame" | layer index | name | "none", edges}`). Each
pinned edge keeps its distance to the *same* edge of the target, measured when the pin is made:
a box pinned on all four edges to a text layer grows with the text; a shape pinned on two opposite
edges is resized, a text layer only moves. Pins resolve in dependency order (cycles are ignored).
Moving a pinned layer keeps the pin at its new distance. Layers get a stable `uid` for this.
**Time (intro / outro).** `graphics.setResponsiveTime {introFrames, outroFrames}` protects the
first and last part of a graphic: when the clip is trimmed or extended, keyframes inside the intro
keep their timing from the clip start, those inside the outro from the clip end, and the middle is
stretched or squeezed (`graphic_design::remap_time`). Trimming the head keeps the intro playing
from the new start.
**Rolls and crawls.** `graphics.setRoll {mode: off | roll | crawlLeft | crawlRight,
startOffScreen, endOffScreen, prerollFrames, easeInFrames, easeOutFrames, postrollFrames}` (the
Roll options of Responsive Design – Time) moves the whole graphic: up for a roll, sideways for a
crawl. *Start Off Screen* starts with the content just outside the frame (below it / right of it /
left of it), otherwise where it was laid out; *End Off Screen* lets it leave completely, otherwise
it stops when its last edge reaches the frame edge. The speed is constant between the preroll and
the postroll, with linear acceleration over Ease In and deceleration over Ease Out
(`graphic_design::roll_progress`). The offset is exact maths on the union of the layers' bounds
(with background padding), at the clip-relative time.
These settings are `TrackItem::graphic` (`GraphicMeta`, schema v12).
## Graphics templates (`.fcgt`)
FilmCraft's equivalent of motion graphics templates is its **own** format: a `.fcgt` file is
documented, versioned JSON (`filmcraft_project::gtemplate`):
| Field | Meaning |
|---|---|
| `format`, `version` | `"filmcraft.graphicsTemplate"`, `1` (newer versions are refused) |
| `id`, `name`, `category`, `description`, `author`, `license`, `tags` | library information |
| `canvas`, `duration` | frame size it was designed for, default clip length (ticks) |
| `layers` | the graphic's layers, exactly as in a project file, each with a `uid` |
| `graphic` | roll / crawl and responsive time |
| `controls` | editable properties: `{id, name, kind: text\|color\|slider\|checkbox\|font\|position, layer: uid, param, min, max}`; `param: "enabled"` is the layer's visibility |
| `resources.fonts` | optional embedded font files (base64) with their licence; registered on install |
Placing a template on a sequence of another size scales and centres its layers (pin distances
scale too).
**FilmCraft never reads or imports Adobe `.mogrt` files** (or any other application's template
packages): Install refuses `.mogrt` paths without opening them, and the reader rejects ZIP
archives. Built-in templates are original designs made for FilmCraft in code (no image, font or
template files): lower thirds (Slab, Rule, Ticker crawl), titles (Centered, Boxed), end credits
(Roll) and callouts (Pointer, Tag), with the bundled Inter font. Thumbnails are rendered by the
engine at runtime.
| Command | Menu | Params |
|---|---|---|
| `graphics.template.list` | (query) | `query`, `category`, `source` (builtin / user) |
| `graphics.template.apply` | Browse tab: drag or double-click | `template` (id, name or `.fcgt` path), `values` {control: value}, `time`, `track` |
| `graphics.template.export` | Graphics and Titles ▸ Export As Motion Graphics Template… | `clip`, `name`, `category`, `description`, `controls` [{`layer`, `param`, `name`, `kind`, `min`, `max`}], `path` (default: the user folder), `embedFonts` |
| `file.exportGraphicsTemplate` | File ▸ Export ▸ Motion Graphics Template… | as above |
| `graphics.template.install` | Graphics and Titles ▸ Install Motion Graphics Template… | `path` (a `.fcgt`) |
| `graphics.template.remove` | Browse tab | `template` (user templates) |
| `graphics.template.set` / `graphics.template.controls` | Essential Graphics ▸ Edit | `clip`, `control`, `value` / `values` |
| `graphics.template.thumbnail` | (query) | `template`, `width`, `path` (PNG) |
User templates live in `<data dir>/Graphics Templates/`.
## Captions, source graphics and fonts
| Command | Menu | Params |
|---|---|---|
| `graphics.upgradeCaption` | Graphics and Titles ▸ Upgrade Caption to Graphic | `captions` (default: the selection, else the one under the playhead): each caption becomes a graphic clip on a video track with the track style (font, size, colour, background, outline, alignment, position); `<i>`/`<b>`/`<u>` become character styles; one undo step |
| `graphics.upgradeToSourceGraphic` | Graphics and Titles ▸ Upgrade to Source Graphic | `clip`: the graphic gets its own project item (shown in the Project panel); every clip of that item shares its layers — editing one updates the others in the same undo step (`project.source_graphics`) |
| `file.replaceFonts` | Graphics and Titles ▸ Replace Fonts in Projects… | `from` (family or {family, style}), `to`, `toStyle`: graphic layers, character styles, source graphics and caption tracks |
| `graphics.fonts.used` | (query) | fonts in use with counts and whether they are missing |
## Not yet
Per-character styling inside one text layer, responsive design pins and rolls/crawls (M10.3),
motion graphics templates, the Browse tab, mask-with-text, gradient fills, per-layer blend modes, and
Bézier curves in the pen tool (paths are polygons).
Mask-with-text, gradient fills, per-layer blend modes, Bézier curves in the pen tool (paths are
polygons), media layers inside graphics, and template controls grouped into folders.

3
docs/project-files.md
View File

@ -8,7 +8,7 @@ writing them is `crates/format` (`filmcraft-format`); the engine's `file.*` comm
```json
{
"format": "filmcraft.project",
"schema_version": 10,
"schema_version": 12,
"generator": "FilmCraft 0.1.0",
"project": { "name": "…", "settings": { … }, "root": { … }, "items": { … }, "next_id": 48 }
}
@ -35,6 +35,7 @@ writing them is `crates/format` (`filmcraft-format`); the engine's `file.*` comm
| 9 | M10.5 | transcripts of media items (`project.transcripts`); no-op step |
| 10 | M3.10 | clip time interpolation, Hold Filters, Field Options, audio source channels, Modify ▸ Audio Channels map, subclip Restrict Trims; no-op step |
| 11 | M3.11 | search bins (`project.search_bins`), Flash Cue markers, Project Settings safe areas, capture format and scratch disks; no-op step |
| 12 | M10.7 | graphics design data: `TrackItem::graphic` (roll / crawl, responsive time, template link), `EffectInstance::layer` (layer uid, per-character styles, responsive pins), `project.source_graphics`; no-op step |
### Migrations