shifathosn
/
filmcraft
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge main into #1140 (docs/gaps.md) 

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Thomas 2026-10-11 19:34:14 -04:00
parent cfe0c751e9 8ea545b103
commit 6b860ede89
150 changed files with 11314 additions and 846 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
ATTRIBUTION.md
View File

@ -9,6 +9,7 @@ iconography, images or other Adobe assets; open licences only; every asset attri
| Asset | Author | Source | Licence |
|---|---|---|---|
| `crates/ui-egui/src/i18n/id.tsv` | FilmCraft contributors | Original work: clean-room Indonesian interface translations | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/fr.tsv` | FilmCraft contributors | Original work: clean-room French interface translations | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/ru.tsv` | FilmCraft contributors | Original work: clean-room Russian interface translations | MIT OR Apache-2.0 |
| `crates/ui-egui/src/i18n/uk.tsv` | FilmCraft contributors | Original work: clean-room Ukrainian interface translations | MIT OR Apache-2.0 |

2
Cargo.lock generated
View File

@ -1696,6 +1696,7 @@ dependencies = [
"objc2-app-kit 0.3.2",
"objc2-foundation 0.3.2",
"open",
"pollster 0.4.0",
"rfd",
"serde_json",
"sys-locale",
@ -1890,6 +1891,7 @@ dependencies = [
"filmcraft-frame",
"filmcraft-geom",
"filmcraft-interchange",
"filmcraft-isobmff",
"filmcraft-media",
"filmcraft-mxf",
"filmcraft-project",

2
ROADMAP.md
View File

@ -83,7 +83,7 @@ Premiere ships 10 interface languages. Measured from our catalogs (3,498 strings
| Arabic | none (no RTL interface) | 0% |
| French | partial (whole interface; native review welcome) | ~100% |
| Portuguese (Brazil) | menus only | 9.9% |
| Indonesian | none | 0% |
| Indonesian | partial (whole interface; native review welcome) | ~100% |
| Japanese | partial (whole interface) | 99.7% |
| German | none | 0% |
| Korean | none | 0% |

80
apps/filmcraft-cli/src/main.rs
View File

@ -76,7 +76,7 @@ SUBCOMMANDS
import <file>... import media into the project
export <out> [--preset name] [--format f] [--range r] [--start s --end s] [--settings json]
[--gpu-rendering auto|off]
[--scale f] [--quality 0-100] [--no-audio] [--queue]
[--scale f] [--quality 0-100] [--no-audio] [--queue] [--allow-offline]
export the active sequence and wait for it to finish: with an
export preset (`export --list-presets`; built-in or the user's), or
a format (h264|hevc|prores|dnxhr|apv|mjpeg|mxf-op1a|mxf-opatom|png|tiff|bmp|gif|wav|aiff|aac, guessed
@ -85,9 +85,12 @@ SUBCOMMANDS
preset; --scale renders at a fraction of the frame size (0.5 =
half); --quality 0-100 for the formats that take one; --no-audio
leaves the sound out; --queue adds to the export queue and runs it
instead
instead. Offline media (missing, made offline, unreadable) renders
as placeholder frames: the export then lists it in `offlineMedia`,
warns on stderr and exits 1, unless --allow-offline accepts it
export --list-presets [query] list export presets (name, category, format) as JSON
render --seconds S --out f.png [--scale 0.5] render one Program frame to PNG
render --seconds S --out f.png [--scale 0.5] [--allow-offline]
render one Program frame to PNG (offline media: as for export)
probe <media> [--image-sequence]
media info as JSON, with MXF / Ogg / BWF details; with
--image-sequence <media> is the first numbered still of a sequence
@ -113,7 +116,8 @@ VALUES
Times: 254016000000 ticks per second; most commands also take seconds=, frame= or timecode=.
EXIT STATUS
0 success · 1 a command failed · 2 usage error
0 success · 1 a command failed, or export/render drew offline media without --allow-offline
· 2 usage error
";
/// Writes one diagnostic line to `w`, dropping any write error. `eprintln!` panics (exit 101) when stderr's
@ -442,7 +446,9 @@ async fn cli() {
match r {
Ok(v) => {
print(&a, &v);
diag(format_args!("exported {} in {:.1}s", written_paths(&v, out).join(", "), t0.elapsed().as_secs_f64()));
let written = written_paths(&v, out).join(", ");
check_offline(&a, &offline_media(&v), &format!("export: {written}"));
diag(format_args!("exported {written} in {:.1}s", t0.elapsed().as_secs_f64()));
}
Err(e) => fail(format!("export: {e}")),
}
@ -457,10 +463,12 @@ async fn cli() {
let out = a.opt("--out").unwrap_or("frame.png");
s.set_playhead(filmcraft_time::Tick::from_seconds_f64(secs));
let t0 = std::time::Instant::now();
let img = render_at_playhead(&s, scale).unwrap_or_else(|e| fail(e));
let (img, offline) = render_at_playhead(&s, scale).unwrap_or_else(|e| fail(e));
let dt = t0.elapsed();
let png = filmcraft_automation::png_rgba(img.w as u32, img.h as u32, img.over_black_rgba8(), 0).unwrap_or_else(|e| fail(e));
std::fs::write(out, png).unwrap_or_else(|e| fail(format!("{out}: {e}")));
let offline: Vec<Value> = offline.iter().filter_map(|o| serde_json::to_value(o).ok()).collect();
check_offline(&a, &offline, &format!("render: {out}"));
diag(format_args!("rendered {}x{} in {:.1} ms → {out}", img.w, img.h, dt.as_secs_f64() * 1000.0));
}
"mcp" => {
@ -468,7 +476,7 @@ async fn cli() {
usage(format!("unknown mcp option `{opt}`"));
}
let server = match a.opt("--bridge") {
Some(addr) => filmcraft_automation::FilmcraftMcp::bridge(addr).unwrap_or_else(|e| fail(e)),
Some(addr) => filmcraft_automation::FilmcraftMcp::bridge(addr).unwrap_or_else(|e| usage(e)),
None => match Backend::open(&a) {
Backend::Local(s) => filmcraft_automation::FilmcraftMcp::headless(*s),
Backend::Bridge(_) => usage("mcp: use --bridge ADDR to drive a running app"),
@ -500,9 +508,46 @@ fn written_paths(result: &Value, requested: &str) -> Vec<String> {
if paths.is_empty() { vec![requested.to_string()] } else { paths }
}
/// The active sequence at the playhead, or the reason it cannot be rendered (no sequence, bad scale).
fn render_at_playhead(s: &Session, scale: f32) -> Result<filmcraft_engine::render::Image, String> {
s.try_render_program_at(scale, s.playhead()).map_err(|e| e.to_string())
/// The offline items an export result reports (`offlineMedia` of a direct export, or of each
/// queued item), once each.
fn offline_media(result: &Value) -> Vec<Value> {
let lists = std::iter::once(result).chain(result.get("items").and_then(Value::as_array).into_iter().flatten());
let mut out: Vec<Value> = Vec::new();
for o in lists.filter_map(|r| r.get("offlineMedia").and_then(Value::as_array)).flatten() {
if !out.iter().any(|x| x.get("item") == o.get("item")) {
out.push(o.clone());
}
}
out
}
/// One stderr line per offline item that rendered as placeholder frames (#1054).
fn offline_warning(offline: &[Value]) -> String {
let s = |o: &Value, k: &str| o.get(k).and_then(Value::as_str).unwrap_or("").to_string();
let mut msg = format!("filmcraft-cli: warning: {} offline media item(s) rendered as placeholder frames:", offline.len());
for o in offline {
let id = o.get("item").and_then(Value::as_u64).map(|i| i.to_string()).unwrap_or_default();
msg.push_str(&format!("\n item {id} {} ({}): {}", s(o, "name"), s(o, "status"), s(o, "path")));
}
msg
}
/// Warn about offline media on stderr; without `--allow-offline` that fails the run (exit 1), after
/// the output (`what`) was written, so scripts don't take placeholder frames for the real picture.
fn check_offline(a: &Args, offline: &[Value], what: &str) {
if offline.is_empty() {
return;
}
diag(format_args!("{}", offline_warning(offline)));
if !a.flag("--allow-offline") {
fail(format!("{what} has placeholder frames for offline media; relink it (exec media.relink …) or pass --allow-offline"));
}
}
/// The active sequence at the playhead with the offline items it drew, or the reason it cannot be
/// rendered (no sequence, bad scale).
fn render_at_playhead(s: &Session, scale: f32) -> Result<(filmcraft_engine::render::Image, Vec<filmcraft_engine::media_pool::OfflineMedia>), String> {
s.try_render_program_with_offline_at(scale, s.playhead()).map_err(|e| e.to_string())
}
/// `--start` / `--end` ask for a custom range. A lone bound is forwarded too, so the engine reports the missing one
@ -531,7 +576,7 @@ mod format_tests {
let e = super::render_at_playhead(&s, scale).unwrap_err();
assert!(e.contains("finite and positive"), "{e}");
}
assert_eq!(super::render_at_playhead(&s, 0.5).unwrap().w, 8);
assert_eq!(super::render_at_playhead(&s, 0.5).unwrap().0.w, 8);
}
/// The CLI (and MCP / headless runs, which share its entry point) registers the hardware
@ -573,6 +618,19 @@ mod format_tests {
assert_eq!(super::written_paths(&json!({"items": []}), "x.mov"), ["x.mov"]);
}
#[test]
fn offline_media_from_direct_and_queued_results() {
use serde_json::json;
let a = json!({"item": 8, "name": "a.mp4", "path": "/m/a.mp4", "status": "missing", "error": ""});
let b = json!({"item": 9, "name": "b.mov", "path": "/m/b.mov", "status": "offline", "error": ""});
assert_eq!(super::offline_media(&json!({"path": "x.mp4", "offlineMedia": [a]})), std::slice::from_ref(&a));
let queued = json!({"items": [{"offlineMedia": [a, b]}, {"offlineMedia": [a]}, {"offlineMedia": []}]});
assert_eq!(super::offline_media(&queued), [a.clone(), b.clone()]);
assert!(super::offline_media(&json!({"path": "x.mp4"})).is_empty());
let w = super::offline_warning(&[a, b]);
assert!(w.contains("2 offline media item(s)") && w.contains("item 8 a.mp4 (missing): /m/a.mp4") && w.contains("item 9 b.mov (offline)"), "{w}");
}
#[test]
fn export_lone_bound_is_forwarded_as_custom_range() {
use serde_json::json;

55
apps/filmcraft-cli/tests/cli.rs
View File

@ -22,6 +22,9 @@ fn help_and_usage_errors() {
assert_eq!(cli(&[]).status.code(), Some(2));
assert_eq!(cli(&["frobnicate"]).status.code(), Some(2));
assert_eq!(cli(&["exec", "file.newBin", "notkv"]).status.code(), Some(2));
// Invalid --bridge is a usage error (exit 2) for both exec and mcp (#921).
assert_eq!(cli(&["--bridge", "0.0.0.0:9876", "exec", "inspect.project"]).status.code(), Some(2));
assert_eq!(cli(&["mcp", "--bridge", "0.0.0.0:9876"]).status.code(), Some(2));
}
#[test]
@ -291,3 +294,55 @@ fn closed_stdout_still_runs_the_script_and_saves() {
assert_eq!(out.status.code(), Some(1));
let _ = std::fs::remove_dir_all(&dir);
}
/// #1054: exporting or rendering a project whose media is missing fails (exit 1) and names the
/// offline items on stderr and in the result's `offlineMedia`, unless `--allow-offline` accepts the
/// placeholder frames; then it still warns.
#[test]
fn export_with_missing_media_fails_unless_allowed() {
let dir = std::env::temp_dir().join(format!("fc-cli-1054-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let f = |n: &str| dir.join(n).to_str().unwrap().to_string();
let (still, proj) = (f("still.png"), f("p.fcproj"));
let o = cli(&["--demo", "render", "--seconds", "1", "--scale", "0.1", "--out", &still]);
assert!(o.status.success(), "{}", String::from_utf8_lossy(&o.stderr));
json_out(&cli(&["exec", "file.newSequence", "name=s", "width=64", "height=36", "fps=24", "--save-as", &proj]));
let paths = format!("paths=[\"{}\"]", still.replace('\\', "\\\\"));
let item = json_out(&cli(&["--project", &proj, "--save", "exec", "file.import", &paths]))["items"][0].as_u64().unwrap();
json_out(&cli(&["--project", &proj, "--save", "exec", "timeline.place", &format!("item={item}"), "track=V1", "frame=0", "audio=false"]));
let export = |out: &str, extra: &[&str]| {
let mut args = vec!["--project", proj.as_str(), "--compact", "export", out, "--format", "png", "--start", "0", "--end", "0.05"];
args.extend_from_slice(extra);
cli(&args)
};
let on = export(&f("on.png"), &[]);
let v = json_out(&on);
assert_eq!(v["offlineMedia"], serde_json::json!([]), "{v}");
assert!(!String::from_utf8_lossy(&on.stderr).contains("warning"), "{}", String::from_utf8_lossy(&on.stderr));
std::fs::remove_file(&still).unwrap();
let off = export(&f("off.png"), &[]);
let err = String::from_utf8_lossy(&off.stderr);
assert_eq!(off.status.code(), Some(1), "{err}");
assert!(err.contains("still.png") && err.contains("missing") && err.contains("--allow-offline"), "{err}");
let v: Value = serde_json::from_slice(&off.stdout).expect("the result JSON is still printed");
assert_eq!(v["offlineMedia"][0]["item"], item, "{v}");
assert_eq!(v["offlineMedia"][0]["status"], "missing", "{v}");
let allowed = export(&f("allowed.png"), &["--allow-offline"]);
let err = String::from_utf8_lossy(&allowed.stderr);
assert!(allowed.status.success(), "{err}");
assert!(err.contains("still.png") && err.contains("missing"), "still warns: {err}");
assert_eq!(json_out(&allowed)["offlineMedia"][0]["status"], "missing");
let render = cli(&["--project", &proj, "render", "--seconds", "0", "--out", &f("r.png")]);
let err = String::from_utf8_lossy(&render.stderr);
assert_eq!(render.status.code(), Some(1), "{err}");
assert!(err.contains("still.png") && err.contains("--allow-offline"), "{err}");
let render = cli(&["--project", &proj, "render", "--seconds", "0", "--out", &f("r.png"), "--allow-offline"]);
assert!(render.status.success(), "{}", String::from_utf8_lossy(&render.stderr));
assert!(String::from_utf8_lossy(&render.stderr).contains("still.png"));
let _ = std::fs::remove_dir_all(&dir);
}

2
apps/filmcraft/Cargo.toml
View File

@ -41,6 +41,8 @@ open = "5"
[dev-dependencies]
glob = "0.3"
# graphics.rs: enumerate the real adapters in a test.
pollster = "0.4"
[target.'cfg(target_os = "macos")'.dependencies]
muda = "0.21"

63
apps/filmcraft/src/args.rs
View File

@ -15,6 +15,9 @@ Options:
--no-recover start without asking about unsaved changes
--data-dir <dir> folder for auto-save, crash recovery and logs
(also: FILMCRAFT_DATA_DIR)
--graphics <mode> start with this graphics mode and keep it: default,
dx12 (Windows), software (Windows, Linux);
`--graphics default` undoes an automatic fallback
--version print the version and exit
-h, --help print this help and exit
";
@ -28,7 +31,7 @@ pub enum Cli {
}
/// Options of a normal start.
#[derive(Debug, PartialEq)]
#[derive(Clone, Debug, PartialEq)]
pub struct Launch {
pub control_port: Option<u16>,
pub control_input_only: bool,
@ -39,6 +42,8 @@ pub struct Launch {
pub startup_flag: bool,
pub recover: Option<bool>,
pub data_dir: Option<std::path::PathBuf>,
/// `--graphics <mode>`: start with this graphics mode and keep it (see `graphics.rs`).
pub graphics: Option<crate::graphics::Mode>,
}
/// Parse the arguments after the program name. `env_control_port` is the value of
@ -49,7 +54,16 @@ pub struct Launch {
/// a script waiting for it. That holds for the whole line, so `--version --bogus` is an error
/// too, and for a port from the environment exactly as for one from `--control`.
pub fn parse(env_control_port: Option<String>, args: impl IntoIterator<Item = String>) -> Result<Cli, String> {
let mut l = Launch { control_port: None, control_input_only: false, files: Vec::new(), demo: true, startup_flag: false, recover: None, data_dir: None };
let mut l = Launch {
control_port: None,
control_input_only: false,
files: Vec::new(),
demo: true,
startup_flag: false,
recover: None,
data_dir: None,
graphics: None,
};
let (mut help, mut version) = (false, false);
let mut args = args.into_iter();
while let Some(a) = args.next() {
@ -66,6 +80,10 @@ pub fn parse(env_control_port: Option<String>, args: impl IntoIterator<Item = St
"--recover" => l.recover = Some(true),
"--no-recover" => l.recover = Some(false),
"--data-dir" => l.data_dir = Some(args.next().map(std::path::PathBuf::from).ok_or("--data-dir needs a folder")?),
"--graphics" => {
let name = args.next().ok_or("--graphics needs a mode: default, dx12 or software")?;
l.graphics = Some(graphics_mode(&name)?);
}
// everything after `--` is a file, whatever it looks like
"--" => l.files.extend(args.by_ref()),
// macOS adds a process serial number when it launches an app bundle
@ -91,6 +109,17 @@ pub fn parse(env_control_port: Option<String>, args: impl IntoIterator<Item = St
Ok(Cli::Run(l))
}
/// A `--graphics` value: one of this platform's modes.
fn graphics_mode(name: &str) -> Result<crate::graphics::Mode, String> {
use crate::graphics::{LADDER, Mode};
let here = || LADDER.iter().map(|m| m.name()).collect::<Vec<_>>().join(", ");
let mode = Mode::parse(name).ok_or_else(|| format!("--graphics needs one of {}, got `{name}`", here()))?;
if !LADDER.contains(&mode) {
return Err(format!("--graphics {name} is not available on this platform (use one of {})", here()));
}
Ok(mode)
}
#[cfg(test)]
mod tests {
use super::*;
@ -166,7 +195,16 @@ mod tests {
let none = launch(&[]);
assert_eq!(
none,
Launch { control_port: None, control_input_only: false, files: vec![], demo: true, startup_flag: false, recover: None, data_dir: None }
Launch {
control_port: None,
control_input_only: false,
files: vec![],
demo: true,
startup_flag: false,
recover: None,
data_dir: None,
graphics: None
}
);
let l = launch(&["--control", "9876", "--empty", "--no-recover", "--data-dir", "/tmp/fc", "cut.fcproj", "a.mp4"]);
assert_eq!(l.control_port, Some(9876));
@ -188,4 +226,23 @@ mod tests {
assert_eq!(parse_args(&["--control-input-only", "--help"]), Ok(Cli::Help));
assert!(USAGE.contains("--control-input-only"));
}
#[test]
fn graphics_picks_one_of_this_platforms_modes() {
use crate::graphics::{LADDER, Mode};
assert_eq!(launch(&[]).graphics, None);
assert_eq!(launch(&["--graphics", "default"]).graphics, Some(Mode::Default));
for mode in LADDER {
assert_eq!(launch(&["--graphics", mode.name()]).graphics, Some(*mode));
}
// a mode this platform has not is refused, not ignored
for mode in [Mode::Dx12, Mode::Software].into_iter().filter(|m| !LADDER.contains(m)) {
assert!(parse_args(&["--graphics", mode.name()]).unwrap_err().contains("not available"));
}
assert!(parse_args(&["--graphics"]).is_err(), "missing mode");
assert!(parse_args(&["--graphics", "vulkan"]).unwrap_err().contains("vulkan"));
assert!(parse_args(&["--graphics", "DEFAULT"]).is_err());
assert!(parse_args(&["--graphics", &"x".repeat(1 << 16)]).is_err());
assert!(USAGE.contains("--graphics"));
}
}

347
apps/filmcraft/src/audio.rs
View File

@ -1,18 +1,33 @@
//! cpal audio output: the playback master clock. Settings ▸ Audio Hardware picks the host
//! ("Device Class"), the output device, the I/O buffer size and the sample rate.
//!
//! A stream the device drops while playing (an ALSA xrun the backend does not recover from, which
//! cpal reports as `poll() returned POLLERR`; an unplugged device) is restarted by
//! [`AudioOut::recover`], keeping the played-frame clock and the play-ahead mixer, so sound
//! resumes where it stopped instead of playback going silent (#840).
use std::panic::{AssertUnwindSafe, catch_unwind};
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::time::{Duration, Instant};
use std::sync::{Arc, Mutex, PoisonError};
use std::sync::{Arc, Mutex, PoisonError, TryLockError};
use cpal::traits::{DeviceTrait, HostTrait, StreamTrait};
use filmcraft_engine::settings::AudioHardwarePrefs;
use filmcraft_ui_egui::{AudioDevices, AudioOut};
/// The playback mixer callback handed to [`AudioOut::start`].
type Fill = Box<dyn FnMut(&mut [f32], usize) + Send>;
pub struct CpalOut {
stream: Option<cpal::Stream>,
/// The running playback's mixer, shared with the stream so a restarted stream continues it.
fill: Option<Arc<Mutex<Fill>>>,
played: Arc<AtomicU64>,
failed: Arc<AtomicBool>,
health: Arc<Health>,
watchdog: Watchdog,
/// Time base for the watchdog.
epoch: Instant,
rate: u32,
channels: u16,
hw: AudioHardwarePrefs,
@ -128,8 +143,11 @@ impl CpalOut {
};
Self {
stream: None,
fill: None,
played: Arc::new(AtomicU64::new(0)),
failed: Arc::new(AtomicBool::new(false)),
health: Arc::new(Health::default()),
watchdog: Watchdog::default(),
epoch: Instant::now(),
rate: cfg.as_ref().map_or(48_000, |c| c.sample_rate().0),
channels: cfg.as_ref().map_or(2, |c| c.channels()),
hw: AudioHardwarePrefs::default(),
@ -153,19 +171,14 @@ impl CpalOut {
Ok(exact.unwrap_or(default))
}
fn converted_stream<T>(
&self,
dev: &cpal::Device,
config: &cpal::StreamConfig,
mut fill: Box<dyn FnMut(&mut [f32], usize) + Send>,
) -> Result<cpal::Stream, cpal::BuildStreamError>
fn converted_stream<T>(&self, dev: &cpal::Device, config: &cpal::StreamConfig, fill: Arc<Mutex<Fill>>) -> Result<cpal::Stream, cpal::BuildStreamError>
where
T: cpal::SizedSample + cpal::FromSample<f32>,
{
let channels = usize::from(config.channels);
let played = self.played.clone();
let failed = self.failed.clone();
let error_flag = failed.clone();
let health = self.health.clone();
let error_health = health.clone();
let mut scratch = Vec::new();
dev.build_output_stream(
config,
@ -173,39 +186,23 @@ impl CpalOut {
// Reuse the conversion buffer. A pathological driver buffer must not panic the audio thread.
if buf.len() > 1_048_576 || scratch.try_reserve(buf.len().saturating_sub(scratch.len())).is_err() {
buf.fill(T::EQUILIBRIUM);
if !failed.swap(true, Ordering::SeqCst) {
if !health.failed.swap(true, Ordering::SeqCst) {
eprintln!("filmcraft: unable to allocate the audio conversion buffer");
}
return;
}
scratch.resize(buf.len(), 0.0);
fill(&mut scratch, channels);
render(&fill, &mut scratch, channels, &played, &health);
convert_samples(&scratch, buf);
played.fetch_add((buf.len() / channels) as u64, Ordering::SeqCst);
},
move |e| stream_error(&error_flag, e),
move |e| stream_error(&error_health, e),
None,
)
}
}
fn stream_error(failed: &AtomicBool, e: cpal::StreamError) {
if !failed.swap(true, Ordering::SeqCst) {
eprintln!("filmcraft: audio stream error: {e}");
}
}
fn convert_samples<T: cpal::Sample + cpal::FromSample<f32>>(input: &[f32], output: &mut [T]) {
// Integer conversion represents [-1, 1). In particular I24 cannot represent +1 exactly.
let upper = f32::from_bits(1.0_f32.to_bits() - 1);
for (sample, converted) in input.iter().zip(output) {
*converted = T::from_sample(if sample.is_finite() { sample.clamp(-1.0, upper) } else { 0.0 });
}
}
impl AudioOut for CpalOut {
fn start(&mut self, mut fill: Box<dyn FnMut(&mut [f32], usize) + Send>) -> Result<u32, String> {
self.stop();
/// Open the configured output device and play `fill` on it. `expect` is the (sample rate,
/// channels) a restarted stream must keep: the mixer was set up for them.
fn open(&mut self, fill: Arc<Mutex<Fill>>, expect: Option<(u32, u16)>) -> Result<(), String> {
let host = host(&self.hw.device_class);
let DeviceChoice { device: dev, fallback } = output_device(&host, &self.hw.default_output, &self.fallback).ok_or("no output device")?;
self.note = fallback;
@ -214,6 +211,11 @@ impl AudioOut for CpalOut {
if channels == 0 || cfg.sample_rate().0 == 0 {
return Err("the output device returned an invalid channel count or sample rate".into());
}
if let Some((rate, ch)) = expect
&& (rate, ch) != (cfg.sample_rate().0, cfg.channels())
{
return Err(format!("the output device now runs at {} Hz with {} channels", cfg.sample_rate().0, cfg.channels()));
}
let mut config: cpal::StreamConfig = cfg.clone().into();
if let cpal::SupportedBufferSize::Range { min, max } = cfg.buffer_size()
&& (*min..=*max).contains(&self.hw.buffer_size)
@ -222,20 +224,18 @@ impl AudioOut for CpalOut {
}
self.rate = cfg.sample_rate().0;
self.channels = cfg.channels();
self.played.store(0, Ordering::SeqCst);
self.failed.store(false, Ordering::SeqCst);
let played = self.played.clone();
let failed = self.failed.clone();
let stream = match cfg.sample_format() {
cpal::SampleFormat::F32 => dev.build_output_stream(
&config,
move |buf: &mut [f32], _| {
fill(buf, channels);
played.fetch_add((buf.len() / channels) as u64, Ordering::SeqCst);
},
move |e| stream_error(&failed, e),
None,
),
cpal::SampleFormat::F32 => {
let played = self.played.clone();
let health = self.health.clone();
let error_health = health.clone();
dev.build_output_stream(
&config,
move |buf: &mut [f32], _| render(&fill, buf, channels, &played, &health),
move |e| stream_error(&error_health, e),
None,
)
}
cpal::SampleFormat::I8 => self.converted_stream::<i8>(&dev, &config, fill),
cpal::SampleFormat::I16 => self.converted_stream::<i16>(&dev, &config, fill),
cpal::SampleFormat::I24 => self.converted_stream::<cpal::I24>(&dev, &config, fill),
@ -251,10 +251,172 @@ impl AudioOut for CpalOut {
.map_err(|e| e.to_string())?;
stream.play().map_err(|e| e.to_string())?;
self.stream = Some(stream);
Ok(())
}
/// Replace a dropped stream with a new one on the same mixer, clock and format.
fn restart(&mut self) -> Result<(), String> {
let fill = self.fill.clone().ok_or("nothing is playing")?;
let expect = (self.rate, self.channels);
// The old stream (and its audio thread) goes first: only one stream ever runs the mixer.
self.stream = None;
self.health.clear_errors();
self.open(fill, Some(expect))
}
}
/// What the audio thread reports to the UI thread.
#[derive(Default)]
struct Health {
/// The device went away (unplugged): restart the stream at once.
lost: AtomicBool,
/// A backend error since the last check (cpal keeps the stream alive after these): restart
/// only if the stream also stopped consuming samples.
errored: AtomicBool,
/// Unrecoverable: the mixer panicked, a buffer could not be allocated, or restarts ran out.
/// [`AudioOut::played_frames`] reports `None` and playback goes on without sound.
failed: AtomicBool,
/// The first error of a stream has been printed.
reported: AtomicBool,
}
impl Health {
fn clear_errors(&self) {
self.lost.store(false, Ordering::SeqCst);
self.errored.store(false, Ordering::SeqCst);
self.reported.store(false, Ordering::SeqCst);
}
}
/// How long a stream that reported an error may go without consuming samples before it is
/// restarted. Well under the UI's `AUDIO_STALL_S`, which gives up on the device clock.
const STALL_RESTART: Duration = Duration::from_millis(300);
/// At most this many restarts within [`RESTART_WINDOW`]; a device that keeps failing gives up.
const MAX_RESTARTS: usize = 3;
const RESTART_WINDOW: Duration = Duration::from_secs(10);
/// What [`Watchdog::check`] decides about the running stream.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum Verdict {
Healthy,
Restart,
GiveUp,
}
/// Decides when a stream that reported errors is restarted (UI thread, every frame).
#[derive(Default)]
struct Watchdog {
/// Played-frame count last seen, and when it last changed.
seen: (u64, Duration),
/// An error was reported and the stream has not consumed samples since.
pending_error: bool,
/// When recent restarts happened (at most [`MAX_RESTARTS`] kept).
restarts: Vec<Duration>,
}
impl Watchdog {
fn reset(&mut self, now: Duration) {
*self = Self { seen: (0, now), ..Self::default() };
}
fn check(&mut self, now: Duration, played: u64, errored: bool, lost: bool) -> Verdict {
if played != self.seen.0 {
self.seen = (played, now);
self.pending_error = false;
}
// An error that arrives with the stream still consuming samples is only a hiccup.
self.pending_error |= errored;
let stalled = now.saturating_sub(self.seen.1) >= STALL_RESTART;
if !lost && !(self.pending_error && stalled) {
return Verdict::Healthy;
}
self.restarts.retain(|t| now.saturating_sub(*t) < RESTART_WINDOW);
if self.restarts.len() >= MAX_RESTARTS {
return Verdict::GiveUp;
}
self.restarts.push(now);
self.seen = (played, now);
self.pending_error = false;
Verdict::Restart
}
}
/// One device callback: run the shared mixer into `buf` and advance the clock. Never panics: a
/// panicking mixer plays silence and fails the stream, and a mixer held by another stream (never
/// expected: a restart drops the old stream first) plays silence without advancing the clock.
fn render(fill: &Mutex<Fill>, buf: &mut [f32], channels: usize, played: &AtomicU64, health: &Health) {
let mut guard = match fill.try_lock() {
Ok(guard) => guard,
Err(TryLockError::Poisoned(poisoned)) => poisoned.into_inner(),
Err(TryLockError::WouldBlock) => {
buf.fill(0.0);
return;
}
};
if health.failed.load(Ordering::SeqCst) || catch_unwind(AssertUnwindSafe(|| (*guard)(buf, channels))).is_err() {
buf.fill(0.0);
if !health.failed.swap(true, Ordering::SeqCst) {
eprintln!("filmcraft: the audio mixer failed; playing without sound");
}
return;
}
played.fetch_add((buf.len() / channels.max(1)) as u64, Ordering::SeqCst);
}
fn stream_error(health: &Health, e: cpal::StreamError) {
match e {
cpal::StreamError::DeviceNotAvailable => health.lost.store(true, Ordering::SeqCst),
cpal::StreamError::BackendSpecific { .. } => health.errored.store(true, Ordering::SeqCst),
}
if !health.reported.swap(true, Ordering::SeqCst) {
eprintln!("filmcraft: audio stream error: {e}");
}
}
fn convert_samples<T: cpal::Sample + cpal::FromSample<f32>>(input: &[f32], output: &mut [T]) {
// Integer conversion represents [-1, 1). In particular I24 cannot represent +1 exactly.
let upper = f32::from_bits(1.0_f32.to_bits() - 1);
for (sample, converted) in input.iter().zip(output) {
*converted = T::from_sample(if sample.is_finite() { sample.clamp(-1.0, upper) } else { 0.0 });
}
}
impl AudioOut for CpalOut {
fn start(&mut self, fill: Box<dyn FnMut(&mut [f32], usize) + Send>) -> Result<u32, String> {
self.stop();
self.played.store(0, Ordering::SeqCst);
self.health = Arc::new(Health::default());
self.watchdog.reset(self.epoch.elapsed());
let fill = Arc::new(Mutex::new(fill));
self.open(fill.clone(), None)?;
self.fill = Some(fill);
Ok(self.rate)
}
fn stop(&mut self) {
self.stream = None;
self.fill = None;
}
fn recover(&mut self) {
if self.stream.is_none() || self.health.failed.load(Ordering::SeqCst) {
return;
}
let lost = self.health.lost.swap(false, Ordering::SeqCst);
let errored = self.health.errored.swap(false, Ordering::SeqCst);
match self.watchdog.check(self.epoch.elapsed(), self.played.load(Ordering::SeqCst), errored, lost) {
Verdict::Healthy => {}
Verdict::Restart => {
log::warn!("the audio output stream {}; restarting it", if lost { "lost its device" } else { "stopped after an error" });
if let Err(e) = self.restart() {
log::warn!("the audio output stream could not be restarted: {e}");
self.stream = None;
self.health.failed.store(true, Ordering::SeqCst);
}
}
Verdict::GiveUp => {
log::warn!("the audio output stream keeps failing ({MAX_RESTARTS} restarts within {} s)", RESTART_WINDOW.as_secs());
self.health.failed.store(true, Ordering::SeqCst);
}
}
}
fn sample_rate(&self) -> u32 {
self.rate
@ -263,7 +425,7 @@ impl AudioOut for CpalOut {
self.channels as usize
}
fn played_frames(&self) -> Option<u64> {
self.stream.as_ref().filter(|_| !self.failed.load(Ordering::SeqCst)).map(|_| self.played.load(Ordering::SeqCst))
self.stream.as_ref().filter(|_| !self.health.failed.load(Ordering::SeqCst)).map(|_| self.played.load(Ordering::SeqCst))
}
fn devices(&self) -> AudioDevices {
let h = host(&self.hw.device_class);
@ -327,11 +489,92 @@ mod tests {
}
#[test]
fn failed_stream_relinquishes_the_playback_clock() {
let failed = AtomicBool::new(false);
stream_error(&failed, cpal::StreamError::DeviceNotAvailable);
assert!(failed.load(Ordering::SeqCst));
fn stream_errors_ask_for_a_restart_instead_of_failing_playback() {
let health = Health::default();
stream_error(&health, cpal::StreamError::BackendSpecific { err: cpal::BackendSpecificError { description: "`alsa::poll()` returned POLLERR".into() } });
assert!(health.errored.load(Ordering::SeqCst) && !health.lost.load(Ordering::SeqCst));
stream_error(&health, cpal::StreamError::DeviceNotAvailable);
assert!(health.lost.load(Ordering::SeqCst));
assert!(!health.failed.load(Ordering::SeqCst), "a stream error alone must not silence playback");
let output = CpalOut::new();
assert_eq!(output.played_frames(), None);
assert_eq!(output.played_frames(), None, "no stream, no clock");
}
fn ms(n: u64) -> Duration {
Duration::from_millis(n)
}
#[test]
fn an_error_while_samples_keep_flowing_is_only_a_hiccup() {
let mut w = Watchdog::default();
w.reset(ms(0));
assert_eq!(w.check(ms(10), 480, true, false), Verdict::Healthy);
// the stream keeps consuming samples after the error: never restarted
for (i, t) in (20..2_000).step_by(20).enumerate() {
assert_eq!(w.check(ms(t), 480 * (i as u64 + 2), false, false), Verdict::Healthy);
}
}
#[test]
fn a_stream_that_errors_and_stops_consuming_is_restarted() {
let mut w = Watchdog::default();
w.reset(ms(0));
assert_eq!(w.check(ms(10), 480, false, false), Verdict::Healthy);
// POLLERR: the backend reports errors and no longer calls back
assert_eq!(w.check(ms(20), 480, true, false), Verdict::Healthy);
assert_eq!(w.check(ms(200), 480, true, false), Verdict::Healthy);
assert_eq!(w.check(ms(310), 480, false, false), Verdict::Restart);
// the restarted stream gets a fresh grace period
assert_eq!(w.check(ms(400), 480, false, false), Verdict::Healthy);
// a stall without any reported error is left to the UI's own watchdog
assert_eq!(w.check(ms(2_000), 480, false, false), Verdict::Healthy);
}
#[test]
fn a_lost_device_restarts_at_once_and_repeated_failures_give_up() {
let mut w = Watchdog::default();
w.reset(ms(0));
assert_eq!(w.check(ms(5), 0, false, true), Verdict::Restart);
assert_eq!(w.check(ms(10), 0, false, true), Verdict::Restart);
assert_eq!(w.check(ms(15), 0, false, true), Verdict::Restart);
assert_eq!(w.check(ms(20), 0, false, true), Verdict::GiveUp);
// sporadic failures far apart keep being recovered
assert_eq!(w.check(ms(10_006), 0, false, true), Verdict::Restart);
}
#[test]
fn the_device_callback_never_panics_and_counts_played_frames() {
let (played, health) = (AtomicU64::new(0), Health::default());
let fill: Mutex<Fill> = Mutex::new(Box::new(|buf: &mut [f32], _| buf.fill(0.25)));
let mut buf = [1.0_f32; 8];
render(&fill, &mut buf, 2, &played, &health);
assert_eq!((buf, played.load(Ordering::SeqCst)), ([0.25; 8], 4));
// a mixer still held elsewhere plays silence without moving the clock
{
let _held = fill.lock().unwrap();
render(&fill, &mut buf, 2, &played, &health);
}
assert_eq!((buf, played.load(Ordering::SeqCst)), ([0.0; 8], 4));
// zero channels from a broken driver must not divide by zero
render(&fill, &mut buf, 0, &played, &health);
assert_eq!(played.load(Ordering::SeqCst), 12);
}
#[test]
fn a_panicking_mixer_plays_silence_and_fails_the_stream() {
let (played, health) = (AtomicU64::new(0), Health::default());
let fill: Mutex<Fill> = Mutex::new(Box::new(|buf: &mut [f32], _| {
buf.fill(0.5);
panic!("mixer bug");
}));
let mut buf = [1.0_f32; 4];
render(&fill, &mut buf, 2, &played, &health);
assert_eq!(buf, [0.0; 4]);
assert_eq!(played.load(Ordering::SeqCst), 0);
assert!(health.failed.load(Ordering::SeqCst));
// later callbacks stay silent instead of re-entering the broken mixer
buf = [1.0; 4];
render(&fill, &mut buf, 2, &played, &health);
assert_eq!(buf, [0.0; 4]);
}
}

831
apps/filmcraft/src/graphics.rs
View File

@ -1,33 +1,487 @@
//! Windows graphics instance configuration, applied before wgpu can initialize any driver.
//! Graphics start-up: the wgpu backends and adapter the window is created with, and the fallback
//! when a start fails (docs/gaps.md G4, #820).
//!
//! A graphics driver can crash inside wgpu instance or device creation (Intel's Vulkan driver
//! `igvk64.dll`, #512 #413; AMD's OpenGL driver `atio6axx.dll`, #477), where no Rust error
//! handling runs, so every launch died the same way until the user set `WGPU_BACKEND` by hand.
//!
//! # When a start counts as failed
//! The data folder keeps the mode to start with and the failed starts in a row in it
//! ([`STATE_FILE`]). Each start also leaves its own attempt file (`graphics-start/<pid>-<n>.json`,
//! so concurrent launches never write the same file) that is removed once the window has painted
//! two frames, when the process ends normally, or [`DEATH_WINDOW`] after launch, whatever happened
//! by then. An attempt file found by a later launch therefore means a process that died within
//! [`DEATH_WINDOW`] of its start without painting: a driver crash, typically. A start that hangs,
//! or is force-quit or crashes later, is not counted; neither is another instance that is still
//! starting (its attempt is younger than [`DEATH_WINDOW`] and is left alone). After
//! [`FAILURES_TO_MOVE_ON`] such failures in a row in the same mode, the launch moves on to the next
//! [`Mode`] of the platform's [`LADDER`] and keeps it; a start that paints resets the count. A
//! start-up error that wgpu does report (no adapter, device creation failed) moves on at once,
//! within the same process.
//!
//! # Getting back
//! A fallback mode shows a notice with a reset button in the window (`graphics.notice.reset`,
//! [`GraphicsStart::reset_to_default`]); `--graphics default` does the same from the command
//! line, and `--graphics dx12|software` picks a mode by hand. An explicit `WGPU_BACKEND` leaves
//! the saved state alone: the user is in charge.
//!
//! All files are written beside and renamed into place, so a reader never sees half a file; two
//! instances updating [`STATE_FILE`] at the same moment can lose one update (a failure counted
//! once less, a confirmation repeated by the next start), never corrupt it.
use eframe::{NativeOptions, egui_wgpu::WgpuSetup, wgpu::Backends};
use std::io::Read as _;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};
use std::time::{Duration, SystemTime, UNIX_EPOCH};
/// Restrict the wgpu instance to the primary backends (Vulkan, DirectX 12, Metal), i.e. leave
/// OpenGL out, unless `backend_override` (normally `Backends::from_env()`, i.e. `WGPU_BACKEND`)
/// names other backends.
pub fn configure(options: &mut NativeOptions, backend_override: Option<Backends>) {
if let WgpuSetup::CreateNew(create) = &mut options.wgpu_options.wgpu_setup {
// eframe's default includes GL. Creating that backend can crash inside AMD's
// atio6axx.dll before adapter selection or any Rust error handling runs, so the
// window never appears. Choosing an adapter afterwards is too late: only leaving GL
// out of the instance avoids it. Keep Vulkan and DX12 both: forcing DX12 alone sends
// shaders through FXC (which rejects fx.wgsl, silently falling back to the CPU) and
// leaves machines without DX12 with no backend at all.
create.instance_descriptor.backends = backend_override.unwrap_or(Backends::PRIMARY);
use eframe::{
NativeOptions,
egui_wgpu::WgpuSetup,
wgpu::{self, Backends},
};
/// The saved graphics state in the data folder: `{"mode": "<mode>", "failures": <n>}`.
pub const STATE_FILE: &str = "graphics-start.json";
/// The folder in the data folder with one attempt file per start that has not painted yet.
pub const ATTEMPTS_DIR: &str = "graphics-start";
/// How long after launch a start that has not painted may die and still count as failed. A
/// driver crash in instance or device creation happens within a second or two.
pub const DEATH_WINDOW: Duration = Duration::from_secs(15);
/// Failed starts in a row, in the same mode, that move on to the next mode.
pub const FAILURES_TO_MOVE_ON: u32 = 2;
/// Attempt files looked at per launch; anything beyond is left for a later launch.
const MAX_ATTEMPTS: usize = 64;
/// The state and attempt files are a few bytes; anything longer is not ours.
const MAX_FILE: u64 = 4096;
/// How the window's graphics device is created.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Mode {
/// The platform default: Vulkan or DirectX 12 on Windows (no OpenGL, see [`apply`]), Metal on
/// macOS, Vulkan or OpenGL elsewhere.
Default,
/// DirectX 12 only (Windows), for Vulkan drivers that crash at start-up.
Dx12,
/// A software adapter (WARP on Windows, lavapipe / llvmpipe elsewhere); the preview then
/// composites on the CPU (`FilmcraftApp::set_wgpu`).
Software,
}
impl Mode {
/// The name in [`STATE_FILE`] and on the command line (`--graphics <name>`).
pub fn name(self) -> &'static str {
match self {
Mode::Default => "default",
Mode::Dx12 => "dx12",
Mode::Software => "software",
}
}
/// The English name shown in the interface (translated there).
pub fn label(self) -> &'static str {
match self {
Mode::Default => "default graphics",
Mode::Dx12 => "DirectX 12 only",
Mode::Software => "software rendering",
}
}
pub fn parse(name: &str) -> Option<Mode> {
[Mode::Default, Mode::Dx12, Mode::Software].into_iter().find(|m| m.name() == name)
}
}
/// The modes tried in order on this platform. macOS has only Metal, and no software adapter.
pub const LADDER: &[Mode] = if cfg!(target_os = "windows") {
&[Mode::Default, Mode::Dx12, Mode::Software]
} else if cfg!(target_os = "macos") {
&[Mode::Default]
} else {
&[Mode::Default, Mode::Software]
};
/// The mode after `mode` in `ladder`, if any.
fn next_in(ladder: &[Mode], mode: Mode) -> Option<Mode> {
let i = ladder.iter().position(|m| *m == mode)?;
ladder.get(i.checked_add(1)?).copied()
}
/// [`STATE_FILE`]: the mode to start with and the failed starts in a row in it.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
struct Saved {
mode: Mode,
failures: u32,
}
/// An attempt file left by a start: its process id (from the file name), mode and launch time.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
struct Attempt {
pid: u32,
mode: Mode,
/// Seconds since the Unix epoch.
started: u64,
}
/// What a launch makes of the saved state and the attempt files it found.
#[derive(Debug, PartialEq, Eq)]
struct Decision {
/// The state to start with and save.
saved: Saved,
/// Moved on to the next mode because the last starts failed.
moved_on: bool,
/// Per attempt: done with (counted, or not ours to count), delete it.
delete: Vec<bool>,
}
/// This launch's mode from the saved state and the attempts other starts left. `now` is in Unix
/// seconds, `pid` is this process (an attempt with our process id is a dead process whose id was
/// reused), `window` is [`DEATH_WINDOW`] in seconds.
fn decide(saved: Option<Saved>, attempts: &[Attempt], now: u64, pid: u32, window: u64, ladder: &[Mode]) -> Decision {
let first = ladder.first().copied().unwrap_or(Mode::Default);
// Nothing saved, or a mode this platform does not use (a data folder copied from another OS).
let saved = saved.filter(|s| ladder.contains(&s.mode)).unwrap_or(Saved { mode: first, failures: 0 });
let mut failures = saved.failures;
let mut delete = Vec::with_capacity(attempts.len());
for a in attempts {
if a.started > now.saturating_add(window) {
// From the future (a clock that jumped back): unknown, and not worth keeping.
delete.push(true);
continue;
}
let dead = a.pid == pid || now.saturating_sub(a.started) >= window;
// A younger attempt is another instance starting right now: leave it to that instance.
delete.push(dead);
// Only failures of the mode we would start in count; older ones are history.
if dead && a.mode == saved.mode {
failures = failures.saturating_add(1);
}
}
if failures >= FAILURES_TO_MOVE_ON {
return match next_in(ladder, saved.mode) {
Some(next) => Decision { saved: Saved { mode: next, failures: 0 }, moved_on: true, delete },
// Nothing left to try: the last mode is still the best chance.
None => Decision { saved: Saved { mode: saved.mode, failures: 0 }, moved_on: false, delete },
};
}
Decision { saved: Saved { mode: saved.mode, failures }, moved_on: false, delete }
}
fn read_small(path: &Path) -> Option<String> {
let mut text = String::new();
std::fs::File::open(path).ok()?.take(MAX_FILE).read_to_string(&mut text).ok()?;
Some(text)
}
/// [`Saved`] from its text; `None` for anything else. A missing failure count is none.
fn parse_state(text: &str) -> Option<Saved> {
let v: serde_json::Value = serde_json::from_str(text).ok()?;
let mode = Mode::parse(v.get("mode")?.as_str()?)?;
let failures = match v.get("failures") {
None => 0,
Some(n) => u32::try_from(n.as_u64()?).ok()?,
};
Some(Saved { mode, failures })
}
fn state_json(s: Saved) -> String {
serde_json::json!({ "mode": s.mode.name(), "failures": s.failures }).to_string()
}
/// An attempt from its file name (`<pid>-<n>.json`) and text; `None` for anything else.
fn parse_attempt(file_name: &str, text: &str) -> Option<Attempt> {
let pid = file_name.strip_suffix(".json")?.split_once('-')?.0.parse().ok()?;
let v: serde_json::Value = serde_json::from_str(text).ok()?;
Some(Attempt { pid, mode: Mode::parse(v.get("mode")?.as_str()?)?, started: v.get("started")?.as_u64()? })
}
fn attempt_json(mode: Mode, started: u64) -> String {
serde_json::json!({ "mode": mode.name(), "started": started }).to_string()
}
/// Write `text` beside `path` (a name only this process uses) and rename it into place, so a
/// start that dies mid-write, or another instance reading, never sees half a file.
fn write_atomic(path: &Path, text: &str) -> std::io::Result<()> {
if let Some(dir) = path.parent() {
std::fs::create_dir_all(dir)?;
}
let mut tmp = path.as_os_str().to_owned();
tmp.push(format!(".{}.tmp", std::process::id()));
let tmp = PathBuf::from(tmp);
std::fs::write(&tmp, text)?;
std::fs::rename(&tmp, path).inspect_err(|_| {
let _ = std::fs::remove_file(&tmp);
})
}
fn remove_quietly(path: &Path) {
match std::fs::remove_file(path) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => log::warn!("can't remove {}: {e}", path.display()),
_ => {}
}
}
/// The attempt files in `dir` (at most [`MAX_ATTEMPTS`]): path and contents, `None` when unreadable.
fn list_attempts(dir: &Path) -> Vec<(PathBuf, Option<Attempt>)> {
let Ok(entries) = std::fs::read_dir(dir) else { return Vec::new() };
entries
.filter_map(Result::ok)
.filter_map(|e| {
let name = e.file_name().into_string().ok()?;
name.ends_with(".json").then(|| {
let path = e.path();
let attempt = read_small(&path).and_then(|text| parse_attempt(&name, &text));
(path, attempt)
})
})
.take(MAX_ATTEMPTS)
.collect()
}
fn unix_now() -> u64 {
SystemTime::now().duration_since(UNIX_EPOCH).map_or(0, |d| d.as_secs())
}
/// Attempt numbers within this process, so each attempt file has its own name.
static NEXT_ATTEMPT: AtomicU32 = AtomicU32::new(0);
/// This launch's graphics mode and its files in the data folder.
#[derive(Clone, Debug)]
pub struct GraphicsStart {
/// [`STATE_FILE`]; `None` without a data folder or with `WGPU_BACKEND` set.
state: Option<PathBuf>,
/// This start's attempt file (with `state`).
attempt: Option<PathBuf>,
mode: Mode,
/// [`Self::reset_to_default`] was used: a later [`Self::confirm`] must not save this mode over it.
reset: Arc<AtomicBool>,
}
impl GraphicsStart {
/// Pick this launch's mode from the data folder and record the attempt. `forced`
/// (`--graphics`) starts in that mode and saves it. `explicit_backend` (`WGPU_BACKEND` set)
/// leaves the data folder alone and starts in `forced`, else [`Mode::Default`].
pub fn begin(data_dir: Option<&Path>, explicit_backend: bool, forced: Option<Mode>) -> Self {
let start = Self::begin_at(data_dir, explicit_backend, forced, unix_now(), std::process::id());
if let Some(path) = start.attempt.clone() {
expire_later(path);
}
start
}
/// [`Self::begin`] at `now` (Unix seconds) as process `pid`, without the expiry timer.
fn begin_at(data_dir: Option<&Path>, explicit_backend: bool, forced: Option<Mode>, now: u64, pid: u32) -> Self {
let first = LADDER.first().copied().unwrap_or(Mode::Default);
let forced = forced.map(|m| if LADDER.contains(&m) { m } else { first });
let reset = Arc::new(AtomicBool::new(false));
let Some(dir) = data_dir.filter(|_| !explicit_backend) else {
return Self { state: None, attempt: None, mode: forced.unwrap_or(Mode::Default), reset };
};
let state = dir.join(STATE_FILE);
let attempts_dir = dir.join(ATTEMPTS_DIR);
let saved = read_small(&state).and_then(|t| parse_state(&t));
let found = list_attempts(&attempts_dir);
let known: Vec<Attempt> = found.iter().filter_map(|(_, a)| *a).collect();
let decision = decide(saved, &known, now, pid, DEATH_WINDOW.as_secs(), LADDER);
let mut to_delete = decision.delete.iter();
for (path, attempt) in &found {
// unreadable files in our folder are debris (a different version, a damaged disk)
if attempt.is_none() || to_delete.next().copied().unwrap_or(false) {
remove_quietly(path);
}
}
let file = state.display();
let next = match forced {
Some(mode) => {
log::info!("graphics mode {} chosen with --graphics", mode.name());
Saved { mode, failures: 0 }
}
None => {
if decision.moved_on {
let failed = saved.map_or(first, |s| s.mode).name();
log::warn!(
"the last {FAILURES_TO_MOVE_ON} starts with graphics mode {failed} died before their window painted; starting with {} (`--graphics default` or the notice in the window goes back)",
decision.saved.mode.name()
);
} else if decision.saved.failures > 0 {
log::warn!("the last start with graphics mode {} died before its window painted", decision.saved.mode.name());
} else if decision.saved.mode != Mode::Default {
log::info!("graphics mode {} after earlier failed starts ({file}; `--graphics default` goes back)", decision.saved.mode.name());
}
decision.saved
}
};
if saved != Some(next)
&& let Err(e) = write_atomic(&state, &state_json(next))
{
log::warn!("can't write {file}: {e}");
}
let start = Self { state: Some(state), attempt: None, mode: next.mode, reset };
start.with_new_attempt(&attempts_dir, now, pid)
}
/// Record a new attempt file for the current mode.
fn with_new_attempt(mut self, dir: &Path, now: u64, pid: u32) -> Self {
let n = NEXT_ATTEMPT.fetch_add(1, Ordering::Relaxed);
let path = dir.join(format!("{pid}-{n}.json"));
match write_atomic(&path, &attempt_json(self.mode, now)) {
Ok(()) => self.attempt = Some(path),
Err(e) => log::warn!("can't write {}: {e}", path.display()),
}
self
}
pub fn mode(&self) -> Mode {
self.mode
}
/// Running in a fallback mode, i.e. not the platform default.
pub fn degraded(&self) -> bool {
self.mode != Mode::Default
}
/// Move on to the next mode after a start-up error wgpu reported; `false` when none is left.
/// The error is certain, so the next mode is saved at once.
pub fn step_down(&mut self) -> bool {
let Some(next) = next_in(LADDER, self.mode) else {
return false;
};
log::warn!("graphics mode {} failed to start; trying {}", self.mode.name(), next.name());
self.mode = next;
if let Some(state) = self.state.clone() {
self.save(&state, Saved { mode: next, failures: 0 });
if let Some(old) = self.attempt.take() {
remove_quietly(&old);
if let Some(dir) = old.parent() {
let path = self.clone().with_new_attempt(dir, unix_now(), std::process::id()).attempt;
if let Some(path) = &path {
expire_later(path.clone());
}
self.attempt = path;
}
}
}
true
}
/// The window is up: keep this mode, and the failed starts in a row are over.
pub fn confirm(&self) {
self.finish();
let Some(state) = &self.state else { return };
if self.reset.load(Ordering::Relaxed) {
return;
}
let ok = Saved { mode: self.mode, failures: 0 };
if read_small(state).and_then(|t| parse_state(&t)) != Some(ok) {
self.save(state, ok);
}
}
/// This start is over, whether it painted or not (the window closed, or every mode failed
/// with an error that was reported): its attempt no longer means a crash.
pub fn finish(&self) {
if let Some(path) = &self.attempt {
remove_quietly(path);
}
}
/// Start with the default mode next time (the notice's reset button). This process keeps
/// its mode: the graphics device can't be swapped under a running window.
pub fn reset_to_default(&self) -> Result<(), String> {
self.reset.store(true, Ordering::Relaxed);
let Some(state) = &self.state else { return Ok(()) };
let first = LADDER.first().copied().unwrap_or(Mode::Default);
write_atomic(state, &state_json(Saved { mode: first, failures: 0 })).map_err(|e| format!("can't write {}: {e}", state.display()))?;
log::info!("graphics mode reset to {} for the next start", first.name());
Ok(())
}
fn save(&self, path: &Path, s: Saved) {
if let Err(e) = write_atomic(path, &state_json(s)) {
log::warn!("can't write {}: {e}", path.display());
}
}
}
/// Remove the attempt file at `path` [`DEATH_WINDOW`] from now: a start still alive then is not
/// counted as a failed one, painted or not.
fn expire_later(path: PathBuf) {
let spawned = std::thread::Builder::new().name("graphics-start".into()).spawn(move || {
let _ = std::panic::catch_unwind(move || {
std::thread::sleep(DEATH_WINDOW);
remove_quietly(&path);
});
});
if let Err(e) = spawned {
log::warn!("can't start the graphics start timer: {e}");
}
}
/// Configure `options` for `mode`, before eframe creates the wgpu instance. `backend_override`
/// is normally `Backends::from_env()`, i.e. `WGPU_BACKEND`.
pub fn apply(options: &mut NativeOptions, mode: Mode, backend_override: Option<Backends>) {
apply_for(options, mode, backend_override, cfg!(target_os = "windows"));
}
fn apply_for(options: &mut NativeOptions, mode: Mode, backend_override: Option<Backends>, windows: bool) {
let WgpuSetup::CreateNew(create) = &mut options.wgpu_options.wgpu_setup else {
return;
};
match mode {
// eframe's default (`WGPU_BACKEND`, else PRIMARY | GL), except on Windows: there it
// includes GL. Creating that backend can crash inside AMD's atio6axx.dll before adapter
// selection or any Rust error handling runs, so the window never appears. Choosing an
// adapter afterwards is too late: only leaving GL out of the instance avoids it. Keep
// Vulkan and DX12 both: forcing DX12 alone sends shaders through FXC (which rejects
// fx.wgsl, silently falling back to the CPU) and leaves machines without DX12 with no
// backend at all.
Mode::Default if windows => create.instance_descriptor.backends = backend_override.unwrap_or(Backends::PRIMARY),
Mode::Default => {}
// Without Vulkan, whose driver is what crashes; GPU effects may fall back to the CPU.
Mode::Dx12 => create.instance_descriptor.backends = Backends::DX12,
Mode::Software => {
create.instance_descriptor.backends = software_backends(windows);
create.native_adapter_selector = Some(Arc::new(pick_software));
}
}
}
/// The backends a software adapter is looked for in: WARP is a DX12 adapter, and GL stays out on
/// Windows for the reason in [`apply_for`].
fn software_backends(windows: bool) -> Backends {
if windows { Backends::DX12 } else { Backends::VULKAN | Backends::GL }
}
/// The first software adapter that can draw to the window's surface.
fn pick_software(adapters: &[wgpu::Adapter], surface: Option<&wgpu::Surface<'_>>) -> Result<wgpu::Adapter, String> {
adapters
.iter()
.find(|a| filmcraft_ui_egui::software_rasterizer(&a.get_info()).is_some() && surface.is_none_or(|s| a.is_surface_supported(s)))
.cloned()
.ok_or_else(|| format!("no software graphics adapter (WARP, lavapipe, llvmpipe) among {} adapters", adapters.len()))
}
/// Whether `err` from `eframe::run_native` is a graphics start-up failure that another mode may
/// avoid (as opposed to a display, event-loop or app error).
pub fn is_graphics_failure(err: &eframe::Error) -> bool {
matches!(err, eframe::Error::Wgpu(_))
}
#[cfg(test)]
mod tests {
use super::*;
fn configured_backends(backend_override: Option<Backends>) -> Backends {
const WINDOWS: &[Mode] = &[Mode::Default, Mode::Dx12, Mode::Software];
const LINUX: &[Mode] = &[Mode::Default, Mode::Software];
const MACOS: &[Mode] = &[Mode::Default];
fn configured(mode: Mode, backend_override: Option<Backends>, windows: bool) -> (Backends, bool) {
let mut options = NativeOptions::default();
configure(&mut options, backend_override);
apply_for(&mut options, mode, backend_override, windows);
let WgpuSetup::CreateNew(create) = options.wgpu_options.wgpu_setup else {
panic!("default native options must create a wgpu instance");
};
create.instance_descriptor.backends
(create.instance_descriptor.backends, create.native_adapter_selector.is_some())
}
#[test]
@ -37,16 +491,349 @@ mod tests {
panic!("default native options must create a wgpu instance");
};
assert!(eframe_default.instance_descriptor.backends.contains(Backends::GL));
let configured = configured_backends(None);
assert_eq!(configured, Backends::PRIMARY);
assert!(!configured.contains(Backends::GL));
assert!(configured.contains(Backends::DX12) && configured.contains(Backends::VULKAN));
let (backends, selector) = configured(Mode::Default, None, true);
assert_eq!(backends, Backends::PRIMARY);
assert!(!backends.contains(Backends::GL));
assert!(backends.contains(Backends::DX12) && backends.contains(Backends::VULKAN));
assert!(!selector);
}
#[test]
fn explicit_backend_override_is_preserved() {
for backends in [Backends::VULKAN, Backends::GL, Backends::DX12, Backends::VULKAN | Backends::DX12, Backends::empty()] {
assert_eq!(configured_backends(Some(backends)), backends);
assert_eq!(configured(Mode::Default, Some(backends), true).0, backends);
}
}
#[test]
fn fallback_modes_leave_out_the_crashing_backends() {
// DX12 only: Intel's and AMD's Vulkan drivers are what crashed (#512, #413).
assert_eq!(configured(Mode::Dx12, None, true), (Backends::DX12, false));
// Software on Windows is WARP (DX12), never GL (atio6axx.dll, #477).
assert_eq!(configured(Mode::Software, None, true), (Backends::DX12, true));
assert_eq!(configured(Mode::Software, None, false), (Backends::VULKAN | Backends::GL, true));
// Off Windows the default is eframe's own.
let WgpuSetup::CreateNew(eframe_default) = NativeOptions::default().wgpu_options.wgpu_setup else {
panic!("default native options must create a wgpu instance");
};
assert_eq!(configured(Mode::Default, None, false), (eframe_default.instance_descriptor.backends, false));
}
const W: u64 = 15;
const NOW: u64 = 1_000_000;
const ME: u32 = 7;
fn saved(mode: Mode, failures: u32) -> Option<Saved> {
Some(Saved { mode, failures })
}
fn dead(mode: Mode) -> Attempt {
Attempt { pid: 1, mode, started: NOW - 100 }
}
fn mode_after(s: Option<Saved>, attempts: &[Attempt], ladder: &[Mode]) -> (Mode, u32, bool) {
let d = decide(s, attempts, NOW, ME, W, ladder);
(d.saved.mode, d.saved.failures, d.moved_on)
}
#[test]
fn two_failed_starts_in_a_row_move_on_to_the_next_mode_and_keep_it() {
// first launch
assert_eq!(mode_after(None, &[], WINDOWS), (Mode::Default, 0, false));
// one start died: counted, but the same mode is tried again
assert_eq!(mode_after(None, &[dead(Mode::Default)], WINDOWS), (Mode::Default, 1, false));
// the second in a row moves on: DX12, then software
assert_eq!(mode_after(saved(Mode::Default, 1), &[dead(Mode::Default)], WINDOWS), (Mode::Dx12, 0, true));
assert_eq!(mode_after(saved(Mode::Dx12, 1), &[dead(Mode::Dx12)], WINDOWS), (Mode::Software, 0, true));
assert_eq!(mode_after(saved(Mode::Default, 1), &[dead(Mode::Default)], LINUX), (Mode::Software, 0, true));
// two dead instances found at once count as two
assert_eq!(mode_after(None, &[dead(Mode::Default), dead(Mode::Default)], WINDOWS), (Mode::Dx12, 0, true));
// a saved mode is kept
assert_eq!(mode_after(saved(Mode::Dx12, 0), &[], WINDOWS), (Mode::Dx12, 0, false));
assert_eq!(mode_after(saved(Mode::Default, 1), &[], WINDOWS), (Mode::Default, 1, false));
// nothing left to try: stay
assert_eq!(mode_after(saved(Mode::Software, 1), &[dead(Mode::Software)], WINDOWS), (Mode::Software, 0, false));
assert_eq!(mode_after(saved(Mode::Default, 5), &[dead(Mode::Default)], MACOS), (Mode::Default, 0, false));
// a mode this platform has not: start over
assert_eq!(mode_after(saved(Mode::Dx12, 1), &[dead(Mode::Dx12)], LINUX), (Mode::Default, 0, false));
assert_eq!(mode_after(saved(Mode::Software, 0), &[], MACOS), (Mode::Default, 0, false));
assert_eq!(mode_after(None, &[], &[]), (Mode::Default, 0, false));
// a hostile count doesn't overflow
assert_eq!(mode_after(saved(Mode::Default, u32::MAX), &[dead(Mode::Default)], WINDOWS), (Mode::Dx12, 0, true));
}
#[test]
fn only_starts_that_died_soon_after_launch_in_the_current_mode_count() {
let starting = Attempt { pid: 2, mode: Mode::Default, started: NOW - 3 };
let just_now = Attempt { pid: 3, mode: Mode::Default, started: NOW };
let future = Attempt { pid: 4, mode: Mode::Default, started: NOW + 3600 };
let older_mode = dead(Mode::Dx12);
let mine = Attempt { pid: ME, mode: Mode::Default, started: NOW - 1 };
let d = decide(saved(Mode::Default, 0), &[starting, just_now, future, older_mode, mine], NOW, ME, W, WINDOWS);
// another instance still starting is left alone; one from the future is dropped; a
// failure in another mode is history; our own process id can only be a dead process
assert_eq!(d.delete, [false, false, true, true, true]);
assert_eq!((d.saved, d.moved_on), (Saved { mode: Mode::Default, failures: 1 }, false));
// the clock a little behind another instance's: still starting
let ahead = Attempt { pid: 5, mode: Mode::Default, started: NOW + 2 };
assert_eq!(decide(None, &[ahead], NOW, ME, W, WINDOWS).delete, [false]);
}
#[test]
fn the_files_round_trip_and_reject_anything_else() {
for mode in WINDOWS {
for failures in [0, 1, 2, u32::MAX] {
assert_eq!(parse_state(&state_json(Saved { mode: *mode, failures })), saved(*mode, failures));
}
assert_eq!(parse_attempt("12-0.json", &attempt_json(*mode, NOW)), Some(Attempt { pid: 12, mode: *mode, started: NOW }));
}
// without a count (an earlier build of this change wrote `pending`): none
assert_eq!(parse_state(r#"{"mode":"dx12","pending":true}"#), saved(Mode::Dx12, 0));
for bad in [
"",
"{",
"null",
"[]",
r#"{"mode":"vulkan","failures":1}"#,
r#"{"failures":1}"#,
r#"{"mode":1,"failures":1}"#,
r#"{"mode":"dx12","failures":"one"}"#,
r#"{"mode":"dx12","failures":-1}"#,
r#"{"mode":"dx12","failures":4294967296}"#,
] {
assert_eq!(parse_state(bad), None, "{bad}");
}
let good = attempt_json(Mode::Default, NOW);
for (name, text) in [
("12-0.json", "{"),
("12-0.json", r#"{"mode":"default"}"#),
("12-0.json", r#"{"mode":"default","started":-5}"#),
("x-0.json", good.as_str()),
("12.json", good.as_str()),
("-0.json", good.as_str()),
("99999999999-0.json", good.as_str()),
("12-0.txt", good.as_str()),
] {
assert_eq!(parse_attempt(name, text), None, "{name} {text}");
}
}
fn temp_dir(name: &str) -> PathBuf {
let dir = std::env::temp_dir().join(format!("filmcraft-graphics-{name}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
dir
}
fn state_of(dir: &Path) -> Option<Saved> {
read_small(&dir.join(STATE_FILE)).and_then(|t| parse_state(&t))
}
fn attempt_files(dir: &Path) -> usize {
std::fs::read_dir(dir.join(ATTEMPTS_DIR)).map_or(0, |d| d.filter_map(Result::ok).count())
}
/// A launch as another process (`pid`) at `now`.
fn launch(dir: &Path, pid: u32, now: u64) -> GraphicsStart {
GraphicsStart::begin_at(Some(dir), false, None, now, pid)
}
#[test]
fn a_start_that_paints_keeps_its_mode_and_clears_its_attempt() {
let dir = temp_dir("paint");
let start = launch(&dir, 10, NOW);
assert_eq!(start.mode(), Mode::Default);
assert!(!start.degraded());
assert_eq!(state_of(&dir), saved(Mode::Default, 0));
assert_eq!(attempt_files(&dir), 1);
start.confirm();
assert_eq!(attempt_files(&dir), 0);
// a window closed before it painted is no failure either
let start = launch(&dir, 11, NOW + 60);
start.finish();
assert_eq!(attempt_files(&dir), 0);
assert_eq!(launch(&dir, 12, NOW + 120).mode(), Mode::Default);
assert_eq!(state_of(&dir), saved(Mode::Default, 0));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn two_dead_starts_move_on_and_a_painted_one_resets_the_count() {
let dir = temp_dir("dead");
let second = LADDER.get(1).copied().unwrap_or(Mode::Default);
// dies without painting
let _ = launch(&dir, 20, NOW);
// the next launch counts it, tries the same mode again and dies too
let again = launch(&dir, 21, NOW + 60);
assert_eq!(again.mode(), Mode::Default);
assert_eq!(state_of(&dir), saved(Mode::Default, 1));
// two in a row: the next mode, kept
let moved = launch(&dir, 22, NOW + 120);
assert_eq!(moved.mode(), second);
assert_eq!(moved.degraded(), second != Mode::Default);
moved.confirm();
assert_eq!(launch(&dir, 23, NOW + 180).mode(), second);
// one failure, then a start that paints: the count starts over
let _ = std::fs::remove_dir_all(&dir);
let _ = launch(&dir, 30, NOW);
let painted = launch(&dir, 31, NOW + 60);
painted.confirm();
assert_eq!(state_of(&dir), saved(Mode::Default, 0));
let _ = launch(&dir, 32, NOW + 120);
assert_eq!(launch(&dir, 33, NOW + 180).mode(), Mode::Default);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn concurrent_launches_neither_count_nor_disturb_each_other() {
let dir = temp_dir("race");
// two instances start within the same seconds; neither has painted yet
let a = launch(&dir, 40, NOW);
let b = launch(&dir, 41, NOW + 1);
let c = launch(&dir, 42, NOW + 2);
assert_eq!((a.mode(), b.mode(), c.mode()), (Mode::Default, Mode::Default, Mode::Default));
assert_eq!(state_of(&dir), saved(Mode::Default, 0));
assert_eq!(attempt_files(&dir), 3, "each instance has its own attempt file");
b.confirm();
a.confirm();
c.confirm();
assert_eq!(attempt_files(&dir), 0);
assert_eq!(launch(&dir, 43, NOW + 60).mode(), Mode::Default);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn corrupt_huge_or_foreign_files_start_with_the_default() {
let dir = temp_dir("corrupt");
let _ = std::fs::create_dir_all(dir.join(ATTEMPTS_DIR));
let huge = format!(r#"{{"mode":"software","failures":1,"pad":"{}"}}"#, "x".repeat(10_000));
for text in ["garbage", huge.as_str()] {
let _ = std::fs::write(dir.join(STATE_FILE), text);
let _ = std::fs::write(dir.join(ATTEMPTS_DIR).join("50-0.json"), text);
let _ = std::fs::write(dir.join(ATTEMPTS_DIR).join("notes.txt"), text);
let start = launch(&dir, 51, NOW);
assert_eq!(start.mode(), Mode::Default);
// replaced with a well-formed state; the unreadable attempt is gone, other files stay
assert_eq!(state_of(&dir), saved(Mode::Default, 0));
assert!(!dir.join(ATTEMPTS_DIR).join("50-0.json").exists());
assert!(dir.join(ATTEMPTS_DIR).join("notes.txt").exists());
start.finish();
}
// a data folder that is a file: nothing can be written, the default still starts
let file = dir.join("not-a-folder");
let _ = std::fs::write(&file, "x");
let mut start = GraphicsStart::begin_at(Some(&file), false, None, NOW, 52);
assert_eq!(start.mode(), Mode::Default);
start.confirm();
let _ = start.step_down();
assert!(start.reset_to_default().is_err());
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn many_attempt_files_are_bounded_per_launch() {
let dir = temp_dir("many");
let attempts = dir.join(ATTEMPTS_DIR);
let _ = std::fs::create_dir_all(&attempts);
let n = MAX_ATTEMPTS + 10;
for pid in 0..n {
let _ = std::fs::write(attempts.join(format!("{}-0.json", pid + 1000)), attempt_json(Mode::Software, NOW + 1));
}
// all of them belong to instances still starting: none is counted or removed
let start = launch(&dir, 60, NOW);
assert_eq!(start.mode(), Mode::Default);
assert_eq!(attempt_files(&dir), n + 1);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn graphics_flag_picks_the_mode_and_default_undoes_a_fallback() {
let dir = temp_dir("forced");
let last = LADDER.last().copied().unwrap_or(Mode::Default);
let _ = write_atomic(&dir.join(STATE_FILE), &state_json(Saved { mode: last, failures: 1 }));
launch(&dir, 70, NOW).confirm();
// `--graphics default`
let start = GraphicsStart::begin_at(Some(&dir), false, Some(Mode::Default), NOW + 60, 71);
assert_eq!(start.mode(), Mode::Default);
assert_eq!(state_of(&dir), saved(Mode::Default, 0));
start.confirm();
assert_eq!(launch(&dir, 72, NOW + 120).mode(), Mode::Default);
// `--graphics <the last mode>` is kept for the next launches
GraphicsStart::begin_at(Some(&dir), false, Some(last), NOW + 180, 73).confirm();
assert_eq!(launch(&dir, 74, NOW + 240).mode(), last);
// a mode this platform has not starts with the first one
if !LADDER.contains(&Mode::Dx12) {
assert_eq!(GraphicsStart::begin_at(Some(&dir), false, Some(Mode::Dx12), NOW + 300, 75).mode(), Mode::Default);
}
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn reset_to_default_wins_over_a_later_confirm() {
let dir = temp_dir("reset");
let last = LADDER.last().copied().unwrap_or(Mode::Default);
let _ = write_atomic(&dir.join(STATE_FILE), &state_json(Saved { mode: last, failures: 0 }));
let start = launch(&dir, 80, NOW);
assert_eq!(start.mode(), last);
// the notice's button, from a clone of the start (the app holds one)
assert_eq!(start.clone().reset_to_default(), Ok(()));
assert_eq!(state_of(&dir), saved(Mode::Default, 0));
start.confirm();
assert_eq!(state_of(&dir), saved(Mode::Default, 0));
assert_eq!(launch(&dir, 81, NOW + 60).mode(), Mode::Default);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn an_explicit_backend_leaves_the_data_folder_alone() {
let dir = temp_dir("explicit");
let _ = write_atomic(&dir.join(STATE_FILE), &state_json(Saved { mode: Mode::Software, failures: 1 }));
let mut start = GraphicsStart::begin_at(Some(&dir), true, None, NOW, 90);
assert_eq!(start.mode(), Mode::Default);
start.confirm();
let _ = start.step_down();
assert_eq!(start.reset_to_default(), Ok(()));
assert_eq!(state_of(&dir), saved(Mode::Software, 1));
assert_eq!(attempt_files(&dir), 0);
// with --graphics too, that mode starts, still without touching the folder
let last = LADDER.last().copied().unwrap_or(Mode::Default);
assert_eq!(GraphicsStart::begin_at(Some(&dir), true, Some(last), NOW, 91).mode(), last);
assert_eq!(state_of(&dir), saved(Mode::Software, 1));
// no data folder: nothing is written, the default still starts
assert_eq!(GraphicsStart::begin_at(None, false, None, NOW, 92).mode(), Mode::Default);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn step_down_walks_the_ladder_and_saves_each_mode_at_once() {
let dir = temp_dir("step");
let mut start = launch(&dir, ME, NOW);
for expected in LADDER.iter().skip(1) {
assert!(start.step_down());
assert_eq!(start.mode(), *expected);
assert_eq!(state_of(&dir), saved(*expected, 0));
assert_eq!(attempt_files(&dir), 1, "one attempt file per process");
}
assert!(!start.step_down(), "nothing after the last mode");
start.finish();
assert_eq!(attempt_files(&dir), 0);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn the_software_selector_only_picks_a_software_adapter() {
if wgpu::Instance::enabled_backend_features().is_empty() {
return;
}
// only the backends software mode uses: no driver this test loads is one the app avoids
let backends = software_backends(cfg!(target_os = "windows"));
let instance = wgpu::Instance::new(wgpu::InstanceDescriptor { backends, ..wgpu::InstanceDescriptor::new_without_display_handle() });
let adapters = pollster::block_on(instance.enumerate_adapters(backends));
match pick_software(&adapters, None) {
Ok(adapter) => assert!(filmcraft_ui_egui::software_rasterizer(&adapter.get_info()).is_some()),
Err(why) => {
assert!(adapters.iter().all(|a| filmcraft_ui_egui::software_rasterizer(&a.get_info()).is_none()), "{why}");
}
}
assert!(pick_software(&[], None).is_err());
}
}

371
apps/filmcraft/src/main.rs
View File

@ -31,7 +31,6 @@ mod control_server;
#[cfg(target_os = "linux")]
mod dev_icon;
mod file_filters;
#[cfg(any(target_os = "windows", test))]
mod graphics;
mod logging;
#[cfg(target_os = "macos")]
@ -44,6 +43,8 @@ use filmcraft_engine::Session;
use filmcraft_engine::autosave::{AutosaveConfig, default_data_dir};
use filmcraft_ui_egui::FilmcraftApp;
use serde_json::json;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
/// Reverse-DNS app id: Linux `.desktop` file, hicolor icon name and Wayland app id.
const APP_ID: &str = "ai.storyteller.filmcraft";
@ -80,7 +81,7 @@ fn main() -> eframe::Result {
let logger = logging::install();
// lossy, so a value that is not Unicode is reported like any other bad port instead of ignored
let env_port = std::env::var_os("FILMCRAFT_CONTROL_PORT").map(|p| p.to_string_lossy().into_owned());
let Launch { control_port, control_input_only, files, demo, startup_flag, recover, data_dir } = match args::parse(env_port, std::env::args().skip(1)) {
let launch = match args::parse(env_port, std::env::args().skip(1)) {
Ok(Cli::Run(l)) => l,
Ok(Cli::Help) => {
tell(args::USAGE, false);
@ -95,7 +96,8 @@ fn main() -> eframe::Result {
std::process::exit(2);
}
};
let log_dir = data_dir.clone().or_else(default_data_dir).map(|d| d.join(logging::LOG_DIR));
let data_root = launch.data_dir.clone().or_else(default_data_dir);
let log_dir = data_root.as_ref().map(|d| d.join(logging::LOG_DIR));
log::info!("FilmCraft {} ({} {})", env!("CARGO_PKG_VERSION"), std::env::consts::OS, std::env::consts::ARCH);
// The log file lives next to the crash logs; opened after the arguments, so `--help`,
// `--version` and usage errors leave no file behind. Records logged until now go in first.
@ -118,165 +120,38 @@ fn main() -> eframe::Result {
// OS hardware video decoders (VideoToolbox on macOS) in front of our own; Settings ▸ Playback ▸
// Hardware decoding switches them off. Unsupported streams and failures use our decoders.
register_hardware_decoders();
register_gpu_frame_renderer();
let options = eframe::NativeOptions {
viewport: egui::ViewportBuilder::default()
.with_title("FilmCraft")
.with_inner_size([1600.0, 980.0])
.with_min_inner_size([900.0, 560.0])
.with_drag_and_drop(true)
.with_fullsize_content_view(true)
.with_titlebar_shown(false)
.with_title_shown(false)
// Started for an agent (`--control`): open without taking the user's keyboard focus.
.with_active(control_port.is_none())
.with_icon(app_icon())
// Wayland matches the window to `packaging/linux/ai.storyteller.filmcraft.desktop`.
.with_app_id(APP_ID),
event_loop_builder: agent_event_loop(control_port.is_some()),
..Default::default()
// Which graphics backends and adapter the window starts with: `--graphics`, else the saved
// mode, which moves on after starts that died before their window painted (a driver crash),
// see graphics.rs.
let backend_override = eframe::wgpu::Backends::from_env();
let mut graphics = graphics::GraphicsStart::begin(data_root.as_deref(), backend_override.is_some(), launch.graphics);
// The GPU export renderer creates its own wgpu instance with every backend; after a start
// fell back, that would load the driver that failed. Exports then composite on the CPU.
if graphics.mode() == graphics::Mode::Default {
register_gpu_frame_renderer();
}
let started = loop {
let mut options = native_options(launch.control_port.is_some());
// Before eframe creates the wgpu instance (e.g. no OpenGL on Windows, see graphics.rs).
graphics::apply(&mut options, graphics.mode(), backend_override);
let app_created = Arc::new(AtomicBool::new(false));
let result = eframe::run_native("FilmCraft", options, app_creator(launch.clone(), graphics.clone(), Arc::clone(&app_created)));
let app_created = app_created.load(Ordering::Relaxed);
match &result {
// wgpu reported the failure (no adapter, device creation failed) before the app
// existed: try the next mode in this process.
Err(e) if !app_created && graphics::is_graphics_failure(e) && graphics.step_down() => {
log::warn!("graphics start failed: {e}");
continue;
}
// Closed before the second frame: the window still came up.
Ok(()) if app_created => graphics.confirm(),
_ => {}
}
break result;
};
// Before eframe creates the wgpu instance: leave OpenGL out on Windows (see graphics.rs).
#[cfg(target_os = "windows")]
let options = {
let mut options = options;
graphics::configure(&mut options, eframe::wgpu::Backends::from_env());
options
};
let started = eframe::run_native(
"FilmCraft",
options,
Box::new(move |cc| {
let mut session = Session::default();
if let Some(dir) = data_dir.clone().or_else(default_data_dir) {
let mut cfg = AutosaveConfig::new(dir);
cfg.local_offset = local_offset;
if let Err(e) = session.start_autosave(cfg) {
log::warn!("auto-save and crash recovery unavailable: {e}");
}
}
// voice-over recording reads the microphone through cpal
session.voiceover.input = Some(Box::new(audio_in::CpalIn::new(&session.prefs.audio_hardware.device_class)));
let project = files.iter().find(|f| f.ends_with(".fcproj")).cloned();
if let Some(p) = project {
if let Err(e) = session.execute("file.open", json!({"path": p})) {
log::error!("{e}");
}
} else if !startup_flag && session.prefs.general.at_startup == "openMostRecent" {
// Settings ▸ General ▸ At Startup ▸ Open Most Recent
let recent = session.prefs.general.recent_projects.iter().find(|p| std::path::Path::new(p).exists()).cloned();
match recent {
Some(p) => {
if let Err(e) = session.execute("file.open", json!({"path": p})) {
log::error!("{e}");
}
}
None => {
let _ = session.execute("file.openDemoProject", json!({}));
}
}
} else if !startup_flag && session.prefs.general.at_startup == "emptyProject" {
} else if demo {
let _ = session.execute("file.openDemoProject", json!({}));
}
let media: Vec<String> = files.iter().filter(|f| !f.ends_with(".fcproj")).cloned().collect();
if !media.is_empty() {
let _ = session.execute("file.import", json!({"paths": media}));
}
if recover == Some(true) && !session.recovery_candidates().is_empty() {
let id = session.recovery_candidates()[0].id.clone();
match session.execute("file.recover", json!({"id": id})) {
Ok(r) => log::info!("recovered {r}"),
Err(e) => log::error!("recovery failed: {e}"),
}
}
let mut app = FilmcraftApp::new(session);
if recover == Some(false) {
app.dialog = None;
}
app.integrated_titlebar = cfg!(target_os = "macos");
if let Some(rs) = cc.wgpu_render_state.clone()
&& std::env::var_os("FILMCRAFT_CPU_COMPOSITE").is_none()
{
app.set_wgpu(rs);
}
// Keep device selection available even when the default device is unavailable.
// Settings ▸ Audio Hardware is applied on the first frame (`apply_prefs`).
app.audio = Some(Box::new(audio::CpalOut::new()));
app.hooks.pick_files = Some(Box::new(|exts: &[&str]| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("Media"), &file_filters::extensions(exts))
.pick_files()
.unwrap_or_default()
.into_iter()
.map(|p| p.to_string_lossy().to_string())
.collect()
}));
// Link Media ▸ Locate…, Attach Proxies, Reconnect Full Resolution: one path, not imported.
app.hooks.pick_file_for_relink = Some(Box::new(|exts: &[&str], _hint| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("Media"), &file_filters::extensions(exts))
.pick_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_save = Some(Box::new(|name: &str| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("FilmCraft Project"), &file_filters::extensions(&["fcproj"]))
.set_file_name(name)
.save_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_save_as = Some(Box::new(|filter: &str, exts: &[&str], name: &str| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t(filter), &file_filters::extensions(exts))
.set_file_name(name)
.save_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_folder = Some(Box::new(|| rfd::FileDialog::new().pick_folder().map(|p| p.to_string_lossy().to_string())));
app.hooks.pick_folder_at =
Some(Box::new(|dir: &str| rfd::FileDialog::new().set_directory(dir).pick_folder().map(|p| p.to_string_lossy().into_owned())));
app.hooks.open_path = Some(Box::new(open_path));
// Help menu, About and Discord links (#642): eframe is built without its `links`
// feature, so `ctx.open_url` does nothing on the desktop. Detached, so a launcher
// that waits for the browser never freezes the UI.
app.hooks.open_url = Some(Box::new(|url: &str| open::that_detached(url).map_err(|e| e.to_string())));
app.hooks.clipboard_media = Some(Box::new(clipboard_media));
// Settings ▸ General ▸ Interface Language ▸ System Language (#218).
app.hooks.cursor_screen_position = Some(Box::new(filmcraft_platform::cursor::cursor_screen_position));
app.hooks.system_languages = Some(Box::new(|| sys_locale::get_locales().collect()));
// Settings ▸ Appearance ▸ Appearance Mode ▸ Sync with system on Linux desktops whose
// compositor reports no theme to winit (no polling: see appearance.rs).
app.hooks.system_theme = appearance::service();
app.hooks.raise_without_focus = Some(Box::new(|| {
window_raise::raise_without_focus();
}));
app.hooks.pick_open_file = Some(Box::new(|filter: &str, exts: &[&str]| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t(filter), &file_filters::extensions(exts))
.pick_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_open_project = Some(Box::new(|| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("FilmCraft Project"), &file_filters::extensions(&["fcproj"]))
.pick_file()
.map(|p| p.to_string_lossy().to_string())
}));
#[cfg(target_os = "macos")]
{
let (rx, update) = native_menu::install(&app, cc.egui_ctx.clone());
app.command_inbox = Some(rx);
app.hooks.shortcuts_changed = Some(update);
app.ui.show_menu_bar = false;
}
if let Some(port) = control_port {
let rx = control_server::start(port, cc.egui_ctx.clone());
app = if control_input_only { app.with_control_input_only(rx) } else { app.with_control(rx) };
}
Ok(Box::new(app))
}),
);
// However it ended, this start no longer counts as one that died.
graphics.finish();
// The window could not be created (no usable GPU / graphics driver, no display): say so
// instead of exiting silently, and keep the reason in the crash log.
if let Err(e) = &started {
@ -288,6 +163,179 @@ fn main() -> eframe::Result {
started
}
/// The main window's options; built for each start attempt (they hold a one-shot hook).
fn native_options(agent: bool) -> eframe::NativeOptions {
eframe::NativeOptions {
viewport: egui::ViewportBuilder::default()
.with_title("FilmCraft")
.with_inner_size([1600.0, 980.0])
.with_min_inner_size([900.0, 560.0])
.with_drag_and_drop(true)
.with_fullsize_content_view(true)
.with_titlebar_shown(false)
.with_title_shown(false)
// Started for an agent (`--control`): open without taking the user's keyboard focus.
.with_active(!agent)
.with_icon(app_icon())
// Wayland matches the window to `packaging/linux/ai.storyteller.filmcraft.desktop`.
.with_app_id(APP_ID),
event_loop_builder: agent_event_loop(agent),
..Default::default()
}
}
/// Creates the app once eframe has its window and graphics device; sets `created` when it runs,
/// and confirms the graphics mode once two frames have painted.
fn app_creator(launch: Launch, graphics: graphics::GraphicsStart, created: Arc<AtomicBool>) -> eframe::AppCreator<'static> {
let Launch { control_port, control_input_only, files, demo, startup_flag, recover, data_dir, graphics: _ } = launch;
Box::new(move |cc| {
created.store(true, Ordering::Relaxed);
// A fallback graphics mode says so in the window, with a button back to the default.
let notice = graphics.degraded().then(|| {
let start = graphics.clone();
filmcraft_ui_egui::graphics_notice::GraphicsNotice::new(graphics.mode().label(), Box::new(move || start.reset_to_default()))
});
let confirmed = AtomicBool::new(false);
cc.egui_ctx.on_end_pass(
"graphics-start",
Arc::new(move |ui| {
if ui.ctx().cumulative_frame_nr() >= 2 && !confirmed.swap(true, Ordering::Relaxed) {
graphics.confirm();
}
}),
);
let mut session = Session::default();
if let Some(dir) = data_dir.clone().or_else(default_data_dir) {
let mut cfg = AutosaveConfig::new(dir);
cfg.local_offset = local_offset;
if let Err(e) = session.start_autosave(cfg) {
log::warn!("auto-save and crash recovery unavailable: {e}");
}
}
// voice-over recording reads the microphone through cpal
session.voiceover.input = Some(Box::new(audio_in::CpalIn::new(&session.prefs.audio_hardware.device_class)));
let project = files.iter().find(|f| f.ends_with(".fcproj")).cloned();
if let Some(p) = project {
if let Err(e) = session.execute("file.open", json!({"path": p})) {
log::error!("{e}");
}
} else if !startup_flag && session.prefs.general.at_startup == "openMostRecent" {
// Settings ▸ General ▸ At Startup ▸ Open Most Recent
let recent = session.prefs.general.recent_projects.iter().find(|p| std::path::Path::new(p).exists()).cloned();
match recent {
Some(p) => {
if let Err(e) = session.execute("file.open", json!({"path": p})) {
log::error!("{e}");
}
}
None => {
let _ = session.execute("file.openDemoProject", json!({}));
}
}
} else if !startup_flag && session.prefs.general.at_startup == "emptyProject" {
} else if demo {
let _ = session.execute("file.openDemoProject", json!({}));
}
let media: Vec<String> = files.iter().filter(|f| !f.ends_with(".fcproj")).cloned().collect();
if !media.is_empty() {
let _ = session.execute("file.import", json!({"paths": media}));
}
if recover == Some(true) && !session.recovery_candidates().is_empty() {
let id = session.recovery_candidates()[0].id.clone();
match session.execute("file.recover", json!({"id": id})) {
Ok(r) => log::info!("recovered {r}"),
Err(e) => log::error!("recovery failed: {e}"),
}
}
let mut app = FilmcraftApp::new(session);
app.graphics_notice = notice;
if recover == Some(false) {
app.dialog = None;
}
app.integrated_titlebar = cfg!(target_os = "macos");
if let Some(rs) = cc.wgpu_render_state.clone()
&& std::env::var_os("FILMCRAFT_CPU_COMPOSITE").is_none()
{
app.set_wgpu(rs);
}
// Keep device selection available even when the default device is unavailable.
// Settings ▸ Audio Hardware is applied on the first frame (`apply_prefs`).
app.audio = Some(Box::new(audio::CpalOut::new()));
app.hooks.pick_files = Some(Box::new(|exts: &[&str]| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("Media"), &file_filters::extensions(exts))
.pick_files()
.unwrap_or_default()
.into_iter()
.map(|p| p.to_string_lossy().to_string())
.collect()
}));
// Link Media ▸ Locate…, Attach Proxies, Reconnect Full Resolution: one path, not imported.
app.hooks.pick_file_for_relink = Some(Box::new(|exts: &[&str], _hint| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("Media"), &file_filters::extensions(exts))
.pick_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_save = Some(Box::new(|name: &str| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("FilmCraft Project"), &file_filters::extensions(&["fcproj"]))
.set_file_name(name)
.save_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_save_as = Some(Box::new(|filter: &str, exts: &[&str], name: &str| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t(filter), &file_filters::extensions(exts))
.set_file_name(name)
.save_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_folder = Some(Box::new(|| rfd::FileDialog::new().pick_folder().map(|p| p.to_string_lossy().to_string())));
app.hooks.pick_folder_at =
Some(Box::new(|dir: &str| rfd::FileDialog::new().set_directory(dir).pick_folder().map(|p| p.to_string_lossy().into_owned())));
app.hooks.open_path = Some(Box::new(open_path));
// Help menu, About and Discord links (#642): eframe is built without its `links`
// feature, so `ctx.open_url` does nothing on the desktop. Detached, so a launcher
// that waits for the browser never freezes the UI.
app.hooks.open_url = Some(Box::new(|url: &str| open::that_detached(url).map_err(|e| e.to_string())));
app.hooks.clipboard_media = Some(Box::new(clipboard_media));
// Settings ▸ General ▸ Interface Language ▸ System Language (#218).
app.hooks.cursor_screen_position = Some(Box::new(filmcraft_platform::cursor::cursor_screen_position));
app.hooks.system_languages = Some(Box::new(|| sys_locale::get_locales().collect()));
// Settings ▸ Appearance ▸ Appearance Mode ▸ Sync with system on Linux desktops whose
// compositor reports no theme to winit (no polling: see appearance.rs).
app.hooks.system_theme = appearance::service();
app.hooks.raise_without_focus = Some(Box::new(|| {
window_raise::raise_without_focus();
}));
app.hooks.pick_open_file = Some(Box::new(|filter: &str, exts: &[&str]| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t(filter), &file_filters::extensions(exts))
.pick_file()
.map(|p| p.to_string_lossy().to_string())
}));
app.hooks.pick_open_project = Some(Box::new(|| {
rfd::FileDialog::new()
.add_filter(filmcraft_ui_egui::i18n::t("FilmCraft Project"), &file_filters::extensions(&["fcproj"]))
.pick_file()
.map(|p| p.to_string_lossy().to_string())
}));
#[cfg(target_os = "macos")]
{
let (rx, update) = native_menu::install(&app, cc.egui_ctx.clone());
app.command_inbox = Some(rx);
app.hooks.shortcuts_changed = Some(update);
app.ui.show_menu_bar = false;
}
if let Some(port) = control_port {
let rx = control_server::start(port, cc.egui_ctx.clone());
app = if control_input_only { app.with_control_input_only(rx) } else { app.with_control(rx) };
}
Ok(Box::new(app))
})
}
/// Files copied in the file manager, else an image, on the system clipboard (Paste, #611).
fn clipboard_media() -> Option<filmcraft_ui_egui::ClipboardMedia> {
let mut clipboard = arboard::Clipboard::new().ok()?;
@ -319,6 +367,7 @@ fn open_path(path: &str, reveal: bool) -> Result<(), String> {
}
c
} else {
// Linux file managers have no select-file flag: open the parent folder instead.
let mut c = std::process::Command::new("xdg-open");
let p = std::path::Path::new(path);
c.arg(if reveal { p.parent().unwrap_or(p) } else { p });

35
apps/filmcraft/src/native_menu.rs
View File

@ -4,7 +4,7 @@
use std::sync::mpsc::{Receiver, channel};
use filmcraft_ui_egui::FilmcraftApp;
use filmcraft_ui_egui::menus::{MENUS, MenuItem as Item, menu_items};
use filmcraft_ui_egui::menus::{MENUS, MenuItem as Item, NATIVE_APP_MENU, menu_items};
use muda::accelerator::Accelerator;
use muda::{Menu, MenuEvent, MenuItem, PredefinedMenuItem, Submenu};
@ -28,25 +28,39 @@ pub fn install(app: &FilmcraftApp, ctx: egui::Context) -> (Receiver<String>, Sho
let mut titles: Vec<(String, Submenu)> = Vec::new();
let bar = Menu::new();
let app_menu = Submenu::new("FilmCraft", true);
let [settings_label, hide_label, hide_others_label, quit_label] = NATIVE_APP_MENU;
// FilmCraft ▸ Settings ▸ <category> (Premiere's app-menu layout; General is Cmd+,)
let settings = Submenu::new(app.ui.language.tr("Settings"), true);
titles.push(("Settings".into(), settings.clone()));
let settings = Submenu::new(app.ui.language.tr(settings_label), true);
titles.push((settings_label.into(), settings.clone()));
for it in items.iter().filter(|i| i.id.starts_with("app.settings.")) {
let label = it.label.trim_end_matches('…').to_string();
let mi = MenuItem::with_id(it.id.clone(), label, true, it.shortcut.as_deref().and_then(accel));
native.push((it.id.clone(), mi.clone()));
let _ = settings.append(&mi);
}
// About and Discord repeat Help menu commands: same translated labels, updated with them
let label = |id: &str, english: &str| items.iter().find(|i| i.id == id).map_or_else(|| english.to_string(), |i| i.label.clone());
let about = MenuItem::with_id("app.about", label("app.about", "About FilmCraft"), true, None);
let discord = MenuItem::with_id("help.discord", label("help.discord", "Join the ArtCraft Discord…"), true, None);
native.push(("app.about".into(), about.clone()));
native.push(("help.discord".into(), discord.clone()));
// the predefined items keep their native actions; our text lets them follow the language
let predefined = [
(hide_label, PredefinedMenuItem::hide(Some(app.ui.language.tr(hide_label)))),
(hide_others_label, PredefinedMenuItem::hide_others(Some(app.ui.language.tr(hide_others_label)))),
(quit_label, PredefinedMenuItem::quit(Some(app.ui.language.tr(quit_label)))),
];
let [(_, hide), (_, hide_others), (_, quit)] = &predefined;
let _ = app_menu.append_items(&[
&MenuItem::with_id("app.about", "About FilmCraft", true, None),
&MenuItem::with_id("help.discord", "Join the ArtCraft Discord…", true, None),
&about,
&discord,
&PredefinedMenuItem::separator(),
&settings,
&PredefinedMenuItem::separator(),
&PredefinedMenuItem::hide(None),
&PredefinedMenuItem::hide_others(None),
hide,
hide_others,
&PredefinedMenuItem::separator(),
&PredefinedMenuItem::quit(None),
quit,
]);
let _ = bar.append(&app_menu);
for top in MENUS {
@ -109,12 +123,17 @@ pub fn install(app: &FilmcraftApp, ctx: egui::Context) -> (Receiver<String>, Sho
filmcraft_ui_egui::i18n::Language::Ru
} else if checked("app.language.french") {
filmcraft_ui_egui::i18n::Language::Fr
} else if checked("app.language.indonesian") {
filmcraft_ui_egui::i18n::Language::Id
} else {
filmcraft_ui_egui::i18n::Language::En
};
for (title, submenu) in &titles {
submenu.set_text(language.tr(title));
}
for (label, item) in &predefined {
item.set_text(language.tr(label));
}
for (id, mi) in &native {
if let Some(it) = items.iter().find(|i| &i.id == id) {
mi.set_text(&it.label);

3
contributors/people.toml
View File

@ -70,6 +70,9 @@ sync_github_name = false
[people.ossini]
sync_github_name = true
[people.opentechnepal]
sync_github_name = true
[people.vangel76]
real_name = "Valentin Klein"
display_name = "Valentin Klein"

30
crates/automation/src/bridge.rs
View File

@ -47,14 +47,14 @@ impl BridgeClient {
}
let mut start = self.call("engine.execute", json!({"command": id, "params": params})).await?;
let Some(job) = start.get("job").and_then(Value::as_u64) else { return Ok(start) };
let result = loop {
let (result, offline) = loop {
tokio::time::sleep(JOB_POLL).await;
let jobs = self.call("engine.execute", json!({"command": "jobs.list", "params": {}})).await?;
let Some(j) = jobs.as_array().and_then(|a| a.iter().find(|j| j["id"].as_u64() == Some(job))).cloned() else {
break Value::Null;
break (Value::Null, None);
};
if j["finished"].as_bool() == Some(true) {
break j["result"].clone();
break (j["result"].clone(), j.get("offlineMedia").cloned());
}
};
if let Some(e) = result.get("error").and_then(Value::as_str) {
@ -62,6 +62,9 @@ impl BridgeClient {
}
if let Some(o) = start.as_object_mut() {
o.insert("result".into(), result);
if let Some(off) = offline {
o.insert("offlineMedia".into(), off);
}
}
Ok(start)
}
@ -121,6 +124,11 @@ mod tests {
/// A stand-in for the app's control server: the export starts job 7, which `jobs.list` reports
/// running twice and then finished with `result`. Every request is recorded.
async fn fake_app(result: Value) -> (String, Arc<Mutex<Vec<Value>>>) {
fake_app_with(result, None).await
}
/// [`fake_app`] whose finished job also lists `offline` as its `offlineMedia`.
async fn fake_app_with(result: Value, offline: Option<Value>) -> (String, Arc<Mutex<Vec<Value>>>) {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap().to_string();
let seen = Arc::new(Mutex::new(Vec::new()));
@ -138,7 +146,11 @@ mod tests {
Some("jobs.list") => {
polls += 1;
let finished = polls >= 3;
json!([{"id": 7, "finished": finished, "result": if finished { result.clone() } else { Value::Null }}])
let mut j = json!({"id": 7, "finished": finished, "result": if finished { result.clone() } else { Value::Null }});
if let Some(o) = offline.clone().filter(|_| finished) {
j["offlineMedia"] = o;
}
json!([j])
}
_ => json!(null),
};
@ -165,6 +177,16 @@ mod tests {
assert_eq!(seen.iter().filter(|q| q["params"]["command"] == "jobs.list").count(), 3);
}
/// The offline media a polled export drew comes back as in a blocking export's reply (#1054).
#[tokio::test]
async fn a_polled_export_reports_its_offline_media() {
let off = json!([{"item": 8, "status": "missing"}]);
let (addr, _) = fake_app_with(json!({"frames": 3}), Some(off.clone())).await;
let b = BridgeClient::new(addr).unwrap();
let r = b.execute("file.exportMedia", json!({"path": "/tmp/out.mov", "wait": true})).await.unwrap();
assert_eq!(r["offlineMedia"], off, "{r}");
}
#[tokio::test]
async fn a_failed_blocking_export_is_an_error() {
let (addr, _) = fake_app(json!({"error": "encode: disk full"})).await;

4
crates/automation/src/long_job.rs
View File

@ -72,6 +72,7 @@ impl FilmcraftMcp {
rep.report(done, total, j["status"].as_str().unwrap_or_default()).await;
}
if finished {
let offline = state.as_ref().and_then(|j| j.get("offlineMedia").cloned());
let result = state.map(|j| j["result"].clone()).unwrap_or(Value::Null);
if let Some(e) = result.get("error").and_then(Value::as_str) {
return CallToolResult::error(vec![Content::text(format!("export failed: {e}"))]);
@ -79,6 +80,9 @@ impl FilmcraftMcp {
let mut out = start;
if let Some(o) = out.as_object_mut() {
o.insert("result".into(), result);
if let Some(off) = offline {
o.insert("offlineMedia".into(), off);
}
}
return CallToolResult::success(vec![Content::text(serde_json::to_string_pretty(&out).unwrap_or_default())]);
}

9
crates/av1/README.md
View File

@ -70,12 +70,15 @@ with libdav1d: **bit-exact** on all 6 fixtures.
wasm32, or without the default `threads` feature, everything runs on the calling thread). Headers are parsed in order; each frame is
then decoded by a frame worker once the reference frames it reads are finished, so independent
frames (hierarchical GOP layers) decode concurrently; tiles of a frame decode in parallel into
private buffers that are merged afterwards. Results never depend on scheduling (the oracle tests
compare 1 thread, many threads and libdav1d). With frame threads, pictures may be returned by a
private buffers that are merged afterwards; the in-loop filters (deblocking, CDEF, loop
restoration) each run in parallel bands of 64 rows (`postfilter.rs` explains why the bands are
independent). Results never depend on scheduling (the oracle tests compare 1 thread, many threads
and libdav1d). With frame threads, pictures may be returned by a
later `decode_pts` call or by `flush`; each carries the `pts` of the temporal unit that showed it.
`tests/perf.rs` (ignored) prints single-threaded fps (thread CPU time), per-stage ms/frame
(`Decoder::stats()`) and all-core fps for 1080p fixtures. The hot loops (sub-pixel filters,
(`Decoder::stats()`) and all-core fps with its per-stage ms/frame for 1080p fixtures (key frames,
GOPs, tiles, low delay) and a single-tile 2160p GOP; `AV1_PERF_ONLY=a,b` picks fixtures. The hot loops (sub-pixel filters,
CDEF, inverse transforms, deblocking, entropy decoding) are written for auto-vectorisation.
Measured 2026-10-01 on a 14-core Apple Silicon Mac (release build, SVT-AV1 preset 8 1080p

53
crates/av1/src/cdef.rs
View File

@ -1,35 +1,45 @@
//! CDEF (7.15).
use crate::frame::{FrameBuf, Plane};
use crate::frame::{FrameBuf, RowBand};
use crate::par::Pool;
use crate::postfilter::bands;
use crate::spec_tables::*;
use crate::state::FrameState;
/// Run CDEF on `fs.cur`; returns the CdefFrame, or None when no block is filtered (CdefFrame
/// equals CurrFrame).
pub(crate) fn apply(fs: &FrameState) -> Option<FrameBuf> {
/// equals CurrFrame). Bands of 64 luma rows (whole 8x8 blocks) are filtered in parallel: every
/// block reads only CurrFrame and writes only its own samples.
pub(crate) fn apply(fs: &FrameState, pool: &Pool) -> Option<FrameBuf> {
let fh = &fs.fh;
if fh.coded_lossless || fh.allow_intrabc || !fs.seq.enable_cdef {
return None;
}
let (mi_rows, mi_cols) = (fh.mi_rows as usize, fh.mi_cols as usize);
let mut out = fs.cur.clone();
let mut any = false;
let mut r = 0;
while r < mi_rows {
let mut c = 0;
while c < mi_cols {
let idx = fs.cdef_idx(r & !15, c & !15);
if idx >= 0 && cdef_block(fs, &mut out, r, c, idx as usize) {
any = true;
let ranges = bands(pool, mi_rows * 4);
let mut planes = out.row_bands(&ranges);
let any = pool.map_mut(&mut planes, |b, band| {
let (start, end) = ranges[b];
let mut any = false;
let mut r = start / 4;
while r < mi_rows.min(end / 4) {
let mut c = 0;
while c < mi_cols {
let idx = fs.cdef_idx(r & !15, c & !15);
if idx >= 0 && cdef_block(fs, band, r, c, idx as usize) {
any = true;
}
c += 2;
}
c += 2;
r += 2;
}
r += 2;
}
any.then_some(out)
any
});
drop(planes);
any.contains(&true).then_some(out)
}
fn cdef_block(fs: &FrameState, out: &mut FrameBuf, r: usize, c: usize, idx: usize) -> bool {
fn cdef_block(fs: &FrameState, out: &mut [RowBand; 3], r: usize, c: usize, idx: usize) -> bool {
let mi = &fs.mi;
let sk = |rr: usize, cc: usize| -> bool { if rr < mi.rows && cc < mi.cols { mi.skip[mi.idx(rr, cc)] } else { true } };
let skip = sk(r, c) && sk(r + 1, c) && sk(r, c + 1) && sk(r + 1, c + 1);
@ -125,7 +135,7 @@ fn constrain(diff: i32, threshold: i32, adj: u32) -> i32 {
}
#[allow(clippy::too_many_arguments)]
fn filter(fs: &FrameState, out: &mut FrameBuf, plane: usize, r: usize, c: usize, pri: i32, sec: i32, damping: i32, dir: usize) {
fn filter(fs: &FrameState, out: &mut [RowBand; 3], plane: usize, r: usize, c: usize, pri: i32, sec: i32, damping: i32, dir: usize) {
if pri == 0 && sec == 0 {
// Every tap contributes 0 and the clamp to [min, max] keeps x: CdefFrame = CurrFrame.
return;
@ -169,7 +179,7 @@ fn filter(fs: &FrameState, out: &mut FrameBuf, plane: usize, r: usize, c: usize,
let sec_taps = [CDEF_SEC_TAPS[pri_tap][0] as i32, CDEF_SEC_TAPS[pri_tap][1] as i32];
let adj = |t: i32| if t == 0 { 0 } else { (damping - (31 - (t as u32).leading_zeros()) as i32).max(0) as u32 };
let (pri_adj, sec_adj) = (adj(pri), adj(sec));
let dst = &mut out.planes[plane];
let dst = &mut out[plane];
if all_avail {
// Every tap is available: whole rows at a time (constrain() with threshold 0 is 0).
let mut taps = [(0isize, 0i32, 0i32, 0u32); 12];
@ -230,7 +240,12 @@ fn filter(fs: &FrameState, out: &mut FrameBuf, plane: usize, r: usize, c: usize,
/// CDEF of a block whose taps are all available, `W` samples per row. Samples (at most 12
/// bits), differences and the weighted tap sum all fit in i16, which doubles the SIMD width.
#[inline(always)]
fn filter_rows<const W: usize>(buf: &[i32; PAD_STRIDE * PAD_STRIDE], taps: &[(isize, i32, i32, u32); 12], dst: &mut Plane, (y0, x0, h): (usize, usize, usize)) {
fn filter_rows<const W: usize>(
buf: &[i32; PAD_STRIDE * PAD_STRIDE],
taps: &[(isize, i32, i32, u32); 12],
dst: &mut RowBand,
(y0, x0, h): (usize, usize, usize),
) {
let mut b16 = [0i16; PAD_STRIDE * PAD_STRIDE];
for (d, &s) in b16.iter_mut().zip(buf.iter()) {
*d = s as i16;

89
crates/av1/src/frame.rs
View File

@ -106,6 +106,48 @@ impl Plane {
dst.row_from_mut(self.oy + y, self.ox)[..self.stride].copy_from_slice(s);
}
}
/// Disjoint mutable bands of whole rows, for filtering in parallel: one per `[start, end)`
/// range of rows (relative to `oy`; ascending and not overlapping, else the later band is
/// cut to what is left). Ranges are clipped to the stored rows.
pub fn row_bands(&mut self, ranges: &[(usize, usize)]) -> Vec<RowBand<'_>> {
let (stride, oy) = (self.stride.max(1), self.oy);
let mut out = Vec::with_capacity(ranges.len());
let mut rest = self.data.as_mut_slice();
// First row of `rest`.
let mut y = 0usize;
for &(start, end) in ranges {
let skip = start.saturating_sub(y).saturating_mul(stride).min(rest.len());
rest = std::mem::take(&mut rest).split_at_mut(skip).1;
y = y.saturating_add(skip / stride);
let len = end.saturating_sub(y).saturating_mul(stride).min(rest.len());
let (band, tail) = std::mem::take(&mut rest).split_at_mut(len);
out.push(RowBand { data: band, stride, y0: oy.saturating_add(y) });
y = y.saturating_add(len / stride);
rest = tail;
}
out
}
}
/// Mutable whole rows [y0, y0 + data.len() / stride) of a plane, addressed with the plane's
/// coordinates (see [`Plane::row_bands`]).
pub struct RowBand<'a> {
pub data: &'a mut [u16],
pub stride: usize,
pub y0: usize,
}
impl RowBand<'_> {
#[inline(always)]
pub fn set(&mut self, x: usize, y: usize, v: u16) {
self.data[(y - self.y0) * self.stride + x] = v;
}
#[inline(always)]
pub fn row_mut(&mut self, y: usize) -> &mut [u16] {
let o = (y - self.y0) * self.stride;
&mut self.data[o..o + self.stride]
}
}
/// A frame's three planes, sized to the macroblock-aligned coded area plus margins.
@ -158,6 +200,16 @@ impl FrameBuf {
}
}
/// [`Plane::row_bands`] of all three planes for ranges of luma rows (chroma rows scaled
/// down): element `i` holds the bands of range `i`.
pub fn row_bands(&mut self, luma: &[(usize, usize)]) -> Vec<[RowBand<'_>; 3]> {
let ssy = self.subsampling_y;
let chroma: Vec<(usize, usize)> = luma.iter().map(|&(s, e)| (s >> ssy, e >> ssy)).collect();
let [p0, p1, p2] = &mut self.planes;
let (b1, b2) = (p1.row_bands(&chroma), p2.row_bands(&chroma));
p0.row_bands(luma).into_iter().zip(b1).zip(b2).map(|((y, u), v)| [y, u, v]).collect()
}
pub fn plane_width(&self, plane: usize) -> usize {
if plane == 0 { self.width } else { (self.width + self.subsampling_x) >> self.subsampling_x }
}
@ -285,3 +337,40 @@ impl MiInfo {
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn rows(b: &RowBand) -> (usize, usize) {
(b.y0, b.data.len() / b.stride)
}
#[test]
fn row_bands_split_whole_rows() {
let mut p = Plane::new(8, 20);
for (i, v) in p.data.iter_mut().enumerate() {
*v = (i / 8) as u16;
}
let bands = p.row_bands(&[(0, 8), (8, 16), (16, usize::MAX)]);
assert_eq!(bands.iter().map(rows).collect::<Vec<_>>(), [(0, 8), (8, 8), (16, 4)]);
// Each band's row y holds the samples of plane row y.
for mut b in bands {
let y = b.y0 + 1;
assert!(b.row_mut(y).iter().all(|&v| v as usize == y));
}
// Gaps between bands (the loop filter's even / odd turns).
let bands = p.row_bands(&[(0, 4), (10, 14), (18, 30)]);
assert_eq!(bands.iter().map(rows).collect::<Vec<_>>(), [(0, 4), (10, 4), (18, 2)]);
}
#[test]
fn row_bands_never_overlap_or_panic() {
let mut p = Plane::new(4, 10);
// Overlapping, descending and out-of-range ranges are cut to what is left.
let bands = p.row_bands(&[(2, 6), (4, 8), (1, 3), (9, 2), (50, 60), (usize::MAX, usize::MAX)]);
assert_eq!(bands.iter().map(rows).collect::<Vec<_>>(), [(2, 4), (6, 2), (8, 0), (9, 0), (10, 0), (10, 0)]);
let mut empty = Plane::default();
assert!(empty.row_bands(&[(0, 64), (64, usize::MAX)]).iter().all(|b| b.data.is_empty()));
}
}

165
crates/av1/src/postfilter.rs
View File

@ -1,20 +1,33 @@
//! In-loop post filters run by decode_frame_wrapup: loop filter (7.14), CDEF (7.15),
//! super-resolution upscaling (7.16) and loop restoration (7.17).
use crate::frame::RowBand;
use crate::spec_tables::*;
use crate::state::FrameState;
use crate::stats::{DecodeStats, Stage, Timer};
/// The post-filters run on the frame worker; `_pool` is for row-parallel filtering (not yet used).
/// `draft`: only super-resolution (which sets the output size) runs.
pub(crate) fn apply(fs: &mut FrameState, stats: &mut DecodeStats, _pool: &crate::par::Pool, draft: bool) {
/// Luma rows per band of the row-parallel filters: whole 64x64 CDEF filter blocks, and no
/// transform block crosses a band edge.
const BAND_ROWS: usize = 64;
/// The [start, end) luma rows of the bands covering `rows`: [`BAND_ROWS`] each (the last one
/// runs to the end of the plane), or one band for everything when `pool` is sequential.
pub(crate) fn bands(pool: &crate::par::Pool, rows: usize) -> Vec<(usize, usize)> {
let n = if pool.is_parallel() { rows.div_ceil(BAND_ROWS).max(1) } else { 1 };
(0..n).map(|b| (b * BAND_ROWS, if b + 1 == n { usize::MAX } else { (b + 1) * BAND_ROWS })).collect()
}
/// The post-filters run on the frame worker, each in row bands on `pool` (the result does not
/// depend on how the bands are scheduled). `draft`: only super-resolution (which sets the
/// output size) runs.
pub(crate) fn apply(fs: &mut FrameState, stats: &mut DecodeStats, pool: &crate::par::Pool, draft: bool) {
let mut t = Timer::start();
let lvl = fs.fh.lf.level;
if (lvl[0] != 0 || lvl[1] != 0) && !draft {
loop_filter(fs);
loop_filter(fs, pool);
}
stats.add(Stage::LoopFilter, t.lap());
let cdef = if draft { None } else { crate::cdef::apply(fs) };
let cdef = if draft { None } else { crate::cdef::apply(fs, pool) };
stats.add(Stage::Cdef, t.lap());
let (up_cur, up_cdef) = if fs.fh.use_superres {
let c = crate::restoration::upscale(fs, &fs.cur);
@ -25,7 +38,7 @@ pub(crate) fn apply(fs: &mut FrameState, stats: &mut DecodeStats, _pool: &crate:
};
stats.add(Stage::Superres, t.lap());
fs.cur = if fs.fh.lr.uses_lr && !draft {
crate::restoration::loop_restoration(fs, &up_cur, up_cdef.as_ref().unwrap_or(&up_cur))
crate::restoration::loop_restoration(fs, &up_cur, up_cdef.as_ref().unwrap_or(&up_cur), pool)
} else {
up_cdef.unwrap_or(up_cur)
};
@ -35,28 +48,102 @@ pub(crate) fn apply(fs: &mut FrameState, stats: &mut DecodeStats, _pool: &crate:
// ---------------------------------------------------------------------------------------------
// Loop filter (7.14)
fn loop_filter(fs: &mut FrameState) {
let (mi_rows, mi_cols) = (fs.fh.mi_rows as usize, fs.fh.mi_cols as usize);
for plane in 0..fs.num_planes {
if plane == 0 || fs.fh.lf.level[1 + plane] != 0 {
for pass in 0..2 {
let row_step = if plane == 0 { 1 } else { 1 << fs.ssy };
let col_step = if plane == 0 { 1 } else { 1 << fs.ssx };
let mut row = 0;
while row < mi_rows {
let mut col = 0;
while col < mi_cols {
edge(fs, plane, pass, row, col);
col += col_step;
}
row += row_step;
/// Luma rows above a band that its horizontal edges read or write (p6 .. p0 of the widest
/// filter, rounded up to whole 4x4 rows); chroma: `LF_MARGIN >> subsampling_y`.
const LF_MARGIN: usize = 8;
/// The loop filter in row bands, all of a frame's in one parallel pass (each join costs a
/// worker wake-up, which is slow on a busy machine).
///
/// Vertical edges (pass 0) only touch the 4 rows of their own edge. A horizontal edge (pass 1)
/// reads and writes at most `filter_size / 2` samples on each side, never more than the
/// transform blocks on either side are high, so no two edges touch the same sample and
/// any order gives the spec's raster-order result. Transform blocks never cross a band edge, so
/// the only edges reaching outside a band are those on its top row, whose p side lies in the
/// band above. Each band therefore also filters the vertical edges of a copy of the
/// `LF_MARGIN` rows above it (the same work as the band above does on them), runs its top
/// horizontal edges against that copy, and afterwards the samples those edges changed are
/// written back; the band above never touches them.
fn loop_filter(fs: &mut FrameState, pool: &crate::par::Pool) {
let luma = bands(pool, fs.fh.mi_rows as usize * 4);
let planes: Vec<usize> = (0..fs.num_planes).filter(|&p| p == 0 || fs.fh.lf.level[1 + p] != 0).collect();
let mut cur = std::mem::take(&mut fs.cur);
let f: &FrameState = fs;
// Unfiltered copies of the rows above each band but the first.
let margins: Vec<[Vec<u16>; 3]> = luma
.iter()
.map(|&(start, _)| {
std::array::from_fn(|p| {
let sub_y = if p == 0 { 0 } else { f.ssy };
let pl = &cur.planes[p];
let (y0, y1) = ((start.saturating_sub(LF_MARGIN)) >> sub_y, start >> sub_y);
let rows = pl.data.get(y0 * pl.stride..y1 * pl.stride);
if start == 0 || !planes.contains(&p) { Vec::new() } else { rows.map(<[u16]>::to_vec).unwrap_or_default() }
})
})
.collect();
let mut bands: Vec<([RowBand; 3], [Vec<u16>; 3])> = cur.row_bands(&luma).into_iter().zip(margins).collect();
let changed = pool.map_mut(&mut bands, |b, (band, margin)| {
let mut changed: [Vec<(usize, u16)>; 3] = Default::default();
for &p in &planes {
changed[p] = filter_band(f, p, luma[b], &mut band[p], &mut margin[p]);
}
changed
});
drop(bands);
for (&(start, _), ch) in luma.iter().zip(&changed) {
for (p, list) in ch.iter().enumerate() {
let sub_y = if p == 0 { 0 } else { f.ssy };
let pl = &mut cur.planes[p];
let first = (start.saturating_sub(LF_MARGIN) >> sub_y) * pl.stride;
for &(i, v) in list {
if let Some(d) = pl.data.get_mut(first + i) {
*d = v;
}
}
}
}
fs.cur = cur;
}
fn edge(fs: &mut FrameState, plane: usize, pass: usize, row: usize, col: usize) {
/// Both loop filter passes over the band of `plane` holding luma rows [start, end); `margin` is
/// an unfiltered copy of the rows above the band (empty for the first band). Returns the samples
/// of `margin` that the band's top horizontal edges changed, as (index, value).
fn filter_band(fs: &FrameState, plane: usize, (start, end): (usize, usize), band: &mut RowBand, margin: &mut [u16]) -> Vec<(usize, u16)> {
let mi_rows = fs.fh.mi_rows as usize;
let (r0, r1) = (start / 4, mi_rows.min(end / 4));
let sub_y = if plane == 0 { 0 } else { fs.ssy };
let edges = |band: &mut RowBand, mut above: Option<&mut RowBand>, pass: usize, rows: std::ops::Range<usize>| {
let row_step = if plane == 0 { 1 } else { 1 << fs.ssy };
let col_step = if plane == 0 { 1 } else { 1 << fs.ssx };
let mi_cols = fs.fh.mi_cols as usize;
let mut row = rows.start;
while row < rows.end {
let mut col = 0;
while col < mi_cols {
edge(fs, band, above.as_deref_mut(), plane, pass, row, col);
col += col_step;
}
row += row_step;
}
};
edges(band, None, 0, r0..r1);
if margin.is_empty() {
edges(band, None, 1, r0..r1);
return Vec::new();
}
let stride = band.stride;
let y0 = start.saturating_sub(LF_MARGIN) >> sub_y;
let mut above = RowBand { data: margin, stride, y0 };
edges(&mut above, None, 0, r0.saturating_sub(LF_MARGIN / 4)..r0);
let before = above.data.to_vec();
edges(band, Some(&mut above), 1, r0..r1);
above.data.iter().zip(&before).enumerate().filter(|(_, (a, b))| a != b).map(|(i, (&a, _))| (i, a)).collect()
}
/// One loop filter edge (7.14.2) at mode info position (row, col): 4 lines of samples across
/// it. `above` holds the rows above `band` for horizontal edges on the band's top row.
fn edge(fs: &FrameState, band: &mut RowBand, above: Option<&mut RowBand>, plane: usize, pass: usize, row: usize, col: usize) {
let (sub_x, sub_y) = if plane == 0 { (0, 0) } else { (fs.ssx, fs.ssy) };
let (dx, dy) = if pass == 0 { (1i32, 0i32) } else { (0, 1) };
let x = col * 4;
@ -102,16 +189,40 @@ fn edge(fs: &mut FrameState, plane: usize, pass: usize, row: usize, col: usize)
return;
}
let t = LfThresh::new(limit, blimit, thresh, fs.bit_depth);
let pl = &mut fs.cur.planes[plane];
let stride = pl.stride;
let (step, along) = if pass == 0 { (1, stride) } else { (stride, 1) };
let base = yp * stride + xp;
let chroma = plane > 0;
let stride = band.stride;
if let Some(above) = above.filter(|_| pass == 1 && yp == band.y0) {
// The p side is in the rows above the band: filter each line in a copy that spans both.
let (n_above, n_band) = (above.data.len() / stride.max(1), band.data.len() / stride.max(1));
for k in 0..4 {
let x = xp + k;
let mut line = [0u16; 2 * LF_LINE];
for j in 1..=n_above.min(LF_LINE) {
line[LF_LINE - j] = above.data[(n_above - j) * stride + x];
}
for j in 0..n_band.min(LF_LINE) {
line[LF_LINE + j] = band.data[j * stride + x];
}
sample_filter(&mut line, LF_LINE, 1, chroma, filter_size, t);
for j in 1..=n_above.min(LF_LINE) {
above.data[(n_above - j) * stride + x] = line[LF_LINE - j];
}
for j in 0..n_band.min(LF_LINE) {
band.data[j * stride + x] = line[LF_LINE + j];
}
}
return;
}
let (step, along) = if pass == 0 { (1, stride) } else { (stride, 1) };
let base = (yp - band.y0) * stride + xp;
for k in 0..4 {
sample_filter(&mut pl.data, base + k * along, step, chroma, filter_size, t);
sample_filter(band.data, base + k * along, step, chroma, filter_size, t);
}
}
/// Samples on each side of an edge that the sample filter can read (at most p6 .. q6).
const LF_LINE: usize = 8;
/// Adaptive filter strength (7.14.4): (lvl, limit, blimit, thresh).
fn strength(fs: &FrameState, row: usize, col: usize, plane: usize, pass: usize) -> (i32, i32, i32, i32) {
let mi = &fs.mi;

94
crates/av1/src/restoration.rs
View File

@ -1,6 +1,8 @@
//! Super-resolution upscaling (7.16) and loop restoration (7.17).
use crate::frame::{FrameBuf, Plane};
use crate::frame::{FrameBuf, Plane, RowBand};
use crate::par::Pool;
use crate::postfilter::bands;
use crate::spec_tables::*;
use crate::state::FrameState;
@ -70,56 +72,66 @@ impl LrCtx<'_> {
}
/// Loop restoration (7.17): returns LrFrame given UpscaledCurrFrame and UpscaledCdefFrame.
pub(crate) fn loop_restoration(fs: &FrameState, cur: &FrameBuf, cdef: &FrameBuf) -> FrameBuf {
/// Bands of 64 luma rows are restored in parallel: every unit reads only the two input frames
/// and writes only its own samples.
pub(crate) fn loop_restoration(fs: &FrameState, cur: &FrameBuf, cdef: &FrameBuf, pool: &Pool) -> FrameBuf {
let fh = &fs.fh;
let mut lr = cdef.clone();
let up_w = fh.upscaled_width as i32;
let fr_h = fh.frame_height as i32;
let bd = fs.bit_depth;
for plane in 0..fs.num_planes {
let rtype_frame = fh.lr.frame_restoration_type[plane];
if rtype_frame == RESTORE_NONE as u8 {
continue;
}
let (sub_x, sub_y) = if plane > 0 { (fs.ssx as i32, fs.ssy as i32) } else { (0, 0) };
let unit_size = fh.lr.loop_restoration_size[plane] as i32;
let unit_rows = fs.lr_unit_rows[plane] as i32;
let unit_cols = fs.lr_unit_cols[plane] as i32;
let plane_end_x = ((up_w + sub_x) >> sub_x) - 1;
let plane_end_y = ((fr_h + sub_y) >> sub_y) - 1;
let mut row = 0;
while row * 4 < fr_h {
let luma_y = row * 4;
let stripe_num = (luma_y + 8) / 64;
let stripe_start_y = (-8 + stripe_num * 64) >> sub_y;
let stripe_end_y = stripe_start_y + (64 >> sub_y) - 1;
let ctx = LrCtx { cur: &cur.planes[plane], cdef: &cdef.planes[plane], plane_end_x, plane_end_y, stripe_start_y, stripe_end_y };
let unit_row = (unit_rows - 1).min(((row * 4 + 8) >> sub_y) / unit_size);
let mut col = 0;
while col * 4 < up_w {
let unit_col = (unit_cols - 1).min(((col * 4) >> sub_x) / unit_size);
let x = (col * 4) >> sub_x;
let y = (row * 4) >> sub_y;
let w = (4 >> sub_x).min(plane_end_x - x + 1);
let h = (4 >> sub_y).min(plane_end_y - y + 1);
if w > 0 && h > 0 {
let ui = (unit_row * unit_cols + unit_col) as usize;
match fs.lr.lr_type[plane][ui] {
t if t == RESTORE_WIENER as u8 => wiener(fs, &ctx, &mut lr.planes[plane], plane, ui, x, y, w, h, bd),
t if t == RESTORE_SGRPROJ as u8 => self_guided(fs, &ctx, &mut lr.planes[plane], plane, ui, x, y, w, h, bd),
_ => {}
}
}
col += 1;
let ranges = bands(pool, fh.frame_height as usize);
let mut planes = lr.row_bands(&ranges);
pool.map_mut(&mut planes, |b, band| {
// Rows of 4 luma samples in this band.
let (start, end) = ranges[b];
let (row0, row1) = ((start / 4) as i32, (end / 4).min(fh.frame_height as usize) as i32);
for (plane, out) in band.iter_mut().enumerate().take(fs.num_planes) {
let rtype_frame = fh.lr.frame_restoration_type[plane];
if rtype_frame == RESTORE_NONE as u8 {
continue;
}
let (sub_x, sub_y) = if plane > 0 { (fs.ssx as i32, fs.ssy as i32) } else { (0, 0) };
let unit_size = fh.lr.loop_restoration_size[plane] as i32;
let unit_rows = fs.lr_unit_rows[plane] as i32;
let unit_cols = fs.lr_unit_cols[plane] as i32;
let plane_end_x = ((up_w + sub_x) >> sub_x) - 1;
let plane_end_y = ((fr_h + sub_y) >> sub_y) - 1;
let mut row = row0;
while row < row1 && row * 4 < fr_h {
let luma_y = row * 4;
let stripe_num = (luma_y + 8) / 64;
let stripe_start_y = (-8 + stripe_num * 64) >> sub_y;
let stripe_end_y = stripe_start_y + (64 >> sub_y) - 1;
let ctx = LrCtx { cur: &cur.planes[plane], cdef: &cdef.planes[plane], plane_end_x, plane_end_y, stripe_start_y, stripe_end_y };
let unit_row = (unit_rows - 1).min(((row * 4 + 8) >> sub_y) / unit_size);
let mut col = 0;
while col * 4 < up_w {
let unit_col = (unit_cols - 1).min(((col * 4) >> sub_x) / unit_size);
let x = (col * 4) >> sub_x;
let y = (row * 4) >> sub_y;
let w = (4 >> sub_x).min(plane_end_x - x + 1);
let h = (4 >> sub_y).min(plane_end_y - y + 1);
if w > 0 && h > 0 {
let ui = (unit_row * unit_cols + unit_col) as usize;
match fs.lr.lr_type[plane][ui] {
t if t == RESTORE_WIENER as u8 => wiener(fs, &ctx, out, plane, ui, x, y, w, h, bd),
t if t == RESTORE_SGRPROJ as u8 => self_guided(fs, &ctx, out, plane, ui, x, y, w, h, bd),
_ => {}
}
}
col += 1;
}
row += 1;
}
row += 1;
}
}
});
drop(planes);
lr
}
#[allow(clippy::too_many_arguments)]
fn wiener(fs: &FrameState, ctx: &LrCtx, out: &mut Plane, plane: usize, ui: usize, x: i32, y: i32, w: i32, h: i32, bd: u32) {
fn wiener(fs: &FrameState, ctx: &LrCtx, out: &mut RowBand, plane: usize, ui: usize, x: i32, y: i32, w: i32, h: i32, bd: u32) {
let round0: u32 = if bd == 12 { 5 } else { 3 };
let round1: u32 = if bd == 12 { 9 } else { 11 };
let coef = &fs.lr.lr_wiener[plane][ui];
@ -163,7 +175,7 @@ fn wiener(fs: &FrameState, ctx: &LrCtx, out: &mut Plane, plane: usize, ui: usize
}
#[allow(clippy::too_many_arguments)]
fn self_guided(fs: &FrameState, ctx: &LrCtx, out: &mut Plane, plane: usize, ui: usize, x: i32, y: i32, w: i32, h: i32, bd: u32) {
fn self_guided(fs: &FrameState, ctx: &LrCtx, out: &mut RowBand, plane: usize, ui: usize, x: i32, y: i32, w: i32, h: i32, bd: u32) {
let set = fs.lr.lr_sgr_set[plane][ui] as usize;
let mut flt0 = [[0i32; 4]; 4];
let mut flt1 = [[0i32; 4]; 4];

18
crates/av1/tests/common/mod.rs
View File

@ -16,6 +16,11 @@ pub fn ffmpeg_dav1d() -> Option<PathBuf> {
ffmpeg_with(&[("-decoders", "libdav1d")])
}
/// ffmpeg with libaom (4:2:2 and 4:4:4 fixtures, which SVT-AV1 cannot make) and libdav1d.
pub fn ffmpeg_libaom() -> Option<PathBuf> {
ffmpeg_with(&[("-encoders", "libaom-av1"), ("-decoders", "libdav1d")])
}
/// Builds such as the Windows "essentials" ones lack these libraries; skip rather than fail.
fn ffmpeg_with(needs: &[(&str, &str)]) -> Option<PathBuf> {
let ff = filmcraft_testkit::ffmpeg_or_skip("av1 oracle")?;
@ -38,9 +43,11 @@ pub fn run(ff: &Path, args: &[&str]) {
assert!(out.status.success(), "ffmpeg {:?} failed: {}", args, String::from_utf8_lossy(&out.stderr));
}
/// An AV1 fixture: a lavfi source encoded by ffmpeg's libsvtav1 into IVF.
/// An AV1 fixture: a lavfi source encoded by ffmpeg (libsvtav1 unless [`Spec::encoder`] says
/// otherwise) into IVF.
pub struct Spec {
pub name: &'static str,
pub encoder: &'static str,
pub lavfi: String,
pub frames: u32,
pub pix_fmt: &'static str,
@ -50,7 +57,12 @@ pub struct Spec {
impl Spec {
pub fn new(name: &'static str, lavfi: impl Into<String>, frames: u32, pix_fmt: &'static str, enc: &[&str]) -> Spec {
Spec { name, lavfi: lavfi.into(), frames, pix_fmt, enc: enc.iter().map(|s| s.to_string()).collect() }
Spec { name, encoder: "libsvtav1", lavfi: lavfi.into(), frames, pix_fmt, enc: enc.iter().map(|s| s.to_string()).collect() }
}
pub fn encoder(mut self, encoder: &'static str) -> Spec {
self.encoder = encoder;
self
}
}
@ -60,7 +72,7 @@ pub fn make(ff: &Path, spec: &Spec) -> PathBuf {
let tmp = filmcraft_testkit::temp_path(&path);
let frames = spec.frames.to_string();
let t = tmp.to_str().unwrap().to_string();
let mut args: Vec<&str> = vec!["-f", "lavfi", "-i", spec.lavfi.as_str(), "-frames:v", frames.as_str(), "-pix_fmt", spec.pix_fmt, "-c:v", "libsvtav1"];
let mut args: Vec<&str> = vec!["-f", "lavfi", "-i", spec.lavfi.as_str(), "-frames:v", frames.as_str(), "-pix_fmt", spec.pix_fmt, "-c:v", spec.encoder];
args.extend(spec.enc.iter().map(String::as_str));
args.extend(["-f", "ivf", t.as_str()]);
run(ff, &args);

16
crates/av1/tests/oracle_inter.rs
View File

@ -101,3 +101,19 @@ fn inter_1080p() {
check_bit_exact(&ff, &s);
}
}
/// 4:2:2 and 4:4:4 (libaom; SVT-AV1 only makes 4:2:0): the post-filters run in row bands whose
/// chroma rows depend on the vertical subsampling, so these check the band edges there too.
#[test]
fn inter_chroma_444_422() {
let Some(ff) = ffmpeg_libaom() else { return };
let p = ["-cpu-used", "8", "-crf", "40", "-enable-restoration", "1"];
let src = "testsrc2=s=416x400:r=25,noise=alls=8:allf=t";
for s in [
Spec::new("inter_aom_444", src, 6, "yuv444p", &p).encoder("libaom-av1"),
Spec::new("inter_aom_422", src, 6, "yuv422p", &p).encoder("libaom-av1"),
Spec::new("inter_aom_444_10bit", src, 6, "yuv444p10le", &p).encoder("libaom-av1"),
] {
check_bit_exact(&ff, &s);
}
}

47
crates/av1/tests/perf.rs
View File

@ -51,8 +51,8 @@ fn run(frames: &[Vec<u8>], threads: usize) -> Run {
}
fn bench(spec: &Spec) {
// AV1_PERF_ONLY=<substring> limits the run to matching fixtures.
if std::env::var("AV1_PERF_ONLY").is_ok_and(|f| !spec.name.contains(f.as_str())) {
// AV1_PERF_ONLY=<substring>[,<substring>...] limits the run to matching fixtures.
if std::env::var("AV1_PERF_ONLY").is_ok_and(|f| !f.split(',').any(|f| spec.name.contains(f))) {
return;
}
let Some(ff) = ffmpeg() else { return };
@ -68,7 +68,6 @@ fn bench(spec: &Spec) {
}
}
let b = best.unwrap();
let n = b.stats.frames.max(1) as f64;
let cpu_fps = b.cpu.map(|c| format!("{:6.1}", b.shown as f64 / c)).unwrap_or_else(|| " ?".into());
println!(
"{:<26} {:>3} shown {:>3} decoded {:>3} tiles/frame | 1 thread: {} fps (thread CPU) {:6.1} fps (wall)",
@ -79,24 +78,35 @@ fn bench(spec: &Spec) {
cpu_fps,
b.shown as f64 / b.wall
);
let total = b.stats.total_secs();
let mut line = String::from(" ms/frame:");
println!("{}", stage_line(" ms/frame:", &b.stats));
// All cores: best of 3 by wall time.
let threads = std::thread::available_parallelism().map(|n| n.get()).unwrap_or(1);
let mut best: Option<Run> = None;
for _ in 0..3 {
let r = run(&frames, threads);
assert_eq!(r.shown, b.shown);
if best.as_ref().is_none_or(|b| r.wall < b.wall) {
best = Some(r);
}
}
let m = best.unwrap();
println!(" {threads} threads: {:6.1} fps (wall, load average {})", b.shown as f64 / m.wall, load_average());
// Per-stage wall time on the frame workers (stages of different frames overlap).
println!("{}", stage_line(&format!(" {threads} threads, ms/frame:"), &m.stats));
}
/// Per-stage milliseconds per frame and share of the total.
fn stage_line(label: &str, stats: &DecodeStats) -> String {
let n = stats.frames.max(1) as f64;
let total = stats.total_secs();
let mut line = String::from(label);
for s in Stage::ALL {
let v = b.stats.secs(s);
let v = stats.secs(s);
if v > 0.0005 {
line += &format!(" {} {:.2} ({:.0}%)", s.name(), 1e3 * v / n, 100.0 * v / total);
}
}
println!("{line}");
// All cores: best of 3 by wall time.
let threads = std::thread::available_parallelism().map(|n| n.get()).unwrap_or(1);
let mut best_wall = f64::INFINITY;
for _ in 0..3 {
let r = run(&frames, threads);
assert_eq!(r.shown, b.shown);
best_wall = best_wall.min(r.wall);
}
println!(" {threads} threads: {:6.1} fps (wall, load average {})", b.shown as f64 / best_wall, load_average());
line
}
#[test]
@ -110,4 +120,9 @@ fn decode_throughput() {
// Higher quality (more coefficients, finer partitions) and a multi-tile stream (4x2 tiles).
bench(&Spec::new("perf_1080p_gop_hq", src, 60, "yuv420p", &p("18", "keyint=60")));
bench(&Spec::new("perf_1080p_gop_tiles", src, 60, "yuv420p", &p("30", "keyint=60:tile-columns=2:tile-rows=1")));
// Low delay (each frame references the previous one, as screen and camera recorders encode):
// frames cannot overlap, so a frame's own parallelism decides the speed.
bench(&Spec::new("perf_1080p_lowdelay", src, 60, "yuv420p", &p("30", "keyint=60:pred-struct=1")));
// One tile at 2160p: the post-filters' row bands are most of the parallelism there.
bench(&Spec::new("perf_2160p_gop", "testsrc2=s=3840x2160:r=25,noise=alls=6:allf=t", 30, "yuv420p", &p("30", "keyint=30")));
}

29
crates/codecs/src/audio_timing_tests.rs
View File

@ -176,3 +176,32 @@ fn playing_at_another_rate_leaves_no_holes() {
let tail: Vec<Vec<f32>> = got.iter().map(|c| c[2048..].to_vec()).collect();
assert_eq!(holes(&tail), 0);
}
/// #790: the encoder priming stored as `CodecDelay` (1024 samples: 21.333 ms at 48 kHz, 23.220 ms
/// at 44.1 kHz) is trimmed exactly. The demuxer takes it off the millisecond timestamps rounded to
/// whole milliseconds; starting the first packet there played the track 16 (48 kHz) or 10 (44.1 kHz)
/// samples late.
#[test]
fn matroska_codec_delay_is_trimmed_to_the_sample() {
for rate in [48_000u32, 44_100] {
let (asc, units) = aac(rate);
let want = continuous(&asc, rate, &units);
let delay_ns = (1024 * 1_000_000_000 + rate as i64 / 2) / rate as i64;
let mut spec = TrackSpec::new(TrackKind::Audio, "A_AAC");
spec.codec_private = asc.clone();
spec.audio = Some((rate as f64, 2, None));
spec.codec_delay_ns = delay_ns as u64;
let mut w = MkvWriter::new(Cursor::new(Vec::new()), vec![spec], MuxOptions::default()).unwrap();
for (k, u) in units.iter().enumerate() {
let start_ns = (k as i64 * 1024 * 1_000_000_000 + rate as i64 / 2) / rate as i64;
w.write_frame(0, start_ns - delay_ns, true, u, None).unwrap();
}
let bytes: Arc<[u8]> = w.finish().unwrap().into_inner().into();
let src = MkvSource::open("primed.mkv", bytes).unwrap();
let frames = units.len().saturating_sub(2) * 1024;
let got = played(&src, rate, frames);
let trimmed: Vec<Vec<f32>> = want.iter().map(|c| c[1024..].to_vec()).collect();
let (n, first) = mismatches(&got, &trimmed, 0);
assert_eq!(n, 0, "{rate} Hz: {n} samples differ from the decode with the priming trimmed, first (channel, sample) {first:?}");
}
}

139
crates/codecs/src/gop.rs
View File

@ -152,6 +152,10 @@ struct State {
bytes: usize,
/// When a frame was last asked of this cache.
last_used: Instant,
/// Presentation pts of the frame the last request wanted: eviction keeps a window around it
/// (not around the newest decoded picture). Walking a long GOP dumps pictures in decode
/// order; a window around those would throw away the playhead frame and force another walk.
pin: i64,
/// Only background requests (thumbnails, [`filmcraft_media::cancel::with_background`]) have
/// used the decoder since it was made: it is given up as soon as another background request
/// is done, whatever the caches' recency.
@ -457,6 +461,7 @@ impl GopCache {
drafts: Default::default(),
bytes: 0,
last_used: Instant::now(),
pin: i64::MIN,
background: false,
}),
pool: pool.clone(),
@ -509,10 +514,19 @@ impl GopCache {
st.bytes -= old.byte_size();
filmcraft_frame::pool::recycle(old);
}
// evict frames far from the most recent (keep a window around the working position)
// Evict frames far from the request (pin), not the newest decode. A frame-threaded
// decoder can emit a burst past the playhead; pivoting on those pts used to drop the
// wanted picture and walk the GOP again.
let pivot = if st.pin != i64::MIN { st.pin } else { pts };
while st.bytes > budget && st.frames.len() > 2 {
let (Some(&first), Some(&last)) = (st.frames.keys().next(), st.frames.keys().next_back()) else { break };
let victim = if pts - first > last - pts { first } else { last };
let mut victim = if pivot - first > last - pivot { first } else { last };
if victim == pivot || victim == pts {
victim = if victim == first { last } else { first };
if victim == pivot || victim == pts {
break;
}
}
if let Some(v) = st.frames.remove(&victim) {
st.drafts.remove(&victim);
st.bytes -= v.byte_size();
@ -583,6 +597,7 @@ impl GopCache {
let mut st = self.shared.state.lock().unwrap_or_else(|e| e.into_inner());
let _decoding = DecodingGuard::enter(me);
st.last_used = Instant::now();
st.pin = want_pts;
let background = filmcraft_media::cancel::background();
if let Some(f) = st.cached(want_pts, draft).cloned() {
HITS.fetch_add(1, Ordering::Relaxed);
@ -919,19 +934,27 @@ mod tests {
fn intra_frames_decode_in_parallel_once_each() {
let s = Arc::new(samples(64, 1, 0, true));
let c = Arc::new(GopCache::new(None));
std::thread::scope(|scope| {
for t in 0..4usize {
let (s, c) = (s.clone(), c.clone());
scope.spawn(move || {
for k in 0..64usize {
let i = (k * 7 + t * 16) % 64;
assert_eq!(index_of(&c.frame(&*s, i as i64 * 1000).expect("frame")), i);
}
});
}
});
// a frame two threads miss at the same moment may decode twice; nothing more
assert!(s.decodes.load(Ordering::Relaxed) <= 64 + 4 * 4);
// First decode 64 distinct cold frames: each of the four workers owns
// 16 indices. Multiplication by seven permutes them without overlap.
// Then let every worker request all 64 cached frames in overlapping order.
// Concurrent misses for the same cold frame may legitimately decode twice,
// so they cannot support a scheduler-independent duplicate allowance.
for requests_per_worker in [16usize, 64] {
let start = Arc::new(std::sync::Barrier::new(4));
std::thread::scope(|scope| {
for t in 0..4usize {
let (s, c, start) = (s.clone(), c.clone(), start.clone());
scope.spawn(move || {
start.wait();
for k in 0..requests_per_worker {
let i = (k * 7 + t * 16) % 64;
assert_eq!(index_of(&c.frame(&*s, i as i64 * 1000).expect("frame")), i);
}
});
}
});
assert_eq!(s.decodes.load(Ordering::Relaxed), 64, "cold frames decode once; warm requests hit the cache");
}
assert_eq!(s.resets.load(Ordering::Relaxed), 0);
}
@ -1364,4 +1387,90 @@ mod tests {
}
assert_eq!(s.resets.load(Ordering::Relaxed), 1);
}
/// Many pictures in one GOP; a decoder that emits a burst past the wanted sample (frame-
/// threaded H.264). Eviction used to pivot on the newest pts and drop the playhead frame.
struct Burst {
n: usize,
/// Pictures held before a decode dumps them all.
burst: usize,
held: VecDeque<i64>,
resets: Arc<AtomicUsize>,
decodes: Arc<AtomicUsize>,
}
impl VideoDecoder for Burst {
fn decode(&mut self, _sample: &[u8], pts: i64) -> crate::Result<Vec<DecodedFrame>> {
self.decodes.fetch_add(1, Ordering::Relaxed);
self.held.push_back(pts);
if self.held.len() < self.burst {
return Ok(Vec::new());
}
Ok(self.held.drain(..).map(|p| picture((p, false))).collect())
}
fn flush(&mut self) -> Vec<DecodedFrame> {
self.held.drain(..).map(|p| picture((p, false))).collect()
}
fn reset(&mut self) {
self.resets.fetch_add(1, Ordering::Relaxed);
self.held.clear();
}
fn name(&self) -> &str {
"burst"
}
}
impl VideoSamples for Burst {
fn count(&self) -> usize {
self.n
}
fn pts(&self, i: usize) -> i64 {
i as i64 * 1000
}
fn sync_before(&self, _i: usize) -> usize {
0
}
fn sample_at(&self, t: i64) -> Option<usize> {
let i = (t / 1000) as usize;
(i < self.n).then_some(i)
}
fn read(&self, i: usize) -> crate::Result<Vec<u8>> {
Ok(vec![i as u8])
}
fn make_decoder(&self) -> crate::Result<Box<dyn VideoDecoder>> {
Ok(Box::new(Burst { n: self.n, burst: self.burst, held: VecDeque::new(), resets: self.resets.clone(), decodes: self.decodes.clone() }))
}
}
#[test]
fn long_gop_seek_keeps_the_target_and_does_not_walk_twice() {
let s = Burst { n: 200, burst: 90, held: VecDeque::new(), resets: Default::default(), decodes: Default::default() };
let mut c = GopCache::new(None);
c.budget = 0;
// Want a picture early in the GOP; the burst dumps 90 pictures at once (past MIN_FRAMES).
assert_eq!(index_of(&c.frame(&s, 10_000).expect("frame")), 10);
assert_eq!(s.resets.load(Ordering::Relaxed), 1);
let decodes = s.decodes.load(Ordering::Relaxed);
assert!(decodes <= 90 + 1, "walked {decodes} samples");
// The wanted picture is still cached: asking again must not restart or re-feed.
assert_eq!(index_of(&c.frame(&s, 10_000).expect("cached")), 10);
assert_eq!(s.resets.load(Ordering::Relaxed), 1);
assert_eq!(s.decodes.load(Ordering::Relaxed), decodes);
// Playback continues from the same run.
assert_eq!(index_of(&c.frame(&s, 11_000).expect("frame")), 11);
assert_eq!(s.resets.load(Ordering::Relaxed), 1);
}
#[test]
fn long_gop_continue_does_not_restart_for_the_next_frame() {
let s = samples(300, 250, 8, false);
let mut c = GopCache::new(None);
c.budget = 0;
assert_eq!(index_of(&c.frame(&s, 80_000).expect("frame")), 80);
let resets = s.resets.load(Ordering::Relaxed);
let decodes = s.decodes.load(Ordering::Relaxed);
assert_eq!(index_of(&c.frame(&s, 81_000).expect("frame")), 81);
assert_eq!(s.resets.load(Ordering::Relaxed), resets);
assert!(s.decodes.load(Ordering::Relaxed) - decodes <= 8 + 2);
}
}

4
crates/codecs/src/hw.rs
View File

@ -258,7 +258,7 @@ impl NalStreamInfo {
bit_depth_chroma: sps.bit_depth_chroma,
interlaced: !sps.frame_mbs_only,
profile_idc: sps.profile_idc,
color: vui_color(crop.2, crop.3, vui.matrix_coefficients, vui.transfer_characteristics, vui.full_range),
color: vui_color(crop.2, crop.3, vui.colour_primaries, vui.matrix_coefficients, vui.transfer_characteristics, vui.full_range),
par: sar_par(vui.sar),
// as the software decoder's DPB (`max_reorder`)
reorder: sps.max_num_reorder_frames().min(sps.max_dpb_frames()),
@ -304,7 +304,7 @@ impl NalStreamInfo {
bit_depth_chroma: sps.bit_depth_chroma,
interlaced: false,
profile_idc: sps.ptl.profile_idc,
color: vui_color(crop.2, crop.3, vui.matrix_coefficients, vui.transfer_characteristics, vui.full_range),
color: vui_color(crop.2, crop.3, vui.colour_primaries, vui.matrix_coefficients, vui.transfer_characteristics, vui.full_range),
par: sar_par(vui.sar),
reorder: sps.max_num_reorder as usize,
})

48
crates/codecs/src/mkv.rs
View File

@ -71,15 +71,14 @@ pub struct MkvSource {
/// length (AAC, MPEG audio, AC-3, E-AC-3) therefore run on from one another, resynchronising to the
/// timestamp only across gaps of more than half a packet (and more than two ticks); see
/// [`crate::audio::contiguous_starts`]. Opus packets: [`opus_starts`]. Other codecs start at their
/// timestamps.
/// timestamps. Timestamps are taken with the exact `CodecDelay` off (see [`stamp_to_samples`]).
fn audio_starts(file: &MkvFile, bytes: &crate::Src, ti: usize, rate: i64) -> Vec<i64> {
use crate::audio::{FixedFrames, PacketTime, contiguous_starts, fixed_packet_samples};
let Some(t) = file.tracks.get(ti) else { return Vec::new() };
if let Some(h) = opus_head(&t.codec) {
return opus_starts(file, bytes, ti, &h, rate);
}
let (n, d) = tb(t);
let at = |pts: i64| (pts as i128 * n as i128 * rate as i128 / d as i128) as i64;
let at = stamp_to_samples(t, rate, codec_delay_samples(t, rate));
let first = || file.read_sample(bytes, ti, 0).unwrap_or_default();
let out_rate = u32::try_from(rate).unwrap_or(0);
let frame = match &t.codec {
@ -98,18 +97,13 @@ fn audio_starts(file: &MkvFile, bytes: &crate::Src, ti: usize, rate: i64) -> Vec
/// Sample-exact Opus packet start positions (48 kHz frames, pre-skip removed).
///
/// The demuxer also subtracts a rounded `CodecDelay` from the timestamps. Starts are accumulated
/// from each packet's TOC duration, resynchronising to the timestamp only across real gaps (more
/// than two ticks off). Pre-skip is the exact `CodecDelay` (or the header's pre-skip when it is
/// absent).
/// Starts are accumulated from each packet's TOC duration, resynchronising to the timestamp only
/// across real gaps (more than two ticks off). Pre-skip is the exact `CodecDelay` (or the header's
/// pre-skip when it is absent).
fn opus_starts(file: &MkvFile, bytes: &crate::Src, ti: usize, head: &filmcraft_opus::OpusHead, rate: i64) -> Vec<i64> {
let Some(t) = file.tracks.get(ti) else { return Vec::new() };
let (n, d) = tb(t);
let delay_ns = t.codec_delay_ns as i128;
// The demuxer's rounding of CodecDelay to ticks, undone here.
let delay_ticks = codec_delay_ticks(t);
let skip = if t.codec_delay_ns > 0 { (delay_ns * rate as i128 / 1_000_000_000) as i64 } else { head.pre_skip as i64 };
let at = |pts: i64| ((pts.saturating_add(delay_ticks) as i128 * n as i128 * rate as i128 / d as i128) as i64).saturating_sub(skip);
let skip = if t.codec_delay_ns > 0 { codec_delay_samples(t, rate) } else { head.pre_skip as i64 };
let at = stamp_to_samples(t, rate, skip);
let packets = t.samples.iter().enumerate().map(|(i, s)| crate::audio::PacketTime {
stamp: at(s.pts),
stamped: own_stamp(s),
@ -118,6 +112,28 @@ fn opus_starts(file: &MkvFile, bytes: &crate::Src, ti: usize, head: &filmcraft_o
crate::audio::contiguous_starts(packets, stamp_tolerance(t, rate))
}
/// A block timestamp (with the demuxer's rounded `CodecDelay` taken off) as a position in sample
/// frames at `rate`, `skip` frames of priming removed (#790). The demuxer's rounding is undone first:
/// `CodecDelay` is rarely a whole number of ticks (1024 AAC samples at 48 kHz are 21.333 ms, 256
/// AC-3 samples at 44.1 kHz 5.805 ms), so starting the first packet at the rounded delay played the
/// whole track up to 16 samples late.
fn stamp_to_samples(t: &filmcraft_matroska::Track, rate: i64, skip: i64) -> impl Fn(i64) -> i64 {
let (n, d) = tb(t);
let delay_ticks = codec_delay_ticks(t);
move |pts: i64| {
let ticks = pts.saturating_add(delay_ticks) as i128;
let at = ticks.checked_mul(n as i128).and_then(|x| x.checked_mul(rate as i128)).map_or(ticks.signum() * i128::MAX, |x| x / d as i128);
i64::try_from(at).unwrap_or(if at < 0 { i64::MIN } else { i64::MAX }).saturating_sub(skip)
}
}
/// `CodecDelay` in sample frames at `rate`, rounded to the nearest (the element holds whole
/// nanoseconds: 1024 samples at 48 kHz are stored as 21 333 333 ns).
fn codec_delay_samples(t: &filmcraft_matroska::Track, rate: i64) -> i64 {
let ns = (t.codec_delay_ns as i128).saturating_mul(rate.max(0) as i128);
i64::try_from(ns.saturating_add(500_000_000) / 1_000_000_000).unwrap_or(i64::MAX)
}
/// `CodecDelay` in timestamp ticks, rounded as the demuxer rounds it (half away from zero) before
/// subtracting it from every timestamp of the track.
fn codec_delay_ticks(t: &filmcraft_matroska::Track) -> i64 {
@ -358,7 +374,11 @@ impl MkvSource {
let rate = match t.default_duration_ns {
Some(ns) if ns > 0 => FrameRate::from_f64(1e9 / ns as f64),
_ => {
let mut d: Vec<i64> = t.samples.windows(2).take(240).map(|p| (p[1].pts - p[0].pts).abs()).filter(|d| *d > 0).collect();
// blocks are stored in decode order: with B-frames, neighbours in the file are
// up to a few frames apart, so step between presentation times (#923)
let mut pts: Vec<i64> = t.samples.iter().take(241).map(|s| s.pts).collect();
pts.sort_unstable();
let mut d: Vec<i64> = pts.windows(2).map(|p| p[1].saturating_sub(p[0])).filter(|d| *d > 0).collect();
d.sort_unstable();
let (n, dd) = tb(t);
let step = d.get(d.len() / 2).copied().unwrap_or(1).max(1) as f64 * n as f64 / dd as f64;

185
crates/codecs/src/mpeg.rs
View File

@ -7,8 +7,9 @@
//! first random-access picture; display order comes from the PTS (or, without timestamps, the
//! GOP and temporal reference). Leading pictures of an open GOP decode from the previous GOP.
//! - Audio: MPEG audio layers I-III (symphonia), AAC in ADTS or LATM (our decoder), AC-3 (our
//! decoder), Blu-ray / AVCHD and DVD LPCM. Audio frames are placed on the timeline from their
//! PTS relative to the first video frame, sample-contiguously (re-anchored only at gaps).
//! decoder), Blu-ray / AVCHD and DVD LPCM. Every audio PID is a stream (capped). Audio frames
//! are placed on the timeline from their PTS relative to the first video frame,
//! sample-contiguously (re-anchored only at gaps).
use std::collections::{HashMap, VecDeque};
use std::sync::{Arc, Mutex};
@ -104,7 +105,8 @@ pub struct MpegSource {
es: Option<(Arc<[u8]>, Vec<std::ops::Range<usize>>)>,
video: Option<VideoTrack>,
gop: GopCache,
audio: Option<AudioTrack>,
/// Playable audio tracks in file order: `info.audio_streams[k]` describes `audios[k]`.
audios: Vec<AudioTrack>,
/// Why the video or audio stream cannot be decoded.
unsupported_video: Option<String>,
unsupported_audio: Option<String>,
@ -201,8 +203,8 @@ impl MpegSource {
log::warn!("{name}: {w}");
}
let vstream = file.find(Kind::Video, |c| vkind(c).is_some()).or_else(|| file.find(Kind::Video, |_| true));
let astream = file.find(Kind::Audio, audio_supported).or_else(|| file.find(Kind::Audio, |_| true));
if vstream.is_none() && astream.is_none() {
let has_audio = file.streams.iter().any(|s| s.kind() == Kind::Audio && !s.units.is_empty());
if vstream.is_none() && !has_audio {
return Err(CodecError::Unsupported(format!("{}: no video or audio streams", file.format.name())));
}
let mut src = MpegSource {
@ -221,7 +223,7 @@ impl MpegSource {
es: None,
video: None,
gop: GopCache::new(None),
audio: None,
audios: Vec::new(),
unsupported_video: None,
unsupported_audio: None,
};
@ -256,7 +258,7 @@ impl MpegSource {
src.video = Some(track);
src.describe_video(rate_from_pts)?;
if let Some(file) = src.file.take() {
src.setup_audio(file, astream, origin);
src.setup_audio(file, origin);
}
}
None => {
@ -273,12 +275,12 @@ impl MpegSource {
bitrate: None,
hdr: None,
});
src.setup_audio(file, astream, None);
src.setup_audio(file, None);
}
}
} else {
src.info.kind = MediaKind::AudioOnly;
src.setup_audio(file, astream, None);
src.setup_audio(file, None);
}
Ok(src)
}
@ -326,7 +328,7 @@ impl MpegSource {
es: Some((all, aus)),
video: Some(track),
gop: GopCache::new(None),
audio: None,
audios: Vec::new(),
unsupported_video: None,
unsupported_audio: None,
};
@ -347,15 +349,15 @@ impl MpegSource {
/// Timeline position (samples at the audio stream's rate) of the first audio frame, and the
/// number of samples in the audio stream.
pub fn audio_extent(&self) -> Option<(i64, i64)> {
let a = self.audio.as_ref()?;
let a = self.audios.first()?;
let first = *a.starts.first()?;
let end = a.starts.last()? + *a.lens.last()? as i64;
Some((first, end - first))
}
/// The demuxed stream indices of the presented video and audio.
/// The demuxed stream indices of the presented video and first audio.
pub fn stream_indices(&self) -> (Option<usize>, Option<usize>) {
(self.video.as_ref().filter(|_| self.es.is_none()).map(|v| v.stream), self.audio.as_ref().map(|a| a.stream))
(self.video.as_ref().filter(|_| self.es.is_none()).map(|v| v.stream), self.audios.first().map(|a| a.stream))
}
/// Presentation timestamps (90 kHz) of the presented video frames in display order. Frames
@ -475,15 +477,31 @@ impl MpegSource {
Ok(())
}
/// Choose and index the audio stream; `origin`: PTS of the first video frame.
fn setup_audio(&mut self, file: File, astream: Option<usize>, origin: Option<i64>) {
let track = astream.and_then(|ai| self.audio_track(&file, ai, origin));
/// Index every audio stream the container carries (capped); `origin`: PTS of the first video frame.
fn setup_audio(&mut self, file: File, origin: Option<i64>) {
let indices: Vec<usize> = file
.streams
.iter()
.enumerate()
.filter(|(_, s)| s.kind() == Kind::Audio && !s.units.is_empty())
.map(|(i, _)| i)
.take(filmcraft_media::MAX_AUDIO_STREAMS)
.collect();
let mut tracks = Vec::new();
for ai in indices {
if let Some(t) = self.audio_track(&file, ai, origin) {
tracks.push(t);
}
}
self.file = Some(file);
self.audio = track;
self.audios = tracks;
}
/// Mark the audio stream as not decodable (it still shows in the media info).
/// Mark the audio stream as not decodable (it still shows in the media info when nothing plays).
fn audio_unsupported(&mut self, codec_name: &str, why: String) -> Option<AudioTrack> {
if !self.info.audio_streams.is_empty() {
return None;
}
self.unsupported_audio = Some(why);
self.info.audio_streams =
vec![AudioStreamInfo { sample_rate: 48_000, channels: 2, codec: format!("{codec_name} (unsupported)"), bits_per_sample: None }];
@ -594,14 +612,16 @@ impl MpegSource {
next = Some(s + n as i64);
}
let total = next.unwrap_or(0).max(0);
if self.info.video.is_none() || self.unsupported_video.is_some() {
if self.unsupported_audio.take().is_some() {
self.info.audio_streams.clear();
}
if self.info.audio_streams.is_empty() && (self.info.video.is_none() || self.unsupported_video.is_some()) {
self.info.duration = Tick::from_units(total, rate.max(1) as i64);
if self.info.video.is_none() {
self.info.kind = MediaKind::AudioOnly;
}
}
self.info.audio_streams =
vec![AudioStreamInfo { sample_rate: rate.max(1), channels: channels.max(1) as u32, codec: codec_name, bits_per_sample: bits }];
self.info.audio_streams.push(AudioStreamInfo { sample_rate: rate.max(1), channels: channels.max(1) as u32, codec: codec_name, bits_per_sample: bits });
Some(AudioTrack {
stream: ai,
codec,
@ -783,10 +803,16 @@ impl MediaSource for MpegSource {
}
fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
if let Some(why) = &self.unsupported_audio {
self.audio_stream(0, start, frames, sample_rate)
}
fn audio_stream(&self, stream: usize, start: i64, frames: usize, sample_rate: u32) -> Result<AudioBuffer, MediaError> {
if self.audios.is_empty()
&& let Some(why) = &self.unsupported_audio
{
return Err(MediaError::Unsupported(why.clone()));
}
let a = self.audio.as_ref().ok_or(MediaError::NoStream("audio"))?;
let a = self.audios.get(stream).ok_or(MediaError::NoStream("audio"))?;
let ch = a.channels;
let ratio = a.rate as f64 / sample_rate as f64;
let exact = a.rate == sample_rate;
@ -852,6 +878,7 @@ pub fn reader_opener(name: &str, head: &[u8], reader: &filmcraft_media::SharedRe
#[cfg(test)]
mod tests {
use super::*;
use filmcraft_media::MediaSource;
/// A mutated transport stream whose PTS span gave ~0 fps panicked on open with "attempt to
/// divide by zero" (`FrameRate::tick_of`).
@ -868,6 +895,118 @@ mod tests {
assert_eq!(stream_rate(None, Some(50.0)), FrameRate::from_f64(50.0));
}
/// MPEG-1 layer II frame, 48 kHz 192 kb/s (576 bytes).
fn mp2_frame(k: u8) -> Vec<u8> {
let mut v = vec![0xFF, 0xFD, 0xA4, 0x00];
v.extend((0..572).map(|i| (i as u8) ^ k));
v
}
fn ts_bytes(prefix: u8, v: i64) -> [u8; 5] {
let v = v & ((1 << 33) - 1);
[(prefix << 4) | (((v >> 30) & 7) as u8) << 1 | 1, (v >> 22) as u8, (((v >> 15) & 0x7F) as u8) << 1 | 1, (v >> 7) as u8, ((v & 0x7F) as u8) << 1 | 1]
}
fn pes(stream_id: u8, pts: Option<i64>, payload: &[u8]) -> Vec<u8> {
let mut h = Vec::new();
if let Some(p) = pts {
h.extend(ts_bytes(2, p));
}
let flags = if pts.is_some() { 0x80 } else { 0 };
let len = 3 + h.len() + payload.len();
let mut v = vec![0, 0, 1, stream_id, (len >> 8) as u8, len as u8, 0x80, flags, h.len() as u8];
v.extend(h);
v.extend_from_slice(payload);
v
}
/// Tiny TS muxer: PAT, PMT and PES packets in 188-byte packets.
struct TsMux {
out: Vec<u8>,
cc: std::collections::HashMap<u16, u8>,
}
impl TsMux {
fn packet(&mut self, pid: u16, pusi: bool, payload: &[u8]) -> usize {
let cc = self.cc.entry(pid).or_insert(0);
let mut p = vec![0x47, (pusi as u8) << 6 | (pid >> 8) as u8, pid as u8];
let n = payload.len().min(184);
if n < 184 {
let al = 183 - n;
p.push(0x30 | *cc);
p.push(al as u8);
if al > 0 {
p.push(0);
p.extend(std::iter::repeat_n(0xFF, al - 1));
}
} else {
p.push(0x10 | *cc);
}
*cc = (*cc + 1) & 15;
p.extend_from_slice(&payload[..n]);
self.out.extend(&p);
n
}
fn section(&mut self, pid: u16, body: &[u8]) {
let mut s = body.to_vec();
s.extend(crc32_mpeg(&s).to_be_bytes());
let mut payload = vec![0];
payload.extend(s);
self.packet(pid, true, &payload);
}
fn psi(&mut self, streams: &[(u8, u16)]) {
let pat = [0x00, 0xB0, 13, 0, 1, 0xC1, 0, 0, 0, 1, 0xE1, 0x00];
self.section(0, &pat);
let mut pmt = vec![0x02, 0xB0, 0, 0, 1, 0xC1, 0, 0, 0xE1, 0x00, 0xF0, 0];
for &(st, pid) in streams {
pmt.extend([st, 0xE0 | (pid >> 8) as u8, pid as u8, 0xF0, 0]);
}
let len = pmt.len() - 3 + 4;
pmt[1] = 0xB0 | (len >> 8) as u8;
pmt[2] = len as u8;
self.section(0x100, &pmt);
}
fn pes(&mut self, pid: u16, data: &[u8]) {
let mut rest = data;
let mut first = true;
while !rest.is_empty() {
let n = self.packet(pid, first, rest);
rest = &rest[n..];
first = false;
}
}
}
fn crc32_mpeg(data: &[u8]) -> u32 {
let mut crc = 0xFFFF_FFFFu32;
for &b in data {
crc ^= (b as u32) << 24;
for _ in 0..8 {
crc = if crc & 0x8000_0000 != 0 { (crc << 1) ^ 0x04C1_1DB7 } else { crc << 1 };
}
}
crc
}
/// Dual MPEG audio in one transport stream: import used to keep only the first PID.
#[test]
fn every_audio_pid_is_a_stream() {
let mut mux = TsMux { out: Vec::new(), cc: Default::default() };
mux.psi(&[(0x03, 0x1100), (0x03, 0x1101)]);
for k in 0..4u16 {
mux.pes(0x1100, &pes(0xC0, Some(90_000 + 2160 * k as i64), &mp2_frame(k as u8)));
mux.pes(0x1101, &pes(0xC1, Some(90_000 + 2160 * k as i64), &mp2_frame(k as u8 + 16)));
}
let source = MpegSource::open("dual.ts", mux.out.into()).unwrap();
assert_eq!(source.info().audio_streams.len(), 2, "{:?}", source.info().audio_streams);
assert_eq!(source.info().audio_streams.iter().map(|a| a.sample_rate).collect::<Vec<_>>(), vec![48_000, 48_000]);
let a0 = source.audio(0, 64, 48_000).unwrap();
let a1 = source.audio_stream(1, 0, 64, 48_000).unwrap();
assert_eq!(a0.channels.len(), 2);
assert_eq!(a1.channels.len(), 2);
assert!(source.audio_stream(2, 0, 64, 48_000).is_err());
}
#[test]
fn presentation_drops_leading_pictures_and_orders_by_key() {
// decode order I(2) B(0) B(1) P(5) B(3) B(4): the leading B pictures are not presented

31
crates/codecs/src/tests.rs
View File

@ -229,6 +229,37 @@ fn matroska_prores_pcm() {
check_mkv("blue_prores_pcm.mkv", &["-c:v", "prores_ks", "-profile:v", "2"], &["-c:a", "pcm_s16le"]);
}
/// `bytes` with its first `DefaultDuration` element (0x23E383) overwritten by an EBML Void of the
/// same length, as a muxer that leaves the optional element out would write the track.
fn without_default_duration(bytes: &[u8]) -> Vec<u8> {
let mut out = bytes.to_vec();
let at = out.windows(3).position(|w| w == [0x23, 0xE3, 0x83]).expect("DefaultDuration");
// ffmpeg writes a one-byte size (0x80 | n)
let total = 4 + (out[at + 3] & 0x7F) as usize;
out[at] = 0xEC;
out[at + 1] = 0x80 | (total - 2) as u8;
out[at + 2..at + total].fill(0);
out
}
#[test]
fn matroska_reordered_video_without_default_duration_keeps_its_rate() {
// blocks are in decode order, so B-frames put file neighbours 2–3 frames apart (#923)
for (bf, name) in [(0, "rate25_bf0.mkv"), (1, "rate25_bf1.mkv"), (2, "rate25_bf2.mkv")] {
let params = format!("bframes={bf}:b-adapt=0:b-pyramid=0:keyint=250:scenecut=0");
let args =
["-f", "lavfi", "-i", "testsrc2=size=64x48:rate=25", "-frames:v", "25", "-c:v", "libx264", "-pix_fmt", "yuv420p", "-x264-params", &params, "-an"];
let Some(control) = fixture(name, &args) else { return };
let stripped = without_default_duration(&control);
for (label, bytes) in [("with DefaultDuration", control.to_vec()), ("without", stripped)] {
let src = crate::open_bytes(name, bytes.into()).expect("open");
let rate = src.info().video.as_ref().expect("video").frame_rate;
assert_eq!((rate.num, rate.den), (25, 1), "{bf} B-frames, {label}");
assert_eq!(src.info().duration, Tick::from_rational(1, 1, 1), "{bf} B-frames, {label}");
}
}
}
/// Path of a fixture under `target/fixtures/codecs/`, generated with ffmpeg when missing
/// (`None` when ffmpeg or the encoder is unavailable).
fn fixture_path(name: &str, args: &[&str]) -> Option<PathBuf> {

15
crates/codecs/src/video.rs
View File

@ -140,11 +140,14 @@ pub(crate) fn tight_plane<T: Copy>(src: Vec<T>, stride: usize, w: usize, h: usiz
}
/// Colour of an H.264 / HEVC picture from its VUI code points (ITU-T H.273), as the decoders
/// report it: BT.709 by default with the matrix guessed from the size, the signalled matrix and
/// transfer when known, and full range when flagged. Shared by the software and hardware
/// decoders so the two are interchangeable.
pub fn vui_color(width: u32, height: u32, matrix: u8, transfer: u8, full_range: bool) -> filmcraft_color::ColorInfo {
/// report it: BT.709 by default with the matrix guessed from the size, the signalled primaries,
/// matrix and transfer when known, and full range when flagged. Shared by the software and
/// hardware decoders so the two are interchangeable.
pub fn vui_color(width: u32, height: u32, primaries: u8, matrix: u8, transfer: u8, full_range: bool) -> filmcraft_color::ColorInfo {
let mut color = filmcraft_color::ColorInfo { matrix: filmcraft_frame::default_matrix(width, height), ..filmcraft_color::ColorInfo::REC709 };
if let Some(p) = primaries_from_code(primaries) {
color.primaries = p;
}
if let Some(m) = filmcraft_color::Matrix::from_code(matrix) {
color.matrix = m;
}
@ -250,7 +253,7 @@ impl H264Decoder {
let y = tight_plane(p.y, p.y_stride, w, h);
let u = tight_plane(p.u, p.uv_stride, cw, ch);
let v = tight_plane(p.v, p.uv_stride, cw, ch);
let color = vui_color(p.width, p.height, p.color.matrix, p.color.transfer, p.color.full_range);
let color = vui_color(p.width, p.height, p.color.primaries, p.color.matrix, p.color.transfer, p.color.full_range);
let par = sar_par(p.sar);
let frame = VideoFrame {
width: p.width,
@ -371,7 +374,7 @@ impl HevcDecoder {
alpha: None,
},
};
let color = vui_color(p.width, p.height, p.color.matrix, p.color.transfer, p.color.full_range);
let color = vui_color(p.width, p.height, p.color.primaries, p.color.matrix, p.color.transfer, p.color.full_range);
let par = sar_par(p.sar);
let frame = VideoFrame { width: p.width, height: p.height, data, color, par, pts: filmcraft_time::Tick::ZERO };
DecodedFrame { pts: p.pts, frame, draft: p.draft }

18
crates/codecs/tests/eac3_oracle.rs
View File

@ -53,12 +53,18 @@ fn eac3_in_mp4_matroska_and_transport_streams() {
};
let ours = src.audio(start, n, 48_000).unwrap().channels;
assert_eq!(ours.len(), ch, "{name}");
// Matroska stores CodecDelay (the encoder's 256-sample priming) in ns and timestamps in
// ms, and we place packets at the rounded value (240 samples): a shared Matroska timing
// limit (AAC and AC-3 alike), not a decoding one. Elsewhere the samples line up exactly.
let (worst, lag) = (-32..=32).map(|lag| (snr(&ours, &want, ch, lag), lag)).fold((f64::MIN, 0), |a, b| if b.0 > a.0 { b } else { a });
let max_lag = if name.ends_with(".mkv") { 16 } else { 0 };
assert!(lag.abs() <= max_lag, "{name}: decode is {lag} samples off ffmpeg's");
// Matroska stores the encoder's 256-sample priming as CodecDelay, which we trim exactly
// (checked to the sample against the encoded signal in `matroska_audio_oracle`). Whether
// ffmpeg's CLI trims it too depends on its version (9.x does; 2023-24 builds keep the
// priming samples), so for .mkv its decode may start 256 samples earlier than ours.
// Elsewhere the samples line up exactly, whatever the version.
let lags: Vec<i64> = match name.ends_with(".mkv") {
true => (-32..=32).chain(-256 - 32..=-256 + 32).collect(),
false => (-32..=32).collect(),
};
let (worst, lag) = lags.into_iter().map(|lag| (snr(&ours, &want, ch, lag), lag)).fold((f64::MIN, 0), |a, b| if b.0 > a.0 { b } else { a });
let ok = if name.ends_with(".mkv") { lag == 0 || lag == -256 } else { lag == 0 };
assert!(ok, "{name}: decode is {lag} samples off ffmpeg's");
// a window read on a fresh source (random access) gives the same samples
let fresh = filmcraft_codecs::open_bytes(name, bytes(&f)).unwrap();
let at = n / 3;

61
crates/codecs/tests/frame_color_oracle.rs Normal file
View File

@ -0,0 +1,61 @@
//! Decoded H.264 / HEVC frames carry the colour primaries their VUI signals (#924). ffmpeg writes
//! MP4 without a `colr` box unless asked, so for most files the bitstream is the only place the
//! primaries are described: stream info already read them, the decoded frames reported BT.709.
mod common;
use common::*;
use filmcraft_color::{ColorInfo, ColorSpace, Matrix, Primaries, Range, Transfer};
use filmcraft_media::FrameRequest;
use filmcraft_time::Tick;
/// One 64×64 frame of `codec` with the colour `params` (x264 / x265 syntax), and our stream and
/// frame colour for it.
fn colors(ff: &std::path::Path, name: &str, codec: &[&str]) -> Option<(ColorInfo, ColorInfo)> {
let mut args = vec!["-f", "lavfi", "-i", "testsrc2=size=64x64:rate=1", "-frames:v", "1"];
args.extend_from_slice(codec);
let path = fixture(ff, name, &args)?;
let isobmff = filmcraft_isobmff::open(bytes(&path)).unwrap();
let track = isobmff.tracks.iter().find(|t| t.kind == filmcraft_isobmff::TrackKind::Video).unwrap();
assert!(track.entries[0].video.as_ref().unwrap().color.is_none(), "{name}: the fixture has a colr box");
let src = filmcraft_codecs::open_bytes(name, bytes(&path)).unwrap();
let stream = src.info().video.as_ref().unwrap().color;
let frame = src.video_frame(FrameRequest::full(Tick::ZERO)).unwrap().color;
Some((stream, frame))
}
#[test]
fn decoded_frames_keep_the_signalled_primaries() {
let ff = filmcraft_testkit::require_ffmpeg!();
let x264 = |p: &'static str| ["-c:v", "libx264", "-pix_fmt", "yuv420p", "-x264-params", p];
let x265 = |p: &'static str, pix: &'static str| ["-c:v", "libx265", "-pix_fmt", pix, "-tag:v", "hvc1", "-x265-params", p];
let sdr2020 = ColorInfo { matrix: Matrix::Bt709, transfer: Transfer::Bt709, primaries: Primaries::Bt2020, range: Range::Limited };
let cases: [(&str, Vec<&str>, ColorInfo, ColorSpace); 4] = [
("prim2020_h264.mp4", x264("colorprim=bt2020:transfer=bt709:colormatrix=bt709").to_vec(), sdr2020, ColorSpace::Rec2020),
("prim2020_hevc.mp4", x265("log-level=error:colorprim=bt2020:transfer=bt709:colormatrix=bt709", "yuv420p").to_vec(), sdr2020, ColorSpace::Rec2020),
// control: BT.709 throughout
(
"prim709_h264.mp4",
x264("colorprim=bt709:transfer=bt709:colormatrix=bt709").to_vec(),
ColorInfo { primaries: Primaries::Bt709, ..sdr2020 },
ColorSpace::Rec709,
),
// HDR keeps its transfer and now its primaries too
(
"prim2020_pq_hevc.mp4",
x265("log-level=error:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc", "yuv420p10le").to_vec(),
ColorInfo { matrix: Matrix::Bt2020Ncl, transfer: Transfer::Pq, primaries: Primaries::Bt2020, range: Range::Limited },
ColorSpace::Rec2100Pq,
),
];
for (name, codec, want, space) in cases {
let Some((stream, frame)) = colors(&ff, name, &codec) else {
eprintln!("SKIPPED ({name}): this ffmpeg can't encode it");
continue;
};
println!("{name}: stream {stream:?}, frame {frame:?}");
assert_eq!(stream, want, "{name}: stream");
assert_eq!(frame, want, "{name}: decoded frame");
assert_eq!(ColorSpace::from_info(&frame), space, "{name}");
}
}

97
crates/codecs/tests/matroska_audio_oracle.rs
View File

@ -48,3 +48,100 @@ fn matroska_audio_plays_without_holes() {
assert!(r_ours < 2.0 * r_theirs + 0.005, "{name}: clicks between packets (|Δ²| {r_ours} vs ffmpeg's {r_theirs})");
}
}
/// Mean |difference| between `ours[i]` and `theirs[i + lag]` over `from..from + len`.
fn mean_err(ours: &[f32], theirs: &[f32], from: usize, len: usize, lag: i64) -> f64 {
(from..from + len).map(|i| (ours[i] as f64 - theirs[(i as i64 + lag) as usize] as f64).abs()).sum::<f64>() / len as f64
}
/// The lag in `lags` at which `ours` best matches `theirs` (least [`mean_err`]), and that error.
fn best_lag(ours: &[f32], theirs: &[f32], from: usize, len: usize, lags: std::ops::RangeInclusive<i64>) -> (i64, f64) {
lags.map(|lag| (lag, mean_err(ours, theirs, from, len, lag))).min_by(|a, b| a.1.total_cmp(&b.1)).unwrap()
}
/// A reference signal we control: low-passed white noise from a fixed xorshift seed. Its
/// autocorrelation dies out within a few samples, so even a one-sample misalignment shows.
fn noise(n: usize) -> Vec<f32> {
let (mut state, mut y) = (0x9E37_79B9_7F4A_7C15u64, 0f32);
(0..n)
.map(|_| {
state ^= state << 13;
state ^= state >> 7;
state ^= state << 17;
let white = (state >> 40) as f32 / (1u64 << 24) as f32 - 0.5;
y = 0.5 * y + 0.4 * white;
y
})
.collect()
}
/// #790: ffmpeg stores the encoder priming as `CodecDelay` (1024 samples = 21.333 ms for AAC at
/// 48 kHz, 256 = 5.805 ms for AC-3 at 44.1 kHz, 1105 = 25.057 ms for MP3), and block timestamps to
/// the millisecond. Taking the rounded delay as the start of the first packet played the whole
/// track up to 16 samples late.
///
/// The strict check is version-independent: ffmpeg encodes a signal we generate, and our decode
/// must line up with that signal to the sample (the stream's `CodecDelay` trimmed exactly). ffmpeg's
/// own decode is compared only after a bounded best-lag search: how its CLI handles Matroska
/// priming depends on the version (9.x trims `CodecDelay`, 2023-24 builds keep the priming
/// samples), so its lag is reported, not asserted.
#[test]
fn matroska_audio_is_sample_aligned_with_ffmpeg() {
let ff = filmcraft_testkit::require_ffmpeg!();
let cases: [(&str, u32, &[&str]); 6] = [
("ref_aac_48k.mkv", 48_000, &["-c:a", "aac", "-b:a", "256k"]),
("ref_aac_44k.mkv", 44_100, &["-c:a", "aac", "-b:a", "256k"]),
("ref_ac3_44k.mkv", 44_100, &["-c:a", "ac3", "-b:a", "448k"]),
("ref_ac3_48k.mkv", 48_000, &["-c:a", "ac3", "-b:a", "448k"]),
("ref_eac3_48k.mkv", 48_000, &["-c:a", "eac3", "-b:a", "384k"]),
("ref_mp3_44k.mkv", 44_100, &["-c:a", "libmp3lame", "-b:a", "320k"]),
];
for (name, rate, codec) in cases {
let reference = noise(rate as usize * 2);
let raw = format!("ref_noise_{rate}.f32");
let raw_path = filmcraft_testkit::fixtures::generate(&dir().join(&raw), |tmp| {
std::fs::write(tmp, reference.iter().flat_map(|v| v.to_le_bytes()).collect::<Vec<u8>>()).is_ok()
})
.unwrap();
let (raw_arg, rate_arg) = (raw_path.to_str().unwrap().to_string(), rate.to_string());
let args: Vec<&str> = ["-f", "f32le", "-ar", &rate_arg, "-ac", "1", "-i", &raw_arg].iter().copied().chain(codec.iter().copied()).collect();
let Some(path) = fixture(&ff, name, &args) else {
eprintln!("SKIPPED ({name}): this ffmpeg can't encode it");
continue;
};
let file = bytes(&path);
let mkv = filmcraft_matroska::open(file.clone()).unwrap();
let delay_ns = mkv.tracks.first().map_or(0, |t| t.codec_delay_ns) as i64;
let delay = (delay_ns * rate as i64 + 500_000_000) / 1_000_000_000;
assert!(delay > 0, "{name}: no CodecDelay");
let src = filmcraft_codecs::open_bytes(name, file).unwrap();
let frames = rate as usize * 3 / 2;
let mut ours = Vec::with_capacity(frames);
while ours.len() < frames {
let n = 1000.min(frames - ours.len());
ours.extend_from_slice(&src.audio(ours.len() as i64, n, rate).unwrap().channels[0]);
}
let (from, len, bound) = (rate as usize / 4, rate as usize / 4, delay + 32);
// strict: the encoder's input, to the sample
let (lag, err) = best_lag(&ours, &reference, from, len, -bound..=bound);
let level = reference[from..from + len].iter().map(|v| v.abs() as f64).sum::<f64>() / len as f64;
assert_eq!(lag, 0, "{name}: our decode is {lag} samples off the encoded signal (CodecDelay {delay} samples)");
assert!(err < 0.25 * level, "{name}: decode error {err:.2e} against a signal level of {level:.2e}");
// ffmpeg's decode, wherever its version puts the priming
let theirs = ffmpeg_audio_f32(&ff, &path, &[]);
assert!(theirs.len() >= frames, "{name}: ffmpeg decoded {} samples", theirs.len());
let (ff_lag, ff_err) = best_lag(&ours, &theirs, from, len, -bound..=bound);
let priming = match ff_lag {
0 => "trims CodecDelay".to_string(),
l if l == delay => "keeps the priming samples".to_string(),
l => format!("is {l} samples off"),
};
println!("{name}: CodecDelay {delay} samples; vs the encoded signal lag 0, error {err:.2e} (level {level:.2e}); ffmpeg {priming}, error {ff_err:.2e}");
assert!(ff_err < 1e-3, "{name}: differs from ffmpeg's decode by {ff_err:.2e} at its best lag {ff_lag}");
// read from the middle, as a seek does
let at = rate as i64 / 2;
let mid = src.audio(at, 2000, rate).unwrap().channels[0].clone();
let diff = mid.iter().zip(&ours[at as usize..]).map(|(a, b)| (a - b).abs()).fold(0f32, f32::max);
assert!(diff < 1e-3, "{name}: a read from {at} differs from continuous decoding by {diff}");
}
}

92
crates/codecs/tests/media_oracle.rs
View File

@ -171,6 +171,98 @@ fn dpx_stills_every_layout() {
}
}
/// A big-endian DPX file holding `data` as its one image element (`w` × `h`, the given
/// descriptor, bit depth, packing and colorimetric code; reference codes undefined).
fn dpx_file(w: u32, h: u32, descriptor: u8, bits: u8, packing: u16, colorimetric: u8, data: &[u8]) -> Vec<u8> {
let mut f = vec![0u8; 2048];
let put = |f: &mut Vec<u8>, at: usize, v: &[u8]| f[at..at + v.len()].copy_from_slice(v);
put(&mut f, 0, b"SDPX");
put(&mut f, 4, &2048u32.to_be_bytes());
put(&mut f, 8, b"V2.0");
put(&mut f, 16, &(2048 + data.len() as u32).to_be_bytes());
put(&mut f, 770, &1u16.to_be_bytes());
put(&mut f, 772, &w.to_be_bytes());
put(&mut f, 776, &h.to_be_bytes());
let el = 780;
put(&mut f, el + 4, &u32::MAX.to_be_bytes());
put(&mut f, el + 12, &u32::MAX.to_be_bytes());
f[el + 20] = descriptor;
f[el + 22] = colorimetric;
f[el + 23] = bits;
put(&mut f, el + 24, &packing.to_be_bytes());
put(&mut f, el + 28, &2048u32.to_be_bytes());
f.extend_from_slice(data);
f
}
/// `pix_fmt` raw frame of ffmpeg's test picture at `w` × `h`.
fn raw_picture(ff: &Path, w: u32, h: u32, pix_fmt: &str) -> Vec<u8> {
let alpha = if pix_fmt.contains('a') { ",format=rgba,geq=r='r(X,Y)':g='g(X,Y)':b='b(X,Y)':a='X*2+Y'" } else { "" };
let vf = format!("scale={w}:{h}{alpha}");
ffmpeg_out(ff, &["-f", "lavfi", "-i", "testsrc2=size=96x64", "-frames:v", "1", "-vf", &vf, "-f", "rawvideo", "-pix_fmt", pix_fmt, "-"])
}
/// A DPX file we built must import exactly like ffmpeg decodes it, within `tol` of its 16-bit
/// decode rounded to 8 bits.
fn dpx_matches(ff: &Path, name: &str, file: Vec<u8>, tol: u8) {
let path = dir().join(name);
std::fs::create_dir_all(dir()).unwrap();
std::fs::write(&path, &file).unwrap();
let src = filmcraft_codecs::open_bytes(name, file.into()).unwrap_or_else(|e| panic!("{name}: {e}"));
assert_eq!(src.info().kind, MediaKind::Still, "{name}");
let ours = rgba(&src.video_frame(FrameRequest::full(filmcraft_time::Tick(0))).unwrap());
let wide = ffmpeg_out(ff, &["-i", path.to_str().unwrap(), "-f", "rawvideo", "-pix_fmt", "rgba64le", "-"]);
let want: Vec<u8> = wide.as_chunks::<2>().0.iter().map(|c| ((u32::from(u16::from_le_bytes(*c)) * 255 + 32767) / 65535) as u8).collect();
assert_eq!(ours.len(), want.len(), "{name}");
let d = ours.iter().zip(&want).map(|(a, b)| a.abs_diff(*b)).max().unwrap();
assert!(d <= tol, "{name}: differs by {d}");
eprintln!("{name}: max diff {d}");
}
/// Packed 12-bit RGB (packing 0: components back to back, low bits first, lines on 32-bit
/// boundaries) and 8-bit Y'CbCr (4:2:2, 4:4:4, 4:4:4:4 with alpha; BT.709, BT.601 and unspecified
/// colorimetry), layouts ffmpeg reads but does not write, built here from ffmpeg's test picture
/// (#841).
#[test]
fn dpx_packed_and_ycbcr_match_ffmpeg() {
let ff = filmcraft_testkit::require_ffmpeg!();
let (w, h) = (95u32, 63u32);
let rgb: Vec<u32> = raw_picture(&ff, w, h, "rgb48le").as_chunks::<2>().0.iter().map(|c| u32::from(u16::from_le_bytes(*c)) >> 4).collect();
let mut data = Vec::new();
for line in rgb.chunks(w as usize * 3) {
let (mut acc, mut n) = (0u64, 0u32);
for &v in line {
acc |= u64::from(v) << n;
n += 12;
while n >= 32 {
data.extend_from_slice(&(acc as u32).to_be_bytes());
acc >>= 32;
n -= 32;
}
}
if n > 0 {
data.extend_from_slice(&(acc as u32).to_be_bytes());
}
}
dpx_matches(&ff, "dpx_packed12.dpx", dpx_file(w, h, 50, 12, 0, 6, &data), 1);
// Y'CbCr: even width, scan lines a multiple of 4 bytes
let (w, h) = (96u32, 64u32);
let px = (w * h) as usize;
let uyvy = raw_picture(&ff, w, h, "uyvy422");
let yuv = raw_picture(&ff, w, h, "yuv444p");
let yuva = raw_picture(&ff, w, h, "yuva444p");
let interleave = |planes: &[u8], order: &[usize]| -> Vec<u8> { (0..px).flat_map(|i| order.iter().map(move |&p| planes[p * px + i])).collect() };
// planes are Y, U (Cb), V (Cr), A; DPX stores Cb Y Cr A
let cbycr = interleave(&yuv, &[1, 0, 2]);
let cbycra = interleave(&yuva, &[1, 0, 2, 3]);
for colorimetric in [6u8, 7, 0] {
dpx_matches(&ff, &format!("dpx_cbycry_{colorimetric}.dpx"), dpx_file(w, h, 100, 8, 0, colorimetric, &uyvy), 1);
dpx_matches(&ff, &format!("dpx_cbycr_{colorimetric}.dpx"), dpx_file(w, h, 102, 8, 0, colorimetric, &cbycr), 1);
}
dpx_matches(&ff, "dpx_cbycra.dpx", dpx_file(w, h, 103, 8, 0, 6, &cbycra), 1);
}
#[test]
fn missing_frames_hold_the_previous_frame() {
let ff = filmcraft_testkit::require_ffmpeg!();

69
crates/codecs/tests/rf64_metadata.rs Normal file
View File

@ -0,0 +1,69 @@
//! #925: a chunk after an RF64 `data` sentinel must not become audio.
use std::sync::Arc;
use filmcraft_time::Tick;
fn chunk(id: &[u8; 4], body: &[u8]) -> Vec<u8> {
let mut out = id.to_vec();
out.extend_from_slice(&(body.len() as u32).to_le_bytes());
out.extend_from_slice(body);
if body.len() % 2 == 1 {
out.push(0);
}
out
}
/// Four 16-bit mono frames of silence at 48 kHz, as RIFF or RF64, optionally with a trailing iXML.
fn fixture(rf64: bool, trailing: bool) -> Vec<u8> {
let mut fmt = Vec::new();
for v in [1u16, 1] {
fmt.extend_from_slice(&v.to_le_bytes());
}
fmt.extend_from_slice(&48_000u32.to_le_bytes());
fmt.extend_from_slice(&96_000u32.to_le_bytes());
fmt.extend_from_slice(&2u16.to_le_bytes());
fmt.extend_from_slice(&16u16.to_le_bytes());
let mut out = if rf64 { b"RF64\xff\xff\xff\xffWAVE".to_vec() } else { b"RIFF\0\0\0\0WAVE".to_vec() };
if rf64 {
let mut ds = vec![0u8; 28];
ds[8..16].copy_from_slice(&8u64.to_le_bytes());
ds[16..24].copy_from_slice(&4u64.to_le_bytes());
out.extend(chunk(b"ds64", &ds));
}
out.extend(chunk(b"fmt ", &fmt));
let offset = out.len();
out.extend(chunk(b"data", &[0u8; 8]));
if rf64 {
out[offset + 4..offset + 8].copy_from_slice(&u32::MAX.to_le_bytes());
}
if trailing {
out.extend(chunk(b"iXML", b"<BWFXML></BWFXML>"));
}
let size = out.len() - 8;
if rf64 {
out[20..28].copy_from_slice(&(size as u64).to_le_bytes());
} else {
out[4..8].copy_from_slice(&(size as u32).to_le_bytes());
}
out
}
#[test]
fn rf64_metadata_stays_outside_audio() {
for (name, rf64, trailing) in [("rf64.wav", true, false), ("riff_ixml.wav", false, true), ("rf64_ixml.wav", true, true)] {
let bytes: Arc<[u8]> = fixture(rf64, trailing).into();
let src = filmcraft_codecs::open_bytes(name, bytes.clone()).unwrap();
let stream = filmcraft_media::reader::open_reader(
name,
Arc::new(filmcraft_media::reader::MemReader(bytes)),
&filmcraft_codecs::reader_openers(),
&filmcraft_codecs::openers(),
)
.unwrap();
for s in [&src, &stream] {
assert_eq!(s.info().duration, Tick::from_units(4, 48_000), "{name}");
assert_eq!(s.audio(0, 20, 48_000).unwrap().channels.remove(0), vec![0.0; 20], "{name}");
}
}
}

2
crates/color/src/lib.rs
View File

@ -20,7 +20,7 @@ pub use grade::GradeSpace;
pub use log::LogCurve;
pub use lut::{Lut, Lut1d, Lut3d, LutFormat};
pub use spaces::{ColorPipeline, ColorSpace, Curve, Gamut, WorkingSpace};
pub use transform::{DecodeTable, InputTransform, OutputTransform, REFERENCE_WHITE_NITS, ToneMap};
pub use transform::{DecodeTable, GpuCurve, GpuInputParams, InputTransform, OutputTransform, REFERENCE_WHITE_NITS, ToneMap};
/// YUV ↔ RGB matrix coefficients.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash, Serialize, Deserialize)]

15
crates/color/src/lut.rs
View File

@ -186,7 +186,7 @@ impl LutFormat {
}
fn floats(line: &str) -> Result<Vec<f32>, String> {
line.split_whitespace().map(|x| x.parse::<f32>().map_err(|_| format!("not a number: {x:?}"))).collect()
line.split_whitespace().map(|x| x.parse::<f32>().ok().filter(|v| v.is_finite()).ok_or_else(|| format!("not a finite number: {x:?}"))).collect()
}
impl Lut {
@ -252,6 +252,9 @@ impl Lut {
if v.len() != 2 {
return Err(err(&format!("{key} needs 2 values")));
}
if v[1] <= v[0] {
return Err(err(&format!("{key} maximum must exceed its minimum")));
}
let r = Some([v[0], v[1]]);
if key == "LUT_1D_INPUT_RANGE" { range1 = r } else { range3 = r }
}
@ -591,6 +594,16 @@ mod tests {
assert!(Lut::parse_cube("TITLE \"x\"\n").is_err());
assert!(Lut::parse_cube("LUT_3D_SIZE 2\n0 0 0\nLUT_1D_SIZE 2\n").is_err());
assert!(Lut::parse_cube("LUT_3D_SIZE 2\nDOMAIN_MIN 1 1 1\nDOMAIN_MAX 0 0 0\n0 0 0\n0 0 0\n0 0 0\n0 0 0\n0 0 0\n0 0 0\n0 0 0\n0 0 0\n").is_err());
for key in ["LUT_1D_INPUT_RANGE", "LUT_3D_INPUT_RANGE"] {
let (size, rows) = if key.starts_with("LUT_1D") { ("LUT_1D_SIZE 2", 2) } else { ("LUT_3D_SIZE 2", 8) };
let table = "0 0 0\n".repeat(rows);
for range in ["0 0", "1 0", "NaN 1", "0 inf"] {
assert!(Lut::parse_cube(&format!("{size}\n{key} {range}\n{table}")).is_err(), "accepted {key} {range}");
}
assert!(Lut::parse_cube(&format!("{size}\n{key} -0.5 2\n{table}")).is_ok());
}
assert!(Lut::parse_cube("LUT_1D_SIZE 2\n0 0 0\nNaN 1 1\n").is_err());
assert!(Lut::parse_cube("LUT_1D_SIZE 2\nDOMAIN_MAX inf 1 1\n0 0 0\n1 1 1\n").is_err());
// comments, blank lines, Windows newlines, inline comments
let txt = "# hi\r\nTITLE \"t\"\r\n\r\nLUT_3D_SIZE 2 # two\r\n0 0 0\n1 0 0\n0 1 0\n1 1 0\n0 0 1\n1 0 1\n0 1 1\n1 1 1\n";
let l = Lut3d::parse_cube(txt).unwrap();

67
crates/color/src/transform.rs
View File

@ -259,6 +259,54 @@ impl InputTransform {
pub fn convert(&self, rgb: [f32; 3]) -> [f32; 3] {
self.apply(rgb.map(|v| self.table.lookup(v)))
}
/// The transform as plain numbers for a shader, which evaluates the curve and the BT.2390
/// EETF directly instead of sampling the tables. `None` for what a shader does not do: log
/// curves and the camera-gamut compression.
pub fn gpu_params(&self) -> Option<GpuInputParams> {
let curve = match self.source.curve() {
Curve::Sdr | Curve::Srgb => GpuCurve::Sdr,
Curve::Linear => GpuCurve::Linear,
Curve::Pq => GpuCurve::Pq,
Curve::Hlg => GpuCurve::Hlg,
Curve::Log(_) => return None,
};
if self.compress {
return None;
}
Some(GpuInputParams {
curve,
hlg: self.hlg,
matrix: (!self.identity_matrix).then_some(self.matrix),
tone: self.tone.as_ref().map(|t| (t.src_peak_nits as f32, t.dst_peak_nits as f32)),
desaturate: self.desaturate,
})
}
}
/// How a shader decodes a channel ([`decode_channel`] without the table): SDR with the sRGB curve,
/// linear, PQ in working units (cd/m² over [`REFERENCE_WHITE_NITS`]), HLG to scene light.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum GpuCurve {
Sdr,
Linear,
Pq,
Hlg,
}
/// [`InputTransform`] for a shader ([`InputTransform::gpu_params`]), stage by stage as
/// [`InputTransform::apply`] runs them.
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct GpuInputParams {
pub curve: GpuCurve,
/// HLG OOTF: BT.2020 luma weights and the scale to working units.
pub hlg: Option<([f32; 3], f32)>,
/// Into the working primaries (row-major), when they differ.
pub matrix: Option<[[f32; 3]; 3]>,
/// BT.2390 tone mapping (source peak, target peak) in cd/m², on max(R,G,B) as a ratio.
pub tone: Option<(f32, f32)>,
/// Luma weights to desaturate into the working gamut with ([`desaturate_into_gamut`]).
pub desaturate: Option<[f32; 3]>,
}
/// Whether gamut `a` lies inside gamut `b` (so no compression is needed going a → b).
@ -462,6 +510,25 @@ mod tests {
}
}
#[test]
fn gpu_params_cover_display_referred_media_only() {
// HLG / PQ / BT.2020 into Rec. 709: every stage the shader has (#965)
let p = |cs| InputTransform::new(cs, Range::Limited, &ColorPipeline::REC709, None).gpu_params();
let hlg = p(ColorSpace::Rec2100Hlg).unwrap();
assert_eq!(hlg.curve, GpuCurve::Hlg);
assert!(hlg.hlg.is_some() && hlg.matrix.is_some() && hlg.desaturate.is_some());
assert_eq!(hlg.tone, Some((HDR_PEAK_NITS as f32, REFERENCE_WHITE_NITS as f32)));
let pq = InputTransform::new(ColorSpace::Rec2100Pq, Range::Limited, &ColorPipeline::REC709, Some(4000.0)).gpu_params().unwrap();
assert_eq!((pq.curve, pq.hlg, pq.tone), (GpuCurve::Pq, None, Some((4000.0, REFERENCE_WHITE_NITS as f32))));
let sdr2020 = p(ColorSpace::Rec2020).unwrap();
assert_eq!((sdr2020.curve, sdr2020.tone), (GpuCurve::Sdr, None));
assert!(sdr2020.matrix.is_some() && sdr2020.desaturate.is_some());
// log curves and camera-gamut compression stay on the CPU
for cs in ColorSpace::ALL.into_iter().filter(|c| c.is_log()) {
assert_eq!(p(cs), None, "{cs:?}");
}
}
#[test]
fn sdr_in_hdr_keeps_values_and_display_tone_maps() {
let pipe = hdr(WorkingSpace::Rec2100Pq);

24
crates/edit/src/lib.rs
View File

@ -247,6 +247,15 @@ fn track_mut(seq: &mut Sequence, id: TrackId) -> Result<&mut Track> {
seq.track_mut(id).ok_or(EditError::NoTrack(id))
}
/// The clip's track, or [`EditError::Locked`] when that track may not change.
fn unlocked_track(seq: &Sequence, clip: ClipId) -> Result<TrackId> {
let (tid, _) = seq.find_item(clip).ok_or(EditError::NoItem(clip))?;
if seq.track(tid).ok_or(EditError::NoTrack(tid))?.locked {
return Err(EditError::Locked);
}
Ok(tid)
}
/// Overwrite edits: each item replaces material on its track.
pub fn overwrite(seq: &mut Sequence, placements: Vec<(TrackId, TrackItem)>, ctx: &mut EditCtx) -> Result<Vec<ClipId>> {
let mut links = HashMap::new();
@ -633,7 +642,8 @@ fn neighbours(track: &Track, id: ClipId) -> (Tick, Tick) {
/// Compute the clamped delta a trim may apply (for UI feedback while dragging).
pub fn clamp_trim(seq: &Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delta: Tick, ctx: &EditCtx) -> Result<Tick> {
let (tid, it) = seq.find_item(clip).ok_or(EditError::NoItem(clip))?;
let tid = unlocked_track(seq, clip)?;
let (_, it) = seq.find_item(clip).ok_or(EditError::NoItem(clip))?;
let tr = seq.track(tid).ok_or(EditError::NoTrack(tid))?;
let (prev_end, next_start) = neighbours(tr, clip);
let media = media_len(ctx, it);
@ -703,6 +713,7 @@ pub fn move_edge(it: &mut TrackItem, edge: Edge, d: Tick, move_start: bool) {
/// Trim one edge of an item (Selection-tool edge drag = Regular; Ripple tool = Ripple).
/// Linked partners should be trimmed by the caller with the same delta.
pub fn trim(seq: &mut Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delta: Tick, ctx: &mut EditCtx) -> Result<Tick> {
unlocked_track(seq, clip)?;
let d = clamp_trim(seq, clip, edge, mode, delta, ctx)?;
if d == Tick::ZERO {
return Ok(d);
@ -759,6 +770,9 @@ pub fn trim(seq: &mut Sequence, clip: ClipId, edge: Edge, mode: TrimMode, delta:
/// overwrite material on a sync-locked track, or when such a linked clip has no room to follow.
pub fn ripple_trim_group(seq: &mut Sequence, clips: &[ClipId], edge: Edge, delta: Tick, ctx: &mut EditCtx) -> Result<Tick> {
let Some(&first) = clips.first() else { return Ok(Tick::ZERO) };
for c in clips {
unlocked_track(seq, *c)?;
}
let mut d = delta;
for c in clips {
let x = clamp_trim(seq, *c, edge, TrimMode::Ripple, d, ctx)?;
@ -861,6 +875,8 @@ pub fn ripple_trim_group(seq: &mut Sequence, clips: &[ClipId], edge: Edge, delta
/// Rolling edit between two adjacent items on one track: moves the cut by `delta`.
pub fn roll(seq: &mut Sequence, left: ClipId, right: ClipId, delta: Tick, ctx: &mut EditCtx) -> Result<Tick> {
unlocked_track(seq, left)?;
unlocked_track(seq, right)?;
let (_, l) = seq.find_item(left).ok_or(EditError::NoItem(left))?;
let (_, r) = seq.find_item(right).ok_or(EditError::NoItem(right))?;
if l.end() != r.start {
@ -908,6 +924,7 @@ pub fn roll(seq: &mut Sequence, left: ClipId, right: ClipId, delta: Tick, ctx: &
/// Slip: change which part of the media an item shows without moving it.
pub fn slip(seq: &mut Sequence, clip: ClipId, delta_media: Tick, ctx: &mut EditCtx) -> Result<Tick> {
unlocked_track(seq, clip)?;
let (_, it) = seq.find_item(clip).ok_or(EditError::NoItem(clip))?;
let used = src_of(it.duration, it.speed);
let max_in = media_len(ctx, it).map(|m| m - used).unwrap_or(Tick::MAX);
@ -921,6 +938,7 @@ pub fn slip(seq: &mut Sequence, clip: ClipId, delta_media: Tick, ctx: &mut EditC
/// Slide: move an item between its neighbours, trimming them to keep the gapless span.
pub fn slide(seq: &mut Sequence, clip: ClipId, delta: Tick, ctx: &mut EditCtx) -> Result<Tick> {
unlocked_track(seq, clip)?;
let (tid, it) = seq.find_item(clip).ok_or(EditError::NoItem(clip))?;
let it = it.clone();
let tr = seq.track(tid).ok_or(EditError::NoTrack(tid))?;
@ -1001,6 +1019,7 @@ pub fn slide_items(seq: &mut Sequence, clips: &[ClipId], delta: Tick, ctx: &mut
/// Rate stretch: change an item's duration by dragging an edge, adjusting speed to keep the same media.
pub fn rate_stretch(seq: &mut Sequence, clip: ClipId, edge: Edge, delta: Tick, ctx: &mut EditCtx) -> Result<f64> {
unlocked_track(seq, clip)?;
let (tid, it) = seq.find_item(clip).ok_or(EditError::NoItem(clip))?;
let tr = seq.track(tid).ok_or(EditError::NoTrack(tid))?;
let (prev_end, next_start) = neighbours(tr, clip);
@ -1038,6 +1057,9 @@ pub fn set_speed_group(seq: &mut Sequence, clips: &[ClipId], speed: f64, reverse
if !(speed.is_finite() && speed > 0.0) {
return Err(EditError::Other("speed must be positive".into()));
}
for clip in clips {
unlocked_track(seq, *clip)?;
}
let mut work = seq.clone();
// per member track: where its later material starts, and by how much the member's length changed
let mut origins: Vec<(TrackId, Tick, Tick)> = Vec::new();

11
crates/edit/src/tests.rs
View File

@ -274,6 +274,17 @@ fn regular_trim_respects_neighbours_and_handles() {
fx.seq.check().unwrap();
}
#[test]
fn trim_on_a_locked_track_is_refused() {
let mut fx = Fx::new();
let v1 = fx.v(0);
let a = fx.put(v1, 0, 100, 0);
fx.seq.track_mut(v1).unwrap().locked = true;
let mut n = fx.next;
assert_eq!(trim(&mut fx.seq, a, Edge::Out, TrimMode::Regular, -f(20), &mut Fx::ctx(&mut n)), Err(EditError::Locked));
assert_eq!(fx.spans(v1), vec![(0, 100)]);
}
#[test]
fn held_regular_in_trim_stops_at_previous_clip() {
let mut fx = Fx::new();

1
crates/engine/Cargo.toml
View File

@ -46,4 +46,5 @@ web-time = "1"
[dev-dependencies]
filmcraft-testkit = { workspace = true }
filmcraft-isobmff = { workspace = true }
image = { version = "0.25", default-features = false, features = ["png"] }

26
crates/engine/src/audio_placement_tests.rs
View File

@ -168,6 +168,32 @@ fn missing_audio_tracks_are_cloned_from_the_destination() {
assert_eq!(again, ids);
}
/// #823: a container with two PCM tracks imports both streams, and overwrite places linked clips.
#[test]
fn import_keeps_every_audio_stream_and_places_linked_clips() {
use filmcraft_isobmff::{Brand, Mp4Writer, PcmConfig, SampleEntry, TrackConfig, WriteSample, WriterOptions};
let mut mux = Mp4Writer::new(std::io::Cursor::new(Vec::new()), WriterOptions::new(Brand::Mov)).unwrap();
for (rate, value) in [(48_000u32, 8192i16), (24_000, -16384)] {
let entry = SampleEntry::pcm(PcmConfig { bits: 16, float: false, big_endian: false, signed: true, channels: 1, sample_rate: rate as f64 });
let track = mux.add_track(TrackConfig::new(entry, rate)).unwrap();
let data: Vec<u8> = (0..rate / 10).flat_map(|_| value.to_le_bytes()).collect();
mux.write_sample(track, WriteSample { data: &data, duration: rate / 10, composition_offset: 0, is_sync: true }).unwrap();
}
let bytes = mux.finish().unwrap().into_inner();
let mut s = Session::default();
let item = crate::commands::import_bytes(&mut s, "/two-audio.mov", bytes.into(), None).unwrap();
let streams = s.project.item(item).unwrap().as_media().unwrap().info.audio_streams.len();
assert_eq!(streams, 2);
s.execute("file.newSequence", json!({"name": "S", "fromItem": item.0, "video": 1, "audio": 1})).unwrap();
let seq = s.state.active_sequence.unwrap();
let q = s.project.sequence(seq).unwrap();
assert_eq!(q.audio_tracks.len(), 2);
let (a, b) = (&q.audio_tracks[0].items[0], &q.audio_tracks[1].items[0]);
assert_eq!((a.audio_stream, b.audio_stream), (0, 1));
assert_eq!(a.link, b.link);
assert!(a.link.is_some());
}
#[test]
fn hostile_track_counts_are_an_error_not_a_crash() {
let s = demo();

17
crates/engine/src/autosave.rs
View File

@ -572,15 +572,16 @@ pub fn scan_candidates(data_dir: &Path, skip: Option<&Path>) -> Vec<RecoveryCand
if !dir.is_dir() || Some(dir.as_path()) == skip || !lock_is_stale(&dir) {
continue;
}
let meta = fs::read(dir.join(META)).ok().and_then(|b| serde_json::from_slice::<JournalMeta>(&b).ok());
match meta {
Some(meta) if dir.join(SNAPSHOT).is_file() => {
out.push(RecoveryCandidate { id: e.file_name().to_string_lossy().into_owned(), dir, meta });
}
_ => {
let _ = fs::remove_dir_all(&dir);
}
let snap = dir.join(SNAPSHOT);
if !snap.is_file() {
// Empty leftover dirs (clean exit, or a crash before the first snapshot).
let _ = fs::remove_dir_all(&dir);
continue;
}
// Keep the snapshot even when session.json is missing or unreadable: the project
// is still recoverable. Meta fields fall back to defaults.
let meta = fs::read(dir.join(META)).ok().and_then(|b| serde_json::from_slice::<JournalMeta>(&b).ok()).unwrap_or_default();
out.push(RecoveryCandidate { id: e.file_name().to_string_lossy().into_owned(), dir, meta });
}
out.sort_by(|a, b| b.meta.saved_unix.cmp(&a.meta.saved_unix).then(b.id.cmp(&a.id)));
out

22
crates/engine/src/autosave_tests.rs
View File

@ -176,6 +176,28 @@ fn auto_save_ring_rotates_and_can_save_the_project() {
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn scan_keeps_snapshot_when_session_metadata_is_unreadable() {
// #894: a stale session with a snapshot must not be deleted because session.json is missing
// or corrupt — the recovered project would vanish from the list and the file would be gone.
let d = temp_dir("recover-nometa");
let root = autosave::recovery_root(&d);
let sess = root.join("123-1");
std::fs::create_dir_all(&sess).unwrap();
let snap = filmcraft_format::encode(&filmcraft_project::Project::default(), false);
std::fs::write(sess.join("snapshot.fcproj"), snap).unwrap();
std::fs::write(sess.join("session.json"), b"not json {").unwrap();
let found = autosave::scan_candidates(&d, None);
assert_eq!(found.len(), 1);
assert_eq!(found[0].id, "123-1");
assert!(sess.join("snapshot.fcproj").is_file(), "snapshot must survive the scan");
std::fs::remove_file(sess.join("session.json")).unwrap();
let found = autosave::scan_candidates(&d, None);
assert_eq!(found.len(), 1, "missing metadata still lists the snapshot");
assert!(sess.join("snapshot.fcproj").is_file());
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn portable_marker_keeps_data_next_to_the_executable() {
// #139: a portable install must not write to %APPDATA%.

4
crates/engine/src/clip_ops.rs
View File

@ -1715,8 +1715,8 @@ fn replace_with(s: &mut Session, p: &Value, how: Replace) -> Result<Value> {
}
let Some((_, it)) = q.find_item_mut(*c) else { continue };
let src_in = match how {
// the Source monitor frame lines up with the sequence playhead
Replace::MatchFrame => src_ph - Tick(((ph - it.start).0 as f64 * it.speed.abs()).round() as i64),
// the Source monitor frame lines up with the sequence playhead (reverse-aware)
Replace::MatchFrame => src_ph - (it.moving_source_time_at(ph) - it.source_in),
_ if keep_in => {
let kept = keep_within.map(|(first, last)| it.source_in.clamp(first, last)).unwrap_or(it.source_in);
if kept != it.source_in {

26
crates/engine/src/clip_ops_tests.rs
View File

@ -606,6 +606,32 @@ fn replace_with_clip_from_source_match_frame_and_bin() {
assert!(s.execute("clip.replaceFromBin", json!({"item": music.0})).is_err());
}
/// Match Frame on a reversed clip maps the Source monitor frame through reverse playback, not as
/// if the clip still ran forward (source 50 at sequence 20 used to become source 69).
#[test]
fn replace_from_source_match_frame_on_a_reversed_clip() {
let mut s = demo();
let rate = s.sequence_rate();
s.execute("sequence.linkedSelection", json!({"on": false})).unwrap();
let forest = item_named(&s, "Misty_Forest.mp4");
s.execute("file.newSequence", json!({"name": "MatchFrameReverse"})).unwrap();
s.execute("timeline.place", json!({"item": forest.0, "time": 0})).unwrap();
let c = s.active_sequence().unwrap().video_tracks[0].items[0].clone();
s.execute("timeline.trim", json!({"clip": c.id.0, "edge": "out", "mode": "regular", "delta": (rate.tick_of(60) - c.duration).0})).unwrap();
select(&mut s, &[c.id.0]);
s.execute("clip.speedDuration", json!({"speed": 100, "reverse": true, "ripple": false})).unwrap();
let got = clip(&s, c.id.0);
assert_eq!(rate.frame_at(got.duration), 60);
assert!(got.reverse);
s.execute("source.open", json!({"item": forest.0})).unwrap();
s.execute("playhead.set", json!({"time": rate.tick_of(20).0})).unwrap();
s.execute("source.setPlayhead", json!({"frame": 50})).unwrap();
s.execute("clip.replaceFromSourceMatchFrame", json!({})).unwrap();
let got = clip(&s, c.id.0);
assert_eq!(got.source_time_at(rate.tick_of(20)), rate.tick_of(50));
assert_ne!(got.source_time_at(rate.tick_of(20)), rate.tick_of(69));
}
/// Replace With Clip ▸ From Bin restarts the clip at the new item's In point (as Premiere does);
/// `keepSourceIn` keeps the clip's place in the media instead.
#[test]

221
crates/engine/src/commands.rs
View File

@ -239,6 +239,15 @@ pub(crate) fn time_p(s: &Session, p: &Value, prefix: &str) -> Option<Tick> {
None
}
/// Sequence time from params. Times before 0 are unreachable in the GUI and cut off on export.
pub(crate) fn sequence_time_p(s: &Session, p: &Value, prefix: &str, cmd: &str) -> Result<Option<Tick>> {
let Some(t) = time_p(s, p, prefix) else { return Ok(None) };
if t < Tick::ZERO {
return Err(bad(cmd, "time must not be negative"));
}
Ok(Some(t))
}
pub(crate) fn clip_p(p: &Value, k: &str) -> Option<ClipId> {
u64_p(p, k).map(ClipId)
}
@ -397,20 +406,75 @@ fn move_clips(s: &mut Session, p: &Value) -> Result<Value> {
/// most constrained clip (media end or start) still moves, and no linked partner slips further.
pub(crate) fn slip_together(q: &mut filmcraft_project::Sequence, ctx: &mut edit::EditCtx<'_>, deltas: &[(ClipId, Tick)]) -> Result<()> {
let mut scale = 1.0_f64;
let mut any = false;
for (c, d) in deltas {
if d.0 == 0 {
continue;
}
let x = edit::slip(&mut q.clone(), *c, *d, ctx)?;
scale = scale.min(x.0.abs() as f64 / d.0.abs() as f64);
match edit::slip(&mut q.clone(), *c, *d, ctx) {
Err(edit::EditError::Locked) => continue,
Err(e) => return Err(e.into()),
Ok(x) => {
any = true;
scale = scale.min(x.0.abs() as f64 / d.0.abs() as f64);
}
}
}
if !any && deltas.iter().any(|(_, d)| d.0 != 0) {
return Err(edit::EditError::Locked.into());
}
for (c, d) in deltas {
let dd = if scale < 1.0 { Tick((d.0 as f64 * scale).round() as i64) } else { *d };
edit::slip(q, *c, dd, ctx)?;
match edit::slip(q, *c, dd, ctx) {
Err(edit::EditError::Locked) => continue,
Err(e) => return Err(e.into()),
Ok(_) => {}
}
}
Ok(())
}
/// Adjacent roll pairs that follow `left`/`right` when linked selection is on: the given cut, plus
/// through-edits between the same pieces' linked partners at that same time (Premiere linked
/// selection). Split edits (audio cut at another time) and unlinked clips are not added.
pub fn linked_roll_pairs(seq: &filmcraft_project::Sequence, left: ClipId, right: ClipId, follow_links: bool) -> Vec<(ClipId, ClipId)> {
let mut pairs = vec![(left, right)];
if !follow_links {
return pairs;
}
let Some((_, l)) = seq.find_item(left) else {
return pairs;
};
let Some((_, r)) = seq.find_item(right) else {
return pairs;
};
let cut = l.end();
let left_link = l.link;
let right_link = r.link;
if left_link.is_none() && right_link.is_none() {
return pairs;
}
for tr in seq.all_tracks() {
if tr.locked {
continue;
}
for w in tr.items.windows(2) {
if w[0].id == left && w[1].id == right {
continue;
}
if w[0].end() != w[1].start || w[0].end() != cut {
continue;
}
let left_ok = left_link.is_some_and(|g| w[0].link == Some(g));
let right_ok = right_link.is_some_and(|g| w[1].link == Some(g));
if left_ok && right_ok {
pairs.push((w[0].id, w[1].id));
}
}
}
pairs
}
/// Expand a clip selection with linked partners (when linked selection is on).
pub fn with_links(s: &Session, clips: &[ClipId]) -> Vec<ClipId> {
let Some(seq) = s.active_sequence() else { return clips.to_vec() };
@ -432,12 +496,22 @@ pub fn with_links(s: &Session, clips: &[ClipId]) -> Vec<ClipId> {
/// clamped by each one's media and neighbours.
fn clamp_trim_linked(q: &filmcraft_project::Sequence, clips: &[ClipId], edge: Edge, mode: TrimMode, delta: Tick, ctx: &edit::EditCtx) -> Result<Tick> {
let mut d = delta;
let mut any = false;
for c in clips {
let x = edit::clamp_trim(q, *c, edge, mode, d, ctx)?;
if x.abs() < d.abs() {
d = x;
match edit::clamp_trim(q, *c, edge, mode, d, ctx) {
Err(edit::EditError::Locked) => continue,
Err(e) => return Err(e.into()),
Ok(x) => {
any = true;
if x.abs() < d.abs() {
d = x;
}
}
}
}
if !any {
return Err(edit::EditError::Locked.into());
}
Ok(d)
}
@ -681,8 +755,9 @@ pub(crate) fn place_item(
if s.state.sequences_as_clips && pi.as_sequence().is_some_and(|q| q.multicam.is_none()) {
return crate::sequence_tools::place_sequence_clips(s, item, range, at, vdest, adest, insert, label);
}
let has_v = pi.has_video() && vdest.is_some();
let has_a = pi.has_audio() && adest.is_some();
let (video, audio) = s.project.source_streams(item);
let has_v = video && vdest.is_some();
let has_a = audio && adest.is_some();
if !has_v && !has_a {
return Err(EngineError::Other("no destination track for this media (check source patching)".into()));
}
@ -724,7 +799,7 @@ pub(crate) fn place_item(
let mut a = p.make_track_item(item, TrackKind::Audio, aat, arange, rate).ok_or_else(|| bad(label, "bad item"))?;
// Modify ▸ Audio Channels with several audio clips: one per clip, on the tracks below
let clips: Vec<Vec<u16>> =
p.item(item).and_then(|i| i.as_media()).and_then(|m| m.interpret.audio_channels.as_ref()).map(|m| m.clips.clone()).unwrap_or_default();
p.resolve_media(item).and_then(|(_, m, _)| m.interpret.audio_channels.as_ref()).map(|m| m.clips.clone()).unwrap_or_default();
let streams = p.resolve_media(item).map_or(1, |(_, m, _)| m.info.audio_streams.len());
let specs = audio_placement_specs(&clips, streams);
let first = p.sequence(seq_id).and_then(|q| q.audio_tracks.iter().position(|t| t.id == adest)).unwrap_or(0);
@ -1289,7 +1364,7 @@ fn build() -> Vec<CommandSpec> {
has_seq,
crate::export_tools::export_media
),
query!("jobs.list", "List Jobs", "{}", |s, _| Ok(Value::Array(s.jobs.iter().map(crate::Job::to_json).collect()))),
query!("jobs.list", "List Jobs", "{}", |s, _| Ok(Value::Array(s.jobs.iter().map(|j| crate::export_tools::job_json(s, j)).collect()))),
query!("perf.stats", "Performance Statistics", "{}", |s, _| Ok(crate::perf::stats(s))),
cmd!("jobs.cancel", "Cancel Job", [], None, r#"{"job":id}"#, always, |s, p| {
let id = u64_p(p, "job").ok_or_else(|| bad("jobs.cancel", "need `job`"))?;
@ -1462,7 +1537,12 @@ fn build() -> Vec<CommandSpec> {
}
}
for group in &groups {
edit::set_speed_group(q, group, speed, reverse, ripple, ctx)?;
let unlocked: Vec<ClipId> =
group.iter().copied().filter(|c| q.find_item(*c).is_some_and(|(t, _)| q.track(t).is_some_and(|tr| !tr.locked))).collect();
if unlocked.is_empty() {
return Err(edit::EditError::Locked.into());
}
edit::set_speed_group(q, &unlocked, speed, reverse, ripple, ctx)?;
}
for c in &sel {
if let (Some(m), Some((_, it))) = (interp, q.find_item_mut(*c)) {
@ -1740,7 +1820,7 @@ fn build() -> Vec<CommandSpec> {
"Apply Audio Transition",
["Sequence"],
Some("Cmd+Shift+D"),
r#"{"clip":id?,"effect":str?,"frames":i64?}"#,
r#"{"clip":id?,"effect":"constant_power|Constant Power|…"?,"frames":i64?,"edge":"in"|"out"?,"params":{param:value}?,"reverse":bool?}"#,
has_seq,
|s, p| apply_transition(s, p, TrackKind::Audio)
),
@ -1999,7 +2079,7 @@ fn build() -> Vec<CommandSpec> {
return Ok(v);
}
let (_seq, ph) = marker_list_mut(s)?;
let t = time_p(s, p, "").unwrap_or(ph);
let t = sequence_time_p(s, p, "", "markers.add")?.unwrap_or(ph);
let name = str_p(p, "name").unwrap_or("").to_string();
let comment = str_p(p, "comment").unwrap_or("").to_string();
let color = str_p(p, "color").and_then(Label::from_name).unwrap_or(Label::Green);
@ -2200,9 +2280,10 @@ fn build() -> Vec<CommandSpec> {
return Err(bad("timeline.place", "enable video, audio, or both"));
}
let item = item_p(p, "item").ok_or_else(|| bad("timeline.place", "need `item`"))?;
let at = time_p(s, p, "").unwrap_or(s.playhead());
let at = sequence_time_p(s, p, "", "timeline.place")?.unwrap_or(s.playhead());
let tg = s.targeting();
let pi = s.project.item(item).ok_or_else(|| bad("timeline.place", "no such item"))?;
let (has_video, has_audio) = s.project.source_streams(item);
let is_audio = |t: TrackId| s.active_sequence().is_some_and(|q| q.audio_tracks.iter().any(|x| x.id == t));
// `track` is where the picture goes and `audioTrack` where the sound goes. A caller that
// names an audio track as `track` asks for the sound alone, on that track.
@ -2211,21 +2292,21 @@ fn build() -> Vec<CommandSpec> {
return Err(bad("timeline.place", "`track` names an audio track and `audioTrack` is given as well: name the sound's track once"));
}
(_, Some(t)) if !is_audio(t) => return Err(bad("timeline.place", "`audioTrack` names a video track")),
(Some(t), None) if is_audio(t) && !pi.has_audio() => {
(Some(t), None) if is_audio(t) && !has_audio => {
return Err(bad("timeline.place", "`track` names an audio track and this item has no sound"));
}
(Some(t), None) if is_audio(t) => (None, Some(t)),
(Some(_), None) if !pi.has_video() => {
(Some(_), None) if !has_video => {
return Err(bad("timeline.place", "`track` names a video track and this item has no picture: name an audio track"));
}
(v, a) => (v.or(tg.video_dest), a.or(tg.audio_dest)),
};
let v = v.filter(|_| video);
let a = a.filter(|_| audio);
if p.get("video").is_some() && video && pi.has_video() && v.is_none() {
if p.get("video").is_some() && video && has_video && v.is_none() {
return Err(bad("timeline.place", "no video destination: add or enable a video track"));
}
if p.get("audio").is_some() && audio && pi.has_audio() && a.is_none() {
if p.get("audio").is_some() && audio && has_audio && a.is_none() {
return Err(bad("timeline.place", "no audio destination: add or enable an audio track"));
}
let full = full_duration(s, pi);
@ -2302,6 +2383,9 @@ fn build() -> Vec<CommandSpec> {
.unwrap_or_default();
let clips = with_links(s, &[c]);
let applied = s.edit_sequence(if mode == TrimMode::Ripple { "Ripple Trim" } else { "Trim" }, |q, ctx, st| {
if q.find_item(c).is_some_and(|(t, _)| q.track(t).is_some_and(|tr| tr.locked)) {
return Err(edit::EditError::Locked.into());
}
let before = q.find_item(c).map(|(_, i)| i.range());
// clamp across all linked partners, then apply the common delta
let mut dd = clamp_trim_linked(q, &clips, edge, mode, d, ctx)?;
@ -2309,6 +2393,10 @@ fn build() -> Vec<CommandSpec> {
// the clip and its linked partners ripple as one edit
let mut group = vec![c];
group.extend(clips.iter().copied().filter(|o| *o != c));
group.retain(|id| q.find_item(*id).is_some_and(|(t, _)| q.track(t).is_some_and(|tr| !tr.locked)));
if group.is_empty() {
return Err(edit::EditError::Locked.into());
}
dd = edit::ripple_trim_group(q, &group, edge, dd, ctx)?;
if st.ripple_sequence_markers
&& let Some(r) = before
@ -2322,7 +2410,11 @@ fn build() -> Vec<CommandSpec> {
}
} else {
for c in &clips {
edit::trim(q, *c, edge, mode, dd, ctx)?;
match edit::trim(q, *c, edge, mode, dd, ctx) {
Err(edit::EditError::Locked) => continue,
Err(e) => return Err(e.into()),
Ok(_) => {}
}
}
}
Ok(dd)
@ -2412,8 +2504,26 @@ fn build() -> Vec<CommandSpec> {
.map(Tick)
.or_else(|| p.get("deltaFrames").and_then(Value::as_i64).map(|f| s.sequence_rate().tick_of(f)))
.unwrap_or_default();
let x = s.edit_sequence("Rolling Edit", |q, ctx, _| Ok(edit::roll(q, l, r, d, ctx)?))?;
Ok(json!({"delta": x.0}))
let follow = s.state.linked_selection;
let (x, n, linked) = s.edit_sequence("Rolling Edit", |q, ctx, _| {
let pairs = linked_roll_pairs(q, l, r, follow);
let linked = pairs.len() > 1;
let mut d = d;
for &(pl, pr) in &pairs {
let mut probe = q.clone();
let x = edit::roll(&mut probe, pl, pr, d, ctx)?;
if x.abs() < d.abs() {
d = x;
}
}
if d != Tick::ZERO {
for &(pl, pr) in &pairs {
edit::roll(q, pl, pr, d, ctx)?;
}
}
Ok((d, pairs.len(), linked))
})?;
Ok(json!({"delta": x.0, "pairs": n, "linked": linked}))
}),
cmd!("timeline.slip", "Slip", [], None, r#"{"clip":id,"delta":ticks|"deltaFrames":i64}"#, has_seq, |s, p| {
let c = clip_p(p, "clip").ok_or_else(|| bad("timeline.slip", "need `clip`"))?;
@ -2441,7 +2551,14 @@ fn build() -> Vec<CommandSpec> {
.unwrap_or_default();
let clips = with_links(s, &[c]);
// clamp across all linked partners, then slide them by the common delta
s.edit_sequence("Slide", |q, ctx, _| Ok(edit::slide_items(q, &clips, d, ctx)?))?;
s.edit_sequence("Slide", |q, ctx, _| {
let unlocked: Vec<ClipId> =
clips.iter().copied().filter(|id| q.find_item(*id).is_some_and(|(t, _)| q.track(t).is_some_and(|tr| !tr.locked))).collect();
if unlocked.is_empty() {
return Err(edit::EditError::Locked.into());
}
Ok(edit::slide_items(q, &unlocked, d, ctx)?)
})?;
Ok(Value::Null)
}),
cmd!("timeline.rateStretch", "Rate Stretch", [], None, r#"{"clip":id,"edge":"in|out","delta":ticks}"#, has_seq, |s, p| {
@ -2456,8 +2573,19 @@ fn build() -> Vec<CommandSpec> {
let clips = with_links(s, &[c]);
let sp = s.edit_sequence("Rate Stretch", |q, ctx, _| {
let mut sp = 1.0;
let mut any = false;
for c in &clips {
sp = edit::rate_stretch(q, *c, edge, d, ctx)?;
match edit::rate_stretch(q, *c, edge, d, ctx) {
Err(edit::EditError::Locked) => continue,
Err(e) => return Err(e.into()),
Ok(s) => {
any = true;
sp = s;
}
}
}
if !any {
return Err(edit::EditError::Locked.into());
}
Ok(sp)
})?;
@ -2600,7 +2728,7 @@ fn build() -> Vec<CommandSpec> {
"Set Effect Parameter",
[],
None,
r##"{"clip":id,"effect":"motion"|index,"param":str,"mask":n?,"value":num|[x,y]|"#rrggbb"|bool|path|"option name" (a choice takes its index or its name),"time":ticks? (where the keyframe goes when the parameter is animated; a parameter that is not animated gets a new static value whatever the time),"keyframe":bool? (true: write a keyframe at `time` and animate the parameter),"merge":bool?,"begin":bool?}"##,
r##"{"clip":id,"effect":"motion"|index,"param":str,"mask":n?,"value":num|[x,y]|"#rrggbb"|bool|path|"option name" (a choice takes its index or its name),"time":ticks? (where the keyframe goes when the parameter is animated, kept there even outside the clip; default: the playhead, pinned to the clip; a parameter that is not animated gets a new static value whatever the time),"keyframe":bool? (true: write a keyframe at `time` and animate the parameter),"merge":bool?,"begin":bool?}"##,
has_seq,
|s, p| {
let c = clip_p(p, "clip").ok_or_else(|| bad("effects.setParam", "need `clip`"))?;
@ -2608,8 +2736,10 @@ fn build() -> Vec<CommandSpec> {
let val = p.get("value").cloned().ok_or_else(|| bad("effects.setParam", "need `value`"))?;
let eff = p.get("effect").cloned().unwrap_or(json!("motion"));
let keyframe = bool_p(p, "keyframe").unwrap_or(false);
// an explicit `time` outside the clip keeps its keyframe there, before the start as after
// the end, so a ramp across a cut keeps its slope (#1055); the playhead is pinned to the clip
let at = time_p(s, p, "");
let ph = s.playhead();
let tl = time_p(s, p, "").unwrap_or(ph);
let pq = p.clone();
// `merge`: a drag of one parameter is one undo step (#201); `begin` starts a new one
let merge = bool_p(p, "merge").unwrap_or(false).then(|| format!("setParam:{}:{eff}:{pid}:{}", c.0, p.get("mask").unwrap_or(&Value::Null)));
@ -2618,7 +2748,7 @@ fn build() -> Vec<CommandSpec> {
}
let keyframes = s.edit_sequence_as("Change Effect Parameter", merge.as_deref(), |q, _, _| {
let (_, it) = q.find_item_mut(c).ok_or(filmcraft_edit::EditError::NoItem(c))?;
let mt = it.source_time_at(tl.clamp(it.start, (it.end() - Tick(1)).max(it.start)));
let mt = it.source_time_at(at.unwrap_or_else(|| ph.clamp(it.start, (it.end() - Tick(1)).max(it.start))));
let e = match &eff {
Value::Number(n) => it.effects.get_mut(n.as_u64().unwrap_or(0) as usize),
Value::String(sid) => it.effects.iter_mut().find(|e| &e.effect == sid),
@ -2820,20 +2950,48 @@ fn build() -> Vec<CommandSpec> {
}
items.sort_by_key(|i| i.0);
items.dedup();
let mut used = std::collections::HashSet::new();
for it in s.project.sequences() {
if let Some(q) = it.as_sequence() {
for t in q.all_tracks() {
used.extend(t.items.iter().map(|x| x.item));
}
}
}
let mut gone = Vec::new();
let mut offlined = Vec::new();
for i in items {
let keep = used.contains(&i)
&& s.project.item(i).and_then(filmcraft_project::ProjectItem::as_media).is_some_and(|m| matches!(m.media, MediaRef::File { .. }));
if keep {
offlined.push(i);
} else {
gone.push(i);
}
}
s.edit("Clear", |pr, st| {
for i in &offlined {
if let Some(m) = pr.item_mut(*i).and_then(filmcraft_project::ProjectItem::as_media_mut) {
m.offline = true;
}
}
for b in &bins {
pr.root.remove_bin(*b);
}
for i in &items {
for i in &offlined {
if pr.root.parent_of(*i).is_none() {
pr.root.children.push(filmcraft_project::BinEntry::Item(*i));
}
}
for i in &gone {
pr.items.remove(i);
pr.root.remove_item(*i);
}
// remove track items that referenced deleted media
let seq_ids: Vec<ItemId> = pr.sequences().map(|q| q.id).collect();
for sid in seq_ids {
if let Some(q) = pr.sequence_mut(sid) {
for t in q.all_tracks_mut() {
t.items.retain(|x| !items.contains(&x.item));
t.items.retain(|x| !gone.contains(&x.item));
filmcraft_edit::remove_orphan_transitions(t);
}
}
@ -2841,11 +2999,12 @@ fn build() -> Vec<CommandSpec> {
st.project_selection.clear();
Ok(())
})?;
for i in &items {
for i in gone.iter().chain(&offlined) {
s.media.remove(*i);
}
crate::relink::refresh(s);
s.fix_state();
Ok(json!({"items": items.len(), "bins": bins.len()}))
Ok(json!({"items": gone.len(), "bins": bins.len(), "offline": offlined.len()}))
}),
cmd!("project.moveToBin", "Move to Bin", [], None, r#"{"items":[id|binId]?,"bin":binId|null}"#, always, |s, p| {
let items: Vec<ItemId> = match p.get("items").and_then(Value::as_array) {

6
crates/engine/src/explicit_targets_tests.rs
View File

@ -234,15 +234,15 @@ fn project_delete_takes_explicit_bins_and_items_without_a_selection() {
assert_eq!(disabled(s.execute("project.delete", json!({}))), "select an item in the Project panel");
// an empty bin, by its id alone
let r = s.execute("project.delete", json!({"items": [empty]})).unwrap();
assert_eq!(r, json!({"items": 0, "bins": 1}));
assert_eq!(r, json!({"items": 0, "bins": 1, "offline": 0}));
assert!(s.project.root.find_bin(filmcraft_project::BinId(empty)).is_none());
// a bin with a clip in it, and an item, each by id alone
let r = s.execute("project.delete", json!({"items": [outer]})).unwrap();
assert_eq!(r, json!({"items": 1, "bins": 1}));
assert_eq!(r, json!({"items": 1, "bins": 1, "offline": 0}));
assert!(s.project.item(ocean).is_none());
let dunes = item_named(&s, "Desert_Dunes.mp4");
let r = s.execute("project.delete", json!({"items": [dunes.0]})).unwrap();
assert_eq!(r, json!({"items": 1, "bins": 0}));
assert_eq!(r, json!({"items": 1, "bins": 0, "offline": 0}));
assert_eq!(s.project.items.len(), before - 2);
assert!(s.state.project_selection.is_empty(), "the selection is left as it was");
assert!(!s.is_enabled("project.delete"), "menu enablement still follows the selection");

22
crates/engine/src/export_tools.rs
View File

@ -440,8 +440,9 @@ pub fn spawn_export(s: &mut Session, project: Arc<Project>, seq: ItemId, mut set
let id = s.jobs.iter().map(|j| j.id).max().unwrap_or(0) + 1;
let job = crate::Job { id, label, progress: Default::default(), result: Default::default() };
// Export always renders full-resolution media, whatever the proxy toggle says.
let provider = s.media.full_res_provider(project.clone(), s.services.clone());
let provider = s.media.full_res_provider(project.clone(), s.services.clone()).recording();
let (prog, res) = (job.progress.clone(), job.result.clone());
let offline = s.export_offline.entry(id).or_default().clone();
if cfg!(target_arch = "wasm32") && !wait && filmcraft_export::stepped(format) {
// No threads: the host advances the export between UI frames (`Session::pump_jobs`).
let mut exporter = filmcraft_export::Exporter::new(project, seq, &settings, &prog).map_err(|e| EngineError::Other(e.to_string()))?;
@ -452,6 +453,7 @@ pub fn spawn_export(s: &mut Session, project: Arc<Project>, seq: ItemId, mut set
}
let run = move || {
let r = filmcraft_export::export(&project, seq, &settings, &provider, &prog).map_err(|e| e.to_string());
*offline.lock().unwrap_or_else(|x| x.into_inner()) = provider.offline_used();
if let Err(e) = &r {
*prog.error.lock().unwrap_or_else(|x| x.into_inner()) = Some(e.clone());
prog.finished.store(true, Ordering::Relaxed);
@ -514,6 +516,7 @@ pub fn export_media(s: &mut Session, p: &Value) -> Result<Value> {
return Err(EngineError::Other(format!("export failed: {e}")));
}
out["result"] = v["result"].clone();
out["offlineMedia"] = offline_media_json(s, id);
}
Ok(out)
}
@ -568,6 +571,22 @@ impl ExportQueue {
}
}
/// A `jobs.list` entry: [`crate::Job::to_json`], plus `offlineMedia` for an export job, so callers
/// that poll an export (the bridge, MCP long exports) can report it like `file.exportMedia` does.
pub(crate) fn job_json(s: &Session, j: &crate::Job) -> Value {
let mut v = j.to_json();
if s.export_offline.contains_key(&j.id) {
v["offlineMedia"] = offline_media_json(s, j.id);
}
v
}
/// The offline items export job `job` drew the slate for, as JSON (`[]` while it runs or when none).
fn offline_media_json(s: &Session, job: u64) -> Value {
let list = s.export_offline.get(&job).map(|o| o.lock().unwrap_or_else(|e| e.into_inner()).clone()).unwrap_or_default();
serde_json::to_value(list).unwrap_or_else(|_| json!([]))
}
fn item_json(s: &Session, it: &QueueItem) -> Value {
let job = it.job.and_then(|id| s.jobs.iter().find(|j| j.id == id));
let (progress, status_text) = match job {
@ -588,6 +607,7 @@ fn item_json(s: &Session, it: &QueueItem) -> Value {
"statusText": status_text,
"job": it.job,
"error": it.error,
"offlineMedia": it.job.map_or_else(|| json!([]), |j| offline_media_json(s, j)),
})
}

23
crates/engine/src/lib.rs
View File

@ -357,6 +357,9 @@ pub struct Session {
pub journal: Vec<(String, Value)>,
/// Background jobs (exports, render previews…).
pub jobs: Vec<Job>,
/// Export job → the offline items it drew the slate for, filled when the export finishes
/// (#1054; `offlineMedia` in `file.exportMedia` and queue results).
pub export_offline: std::collections::HashMap<u64, Arc<std::sync::Mutex<Vec<media_pool::OfflineMedia>>>>,
/// User preferences (`prefs.*` commands) and where they persist (None = not persisted).
pub prefs: autosave::Preferences,
pub prefs_path: Option<std::path::PathBuf>,
@ -476,6 +479,7 @@ impl Session {
events: Vec::new(),
journal: Vec::new(),
jobs: Vec::new(),
export_offline: Default::default(),
prefs: Default::default(),
prefs_path: None,
persistence: None,
@ -1024,11 +1028,22 @@ impl Session {
/// [`Session::render_program_at`], with the reason when nothing can be rendered.
pub fn try_render_program_at(&self, scale: f32, t: Tick) -> Result<filmcraft_render::Image> {
self.render_program_with(scale, t, &self.media.provider(self.project.clone(), self.services.clone()))
}
/// [`Session::try_render_program_at`], also naming the offline items the frame drew the slate
/// for (#1054).
pub fn try_render_program_with_offline_at(&self, scale: f32, t: Tick) -> Result<(filmcraft_render::Image, Vec<media_pool::OfflineMedia>)> {
let provider = self.media.provider(self.project.clone(), self.services.clone()).recording();
let img = self.render_program_with(scale, t, &provider)?;
Ok((img, provider.offline_used()))
}
fn render_program_with(&self, scale: f32, t: Tick, provider: &dyn filmcraft_render::SourceProvider) -> Result<filmcraft_render::Image> {
let seq = self.renderable_sequence(scale)?;
let t = self.sequence_rate().snap(t.max(Tick::ZERO));
let provider = self.media.provider(self.project.clone(), self.services.clone());
let opts = filmcraft_render::RenderOptions { scale, captions: true, ..Default::default() };
Ok(filmcraft_render::render_sequence(&self.project, seq, t, opts, &provider))
Ok(filmcraft_render::render_sequence(&self.project, seq, t, opts, provider))
}
/// The active sequence, if a frame of it at `scale` is within the image size limits.
@ -1114,6 +1129,8 @@ mod keyboard_tests;
#[cfg(test)]
mod linked_speed_tests;
#[cfg(test)]
mod locked_clip_edits_tests;
#[cfg(test)]
mod masks_tests;
#[cfg(test)]
mod media_browser_tests;
@ -1156,6 +1173,8 @@ mod remix_tests;
#[cfg(test)]
mod ripple_delete_tests;
#[cfg(test)]
mod roll_linked_tests;
#[cfg(test)]
mod scopes_tests;
#[cfg(test)]
mod sequence_inspect_tests;

84
crates/engine/src/locked_clip_edits_tests.rs Normal file
View File

@ -0,0 +1,84 @@
//! #911: a locked track refuses timing and source-range edits the way a locked destination
//! refuses `timeline.move`.
use super::*;
use filmcraft_project::ClipId;
use serde_json::json;
fn fixture() -> (Session, u64) {
let mut s = Session::default();
s.execute("file.newProject", json!({"name": "Lock check"})).unwrap();
s.execute("file.newSequence", json!({"name": "Sequence", "fps": 25, "video": 1, "audio": 0, "width": 16, "height": 16})).unwrap();
s.execute("file.newOfflineFile", json!({"name": "Media", "seconds": 4, "fps": 25, "video": true, "audio": false, "width": 16, "height": 16})).unwrap();
let item = s.project.items.values().find(|i| i.as_media().is_some()).map(|i| i.id.0).unwrap();
s.execute("timeline.place", json!({"item": item, "track": "V1", "time": 0, "sourceIn": 0, "duration": 1_016_064_000_000i64})).unwrap();
let clip = s.active_sequence().unwrap().video_tracks[0].items[0].id.0;
(s, clip)
}
fn v1_clip(s: &Session, id: u64) -> filmcraft_project::TrackItem {
s.active_sequence().unwrap().find_item(ClipId(id)).unwrap().1.clone()
}
/// Place a 100-frame clip, lock V1, trim Out by −20 frames: the command errors and duration stays 100.
#[test]
fn locked_track_refuses_trim_out_and_keeps_duration() {
let (mut s, clip) = fixture();
let before = v1_clip(&s, clip);
assert_eq!(s.sequence_rate().frame_at(before.duration), 100);
s.execute("timeline.setTrack", json!({"track": "V1", "locked": true})).unwrap();
let err = s.execute("timeline.trim", json!({"clip": clip, "edge": "out", "mode": "regular", "deltaFrames": -20})).unwrap_err();
assert!(err.to_string().contains("locked"), "{err}");
let after = v1_clip(&s, clip);
assert_eq!(after.duration, before.duration);
assert_eq!(after.start, before.start);
assert_eq!(after.source_in, before.source_in);
assert_eq!(after.speed, before.speed);
}
#[test]
fn unlocked_track_still_trims_out() {
let (mut s, clip) = fixture();
s.execute("timeline.trim", json!({"clip": clip, "edge": "out", "mode": "regular", "deltaFrames": -20})).unwrap();
assert_eq!(s.sequence_rate().frame_at(v1_clip(&s, clip).duration), 80);
}
#[test]
fn locked_track_refuses_move_on_the_same_track() {
let (mut s, clip) = fixture();
let before = v1_clip(&s, clip);
s.execute("timeline.setTrack", json!({"track": "V1", "locked": true})).unwrap();
let err = s.execute("timeline.move", json!({"moves": [{"clip": clip, "track": "V1", "time": 1_016_064_000_000i64}], "linked": false})).unwrap_err();
assert!(err.to_string().contains("locked"), "{err}");
assert_eq!(v1_clip(&s, clip).duration, before.duration);
}
#[test]
fn locked_track_refuses_roll_slip_slide_speed() {
let mut s = Session::default();
s.execute("file.newProject", json!({"name": "Lock check"})).unwrap();
s.execute("file.newSequence", json!({"name": "Sequence", "fps": 25, "video": 1, "audio": 0, "width": 16, "height": 16})).unwrap();
s.execute("file.newOfflineFile", json!({"name": "Media", "seconds": 4, "fps": 25, "video": true, "audio": false, "width": 16, "height": 16})).unwrap();
let item = s.project.items.values().find(|i| i.as_media().is_some()).map(|i| i.id.0).unwrap();
let frame = 10_160_640_000i64;
s.execute("timeline.place", json!({"item": item, "track": "V1", "time": 0, "sourceIn": frame * 10, "duration": frame * 60})).unwrap();
s.execute("timeline.place", json!({"item": item, "track": "V1", "time": frame * 60, "sourceIn": 0, "duration": frame * 100})).unwrap();
let (left, right) = {
let q = s.active_sequence().unwrap();
(q.video_tracks[0].items[0].id.0, q.video_tracks[0].items[1].id.0)
};
s.execute("timeline.setTrack", json!({"track": "V1", "locked": true})).unwrap();
let snap = s.project.clone();
for cmd in [
json!({"id": "timeline.roll", "p": {"left": left, "right": right, "deltaFrames": 10}}),
json!({"id": "timeline.slip", "p": {"clip": left, "deltaFrames": -10}}),
json!({"id": "timeline.slide", "p": {"clip": left, "deltaFrames": 10}}),
json!({"id": "timeline.rateStretch", "p": {"clip": left, "edge": "out", "deltaFrames": 20}}),
json!({"id": "clip.speedDuration", "p": {"clips": [left], "speed": 200}}),
json!({"id": "timeline.trim", "p": {"clip": left, "edge": "in", "mode": "regular", "deltaFrames": 20}}),
] {
let err = s.execute(cmd["id"].as_str().unwrap(), cmd["p"].clone()).unwrap_err();
assert!(err.to_string().contains("locked"), "{}: {err}", cmd["id"]);
assert_eq!(*s.project, *snap, "{} must leave the sequence unchanged", cmd["id"]);
}
}

9
crates/engine/src/masks.rs
View File

@ -91,9 +91,12 @@ pub(crate) fn commands() -> Vec<CommandSpec> {
/// A mask path from JSON: the serialized [`MaskPath`] object, an array of vertices
/// (`{"p":[x,y],"in":[x,y]?,"out":[x,y]?}`), or an array of `[x, y]` / `{"x","y"}` points.
pub fn path_from_json(v: &Value) -> Option<MaskPath> {
if let Ok(p) = serde_json::from_value::<MaskPath>(v.clone()) {
return Some(p);
}
// The vertex forms first: serde also accepts `{"p","in","out"}` vertices (the `masks.list` /
// `masks.add` form) but silently drops the unknown `in` / `out` tangents.
vertex_path(v).or_else(|| serde_json::from_value::<MaskPath>(v.clone()).ok())
}
fn vertex_path(v: &Value) -> Option<MaskPath> {
let arr = v.as_array().or_else(|| v.get("vertices").and_then(Value::as_array))?;
let closed = v.get("closed").and_then(Value::as_bool).unwrap_or(true);
let mut vertices = Vec::new();

19
crates/engine/src/masks_tests.rs
View File

@ -380,3 +380,22 @@ fn rounding_an_animated_mask_rounds_every_keyframe_and_only_the_chosen_corners()
assert_eq!(v.len(), 6);
assert!((v[0]["p"][0].as_f64().unwrap() - 450.0).abs() < 1e-6, "{}", v[0]);
}
/// `path_to_json` → `path_from_json` keeps the Bézier tangents (they used to come back as zero, so
/// a pen mask's dragged-out curves were committed as corners; #853).
#[test]
fn path_json_round_trip_keeps_tangents() {
use filmcraft_geom::Vec2;
use filmcraft_project::mask::{MaskPath, MaskVertex};
let path = MaskPath {
vertices: vec![
MaskVertex { p: Vec2::new(10.0, 20.0), t_in: Vec2::new(-5.0, 0.0), t_out: Vec2::new(5.0, 0.0) },
MaskVertex::corner(Vec2::new(80.0, 20.0)),
MaskVertex { p: Vec2::new(40.0, 90.0), t_in: Vec2::new(3.0, -4.0), t_out: Vec2::new(-3.0, 4.0) },
],
closed: true,
};
assert_eq!(crate::masks::path_from_json(&crate::masks::path_to_json(&path)), Some(path.clone()));
// the serialized project form still parses
assert_eq!(crate::masks::path_from_json(&serde_json::to_value(&path).unwrap()), Some(path));
}

52
crates/engine/src/media_pool.rs
View File

@ -12,7 +12,7 @@
//! compositor derives its pixel scale from the frame it gets, so effects and Motion render the
//! same picture at lower resolution. Export asks for [`MediaPool::full_res_provider`].
use std::collections::HashMap;
use std::collections::{BTreeSet, HashMap};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex, RwLock};
@ -32,6 +32,17 @@ pub struct OfflineStatus {
pub error: String,
}
/// An offline item an export or a rendered frame drew the slate for (#1054). `status` is the
/// word `media.status` uses: `missing`, `offline` (made offline) or `unreadable`.
#[derive(Clone, Debug, PartialEq, serde::Serialize)]
pub struct OfflineMedia {
pub item: u64,
pub name: String,
pub path: String,
pub status: &'static str,
pub error: String,
}
pub struct MediaPool {
/// item → (reference key, source)
sources: RwLock<HashMap<ItemId, (String, SharedSource)>>,
@ -316,12 +327,12 @@ impl MediaPool {
/// A render-side provider bound to a project snapshot (proxies when enabled).
pub fn provider(self: &Arc<Self>, project: Arc<Project>, services: Arc<dyn Services>) -> PoolProvider {
PoolProvider { pool: self.clone(), project, services, full_res: false }
PoolProvider { pool: self.clone(), project, services, full_res: false, used: None }
}
/// A provider that always reads full-resolution media (export).
pub fn full_res_provider(self: &Arc<Self>, project: Arc<Project>, services: Arc<dyn Services>) -> PoolProvider {
PoolProvider { pool: self.clone(), project, services, full_res: true }
PoolProvider { pool: self.clone(), project, services, full_res: true, used: None }
}
}
@ -332,10 +343,45 @@ pub struct PoolProvider {
pub services: Arc<dyn Services>,
/// Ignore proxies (export always renders full resolution).
pub full_res: bool,
/// Items asked for so far, when recording ([`PoolProvider::recording`]).
used: Option<Mutex<BTreeSet<ItemId>>>,
}
impl PoolProvider {
/// This provider, remembering the items it is asked for so [`PoolProvider::offline_used`] can
/// tell which of them rendered offline.
pub fn recording(mut self) -> Self {
self.used = Some(Mutex::new(BTreeSet::new()));
self
}
/// The offline items among those asked for since [`PoolProvider::recording`], once each, in
/// item order (subclips count as their media). Empty when not recording.
pub fn offline_used(&self) -> Vec<OfflineMedia> {
let Some(used) = &self.used else { return Vec::new() };
let used = used.lock().unwrap_or_else(|e| e.into_inner()).clone();
let media: BTreeSet<ItemId> = used.into_iter().filter_map(|i| self.project.resolve_media(i).map(|(id, _, _)| id)).collect();
media
.into_iter()
.filter_map(|id| {
let st = self.pool.offline_status(id)?;
let status = match st.reason {
OfflineReason::Missing => "missing",
OfflineReason::MadeOffline => "offline",
OfflineReason::Unreadable => "unreadable",
};
let name = self.project.item(id).map(|i| i.name.clone()).unwrap_or_default();
Some(OfflineMedia { item: id.0, name, path: st.path, status, error: st.error })
})
.collect()
}
}
impl filmcraft_render::SourceProvider for PoolProvider {
fn source(&self, item: ItemId) -> Option<SharedSource> {
if let Some(used) = &self.used {
used.lock().unwrap_or_else(|e| e.into_inner()).insert(item);
}
if self.full_res { self.pool.full_res_source(&self.project, item, &*self.services) } else { self.pool.source_for(&self.project, item, &*self.services) }
}
}

8
crates/engine/src/mixer_tests.rs
View File

@ -520,3 +520,11 @@ fn surround_sequence_settings_and_panner_lanes() {
assert!(s.execute("file.newSequence", json!({"mix": "7.1"})).is_err());
s.execute("file.newSequence", json!({"trackType": "Standard"})).unwrap();
}
#[test]
fn audio_transition_schema_lists_edge_params_and_reverse() {
let params = find_command("sequence.applyAudioTransition").unwrap().params;
for key in ["\"edge\"", "\"params\"", "\"reverse\"", "constant_power"] {
assert!(params.contains(key), "schema `{params}` is missing {key}");
}
}

3
crates/engine/src/project_tools.rs
View File

@ -14,6 +14,7 @@
//! | `file.projectSettings.general` / `.scratchDisks` | File ▸ Project Settings ▸ General… / Scratch Disks… |
//! | `edit.find` / `edit.findNext` | Edit ▸ Find… (⌘F) / Find Next |
//! | `edit.editOriginal` | Edit ▸ Edit Original (⌘E): the frontend opens the files |
//! | `file.reveal` | Project panel ▸ Reveal in Finder / Show in Explorer: the frontend reveals the media file |
//! | `clip.editOffline` | Clip ▸ Edit Offline… |
//! | `clip.sourceSettings` | Clip ▸ Source Settings… |
//! | `clip.restoreCaptionsFromSource` | Clip ▸ Restore Captions from Source Clip (always disabled: no embedded caption reader) |
@ -163,6 +164,8 @@ pub(crate) fn commands() -> Vec<CommandSpec> {
spec("edit.find", "Find…", &["Edit"], Some("Cmd+F"), FIND_PARAMS, always, find),
spec("edit.findNext", "Find Next", &["Edit"], None, "{}", has_find, find_next),
spec("edit.editOriginal", "Edit Original", &["Edit"], Some("Cmd+E"), r#"{"items":[id]?}"#, has_original, edit_original),
// Project panel context menu (not in the File menu): reveal the clip's media file.
query("file.reveal", "Reveal in Finder", &[], None, r#"{"items":[id]?}"#, has_file_selection, edit_original),
// ---------------- Clip ----------------
spec(
"clip.editOffline",

46
crates/engine/src/project_tools_tests.rs
View File

@ -361,6 +361,18 @@ fn edit_original_update_metadata_and_clip_items() {
assert!(std::fs::read_to_string(&side).unwrap().contains("someone else"));
}
#[test]
fn file_reveal_enabled_for_imported_clip() {
let dir = tmp_dir("reveal");
let path = dir.join("take.mp4");
make_movie(&path, DemoScene::Aurora, 64, 36, 8);
let (mut s, items, _) = session_with(&[&path]);
assert!(!s.is_enabled("file.reveal"), "disabled with no Project panel selection");
s.execute("project.select", json!({"items": [items[0].0]})).unwrap();
assert!(s.is_enabled("file.reveal"));
assert_eq!(s.execute("file.reveal", json!({})).unwrap()["paths"], json!([path.to_string_lossy()]));
}
#[test]
fn edit_offline_sets_the_log_fields() {
let mut s = demo();
@ -469,13 +481,13 @@ fn clear_deletes_a_bin_with_everything_in_it() {
select(&mut s);
// an empty bin, by its id: gone, nothing else touched
let r = s.execute("project.delete", json!({"items": [empty]})).unwrap();
assert_eq!(r, json!({"items": 0, "bins": 1}));
assert_eq!(r, json!({"items": 0, "bins": 1, "offline": 0}));
assert!(s.project.root.find_bin(filmcraft_project::BinId(empty)).is_none());
assert_eq!(s.project.items.len(), before);
// a bin holding a sub-bin holding a clip: all three go
select(&mut s);
let r = s.execute("project.delete", json!({"items": [outer]})).unwrap();
assert_eq!(r, json!({"items": 1, "bins": 1}));
assert_eq!(r, json!({"items": 1, "bins": 1, "offline": 0}));
assert!(s.project.root.find_bin(filmcraft_project::BinId(outer)).is_none());
assert!(s.project.root.find_bin(filmcraft_project::BinId(inner)).is_none());
assert!(s.project.item(ocean).is_none());
@ -487,7 +499,7 @@ fn clear_deletes_a_bin_with_everything_in_it() {
// the clip, its bin and the bin around it all named: each counted once
select(&mut s);
let r = s.execute("project.delete", json!({"items": [ocean.0, inner, outer]})).unwrap();
assert_eq!(r, json!({"items": 1, "bins": 1}));
assert_eq!(r, json!({"items": 1, "bins": 1, "offline": 0}));
assert_eq!(s.project.items.len(), before - 1);
s.execute("edit.undo", json!({})).unwrap();
// the project's own top bin is never cleared
@ -496,3 +508,31 @@ fn clear_deletes_a_bin_with_everything_in_it() {
s.execute("project.delete", json!({"items": [root]})).unwrap();
assert_eq!(s.project.items.len(), before);
}
#[test]
fn deleting_media_used_on_the_timeline_makes_it_offline() {
let mut s = Session::default();
s.execute("file.newProject", json!({"name": "Offline"})).unwrap();
s.execute("file.newSequence", json!({"name": "Seq", "fps": 25, "video": 1, "audio": 0, "width": 16, "height": 16})).unwrap();
let item = s
.execute("file.newOfflineFile", json!({"name": "Clip", "seconds": 2, "fps": 25, "video": true, "audio": false, "width": 16, "height": 16}))
.unwrap()["item"]
.as_u64()
.unwrap();
s.execute("timeline.place", json!({"item": item, "track": "V1", "time": 0})).unwrap();
let clips_before = s.project.sequences().next().unwrap().as_sequence().unwrap().video_tracks[0].items.len();
let r = s.execute("project.delete", json!({"items": [item]})).unwrap();
assert_eq!(r["offline"], 1);
assert_eq!(r["items"], 0);
assert!(s.project.item(ItemId(item)).unwrap().as_media().unwrap().offline);
assert_eq!(s.project.sequences().next().unwrap().as_sequence().unwrap().video_tracks[0].items.len(), clips_before);
let unused = s
.execute("file.newOfflineFile", json!({"name": "Spare", "seconds": 1, "fps": 25, "video": true, "audio": false, "width": 16, "height": 16}))
.unwrap()["item"]
.as_u64()
.unwrap();
let r = s.execute("project.delete", json!({"items": [unused]})).unwrap();
assert_eq!(r["items"], 1);
assert_eq!(r["offline"], 0);
assert!(s.project.item(ItemId(unused)).is_none());
}

69
crates/engine/src/relink_tests.rs
View File

@ -201,3 +201,72 @@ fn explicit_folder_remaps_normalize_both_prefixes_and_respect_boundaries() {
assert_eq!(apply_remap("C:/shoot/Media/a.mov", r"C:\shoot\Media", r"D:\archive\Media"), Some("D:/archive/Media/a.mov".into()));
assert_eq!(apply_remap("C:/shoot/Media2/a.mov", r"C:\shoot\Media", r"D:\archive\Media"), None);
}
/// #1054: an export (or a rendered frame) that draws the offline slate names the offline items it
/// used, with their status, so a headless caller can tell placeholder frames from real ones. Media
/// outside the exported range is not reported.
#[test]
fn export_names_the_offline_media_it_rendered() {
let root = tmp_dir("relink-export-offline");
let (path, items) = saved_project(&root);
let out = |name: &str| root.join(name).to_string_lossy().into_owned();
let mut s = open(&path);
let r = s
.execute("file.exportMedia", json!({"path": out("on.png"), "format": "png", "range": "custom", "startSeconds": 0, "endSeconds": 0.05, "wait": true}))
.unwrap();
assert_eq!(r["offlineMedia"], json!([]), "{r}");
std::fs::remove_file(root.join("Media/a.mov")).unwrap();
let mut s = open(&path);
// b.mov (frames 12..24) is still there
let r = s
.execute("file.exportMedia", json!({"path": out("b.png"), "format": "png", "range": "custom", "startSeconds": 0.6, "endSeconds": 0.65, "wait": true}))
.unwrap();
assert_eq!(r["offlineMedia"], json!([]), "{r}");
let r = s.execute("file.exportMedia", json!({"path": out("all.wav"), "format": "wav", "wait": true})).unwrap();
let off = r["offlineMedia"].as_array().unwrap();
assert_eq!(off.len(), 1, "{r}");
assert_eq!(off[0]["item"], json!(items[0].0));
assert_eq!(off[0]["name"], json!("a.mov"));
assert_eq!(off[0]["status"], json!("missing"));
assert!(off[0]["path"].as_str().unwrap().ends_with("a.mov"), "{r}");
// made offline on purpose
s.execute("media.makeOffline", json!({"items": [items[1].0]})).unwrap();
let r = s
.execute("file.exportMedia", json!({"path": out("b2.png"), "format": "png", "range": "custom", "startSeconds": 0.6, "endSeconds": 0.65, "wait": true}))
.unwrap();
assert_eq!(
r["offlineMedia"].as_array().map(|a| a.iter().map(|o| (o["item"].clone(), o["status"].clone())).collect::<Vec<_>>()),
Some(vec![(json!(items[1].0), json!("offline"))]),
"{r}"
);
// the queue reports it per item
let r = s
.execute(
"export.queue.add",
json!({"path": out("q.png"), "format": "png", "range": "custom", "startSeconds": 0.6, "endSeconds": 0.65, "start": true, "wait": true}),
)
.unwrap();
assert_eq!(r["items"][0]["offlineMedia"][0]["status"], json!("offline"), "{r}");
// one rendered frame
let (_, off) = s.try_render_program_with_offline_at(0.5, s.sequence_rate().tick_of(3)).unwrap();
assert_eq!(off.iter().map(|o| (o.item, o.status)).collect::<Vec<_>>(), [(items[0].0, "missing")]);
let (_, off) = s.try_render_program_with_offline_at(0.5, s.sequence_rate().tick_of(30)).unwrap();
assert!(off.is_empty(), "nothing on the timeline there: {off:?}");
let _ = std::fs::remove_dir_all(&root);
}
/// #1054: `jobs.list` carries an export job's `offlineMedia`, for callers that poll the job (the
/// bridge and MCP long exports) instead of waiting in `file.exportMedia`.
#[test]
fn jobs_list_reports_an_exports_offline_media() {
let root = tmp_dir("relink-jobs-offline");
let (path, items) = saved_project(&root);
std::fs::remove_file(root.join("Media/b.mov")).unwrap();
let mut s = open(&path);
let out = root.join("all.wav").to_string_lossy().into_owned();
let job = s.execute("file.exportMedia", json!({"path": out, "format": "wav", "wait": true})).unwrap()["job"].clone();
let jobs = s.execute("jobs.list", json!({})).unwrap();
let j = jobs.as_array().unwrap().iter().find(|j| j["id"] == job).unwrap();
assert_eq!(j["offlineMedia"][0]["item"], json!(items[1].0), "{j}");
let _ = std::fs::remove_dir_all(&root);
}

86
crates/engine/src/roll_linked_tests.rs Normal file
View File

@ -0,0 +1,86 @@
//! #855: a rolling edit on a linked A/V cut must move the sound cut with the picture.
use filmcraft_project::ClipId;
use filmcraft_time::Tick;
use serde_json::json;
use crate::Session;
fn range(s: &Session, c: ClipId) -> (Tick, Tick) {
s.active_sequence().unwrap().find_item(c).map(|(_, i)| (i.start, i.end())).unwrap()
}
fn source_in(s: &Session, c: ClipId) -> Tick {
s.active_sequence().unwrap().find_item(c).unwrap().1.source_in
}
/// Two consecutive 48-frame A/V clips (picture on V1, linked sound on A1).
fn two_clips(s: &mut Session) -> [(ClipId, ClipId); 2] {
s.execute("file.openDemoProject", json!({})).unwrap();
s.execute("file.newSequence", json!({"name": "Roll", "video": 1, "audio": 1})).unwrap();
let rate = s.sequence_rate();
let item = s.project.items.values().find(|i| i.has_video() && i.has_audio() && i.as_media().is_some()).map(|i| i.id).unwrap();
for n in 0..2 {
s.execute("timeline.place", json!({"item": item.0, "frame": 48 * n, "sourceIn": rate.tick_of(48).0, "duration": rate.tick_of(48).0})).unwrap();
}
let q = s.active_sequence().unwrap();
[0, 1].map(|n| (q.video_tracks[0].items[n].id, q.audio_tracks[0].items[n].id))
}
/// #855: rolling the V1 cut +12 frames used to leave A1, so export played the second clip's audio
/// under the first clip's picture.
#[test]
fn roll_moves_linked_audio_cut() {
let mut s = Session::default();
let [(v0, a0), (v1, a1)] = two_clips(&mut s);
assert!(s.state.linked_selection);
let rate = s.sequence_rate();
let f = |n: i64| rate.tick_of(n);
let r = s.execute("timeline.roll", json!({"left": v0.0, "right": v1.0, "deltaFrames": 12})).unwrap();
assert_eq!(r["linked"], json!(true), "{r}");
assert_eq!(r["pairs"], json!(2), "{r}");
assert_eq!(range(&s, v0), (f(0), f(60)));
assert_eq!(range(&s, v1), (f(60), f(96)));
assert_eq!(range(&s, a0), (f(0), f(60)), "linked sound must roll with the picture");
assert_eq!(range(&s, a1), (f(60), f(96)));
}
/// Linked selection off (or unlinked clips) rolls only the given pair and says so.
#[test]
fn roll_without_linked_selection_leaves_audio() {
let mut s = Session::default();
let [(v0, a0), (v1, a1)] = two_clips(&mut s);
s.execute("sequence.linkedSelection", json!({"on": false})).unwrap();
let rate = s.sequence_rate();
let f = |n: i64| rate.tick_of(n);
let r = s.execute("timeline.roll", json!({"left": v0.0, "right": v1.0, "deltaFrames": 12})).unwrap();
assert_eq!(r["linked"], json!(false), "{r}: clips were not followed as a linked pair");
assert_eq!(range(&s, v0), (f(0), f(60)));
assert_eq!(range(&s, v1), (f(60), f(96)));
assert_eq!(range(&s, a0), (f(0), f(48)), "unlinked A1 cut stays");
assert_eq!(range(&s, a1), (f(48), f(96)));
}
/// #526/#558: reversed linked clips still keep the media on the unmoved timeline edges.
#[test]
fn roll_linked_reversed_clips_keeps_media_mapping() {
let mut s = Session::default();
let [(v0, a0), (v1, a1)] = two_clips(&mut s);
s.execute("timeline.select", json!({"clips": [v0.0, a0.0, v1.0, a1.0]})).unwrap();
s.execute("clip.speedDuration", json!({"speed": 100, "reverse": true})).unwrap();
let rate = s.sequence_rate();
let f = |n: i64| rate.tick_of(n);
let left_in = source_in(&s, v0);
let right_in = source_in(&s, v1);
s.execute("timeline.roll", json!({"left": v0.0, "right": v1.0, "deltaFrames": 12})).unwrap();
assert_eq!(range(&s, v0), (f(0), f(60)));
assert_eq!(range(&s, a0), (f(0), f(60)));
assert_eq!(range(&s, v1), (f(60), f(96)));
assert_eq!(range(&s, a1), (f(60), f(96)));
// reversed left: source_in moves so the clip's In still shows the same media
assert_eq!(source_in(&s, v0), left_in - f(12));
assert_eq!(source_in(&s, a0), left_in - f(12));
// reversed right: source_in stays so the clip's Out still shows the same media
assert_eq!(source_in(&s, v1), right_in);
assert_eq!(source_in(&s, a1), right_in);
}

6
crates/engine/src/scene_detect.rs
View File

@ -9,8 +9,10 @@
//!
//! - **Apply a cut at each detected cut point**: the clip (and its linked audio) is cut at the
//! first frame of each new shot;
//! - **Create a subclip for each cut**: one subclip per shot, in a new bin "<clip> Scenes";
//! - **Generate clip markers**: a marker on the clip's master clip (media time) at each cut.
//! - **Create a bin of subclips from each detected cut point**: one subclip per shot, in a new
//! bin "<clip> Scenes";
//! - **Create a clip marker at each detected cut point**: a marker on the clip's master clip
//! (media time) at each cut.
use std::sync::{Arc, Mutex};

4
crates/engine/src/sequence_extras.rs
View File

@ -20,7 +20,7 @@ use filmcraft_project::{ClipId, ItemKind, Label, Marker, MarkerId, MarkerKind, S
use filmcraft_time::{Tick, TimeRange};
use serde_json::{Value, json};
use crate::commands::{CommandSpec, bad, bool_p, f64_p, has_seq, str_p, time_p, u64_p};
use crate::commands::{CommandSpec, bad, bool_p, f64_p, has_seq, sequence_time_p, str_p, u64_p};
use crate::{EngineError, Result, Session};
type Run = fn(&mut Session, &Value) -> Result<Value>;
@ -483,7 +483,7 @@ fn show_active_only(s: &mut Session, p: &Value) -> Result<Value> {
// ---------------------------------------------------------------------------------------------
fn add_flash_cue(s: &mut Session, p: &Value) -> Result<Value> {
let t = s.sequence_rate().snap(time_p(s, p, "").unwrap_or(s.playhead()));
let t = s.sequence_rate().snap(sequence_time_p(s, p, "", "markers.addFlashCue")?.unwrap_or(s.playhead()));
let n = s.active_sequence().map(|q| q.markers.iter().filter(|m| m.kind == MarkerKind::FlashCue).count()).unwrap_or(0) + 1;
let name = str_p(p, "name").map(str::to_string).unwrap_or_else(|| format!("Cue Point {n}"));
let comment = str_p(p, "comment").unwrap_or("").to_string();

13
crates/engine/src/sequence_tools.rs
View File

@ -31,7 +31,7 @@ use filmcraft_project::{AudioChannels, ClipId, ItemId, Label, Marker, MarkerId,
use filmcraft_time::{Tick, TimeRange};
use serde_json::{Value, json};
use crate::commands::{CommandSpec, always, bad, bool_p, clips_p, has_seq, str_p, time_p, track_p, with_links};
use crate::commands::{CommandSpec, always, bad, bool_p, clips_p, has_seq, sequence_time_p, str_p, time_p, track_p, with_links};
use crate::{EngineError, Result, Session};
type Run = fn(&mut Session, &Value) -> Result<Value>;
@ -206,7 +206,7 @@ fn groups() -> Vec<(&'static str, Vec<CommandSpec>)> {
r#"{"time":ticks?,"name":str?,"comment":str?,"color":label?}"#,
has_seq,
|s, p| {
let t = time_p(s, p, "").unwrap_or(s.playhead());
let t = sequence_time_p(s, p, "", "markers.addChapter")?.unwrap_or(s.playhead());
let n = s.active_sequence().map(|q| q.markers.iter().filter(|m| m.kind == MarkerKind::Chapter).count()).unwrap_or(0) + 1;
let name = str_p(p, "name").map(str::to_string).unwrap_or_else(|| format!("Chapter {n}"));
let id = add_marker(s, "Add Chapter Marker", t, Tick::ZERO, MarkerKind::Chapter, name, p)?;
@ -735,11 +735,12 @@ pub(crate) fn wants_new_track(p: &Value) -> bool {
pub(crate) fn place_on_new_tracks(s: &mut Session, p: &Value) -> Result<Value> {
let cmd = "timeline.place";
let item = crate::commands::item_p(p, "item").ok_or_else(|| bad(cmd, "need `item`"))?;
let pi = s.project.item(item).ok_or_else(|| bad(cmd, "no such item"))?;
s.project.item(item).ok_or_else(|| bad(cmd, "no such item"))?;
let (video, audio) = s.project.source_streams(item);
let is_new = |k: &str| p.get(k).and_then(Value::as_str).is_some_and(|v| v.eq_ignore_ascii_case("new"));
let enabled = |k: &str| p.get(k).and_then(Value::as_bool).unwrap_or(true);
let new_v = is_new("track") && pi.has_video() && enabled("video");
let new_a = is_new("audioTrack") && pi.has_audio() && enabled("audio");
let new_v = is_new("track") && video && enabled("video");
let new_a = is_new("audioTrack") && audio && enabled("audio");
if !new_v && !new_a {
return Err(bad(cmd, "no new track to add: the item has no picture or sound for it"));
}
@ -977,7 +978,7 @@ fn add_marker(s: &mut Session, label: &str, t: Tick, dur: Tick, kind: MarkerKind
/// `duration` say otherwise) at the playhead.
fn add_range_marker(s: &mut Session, p: &Value) -> Result<Value> {
let rate = s.sequence_rate();
let t = time_p(s, p, "").unwrap_or(s.playhead());
let t = sequence_time_p(s, p, "", "markers.addRange")?.unwrap_or(s.playhead());
let dur = p
.get("duration")
.and_then(Value::as_i64)

3
crates/engine/src/settings.rs
View File

@ -34,7 +34,7 @@ pub const PREFS_VERSION: u32 = 2;
#[serde(rename_all = "camelCase", default)]
pub struct GeneralPrefs {
/// Interface language (persisted independently of projects): `en`, `ja`, `es`, `pt-br`, `uk`,
/// `zh-cn`, `de`, `ru`, `fr`, or `system` (the default: the operating system's preferred language when the
/// `zh-cn`, `de`, `ru`, `fr`, `id`, or `system` (the default: the operating system's preferred language when the
/// interface has it, otherwise English).
pub interface_language: String,
/// "At Startup": `showHome` (FilmCraft: the demo project), `openMostRecent`, `emptyProject`.
@ -777,6 +777,7 @@ static CATEGORIES: &[Category] = &[
("de", "Deutsch"),
("ru", "Русский"),
("fr", "Français"),
("id", "Bahasa Indonesia"),
]),
true,
),

6
crates/engine/src/settings_tests.rs
View File

@ -38,6 +38,12 @@ fn interface_language_persists_and_rejects_hostile_values() {
assert_eq!(Preferences::load(&path).general.interface_language, "de");
set(&mut s, "general.interfaceLanguage", json!("fr"));
assert_eq!(Preferences::load(&path).general.interface_language, "fr");
// Indonesian is stored as `id`, not a locale tag or the legacy `in` code
for value in [json!("id-ID"), json!("in"), json!("ID")] {
assert!(s.execute("prefs.set", json!({"key": "general.interfaceLanguage", "value": value})).is_err());
}
set(&mut s, "general.interfaceLanguage", json!("id"));
assert_eq!(Preferences::load(&path).general.interface_language, "id");
// back to following the operating system (#218)
set(&mut s, "general.interfaceLanguage", json!("system"));
assert_eq!(Preferences::load(&path).general.interface_language, "system");

103
crates/engine/src/source_placement_tests.rs
View File

@ -1,5 +1,5 @@
use super::*;
use filmcraft_project::ItemId;
use filmcraft_project::{ItemId, ItemKind};
use filmcraft_time::{Tick, TimeRange};
use serde_json::json;
@ -83,6 +83,107 @@ fn audio_only_source_drops_only_sound() {
assert_eq!(s.active_sequence().unwrap().audio_tracks[1].items.len(), 1);
}
fn subclip_session(video: bool, audio: bool) -> (Session, ItemId) {
let mut s = Session::default();
s.execute("file.newSequence", json!({"name":"Subclip test","fps":24})).unwrap();
let parent =
s.execute("file.newOfflineFile", json!({"name":"Parent","video":video,"audio":audio,"fps":24,"seconds":10})).unwrap()["item"].as_u64().unwrap();
let sub = s.execute("clip.makeSubclip", json!({"item":parent,"startFrame":10,"endFrame":60})).unwrap()["item"].as_u64().unwrap();
(s, ItemId(sub))
}
fn assert_subclip_placement(s: &Session, item: ItemId, expected: (usize, usize)) {
assert_eq!(counts(s), expected);
let q = s.active_sequence().unwrap();
let ItemKind::Subclip { range, .. } = &s.project.item(item).unwrap().kind else { panic!("expected a subclip") };
for clip in q.all_tracks().flat_map(|t| t.items.iter()) {
assert_eq!(clip.item, item);
assert_eq!(clip.source_in, range.start);
assert_eq!(clip.duration, q.settings.frame_rate.snap_nearest(range.duration));
}
if expected == (1, 1) {
let video = &q.video_tracks[0].items[0];
let audio = &q.audio_tracks[0].items[0];
assert!(video.link.is_some());
assert_eq!(video.link, audio.link);
}
}
#[test]
fn subclip_placement_keeps_parent_streams_range_and_history() {
for (video, audio, expected) in [(true, true, (1, 1)), (false, true, (0, 1)), (true, false, (1, 0))] {
let (mut s, sub) = subclip_session(video, audio);
let before = s.project.to_json();
s.execute("timeline.place", json!({"item":sub.0,"track":"V1","audioTrack":"A1","time":0,"video":true,"audio":true})).unwrap();
assert_subclip_placement(&s, sub, expected);
let placed = s.project.to_json();
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(s.project.to_json(), before);
s.execute("edit.redo", json!({})).unwrap();
assert_eq!(s.project.to_json(), placed);
s.project = std::sync::Arc::new(Project::from_json(&placed).unwrap());
assert_subclip_placement(&s, sub, expected);
}
}
#[test]
fn subclip_audio_can_be_placed_on_an_audio_track() {
for video in [false, true] {
let (mut s, sub) = subclip_session(video, true);
s.execute("timeline.place", json!({"item":sub.0,"track":"A1","time":0,"video":false,"audio":true})).unwrap();
assert_subclip_placement(&s, sub, (0, 1));
}
let (mut s, sub) = subclip_session(true, false);
let before = s.project.to_json();
assert!(s.execute("timeline.place", json!({"item":sub.0,"track":"A1","time":0,"video":false,"audio":true})).is_err());
assert_eq!(s.project.to_json(), before);
}
#[test]
fn source_insert_and_overwrite_keep_subclip_sound() {
for command in ["source.insert", "source.overwrite"] {
for (video, audio, expected) in [(true, true, (1, 1)), (false, true, (0, 1)), (true, false, (1, 0))] {
let (mut s, sub) = subclip_session(video, audio);
s.execute("source.open", json!({"item":sub.0})).unwrap();
s.execute(command, json!({})).unwrap();
assert_subclip_placement(&s, sub, expected);
}
}
}
#[test]
fn subclip_placement_can_add_tracks_for_parent_streams() {
for (video, audio, expected) in [(true, true, (1, 1)), (false, true, (0, 1)), (true, false, (1, 0))] {
let (mut s, sub) = subclip_session(video, audio);
let before = s.project.to_json();
s.execute("timeline.place", json!({"item":sub.0,"track":"new","audioTrack":"new","time":0})).unwrap();
assert_eq!(counts(&s), expected);
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(s.project.to_json(), before);
}
}
#[test]
fn subclip_placement_keeps_parent_channel_map_and_all_audio_streams() {
let (mut s, sub) = subclip_session(true, true);
let parent = s.project.resolve_media(sub).unwrap().0;
let m = std::sync::Arc::make_mut(&mut s.project).item_mut(parent).unwrap().as_media_mut().unwrap();
m.info.audio_streams.push(m.info.audio_streams[0].clone());
m.interpret.audio_channels = Some(filmcraft_project::AudioChannelMap { format: filmcraft_project::AudioChannels::Mono, clips: vec![vec![1], vec![0]] });
s.execute("timeline.place", json!({"item":sub.0,"track":"V1","audioTrack":"A1","time":0})).unwrap();
assert_eq!(counts(&s), (1, 3));
let q = s.active_sequence().unwrap();
let link = q.video_tracks[0].items[0].link;
assert!(link.is_some());
for (index, (stream, channels)) in [(0, vec![1]), (0, vec![0]), (1, vec![])].into_iter().enumerate() {
let clip = &q.audio_tracks[index].items[0];
assert_eq!(clip.item, sub);
assert_eq!(clip.audio_stream, stream);
assert_eq!(clip.source_channels, channels);
assert_eq!(clip.link, link);
}
}
#[test]
fn malformed_source_spans_are_bounded() {
let s = setup();

84
crates/engine/src/tests.rs
View File

@ -381,6 +381,59 @@ fn set_param_writes_a_keyframe_when_asked_or_animated() {
assert_eq!(s.execute("effects.setParam", odd).unwrap()["keyframes"], 2);
}
/// A keyframe set with a `time` outside the clip stays at that time (#1055), before the start as
/// after the end: a ramp that spans a cut shows its true slice, and a clip-end keyframe is not
/// overwritten by a later one. Only the playhead fallback is pinned to the clip.
#[test]
fn set_param_keeps_a_keyframe_outside_the_clip_at_its_time() {
let mut s = demo();
let it = s.active_sequence().unwrap().video_tracks[0].items[1].clone();
let (c, start, end) = (it.id.0, it.start, it.end());
let opacity = |s: &Session| {
let it = s.active_sequence().unwrap().find_item(filmcraft_project::ClipId(c)).unwrap().1;
let p = &it.effect("opacity").unwrap().params["opacity"];
(p.keyframes.iter().map(|k| (k.time, k.value.as_f64().unwrap())).collect::<Vec<_>>(), it.clone())
};
let set = |value: f64, time: Tick| json!({"clip": c, "effect": "opacity", "param": "opacity", "value": value, "time": time.0, "keyframe": true});
let len = end - start;
// Case A: a keyframe exactly at the clip end, then one past it, gives three keyframes
s.execute("effects.setParam", set(100.0, start)).unwrap();
s.execute("effects.setParam", set(100.0, end)).unwrap();
let r = s.execute("effects.setParam", set(20.0, end + len)).unwrap();
assert_eq!(r["keyframes"], 3, "{r}");
let (keys, it) = opacity(&s);
assert_eq!(keys, vec![(it.source_time_at(start), 100.0), (it.source_time_at(end), 100.0), (it.source_time_at(end + len), 20.0)]);
// Case B: a ramp past the end is not compressed; halfway through the clip is a quarter of the way down
s.undo();
s.undo();
let r = s.execute("effects.setParam", set(0.0, end + len)).unwrap();
assert_eq!(r["keyframes"], 2, "{r}");
let (_, it) = opacity(&s);
let p = it.effect("opacity").unwrap().params["opacity"].clone();
let mid = start + Tick(len.0 / 2);
assert!((p.f64_at(it.source_time_at(mid)) - 75.0).abs() < 0.5, "{}", p.f64_at(it.source_time_at(mid)));
// before the start the same holds: the keyframe keeps its time and the ramp its slope
s.undo();
s.undo();
s.execute("effects.setParam", set(0.0, start - len)).unwrap();
s.execute("effects.setParam", set(100.0, end)).unwrap();
let (keys, it) = opacity(&s);
assert_eq!(keys, vec![(it.source_time_at(start - len), 0.0), (it.source_time_at(end), 100.0)]);
let p = it.effect("opacity").unwrap().params["opacity"].clone();
assert!((p.f64_at(it.source_time_at(start)) - 50.0).abs() < 0.5, "{}", p.f64_at(it.source_time_at(start)));
// hostile times saturate instead of overflowing
for t in [i64::MIN, i64::MAX] {
s.execute("effects.setParam", json!({"clip": c, "effect": "opacity", "param": "opacity", "value": 50.0, "time": t})).unwrap();
}
// with no `time`, a playhead past the clip still lands on its last frame
s.undo();
s.undo();
s.execute("playhead.set", json!({"time": (end + len).0})).unwrap();
s.execute("effects.setParam", json!({"clip": c, "effect": "opacity", "param": "opacity", "value": 40.0})).unwrap();
let (keys, it) = opacity(&s);
assert!(keys.contains(&(it.source_time_at(end - Tick(1)), 40.0)), "{keys:?}");
}
/// A choice parameter takes its option name as well as its index, as transition parameters do
/// (`effects.list {"detail": true}` lists the names).
#[test]
@ -958,3 +1011,34 @@ fn enable_flips_each_selected_clip() {
s.execute("clip.enable", json!({"clips": [a, b]})).unwrap();
assert_eq!((enabled(&s, a), enabled(&s, b)), (true, true));
}
/// Times before sequence start are unreachable in the GUI and cut off on export.
#[test]
fn negative_place_and_marker_times_are_refused() {
let mut s = demo();
let item = s.active_sequence().unwrap().video_tracks[0].items[0].item;
let before = s.project.clone();
let n_markers = s.active_sequence().unwrap().markers.len();
let n_clips: usize = s.active_sequence().unwrap().video_tracks.iter().map(|t| t.items.len()).sum();
for (cmd, params) in [
("timeline.place", json!({"item": item.0, "frame": -12})),
("timeline.place", json!({"item": item.0, "seconds": -1.0})),
("timeline.place", json!({"item": item.0, "time": -1})),
("markers.add", json!({"frame": -10})),
("markers.add", json!({"seconds": -0.5})),
("markers.add", json!({"time": i64::MIN})),
("markers.addChapter", json!({"frame": -1})),
("markers.addRange", json!({"time": -1, "durationFrames": 10})),
("markers.addFlashCue", json!({"frame": -3})),
] {
let e = s.execute(cmd, params.clone()).expect_err(&format!("{cmd} {params}"));
assert!(matches!(&e, EngineError::BadParams { cmd: c, msg } if c == cmd && msg.contains("negative")), "{cmd} {params}: {e:?}");
assert_eq!(*s.project, *before);
}
assert_eq!(s.active_sequence().unwrap().markers.len(), n_markers);
let n_clips_after: usize = s.active_sequence().unwrap().video_tracks.iter().map(|t| t.items.len()).sum();
assert_eq!(n_clips_after, n_clips);
s.execute("timeline.place", json!({"item": item.0, "frame": 0})).unwrap();
s.execute("markers.add", json!({"frame": 0, "name": "start"})).unwrap();
assert!(s.active_sequence().unwrap().markers.iter().any(|m| m.start == Tick::ZERO && m.name == "start"));
}

58
crates/export/src/audio_out.rs
View File

@ -1,6 +1,6 @@
//! The export audio path: the sequence mix at the output sample rate, folded to the output
//! channel count, with Export ▸ Effects ▸ Loudness Normalization (a BS.1770-4 measuring pass over
//! the export range, then a linear gain and a look-ahead true-peak limiter).
//! the export range, then bounded gain refinement through a look-ahead true-peak limiter).
//!
//! The limiter's look-ahead delays its output; [`AudioOut`] mixes that many samples ahead (silence
//! past the end of the range) and drops the first ones, so the exported audio stays aligned with
@ -20,6 +20,7 @@ use crate::{ExportError, ExportSettings, Result};
#[derive(Clone, Copy, Debug, Default, PartialEq, serde::Serialize)]
pub struct LoudnessReport {
pub measured_lufs: f64,
/// Applied gain before the limiter, including bounded loudness correction.
pub gain_db: f64,
}
@ -84,11 +85,62 @@ impl AudioOut {
}
let measured = meter.integrated();
let gain_db = filmcraft_audio_dsp::normalize_gain_db(measured, ln.target_lufs);
self.gain = 10f64.powf(gain_db / 20.0) as f32;
self.loudness = Some(LoudnessReport { measured_lufs: measured, gain_db });
// Limiting changes integrated loudness. Measure the actual limited signal and
// refine its INPUT gain; the peak ceiling remains in force on every pass.
// Four trials and at most 9 dB of extra drive bound both work and compression.
let start = self.out;
let mut trial_gain = gain_db;
let mut best_gain = gain_db;
let mut best_error = f64::INFINITY;
let mut previous = None;
if measured.is_finite() && ln.target_lufs.is_finite() {
for _ in 0..4 {
self.rewind(start, trial_gain);
let mut limited = filmcraft_audio_dsp::LoudnessMeter::new(self.sr as f64, self.channels);
while self.out < self.end {
if cancel() {
self.rewind(start, best_gain);
return Err(ExportError::Cancelled);
}
let until = self.out.saturating_add(self.sr as i64).min(self.end);
if let Some(buf) = self.pull(until, sources) {
let refs: Vec<&[f32]> = buf.iter().map(Vec::as_slice).collect();
limited.process(&refs);
}
}
let achieved = limited.integrated();
if !achieved.is_finite() {
break;
}
let error = ln.target_lufs - achieved;
if error.abs() < best_error {
best_error = error.abs();
best_gain = trial_gain;
}
// Stop on convergence or a ceiling-limited plateau. An impossible target
// must not cause unbounded passes or ever bypass the limiter.
if error.abs() <= 0.1 || previous.is_some_and(|last: f64| (achieved - last).abs() < 0.02) {
break;
}
previous = Some(achieved);
trial_gain += error.clamp(-3.0, 3.0);
}
}
self.rewind(start, best_gain);
self.loudness = Some(LoudnessReport { measured_lufs: measured, gain_db: best_gain });
Ok(())
}
/// Restore the output cursor and limiter history between measuring passes and delivery.
fn rewind(&mut self, start: i64, gain_db: f64) {
self.out = start;
self.next_in = start;
self.gain = 10f64.powf(gain_db / 20.0) as f32;
if let Some(limiter) = self.limiter.as_mut() {
limiter.reset();
}
}
/// Mix `n` samples from `pos`, folded to the output channels (silence past the range end).
/// Stereo output of a 5.1 Mix is the ITU-R BS.775 downmix; 5.1 output of a stereo Mix places it
/// on L and R; 5.1 is in L, R, C, LFE, Ls, Rs order.

79
crates/export/src/settings_tests.rs
View File

@ -518,7 +518,7 @@ fn loudness_normalization_hits_target_and_true_peak() {
assert!((i - target).abs() < 0.5, "target {target}: measured {i:.2} LUFS");
assert!(tp <= -0.9, "true peak {tp:.2} dBTP");
let rep = prog.loudness.lock().unwrap().unwrap();
assert!((rep.measured_lufs - (target - rep.gain_db)).abs() < 1e-6);
assert!((rep.measured_lufs + rep.gain_db - target).abs() < 0.1, "unlimited tone needs no correction");
}
// a target the peaks cannot reach: the limiter holds the true-peak ceiling
let path = dir.path("hot.wav");
@ -870,3 +870,80 @@ fn gpu_rendering_is_off_by_default() {
let auto: ExportSettings = serde_json::from_value(serde_json::json!({"format": "h264", "gpuRendering": "auto"})).unwrap();
assert_eq!(auto.gpu_rendering, crate::GpuRendering::Auto);
}
/// Brief large peaks over a quiet bed exercise loudness lost during limiter release.
struct CrestSource(filmcraft_media::SharedSource, Arc<std::sync::atomic::AtomicUsize>);
impl MediaSource for CrestSource {
fn info(&self) -> &filmcraft_media::MediaInfo {
self.0.info()
}
fn video_frame(&self, req: FrameRequest) -> filmcraft_media::Result<Arc<filmcraft_frame::VideoFrame>> {
self.0.video_frame(req)
}
fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> filmcraft_media::Result<filmcraft_frame::AudioBuffer> {
self.1.fetch_add(1, Ordering::Relaxed);
let samples: Vec<f32> = (0..frames)
.map(|i| {
let at = start + i as i64;
let phase = at.rem_euclid(sample_rate as i64 / 4);
let amplitude = if phase < sample_rate as i64 / 500 { 0.9 } else { 0.08 };
amplitude * (std::f32::consts::TAU * 1000.0 * at as f32 / sample_rate as f32).sin()
})
.collect();
Ok(filmcraft_frame::AudioBuffer { sample_rate, channels: vec![samples.clone(), samples] })
}
}
#[test]
fn loudness_normalization_corrects_limiter_loss_and_bounds_impossible_targets() {
use std::cell::Cell;
let (p, seq, mut sources) = matte([0.0, 0.0, 0.0, 1.0], 64, 36, Some(-6.0));
let reads = Arc::new(std::sync::atomic::AtomicUsize::new(0));
for source in sources.0.values_mut() {
*source = Arc::new(CrestSource(source.clone(), reads.clone()));
}
let range = TimeRange::new(Tick::ZERO, Tick(TICKS_PER_SECOND));
let mut settings = ExportSettings::default();
settings.effects.loudness = LoudnessNormalization { enabled: true, target_lufs: -14.0, true_peak_dbtp: -2.0 };
for target in [-14.0, 2.0] {
settings.effects.loudness.target_lufs = target;
let mut audio = audio_out::AudioOut::new(p.clone(), seq, &settings, range).unwrap();
let chunks = Cell::new(0);
audio
.measure(&settings, &sources, &|| {
chunks.set(chunks.get() + 1);
false
})
.unwrap();
let report = audio.loudness.unwrap();
let result = audio.rest(&sources).unwrap();
let (integrated, peak) = lufs(&result, 48_000);
eprintln!("target {target}: output {integrated:.3} LUFS, {peak:.3} dBTP, gain {:.3}, chunks {}", report.gain_db, chunks.get());
assert!(chunks.get() <= 5, "one raw pass plus at most four limited trials");
assert_eq!(result[0].len(), 48_000, "measuring must rewind output and limiter latency");
assert!(peak <= -1.8, "limiting must remain enabled: {peak}");
if target == -14.0 {
assert!((integrated - target).abs() < 0.15, "achieved {integrated}");
assert!(report.gain_db > target - report.measured_lufs + 0.3, "fixture must require limiter correction");
} else {
assert!(integrated < target - 1.0, "impossible target sacrifices loudness, not peak ceiling");
}
}
// Cancel after raw measurement and one limited trial, before further refinement.
// Record source reads at cancellation to prove no further mixing occurs.
let mut audio = audio_out::AudioOut::new(p, seq, &settings, range).unwrap();
let chunks = Cell::new(0);
let reads_at_cancel = Cell::new(None);
let cancel = || {
chunks.set(chunks.get() + 1);
if chunks.get() > 2 {
reads_at_cancel.set(Some(reads.load(Ordering::Relaxed)));
true
} else {
false
}
};
assert!(matches!(audio.measure(&settings, &sources, &cancel), Err(ExportError::Cancelled)));
assert_eq!(chunks.get(), 3);
assert_eq!(reads_at_cancel.get(), Some(reads.load(Ordering::Relaxed)));
}

25
crates/gpu/src/blend_tests.rs
View File

@ -162,14 +162,15 @@ fn gpu_blend_modes_match_cpu() {
layers: vec![
// opaque YUV on the left, a partially transparent ramp on the right; the top-right
// corner stays empty (destination alpha 0)
PlanLayer { frame: yuv_frame(640, 360), matrix: Affine::scale(0.35, 0.5), opacity: 1.0, blend: Blend::Normal, fx: None },
PlanLayer { frame: base.clone(), matrix: Affine::translate(200.0, 40.0), opacity: 0.9, blend: Blend::Normal, fx: None },
PlanLayer { frame: yuv_frame(640, 360), matrix: Affine::scale(0.35, 0.5), opacity: 1.0, blend: Blend::Normal, fx: None, input: None },
PlanLayer { frame: base.clone(), matrix: Affine::translate(200.0, 40.0), opacity: 0.9, blend: Blend::Normal, fx: None, input: None },
PlanLayer {
frame: top.clone(),
matrix: Affine::motion(Vec2::new(170.0, 95.0), Vec2::new(1.1, 1.1), 17.0, Vec2::new(80.0, 60.0)),
opacity: 0.8,
blend: mode,
fx: None,
input: None,
},
PlanLayer {
frame: demo.clone(),
@ -177,6 +178,7 @@ fn gpu_blend_modes_match_cpu() {
opacity: 0.6,
blend: mode,
fx: None,
input: None,
},
],
};
@ -235,8 +237,8 @@ fn gpu_blend_edge_cases_match_cpu() {
width: n,
height: n,
layers: vec![
PlanLayer { frame: back.clone(), matrix: Affine::IDENTITY, opacity: 1.0, blend: Blend::Normal, fx: None },
PlanLayer { frame: src.clone(), matrix: Affine::IDENTITY, opacity, blend: mode, fx: None },
PlanLayer { frame: back.clone(), matrix: Affine::IDENTITY, opacity: 1.0, blend: Blend::Normal, fx: None, input: None },
PlanLayer { frame: src.clone(), matrix: Affine::IDENTITY, opacity, blend: mode, fx: None, input: None },
],
};
let cpu = execute_cpu(&plan);
@ -276,8 +278,8 @@ fn gpu_dissolve_pattern_is_exact() {
width: w,
height: h,
layers: vec![
PlanLayer { frame: red.clone(), matrix: Affine::scale(w as f64, h as f64), opacity: 1.0, blend: Blend::Normal, fx: None },
PlanLayer { frame: white.clone(), matrix: Affine::scale(w as f64, h as f64), opacity: op, blend: Blend::Dissolve, fx: None },
PlanLayer { frame: red.clone(), matrix: Affine::scale(w as f64, h as f64), opacity: 1.0, blend: Blend::Normal, fx: None, input: None },
PlanLayer { frame: white.clone(), matrix: Affine::scale(w as f64, h as f64), opacity: op, blend: Blend::Dissolve, fx: None, input: None },
],
};
let cpu = execute_cpu(&plan).over_black_rgba8();
@ -303,7 +305,7 @@ fn normal_fast_path_and_blend_layers_off_output() {
};
let mut c = GpuCompositor::new(&dev, &q);
let grey = Arc::new(VideoFrame::rgba_f32(4, 4, vec![0.2; 64]));
let layer = |m: Affine, blend: Blend| PlanLayer { frame: grey.clone(), matrix: m, opacity: 1.0, blend, fx: None };
let layer = |m: Affine, blend: Blend| PlanLayer { frame: grey.clone(), matrix: m, opacity: 1.0, blend, fx: None, input: None };
let normal = FramePlan::Layers { width: 32, height: 16, layers: vec![layer(Affine::scale(8.0, 4.0), Blend::Normal)] };
c.composite(&normal);
assert!(c.backdrop.is_none());
@ -329,7 +331,14 @@ fn normal_fast_path_and_blend_layers_off_output() {
#[test]
fn quad_bounds_clamps_and_handles_non_finite() {
let l = |m: Affine| PlanLayer { frame: Arc::new(VideoFrame::rgba_f32(10, 10, vec![0.0; 400])), matrix: m, opacity: 1.0, blend: Blend::Screen, fx: None };
let l = |m: Affine| PlanLayer {
frame: Arc::new(VideoFrame::rgba_f32(10, 10, vec![0.0; 400])),
matrix: m,
opacity: 1.0,
blend: Blend::Screen,
fx: None,
input: None,
};
assert_eq!(quad_bounds(&l(Affine::translate(5.0, 5.0)), 100, 100), Some((4, 4, 12, 12)));
assert_eq!(quad_bounds(&l(Affine::translate(-5.0, 95.0)), 100, 100), Some((0, 94, 6, 6)));
assert_eq!(quad_bounds(&l(Affine::translate(f64::NAN, 0.0)), 100, 100), Some((0, 0, 100, 100)));

87
crates/gpu/src/composite.wgsl
View File

@ -15,6 +15,13 @@ struct U {
p1: vec4<f32>, // y_off y_scale c_off c_scale (code units)
p2: vec4<f32>, // kr kb code_scale footprint
p3: vec4<f32>, // blend mode, alpha-plane scale (0: none), effect-source integer decimation, unused
// Colour management of HDR / wide-gamut Y'CbCr (`filmcraft_color::InputTransform`, #965):
q0: vec4<f32>, // on, curve (0 sdr, 1 linear, 2 pq, 3 hlg), HLG OOTF scale (0: none), reference white cd/m²
q1: vec4<f32>, // HLG OOTF luma weights, gamut matrix on
q2: vec4<f32>, // gamut matrix row 0, tone mapping on
q3: vec4<f32>, // gamut matrix row 1, tone-mapping source peak cd/m²
q4: vec4<f32>, // gamut matrix row 2, tone-mapping target peak cd/m²
q5: vec4<f32>, // desaturation luma weights, desaturation on
};
@group(0) @binding(0) var<uniform> u: U;
@ -85,6 +92,82 @@ fn to_linear(v: vec3<f32>) -> vec3<f32> {
return srgb_to_linear(clamp(v, vec3(0.0), vec3(1.0)));
}
// ---- colour management (`filmcraft_color::transform`), for `u.q0.x` = 1
// SMPTE ST 2084 on a 0..1 scale (1 = 10 000 cd/m²) and back.
fn pq1(e: f32) -> f32 {
let m1 = 0.1593017578125; let m2 = 78.84375; let c1 = 0.8359375; let c2 = 18.8515625; let c3 = 18.6875;
let p = pow(max(e, 0.0), 1.0 / m2);
return pow(max(p - c1, 0.0) / (c2 - c3 * p), 1.0 / m1);
}
fn pq1_inv(y: f32) -> f32 {
let m1 = 0.1593017578125; let m2 = 78.84375; let c1 = 0.8359375; let c2 = 18.8515625; let c3 = 18.6875;
let p = pow(clamp(y, 0.0, 1.0), m1);
return pow((c1 + c2 * p) / (1.0 + c3 * p), m2);
}
// ITU-R BT.2390 EETF (`bt2390_eetf`), source peak s to target peak d, in cd/m².
fn eetf(nits: f32, s: f32, d: f32) -> f32 {
if d >= s || nits <= 0.0 { return clamp(nits, 0.0, max(s, d)); }
let smax = pq1_inv(s / 10000.0);
let e1 = min(pq1_inv(nits / 10000.0) / smax, 1.0);
let max_lum = pq1_inv(d / 10000.0) / smax;
let ks = 1.5 * max_lum - 0.5;
var e2 = e1;
if e1 >= ks {
let t = (e1 - ks) / (1.0 - ks);
let t2 = t * t; let t3 = t2 * t;
e2 = (2.0 * t3 - 3.0 * t2 + 1.0) * ks + (t3 - 2.0 * t2 + t) * (1.0 - ks) + (-2.0 * t3 + 3.0 * t2) * max_lum;
}
return pq1(e2 * smax) * 10000.0;
}
// Signal → working linear: `InputTransform::convert` (curve, HLG OOTF, gamut matrix, tone
// mapping, desaturation into the working gamut). The CPU's decode tables clamp the signal to 0..1.
fn managed(rgb: vec3<f32>) -> vec3<f32> {
let v = clamp(rgb, vec3(0.0), vec3(1.0));
let curve = u32(u.q0.y);
let white = u.q0.w;
var c = v;
if curve == 0u { c = srgb_to_linear(v); }
if curve == 2u { c = vec3(pq1(v.r), pq1(v.g), pq1(v.b)) * (10000.0 / white); }
if curve == 3u { c = hlg_inv(v); }
if u.q0.z > 0.0 {
let ys = max(dot(u.q1.xyz, c), 0.0);
var g = 0.0;
if ys > 0.0 { g = u.q0.z * pow(ys, 0.2); }
c = c * g;
}
if u.q1.w > 0.5 {
c = vec3(dot(u.q2.xyz, c), dot(u.q3.xyz, c), dot(u.q4.xyz, c));
}
if u.q2.w > 0.5 {
let m = max(max(c.r, c.g), c.b);
if m > 0.0 {
let nits = m * white;
var k = 1.0;
if nits >= u.q3.w {
k = u.q4.w / white / m;
} else if nits > 1e-6 {
k = eetf(nits, u.q3.w, u.q4.w) / nits;
}
c = c * k;
}
}
if u.q5.w > 0.5 {
let mn = min(min(c.r, c.g), c.b);
if mn < 0.0 {
let y = dot(u.q5.xyz, c);
if y <= 0.0 {
c = vec3(0.0);
} else {
c = max(y + (c - y) * (y / (y - mn)), vec3(0.0));
}
}
}
return c;
}
// One linear premultiplied sample at source position p.
fn sample(p: vec2<f32>) -> vec4<f32> {
let kind = u32(u.p0.y);
@ -113,7 +196,9 @@ fn sample(p: vec2<f32>) -> vec4<f32> {
if u.p3.y > 0.0 {
a = clamp(load4(tex3, p).r * u.p3.y, 0.0, 1.0);
}
return vec4(to_linear(vec3(R, G, B)) * a, a);
var lin = to_linear(vec3(R, G, B));
if u.q0.x > 0.5 { lin = managed(vec3(R, G, B)); }
return vec4(lin * a, a);
}
// The layer's premultiplied linear colour at this pixel, opacity applied.

7
crates/gpu/src/fx.wgsl
View File

@ -383,7 +383,12 @@ fn color_op(op: u32, c: vec3<f32>) -> vec3<f32> {
case OP_EXTRACT: {
let lo = u.p0.x; let hi = u.p0.y; let soft = u.p0.z;
let l = linear_to_srgb1(max(luma709(c), 0.0));
let inside = min(clamp((l - lo) / soft, 0.0, 1.0), clamp((hi - l) / soft, 0.0, 1.0));
// Hard cutoffs avoid both division by zero and an artificial narrow fade
// that amplifies transfer-function rounding relative to the CPU reference.
var inside = select(0.0, 1.0, l > lo && l < hi);
if soft > 0.0 {
inside = min(clamp((l - lo) / soft, 0.0, 1.0), clamp((hi - l) / soft, 0.0, 1.0));
}
return vec3(select(inside, 1.0 - inside, u.i1.x != 0u));
}
case OP_INVERT: {

37
crates/gpu/src/fx_tests.rs
View File

@ -126,8 +126,14 @@ fn cases() -> Vec<(&'static str, Vec<(&'static str, ParamValue)>, bool)> {
("gamma_correction", vec![("gamma", fl(-3.0))], false),
("levels", vec![("in_black", fl(20.0)), ("in_white", fl(230.0)), ("out_black", fl(10.0)), ("out_white", fl(240.0)), ("gamma", fl(150.0))], false),
("levels", vec![("in_black", fl(200.0)), ("in_white", fl(0.0)), ("gamma", fl(5.0)), ("out_white", fl(-50.0))], true),
("extract", vec![], true),
("extract", vec![("invert", ParamValue::Bool(true))], true),
("extract", vec![("black", fl(40.0)), ("white", fl(200.0)), ("softness", fl(30.0))], true),
("extract", vec![("black", fl(90.0)), ("white", fl(160.0)), ("invert", ParamValue::Bool(true))], true),
("extract", vec![("black", fl(90.0)), ("white", fl(160.0)), ("softness", fl(1.0))], true),
("extract", vec![("black", fl(40.0)), ("white", fl(200.0)), ("softness", fl(100.0))], true),
("extract", vec![("black", fl(160.0)), ("white", fl(90.0))], true),
("extract", vec![("black", fl(90.0)), ("white", fl(90.0))], true),
("invert", vec![("blend", fl(30.0))], false),
("invert", vec![("channel", ParamValue::Choice(1))], false),
("invert", vec![("channel", ParamValue::Choice(2)), ("blend", fl(70.0))], false),
@ -329,6 +335,34 @@ fn gpu_effects_match_cpu_exactly() {
assert!(failures.is_empty(), "GPU effects differ from the CPU:\n{}", failures.join("\n"));
}
#[test]
fn gpu_extract_zero_softness_has_no_fractional_edge() {
let Some((dev, q)) = device() else {
eprintln!("no GPU adapter; skipping");
return;
};
let mut c = GpuCompositor::new(&dev, &q);
let (w, h) = (67, 41);
let px = picture(w, h);
let frame = VideoFrame::rgba_f32(w, h, px.clone());
let mut masks = Vec::new();
for invert in [false, true] {
let e = effect("extract", &[("black", fl(90.0)), ("white", fl(160.0)), ("invert", ParamValue::Bool(invert))]);
let op = FxOp::eval(&e, &cx(Tick::ZERO, 1.0), w as usize, h as usize).unwrap();
let (_, _, gpu) = c.effect_image(&frame, &LayerFx { size: (w, h), decimation: 1, ops: vec![op] }).unwrap();
for (src, out) in px.as_chunks::<4>().0.iter().zip(gpu.as_chunks::<4>().0.iter()) {
assert_eq!(out[3], src[3], "Extract must preserve alpha");
assert!(out[0] == 0.0 || out[0] == out[3], "zero softness produced a fractional edge: {out:?}");
assert_eq!(out[0], out[1]);
assert_eq!(out[0], out[2]);
}
masks.push(gpu);
}
for ((normal, inverted), src) in masks[0].as_chunks::<4>().0.iter().zip(masks[1].as_chunks::<4>().0.iter()).zip(px.as_chunks::<4>().0.iter()) {
assert_eq!(normal[0] + inverted[0], src[3], "inversion must complement the premultiplied mask");
}
}
/// Chains run in order (and Unsharp keeps its original while blurring, between other blurs).
#[test]
fn gpu_effect_chains_match_cpu() {
@ -578,6 +612,7 @@ fn gpu_effect_layers_match_cpu_plan() {
opacity: 1.0,
blend: Blend::Normal,
fx: Some(Arc::new(LayerFx { size: (320, 180), decimation: 2, ops: ops(&chain_c, 320, 180, 0.5) })),
input: None,
},
PlanLayer {
frame: ramp.clone(),
@ -585,6 +620,7 @@ fn gpu_effect_layers_match_cpu_plan() {
opacity: 0.8,
blend: mode,
fx: Some(Arc::new(LayerFx { size: (160, 120), decimation: 1, ops: ops(&chain_a, 160, 120, 1.0) })),
input: None,
},
PlanLayer::new(ramp.clone(), Affine::translate(10.0, 20.0), 0.5, Blend::Normal),
PlanLayer {
@ -593,6 +629,7 @@ fn gpu_effect_layers_match_cpu_plan() {
opacity: 0.7,
blend: mode,
fx: Some(Arc::new(LayerFx { size: (320, 180), decimation: 1, ops: ops(&chain_b, 320, 180, 1.0) })),
input: None,
},
],
};

51
crates/gpu/src/lib.rs
View File

@ -586,10 +586,11 @@ impl GpuCompositor {
chroma: up.chroma,
size: (f.width, f.height),
color: f.color,
input: None,
})
}
fn uniforms(src: &SrcInfo, m: &filmcraft_geom::Affine, opacity: f32, blend: Blend, out: (u32, u32)) -> [f32; 28] {
fn uniforms(src: &SrcInfo, m: &filmcraft_geom::Affine, opacity: f32, blend: Blend, out: (u32, u32)) -> [f32; UNIFORM_FLOATS] {
let (kr, kb) = src.color.matrix.kr_kb();
let bits_scale = src.code_levels / 256.0; // 2^(bits - 8), independent of texture encoding
let (yo, ys, co, cs) = match (src.color.range, src.kind) {
@ -609,7 +610,31 @@ impl GpuCompositor {
let footprint = (1.0 / sx.min(sy).max(1e-6)) as f32;
let taps = if footprint > 1.25 { footprint.ceil().min(4.0) } else { 1.0 };
let _ = Matrix::Bt709;
[
// colour management (q0..q5), stage by stage as `InputTransform::apply`
let q = src.input.map_or([0.0; 24], |p| {
let curve = match p.curve {
filmcraft_color::GpuCurve::Sdr => 0.0,
filmcraft_color::GpuCurve::Linear => 1.0,
filmcraft_color::GpuCurve::Pq => 2.0,
filmcraft_color::GpuCurve::Hlg => 3.0,
};
let (hl, hk) = p.hlg.unwrap_or(([0.0; 3], 0.0));
let mx = p.matrix.unwrap_or([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]);
let (sp, dp) = p.tone.unwrap_or((0.0, 0.0));
let dl = p.desaturate.unwrap_or([0.0; 3]);
let on = |b: bool| if b { 1.0 } else { 0.0 };
#[rustfmt::skip]
let q = [
1.0, curve, hk, filmcraft_color::REFERENCE_WHITE_NITS as f32,
hl[0], hl[1], hl[2], on(p.matrix.is_some()),
mx[0][0], mx[0][1], mx[0][2], on(p.tone.is_some()),
mx[1][0], mx[1][1], mx[1][2], sp,
mx[2][0], mx[2][1], mx[2][2], dp,
dl[0], dl[1], dl[2], on(p.desaturate.is_some()),
];
q
});
let base = [
m.a as f32,
m.b as f32,
m.c as f32,
@ -638,10 +663,14 @@ impl GpuCompositor {
src.alpha,
0.0,
0.0,
]
];
let mut u = [0.0; UNIFORM_FLOATS];
u[..28].copy_from_slice(&base);
u[28..].copy_from_slice(&q);
u
}
fn uniform_buffer(&self, u: &[f32; 28]) -> wgpu::Buffer {
fn uniform_buffer(&self, u: &[f32; UNIFORM_FLOATS]) -> wgpu::Buffer {
let bytes: Vec<u8> = u.iter().flat_map(|v| v.to_le_bytes()).collect();
let buf = self.device.create_buffer(&wgpu::BufferDescriptor {
label: Some("layer-u"),
@ -679,7 +708,8 @@ impl GpuCompositor {
},
None => VideoFrame::rgba_f32(img.w as u32, img.h as u32, img.px.clone()),
};
owned = [PlanLayer { frame: Arc::new(frame), matrix: filmcraft_geom::Affine::IDENTITY, opacity: 1.0, blend: Blend::Normal, fx: None }];
owned =
[PlanLayer { frame: Arc::new(frame), matrix: filmcraft_geom::Affine::IDENTITY, opacity: 1.0, blend: Blend::Normal, fx: None, input: None }];
(img.w as u32, img.h as u32, &owned[..])
}
FramePlan::Composite { width, height, steps } => return self.composite_steps(*width, *height, steps, prep),
@ -736,7 +766,10 @@ impl GpuCompositor {
chroma: (1, 1),
size: (l.frame.width, l.frame.height),
color: l.frame.color,
input: None,
});
// HDR / wide-gamut Y'CbCr: the planner's input transform, in the shader (#965)
let src = SrcInfo { input: l.input.as_ref().filter(|_| src.kind == 2).and_then(|t| t.gpu_params()), ..src };
let views = self.uploads.get(k).map_or_else(|| std::array::from_fn(|_| self.dummy.clone()), |up| up.views.clone());
let job = match &l.fx {
Some(lfx) => {
@ -768,7 +801,8 @@ impl GpuCompositor {
// what the layer draws: the effect result (linear premultiplied RGBA) or the frame
let (u, tex) = match &job {
Some(j) => {
let s = SrcInfo { kind: 1, code_scale: 1.0, code_levels: 1.0, alpha: 0.0, chroma: l.size(), size: l.size(), color: l.frame.color };
let s =
SrcInfo { kind: 1, code_scale: 1.0, code_levels: 1.0, alpha: 0.0, chroma: l.size(), size: l.size(), color: l.frame.color, input: None };
(Self::uniforms(&s, &l.matrix, l.opacity, l.blend, (w, h)), [j.result.clone(), self.dummy.clone(), self.dummy.clone(), self.dummy.clone()])
}
None => {
@ -1108,8 +1142,13 @@ struct SrcInfo {
chroma: (u32, u32),
size: (u32, u32),
color: filmcraft_color::ColorInfo,
/// Colour management beyond the default decode (a Y'CbCr layer's [`PlanLayer::input`]).
input: Option<filmcraft_color::GpuInputParams>,
}
/// Floats in a layer's uniform block (`U` in `composite.wgsl`: 13 vec4).
const UNIFORM_FLOATS: usize = 52;
/// A render pass drawing into the accumulator.
fn accum_pass<'e>(enc: &'e mut wgpu::CommandEncoder, view: &wgpu::TextureView, load: wgpu::LoadOp<wgpu::Color>) -> wgpu::RenderPass<'e> {
enc.begin_render_pass(&wgpu::RenderPassDescriptor {

68
crates/gpu/src/tests.rs
View File

@ -23,6 +23,7 @@ fn yuv_code_range_uniforms() {
chroma: (1, 1),
size: (1, 1),
color: filmcraft_color::ColorInfo { range, ..filmcraft_color::ColorInfo::REC709 },
input: None,
};
let u = GpuCompositor::uniforms(&src, &Affine::IDENTITY, 1.0, Blend::Normal, (1, 1));
// Full: 0..2^bits-1, neutral 2^(bits-1). Limited: BT.709 8-bit codes * 2^(bits-8).
@ -122,13 +123,14 @@ fn gpu_matches_cpu_plan() {
width: w,
height: h,
layers: vec![
PlanLayer { frame: yuv_frame(640, 360), matrix: Affine::scale(0.5, 0.5), opacity: 1.0, blend: Blend::Normal, fx: None },
PlanLayer { frame: yuv_frame(640, 360), matrix: Affine::scale(0.5, 0.5), opacity: 1.0, blend: Blend::Normal, fx: None, input: None },
PlanLayer {
frame: rgba,
matrix: Affine::motion(Vec2::new(200.0, 100.0), Vec2::new(0.4, 0.4), 12.0, Vec2::new(160.0, 90.0)),
opacity: 0.7,
blend: Blend::Normal,
fx: None,
input: None,
},
],
};
@ -287,8 +289,8 @@ fn prepared_upload_matches_inline_conversion() {
width: w as usize,
height: h as usize,
layers: vec![
PlanLayer { frame: yuv16, matrix: Affine::IDENTITY, opacity: 1.0, blend: Blend::Normal, fx: None },
PlanLayer { frame: rgbaf, matrix: Affine::scale(0.5, 0.5), opacity: 0.8, blend: Blend::Normal, fx: None },
PlanLayer { frame: yuv16, matrix: Affine::IDENTITY, opacity: 1.0, blend: Blend::Normal, fx: None, input: None },
PlanLayer { frame: rgbaf, matrix: Affine::scale(0.5, 0.5), opacity: 0.8, blend: Blend::Normal, fx: None, input: None },
],
};
let image = FramePlan::Image(filmcraft_render::Image { w: w as usize, h: h as usize, px: f32_layer });
@ -317,8 +319,11 @@ fn upload_cache_keeps_buffers_alive() {
let mut c = GpuCompositor::new(&dev, &q);
let px = Arc::new(vec![200u8; 16 * 8 * 4]);
let frame = Arc::new(VideoFrame { width: 16, height: 8, data: PixelData::Rgba8(px.clone()), ..(*yuv_frame(16, 8)).clone() });
let plan =
FramePlan::Layers { width: 16, height: 8, layers: vec![PlanLayer { frame, matrix: Affine::IDENTITY, opacity: 1.0, blend: Blend::Normal, fx: None }] };
let plan = FramePlan::Layers {
width: 16,
height: 8,
layers: vec![PlanLayer { frame, matrix: Affine::IDENTITY, opacity: 1.0, blend: Blend::Normal, fx: None, input: None }],
};
c.composite(&plan);
drop(plan);
assert!(Arc::strong_count(&px) > 1, "cached upload must own its pixel buffer");
@ -467,8 +472,8 @@ fn gpu_draws_yuv_alpha_plane_like_the_cpu() {
width: w,
height: h,
layers: vec![
PlanLayer { frame: background.clone(), matrix: Affine::IDENTITY, opacity: 1.0, blend: Blend::Normal, fx: None },
PlanLayer { frame: yuv_alpha_frame(320, 180, bits), matrix: Affine::IDENTITY, opacity: 0.9, blend: Blend::Normal, fx: None },
PlanLayer { frame: background.clone(), matrix: Affine::IDENTITY, opacity: 1.0, blend: Blend::Normal, fx: None, input: None },
PlanLayer { frame: yuv_alpha_frame(320, 180, bits), matrix: Affine::IDENTITY, opacity: 0.9, blend: Blend::Normal, fx: None, input: None },
],
};
let cpu = execute_cpu(&plan).over_black_rgba8();
@ -484,7 +489,7 @@ fn gpu_draws_yuv_alpha_plane_like_the_cpu() {
let bg = execute_cpu(&FramePlan::Layers {
width: w,
height: h,
layers: vec![PlanLayer { frame: background.clone(), matrix: Affine::IDENTITY, opacity: 1.0, blend: Blend::Normal, fx: None }],
layers: vec![PlanLayer { frame: background.clone(), matrix: Affine::IDENTITY, opacity: 1.0, blend: Blend::Normal, fx: None, input: None }],
})
.over_black_rgba8();
let (l, r, b0) = (at(&gpu, 1), at(&gpu, w - 2), at(&bg, 1));
@ -500,3 +505,50 @@ fn fast_half_float_conversion_matches_the_reference_for_every_value() {
assert!(fast.to_bits() == slow.to_bits() || (fast.is_nan() && slow.is_nan()), "{h:#06x}: {fast} vs {slow}");
}
}
/// HDR and wide-gamut Y'CbCr on a Rec. 709 sequence (#965): the shader's colour management
/// (curve, HLG OOTF, BT.2020 → 709 matrix, BT.2390 tone mapping, desaturation) matches the CPU
/// running the same `InputTransform`.
#[test]
fn hdr_input_gpu_matches_cpu() {
use filmcraft_color::{ColorInfo, ColorPipeline, ColorSpace, InputTransform, Matrix, Primaries, Transfer};
let Some((dev, q)) = device() else {
eprintln!("no GPU adapter; skipping");
return;
};
let mut c = GpuCompositor::new(&dev, &q);
// a 10-bit luma ramp (to the top code) under neutral, saturated and extreme chroma, 4:4:4
let (w, h) = (256u32, 6u32);
let chroma = [(512, 512), (300, 760), (760, 300), (64, 512), (960, 960), (512, 64)];
let y: Vec<u16> = (0..h).flat_map(|_| (0..w).map(|x| (64 + x * 876 / 255) as u16)).collect();
let cb: Vec<u16> = (0..h).flat_map(|r| std::iter::repeat_n(chroma[r as usize].0, w as usize)).collect();
let cr: Vec<u16> = (0..h).flat_map(|r| std::iter::repeat_n(chroma[r as usize].1, w as usize)).collect();
let data = PixelData::Yuv16 { planes: [Arc::new(y), Arc::new(cb), Arc::new(cr)], chroma: Chroma::C444, bits: 10, alpha: None };
let hdr = |transfer| ColorInfo { matrix: Matrix::Bt2020Ncl, transfer, primaries: Primaries::Bt2020, range: Range::Limited };
let cases = [
("HLG", hdr(Transfer::Hlg), None),
("PQ", hdr(Transfer::Pq), None),
("PQ 4000 cd/m²", hdr(Transfer::Pq), Some(4000.0)),
("BT.2020 SDR", ColorInfo { primaries: Primaries::Bt2020, ..ColorInfo::REC709 }, None),
];
for (name, color, peak) in cases {
let frame = Arc::new(VideoFrame { width: w, height: h, data: data.clone(), color, par: (1, 1), pts: Default::default() });
let cs = ColorSpace::from_info(&color);
let t = Arc::new(InputTransform::new(cs, color.range, &ColorPipeline::REC709, peak));
assert!(t.gpu_params().is_some(), "{name}");
let layer = PlanLayer { input: Some(t), ..PlanLayer::new(frame.clone(), Affine::IDENTITY, 1.0, Blend::Normal) };
let plan = FramePlan::Layers { width: w as usize, height: h as usize, layers: vec![layer] };
let cpu = execute_cpu(&plan).over_black_rgba8();
c.composite(&plan);
let (_, _, gpu) = c.read_output().expect("readback");
let error = cpu.iter().zip(&gpu).map(|(a, b)| a.abs_diff(*b)).max().unwrap_or(0);
// and the stage does something: the default decode of the same frame is far off
let plain = FramePlan::Layers { width: w as usize, height: h as usize, layers: vec![PlanLayer::new(frame, Affine::IDENTITY, 1.0, Blend::Normal)] };
c.composite(&plain);
let (_, _, unmanaged) = c.read_output().expect("readback");
let apart = cpu.iter().zip(&unmanaged).map(|(a, b)| a.abs_diff(*b)).max().unwrap_or(0);
eprintln!("{name}: max error {error}, unmanaged differs by up to {apart}");
assert!(error <= 2, "{name}: GPU differs from the CPU by up to {error} codes");
assert!(apart > 20, "{name}: colour management changed nothing ({apart})");
}
}

6
crates/h264/tests/common/mod.rs
View File

@ -4,6 +4,11 @@
use filmcraft_h264::{Decoder, Picture};
use std::path::{Path, PathBuf};
use std::process::Command;
use std::sync::{Mutex, PoisonError};
// Tests share fixture names within each binary. Serialize the existence check,
// generation and publication so a late producer cannot replace a file in use.
static FIXTURE_GENERATION: Mutex<()> = Mutex::new(());
/// One encoder configuration of the fixture matrix.
pub struct Fixture {
@ -334,6 +339,7 @@ fn run(cmd: &mut Command) -> bool {
/// Returns None (with a message) when ffmpeg is unavailable.
pub fn ensure(f: &Fixture) -> Option<(PathBuf, PathBuf)> {
let ff = filmcraft_testkit::ffmpeg_or_skip(f.name)?;
let _generation = FIXTURE_GENERATION.lock().unwrap_or_else(PoisonError::into_inner);
let dir = fixtures_dir();
let h264 = dir.join(format!("{}.h264", f.name));
let yuv = dir.join(format!("{}.yuv", f.name));

47
crates/h264/tests/conformance.rs
View File

@ -148,3 +148,50 @@ fn generate_fixtures() {
filmcraft_testkit::fixtures::generate_and_report(&format!("h264/{}", f.name), &outs, || common::ensure(f));
}
}
/// Concurrent consumers must publish one complete fixture without replacing an open file.
#[test]
fn concurrent_fixture_publication_is_safe() {
if filmcraft_testkit::ffmpeg_or_skip("concurrent_fixture_publication").is_none() {
return;
}
let stamp = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos();
let name = Box::leak(format!("concurrent_fixture_{}_{stamp}", std::process::id()).into_boxed_str());
let fixture =
common::Fixture { name, source: "testsrc2", width: 64, height: 48, frames: 2, filter: "", args: &["-preset", "ultrafast", "-x264-params", "keyint=1"] };
let open_published = |path: &std::path::Path| {
let mut options = std::fs::OpenOptions::new();
options.read(true);
#[cfg(windows)]
{
use std::os::windows::fs::OpenOptionsExt;
// Like the external decoder, allow reads but deny replacement while open.
options.share_mode(1);
}
options.open(path).unwrap()
};
let start = std::sync::Barrier::new(4);
let results = std::thread::scope(|scope| {
let jobs: Vec<_> = (0..4)
.map(|_| {
scope.spawn(|| {
start.wait();
let paths = common::ensure(&fixture).expect("software fixture must generate");
// Keep both published files open until every producer has finished.
let files = (open_published(&paths.0), open_published(&paths.1));
(paths, files)
})
})
.collect();
jobs.into_iter().map(|j| j.join().expect("fixture producer panicked")).collect::<Vec<_>>()
});
let paths = results[0].0.clone();
for (got, _) in &results {
assert_eq!(got, &paths);
}
assert!(!std::fs::read(&paths.0).unwrap().is_empty());
assert_eq!(std::fs::metadata(&paths.1).unwrap().len(), 2 * (64 * 48 + 2 * 32 * 24));
drop(results);
std::fs::remove_file(paths.0).unwrap();
std::fs::remove_file(paths.1).unwrap();
}

8
crates/interchange/src/otio.rs
View File

@ -273,9 +273,10 @@ impl Exp<'_, '_> {
let mut s = self.stack_json(base, &it.name, Some((it.source_in, it.duration, rate)));
self.stack.pop();
s["effects"] = json!(effects);
s["markers"] = json!(markers);
s["enabled"] = json!(it.enabled);
s["metadata"]["filmcraft"]["clip"] = meta["filmcraft"].clone();
// Nested sequence markers stay on the Stack; clip instance markers live on the clip metadata.
s["metadata"]["filmcraft"]["clip"]["markers"] = json!(markers);
return s;
}
let _ = generator_of(self.p, base);
@ -665,7 +666,10 @@ impl Imp<'_, '_> {
let name = name_of(c);
let mut ti = self.b.clip(item, kind, if name.is_empty() { "Nested Sequence" } else { &name }, pos, dur, src_in);
let clip_meta = meta.and_then(|m| m.get("clip")).cloned();
// Stack markers belong to the nested sequence, not the clip instance.
// Stack markers belong to the nested sequence; instance markers are on filmcraft.clip.
if let Some(ref cm) = clip_meta {
ti.markers = self.markers(cm);
}
self.apply_clip_meta(&mut ti, c, clip_meta.as_ref());
Some(ti)
}

77
crates/interchange/tests/nested_sequence_markers.rs Normal file
View File

@ -0,0 +1,77 @@
//! Nested sequence markers vs clip instance markers in OTIO (issue #901).
//!
//! FCP7 XML and FCPXML already keep these on the right objects. OTIO must not drop the nested
//! sequence's markers when the using clip has none, and must not put clip instance markers on the
//! nested sequence.
mod support;
use filmcraft_interchange::{ExportOptions, Format};
use filmcraft_project::{Label, Marker, MarkerId, MarkerKind, TrackKind};
use filmcraft_time::{FrameRate, Tick};
use support::*;
fn nest_project(clip_has_marker: bool) -> (filmcraft_project::Project, filmcraft_project::ItemId, filmcraft_project::ItemId) {
let r = FrameRate::FPS_24;
let mut p = filmcraft_project::Project::new("P");
let a = media(&mut p, "/m/a.mov", true, true, r);
let nest = sequence(&mut p, "Inner", r, false);
clip(&mut p, nest, TrackKind::Video, 0, a, 0, 48, 0);
{
let q = p.sequence_mut(nest).unwrap();
q.markers.push(Marker {
id: MarkerId(1),
start: r.tick_of(12),
duration: Tick::ZERO,
name: "on nest".into(),
comment: String::new(),
kind: MarkerKind::Comment,
color: Label::Green,
});
}
let s = sequence(&mut p, "Outer", r, false);
let nest_clip = clip(&mut p, s, TrackKind::Video, 0, nest, 0, 48, 0);
if clip_has_marker {
let q = p.sequence_mut(s).unwrap();
let (_, it) = q.find_item_mut(nest_clip).unwrap();
it.markers.push(Marker {
id: MarkerId(2),
start: r.tick_of(6),
duration: Tick::ZERO,
name: "on clip".into(),
comment: String::new(),
kind: MarkerKind::Comment,
color: Label::Yellow,
});
}
(p, s, nest)
}
fn names(ms: &[Marker]) -> Vec<&str> {
ms.iter().map(|m| m.name.as_str()).collect()
}
#[test]
fn otio_keeps_nested_sequence_markers_when_clip_has_none() {
let (p, s, nest) = nest_project(false);
let (imp, text, rep) = roundtrip(&p, s, Format::Otio, &ExportOptions::default());
assert!(!rep.has_warnings(), "{rep}");
let outer = only_seq(&imp);
let nest_clip = &imp.project.sequence(outer).unwrap().video_tracks[0].items[0];
assert!(nest_clip.markers.is_empty(), "clip instance must stay marker-free\n{text}");
let inner = nest_clip.item;
assert_eq!(imp.project.item(inner).unwrap().name, "Inner");
assert_eq!(names(&imp.project.sequence(inner).unwrap().markers), names(&p.sequence(nest).unwrap().markers), "{text}");
}
#[test]
fn otio_keeps_clip_instance_markers_off_the_nested_sequence() {
let (p, s, nest) = nest_project(true);
let (imp, text, _) = roundtrip(&p, s, Format::Otio, &ExportOptions::default());
let outer = only_seq(&imp);
let nest_clip = &imp.project.sequence(outer).unwrap().video_tracks[0].items[0];
assert_eq!(names(&nest_clip.markers), vec!["on clip"], "{text}");
assert_eq!(nest_clip.markers[0].start, FrameRate::FPS_24.tick_of(6));
let inner = nest_clip.item;
assert_eq!(names(&imp.project.sequence(inner).unwrap().markers), names(&p.sequence(nest).unwrap().markers), "{text}");
}

216
crates/media/src/dpx.rs
View File

@ -1,16 +1,23 @@
//! DPX (`.dpx`) stills, decoded clean-room from SMPTE ST 268 (Digital Picture Exchange).
//!
//! The first image element is decoded: luma, RGB, RGBA or ABGR, 8, 10, 12 or 16 bits per
//! component, big-endian (`SDPX`) or little-endian (`XPDS`) files. 10-bit components are filled into
//! 32-bit words (packing 1, method A: two padding bits at the bottom; packing 2, method B: at the
//! top), 12-bit components fill 16-bit words the same two ways. Scan lines start on a 32-bit
//! boundary and may carry end-of-line padding. Orientations 0–3 (either scan direction, either line
//! order) are honoured. Code values map linearly onto 8 bits (full range); a logarithmic (printing
//! density) transfer is shown as stored, without a film-log conversion.
//! The first image element is decoded: luma, RGB, RGBA, ABGR, or Y'CbCr (CbYCrY 4:2:2, CbYACrYA
//! 4:2:2:4, CbYCr 4:4:4, CbYCrA 4:4:4:4), 8, 10, 12 or 16 bits per component, big-endian (`SDPX`) or
//! little-endian (`XPDS`) files. 10-bit components are filled into 32-bit words (packing 1, method
//! A: two padding bits at the bottom; packing 2, method B: at the top), 12-bit components fill
//! 16-bit words the same two ways; packed 10/12-bit components (packing 0) follow each other with no
//! padding, the first in the low bits of a 32-bit word, a component crossing into the next word
//! continuing in its low bits. Scan lines start on a 32-bit boundary and may carry end-of-line
//! padding. Orientations 0–3 (either scan direction, either line order) are honoured. RGB and luma
//! code values map linearly onto 8 bits (full range); a logarithmic (printing density) transfer is
//! shown as stored, without a film-log conversion. Y'CbCr is converted with the matrix the
//! colorimetric field names (BT.709, or BT.601 for the 601 and composite video codes; by picture
//! size when unspecified), in video range unless the reference low / high data codes span the
//! whole code range; 4:2:2 chroma is shared by the two pixels of a pair.
//!
//! Not decoded (an "unsupported" error): run-length encoded elements, the 10/12-bit "packed"
//! layout (packing 0), Y'CbCr descriptors, 1-bit and floating-point components, transposed
//! orientations.
//! Not decoded (an "unsupported" error): run-length encoded elements, 1-bit and floating-point
//! components, transposed orientations.
use filmcraft_color::{Matrix, Range, normalize_c, normalize_y, ycbcr_to_rgb};
use crate::{MediaError, Result, still::Raster};
@ -63,6 +70,8 @@ enum Packing {
Words { shift: u32 },
/// Three 10-bit components per 32-bit word, the first in the high bits, above `pad` bits.
Ten { pad: u32 },
/// `bits`-bit components back to back in 32-bit words, low bits first (packing 0).
Packed { bits: u32 },
}
impl Packing {
@ -73,6 +82,7 @@ impl Packing {
Packing::Bytes => comps,
Packing::Words { .. } => comps.checked_mul(2)?,
Packing::Ten { .. } => comps.div_ceil(3).checked_mul(4)?,
Packing::Packed { bits } => comps.checked_mul(bits as usize)?.div_ceil(32).checked_mul(4)?,
};
raw.checked_next_multiple_of(4)
}
@ -93,6 +103,69 @@ impl Packing {
}
}
}
Packing::Packed { bits } => {
let mask = (1u64 << bits) - 1;
for i in 0..comps {
let bit = i * bits as usize;
let at = bit / 32 * 4;
let pair = u64::from(order.u32(line, at)) | (u64::from(order.u32(line, at + 4)) << 32);
out.push(((pair >> (bit % 32)) & mask) as u16);
}
}
}
}
}
/// What the stored components of a pixel are.
#[derive(Clone, Copy)]
enum Components {
/// `n` components per pixel; the indices of R, G, B and A among them (alpha `None`: opaque).
Rgb { n: usize, map: [Option<usize>; 4] },
/// `n` components per pixel: Cb (4:2:2: Cb on even pixels, Cr on odd ones), Y', then Cr (4:4:4
/// only), then alpha when `alpha`.
YCbCr { n: usize, sub: bool, alpha: bool, matrix: Matrix, range: Range },
}
impl Components {
fn per_pixel(self) -> usize {
match self {
Components::Rgb { n, .. } | Components::YCbCr { n, .. } => n,
}
}
fn has_alpha(self) -> bool {
match self {
Components::Rgb { map, .. } => map[3].is_some(),
Components::YCbCr { alpha, .. } => alpha,
}
}
/// RGBA8 of pixel `x` from the unpacked component values of its scan line (`bits` per value).
fn pixel(self, line: &[u16], x: usize, bits: u32) -> [u8; 4] {
let max = (1u32 << bits) - 1;
let to8 = |v: u32| ((v.min(max) * 255 + max / 2) / max) as u8;
let get = |i: usize| line.get(i).map(|&v| u32::from(v));
match self {
Components::Rgb { n, map } => map.map(|slot| match slot {
Some(i) => get(x * n + i).map_or(0, to8),
None => 255,
}),
Components::YCbCr { n, sub, alpha, matrix, range } => {
let base = x * n;
let (cb, cr) = if sub {
let pair = (x & !1) * n;
(get(pair), get(pair + n))
} else {
(get(base), get(base + 2))
};
// a missing Cr (odd width, 4:2:2) is neutral
let mid = 1u32 << (bits - 1);
let y = normalize_y(get(base + 1).unwrap_or(0), bits, range);
let rgb = ycbcr_to_rgb(y, normalize_c(cb.unwrap_or(mid), bits, range), normalize_c(cr.unwrap_or(mid), bits, range), matrix);
let c = |v: f32| (v.clamp(0.0, 1.0) * 255.0).round() as u8;
let a = if alpha { get(base + n - 1).map_or(0, to8) } else { 255 };
[c(rgb[0]), c(rgb[1]), c(rgb[2]), a]
}
}
}
}
@ -110,7 +183,10 @@ pub fn decode(name: &str, bytes: &[u8]) -> Result<Raster> {
let orientation = order.u16(bytes, 768);
let (w, h) = (order.u32(bytes, 772) as usize, order.u32(bytes, 776) as usize);
let el = ELEMENT;
let low_code = order.u32(bytes, el + 4);
let high_code = order.u32(bytes, el + 12);
let descriptor = bytes.get(el + 20).copied().unwrap_or(0);
let colorimetric = bytes.get(el + 22).copied().unwrap_or(0);
let bits = bytes.get(el + 23).copied().unwrap_or(0);
let packing = order.u16(bytes, el + 24);
let encoding = order.u16(bytes, el + 26);
@ -120,14 +196,6 @@ pub fn decode(name: &str, bytes: &[u8]) -> Result<Raster> {
return Err(err(name, "empty picture"));
}
let pixels = w.checked_mul(h).filter(|&n| n <= MAX_PIXELS).ok_or_else(|| err(name, "picture too large"))?;
// component order per pixel: indices of R, G, B, A (None: opaque) in the stored components
let (n, map): (usize, [Option<usize>; 4]) = match descriptor {
6 => (1, [Some(0), Some(0), Some(0), None]),
50 => (3, [Some(0), Some(1), Some(2), None]),
51 => (4, [Some(0), Some(1), Some(2), Some(3)]),
52 => (4, [Some(3), Some(2), Some(1), Some(0)]),
d => return Err(unsupported(name, format!("descriptor {d}"))),
};
if encoding != 0 {
return Err(unsupported(name, "run-length encoding".into()));
}
@ -138,6 +206,7 @@ pub fn decode(name: &str, bytes: &[u8]) -> Result<Raster> {
(10, 2) => Packing::Ten { pad: 0 },
(12, 1) => Packing::Words { shift: 4 },
(12, 2) => Packing::Words { shift: 0 },
(10 | 12, 0) => Packing::Packed { bits: u32::from(bits) },
(b, p) => return Err(unsupported(name, format!("{b}-bit components with packing {p}"))),
};
let (flip_x, flip_y) = match orientation {
@ -147,6 +216,27 @@ pub fn decode(name: &str, bytes: &[u8]) -> Result<Raster> {
3 => (true, true),
o => return Err(unsupported(name, format!("orientation {o}"))),
};
let max = (1u32 << bits) - 1;
// video range unless the reference codes span every code value
let range = if low_code == 0 && high_code == max { Range::Full } else { Range::Limited };
let matrix = match colorimetric {
5 | 6 => Matrix::Bt709,
7..=10 => Matrix::Bt601,
_ => filmcraft_frame::default_matrix(w as u32, h as u32),
};
let ycbcr = |n, sub, alpha| Components::YCbCr { n, sub, alpha, matrix, range };
let components = match descriptor {
6 => Components::Rgb { n: 1, map: [Some(0), Some(0), Some(0), None] },
50 => Components::Rgb { n: 3, map: [Some(0), Some(1), Some(2), None] },
51 => Components::Rgb { n: 4, map: [Some(0), Some(1), Some(2), Some(3)] },
52 => Components::Rgb { n: 4, map: [Some(3), Some(2), Some(1), Some(0)] },
100 => ycbcr(2, true, false),
101 => ycbcr(3, true, true),
102 => ycbcr(3, false, false),
103 => ycbcr(4, false, true),
d => return Err(unsupported(name, format!("descriptor {d}"))),
};
let n = components.per_pixel();
let comps = w.checked_mul(n).ok_or_else(|| err(name, "picture too large"))?;
let pad = if eol_padding == UNDEFINED { 0 } else { eol_padding as usize };
let stride = layout.line_bytes(comps).and_then(|l| l.checked_add(pad)).ok_or_else(|| err(name, "picture too large"))?;
@ -157,8 +247,7 @@ pub fn decode(name: &str, bytes: &[u8]) -> Result<Raster> {
// the last line needs no end-of-line padding
let size = stride.checked_mul(h - 1).and_then(|s| s.checked_add(stride - pad)).ok_or_else(|| err(name, "picture too large"))?;
let data = offset.checked_add(size).and_then(|end| bytes.get(offset..end)).ok_or_else(|| err(name, "truncated image data"))?;
let max = (1u32 << bits) - 1;
let to8 = |v: u16| ((u32::from(v).min(max) * 255 + max / 2) / max) as u8;
let bits = u32::from(bits);
let mut rgba = vec![0u8; pixels * 4];
let mut line = Vec::with_capacity(comps);
for y in 0..h {
@ -168,19 +257,12 @@ pub fn decode(name: &str, bytes: &[u8]) -> Result<Raster> {
for x in 0..w {
let dx = if flip_x { w - 1 - x } else { x };
let o = (dy * w + dx) * 4;
let Some(px) = rgba.get_mut(o..o + 4) else { continue };
for (c, slot) in map.iter().enumerate() {
let v = match slot {
Some(i) => line.get(x * n + i).map_or(0, |&v| to8(v)),
None => 255,
};
if let Some(d) = px.get_mut(c) {
*d = v;
}
if let Some(px) = rgba.get_mut(o..o + 4) {
px.copy_from_slice(&components.pixel(&line, x, bits));
}
}
}
let has_alpha = map[3].is_some() && rgba.as_chunks::<4>().0.iter().any(|p| p[3] != 255);
let has_alpha = components.has_alpha() && rgba.as_chunks::<4>().0.iter().any(|p| p[3] != 255);
Ok(Raster { width: w as u32, height: h as u32, rgba, has_alpha })
}
@ -250,14 +332,84 @@ mod tests {
assert_eq!(r.rgba, [255, 0, 0, 128]);
}
/// Packing 0: components back to back, low bits first, each line on a 32-bit boundary.
fn packed(lines: &[&[u32]], bits: u32, big: bool) -> Vec<u8> {
let mut out = Vec::new();
for line in lines {
let (mut acc, mut n) = (0u64, 0u32);
for &v in *line {
acc |= u64::from(v) << n;
n += bits;
while n >= 32 {
let w = acc as u32;
out.extend_from_slice(&if big { w.to_be_bytes() } else { w.to_le_bytes() });
acc >>= 32;
n -= 32;
}
}
if n > 0 {
let w = acc as u32;
out.extend_from_slice(&if big { w.to_be_bytes() } else { w.to_le_bytes() });
}
}
out
}
#[test]
fn packed_ten_and_twelve_bit_components_cross_words() {
// 2 × 2 RGB 10-bit: 6 components (60 bits) per line, the fourth straddles two words
let lines: [&[u32]; 2] = [&[1023, 0, 512, 4, 1019, 1023], &[0, 1023, 0, 1023, 1023, 1023]];
for big in [true, false] {
let r = decode("a.dpx", &file(big, 2, 2, 50, 10, 0, 0, &packed(&lines, 10, big))).unwrap();
assert_eq!(r.rgba, [255, 0, 128, 255, 1, 254, 255, 255, 0, 255, 0, 255, 255, 255, 255, 255]);
}
// 3 × 1 RGB 12-bit: 9 components, 108 bits in 4 words
let line: [&[u32]; 1] = [&[4095, 0, 2048, 16, 4079, 0, 0, 0, 4095]];
let r = decode("a.dpx", &file(true, 3, 1, 50, 12, 0, 0, &packed(&line, 12, true))).unwrap();
assert_eq!(r.rgba, [255, 0, 128, 255, 1, 254, 0, 255, 0, 0, 255, 255]);
}
#[test]
fn ycbcr_descriptors_convert_to_rgb() {
// 8-bit CbYCrY, video range, BT.709 (colorimetric 6): black, white, then a red pixel pair
// (the 709 code triple of red is rounded, hence green 1)
let mut f = file(true, 4, 1, 100, 8, 0, 0, &[128, 16, 128, 235, 102, 63, 240, 63]);
f[ELEMENT + 22] = 6;
let r = decode("a.dpx", &f).unwrap();
assert_eq!(r.rgba, [0, 0, 0, 255, 255, 255, 255, 255, 255, 1, 0, 255, 255, 1, 0, 255]);
assert!(!r.has_alpha);
// red in BT.601 (colorimetric 7): the 601 code triple, rounded
let mut f = file(true, 2, 1, 100, 8, 0, 0, &[90, 81, 240, 81]);
f[ELEMENT + 22] = 7;
assert_eq!(decode("a.dpx", &f).unwrap().rgba, [254, 0, 0, 255, 254, 0, 0, 255]);
// 10-bit CbYCr filled (method A), full range from the reference codes: mid grey
let word = |a: u32, b: u32, c: u32| ((a << 22) | (b << 12) | (c << 2)).to_be_bytes();
let mut f = file(true, 1, 1, 102, 10, 1, 0, &word(512, 512, 512));
f[ELEMENT + 4..ELEMENT + 8].copy_from_slice(&0u32.to_be_bytes());
f[ELEMENT + 12..ELEMENT + 16].copy_from_slice(&1023u32.to_be_bytes());
assert_eq!(decode("a.dpx", &f).unwrap().rgba, [128, 128, 128, 255]);
// 16-bit CbYCrA: white at half opacity
let v = |x: u16| x.to_be_bytes();
let data: Vec<u8> = [v(32768), v(60160), v(32768), v(32768)].concat();
let r = decode("a.dpx", &file(true, 1, 1, 103, 16, 0, 0, &data)).unwrap();
assert_eq!(r.rgba, [255, 255, 255, 128]);
assert!(r.has_alpha);
// 8-bit CbYACrYA, 3 pixels: the last pixel has no Cr and is neutral
let r = decode("a.dpx", &file(true, 3, 1, 101, 8, 0, 0, &[128, 235, 255, 128, 16, 0, 128, 235, 255, 0, 0, 0])).unwrap();
assert_eq!(r.rgba, [255, 255, 255, 255, 0, 0, 0, 0, 255, 255, 255, 255]);
assert!(r.has_alpha);
}
#[test]
fn hostile_headers_fail_cleanly() {
// huge dimensions with no data, truncated data, unsupported layouts
assert!(decode("a.dpx", &file(true, u32::MAX, u32::MAX, 50, 10, 1, 0, &[])).is_err());
assert!(decode("a.dpx", &file(true, 4096, 4096, 50, 10, 1, 0, &[0; 64])).is_err());
assert!(decode("a.dpx", &file(true, 2, 2, 50, 10, 1, 0, &[0; 12])).is_err());
assert!(decode("a.dpx", &file(true, 1, 1, 100, 8, 0, 0, &[0; 4])).is_err());
assert!(decode("a.dpx", &file(true, 1, 1, 50, 10, 0, 0, &[0; 4])).is_err());
assert!(decode("a.dpx", &file(true, 1, 1, 150, 8, 0, 0, &[0; 4])).is_err());
assert!(decode("a.dpx", &file(true, 1, 1, 50, 10, 3, 0, &[0; 4])).is_err());
assert!(decode("a.dpx", &file(true, 1, 1, 50, 16, 0, 0, &[0; 4])).is_err());
assert!(decode("a.dpx", &file(true, 3, 3, 100, 12, 0, 0, &[0; 20])).is_err());
assert!(decode("a.dpx", &file(true, 1, 1, 50, 32, 0, 0, &[0; 16])).is_err());
assert!(decode("a.dpx", &file(true, 1, 1, 50, 8, 0, 5, &[0; 4])).is_err());
assert!(decode("a.dpx", &file(true, 0, 1, 50, 8, 0, 0, &[0; 4])).is_err());

14
crates/media/src/wav.rs
View File

@ -62,6 +62,7 @@ impl WavSource {
let mut data = None;
let mut bext_tr = None;
let mut ixml_tr = None;
let mut ds64_data = None;
for _ in 0..MAX_CHUNKS {
let Some(body) = pos.checked_add(8).filter(|&b| b <= len) else { break };
let mut hdr = [0u8; 8];
@ -69,8 +70,12 @@ impl WavSource {
let id = &hdr[0..4];
let mut clen = u64::from(le32(&hdr, 4));
let rest = len - body;
if id == b"data" && (clen == 0xFFFF_FFFF || clen > rest) {
clen = rest;
if id == b"data" {
// RF64 sentinel: the real size is ds64.dataSize (EBU 3306 §3.4), else the rest of the file
if clen == 0xFFFF_FFFF {
clen = ds64_data.unwrap_or(rest);
}
clen = clen.min(rest);
}
if clen > rest {
break;
@ -87,6 +92,11 @@ impl WavSource {
}
fmt = Some((tag, ch, rate, bits));
}
// ds64: riffSize u64, dataSize u64, sampleCount u64, ...
b"ds64" if clen >= 16 => {
let b = read_range(&*reader, body + 8, 8).map_err(io)?;
ds64_data = Some(u64::from(le32(&b, 0)) | u64::from(le32(&b, 4)) << 32);
}
b"data" => data = Some((body, clen)),
// bext: Description 256, Originator 32, OriginatorReference 32, OriginationDate 10,
// OriginationTime 8, then TimeReference (low u32, high u32)

23
crates/project/src/lib.rs
View File

@ -1595,9 +1595,10 @@ impl Project {
// a multi-camera source sequence edits in as a multi-camera clip showing its first angle
let multicam =
it.as_sequence().and_then(|q| q.multicam.as_ref()).map(|m| MulticamSel { enabled: true, angle: m.first_video_angle().unwrap_or(0) as u32 });
let source_channels = match (kind, &it.kind) {
(TrackKind::Audio, ItemKind::Media(m)) => m.interpret.audio_channels.as_ref().and_then(|a| a.clips.first().cloned()).unwrap_or_default(),
_ => Vec::new(),
let source_channels = if kind == TrackKind::Audio {
self.resolve_media(item).and_then(|(_, m, _)| m.interpret.audio_channels.as_ref()).and_then(|a| a.clips.first().cloned()).unwrap_or_default()
} else {
Vec::new()
};
let id = ClipId(self.alloc_id());
let dur = seq_rate.snap_nearest(source.duration).max(seq_rate.frame_duration());
@ -1671,6 +1672,16 @@ fn resolve_auto_points_sized(e: &mut EffectInstance, frame: (u32, u32), source:
}
impl Project {
/// Available picture and sound, resolving a subclip to its parent media. Missing parents,
/// cycles and overly deep subclip chains carry neither stream.
pub fn source_streams(&self, item: ItemId) -> (bool, bool) {
let Some(pi) = self.item(item) else { return (false, false) };
match &pi.kind {
ItemKind::Subclip { .. } => self.resolve_media(item).map_or((false, false), |(_, m, _)| (m.info.has_video(), m.info.has_audio())),
_ => (pi.has_video(), pi.has_audio()),
}
}
/// Resolve media through at most sixteen items, preserving the outermost subclip range.
/// Missing parents, cycles and longer chains are unavailable media.
pub fn resolve_media(&self, item: ItemId) -> Option<(ItemId, &MediaClip, Option<TimeRange>)> {
@ -1895,6 +1906,7 @@ mod tests {
fn media_resolution_bounds_depth_and_preserves_the_outer_range() {
let (mut p, media, _) = demo_project();
assert_eq!(p.resolve_media(media).map(|(id, _, range)| (id, range)), Some((media, None)));
let streams = p.source_streams(media);
let mut parent = media;
let mut last_range = TimeRange::default();
for n in 0..15 {
@ -1902,9 +1914,14 @@ mod tests {
parent = p.add_item("Sub", Label::Iris, ItemKind::Subclip { parent, range: last_range, restrict_trims: false }, None);
}
assert_eq!(p.resolve_media(parent).map(|(id, _, range)| (id, range)), Some((media, Some(last_range))));
assert_eq!(p.source_streams(parent), streams);
let too_deep = p.add_item("Too deep", Label::Iris, ItemKind::Subclip { parent, range: last_range, restrict_trims: false }, None);
assert!(p.resolve_media(too_deep).is_none());
assert!(p.resolve_media(ItemId(u64::MAX)).is_none());
assert_eq!(p.source_streams(too_deep), (false, false));
assert_eq!(p.source_streams(ItemId(u64::MAX)), (false, false));
p.item_mut(parent).unwrap().kind = ItemKind::Subclip { parent, range: last_range, restrict_trims: false };
assert_eq!(p.source_streams(parent), (false, false));
}
#[test]

25
crates/render/src/audio.rs
View File

@ -351,7 +351,8 @@ pub fn clip_is_mono(item: &TrackItem, sources: &dyn SourceProvider) -> Option<bo
}
/// The clip's audio (after speed/reverse) for timeline samples `[x0, x0 + len)` as `w` channels
/// (2: stereo, mono sources on both sides; 6: 5.1), with silence outside the clip.
/// (2: stereo, mono sources on both sides, 5.1 sources folded down; 6: 5.1), with silence outside
/// the clip.
fn raw_channels(item: &TrackItem, src: SourceAudio, x0: i64, len: usize, sr: u32, w: usize) -> Vec<Vec<f32>> {
let mut out = vec![vec![0.0f32; len]; w];
let item_s0 = item.start.to_units_floor(sr as i64);
@ -394,6 +395,17 @@ fn raw_channels(item: &TrackItem, src: SourceAudio, x0: i64, len: usize, sr: u32
return out;
}
if w == 2 {
// a 5.1 source (or a six-channel pick) folds down with ITU-R BS.775, as the Mix and the
// Source Monitor do: picking its first two channels would drop C (dialogue) and surrounds
if let Some(six) = source_51_fold(item, buf.channel_count()) {
let picked: Vec<&[f32]> = six.iter().filter_map(|&c| buf.channels.get(c).map(Vec::as_slice)).collect();
let conv = chans::convert(&picked, Layout::Surround51, Layout::Stereo, Mixdown::FrontRear);
for (dst, ch) in out.iter_mut().zip(&conv) {
let m = n.min(ch.len());
dst[off..off + m].copy_from_slice(&ch[..m]);
}
return out;
}
let (left, right) = source_pair(item, buf.channel_count());
for (c, dst) in out.iter_mut().enumerate() {
let ch = &buf.channels[if c == 0 { left } else { right }];
@ -431,6 +443,17 @@ pub fn source_51(item: &TrackItem, available: usize) -> Vec<usize> {
}
}
/// The six source channels (L, R, C, LFE, Ls, Rs) a clip on a stereo track folds down: a six-channel
/// pick (Modify ▸ Audio Channels), else all of a six-channel source. `None` plays [`source_pair`]
/// (sources with another channel count have no known 5.1 layout).
fn source_51_fold(item: &TrackItem, available: usize) -> Option<Vec<usize>> {
match item.source_channels.len() {
0 if available == 6 => Some((0..6).collect()),
6.. => Some(source_51(item, available)),
_ => None,
}
}
/// The source channels (left, right) an audio clip plays: its `source_channels` (one = mono on
/// both sides; Modify ▸ Audio Channels, Breakout to Mono), else the first two (mono sources on
/// both sides). Channels the source doesn't have fall back to the first.

17
crates/render/src/colorman.rs
View File

@ -3,7 +3,10 @@
//! transform. The maths lives in `filmcraft_color::transform`.
//!
//! A sequence whose pipeline is "plain" (Rec. 709 working space without wide gamut) and media
//! whose metadata says Rec. 709 / sRGB / linear take the original fast path unchanged.
//! whose metadata says Rec. 709 / sRGB / linear take the original fast path unchanged. On such a
//! sequence, display-referred HDR and wide-gamut Y'CbCr (PQ, HLG, BT.2020, P3) stays on the GPU
//! path as well, with its input transform run in the compositor's shader ([`gpu_input`]); log
//! media and the other pipelines are converted here, on the CPU.
use std::collections::HashMap;
use std::sync::{Arc, Mutex, OnceLock};
@ -51,6 +54,18 @@ pub fn input_transform(cs: ColorSpace, range: Range, pipe: &ColorPipeline, peak_
t
}
/// The input transform of a frame the GPU compositor can colour manage itself (#965): a Y'CbCr
/// frame of display-referred HDR or wide-gamut media (PQ, HLG, BT.2020, P3) on a plain pipeline,
/// whose working → monitor transform is the identity. None for log media (camera-gamut
/// compression), RGB frames and other pipelines, which [`decode`] converts on the CPU.
pub fn gpu_input(project: &Project, item: ItemId, pipe: &ColorPipeline, cs: ColorSpace, frame: &VideoFrame) -> Option<Arc<InputTransform>> {
if !pipe.is_plain() || !matches!(frame.data, filmcraft_frame::PixelData::Yuv8 { .. } | filmcraft_frame::PixelData::Yuv16 { .. }) {
return None;
}
let t = input_transform(cs, frame.color.range, pipe, source_peak_nits(project, item));
t.gpu_params().map(|_| t)
}
/// Decode a frame of `item` (box-decimated by `n`) into the working space of `pipe`.
pub fn decode(project: &Project, item: ItemId, frame: &VideoFrame, n: usize, pipe: &ColorPipeline) -> Image {
let cs = source_space(project, item, frame);

9
crates/render/src/effects.rs
View File

@ -77,6 +77,13 @@ pub(crate) fn dec(c: [f32; 3]) -> [f32; 3] {
[srgb_to_linear(c[0].clamp(0.0, 1.0)), srgb_to_linear(c[1].clamp(0.0, 1.0)), srgb_to_linear(c[2].clamp(0.0, 1.0))]
}
/// Stable seed for the *exact* frame timestamp. A fixed 30 Hz quantizer
/// would reuse the same noise field at 60/120 fps and can repeat at 30 fps
/// through floating-point rounding of seconds.
pub(crate) fn noise_frame_seed(cx: &FxCtx) -> u64 {
cx.t.0 as u64
}
pub(crate) fn hash3(x: usize, y: usize, z: u64) -> f32 {
let mut h = (x as u64).wrapping_mul(0x9E37_79B9_7F4A_7C15) ^ (y as u64).wrapping_mul(0xC2B2_AE3D_27D4_EB4F) ^ z.wrapping_mul(0x1656_67B1_9E37_79F9);
h ^= h >> 31;
@ -110,7 +117,7 @@ pub fn apply(img: &mut Image, e: &EffectInstance, cx: &FxCtx) {
let amt = f(e, "amount", cx) / 100.0;
let colored = b(e, "color");
let clip = b(e, "clip");
let frame = (cx.seconds * 30.0) as u64;
let frame = noise_frame_seed(cx);
let w = img.w;
img.px.par_chunks_mut(w * 4).enumerate().for_each(|(y, row)| {
for x in 0..w {

76
crates/render/src/gpufx.rs
View File

@ -320,12 +320,17 @@ impl FxOp {
g: 100.0 / f(e, "gamma", cx).max(1.0),
}
}
"extract" => FxOp::Extract {
lo: f(e, "black", cx) / 255.0,
hi: f(e, "white", cx) / 255.0,
soft: f(e, "softness", cx) / 100.0 * 0.2 + 1e-4,
invert: b(e, "invert"),
},
"extract" => {
let softness = f(e, "softness", cx);
FxOp::Extract {
lo: f(e, "black", cx) / 255.0,
hi: f(e, "white", cx) / 255.0,
// Zero softness is a hard cutoff. A tiny substitute fade amplifies
// CPU/GPU transfer-function rounding and introduces unwanted grey edges.
soft: if softness <= 0.0 { 0.0 } else { softness / 100.0 * 0.2 + 1e-4 },
invert: b(e, "invert"),
}
}
"invert" => FxOp::Invert { channel: choice(e, "channel"), blend: f(e, "blend", cx) / 100.0 },
"posterize" => FxOp::Posterize { n: f(e, "levels", cx).max(2.0) - 1.0 },
"gaussian_blur" | "gaussian_blur_legacy" => {
@ -730,7 +735,12 @@ impl FxOp {
let (lo, hi, soft, inv) = (*lo, *hi, *soft, *invert);
img.map_rgb(|c, _, _| {
let l = linear_to_srgb(luma709(c[0], c[1], c[2]).max(0.0));
let inside = ((l - lo) / soft).clamp(0.0, 1.0).min(((hi - l) / soft).clamp(0.0, 1.0));
let inside = if soft <= 0.0 {
// Match the feathered function's limiting value at the cutoffs.
if l > lo && l < hi { 1.0 } else { 0.0 }
} else {
((l - lo) / soft).clamp(0.0, 1.0).min(((hi - l) / soft).clamp(0.0, 1.0))
};
let v = if inv { 1.0 - inside } else { inside };
[v, v, v]
});
@ -1086,3 +1096,55 @@ fn resample_cpu(src: &Image, inv: &Affine, rect: [u32; 4]) -> Image {
});
out
}
#[cfg(test)]
mod tests {
use super::*;
use filmcraft_project::{ParamValue, find_effect};
use filmcraft_time::Tick;
#[test]
fn extract_zero_softness_has_no_fractional_edge() {
let cx = FxCtx {
t: Tick::ZERO,
px_scale: 1.0,
seconds: 0.0,
timecode: "",
clip_name: "",
project: None,
env: None,
working: filmcraft_color::WorkingSpace::Rec709,
};
// Both sides of each cutoff, including samples inside the former artificial fade.
let lo = 90.0 / 255.0;
let hi = 160.0 / 255.0;
let samples = [lo - 0.00005, lo + 0.00005, hi - 0.00005, hi + 0.00005];
for softness in [0.0, -1.0] {
for invert in [false, true] {
for alpha in [1.0, 0.5] {
let mut e = find_effect("extract").unwrap().instance();
for (key, value) in [("black", 90.0), ("white", 160.0), ("softness", softness)] {
e.params.get_mut(key).unwrap().value = ParamValue::Float(value);
}
e.params.get_mut("invert").unwrap().value = ParamValue::Bool(invert);
let mut img = Image {
w: 4,
h: 1,
px: samples
.iter()
.flat_map(|v| {
let c = filmcraft_color::srgb_to_linear(*v) * alpha;
[c, c, c, alpha]
})
.collect(),
};
FxOp::eval(&e, &cx, 4, 1).unwrap().apply(&mut img);
for (pixel, inside) in img.px.as_chunks::<4>().0.iter().zip([false, true, true, false]) {
let value = if inside != invert { alpha } else { 0.0 };
assert_eq!(*pixel, [value, value, value, alpha], "softness={softness}, invert={invert}");
}
}
}
}
}
}

19
crates/render/src/mixer_tests.rs
View File

@ -638,6 +638,25 @@ fn a_51_track_passes_through_a_51_mix_and_folds_into_stereo() {
assert!(close(meters[&crate::mixer::MASTER][5], 0.6, 1e-6));
}
#[test]
fn a_51_source_on_a_stereo_track_folds_down_instead_of_dropping_centre_and_surrounds() {
// the default after New Sequence From Clip: a 5.1 file on a stereo track in a stereo Mix
let mut r = Rig::with_channels(&[chan_dc], 6);
let k = std::f32::consts::FRAC_1_SQRT_2;
let (lo, ro) = (0.1 + k * 0.3 + k * 0.5, 0.2 + k * 0.3 + k * 0.6);
let st = mix_all(&r, 0, 256);
assert_eq!(st.len(), 2);
assert!(close(st[0][9], lo, 1e-6) && close(st[1][9], ro, 1e-6), "BS.775 fold, got {} / {}", st[0][9], st[1][9]);
// a six-channel pick folds the picked channels (here the source reversed: Rs, Ls, LFE, C, R, L)
r.track(0).items[0].source_channels = vec![5, 4, 3, 2, 1, 0];
let st = mix_all(&r, 0, 256);
assert!(close(st[0][9], 0.6 + k * 0.4 + k * 0.2, 1e-6) && close(st[1][9], 0.5 + k * 0.4 + k * 0.1, 1e-6));
// a stereo pick still plays exactly those two channels
r.track(0).items[0].source_channels = vec![2, 4];
let st = mix_all(&r, 0, 256);
assert!(close(st[0][9], 0.3, 1e-6) && close(st[1][9], 0.5, 1e-6));
}
#[test]
fn stereo_track_through_a_51_submix_into_a_stereo_mix() {
let mut r = Rig::new(&[chan_dc]);

40
crates/render/src/plan.rs
View File

@ -39,6 +39,10 @@ pub struct PlanLayer {
pub blend: Blend,
/// Standard effects to run on the frame before it is placed (None: draw the frame directly).
pub fx: Option<Arc<LayerFx>>,
/// Colour management of a Y'CbCr frame that needs more than the default decode (HDR, BT.2020,
/// P3 media on a Rec. 709 sequence; [`crate::colorman::gpu_input`]): run on the GPU as on the
/// CPU. None: the default decode.
pub input: Option<Arc<filmcraft_color::InputTransform>>,
}
/// The GPU effect stage of a layer: the frame is decoded into a working image of `size` (the
@ -54,7 +58,7 @@ pub struct LayerFx {
impl PlanLayer {
/// A layer drawing `frame` directly.
pub fn new(frame: Arc<VideoFrame>, matrix: Affine, opacity: f32, blend: Blend) -> Self {
Self { frame, matrix, opacity, blend, fx: None }
Self { frame, matrix, opacity, blend, fx: None, input: None }
}
/// Size of the picture the matrix places: the working image with effects, else the frame.
@ -408,10 +412,13 @@ fn push_item(
let time = crate::video_source_time(item, t, src.info().frame_rate());
let Ok(frame) = src.video_frame(FrameRequest { time, scale: want }) else { return };
let cs = crate::colorman::source_space(project, item.item, &frame);
// log / HDR / wide-gamut media is converted on the CPU (below), and so are blended
// HDR and wide-gamut media on a Rec. 709 sequence is converted on the GPU too (#965); log
// media and other pipelines are converted on the CPU (below), and so are blended
// in-between frames (Frame Blending / Optical Flow on speed-changed clips)
let pipe = &seq.settings.color;
let input = if crate::colorman::needs_management(pipe, cs, &frame) { crate::colorman::gpu_input(project, item.item, pipe, cs, &frame) } else { None };
let plain =
!crate::colorman::needs_management(&seq.settings.color, cs, &frame) && crate::interpolation_blend(item, t, src.info().frame_rate()).is_none();
(input.is_some() || !crate::colorman::needs_management(pipe, cs, &frame)) && crate::interpolation_blend(item, t, src.info().frame_rate()).is_none();
if plain && !chain.is_empty() {
// GPU effect stage: the working image the CPU would decode (`base_layer`), the
// effects evaluated for it, placed as `item_layer` places it
@ -435,7 +442,7 @@ fn push_item(
.then_apply(&motion)
.then_apply(&Affine::scale(1.0 / px_scale as f64, 1.0 / px_scale as f64));
let fx = LayerFx { size: (lw as u32, lh as u32), decimation: n as u32, ops };
out.push(PlanLayer { frame, matrix: m, opacity: op * extra_opacity, blend: bl, fx: Some(Arc::new(fx)) });
out.push(PlanLayer { frame, matrix: m, opacity: op * extra_opacity, blend: bl, fx: Some(Arc::new(fx)), input });
return;
}
} else if plain {
@ -450,7 +457,7 @@ fn push_item(
};
let px_scale = frame.width as f64 / size.0.max(1) as f64;
let m = Affine::scale(opts.scale as f64, opts.scale as f64).then_apply(&motion).then_apply(&Affine::scale(1.0 / px_scale, 1.0 / px_scale));
out.push(PlanLayer::new(frame, m, op * extra_opacity, bl));
out.push(PlanLayer { input, ..PlanLayer::new(frame, m, op * extra_opacity, bl) });
return;
}
}
@ -468,8 +475,12 @@ fn near_identity(m: &Affine) -> bool {
/// A layer's working image with its effects applied, on the CPU (the reference for the GPU
/// effect stage).
pub fn effect_image(frame: &VideoFrame, fx: &LayerFx) -> crate::Image {
let (w, h, px) = frame.to_linear_f32_decimated(fx.decimation.max(1) as usize);
let mut img = crate::Image { w, h, px };
effect_image_with(frame, fx, None)
}
/// [`effect_image`] of a colour-managed layer ([`PlanLayer::input`]).
pub fn effect_image_with(frame: &VideoFrame, fx: &LayerFx, input: Option<&filmcraft_color::InputTransform>) -> crate::Image {
let mut img = linear_image(frame, fx.decimation.max(1) as usize, input);
for op in &fx.ops {
op.apply(&mut img);
}
@ -506,12 +517,23 @@ pub fn execute_cpu(plan: &FramePlan) -> crate::Image {
}
}
/// A layer's frame (box-decimated by `n`) as a linear working image: the default decode, or
/// `input`'s table and pixel stage as `colorman::decode` runs them.
fn linear_image(frame: &VideoFrame, n: usize, input: Option<&filmcraft_color::InputTransform>) -> crate::Image {
let (w, h, px) = frame.to_linear_f32_decimated_with(n, input.map(|t| &t.table));
let mut img = crate::Image { w, h, px };
if let Some(t) = input.filter(|t| t.has_pixel_stage()) {
img.map_rgb(|c, _, _| t.apply(c));
}
img
}
fn execute_layers(canvas: &mut crate::Image, layers: &[PlanLayer]) {
let (width, height) = (canvas.w, canvas.h);
for l in layers {
let src = match &l.fx {
Some(fx) => effect_image(&l.frame, fx),
None => crate::Image { w: l.frame.width as usize, h: l.frame.height as usize, px: l.frame.to_linear_f32() },
Some(fx) => effect_image_with(&l.frame, fx, l.input.as_deref()),
None => linear_image(&l.frame, 1, l.input.as_deref()),
};
let placed = if l.matrix == Affine::scale(width as f64, height as f64) && src.w == 1 && src.h == 1 {
crate::Image::filled(width, height, src.get(0, 0))

25
crates/render/src/region_tests.rs
View File

@ -299,3 +299,28 @@ fn a_region_layer_expands_to_the_whole_layer() {
assert_eq!(full.px.iter().filter(|v| **v == 1.0).count(), 2 * 2 * 4);
assert!(full.px[..(2 * 6 + 4) * 4].iter().all(|v| *v == 0.0));
}
/// HDR and wide-gamut Y'CbCr on a Rec. 709 sequence is planned as a GPU layer carrying its input
/// transform, not converted to a float image on the CPU (#965: 4K HLG / PQ playback fell to a
/// third of real time), and the plan still renders what the reference renderer does.
#[test]
fn hdr_media_is_planned_for_the_gpu_with_its_colour_management() {
use filmcraft_color::{ColorInfo, Matrix, Primaries, Range, Transfer};
let hdr = |transfer| ColorInfo { matrix: Matrix::Bt2020Ncl, transfer, primaries: Primaries::Bt2020, range: Range::Limited };
for (name, color) in
[("HLG", hdr(Transfer::Hlg)), ("PQ", hdr(Transfer::Pq)), ("BT.2020 SDR", ColorInfo { primaries: Primaries::Bt2020, ..ColorInfo::REC709 })]
{
let f = Arc::new(VideoFrame { color, ..(*camera(W, H)).clone() });
let (p, seq, map) = scene(&[Spec::new(f)]);
let opts = RenderOptions::default();
let plan = plan::plan_frame(&p, seq, Tick::ZERO, opts, &map);
let plan::FramePlan::Layers { layers, .. } = &plan else { panic!("{name}: not a layer plan") };
assert_eq!(layers.len(), 1, "{name}");
assert!(matches!(layers[0].frame.data, PixelData::Yuv8 { .. }), "{name}: converted on the CPU");
assert!(layers[0].input.is_some(), "{name}: no colour management for the GPU");
let reference = render_sequence(&p, seq, Tick::ZERO, opts, &map);
let got = plan::execute_cpu(&plan);
let max = got.px.iter().zip(&reference.px).map(|(a, b)| (a - b).abs()).fold(0f32, f32::max);
assert!(max < 1e-4, "{name}: plan differs from the reference by {max}");
}
}

2
crates/render/src/vfx/color.rs
View File

@ -337,7 +337,7 @@ pub fn noise(img: &mut Image, e: &EffectInstance, cx: &FxCtx) {
let colored = bv(e, "color");
let clip = bv(e, "clip");
let grain = (fv(e, "grain_size", cx).max(1.0) * cx.px_scale).max(1.0);
let frame = (cx.seconds * 30.0) as u64;
let frame = crate::effects::noise_frame_seed(cx);
let w = img.w;
img.px.par_chunks_mut(w * 4).enumerate().for_each(|(y, row)| {
for x in 0..w {

42
crates/render/tests/noise_frame_cadence.rs Normal file
View File

@ -0,0 +1,42 @@
use filmcraft_project::{ParamValue, find_effect};
use filmcraft_render::{
Image,
effects::{FxCtx, apply},
};
use filmcraft_time::FrameRate;
fn render_noise(effect_id: &str, rate: FrameRate, frame: i64, amount: f64) -> Vec<f32> {
let mut image = Image::filled(16, 16, [0.5, 0.5, 0.5, 1.0]);
let mut effect = find_effect(effect_id).unwrap().instance();
effect.param_mut("amount").unwrap().value = ParamValue::Float(amount);
let tick = rate.tick_of(frame);
let context = FxCtx {
t: tick,
px_scale: 1.0,
seconds: tick.seconds(),
timecode: "00:00:00:00",
clip_name: "generated matte",
project: None,
env: None,
working: filmcraft_color::WorkingSpace::Rec709,
};
apply(&mut image, &effect, &context);
image.px
}
#[test]
fn both_noise_variants_change_on_every_sequence_frame() {
for effect_id in ["noise", "noise_legacy"] {
for rate in [FrameRate::FPS_24, FrameRate::FPS_30, FrameRate::FPS_60, FrameRate::FPS_120] {
for frame in 0..5 {
let this = render_noise(effect_id, rate, frame, 100.0);
let next = render_noise(effect_id, rate, frame + 1, 100.0);
assert_ne!(this, next, "{effect_id} {rate:?} repeated frames {frame} and {}", frame + 1);
assert_eq!(this, render_noise(effect_id, rate, frame, 100.0), "same timestamp must be deterministic");
}
}
// Floating-point second-to-30Hz conversion also repeated frames 122/123 at 30 fps.
assert_ne!(render_noise(effect_id, FrameRate::FPS_30, 122, 100.0), render_noise(effect_id, FrameRate::FPS_30, 123, 100.0));
assert_eq!(render_noise(effect_id, FrameRate::FPS_60, 0, 0.0), [0.5, 0.5, 0.5, 1.0].repeat(16 * 16));
}
}

59
crates/render/tests/underline_direction.rs Normal file
View File

@ -0,0 +1,59 @@
//! Horizontal vs vertical graphic text with underline (letter A, 60 px).
use filmcraft_geom::{Affine, Vec2};
use filmcraft_project::graphic::{new_text_layer, new_vertical_text_layer};
use filmcraft_project::{EffectInstance, ParamValue};
use filmcraft_render::graphic_clip::render_graphic_layers;
use filmcraft_render::image::Image;
use filmcraft_time::Tick;
fn set(e: &mut EffectInstance, k: &str, v: ParamValue) {
e.params.get_mut(k).unwrap_or_else(|| panic!("{k}")).value = v;
}
fn draw(layer: EffectInstance) -> Image {
let mut img = Image::new(320, 200);
render_graphic_layers(&[layer], Tick::ZERO, (320, 200), &Affine::IDENTITY, &mut img);
img
}
fn ink_box(img: &Image) -> [usize; 4] {
let mut b: Option<[usize; 4]> = None;
for y in 0..img.h {
for x in 0..img.w {
if img.get(x, y)[3] > 0.4 {
b = Some(match b {
Some(a) => [a[0].min(x), a[1].min(y), a[2].max(x), a[3].max(y)],
None => [x, y, x, y],
});
}
}
}
b.expect("ink")
}
fn count(img: &Image) -> usize {
(0..img.h).flat_map(|y| (0..img.w).map(move |x| (x, y))).filter(|&(x, y)| img.get(x, y)[3] > 0.4).count()
}
#[test]
fn underline_horizontal_vs_vertical_letter_a() {
let pos = Vec2::new(80.0, 100.0);
let mut h = new_text_layer("A", pos, 60.0);
let h_plain = draw(h.clone());
set(&mut h, "underline", ParamValue::Bool(true));
let h_ul = draw(h);
let mut v = new_vertical_text_layer("A", Vec2::new(200.0, 100.0), 60.0);
let v_plain = draw(v.clone());
set(&mut v, "underline", ParamValue::Bool(true));
let v_ul = draw(v);
assert!(count(&h_ul) > count(&h_plain) + 20, "horizontal underline adds ink");
assert!(count(&v_ul) > count(&v_plain) + 20, "vertical underline adds ink");
let hb = ink_box(&h_ul);
let hp = ink_box(&h_plain);
let vb = ink_box(&v_ul);
let vp = ink_box(&v_plain);
assert!(hb[3] > hp[3] + 1, "horizontal underline sits below the letter: plain {hp:?} ul {hb:?}");
assert!(vb[2] > vp[2] + 1, "vertical underline sits to the right of the letter: plain {vp:?} ul {vb:?}");
}

36
crates/text/src/layout.rs
View File

@ -682,7 +682,8 @@ pub fn layout_rich_uncached(text: &str, style: &TextStyle, runs: &[StyleRun], pa
lay.underline_runs.push(*si);
}
} else if style.underline && pl.content > 0.0 {
lay.underlines.push([x, baseline + pm.underline_pos, x + pl.content, baseline + pm.underline_pos + pm.underline_thickness]);
let y = baseline - style.baseline_shift + pm.underline_pos;
lay.underlines.push([x, y, x + pl.content, y + pm.underline_thickness]);
lay.underline_runs.push(0);
}
lay.lines.push(Line {
@ -724,6 +725,7 @@ fn layout_vertical(text: &str, sty: &Styles, para: &ParagraphStyle, col_w: f32,
let row = (pm.ascent + pm.descent).max(style.size * 0.5) + style.tracking * style.size / 1000.0;
let flat = ParagraphStyle { vertical: true, ..Default::default() };
lay.vertical = true;
let rich = sty.rich();
let mut base = 0usize;
let mut ncols = 0usize;
for (c, p) in text.split('\n').enumerate() {
@ -761,6 +763,21 @@ fn layout_vertical(text: &str, sty: &Styles, para: &ParagraphStyle, col_w: f32,
}
g
}));
// Underline runs down the column, to the right of the cell (the "below" side of vertical-rl).
let y_top = baseline - pl.ascent;
if rich {
for (_x0, x1, si) in &pl.underlines {
let st = sty.list[*si as usize];
let um = fonts::face(sty.prim[*si as usize].face).metrics(st.size);
let x = *x1 + shift - st.baseline_shift + um.underline_pos;
lay.underlines.push([x, y_top, x + um.underline_thickness, y_top + row]);
lay.underline_runs.push(*si);
}
} else if style.underline && pl.content > 0.0 {
let x = shift + pl.left_trim + pl.content + pm.underline_pos;
lay.underlines.push([x, y_top, x + pm.underline_thickness, y_top + row]);
lay.underline_runs.push(0);
}
lay.lines.push(Line {
range: pl.range,
baseline,
@ -998,6 +1015,23 @@ mod tests {
assert_eq!(h.lines.len(), 2);
}
#[test]
fn vertical_underline_runs_down_the_column() {
let s = TextStyle { underline: true, ..st(60.0) };
let h = layout("A", &s, &ParagraphStyle::default());
let v = layout("A", &s, &ParagraphStyle { vertical: true, ..Default::default() });
assert_eq!(h.underlines.len(), 1);
assert_eq!(v.underlines.len(), 1, "vertical cells get an underline");
let [hx0, hy0, hx1, hy1] = h.underlines[0];
let [vx0, vy0, vx1, vy1] = v.underlines[0];
assert!(hx1 - hx0 > hy1 - hy0, "horizontal underline is a wide strip");
assert!(vy1 - vy0 > vx1 - vx0, "vertical underline is a tall strip");
let runs = [StyleRun { range: 0..1, style: s.clone() }];
let rich = layout_rich("A", &st(60.0), &runs, &ParagraphStyle { vertical: true, ..Default::default() });
assert_eq!(rich.underlines.len(), 1);
assert_eq!(rich.underline_runs, vec![1]);
}
#[test]
fn vertical_cells_keep_combining_marks_with_their_base() {
let style = st(32.0);

25
crates/text/tests/underline_shift.rs Normal file
View File

@ -0,0 +1,25 @@
//! Whole-style underline must follow `baseline_shift` the same way character-style runs do.
use filmcraft_text::{ParagraphStyle, StyleRun, TextStyle, layout, layout_rich};
fn style(shift: f32) -> TextStyle {
TextStyle { size: 100.0, underline: true, baseline_shift: shift, ..Default::default() }
}
fn underline_y(shift: f32) -> (f32, f32) {
let para = ParagraphStyle::default();
let s = style(shift);
let plain = layout("Hi", &s, &para);
let rich = layout_rich("Hi", &s, &[StyleRun { range: 0..2, style: s.clone() }], &para);
(plain.underlines[0][1], rich.underlines[0][1])
}
#[test]
fn whole_style_underline_follows_baseline_shift() {
let (p0, r0) = underline_y(0.0);
for shift in [40.0, -40.0] {
let (plain, rich) = underline_y(shift);
assert!((plain - rich).abs() < 1e-3, "plain {plain} vs rich {rich} at shift {shift}");
assert!((plain - (p0 - shift)).abs() < 1e-3, "plain underline y {plain} should move with shift {shift}");
assert!((rich - (r0 - shift)).abs() < 1e-3, "rich underline y {rich} should move with shift {shift}");
}
}

2
crates/ui-egui/README.md
View File

@ -18,7 +18,7 @@ the committed span and undo. Set `FILMCRAFT_UI_SNAPSHOT_DIR` to render screensho
## Localisation
Edit > Language offers English, Japanese, Spanish, Brazilian Portuguese, Ukrainian, Simplified
Chinese, German, Russian and French. The language is stored in the engine's `general.interfaceLanguage` preference,
Chinese, German, Russian, French and Indonesian (which covers the whole interface). The language is stored in the engine's `general.interfaceLanguage` preference,
restored on startup, and also available in Settings > General.
Its default, System Language (`system`), follows the operating system: the first of the user's
preferred languages that the interface has (the host supplies them through

15
crates/ui-egui/examples/bench/playback.rs
View File

@ -347,11 +347,16 @@ impl Bench {
}
}
for r in &recs {
rep.latency.push(ms(r.finished - r.queued));
rep.service.push(ms(r.finished - r.started));
rep.job_cpu.push(ms(r.cpu));
rep.src_wall.push(ms(r.source_wall));
rep.src_cpu.push(ms(r.source_cpu));
// Preroll jobs run before the play clock; counting them as on-clock latency made
// the LongGOP seek look like a mid-play stall (`lat.max` ≈ `preroll_ms`).
let on_clock = r.queued >= anchor && due_at.contains_key(&r.key.frame) && r.key.target == target;
if on_clock {
rep.latency.push(ms(r.finished - r.queued));
rep.service.push(ms(r.finished - r.started));
rep.job_cpu.push(ms(r.cpu));
rep.src_wall.push(ms(r.source_wall));
rep.src_cpu.push(ms(r.source_cpu));
}
if !due_at.contains_key(&r.key.frame) || r.key.target != target {
rep.wasted_jobs += 1;
}

40
crates/ui-egui/src/dock.rs
View File

@ -581,7 +581,9 @@ pub fn draw_group_chrome(
t: &Tokens,
reg: &mut crate::automation::Registry,
) -> Vec<DockAction> {
draw_group_chrome_details(ui, g, focused, seqs, t, reg).actions
let chrome = draw_group_chrome_details(ui, g, seqs, t, reg);
paint_group_focus(ui, g, focused, t);
chrome.actions
}
#[derive(Default)]
@ -590,14 +592,7 @@ pub struct Chrome {
pub tabs: Vec<(PanelKind, Rect)>,
}
pub fn draw_group_chrome_details(
ui: &mut egui::Ui,
g: &Group,
focused: PanelKind,
seqs: &SeqTabs,
t: &Tokens,
reg: &mut crate::automation::Registry,
) -> Chrome {
pub fn draw_group_chrome_details(ui: &mut egui::Ui, g: &Group, seqs: &SeqTabs, t: &Tokens, reg: &mut crate::automation::Registry) -> Chrome {
let mut actions = Vec::new();
let mut tab_rects: Vec<(PanelKind, Rect)> = Vec::new();
let painter = ui.painter().clone();
@ -687,6 +682,22 @@ pub fn draw_group_chrome_details(
list_opened = !open;
}
}
// The wheel over the strip steps through the tabs: down = next, up = previous, one tab
// per wheel notch (trackpad scrolling adds up to a notch first), stopping at the ends.
let wheel_id = egui::Id::new(("tab-wheel", g.path.clone()));
if ui.rect_contains_pointer(strip) && !tabs.is_empty() {
const NOTCH: f32 = 40.0;
let total = ui.ctx().data(|d| d.get_temp::<f32>(wheel_id)).unwrap_or(0.0) + crate::panels::timeline::wheel_input(ui.ctx()).delta.y;
let steps = (total / NOTCH).trunc();
ui.ctx().data_mut(|d| d.insert_temp(wheel_id, total - steps * NOTCH));
let to = (active_at as i64).saturating_sub(steps as i64).clamp(0, tabs.len() as i64 - 1) as usize;
if let Some(tab) = tabs.get(to).filter(|_| to != active_at) {
activate(tab, &mut actions);
actions.push(DockAction::Focus(tab.panel));
}
} else {
ui.ctx().data_mut(|d| d.remove::<f32>(wheel_id));
}
let mut x = strip.min.x + 12.0;
// sequence tabs as drawn (for dragging one past its neighbours), and the one being dragged
let mut seq_rects: Vec<(u64, Rect)> = Vec::new();
@ -836,10 +847,6 @@ pub fn draw_group_chrome_details(
ui.ctx().data_mut(|d| d.remove::<bool>(list_id));
}
}
// focus outline
if active_panel == Some(focused) {
painter.rect_stroke(g.rect, 0.0, Stroke::new(1.0, t.focus), StrokeKind::Inside);
}
// clicking anywhere in the panel focuses it
if let Some(p) = active_panel
&& ui.rect_contains_pointer(g.rect)
@ -850,6 +857,13 @@ pub fn draw_group_chrome_details(
Chrome { actions, tabs: tab_rects }
}
/// Paint once after the group's contents so child backgrounds cannot hide its focus (#1075).
pub(crate) fn paint_group_focus(ui: &egui::Ui, group: &Group, focused: PanelKind, tokens: &Tokens) {
if group.panels.get(group.active).copied() == Some(focused) {
ui.painter().rect_stroke(group.rect, 0.0, Stroke::new(1.0, tokens.focus), StrokeKind::Inside);
}
}
fn paint_widget_focus(painter: &egui::Painter, response: &egui::Response, tokens: &Tokens) {
if response.has_focus() {
painter.rect_stroke(response.rect.shrink(2.0), 2.0, Stroke::new(1.5, tokens.focus), StrokeKind::Inside);

8
crates/ui-egui/src/dock_ui.rs
View File

@ -191,7 +191,7 @@ impl FilmcraftApp {
let mut actions = vec![];
let mut targets = vec![];
for group in &groups {
let chrome = dock::draw_group_chrome_details(ui, group, self.ui.focused, &seqs, &t, &mut self.auto);
let chrome = dock::draw_group_chrome_details(ui, group, &seqs, &t, &mut self.auto);
if let Some(target) = target(group, chrome.tabs, true) {
targets.push(target);
}
@ -205,6 +205,9 @@ impl FilmcraftApp {
self.panel_body(ui, panel, group.content);
}
}
for group in &groups {
dock::paint_group_focus(ui, group, self.ui.focused, &t);
}
let floating_order = self.floating_order(&ctx);
for index in floating_order.into_iter().filter(|_| maximized.is_none()) {
if let Some(target) = self.floating_group(&ctx, index, &seqs, &mut actions) {
@ -279,7 +282,7 @@ impl FilmcraftApp {
panels: floating.panels.clone(),
active: floating.active,
};
let chrome = dock::draw_group_chrome_details(ui, &group, self.ui.focused, seqs, &t, &mut self.auto);
let chrome = dock::draw_group_chrome_details(ui, &group, seqs, &t, &mut self.auto);
actions.extend(chrome.actions);
if let Some(active) = group.panels.get(group.active).copied() {
self.panel_body(ui, active, group.content);
@ -307,6 +310,7 @@ impl FilmcraftApp {
if next != rect {
let _ = self.dock_command(ctx, Action::MoveFloating { panel, rect: [next.min.x, next.min.y, next.width(), next.height()] });
}
dock::paint_group_focus(ui, &group, self.ui.focused, &t);
output = target(&group, chrome.tabs, false);
});
output

7
crates/ui-egui/src/frames.rs
View File

@ -585,7 +585,9 @@ impl FrameServer {
// the frame due now; once the clock runs, the frames before it are late
self.request_job(key, rate.tick_of(key.frame), scale, project, 0, false, (!preroll && dir > 0).then_some(Tick::ZERO));
}
let ahead = prefetch_depth(speed);
// During preroll the clock has not started: only the frames that gate start, not a
// prefetch run that walks the rest of a LongGOP and evicts the playhead pictures.
let ahead = if preroll { (PREROLL_FRAMES - 1).max(0) } else { prefetch_depth(speed) };
for i in 0..ahead {
let f = key.frame + (lead + i * stride) * dir * step;
if f >= 0 {
@ -914,7 +916,7 @@ fn plan_job(job: &Job, seq: ItemId, pool: &Arc<MediaPool>, services: &Arc<dyn Se
// captions (they are not part of the preview hash), so captions are layered on live.
let (w, h) = filmcraft_render::output_size(q, job.scale);
let matrix = filmcraft_geom::Affine::scale(w as f64 / frame.width.max(1) as f64, h as f64 / frame.height.max(1) as f64);
let mut layers = vec![filmcraft_render::plan::PlanLayer { frame, matrix, opacity: 1.0, blend: filmcraft_render::Blend::Normal, fx: None }];
let mut layers = vec![filmcraft_render::plan::PlanLayer { frame, matrix, opacity: 1.0, blend: filmcraft_render::Blend::Normal, fx: None, input: None }];
for o in filmcraft_render::caption_overlays(q, job.time, w, h) {
layers.push(filmcraft_render::plan::PlanLayer {
frame: Arc::new(filmcraft_frame::VideoFrame::rgba_f32(o.w as u32, o.h as u32, o.px)),
@ -922,6 +924,7 @@ fn plan_job(job: &Job, seq: ItemId, pool: &Arc<MediaPool>, services: &Arc<dyn Se
opacity: 1.0,
blend: filmcraft_render::Blend::Normal,
fx: None,
input: None,
});
}
return (filmcraft_render::plan::FramePlan::Layers { width: w, height: h, layers }, true);

83
crates/ui-egui/src/graphics_notice.rs Normal file
View File

@ -0,0 +1,83 @@
//! The notice shown while the window runs in a fallback graphics mode (#820): the desktop app
//! moved on to DirectX 12 only or a software adapter after earlier starts died in the graphics
//! driver. It says so and offers a one-click way back to the default mode for the next start.
//!
//! Automation ids: `graphics.notice` (the notice), `graphics.notice.reset` (Use Default Graphics),
//! `graphics.notice.close`.
use crate::{FilmcraftApp, i18n};
/// Puts the default graphics mode back for the next start; the error says what could not be saved.
pub type ResetFn = Box<dyn FnMut() -> Result<(), String>>;
/// A fallback graphics mode, as the host reports it.
pub struct GraphicsNotice {
/// The mode's English name ("DirectX 12 only", "software rendering"), translated when shown.
pub mode: String,
reset: ResetFn,
/// The default mode is saved for the next start.
pub reset_done: bool,
/// Closed for this session.
pub closed: bool,
}
impl GraphicsNotice {
pub fn new(mode: &str, reset: ResetFn) -> Self {
Self { mode: mode.to_string(), reset, reset_done: false, closed: false }
}
/// Use Default Graphics: save the default mode for the next start.
pub fn reset(&mut self) -> Result<(), String> {
(self.reset)()?;
self.reset_done = true;
Ok(())
}
}
/// Draw the notice (bottom right, above the status bar) until it is closed.
pub(crate) fn show(app: &mut FilmcraftApp, ctx: &egui::Context) {
let Some(mut notice) = app.graphics_notice.take() else { return };
if notice.closed {
app.graphics_notice = Some(notice);
return;
}
let title = tlf!("Safe graphics mode: {mode}", mode = i18n::t(&notice.mode));
let (mut reset, mut close) = (false, false);
let shown = egui::Window::new("graphics-notice")
.id(egui::Id::new("graphics-notice"))
.title_bar(false)
.collapsible(false)
.resizable(false)
.anchor(egui::Align2::RIGHT_BOTTOM, [-12.0, -32.0])
.show(ctx, |ui| {
ui.set_max_width(360.0);
ui.label(egui::RichText::new(&title).strong());
ui.add_space(4.0);
if notice.reset_done {
ui.label(tl!("The default graphics are used from the next start."));
} else {
ui.label(tl!("An earlier start failed with the default graphics. Updating the graphics driver usually fixes this."));
}
ui.add_space(6.0);
ui.horizontal(|ui| {
if !notice.reset_done {
let label = tl!("Use Default Graphics");
let r = ui.button(label);
app.auto.add_response("graphics.notice.reset", &r, label);
reset = r.clicked();
}
let label = tl!("Close");
let c = ui.button(label);
app.auto.add_response("graphics.notice.close", &c, label);
close = c.clicked();
});
});
if let Some(shown) = shown {
app.auto.add("graphics.notice", shown.response.rect, &title);
}
if reset && let Err(e) = notice.reset() {
app.ui.status = e;
}
notice.closed |= close;
app.graphics_notice = Some(notice);
}

5
crates/ui-egui/src/i18n/de.tsv
View File

@ -6,7 +6,7 @@
# apps, text addresses the user informally ("du") and commands use the infinitive ("Projekt
# öffnen…"). Product, format and feature names (FilmCraft, ArtCraft, Lumetri, Essential Graphics,
# EDL, AAF…) keep their spelling. This catalog covers every menu, including the native macOS
# Settings menu; anything else is shown in English.
# app menu; anything else is shown in English.
#
# Terms: track = Spur, bin = Ablage, marker = Marker, playhead = Abspielposition, In/Out = In-/Out-
# Punkt, transition = Übergang, frame hold = Standbild, panel = Bedienfeld, workspace =
@ -180,6 +180,8 @@
Guide Templates Hilfslinienvorlagen
Help Hilfe
Hide All Caption Tracks Alle Untertitelspuren ausblenden
Hide FilmCraft FilmCraft ausblenden
Hide Others Andere ausblenden
High Quality Playback Wiedergabe in hoher Qualität
History Verlauf
Import From Importieren aus
@ -269,6 +271,7 @@
Properties Eigenschaften
Proxy Proxy
Purple Lila
Quit FilmCraft FilmCraft beenden
Reconnect Full Resolution Media… Medien in voller Auflösung neu verknüpfen…
Recover Unsaved Changes… Nicht gespeicherte Änderungen wiederherstellen…
Rectangle Rechteck

Can't render this file because it contains an unexpected character in line 6 and column 45.

11
crates/ui-egui/src/i18n/es.tsv
View File

@ -2804,8 +2804,8 @@
Ignore Audio Ignorar audio
Ignore Video Ignorar video
Apply a cut at each detected cut point Aplicar un corte en cada punto de corte detectado
Create a subclip for each detected cut point Crear un subclip para cada punto de corte detectado
Generate clip markers at each detected cut point Generar marcadores de clip en cada punto de corte detectado
Create a bin of subclips from each detected cut point Crear una bandeja de subclips a partir de cada punto de corte detectado
Create a clip marker at each detected cut point Crear un marcador de clip en cada punto de corte detectado
Sensitivity: Sensibilidad:
Normalize Mix Track to: Normalizar pista de mezcla a:
Disabled clips Clips desactivados
@ -3489,6 +3489,7 @@
Click to select, or click in empty space and drag to marquee select. Use Shift, Alt, and Ctrl for other options. Haz clic para seleccionar, o haz clic en un espacio vacío y arrastra para seleccionar con un marco. Usa Mayús, Alt y Ctrl para otras opciones.
Click to zoom in; Alt-click to zoom out. Haz clic para acercar; Alt+clic para alejar.
Drag: move · Alt-click: smooth/corner · Ctrl-click: delete Arrastrar: mover · Alt+clic: suave/esquina · Ctrl+clic: eliminar
Drag inward to round the corners Arrastrar hacia dentro para redondear las esquinas
Include Alpha Channel Incluir canal alfa
Sequence Settings Ajustes de secuencia
Color Management Gestión del color
@ -3562,3 +3563,9 @@
Edit Project Notes Editar notas del proyecto
Notes for this project: to-dos, instructions, frame numbers… Saved with the project. Notas de este proyecto: tareas pendientes, instrucciones, números de fotograma… Se guardan con el proyecto.
Use Effect Shutter Angle Usar el ángulo de obturación del efecto
Safe graphics mode: {mode} Modo gráfico seguro: {mode}
An earlier start failed with the default graphics. Updating the graphics driver usually fixes this. Un inicio anterior falló con los gráficos predeterminados. Actualizar el controlador gráfico suele solucionarlo.
Use Default Graphics Usar gráficos predeterminados
The default graphics are used from the next start. Los gráficos predeterminados se usarán a partir del próximo inicio.
DirectX 12 only Solo DirectX 12
software rendering renderizado por software

Can't render this file because it contains an unexpected character in line 3 and column 95.

Some files were not shown because too many files have changed in this diff Show More