From 492fa6de0a049c22afcc690afa36930d295261f3 Mon Sep 17 00:00:00 2001 From: bflatastic Date: Fri, 9 Oct 2026 23:20:03 +0700 Subject: [PATCH] Effects: OCIO custom configs read Blender's v2 transforms and warn (#410) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cause: the config reader didn't know ColorSpaceTransform or ExponentWithLinearTransform and passed colours through them silently. Blender's Linear Rec.709 -> sRGB therefore stayed in its E-white XYZ reference space (the blue cast) without the sRGB curve. Display colorspaces, view transforms, v2 views, looks, AllocationTransform, BuiltinTransform and .spi1d / .spimtx files were missing too, and nothing told the user. Fix (from the OCIO v2 documentation and the CLF "moncurve", not OCIO's source): ExponentWithLinear, ColorSpace, Look, DisplayView, Allocation and Builtin transforms (identity, ACES AP0/AP1 matrices, ACEScct / ACEScc, the sRGB / P3 / Rec.1886 / gamma / PQ displays). RangeTransform now clamps and takes one-sided ranges. Display colorspaces meet the scene reference through the default view transform. Added v2 views, view looks, the looks section and .spi1d / .spi3d / .spimtx files. A conversion is built once as a flat list of steps (Config::path / display_path) for both the CPU effect and the GPU colour program (new moncurve and clamp ops and the ACEScc curve; a program over the kernel's 255 ops renders on the CPU). Named-transform resolution is limited to 16 levels and 4096 steps, and YAML nesting to 64 levels, so hostile or looping configs stop. Transforms that are still unsupported (Grading Primary / Tone, ACES output, HLG…), unreadable configs and unknown spaces are now reported: a warning under the effect in Effect Controls, and the new effect.warning query. Tests: ocio_config::blender_like_config_converts_linear_rec709_to_srgb (CST, display colorspaces, v1 / v2 views, allocation, looks, builtin, range, loops), moncurve_matches_srgb_and_inverts, builtin_styles, yaml_nesting_is_bounded, ocio::spi_files_parse, tests_effects::ocio_effect_warns_about_unsupported_config_transforms, gpu tests_fx_lut::ocio_custom_config_v2 (GPU = CPU). Co-Authored-By: Claude Opus 5.5 --- crates/effects/src/lib.rs | 12 + crates/effects/src/ocio.rs | 387 +++++-- crates/effects/src/ocio_config.rs | 1061 +++++++++++++++--- crates/engine/src/commands/effect.rs | 19 + crates/engine/src/tests_effects.rs | 29 + crates/gpu/src/fx_lut.rs | 22 +- crates/gpu/src/shaders/fx_lut.wgsl | 67 ++ crates/gpu/src/tests_fx_lut.rs | 102 ++ crates/ui-egui/src/panels/effect_controls.rs | 9 + docs/agents.md | 3 + docs/parity.md | 5 +- 11 files changed, 1483 insertions(+), 233 deletions(-) diff --git a/crates/effects/src/lib.rs b/crates/effects/src/lib.rs index 9c0fe95b..6d661531 100644 --- a/crates/effects/src/lib.rs +++ b/crates/effects/src/lib.rs @@ -238,6 +238,18 @@ pub fn flatten_params(g: &PropGroup, eval: &mut dyn FnMut(&Property) -> Value) - Params { values } } +/// The warning shown with effect instance `g` (Effect Controls, `effect.warning`), its +/// parameters evaluated by `eval`: settings it can't render as asked, such as an OCIO config it +/// can't read or transforms it passes through. `None` when it renders as asked. +pub fn warning(g: &PropGroup, eval: &mut dyn FnMut(&Property) -> Value) -> Option { + let GroupKind::Effect { effect } = &g.kind else { return None }; + // Only the effects that can warn evaluate their parameters. + if !ocio::CUSTOM_CONFIG_FX.contains(&effect.as_str()) { + return None; + } + ocio::warning(effect, &flatten_params(g, eval)) +} + /// A layer image in flight: point `p` sits at pixel `p * scale + offset`. Points are in layer /// space, or in effect space while the layer's effects run (see [`Buf::rebase`]). #[derive(Clone, Debug, Default)] diff --git a/crates/effects/src/ocio.rs b/crates/effects/src/ocio.rs index c11c20cd..d00f6e9e 100644 --- a/crates/effects/src/ocio.rs +++ b/crates/effects/src/ocio.rs @@ -17,9 +17,12 @@ //! * CDL maths follows the ASC CDL v1.2 specification (slope, offset, power, then saturation with //! Rec. 709 luma weights); the "No Clamp" style mirrors power around zero; //! * File Transform reads `.cube` (Adobe/Resolve), `.3dl` (Autodesk Lustre/Flame integer -//! lattices), `.csp` (cineSpace, with its per-channel pre-LUT shaper) and ASC `.cc` / `.ccc` / -//! `.cdl` XML, with nearest, trilinear or tetrahedral interpolation and an inverse direction -//! (exact for 1D/CDL, iterative for 3D lattices). +//! lattices), `.csp` (cineSpace, with its per-channel pre-LUT shaper), Sony Imageworks +//! `.spi1d` / `.spi3d` / `.spimtx` and ASC `.cc` / `.ccc` / `.cdl` XML, with nearest, trilinear +//! or tetrahedral interpolation and an inverse direction (exact for 1D/CDL/matrices, iterative +//! for 3D lattices); +//! * Configuration ▸ Custom reads a `.ocio` file instead ([`crate::ocio_config`]); what it can't +//! apply is reported by [`warning`]. use std::collections::HashMap; use std::sync::{Arc, Mutex, OnceLock}; @@ -36,13 +39,13 @@ use crate::{Buf, EffectCtx, EffectSpec, num, p, popup, slider}; type M3 = [[f64; 3]; 3]; -const D65: [f64; 2] = [0.3127, 0.3290]; +pub(crate) const D65: [f64; 2] = [0.3127, 0.3290]; const ACES_WHITE: [f64; 2] = [0.32168, 0.33767]; const AP0: [[f64; 2]; 3] = [[0.7347, 0.2653], [0.0, 1.0], [0.0001, -0.0770]]; const AP1: [[f64; 2]; 3] = [[0.713, 0.293], [0.165, 0.830], [0.128, 0.044]]; -const REC709: [[f64; 2]; 3] = [[0.640, 0.330], [0.300, 0.600], [0.150, 0.060]]; -const REC2020: [[f64; 2]; 3] = [[0.708, 0.292], [0.170, 0.797], [0.131, 0.046]]; -const P3: [[f64; 2]; 3] = [[0.680, 0.320], [0.265, 0.690], [0.150, 0.060]]; +pub(crate) const REC709: [[f64; 2]; 3] = [[0.640, 0.330], [0.300, 0.600], [0.150, 0.060]]; +pub(crate) const REC2020: [[f64; 2]; 3] = [[0.708, 0.292], [0.170, 0.797], [0.131, 0.046]]; +pub(crate) const P3: [[f64; 2]; 3] = [[0.680, 0.320], [0.265, 0.690], [0.150, 0.060]]; const IDENTITY: M3 = [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]; /// Transfer functions (encoded ↔ linear). @@ -52,10 +55,16 @@ pub enum Tf { Srgb, Gamma(f64), AcesCct, + /// ACEScc (Academy S-2014-003), the pure-log grading encoding. + AcesCc, /// SMPTE ST 2084 with 1.0 = 100 cd/m². Pq, } +/// ACEScc: the lowest code value (linear 0 and below) and the end of its toe (linear 2⁻¹⁵). +const ACESCC_MIN: f64 = (-16.0 + 9.72) / 17.52; +const ACESCC_TOE: f64 = (-15.0 + 9.72) / 17.52; + impl Tf { pub fn decode(self, v: f64) -> f64 { match self { @@ -75,6 +84,15 @@ impl Tf { 65504.0 } } + Tf::AcesCc => { + if v < ACESCC_TOE { + (2f64.powf(v * 17.52 - 9.72) - 2f64.powi(-16)) * 2.0 + } else if v < (65504f64.log2() + 9.72) / 17.52 { + 2f64.powf(v * 17.52 - 9.72) + } else { + 65504.0 + } + } Tf::Pq => { let (m1, m2, c1, c2, c3) = pq_consts(); let e = v.max(0.0).powf(1.0 / m2); @@ -99,6 +117,15 @@ impl Tf { (v.log2() + 9.72) / 17.52 } } + Tf::AcesCc => { + if v <= 0.0 { + ACESCC_MIN + } else if v < 2f64.powi(-15) { + ((2f64.powi(-16) + v * 0.5).log2() + 9.72) / 17.52 + } else { + (v.log2() + 9.72) / 17.52 + } + } Tf::Pq => { let (m1, m2, c1, c2, c3) = pq_consts(); let y = (v / 100.0).max(0.0).powf(m1); @@ -123,13 +150,13 @@ pub struct Space { pub raw: bool, } -const fn sp(name: &'static str, prims: [[f64; 2]; 3], white: [f64; 2], tf: Tf) -> Space { +pub(crate) const fn sp(name: &'static str, prims: [[f64; 2]; 3], white: [f64; 2], tf: Tf) -> Space { Space { name, prims: Some((prims, white)), tf, raw: false } } /// The built-in configuration's colour spaces, in popup order. -const ACES2065_1: Space = sp("ACES2065-1", AP0, ACES_WHITE, Tf::Linear); -const ACES_CG: Space = sp("ACEScg", AP1, ACES_WHITE, Tf::Linear); +pub(crate) const ACES2065_1: Space = sp("ACES2065-1", AP0, ACES_WHITE, Tf::Linear); +pub(crate) const ACES_CG: Space = sp("ACEScg", AP1, ACES_WHITE, Tf::Linear); const ACES_CCT: Space = sp("ACEScct", AP1, ACES_WHITE, Tf::AcesCct); pub const SPACES: &[Space] = &[ @@ -332,6 +359,92 @@ pub enum FileXform { Lut { lut: Lut, shaper: Option<[Vec<(f32, f32)>; 3]> }, /// CDL corrections by id (in file order). Cdl(Vec<(String, Cdl)>), + /// A matrix with offset (Sony Imageworks `.spimtx`), as a MatrixTransform's. + Matrix { m: [f64; 16], offset: [f64; 4] }, +} + +/// Parse a Sony Imageworks `.spimtx` matrix: three rows of four numbers, three matrix columns +/// and an offset in 16-bit code values (÷ 65535). +pub fn parse_spimtx(text: &str) -> Option { + let v: Vec = text.split_whitespace().map(|t| t.parse().ok()).collect::>()?; + if v.len() != 12 || v.iter().any(|x| !x.is_finite()) { + return None; + } + let mut m = [0.0; 16]; + let mut offset = [0.0; 4]; + for (r, row) in v.chunks_exact(4).enumerate() { + m[r * 4..r * 4 + 3].copy_from_slice(&row[..3]); + offset[r] = row[3] / 65535.0; + } + m[15] = 1.0; + Some(FileXform::Matrix { m, offset }) +} + +/// Parse a Sony Imageworks `.spi1d` 1D LUT: `From `, `Length `, `Components +/// <1 | 3>`, then `{` the values `}`. +pub fn parse_spi1d(text: &str) -> Option { + let (mut from, mut len, mut comps) = ([0.0f32, 1.0], 0usize, 1usize); + let mut values: Vec = vec![]; + let mut body = false; + for line in text.lines().map(str::trim).filter(|l| !l.is_empty()) { + if body { + if line.starts_with('}') { + break; + } + for t in line.split_whitespace() { + values.push(t.parse().ok()?); + } + continue; + } + let mut t = line.split_whitespace(); + match t.next()? { + "{" => body = true, + "From" => from = [t.next()?.parse().ok()?, t.next()?.parse().ok()?], + "Length" => len = t.next()?.parse().ok()?, + "Components" => comps = t.next()?.parse().ok()?, + _ => {} + } + } + // A 1D table is interpolated by index, so cap it like the lattices (2²⁴ entries). + if !(2..=1 << 24).contains(&len) + || !matches!(comps, 1 | 3) + || values.len() != len.checked_mul(comps)? + || from[1] <= from[0] + || from.iter().any(|v| !v.is_finite()) + { + return None; + } + let d1 = values.chunks_exact(comps).map(|c| if comps == 3 { [c[0], c[1], c[2]] } else { [c[0]; 3] }).collect(); + Some(Lut { n1: len, d1, n3: 0, d3: vec![], dmin: [from[0]; 3], dmax: [from[1]; 3] }) +} + +/// Parse a Sony Imageworks `.spi3d` lattice: `SPILUT 1.0`, `3 3`, the sizes, then one +/// `r g b R G B` line per lattice point (indices, then the colour). +pub fn parse_spi3d(text: &str) -> Option { + let mut lines = text.lines().map(str::trim).filter(|l| !l.is_empty()); + if !lines.next()?.starts_with("SPILUT") { + return None; + } + lines.next()?; + let size: Vec = lines.next()?.split_whitespace().map(|t| t.parse().ok()).collect::>()?; + let n = *size.first()?; + if size.len() != 3 || size.iter().any(|s| *s != n) || !(2..=129).contains(&n) { + return None; + } + let mut d3 = vec![[0.0f32; 3]; n * n * n]; + let mut seen = 0usize; + for line in lines { + let v: Vec<&str> = line.split_whitespace().collect(); + let [r, g, b, cr, cg, cb] = v.as_slice() else { return None }; + let (r, g, b): (usize, usize, usize) = (r.parse().ok()?, g.parse().ok()?, b.parse().ok()?); + if r >= n || g >= n || b >= n { + return None; + } + // Our lattice is red fastest. + *d3.get_mut(r + g * n + b * n * n)? = [cr.parse().ok()?, cg.parse().ok()?, cb.parse().ok()?]; + seen += 1; + } + (seen == n * n * n).then_some(Lut { n1: 0, d1: vec![], n3: n, d3, dmin: [0.0; 3], dmax: [1.0; 3] }) } /// Parse an Autodesk `.3dl` lattice (integers, blue fastest; an optional input-mesh line). @@ -470,6 +583,15 @@ pub fn parse_file(text: &str, ext: &str) -> Option { if ext == "csp" || text.trim_start().starts_with("CSPLUTV100") { return parse_csp(text).map(|(lut, s)| FileXform::Lut { lut, shaper: Some(s) }); } + if ext == "spimtx" { + return parse_spimtx(text); + } + if ext == "spi1d" || text.trim_start().starts_with("Version") { + return parse_spi1d(text).map(|lut| FileXform::Lut { lut, shaper: None }); + } + if ext == "spi3d" || text.trim_start().starts_with("SPILUT") { + return parse_spi3d(text).map(|lut| FileXform::Lut { lut, shaper: None }); + } if ext == "cube" || text.contains("LUT_3D_SIZE") || text.contains("LUT_1D_SIZE") { return parse_cube(text).map(|lut| FileXform::Lut { lut, shaper: None }); } @@ -514,6 +636,7 @@ impl FileXform { let o = if inverse { cdl.invert(c64, true) } else { cdl.apply(c64, true) }; o.map(|v| v as f32) } + FileXform::Matrix { m, offset } => crate::ocio_config::matrix_apply(m, offset, c.map(|v| v as f64), inverse).map(|v| v as f32), FileXform::Lut { lut, shaper } => { let fwd = |c: [f32; 3]| { let c = match shaper { @@ -656,33 +779,86 @@ fn space_at(i: u32) -> &'static Space { &SPACES[(i as usize).min(SPACES.len() - 1)] } -/// The custom config chosen by an effect (Configuration ▸ Custom with a readable file). -fn custom_config(ctx: &EffectCtx) -> Option> { - if ctx.params.e("config") != 1 { - return None; +/// The effects that take Configuration ▸ Custom (a `.ocio` file). +pub(crate) const CUSTOM_CONFIG_FX: [&str; 2] = ["ec.color.ociocolorspace", "ec.color.ociodisplay"]; + +/// Effect `id`'s transform under Configuration ▸ Custom: the config's conversion from the source +/// space to the destination space (Color Space Transform) or through a display's view (Display +/// Transform), see [`crate::ocio_config`]. `Ok(None)` with the built-in configuration; an error +/// (the pixels pass through) when the config can't be read or has no such colour space. +fn custom_xf(id: &str, pr: &crate::Params) -> Result, String> { + use crate::ocio_config::Xf; + if !CUSTOM_CONFIG_FX.contains(&id) || pr.e("config") != 1 { + return Ok(None); } - crate::ocio_config::load_config(ctx.params.s("configFile")) + let cfg = crate::ocio_config::load_config(pr.s("configFile")).map_err(|e| format!("Can't read the OCIO config: {e}"))?; + // A space by its typed name, else by the built-in popup's name. + let space = |name: &str, popup: u32| { + let name = if name.trim().is_empty() { space_at(popup).name } else { name.trim() }; + cfg.space(name).ok_or_else(|| format!("The OCIO config has no color space `{name}`")) + }; + let a = space(pr.s("sourceName"), pr.e("source"))?; + let inverse = pr.e("direction") == 1; + if id == "ec.color.ociodisplay" { + let (dn, vn) = display_view_names(pr); + let xf = cfg.display_path(a, dn, vn); + return Ok(Some(if inverse { Xf::Inverse(Box::new(xf)) } else { xf })); + } + let z = space(pr.s("destinationName"), pr.e("destination"))?; + // Inverse: the conversion the other way (OCIO's inverse ColorSpaceTransform). + Ok(Some(if inverse { cfg.path(z, a) } else { cfg.path(a, z) })) } -/// A space of a custom config by its typed name, else by the built-in popup's name. -fn custom_space<'c>(cfg: &'c crate::ocio_config::Config, name: &str, popup: u32) -> Option<&'c crate::ocio_config::ConfigSpace> { - cfg.space(name).or_else(|| cfg.space(space_at(popup).name)) +/// The display and view a Display Transform names: the typed names, else the popups'. +fn display_view_names(pr: &crate::Params) -> (&str, &str) { + let display = DISPLAYS.get(pr.e("display") as usize).map_or("", |d| d.0); + let view = VIEWS.get(pr.e("view") as usize).copied().unwrap_or(""); + let typed = |k: &str, popup| if pr.s(k).trim().is_empty() { popup } else { pr.s(k).trim() }; + (typed("displayName", display), typed("viewName", view)) +} + +/// Effect Controls' warning for effect `id`: a custom OCIO configuration that can't be read, has +/// no such colour space, display or view, or uses transforms EffectCraft can't apply (colours +/// pass through those steps, #410). `None` when the effect renders as asked. +pub fn warning(id: &str, pr: &crate::Params) -> Option { + let xf = match custom_xf(id, pr) { + Ok(x) => x?, + Err(e) => return Some(format!("{e}: the colors pass through.")), + }; + let mut notes = vec![]; + if id == "ec.color.ociodisplay" + && let Ok(cfg) = crate::ocio_config::load_config(pr.s("configFile")) + { + let (dn, vn) = display_view_names(pr); + if let Some((d, v)) = cfg.view(dn, vn) + && (!d.eq_ignore_ascii_case(dn) || !v.name.eq_ignore_ascii_case(vn)) + { + notes.push(format!("The OCIO config has no display `{dn}` with a view `{vn}`: showing {d} / {}.", v.name)); + } + } + let missing = xf.unsupported_list(); + if !missing.is_empty() { + notes.push(format!("Not supported, the colors pass through: {}.", missing.join(", "))); + } + (!notes.is_empty()).then(|| notes.join(" ")) +} + +/// Configuration ▸ Custom: render through the config's transform. `false` with the built-in +/// configuration (the caller renders it). +fn custom_render(id: &str, ctx: &EffectCtx, b: &mut Buf) -> bool { + match custom_xf(id, ctx.params) { + Ok(None) => false, + Ok(Some(xf)) => { + map_px(b, |c| xf.apply(c.map(|v| v as f64), false).map(|v| v as f32)); + true + } + // A config that can't be read, or an unknown space: the pixels pass through. + Err(_) => true, + } } fn ocio_cst(ctx: &EffectCtx, mut b: Buf) -> Buf { - // Custom configuration that can't be read: pixels pass through. - if ctx.params.e("config") == 1 && custom_config(ctx).is_none() { - return b; - } - if let Some(cfg) = custom_config(ctx) { - let (Some(a), Some(z)) = ( - custom_space(&cfg, ctx.params.s("sourceName"), ctx.params.e("source")), - custom_space(&cfg, ctx.params.s("destinationName"), ctx.params.e("destination")), - ) else { - return b; - }; - let (a, z) = if ctx.params.e("direction") == 1 { (z, a) } else { (a, z) }; - map_px(&mut b, |c| cfg.convert(a, z, c.map(|v| v as f64)).map(|v| v as f32)); + if custom_render("ec.color.ociocolorspace", ctx, &mut b) { return b; } let (a, z) = (space_at(ctx.params.e("source")), space_at(ctx.params.e("destination"))); @@ -720,19 +896,7 @@ fn ocio_cdl(ctx: &EffectCtx, mut b: Buf) -> Buf { } fn ocio_display(ctx: &EffectCtx, mut b: Buf) -> Buf { - // Custom configuration that can't be read: pixels pass through. - if ctx.params.e("config") == 1 && custom_config(ctx).is_none() { - return b; - } - if let Some(cfg) = custom_config(ctx) { - // The display's view names a colour space of the config. - let display = DISPLAYS.get(ctx.params.e("display") as usize).map_or("", |d| d.0); - let view = VIEWS.get(ctx.params.e("view") as usize).copied().unwrap_or(""); - let dn = if ctx.params.s("displayName").is_empty() { display } else { ctx.params.s("displayName") }; - let vn = if ctx.params.s("viewName").is_empty() { view } else { ctx.params.s("viewName") }; - let (Some(a), Some(z)) = (custom_space(&cfg, ctx.params.s("sourceName"), ctx.params.e("source")), cfg.view_space(dn, vn)) else { return b }; - let (a, z) = if ctx.params.e("direction") == 1 { (z, a) } else { (a, z) }; - map_px(&mut b, |c| cfg.convert(a, z, c.map(|v| v as f64)).map(|v| v as f32)); + if custom_render("ec.color.ociodisplay", ctx, &mut b) { return b; } let input = space_at(ctx.params.e("source")); @@ -899,9 +1063,26 @@ pub enum ColorOp { Pow([f32; 3]), /// LogTransform / LogAffineTransform, forward or inverse (as `ocio_config::Xf::apply`). LogAffine { base: f32, log_slope: [f32; 3], log_offset: [f32; 3], lin_slope: [f32; 3], lin_offset: [f32; 3], inverse: bool }, + /// ExponentWithLinearTransform, forward (decode) or inverse (as + /// [`crate::ocio_config::moncurve`]). + MonCurve { gamma: [f32; 3], offset: [f32; 3], inverse: bool, mirror: bool }, + /// Clamp each channel to `lo..=hi` (a clamping RangeTransform; ±`f32::MAX` = unbounded). + Clamp { lo: [f32; 3], hi: [f32; 3] }, } -/// The ops of a LUT / CDL file transform (as [`FileXform::apply`]). +/// The ops of a matrix-and-offset transform (MatrixTransform, `.spimtx`), forward or inverse +/// (as [`crate::ocio_config::matrix_apply`]). +fn matrix_ops(m: &[f64; 16], offset: &[f64; 4], inverse: bool, ops: &mut Vec) { + if !inverse { + let m = [0, 1, 2].map(|r| [m[r * 4] as f32, m[r * 4 + 1] as f32, m[r * 4 + 2] as f32]); + ops.push(ColorOp::Affine { m, pre: [0.0; 3], post: [offset[0] as f32, offset[1] as f32, offset[2] as f32] }); + } else if let Some(inv) = crate::ocio_config::mat3_inverse(m) { + let m = inv.map(|r| r.map(|v| v as f32)); + ops.push(ColorOp::Affine { m, pre: [-offset[0] as f32, -offset[1] as f32, -offset[2] as f32], post: [0.0; 3] }); + } +} + +/// The ops of a LUT / CDL / matrix file transform (as [`FileXform::apply`]). fn file_ops(x: &FileXform, interp: u32, inverse: bool, ccc_id: &str, ops: &mut Vec) { match x { FileXform::Cdl(list) => { @@ -911,6 +1092,7 @@ fn file_ops(x: &FileXform, interp: u32, inverse: bool, ccc_id: &str, ops: &mut V FileXform::Lut { lut, shaper } => { ops.push(ColorOp::Lut { lut: Arc::new(lut.clone()), shaper: shaper.clone().map(Arc::new), interp, inverse }); } + FileXform::Matrix { m, offset } => matrix_ops(m, offset, inverse, ops), } } @@ -920,16 +1102,12 @@ fn config_xf_ops(x: &crate::ocio_config::Xf, inverse: bool, ops: &mut Vec config_xf_ops(x, !inverse, ops), - Xf::Matrix { m, offset } => { - if !inverse { - let m = [0, 1, 2].map(|r| [m[r * 4] as f32, m[r * 4 + 1] as f32, m[r * 4 + 2] as f32]); - ops.push(ColorOp::Affine { m, pre: [0.0; 3], post: [offset[0] as f32, offset[1] as f32, offset[2] as f32] }); - } else if let Some(inv) = crate::ocio_config::mat3_inverse(m) { - let m = inv.map(|r| r.map(|v| v as f32)); - ops.push(ColorOp::Affine { m, pre: [-offset[0] as f32, -offset[1] as f32, -offset[2] as f32], post: [0.0; 3] }); - } - } + Xf::Matrix { m, offset } => matrix_ops(m, offset, inverse, ops), Xf::File { xf: Some(x), interp, ccc, .. } => file_ops(x, *interp, inverse, ccc, ops), + Xf::ExponentLinear { gamma, offset, mirror } => ops.push(ColorOp::MonCurve { gamma: f3(gamma), offset: f3(offset), inverse, mirror: *mirror }), + // A BuiltinTransform's curve: forward decodes. + Xf::Curve(tf) => ops.push(if inverse { ColorOp::Encode(*tf) } else { ColorOp::Decode(*tf) }), + Xf::Named(_) => {} Xf::File { xf: None, .. } | Xf::Unsupported(_) => {} Xf::Exponent(e) => ops.push(ColorOp::Pow([0, 1, 2].map(|k| (if inverse { 1.0 / e[k].max(1e-9) } else { e[k] }) as f32))), Xf::Log { base } => { @@ -944,11 +1122,14 @@ fn config_xf_ops(x: &crate::ocio_config::Xf, inverse: bool, ops: &mut Vec ops.push(ColorOp::Cdl { cdl: *cdl, clamp: true, inverse }), - Xf::Range { min_in, max_in, min_out, max_out } => { - let (a0, a1, b0, b1) = if inverse { (*min_out, *max_out, *min_in, *max_in) } else { (*min_in, *max_in, *min_out, *max_out) }; - let k = if (a1 - a0).abs() > 1e-12 { (b1 - b0) / (a1 - a0) } else { 0.0 }; + Xf::Range(r) => { + let (k, off, lo, hi) = r.parts(inverse); let k = k as f32; - ops.push(ColorOp::Affine { m: [[k, 0.0, 0.0], [0.0, k, 0.0], [0.0, 0.0, k]], pre: [-a0 as f32; 3], post: [b0 as f32; 3] }); + ops.push(ColorOp::Affine { m: [[k, 0.0, 0.0], [0.0, k, 0.0], [0.0, 0.0, k]], pre: [0.0; 3], post: [off as f32; 3] }); + if lo.is_finite() || hi.is_finite() { + let bound = |v: f64| (v as f32).clamp(-f32::MAX, f32::MAX); + ops.push(ColorOp::Clamp { lo: [bound(lo); 3], hi: [bound(hi); 3] }); + } } Xf::Group(list) => { if inverse { @@ -960,23 +1141,6 @@ fn config_xf_ops(x: &crate::ocio_config::Xf, inverse: bool, ops: &mut Vec) { - if a.is_data || z.is_data || a.name == z.name { - return; - } - match (&a.to_ref, &a.from_ref) { - (Some(x), _) => config_xf_ops(x, false, ops), - (None, Some(x)) => config_xf_ops(x, true, ops), - _ => {} - } - match (&z.from_ref, &z.to_ref) { - (Some(x), _) => config_xf_ops(x, false, ops), - (None, Some(x)) => config_xf_ops(x, true, ops), - _ => {} - } -} - /// How a colour program meets premultiplied pixels. #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum Straight { @@ -1004,6 +1168,16 @@ fn xform_ops(x: &Xform, ops: &mut Vec) { pub fn color_program(id: &str, ctx: &EffectCtx) -> Option<(Vec, Straight)> { let pr = ctx.params; let mut ops = vec![]; + // Configuration ▸ Custom compiles the config's transforms (a config that can't be read, or + // names unknown spaces, passes the pixels through). + match custom_xf(id, pr) { + Ok(None) => {} + Ok(Some(xf)) => { + config_xf_ops(&xf, false, &mut ops); + return Some((ops, Straight::Unpremul)); + } + Err(_) => return Some((ops, Straight::Unpremul)), + } match id { "ec.utility.applylut" => { if let Some(lut) = crate::utility::load_lut(pr.s("lut")) { @@ -1012,18 +1186,6 @@ pub fn color_program(id: &str, ctx: &EffectCtx) -> Option<(Vec, Straigh return Some((ops, Straight::Divide)); } "ec.color.ociocolorspace" => { - if pr.e("config") == 1 { - // A custom config (one that can't be read, or names unknown spaces: the pixels - // pass through). - if let Some(cfg) = custom_config(ctx) - && let (Some(a), Some(z)) = - (custom_space(&cfg, pr.s("sourceName"), pr.e("source")), custom_space(&cfg, pr.s("destinationName"), pr.e("destination"))) - { - let (a, z) = if pr.e("direction") == 1 { (z, a) } else { (a, z) }; - config_ops(a, z, &mut ops); - } - return Some((ops, Straight::Unpremul)); - } let (a, z) = (space_at(pr.e("source")), space_at(pr.e("destination"))); let (a, z) = if pr.e("direction") == 1 { (z, a) } else { (a, z) }; if a.name != z.name { @@ -1037,19 +1199,6 @@ pub fn color_program(id: &str, ctx: &EffectCtx) -> Option<(Vec, Straigh } } "ec.color.ociodisplay" => { - if pr.e("config") == 1 { - if let Some(cfg) = custom_config(ctx) { - let display = DISPLAYS.get(pr.e("display") as usize).map_or("", |d| d.0); - let view = VIEWS.get(pr.e("view") as usize).copied().unwrap_or(""); - let dn = if pr.s("displayName").is_empty() { display } else { pr.s("displayName") }; - let vn = if pr.s("viewName").is_empty() { view } else { pr.s("viewName") }; - if let (Some(a), Some(z)) = (custom_space(&cfg, pr.s("sourceName"), pr.e("source")), cfg.view_space(dn, vn)) { - let (a, z) = if pr.e("direction") == 1 { (z, a) } else { (a, z) }; - config_ops(a, z, &mut ops); - } - } - return Some((ops, Straight::Unpremul)); - } let input = space_at(pr.e("source")); let (display, view, inverse) = (pr.e("display") as usize, pr.e("view") as usize, pr.e("direction") == 1); let (_, prims, tf) = DISPLAYS[display.min(DISPLAYS.len() - 1)]; @@ -1474,6 +1623,44 @@ mod tests { v } + /// Sony Imageworks `.spi1d`, `.spi3d` and `.spimtx` (Blender's configs use them, #410), and + /// malformed ones rejected without panicking. + #[test] + fn spi_files_parse() { + let spi1d = "Version 1\nFrom 0.0 2.0\nLength 3\nComponents 1\n{\n0.0\n0.25\n1.0\n}\n"; + let x = parse_file(spi1d, "spi1d").unwrap(); + let o = x.apply([1.0, 0.5, 2.0], 1, false, ""); + assert!((o[0] - 0.25).abs() < 1e-6 && (o[1] - 0.125).abs() < 1e-6 && (o[2] - 1.0).abs() < 1e-6, "{o:?}"); + let back = x.apply(o, 1, true, ""); + assert!((back[0] - 1.0).abs() < 1e-5 && (back[2] - 2.0).abs() < 1e-5, "{back:?}"); + let three = parse_spi1d("Version 1\nFrom 0 1\nLength 2\nComponents 3\n{\n0 0 0\n1 0.5 2\n}\n").unwrap(); + assert_eq!(three.d1, vec![[0.0; 3], [1.0, 0.5, 2.0]]); + let mut spi3d = String::from("SPILUT 1.0\n3 3\n2 2 2\n"); + for (r, g, b) in (0..8).map(|i| (i & 1, (i >> 1) & 1, i >> 2)) { + spi3d += &format!("{r} {g} {b} {} {} {}\n", r as f32 * 0.5, g, b); + } + let x = parse_file(&spi3d, "spi3d").unwrap(); + let o = x.apply([1.0, 0.5, 0.25], 1, false, ""); + assert!((o[0] - 0.5).abs() < 1e-6 && (o[1] - 0.5).abs() < 1e-6 && (o[2] - 0.25).abs() < 1e-6, "{o:?}"); + let m = parse_file("2 0 0 6553.5\n0 1 0 0\n0 0 1 0\n", "spimtx").unwrap(); + let o = m.apply([0.25, 0.5, 1.0], 1, false, ""); + assert!((o[0] - 0.6).abs() < 1e-6 && o[1] == 0.5 && o[2] == 1.0, "{o:?}"); + assert!((m.apply(o, 1, true, "")[0] - 0.25).abs() < 1e-6); + for bad in [ + "Version 1\nFrom 0 1\nLength 99999999999\nComponents 1\n{\n0\n}\n", + "Version 1\nFrom 1 0\nLength 2\nComponents 1\n{\n0\n1\n}\n", + "Version 1\nFrom 0 1\nLength 3\nComponents 1\n{\n0\n1\n}\n", + "Version 1\nFrom 0 1\nLength 2\nComponents 2\n{\n0 0\n1 1\n}\n", + ] { + assert!(parse_spi1d(bad).is_none(), "{bad}"); + } + assert!(parse_spi3d("SPILUT 1.0\n3 3\n4000 4000 4000\n").is_none()); + assert!(parse_spi3d("SPILUT 1.0\n3 3\n2 2 2\n9 0 0 1 1 1\n").is_none()); + assert!(parse_spi3d("SPILUT 1.0\n3 3\n2 2 2\n0 0 0 1 1 1\n").is_none(), "incomplete lattice"); + assert!(parse_spimtx("1 0 0\n0 1 0\n").is_none()); + assert!(parse_spimtx("NaN 0 0 0 0 1 0 0 0 0 1 0").is_none()); + } + #[test] fn lut_files_parse_and_interpolate() { // .cube with a non-linear lattice: trilinear between lattice points. diff --git a/crates/effects/src/ocio_config.rs b/crates/effects/src/ocio_config.rs index 687534e9..5ea84441 100644 --- a/crates/effects/src/ocio_config.rs +++ b/crates/effects/src/ocio_config.rs @@ -1,33 +1,60 @@ //! Custom OpenColorIO configuration files (`.ocio`) for the OCIO effects, read from the -//! published configuration format (a YAML document), not from OpenColorIO's source. +//! published configuration format (a YAML document) and the OpenColorIO v2 documentation of its +//! transforms, not from OpenColorIO's source. //! //! Supported subset: //! //! * `search_path` (a string with `:`-separated entries or a list), resolved against the //! config's folder, for `FileTransform` sources; //! * `roles` (any role name can stand for its colour space); -//! * `colorspaces`: `name`, `aliases`, `isdata`, and the transforms to / from the reference -//! space — `to_reference` / `from_reference` (OCIO v1) or `to_scene_reference` / -//! `from_scene_reference` (v2); a space with only one direction is inverted for the other; -//! * `displays`: display → views (`! {name, colorspace}`); -//! * transforms: `MatrixTransform` (`matrix` 4×4 row-major, `offset`), `FileTransform` (the LUT -//! and CDL files [`super::ocio::parse_file`] reads; `interpolation`, `ccc_id`), -//! `ExponentTransform` (`value`), `LogTransform` (`base`), `LogAffineTransform` (`base`, -//! `logSideSlope`, `logSideOffset`, `linSideSlope`, `linSideOffset`), `CDLTransform` (`slope`, -//! `offset`, `power`, `sat`), `RangeTransform` (`min_in_value`, `max_in_value`, -//! `min_out_value`, `max_out_value`) and `GroupTransform` (`children`), each with -//! `direction: inverse`. Other transforms are reported by [`Config::unsupported`] and pass -//! colours through. +//! * `colorspaces` (scene-referred) and `display_colorspaces` (display-referred): `name`, +//! `aliases`, `isdata`, and the transforms to / from their reference space — +//! `to_reference` / `from_reference` (OCIO v1), `to_scene_reference` / `from_scene_reference` +//! and `to_display_reference` / `from_display_reference` (v2); a space with only one direction +//! is inverted for the other; +//! * the two reference spaces of v2: a conversion between a scene-referred and a +//! display-referred space goes through the default view transform (`default_view_transform`, +//! else the first scene-referred one in `view_transforms`), as OCIO documents; +//! * `displays`: display → views, either `! {name, colorspace, looks}` or +//! `! {name, view_transform, display_colorspace, looks}` (`` too); +//! * `looks` (`name`, `process_space`, `transform`, `inverse_transform`) and look strings +//! (`+A, -B`, `A | B` = the first one the config has); +//! * transforms: `MatrixTransform` (`matrix` 4×4 row-major, `offset`), `FileTransform` (the LUT, +//! CDL and matrix files [`super::ocio::parse_file`] reads; `interpolation`, `ccc_id`), +//! `ExponentTransform` (`value`), `ExponentWithLinearTransform` (`gamma`, `offset`, `style: +//! mirror`; the power curve with a linear toe that the Academy CLF specification calls +//! "moncurve"), `LogTransform` (`base`), `LogAffineTransform` (`base`, `logSideSlope`, +//! `logSideOffset`, `linSideSlope`, `linSideOffset`), `CDLTransform` (`slope`, `offset`, +//! `power`, `sat`), `RangeTransform` (any of `min_in_value`, `max_in_value`, +//! `min_out_value`, `max_out_value`; clamping unless `style: noClamp`), `AllocationTransform` +//! (`allocation: uniform | lg2`, `vars`), `ColorSpaceTransform` (`src`, `dst`, `data_bypass`), +//! `LookTransform` (`src`, `dst`, `looks`), `DisplayViewTransform` (`src`, `display`, `view`, +//! `looks_bypass`, `data_bypass`), `BuiltinTransform` (the styles [`builtin`] lists) and +//! `GroupTransform` (`children`), each with `direction: inverse`. +//! +//! Other transforms (`GradingPrimaryTransform`, `GradingToneTransform`, the ACES output +//! transforms…) pass colours through; [`Config::unsupported`] lists them for a config and +//! [`Xf::unsupported_list`] for one conversion, and the effects show that as a warning. //! //! The YAML reader covers what configs use: block mappings and sequences, flow `{}` / `[]` //! collections (also spanning lines), `!` tags, quoted scalars, `|` / `>` block scalars -//! and comments. +//! and comments. Nesting deeper than [`MAX_YAML_DEPTH`] is cut off, and a conversion stops +//! resolving named transforms [`MAX_NAMED_DEPTH`] levels down or past [`MAX_OPS`] steps +//! (a config that refers to itself in a loop), so a hostile config can't exhaust the stack. use std::collections::HashMap; use std::path::{Path, PathBuf}; use std::sync::{Arc, Mutex, OnceLock}; -use super::ocio::{Cdl, FileXform, load_file}; +use super::ocio::{ACES_CG, ACES2065_1, Cdl, D65, FileXform, P3, REC709, REC2020, Space, Tf, load_file, matrix, sp}; + +/// Deepest YAML nesting read (configs use about ten levels). +pub const MAX_YAML_DEPTH: usize = 64; +/// Deepest chain of named transforms (a ColorSpaceTransform naming a space whose transform +/// names another…) one conversion resolves. +pub const MAX_NAMED_DEPTH: usize = 16; +/// Most steps one conversion has. +pub const MAX_OPS: usize = 4096; // ---------------------------------------------------------------- YAML subset @@ -81,6 +108,18 @@ impl Yaml { _ => vec![], } } + /// A string value of key `k` (`""` when missing). + fn text(&self, k: &str) -> &str { + self.get(k).and_then(Yaml::str).unwrap_or("") + } + /// A boolean value of key `k` (`true` / `false`), `default` when missing. + fn flag(&self, k: &str, default: bool) -> bool { + match self.text(k).trim().to_ascii_lowercase().as_str() { + "true" | "yes" => true, + "false" | "no" => false, + _ => default, + } + } } /// Strip a trailing comment (a `#` outside quotes, at the start or after whitespace). @@ -131,13 +170,13 @@ fn split_key(s: &str) -> Option<(String, &str)> { /// Bracket balance of a flow fragment (outside quotes). fn balance(s: &str) -> i32 { - let (mut sq, mut dq, mut d) = (false, false, 0); + let (mut sq, mut dq, mut d) = (false, false, 0i32); for c in s.chars() { match c { '\'' if !dq => sq = !sq, '"' if !sq => dq = !dq, - '{' | '[' if !sq && !dq => d += 1, - '}' | ']' if !sq && !dq => d -= 1, + '{' | '[' if !sq && !dq => d = d.saturating_add(1), + '}' | ']' if !sq && !dq => d = d.saturating_sub(1), _ => {} } } @@ -147,6 +186,7 @@ fn balance(s: &str) -> i32 { struct Flow<'a> { s: &'a [u8], i: usize, + depth: usize, } impl Flow<'_> { @@ -158,6 +198,17 @@ impl Flow<'_> { fn peek(&self) -> Option { self.s.get(self.i).copied() } + /// A nested value (cut off past [`MAX_YAML_DEPTH`]: the rest of the text is skipped). + fn nested(&mut self) -> Yaml { + if self.depth >= MAX_YAML_DEPTH { + self.i = self.s.len(); + return Yaml::Null; + } + self.depth += 1; + let v = self.value(); + self.depth -= 1; + v + } fn value(&mut self) -> Yaml { self.ws(); match self.peek() { @@ -183,7 +234,7 @@ impl Flow<'_> { if self.peek() == Some(b':') { self.i += 1; } - let v = self.value(); + let v = self.nested(); m.push((unquote(&k), v)); } Yaml::Map(m) @@ -203,7 +254,7 @@ impl Flow<'_> { self.i += 1; continue; } - _ => v.push(self.value()), + _ => v.push(self.nested()), } } Yaml::Seq(v) @@ -216,7 +267,7 @@ impl Flow<'_> { let tag = tag_name(std::str::from_utf8(&self.s[start..self.i]).unwrap_or("")); self.ws(); let inner = match self.peek() { - Some(b'{') | Some(b'[') => self.value(), + Some(b'{') | Some(b'[') => self.nested(), _ => Yaml::Null, }; Yaml::Tagged(tag, Box::new(inner)) @@ -257,8 +308,8 @@ fn tag_name(t: &str) -> String { t.trim().trim_start_matches('!').trim_start_matches('<').trim_end_matches('>').to_string() } -fn flow(s: &str) -> Yaml { - Flow { s: s.as_bytes(), i: 0 }.value() +fn flow(s: &str, depth: usize) -> Yaml { + Flow { s: s.as_bytes(), i: 0, depth }.value() } struct Lines { @@ -281,13 +332,13 @@ impl Lines { } /// The value of a `key:` / `- ` remainder `rest` whose children are indented more than - /// `indent` (from line `*i`). - fn value(&self, rest: &str, indent: usize, i: &mut usize) -> Yaml { + /// `indent` (from line `*i`), `depth` levels down. + fn value(&self, rest: &str, indent: usize, i: &mut usize, depth: usize) -> Yaml { let rest = rest.trim(); if rest.is_empty() { return match self.v.get(*i) { // A sequence may sit at the key's own indent. - Some((ind, c)) if *ind > indent || (*ind == indent && (c == "-" || c.starts_with("- "))) => self.block(*ind, i), + Some((ind, c)) if *ind > indent || (*ind == indent && (c == "-" || c.starts_with("- "))) => self.block(*ind, i, depth + 1), _ => Yaml::Null, }; } @@ -305,7 +356,7 @@ impl Lines { if rest.starts_with('!') && !rest.contains('{') && !rest.contains('[') { let tag = tag_name(rest); let inner = match self.v.get(*i) { - Some((ind, _)) if *ind > indent => self.block(*ind, i), + Some((ind, _)) if *ind > indent => self.block(*ind, i, depth + 1), _ => Yaml::Null, }; return Yaml::Tagged(tag, Box::new(inner)); @@ -313,18 +364,25 @@ impl Lines { if rest.starts_with('{') || rest.starts_with('[') || (rest.starts_with('!') && (rest.contains('{') || rest.contains('['))) { // Flow collections may continue on the next lines. let mut text = rest.to_string(); - while balance(&text) > 0 && *i < self.v.len() { + while balance(&text) > 0 + && let Some((_, next)) = self.v.get(*i) + { text.push(' '); - text.push_str(&self.v[*i].1); + text.push_str(next); *i += 1; } - return flow(&text); + return flow(&text, depth); } Yaml::Str(unquote(rest)) } - /// A block node whose lines start at `indent`. - fn block(&self, indent: usize, i: &mut usize) -> Yaml { + /// A block node whose lines start at `indent`, `depth` levels down (cut off past + /// [`MAX_YAML_DEPTH`]: the rest of the document is skipped). + fn block(&self, indent: usize, i: &mut usize, depth: usize) -> Yaml { + if depth >= MAX_YAML_DEPTH { + *i = self.v.len(); + return Yaml::Null; + } let Some((_, first)) = self.v.get(*i) else { return Yaml::Null }; if first == "-" || first.starts_with("- ") { let mut items = vec![]; @@ -337,27 +395,27 @@ impl Lines { // `- key: value` opens a mapping whose keys are indented to the key. if split_key(&rest).is_some() && !rest.starts_with('{') && !rest.starts_with('!') { let inner = indent + 1 + (c.len() - 1 - c[1..].trim_start().len()); - items.push(self.map_from(Some(rest), inner.max(indent + 2), i)); + items.push(self.map_from(Some(rest), inner.max(indent + 2), i, depth + 1)); } else { - items.push(self.value(&rest, indent, i)); + items.push(self.value(&rest, indent, i, depth)); } } return Yaml::Seq(items); } if split_key(first).is_some() { - return self.map_from(None, indent, i); + return self.map_from(None, indent, i, depth); } *i += 1; Yaml::Str(unquote(first)) } /// A block mapping at `indent`; `first` is an entry already taken from a `- ` line. - fn map_from(&self, first: Option, indent: usize, i: &mut usize) -> Yaml { + fn map_from(&self, first: Option, indent: usize, i: &mut usize, depth: usize) -> Yaml { let mut m = vec![]; if let Some(f) = first && let Some((k, rest)) = split_key(&f) { - let v = self.value(rest, indent, i); + let v = self.value(rest, indent, i, depth); m.push((k, v)); } while let Some((ind, c)) = self.v.get(*i) { @@ -371,7 +429,7 @@ impl Lines { } let Some((k, rest)) = split_key(c) else { break }; *i += 1; - let v = self.value(rest, indent, i); + let v = self.value(rest, indent, i, depth); m.push((k, v)); } Yaml::Map(m) @@ -383,12 +441,13 @@ pub fn parse_yaml(text: &str) -> Yaml { let l = Lines::new(text); let mut i = 0; let indent = l.v.first().map_or(0, |(ind, _)| *ind); - l.block(indent, &mut i) + l.block(indent, &mut i, 0) } // ---------------------------------------------------------------- transforms -/// A colour transform of a config. +/// A colour transform of a config. As in OCIO, the forward direction of ExponentTransform, +/// ExponentWithLinearTransform and [`Xf::Curve`] decodes (encoded → linear). #[derive(Clone, Debug)] pub enum Xf { /// 4×4 row-major matrix (RGB of the first three rows/columns) plus offset. @@ -403,6 +462,13 @@ pub enum Xf { ccc: String, }, Exponent([f64; 3]), + /// ExponentWithLinearTransform ([`moncurve`]); `mirror` = `style: mirror` (else negative + /// values continue the linear segment). + ExponentLinear { + gamma: [f64; 3], + offset: [f64; 3], + mirror: bool, + }, Log { base: f64, }, @@ -414,29 +480,97 @@ pub enum Xf { lin_offset: [f64; 3], }, Cdl(Cdl), - Range { - min_in: f64, - max_in: f64, - min_out: f64, - max_out: f64, - }, + Range(RangeXf), + /// A transfer function of the built-in configuration (BuiltinTransform), forward = decode. + Curve(Tf), Group(Vec), Inverse(Box), + /// A transform that names other parts of the config, replaced by their transforms when a + /// conversion is built ([`Config::path`]). + Named(Box), /// An unsupported transform (by tag): identity. Unsupported(String), } -fn vec3_of(y: Option<&Yaml>, d: f64) -> [f64; 3] { - let v = y.map(Yaml::nums).unwrap_or_default(); - match v.len() { - 0 => [d; 3], - 1 | 2 => [v[0]; 3], - _ => [v[0], v[1], v[2]], +/// A transform naming parts of the config (see [`Xf::Named`]). +#[derive(Clone, Debug)] +pub enum Named { + /// ColorSpaceTransform: `src` → `dst`. + Space { src: String, dst: String, data_bypass: bool }, + /// LookTransform: `src` → the looks → `dst`. + Look { src: String, dst: String, looks: String }, + /// DisplayViewTransform: `src` through a display's view. + DisplayView { src: String, display: String, view: String, looks_bypass: bool, data_bypass: bool }, +} + +/// RangeTransform: each bound pair is optional (`min_in` with `min_out`, `max_in` with +/// `max_out`). +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct RangeXf { + pub min_in: Option, + pub max_in: Option, + pub min_out: Option, + pub max_out: Option, + /// Clamp to the output range (`style: Clamp`, the default). + pub clamp: bool, +} + +impl RangeXf { + /// (scale, offset, low, high) of the forward (or inverse) range: `v · scale + offset` + /// clamped to `low..=high` (±∞ = unbounded). Both pairs map the input range onto the + /// output range; one pair only shifts (and clamps on that side). + pub fn parts(&self, inverse: bool) -> (f64, f64, f64, f64) { + let (lo_in, hi_in, lo_out, hi_out) = + if inverse { (self.min_out, self.max_out, self.min_in, self.max_in) } else { (self.min_in, self.max_in, self.min_out, self.max_out) }; + let lo = lo_in.and(lo_out); + let hi = hi_in.and(hi_out); + let (k, off) = match (lo_in, hi_in, lo_out, hi_out) { + (Some(a0), Some(a1), Some(b0), Some(b1)) if (a1 - a0).abs() > 1e-12 => { + let k = (b1 - b0) / (a1 - a0); + (k, b0 - a0 * k) + } + (Some(a0), _, Some(b0), _) => (1.0, b0 - a0), + (_, Some(a1), _, Some(b1)) => (1.0, b1 - a1), + _ => (1.0, 0.0), + }; + let bound = |v: Option, unbounded: f64| v.filter(|_| self.clamp).filter(|v| v.is_finite()).unwrap_or(unbounded); + (k, off, bound(lo, f64::NEG_INFINITY), bound(hi, f64::INFINITY)) + } + + fn apply(&self, c: [f64; 3], inverse: bool) -> [f64; 3] { + let (k, off, lo, hi) = self.parts(inverse); + // Comparisons, not `clamp`: never panics on a reversed range and keeps NaN. + c.map(|v| { + let v = v * k + off; + if v < lo { + lo + } else if v > hi { + hi + } else { + v + } + }) } } -fn mat_apply(m: &[f64; 16], off: &[f64; 4], c: [f64; 3]) -> [f64; 3] { - [0, 1, 2].map(|r| m[r * 4] * c[0] + m[r * 4 + 1] * c[1] + m[r * 4 + 2] * c[2] + off[r]) +fn vec3_of(y: Option<&Yaml>, d: f64) -> [f64; 3] { + let v = y.map(Yaml::nums).unwrap_or_default(); + match v.as_slice() { + [] => [d; 3], + [a] | [a, _] => [*a; 3], + [a, b, c, ..] => [*a, *b, *c], + } +} + +/// A matrix-and-offset transform (MatrixTransform, `.spimtx`) applied forward or inverse. +pub fn matrix_apply(m: &[f64; 16], off: &[f64; 4], c: [f64; 3], inverse: bool) -> [f64; 3] { + if !inverse { + return [0, 1, 2].map(|r| m[r * 4] * c[0] + m[r * 4 + 1] * c[1] + m[r * 4 + 2] * c[2] + off[r]); + } + match mat3_inverse(m) { + Some(inv) => effectcraft_color::space::mul_vec(&inv, [c[0] - off[0], c[1] - off[1], c[2] - off[2]]), + None => c, + } } pub(crate) fn mat3_inverse(m: &[f64; 16]) -> Option<[[f64; 3]; 3]> { @@ -450,6 +584,69 @@ fn log_base(v: f64, base: f64) -> f64 { v.max(f64::MIN_POSITIVE).ln() / base.ln() } +/// ExponentWithLinearTransform (the Academy CLF "moncurve"): forward decodes, `((v + offset) / +/// (1 + offset))^gamma` above the break point `offset / (gamma − 1)` and a straight line through +/// 0 that meets it below; inverse encodes. `mirror` mirrors negative values instead of +/// continuing the line. A gamma of at most 1 or a non-positive offset is a plain power curve. +pub fn moncurve(v: f64, gamma: f64, offset: f64, inverse: bool, mirror: bool) -> f64 { + if mirror && v < 0.0 { + return -moncurve(-v, gamma, offset, inverse, false); + } + if gamma <= 1.0 || offset <= 0.0 { + let p = if inverse { 1.0 / gamma.max(1e-9) } else { gamma }; + return v.max(0.0).powf(p); + } + let x_break = offset / (gamma - 1.0); + let y_break = ((x_break + offset) / (1.0 + offset)).powf(gamma); + // Slope of the linear segment (y = x · slope). + let slope = y_break / x_break; + if !inverse { + if v >= x_break { ((v + offset) / (1.0 + offset)).powf(gamma) } else { v * slope } + } else if v >= y_break { + (1.0 + offset) * v.powf(1.0 / gamma) - offset + } else { + v / slope + } +} + +/// A 3×3 matrix as a MatrixTransform. +fn m3(m: [[f64; 3]; 3]) -> Xf { + Xf::Matrix { m: [m[0][0], m[0][1], m[0][2], 0.0, m[1][0], m[1][1], m[1][2], 0.0, m[2][0], m[2][1], m[2][2], 0.0, 0.0, 0.0, 0.0, 1.0], offset: [0.0; 4] } +} + +/// The sRGB curve as OCIO defines it: ExponentWithLinearTransform with gamma 2.4, offset 0.055. +const SRGB_CURVE: Xf = Xf::ExponentLinear { gamma: [2.4; 3], offset: [0.055; 3], mirror: false }; + +/// A BuiltinTransform by its documented `style` (case-insensitive), built from the published +/// primaries and curves (Bradford adaptation where the name says `BFD`; PQ with 1.0 = 100 nits). +/// `None` for the styles not implemented (the ACES output transforms and gamut compression, +/// camera log curves, HLG). +pub fn builtin(style: &str) -> Option { + let xyz = Space { name: "XYZ", prims: None, tf: Tf::Linear, raw: false }; + let lin = |prims| sp("linear", prims, D65, Tf::Linear); + // CIE XYZ D65 → a display's linear RGB, then its encoding (inverse = encode). + let display = |prims, curve: Xf| Some(Xf::Group(vec![m3(matrix(&xyz, &lin(prims))), Xf::Inverse(Box::new(curve))])); + let gamma = |g: f64| Xf::Exponent([g; 3]); + let s = style.trim().to_ascii_uppercase(); + match s.as_str() { + "IDENTITY" => Some(Xf::Group(vec![])), + "UTILITY - ACES-AP0_TO_CIE-XYZ-D65_BFD" => Some(m3(ACES2065_1.to_ref(true))), + "UTILITY - ACES-AP1_TO_CIE-XYZ-D65_BFD" => Some(m3(ACES_CG.to_ref(true))), + "UTILITY - ACES-AP1_TO_LINEAR-REC709_BFD" => Some(m3(matrix(&ACES_CG, &lin(REC709)))), + "ACESCCT_TO_ACES2065-1" => Some(Xf::Group(vec![Xf::Curve(Tf::AcesCct), m3(matrix(&ACES_CG, &ACES2065_1))])), + "ACESCC_TO_ACES2065-1" => Some(Xf::Group(vec![Xf::Curve(Tf::AcesCc), m3(matrix(&ACES_CG, &ACES2065_1))])), + "DISPLAY - CIE-XYZ-D65_TO_SRGB" => display(REC709, SRGB_CURVE), + "DISPLAY - CIE-XYZ-D65_TO_DISPLAYP3" => display(P3, SRGB_CURVE), + "DISPLAY - CIE-XYZ-D65_TO_REC.1886-REC.709" => display(REC709, gamma(2.4)), + "DISPLAY - CIE-XYZ-D65_TO_REC.1886-REC.2020" => display(REC2020, gamma(2.4)), + "DISPLAY - CIE-XYZ-D65_TO_G2.2-REC.709" => display(REC709, gamma(2.2)), + "DISPLAY - CIE-XYZ-D65_TO_G2.6-P3-D65" => display(P3, gamma(2.6)), + "DISPLAY - CIE-XYZ-D65_TO_REC.2100-PQ" => display(REC2020, Xf::Curve(Tf::Pq)), + "DISPLAY - CIE-XYZ-D65_TO_ST2084-P3-D65" => display(P3, Xf::Curve(Tf::Pq)), + _ => None, + } +} + impl Xf { fn from_yaml(y: &Yaml, dir: &Path, search: &[String]) -> Xf { let tag = y.tag().unwrap_or(""); @@ -457,7 +654,7 @@ impl Xf { Yaml::Tagged(_, b) => b.as_ref(), other => other, }; - let inverse = body.get("direction").and_then(Yaml::str).is_some_and(|d| d.eq_ignore_ascii_case("inverse")); + let inverse = body.text("direction").eq_ignore_ascii_case("inverse"); let xf = match tag { "MatrixTransform" => { let v = body.get("matrix").map(Yaml::nums).unwrap_or_default(); @@ -479,7 +676,7 @@ impl Xf { Xf::Matrix { m, offset } } "FileTransform" => { - let src = body.get("src").and_then(Yaml::str).unwrap_or("").to_string(); + let src = body.text("src").to_string(); let interp = match body.get("interpolation").and_then(Yaml::str).unwrap_or("linear").to_ascii_lowercase().as_str() { "nearest" => 0, "tetrahedral" | "best" => 2, @@ -490,6 +687,11 @@ impl Xf { Xf::File { src, xf, interp, ccc } } "ExponentTransform" => Xf::Exponent(vec3_of(body.get("value"), 1.0)), + "ExponentWithLinearTransform" => Xf::ExponentLinear { + gamma: vec3_of(body.get("gamma"), 1.0), + offset: vec3_of(body.get("offset"), 0.0), + mirror: body.text("style").eq_ignore_ascii_case("mirror"), + }, "LogTransform" => Xf::Log { base: body.get("base").and_then(Yaml::f).unwrap_or(2.0) }, "LogAffineTransform" => Xf::LogAffine { base: body.get("base").and_then(Yaml::f).unwrap_or(2.0), @@ -504,34 +706,64 @@ impl Xf { power: vec3_of(body.get("power"), 1.0), sat: body.get("sat").and_then(Yaml::f).unwrap_or(1.0), }), - "RangeTransform" => Xf::Range { - min_in: body.get("min_in_value").and_then(Yaml::f).unwrap_or(0.0), - max_in: body.get("max_in_value").and_then(Yaml::f).unwrap_or(1.0), - min_out: body.get("min_out_value").and_then(Yaml::f).unwrap_or(0.0), - max_out: body.get("max_out_value").and_then(Yaml::f).unwrap_or(1.0), - }, + "RangeTransform" => Xf::Range(RangeXf { + min_in: body.get("min_in_value").and_then(Yaml::f), + max_in: body.get("max_in_value").and_then(Yaml::f), + min_out: body.get("min_out_value").and_then(Yaml::f), + max_out: body.get("max_out_value").and_then(Yaml::f), + clamp: !body.text("style").eq_ignore_ascii_case("noclamp"), + }), + // Uniform: the `vars` range onto 0..1; lg2: its log₂ (after adding the optional + // third var) onto 0..1. + "AllocationTransform" => { + let lg2 = body.text("allocation").eq_ignore_ascii_case("lg2"); + let vars = body.get("vars").map(Yaml::nums).unwrap_or_default(); + let (lo, hi) = match vars.as_slice() { + [lo, hi, ..] => (*lo, *hi), + _ if lg2 => (-15.0, 6.0), + _ => (0.0, 1.0), + }; + let span = hi - lo; + if !span.is_finite() || span.abs() < 1e-12 { + Xf::Unsupported("AllocationTransform (empty range)".into()) + } else if lg2 { + let off = vars.get(2).copied().unwrap_or(0.0); + Xf::LogAffine { base: 2.0, log_slope: [1.0 / span; 3], log_offset: [-lo / span; 3], lin_slope: [1.0; 3], lin_offset: [off; 3] } + } else { + Xf::Range(RangeXf { min_in: Some(lo), max_in: Some(hi), min_out: Some(0.0), max_out: Some(1.0), clamp: false }) + } + } + "BuiltinTransform" => { + let style = body.text("style"); + builtin(style).unwrap_or_else(|| Xf::Unsupported(format!("BuiltinTransform `{style}`"))) + } + "ColorSpaceTransform" => Xf::Named(Box::new(Named::Space { + src: body.text("src").to_string(), + dst: body.text("dst").to_string(), + data_bypass: body.flag("data_bypass", true), + })), + "LookTransform" => { + Xf::Named(Box::new(Named::Look { src: body.text("src").to_string(), dst: body.text("dst").to_string(), looks: body.text("looks").to_string() })) + } + "DisplayViewTransform" => Xf::Named(Box::new(Named::DisplayView { + src: body.text("src").to_string(), + display: body.text("display").to_string(), + view: body.text("view").to_string(), + looks_bypass: body.flag("looks_bypass", false), + data_bypass: body.flag("data_bypass", true), + })), "GroupTransform" => Xf::Group(body.get("children").map(|c| c.seq().iter().map(|t| Xf::from_yaml(t, dir, search)).collect()).unwrap_or_default()), other => Xf::Unsupported(other.to_string()), }; if inverse { Xf::Inverse(Box::new(xf)) } else { xf } } - /// Apply forward (or inverse) to linear-or-encoded RGB. + /// Apply forward (or inverse) to linear-or-encoded RGB. Named transforms pass through: + /// apply the conversions [`Config::path`] builds, where they are resolved. pub fn apply(&self, c: [f64; 3], inverse: bool) -> [f64; 3] { match self { Xf::Inverse(x) => x.apply(c, !inverse), - Xf::Matrix { m, offset } => { - if !inverse { - return mat_apply(m, offset, c); - } - match mat3_inverse(m) { - Some(inv) => { - let d = [c[0] - offset[0], c[1] - offset[1], c[2] - offset[2]]; - effectcraft_color::space::mul_vec(&inv, d) - } - None => c, - } - } + Xf::Matrix { m, offset } => matrix_apply(m, offset, c, inverse), Xf::File { xf, interp, ccc, .. } => match xf { Some(x) => x.apply(c.map(|v| v as f32), *interp, inverse, ccc).map(|v| v as f64), None => c, @@ -540,6 +772,7 @@ impl Xf { let p = if inverse { 1.0 / e[k].max(1e-9) } else { e[k] }; c[k].max(0.0).powf(p) }), + Xf::ExponentLinear { gamma, offset, mirror } => [0, 1, 2].map(|k| moncurve(c[k], gamma[k], offset[k], inverse, *mirror)), Xf::Log { base } => { if inverse { c.map(|v| base.powf(v)) @@ -561,10 +794,13 @@ impl Xf { cdl.apply(c, true) } } - Xf::Range { min_in, max_in, min_out, max_out } => { - let (a0, a1, b0, b1) = if inverse { (*min_out, *max_out, *min_in, *max_in) } else { (*min_in, *max_in, *min_out, *max_out) }; - let k = if (a1 - a0).abs() > 1e-12 { (b1 - b0) / (a1 - a0) } else { 0.0 }; - c.map(|v| b0 + (v - a0) * k) + Xf::Range(r) => r.apply(c, inverse), + Xf::Curve(tf) => { + if inverse { + c.map(|v| tf.encode(v)) + } else { + c.map(|v| tf.decode(v)) + } } Xf::Group(list) => { let mut v = c; @@ -579,7 +815,7 @@ impl Xf { } v } - Xf::Unsupported(_) => c, + Xf::Named(_) | Xf::Unsupported(_) => c, } } @@ -592,10 +828,23 @@ impl Xf { _ => {} } } + + /// The transforms of this one that can't be applied (and pass colours through), sorted and + /// without repeats: for warnings. + pub fn unsupported_list(&self) -> Vec { + let mut out = vec![]; + self.unsupported(&mut out); + out.sort(); + out.dedup(); + out + } } /// Find a FileTransform source on the search path (relative to the config's folder). fn resolve(src: &str, dir: &Path, search: &[String]) -> Option { + if src.trim().is_empty() { + return None; + } let p = Path::new(src); if p.is_absolute() { return p.exists().then(|| p.to_path_buf()); @@ -614,27 +863,48 @@ pub struct ConfigSpace { pub aliases: Vec, pub family: String, pub is_data: bool, + /// Display-referred (`display_colorspaces`): its reference is the display reference space. + pub display: bool, pub to_ref: Option, pub from_ref: Option, } -impl ConfigSpace { - fn to_reference(&self, c: [f64; 3]) -> [f64; 3] { - match (&self.to_ref, &self.from_ref) { - (Some(x), _) => x.apply(c, false), - (None, Some(x)) => x.apply(c, true), - _ => c, - } - } - fn reference_to(&self, c: [f64; 3]) -> [f64; 3] { - match (&self.from_ref, &self.to_ref) { - (Some(x), _) => x.apply(c, false), - (None, Some(x)) => x.apply(c, true), - _ => c, - } +/// A view of a display: a colour space (OCIO v1 style), or a view transform and a display +/// colour space (v2); either with looks. +#[derive(Clone, Debug, Default)] +pub struct View { + pub name: String, + pub colorspace: String, + pub view_transform: String, + pub display_colorspace: String, + pub looks: String, +} + +/// A view transform: scene reference → display reference (or display → display). +#[derive(Clone, Debug)] +pub struct ViewTransform { + pub name: String, + pub from_scene: Option, + pub to_scene: Option, + pub from_display: Option, + pub to_display: Option, +} + +impl ViewTransform { + fn scene(&self) -> bool { + self.from_scene.is_some() || self.to_scene.is_some() } } +/// A look: a transform applied in its process space. +#[derive(Clone, Debug)] +pub struct Look { + pub name: String, + pub process_space: String, + pub transform: Option, + pub inverse_transform: Option, +} + /// A parsed `.ocio` configuration. #[derive(Clone, Debug, Default)] pub struct Config { @@ -642,8 +912,11 @@ pub struct Config { pub search_path: Vec, pub roles: Vec<(String, String)>, pub spaces: Vec, - /// Display → [(view, colour space)]. - pub displays: Vec<(String, Vec<(String, String)>)>, + /// Display → views. + pub displays: Vec<(String, Vec)>, + pub view_transforms: Vec, + pub default_view_transform: String, + pub looks: Vec, } impl Config { @@ -662,18 +935,22 @@ impl Config { Some(Yaml::Map(m)) => m.iter().filter_map(|(k, v)| v.str().map(|s| (k.clone(), s.to_string()))).collect(), _ => vec![], }; + // The first of `keys` an entry has, as a transform. + let xf = |entry: &Yaml, keys: &[&str]| { + keys.iter().find_map(|k| entry.get(k)).filter(|v| !matches!(v, Yaml::Null)).map(|v| Xf::from_yaml(v, dir, &search_path)) + }; let mut spaces = vec![]; - for list in ["colorspaces", "display_colorspaces"] { + for (list, display) in [("colorspaces", false), ("display_colorspaces", true)] { for cs in y.get(list).map(Yaml::seq).unwrap_or(&[]) { let Some(name) = cs.get("name").and_then(Yaml::str) else { continue }; - let xf = |keys: &[&str]| keys.iter().find_map(|k| cs.get(k)).filter(|v| !matches!(v, Yaml::Null)).map(|v| Xf::from_yaml(v, dir, &search_path)); spaces.push(ConfigSpace { name: name.to_string(), aliases: cs.get("aliases").map(|a| a.seq().iter().filter_map(|s| s.str().map(str::to_string)).collect()).unwrap_or_default(), - family: cs.get("family").and_then(Yaml::str).unwrap_or("").to_string(), - is_data: cs.get("isdata").and_then(Yaml::str).is_some_and(|v| v == "true"), - to_ref: xf(&["to_reference", "to_scene_reference", "to_display_reference"]), - from_ref: xf(&["from_reference", "from_scene_reference", "from_display_reference"]), + family: cs.text("family").to_string(), + is_data: cs.flag("isdata", false), + display, + to_ref: xf(cs, &["to_reference", "to_scene_reference", "to_display_reference"]), + from_ref: xf(cs, &["from_reference", "from_scene_reference", "from_display_reference"]), }); } } @@ -684,13 +961,60 @@ impl Config { Some(Yaml::Map(m)) => m .iter() .map(|(d, views)| { - let v = views.seq().iter().filter_map(|v| Some((v.get("name")?.str()?.to_string(), v.get("colorspace")?.str()?.to_string()))).collect(); + let v = views + .seq() + .iter() + .filter(|v| !v.text("name").is_empty()) + .map(|v| View { + name: v.text("name").to_string(), + colorspace: v.text("colorspace").to_string(), + view_transform: v.text("view_transform").to_string(), + display_colorspace: v.text("display_colorspace").to_string(), + looks: v.text("looks").to_string(), + }) + .collect(); (d.clone(), v) }) .collect(), _ => vec![], }; - Ok(Config { name: y.get("name").and_then(Yaml::str).unwrap_or("").to_string(), search_path, roles, spaces, displays }) + let view_transforms = y + .get("view_transforms") + .map(Yaml::seq) + .unwrap_or(&[]) + .iter() + .filter(|v| !v.text("name").is_empty()) + .map(|v| ViewTransform { + name: v.text("name").to_string(), + from_scene: xf(v, &["from_scene_reference", "from_reference"]), + to_scene: xf(v, &["to_scene_reference", "to_reference"]), + from_display: xf(v, &["from_display_reference"]), + to_display: xf(v, &["to_display_reference"]), + }) + .collect(); + let looks = y + .get("looks") + .map(Yaml::seq) + .unwrap_or(&[]) + .iter() + .filter(|l| !l.text("name").is_empty()) + .map(|l| Look { + name: l.text("name").to_string(), + process_space: l.text("process_space").to_string(), + transform: xf(l, &["transform"]), + inverse_transform: xf(l, &["inverse_transform"]), + }) + .collect(); + Ok(Config { + name: y.text("name").to_string(), + search_path, + roles, + spaces, + displays, + view_transforms, + default_view_transform: y.text("default_view_transform").to_string(), + looks, + }) } pub fn space_names(&self) -> Vec<&str> { @@ -704,29 +1028,58 @@ impl Config { by(n).or_else(|| self.roles.iter().find(|(r, _)| r.eq_ignore_ascii_case(n)).and_then(|(_, s)| by(s))) } - /// The colour space of a display's view. - pub fn view_space(&self, display: &str, view: &str) -> Option<&ConfigSpace> { - let (_, views) = self.displays.iter().find(|(d, _)| d.eq_ignore_ascii_case(display.trim())).or(self.displays.first())?; - let (_, cs) = views.iter().find(|(v, _)| v.eq_ignore_ascii_case(view.trim())).or(views.first())?; - self.space(cs) + /// A display's view (case-insensitive): the display named (else the first) and its view + /// named (else its first). + pub fn view(&self, display: &str, view: &str) -> Option<(&str, &View)> { + let (d, views) = self.displays.iter().find(|(d, _)| d.eq_ignore_ascii_case(display.trim())).or(self.displays.first())?; + let v = views.iter().find(|v| v.name.eq_ignore_ascii_case(view.trim())).or(views.first())?; + Some((d.as_str(), v)) + } + + fn look(&self, name: &str) -> Option<&Look> { + self.looks.iter().find(|l| l.name.eq_ignore_ascii_case(name.trim())) + } + + fn view_transform(&self, name: &str) -> Option<&ViewTransform> { + self.view_transforms.iter().find(|v| v.name.eq_ignore_ascii_case(name.trim())) + } + + /// The view transform between the scene and display references: `default_view_transform`, + /// else the first scene-referred one. + fn default_view_transform(&self) -> Option<&ViewTransform> { + self.view_transform(&self.default_view_transform).filter(|v| v.scene()).or_else(|| self.view_transforms.iter().find(|v| v.scene())) + } + + /// The conversion from space `a` to space `b` as one flat group of steps (named transforms + /// resolved; data spaces pass through). + pub fn path(&self, a: &ConfigSpace, b: &ConfigSpace) -> Xf { + let mut p = Build::new(self); + p.path(a, b, 0); + p.done() + } + + /// Space `input` shown on `display` through `view` ([`Config::view`]): its looks, then the + /// view's colour space, or its view transform and display colour space. + pub fn display_path<'a>(&'a self, input: &'a ConfigSpace, display: &str, view: &str) -> Xf { + let mut p = Build::new(self); + p.display_view(input, display, view, false, true, 0); + p.done() } /// Convert one colour from space `a` to space `b` (through the reference space; data - /// spaces pass through). + /// spaces pass through). For many colours, build the [`Config::path`] once. pub fn convert(&self, a: &ConfigSpace, b: &ConfigSpace, c: [f64; 3]) -> [f64; 3] { - if a.is_data || b.is_data || a.name == b.name { - return c; - } - b.reference_to(a.to_reference(c)) + self.path(a, b).apply(c, false) } - /// Transforms this module can't apply (by tag), for warnings. + /// Transforms this module can't apply anywhere in the config (by tag), for warnings. pub fn unsupported(&self) -> Vec { let mut out = vec![]; - for s in &self.spaces { - for x in [&s.to_ref, &s.from_ref].into_iter().flatten() { - x.unsupported(&mut out); - } + let spaces = self.spaces.iter().flat_map(|s| [&s.to_ref, &s.from_ref]); + let views = self.view_transforms.iter().flat_map(|v| [&v.from_scene, &v.to_scene, &v.from_display, &v.to_display]); + let looks = self.looks.iter().flat_map(|l| [&l.transform, &l.inverse_transform]); + for x in spaces.chain(views).chain(looks).flatten() { + x.unsupported(&mut out); } out.sort(); out.dedup(); @@ -734,13 +1087,228 @@ impl Config { } } -type ConfigCache = Mutex>>>; +/// Builds one conversion of a config as a flat list of steps, resolving named transforms +/// (limited: [`MAX_NAMED_DEPTH`] levels, [`MAX_OPS`] steps and as many resolutions). +struct Build<'c> { + cfg: &'c Config, + out: Vec, + /// Named transforms still allowed to resolve. + budget: usize, + /// Set when a limit cut the conversion short. + full: bool, +} -/// Load (and cache) a config from a `.ocio` path or the config text itself. -pub fn load_config(src: &str) -> Option> { +impl<'c> Build<'c> { + fn new(cfg: &'c Config) -> Self { + Build { cfg, out: vec![], budget: MAX_OPS, full: false } + } + + fn done(mut self) -> Xf { + if self.full { + self.out.push(Xf::Unsupported(format!("the rest of a conversion of more than {MAX_OPS} steps"))); + } + Xf::Group(self.out) + } + + fn push(&mut self, x: Xf) { + if self.out.len() < MAX_OPS { + self.out.push(x); + } else { + self.full = true; + } + } + + /// Transform `x` forward, or inverse (its steps reversed and each inverted). + fn xf(&mut self, x: &Xf, inverse: bool, depth: usize) { + if self.full { + return; + } + let start = self.out.len(); + match x { + Xf::Group(list) => list.iter().for_each(|x| self.xf(x, false, depth)), + Xf::Inverse(x) => self.xf(x, true, depth), + Xf::Named(n) => self.named(n, depth), + leaf => self.push(leaf.clone()), + } + if inverse && let Some(steps) = self.out.get_mut(start..) { + steps.reverse(); + for s in steps.iter_mut() { + *s = match std::mem::replace(s, Xf::Group(vec![])) { + Xf::Inverse(x) => *x, + x @ Xf::Unsupported(_) => x, + x => Xf::Inverse(Box::new(x)), + }; + } + } + } + + fn space(&mut self, name: &str, what: &str) -> Option<&'c ConfigSpace> { + let s = self.cfg.space(name); + if s.is_none() { + self.push(Xf::Unsupported(format!("{what} (no color space `{name}`)"))); + } + s + } + + fn named(&mut self, n: &Named, depth: usize) { + if depth >= MAX_NAMED_DEPTH { + self.push(Xf::Unsupported(format!("transforms nested more than {MAX_NAMED_DEPTH} deep"))); + return; + } + let Some(budget) = self.budget.checked_sub(1) else { + self.full = true; + return; + }; + self.budget = budget; + let depth = depth + 1; + match n { + Named::Space { src, dst, data_bypass } => { + let (Some(a), Some(b)) = (self.space(src, "ColorSpaceTransform"), self.space(dst, "ColorSpaceTransform")) else { return }; + if *data_bypass { + self.path(a, b, depth); + } else { + self.convert(a, b, depth); + } + } + Named::Look { src, dst, looks } => { + let (Some(a), Some(b)) = (self.space(src, "LookTransform"), self.space(dst, "LookTransform")) else { return }; + let cur = self.looks(a, looks, depth); + self.path(cur, b, depth); + } + Named::DisplayView { src, display, view, looks_bypass, data_bypass } => { + let Some(a) = self.space(src, "DisplayViewTransform") else { return }; + self.display_view(a, display, view, *looks_bypass, *data_bypass, depth); + } + } + } + + /// Space `s` → its reference space. + fn space_to_ref(&mut self, s: &ConfigSpace, depth: usize) { + match (&s.to_ref, &s.from_ref) { + (Some(x), _) => self.xf(x, false, depth), + (None, Some(x)) => self.xf(x, true, depth), + _ => {} + } + } + + /// Its reference space → space `s`. + fn ref_to_space(&mut self, s: &ConfigSpace, depth: usize) { + match (&s.from_ref, &s.to_ref) { + (Some(x), _) => self.xf(x, false, depth), + (None, Some(x)) => self.xf(x, true, depth), + _ => {} + } + } + + /// Between the reference spaces: scene → display through the default view transform, or + /// back (nothing when they are the same). + fn bridge(&mut self, from_display: bool, to_display: bool, depth: usize) { + if from_display == to_display { + return; + } + let Some(vt) = self.cfg.default_view_transform() else { + self.push(Xf::Unsupported("the scene ↔ display reference conversion (the config has no view transform)".into())); + return; + }; + let (fwd, back) = (&vt.from_scene, &vt.to_scene); + match (to_display, fwd, back) { + (true, Some(x), _) | (false, _, Some(x)) => self.xf(x, false, depth), + (true, None, Some(x)) | (false, Some(x), None) => self.xf(x, true, depth), + _ => {} + } + } + + /// Space `a` → space `b`, whatever `isdata` says. + fn convert(&mut self, a: &ConfigSpace, b: &ConfigSpace, depth: usize) { + if a.name == b.name { + return; + } + self.space_to_ref(a, depth); + self.bridge(a.display, b.display, depth); + self.ref_to_space(b, depth); + } + + /// Space `a` → space `b`; data spaces pass through. + fn path(&mut self, a: &ConfigSpace, b: &ConfigSpace, depth: usize) { + if !(a.is_data || b.is_data) { + self.convert(a, b, depth); + } + } + + /// Apply a look string from space `cur`, each look in its process space; the space the + /// colours end in. + fn looks(&mut self, mut cur: &'c ConfigSpace, spec: &str, depth: usize) -> &'c ConfigSpace { + for item in spec.split([',', ':']).map(str::trim).filter(|t| !t.is_empty()) { + let (inverse, names) = match item.strip_prefix('-') { + Some(n) => (true, n), + None => (false, item.trim_start_matches('+')), + }; + // `A | B`: the first look the config has. + let Some(look) = names.split('|').find_map(|n| self.cfg.look(n)) else { + self.push(Xf::Unsupported(format!("Look `{}` (not in the config)", names.trim()))); + continue; + }; + if let Some(ps) = self.cfg.space(&look.process_space) { + self.path(cur, ps, depth); + cur = ps; + } + match (inverse, &look.transform, &look.inverse_transform) { + (false, Some(x), _) | (true, _, Some(x)) => self.xf(x, false, depth), + (false, None, Some(x)) | (true, Some(x), None) => self.xf(x, true, depth), + _ => {} + } + } + cur + } + + fn display_view(&mut self, input: &'c ConfigSpace, display: &str, view: &str, looks_bypass: bool, data_bypass: bool, depth: usize) { + if input.is_data && data_bypass { + return; + } + let cfg = self.cfg; + let Some((dname, v)) = cfg.view(display, view) else { + self.push(Xf::Unsupported(format!("display `{display}` (the config has no displays)"))); + return; + }; + let cur = if looks_bypass { input } else { self.looks(input, &v.looks, depth) }; + if v.view_transform.is_empty() { + if let Some(cs) = self.space(&v.colorspace, "View") { + self.path(cur, cs, depth); + } + return; + } + let Some(vt) = cfg.view_transform(&v.view_transform) else { + self.push(Xf::Unsupported(format!("View (no view transform `{}`)", v.view_transform))); + return; + }; + let dcs_name = if v.display_colorspace == "" { dname } else { v.display_colorspace.as_str() }; + let Some(dcs) = self.space(dcs_name, "View") else { return }; + // To the view transform's input reference, through it to the display reference, then + // to the display colour space. + let scene = vt.scene(); + self.space_to_ref(cur, depth); + self.bridge(cur.display, !scene, depth); + let (fwd, back) = if scene { (&vt.from_scene, &vt.to_scene) } else { (&vt.from_display, &vt.to_display) }; + match (fwd, back) { + (Some(x), _) => self.xf(x, false, depth), + (None, Some(x)) => self.xf(x, true, depth), + _ => {} + } + if !dcs.is_data { + self.bridge(true, dcs.display, depth); + self.ref_to_space(dcs, depth); + } + } +} + +type ConfigCache = Mutex, String>>>; + +/// Load (and cache) a config from a `.ocio` path or the config text itself; why not, when it +/// can't be read. +pub fn load_config(src: &str) -> Result, String> { let t = src.trim(); if t.is_empty() { - return None; + return Err("no config file chosen".into()); } static CACHE: OnceLock = OnceLock::new(); let cache = CACHE.get_or_init(|| Mutex::new(HashMap::new())); @@ -753,9 +1321,8 @@ pub fn load_config(src: &str) -> Option> { Config::parse(src, Path::new(".")) } else { let p = Path::new(t); - std::fs::read_to_string(p).map_err(|e| e.to_string()).and_then(|text| Config::parse(&text, p.parent().unwrap_or(Path::new(".")))) + std::fs::read_to_string(p).map_err(|e| format!("{t}: {e}")).and_then(|text| Config::parse(&text, p.parent().unwrap_or(Path::new(".")))) } - .ok() .map(Arc::new); if let Ok(mut m) = cache.lock() { m.insert(src.to_string(), parsed.clone()); @@ -830,6 +1397,12 @@ colorspaces: (0..3).all(|i| (a[i] - b[i]).abs() < tol) } + fn temp_dir(name: &str) -> PathBuf { + let dir = std::env::temp_dir().join(format!("ec-ocio-{name}-{}", std::process::id())); + std::fs::create_dir_all(dir.join("luts")).unwrap(); + dir + } + #[test] fn yaml_subset() { let y = parse_yaml("a: 1\nb:\n - x\n - {k: [1, 2], s: 'q: r'}\nc: ! {v: 3}\nd:\n- 4\n- 5\n"); @@ -842,10 +1415,22 @@ colorspaces: assert_eq!(y.get("d").unwrap().seq().len(), 2, "a sequence at the key's indent"); } + /// Hostile nesting (flow brackets and block indentation) is cut off instead of exhausting + /// the stack. + #[test] + fn yaml_nesting_is_bounded() { + let flow = format!("a: {}\nb: 1\n", "[".repeat(200_000)); + assert!(matches!(parse_yaml(&flow), Yaml::Map(_))); + let block: String = (0..2_000).map(|i| format!("{}k{i}:\n", " ".repeat(i))).collect(); + assert!(matches!(parse_yaml(&block), Yaml::Map(_))); + let dashes: String = (0..2_000).map(|i| format!("{}- x{i}:\n", " ".repeat(2 * i))).collect(); + let _ = parse_yaml(&dashes); + assert!(Config::parse(&flow, Path::new(".")).is_err()); + } + #[test] fn parses_spaces_roles_displays_and_converts() { - let dir = std::env::temp_dir().join(format!("ec-ocio-{}", std::process::id())); - std::fs::create_dir_all(dir.join("luts")).unwrap(); + let dir = temp_dir("basic"); // A 1D LUT that doubles (0..1 → 0..2). std::fs::write(dir.join("luts/double.cube"), "LUT_1D_SIZE 2\nDOMAIN_MIN 0 0 0\nDOMAIN_MAX 1 1 1\n0 0 0\n2 2 2\n").unwrap(); let c = Config::parse(CONFIG, &dir).unwrap(); @@ -856,7 +1441,7 @@ colorspaces: assert_eq!(c.space("scene_linear").unwrap().name, "lin"); assert_eq!(c.space("G22").unwrap().name, "gamma22"); assert_eq!(c.space("gamma 2.2").unwrap().name, "gamma22"); - assert_eq!(c.view_space("sRGB", "Standard").unwrap().name, "gamma22"); + assert_eq!(c.view("sRGB", "Standard").unwrap().1.colorspace, "gamma22"); let sp = |n: &str| c.space(n).unwrap(); // gamma22 → lin: decode with the exponent. let g = c.convert(sp("gamma22"), sp("lin"), [0.5, 0.25, 1.0]); @@ -864,6 +1449,8 @@ colorspaces: // lin → gamma22: the inverse (only to_reference given). let back = c.convert(sp("lin"), sp("gamma22"), g); assert!(close(back, [0.5, 0.25, 1.0], 1e-9), "{back:?}"); + // The display's view is that conversion. + assert!(close(c.display_path(sp("lin"), "sRGB", "Standard").apply(g, false), [0.5, 0.25, 1.0], 1e-9)); // lin → half: matrix and offset; and back through the inverse matrix. let h = c.convert(sp("lin"), sp("half"), [0.4, 0.6, 1.0]); assert!(close(h, [0.3, 0.4, 0.6], 1e-9), "{h:?}"); @@ -883,13 +1470,16 @@ colorspaces: // Data spaces pass through; unsupported transforms are reported. assert_eq!(c.convert(sp("raw"), sp("gamma22"), [0.3, 0.3, 0.3]), [0.3, 0.3, 0.3]); assert_eq!(c.unsupported(), vec!["ExposureContrastTransform".to_string()]); + assert_eq!(c.path(sp("lin"), sp("weird")).unsupported_list(), vec!["ExposureContrastTransform".to_string()]); + assert!(c.path(sp("lin"), sp("half")).unsupported_list().is_empty()); // Loading from a file path (cached), and errors. std::fs::write(dir.join("config.ocio"), CONFIG).unwrap(); let loaded = load_config(&dir.join("config.ocio").to_string_lossy()).unwrap(); assert_eq!(loaded.spaces.len(), 8); - assert!(load_config("/no/such/config.ocio").is_none()); + assert!(load_config("/no/such/config.ocio").is_err()); assert!(Config::parse("just: text\n", &dir).is_err()); } + /// The OCIO Color Space and Display Transform effects with Configuration ▸ Custom. #[test] fn effects_use_a_custom_config() { @@ -925,4 +1515,217 @@ colorspaces: ); assert_eq!(o.img.data, img.data); } + + /// A config laid out like Blender 4.x / 5.x's (written for this test): the reference space is + /// CIE XYZ with the E white, the D65 spaces adapt through a `.spimtx` matrix, sRGB is a + /// ColorSpaceTransform plus an ExponentWithLinearTransform, the display-referred spaces meet + /// the scene through the default view transform, and a log space uses an + /// AllocationTransform. + fn blender_like(dir: &Path) -> Config { + use effectcraft_color::space::{invert, rgb_to_xyz}; + let e_to_d65 = crate::ocio::bradford([1.0 / 3.0, 1.0 / 3.0], D65); + let rows = |m: [[f64; 3]; 3], sep: &str, end: &str| m.iter().map(|r| format!("{}{sep}{}{sep}{}{end}", r[0], r[1], r[2])).collect::>(); + std::fs::write(dir.join("luts/e_to_d65.spimtx"), rows(e_to_d65, " ", " 0\n").concat()).unwrap(); + let xyz_to_709 = rows(invert(&rgb_to_xyz(REC709, D65)), ", ", ", 0").join(", "); + let text = format!( + r#"ocio_profile_version: 2.4 +search_path: luts +roles: + reference: Linear CIE-XYZ E + scene_linear: Linear Rec.709 + cie_xyz_d65_interchange: XYZ D65 Display +default_view_transform: Standard +view_transforms: + - ! + name: Standard + from_scene_reference: ! {{src: e_to_d65.spimtx, interpolation: linear}} +displays: + sRGB: + - ! {{name: Standard, colorspace: sRGB}} + - ! {{name: Display, view_transform: Standard, display_colorspace: sRGB Display}} + - ! {{name: Log, colorspace: Log Encoding}} + - ! {{name: Log Brighter, colorspace: Log Encoding, looks: +Brighter}} + - ! {{name: Log Darker, colorspace: Log Encoding, looks: "-Brighter, Missing"}} + - ! {{name: Raw, colorspace: Non-Color}} +display_colorspaces: + - ! + name: XYZ D65 Display + isdata: false + - ! + name: sRGB Display + from_display_reference: ! + children: + - ! {{src: cie_xyz_d65_interchange, dst: Linear Rec.709}} + - ! {{gamma: 2.4, offset: 0.055, direction: inverse}} +colorspaces: + - ! + name: Linear CIE-XYZ E + aliases: [Linear CIE-XYZ I-E] + - ! + name: Linear CIE-XYZ D65 + from_scene_reference: ! {{src: e_to_d65.spimtx, interpolation: linear}} + - ! + name: Linear Rec.709 + from_scene_reference: ! + children: + - ! {{src: Linear CIE-XYZ I-E, dst: Linear CIE-XYZ D65}} + - ! {{matrix: [{xyz_to_709}, 0, 0, 0, 1]}} + - ! + name: sRGB + from_scene_reference: ! + children: + - ! {{src: Linear CIE-XYZ E, dst: Linear Rec.709}} + - ! {{gamma: 2.4, offset: 0.055, direction: inverse}} + - ! + name: Log Encoding + from_scene_reference: ! + children: + - ! {{src: Linear CIE-XYZ E, dst: Linear Rec.709}} + - ! {{allocation: lg2, vars: [-12, 12]}} + - ! + name: ACEScg + from_scene_reference: ! + children: + - ! {{src: Linear CIE-XYZ E, dst: Linear CIE-XYZ D65}} + - ! {{style: "UTILITY - ACES-AP1_to_CIE-XYZ-D65_BFD", direction: inverse}} + - ! + name: Clipped + from_scene_reference: ! + children: + - ! {{src: Linear CIE-XYZ E, dst: Linear Rec.709}} + - ! {{min_in_value: 0, min_out_value: 0}} + - ! + name: Graded + from_scene_reference: ! + children: + - ! {{src: Linear CIE-XYZ E, dst: Linear Rec.709}} + - ! + shadows: {{rgb: [0.2, 0.2, 0.2], master: 0.35, start: 0.4, pivot: 0.1}} + - ! + name: Loop + from_scene_reference: ! {{src: Linear CIE-XYZ E, dst: Loop}} + - ! + name: Fan + from_scene_reference: ! + children: + - ! {{src: Linear CIE-XYZ E, dst: Fan}} + - ! {{src: Linear CIE-XYZ E, dst: Fan}} + - ! {{src: Linear CIE-XYZ E, dst: Fan}} + - ! + name: Non-Color + isdata: true +looks: + - ! + name: Brighter + process_space: Log Encoding + transform: ! {{offset: [0.05, 0.05, 0.05]}} +"# + ); + Config::parse(&text, dir).unwrap() + } + + /// The sRGB encoding (IEC 61966-2-1). + fn srgb(v: f64) -> f64 { + if v <= 0.003_130_8 { v * 12.92 } else { 1.055 * v.powf(1.0 / 2.4) - 0.055 } + } + + /// #410: with Blender's config, Linear Rec.709 → sRGB got no sRGB curve and a blue cast: + /// ColorSpaceTransform and ExponentWithLinearTransform passed colours through, so the + /// colours stayed in the E-white XYZ reference. Every route to sRGB now encodes greys to + /// neutral sRGB values (0.18 → 0.4614). + #[test] + fn blender_like_config_converts_linear_rec709_to_srgb() { + let dir = temp_dir("blender"); + let c = blender_like(&dir); + assert!(c.unsupported().iter().all(|t| t == "GradingToneTransform"), "{:?}", c.unsupported()); + let sp = |n: &str| c.space(n).unwrap(); + let lin = sp("scene_linear"); + let routes = [ + ("Color Space Transform", c.path(lin, sp("sRGB"))), + ("display-referred sRGB", c.path(lin, sp("sRGB Display"))), + ("v1 view", c.display_path(lin, "sRGB", "Standard")), + ("v2 view", c.display_path(lin, "sRGB", "Display")), + ]; + for (route, x) in &routes { + assert!(x.unsupported_list().is_empty(), "{route}: {:?}", x.unsupported_list()); + for v in [0.0, 0.001, 0.18, 0.5, 1.0, 4.0] { + let o = x.apply([v; 3], false); + assert!(close(o, [srgb(v); 3], 2e-5), "{route}: {v} → {o:?}, want {}", srgb(v)); + } + let red = x.apply([1.0, 0.0, 0.0], false); + assert!(close(red, [1.0, 0.0, 0.0], 1e-4), "{route}: red {red:?}"); + // And back. + assert!(close(x.apply([srgb(0.18); 3], true), [0.18; 3], 1e-6), "{route}"); + } + // AllocationTransform lg2 and a look in its process space; data passes through. + let log = |v: f64| (v.log2() + 12.0) / 24.0; + let o = c.display_path(lin, "sRGB", "Log").apply([0.18; 3], false); + assert!(close(o, [log(0.18); 3], 1e-6), "{o:?}"); + let o = c.display_path(lin, "sRGB", "Log Brighter").apply([0.18; 3], false); + assert!(close(o, [log(0.18) + 0.05; 3], 1e-6), "{o:?}"); + let darker = c.display_path(lin, "sRGB", "Log Darker"); + assert!(close(darker.apply([0.18; 3], false), [log(0.18) - 0.05; 3], 1e-6)); + assert_eq!(darker.unsupported_list(), vec!["Look `Missing` (not in the config)".to_string()]); + assert_eq!(c.display_path(lin, "sRGB", "Raw").apply([0.3, 2.0, -1.0], false), [0.3, 2.0, -1.0]); + // BuiltinTransform: the built-in config's Linear Rec.709 → ACEScg. + let want = crate::ocio::Xform::new(&sp_builtin("Linear Rec.709 (sRGB)"), &sp_builtin("ACEScg")).apply([1.0, 0.25, 0.0]); + let got = c.convert(lin, sp("ACEScg"), [1.0, 0.25, 0.0]); + assert!(close(got, want.map(f64::from), 1e-4), "{got:?} vs {want:?}"); + // RangeTransform with only a minimum clamps below and shifts nothing. + let o = c.convert(lin, sp("Clipped"), [-0.5, 0.5, 2.0]); + assert!(close(o, [0.0, 0.5, 2.0], 1e-6), "{o:?}"); + // Unsupported steps are listed (and pass through); loops stop. + assert_eq!(c.path(lin, sp("Graded")).unsupported_list(), vec!["GradingToneTransform".to_string()]); + assert!(c.path(lin, sp("Loop")).unsupported_list()[0].contains("nested")); + let fan = c.path(lin, sp("Fan")); + assert!(!fan.unsupported_list().is_empty()); + assert!(matches!(&fan, Xf::Group(v) if v.len() <= MAX_OPS + 1)); + } + + fn sp_builtin(name: &str) -> Space { + *crate::ocio::SPACES.iter().find(|s| s.name == name).unwrap() + } + + /// ExponentWithLinearTransform is the sRGB curve at gamma 2.4 / offset 0.055, continuous at + /// its break point and inverted exactly; mirror mirrors negatives. + #[test] + fn moncurve_matches_srgb_and_inverts() { + for v in [-0.2, 0.0, 0.002, 0.039, 0.0393, 0.2, 0.5, 1.0, 3.0] { + let lin = moncurve(v, 2.4, 0.055, false, false); + assert!((moncurve(lin, 2.4, 0.055, true, false) - v).abs() < 1e-12, "{v}"); + if v >= 0.0 { + assert!((moncurve(srgb(v), 2.4, 0.055, false, false) - v).abs() < 2e-5, "{v}"); + } + } + let xb = 0.055 / 1.4; + assert!((moncurve(xb - 1e-12, 2.4, 0.055, false, false) - moncurve(xb + 1e-12, 2.4, 0.055, false, false)).abs() < 1e-9); + assert_eq!(moncurve(-0.5, 2.4, 0.055, false, true), -moncurve(0.5, 2.4, 0.055, false, false)); + assert_eq!(moncurve(-0.5, 1.0, 0.0, false, false), 0.0); + } + + #[test] + fn builtin_styles() { + for style in ["IDENTITY", "DISPLAY - CIE-XYZ-D65_to_sRGB", "display - cie-xyz-d65_to_rec.2100-pq", "ACEScc_to_ACES2065-1", "ACEScct_to_ACES2065-1"] { + assert!(builtin(style).is_some(), "{style}"); + } + assert!(builtin("ACES-OUTPUT - ACES2065-1_to_CIE-XYZ-D65 - SDR-VIDEO_1.0").is_none()); + // D65 white in XYZ is display white. + let xyz = rgb_white_xyz(); + for style in ["DISPLAY - CIE-XYZ-D65_to_sRGB", "DISPLAY - CIE-XYZ-D65_to_REC.1886-REC.709", "DISPLAY - CIE-XYZ-D65_to_DisplayP3"] { + let o = builtin(style).unwrap().apply(xyz, false); + assert!(close(o, [1.0; 3], 1e-6), "{style}: {o:?}"); + } + // PQ: 1.0 = 100 nits → code value ≈ 0.508. + let pq = builtin("DISPLAY - CIE-XYZ-D65_to_REC.2100-PQ").unwrap().apply(xyz, false); + assert!((pq[0] - 0.508).abs() < 1e-3, "{pq:?}"); + // ACEScc and ACEScct of 0.18 (ACES2065-1 grey stays grey). + for (style, code) in [("ACEScc_to_ACES2065-1", 0.413_588_402_492_442_1), ("ACEScct_to_ACES2065-1", 0.413_588_402_492_442_1)] { + let o = builtin(style).unwrap().apply([code; 3], false); + assert!(close(o, [0.18; 3], 1e-6), "{style}: {o:?}"); + } + } + + fn rgb_white_xyz() -> [f64; 3] { + [D65[0] / D65[1], 1.0, (1.0 - D65[0] - D65[1]) / D65[1]] + } } diff --git a/crates/engine/src/commands/effect.rs b/crates/engine/src/commands/effect.rs index 4f0c185e..68684650 100644 --- a/crates/engine/src/commands/effect.rs +++ b/crates/engine/src/commands/effect.rs @@ -126,6 +126,19 @@ fn find_fx(s: &Session, p: &Value, cmd: &str) -> Result<(effectcraft_project::It Ok((cid, lid, g.uid)) } +/// `effect.warning`: what Effect Controls warns about an effect (a custom OCIO config it can't +/// read, or transforms it passes through), `null` when it renders as asked. +fn warning(s: &mut Session, p: &Value) -> Result { + let cmd = "effect.warning"; + let (cid, lid, uid) = find_fx(s, p, cmd)?; + let comp = s.project.comp(cid).ok_or(EngineError::NoComp)?; + let layer = comp.layer(lid).ok_or(EngineError::NoComp)?; + let g = layer.effects().and_then(|fx| fx.groups().find(|g| g.uid == uid)).ok_or_else(|| bad(cmd, "no such effect"))?; + let t = super::time_p(s, p, Some(comp)); + let ctx = effectcraft_render::EvalCtx { project: &s.project, comp_id: cid, comp, time: t, expr: s.expr.as_deref(), footage: None }; + Ok(json!({"effect": uid, "warning": effectcraft_effects::warning(g, &mut |pr| ctx.value(layer, pr))})) +} + /// A colour parameter's eyedropper on a keyer (Effect Controls): the colour of the effect's input /// (the layer's pixels before this effect, so a keyed screen can still be picked) at a /// layer-space point, set on the parameter. `average` takes the 5 × 5 pixels around it. @@ -457,6 +470,12 @@ pub fn specs() -> Vec { cmd!("effect.paste", "Paste Effects", [], None, "{layers?} — adds the copied effects to the layers", has_layers, paste), cmd!("effect.reset", "Reset Effect", [], None, "{layer?, effect}", has_layers, reset), crate::query!("effect.list", "List Effects", "{filter?}", list), + crate::query!( + "effect.warning", + "Effect Warning", + "{layer?, effect?: index|uid|name (default: the selected effect), time?} → {effect, warning: string|null} — what Effect Controls warns about (an OCIO config it can't read, transforms it passes through)", + warning + ), crate::query!( "effect.plugins.list", "List Effect Plug-ins", diff --git a/crates/engine/src/tests_effects.rs b/crates/engine/src/tests_effects.rs index b9abd9a1..2f164991 100644 --- a/crates/engine/src/tests_effects.rs +++ b/crates/engine/src/tests_effects.rs @@ -317,3 +317,32 @@ fn opening_a_schema_1_project_moves_shape_layer_effect_points_into_effect_space( let _ = std::fs::remove_dir_all(dir); check(&s3); } + +/// #410: `effect.warning` reports what an OCIO effect can't render as asked (Effect Controls +/// shows it under the effect's name): a transform of its custom config that passes colours +/// through, a colour space the config lacks or a config it can't read; nothing for the +/// built-in config. +#[test] +fn ocio_effect_warns_about_unsupported_config_transforms() { + let (mut s, a, _) = comp_with_two_solids(); + let fx = apply(&mut s, a, "OCIO Color Space Transform"); + let warning = |s: &mut Session| s.execute("effect.warning", json!({"layer": a, "effect": fx})).unwrap()["warning"].clone(); + assert!(warning(&mut s).is_null(), "built-in config"); + let set = |s: &mut Session, param: &str, value: serde_json::Value| { + let uid = layer(s, a).effects().unwrap().groups().next().unwrap().get(param).unwrap().uid; + s.execute("prop.set", json!({"layer": a, "prop": uid, "value": value})).unwrap(); + }; + let config = "ocio_profile_version: 2\ncolorspaces:\n - !\n name: lin\n - !\n name: graded\n from_scene_reference: ! {}\n"; + set(&mut s, "config", json!(1)); + set(&mut s, "configFile", json!(config)); + set(&mut s, "sourceName", json!("lin")); + set(&mut s, "destinationName", json!("graded")); + let w = warning(&mut s); + assert!(w.as_str().is_some_and(|w| w.contains("GradingToneTransform")), "{w}"); + set(&mut s, "destinationName", json!("lin")); + assert!(warning(&mut s).is_null()); + set(&mut s, "destinationName", json!("nowhere")); + assert!(warning(&mut s).as_str().is_some_and(|w| w.contains("no color space `nowhere`"))); + set(&mut s, "configFile", json!("/no/such/config.ocio")); + assert!(warning(&mut s).as_str().is_some_and(|w| w.starts_with("Can't read the OCIO config"))); +} diff --git a/crates/gpu/src/fx_lut.rs b/crates/gpu/src/fx_lut.rs index fa145241..cf8df08d 100644 --- a/crates/gpu/src/fx_lut.rs +++ b/crates/gpu/src/fx_lut.rs @@ -10,8 +10,10 @@ //! iterative 3D inverse): hardware 3D-texture filtering would round the interpolation weights //! and cannot do tetrahedral interpolation. The CPU evaluates transfer functions, CDLs and //! Color Profile Converter's matrix in f64; the kernel in f32 (within the 1e-3 tolerance). -//! Custom `.ocio` configurations compile their colour spaces' transforms (matrix and offset, -//! exponent, log / log-affine, range, CDL, file LUTs, groups, inverses) to the same program. +//! Custom `.ocio` configurations compile their conversions (matrix and offset, exponent, +//! exponent with linear segment, log / log-affine, allocation, range with its clamp, CDL, file +//! LUTs and matrices, built-in curves, with colour space / look / display-view transforms +//! resolved) to the same program; one longer than the kernel's 255 ops renders on the CPU. use effectcraft_effects::{ColorOp, EffectCtx, Lut, Straight, Tf}; @@ -57,9 +59,13 @@ fn tf_code(t: Tf) -> [f32; 2] { Tf::Gamma(g) => [2.0, g as f32], Tf::AcesCct => [3.0, 0.0], Tf::Pq => [4.0, 0.0], + Tf::AcesCc => [5.0, 0.0], } } +/// Most ops `fxl_run` interprets (its loop guard); longer programs render on the CPU. +const MAX_OPS: usize = 255; + /// A LUT in the layout `fxl_lut_*` read at the returned offset: n1, n3, domain min (3), domain /// max (3), the 1D table (n1 × RGB), then the lattice (n3³ × RGB, red fastest). pub(crate) fn push_lut(d: &mut Vec, lut: &Lut) -> f32 { @@ -75,6 +81,9 @@ pub(crate) fn push_lut(d: &mut Vec, lut: &Lut) -> f32 { /// Program layout: a stream of (opcode, arguments…) ending in 0, followed by LUT data. fn encode(ops: &[ColorOp]) -> Option> { + if ops.len() > MAX_OPS { + return None; + } let mut prog: Vec = vec![]; let mut tables: Vec> = vec![]; // LUT offsets are patched once the program's length is known. @@ -123,6 +132,15 @@ fn encode(ops: &[ColorOp]) -> Option> { prog.extend(log_slope.iter().chain(log_offset).chain(lin_slope).chain(lin_offset)); prog.extend([*inverse as u32 as f32, f64::MIN_POSITIVE.ln() as f32]); } + ColorOp::MonCurve { gamma, offset, inverse, mirror } => { + prog.push(12.0); + prog.extend(gamma.iter().chain(offset)); + prog.extend([*inverse as u32 as f32, *mirror as u32 as f32]); + } + ColorOp::Clamp { lo, hi } => { + prog.push(13.0); + prog.extend(lo.iter().chain(hi)); + } ColorOp::Lut { lut, shaper, interp, inverse } => { prog.extend([8.0, 0.0, *interp as f32, *inverse as u32 as f32, -1.0]); let mut t = vec![]; diff --git a/crates/gpu/src/shaders/fx_lut.wgsl b/crates/gpu/src/shaders/fx_lut.wgsl index 83beb623..87966df7 100644 --- a/crates/gpu/src/shaders/fx_lut.wgsl +++ b/crates/gpu/src/shaders/fx_lut.wgsl @@ -266,6 +266,16 @@ fn fxl_decode(tf: u32, g: f32, v: f32) -> f32 { let l = powz(max(e - FXL_PQ_C1, 0.0) / (FXL_PQ_C2 - FXL_PQ_C3 * e), 1.0 / FXL_PQ_M1); return l * 100.0; } + case 5u: { + // ACEScc. + if (v < (-15.0 + 9.72) / 17.52) { + return (exp2(v * 17.52 - 9.72) - exp2(-16.0)) * 2.0; + } + if (v < (log2(65504.0) + 9.72) / 17.52) { + return exp2(v * 17.52 - 9.72); + } + return 65504.0; + } default: { return v; } @@ -297,6 +307,16 @@ fn fxl_encode(tf: u32, g: f32, v: f32) -> f32 { let y = powz(max(v / 100.0, 0.0), FXL_PQ_M1); return powz((FXL_PQ_C1 + FXL_PQ_C2 * y) / (1.0 + FXL_PQ_C3 * y), FXL_PQ_M2); } + case 5u: { + // ACEScc. + if (v <= 0.0) { + return (-16.0 + 9.72) / 17.52; + } + if (v < exp2(-15.0)) { + return (log2(exp2(-16.0) + v * 0.5) + 9.72) / 17.52; + } + return (log2(v) + 9.72) / 17.52; + } default: { return v; } @@ -383,6 +403,37 @@ fn fxl_tonemap(v: f32, inverse: bool) -> f32 { return (-b + sqrt(b * b - 4.0 * a * c)) / (2.0 * a); } +// ExponentWithLinearTransform (ocio_config::moncurve): forward decodes; a gamma of at most 1 or +// a non-positive offset is a plain power curve. +fn fxl_moncurve(v0: f32, g: f32, o: f32, inv: bool, mirror: bool) -> f32 { + var v = v0; + var sign = 1.0; + if (mirror && v < 0.0) { + v = -v; + sign = -1.0; + } + if (g <= 1.0 || o <= 0.0) { + var p = g; + if (inv) { + p = 1.0 / max(g, 1e-9); + } + return sign * powz(max(v, 0.0), p); + } + let xb = o / (g - 1.0); + let yb = powz((xb + o) / (1.0 + o), g); + let slope = yb / xb; + if (!inv) { + if (v >= xb) { + return sign * powz((v + o) / (1.0 + o), g); + } + return sign * v * slope; + } + if (v >= yb) { + return sign * ((1.0 + o) * powz(v, 1.0 / g) - o); + } + return sign * v / slope; +} + // Run the colour program at data[0..] on a straight colour. fn fxl_run(c0: vec3) -> vec3 { var c = c0; @@ -485,6 +536,22 @@ fn fxl_run(c0: vec3) -> vec3 { } pc += 16u; } + case 12u: { + // ExponentWithLinearTransform: gamma (3), offset (3), inverse, mirror. + let g = fxl_d3(pc + 1u); + let o = fxl_d3(pc + 4u); + let inv = data[pc + 7u] != 0.0; + let mirror = data[pc + 8u] != 0.0; + for (var i = 0u; i < 3u; i++) { + c[i] = fxl_moncurve(c[i], g[i], o[i], inv, mirror); + } + pc += 9u; + } + case 13u: { + // Clamp (a clamping RangeTransform): low (3), high (3). + c = min(max(c, fxl_d3(pc + 1u)), fxl_d3(pc + 4u)); + pc += 7u; + } default: { return c; } diff --git a/crates/gpu/src/tests_fx_lut.rs b/crates/gpu/src/tests_fx_lut.rs index 0a30b907..b74e90f0 100644 --- a/crates/gpu/src/tests_fx_lut.rs +++ b/crates/gpu/src/tests_fx_lut.rs @@ -241,3 +241,105 @@ fn ocio_custom_config() { } effect_case("ec.color.ociocolorspace", &[("config", e(1)), ("configFile", s("/missing.ocio"))]); } + +/// A config with the OCIO v2 transforms of Blender's (#410): an E-white XYZ reference, +/// ColorSpaceTransform, ExponentWithLinearTransform, display-referred spaces through the +/// default view transform, AllocationTransform, BuiltinTransform (a matrix, the PQ and ACEScc +/// curves), a minimum-only RangeTransform and a look in a view. +const CONFIG_V2: &str = r#"ocio_profile_version: 2 +roles: + reference: XYZ E + scene_linear: Linear Rec.709 +default_view_transform: Standard +view_transforms: + - ! + name: Standard + from_scene_reference: ! {matrix: [0.9531, -0.0266, 0.0239, 0, -0.0382, 1.0288, 0.0094, 0, 0.0026, -0.0030, 1.0893, 0, 0, 0, 0, 1]} +displays: + sRGB: + - ! {name: Standard, view_transform: Standard, display_colorspace: sRGB Display} + - ! {name: Log, colorspace: Log, looks: +Lift} + - ! {name: PQ, view_transform: Standard, display_colorspace: PQ Display} +display_colorspaces: + - ! + name: XYZ D65 + - ! + name: sRGB Display + from_display_reference: ! + children: + - ! {src: XYZ D65, dst: Linear Rec.709} + - ! {gamma: 2.4, offset: 0.055, direction: inverse} + - ! + name: PQ Display + from_display_reference: ! {style: DISPLAY - CIE-XYZ-D65_to_REC.2100-PQ} +colorspaces: + - ! + name: XYZ E + - ! + name: Linear Rec.709 + from_scene_reference: ! + children: + - ! {matrix: [0.9531, -0.0266, 0.0239, 0, -0.0382, 1.0288, 0.0094, 0, 0.0026, -0.0030, 1.0893, 0, 0, 0, 0, 1]} + - ! {matrix: [3.2410, -1.5374, -0.4986, 0, -0.9692, 1.8760, 0.0416, 0, 0.0556, -0.2040, 1.0570, 0, 0, 0, 0, 1]} + - ! + name: sRGB + from_scene_reference: ! + children: + - ! {src: XYZ E, dst: Linear Rec.709} + - ! {gamma: 2.4, offset: 0.055, direction: inverse} + - ! + name: Log + from_scene_reference: ! + children: + - ! {src: XYZ E, dst: Linear Rec.709} + - ! {allocation: lg2, vars: [-10, 6.5]} + - ! + name: ACEScg + from_scene_reference: ! + children: + - ! {src: XYZ E, dst: XYZ D65} + - ! {style: "UTILITY - ACES-AP1_to_CIE-XYZ-D65_BFD", direction: inverse} + - ! + name: ACEScc + to_scene_reference: ! + children: + - ! {style: ACEScc_to_ACES2065-1} + - ! {matrix: [0.9525, 0, 0.0001, 0, 0.3439, 0.7281, -0.0721, 0, 0, 0, 1.0088, 0, 0, 0, 0, 1]} + - ! + name: Floor + from_scene_reference: ! + children: + - ! {src: XYZ E, dst: sRGB} + - ! {min_in_value: 0.1, min_out_value: 0.1} + - ! + name: Uniform + from_scene_reference: ! {allocation: uniform, vars: [-0.5, 2]} +looks: + - ! + name: Lift + process_space: Log + transform: ! {offset: [0.05, 0.02, 0], sat: 0.8} +"#; + +#[test] +fn ocio_custom_config_v2() { + let cfg = [("config", e(1)), ("configFile", s(CONFIG_V2))]; + for (a, z) in [ + ("Linear Rec.709", "sRGB"), + ("Linear Rec.709", "sRGB Display"), + ("sRGB", "Log"), + ("ACEScg", "sRGB"), + ("ACEScc", "Linear Rec.709"), + ("sRGB", "Floor"), + ("Linear Rec.709", "Uniform"), + ] { + for dir in 0..2 { + let vals = [cfg.as_slice(), &[("sourceName", s(a)), ("destinationName", s(z)), ("direction", e(dir))]].concat(); + effect_case("ec.color.ociocolorspace", &vals); + } + } + for (view, dir) in [("Standard", 0), ("Standard", 1), ("Log", 0), ("Log", 1), ("PQ", 0)] { + let vals = [cfg.as_slice(), &[("sourceName", s("Linear Rec.709")), ("displayName", s("sRGB")), ("viewName", s(view)), ("direction", e(dir))]].concat(); + effect_case("ec.color.ociodisplay", &vals); + } +} diff --git a/crates/ui-egui/src/panels/effect_controls.rs b/crates/ui-egui/src/panels/effect_controls.rs index e5dbc2b8..1bdd59fd 100644 --- a/crates/ui-egui/src/panels/effect_controls.rs +++ b/crates/ui-egui/src/panels/effect_controls.rs @@ -1056,6 +1056,15 @@ pub fn show(app: &mut EffectcraftApp, ui: &mut egui::Ui, rect: Rect) { item(ui, "Move Up", fi > 0, "effect.reorder", json!({"layer": lid, "effect": g.uid, "index": fi.max(1)})); item(ui, "Move Down", fi + 1 < fx.len(), "effect.reorder", json!({"layer": lid, "effect": g.uid, "index": fi + 2})); }); + // What the effect can't render as asked (an OCIO config it can't read, transforms it + // passes through), under its name. + if let Some(w) = effectcraft_engine::effects::warning(g, &mut |pr| ectx.value(&layer, pr)) { + let galley = bp.layout(format!("⚠ {w}"), Tokens::ui(11.0), t.warning, (body.width() - 20.0).max(60.0)); + let wr = Rect::from_min_size(pos2(body.min.x + 10.0, y + 3.0), galley.size()); + bp.galley(wr.min, galley, t.warning); + app.auto.add(&format!("effectControls.effect.{}.warning", g.uid), wr, &w); + y += wr.height() + 6.0; + } if open { let eh = editor_height(effect, g, body.width()); if eh > 0.0 { diff --git a/docs/agents.md b/docs/agents.md index 26e32f8c..0c343f5c 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -222,6 +222,9 @@ build real projects through these interfaces; they are worked examples of everyt * **Colour of footage**: `file.interpretFootage {"items":[id], "preserveRgb":true}` is Interpret Footage ▸ Preserve RGB: the file's values unconverted (float EXRs stay linear), for a per-layer OCIO conversion or data passes. +* **OCIO effects**: with Configuration ▸ Custom, `configFile` is a `.ocio` path or the config's + text; `effect.warning {layer, effect}` says what Effect Controls warns about (a config it + can't read, a colour space it lacks, transforms that pass colours through) or `null`. * **Multi-layer OpenEXR**: the footage shows the beauty pass (an unnamed RGBA layer, else Combined / Beauty / Image / RGBA…, never Cryptomatte or depth). `layer.channels {layer}` lists a footage layer's EXR layers and diff --git a/docs/parity.md b/docs/parity.md index 215a995c..844ba4f7 100644 --- a/docs/parity.md +++ b/docs/parity.md @@ -192,7 +192,7 @@ Mocha and Cineware are third-party and not counted. |---|---|---| | 3D Channel | 3D Channel Extract, Cryptomatte, Depth Matte, Depth of Field, EXtractoR, Fog 3D, ID Matte, IDentifier | Read auxiliary channels: multi-layer OpenEXR (any `layer.channel`, Cryptomatte manifests from the header; EXtractoR picks a layer and channels from popups of the footage's layers and channels, `layer.channels`, #295; the footage itself shows the beauty pass, never Cryptomatte or depth, #412) or the compositor's own depth / layer-ID / normal / UV / Cryptomatte pass for precomps of 3D comps. | | Immersive Video | VR Blur, VR Chromatic Aberrations, VR Color Gradients, VR Converter, VR De-Noise, VR Digital Glitch, VR Fractal Noise, VR Glow, VR Plane to Sphere, VR Rotate Sphere, VR Sharpen, VR Sphere to Plane | Seam-aware equirectangular processing (wrap-around, pole continuation, latitude-scaled filters), mono / over-under / side-by-side layouts; VR Converter between equirectangular, cube maps (4:3, 3:2, 6:1), sphere map and 2D. | -| Color Correction | OCIO CDL / Color Space / Display / File / Look Transform, Color Stabilizer | Built-in minimal OCIO-style config (ACES2065-1, ACEScg, ACEScct, sRGB, Rec.709, Rec.2020, Display P3, linear variants, XYZ) from published matrices and transfer functions; `.cube` / `.3dl` / `.csp` LUTs and ASC `.cc` / `.ccc` / `.cdl`. Custom `.ocio` config files are read (Configuration ▸ Custom: colorspaces, roles, displays/views; Matrix, File, Exponent, Log, LogAffine, CDL, Range and Group transforms). | +| Color Correction | OCIO CDL / Color Space / Display / File / Look Transform, Color Stabilizer | Built-in minimal OCIO-style config (ACES2065-1, ACEScg, ACEScct, sRGB, Rec.709, Rec.2020, Display P3, linear variants, XYZ) from published matrices and transfer functions; `.cube` / `.3dl` / `.csp` / `.spi1d` / `.spi3d` / `.spimtx` LUTs and ASC `.cc` / `.ccc` / `.cdl`. Custom `.ocio` config files are read from the OCIO v2 documentation (Configuration ▸ Custom: colorspaces and display colorspaces with the scene / display reference spaces joined by the default view transform, roles, displays with v1 and v2 views, view transforms, looks; Matrix, File, Exponent, ExponentWithLinear, Log, LogAffine, CDL, Range, Allocation, ColorSpace, Look, DisplayView, Builtin (matrices and display curves) and Group transforms), so Blender 4.x / 5.x configs give the Standard, AgX, Filmic and False Color views (#410). Grading Primary / Tone transforms, the ACES output and gamut-compression builtins and HLG pass colours through, and the effect shows a warning naming them (`effect.warning`). | | Distort | CC Flo Motion, Liquify, Rolling Shutter Repair | Liquify strokes are effect data replayed into a displacement mesh (`liquify.stroke` / `liquify.clear` commands; no interactive viewer brush yet). Rolling Shutter Repair uses the KLT tracker. | | Blur & Sharpen | Camera-Shake Deblur, CC Radial Blur | Deblur substitutes aligned patches from sharper neighbouring frames. | | Audio | Compressor, Distortion, Gate | Applied in the mixdown like the other audio effects. | @@ -753,7 +753,8 @@ composited at 8 and 32 bpc: cineSpace shapers are read from a storage buffer with the CPU's nearest / trilinear / tetrahedral interpolation and inverses (not a hardware 3D texture: its filtering rounds the weights and cannot do tetrahedral). Lumetri's Input LUT and Look no longer force the CPU. - Custom `.ocio` configurations render on the CPU. Exception: nearest-neighbour lattice lookups + Custom `.ocio` configurations compile to the same programs (one of more than 255 steps + renders on the CPU). Exception: nearest-neighbour lattice lookups may pick the other lattice point for 8 bpc inputs exactly on a rounding boundary (up to 1 % of pixels allowed). - **Simulation render passes** (`gpu::fx_sim`): CC Rainfall, CC Snowfall, CC Star Burst, CC