diff --git a/ROADMAP.md b/ROADMAP.md index bd1f716e..a0af5206 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -48,8 +48,8 @@ Priorities, in order. Each points at tracker rows in [`docs/parity.md`](docs/par 1. **Camera colour calibration of our own** (LR-PROF-CAMERACOLOR, P0): fit each camera to its own embedded JPEG, use matrices the files carry themselves, then chart shots. Sony ARW, Nikon NEF, Panasonic RW2, Fujifilm RAF, Canon CR3/CR2, - Pentax PEF and Samsung SRW have the file-local fit (matrix + tone curve from their own JPEG; CR2 and PEF since issue #310, not yet - checked on the corpus); then ORF, then validate fidelity. + Pentax PEF, Samsung SRW and Olympus ORF have the file-local fit (matrix + tone curve from their own JPEG; CR2 and PEF + since issue #310, not yet checked on the corpus); then validate fidelity. 2. **Raw formats, clean-room** (LR-IMP-FORMATS, P0): Remaining **unverified CR3 CRX** variants, compressed ORF, NEF lossy-after-split, sRAW. Decided 2026-10-05: write our own decoders from prose descriptions (never decoder source, no LGPL dependency); compressed NEF (#86) is the template. @@ -189,7 +189,7 @@ Not decoded yet — preview only (no permissively licensed description; black-bo alignment or reset predictors at the split row). The other Nikon Huffman variants are decoded (`crates/raw/src/vendor/nefc.rs` documents the analysis). - **Olympus compressed ORF**, **Canon CR3 unverified CRX variants** (M11.1), **Canon sRAW/mRAW**. -**Camera colour matrices:** ARW, NEF, RW2, RAF, CR3, CR2, PEF and Samsung SRW files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged +**Camera colour matrices:** ARW, NEF, RW2, RAF, CR3, CR2, PEF, Samsung SRW and Olympus ORF files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged `matrix_is_fallback`) with the file's as-shot white-balance multipliers. Clean sources to evaluate next: manufacturer matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax equivalents; Panasonic RW2 files carry none) and our own chart-based calibration (M11.4). Adobe matrices are never used. diff --git a/crates/catalog/src/model.rs b/crates/catalog/src/model.rs index 9825fa8c..62fbaa8f 100644 --- a/crates/catalog/src/model.rs +++ b/crates/catalog/src/model.rs @@ -301,7 +301,7 @@ pub struct Analysis { /// as shot), as for rendered photographs. `format` is the file's extension or Lightroom's /// `fileFormat` name, in any case. See [`Photo::relative_wb`]. pub fn relative_wb_format(format: &str) -> bool { - ["ARW", "NEF", "NRW", "RW2", "RWL", "RAW", "RAF", "CR3", "CR2", "PEF", "SRW"].iter().any(|f| format.eq_ignore_ascii_case(f)) + ["ARW", "NEF", "NRW", "RW2", "RWL", "RAW", "RAF", "CR3", "CR2", "PEF", "SRW", "ORF"].iter().any(|f| format.eq_ignore_ascii_case(f)) } impl Photo { @@ -344,7 +344,7 @@ impl Photo { pub fn develops_raw(&self) -> bool { self.kind == MediaKind::Raw && self.preview_only.is_none() } - /// The current ARW, NEF, RW2, RAF, CR3, CR2, PEF and SRW readers have vendor WB multipliers but no measured camera + /// The current ARW, NEF, RW2, RAF, CR3, CR2, PEF, SRW and ORF readers have vendor WB multipliers but no measured camera /// illuminant. Use adjustments relative to the camera's as-shot look, as for rendered /// photographs (the engine's `camera_preview::file_local_look` covers the same formats; RWL and /// RAW are Leica's and the oldest Panasonic bodies' names for RW2 files). @@ -489,6 +489,9 @@ mod edited_tests { ("a.raf", "rAf", true), ("a.cr3", "cr3", true), ("a.srw", "SRW", true), + ("a.cr2", "CR2", true), + ("a.pef", "pef", true), + ("a.orf", "ORF", true), ("a.dng", "DNG", false), ] { let mut p = Photo::new(PhotoId(1), Source::Demo { scene: 0 }, name, format, 10, 10, "2026-10-01T00:00:00"); diff --git a/crates/engine/src/camera_preview.rs b/crates/engine/src/camera_preview.rs index f5f55764..fa6731b3 100644 --- a/crates/engine/src/camera_preview.rs +++ b/crates/engine/src/camera_preview.rs @@ -1,5 +1,5 @@ //! Estimate the starting look of a raw without a camera colour matrix (Sony ARW, Nikon NEF, Panasonic -//! RW2, Fujifilm RAF, Canon CR3) from its own JPEG. Colour and luminance are fitted separately; the JPEG supplies correspondences only, +//! RW2, Fujifilm RAF, Canon CR2/CR3, Pentax PEF, Samsung SRW, Olympus ORF) from its own JPEG. Colour and luminance are fitted separately; the JPEG supplies correspondences only, //! never output pixels or a replacement for RAW editing. //! A global matrix can't follow the camera's hue-dependent rendering (the best matrix rendered a //! lime shirt olive that the camera kept lime): a hue/saturation table fitted to the residuals @@ -42,6 +42,7 @@ pub(crate) fn file_local_look(format: RawFormat) -> bool { | RawFormat::Cr2 | RawFormat::Pef | RawFormat::Srw + | RawFormat::Orf ) } @@ -1401,6 +1402,7 @@ mod tests { RawFormat::Cr2, RawFormat::Pef, RawFormat::Srw, + RawFormat::Orf, ]; assert!(local.into_iter().all(file_local_look)); // DNG carries its own colour model diff --git a/crates/engine/src/media.rs b/crates/engine/src/media.rs index f76dc6c9..90a0d416 100644 --- a/crates/engine/src/media.rs +++ b/crates/engine/src/media.rs @@ -62,7 +62,7 @@ impl SettingsHashes { } /// Bump when the pipeline's output changes, to invalidate cached thumbnails. -pub const RENDER_CACHE_VERSION: u64 = 29; +pub const RENDER_CACHE_VERSION: u64 = 30; /// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache). pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512]; diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index d51122e3..7b0f6cae 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -14,7 +14,7 @@ fn active_dev(s: &Session) -> lightcraft_develop::DevelopSettings { fn changing_one_wb_control_resolves_as_shot_without_stale_tint() { use lightcraft_catalog::{Photo, PhotoId, Source}; use lightcraft_develop::{DevelopSettings, WbMode}; - for format in ["ARW", "NEF", "NRW", "RAF", "CR2", "PEF"] { + for format in ["ARW", "NEF", "NRW", "RAF", "CR2", "PEF", "ORF"] { let mut s = demo(); let id = PhotoId(100); let name = format!("synthetic.{format}"); diff --git a/docs/camera-preview-colour.md b/docs/camera-preview-colour.md index 76ab7ab6..813b1e16 100644 --- a/docs/camera-preview-colour.md +++ b/docs/camera-preview-colour.md @@ -1,6 +1,6 @@ -# Sony ARW, Nikon NEF, Panasonic RW2, Fujifilm RAF, Canon CR3/CR2 and Pentax PEF starting look +# Sony ARW, Nikon NEF, Panasonic RW2, Fujifilm RAF, Canon CR3/CR2, Pentax PEF, Samsung SRW and Olympus ORF starting look -ARW, NEF, RW2, RAF, CR3, CR2, PEF and SRW decoding supply sensor data and camera white-balance multipliers, but currently no measured camera colour matrix. Treating camera RGB as linear sRGB produces a dark, muted default rendering. The loader now estimates a file-local starting look from that file's embedded camera JPEG, only when a usable camera matrix is absent. All output pixels still come from the RAW sensor data; there is no JPEG replacement or uniform saturation boost. +ARW, NEF, RW2, RAF, CR3, CR2, PEF, SRW and ORF decoding supply sensor data and camera white-balance multipliers, but currently no measured camera colour matrix. Treating camera RGB as linear sRGB produces a dark, muted default rendering. The loader now estimates a file-local starting look from that file's embedded camera JPEG, only when a usable camera matrix is absent. All output pixels still come from the RAW sensor data; there is no JPEG replacement or uniform saturation boost. ## Colour and tone are separate @@ -118,3 +118,13 @@ CR2 and PEF decode without a camera colour matrix too, but were left out of the flat and desaturated: the camera's preview first, then the neutral fallback). They now use the same fit, gates and relative white balance. This change was not measured on the corpus: the gates keep the fallback for any file whose fit doesn't pass, so a file can only stay as before or move closer to its camera JPEG. Per-model numbers are still to come. + +### Olympus ORF + +The ORF decoder (uncompressed files) supplies the as-shot white balance (ImageProcessing `WB_RBLevels`) and no colour +matrix, so ORF takes the same fit, gates and relative white balance. The reference is the camera's preview JPEG (on the +E-1 the 1280 × 960 one the maker note points to). All three decodable corpus files are accepted (held-out RMS against +the neutral fallback): E-1 0.126 → 0.014, E-400 0.087 → 0.022, XZ-2 0.061 → 0.023. Compressed ORFs (E-M1, E-M10 III) +still open from their embedded JPEG. Against Lightroom Classic's render of a CC0 E-1 photo (2000 px, ΔE00 of the +unedited render) the starting look moved from 11.25 to 8.30. Lightroom frames that E-1 file about 4 × 11 sensor pixels +differently from the crop the file records (`CropLeft`/`CropTop` 36, 34); the file's own crop is kept. diff --git a/docs/parity.md b/docs/parity.md index 38740af9..8872fc2d 100644 --- a/docs/parity.md +++ b/docs/parity.md @@ -296,7 +296,7 @@ under the UI command that wraps it (a no-op error); the UI command now wins. | LR-PROF-ADOBE | Standard raw looks (own equivalents) | P0 | ✅ | `crates/engine/src/presets.rs` (`PROFILES`), `crates/pipeline/src/profiles.rs` | six own looks: Color, Neutral, Vivid, Landscape, Portrait, Monochrome | | LR-PROF-ADAPTIVE | Adaptive profiles | P2 | ⬜ | | | | LR-PROF-CAMERA | Camera-matching looks | P2 | ⬜ | | | -| LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs`, `crates/raw/src/profile.rs`, `crates/raw/src/gaintable.rs`, `crates/engine/src/camera_preview.rs` | DNG files use the colour matrices and the camera-profile look they carry (`ProfileHueSatMap`, `ProfileLookTable`, `ProfileToneCurve`, read from the file at run time per DNG spec ch. 6 — Lightroom-converted DNGs rendered flat and muted without them, issue #138). A DNG's `ProfileGainTableMap` / `ProfileGainTableMap2` (Apple ProRAW's local tone map; both versions, DNG 1.7 precedence, the file's byte order with a big-endian fallback) is read and written back by DNG export and not rendered by default, as Lightroom Classic renders ProRAW without it (measured on one CC0 iPhone 12 Pro ProRAW); the Profile option "Camera local tone mapping" (`profile.camera_local_tone`, per photo, off by default, `crates/pipeline/src/local_tone.rs`) renders it as the camera does, per output pixel in the scene-linear stage (CPU, also under the GPU path); Sony ARW, Nikon NEF, Panasonic RW2, Fujifilm RAF, Canon CR3 and CR2, Pentax PEF and Samsung SRW get a guarded file-local fit to their own embedded JPEG — chromaticity matrix + hue/saturation/value table + tone and chroma curves, relative WB — or, with a bundled or local per-model profile pooled from many photos (`lightcraft-cli calibrate`, ARW, NEF and RAF so far, `crates/engine/src/camera_profiles.rs`), the profile's colour and only their own tone/chroma curves (on 29 held-out ILCE-7M4 photos mean ΔE vs the camera JPEG 3.54 → 3.10; bundled ILCE-7CR from 125 photos: 26 held-out 4.07 → 3.29, and every one of the 131 photos accepted, 8 of which fell back to neutral without it), kept with smart previews (see docs/camera-preview-colour.md); Sony ARWs shot with DRO on (maker note `0xb025`, on by default) keep that colour but their tone curve is lowered to Sony's curve without DRO, whose lifted shadows are in the camera JPEG only (issue #244: 8 ILCE-7CR photos +6.4 → +0.8 L* from Lightroom's default, median ΔE 7.5 → 3.3); ARW: on 10 public raw.pixls.us samples from 8 bodies 7 fits were accepted, mean ΔE vs the camera JPEG 17–26 → 3–10; NEF: 13 of 13 decodable samples (D750, D780, D850, D7500, Z 50 and local D7500 shots) accepted, ΔE 13–46 → 3–8 (issue #150); RW2 / RWL: 140 of 174 public samples accepted, from 93 of the 114 bodies with a preview, median ΔE 5.0 with matching lightness (13 of the 14 corpus files with a preview: ΔE 9.6–27 → 2.8–8.9; most rejections are compacts and kit zooms whose camera JPEG is distortion-corrected); a fit that fails the gates on all pixels is retried away from edges, where a lens-corrected camera JPEG still lines up (issue #232: the six distortion-corrected public ILCE-7RM2 samples ΔE 16.6–16.9 → 3.7–3.8, the corpus GX80 19.1 → 7.9); RAF: bundled X-H2S/X-T4 profiles from 201/545 photos; on 150 withheld files mean ΔE improves slightly over per-file fitting (X-H2S 1.70 → 1.62, X-T4 2.43 → 2.34); 23 of 24 references from 13 bodies accepted, including the dark X-H2S samples (8.74 → 2.25) (`docs/camera-preview-colour.md`); this is not measured camera calibration; other raws and rejected fits use a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today | +| LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs`, `crates/raw/src/profile.rs`, `crates/raw/src/gaintable.rs`, `crates/engine/src/camera_preview.rs` | DNG files use the colour matrices and the camera-profile look they carry (`ProfileHueSatMap`, `ProfileLookTable`, `ProfileToneCurve`, read from the file at run time per DNG spec ch. 6 — Lightroom-converted DNGs rendered flat and muted without them, issue #138). A DNG's `ProfileGainTableMap` / `ProfileGainTableMap2` (Apple ProRAW's local tone map; both versions, DNG 1.7 precedence, the file's byte order with a big-endian fallback) is read and written back by DNG export and not rendered by default, as Lightroom Classic renders ProRAW without it (measured on one CC0 iPhone 12 Pro ProRAW); the Profile option "Camera local tone mapping" (`profile.camera_local_tone`, per photo, off by default, `crates/pipeline/src/local_tone.rs`) renders it as the camera does, per output pixel in the scene-linear stage (CPU, also under the GPU path); Sony ARW, Nikon NEF, Panasonic RW2, Fujifilm RAF, Canon CR3 and CR2, Pentax PEF, Samsung SRW and Olympus ORF get a guarded file-local fit to their own embedded JPEG — chromaticity matrix + hue/saturation/value table + tone and chroma curves, relative WB — or, with a bundled or local per-model profile pooled from many photos (`lightcraft-cli calibrate`, ARW, NEF and RAF so far, `crates/engine/src/camera_profiles.rs`), the profile's colour and only their own tone/chroma curves (on 29 held-out ILCE-7M4 photos mean ΔE vs the camera JPEG 3.54 → 3.10; bundled ILCE-7CR from 125 photos: 26 held-out 4.07 → 3.29, and every one of the 131 photos accepted, 8 of which fell back to neutral without it), kept with smart previews (see docs/camera-preview-colour.md); Sony ARWs shot with DRO on (maker note `0xb025`, on by default) keep that colour but their tone curve is lowered to Sony's curve without DRO, whose lifted shadows are in the camera JPEG only (issue #244: 8 ILCE-7CR photos +6.4 → +0.8 L* from Lightroom's default, median ΔE 7.5 → 3.3); ARW: on 10 public raw.pixls.us samples from 8 bodies 7 fits were accepted, mean ΔE vs the camera JPEG 17–26 → 3–10; NEF: 13 of 13 decodable samples (D750, D780, D850, D7500, Z 50 and local D7500 shots) accepted, ΔE 13–46 → 3–8 (issue #150); RW2 / RWL: 140 of 174 public samples accepted, from 93 of the 114 bodies with a preview, median ΔE 5.0 with matching lightness (13 of the 14 corpus files with a preview: ΔE 9.6–27 → 2.8–8.9; most rejections are compacts and kit zooms whose camera JPEG is distortion-corrected); a fit that fails the gates on all pixels is retried away from edges, where a lens-corrected camera JPEG still lines up (issue #232: the six distortion-corrected public ILCE-7RM2 samples ΔE 16.6–16.9 → 3.7–3.8, the corpus GX80 19.1 → 7.9); RAF: bundled X-H2S/X-T4 profiles from 201/545 photos; on 150 withheld files mean ΔE improves slightly over per-file fitting (X-H2S 1.70 → 1.62, X-T4 2.43 → 2.34); 23 of 24 references from 13 bodies accepted, including the dark X-H2S samples (8.74 → 2.25); ORF: the 3 decodable corpus files (E-1, E-400, XZ-2) accepted (`docs/camera-preview-colour.md`); this is not measured camera calibration; other raws and rejected fits use a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today | | LR-PROF-CREATIVE | Creative profiles (own) | P2 | ✅ | `cmd:develop.profile`, `crates/pipeline/src/profiles.rs` | 16 own looks in Film / Cinematic / Muted / B&W (tone + point-curve fades, colour grading, mixer / B&W mix); scale with `ctl:profile.amount`; sliders untouched | | LR-PROF-LEGACY | Legacy profiles | P2 | ⬜ | | | | LR-PROF-NONRAW | Profiles for non-raw files | P0 | ✅ | `cmd:develop.profile` | same looks apply to JPEG/TIFF | diff --git a/docs/whats-new.md b/docs/whats-new.md index b88d91ad..72f50623 100644 --- a/docs/whats-new.md +++ b/docs/whats-new.md @@ -37,6 +37,10 @@ belonged to another command moves to the new one. Menus show the new keys; agents use `app.setShortcut`. ### RAW decoding +- Olympus ORF raws get the starting look fitted to the camera's own JPEG too, instead of opening flat and grey (the + E-1, E-400 and XZ-2 corpus files: all accepted), with white balance relative to the as-shot look as for the other + formats. On an E-1 photo the starting render moved from ΔE00 11.3 to 8.3 against Lightroom's. Photos already + imported pick it up when re-rendered. - Samsung SRW files without compression now open as raws: NX5, NX10, NX11, NX20, NX200, NX210, NX1000, NX1100, EX1 and WB2000. The compressed ones (NX1, NX30, NX300, NX500, NX2000, NX3000, NX3300, NX mini) still open from their camera JPEG. Photos already imported pick the change up on Reload.