shifathosn
/
lightcraft
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #86 from storytold/fix/issue-10-nef-compressed 

Issue #10: Raw: Nikon compressed NEF (lossless + lossy)
This commit is contained in:
Brandon Thomas 2026-10-05 06:08:28 -04:00 committed by GitHub
parent 8532d4ea56 6592e62912
commit 92bd3aa688
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
9 changed files with 611 additions and 45 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
ROADMAP.md
View File

@ -11,7 +11,7 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revi
| M0 | Skeleton + visual shell | workspace, xtask CI + layering, geom/color/raster, develop model, pipeline v0, catalog v0, engine commands, Lightroom-look UI (grid, loupe, filmstrip, Edit panel), control channel, MCP, web build | 3–5 | ✅ |
| M1 | Library core | import (JPEG/PNG/TIFF/WebP), EXIF/XMP, persistent catalog (op log + snapshots), albums, ratings/flags/labels, filter/search/sort, thumbnail cache, 100k-photo grid | 6–10 | ✅ (100k-photo grid scale test pending) |
| M2 | Pipeline v1 (quality) | WB temp/tint, profiles, local tone mapping (highlights/shadows), curves, HSL, point colour, colour grading, texture/clarity/dehaze, vignette, grain, B&W, auto tone/WB, histogram, before/after | 10–15 | ✅ (look tuning vs our references ongoing) |
| M3 | RAW I | TIFF/DNG (LJ92, deflate, tiles, opcodes), demosaic (AHD/PPG/bilinear), highlight recovery, DNG colour model, CR2, NEF, ARW, embedded previews | 10–15 | 🚧 (DNG, CR2, ARW, NEF uncompressed, embedded previews ✅; NEF Huffman ⬜) |
| M3 | RAW I | TIFF/DNG (LJ92, deflate, tiles, opcodes), demosaic (AHD/PPG/bilinear), highlight recovery, DNG colour model, CR2, NEF, ARW, embedded previews | 10–15 | 🚧 (DNG, CR2, ARW, NEF uncompressed + Huffman lossless/lossy, embedded previews ✅; NEF lossy-after-split ⬜) |
| M4 | Crop, geometry, optics | crop tool + overlays, straighten, Upright (auto/level/vertical/full/guided), manual transforms, CA, defringe, manual lens corrections | 6–10 | ✅ |
| M5 | Performance | source pyramids, wgpu compute pipeline (CPU oracle), draft/full renders, prefetch, budgets (16 ms slider updates on 24 MP) | 10–15 | 🚧 (stage cache, source pyramid, wgpu pipeline, prefetch, memory budget ✅; colour NR at half resolution, GPU histogram ⬜) |
| M6 | Masking | brush, linear/radial gradients, colour/luminance/depth range, add/subtract/intersect/invert, all local adjustments, masks panel | 8–12 | 🚧 (brush/linear/radial/colour/luminance range, add/subtract/intersect, masks panel ✅; depth range, AI masks ⬜) |
@ -19,7 +19,7 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revi
| M8 | Heal / Remove | content-aware remove (PatchMatch), heal, clone, brush spots, visualize spots, red/pet eye | 6–10 | 🚧 (heal, clone, auto source, visualize spots, red/pet eye ✅; PatchMatch remove ⬜) |
| M9 | Presets, profiles, versions, sync | preset browser + amount, create/import presets, profile browser, versions, history, copy/paste/sync settings | 5–8 | ✅ |
| M10 | Export & share | export dialog (JPEG/PNG/TIFF/DNG/AVIF/JXL/original), sizing, sharpening, metadata, watermark, naming, batch jobs, XMP sidecars, HDR export | 6–10 | 🚧 (all formats incl. DNG/original, sizing, presets, background jobs ✅; JXL encode, HDR export ⬜) |
| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 packed, PEF, ORF uncompressed ✅; CR3, compressed NEF/ORF/RAF ⬜) |
| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 packed, PEF, ORF uncompressed ✅; CR3, compressed ORF/RAF ⬜) |
| M12 | AI & smart features | subject/sky/background/people/object masks, semantic search, faces/People (permissively licensed models, pure-Rust inference) | 20–40 | ⬜ |
| M13 | Merge | HDR merge (deghost), panorama (projections, boundary warp, fill edges), HDR panorama | 10–15 | ✅ |
| M14 | Video | import/playback/trim via FilmCraft crates, global edits + presets on video, video export | 6–10 | ⬜ |
@ -50,7 +50,7 @@ trip, slideshow, auto import…) in ≈ 12 h on 2026-10-02; four to six parallel
| Work package | Tracker rows | Agent-hours | Risk |
|---|---|---:|---|
| Remaining P0/P1 UI and library features (folder rename/move, keyword painter, people view…) | ≈ 8 | 5–10 | low |
| Raw codecs: CR3 (CRX), compressed NEF / ORF / RAF, RW2 v4, HEIC/AVIF decode, JPEG XL DNG | LR-IMP-FORMATS | 40–80 | **high** — clean-room black-box analysis, no permissive specs |
| Raw codecs: CR3 (CRX), compressed ORF / RAF, NEF lossy-after-split, RW2 v4, HEIC/AVIF decode, JPEG XL DNG | LR-IMP-FORMATS | 40–80 | **high** — clean-room black-box analysis, no permissive specs |
| Lens-profile database of our own (calibration targets, fitting, data) | LR-EDIT-OPTICS-PROFILE | 15–30 | data collection |
| Video: playback, trim, edits, export (pure-Rust decode, ideally shared with FilmCraft) | R. Video | 20–40 | medium |
| AI: subject / sky / background / people / object masks, object-aware remove, AI denoise, super resolution, lens blur, people & faces, natural-language search, culling | ≈ 30 | 80–150 | **high** — permissively licensed weights, pure-Rust inference, maybe training |
@ -94,16 +94,16 @@ The milestone estimates in the table above were made before work started and are
## Raw format coverage and known gaps
Decoded (CC0 corpus from raw.pixls.us, `cargo xtask corpus --download`, `crates/raw/tests/corpus.rs`): DNG (uncompressed,
LJ92, lossy JPEG / Smart Previews, Deflate, float, linear), CR2, ARW (uncompressed, ARW2, LJ92), NEF/NRW uncompressed, RAF uncompressed (Bayer and
LJ92, lossy JPEG / Smart Previews, Deflate, float, linear), CR2, ARW (uncompressed, ARW2, LJ92), NEF/NRW uncompressed and Huffman-compressed (lossless, lossy type 1/2, 12/14-bit), RAF uncompressed (Bayer and
X-Trans), RW2 packed 12/14-bit, PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Every
supported container also yields its embedded JPEG preview (CR3 too), and the engine shows that preview for raw variants
it can't decode yet.
Not decoded yet — preview only (no permissively licensed description; black-box analysis incomplete):
- **Nikon Huffman NEF** (lossless / lossy). Black-box findings so far: maker note `0x0096` holds per-parity predictor
seeds (2048 for 14-bit lossless); ≈ 9 bits per pixel; the category-0 code word is a 6-bit rotation of `011111`. The
Huffman tables are not stored in the files, and an automatic table search (beam search over prefix codes scored by
smoothness and bit rate, validated on Pentax files whose tables *are* stored) did not converge.
- **Nikon "lossy after split" NEF** (non-zero split row in maker note `0x0096`, e.g. some D3400/D5000/D5200/D5500/D5600
files): rows above the split decode with the regular lossy table; from the split row on a different code is used
that our black-box analysis has not recovered yet (none of the four regular tables fits, also not with byte
alignment or reset predictors at the split row). The other Nikon Huffman variants are decoded (`crates/raw/src/vendor/nefc.rs` documents the analysis).
- **Panasonic RW2 raw format 4** (quantised): the block layout is known (0x4000-byte chunks rotated by 0x1ff8; 128-bit
blocks of two 12-bit seeds plus four groups of a 2-bit scale and three 8-bit codes; scales 0/1 are ×1/×2 differences
from the same-colour pixel two to the left), the reconstruction rule for scales 2/3 is not established.

6
crates/raw/src/lib.rs
View File

@ -11,10 +11,10 @@
//! and produces camera → linear Rec.2020 D65 matrices.
//!
//! Formats: DNG (uncompressed, lossless JPEG, lossy JPEG (Smart Previews), Deflate incl. floating point, tiled/stripped, CFA and LinearRaw),
//! Canon CR2, Nikon NEF/NRW (uncompressed), Sony ARW (uncompressed, ARW2, lossless), Fujifilm RAF (uncompressed Bayer
//! Canon CR2, Nikon NEF/NRW (uncompressed, Huffman lossless / lossy compressed), Sony ARW (uncompressed, ARW2, lossless), Fujifilm RAF (uncompressed Bayer
//! and X-Trans), Panasonic RW2 (packed 12/14-bit), Pentax PEF (uncompressed, Huffman), Olympus ORF (uncompressed).
//! [`embedded_preview`] covers all of them plus CR3. Variants we can't decode yet (Nikon Huffman NEF, Panasonic
//! quantised RW2, compressed ORF/RAF, CR3) return [`RawError::Unsupported`]; each vendor module documents its sources
//! [`embedded_preview`] covers all of them plus CR3. Variants we can't decode yet (Nikon "lossy after split" NEF,
//! Panasonic quantised RW2, compressed ORF/RAF, CR3) return [`RawError::Unsupported`]; each vendor module documents its sources
//! (public specifications, tag-name documentation, black-box analysis of CC0 samples) and gaps. Non-DNG files carry no
//! colour matrix: [`color`] falls back to a documented neutral model. The decoders never panic on malformed input.
#![forbid(unsafe_code)]

1
crates/raw/src/vendor/mod.rs vendored
View File

@ -3,6 +3,7 @@
pub mod arw;
pub mod cr2;
pub mod nef;
pub mod nefc;
pub mod orf;
pub mod pef;
pub mod raf;

81
crates/raw/src/vendor/nef.rs vendored
View File

@ -1,22 +1,22 @@
//! Nikon NEF / NRW — uncompressed variants only.
//! Nikon NEF / NRW — uncompressed and Huffman-compressed (lossless, lossy) variants.
//!
//! Sources: TIFF 6.0 (the raw image is a standard CFA SubIFD), Laurent Clévy's NEF structure notes (prose: IFD
//! layout, SubIFDs, maker note header) and the ExifTool Nikon tag-name documentation (`0x000c` WB_RBLevels,
//! `0x003d` BlackLevel). Nikon's Huffman-compressed variants (compression 34713: lossless, lossy type 1/2) are
//! **not** implemented: the only descriptions we found are derived from GPL code, which our clean-room rules
//! forbid. The coding is T.81-style (difference category + additional bits, as in Pentax PEF), but unlike PEF the
//! Huffman tables are not stored in the files and our black-box table search has not converged (see ROADMAP.md).
//! They are reported as [`RawError::Unsupported`]; their embedded previews still work.
//! `0x003d` BlackLevel, `0x0096` NEFLinearizationTable). The Huffman-compressed data (compression 34713) is decoded
//! by [`super::nefc`], which documents its clean-room sources; files it can't decode yet ("lossy after split")
//! are reported as [`RawError::Unsupported`] and their embedded previews still work.
use super::white_from_data;
use super::{nefc, white_from_data};
use crate::tiffraw::{Packing, read_image};
use crate::{BlackLevel, Cfa, ColorData, OpcodeLists, RawData, RawError, RawFormat, RawImage, Rect, Result};
use lightcraft_geom::Orientation;
use lightcraft_tiff::image::ImageInfo;
use lightcraft_tiff::tags::{self as t, photometric};
use lightcraft_tiff::{Ifd, Tiff, makernote};
use lightcraft_tiff::{ByteOrder, Ifd, Tiff, makernote};
const WB_RB_LEVELS: u16 = 0x000c;
const BLACK_LEVEL: u16 = 0x003d;
const LINEARIZATION_TABLE: u16 = 0x0096;
fn raw_ifd(tiff: &Tiff) -> Option<&Ifd> {
tiff.all_ifds()
@ -47,30 +47,15 @@ pub(crate) fn decode(bytes: &[u8]) -> Result<RawImage> {
let ifd0 = &tiff.ifds[0];
let raw = raw_ifd(&tiff).ok_or_else(|| RawError::Unsupported("NEF without a CFA image IFD".into()))?;
let info = raw.image()?;
if info.compression == t::compression::NIKON {
return Err(RawError::Unsupported("Nikon Huffman-compressed NEF (no clean-room description available)".into()));
}
let (w, h) = (info.width as usize, info.height as usize);
let bits = info.bits() as u32;
let row_samples = w * info.samples_per_pixel as usize;
let chunks = info.chunks(bytes.len() as u64);
let rows_in_first = chunks.first().map(|c| c.height as usize).unwrap_or(h).max(1);
let bytes_per_row = chunks.first().map(|c| c.len as usize / rows_in_first).unwrap_or(0);
let packing = if bytes_per_row >= row_samples * 2 {
Packing::Word16
} else if bits == 12 && bytes_per_row * 8 >= row_samples * 12 && bytes_per_row * 8 < row_samples * 13 {
Packing::Msb
} else if info.compression == 1 && bits != 8 && bits != 16 {
return Err(RawError::Unsupported(format!("NEF uncompressed packing ({bytes_per_row} bytes per {row_samples}-sample row)")));
} else {
Packing::Msb
};
let data = read_image(bytes, &info, tiff.order, packing)?;
let RawData::U16(ref samples) = data else { return Err(RawError::Unsupported("float NEF".into())) };
let make = ifd0.string(t::MAKE).unwrap_or_default();
let mn =
tiff.exif().and_then(|e| e.get(t::MAKER_NOTE)).and_then(|e| makernote::parse_makernote(bytes, e.offset, e.count() as u64, tiff.order, &make));
let data =
if info.compression == t::compression::NIKON { compressed(bytes, &info, mn.as_ref())? } else { uncompressed(bytes, &info, tiff.order)? };
let RawData::U16(ref samples) = data else { return Err(RawError::Unsupported("float NEF".into())) };
let black = match mn.as_ref().and_then(|m| m.ifd.f64s(BLACK_LEVEL)).as_deref() {
Some([a, b, c, d]) if [a, b, c, d].iter().all(|v| **v < 16384.0) => {
BlackLevel { repeat_rows: 2, repeat_cols: 2, values: vec![*a as f32, *b as f32, *c as f32, *d as f32], ..Default::default() }
@ -114,6 +99,46 @@ pub(crate) fn decode(bytes: &[u8]) -> Result<RawImage> {
Ok(img)
}
/// Uncompressed strips: 16-bit words or 12-bit MSB-packed, told apart by the strip size.
fn uncompressed(bytes: &[u8], info: &ImageInfo, order: ByteOrder) -> Result<RawData> {
let (w, h) = (info.width as usize, info.height as usize);
let bits = info.bits() as u32;
let row_samples = w * info.samples_per_pixel as usize;
let chunks = info.chunks(bytes.len() as u64);
let rows_in_first = chunks.first().map(|c| c.height as usize).unwrap_or(h).max(1);
let bytes_per_row = chunks.first().map(|c| c.len as usize / rows_in_first).unwrap_or(0);
let packing = if bytes_per_row >= row_samples * 2 {
Packing::Word16
} else if bits == 12 && bytes_per_row * 8 >= row_samples * 12 && bytes_per_row * 8 < row_samples * 13 {
Packing::Msb
} else if info.compression == 1 && bits != 8 && bits != 16 {
return Err(RawError::Unsupported(format!("NEF uncompressed packing ({bytes_per_row} bytes per {row_samples}-sample row)")));
} else {
Packing::Msb
};
read_image(bytes, info, order, packing)
}
/// Nikon Huffman-compressed strip (see [`super::nefc`]); the decode table is maker note `0x0096`.
fn compressed(bytes: &[u8], info: &ImageInfo, mn: Option<&makernote::MakerNote>) -> Result<RawData> {
// (some Z bodies label uncompressed, row-padded data 34713 without a table: not handled here yet)
let table = mn
.and_then(|m| Some((m.ifd.bytes(LINEARIZATION_TABLE)?, m.order)))
.ok_or_else(|| RawError::Unsupported("Nikon compressed NEF without a linearization table (maker note 0x96)".into()))?;
if info.samples_per_pixel != 1 {
return Err(RawError::Unsupported(format!("Nikon compressed NEF with {} samples per pixel", info.samples_per_pixel)));
}
let bits = info.bits() as u32;
let table = nefc::parse_table(table.0, table.1, bits)?;
// one strip in every sample; several would be contiguous parts of the same bit stream
let chunks = info.chunks(bytes.len() as u64);
let (Some(first), Some(last)) = (chunks.first(), chunks.last()) else { return Err(RawError::Corrupt("NEF: no image data".into())) };
let end = last.offset.saturating_add(last.len).min(bytes.len() as u64);
let src = usize::try_from(first.offset).ok().zip(usize::try_from(end).ok()).and_then(|(a, b)| bytes.get(a..b));
let src = src.ok_or_else(|| RawError::Corrupt("NEF: image data outside the file".into()))?;
Ok(RawData::U16(nefc::decode(src, info.width as usize, info.height as usize, bits, &table)?))
}
#[cfg(test)]
mod tests {
use super::*;
@ -159,7 +184,7 @@ mod tests {
}
#[test]
fn compressed_is_unsupported() {
fn compressed_without_table_is_unsupported() {
let bytes = nef(34713, 14, vec![vec![0; 64]], 8, 8, 8);
assert!(matches!(crate::decode(&bytes), Err(RawError::Unsupported(_))));
}

481
crates/raw/src/vendor/nefc.rs vendored Normal file
View File

@ -0,0 +1,481 @@
//! Nikon Huffman-compressed NEF image data (TIFF compression 34713): "lossless compressed" and "lossy compressed"
//! (type 1 and type 2) at 12 and 14 bits.
//!
//! **Clean-room.** No raw-decoder source code (dcraw, LibRaw, rawspeed, rawloader/rawler, darktable, RawTherapee,
//! libopenraw, ExifTool's Perl code, …) was read or consulted. What we used:
//!
//! - Prose format descriptions: Laurent Clévy, "Nikon Electronic File (NEF) format" (<http://lclevy.free.fr/nef/>),
//! the maker-note tag table and the "0x96 (linearization table) tag format" table (version bytes, four `u16`
//! predictor seeds at offset 2, curve size at 10, curve at 12, split value at 562 for lossy type 2, "lossy type
//! 2 reads an incomplete table and interpolation is required"); Bill Claff, "NEF Compression"
//! (<https://www.photonstophotos.net/NikonInfo/NEF_Compression.htm>): a lossy encoding curve followed by a
//! non-adaptive (fixed-table) Huffman stage, and the number of encoded values per camera (683, 769, 1025, 2753,
//! 3073, 4097 …), which confirmed our curve interpolation; the libopenraw format notes
//! (<https://libopenraw.freedesktop.org/formats/nef/>, prose only); ExifTool's Nikon tag-name documentation
//! (`0x0093` NEFCompression, `0x0096` NEFLinearizationTable); ITU-T T.81 Annex F/H for the "difference category
//! + additional bits" coding and canonical (BITS/HUFFVAL) Huffman codes.
//! - Our own black-box analysis of CC0 samples from raw.pixls.us. Nikon's Huffman tables are not stored in the
//! files and no prose source lists them, so they were recovered from the data: several cameras (D5100, D300,
//! D700) were shot with the same scene in compressed and uncompressed modes; a beam search assigned code words
//! to difference categories so that the decoded compressed image matched the uncompressed one (likelihood of
//! the decoded values given the reference, minus the bits consumed, plus a description-length penalty per new
//! code word). Each table was then verified on ~45 files from ~35 bodies (D40 … D850, Df, Z 6, Z 50, 1 J1): with
//! the right table the whole strip decodes and ends within the last byte of the data, with every value inside
//! the encoded range. The predictor and difference coding are the same as Pentax PEF (see `pef.rs`).
//!
//! Format, as established:
//! - One strip, MSB-first bit stream, no byte stuffing, continuous across rows. Per pixel a canonical Huffman code
//! gives the category `n`, followed by `n` additional bits (T.81: a leading 0 means a negative difference).
//! - Each pixel is predicted from the same-colour pixel two to the left; the first two pixels of a row from the
//! first two pixels of the previous row of the same parity, starting at the seeds of maker note `0x0096`.
//! - Lossless (`0x0096` version `0x46`): the decoded values are the samples. Lossy (version `0x44`): they index a
//! curve. Type 1 (`0x44 0x10`) stores the full curve; type 2 (`0x44 0x20`, `0x44 0x40`) stores 257 points,
//! one every `2^bits / 256` codes, linearly interpolated.
//! - Values in the maker note are in the maker note's byte order (newer bodies write little-endian notes).
//!
//! Not supported (returned as [`RawError::Unsupported`], the embedded preview is used instead):
//! - "Lossy after split" files (non-zero split row at `0x0096` offset 562): from that row on a different table
//! is used which our analysis has not recovered yet.
//! - Other `0x0096` versions, bit depths other than 12/14.
//! - Code words that never occurred in any sample (the lossy 12-bit 8-bit code word `11111110`, and code words
//! longer than 9 bits in the lossy tables) are rejected as corrupt; they cannot code a difference within the
//! lossy value range anyway.
use super::pef::{Bits, Huffman, diff};
use crate::{MAX_SAMPLES, RawError, Result};
use lightcraft_tiff::ByteOrder;
/// A table code word whose symbol was never observed (see the module docs).
const UNSEEN: u8 = u8::MAX;
/// Canonical Huffman table: number of codes of length 1..=16 and the symbols (difference categories) in code order.
struct Table {
counts: &'static [u8],
symbols: &'static [u8],
}
const LOSSLESS_12: Table = Table { counts: &[0, 1, 4, 2, 3, 1, 2], symbols: &[5, 4, 6, 3, 7, 2, 8, 1, 9, 0, 10, 11, 12] };
const LOSSLESS_14: Table = Table { counts: &[0, 1, 4, 2, 2, 3, 1, 2], symbols: &[7, 6, 8, 5, 9, 4, 10, 3, 11, 12, 2, 0, 1, 13, 14] };
const LOSSY_12: Table = Table { counts: &[0, 1, 5, 1, 1, 1, 1, 1, 1], symbols: &[5, 4, 3, 6, 2, 7, 1, 0, 8, 9, UNSEEN, 10] };
const LOSSY_14: Table = Table { counts: &[0, 1, 4, 3, 1, 1, 1, 1, 1], symbols: &[5, 6, 4, 7, 8, 3, 9, 2, 1, 0, 10, 11, 12] };
impl Table {
/// `(left-aligned 12-bit code, length)` per symbol, for [`Huffman::new`].
fn codes(&self) -> Vec<(u16, u8)> {
let mut out = vec![(0u16, 0u8); 17];
let mut code = 0u32;
let mut symbols = self.symbols.iter();
for (i, &n) in self.counts.iter().enumerate() {
let len = i as u32 + 1;
for _ in 0..n {
if let Some(&s) = symbols.next()
&& let Some(slot) = out.get_mut(s as usize)
{
*slot = ((code << (12 - len)) as u16, len as u8);
}
code += 1;
}
code <<= 1;
}
out
}
}
/// How the decoded values map to samples.
#[derive(Clone, Debug, PartialEq)]
pub(crate) enum Encoding {
/// The decoded values are the samples (`0..2^bits`).
Lossless,
/// The decoded values index this curve.
Lossy(Vec<u16>),
}
/// Maker note `0x0096` (NEF linearization table), parsed.
#[derive(Clone, Debug, PartialEq)]
pub(crate) struct DecodeTable {
pub encoding: Encoding,
/// Predictor seeds `[row parity][column]`.
pub seeds: [[i32; 2]; 2],
}
/// Parse maker note `0x0096` for a `bits`-bit image. `order` is the maker note's byte order.
pub(crate) fn parse_table(t: &[u8], order: ByteOrder, bits: u32) -> Result<DecodeTable> {
let u16_at = |o: usize| t.get(o..o + 2).and_then(|s| <[u8; 2]>::try_from(s).ok()).map(|b| order.u16(b));
let short = || RawError::Corrupt(format!("NEF: linearization table too short ({} bytes)", t.len()));
if bits != 12 && bits != 14 {
return Err(RawError::Unsupported(format!("Nikon compressed NEF with {bits}-bit samples")));
}
let (v0, v1) = (t.first().copied().ok_or_else(short)?, t.get(1).copied().ok_or_else(short)?);
let mut seeds = [[0i32; 2]; 2];
for (i, s) in seeds.iter_mut().flatten().enumerate() {
*s = u16_at(2 + 2 * i).ok_or_else(short)? as i32;
}
let encoding = match (v0, v1) {
(0x46, _) => Encoding::Lossless,
(0x44, 0x10 | 0x20 | 0x40) => {
let n = u16_at(10).ok_or_else(short)? as usize;
let points: Vec<u16> = (0..n).map(|i| u16_at(12 + 2 * i)).collect::<Option<_>>().ok_or_else(short)?;
if n < 2 {
return Err(RawError::Corrupt(format!("NEF: linearization curve of {n} points")));
}
if v1 == 0x10 {
Encoding::Lossy(points)
} else {
let split = u16_at(562).unwrap_or(0);
if split != 0 {
return Err(RawError::Unsupported(format!(
"Nikon \"lossy after split\" compressed NEF (split at row {split}) is not decoded yet"
)));
}
// type 2: `n` points, one every 2^bits / (n - 1) codes
let range = 1usize << bits;
if n - 1 > range || !range.is_multiple_of(n - 1) {
return Err(RawError::Unsupported(format!("NEF: {n}-point lossy curve for {bits}-bit data")));
}
let step = range / (n - 1);
let mut curve = Vec::with_capacity(range + 1);
for pair in points.windows(2) {
let &[a, b] = pair else { continue };
let (a, b) = (a as u32, b as u32);
for k in 0..step as u32 {
let s = step as u32;
curve.push(((a * (s - k) + b * k) / s) as u16);
}
}
curve.extend(points.last().copied());
Encoding::Lossy(curve)
}
}
_ => return Err(RawError::Unsupported(format!("Nikon compressed NEF version {v0:#04x} {v1:#04x}"))),
};
Ok(DecodeTable { encoding, seeds })
}
/// Decode a `w × h` Nikon Huffman-compressed strip `src` with `bits`-bit samples.
pub(crate) fn decode(src: &[u8], w: usize, h: usize, bits: u32, table: &DecodeTable) -> Result<Vec<u16>> {
if bits != 12 && bits != 14 {
return Err(RawError::Unsupported(format!("Nikon compressed NEF with {bits}-bit samples")));
}
let n = w.checked_mul(h).filter(|&n| n > 0 && n <= MAX_SAMPLES).ok_or(RawError::Limit("NEF image size"))?;
// every code word is at least 2 bits long: a strip that can't hold the image is truncated (also bounds the
// allocation by the file size)
if n / 4 > src.len() {
return Err(RawError::Corrupt(format!("NEF: {} bytes of compressed data for {w}x{h} pixels", src.len())));
}
let huff = Huffman::new(
&match (&table.encoding, bits) {
(Encoding::Lossless, 12) => LOSSLESS_12,
(Encoding::Lossless, _) => LOSSLESS_14,
(Encoding::Lossy(_), 12) => LOSSY_12,
(Encoding::Lossy(_), _) => LOSSY_14,
}
.codes(),
)?;
let (lut, max): (&[u16], i32) = match &table.encoding {
Encoding::Lossless => (&[], (1i32 << bits) - 1),
Encoding::Lossy(curve) => (curve, curve.len() as i32 - 1),
};
let mut out = vec![0u16; n];
let mut stream = Bits::new(src);
let mut vpred = table.seeds;
let mut clipped = 0usize;
for (y, row) in out.chunks_exact_mut(w).enumerate() {
let seeds = vpred.get_mut(y & 1).ok_or(RawError::Limit("NEF row"))?;
let mut hpred = [0i32; 2];
for (x, o) in row.iter_mut().enumerate() {
let d = diff(&mut stream, &huff).ok_or_else(|| RawError::Corrupt(format!("NEF: invalid Huffman code at row {y}")))?;
let p = if x < 2 {
let s = seeds.get_mut(x).ok_or(RawError::Limit("NEF column"))?;
*s += d;
hpred[x & 1] = *s;
*s
} else {
hpred[x & 1] += d;
hpred[x & 1]
};
let v = p.clamp(0, max);
clipped += (v != p) as usize;
*o = if lut.is_empty() { v as u16 } else { lut.get(v as usize).copied().unwrap_or(0) };
}
// the camera pads the strip; reading more than a few bytes past its end means truncated or corrupt data
if stream.consumed_bits() > src.len() * 8 + 64 {
return Err(RawError::Corrupt(format!("NEF: compressed data ends at row {y} of {h}")));
}
}
// a wrong table / corrupt stream drifts out of range quickly; real files stay inside (a handful of edge pixels)
if clipped > n / 1000 + 16 {
return Err(RawError::Corrupt(format!("NEF: {clipped} decoded values out of range")));
}
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
/// MSB-first bit writer for the test encoder.
#[derive(Default)]
struct Writer {
out: Vec<u8>,
acc: u64,
n: u32,
}
impl Writer {
fn put(&mut self, v: u32, len: u32) {
for i in (0..len).rev() {
self.acc = (self.acc << 1) | ((v >> i) & 1) as u64;
self.n += 1;
if self.n == 8 {
self.out.push(self.acc as u8);
self.acc = 0;
self.n = 0;
}
}
}
fn finish(mut self) -> Vec<u8> {
if self.n > 0 {
self.out.push((self.acc << (8 - self.n)) as u8);
}
self.out.extend_from_slice(&[0; 4]);
self.out
}
}
fn category(d: i32) -> u32 {
32 - d.unsigned_abs().leading_zeros()
}
/// Encode `values` (decoded domain: samples or curve indices) the way the camera does.
fn encode(values: &[i32], w: usize, table: &Table, seeds: [[i32; 2]; 2]) -> Vec<u8> {
let codes = table.codes();
let mut wr = Writer::default();
let mut vpred = seeds;
for (y, row) in values.chunks(w).enumerate() {
for (x, &v) in row.iter().enumerate() {
let pred = if x < 2 { vpred[y & 1][x] } else { row[x - 2] };
if x < 2 {
vpred[y & 1][x] = v;
}
let d = v - pred;
let k = category(d);
let (code, len) = codes[k as usize];
assert!(len > 0, "no code for category {k}");
wr.put(code as u32 >> (12 - len), len as u32);
let extra = if d < 0 { d + (1 << k) - 1 } else { d };
wr.put(extra as u32, k);
}
}
wr.finish()
}
/// Deterministic test image: smooth gradient + texture + hard edges, within `0..=max`.
fn image(w: usize, h: usize, max: i32, seed: u32) -> Vec<i32> {
let mut s = seed;
(0..w * h)
.map(|i| {
let (x, y) = ((i % w) as i32, (i / w) as i32);
s = s.wrapping_mul(1_664_525).wrapping_add(1_013_904_223);
let noise = (s >> 24) as i32 % 9 - 4;
let base = (x * 37 + y * 11) % (max / 2) + if (x / 7 + y / 5) % 3 == 0 { max / 3 } else { 0 };
(base + noise).clamp(0, max)
})
.collect()
}
fn lossless(bits: u32) -> DecodeTable {
let s = 1 << (bits - 3);
DecodeTable { encoding: Encoding::Lossless, seeds: [[s, s], [s, s]] }
}
#[test]
fn tables_are_complete_prefix_codes() {
for t in [LOSSLESS_12, LOSSLESS_14, LOSSY_12, LOSSY_14] {
assert_eq!(t.counts.iter().map(|&c| c as usize).sum::<usize>(), t.symbols.len());
let kraft: f64 = t.counts.iter().enumerate().map(|(i, &c)| c as f64 / (1u64 << (i + 1)) as f64).sum();
assert!(kraft <= 1.0);
assert!(Huffman::new(&t.codes()).is_ok());
}
assert_eq!(LOSSLESS_14.codes()[7], (0b00 << 10, 2));
assert_eq!(LOSSLESS_14.codes()[0], (0b111110 << 6, 6));
assert_eq!(LOSSY_12.codes()[10], (0b111111110 << 3, 9));
}
#[test]
fn lossless_roundtrip_12_and_14() {
for (bits, table) in [(12, LOSSLESS_12), (14, LOSSLESS_14)] {
let (w, h) = (64, 9);
let max = (1 << bits) - 1;
let mut img = image(w, h, max, bits);
// extreme jumps exercise the longest categories
img[5] = 0;
img[7] = max;
img[w + 9] = max;
let t = lossless(bits);
let src = encode(&img, w, &table, t.seeds);
let out = decode(&src, w, h, bits, &t).unwrap();
assert_eq!(out, img.iter().map(|&v| v as u16).collect::<Vec<_>>(), "{bits}-bit");
}
}
fn lossy_table(bits: u32, order: ByteOrder) -> Vec<u8> {
// version 0x44 0x20, seeds, 257 points of a concave curve, padding up to the split value (0)
let mut t = vec![0x44, 0x20];
let put = |t: &mut Vec<u8>, v: u16| t.extend_from_slice(&if order == ByteOrder::Big { v.to_be_bytes() } else { v.to_le_bytes() });
for _ in 0..4 {
put(&mut t, 300);
}
put(&mut t, 257);
let max = ((1u32 << bits) - 1) as f64;
for i in 0..257u32 {
put(&mut t, (max * (i as f64 / 64.0).min(1.0).sqrt()) as u16);
}
t.resize(624, 0);
t
}
#[test]
fn lossy_type2_curve_and_roundtrip() {
for (bits, table) in [(12u32, LOSSY_12), (14, LOSSY_14)] {
for order in [ByteOrder::Big, ByteOrder::Little] {
let t = parse_table(&lossy_table(bits, order), order, bits).unwrap();
assert_eq!(t.seeds, [[300, 300], [300, 300]]);
let Encoding::Lossy(curve) = &t.encoding else { panic!() };
let step = (1usize << bits) / 256;
assert_eq!(curve.len(), (1 << bits) + 1);
assert_eq!(curve[64 * step], (1 << bits) - 1);
assert!(curve.windows(2).all(|p| p[0] <= p[1]));
// the encoded range is the curve's rising part
let (w, h) = (48, 7);
let img = image(w, h, 64 * step as i32 - 1, 7);
let src = encode(&img, w, &table, t.seeds);
let out = decode(&src, w, h, bits, &t).unwrap();
assert_eq!(out, img.iter().map(|&v| curve[v as usize]).collect::<Vec<_>>());
}
}
}
#[test]
fn lossy_type1_full_curve() {
let mut t = vec![0x44, 0x10, 0, 10, 0, 10, 0, 10, 0, 10, 0, 5];
for v in [0u16, 100, 400, 900, 1600] {
t.extend_from_slice(&v.to_be_bytes());
}
let parsed = parse_table(&t, ByteOrder::Big, 12).unwrap();
assert_eq!(parsed.encoding, Encoding::Lossy(vec![0, 100, 400, 900, 1600]));
let img = vec![0, 4, 1, 3, 2, 2, 4, 0];
let src = encode(&img, 4, &LOSSY_12, parsed.seeds);
assert_eq!(decode(&src, 4, 2, 12, &parsed).unwrap(), vec![0, 1600, 100, 900, 400, 400, 1600, 0]);
}
#[test]
fn rejects_unsupported_and_malformed_tables() {
let unsupported = |r: Result<DecodeTable>| matches!(r, Err(RawError::Unsupported(_)));
let corrupt = |r: Result<DecodeTable>| matches!(r, Err(RawError::Corrupt(_)));
let mut split = lossy_table(12, ByteOrder::Big);
split[562..564].copy_from_slice(&345u16.to_be_bytes());
assert!(unsupported(parse_table(&split, ByteOrder::Big, 12)));
assert!(unsupported(parse_table(&[0x49, 0x30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], ByteOrder::Big, 12)));
assert!(unsupported(parse_table(&lossy_table(12, ByteOrder::Big), ByteOrder::Big, 16)));
assert!(corrupt(parse_table(&[], ByteOrder::Big, 12)));
assert!(corrupt(parse_table(&[0x46, 0x30, 8], ByteOrder::Big, 14)));
assert!(corrupt(parse_table(&lossy_table(12, ByteOrder::Big)[..100], ByteOrder::Big, 12)));
// 300 points don't divide the 12-bit range
let mut odd = lossy_table(12, ByteOrder::Big);
odd[10..12].copy_from_slice(&300u16.to_be_bytes());
assert!(parse_table(&odd, ByteOrder::Big, 12).is_err());
assert!(parse_table(&[0x44, 0x20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0], ByteOrder::Big, 12).is_err());
}
/// A minimal NEF: CFA SubIFD with one compressed strip, Exif maker note (`Nikon\0` v2 header + embedded TIFF in
/// `mn_order`) holding the linearization table.
fn nef_file(strip: Vec<u8>, w: u32, h: u32, bits: u16, table: Vec<u8>, mn_order: ByteOrder) -> Vec<u8> {
use lightcraft_tiff::tags::{self as t, photometric};
use lightcraft_tiff::{IfdBuilder, ImageData, TiffWriter, Value};
let mut mn = IfdBuilder::new();
mn.set(0x0096, Value::Undefined(table));
let mut note = b"Nikon\0\x02\x10\0\0".to_vec();
note.extend(TiffWriter::new(mn_order, false).write(&[mn]).unwrap());
let mut exif = IfdBuilder::new();
exif.set(t::MAKER_NOTE, Value::Undefined(note));
let mut raw = IfdBuilder::new();
raw.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![0]));
raw.set(t::IMAGE_WIDTH, Value::Long(vec![w]));
raw.set(t::IMAGE_LENGTH, Value::Long(vec![h]));
raw.set(t::BITS_PER_SAMPLE, Value::Short(vec![bits]));
raw.set(t::COMPRESSION, Value::Short(vec![t::compression::NIKON]));
raw.set(t::PHOTOMETRIC, Value::Short(vec![photometric::CFA]));
raw.set(t::CFA_REPEAT_PATTERN_DIM, Value::Short(vec![2, 2]));
raw.set(t::CFA_PATTERN_EP, Value::Byte(vec![0, 1, 1, 2]));
raw.set_image(ImageData::Strips { rows_per_strip: h, strips: vec![strip] });
let mut ifd0 = IfdBuilder::new();
ifd0.set(t::MAKE, Value::Ascii("NIKON CORPORATION".into()));
ifd0.set(t::MODEL, Value::Ascii("NIKON TEST".into()));
ifd0.set_child(t::EXIF_IFD, exif);
ifd0.add_sub_ifd(raw);
TiffWriter::new(ByteOrder::Big, false).write(&[ifd0]).unwrap()
}
#[test]
fn decodes_through_the_nef_container() {
// lossless 14-bit, big-endian maker note
let (w, h) = (40usize, 6usize);
let img = image(w, h, 16383, 11);
let mut table = vec![0x46, 0x30];
for _ in 0..4 {
table.extend_from_slice(&2048u16.to_be_bytes());
}
table.resize(46, 0);
let src = encode(&img, w, &LOSSLESS_14, [[2048; 2]; 2]);
let bytes = nef_file(src, w as u32, h as u32, 14, table, ByteOrder::Big);
let r = crate::decode(&bytes).unwrap();
assert_eq!(r.data, crate::RawData::U16(img.iter().map(|&v| v as u16).collect()));
assert_eq!(r.cfa.as_ref().unwrap().name(), "RGGB");
assert_eq!(crate::probe_info(&bytes).unwrap(), r.info());
// lossy 12-bit type 2, little-endian maker note (newer bodies)
let table = lossy_table(12, ByteOrder::Little);
let t = parse_table(&table, ByteOrder::Little, 12).unwrap();
let Encoding::Lossy(curve) = &t.encoding else { panic!() };
let img = image(w, h, 1023, 5);
let src = encode(&img, w, &LOSSY_12, t.seeds);
let r = crate::decode(&nef_file(src.clone(), w as u32, h as u32, 12, table.clone(), ByteOrder::Little)).unwrap();
assert_eq!(r.data, crate::RawData::U16(img.iter().map(|&v| curve[v as usize]).collect()));
// the split variant falls back to the preview
let mut split = table;
split[562..564].copy_from_slice(&3u16.to_le_bytes());
let e = crate::decode(&nef_file(src, w as u32, h as u32, 12, split, ByteOrder::Little));
assert!(matches!(e, Err(RawError::Unsupported(_))), "{e:?}");
}
#[test]
fn truncated_and_corrupted_streams_error_never_panic() {
let (w, h) = (64, 16);
let t = lossless(14);
let img = image(w, h, 16383, 3);
let src = encode(&img, w, &LOSSLESS_14, t.seeds);
// truncation: too short for the image, or ends early
for cut in [0, 1, 10, src.len() / 2, src.len() - 40] {
assert!(decode(&src[..cut], w, h, 14, &t).is_err(), "cut {cut}");
}
// garbage and bit flips: an error or (rarely) some image, never a panic
let mut s = 12345u32;
for i in 0..200 {
let mut bad = src.clone();
for _ in 0..1 + i % 8 {
s = s.wrapping_mul(1_103_515_245).wrapping_add(12345);
let at = (s >> 8) as usize % bad.len();
bad[at] ^= 1 << (s % 8);
}
let _ = decode(&bad, w, h, 14, &t);
let noise: Vec<u8> = (0..src.len()).map(|k| ((k as u32).wrapping_mul(2_654_435_761) >> (i % 24)) as u8).collect();
let _ = decode(&noise, w, h, 14, &t);
let _ = decode(&noise, w, h, 12, &lossless(12));
}
// all-ones never forms a valid lossy 12-bit code
assert!(decode(&[0xff; 64], 8, 8, 12, &parse_table(&lossy_table(12, ByteOrder::Big), ByteOrder::Big, 12).unwrap()).is_err());
// absurd dimensions are limited before allocating
assert!(decode(&src, usize::MAX, 2, 14, &t).is_err());
assert!(decode(&src, 1 << 20, 1 << 20, 14, &t).is_err());
assert!(decode(&src, 0, 2, 14, &t).is_err());
}
}

4
crates/raw/src/vendor/pef.rs vendored
View File

@ -99,6 +99,10 @@ impl<'a> Bits<'a> {
self.skip(k);
v
}
/// Bits handed out so far (may exceed the source length: zeros are read past its end).
pub fn consumed_bits(&self) -> usize {
self.pos * 8 - self.n as usize
}
pub fn overrun(&self) -> bool {
self.pos > self.src.len() + 8
}

55
crates/raw/tests/corpus.rs
View File

@ -23,8 +23,6 @@ const KNOWN_UNSUPPORTED: &[&str] = &[
"rw2-panasonic-gx80", // "
"rw2-panasonic-g9-b", // "
"orf-olympus-em", // Olympus compressed ORF
"nef-nikon-d5100-lossless", // Nikon Huffman NEF
"nef-nikon-d7000-lossy", // "
"sraw", // Canon sRAW / mRAW
];
@ -94,3 +92,56 @@ fn corpus_raw_decodes() {
}
eprintln!("corpus/raw: {ok} decoded, {unsupported} known-unsupported (preview only)");
}
/// Green-channel means of 32×32 blocks.
fn block_means(img: &lightcraft_raw::RawImage) -> Vec<f64> {
let lightcraft_raw::RawData::U16(d) = &img.data else { panic!("float data") };
let (w, h, b) = (img.width, img.height, 32);
let mut out = Vec::new();
for by in 0..h / b {
for bx in 0..w / b {
let (mut s, mut n) = (0f64, 0f64);
for y in by * b..by * b + b {
for x in (bx * b..bx * b + b).step_by(2) {
s += d[y * w + x + (y & 1)] as f64;
n += 1.0;
}
}
out.push(s / n);
}
}
out
}
fn correlation(a: &[f64], b: &[f64]) -> f64 {
let n = a.len() as f64;
let (ma, mb) = (a.iter().sum::<f64>() / n, b.iter().sum::<f64>() / n);
let cov: f64 = a.iter().zip(b).map(|(x, y)| (x - ma) * (y - mb)).sum();
let va: f64 = a.iter().map(|x| (x - ma).powi(2)).sum();
let vb: f64 = b.iter().map(|y| (y - mb).powi(2)).sum();
cov / (va * vb).sqrt()
}
/// raw.pixls.us has the same D5100 scene as 14-bit lossless compressed and uncompressed NEF: the Huffman decode must
/// match the uncompressed image (up to the small differences between two exposures).
#[test]
fn corpus_nef_compressed_matches_uncompressed() {
let dir = corpus_root().join("raw");
let (Ok(a), Ok(b)) = (std::fs::read(dir.join("nef-nikon-d5100-lossless.nef")), std::fs::read(dir.join("nef-nikon-d5100-uncompressed.nef")))
else {
eprintln!("skip: D5100 NEF pair absent");
return;
};
let (a, b) = (decode(&a).unwrap(), decode(&b).unwrap());
assert_eq!((a.width, a.height, a.bits), (b.width, b.height, b.bits));
let r = correlation(&block_means(&a), &block_means(&b));
eprintln!("D5100 lossless vs uncompressed: block correlation {r:.4}");
assert!(r > 0.98, "correlation {r}");
// the lossy 12-bit D7000 file decodes into the curve's range and isn't flat
if let Ok(c) = std::fs::read(dir.join("nef-nikon-d7000-lossy12.nef")) {
let c = decode(&c).unwrap();
let m = block_means(&c);
let (lo, hi) = m.iter().fold((f64::MAX, 0f64), |(l, h), &v| (l.min(v), h.max(v)));
assert!(hi <= 4095.0 && hi - lo > 500.0, "D7000 block means {lo}..{hi}");
}
}

8
docs/parity.md
View File

@ -57,9 +57,9 @@ Weighted completion (✅ = 1, 🟡 = ½, 🚫 left out): **79.4%** of 502 in-sco
Ordered by tier, then user value, then (low) effort. Take the first one nobody is working on.
1. **LR-IMP-FORMATS** (P0) — CR3, compressed NEF/RAF/ORF, RW2 v4 (preview only today: such photos are marked
`preview_only` and the UI says they are edited from the camera's embedded JPEG, issue #10); HEIC/AVIF decode (no
permissive pure-Rust decoder yet). High value, high effort (clean sources needed).
1. **LR-IMP-FORMATS** (P0) — CR3, compressed RAF/ORF, RW2 v4, Nikon "lossy after split" NEF (preview only today:
such photos are marked `preview_only` and the UI says they are edited from the camera's embedded JPEG); HEIC/AVIF
decode (no permissive pure-Rust decoder yet). High value, high effort (clean sources needed).
2. **LR-EDIT-OPTICS-PROFILE** (P1) — a lens-profile database of our own (embedded DNG/maker corrections work today).
## Shortcuts: conflicts and missing bindings
@ -110,7 +110,7 @@ under the UI command that wraps it (a no-op error); the UI command now wins.
| LR-IMP-PROFILES | Import profiles & presets | P1 | ✅ | `cmd:file.importPresets`, `cmd:preset.import`, `cmd:profile.import`, `crates/engine/src/preset_import.rs`, `crates/engine/src/preset_luminar.rs` | presets: .lcpreset, XMP, classic .lrtemplate, photos carrying edits ("DNG presets"), Luminar looks (.lmp, .mplumpack collections; sliders with a counterpart), .zip bundles, folders (folder → group), drag & drop; masks carried over; unmapped settings reported; profiles: .cube 3D LUTs; Adobe profile formats deliberately unsupported |
| LR-IMP-LOCAL | Work on files in place | P0 | ✅ | `cmd:library.browse`, `cmd:photo.addToLibrary`, `cmd:library.import` (mode add), `cmd:local.addRoot`, `cmd:local.hide`, `cmd:local.restoreHidden`, `crates/engine/src/cmd/browse.rs`, `crates/ui-egui/src/panels/left.rs` (`local_section`), `crates/ui-egui/src/panels/grid.rs` (`folder_header`) | Local: browse Pictures / Desktop / Downloads / Home or any folder without adding it (breadcrumb, Include subfolders, Add N to My Photos); browsed photos stay out of All Photos, albums and counts; folders picked with Browse Folder… (or Keep in Local / Add to Local on a folder row) stay listed across restarts, and browsing a folder inside a listed one highlights it inside that tree (opened down to it, siblings reachable) instead of adding a root; edits go to XMP sidecars; importing promotes them; right-click a top-level location → Remove from Local hides the shortcut only (nothing on disk or in the catalog changes), saved with the UI state, and “Show N hidden locations” (or Browse Folder… on it) brings it back; a folder is one location however its path is spelled (`/` or `\`, trailing separator, `.`/`..`, drive-letter case; `folder_key` in `crates/catalog/src/query.rs`); browsed photos nobody changed are forgotten once their folder has not been browsed for 30 days (Settings → Performance → Local folders, 0 = never; checked when the library opens, or `cmd:library.forgetLocal` with `dryRun`; files and sidecars stay, browsing again brings them back; see `crates/catalog/src/local.rs` for what counts as a change) |
| LR-IMP-SIDECAR-SPLIT | Separate XMP sidecar variants | P2 | ⬜ | `cmd:library.xmpPreferences` | sidecar naming option exists (stem/full), no split sidecars |
| LR-IMP-FORMATS | Supported formats | P0 | 🟡 | `crates/codecs/src/lib.rs`, `crates/raw/src/lib.rs` | JPEG, PNG, TIFF, WebP, JXL, PSD, GIF, BMP; DNG (incl. lossy / Smart Preview DNG), CR2, ARW, NEF, RAF, RW2, PEF, ORF. Missing: CR3, compressed NEF/RAF/ORF, RW2 v4 (preview only: imported with `preview_only` = the decoder's reason, rendered as a rendered JPEG, flagged by a grid badge and loupe/Edit/Info notices, `previewOnly` in `catalog.query`; Reload clears it once the file decodes), HEIC/AVIF decode |
| LR-IMP-FORMATS | Supported formats | P0 | 🟡 | `crates/codecs/src/lib.rs`, `crates/raw/src/lib.rs` | JPEG, PNG, TIFF, WebP, JXL, PSD, GIF, BMP; DNG (incl. lossy / Smart Preview DNG), CR2, ARW, NEF (uncompressed, lossless and lossy compressed 12/14-bit, `crates/raw/src/vendor/nefc.rs`), RAF, RW2, PEF, ORF. Missing: CR3, compressed RAF/ORF, RW2 v4, NEF "lossy after split" (preview only: imported with `preview_only` = the decoder's reason, rendered as a rendered JPEG, flagged by a grid badge and loupe/Edit/Info notices, `previewOnly` in `catalog.query`; Reload clears it once the file decodes), HEIC/AVIF decode |
| LR-IMP-CULL-AT-IMPORT | Culling analysis at import | P2 | 🟡 | `cmd:photo.analyze` | run Assisted Culling on the imported photos (they're selected after an import); not automatic |
| LR-IMP-MOVE | Move on import [Classic] | P1 | ✅ | `cmd:library.import` (`mode` move), `crates/engine/src/import_move.rs`, `crates/engine/src/tests_import_move.rs`, `crates/ui-egui/src/import.rs` | Transfer ▸ Move with Copy's destination, folders (day / month / one folder / custom template), rename template and example destination; XMP sidecars (both namings) move along; each source is removed only after its destination is written (hard link on the same volume, else copied, synced and compared byte for byte) and its catalog record is saved; failed, duplicate, unchecked files keep their sources, taken names get -1, -2…, never overwritten; files already in the destination/library are added in place; sources that can't be removed (read-only card) are kept and reported (`kept`). Undo removes the photos from the library but leaves the moved files at the destination. No Copy as DNG while moving |
| LR-IMP-DNG-CONVERT | Convert to DNG on import [Classic] | P2 | ✅ | `cmd:library.import` (`dng`), `crates/ui-egui/src/import.rs` | copy imports: Raw files ▸ Copy as DNG (lossless; the card is untouched) |

4
docs/whats-new.md
View File

@ -45,3 +45,7 @@
- Slideshow, second window, All Metadata, System Info.
- Edit in External Editor (⇧⌘E): a 16-bit TIFF copy, stacked, refreshed when you come back.
- Lossy DNG files and Smart Previews open as raw photos.
- Compressed Nikon NEFs (lossless and lossy compressed, 12- and 14-bit — e.g. D3200, D5100, D7000, D750, D850, Z 50)
now develop from the raw data instead of the camera's embedded JPEG, so a B&W or other picture style set in the
camera no longer gets baked in. Photos already imported as "preview only" switch over on Reload. (Files that
Nikon splits into two differently compressed halves still use the preview for now.)