- tiffraw: Mode::Header parses the first LJ92 frame header (Compression 7),
so probe_info reports Unsupported for subsampled frames (Sony lossless
M/S) as the full decode does, instead of claiming the file decodes.
- corpus: CC0 ILCE-7M4 lossless L/M/S samples from raw.pixls.us in
`cargo xtask corpus`; M/S listed as known unsupported.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DNG compression 34892: each tile / strip is a baseline JPEG of 8-bit samples (decoded with
zune-jpeg, pure Rust), then the usual linearization table, levels and crop apply. Per the DNG
specification. Lossy DNGs (Smart Previews, "lossy compression" DNG exports, many phone DNGs)
now open as raw files instead of falling back to their embedded preview. Unit test with
synthetic JPEG tiles (edge tile cut short); the corpus sample decodes (5760×3840 in 144 ms).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sony's lossless-compressed raw (Compression 7, e.g. ILCE-7M4) stores
512×512 tiles as LJ92 frames of 256×256 with four components: each
frame sample is one 2×2 CFA cell (top-left, top-right, bottom-left,
bottom-right). The generic TIFF path reads a frame as a row-major sample
stream (DNG's convention), which interleaved each tile's left and right
halves row by row: vertical bands, scrambled CFA, purple cast.
arw.rs now detects the layout from the first tile's frame header and
places the cells itself; other lossless-JPEG layouts keep the generic
path. Component order verified on a real ILCE-7M4 file (green
differences to the diagonal neighbour cell match those within a cell).
RENDER_CACHE_VERSION bumped so cached thumbnails of affected files are
re-rendered.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCLtGBctp1CxX2MdCuKCrm
A clipped pixel is never valid at full resolution, so its diffused chromaticity was always the
bilinear sample of the half-resolution pyramid level; the full-resolution chroma field (288 MB
at 24 MP) and its fill pass were wasted work. The pyramid now starts at half resolution straight
from the image. Bit-identical to the previous implementation (kept as a test reference over odd
sizes and clipping densities). 24 MP: 319 -> 145 ms isolated; real CR2/ARW loads 0.8-1.1 s ->
0.3-0.5 s under load. Runs on every full-resolution raw load (1:1 loupe, export).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- memory::budget (default min(25% RAM, 1.5 GiB); LIGHTCRAFT_MEMORY_MB, app.memoryBudget):
half for engine caches (sources + rendered previews, LRU across them via a global
use tick; the photo on screen stays), a quarter for decodes in flight (WorkGate:
thumbnails, prefetch and import probes wait; loupe/export loads never wait), an
eighth for the GPU free pool.
- GPU: best-fit buffer reuse (<= 25% slack), buffers released outside a render are
pooled at once (no stranded retired lists), pool trimmed after 3 s idle.
- Decodes drop file bytes and raw samples early, colour/crop/orient in place.
- macOS: apps install malloc_zone_pressure_relief as the release hook (after each
decode, import and idle trim): freed large blocks no longer stay resident.
14 raws + loupe + 12 steps (M4 Pro): peak RSS 1.9-2.3 GB -> 0.83-0.99 GB, footprint
2.5-2.6 GB -> 0.91-0.96 GB, heap peak 930 -> 656 MB; stepping median ~13 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decoders take a Mode: Header reads everything but the samples (DNG, CR2, ARW,
RAF, RW2; PEF when it has crop tags). NEF/ORF (and PEF without crop tags)
derive geometry/CFA from the samples and still unpack them. decode() stays
bit-exact (debug-hash of every corpus/raw decode identical before/after); the
corpus test asserts probe_info == decode().info() for all 37 files.
Import of 14 corpus raws: 3.4 s -> 1.4-2.0 s; peak RSS through import
1.41-1.49 GB -> 1.12-1.22 GB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- lightcraft-raw: RawImage::develop_binned(k) — k×k superpixel binning of the CFA
(Bayer k even, X-Trans k multiple of 3), black/white normalised row by row, default
crop only, clipping preserved for highlight reconstruction. 2560 px previews of a
24 MP raw bin 2×, thumbnails 4–8×; exports / 1:1 still demosaic (AHD) at full size.
- highlight::reconstruct and its pyramid fill are rayon-parallel (bit-exact).
- X-Trans directional demosaic: precomputed per-phase neighbour offsets and no
reflection in the interior, no per-pixel allocation (bit-exact, tested against the
reference formulation): 2.9 s → 0.75 s CPU at 24 MP.
- files::load_bytes runs on a rayon worker so its short parallel loops don't each
wait for the pool to wake (dominant cost on a loaded machine); LIGHTCRAFT_PROFILE
prints per-stage source timings; files::bin_factor picks the block size.
- render_bench ONLY=source: per-file decode / normalise / demosaic / binning / load.
- Fix: u64 overflow on hostile IFD sizes in DNG/ARW/NEF raw-IFD selection (proptest).
load_bytes(2560) CPU time, ARW a7 III 24 MP: 2347 → 240 ms; X-T20: 3120 → 1220 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- embedded_preview: CR3 PRVW/THMB boxes (plus RAF, RW2, ORF, PEF from the
format commits); stops at the first valid box of each kind
- engine: raw variants that report Unsupported are probed and shown from
their embedded JPEG (orientation from the JPEG or the raw), not an error
- crates/raw/tests/corpus.rs: every corpus/raw file is recognised, has a
JPEG preview (DNG optional) and decodes or is a known-unsupported
variant; prints decode timings
- xtask corpus --download: 37 CC0 raw.pixls.us samples (matched by SHA-256)
- ROADMAP/crate docs: format coverage, NEF Huffman / RW2 format 4 /
compressed ORF+RAF gaps, neutral colour-matrix fallback
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
16-bit LE words (12-bit-in-top-bits detected and shifted), 12-bit rows
of LE 32-bit words read MSB-first (black-box), maker-note
ImageProcessing crop, BlackLevel2, WB_RBLevels for old- and new-style
notes, CFA phase from the data (GRBG/RGGB), CameraSettings preview.
Compressed ORF reports Unsupported (documented gap).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Huffman table from maker note 0x0220 (codes/lengths, black-box layout),
T.81 difference categories, same-colour horizontal prediction with
per-row-parity vertical seeds; BlackPoint, WB levels, maker-note image
area or dark-column trimming. Unit round-trip test with our own table.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Black-box derived layout: 0x4000-byte chunks stored rotated by 0x1ff8,
16-byte LE blocks read LSB-first (10x12-bit or 9x14-bit samples).
Sensor borders, CFA code, per-channel black, linearity limit, WB levels,
ISO; JpgFromRaw preview via a generic undefined-tag JPEG candidate.
The quantised raw format 4 reports Unsupported (documented gap).
Adds a rawdump debug example.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RAF header/record parsing, raw IFD (0xf000) with 16-bit, 12-bit LSB and
32-bit-word MSB packings found by black-box analysis of CC0 samples,
X-Trans layout (reversed record, anchored at 0,0), crop, black levels,
WB_GRB levels, metadata and preview from the embedded JPEG. Compressed
RAF reports Unsupported. Colour uses the documented neutral fallback.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
16-bit word and 12-bit MSB-packed uncompressed NEF, maker-note black
levels and WB_RBLevels, masked trailing column trim, robust white
estimate ignoring 0xFFFF padding. Huffman-compressed NEF is reported as
unsupported (no clean-room description available).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ARW2 implemented from Dietz, EI 2016 (128-bit blocks, interleaved 16-pixel
sets, 7-bit scaled deltas, 5-segment tone curve from 0x7010); black/white/
WB/CFA/crop from the raw IFD's documented Sony tags. Verified visually on
A7 III / A7 IV samples; ~500-850 MP/s row-parallel decode.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IFD#3 lossless JPEG with 0xc640 slice reassembly, SensorInfo borders,
masked-column black levels per CFA position, saturation-plateau white,
as-shot WB from ColorBalance. Verified bit-identical to Adobe's DNG of
the same 5D Mark III frame. sRAW/mRAW reported as unsupported.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Classic + BigTIFF, both byte orders, all field types, SubIFDs and Exif/GPS/
Interop pointer IFDs, strips/tiles layout, maker-note offset-base helpers,
IFD writer; hardened against malformed input (proptests). Placeholder
manifests for lightcraft-meta and lightcraft-raw.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brandon Thomas
Christian Bretz