`lightcraft_raw::sparse::header_plan` says which byte ranges of a raw file the header
probe (`probe_info`, `lightcraft_meta::extract`) reads, given a buffer of the file's length
in which only some ranges are loaded: everything but the bulk sample data (large TIFF
strips/tiles minus the first chunk's coding headers, a CR3's image tracks minus the raw
track's CRX headers and QP maps, a RAF's CFA strip, an RW2's raw data). `preview_plan`
also reads what `embedded_preview` needs, for raws shown from their preview. NEF, ORF and
PEF (their header probe looks at samples) and anything else are read whole.
The corpus test checks, for every raw.pixls.us sample the plan describes, that probing the
partly loaded buffer gives exactly what the whole file gives, with the skipped bytes zero and
again 0xFF: CR3 2.7 %, DNG 2.4 %, CR2 3 %, RAF 6 %, RW2 7 %, ARW 8 % of their bytes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HsvTable::apply took the value through linear -> sRGB -> linear even when the table's value
scale is 1 (every hue/saturation-only table, as fitted camera looks are), and wrapped hues with
rem_euclid (an fmodf call) three times per pixel. The value is now kept as it is for a unit
scale and the sRGB-encoded value is reused otherwise; hues are wrapped by a helper that equals
rem_euclid bit for bit (tested) without the call for values within one period.
2048 px render of a 24 MP ARW, single thread: 16.4 -> 15.1 G instructions, 4.94 -> 4.76 G cycles.
Sources: own design.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The CRX 0x200 QP map opens with its first symbol as a 32-bit literal; every
later symbol is Rice coded with a limited-length escape: 23 zeros, the
terminating 1 and the symbol in 8 bits (the ITU-T T.87 layout for a 32-bit
limit and 8-bit symbols, without its minus-one offset). The old rule (an
escape after 16 zeros with a 16-bit literal) matched only the first symbol,
which is all the R8 and R7 maps need. R5 Mark II maps (QP 150-162) jump by up
to 12 where the Rice parameter is 0, giving 18- to 22-zero prefixes that
misparsed ("Canon CRX adaptive QP value -21087"): 174 of 306 sampled files
from a private R5 Mark II library opened as previews. Every QP map of its
33,665 C-RAW files now decodes to the exact end of its entropy stream; the six
oracle fixtures are unchanged. Established by black-box analysis of the
bitstream; no third-party decoder source consulted.
Adds the four CC0 R5 Mark II samples (raw.pixls.us 7881-7884) to the corpus
with pinned sensor hashes (the crop-mode framing is left to #754), bumps
RENDER_CACHE_VERSION to 43, and updates cr3.md, raw-parity (235 verified
models), coverage, gaps, the checklist rows, ROADMAP, target-app-parity and
What's New.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Sony body shot in Adobe RGB writes ColorSpace Uncalibrated with InteropIndex R03 in the raw's EXIF and leaves the
embedded JPEG untagged, so the look was fitted against the JPEG decoded as sRGB. The EXIF fallback now applies to
all makes (the Nikon maker note still wins); DNGs are skipped, since their previews come from the converter.
Canon EOS R50 C-RAW files reach QP 168 (quant 44, step 102, measured about 100). The quantization step and the adaptive QP map range accept it; anything above stays rejected. Bumps RENDER_CACHE_VERSION since such files change from the embedded preview to a raw decode.
The 2668-entry table read off the black-box oracle's output is gone.
arw6_curve.rs now builds the 4096-entry table at first use from our own
measurement: piecewise linear through knots u_k = 128 k, c_k = u_k for
k <= 11 and 356.89 + 655.646 ln(k - 6) above, rounded half up, fitted to
the 7 same-exposure lossless (LJ92) / ARW6 pairs of the CC0 ILCE-7RM6 set.
A unit test pins the whole table (sum 27776336, weighted 92792980288),
its anchors, the identity through 1408 and strict monotonicity.
The corpus sums become regression values of the new output; the 12-bit
codes before the curve stay verified against the oracle (Phase 0).
docs/arw6-compression.md describes the curve, how it was measured, its
uncertainty, the structure and x2-scale evidence, why it is not in the
file, and the after-the-fact comparison with the oracle's curve; the
tracker and docs no longer call the output bit-exact against LibRaw.
Co-Authored-By: Claude <noreply@anthropic.com>
docs/superpowers/ held the working spec and plan, which pointed at the
local plan/ folder. The ARW6 module docs now cite the sections of
docs/arw6-compression.md, which gains the facts they relied on from the
spec: the NP field of word 0, the stream header word layout and how the
first half's residual lines are counted.
Co-Authored-By: Claude <noreply@anthropic.com>
`cargo xtask corpus --download` now fetches one full-frame Compressed and
one APS-C Compressed HQ file (about 94 MB) instead of all 12 (about
630 MB); the corpus test checks those two and the lossless-sibling
regression test is gone, since nothing would download those files.
docs/arw6-compression.md keeps the full table with source paths and
SHA-256, says which two xtask fetches and how to fetch the rest with
`curl -L`, and records the CC0 1.0 statement of RawDB's set record
(accessed 2026-10-10).
Co-Authored-By: Claude <noreply@anthropic.com>
The RawDB entries in xtask carry their SHA-256 as the third field, which
`cargo xtask corpus --download` checks; the corpus test keeps only the
sensor sum checks.
Co-Authored-By: Claude <noreply@anthropic.com>
Since #281, probe takes a Sony file as a raw only with raw evidence, and
two layouts have none of the kinds has_raw_ifd looks for: SRF (DSC-F828,
an RGB-looking IFD0 of 14-bit samples) and the DSLR-A100 (raw data
addressed by SubIFDs). Both were "not a readable photo". For Sony, IFD0
samples of 12 or 14 bits, or model DSLR-A100 with an old-style JPEG IFD0
(compression 6) and SubIFDs, now count too; an 8- or 16-bit camera TIFF,
or an exported A100 TIFF with a SubIFDs thumbnail, stays an image. The public F828 and
A100 samples open through their previews as on main 5436951; 133 other Sony files unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The R5 Mark II's APS-C crop-mode files (5376 x 3574) state an IAD1 active area of x 132..5243,
but columns 132..~256 sit at the black level and the image runs to the right edge (measured on
raw.pixls.us 7883/7884), while their recommended crop, x 272..5359 (5088 x 3392, the camera JPEG's
size), lies on the image. The recommended crop didn't fit inside the stated active area, so the
whole active area was kept: a black band down the left and ~130 image columns lost.
When the recommended crop doesn't fit inside the stated active area, it is now the valid area.
On 7883 the render's high-pass correlation with the camera JPEG goes from -0.13 to 0.99. The
active area and crop of the other 108 decodable CC0 CR3s on raw.pixls.us are unchanged.
RENDER_CACHE_VERSION 49.
Raws without matrices in the file (ARW, NEF, RW2, RAF, CR2, CR3, PEF, SRW,
ORF) were all edited on the relative white-balance scale (6500 K / 0 = as
shot), even when lightcraft_raw::spectral has measured matrices for the
camera. Those 52 bodies (ILCE-7M3, ILCE-7M4, D850, EOS 5D Mark III...) now
edit in Kelvin like a DNG and like Lightroom.
Design:
- One decision on both sides: spectral::find_camera("Make Model") backs
catalog::relative_wb_camera (Photo::relative_wb); files::WbScaleOf decides
the same from the decoded file. A corpus test asserts they agree.
Photo::fitted_look keeps the old predicate for smart previews.
- The matrices only convert: As Shot is the file's multipliers read through
the model's ColorMatrix1/2 (spectral::as_shot_white_through). The look
(JPEG fit / profile / spectral fallback) is unchanged, and the slider
adapts from that white with Bradford. The camera-space rebalance stays
reserved for files with their own matrices.
- Presets resolve to their Kelvin values (WbMode::preset, sources cited).
- The Color panel shows the white balance in effect (effective_wb).
- XMP: crs::Target::for_file takes the camera; matrix cameras read
crs:Temperature as Kelvin (marked "scale": "kelvin"), others keep the
#510 as-shot shift.
Migration (no look changes):
- WhiteBalance.scale: WbScale { Legacy (absent in JSON), Kelvin }. A
white balance without a scale keeps its old meaning. On a spectral-Kelvin
raw (SourceInfo.legacy_relative_wb) the pipeline composes the old
adaptation (6500 -> value) with the new one (as shot -> ...) in Bradford
cone space (local::legacy_white). Von Kries scalings compose exactly, so
the matrix matches to f32 and the pixels are identical
(pipeline legacy_relative_white_balance_renders_as_before_on_a_kelvin_raw,
media old_relative_white_balance_on_a_kelvin_camera_keeps_its_look...).
- Every white-balance writer (develop.set/adjust/quickAdjust/wb/wbPick/
reset) first runs local::normalize_wb: a legacy value becomes the Custom
Kelvin value with the same look and is marked Kelvin. Partials without a
scale (old presets, develop.merge) stay legacy.
- Catalog format 7: Op::SetAsShotWb replaces the 6500/0 as-shot recorded
by older imports when the file is next decoded or reloaded (system op,
not an undo step). Format-version test updated.
- The web wire carries legacy_relative_wb.
Docs: parity (LR-EDIT-COLOR-TEMP), camera-preview-colour, xmp-interop,
lightroom-catalog-import, control-protocol, mcp, whats-new.
white_from_data anchored the plateau search on the absolute maximum, so a
handful of stray samples above saturation (Canon EOS 70D CR2: 83 of 20.6M,
up to 14426 vs. a plateau at 13583) made it fall back to 2^bits-1. The
image rendered ~0.3 EV dark and clipped highlights were never seen as
clipped. Skip up to half a plateau's worth of strays when locating the top.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The D4 records no CropArea. Its 4992 x 3292 data keeps 2 columns of optical
black left of the image and 42 right of it (then 8 empty ones), which showed
as purple stripes on both sides; the camera's own 4928 x 3280 JPEG sits at
(8, 6). Green saturates at 15546-15672 while red and blue run on to 16383,
so the data's own top left clipped highlights magenta; one level of 15520
keeps them white without moving the white balance.
Measured on 148 uncompressed 14-bit NEFs from one body by aligning the
sensor data with the embedded camera JPEG.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Fujifilm RAFs carry per-shot GeometricDistortionParams (0xf00b) and VignettingParams (0xf010)
in the raw IFD. Derived by measurement against the cameras' own JPEGs and LightCraft's
uncorrected renders: [scale, n knots, n values], scale x n = half diagonal of the recorded image,
last knot = farthest corner, distortion in percent indexed by the source radius with the
frame-filling zoom, vignetting as 100/V gain. They become a WarpRectilinear and a
FixVignetteRadial for 59 validated bodies (same path as the Sony ARW and Panasonic RW2
corrections); unmatched bodies and tables with another scale stay uncorrected.
On 67 body/lens combinations the corrected render's radial shape error against the camera JPEG
falls from a median of 0.72 % to 0.05 % of the half diagonal. The lateral CA table (0xf00f) is
real but is not applied: our demosaic already registers the colour planes and the stored shift
over-corrects. See docs/fuji-lens-corrections.md.
RENDER_CACHE_VERSION 48.
`probe_info` (every file an import adds) ran the whole Nikon Huffman
decode on a compressed NEF because `nef::decode` had no header mode and
the masked-trailing-column test wanted the samples: 386 ms per 24 MP
file, the full decode time, where every other format probes in < 1 ms.
The header has no source for those columns (the D5100 and D7000 write 8
masked columns at value 0, compressed and uncompressed alike, with no
CropArea or other maker-note tag), so the probe cannot skip the test.
Instead both modes now judge the active width from the first 128 rows
(`nef::MASK_ROWS`) against the nominal white level: the header probe
decodes only those rows (`nefc::decode_rows`, the stream is sequential;
uncompressed files read only the strips holding them) and the full
decode uses the same rule, so `probe_info == RawImage::info` still
holds. On every corpus NEF/NRW the answer is the same for 8 to 512 rows
as for the whole frame. The whole-image size and truncation checks still
apply in both modes; the "lossy after split" end-of-strip rule is only
checked when the whole strip is decoded.
Release, same files: probe 66 -> 3.4 ms (D5100 lossless), 72 -> 3.2 ms
(D7000 lossy 12), 88 -> 3.1 ms (D7500 lossless 12), 92 -> 3.7 ms (Z f
lossless), 23 -> 1.5 ms (D5100 uncompressed), 54 -> 2.7 ms (B700 NRW).
Tests: a synthetic lossless NEF whose stream is cut off after two
thirds of the rows probes to the same info as the whole file while the
full decode reports the truncation; synthetic uncompressed NEFs with
masked columns in the top rows, lower rows, all or none give the same
active area in both modes. Both failed before the fix. Docs: the
`probe_info` doc comment, docs/parity.md (LR-IMP-FORMATS),
docs/gpu-pipeline.md.
Issue #697 (ILCE-7CM2 lossless compressed M-size ARW "washed out" in 0.4
and 0.5) is the 512-vs-1024 YCbCr pedestal of #535 item 3, fixed on main
by 3100cc1a after v0.5.0. Measured on the reporter's file: 0.5.0 renders
the darkest half of the dusk scene +11 L* and blue against the camera
JPEG, with the lamplight turned white (mean dE 11.8); main is within
+3 L* (dE 8.9). Its darkest 0.1 % per channel sits at 1017/1020/1024,
confirming the 1024 pedestal on a fifth body.
- Corpus: two CC0 raw.pixls.us ILCE-7CM2 files, lossless L and the
compressed "M-size" coding, which is Bayer at 4736x3132 with black 512
(only the lossless M/S codings are YCbCr), so a size-based pedestal
rule would break it.
- Tests: corpus_sony_a7cm2_codings (model, layout, levels, crop, as-shot
gains, header/full agreement) and both files in the #535 data-floor
test.
- docs/parity.md: the ILCE-7CM2 verification on the Sony coverage row.
Claude
ingolfteetz
Andrew Frahn
radialmonster
dev-null-undefined
Alex Chambers
Jason Baldwin
TicT4x
Unemars
Jason Baldwin