65 lines
4.5 KiB
Markdown
65 lines
4.5 KiB
Markdown
# Photo Merge (HDR, Panorama, HDR Panorama)
|
||
|
||
`crates/merge` (layer L3, no UI dependencies) implements the algorithms; `crates/engine/src/merge.rs`
|
||
plans, runs and finishes merges in the library; the desktop UI shows a dialog with a live preview.
|
||
All algorithms are our own implementations from published papers (cited in the module docs).
|
||
|
||
## Using it
|
||
|
||
- **App:** select the photos, then *Photo Merge ▸ HDR…* (`Ctrl+H`), *Panorama…* (`Ctrl+M`) or
|
||
*HDR Panorama…* from the photo context menu (or the `dialog.merge*` commands). The dialog previews
|
||
the merge at ≤ 1024 px and re-runs the preview when an option changes; *Merge* runs the full merge
|
||
in the background (a row in the activity stack shows its progress, with ✕ to stop it). The result is written next to the first photo as
|
||
`<name>-HDR.dng`, `-Pano.dng` or `-HDR-Pano.dng`, imported and selected.
|
||
- **Commands** (CLI, MCP, control channel):
|
||
- `merge.hdr {ids?, align=true, deghost=none|low|medium|high, autoSettings=true, stack=false, preview=false, showOverlay=false, previewPath?}`
|
||
- `merge.panorama {ids?, projection=auto|spherical|cylindrical|perspective, boundaryWarp=0..100, autoCrop=false, fillEdges=false, autoSettings=true, maxMegapixels=40, preview, previewPath?}`
|
||
- `merge.hdrPanorama {ids?, bracket=0 (from EXIF), …both sets of options}`
|
||
- **CLI:** `lightcraft-cli merge hdr|panorama|hdr-panorama [options] FILES…`;
|
||
`lightcraft-cli synth-merge hdr|panorama -o DIR` writes synthetic inputs (procedural scene).
|
||
|
||
## What it does
|
||
|
||
**HDR** (`hdr.rs`): frames ordered by EXIF exposure (or median luminance); the middle one is the
|
||
reference. Auto Align: features on exposure-normalised luminance, RANSAC translation / similarity /
|
||
homography (the simplest model that explains ≥ 97 % of the matches). Exposure ratios are measured
|
||
(median log ratio of well-exposed pixels between brightness neighbours), EXIF is the fallback.
|
||
Radiance is a weighted mean in linear light, weight `k · (1 − smoothstep(0.8·clip, 0.96·clip, max
|
||
channel))`. Deghost: a per-pixel consistency test against the reference radiance (with a noise
|
||
floor), dilated and feathered masks; the union is the deghost overlay. Output: 16-bit float DNG.
|
||
|
||
**Panorama** (`pano/`): MOPS-style features, pairwise RANSAC homographies with Brown & Lowe's
|
||
verification, focal from homographies (or EXIF 35 mm), rotations chained along a maximum spanning
|
||
tree, Levenberg–Marquardt bundle adjustment of rotations + focal lengths, straightening.
|
||
Projections: spherical, cylindrical, perspective; *Auto* picks perspective for compact fields of
|
||
view < 110°, cylindrical when the vertical field is < 90°, else spherical. Exposure: EXIF, then
|
||
gain compensation from the overlaps. Seams: each pixel goes to the image with the highest
|
||
centre-weighted feather weight; multi-band blending (Laplacian pyramids) in log space. Boundary
|
||
Warp: a separable stretch of the smoothed valid-area edges to the canvas edges. Auto Crop: the
|
||
largest rectangle of image data (set as the develop crop, so it stays adjustable). Fill Edges:
|
||
push–pull diffusion. Output: 16-bit integer DNG (16-bit float for HDR panoramas).
|
||
|
||
**DNG output** (`output.rs`): LinearRaw, 3 samples, Deflate tiles; values scaled below 1.0 with the
|
||
scale in `BaselineExposure`; raw sources keep their DNG colour tags (camera RGB), other sources get
|
||
`ColorMatrix1`/`ForwardMatrix1` of their RGB space.
|
||
|
||
## Measured (synthetic tests, `cargo test -p lightcraft-merge`)
|
||
|
||
- HDR, 3 brackets ±2 EV with 14-bit noise and handheld shifts: alignment error < 0.3 px, exposure
|
||
error < 0.02 EV, radiance median log error < 0.03 (noise-limited, bias < 0.01), recovered
|
||
highlights < 0.05; moving object: ghost error 0.2–2.0 without deghosting, < 0.06 with High.
|
||
- Panorama, 5 views over 182° with a 1.3× exposure drift: relative rotations < 0.1°, focal < 1 %,
|
||
BA RMS ≈ 0.35 px; median log error vs ground truth < 0.02; no seam steps.
|
||
|
||
## Limits (known gaps)
|
||
|
||
- No lens-distortion model in the panorama bundle adjustment (rectilinear lenses assumed); no
|
||
360° wrap-around; flat (translating-camera) scans are fitted with the rotation model.
|
||
- No camera response calibration for JPEG brackets (sRGB decoding is taken as linear light).
|
||
- Boundary Warp is a separable stretch, not a content-preserving mesh warp; Fill Edges is diffusion,
|
||
not patch synthesis.
|
||
- *Create Stack*: the result goes on top of a collapsed stack of its sources (`Catalog::stack_with_ops`); its
|
||
caption also lists them.
|
||
- Panorama output is limited to `maxMegapixels` (default 40) — blending memory is ~25 bytes/pixel.
|
||
- The browser build has no file access for merges yet.
|