lightcraft/docs/gpu-pipeline.md

GPU develop pipeline (M5.2)

The CPU pipeline (crates/pipeline) is the reference ("oracle"). lightcraft-gpu (crates/gpu, L3) evaluates the same stages with wgpu compute shaders (WGSL) on Metal / Vulkan / DX12. Everything is pure Rust (wgpu, naga); the drivers are the system's. No GL backend is compiled in.

Where it is used

• lightcraft_engine::media::develop (called by every RenderJob): loupe / before / compare views, render_now (CLI, MCP, control channel renders) and exports render on the GPU when one is available; grid/filmstrip thumbnails (many small jobs in parallel) stay on the CPU.
• Anything the GPU path cannot do returns None and the CPU renders instead: no adapter (CI machines, software-only adapters), LIGHTCRAFT_GPU=0 (whole process), the app.gpu {enabled} command (runtime preference; ui.inspect → perf.gpu shows the adapter), an output larger than the device's storage-buffer limit, or a device error / panic during a render (the process then stays on the CPU).
• wasm32: the crate compiles to the CPU fallback (WebGPU needs asynchronous device creation and a device shared with the canvas — open item below).

Structure

• Shared parameters, so the two implementations cannot drift apart: pipeline::plan (effective settings, frame, output size, stage-cache keys), Frame::sample_plan, local::{wb_matrix_for, nr_params, plane_sigmas, guided_fast_step, airlight_of}, finish::{FinishParams, mask_terms}, masks::brush_dabs; exact tables (tone LUT, sRGB LUT, curve LUTs, resample taps) and the OkLab matrices are uploaded / generated into the WGSL prelude from the CPU values.
• Buffers, not textures: images are array<f32> with the CPU's interleaved layout (RGB, 1–3 channels), so upload / readback are plain copies of Rgb32f / Plane / Rgba8 data.
• Kernels (crates/gpu/src/wgsl/): geom (orientation pixel map, bilinear through the crop/straighten/flip affine or the full lens + perspective warp), resize (separable resample with the CPU's taps; Mitchell prefilter, box/bilinear for the fast guided filter), blur (box passes with running sums over pixel chunks; three each way = the CPU's Gaussian), map (log luminance, dark channel, guided-filter steps, white balance, luminance / colour NR, airlight sampling), mask (linear / radial / luminance / colour range / brush shapes, combine, finalize), finish (the whole per-pixel stage) — each mirrors a named CPU function.
• Per-stage hybrid: defringe and spot removal (rare, CPU-only for now) download the resampled image, run on the CPU and upload; heuristic mask shapes (Sky, Subject, Background, …) and brushes of more than 4096 dabs are evaluated on the CPU and uploaded; sources over the buffer limit are resampled on the CPU.
• Stage cache: GpuStages mirrors StageCache (same keys: source identity + geo, lin_key, per-plane radii) and lives in it as an extension, so the UI needs no change. The uploaded source is kept per view. A tone / colour / exposure drag re-runs one dispatch of finish plus the readback.
• One command encoder per render; the only mid-render readback is the airlight samples (dehaze, when the dark channel is recomputed). Dropped buffers are recycled (exact size, ≤ 2 GB pool) once the thread that dropped them has submitted — allocating and zero-filling fresh 100–300 MB buffers per pass cost as much as the passes.
• LIGHTCRAFT_PROFILE=1 prints GPU stage timings (each stage is then submitted and waited for).

Correctness: CPU oracle and equivalence tests

crates/gpu/tests/equivalence.rs renders the same settings on both and compares 8-bit sRGB: bounds mean |Δ| < 0.5 LSB and max |Δ| ≤ 3 LSB per channel. Measured (Apple M4 Pro, Metal): max 1 LSB, mean ≤ 0.0003 LSB for every case — 26 settings cases (all tone/colour/effects tools, 3 vignette styles, grain, curves, grading, B&W, WB, NR, crop/straighten/flip, lens + perspective, three mask sets incl. CPU-evaluated shapes, spots + defringe), all 8 orientations ± crop, embedded DNG lens data, display-referred sources, draft quality, full-size renders; the 24 MP export in render_bench also differs by max 1 LSB. Cached (slider-drag) GPU renders are bit-identical to fresh ones. The tests skip (pass with a note) when no adapter exists.

Remaining differences come from f32 vs f64 coordinate math, fast-math transcendental functions on Metal and running-sum order in the box filters — all far below one 8-bit step.

Measurements

render_bench on corpus/raw/arw-sony-a7m3-compressed.arw (24 MP; loupe = 2560 px preview → 1920×1280), min of 7 runs, wall clock, on a heavily shared machine (load average 60–110 on 14 cores, so CPU numbers are pessimistic; the GPU numbers are less affected):

scenario	CPU	GPU
loupe 1920×1280 cold (new photo, incl. upload)	415 ms	32 ms
loupe draft 1152×768 cold	169 ms	21 ms
exposure drag 1920×1280 (warm)	22–42 ms	3.9 ms
highlights drag, draft (warm)	14 ms	3.2 ms
clarity drag, draft (warm)	16–23 ms	3.0 ms
NR drag, draft / 1920×1280 (warm)	186 / 301–471 ms	10 / 18 ms
per-pixel stage 1920×1280, + vibrance/saturation	53–65 ms	5 ms
export render 6000×4000	1037–1414 ms	291–330 ms

GPU timings include the readback of the 8-bit result and the histogram. Device creation + kernel compilation: ~0.4 s once per process — the desktop app starts it on a background thread at launch (lightcraft_gpu::warm_up), so the first loupe render doesn't wait for it; other processes create the device on their first GPU render. lightcraft_gpu::ready() asks without blocking.

Opening a photo (M5.4)

• The loupe shows a stand-in at once (media::QuickJob): the photo's cached view render for its current settings, else (raws with their import look) the embedded camera JPEG, else a thumbnail-level render; the full render replaces it in place.
• Raw sources for previews (2560 px) and thumbnails are binned straight from the mosaic (RawImage::develop_binned, 2× for a 24 MP preview); only exports / 1:1 demosaic at full size.
• The next and previous photos in filmstrip order are prepared in the background once the current one is rendered (decoded source kept, view render cached): stepping through photos shows the developed image in ~50 ms.

Memory (M5.6)

• library.memory reports what the engine's caches hold (decoded thumbnail / preview / full-size sources, rendered previews) and the GPU renderer's device buffers (allocated, of which pooled and retired); ui.inspect → memory adds the loupe's stage caches (CPU images, GPU buffers) and the textures.
• Heap profile: build lightcraft-cli with --features dhat-heap; library.memory then also reports live/peak heap bytes and the run writes dhat-heap.json (LIGHTCRAFT_DHAT_FILE), whose allocation sites at the peak (t-gmax) show who holds the memory.
• Measuring the scenario (import 14 raws from corpus/raw, open the loupe, step 12 times): /usr/bin/time -l lightcraft-cli snapshot <files> --script steps.jsonl -o out.png → "maximum resident set size" and "peak memory footprint". Run it several times: the high-water mark is noisy (allocator caching, scheduling). On Apple silicon GPU buffers count in the footprint.
• Imports read raw headers only (lightcraft_raw::probe_info): no pixel data is decompressed.
• One budget (memory::budget, default min(25 % of RAM, 1.5 GiB); LIGHTCRAFT_MEMORY_MB or app.memoryBudget {mb}): half for the engine caches (decoded thumbnail / preview / full sources and rendered previews, evicted least recently used across them; the photo on screen stays), a quarter for decodes in flight (memory::work_gate: grid thumbnails, neighbour prefetch and import probes wait for room, loupe and export loads never wait), an eighth for the GPU buffer pool (reuse by best fit within 25 %; freed after 3 s idle).
• Decodes free the file's bytes and the raw samples as soon as they are consumed and colour / crop / orient in place. macOS keeps freed large blocks resident ("reusable") until the system is short of memory, so the resident size climbed with every photo decoded; the apps install malloc_zone_pressure_relief as memory::set_release_hook, called after each decode, import and idle trim.
• Result (M4 Pro, 14 raws, loupe, 12 steps): peak RSS 1.9–2.3 GB → 0.83–0.99 GB, peak footprint 2.5–2.6 GB → 0.91–0.96 GB, heap peak 930 → ~600 MB; stepping median ~13 ms (unchanged).

Open items

• Keep the loupe texture on the GPU (render into an egui-wgpu texture on eframe's device instead of reading back); share the device with eframe.
• WebGPU in the browser build (async init, single device per canvas).
• Defringe and spot removal kernels; Sky/Subject heuristics (replaced by the segmenter in M12).
• Tiling for sources / outputs beyond the buffer limit (60 MP+ on smaller adapters): tile with overlap equal to the largest filter radius.
• Large-radius blurs at full size (dehaze dark channel at 24 MP) dominate the export: a summed-area table or a downsampled dark channel would cut them further.
• GPU histogram (atomics) to skip the CPU pass over the readback.