M3.2: reject stale selections and disclose approximate masks

Tie background results to their rendered source, borrow cached mattes for the
approximation notice, and cover provenance and matte access with regressions.

Co-authored-by: Codex <noreply@openai.com>
(cherry picked from commit 9922a180f57404bfb90fcb768dcea7dd129d145a)
This commit is contained in:
grlessa 2026-10-11 15:25:29 -03:00 committed by bflatastic
parent 1d02ae775a
commit aedd3cbbb8
9 changed files with 202 additions and 13 deletions

View File

@ -553,9 +553,8 @@ impl MediaCache {
}
}
/// The facts of photo `id`'s cached sources: the decoder's when one has them, else `header`
/// (with a smart preview's stored camera tone curve).
fn source_facts(&self, id: PhotoId, header: SourceInfo) -> SourceInfo {
/// Prefer a cached source with decoder facts, preserving thumbnail/preview/full priority.
fn cached_source(&self, id: PhotoId) -> Option<&DecodedSource> {
let cached = || {
self.thumbs
.peek(&id)
@ -563,7 +562,12 @@ impl MediaCache {
.chain(self.previews.iter().filter(|e| e.0 == id).map(|e| &e.1))
.chain(self.full.iter().filter(|e| e.0 == id).map(|e| &e.1))
};
match cached().find(|s| s.info.is_some()).or_else(|| cached().next()) {
cached().find(|s| s.info.is_some()).or_else(|| cached().next())
}
/// The decoder's cached facts, else `header` (with a smart preview's stored camera tone curve).
fn source_facts(&self, id: PhotoId, header: SourceInfo) -> SourceInfo {
match self.cached_source(id) {
Some(s) => s.info_or(header),
None => header,
}
@ -1545,6 +1549,11 @@ impl crate::Session {
self.media.source_facts(id, header)
}
/// Borrow source mattes for selection notices without constructing render facts each frame.
pub fn source_mattes(&self, id: PhotoId) -> Option<&lightcraft_pipeline::masks::Mattes> {
self.media.cached_source(id).and_then(|s| s.info.as_ref()).and_then(|info| info.mattes.as_deref())
}
/// The source proxy for pixel-statistics commands (auto tone/WB), loading synchronously.
pub fn source_now(&mut self, id: PhotoId, level: SourceLevel) -> Result<Arc<Rgb32f>, String> {
let p = self.catalog.photo(id).ok_or("no such photo")?.clone();
@ -1709,6 +1718,24 @@ mod thumbnail_hash_tests {
#[cfg(test)]
mod tests {
#[test]
fn selection_notices_borrow_the_same_mattes_as_render_facts() {
let mut session = crate::Session::with_demo();
let id = session.active().unwrap();
assert!(session.source_mattes(id).is_none());
let mattes = std::sync::Arc::new(lightcraft_pipeline::masks::Mattes::default());
session.media.insert_source(
id,
super::SourceLevel::Thumb,
super::DecodedSource::new(
std::sync::Arc::new(lightcraft_raster::Rgb32f::new(8, 8)),
Some(lightcraft_pipeline::SourceInfo { mattes: Some(mattes.clone()), ..Default::default() }),
),
);
assert!(std::ptr::eq(session.source_mattes(id).unwrap(), mattes.as_ref()));
assert!(std::sync::Arc::ptr_eq(session.source_info(id).mattes.as_ref().unwrap(), &mattes));
}
use super::*;
#[test]

View File

@ -363,8 +363,14 @@ impl Session {
let mut d = (*p.develop).clone();
d.masks.clear();
let content = crate::media::content_key(p);
// Reloaded lens data and camera-profile changes also change the model's rendered input.
let lens_key = serde_json::to_string(&p.embedded_lens).unwrap_or_default();
let mut h = mix(0xcbf2_9ce4_8422_2325, d.hash64());
h = mix(h, id.0);
h = mix(h, crate::camera_profiles::cache_key());
for b in lens_key.bytes() {
h = mix(h, u64::from(b));
}
for b in content.bytes() {
h = mix(h, u64::from(b));
}
@ -411,7 +417,7 @@ impl Session {
job
}
};
Ok(worker::Job { dir, key: if region.is_some() { 0 } else { key }, render: Some(render), missing, kind, tag, reply })
Ok(worker::Job { dir, key: if region.is_some() { 0 } else { key }, source_key: key, render: Some(render), missing, kind, tag, reply })
}
/// Send a request and wait for its result (CLI, MCP, tests; or when not in background mode).
@ -560,7 +566,8 @@ impl Session {
self.segment_queue(id, kind, tag, Some(region))?;
} else {
let patch = self.segment_wait(id, kind, Some(region))?;
let outcome = worker::Outcome { tag, result: Ok(patch), superseded: false };
let source_key = self.segment_key(id).ok_or("no such photo")?.0;
let outcome = worker::Outcome { tag, source_key, result: Ok(patch), superseded: false };
let mut polled = Polled::default();
self.apply_outcome(outcome, &mut polled);
}
@ -599,6 +606,22 @@ impl Session {
if o.superseded {
return;
}
let photo = match &o.tag {
Tag::Clicks { photo, .. } | Tag::NewPrompt { photo, .. } | Tag::Detail { photo, .. } => Some(*photo),
_ => None,
};
if let Some(photo) = photo
&& self.segment_key(photo).map(|(key, _)| key) != Some(o.source_key)
{
if let Tag::Clicks { photo, mask, comp, hint, exclude } = &o.tag {
let pending = PendingClicks { photo: *photo, mask: *mask, comp: *comp, hint: hint.clone(), exclude: exclude.clone() };
if self.segmenter.pending.as_ref() == Some(&pending) {
self.segmenter.pending = None;
}
}
polled.messages.push("The photo changed while the selection was being computed; try again.".into());
return;
}
match o.tag {
Tag::Wait => {}
Tag::Prepare => {
@ -698,6 +721,71 @@ mod tests {
SegMask::from_logits(side, &l)
}
#[cfg(feature = "sam")]
#[test]
fn stale_prompt_result_does_not_add_a_mask_after_geometry_changes() {
let mut s = Session::with_demo();
let photo = s.active().unwrap();
let source_key = s.segment_key(photo).unwrap().0;
let outcome = worker::Outcome {
source_key,
tag: Tag::NewPrompt { photo, mask: None, op: "add".into(), name: None, text: "sky".into() },
result: Ok(Some(seg_box(16, 0, 0, 16, 8))),
superseded: false,
};
let mut d = (*s.develop_of(photo).unwrap()).clone();
d.geometry.horizontal = 30.0;
s.set_develop(photo, d, "Perspective").unwrap();
let mut polled = Polled::default();
s.apply_outcome(outcome, &mut polled);
assert!(s.develop_of(photo).unwrap().masks.is_empty(), "a mask computed in the old photo frame must not be applied");
assert!(!polled.changed);
assert!(polled.messages.iter().any(|m| m.contains("changed")));
}
#[cfg(feature = "sam")]
#[test]
fn current_prompt_result_is_applied_without_inference_or_model_weights() {
let mut s = Session::with_demo();
let photo = s.active().unwrap();
let source_key = s.segment_key(photo).unwrap().0;
let outcome = worker::Outcome {
source_key,
tag: Tag::NewPrompt { photo, mask: None, op: "add".into(), name: None, text: "sky".into() },
result: Ok(Some(seg_box(16, 0, 0, 16, 8))),
superseded: false,
};
let mut polled = Polled::default();
s.apply_outcome(outcome, &mut polled);
assert!(polled.changed && polled.messages.is_empty());
assert!(matches!(s.develop_of(photo).unwrap().masks[0].components[0].shape, MaskShape::Prompt { seg: Some(_), .. }));
}
#[cfg(feature = "sam")]
#[test]
fn stale_detail_result_does_not_overwrite_an_existing_mask() {
let mut s = Session::with_demo();
let photo = s.active().unwrap();
let seg = seg_box(16, 4, 4, 6, 6);
let shape = MaskShape::Prompt { text: "flower".into(), seg: Some(seg.clone()), detail: vec![], edge: 0.0 };
s.execute("mask.add", &serde_json::json!({"kind":"prompt","text":"flower","seg":seg})).unwrap();
let source_key = s.segment_key(photo).unwrap().0;
let mask = s.active_mask.unwrap();
let mut d = (*s.develop_of(photo).unwrap()).clone();
d.orientation = lightcraft_geom::Orientation::Rotate90;
s.set_develop(photo, d, "Rotate").unwrap();
let outcome = worker::Outcome {
source_key,
tag: Tag::Detail { photo, mask, comp: 0, shape },
result: Ok(Some(seg_box(16, 5, 5, 7, 7))),
superseded: false,
};
let mut polled = Polled::default();
s.apply_outcome(outcome, &mut polled);
assert!(!polled.changed);
assert!(matches!(&s.develop_of(photo).unwrap().masks[0].components[0].shape,MaskShape::Prompt { detail,.. } if detail.is_empty()));
}
#[test]
fn detail_region_frames_the_selection_with_a_margin() {
// a small object in the middle of a square photo

View File

@ -50,6 +50,8 @@ pub struct Job {
pub dir: PathBuf,
/// Which photo and look the request is about (the encode cache's key).
pub key: u64,
/// Identity of the full photo input, including for a detail job whose encode-cache key is zero.
pub source_key: u64,
/// Renders the model input when the cache doesn't hold `key` (always for a detail pass).
pub render: Option<RenderJob>,
/// The photo's file, when it is missing on disk (for a clearer error if rendering fails).
@ -62,6 +64,7 @@ pub struct Job {
/// A finished request: the segmentation (`None`: nothing selected / nothing to report).
pub struct Outcome {
pub tag: Tag,
pub source_key: u64,
pub result: Result<Option<SegMask>, String>,
/// A newer request for the same selection came in; this one was skipped.
pub superseded: bool,
@ -219,7 +222,7 @@ fn run(rx: &mpsc::Receiver<Job>, shared: &Shared, idle: Duration) {
let done = Done(if is_detail { &shared.detail } else { &shared.pending });
if queue.iter().any(|later| supersedes(later, &job)) {
drop(done);
let _ = job.reply.send(Outcome { tag: job.tag, result: Ok(None), superseded: true });
let _ = job.reply.send(Outcome { tag: job.tag, source_key: job.source_key, result: Ok(None), superseded: true });
continue;
}
let result = match std::panic::catch_unwind(AssertUnwindSafe(|| run_job(&mut state, shared, &mut job))) {
@ -237,7 +240,7 @@ fn run(rx: &mpsc::Receiver<Job>, shared: &Shared, idle: Duration) {
}
shared.loaded.store(state.model.is_some() || state.remote.is_some(), Ordering::SeqCst);
drop(done);
let _ = job.reply.send(Outcome { tag: job.tag, result, superseded: false });
let _ = job.reply.send(Outcome { tag: job.tag, source_key: job.source_key, result, superseded: false });
}
}
@ -405,7 +408,7 @@ mod tests {
fn job(tag: Tag, reply: &mpsc::Sender<Outcome>) -> Job {
// an empty folder: loading fails with an error (no model files)
let dir = std::env::temp_dir().join(format!("lc-sam3-worker-none-{}", std::process::id()));
Job { dir, key: 1, render: None, missing: None, kind: Kind::Prepare, tag, reply: reply.clone() }
Job { dir, key: 1, source_key: 1, render: None, missing: None, kind: Kind::Prepare, tag, reply: reply.clone() }
}
#[test]

View File

@ -93,6 +93,17 @@ impl Mattes {
}
}
/// Whether automatic selection is a classical approximation rather than a stored model result
/// or source matte. Frontends should make this visible instead of implying AI segmentation.
pub fn approximate_selection(shape: &MaskShape, mattes: Option<&Mattes>) -> bool {
match shape {
MaskShape::Background => approximate_selection(&MaskShape::Subject, mattes),
MaskShape::Sky | MaskShape::Subject | MaskShape::People { .. } => mattes.is_none_or(|m| m.for_shape(shape).is_empty()),
MaskShape::Object { hint, seg, detail, .. } => !hint.is_empty() && seg.is_none() && detail.is_empty(),
_ => false,
}
}
pub struct Evaluated {
/// The mask's id ([`Mask::id`]).
pub id: u32,
@ -829,6 +840,21 @@ mod tests {
Frame::new(w, h, &DevelopSettings::default(), true)
}
#[test]
fn selection_provenance_distinguishes_heuristics_mattes_and_model_results() {
assert!(approximate_selection(&MaskShape::Sky, None));
assert!(approximate_selection(&MaskShape::Background, None));
let mut m = Mattes::default();
m.push(MatteKind::Sky, Image::filled(2, 2, 255));
assert!(!approximate_selection(&MaskShape::Sky, Some(&m)));
assert!(approximate_selection(&MaskShape::Subject, Some(&m)));
m.push(MatteKind::Person, Image::filled(2, 2, 255));
assert!(!approximate_selection(&MaskShape::Background, Some(&m)));
let seg = SegMask::from_logits(2, &[8.0, -8.0, 8.0, -8.0]);
let shape = MaskShape::Object { hint: vec![Point::new(0.2, 0.5)], exclude: vec![], seg: Some(seg), detail: vec![], edge: 0.0 };
assert!(!approximate_selection(&shape, None));
}
#[test]
fn linear_gradient_ramps() {
let f = frame(100, 50);

View File

@ -118,6 +118,7 @@
"Auto-Tag from Tracklog": "Etiquetar automaticamente pelo tracklog",
"Auto-Tag from Tracklog…": "Etiquetar automaticamente pelo tracklog…",
"Automatic": "Automático",
"Automatic selection is approximate. Refine with Brush or use Object / Describe.": "A seleção automática é aproximada. Refine com Pincel ou use Objeto / Descrever.",
"Automatically Write Changes into XMP": "Gravar alterações automaticamente no XMP",
"Automatically write changes into XMP sidecars": "Grava automaticamente as alterações nos arquivos XMP auxiliares",
"Autumn Gold": "Ouro de outono",

View File

@ -187,6 +187,21 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui, id: PhotoId) {
});
}
});
let mattes = app.session.source_mattes(id);
let approximate =
d.masks.iter().any(|m| m.visible && m.components.iter().any(|c| lightcraft_pipeline::masks::approximate_selection(&c.shape, mattes)));
if approximate {
egui::Frame::NONE.inner_margin(egui::Margin::symmetric(16, 0)).show(ui, |ui| {
ui.add(
egui::Label::new(
egui::RichText::new(crate::i18n::tr("Automatic selection is approximate. Refine with Brush or use Object / Describe."))
.color(t.text_dim)
.size(11.0),
)
.wrap(),
);
});
}
divider(ui);
// mask list
egui::Frame::NONE.inner_margin(egui::Margin { left: 16, right: 16, top: 8, bottom: 8 }).show(ui, |ui| {

View File

@ -90,3 +90,18 @@ fn masking_long_names_and_describe_controls_stay_inside_the_panel() {
}
}
}
#[test]
fn approximate_selection_notice_is_visible_and_wraps_inside_the_panel() {
let source = "Automatic selection is approximate. Refine with Brush or use Object / Describe.";
for locale in [Locale::En, Locale::De, Locale::PtBr] {
for width in [250.0, 270.0, 520.0] {
for scale in [1.0, 2.0] {
let (texts, _) = panel(width, scale, locale, false);
let notice = texts.iter().find(|t| t.galley.job.text == locale.tr(source)).expect("approximation notice must be visible");
assert!(!notice.galley.elided, "the refinement suggestion must remain readable");
assert!(Rect::from_min_size(pos2(0.0, 0.0), vec2(width, 1600.0)).contains_rect(notice.visual_bounding_rect()));
}
}
}
}

View File

@ -236,3 +236,17 @@ python3 -m venv .venv-sam3 && .venv-sam3/bin/pip install -r tools/requirements-s
.venv-sam3/bin/python tools/sam3_reference.py photo.jpg ref.safetensors
LIGHTCRAFT_SAM3_DIR=<model dir> LIGHTCRAFT_SAM3_REF=ref.safetensors cargo test -p lightcraft-segment --release -- --nocapture
```
## Selection provenance and late results
Sky, Subject, Background and People still use a file's semantic mattes when available,
otherwise classical approximate selection. The Masking panel explicitly identifies that
approximation and suggests Brush refinement or Object / Describe. It is not equivalent to
SAM inference; this label does not improve the heuristic's segmentation quality.
A background Object / Describe / Detail result is now tied to the rendered input that
produced it. Changing the photo's look, geometry, orientation, reloaded lens coefficients,
or camera profile while inference runs causes the old result to be discarded with a retry
notice. An unchanged result is accepted normally. Detail jobs carry the full-photo input
identity separately from their temporary encode-cache key. Regression tests inject stored
segmentations without downloading weights or running inference.

View File

@ -294,10 +294,10 @@ differences and missing bindings are in [ui-parity.md](ui-parity.md#shortcuts).
| Id | Feature | Tier | Status | Evidence | Missing / notes |
|---|---|---|---|---|---|
| LR-MASK-PANEL | Masks panel | P0 | ✅ | `cmd:panel.masking`, `cmd:mask.add`, `cmd:mask.select`, `cmd:mask.rename`, `cmd:mask.duplicate` (`invert`), `cmd:mask.move`, `cmd:mask.visible`, `cmd:mask.delete`, `crates/ui-egui/src/panels/masking.rs` | list with per-mask show/hide eye, double-click rename, right-click menu (duplicate, duplicate and invert, invert, hide, move up/down, rename, delete); fits any panel width (create tiles go to three columns, mask actions wrap, long component names are cut short); no drag-to-reorder |
| LR-MASK-SUBJECT | Select subject | P2 | 🟡 | `cmd:mask.add` (`subject`), `crates/pipeline/src/masks.rs` | saliency heuristic, no segmentation model; a photo that carries person mattes (DNG semantic masks: iPhone ProRAW portrait / skin / hair / teeth / glasses mattes) uses them instead (synthetic files only: no corpus file carries person mattes) |
| LR-MASK-SKY | Select sky | P2 | 🟡 | `cmd:mask.add` (`sky`), `crates/pipeline/src/masks.rs` (`Mattes`), `crates/raw/src/semantic.rs` | heuristic; a photo that carries a sky matte (DNG semantic mask, e.g. iPhone ProRAW) uses it instead, through orientation / crop / lens corrections, preview = export (checked on one CC0 iPhone 12 Pro ProRAW and synthetic files) |
| LR-MASK-BACKGROUND | Select background | P2 | 🟡 | `cmd:mask.add` (`background`) | inverse of the subject mask (heuristic, or the photo's person mattes) |
| LR-MASK-OBJECTS | Object selection | P2 | 🟡 | `cmd:mask.add` (`object`, `prompt`), `cmd:mask.objectPoint`, `cmd:mask.refineDetail`, `cmd:segment.prepare`, `cmd:segment.model.status`, `cmd:segment.model.download`, `cmd:segment.model.cancel`, `crates/segment`, `crates/engine/src/segment/mod.rs` | SAM 3 in pure Rust (candle; Metal on macOS, CPU elsewhere): Object tile → click to include, ⌥-click to leave out; Describe tile → a text prompt selects every instance ("sky", "the red car"); both also as Add/Subtract/Intersect components; + / − under the selected mask; comma lists (`car, road`); a zoomed-in detail pass for 5–10× finer edges on small objects; per-selection Edge (hard ↔ soft). The model runs on its own worker thread (the UI never waits; panics become errors), is unloaded after 10 min idle, and is optional: the segmentation is stored with the mask (288² logits), so renders and exports never need it. Weights are never bundled (SAM License): without them the app offers a consented background download (mirrors, resume, timeouts, SHA-256) — **but no default download location is configured yet** (users set `LIGHTCRAFT_SAM3_MIRRORS` or install by hand: the dialog names the three files, shows the model folder and links docs/ai-masks.md). Not verified against Lightroom's Select Object; no brush/box object mode; the first click on a photo waits for its analysis (~4 s on an M4 Pro, much longer on CPU) |
| LR-MASK-SUBJECT | Select subject | P2 | 🟡 | `cmd:mask.add` (`subject`), `crates/pipeline/src/masks.rs` | saliency heuristic, no segmentation model; a photo that carries person mattes (DNG semantic masks: iPhone ProRAW portrait / skin / hair / teeth / glasses mattes) uses them instead (synthetic files only: no corpus file carries person mattes) approximate selections are identified in the panel; stale segmentation results after edits/profile/lens changes are rejected (`crates/engine/src/segment/mod.rs`, `docs/ai-masks.md`) |
| LR-MASK-SKY | Select sky | P2 | 🟡 | `cmd:mask.add` (`sky`), `crates/pipeline/src/masks.rs` (`Mattes`), `crates/raw/src/semantic.rs` | heuristic; a photo that carries a sky matte (DNG semantic mask, e.g. iPhone ProRAW) uses it instead, through orientation / crop / lens corrections, preview = export (checked on one CC0 iPhone 12 Pro ProRAW and synthetic files) approximate selections are identified in the panel; stale segmentation results after edits/profile/lens changes are rejected (`crates/engine/src/segment/mod.rs`, `docs/ai-masks.md`) |
| LR-MASK-BACKGROUND | Select background | P2 | 🟡 | `cmd:mask.add` (`background`) | inverse of the subject mask (heuristic, or the photo's person mattes) approximate selections are identified in the panel; stale segmentation results after edits/profile/lens changes are rejected (`crates/engine/src/segment/mod.rs`, `docs/ai-masks.md`) |
| LR-MASK-OBJECTS | Object selection | P2 | 🟡 | `cmd:mask.add` (`object`, `prompt`), `cmd:mask.objectPoint`, `cmd:mask.refineDetail`, `cmd:segment.prepare`, `cmd:segment.model.status`, `cmd:segment.model.download`, `cmd:segment.model.cancel`, `crates/segment`, `crates/engine/src/segment/mod.rs` | SAM 3 in pure Rust (candle; Metal on macOS, CPU elsewhere): Object tile → click to include, ⌥-click to leave out; Describe tile → a text prompt selects every instance ("sky", "the red car"); both also as Add/Subtract/Intersect components; + / − under the selected mask; comma lists (`car, road`); a zoomed-in detail pass for 5–10× finer edges on small objects; per-selection Edge (hard ↔ soft). The model runs on its own worker thread (the UI never waits; panics become errors), is unloaded after 10 min idle, and is optional: the segmentation is stored with the mask (288² logits), so renders and exports never need it. Weights are never bundled (SAM License): without them the app offers a consented background download (mirrors, resume, timeouts, SHA-256) — **but no default download location is configured yet** (users set `LIGHTCRAFT_SAM3_MIRRORS` or install by hand: the dialog names the three files, shows the model folder and links docs/ai-masks.md). Not verified against Lightroom's Select Object; no brush/box object mode; the first click on a photo waits for its analysis (~4 s on an M4 Pro, much longer on CPU) approximate selections are identified in the panel; stale segmentation results after edits/profile/lens changes are rejected (`crates/engine/src/segment/mod.rs`, `docs/ai-masks.md`) |
| LR-MASK-PEOPLE | People parts | P2 | ⬜ | | shape exists (no UI): uses the photo's person / skin / hair / teeth / glasses mattes when it carries them (DNG semantic masks), else the subject heuristic |
| LR-MASK-LANDSCAPE | Landscape classes | P2 | ⬜ | | shape exists, evaluates empty |
| LR-MASK-BRUSH | Brush mask | P0 | 🟡 | `cmd:tool.brush`, `cmd:mask.brushStroke` (`autoMask`), `crates/pipeline/src/masks.rs`, `crates/ui-egui/src/brush_live.rs` | size/feather/flow/density/erase; Create New Mask starts a separate brush in paint mode; the stroke shows while the button is held with the committed stroke's dab spacing, feather and flow, so fast strokes have no gaps (#517); coverage builds up by passes, not hold time (holding the brush still adds nothing, live or committed, #518); Auto Mask: dabs weighted by similarity to the colour under the dab centre, refined by a guided filter on luminance (CPU + GPU); no A/B brushes, no pressure |