- New L1 crate effectcraft-segment: the MaskModel interface (prompts → foreground probability),
a registry of open-licensed models (licence must be MIT/Apache/BSD/…; URL, size, SHA-256),
a PyTorch checkpoint reader (zip + pickle subset), dependency-free SHA-256, and MobileSAM
(TinyViT-5M + SAM prompt encoder / two-way transformer / mask decoder) on rayon and ndarray's
safe GEMM. Loads the official mobile_sam.pt as published; checked against candle
(embedding 0.3% RMS, 99.9% equal mask pixels); ~0.7 s/frame encoder at 32 threads, cached
per frame, ~0.1 s decoder.
- track::roto: segment_with / propagate_with prompt a model with the strokes or the flow-warped
matte (box, core points, mask); its probability is a strong, unblurred prior for the same
graph cut (strokes stay hard constraints); a failing model or one disagreeing with the flow
(IoU < 0.5) is ignored. Prior gets a weight.
- effects::roto: Version 2.0 / 3.0 use the model chosen in Settings (set_model / model_for);
the model id is part of the chain seed.
- engine: roto_models (roto.models, roto.model.select / install / download / remove), weights in
the models folder (host::config_dir()/models, shared by the app, CLI and MCP), background
loading, downloads through the system curl, verified; Settings ▸ Roto Brush page (prefs
roto.model). host::config_dir moved from the desktop app.
- Tests: segment (kernels vs direct sums, checkpoint reader, SHA-256 vectors, registry), track
(fake model: prior shapes the cut, failing / disagreeing models ignored; real weights with
EFFECTCRAFT_MOBILESAM: base IoU 0.989, propagated ≥ 0.980 vs classic 0.973), engine (registry,
verification, selection; real install + load), MCP.
- Docs: README, ROADMAP, parity (MSK-4 done: 90/2/0), preferences, agents, architecture,
crates/segment/README.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Structure from motion for the 3D Camera Tracker in effectcraft-track
(camtrack): long KLT feature tracks with re-detection, parallax keyframes,
two-view initialisation (normalised 8-point essential matrix in RANSAC or
planar homography decomposition), incremental resection + triangulation,
sparse Levenberg-Marquardt bundle adjustment with the Schur complement and
Huber loss, focal length search (Fixed / Variable Zoom / Specify Angle of
View), tripod pan solve, Auto Detect, ground plane / target / world mapping.
Synthetic-scene tests (path, focal, error, tripod, flat, rendered frames).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Library groundwork for mask tracking (MSK-3) and the Warp Stabilizer (TRK-2),
clean-room from published methods:
- track::klt: Shi-Tomasi corners, pyramidal Lucas-Kanade with forward-backward check
- track::fit: RANSAC + least-squares translation/scale/similarity/affine/homography
- track:😷 MaskTracker following the pixels inside a mask outline per frame
- track::stabilize: incremental clip analysis (serde WarpAnalysis), Gaussian path
smoothing (Matsushita 2006), No Motion, preserve scale, crop / auto-scale with
max-scale relaxation
- path::interp: Mask Interpolation correspondence (arc-length vertex insertion,
shape-context matching) and rigid/linear in-betweens
Engine commands, UI and the effect itself are not wired yet (stubs unchanged).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New L2 crate effectcraft-track: per-point feature/search region tracking with
pyramid normalized cross-correlation and Lucas-Kanade (similarity + gain/bias)
sub-pixel refinement, confidence, adapt/stop/extrapolate/continue actions,
RGB/luminance/saturation channels, blur/enhance pre-processing, and
homography/affine/similarity solves.
Project: Motion Trackers > Tracker n > Track Point n property tree
(GroupKind::Tracker with type, target and options). Engine: track.* commands
(new/motion/stabilize/property/select/setType/setTarget/options/setPoint/
analyze/stop/apply/reset/delete/status), background or blocking analysis
job with progress and cancel (one undo step), apply to Transform, Stabilize
and Corner Pin. Renderer::layer_source for the tracker's frames.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bflatastic
Brandon Thomas