diff --git a/docs/architecture.md b/docs/architecture.md index ff210e5c..b53b059d 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -161,8 +161,14 @@ Half, Third and Quarter resolution render proportionally fewer pixels end to end Acceleration, the default, or Mercury Software Only; `render.backend`). The CPU renderer is the reference and keeps rendering layer content (sources, masks, CPU effects, layer styles) into the layer cache. The render crate defines an `Accelerator` trait; `effectcraft_gpu::Gpu` implements it -on wgpu compute shaders. `RenderOpts::backend` picks `Cpu`, `Gpu` or `Auto` (GPU when an -accelerator is attached and the project's renderer is the GPU). A GPU frame walks the comp like +on wgpu compute shaders. `RenderOpts::backend` picks `Cpu`, `Gpu` or `Auto` (an accelerator +attached and the project's renderer the GPU). Auto renders each comp's top-level frames on +whichever compositor measured faster for it (`render::auto::AutoPick`, kept by the accelerator): +per comp, output scale and path (readback or viewer display) it averages CPU and GPU frame times +(warm-up on both sides, the fastest warm-up frame starting the average, single stalls capped), +picks the GPU unless the CPU is clearly faster, and re-measures the other side every 48 frames. +Light comps (a few small layers, which the CPU composites only within their bounds) then stay on +the CPU instead of paying for full-frame passes and a readback. A GPU frame walks the comp like `draw_comp`: cached layer buffers are uploaded once per buffer, then transformed with the CPU's sampling (nearest, bilinear, Catmull-Rom bicubic, the same minification pre-filter), motion-blur sub-samples are accumulated, and track mattes, Preserve Transparency, layer style passes, @@ -184,16 +190,23 @@ GPU canvas; only the depth of field reads back its 8-byte radius range. Advanced blend modes, track mattes or Preserve Transparency and environment backgrounds take the CPU's 2D compositing path (their scenes still render through `render_3d`). Wireframe outlines draw on the GPU from the CPU's pixel list (`Renderer::wireframe_pixels`). GPU -effects (`effects::GPU_EFFECTS`, 152 of them: blurs, colour correction, keying incl. Key Light, -mattes, channel, stylize, distortion, transitions, generators, noise, grain and time; see +effects (`effects::GPU_EFFECTS`, 166 of them: blurs, colour correction, keying incl. Key Light, +mattes, channel, stylize, distortion and warps (Warp, Bezier Warp, Smear, Reshape, CC Bend It, CC +Page Turn), Cartoon, bevels, shapes, transitions, generators, noise, grain and time; see [effects.md](effects.md)) repeat the CPU effect's steps (padding, box radii, parameters, hashes) as kernels, in one module per family (`gpu::fx_*` with `shaders/fx_*.wgsl`); consecutive GPU effects run as one chain with one upload and one readback. Statistics that need the whole frame (Auto Levels / Contrast / Color, Equalize, Shadow/Highlight, Color Stabilizer, Remove Grain's noise level) are measured on the CPU from one readback and applied on the GPU; effects reading other frames (Echo, Posterize Time) upload the frames the host renders. Settings a kernel cannot -match render on the CPU (`effects::catalog::gpu_supported`). Tests render scenes on both paths and compare them -(≤ 1/255 at 8 bpc, ≤ 1e-3 at 32 bpc); they skip without an adapter. The desktop viewer builds the +match render on the CPU (`effects::catalog::gpu_supported`; e.g. Warp's Fisheye and Twist past +50 % or with a distortion, whose Newton inverse wanders chaotically near the crease). Tests render +scenes on both paths and compare them (≤ 1/255 at 8 bpc, ≤ 1e-3 at 32 bpc); they skip without an +adapter. Working textures (premultiplied RGBA f32) come from a pool: a released texture is reused +once every encoder that could still read it has been submitted, transparent inputs share one +zero texture per size, readback staging buffers are reused, and the 8/16 bpc quantisation after +each layer is fused into the layer's composite kernel; under test, pooled scratch images start +as NaNs (a kernel that skips pixels fails the oracle) and validation errors panic. The desktop viewer builds the `Gpu` on egui-wgpu's device and shows frames from GPU textures without reading them back (`ui-egui::frames`); headless renders, the CLI (unless `--gpu`) and CI use the CPU. On the web (WebGPU) the GPU composites viewer frames; steps that need a readback fall back to the CPU (the @@ -203,8 +216,9 @@ Playground cannons without interacting forces evolve every particle independentl backend (reached through `EffectHost::particles` when the GPU compositor is active) simulates one particle per invocation from the CPU's birth schedule and keeps per-key state checkpoints on the GPU, like `SimCache`; the CPU simulation stays the oracle (tests compare ids and positions). -`effectcraft-cli bench --gpu` reports CPU vs GPU ms/frame, speed-up and pixel agreement for every -comp plus an adjustment-layer comp. +`effectcraft-cli bench --gpu` reports CPU vs GPU ms/frame, speed-up, Auto's ms/frame, pixel +agreement and the steady-state upload / readback MB per frame for every comp plus an +adjustment-layer comp. **Colour and bit depth** (`crates/render/src/color.rs`, `crates/color/src/space.rs`). Pixels are `f32`, but 8 and 16 bpc projects clamp and quantise each layer after its source and masks and diff --git a/docs/parity.md b/docs/parity.md index 4eac5a3c..6165318b 100644 --- a/docs/parity.md +++ b/docs/parity.md @@ -10,7 +10,7 @@ How close EffectCraft is to After Effects 2026, feature by feature, and how much | Unweighted | ≈ 94% (half-credit) / 98% (fractions) | | P0 / P1 / P2 | ≈ 96% / 92% / 85% (half-credit); 99% / 97% / 96% (fractions) | | Features done / partial / missing | 80 / 12 / 0 of 92 | -| **Effects** | **298 of 298** After Effects 2026 effects exist; 42 still simplified ([effects.md](effects.md)); 152 run on the GPU | +| **Effects** | **298 of 298** After Effects 2026 effects exist; 42 still simplified ([effects.md](effects.md)); 166 run on the GPU | | Disabled menu entries left | 0 (the stub list is empty) | | Remaining work | ≈ 19–22 agent-hours at the pace measured so far (≈ 58 on the conservative audit scale) | | **Wall-clock estimate** | **≈ 4–5 hours** with five agents in parallel; ≈ 3 hours for 100% of P0 + P1 | @@ -136,11 +136,11 @@ the disk cache in the formats wave; the 3D Camera Tracker in M12.6.) | Text | ≈ 96% | 0.3 | no extruded strokes (M13.12: the Variable Font Axes animator re-spaces the text — advances follow the animated axes; M13.6: variable font axes in the character style — `layer.setText variations`, the Character panel's Variable Font Axes fields — shape with HVAR / gvar advances and draw at that design-space position; OpenType features — stylistic sets, discretionary ligatures, contextual / stylistic alternates, swash, titling, ordinals, fractions, figure styles, true small caps / all small caps and superior / inferior glyphs with faux fallback — per character with the Character panel's OpenType popup and `text.fontFeatures` landed in M13.2; vertical Roman / Tate-Chu-Yoko, forced LTR paragraphs, caret on animated and path text, Variable Font Axes and Lottie style runs landed in M13.5; extruded, bevelled text in M7.6; per-character styles, paragraph settings, on-canvas editing and the `sourceText` style API in M9.9–M9.10) | | Web | 93% | 0.5 | GPU effects in the browser (WebGPU compositing only on the page), no disk cache, no shared-memory threads inside one engine instance (browser storage, Web Audio, Web Worker renders/analyses, WebGPU viewer and offline install landed in M15.2; viewer frames in frame workers fed by project diffs, Roto Brush propagation in a worker, non-blocking `wait: true` jobs and a browser Media Browser (File System Access folders, browser storage) in M13.10) | | 3D | 88% | 5.5 | multi-view layouts, the Extended Viewer for Advanced 3D comps (Classic 3D Extended Viewer landed in M13.5 UI completion); collapsed precomps of another size seen through the parent's camera render (fixed in M13.2); stereo rigs, orbit nulls, lights controlled by the camera, cameras/lights from glTF models, environment backgrounds, Advanced 3D motion blur, blend modes and track mattes landed in M7.7; Classic 3D iris-shaped bokeh with highlights, progressive depth of field on tilted layers and the focus-link commands landed in M13.5; Advanced 3D depth of field with the iris and highlight options, collapsed precomps as real Advanced 3D geometry and extruded text/shape strokes landed in M13.8; Advanced 3D (glTF/OBJ models, primitives, extruded text and shapes, PBR, image-based light, shadow maps, GPU rasteriser) in M7.4–M7.6; Advanced 3D end to end on the GPU (motion blur, iris depth of field, compositing) in M13.11 | -| Effects | ≈ 85% | 3.0 | GPU versions of the remaining effects (152 run on the GPU since M13.9, EFF-5: keying, mattes, channel, colour correction, stylize, distort, noise, blurs and time effects joined the 58 of M12.7; Warp, Bezier Warp, Reshape, Smear, Cartoon, CC Page Turn and others still render on the CPU) and the missing controls listed as partial in [effects.md](effects.md) (every After Effects effect exists since M9.11, M12.5 and M12.6; parameter names, order, twirl-downs, popups, units and defaults were aligned in M9.12) | +| Effects | ≈ 85% | 3.0 | GPU versions of the remaining effects (166 run on the GPU since M13.13, EFF-5: Warp, Bezier Warp, Reshape, Smear, CC Bend It, CC Page Turn, Cartoon, Color Emboss, Circle, Ellipse, Iris Wipe, Bevel Alpha / Edges and Gaussian Blur (Legacy) joined the 152 of M13.9; simulations, 3D channel, VR, audio-driven, OCIO, text and the CC light / transition families still render on the CPU) and the missing controls listed as partial in [effects.md](effects.md) (every After Effects effect exists since M9.11, M12.5 and M12.6; parameter names, order, twirl-downs, popups, units and defaults were aligned in M9.12) | | Interface | 74% | 5.0 | more Learn tutorials and pixel-level fidelity of dialogs (the Home ▸ Learn tab with interactive tutorials and a UI fidelity pass landed in M13.5 UI completion; visual editors for Lumetri RGB / hue-saturation curves, Colorama's output cycle wheel, Glow's colour map and Reshape's correspondence points (viewer handles), a scrolling Preview panel and AE-style Composition / Timeline tabs (close, label swatch, viewer lock) landed in M13.10; Timeline outline and Project panel columns scroll horizontally, the Layer Style dialog, ROI resize handles, Pan Behind snapping and 3D Reference Axes landed in M13.5; native macOS menu bar, Timeline columns/search/reveal-add, Home screen with recent projects and all AE workspaces landed; viewer rulers/snapping/channels/snapshots landed in M0.13) | | Project | ≈ 68% | 5.5 | auto-save, folder moves, OCIO displays beyond the built-in tone map (Color Engine with OCIO/ACES working spaces, HDR compand/tone mapping, Rec. 2100 PQ/HLG output, Feet + Frames, display colour management, Simulate Output and the locked viewer landed in M7.7; proxies and Interpret Footage fields / pixel aspect / alpha guess landed: PRJ-8, PRJ-3) | | Masks & roto | 74% | 5.0 | Roto Brush's learned (3.0) segmentation model (variable-width mask feather points with the Mask Feather tool landed in M13.5; mask tracking and Mask Interpolation landed in M6.6; Roto Brush & Refine Edge with graph-cut segmentation, flow propagation, edge matting, decontamination and Freeze in M6.7) | -| Preview | 77% | 2.8 | GPU compositing of Advanced 3D layers with blend modes / track mattes and of environment backgrounds (Advanced 3D runs — raster, motion blur, iris depth of field, compositing — and wireframes run on the GPU since M13.11: CPU warm 3394 → GPU warm 290 ms/frame at 1920×1080 (11.7×; Half 968 → 85 ms), GPU ≠ CPU on 0.005 % of pixels, on an M4 Pro under load; Classic 3D bokeh depth of field — iris shapes, highlights, fringe, progressive blur on tilted planes — runs in WGSL since M13.6: 3D Showcase GPU warm 205 → 54 ms/frame at full size on an M4 Pro; the Preview panel's five shortcuts with their own Include / Loop / Cache Before Playback / Range / Play From / Frame Rate / Skip / Resolution / Full Screen / stop options and `playback.settings.get/set` landed in M13.8; Classic 3D runs and adjustment layers composite on the GPU since M12.7; persistent disk cache with the blue cache bar landed; region of interest, snapshots, exposure and Fast Previews landed in M0.13) | +| Preview | 77% | 2.8 | GPU compositing of Advanced 3D layers with blend modes / track mattes and of environment backgrounds (pooled GPU textures and fused quantisation since M13.13: Lower Third GPU warm 21 → 4 ms/frame, and Auto now picks the CPU or the GPU per comp from measured frame times; Advanced 3D runs — raster, motion blur, iris depth of field, compositing — and wireframes run on the GPU since M13.11: CPU warm 3394 → GPU warm 290 ms/frame at 1920×1080 (11.7×; Half 968 → 85 ms), GPU ≠ CPU on 0.005 % of pixels, on an M4 Pro under load; Classic 3D bokeh depth of field — iris shapes, highlights, fringe, progressive blur on tilted planes — runs in WGSL since M13.6: 3D Showcase GPU warm 205 → 54 ms/frame at full size on an M4 Pro; the Preview panel's five shortcuts with their own Include / Loop / Cache Before Playback / Range / Play From / Frame Rate / Skip / Resolution / Full Screen / stop options and `playback.settings.get/set` landed in M13.8; Classic 3D runs and adjustment layers composite on the GPU since M12.7; persistent disk cache with the blue cache bar landed; region of interest, snapshots, exposure and Fast Previews landed in M0.13) | | Tracking | ≈ 96% | 0.3 | face tracking is a classical (skin model + feature components + shape model) fitter, not a learned detector: profile views and occluded faces are weak; Rolling Shutter Ripple is approximated by Subspace Warp's mesh density (face tracking (Outline Only / Detailed Features with Face Track Points and Extract & Copy Face Measurements), Subspace Warp's content-preserving mesh warp on subspace-smoothed trajectories, and radial lens distortion (k1, k2) in the camera bundle adjustment with Undistort Footage landed in M13.3; Rolling Shutter Repair in M9.11; point tracker, mask tracking, Warp Stabilizer and the 3D Camera Tracker in M6.x / M12.5 / M12.6) | | Paint | 0% | 4.0 | Brush, Clone Stamp, Eraser | @@ -204,6 +204,33 @@ axes since M13.6; Advanced 3D's DOF has iris shapes since M13.8). as real geometry lit by the parent (2D collapsed precomps draw nested 3D layers with the parent's renderer); text and shape strokes extrude as bevelled meshes in paint order. +## Update: M13.13 GPU performance and the remaining GPU ports + +- **Small comps on the GPU (PRV-3)**: the GPU frame allocated (and zero-initialised) several + full-frame RGBA f32 textures per layer; working textures now come from a pool (reused once + every encoder that could read them is submitted), transparent inputs share one zero texture, + readback staging buffers are reused and the 8/16 bpc quantisation after each layer is fused + into the layer's composite. `bench --gpu` on an M4 Pro under load (both binaries run back to + back, median of 15): Lower Third 1920×1080 GPU warm 21.0 → 4.1 ms/frame (viewer 11.1 → 2.6; + Half 10.1 → 3.1), EffectCraft Intro 27.5 → 7.8 (viewer 21.7 → 3.6), 3D Showcase ≈ 95 → ≈ 60 + (viewer ≈ 80 → ≈ 40), Adjustment Layers 93 → 36. In steady state the only transfer left is + the frame's own readback (0.0 MB uploads per frame on every demo comp; the adjustment + footprint is now cached and uploaded once). +- **Auto picks per comp**: `Backend::Auto` (Mercury GPU Acceleration) times each comp on both + compositors (`render::AutoPick`: per comp, scale and path; warm-up, capped stalls, re-probe + every 48 frames) and renders on the faster one, in renders and in the viewer. A light comp + stays on the CPU, which composites only the layers' bounds: Lower Third Auto 0.8 ms/frame + (CPU warm 1.7, GPU warm 4.1, which is mostly the 32 MB readback). +- **GPU ports (EFF-5)**: Warp, Bezier Warp, Reshape, Smear, CC Bend It, CC Page Turn, Cartoon, + Color Emboss, Circle, Ellipse, Iris Wipe, Bevel Alpha, Bevel Edges and Gaussian Blur (Legacy) + run on the GPU (166 GPU effects); Median, Median (Legacy) and Dust & Scratches take any radius + (a sliding 512-bin histogram per row segment above radius 4); CC Power Pin's Perspective + below 100 % runs on the GPU. All match the CPU oracle within 1/255 (8 bpc) / 1e-3 (32 bpc) on + every tested pixel except: Smear and Reshape allow 0.1 % (f32 point-in-outline tests on + boundary pixels; measured 0); Warp's Fisheye and Twist bent past 50 % or with a Horizontal / + Vertical Distortion render on the CPU (`gpu_supported`), since their Newton inverse wanders + chaotically near the crease and f32 settles on other pixels. + ## Update: M13.11 Advanced 3D on the GPU, Essential Graphics mirrors, ScriptUI paint - **Advanced 3D on the GPU (PRV-3, 3D-3)**: a whole Advanced 3D run renders on the device