Compare commits

...

106 Commits

Author SHA1 Message Date
bflatastic f0b7b11393 Docs: gaps.md records the community fixes of 7 October (#135, #139, #141, #142)
The ui_angles evidence (#141) moves next to G2's other regression
evidence; the Stability row notes the parent-chain hang, the label colour
panic and the contained GPU failures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 07:54:28 +07:00
BFlat 578f837fa4
Merge pull request #144 from storytold/fix/139-readback-wait
GPU: land #139 (contain GPU initialization errors) with a readback latency fix
2026-10-07 07:53:36 +07:00
bflatastic 80dff0fb0e GPU: bounded readbacks block on the GPU instead of sleeping between polls
#139's bounded read_buffer polled the device without waiting and slept up
to 10 ms between polls, adding up to 10 ms to every readback (the desktop
viewer reads every GPU frame back): the gpu crate's tests took 227 s
against 119 s before. It now waits with PollType::Wait and the remaining
deadline as its timeout (a timeout still retires the context), then takes
the result, giving another thread that polled the same device up to 1 ms
to deliver its callback. The tests take 71–79 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 07:52:53 +07:00
bflatastic bfabc72be1 Merge #139 from dexterlabs1: contain GPU initialization errors and retire failed compositor work 2026-10-07 07:52:46 +07:00
BFlat a36eabee5a
Merge pull request #142 from dexterlabs1/contribution/parent-cycle-current-review
Prevent parenting hangs on malformed loaded ancestor chains
2026-10-07 07:26:44 +07:00
BFlat d0bf941522
Merge pull request #141 from jonassneu/fix/angle-input-validation
UI: cancel numeric edits on Escape and reject non-finite values
2026-10-07 07:24:12 +07:00
BFlat 2a817d8c72
Merge pull request #135 from storybored379/fix/unicode-label-colors
Settings: handle invalid Unicode label colours gracefully
2026-10-07 07:19:40 +07:00
BFlat 7c0cbc5a5c
Merge pull request #128 from EthanSuquet/chore/untrack-nested-target
Untrack crates/hevcenc/target (36 MB of build output) and ignore nested target/
2026-10-07 07:16:33 +07:00
Dexter f5b98f8199 fix(engine): bound parenting validation on malformed ancestor chains 2026-10-06 18:54:13 -05:00
Jonas Neubauer a6685a1eb7 G2: cancel numeric edits on Escape and reject non-finite values 2026-10-07 01:43:57 +02:00
Dexter 94dbccf69f fix: contain GPU initialization and retire failed readbacks 2026-10-06 16:40:12 -05:00
Dexter 2e02523fe9 test: tolerate installed variants of bundled font families 2026-10-06 16:40:12 -05:00
BFlat 2fadd14f09
Merge pull request #134 from storytold/docs/issues-85-117
Docs: user issues #85–#117 in gaps, parity, ROADMAP and README
2026-10-07 02:34:38 +07:00
bflatastic 751d6a5c1a Docs: user issues #85–#117 in gaps, parity, ROADMAP and README
gaps.md records the seven reports of 6 October (#85, #88, #89, #93, #103,
#106, #117), what they found in features marked done, their fixes of 7
October and the regression tests; parity.md's Interface, Animation and
Preview rows, the ROADMAP's reliability note and the README's panels
paragraph (drag and drop between panels) follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 02:34:15 +07:00
bflatastic 63a7483a48 Comments: describe the new drop, angle and cache code without product names
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 02:34:14 +07:00
BFlat 1ea493f257
Merge pull request #133 from storytold/fix/85-88-viewer-drop
Viewer: drop Project items, files and effects onto the comp (#85, #88)
2026-10-07 02:31:41 +07:00
bflatastic 8ef1a28bc4 Viewer: drop Project items, files and effects onto the comp (#85, #88)
The Composition viewer ignored drags from other panels. Now, as in After
Effects:

- A Project item dropped on the viewer becomes a layer centred where it
  lands (above the selected layer, like any new layer); while dragging, the
  viewer is outlined and the layer's frame follows the pointer.
- Media Browser files do the same; files dropped from the OS file manager
  do too when the platform reports where the pointer is during the drag
  (otherwise they import as before).
- An effect from Effects & Presets goes on the layer under the pointer,
  which is outlined while dragging.

layer.addItem and layer.newModel take `position` ([x, y] comp px).
file.import takes `addToComp` with `time`, `index` and `position`;
mediaBrowser.import shares it (file::add_to_comp): layered files go in as
their comps, the comp they were dropped in stays open, and items that
can't be layers are reported in `errors`. viewer::layer_at is the hit
test pick() and the drop share.

Tests: ui_drag_drop.rs (project item and effect dropped on the viewer)
and imported_files_go_into_the_comp_where_they_were_dropped (engine).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 02:31:13 +07:00
BFlat 78f49e09ab
Merge pull request #131 from storytold/fix/89-timeline-drop
Timeline: Project items land where they are dropped (#89)
2026-10-07 02:19:04 +07:00
bflatastic 1a0940d50a Timeline: Project items land where they are dropped (#89)
A Project item (or Media Browser file) dropped in the Timeline always
went above the selected layer, starting where new layers start.

Now, as in After Effects, a line shows where it goes in the stack (above
the first layer row whose middle is below the pointer, else below the
last), and over the time graph a marker with a timecode tag shows its In
point, which follows the pointer (frame-snapped) or, with Shift, the
current time. The layer reorder drag shares the insertion logic.

layer.addItem and layer.newModel take `index` (1-based stack position);
mediaBrowser.import passes `time` and `index` on, one layer under the
other.

Tests: project_items_land_where_they_are_dropped_in_the_timeline
(ui_drag_drop.rs, real pointer drags: outline, time graph, Shift, below
the last layer) and add_item_at_a_stack_index_and_time (engine).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 02:18:38 +07:00
BFlat 9e619d2893
Merge pull request #129 from storytold/fix/106-gpu-out-of-memory
GPU: running out of video memory falls back to the CPU instead of crashing (#106)
2026-10-07 02:06:46 +07:00
bflatastic e64076e432 GPU: running out of video memory falls back to the CPU instead of crashing (#106)
The desktop app shares egui-wgpu's device, which had no uncaptured-error
handler, so wgpu's default one panicked on an out-of-memory error on
every frame thread; the panicking frame stayed "in progress" forever and
never rendered again. The GPU frame budget was a fixed 1 GiB.

- GpuContext::new logs uncaptured errors once its pipelines are built
  (errors while building them still panic there, which the app catches
  to fall back to CPU compositing).
- Gpu::within_memory runs GPU work in an out-of-memory error scope; on an
  error the result is discarded and uploaded layers and pooled textures
  are freed.
- Frame workers render each frame within it; out of memory, the frame
  renders on the CPU and the RAM preview keeps half as many GPU frames
  (down to 64 MiB), since wgpu can't tell how much video memory there is.
- A frame whose render panics renders again on the CPU, or is released,
  so the viewer asks for it again.

Tests: a_frame_whose_render_panics_is_released (failed before),
running_out_of_video_memory_halves_the_gpu_frame_budget and
within_memory_returns_the_work_when_there_is_room.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 02:06:15 +07:00
EthanSuquet 0f583853ad Untrack crates/hevcenc/target (36 MB of build output) and ignore nested target/
Commit c9d5e733 (M13.4, hevcenc deblocking) committed 207 files under
crates/hevcenc/target/agent-codec-hevc/: an agent's CARGO_TARGET_DIR with
rlibs, rmetas, build-script outputs and .rustc_info.json (~36 MB). They
were tracked because .gitignore only ignored the root `/target`.

- git rm --cached the directory; nothing references it.
- .gitignore: `/target` -> `target/`, so a per-crate or per-agent target
  dir is ignored anywhere in the tree. No tracked file lives under a
  directory named `target`, so nothing else becomes ignored.

The blobs stay in history; this keeps them out of every new checkout and
stops the next agent run from committing more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 15:02:20 -04:00
BFlat 0fac647928
Merge pull request #127 from storytold/fix/103-preview-audio-cache
Preview: Audio, Lock and Shy keep the cache; playback with audio waits for frames (#103)
2026-10-07 01:47:01 +07:00
bflatastic 93856382e7 Preview: Audio, Lock and Shy keep the cache; playback with audio waits for frames (#103)
Toggling a layer's Audio, Lock or Shy switch (or Hide Shy Layers) threw
away every cached frame: the RAM preview identity hashes each comp's
address, which any edit replaces, and the layer and disk cache keys
hashed every switch. None of these change pixels.

- Switches::pixels, Layer::same_pixels and Comp::same_pixels (exhaustive
  field lists, so a new field must be considered) and Comp::pixel_form.
- Layer cache keys hash Switches::pixels; the disk cache's comp content
  key hashes Comp::pixel_form.
- Frames::identity: when a new revision's comps draw the same pixels as
  those of the comp's last new identity, that identity (and the project
  holding its comps' addresses) is kept, so the frames stay, also when the
  switch is toggled back.

With audio on, playback skipped frames that weren't rendered yet. Now the
sound waits: until the frames ahead (to the end of the preview, or what
fits in the RAM preview) are cached, every frame shows as it renders,
silently, like a preview without audio; then the sound starts in sync.
Reaching an uncached frame stops the sound instead of skipping frames.

Tests: audio_lock_and_shy_switches_keep_the_frames (frames.rs, nested
comps and toggling back), audio_lock_and_shy_keep_cached_layers (engine),
the disk key in tests_disk_cache, and
preview_with_audio_shows_every_frame_and_sounds_once_cached.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 01:46:35 +07:00
BFlat 4a401a4785
Merge pull request #123 from storytold/fix/93-angle-revolutions
Angles: the revolutions ("0x") scrub and take typing (#93)
2026-10-07 01:29:27 +07:00
bflatastic 27b0eec216 Angles: the revolutions ("0x") scrub and take typing (#93)
An angle's revolutions were painted text: only the degrees could be
dragged or typed, so full turns needed the dial or typing past 360.

fx_widgets::angle_field draws `0x+45.0°` as two hot numbers: the
revolutions (widgets::hot_int_at, ten pixels a turn, fractions adding up
between frames so slow drags still step; a typed value is truncated) and
the degrees, each keeping the other. Effect Controls, the Timeline and
the Properties panel use it, so they read and edit angles alike (the
Timeline showed `-1x-40.0°` and Properties had its own split).

Automation ids: timeline.prop.<uid>.revolutions,
effectControls.prop.<uid>.revolutions, properties.prop.<uid>.revolutions.

Regression test: angle_revolutions_scrub_and_take_typing drags and types
the revolutions in all three panels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 01:29:03 +07:00
BFlat 1a9288f83c
Merge pull request #122 from storytold/claude/issue-117-render-settings-templates
Render Queue: Render Settings templates apply from the menu (#117)
2026-10-07 01:20:01 +07:00
bflatastic 8a415e03ca Render Queue: Render Settings templates apply from the menu (#117)
Choosing Draft Settings (or any entry) in a Render Queue item's Render
Settings menu did nothing when the menu had been moved up to fit the
window. egui hit-tests a fixed-position Area where it was asked to go,
not where it was moved, so the popup's "pressed outside" check was true
over its own entries and the menu closed on the press, before the click.

widgets::pressed_outside tests the press against the rect the popup was
drawn at. Every popup with the same check uses it: popup_menu (Render
Queue, Effect Controls, dropdowns), the colour popup, the Preview and
viewer popups, the panel menu, the Flowchart, OpenType and font menus.

Regression test: ui_render_queue.rs picks Draft Settings from a menu
moved up to fit a 1000 px window; it failed before the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 01:19:25 +07:00
BFlat ea25e402b1
Merge pull request #119 from storytold/claude/issue-94-windows-arm64-docs
Docs: Windows ARM64 builds in the README (#94)
2026-10-07 00:44:23 +07:00
Claude fa25545d8b
Docs: Windows ARM64 builds in the README (#94)
#94 asked for native Windows on ARM builds and for their availability and limits to be
documented. The builds landed in #101 (an arm64 entry in the release matrix, the PE checks in
package.ps1 and the windows-arm64 workflow that installs the MSI on ARM64 hardware), but the
README never said they exist. Get started now lists the installers on the Releases page for
each platform, says the Windows ARM64 build runs without x64 emulation, and says what CI checks
on ARM64 hardware and what it doesn't yet (the app's window and a native test run).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018iLqXiCVkUULoN4eWTD7cB
2026-10-06 17:44:05 +00:00
BFlat c750b299cf
Merge pull request #118 from storytold/claude/system-fonts-rmnw4m
G2: font menus list the installed fonts
2026-10-07 00:37:13 +07:00
storybored379 202417dab3 Settings: handle invalid Unicode label colours gracefully
Use checked hex slices so prefs.set follows the existing invalid-colour fallback when a component splits a UTF-8 character. Cover invalid Unicode placements and valid mixed-case colours through the command.

Co-Authored-By: OpenAI Codex <noreply@openai.com>
2026-10-06 09:58:59 -07:00
Brandon Thomas 588d269d29 Effects: adding an effect brings up Effect Controls (#87)
effect.apply / effect.applyLast run through the menu dispatcher (Effect
menu, Last Effect, an effect dropped on a Timeline layer or on Effect
Controls) now bring up Effect Controls, opened if closed and brought to
the front without taking the focus, on the layer the effect went to
(selecting it if it was dropped on an unselected layer, keeping the new
effect selected). Double-clicking an effect's name in the Timeline opens
it in Effect Controls (layer.select, prop.select, window.panel) instead
of twirling it, as in After Effects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 04bd305b9dccbb34d8c70fe870513208cdae0b7b)
2026-10-06 11:34:27 -04:00
Brandon Thomas aa31ffc2fb Project: opening a comp brings up its Composition panel (#90)
Opening a comp (Project double-click, Open Composition, a Timeline tab)
now reopens the Composition panel if it was closed and brings it (and
the Timeline) to the front, as in After Effects. A closed Composition
panel also reopens in the centre (with the Layer panel or other viewers,
else above the Timeline) instead of beside Effects & Presets, which also
fixes Window > Composition.

App::raise_panel opens/fronts a panel without moving the focus;
show_panel builds on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit adf2c6dac78ce426ea0bf6b4199b7238d8ad99dd)
2026-10-06 11:34:27 -04:00
Claude 0ffa2470b3
Docs: installed fonts in the font menus (gaps, parity, ROADMAP, README, architecture)
gaps.md records the Windows report and its fix under real-user experience and G2; parity.md's
Text row, the ROADMAP's reliability note and the README's Text bullet say text can use any
installed font; architecture.md says when the system font scan runs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019EjTYdt76hTG4Kh2CJFnCr
2026-10-06 15:14:55 +00:00
Claude dda9961889
G2: font menus list the installed fonts
A Windows user reported that the Properties panel's font menu offered only the bundled Inter,
JetBrains Mono and Noto Serif. The system font folders were scanned only when a project asked for
a family the bundled fonts lack, so until then every font menu (Properties, Character, Essential
Graphics) and `file.findMissing {what: fonts}` saw the bundled families alone, on every
platform. Listing the families now scans the system fonts first, and the desktop app starts that
scan on a background thread at launch so the menus are complete when first opened.

The style menus in the Properties and Character panels list the chosen family's own styles
instead of a fixed Regular / Medium / SemiBold / Bold / Italic. Both menus build their lists only
while open.

New query for agents: `text.fonts {query?, rescan?}` (MCP tool `list_fonts`) lists every family
with its styles, origin (bundled / system / user) and own-language name; `rescan` picks up fonts
installed since launch.

Tests: a fresh-process test that the family list includes every family in the platform's font
folders without a prior scan (failed before), and `text.fonts` filtering, rescan and per-family
styles, and the `list_fonts` MCP tool.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019EjTYdt76hTG4Kh2CJFnCr
2026-10-06 15:14:55 +00:00
Brandon Thomas a46ebf4dc7 Puppet: selected pins are filled and ringed, unselected hollow (#96)
A selected pin was drawn only 1 px bigger and slightly brighter. Now, like
selected and unselected mask vertices in the viewer, a selected pin is
filled with its kind colour and ringed in white, and an unselected pin is
hollow (dark centre, coloured outline). Drawing only; hit-testing is
unchanged.

Regression test: selected_puppet_pins_are_filled_and_unselected_hollow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 103b5c41e9e8ea84008593e05af50b01a2800b3e)
2026-10-06 11:09:41 -04:00
Brandon Thomas 7002c92632 Timeline: selected effect rows highlight (#91, #97)
Clicking an effect's name selected it (prop.select with the effect's group
uid) but group rows never painted the selection, so there was no feedback.
Group rows (effects, masks, shape groups...) now fill with row_selected when
selected, exactly like selected property rows.

Regression test: timeline_selected_effect_row_highlights.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 8b140428b39a24c3e1207ffde0430e759e316112)
2026-10-06 11:09:41 -04:00
Brandon Thomas 4b9e701223 Merge remote-tracking branch 'origin/main' into land-prs 2026-10-06 10:33:00 -04:00
Brandon Thomas d4e310b252 Merge fixes for #95 + #71 on current main
- Project panel Clear (#95's focus routing) goes through invoke, so deleting
  items that compositions use still asks first (M3.16).
- ui_project_delete: let a dialog finish egui::Modal's sizing frame before
  clicking its buttons (#95 moved dialogs to egui::Modal).
- #71's i18n / theme tests clear the frame's texture delta; egui asserts on
  unhandled deltas in debug builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 10:22:15 -04:00
Brandon Thomas 0e27c845f7
Merge pull request #101 from storytold/pr/windows-arm64
Windows on ARM64: release builds, PE header check, install test on ARM64 hardware
2026-10-06 10:21:02 -04:00
Brandon Thomas 7e1c2d0329 Windows ARM64 workflow: pin third-party actions by commit
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:51:09 -04:00
Brandon Thomas 370a7a5131 Windows on ARM64: release builds, PE header check, install test on ARM64 hardware
Release builds add an arm64 MSI and portable zip, cross-compiled on the x64
runner (signing and WiX as for x64). package.ps1 checks each binary's PE
header before packaging: Machine must match -Arch, the app must be a GUI
program and the CLI a console program. It skips the CLI smoke run when an
ARM64 build can't run on the host. A new Windows ARM64 workflow packages the
ARM64 MSI, installs it on a Windows 11 ARM64 runner, checks both installed
exes are ARM64, runs the CLI, and uninstalls. Ported from printcraft#66.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:38:55 -04:00
Brandon Thomas dc96725725 audio: allow the deprecated fetch_update while rust-version is 1.95
Rust 1.99 renamed AtomicU32::fetch_update to try_update; try_update is not
available on the workspace's minimum Rust (1.95).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:46:22 -04:00
Brandon Thomas 633fbcfaeb Merge PR #71: Persist Japanese menu language in General settings 2026-10-06 08:42:00 -04:00
Brandon Thomas dfd8aac431 keyframe: never-crash deny lints (craftrules standards/never-crash.md)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 2232354b902238827b2e80a29ef266c2e5bde272)
(cherry picked from commit 95da7943454ca5b2575993d11f9ba046822ce255)
2026-10-06 08:41:33 -04:00
Brandon Thomas 98872a4690 Merge PR #95: Improve AE-style editing, easing fidelity and Windows GPU rendering 2026-10-06 08:41:33 -04:00
Brandon Thomas e6c817bb3f
Merge pull request #99 from storytold/pr/release-0.3.1
Release: EffectCraft v0.3.1
2026-10-06 08:04:15 -04:00
Brandon Thomas e1e0a78be5 Release: EffectCraft v0.3.1
Fixes the Windows installer: the Start Menu shortcut and Add/Remove Programs icon were blank. The shortcut description was wrong. Release builds no longer open a console window behind the app.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:04:07 -04:00
Brandon Thomas 722d7d673c
Merge pull request #98 from storytold/pr/windows-packaging
Windows packaging: Start Menu/Add-Remove icon, no console window, correct description
2026-10-06 07:54:09 -04:00
Brandon Thomas 17b17bcce3 Windows packaging: Start Menu/Add-Remove icon, no console window, correct description
The MSI Icon id had no .ico/.exe extension, which Windows Installer needs for
advertised shortcuts and ARPPRODUCTICON, so both showed a blank icon. Release
builds of the desktop app are now GUI-subsystem (no console window; the CLI stays
a console program), the shortcut/summary description names what EffectCraft does, and
packaging-lint rejects an Icon id without an extension. Found in printcraft#57.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 07:51:06 -04:00
midasdf 25233abbcb G9: persist Japanese menu language without bundled fonts 2026-10-06 16:22:23 +09:00
Dexter 39e6907b5f G2: improve editing reliability and Windows rendering fidelity 2026-10-06 00:00:31 -05:00
BFlat 80c246d843
Merge pull request #92 from storytold/m3-delete-confirm
M3.16: deleting Project items in use asks first (After Effects parity)
2026-10-06 11:36:27 +07:00
bflatastic 9624024dc1 M3.16: deleting Project items that compositions use asks first
As in After Effects, the Delete key and the Project panel's trash button ask before deleting
items that layers in other compositions use, saying how many layers in how many compositions go
with them (Delete / Cancel; Escape cancels). Unused items are deleted at once; agents running
project.delete directly are not asked. project.usage {items?} reports the count, sharing the
deletion's item and usage helpers. Follow-up to #72.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 11:35:57 +07:00
BFlat b46be83ab4
Merge pull request #84 from fortesnow/jp-font-fallback
text: script-aware system font fallback; accept PostScript / full font names
2026-10-06 10:57:36 +07:00
BFlat 86ae2fa218
Merge pull request #72 from jonassneu/fix/project-delete-shortcuts
Project: delete selected items with Delete and Backspace
2026-10-06 10:56:43 +07:00
bflatastic f0578a5940 Merge branch 'main' into pr72-land
# Conflicts:
#	docs/gaps.md
2026-10-06 10:55:43 +07:00
BFlat 42401e7672
Merge pull request #70 from jonassneu/fix/viewer-visible-layer-selection
Viewer: respect solo and visibility in layer selection
2026-10-06 10:54:56 +07:00
fortesnow c8e9efb962 text: script-aware system font fallback and PostScript / full font names
Japanese (and Chinese, Korean) text in a family that lacks the glyphs, such as the default Inter,
rendered as missing-glyph boxes: `fallback_for` only searched the bundled faces and faces some
other layer had already loaded, so the result depended on render order. Now, after those cheap
candidates, it tries the platform's usual family for the character's writing system (kana and
hangul pick their language; Han ideographs follow the locale, Japanese first when it is unknown,
`EFFECTCRAFT_CJK_LOCALE` overrides) in the preferred face's style, and finally any installed face
that covers the character, reading files without keeping them. Results are cached per 256-character
block and preferred face.

`resolve` also accepts a face's full name (`Yu Gothic Bold`) or PostScript name (`YuGothic-Bold`),
which is how After Effects scripts set `textDocument.font`; the sfnt reader now records name id 6.

Tests: script classification, PostScript / full-name resolution on the bundled Inter, and (native,
skipped when no Japanese family is installed) that kana, ideographs and CJK punctuation in Inter
Bold fall back to an installed Japanese family's bold cut, with the cache returning the same face.
2026-10-06 10:41:15 +09:00
BFlat 877afeb25a
Merge pull request #83 from storytold/pr/release-gate
Release gate: ui_agent_audit builds in release mode
2026-10-06 08:37:17 +07:00
bflatastic f25b75486b Release gate: the agent-audit test builds in release mode
`cargo xtask ci` runs clippy and the tests with --release, where egui has no Style::debug (it
exists only with debug_assertions), so ui_agent_audit.rs failed to compile since M13.14. The
debug option is set only in debug builds, and the test, which needs the widget types egui records
with it, is ignored in release builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 08:37:01 +07:00
BFlat 857340036b
Merge pull request #82 from storytold/docs/issue-status
Docs: user issues #41–#47 and #63–#68 in gaps.md
2026-10-06 00:39:08 +07:00
bflatastic 25f353fb2c Docs: gaps.md records the user issues of 5 October and what became of them
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 00:38:43 +07:00
BFlat dd56ae5c00
Merge pull request #81 from storytold/fix/44-project-import
Fix #44: double-click the Project panel's empty area to import
2026-10-06 00:37:36 +07:00
bflatastic df12664b8e Fix #44: double-click the Project panel's empty area to import
As in After Effects, a double-click below the items in the Project panel opens File ▸ Import ▸
File... (rows and the scroll bar keep their own clicks). Dropping files on the window already
imported them; the test now covers both. (On Wayland, winit delivers no file drops at all, so
dragging files in works on X11, macOS and Windows.)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 00:37:05 +07:00
BFlat fdfbca5faf
Merge pull request #80 from storytold/fix/43-wayland-icon
Fix #43: the AppImage's window shows the EffectCraft icon on Wayland
2026-10-06 00:31:29 +07:00
bflatastic b424c321dc Fix #43: the AppImage's window shows the EffectCraft icon on Wayland
Wayland compositors take a window's icon from the desktop entry named by its app ID; the .deb,
.rpm and Flatpak install it, an AppImage installs nothing, so its window got the generic icon.
Started from an AppImage ($APPIMAGE), the app now writes a hidden (NoDisplay) desktop entry
pointing at it, and its 256 px icon, under $XDG_DATA_HOME (~/.local/share) before the window
opens. Nothing is written when an installed package or the user's own entry provides the ID;
the entry follows the AppImage when it moves; failures are only logged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 00:31:06 +07:00
BFlat 72e60dc6fa
Merge pull request #79 from storytold/fix/45-project-details
Fix #45: the Project panel's details stay inside the panel
2026-10-06 00:26:17 +07:00
bflatastic e6d4a64775 Fix #45: the Project panel's details stay inside the panel
The details header (item name, size, duration, codec) and the "Select an item" hint were painted
with no width limit, so in a narrow Project panel they ran past its right edge. They now stop
10 px short of it and end in "…" when they don't fit (widgets::text_fit, a shared helper), and
are registered for automation (project.details.*).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 00:25:57 +07:00
BFlat 83cd5d0f45
Merge pull request #78 from storytold/fix/46-docking
Test #46: docking by the panel menu button and resizing by the gutters
2026-10-06 00:18:37 +07:00
bflatastic efcbc2e412 Test #46: panels dock by a drag from the panel menu button; gutters resize
Covers what issue #46 reports (v0.1.1 on Linux): a panel dragged by its hamburger onto another
group's edge docks there as its own group, and dragging the gap between panels resizes them.
Both work on main (tab dragging was reworked in M3.15 for v0.2.0); this keeps them working.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 00:18:11 +07:00
BFlat bb7b5f51de
Merge pull request #77 from storytold/fix/65-drag-refresh
Fix #65: the viewer keeps up while a layer is dragged
2026-10-06 00:16:25 +07:00
bflatastic 1b8780f6da Fix #65: the viewer keeps up while a layer is dragged
While paused, the app prefetched the frames around the current one as soon as the viewer's
frame finished. During a drag every step changes the project, so those frames were stale at the
next step, and they held the CPU and GPU the next viewer frame needed: on the demo project a
drag step took 102 ms on average to show (up to 413 ms) though a frame renders in 20 ms. Paused
prefetch now waits until edits pause for 0.3 s: 16 ms a step (11 ms render).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 00:16:04 +07:00
BFlat 75640044b8
Merge pull request #76 from storytold/fix/64-double-axes
Fix #64: one 3D compass in the viewer
2026-10-05 23:49:03 +07:00
bflatastic 77c056a8db Fix #64: one 3D compass in the viewer
Two implementations of Show 3D Reference Axes had landed (M13.5 and M14.5) and both drew,
offset by a few pixels: lines, and arrows. Keep the arrowed one (perspective-correct, shown
while a 3D layer is active) and test that only one is drawn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 23:48:41 +07:00
BFlat 0dcbe325ab
Merge pull request #74 from storytold/fix/63-viewer-scale
Fix #63: scaling a layer by its handles follows the pointer
2026-10-05 23:44:30 +07:00
bflatastic 782a67f228 Fix #63: scaling a layer by its handles follows the pointer
The handle drag rescaled incrementally from the current scale every frame: errors built up,
edge handles scaled both axes, and once a factor reached ~0 the drag could never bring it
back. It also tested the handle at the point where egui recognised the drag (past the drag
threshold), so a quick grab often missed the handle and moved the layer instead.

Now, as in After Effects, the drag scales about the anchor point relative to the layer as the
drag began, so the grabbed point stays under the pointer: corners scale both axes, edges one,
Shift keeps the proportions, and dragging past the anchor flips. Selection-tool gestures start
at the press point.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 23:43:50 +07:00
BFlat 4373c24962
Merge pull request #73 from storytold/fix/68-open-links
Fix #68: Discord buttons and other links open the browser again
2026-10-05 23:27:45 +07:00
bflatastic 0286a49f0a Fix #68: open links in the system browser (Discord buttons, Help menu, Settings)
The desktop app built eframe without its `links` feature, so egui's open_url was a no-op
on native: the header and Home Discord buttons, Help menu links and Settings links did nothing.
Enabling it links the `webbrowser` crate (MIT OR Apache-2.0), which opens URLs with the
system browser on macOS, Windows and Linux.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 23:26:29 +07:00
Jonas Neubauer 41f5ee3aec G2 (#67): delete Project items with Delete and Backspace 2026-10-05 18:24:14 +02:00
Jonas Neubauer a0c1b60d61 G2 (#66): respect solo and visibility in viewer selection 2026-10-05 18:01:41 +02:00
Brandon Thomas e5c58600fd
Merge pull request #60 from storytold/pr/freebsd
FreeBSD: system info reports memory, CPU model and OS version
2026-10-05 11:33:17 -04:00
Brandon Thomas b70c808d60 FreeBSD: system info reports memory, CPU model and OS version
sysctl hw.physmem / hw.pagesize / vm free+inactive page counts, hw.model,
and freebsd-version, with a unit test for the sysctl parser. The FreeBSD
CI workflow itself landed with #69.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 11:33:06 -04:00
Brandon Thomas 7f3caeea70
Merge pull request #69 from storytold/pr/gpu-gl-fallback
GPU on wgpu's OpenGL backend (FreeBSD / Linux without Vulkan)
2026-10-05 11:29:02 -04:00
Brandon Thomas 3767332efd GPU compositor declines devices whose limits its pipelines exceed
egui-wgpu creates its device on GL with WebGL2's limits (no storage buffers), so the desktop
app on FreeBSD built the compositor's pipelines on it: validation errors (only logged in
release builds) and wrong frames. GpuContext::new now checks the limits its pipelines bind
(storage buffers and textures per stage, bind groups, workgroup sizes) and returns Err, which
keeps the app on the CPU. ui_gpu skips its GPU-vs-CPU comparison when that happens on GL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 10:52:04 -04:00
Brandon Thomas 6eedf8e2c5 GPU on wgpu's OpenGL backend: renderable depth target, portable shaders, serialised tests
FreeBSD's CI VM (Mesa llvmpipe over GL) failed 22 effectcraft-gpu tests:

- Advanced 3D's camera-depth target was R32Float, which GLES can't render to without
  EXT_color_buffer_float: create_render_pipeline failed validation (release builds only log
  that and draw nothing). The target is now R32Uint holding the f32 bits (renderable on every
  backend, lossless), and GpuContext checks the adapter's format features (render attachment,
  blendable colour) and independent blending up front; when they're missing the render
  pipelines are never built and Advanced 3D renders on the CPU.
- Signed `%` in the shaders: naga emits GLSL `%`, undefined for negative operands, which broke
  the VR effects' horizontal wrap and Cell Pattern's tiling. A shared `imod` never takes `%` of
  a negative number.
- asin/acos: Mesa's are coarse approximations (~1e-4 rad), visible in every lens/sphere/cylinder
  mapping. `asin_p`/`acos_p` (Cephes' asinf polynomial) are f32-accurate on every backend.
- "Could not lock adapter context": the GL backend has one context lock per adapter with a
  timeout, so tests driving devices concurrently panic. Every gpu test helper that hands out a
  device holds one process-wide lock for the calling test; ui_web_job_gpu's two device tests
  share a lock too.
- On GL, up to 1 % of pixels may flip across a decision boundary (division through a
  reciprocal, unfused fma); the GPU-vs-CPU comparisons allow that there only.

Also adds the FreeBSD workflow (from the FreeBSD CI PR) so this branch is checked there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 10:08:21 -04:00
BFlat 24fcbdf3d2
Merge pull request #62 from storytold/pr/release-0.3.0
Release: EffectCraft v0.3.0
2026-10-05 20:42:35 +07:00
bflatastic 044fc32a9c Release: EffectCraft v0.3.0
Bump the workspace version 0.2.0 -> 0.3.0 (cargo xtask version set 0.3.0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 20:42:11 +07:00
BFlat b9ffd889bc
Merge pull request #61 from storytold/m13-face-ml
M13.36: face tracking trained model (MediaPipe Face Landmarker, pure Rust)
2026-10-05 20:40:31 +07:00
bflatastic e43bf66775 M13.36: face tracking trained model — MediaPipe Face Landmarker on a pure-Rust TFLite interpreter
- effectcraft-segment: a FaceModel interface (find a face in a region, follow it; a Topology
  maps the model's points to the tracker's landmarks and outline), a TensorFlow Lite reader and
  interpreter (float ops, float16 weights folded; matches ai-edge-litert to ~1e-6), and
  MediaPipe Face Landmarker (Google, Apache-2.0: BlazeFace + Face Mesh V2, 3.8 MB, pinned
  SHA-256). The registry gains a task and authors per model; installed weights get a NOTICE.
- effectcraft-track: FaceTracker::new_with uses the model, falls back to the classical fitter
  when it finds no face; chin and jaw come from the outline either way so measurements agree.
- Engine: roto_models becomes models (both tasks); face.models / face.model.* commands,
  Settings ▸ Face Tracking, `wait` on select/install/download for scripts; track.mask reports
  faceModel.
- Credits: ATTRIBUTION.md lists the optional models and the Apache-2.0 specs the code follows.
- Docs: README, ROADMAP (G7), gaps, parity (TRK-3 score unchanged pending G1), agents,
  preferences, architecture, segment README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 20:39:45 +07:00
Brandon Thomas e2f027f374
docs: point agents to the shared test corpora (#59)
Real-file test oracles live in storytold/photocraft-corpus, explained in
craftrules standards/test-corpora.md. Never commit large binary fixtures
here; fetch them pinned and sha256-verified.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 08:24:13 -04:00
BFlat 4624c00308
Merge pull request #58 from storytold/m13-roto-ml
M13.35: Roto Brush trained model — swappable segment module, MobileSAM in pure Rust
2026-10-05 17:57:36 +07:00
bflatastic 174ab2b6af Merge main (gap assessment) into M13.35: keep parity numbers, record G7's decision and evidence
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:57:09 +07:00
bflatastic 7579de21c6 M13.35: Roto Brush trained model — swappable segment module, MobileSAM in pure Rust
- New L1 crate effectcraft-segment: the MaskModel interface (prompts → foreground probability),
  a registry of open-licensed models (licence must be MIT/Apache/BSD/…; URL, size, SHA-256),
  a PyTorch checkpoint reader (zip + pickle subset), dependency-free SHA-256, and MobileSAM
  (TinyViT-5M + SAM prompt encoder / two-way transformer / mask decoder) on rayon and ndarray's
  safe GEMM. Loads the official mobile_sam.pt as published; checked against candle
  (embedding 0.3% RMS, 99.9% equal mask pixels); ~0.7 s/frame encoder at 32 threads, cached
  per frame, ~0.1 s decoder.
- track::roto: segment_with / propagate_with prompt a model with the strokes or the flow-warped
  matte (box, core points, mask); its probability is a strong, unblurred prior for the same
  graph cut (strokes stay hard constraints); a failing model or one disagreeing with the flow
  (IoU < 0.5) is ignored. Prior gets a weight.
- effects::roto: Version 2.0 / 3.0 use the model chosen in Settings (set_model / model_for);
  the model id is part of the chain seed.
- engine: roto_models (roto.models, roto.model.select / install / download / remove), weights in
  the models folder (host::config_dir()/models, shared by the app, CLI and MCP), background
  loading, downloads through the system curl, verified; Settings ▸ Roto Brush page (prefs
  roto.model). host::config_dir moved from the desktop app.
- Tests: segment (kernels vs direct sums, checkpoint reader, SHA-256 vectors, registry), track
  (fake model: prior shapes the cut, failing / disagreeing models ignored; real weights with
  EFFECTCRAFT_MOBILESAM: base IoU 0.989, propagated ≥ 0.980 vs classic 0.973), engine (registry,
  verification, selection; real install + load), MCP.
- Docs: README, ROADMAP, parity (MSK-4 done: 90/2/0), preferences, agents, architecture,
  crates/segment/README.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 17:55:02 +07:00
Brandon Thomas aa4656b543 Docs: honest gap assessment (docs/gaps.md), roadmap workstreams G1–G9, README status
parity.md's ≈99% measures breadth only. docs/gaps.md assesses real-world
readiness by dimension (fidelity, .aep interop, stability, platform
reliability, performance, media, AI tools, maturity) with evidence, and
sets prioritised workstreams with done criteria. ROADMAP, README, parity.md,
AGENTS.md and CLAUDE.md point agents at it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 06:32:19 -04:00
BFlat 11d4885764
Merge pull request #57 from storytold/m13-audio-scrub
M13.34: audio scrubbing (Ctrl/Cmd-drag the current time)
2026-10-05 15:19:12 +07:00
bflatastic 5afc25d0d8 M13.34: audio scrubbing (Ctrl/Cmd-drag the current time)
- audio::AudioScrub: an open output that plays a faded snippet (one frame, 30–100 ms) of the
  comp's mix at each new frame, replacing what is still queued; holding still repeats nothing.
  The mixing is shared with the preview feeder (mix_block); AudioFeed::clear.
- The Timeline ruler: Ctrl/Cmd-drag scrubs audio (EffectcraftApp::scrub_audio opens the output
  through the audio device hook on first use). The Audio panel meters follow the snippets; an
  idle second closes the output.
- playback.scrubAudio {time?} for agents.
- Tests: ui_audio_scrub (snippet length, fades, same-frame no-op, replace not pile up, silent
  comps, Cmd-drag in the ruler, idle close) with a fake device. Docs: parity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 15:18:54 +07:00
BFlat 9607be471b
Merge pull request #56 from storytold/m13-nested-markers
M13.33: nested comp markers on the precomp layer bar
2026-10-05 15:12:22 +07:00
bflatastic f013bdd8d7 M13.33: nested comp markers on the precomp layer bar
- commands::markers::nested_markers: a precomp layer's comp markers at their times in the
  containing comp (start and stretch in closed form; with time remapping or a stretch over
  protected regions, the first frame whose source time reaches them), within In–Out.
- markers.nested {layer}: the same list for agents.
- Timeline: drawn on the precomp bar as outlined read-only markers (positions cached per
  revision), hover names them, double-click opens the nested comp at the marker; automation id
  timeline.layer.<id>.nestedMarker.<i>.
- Tests: engine (start, stretch, trim, time remap, non-precomp error), UI (draw, label, open).
  Docs: parity, control protocol.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 15:12:06 +07:00
BFlat 1ac55c33a3
Merge pull request #55 from storytold/m13-new-viewer
M13.32: View ▸ New Viewer — several Composition viewers
2026-10-05 15:00:30 +07:00
bflatastic 97390f15fb M13.32: View ▸ New Viewer — several Composition viewers
- PanelKind::Viewer(n): more Composition viewers (panels::viewers). One is active (the full
  interactive viewer, showing the active comp); the others show their own comp's frame at that
  comp's current time (CPU or GPU frames, their own textures) and become active on a click or
  their tab, their comp becoming the active comp.
- view.newViewer (View menu, the Composition panel menu, Alt+Shift+N): a new viewer of the
  active comp next to the one in use, which is locked. Opening a comp while the active viewer
  is locked shows it in an unlocked viewer, or a new one when every viewer is locked. Closing
  the active viewer hands over to another with its comp. Tabs are named after their comp, with
  their own lock and swatch; Preview full screen maximizes the active viewer.
- UI state: viewers (comp per viewer) and active_viewer (serde).
- Tests: ui_viewers (lock on New Viewer, routing, click to activate, all locked → new viewer,
  close hand-over, tab titles; an ignored snapshot). The bounded-drag frames test asserts what
  holds when freed addresses are reused. Docs: parity, control protocol.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 15:00:11 +07:00
BFlat 6a5abad753
Merge pull request #54 from storytold/m4-content-keyed-frames
M4.14: content-keyed RAM preview frames (M4 done); proxies in the disk cache key
2026-10-05 14:39:33 +07:00
bflatastic 26c714db7f M4.14: content-keyed RAM preview frames; proxies in the disk cache's content key
- FrameKey's identity is the comp's content (frames::comp_content), not the project revision:
  an edit keeps the cached frames of comps it doesn't touch and undo finds the frames of the
  state it returns to. The identity hashes the addresses of the comps a comp draws (~0.1 ms on
  the 1500-layer test project; the full content hash took ~117 ms), by value the footage, solids,
  proxies and project settings; with expressions, every item. Frames::content_of keeps a
  project snapshot per identity with frames (or in flight), so Arc::make_mut always copies an
  edited comp and its address can't be reused meanwhile; past 64 identities the least recently
  used go with their frames (long drags).
- drop_stale is gone (frames of other contents are evicted least-recently-shown first).
- disk_cache::comp_content_key covers items' proxies and Use Proxy (it served stale frames).
- Tests: frames (content keys, edits elsewhere and undo, bounded drags, in-place edit safety),
  ui_ram_preview (end to end), the disk cache proxy key. Docs: roadmap (M4 done), parity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 14:39:17 +07:00
bflatastic 4f7723a7b6 M13.31: Home size column test uses an unambiguous size (12.5 KB rounds to even)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 14:33:44 +07:00
BFlat 0f1cad8bbd
Merge pull request #53 from storytold/ci-packaging-lint
CI: fix Packaging lint (SC2015 in release.yml)
2026-10-05 13:52:21 +07:00
427 changed files with 14199 additions and 1733 deletions

48
.github/workflows/freebsd.yml vendored Normal file
View File

@ -0,0 +1,48 @@
# Builds and tests the workspace on FreeBSD, inside a FreeBSD VM on a Linux runner
# (GitHub has no FreeBSD runners). GPU tests skip themselves when the VM has no adapter.
name: FreeBSD
on:
push:
branches: [main]
pull_request:
paths:
- Cargo.toml
- Cargo.lock
- "**/Cargo.toml"
- .github/workflows/freebsd.yml
workflow_dispatch:
concurrency:
group: freebsd-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
freebsd:
name: FreeBSD 14 (build + test)
runs-on: ubuntu-latest
timeout-minutes: 300
steps:
- uses: actions/checkout@v4
- name: Build and test in a FreeBSD VM
uses: vmactions/freebsd-vm@c46abacb49f09938ca4e1702d15d836285d694cc # v1
with:
release: "14.3"
usesh: true
cpu: 4
mem: 12288
# Nothing to bring back; copying the target dir out of the VM fails on large builds.
copyback: false
prepare: |
pkg install -y git curl pkgconf cmake python3 libxkbcommon wayland libX11 libXcursor libXrandr libXi libxcb mesa-libs vulkan-loader gtk3 fontconfig freetype2 alsa-lib
curl -fsSL https://sh.rustup.rs | sh -s -- -y --profile minimal
run: |
. "$HOME/.cargo/env"
# The checkout is synced in under another owner; let tests that call git read it.
git config --global --add safe.directory "*"
rustc --version
CARGO_PROFILE_DEV_DEBUG=0 cargo build --workspace --all-targets --locked
CARGO_PROFILE_DEV_DEBUG=0 cargo test --workspace --locked

View File

@ -47,6 +47,10 @@ jobs:
- name: XML, desktop entry, AppStream, Flatpak manifest
run: |
xmllint --noout packaging/windows/effectcraft.wxs packaging/macos/*.plist* packaging/linux/*.xml*
# Windows Installer needs Icon ids ending in .ico or .exe, or shortcuts show a blank icon.
if grep -oE '<Icon Id="[^"]*"' packaging/windows/effectcraft.wxs | grep -vE '\.(ico|exe)"$'; then
echo "::error::Icon Id must end in .ico or .exe"; exit 1
fi
desktop-file-validate packaging/linux/ai.storyteller.effectcraft.desktop
sed -e 's/@VERSION@/0.0.0/' -e 's/@DATE@/2026-01-01/' packaging/linux/ai.storyteller.effectcraft.metainfo.xml.in >"$RUNNER_TEMP/ai.storyteller.effectcraft.metainfo.xml"
appstreamcli validate --no-net --explain "$RUNNER_TEMP/ai.storyteller.effectcraft.metainfo.xml"

View File

@ -112,6 +112,9 @@ jobs:
include:
- { arch: x64, target: x86_64-pc-windows-msvc }
- { arch: x86, target: i686-pc-windows-msvc }
# Windows on ARM: cross-compiled on the x64 runner, so signing and WiX run as above.
# .github/workflows/windows-arm64.yml installs and runs it on ARM64 hardware.
- { arch: arm64, target: aarch64-pc-windows-msvc }
env:
EFFECTCRAFT_VERSION: ${{ needs.version.outputs.version }}
EFFECTCRAFT_BUILD_DATE: ${{ needs.version.outputs.date }}

95
.github/workflows/windows-arm64.yml vendored Normal file
View File

@ -0,0 +1,95 @@
# Windows on ARM: package the ARM64 release the way release.yml does (cross-compiled on an x64
# runner, unsigned here), then install that MSI on a Windows 11 ARM64 runner, check the installed
# binaries are ARM64, run the CLI natively, and uninstall. (A native ARM64 `cargo test` job is left
# out for now: PrintCraft saw an intermittent access violation in egui UI tests there.)
name: Windows ARM64
on:
push:
branches: [main]
pull_request:
paths:
- Cargo.toml
- Cargo.lock
- "**/Cargo.toml"
- "**/build.rs"
- .github/workflows/windows-arm64.yml
- packaging/windows/**
schedule:
- cron: "41 4 * * 1" # weekly, so drift in the runner images shows up
workflow_dispatch:
concurrency:
group: windows-arm64-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
jobs:
package:
name: package (cross-compiled on x64)
runs-on: windows-latest
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@89b12181fb390509a0842a86cc55eeb8eb928c1d # stable
with:
targets: aarch64-pc-windows-msvc
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
key: windows-arm64-package
- name: Install WiX
shell: pwsh
run: dotnet tool install --global wix --version 5.0.2
- name: Build and package (unsigned)
shell: pwsh
env:
DIST: ${{ github.workspace }}\dist\arm64
run: ./packaging/windows/package.ps1 -Arch arm64
- uses: actions/upload-artifact@v4
with:
name: effectcraft-windows-arm64
path: dist/arm64/*
if-no-files-found: error
retention-days: 7
install:
name: install and run on ARM64
needs: package
runs-on: windows-11-arm
timeout-minutes: 30
steps:
- uses: actions/download-artifact@v4
with:
name: effectcraft-windows-arm64
path: dist
- name: Install the MSI, check it, run it, uninstall it
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
$msi = (Get-ChildItem dist/*.msi | Select-Object -First 1).FullName
$p = Start-Process msiexec.exe -ArgumentList '/i', "`"$msi`"", '/qn', '/l*v', 'install.log' -Wait -PassThru
if ($p.ExitCode -ne 0) { Get-Content install.log -Tail 60; throw "msiexec /i exited $($p.ExitCode)" }
$dir = Join-Path $env:ProgramFiles 'EffectCraft'
foreach ($exe in 'effectcraft.exe', 'effectcraft-cli.exe') {
$path = Join-Path $dir $exe
if (-not (Test-Path $path)) { throw "$exe not installed in $dir" }
$bytes = [System.IO.File]::ReadAllBytes($path)
$machine = [BitConverter]::ToUInt16($bytes, [BitConverter]::ToInt32($bytes, 0x3C) + 4)
if ($machine -ne 0xAA64) { throw "$exe is for machine 0x$('{0:X}' -f $machine), not ARM64" }
Write-Output "ok $path (ARM64)"
}
# The installed CLI runs natively on ARM64.
$cli = Join-Path $dir 'effectcraft-cli.exe'
$out = & $cli --version 2>&1 | Out-String
if ($LASTEXITCODE -ne 0) { Write-Output $out; throw "effectcraft-cli --version exited $LASTEXITCODE" }
Write-Output "ok effectcraft-cli --version: $($out.Trim())"
$p = Start-Process msiexec.exe -ArgumentList '/x', "`"$msi`"", '/qn' -Wait -PassThru
if ($p.ExitCode -ne 0) { throw "msiexec /x exited $($p.ExitCode)" }
if (Test-Path (Join-Path $dir 'effectcraft.exe')) { throw 'uninstall left effectcraft.exe behind' }
Write-Output 'ok uninstalled'

2
.gitignore vendored
View File

@ -1,6 +1,6 @@
*~
plan/
/target
target/
/dist
/log
.DS_Store

View File

@ -105,8 +105,15 @@ reproduce an image (for example, point lists traced from someone else's icon).
See `CLAUDE.md`: pure Rust, dependency layering (`cargo xtask layers`), exact `Tick` time, everything is
a command, everything is agent-drivable, and the quality gates (`cargo xtask ci`) before every commit.
Shared real-file test corpora (Photoshop-authored PSDs, etc.) live in
[`storytold/photocraft-corpus`](https://github.com/storytold/photocraft-corpus), explained in
[craftrules `standards/test-corpora.md`](https://github.com/storytold/craftrules/blob/main/standards/test-corpora.md).
Never commit large binary fixtures to this repo; fetch them pinned by commit and sha256-verified,
as PhotoCraft does with `cargo xtask corpus`.
## See also
- [docs/gaps.md](docs/gaps.md): where EffectCraft falls short of After Effects and the prioritised workstreams (G1–G9); read it before choosing work
- [CONTRIBUTING.md](CONTRIBUTING.md): setup, gates, commits, how to add things
- [docs/architecture.md](docs/architecture.md): layers, data model, commands, pipeline
- [docs/testing.md](docs/testing.md): oracle tests, criteria, benchmarks

View File

@ -70,6 +70,34 @@ Screenshots of EffectCraft itself, rendered headlessly, showing only procedurall
(sidecars `.attribution`).
- Example effect plug-in: `examples/plugins/posterize-bands` (plug-in API v1, WebAssembly).
## Trained models (optional downloads, not bundled)
The repository and the installers contain no model weights. These models are downloaded only when
a user asks (Settings ▸ Roto Brush, Settings ▸ Face Tracking, or the `roto.model.*` /
`face.model.*` commands), unmodified from their authors' official location, checked against a
pinned SHA-256 and stored in the user's `models` folder with a `<file>.NOTICE.txt` naming their
authors, licence and source. EffectCraft runs them with its own Rust implementation and is not
affiliated with or endorsed by their authors.
| Model | Authors | Source | Licence |
|---|---|---|---|
| MobileSAM (`mobile_sam.pt`) | Chaoning Zhang et al., building on Meta AI's Segment Anything and Microsoft's TinyViT | https://github.com/ChaoningZhang/MobileSAM | Apache-2.0 |
| MediaPipe Face Landmarker (`face_landmarker.task`: BlazeFace short-range detector, Face Mesh V2, blendshapes) | Google LLC | https://developers.google.com/edge/mediapipe/solutions/vision/face_landmarker (model cards: https://storage.googleapis.com/mediapipe-assets/MediaPipe%20BlazeFace%20Model%20Card%20(Short%20Range).pdf, https://storage.googleapis.com/mediapipe-assets/Model%20Card%20MediaPipe%20Face%20Mesh%20V2.pdf) | Apache-2.0 |
## Formats and parameters from other open-source projects
Written in Rust for EffectCraft; no code is copied. They follow published specifications and
configuration from these Apache-2.0 projects:
- `crates/segment/src/tflite.rs` reads TensorFlow Lite files using the field layout of TensorFlow
Lite's `schema.fbs` (The TensorFlow Authors, https://github.com/tensorflow/tensorflow,
Apache-2.0).
- `crates/segment/src/mediapipe.rs` follows MediaPipe's published face landmarker graph: the SSD
anchor layout, detection decoding, crop sizes and the face-mesh point indices it uses (Google
LLC, https://github.com/google-ai-edge/mediapipe, Apache-2.0).
- `crates/segment/src/mobilesam.rs` follows the MobileSAM, TinyViT and Segment Anything
architectures and their Apache-2.0 reference code.
## First-party brand marks
The ArtCraft name, wordmark and logos are trademarks of the ArtCraft Team. They are not open source and are not

View File

@ -6,7 +6,8 @@ EffectCraft is a clean-room, open-source, pure-Rust motion graphics and visual e
`plan/` is maintainer-local (gitignored). Public equivalents: [`ROADMAP.md`](ROADMAP.md), [`docs/`](docs/).
1. Read `plan/STATUS.md` (current milestone, next task, running agents, blockers).
2. Read the task in `plan/execution-plan.md` §3, the relevant section of `plan/architecture.md`, and the crate README/docs you touch. Feature ids: `plan/aftereffects/feature-catalog.md`; effects: `plan/aftereffects/effects-list.md`; UI reference: `plan/aftereffects/README.md`.
3. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask.
3. Read [`docs/gaps.md`](docs/gaps.md): the honest assessment of where we fall short and the prioritised workstreams. Fidelity, reliability and stability (G1–G3) come before new features.
4. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask.
## Non-negotiables
**Read [`AGENTS.md`](AGENTS.md) first; its rules override everything here.** No Adobe assets, every asset licensed + attributed (`cargo xtask assets`).

175
Cargo.lock generated
View File

@ -1134,7 +1134,7 @@ dependencies = [
[[package]]
name = "effectcraft"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"cpal",
"effectcraft-engine",
@ -1155,7 +1155,7 @@ dependencies = [
[[package]]
name = "effectcraft-automation"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-engine",
"effectcraft-host",
@ -1167,14 +1167,14 @@ dependencies = [
[[package]]
name = "effectcraft-av1enc"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"filmcraft-av1",
]
[[package]]
name = "effectcraft-cli"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"cpu-time",
"effectcraft-automation",
@ -1190,7 +1190,7 @@ dependencies = [
[[package]]
name = "effectcraft-color"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"proptest",
"serde",
@ -1198,13 +1198,14 @@ dependencies = [
[[package]]
name = "effectcraft-effects"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-color",
"effectcraft-geom",
"effectcraft-keyframe",
"effectcraft-project",
"effectcraft-raster",
"effectcraft-segment",
"effectcraft-track",
"log",
"rayon",
@ -1214,7 +1215,7 @@ dependencies = [
[[package]]
name = "effectcraft-engine"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-color",
"effectcraft-effects",
@ -1229,6 +1230,7 @@ dependencies = [
"effectcraft-psd",
"effectcraft-raster",
"effectcraft-render",
"effectcraft-segment",
"effectcraft-svg",
"effectcraft-text",
"effectcraft-time",
@ -1246,7 +1248,7 @@ dependencies = [
[[package]]
name = "effectcraft-export"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-av1enc",
"effectcraft-color",
@ -1275,7 +1277,7 @@ dependencies = [
[[package]]
name = "effectcraft-expr"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"boa_engine",
"effectcraft-color",
@ -1292,7 +1294,7 @@ dependencies = [
[[package]]
name = "effectcraft-geom"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"proptest",
"serde",
@ -1300,7 +1302,7 @@ dependencies = [
[[package]]
name = "effectcraft-gpu"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"bytemuck",
"effectcraft-color",
@ -1312,21 +1314,24 @@ dependencies = [
"effectcraft-render",
"effectcraft-text",
"effectcraft-time",
"js-sys",
"log",
"pollster 0.4.0",
"wasm-bindgen",
"web-time",
"wgpu",
]
[[package]]
name = "effectcraft-hevcenc"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"filmcraft-hevc",
]
[[package]]
name = "effectcraft-host"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-engine",
"effectcraft-export",
@ -1348,7 +1353,7 @@ dependencies = [
[[package]]
name = "effectcraft-interchange"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-color",
"effectcraft-keyframe",
@ -1367,7 +1372,7 @@ dependencies = [
[[package]]
name = "effectcraft-keyframe"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-color",
"effectcraft-geom",
@ -1379,7 +1384,7 @@ dependencies = [
[[package]]
name = "effectcraft-lottie"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-color",
"effectcraft-effects",
@ -1396,7 +1401,7 @@ dependencies = [
[[package]]
name = "effectcraft-media"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-model",
"effectcraft-pdf",
@ -1421,7 +1426,7 @@ dependencies = [
[[package]]
name = "effectcraft-model"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-geom",
"image",
@ -1432,14 +1437,14 @@ dependencies = [
[[package]]
name = "effectcraft-opusenc"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"filmcraft-opus",
]
[[package]]
name = "effectcraft-path"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-geom",
"effectcraft-keyframe",
@ -1451,7 +1456,7 @@ dependencies = [
[[package]]
name = "effectcraft-pdf"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-raster",
"effectcraft-svg",
@ -1465,7 +1470,7 @@ dependencies = [
[[package]]
name = "effectcraft-plugin"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-effects",
"effectcraft-keyframe",
@ -1477,7 +1482,7 @@ dependencies = [
[[package]]
name = "effectcraft-project"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-color",
"effectcraft-geom",
@ -1490,14 +1495,14 @@ dependencies = [
[[package]]
name = "effectcraft-psd"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"miniz_oxide 0.8.9",
]
[[package]]
name = "effectcraft-raster"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-color",
"effectcraft-geom",
@ -1506,7 +1511,7 @@ dependencies = [
[[package]]
name = "effectcraft-render"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-color",
"effectcraft-effects",
@ -1528,7 +1533,7 @@ dependencies = [
[[package]]
name = "effectcraft-script"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"boa_engine",
"effectcraft-engine",
@ -1541,9 +1546,18 @@ dependencies = [
"serde_json",
]
[[package]]
name = "effectcraft-segment"
version = "0.3.1"
dependencies = [
"miniz_oxide 0.8.9",
"ndarray",
"rayon",
]
[[package]]
name = "effectcraft-svg"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-geom",
"effectcraft-path",
@ -1555,7 +1569,7 @@ dependencies = [
[[package]]
name = "effectcraft-text"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-keyframe",
"harfrust",
@ -1567,7 +1581,7 @@ dependencies = [
[[package]]
name = "effectcraft-time"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"proptest",
"serde",
@ -1575,9 +1589,10 @@ dependencies = [
[[package]]
name = "effectcraft-track"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-raster",
"effectcraft-segment",
"rayon",
"serde",
"serde_json",
@ -1586,7 +1601,7 @@ dependencies = [
[[package]]
name = "effectcraft-ui-egui"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-color",
"effectcraft-engine",
@ -1614,11 +1629,11 @@ dependencies = [
[[package]]
name = "effectcraft-vp9enc"
version = "0.2.0"
version = "0.3.1"
[[package]]
name = "effectcraft-web"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"effectcraft-engine",
"effectcraft-export",
@ -1732,6 +1747,7 @@ dependencies = [
"raw-window-handle",
"smithay-clipboard",
"web-time",
"webbrowser",
"winit",
]
@ -2313,6 +2329,15 @@ version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aa9a19cbb55df58761df49b23516a86d432839add4af60fc256da840f66ed35b"
[[package]]
name = "form_urlencoded"
version = "1.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cb4cb245038516f5f85277875cdaa4f7d2c9a0fa0468de06ed190163b1581fcf"
dependencies = [
"percent-encoding",
]
[[package]]
name = "futures-concurrency"
version = "7.7.1"
@ -2719,6 +2744,27 @@ dependencies = [
"zerovec",
]
[[package]]
name = "idna"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3b0875f23caa03898994f6ddc501886a45c7d3d62d04d2d90788d47be1b1e4de"
dependencies = [
"idna_adapter",
"smallvec",
"utf8_iter",
]
[[package]]
name = "idna_adapter"
version = "1.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cb68373c0d6620ef8105e855e7745e18b0d00d3bdb07fb532e434244cdb9a714"
dependencies = [
"icu_normalizer",
"icu_properties",
]
[[package]]
name = "image"
version = "0.25.10"
@ -3050,6 +3096,16 @@ dependencies = [
"libc",
]
[[package]]
name = "matrixmultiply"
version = "0.3.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f607c237553f086e7043417a51df26b2eb899d3caff94e6a67592ff992fedc7"
dependencies = [
"autocfg",
"rawpointer",
]
[[package]]
name = "memchr"
version = "2.8.3"
@ -3167,6 +3223,21 @@ dependencies = [
"thiserror 2.0.21",
]
[[package]]
name = "ndarray"
version = "0.17.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "520080814a7a6b4a6e9070823bb24b4531daac8c4627e08ba5de8c5ef2f2752d"
dependencies = [
"matrixmultiply",
"num-complex",
"num-integer",
"num-traits",
"portable-atomic",
"portable-atomic-util",
"rawpointer",
]
[[package]]
name = "ndk"
version = "0.9.0"
@ -4169,6 +4240,12 @@ dependencies = [
"objc2-quartz-core 0.3.2",
]
[[package]]
name = "rawpointer"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "60a357793950651c4ed0f3f52338f53b2f809f32d83a07f72909fa13e4c6c1e3"
[[package]]
name = "rayon"
version = "1.12.0"
@ -5187,6 +5264,18 @@ version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4ac048d71ede7ee76d585517add45da530660ef4390e49b098733c6e897f254"
[[package]]
name = "url"
version = "2.5.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff67a8a4397373c3ef660812acab3268222035010ab8680ec4215f38ba3d0eed"
dependencies = [
"form_urlencoded",
"idna",
"percent-encoding",
"serde",
]
[[package]]
name = "utf16_iter"
version = "1.0.5"
@ -5575,6 +5664,22 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "webbrowser"
version = "1.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "62c35be770821a214dbc362fc26908c853e776c0004294d0b10b8a6bad582f94"
dependencies = [
"jni 0.22.4",
"log",
"ndk-context",
"objc2 0.6.4",
"objc2-app-kit 0.3.2",
"objc2-foundation 0.3.2",
"url",
"web-sys",
]
[[package]]
name = "weezl"
version = "0.1.12"
@ -6318,7 +6423,7 @@ checksum = "e450f9b2ed1dff33c94c12589a87338689467b9c4f5d8a5710bd09a847d2c8a7"
[[package]]
name = "xtask"
version = "0.2.0"
version = "0.3.1"
dependencies = [
"serde_json",
]

View File

@ -4,7 +4,7 @@ members = ["crates/*", "apps/*", "xtask"]
default-members = ["crates/*", "apps/*"]
[workspace.package]
version = "0.2.0"
version = "0.3.1"
edition = "2024"
license = "MIT OR Apache-2.0"
rust-version = "1.95"
@ -22,6 +22,7 @@ effectcraft-project = { path = "crates/project" }
effectcraft-text = { path = "crates/text" }
effectcraft-effects = { path = "crates/effects" }
effectcraft-track = { path = "crates/track" }
effectcraft-segment = { path = "crates/segment" }
effectcraft-model = { path = "crates/model" }
effectcraft-render = { path = "crates/render" }
effectcraft-gpu = { path = "crates/gpu" }

View File

@ -85,7 +85,9 @@ then to go further in a few places where it matters to us:
The panels, menus and shortcuts follow After Effects, so your muscle memory carries over:
Project, Composition, Timeline, Effect Controls, Properties, Effects & Presets, Character,
Paragraph, Align, Info, Preview, Audio and the Render Queue, docked the way you expect.
Paragraph, Align, Info, Preview, Audio and the Render Queue, docked the way you expect. Drag
footage, comps and effects between them: onto the comp viewer, where they land under the pointer,
or into the Timeline, between layers and at the time you point to.
- **Layers of every kind:** solids, shapes, text, footage, nested compositions, nulls,
adjustment layers, cameras and lights; parenting, track mattes, all 38 blend modes, motion blur.
@ -97,8 +99,8 @@ Paragraph, Align, Info, Preview, Audio and the Render Queue, docked the way you
- **Shapes and masks:** shape layers with trim paths, repeaters, round corners, offset, zig zag,
twist, wiggle, merge paths and gradient strokes; masks drawn with the pen tool, with modes,
feather, expansion and vertex editing in the viewer.
- **Text:** point and paragraph text with real shaping, the Character and Paragraph panels, and
text animators with range selectors.
- **Text:** point and paragraph text with real shaping in any font installed on your machine, the
Character and Paragraph panels, and text animators with range selectors.
- **Expressions:** JavaScript with the After Effects object model (`wiggle`, `loopOut`,
`thisComp.layer("…")`, vector maths on arrays), an inline editor and the pick-whip.
- **Timeline like you know it:** twirl layers open to Transform, masks, effects and the rest; drag
@ -179,7 +181,14 @@ See [docs/agents.md](docs/agents.md) and [docs/control-protocol.md](docs/control
## Get started
You need [Rust](https://rustup.rs/) 1.95 or newer.
Installers for each version are on the [Releases](https://github.com/storytold/effectcraft/releases)
page: a universal macOS app; Windows MSIs and portable zips for x64, x86 and ARM64; and Linux
AppImage, deb, rpm and tar.gz for x86_64 and aarch64. The Windows ARM64 build runs natively on
Windows on ARM, without x64 emulation. CI installs that MSI on Windows 11 ARM64 hardware, checks
both programs are ARM64 and runs the command-line tool there, but it doesn't open the app's window
or run the test suite natively on ARM64 yet, so please report anything that behaves differently.
To build it yourself you need [Rust](https://rustup.rs/) 1.95 or newer.
```sh
git clone https://github.com/storytold/effectcraft
@ -195,13 +204,25 @@ tests, layering, asset attribution and the WebAssembly build). See [CONTRIBUTING
## Where it stands
EffectCraft is young. The core of After Effects is in place and gets deeper every week: what is
done and what comes next is in the [ROADMAP](ROADMAP.md). Still to come: learned models for
Roto Brush and face tracking (both use classical methods today), GPU versions of the remaining
CPU-only effects, the Advanced 3D Extended Viewer, and shared-memory threads in the web build. The
web build ([docs/web.md](docs/web.md)) runs the full app in the browser, with renders, analyses
and viewer frames in Web Workers. Projects are saved as `.ecproj`, readable versioned JSON; After
Effects `.aep` files cannot be opened.
EffectCraft is young: its first commit was on 1 October 2026. Nearly every After Effects feature
exists, including all of its effects, but existing is not the same as behaving exactly like After
Effects, and we haven't measured that yet. Today it's a good place to try things and to tell us
what breaks. It isn't yet a replacement for After Effects on client work. In particular:
- **After Effects projects can't be opened.** Projects are saved as `.ecproj`, readable versioned
JSON, but EffectCraft can't open `.aep` / `.aepx` files, and After Effects plug-ins don't run.
- **Behaviour still differs from After Effects in places.** Nothing yet compares our renders with
After Effects automatically, so please report differences.
- **macOS is the most tested platform.** Linux and Windows users have hit basic interaction
bugs, which we're fixing first.
- **The trained models are new.** Roto Brush can use MobileSAM (an optional 40 MB download in
Settings ▸ Roto Brush) and face tracking can use Google's MediaPipe Face Landmarker (an optional
3.8 MB download in Settings ▸ Face Tracking). Neither is compared with After Effects yet.
The [ROADMAP](ROADMAP.md) has the plan and [docs/gaps.md](docs/gaps.md) the full, honest
assessment. The web build ([docs/web.md](docs/web.md)) runs the full app in the browser.
[Bug reports](https://github.com/storytold/effectcraft/issues) are the most useful thing you can
send us right now.
## How it's made

View File

@ -1,5 +1,7 @@
# Roadmap
- 2026-10-06: Settings ▸ General ▸ Language persists English/Japanese menu labels (`general.language`); native UI reuses installed Japanese font fallback, with no bundled CJK font. Dialog and panel contents remain English.
EffectCraft aims to do what After Effects does, with the same panels, menus and behaviour, written
from scratch in Rust. This is where it stands. The milestones overlap; several are worked on at
once.
@ -10,7 +12,7 @@ once.
| M1 | Keyframes: temporal and spatial interpolation, Easy Ease, roving, velocity | Done |
| M2 | Compositing: 38 blend modes, track mattes, parenting, adjustment layers | Done |
| M3 | Project operations: settings dialogs, layer commands, `.ecproj`, undo, After Effects menu bar | Done (unsaved-changes prompts, Solid / Layer Settings, lock-aware layer commands, safe project files; View ▸ New Viewer still opens only the one locked viewer) |
| M4 | Preview: precomps, motion blur, cached playback | Mostly done (Preserve frame rate / resolution when nested, one motion blur gate, RAM preview fixes, Cache Frames When Idle, motion blur of collapsed precomps and of animated shape / text content; left: content-keyed RAM preview frames) |
| M4 | Preview: precomps, motion blur, cached playback | Done (Preserve frame rate / resolution when nested, one motion blur gate, RAM preview fixes, Cache Frames When Idle, motion blur of collapsed precomps and of animated shape / text content, content-keyed RAM preview frames) |
| M5 | Timeline depth: graph editor, keyframe clipboard and dialogs, time remapping, pick-whips, expression editor | Done |
| M6 | Shapes, masks and footage: shape operators, masks and the pen tool, video and image import | Done |
| M7 | 3D: 3D layers, cameras, lights, shadows, depth of field, 3D views and camera tools | In review |
@ -19,20 +21,48 @@ once.
| M10 | Export: render queue, H.264, ProRes, image sequences, GIF, audio | Done |
| M11 | Animation tools: audio playback, meters and waveforms, Lottie import and export, presets, Motion Sketch, Wiggler, Smoother | Done |
| M12 | Performance: layer cache, parallel and GPU compositing, disk cache, motion tracking (done); GPU versions of the remaining CPU-only effects | Mostly done |
| M13 | Puppet tools, paint, Roto Brush, motion tracking, a plug-in API, Timeline depth | Done (Roto Brush and face tracking use classical models; learned models are still to come) |
| M13 | Puppet tools, paint, Roto Brush, motion tracking, a plug-in API, Timeline depth | Done (trained models, open source, downloaded on demand, in a swappable model module: MobileSAM for Roto Brush 2.0 / 3.0, MediaPipe Face Landmarker for face tracking) |
| M15 | The web app (WebAssembly, WebGPU) | Done: browser storage, Web Audio, renders and analyses in Web Workers, viewer frames and GPU effects in frame workers with their own WebGPU devices, the disk cache in the Origin Private File System, a storage manager, offline install ([docs/web.md](docs/web.md)) |
| M14 | Built for agents: MCP server, command-line tool, control channel; Settings, keyboard shortcut editor, auto-save and crash recovery ([docs/preferences.md](docs/preferences.md)) | Done |
## How far from full parity
Measured feature by feature in [docs/parity.md](docs/parity.md) (4 October 2026):
There are two honest answers, and they are far apart. The full assessment, with the evidence, is
in [docs/gaps.md](docs/gaps.md).
- **≈ 99% of After Effects' features, weighted by importance**, counting partial features as half
done (≈ 99.8% with per-feature fractions): 89 of 92 features done, 3 partial, none missing; every
essential (P0) feature done; all 306 effects implemented in full, 280 of them on the GPU.
- **≈ 8.5 agent-hours of work remain**: about **2–3 hours of wall-clock time** with five Claude
Opus 5.5 agents in parallel, ≈ 1 hour without learned models for Roto Brush and face tracking.
- What is left: GPU kernels for the remaining visual effects, and learned-model quality for Roto
Brush and face tracking; in previews, content-keyed RAM preview frames ([parity.md](docs/parity.md#highest-value-gaps-in-order)).
- **Breadth: ≈ 99%.** Almost every After Effects feature exists: 89 of 92 catalogued features
done and 3 partial, all 306 effects implemented, 280 of them on the GPU
([docs/parity.md](docs/parity.md)). This is our own checklist, graded by the agents that built
the features.
- **Real use: ≈ 30–50% (estimated, not yet measured).** Whether someone who uses After Effects for
a living can do client work in EffectCraft. What holds it back:
- **Fidelity is unmeasured.** No test compares our output with After Effects itself, and users
are still finding behaviour bugs in features marked done.
- **After Effects projects can't be opened.** EffectCraft can't read `.aep` / `.aepx` files,
and third-party After Effects plug-ins can't run.
- **Reliability is uneven across platforms.** Early Linux users hit basic problems: viewer
panning, panel docking, drag-and-drop import. A Windows user found the font menus offered only
the bundled fonts (fixed 6 October: they list every installed font). Reports of 6 October
covered drag and drop between panels, a Render Queue menu that ignored clicks, angle
revolutions, and preview with audio and with little video memory (all fixed 7 October).
- **AI tools are new.** Roto Brush's trained model (MobileSAM, M13.35) and face tracking's
(MediaPipe Face Landmarker, M13.36) are optional downloads, not yet compared with After
Effects.
## Where we're going
In priority order (details and "done" criteria in [docs/gaps.md](docs/gaps.md)):
| | Workstream | Why |
|---|---|---|
| G1 | Measure fidelity against After Effects: a corpus of test projects rendered and sampled in both apps, scored per feature | Turns every estimate into a measurement; a feature that behaves differently is not done |
| G2 | Real-user reliability on macOS, Windows and Linux; fix every open user issue | Basic workflows must work everywhere before more features matter |
| G3 | Stability: a green gate on the current toolchain, fuzzing of every input | A crash loses people's work |
| G4 | Open After Effects projects (`.aep` / `.aepx`, pending an owner decision on clean-room scope); relinking moved footage | The biggest barrier to switching |
| G5 | Performance benchmarks at 1080p and 4K on real-world-sized projects | Unknown today |
| G6 | Media depth: encoder efficiency, camera and phone formats | Files are larger than from mature encoders; camera formats are unverified |
| G7 | Learned models: Roto Brush (M13.35: MobileSAM) and face tracking (M13.36: MediaPipe Face Landmarker), open source, optional downloads, swappable model module; next, measure them | Quality against After Effects is unmeasured |
| G8 | Plug-in ecosystem on our WebAssembly plug-in API | After Effects plug-ins can't run here |
| G9 | Localisation, accessibility, user documentation | Reach beyond English-speaking power users |
Come tell us what matters most to you on [Discord](https://discord.gg/artcraft).

View File

@ -204,6 +204,22 @@ pub async fn start(canvas_id: String) -> Result<(), JsValue> {
eframe::WebOptions::default(),
Box::new(move |cc| {
CTX.with(|c| *c.borrow_mut() = Some(cc.egui_ctx.clone()));
// The page owns this shared presentation device; worker devices retain their
// separate owned-device handlers. No browser error-scope future is blocked on.
let gpu_failures = effectcraft_ui_egui::gpu_failure::GpuFailureBridge::new(&cc.egui_ctx);
if let Some(rs) = &cc.wgpu_render_state {
let errors = gpu_failures.clone();
rs.device.on_uncaptured_error(Arc::new(move |error| {
let message = format!("uncaptured GPU error: {error}");
if errors.report(&message, false) {
log::error!("{message}");
}
}));
let lost = gpu_failures.clone();
rs.device.set_device_lost_callback(move |reason, message| {
lost.report(&format!("GPU device lost ({reason:?}): {message}"), true);
});
}
let mut session = session();
session.load_settings();
let recovery = session.begin_recovery();
@ -215,6 +231,7 @@ pub async fn start(canvas_id: String) -> Result<(), JsValue> {
}
api::set_info("restored", json!(restored));
let mut app = EffectcraftApp::new(session);
app.set_gpu_failure_bridge(gpu_failures);
app.ui.start_screen = home;
if let Some(r) = recovery.filter(|r| !restored && r.autosave.is_some()) {
app.offer_recovery(r);

View File

@ -477,8 +477,8 @@ mod tests {
a.execute("comp.new", json!({"name": "C"})).unwrap();
// Untitled: the auto-save lands in the store's file table.
let p = a.autosave_now().unwrap();
assert_eq!(p, "/EffectCraft Auto-Save/Untitled Project auto-save 1.ecproj");
assert!(s.file(&p).is_some());
assert_eq!(Path::new(&p), Path::new("/EffectCraft Auto-Save/Untitled Project auto-save 1.ecproj"));
assert!(s.file(&path_str(Path::new(&p))).is_some());
// The tab closes without a clean exit: the next visit offers the auto-save.
let mut b = Session { config: Some(Arc::new(WebConfig::new(s.clone()))), ..Default::default() };
b.load_settings();

View File

@ -14,7 +14,8 @@ effectcraft-engine = { workspace = true }
effectcraft-host = { workspace = true }
effectcraft-ui-egui = { workspace = true }
egui = { workspace = true }
eframe = { workspace = true, features = ["default_fonts", "wgpu", "accesskit", "wayland", "x11"] }
# `links`: links (Help menu, Discord, Settings) open in the system browser.
eframe = { workspace = true, features = ["default_fonts", "wgpu", "accesskit", "links", "wayland", "x11"] }
serde_json = { workspace = true }
rfd = { workspace = true }
cpal = { workspace = true }

View File

@ -0,0 +1,132 @@
//! Desktop integration for the AppImage. Wayland compositors (GNOME, KDE…) show a window's icon
//! from the desktop entry named by its app ID (`ai.storyteller.effectcraft.desktop`). The .deb,
//! .rpm and Flatpak packages install one; an AppImage installs nothing, so its window got the
//! generic icon. Started from an AppImage (`$APPIMAGE`), the app adds a hidden entry
//! (`NoDisplay=true`: no menu item) pointing at it and its icon under the user's data folder
//! (`$XDG_DATA_HOME`, else `~/.local/share`). Nothing is written when another entry already
//! provides the ID (an installed package, or the user's own), and an entry it wrote is kept up to
//! date when the AppImage moves. Failures are only logged: the icon is cosmetic.
use std::path::{Path, PathBuf};
pub const APP_ID: &str = "ai.storyteller.effectcraft";
/// Marks entries this module wrote (it never touches others).
const MARKER: &str = "X-EffectCraft-AppImage=true";
/// A path as one `Exec` argument (desktop entry spec: quoted, with `"`, `` ` ``, `$` and `\`
/// escaped; `%` doubled so it isn't a field code).
pub fn exec_arg(path: &str) -> String {
let mut s = String::with_capacity(path.len() + 2);
s.push('"');
for c in path.chars() {
// The quoting escape is a backslash, which the key file's string syntax escapes again:
// `"` is written `\\"`, a backslash `\\\\`.
match c {
'"' | '`' | '$' => {
s.push_str("\\\\");
s.push(c);
}
'\\' => s.push_str("\\\\\\\\"),
'%' => s.push_str("%%"),
'\n' | '\r' => {}
c => s.push(c),
}
}
s.push('"');
s
}
/// The hidden desktop entry for the AppImage at `appimage`.
pub fn entry(appimage: &str) -> String {
format!(
"[Desktop Entry]\nType=Application\nName=EffectCraft\nComment=Motion graphics and visual effects compositor\nExec={} %F\nIcon={APP_ID}\nTerminal=false\nStartupWMClass={APP_ID}\nNoDisplay=true\n{MARKER}\n",
exec_arg(appimage)
)
}
/// Write the entry and `icon` (a 256 px PNG) under `data_home`, unless one of `data_dirs` (or
/// `data_home` itself, with an entry we didn't write) already has the ID's entry. Returns whether
/// anything was written.
pub fn integrate(appimage: &str, data_home: &Path, data_dirs: &[PathBuf], icon: &[u8]) -> std::io::Result<bool> {
let name = format!("{APP_ID}.desktop");
if data_dirs.iter().any(|d| d.join("applications").join(&name).exists()) {
return Ok(false);
}
let path = data_home.join("applications").join(&name);
let text = entry(appimage);
let icon_path = data_home.join("icons/hicolor/256x256/apps").join(format!("{APP_ID}.png"));
match std::fs::read_to_string(&path) {
Ok(old) if !old.contains(MARKER) => return Ok(false),
Ok(old) if old == text && icon_path.exists() => return Ok(false),
_ => {}
}
write_atomic(&icon_path, icon)?;
write_atomic(&path, text.as_bytes())?;
Ok(true)
}
fn write_atomic(path: &Path, bytes: &[u8]) -> std::io::Result<()> {
if let Some(dir) = path.parent() {
std::fs::create_dir_all(dir)?;
}
let tmp = path.with_extension("part");
std::fs::write(&tmp, bytes)?;
std::fs::rename(&tmp, path)
}
/// [`integrate`] for this process: when started from an AppImage, with the XDG data folders of
/// the environment.
pub fn integrate_from_env(icon: &[u8]) {
let Some(appimage) = std::env::var_os("APPIMAGE").map(|a| a.to_string_lossy().into_owned()) else { return };
let env_path = |k: &str| std::env::var_os(k).filter(|v| !v.is_empty()).map(PathBuf::from);
let Some(data_home) = env_path("XDG_DATA_HOME").or_else(|| env_path("HOME").map(|h| h.join(".local/share"))) else { return };
let dirs = std::env::var("XDG_DATA_DIRS").ok().filter(|v| !v.is_empty()).unwrap_or_else(|| "/usr/local/share:/usr/share".into());
let data_dirs: Vec<PathBuf> = dirs.split(':').filter(|d| !d.is_empty()).map(PathBuf::from).collect();
match integrate(&appimage, &data_home, &data_dirs, icon) {
Ok(true) => log::info!("AppImage: desktop entry and icon written to {}", data_home.display()),
Ok(false) => {}
Err(e) => log::warn!("AppImage: could not write the desktop entry: {e}"),
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn exec_arguments_are_quoted_and_escaped() {
assert_eq!(exec_arg("/home/a/EffectCraft.AppImage"), "\"/home/a/EffectCraft.AppImage\"");
assert_eq!(exec_arg("/a b/$x\"y`z\\w%"), "\"/a b/\\\\$x\\\\\"y\\\\`z\\\\\\\\w%%\"");
let e = entry("/opt/EffectCraft.AppImage");
assert!(e.contains("Exec=\"/opt/EffectCraft.AppImage\" %F\n") && e.contains("NoDisplay=true\n") && e.contains("Icon=ai.storyteller.effectcraft\n"));
}
#[test]
fn writes_once_follows_moves_and_leaves_other_entries_alone() {
let root = std::env::temp_dir().join(format!("ec-appimage-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
let (home, system) = (root.join("home"), root.join("system"));
let entry_path = home.join("applications/ai.storyteller.effectcraft.desktop");
let icon_path = home.join("icons/hicolor/256x256/apps/ai.storyteller.effectcraft.png");
// First start: entry and icon written; again: nothing to do.
assert!(integrate("/a/EffectCraft.AppImage", &home, std::slice::from_ref(&system), b"png").unwrap());
assert_eq!(std::fs::read(&icon_path).unwrap(), b"png");
assert!(std::fs::read_to_string(&entry_path).unwrap().contains("Exec=\"/a/EffectCraft.AppImage\""));
assert!(!integrate("/a/EffectCraft.AppImage", &home, std::slice::from_ref(&system), b"png").unwrap());
// The AppImage moved: the entry follows it.
assert!(integrate("/b/EffectCraft.AppImage", &home, std::slice::from_ref(&system), b"png").unwrap());
assert!(std::fs::read_to_string(&entry_path).unwrap().contains("Exec=\"/b/EffectCraft.AppImage\""));
// The user's own entry is never replaced.
std::fs::write(&entry_path, "[Desktop Entry]\nName=Mine\n").unwrap();
assert!(!integrate("/c/EffectCraft.AppImage", &home, std::slice::from_ref(&system), b"png").unwrap());
assert_eq!(std::fs::read_to_string(&entry_path).unwrap(), "[Desktop Entry]\nName=Mine\n");
// An installed package provides the entry: nothing is written.
let _ = std::fs::remove_dir_all(&home);
std::fs::create_dir_all(system.join("applications")).unwrap();
std::fs::write(system.join("applications/ai.storyteller.effectcraft.desktop"), "x").unwrap();
assert!(!integrate("/a/EffectCraft.AppImage", &home, std::slice::from_ref(&system), b"png").unwrap());
assert!(!entry_path.exists());
let _ = std::fs::remove_dir_all(&root);
}
}

View File

@ -76,7 +76,7 @@ fn run<T: SizedSample + FromSample<f32>>(
let frames = out.len() / ch.max(1);
stereo.resize(frames * 2, 0.0);
feed.pull(&mut stereo);
for (f, s) in out.chunks_mut(ch.max(1)).zip(stereo.chunks_exact(2)) {
for (f, s) in out.chunks_mut(ch.max(1)).zip(stereo.as_chunks::<2>().0.iter()) {
for (c, o) in f.iter_mut().enumerate() {
// Mono devices, or left and right mapped to the same channel, get the mix.
let v = if ch == 1 || (c == left && c == right) {

View File

@ -4,9 +4,12 @@
//!
//! `--control <port>` (or `EFFECTCRAFT_CONTROL_PORT`) starts a localhost JSON-lines control server;
//! see `effectcraft_ui_egui::control` for the methods.
#![cfg_attr(all(target_os = "windows", not(debug_assertions)), windows_subsystem = "windows")]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
// Built everywhere so its tests run on every platform; only Linux AppImages use it.
#[cfg_attr(not(target_os = "linux"), allow(dead_code))]
mod appimage;
mod audio_out;
mod control_server;
#[cfg(target_os = "macos")]
@ -17,11 +20,15 @@ use serde_json::json;
/// The app icon for the window, taskbar and Dock (assets/app-icon, regenerated by
/// `packaging/icons.sh`). macOS gets the padded variant so it sits on Apple's icon grid in the Dock.
/// The 256 px app icon (Windows, Linux; the AppImage's desktop integration installs it too).
#[cfg(not(target_os = "macos"))]
const ICON_PNG: &[u8] = include_bytes!("../../../assets/app-icon/hicolor/256x256/apps/ai.storyteller.effectcraft.png");
fn app_icon() -> egui::IconData {
#[cfg(target_os = "macos")]
let png: &[u8] = include_bytes!("../../../assets/app-icon/effectcraft-macos-512.png");
#[cfg(not(target_os = "macos"))]
let png: &[u8] = include_bytes!("../../../assets/app-icon/hicolor/256x256/apps/ai.storyteller.effectcraft.png");
let png: &[u8] = ICON_PNG;
eframe::icon_data::from_png_bytes(png).unwrap_or_default()
}
@ -47,6 +54,11 @@ fn main() -> eframe::Result {
_ => files.push(a),
}
}
// The font menus list the installed fonts: read their names while the window opens.
effectcraft_engine::text::fonts::scan_system_in_background();
// Wayland shows the window's icon from its desktop entry: an AppImage brings its own.
#[cfg(target_os = "linux")]
appimage::integrate_from_env(ICON_PNG);
let options = eframe::NativeOptions {
viewport: egui::ViewportBuilder::default()
.with_title("EffectCraft")
@ -57,7 +69,7 @@ fn main() -> eframe::Result {
.with_active(control_port.is_none())
.with_icon(app_icon())
// Wayland matches the window to `ai.storyteller.effectcraft.desktop` by this ID.
.with_app_id("ai.storyteller.effectcraft"),
.with_app_id(appimage::APP_ID),
event_loop_builder: agent_event_loop(control_port.is_some()),
..Default::default()
};
@ -65,6 +77,23 @@ fn main() -> eframe::Result {
"EffectCraft",
options,
Box::new(move |cc| {
// This executable owns the shared device's handlers. Install them before
// EffectCraft's compositor pipelines are built, without changing handlers from
// inside Gpu::new. eframe has already constructed its presentation renderer here.
let gpu_failures = effectcraft_ui_egui::gpu_failure::GpuFailureBridge::new(&cc.egui_ctx);
if let Some(rs) = &cc.wgpu_render_state {
let errors = gpu_failures.clone();
rs.device.on_uncaptured_error(std::sync::Arc::new(move |error| {
let message = format!("uncaptured GPU error: {error}");
if errors.report(&message, false) {
log::error!("{message}");
}
}));
let lost = gpu_failures.clone();
rs.device.set_device_lost_callback(move |reason, message| {
lost.report(&format!("GPU device lost ({reason:?}): {message}"), true);
});
}
let mut session = effectcraft_host::session();
// Lazy open and non-blocking auto-save: footage is checked and auto-saves are
// written on background threads (M13.14).
@ -72,7 +101,7 @@ fn main() -> eframe::Result {
session.autosave.background = true;
// Settings, shortcut presets and the crash-recovery sentinel live in the platform
// config directory. Agent-driven runs (`--control`) skip crash recovery.
if let Some(dir) = config_dir() {
if let Some(dir) = effectcraft_host::config_dir() {
// WebAssembly effect plug-ins in <config>/Plug-ins load before the menus are built.
let plugins = dir.join("Plug-ins");
if plugins.is_dir() {
@ -98,6 +127,7 @@ fn main() -> eframe::Result {
}
let show_home = home.unwrap_or(session.prefs.startup.show_home_on_launch && files.is_empty() && control_port.is_none());
let mut app = EffectcraftApp::new(session);
app.set_gpu_failure_bridge(gpu_failures);
app.ui.start_screen = show_home;
if let Some(r) = recovery {
app.offer_recovery(r);
@ -163,24 +193,6 @@ impl eframe::App for Desktop {
}
}
/// The platform config directory for EffectCraft (`EFFECTCRAFT_CONFIG_DIR` overrides):
/// `~/Library/Application Support/EffectCraft` (macOS), `%APPDATA%\EffectCraft` (Windows),
/// `$XDG_CONFIG_HOME/effectcraft` or `~/.config/effectcraft` (Linux and others).
fn config_dir() -> Option<std::path::PathBuf> {
use std::path::PathBuf;
if let Some(d) = std::env::var_os("EFFECTCRAFT_CONFIG_DIR") {
return Some(PathBuf::from(d));
}
let home = || std::env::var_os("HOME").map(PathBuf::from);
if cfg!(target_os = "macos") {
return home().map(|h| h.join("Library/Application Support/EffectCraft"));
}
if cfg!(target_os = "windows") {
return std::env::var_os("APPDATA").map(|a| PathBuf::from(a).join("EffectCraft"));
}
std::env::var_os("XDG_CONFIG_HOME").map(PathBuf::from).or_else(|| home().map(|h| h.join(".config"))).map(|c| c.join("effectcraft"))
}
/// Driven by an agent (`--control`): don't activate the app on launch, so the user's keyboard
/// focus stays where it is (macOS; winit activates ignoring other apps by default).
fn agent_event_loop(agent: bool) -> Option<eframe::EventLoopBuilderHook> {

View File

@ -94,8 +94,8 @@ impl NativeBar {
let sub = Submenu::new(label, true);
self.fill(&sub, children);
match label.as_str() {
"Window" => sub.set_as_windows_menu_for_nsapp(),
"Help" => sub.set_as_help_menu_for_nsapp(),
"Window" | "ウィンドウ" => sub.set_as_windows_menu_for_nsapp(),
"Help" | "ヘルプ" => sub.set_as_help_menu_for_nsapp(),
_ => {}
}
if let Err(e) = menu.append(&sub) {

View File

@ -60,8 +60,8 @@ pub fn encode_png(w: u32, h: u32, rgba: Vec<u8>, max_side: u32) -> Result<Vec<u8
let mut out = Vec::new();
let enc = PngEncoder::new_with_quality(&mut out, CompressionType::Fast, FilterType::Adaptive);
let raw = img.into_raw();
let r = if raw.chunks_exact(4).all(|p| p[3] == 255) {
let rgb: Vec<u8> = raw.chunks_exact(4).flat_map(|p| [p[0], p[1], p[2]]).collect();
let r = if raw.as_chunks::<4>().0.iter().all(|p| p[3] == 255) {
let rgb: Vec<u8> = raw.as_chunks::<4>().0.iter().flat_map(|p| [p[0], p[1], p[2]]).collect();
enc.write_image(&rgb, w, h, image::ExtendedColorType::Rgb8)
} else {
enc.write_image(&raw, w, h, image::ExtendedColorType::Rgba8)

View File

@ -393,3 +393,31 @@ fn high_level_tools_add_effect_and_report_state() {
let (c, err) = call(&mut s, "add_effect", json!({"layer": l, "effect": "No Such Effect"}));
assert!(err && c[0]["text"].as_str().unwrap().contains("unknown effect"), "{c:?}");
}
#[test]
fn list_fonts_lists_families_with_styles() {
let mut s = server();
let all = call_json(&mut s, "list_fonts", json!({}));
let fams = all["families"].as_array().unwrap();
assert!(fams.iter().any(|f| f["family"] == "Inter" && f["origin"] == "bundled"), "{all}");
let one = call_json(&mut s, "list_fonts", json!({"query": "jetbrains"}));
assert_eq!(one["count"], 1, "{one}");
assert_eq!(one["families"][0]["family"], "JetBrains Mono");
assert!(one["families"][0]["styles"].as_array().unwrap().contains(&json!("Regular")), "{one}");
}
/// Trained models over MCP (Roto Brush and face tracking): listed with their authors and
/// licence, chosen, refused when unknown.
#[test]
fn trained_models_over_mcp() {
let mut s = server();
for (prefix, id) in [("roto", "mobilesam"), ("face", "mediapipe-face")] {
let list = call_json(&mut s, "execute_command", json!({"command": format!("{prefix}.models"), "params": {}}));
let models = list["models"].as_array().unwrap();
assert!(models.iter().any(|m| m["id"] == id && m["licence"] == "Apache-2.0" && m["authors"].is_string() && m["installed"] == false), "{list}");
let r = call_json(&mut s, "execute_command", json!({"command": format!("{prefix}.model.select"), "params": {"id": id}}));
assert_eq!(r["model"], id);
let (c, err) = call(&mut s, "execute_command", json!({"command": format!("{prefix}.model.select"), "params": {"id": "nope"}}));
assert!(err && c[0]["text"].as_str().unwrap().contains("unknown model"), "{c:?}");
}
}

View File

@ -150,6 +150,10 @@ fn list_effects(b: &mut Backend, a: &Value) -> Result<Reply> {
json_reply(b.exec("effect.list", obj(&[("filter", get(a, "filter"))]))?)
}
fn list_fonts(b: &mut Backend, a: &Value) -> Result<Reply> {
json_reply(b.exec("text.fonts", obj(&[("query", get(a, "query")), ("rescan", get(a, "rescan"))]))?)
}
/// `effect.apply` on one layer, then set parameters on the new instance by param id.
fn add_effect(b: &mut Backend, a: &Value) -> Result<Reply> {
let (layer, effect, comp) = (need(a, "layer")?, need(a, "effect")?, get(a, "comp"));
@ -607,6 +611,21 @@ static TOOLS: &[ToolDef] = &[
schema: || schema(json!({"filter": {"type": "string", "description": "Only effects whose id, name or category contains this text."}}), &[]),
run: list_effects,
},
ToolDef {
name: "list_fonts",
description: "The font families text layers can use, bundled and installed on this machine: family, styles, origin (bundled / system / user) and the name in the font's own language. Narrow with `query`; `rescan` picks up fonts installed since launch. Set one with execute_command `layer.setText {\"layer\", \"font\", \"style\"}`.",
bridge_only: false,
schema: || {
schema(
json!({
"query": {"type": "string", "description": "Only families whose name (English or own-language) contains this text."},
"rescan": {"type": "boolean", "description": "Look for fonts installed since launch first."}
}),
&[],
)
},
run: list_fonts,
},
ToolDef {
name: "get_state",
description: "Where things stand: the editor state (active comp, current time, selected layers/keys/items, open comps, tool, work area) plus what this build can do (`app`: version, command and effect counts, GPU effects, import/export formats, After Effects parity summary). Cheap; call it to re-orient after a series of edits.",

View File

@ -75,7 +75,7 @@ fn check(name: &str, cfg: EncoderConfig, pics: &[Pic]) -> Vec<u32> {
let raw = std::fs::read(&output).expect("decoded yuv");
let _ = std::fs::remove_dir_all(&dir);
let samples: Vec<u16> =
if bd == 8 { raw.iter().map(|&b| b as u16).collect() } else { raw.chunks_exact(2).map(|c| u16::from_le_bytes([c[0], c[1]])).collect() };
if bd == 8 { raw.iter().map(|&b| b as u16).collect() } else { raw.as_chunks::<2>().0.iter().map(|c| u16::from_le_bytes([c[0], c[1]])).collect() };
let (cw, ch) = (w.div_ceil(2), h.div_ceil(2));
let fsize = w * h + 2 * cw * ch;
assert_eq!(samples.len(), fsize * pics.len(), "{name}: decoded frame count");

View File

@ -222,7 +222,7 @@ fn description(t: Option<&[u8]>) -> Option<String> {
// The first record: UTF-16BE.
let len = be32(t, 20)? as usize;
let off = be32(t, 24)? as usize;
let u: Vec<u16> = t.get(off..off + len)?.chunks_exact(2).map(|c| u16::from_be_bytes([c[0], c[1]])).collect();
let u: Vec<u16> = t.get(off..off + len)?.as_chunks::<2>().0.iter().map(|c| u16::from_be_bytes(*c)).collect();
Some(String::from_utf16_lossy(&u).trim_end_matches('\0').to_string())
}
_ => None,

View File

@ -15,6 +15,7 @@ effectcraft-color = { workspace = true }
effectcraft-keyframe = { workspace = true }
effectcraft-project = { workspace = true }
effectcraft-raster = { workspace = true }
effectcraft-segment = { workspace = true }
effectcraft-track = { workspace = true }
rayon = { workspace = true }
serde = { workspace = true }

View File

@ -247,7 +247,7 @@ pub fn process(id: &str, params: &Params, buf: &mut [f32], rate: u32, start: f64
match id {
"ec.audio.backwards" => {
if params.b("swapChannels") {
buf.chunks_exact_mut(2).for_each(|f| f.swap(0, 1));
buf.as_chunks_mut::<2>().0.iter_mut().for_each(|f| f.swap(0, 1));
}
}
"ec.audio.basstreble" => {
@ -471,7 +471,7 @@ fn compressor(params: &Params, buf: &mut [f32], sr: f64) {
let limit = db_to_lin(params.f("outputLimit").min(0.0)) as f32;
// Smoothed gain change in dB: attack while the reduction grows, release as it shrinks.
let (mut g, mut gs) = (0.0f64, 0.0f64);
for f in buf.chunks_exact_mut(2) {
for f in buf.as_chunks_mut::<2>().0.iter_mut() {
let peak = f[0].abs().max(f[1].abs()) as f64;
let target = compressor_gain_db(lin_to_db(peak), th, ratio, knee);
let k = if target < g { att } else { rel };
@ -497,7 +497,7 @@ fn gate(params: &Params, buf: &mut [f32], sr: f64) {
// Openness 0..1 (closed..open) is the gain: a closed gate mutes.
let mut g = 0.0f64;
let mut held = 0usize;
for f in buf.chunks_exact_mut(2) {
for f in buf.as_chunks_mut::<2>().0.iter_mut() {
let peak = f[0].abs().max(f[1].abs()) as f64;
let open = if peak >= th {
held = hold;
@ -560,7 +560,7 @@ fn distortion(params: &Params, buf: &mut [f32], sr: f64, start: f64) {
// Sample-and-hold on absolute sample positions, so blocks agree with one long run.
let first = (start * sr).round() as i64;
let mut held = [0.0f32; 2];
for (i, f) in buf.chunks_exact_mut(2).enumerate() {
for (i, f) in buf.as_chunks_mut::<2>().0.iter_mut().enumerate() {
let take = i == 0 || (first + i as i64).rem_euclid(down) == 0;
for ch in 0..2 {
let x = f[ch];
@ -696,7 +696,7 @@ fn stereo_mixer(params: &Params, buf: &mut [f32]) {
let (l_to_l, l_to_r) = pan(params.f("leftPan"));
let (r_to_l, r_to_r) = pan(params.f("rightPan"));
let inv = if params.b("invertPhase") { -1.0 } else { 1.0 };
for f in buf.chunks_exact_mut(2) {
for f in buf.as_chunks_mut::<2>().0.iter_mut() {
let (l, r) = (f[0] * ll, f[1] * rl);
f[0] = (l * l_to_l + r * r_to_l) * inv;
f[1] = (l * l_to_r + r * r_to_r) * inv;

View File

@ -173,7 +173,8 @@ mod tests {
std::fs::write(path, &want).unwrap();
return;
}
let have = std::fs::read_to_string(path).unwrap_or_default();
// A Windows checkout may use CRLF; catalog content must still match exactly.
let have = std::fs::read_to_string(path).unwrap_or_default().replace("\r\n", "\n");
assert!(have == want, "docs/effects.md is stale: UPDATE_DOCS=1 cargo test -p effectcraft-effects --lib effects_doc_is_current");
}

View File

@ -16,15 +16,19 @@
//! decontamination of the edge colours. Outside the segmentation span the layer is transparent;
//! without strokes the effect passes its input through.
//!
//! *Version* is stored but all versions use the same (classic, graph-cut) engine; *Quality ▸
//! Best* segments at a higher working resolution with more iterations.
//! *Version*: 1.0 is the classic graph-cut engine; 2.0 and 3.0 also use the trained model chosen
//! in Settings ▸ Roto Brush ([`set_model`], an `effectcraft_segment::MaskModel` such as
//! MobileSAM), falling back to the classic engine when none is installed. The model's id is part
//! of the chain seed, so choosing another recomputes the mattes. *Quality ▸ Best* segments at a
//! higher working resolution with more iterations.
use std::collections::{BTreeMap, HashMap, VecDeque};
use std::sync::{Arc, Mutex, OnceLock};
use std::sync::{Arc, Mutex, OnceLock, RwLock};
use effectcraft_keyframe::Value;
use effectcraft_project::ParamUi;
use effectcraft_raster::Image;
use effectcraft_segment::MaskModel;
use effectcraft_track::roto::refine::{self, MatteParams};
use effectcraft_track::roto::{self as rb, FrameSeg, RotoData, SegOpts, rle};
use rayon::prelude::*;
@ -173,8 +177,29 @@ pub fn matte_params(params: &Params) -> MatteParams {
}
}
fn model_slot() -> &'static RwLock<Option<Arc<dyn MaskModel>>> {
static M: OnceLock<RwLock<Option<Arc<dyn MaskModel>>>> = OnceLock::new();
M.get_or_init(|| RwLock::new(None))
}
/// The trained model Roto Brush 2.0 / 3.0 use (Settings ▸ Roto Brush; `None`: the classic
/// engine for every version).
pub fn set_model(m: Option<Arc<dyn MaskModel>>) {
if let Ok(mut slot) = model_slot().write() {
*slot = m;
}
}
/// The model an instance segments with: its version's choice of the installed one.
pub fn model_for(params: &Params) -> Option<Arc<dyn MaskModel>> {
if e(params, "version") == 0 {
return None;
}
model_slot().read().ok().and_then(|m| m.clone())
}
/// Seed of the chain keys: what the frames are (the engine-maintained Input Key), the layer
/// size, the frame rate and the segmentation settings.
/// size, the frame rate, the segmentation settings and the model.
pub fn seed(params: &Params, layer_size: [f64; 2], fps: f64) -> u64 {
let mut h = 0xcbf2_9ce4_8422_2325;
fnv(&mut h, s(params, INPUT_KEY).as_bytes());
@ -189,6 +214,7 @@ pub fn seed(params: &Params, layer_size: [f64; 2], fps: f64) -> u64 {
fnv(&mut h, &v.to_bits().to_le_bytes());
}
fnv(&mut h, &[e(params, "quality") as u8, e(params, "version") as u8]);
fnv(&mut h, model_for(params).map_or(effectcraft_segment::CLASSICAL, |m| m.info().id).as_bytes());
h
}
@ -275,13 +301,15 @@ pub struct Chain {
pub keys: BTreeMap<i64, u64>,
pub opts: SegOpts,
pub refine_radius: f64,
/// The trained model (Version 2.0 / 3.0 with one installed).
pub model: Option<Arc<dyn MaskModel>>,
}
impl Chain {
pub fn new(params: &Params, layer_size: [f64; 2], fps: f64, scale: f64) -> Chain {
let data = data(params);
let keys = data.chain_keys(seed(params, layer_size, fps));
Chain { refine_radius: data.refine_radius(), data, keys, opts: seg_opts(params, scale) }
Chain { refine_radius: data.refine_radius(), data, keys, opts: seg_opts(params, scale), model: model_for(params) }
}
/// Frame `f`'s segmentation at this chain's scale: cached, or computed from the nearest
@ -331,7 +359,7 @@ impl Chain {
let img = frame(g)?;
let strokes = self.data.strokes_at(g);
let seg = match (&cur, g == base) {
(_, true) | (None, _) => rb::segment(&img, &strokes, None, &self.opts, self.refine_radius),
(_, true) | (None, _) => rb::segment_with(&img, &strokes, None, &self.opts, self.refine_radius, self.model.as_deref()),
(Some(p), false) => {
let src = self.data.source_of(g)?;
let pimg = match prev_img.take() {
@ -341,7 +369,7 @@ impl Chain {
if pimg.width != img.width || pimg.height != img.height {
return None;
}
rb::propagate(&pimg, p, &img, &strokes, &self.opts, self.refine_radius)
rb::propagate_with(&pimg, p, &img, &strokes, &self.opts, self.refine_radius, self.model.as_deref())
}
};
let seg = Arc::new(seg);

View File

@ -20,6 +20,7 @@ effectcraft-path = { workspace = true }
effectcraft-text = { workspace = true }
effectcraft-effects = { workspace = true }
effectcraft-render = { workspace = true }
effectcraft-segment = { workspace = true }
effectcraft-track = { workspace = true }
effectcraft-model = { workspace = true }
effectcraft-lottie = { workspace = true }

View File

@ -1,7 +1,7 @@
//! File menu.
use effectcraft_color::Label;
use effectcraft_project::{FootageKind, ItemKind, Project};
use effectcraft_project::{FootageKind, ItemId, ItemKind, LayerSource, Project};
use serde_json::{Value, json};
use super::{CommandSpec, always, bad, str_p};
@ -101,6 +101,54 @@ fn revert(s: &mut Session, _: &Value) -> Result<Value> {
open(s, &json!({"path": path}))
}
/// File ▸ Import; with `addToComp` (files dropped on the Timeline or the Composition viewer) they
/// also become layers of the active comp ([`add_to_comp`]).
fn import_cmd(s: &mut Session, p: &Value) -> Result<Value> {
let target = s.active_comp_id();
let mut r = import(s, p)?;
let errors = add_to_comp(s, &r, target, p);
if !errors.is_empty()
&& let Some(e) = r.get_mut("errors").and_then(Value::as_array_mut)
{
e.extend(errors.into_iter().map(Value::from));
}
Ok(r)
}
/// With `addToComp`, put what an import (its result `r`) made into `target`, the comp that was
/// active before it, one layer under the other, at the drop's `time`, `index` and `position`
/// (as in `layer.addItem`): the comps of layered files (their layers' footage stays in the
/// Project panel) and the other footage. `target` stays open. Returns what couldn't be added.
pub(crate) fn add_to_comp(s: &mut Session, r: &Value, target: Option<ItemId>, p: &Value) -> Vec<String> {
let Some(cid) = target.filter(|_| p.get("addToComp").and_then(Value::as_bool).unwrap_or(false)) else { return vec![] };
let ids = |k: &str| r.get(k).and_then(Value::as_array).into_iter().flatten().filter_map(Value::as_u64).collect::<Vec<u64>>();
let comps = ids("comps");
let in_comps: std::collections::HashSet<u64> = comps
.iter()
.filter_map(|c| s.project.comp(ItemId(*c)))
.flat_map(|c| c.layers.iter())
.filter_map(|l| match l.source {
LayerSource::Footage { item } | LayerSource::Comp { item } | LayerSource::Solid { item } => Some(item.0),
_ => None,
})
.collect();
let add: Vec<u64> = comps.iter().copied().chain(ids("items").into_iter().filter(|i| !in_comps.contains(i))).collect();
let mut errors = vec![];
for (k, item) in add.into_iter().enumerate() {
let mut params = json!({"comp": cid.0, "item": item, "time": p.get("time"), "position": p.get("position")});
if let Some(i) = p.get("index").and_then(Value::as_u64) {
params["index"] = json!(i.saturating_add(k as u64));
}
if let Err(e) = s.execute("layer.addItem", params) {
errors.push(e.to_string());
}
}
if s.active_comp_id() != Some(cid) {
s.open_comp(cid);
}
errors
}
pub(crate) fn import(s: &mut Session, p: &Value) -> Result<Value> {
let paths: Vec<String> = match p.get("paths").or(p.get("path")) {
Some(Value::Array(a)) => a.iter().filter_map(|v| v.as_str().map(str::to_string)).collect(),
@ -528,9 +576,9 @@ pub fn specs() -> Vec<CommandSpec> {
"File...",
["File", "Import"],
Some("Cmd+I"),
"{paths: [string], importAs?: footage|composition|compositionLayerSizes (Photoshop, PDF, Illustrator and EPS files), layer?: name|index (footage of one Photoshop layer), page?: number from 1 (PDF / Illustrator page), drag?: bool (dropped files: Settings ▸ Import ▸ Default Drag Import As)}",
"{paths: [string], importAs?: footage|composition|compositionLayerSizes (Photoshop, PDF, Illustrator and EPS files), layer?: name|index (footage of one Photoshop layer), page?: number from 1 (PDF / Illustrator page), drag?: bool (dropped files: Settings ▸ Import ▸ Default Drag Import As), addToComp?: bool (also add them to the active comp, at time?, index?, position? as in layer.addItem)}",
always,
import
import_cmd
),
cmd!(
"file.projectSettings",

View File

@ -46,6 +46,14 @@ pub fn specs() -> Vec<CommandSpec> {
),
fe!("playback.cacheWhenIdle", "Cache Frames When Idle", ["Composition", "Preview"], None, "{value?}", always),
fe!("playback.audio", "Audio", ["Composition", "Preview"], None, "{value?: include audio in previews}", always),
fe!(
"playback.scrubAudio",
"Scrub Audio",
[],
None,
"{time? (s, default the current time)} — plays one frame of the comp's audio there (Ctrl/Cmd-drag the current time)",
has_comp
),
// Viewer.
fe!("view.zoomIn", "Zoom In", ["View"], Some("."), "{}", has_comp),
fe!("view.zoomOut", "Zoom Out", ["View"], Some(","), "{}", has_comp),
@ -64,6 +72,14 @@ pub fn specs() -> Vec<CommandSpec> {
fe!("view.options", "View Options...", ["View"], Some("Cmd+Alt+U"), "{}", has_comp),
fe!("view.layerControls", "Show Layer Controls", ["View"], Some("Cmd+Shift+H"), "{value?}", always),
fe!("view.fullScreen", "Enter Full Screen", ["View"], Some("Ctrl+Cmd+F"), "{}", always),
fe!(
"view.newViewer",
"New Viewer",
["View"],
Some("Alt+Shift+N"),
"{} — another Composition viewer of the active comp; the one in use is locked",
has_comp
),
// Panels and workspaces.
fe!(
"window.panel",

View File

@ -58,7 +58,7 @@ pub(crate) fn label_keys(s: &mut Session, p: &Value) -> Result<Value> {
pub fn amplitudes(buf: &[f32]) -> (f64, f64, f64) {
let n = (buf.len() / 2).max(1) as f64;
let (mut l, mut r) = (0.0f64, 0.0f64);
for c in buf.chunks_exact(2) {
for c in buf.as_chunks::<2>().0 {
l += c[0].abs() as f64;
r += c[1].abs() as f64;
}

View File

@ -24,13 +24,58 @@ pub(crate) fn color_p(p: &Value, k: &str) -> Option<[f32; 3]> {
}
/// Insert a new layer above the selection (or at the top) and select it.
pub(crate) fn insert_layer(proj: &mut Project, st: &mut crate::EditorState, cid: ItemId, mut layer: Layer) -> Result<LayerId> {
pub(crate) fn insert_layer(proj: &mut Project, st: &mut crate::EditorState, cid: ItemId, layer: Layer) -> Result<LayerId> {
insert_layer_at(proj, st, cid, layer, None)
}
/// `index`: an `index` parameter, the layer's 1-based place in the stack (1 = top; past the
/// bottom = the bottom), as 0-based.
pub(crate) fn index_p(p: &Value, cmd: &str) -> Result<Option<usize>> {
match p.get("index") {
None | Some(Value::Null) => Ok(None),
Some(v) => {
v.as_u64().filter(|i| *i >= 1).map(|i| Some(usize::try_from(i - 1).unwrap_or(usize::MAX))).ok_or_else(|| bad(cmd, "index: 1-based stack position"))
}
}
}
/// A `position` parameter: `[x, y]` in comp pixels (finite).
pub(crate) fn position_p(p: &Value, cmd: &str) -> Result<Option<[f64; 2]>> {
match p.get("position") {
None | Some(Value::Null) => Ok(None),
Some(v) => v
.as_array()
.and_then(|a| Some([a.first()?.as_f64()?, a.get(1)?.as_f64()?]))
.filter(|c| c.iter().all(|x| x.is_finite()))
.map(Some)
.ok_or_else(|| bad(cmd, "position: [x, y] in comp pixels")),
}
}
/// Put a new layer's Position at `pos` (comp pixels).
pub(crate) fn place(l: &mut Layer, pos: Option<[f64; 2]>) {
if let Some([x, y]) = pos
&& let Some(pr) = l.props.prop_mut("transform/position")
{
let z = match pr.value {
KV::Vec3(v) => v[2],
_ => 0.0,
};
pr.value = KV::Vec3([x, y, z]);
}
}
/// [`insert_layer`] at stack position `at` (0 = top) instead of above the selected layer.
pub(crate) fn insert_layer_at(proj: &mut Project, st: &mut crate::EditorState, cid: ItemId, mut layer: Layer, at: Option<usize>) -> Result<LayerId> {
// Text and shape layers in Advanced 3D comps get Geometry Options (extrusion, bevels).
if proj.comp(cid).is_some_and(|c| c.renderer == effectcraft_project::Renderer::Advanced3D) {
super::model3d::add_geometry_options(&mut proj.next_id, &mut layer);
}
let comp = proj.comp_mut(cid).ok_or(EngineError::NoComp)?;
let at = st.selected_layers.first().and_then(|id| comp.layers.iter().position(|l| l.id == *id)).unwrap_or(0);
let at = match at {
Some(i) => i.min(comp.layers.len()),
None => st.selected_layers.first().and_then(|id| comp.layers.iter().position(|l| l.id == *id)).unwrap_or(0),
};
let id = layer.id;
comp.layers.insert(at, layer);
st.selected_layers = vec![id];
@ -258,12 +303,17 @@ fn add_item(s: &mut Session, p: &Value) -> Result<Value> {
return Err(bad("layer.addItem", "data files can't be layers; read them in expressions with footage(\"name\").sourceData"));
}
ItemKind::Footage(f) if f.kind == effectcraft_project::FootageKind::Model => {
return super::model3d::new_model(s, &serde_json::json!({"comp": cid.0, "item": item.0, "time": f_p(p, "time")}));
return super::model3d::new_model(
s,
&serde_json::json!({"comp": cid.0, "item": item.0, "time": f_p(p, "time"), "index": p.get("index"), "position": p.get("position")}),
);
}
ItemKind::Footage(f) => (LayerSource::Footage { item }, (f.width, f.height), Some(f.duration)),
ItemKind::Solid(so) => (LayerSource::Solid { item }, (so.width, so.height), None),
ItemKind::Folder => return Err(bad("layer.addItem", "folders can't be layers")),
};
let index = index_p(p, "layer.addItem")?;
let position = position_p(p, "layer.addItem")?;
let fr = comp.frame_rate;
// Settings ▸ General ▸ Create Layers at Composition Start Time (off: at the current time).
let default_start = if s.prefs.general.create_layers_at_comp_start || s.active_comp_id() != Some(cid) { Tick::ZERO } else { s.time() };
@ -273,7 +323,8 @@ fn add_item(s: &mut Session, p: &Value) -> Result<Value> {
l.start_time = start;
l.in_point = start;
l.out_point = fr.snap_nearest((start + dur.unwrap_or(comp.duration)).min(comp.duration)).max(start + fr.frame_duration());
insert_layer(proj, st, cid, l)
place(&mut l, position);
insert_layer_at(proj, st, cid, l, index)
})?;
Ok(json!({"layer": id.0}))
}
@ -612,6 +663,45 @@ pub(crate) fn parent_fixes(s: &Session, cid: ItemId, ids: &[LayerId], parent: Op
.collect()
}
/// Validate the proposed ancestor chain before any authored edit, including when compensation
/// is disabled. Bound both unique ancestors and linear lookup/selection work on loaded graphs.
fn check_parent_chain(comp: &Comp, ids: &[LayerId], parent: Option<LayerId>, max_visits: usize, max_work: usize) -> Result<()> {
let mut seen = std::collections::HashSet::new();
let mut work = 0usize;
let mut cur = parent;
while let Some(id) = cur {
if seen.len() >= max_visits {
return Err(bad("layer.setParent", "parent chain exceeds the safety visit limit"));
}
if !seen.insert(id) {
return Err(bad("layer.setParent", "parent chain contains an existing cycle"));
}
for selected in ids {
if work >= max_work {
return Err(bad("layer.setParent", "parent chain exceeds the safety work limit"));
}
work += 1;
if *selected == id {
return Err(bad("layer.setParent", "parenting would create a cycle"));
}
}
let mut ancestor = None;
for layer in &comp.layers {
if work >= max_work {
return Err(bad("layer.setParent", "parent chain exceeds the safety work limit"));
}
work += 1;
if layer.id == id {
ancestor = Some(layer);
break;
}
}
let layer = ancestor.ok_or_else(|| bad("layer.setParent", "parent chain refers to a missing layer"))?;
cur = layer.parent;
}
Ok(())
}
fn set_parent(s: &mut Session, p: &Value) -> Result<Value> {
let (cid, ids) = layers_p(s, p)?;
let comp = s.project.comp(cid).ok_or(EngineError::NoComp)?;
@ -619,16 +709,9 @@ fn set_parent(s: &mut Session, p: &Value) -> Result<Value> {
None | Some(Value::Null) => None,
Some(v) => Some(resolve_layer(comp, v).ok_or_else(|| bad("layer.setParent", "no such parent layer"))?),
};
// Cycle check.
if let Some(par) = parent {
let mut cur = Some(par);
while let Some(c) = cur {
if ids.contains(&c) {
return Err(bad("layer.setParent", "parenting would create a cycle"));
}
cur = comp.layer(c).and_then(|l| l.parent);
}
}
// A loaded cycle need not include a selected child. Validate before compensation or edit;
// do not impose the renderer's 64-ancestor truncation as a new ordinary hierarchy limit.
check_parent_chain(comp, &ids, parent, 4096, 1_048_576)?;
// Like AE's pick-whip, parenting keeps the layer where it is: its transform is re-expressed
// in the new parent's space (Position, Rotation, Scale), unless `compensate: false`.
let compensate = b_p(p, "compensate").unwrap_or(true);
@ -1242,7 +1325,15 @@ pub fn specs() -> Vec<CommandSpec> {
has_layer_settings,
layer_settings
),
cmd!("layer.addItem", "Add Footage to Comp", ["File"], Some("Cmd+/"), "{item: id|name, time?, duration? (s, for a still)}", has_comp, add_item),
cmd!(
"layer.addItem",
"Add Footage to Comp",
["File"],
Some("Cmd+/"),
"{item: id|name, time? (s, the In point), index? (1-based stack position; default above the selected layer), position? ([x, y] comp px; default the centre), duration? (s, for a still)}",
has_comp,
add_item
),
cmd!("layer.select", "Select Layers", [], None, "{layers: [id|name|#n], add?, toggle?}", has_comp, select),
cmd!("layer.selectNext", "Select Next Layer", [], Some("Cmd+ArrowDown"), "{add?}", has_comp, select_next),
cmd!("layer.selectPrevious", "Select Previous Layer", [], Some("Cmd+ArrowUp"), "{add?}", has_comp, select_prev),
@ -1340,3 +1431,120 @@ pub fn specs() -> Vec<CommandSpec> {
#[allow(dead_code)]
fn unused(_: &Comp) {}
#[cfg(test)]
mod parent_chain_tests {
use super::*;
use std::sync::Arc;
fn fixture() -> (Session, u64, u64, u64) {
let mut s = Session::default();
s.execute("comp.new", json!({"name":"Parent safety","width":16,"height":16,"duration":1})).unwrap();
let a = s.execute("layer.newSolid", json!({"name":"A"})).unwrap()["layer"].as_u64().unwrap();
let b = s.execute("layer.newSolid", json!({"name":"B"})).unwrap()["layer"].as_u64().unwrap();
let c = s.execute("layer.newSolid", json!({"name":"C"})).unwrap()["layer"].as_u64().unwrap();
for id in [a, b, c] {
s.execute("prop.set", json!({"layer":id,"path":"transform/anchor","value":[0,0,0]})).unwrap();
}
s.execute("prop.set", json!({"layer":a,"path":"transform/position","value":[100,200,0]})).unwrap();
s.execute("prop.set", json!({"layer":c,"path":"transform/position","value":[10,20,0]})).unwrap();
s.history.undo.clear();
(s, a, b, c)
}
fn malformed_parent(s: &mut Session, child: u64, parent: u64) {
let cid = s.active_comp_id().unwrap();
Arc::make_mut(&mut s.project).comp_mut(cid).unwrap().layer_mut(LayerId(child)).unwrap().parent = Some(LayerId(parent));
}
fn assert_atomic_error(s: &mut Session, child: u64, parent: u64, compensate: bool, message: &str) {
let before = s.project.clone();
let revision = s.revision;
let undo = s.history.undo.len();
let err = s.execute("layer.setParent", json!({"layers":[child],"parent":parent,"compensate":compensate})).unwrap_err();
assert!(err.to_string().contains(message), "{err}");
assert!(Arc::ptr_eq(&before, &s.project));
assert_eq!(s.revision, revision);
assert_eq!(s.history.undo.len(), undo);
}
#[test]
fn loaded_cycle_excluding_selected_child_is_rejected_atomically() {
for compensate in [true, false] {
let (mut s, a, b, c) = fixture();
// Simulate a malformed loaded project. Never execute this on the unguarded baseline.
malformed_parent(&mut s, a, b);
malformed_parent(&mut s, b, a);
assert_atomic_error(&mut s, c, a, compensate, "existing cycle");
}
}
#[test]
fn dangling_loaded_parent_is_rejected_atomically() {
for compensate in [true, false] {
let (mut s, a, _, c) = fixture();
malformed_parent(&mut s, a, u64::MAX);
assert_atomic_error(&mut s, c, a, compensate, "missing layer");
}
}
#[test]
fn selected_descendant_and_direct_self_are_rejected() {
let (mut s, a, _, c) = fixture();
assert_atomic_error(&mut s, c, c, false, "create a cycle");
s.execute("layer.setParent", json!({"layers":[a],"parent":c,"compensate":false})).unwrap();
assert_atomic_error(&mut s, c, a, true, "create a cycle");
}
#[test]
fn ordinary_parent_and_unparent_keep_explicit_false_and_undo_semantics() {
for compensate in [true, false] {
let (mut s, a, _, c) = fixture();
let old = s.project.clone();
let world = |s: &Session| {
let cid = s.active_comp_id().unwrap();
let comp = s.active_comp().unwrap();
let ctx = effectcraft_render::EvalCtx::new(&s.project, cid, comp, s.time());
ctx.world_matrix(comp.layer(LayerId(c)).unwrap()).apply(effectcraft_geom::Vec3::ZERO)
};
let before = world(&s);
s.execute("layer.setParent", json!({"layers":[c],"parent":a,"compensate":compensate})).unwrap();
let expected = if compensate { before } else { before + effectcraft_geom::vec3(100.0, 200.0, 0.0) };
assert!((world(&s) - expected).length() < 1e-9);
assert_eq!(s.active_comp().unwrap().layer(LayerId(c)).unwrap().parent, Some(LayerId(a)));
assert_eq!(s.history.undo.len(), 1);
let parented = s.project.clone();
assert!(s.undo());
assert!(Arc::ptr_eq(&s.project, &old));
assert!(s.redo());
assert!(Arc::ptr_eq(&s.project, &parented));
s.execute("layer.setParent", json!({"layers":[c],"parent":null,"compensate":compensate})).unwrap();
assert_eq!(s.active_comp().unwrap().layer(LayerId(c)).unwrap().parent, None);
assert!(s.undo());
assert!(Arc::ptr_eq(&s.project, &parented));
}
}
#[test]
fn ancestor_and_lookup_work_budgets_fail_closed() {
let (mut s, a, b, c) = fixture();
malformed_parent(&mut s, a, b);
let comp = s.active_comp().unwrap();
// New solids are inserted above the selection: C,B,A. Walking A->B scans
// three then two layers, and compares the one selected child twice: seven work units.
assert_eq!(comp.layers.iter().map(|l| l.id).collect::<Vec<_>>(), vec![LayerId(c), LayerId(b), LayerId(a)]);
assert!(check_parent_chain(comp, &[LayerId(c)], Some(LayerId(a)), 2, 7).is_ok());
assert!(check_parent_chain(comp, &[LayerId(c)], Some(LayerId(a)), 2, 6).unwrap_err().to_string().contains("work limit"));
assert!(check_parent_chain(comp, &[LayerId(c)], Some(LayerId(a)), 2, 256).is_ok());
assert!(check_parent_chain(comp, &[LayerId(c)], Some(LayerId(a)), 1, 256).unwrap_err().to_string().contains("visit limit"));
assert!(check_parent_chain(comp, &[LayerId(c)], Some(LayerId(a)), 2, 1).unwrap_err().to_string().contains("work limit"));
assert!(check_parent_chain(comp, &[LayerId(c)], None, 0, 0).is_ok(), "unparenting does not traverse ancestors");
}
#[test]
fn well_formed_chain_beyond_renderer_depth_is_not_unnecessarily_rejected() {
let (s, _, _, c) = fixture();
let mut comp = s.active_comp().unwrap().clone();
let template = comp.layers.first().unwrap().clone();
for n in 0..80 {
let mut layer = template.clone();
layer.id = LayerId(10_000 + n);
layer.parent = (n < 79).then_some(LayerId(10_001 + n));
comp.layers.push(layer);
}
assert!(check_parent_chain(&comp, &[LayerId(c)], Some(LayerId(10_000)), 4096, 1_048_576).is_ok());
}
}

View File

@ -5,7 +5,7 @@
//! comp seconds; layer markers are stored in layer time. All commands are undoable.
use effectcraft_color::Label;
use effectcraft_project::{Comp, CuePoint, ItemId, LayerId, LayerSource, Marker};
use effectcraft_project::{Comp, CuePoint, ItemId, Layer, LayerId, LayerSource, Marker, Project};
use effectcraft_time::Tick;
use serde_json::{Value, json};
@ -85,6 +85,56 @@ fn list(s: &mut Session, p: &Value) -> Result<Value> {
Ok(Value::Array(ms.iter().enumerate().map(|(i, m)| marker_json(c, o, i, m)).collect()))
}
/// The nested comp's markers on a precomp layer, at their times in the containing comp `comp`
/// and within the layer's In–Out range, as After Effects shows them on the precomp layer's bar:
/// `(comp time, index in the nested comp, marker)`. The layer's start time and stretch map them;
/// with time remapping, or a stretch over protected regions (Responsive Design), a marker sits at
/// the first frame the layer shows its time.
pub fn nested_markers<'a>(project: &'a Project, comp_id: ItemId, comp: &Comp, layer: &Layer) -> Vec<(Tick, usize, &'a Marker)> {
let LayerSource::Comp { item } = layer.source else { return vec![] };
let Some(nc) = project.comp(item) else { return vec![] };
if nc.markers.is_empty() || item == comp_id {
return vec![];
}
let shown = |t: Tick| t >= layer.in_point && t < layer.out_point;
let stretched_regions = (layer.stretch - 100.0).abs() > 1e-9 && nc.markers.iter().any(|m| m.protected);
if layer.props.get("timeRemap").is_none() && !stretched_regions {
return nc.markers.iter().enumerate().map(|(i, m)| (layer.comp_time(m.time), i, m)).filter(|(t, _, _)| shown(*t)).collect();
}
// The layer's source time at every frame of its range; each marker at the first frame that
// reaches it (within rounding: a thousandth of a nested frame).
let fr = comp.frame_rate;
let (f0, f1) = (fr.frame_at(layer.in_point), fr.frame_at(layer.out_point));
let frames: Vec<(Tick, Tick)> = (f0..f1.min(f0.saturating_add(100_000)))
.map(|f| {
let t = fr.tick_of(f);
(t, effectcraft_render::EvalCtx::new(project, comp_id, comp, t).source_time(layer))
})
.collect();
let eps = nc.frame_duration().0 / 1000;
nc.markers
.iter()
.enumerate()
.filter_map(|(i, m)| frames.iter().find(|(_, src)| src.0.saturating_add(eps) >= m.time.0).map(|(t, _)| (*t, i, m)))
.filter(|(t, _, _)| shown(*t))
.collect()
}
/// `markers.nested`: the nested comp's markers on a precomp layer (see [`nested_markers`]).
fn nested(s: &mut Session, p: &Value) -> Result<Value> {
let cmd = "markers.nested";
let cid = comp_id(s, p)?;
let c = s.project.comp(cid).ok_or(EngineError::NoComp)?;
let lid = p.get("layer").and_then(|v| resolve_layer(c, v)).ok_or_else(|| bad(cmd, "missing `layer` (a precomp layer)"))?;
let l = c.layer(lid).ok_or_else(|| bad(cmd, "no such layer"))?;
let LayerSource::Comp { item } = l.source else { return Err(bad(cmd, "not a precomp layer")) };
let v: Vec<Value> = nested_markers(&s.project, cid, c, l)
.into_iter()
.map(|(t, i, m)| json!({"time": t.seconds(), "comp": item.0, "index": i, "nestedTime": m.time.seconds(), "duration": m.duration.seconds(), "comment": m.comment, "label": m.label.name()}))
.collect();
Ok(json!(v))
}
fn locked(c: &Comp, o: Owner) -> bool {
matches!(o, Owner::Layer(l) if c.layer(l).is_some_and(|l| l.markers_locked))
}
@ -299,6 +349,12 @@ fn has_precomp_layer(s: &Session) -> std::result::Result<(), String> {
pub fn specs() -> Vec<CommandSpec> {
vec![
query!("markers.list", "List Markers", "{layer? (omit for composition markers)}", list),
query!(
"markers.nested",
"Nested Comp Markers",
"{layer: a precomp layer, comp?} — its comp's markers at their times in this comp (as on the layer bar)",
nested
),
cmd!(
"markers.set",
"Marker Settings",

View File

@ -187,6 +187,7 @@ fn track_mask(s: &mut Session, p: &Value) -> Result<Value> {
path,
method,
times: times.clone(),
face_model: if method.is_face() { s.models.face() } else { None },
};
s.state.mask_track_method = method;
let wait = b_p(p, "wait").unwrap_or(false);
@ -195,6 +196,8 @@ fn track_mask(s: &mut Session, p: &Value) -> Result<Value> {
Ok(json!({
"mask": uid,
"method": method.id(),
// The engine face tracking tries first (the classic one when the model finds no face).
"faceModel": method.is_face().then(|| s.models.face().map_or(effectcraft_segment::CLASSICAL, |m| m.info().id)),
"frames": times.len() - 1,
"start": times[0].seconds(),
"end": times.last().map(|t| t.seconds()),

View File

@ -34,7 +34,7 @@ mod layer_time;
pub(crate) mod link;
mod liquify;
mod lottie;
mod markers;
pub mod markers;
mod mask;
pub mod mask_interp;
pub(crate) mod model3d;
@ -133,6 +133,7 @@ pub fn command_specs() -> &'static [CommandSpec] {
SPECS.get_or_init(|| {
let mut v = Vec::new();
v.extend(file::specs());
v.extend(crate::models::specs());
v.extend(edit::specs());
v.extend(comp::specs());
v.extend(layer::specs());

View File

@ -8,7 +8,7 @@ use effectcraft_project::{FootageKind, ItemId, ItemKind, Layer, LayerSource, Nod
use effectcraft_time::Tick;
use serde_json::{Value, json};
use super::layer::{color_p, insert_layer};
use super::layer::{color_p, index_p, insert_layer, insert_layer_at, place, position_p};
use super::{CommandSpec, b_p, bad, comp_id, f_p, has_comp, has_layers, layer_mut, layer_p, layers_p, str_p};
use crate::{EngineError, Result, Session, cmd};
@ -119,6 +119,8 @@ pub(crate) fn new_model(s: &mut Session, p: &Value) -> Result<Value> {
let name = str_p(p, "name").unwrap_or(&it.name).to_string();
let fr = comp.frame_rate;
let start = fr.snap_nearest(f_p(p, "time").map(Tick::from_seconds_f64).unwrap_or(Tick::ZERO));
let index = index_p(p, "layer.newModel")?;
let position = position_p(p, "layer.newModel")?;
let id = s.edit("New 3D Model Layer", None, |proj, st| {
let mut l = build::layer(proj, &comp, &name, LayerSource::Model { item }, (comp.width, comp.height), None);
let g = build::model_geometry_options(&mut Ids(&mut proj.next_id), scale, &clips);
@ -128,7 +130,8 @@ pub(crate) fn new_model(s: &mut Session, p: &Value) -> Result<Value> {
l.start_time = start;
l.in_point = start;
l.out_point = comp.duration.max(start + fr.frame_duration());
insert_layer(proj, st, cid, l)
place(&mut l, position);
insert_layer_at(proj, st, cid, l, index)
})?;
Ok(json!({"layer": id.0, "item": item.0}))
}
@ -349,7 +352,7 @@ pub fn specs() -> Vec<CommandSpec> {
"3D Model Layer",
[],
None,
"{item?: id or name of a 3D model footage item, path?: a .gltf, .glb or .obj file to import, name?, time? (s)}",
"{item?: id or name of a 3D model footage item, path?: a .gltf, .glb or .obj file to import, name?, time? (s), index? (1-based stack position), position? ([x, y] comp px)}",
has_comp,
new_model
),

View File

@ -104,15 +104,14 @@ fn browser_import(s: &mut Session, p: &Value) -> Result<Value> {
if !browser(s)?.fetch(&list, p) {
return Ok(json!({"pending": true, "paths": list}));
}
let r = s.execute("file.import", json!({"paths": paths}))?;
// Add to the active comp (drag into the timeline).
if b_p(p, "addToComp").unwrap_or(false)
&& s.active_comp_id().is_some()
&& let Some(items) = r.get("items").and_then(Value::as_array)
let target = s.active_comp_id();
let mut r = s.execute("file.import", json!({"paths": paths}))?;
// Dragged into the Timeline or the Composition viewer: into the comp, where they were dropped.
let errors = super::file::add_to_comp(s, &r, target, p);
if !errors.is_empty()
&& let Some(e) = r.get_mut("errors").and_then(Value::as_array_mut)
{
for it in items {
s.execute("layer.addItem", json!({"item": it}))?;
}
e.extend(errors.into_iter().map(Value::from));
}
Ok(r)
}
@ -233,7 +232,15 @@ pub fn specs() -> Vec<CommandSpec> {
cmd!("mediaBrowser.go", "Go to Folder", [], None, "{path?: folder | \"..\", importableOnly?}", browser_enabled, browser_go),
cmd!("mediaBrowser.addFavorite", "Add to Favorites", [], None, "{path?}", browser_enabled, |s, p| browser_favorite(s, p, true)),
cmd!("mediaBrowser.removeFavorite", "Remove from Favorites", [], None, "{path?}", browser_enabled, |s, p| browser_favorite(s, p, false)),
cmd!("mediaBrowser.import", "Import", [], None, "{paths, addToComp?}", browser_enabled, browser_import),
cmd!(
"mediaBrowser.import",
"Import",
[],
None,
"{paths, addToComp?, time?, index?, position? (where the layers go, as in layer.addItem)}",
browser_enabled,
browser_import
),
cmd!(
"mediaBrowser.action",
"Media Browser Action",

View File

@ -5,7 +5,7 @@ use effectcraft_project::{ItemId, Project};
use serde_json::{Value, json};
use super::{CommandSpec, always, b_p, bad, str_p};
use crate::{EngineError, Result, Session, cmd};
use crate::{EngineError, Result, Session, cmd, query};
/// `item` (id or name) → id.
fn item_ref(s: &Session, v: &Value, cmd: &str) -> Result<ItemId> {
@ -140,17 +140,40 @@ fn set_comment(s: &mut Session, p: &Value) -> Result<Value> {
Ok(Value::Null)
}
/// The items deleting `ids` removes: them and, for folders, everything inside.
fn doomed_items(project: &Project, ids: &[ItemId]) -> Vec<ItemId> {
project.items.keys().copied().filter(|id| ids.iter().any(|d| is_within(project, *id, *d))).collect()
}
/// What deleting `items` takes with it: (layers using them, in how many comps), counting only
/// the comps that stay.
fn usage_of(project: &Project, items: &[ItemId]) -> (usize, usize) {
let mut layers = 0;
let mut comps = 0;
for (_, comp) in project.comps().filter(|(cid, _)| !items.contains(cid)) {
let n = comp.layers.iter().filter(|l| l.source.item().is_some_and(|i| items.contains(&i))).count();
layers += n;
comps += usize::from(n > 0);
}
(layers, comps)
}
/// What deleting the items (default: selected) would delete: the items (with folder contents)
/// and the layers in other compositions that use them. The Project panel asks before deleting
/// items in use, as After Effects does.
fn usage(s: &mut Session, p: &Value) -> Result<Value> {
let ids = items_p(s, p, "project.usage")?;
let doomed = doomed_items(&s.project, &ids);
let (layers, comps) = usage_of(&s.project, &doomed);
Ok(json!({"items": doomed.len(), "layers": layers, "comps": comps}))
}
/// Delete project items (folders with their contents). Layers that use a deleted item and Render
/// Queue items of deleted comps go too, like After Effects' Edit ▸ Clear in the Project panel.
fn delete(s: &mut Session, p: &Value) -> Result<Value> {
let c = "project.delete";
let ids = items_p(s, p, c)?;
let mut doomed: Vec<ItemId> = vec![];
for id in s.project.items.keys() {
if ids.iter().any(|d| is_within(&s.project, *id, *d)) {
doomed.push(*id);
}
}
let doomed = doomed_items(&s.project, &ids);
let n = doomed.len();
s.edit("Delete Items", None, |proj, st| {
for id in &doomed {
@ -242,6 +265,7 @@ pub fn specs() -> Vec<CommandSpec> {
always,
delete
),
query!("project.usage", "Project Item Usage", "{items?: [id|name] (default: selected)} → {items, layers, comps} that deleting them removes", usage),
cmd!("project.duplicate", "Duplicate Project Items", [], None, "{items?: [id|name] (default: selected)} → {items}", always, duplicate),
]
}

View File

@ -498,6 +498,33 @@ pub fn font_features(s: &mut Session, p: &Value) -> Result<Value> {
}))
}
/// The installed font families (bundled and system), with their styles, where each comes from
/// and its name in the font's own language. `query` keeps the families whose English or native
/// name contains it (case-insensitive); `rescan` first looks for fonts installed since the app
/// started.
pub fn fonts(_s: &mut Session, p: &Value) -> Result<Value> {
use effectcraft_text::fonts;
let added = if p.get("rescan").and_then(Value::as_bool).unwrap_or(false) { fonts::rescan_system() } else { 0 };
let query = p.get("query").and_then(Value::as_str).map(str::to_lowercase).unwrap_or_default();
let native = fonts::native_families();
let mut origin: std::collections::BTreeMap<String, &'static str> = Default::default();
for f in fonts::all_faces() {
origin.entry(f.info.family.clone()).or_insert(f.info.origin);
}
let list: Vec<Value> = effectcraft_text::families()
.into_iter()
.filter(|(f, _)| query.is_empty() || f.to_lowercase().contains(&query) || native.get(f).is_some_and(|n| n.to_lowercase().contains(&query)))
.map(|(f, styles)| {
let mut v = json!({"family": f, "styles": styles, "origin": origin.get(&f).copied().unwrap_or("system")});
if let Some(n) = native.get(&f) {
v["nativeName"] = json!(n);
}
v
})
.collect();
Ok(json!({"count": list.len(), "families": list, "added": added}))
}
fn always_ok(_: &Session) -> std::result::Result<(), String> {
Ok(())
}
@ -527,6 +554,15 @@ pub fn specs() -> Vec<CommandSpec> {
always_ok,
font_features
),
cmd!(
"text.fonts",
"List Fonts",
[],
None,
"{query?, rescan?} → {count, families: [{family, styles, origin: bundled|system|user, nativeName?}], added} (rescan picks up fonts installed since launch)",
always_ok,
fonts
),
cmd!(
"text.setSelection",
"Set Text Selection",

View File

@ -26,6 +26,7 @@ pub mod mask_track;
pub mod media_browser;
pub mod media_cache;
pub mod menus;
pub mod models;
pub mod offload;
pub mod perf;
pub mod prefs;
@ -63,6 +64,7 @@ pub use effectcraft_keyframe as keyframe;
pub use effectcraft_project as project;
pub use effectcraft_raster as raster;
pub use effectcraft_render as render;
pub use effectcraft_segment as segment;
pub use effectcraft_text as text;
pub use effectcraft_time as time;
pub use effectcraft_track as track;
@ -347,6 +349,10 @@ pub struct Session {
pub camera_pending: Vec<(ItemId, LayerId, Uid)>,
/// The running (or finished, not yet polled) Roto Brush propagation / Freeze.
pub roto_job: Option<roto::RotoJob>,
/// Trained models for Roto Brush and face tracking (Settings ▸ Roto Brush / Face Tracking).
pub models: models::Models,
/// Where installed models live (hosts set it; else `models` next to the settings).
pub models_dir: Option<std::path::PathBuf>,
/// Roto Brush instances edited since their last propagation: (comp, layer, effect uid).
pub roto_pending: Vec<(ItemId, LayerId, Uid)>,
pub events: Vec<Event>,
@ -463,6 +469,8 @@ impl Default for Session {
camera_job: None,
camera_pending: vec![],
roto_job: None,
models: Default::default(),
models_dir: None,
roto_pending: vec![],
events: vec![],
journal: vec![],
@ -1010,6 +1018,17 @@ pub fn text_families() -> Vec<String> {
effectcraft_text::families().into_iter().map(|(f, _)| f).collect()
}
/// The styles a family offers (its own faces, in weight order), for the style menus. A family
/// that isn't installed offers the usual styles, which resolve to Inter's.
pub fn font_styles(family: &str) -> Vec<String> {
effectcraft_text::families()
.into_iter()
.find(|(f, _)| f.eq_ignore_ascii_case(family))
.map(|(_, st)| st)
.filter(|st| !st.is_empty())
.unwrap_or_else(|| ["Regular", "Medium", "SemiBold", "Bold", "Italic"].iter().map(|s| s.to_string()).collect())
}
/// The OpenType feature tags (GSUB / GPOS) of the face a family + style resolves to.
pub fn font_features(family: &str, style: &str) -> Vec<String> {
effectcraft_text::fonts::face(effectcraft_text::resolve(family, style).face).features()
@ -1079,6 +1098,8 @@ pub fn font_menu(prefs: &prefs::Prefs) -> Vec<FontRow> {
#[cfg(test)]
mod tests_history;
#[cfg(test)]
mod tests_models;
#[cfg(test)]
mod tests_paint;
#[cfg(test)]
mod tests_panels;

View File

@ -198,6 +198,8 @@ pub(crate) struct MaskWork {
pub method: MaskMethod,
/// Comp times; the first is the start frame.
pub times: Vec<Tick>,
/// Face tracking's trained model (Settings ▸ Face Tracking), if one is in use.
pub face_model: Option<Arc<dyn effectcraft_segment::face::FaceModel>>,
}
async fn run_work(w: MaskWork, shared: &MaskTrackShared) {
@ -275,7 +277,7 @@ async fn run_face<F: std::future::Future<Output = Option<(Arc<effectcraft_raster
) {
let detailed = w.method == MaskMethod::FaceDetailed;
let b = trk::mask::bounds(&flatten(&w.path, 8));
let Some((mut tracker, fit)) = trk::face::FaceTracker::new(&trk::Frame { img: &first.0, offset: first.1 }, b) else {
let Some((mut tracker, fit)) = trk::face::FaceTracker::new_with(&trk::Frame { img: &first.0, offset: first.1 }, b, w.face_model.clone()) else {
let mut s = lock(&shared.state);
s.error = Some("no face found inside the mask (draw the mask around a face)".into());
s.finished = true;

View File

@ -24,7 +24,7 @@ pub fn load_peaks(file: &Path) -> Option<Vec<[f32; 4]>> {
return None;
}
let f = |c: &[u8]| f32::from_le_bytes([c[0], c[1], c[2], c[3]]);
Some(b.chunks_exact(16).map(|c| [f(&c[0..4]), f(&c[4..8]), f(&c[8..12]), f(&c[12..16])]).collect())
Some(b.as_chunks::<16>().0.iter().map(|c| [f(&c[0..4]), f(&c[4..8]), f(&c[8..12]), f(&c[12..16])]).collect())
}
/// Store a summary (atomically: written beside, then renamed).

View File

@ -1046,6 +1046,7 @@ Animation
Reveal Properties with Animation | anim.reveal {"kind":"animation"}
Reveal All Modified Properties | anim.reveal {"kind":"modified"}
View
New Viewer | view.newViewer
Split with New Locked Viewer | view.splitLockedViewer
---
Zoom In | view.zoomIn

463
crates/engine/src/models.rs Normal file
View File

@ -0,0 +1,463 @@
//! Trained models (Settings ▸ Roto Brush and Settings ▸ Face Tracking; `roto.model*` and
//! `face.model*` commands): which models the registry offers (`effectcraft_segment::MODELS`, all
//! open source), which are installed (in the `models` folder), installing one from a file or
//! downloading the official weights (verified against the registry's SHA-256 either way, and kept
//! with a notice naming their authors and licence), choosing one per task, and loading it in the
//! background: into the Roto Brush effect (`effects::roto::set_model`) or the face tracker
//! ([`Models::face`]).
//!
//! Weights are never bundled: the install stays small and nothing is fetched unless asked for.
//! Downloads use the system's `curl` (shipped with Windows 10+, macOS and most Linux systems),
//! so no networking code is linked into EffectCraft; without it, the file can be downloaded in a
//! browser and installed with Install from File. The browser build uses the built-in engines
//! only, for now.
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use effectcraft_segment::face::FaceModel;
use effectcraft_segment::{self as seg, Loaded, ModelInfo, Task};
use serde_json::{Value, json};
use crate::commands::{CommandSpec, always, b_p, bad, str_p};
use crate::{EngineError, Result, Session, cmd, query};
/// What a task's background worker reports.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct Status {
/// What is happening now ("Downloading MobileSAM…").
pub busy: Option<String>,
/// The model in use.
pub active: Option<String>,
/// The last failure, for the Settings page.
pub error: Option<String>,
/// A finished job's message and whether it failed, for a toast (taken by
/// [`Session::poll_models`]).
pub done: Option<(String, bool)>,
}
/// The session's trained models: per task, the worker's status; and the face model in use.
#[derive(Default)]
pub struct Models {
roto: Arc<Mutex<Status>>,
face: Arc<Mutex<Status>>,
face_model: FaceSlot,
}
impl Models {
fn lane(&self, task: Task) -> &Arc<Mutex<Status>> {
match task {
Task::Mask => &self.roto,
Task::Face => &self.face,
}
}
pub fn status(&self, task: Task) -> Status {
self.lane(task).lock().map(|s| s.clone()).unwrap_or_default()
}
/// The face model face tracking uses (`None`: the classic tracker).
pub fn face(&self) -> Option<Arc<dyn FaceModel>> {
self.face_model.lock().ok().and_then(|m| m.clone())
}
fn set_active(&self, task: Task, model: Option<Loaded>) {
activate(self.lane(task), &self.face_model, task, model);
}
}
type FaceSlot = Arc<Mutex<Option<Arc<dyn FaceModel>>>>;
/// Put a loaded model to use (or drop `task`'s): Roto Brush's lives in the effect, the face
/// tracker's in `face`.
fn activate(lane: &Arc<Mutex<Status>>, face: &FaceSlot, task: Task, model: Option<Loaded>) {
let id = model.as_ref().map(|m| match m {
Loaded::Mask(m) => m.info().id.to_string(),
Loaded::Face(m) => m.info().id.to_string(),
});
match (task, model) {
(Task::Mask, Some(Loaded::Mask(m))) => crate::effects::roto::set_model(Some(m)),
(Task::Mask, _) => crate::effects::roto::set_model(None),
(Task::Face, m) => {
if let Ok(mut slot) = face.lock() {
*slot = match m {
Some(Loaded::Face(f)) => Some(f),
_ => None,
};
}
}
}
set(lane, |s| s.active = id);
}
fn set(status: &Arc<Mutex<Status>>, f: impl FnOnce(&mut Status)) {
if let Ok(mut s) = status.lock() {
f(&mut s);
}
}
/// How a task shows in Settings, toasts and the command list.
struct Kind {
/// Command prefix and settings key section.
prefix: &'static str,
label: &'static str,
classic_name: &'static str,
classic_description: &'static str,
}
const ROTO: Kind = Kind {
prefix: "roto",
label: "Roto Brush",
classic_name: "Classic (graph cut)",
classic_description: "Built in: colour models and graph cuts, propagated by optical flow. Always available.",
};
const FACE: Kind = Kind {
prefix: "face",
label: "Face Tracking",
classic_name: "Classic (shape model)",
classic_description: "Built in: a skin colour model, facial feature shapes and a fitted face shape model. Best on front-facing, evenly lit faces. Always available.",
};
fn kind(task: Task) -> &'static Kind {
match task {
Task::Mask => &ROTO,
Task::Face => &FACE,
}
}
impl Session {
/// The folder installed models live in: the host's, else `models` next to the settings.
pub fn models_dir(&self) -> Option<PathBuf> {
self.models_dir.clone().or_else(|| Some(self.config.as_ref()?.dir()?.join("models")))
}
/// Where `m` is installed, if it is (a file of the published size).
pub fn model_path(&self, m: &ModelInfo) -> Option<PathBuf> {
let p = self.models_dir()?.join(m.file_name);
std::fs::metadata(&p).ok().filter(|md| md.len() == m.size).map(|_| p)
}
/// The model chosen for `task` (Settings).
pub fn chosen_model(&self, task: Task) -> &str {
match task {
Task::Mask => &self.prefs.roto.model,
Task::Face => &self.prefs.face.model,
}
}
fn choose_model(&mut self, task: Task, id: &str) {
let slot = match task {
Task::Mask => &mut self.prefs.roto.model,
Task::Face => &mut self.prefs.face.model,
};
*slot = id.to_string();
}
/// Load (or drop) the models Settings chooses; called whenever settings change.
pub fn apply_models(&mut self) {
for task in [Task::Mask, Task::Face] {
self.apply_model(task, false);
}
}
/// Load (or drop) `task`'s chosen model; `wait`: before returning (scripts and agents).
fn apply_model(&mut self, task: Task, wait: bool) {
let want = self.chosen_model(task).to_string();
let lane = self.models.lane(task).clone();
let st = self.models.status(task);
let info = seg::info(&want).filter(|m| m.task == task);
let Some(info) = info else {
if st.active.is_some() {
self.models.set_active(task, None);
}
return;
};
if st.active.as_deref() == Some(info.id) || st.busy.is_some() {
return;
}
let Some(path) = self.model_path(info) else {
self.models.set_active(task, None);
return;
};
let (face_slot, label) = (self.models.face_model.clone(), kind(task).label);
spawn(&lane, label, format!("Loading {}…", info.name), wait, move |status| {
let m = load_file(&path, info.id)?;
activate(status, &face_slot, task, Some(m));
Ok(format!("{label}: {} loaded", info.name))
});
}
/// Finish what the model workers did: a toast, and loading a model that just arrived.
pub fn poll_models(&mut self) {
for task in [Task::Mask, Task::Face] {
let done = self.models.lane(task).lock().ok().and_then(|mut s| s.done.take());
if let Some((message, error)) = done {
self.events.push(crate::Event::Toast { message, error });
self.apply_model(task, false);
}
}
}
}
fn load_file(path: &Path, id: &str) -> std::result::Result<Loaded, String> {
let bytes = std::fs::read(path).map_err(|e| format!("cannot read {}: {e}", path.display()))?;
seg::load(id, &bytes)
}
/// Run `job` on a worker thread (`wait`: right here), reporting through `status` (desktop; the
/// browser build has no trained models yet).
fn spawn(
status: &Arc<Mutex<Status>>,
label: &'static str,
what: String,
wait: bool,
job: impl FnOnce(&Arc<Mutex<Status>>) -> std::result::Result<String, String> + Send + 'static,
) {
if cfg!(target_arch = "wasm32") {
set(status, |s| s.error = Some("trained models are available in the desktop app".into()));
return;
}
set(status, |s| {
s.busy = Some(what);
s.error = None;
});
let st = status.clone();
let run = move || {
let r = job(&st);
set(&st, |s| {
s.busy = None;
match r {
Ok(msg) => s.done = Some((msg, false)),
Err(e) => {
s.done = Some((format!("{label} model: {e}"), true));
s.error = Some(e);
}
}
});
};
if wait {
run();
} else if let Err(e) = std::thread::Builder::new().name("ec-model".into()).spawn(run) {
set(status, |s| {
s.busy = None;
s.error = Some(format!("cannot start: {e}"));
});
}
}
/// The notice kept next to `m`'s weights.
fn notice_path(dir: &Path, m: &ModelInfo) -> PathBuf {
dir.join(format!("{}.NOTICE.txt", m.file_name))
}
/// Copy verified weights into the models folder (atomically), with their notice.
fn install_bytes(dir: &Path, m: &ModelInfo, bytes: &[u8]) -> std::result::Result<PathBuf, String> {
if bytes.len() as u64 != m.size || seg::sha256::hex(bytes) != m.sha256 {
return Err(format!("this is not the published {} file (size or SHA-256 differs)", m.name));
}
use crate::config::FileOps;
let path = dir.join(m.file_name);
let files = crate::config::StdFiles;
files.write(&path, bytes).map_err(|e| format!("cannot write {}: {e}", path.display()))?;
files.write(&notice_path(dir, m), seg::notice(m).as_bytes()).map_err(|e| format!("cannot write the notice: {e}"))?;
Ok(path)
}
fn model_p(p: &Value, cmd: &str, task: Task) -> Result<&'static ModelInfo> {
let id = str_p(p, "id").ok_or_else(|| bad(cmd, "missing `id`"))?;
seg::info(id).filter(|m| m.task == task).ok_or_else(|| bad(cmd, format!("unknown model `{id}` (see {}.models)", kind(task).prefix)))
}
fn list(s: &mut Session, task: Task) -> Result<Value> {
let k = kind(task);
let st = s.models.status(task);
let chosen = s.chosen_model(task).to_string();
let mut v = vec![json!({
"id": seg::CLASSICAL,
"name": k.classic_name,
"description": k.classic_description,
"licence": "MIT OR Apache-2.0",
"installed": true,
"selected": chosen == seg::CLASSICAL,
"active": st.active.is_none(),
})];
for m in seg::models(task) {
v.push(json!({
"id": m.id,
"name": m.name,
"description": m.description,
"authors": m.authors,
"licence": m.licence,
"licenceUrl": m.licence_url,
"homepage": m.homepage,
"url": m.url,
"size": m.size,
"sha256": m.sha256,
"installed": s.model_path(m).is_some(),
"selected": chosen == m.id,
"active": st.active.as_deref() == Some(m.id),
}));
}
Ok(json!({"models": v, "busy": st.busy, "error": st.error, "folder": s.models_dir().map(|d| d.display().to_string())}))
}
/// After a `wait`: the job's failure, as the command's.
fn waited(s: &Session, task: Task, cmd: &str) -> Result<()> {
match s.models.status(task).error {
Some(e) => Err(bad(cmd, e)),
None => Ok(()),
}
}
fn select(s: &mut Session, p: &Value, task: Task) -> Result<Value> {
let cmd = format!("{}.model.select", kind(task).prefix);
let id = str_p(p, "id").ok_or_else(|| bad(&cmd, "missing `id`"))?;
if id != seg::CLASSICAL {
model_p(p, &cmd, task)?;
}
let wait = b_p(p, "wait").unwrap_or(false);
s.choose_model(task, id);
if wait {
s.apply_model(task, true);
waited(s, task, &cmd)?;
}
s.save_prefs();
s.prefs_changed();
Ok(json!({"model": id, "active": s.models.status(task).active}))
}
fn install(s: &mut Session, p: &Value, task: Task) -> Result<Value> {
let cmd = format!("{}.model.install", kind(task).prefix);
let path = str_p(p, "path").ok_or_else(|| bad(&cmd, "missing `path` (the downloaded weights file)"))?;
let dir = s.models_dir().ok_or_else(|| bad(&cmd, "no settings folder to install into"))?;
let bytes = std::fs::read(path).map_err(|e| EngineError::Other(format!("cannot read {path}: {e}")))?;
// Which model it is: the one asked for, or the one whose checksum it has.
let m = match str_p(p, "id") {
Some(_) => model_p(p, &cmd, task)?,
None => {
let h = seg::sha256::hex(&bytes);
seg::models(task).find(|m| m.sha256 == h).ok_or_else(|| bad(&cmd, "this file is not a published model in the registry (SHA-256 differs)"))?
}
};
let dest = install_bytes(&dir, m, &bytes).map_err(|e| bad(&cmd, e))?;
let wait = b_p(p, "wait").unwrap_or(false);
s.apply_model(task, wait);
if wait {
waited(s, task, &cmd)?;
}
Ok(json!({"id": m.id, "path": dest.display().to_string()}))
}
fn download(s: &mut Session, p: &Value, task: Task) -> Result<Value> {
let k = kind(task);
let cmd = format!("{}.model.download", k.prefix);
let m = model_p(p, &cmd, task)?;
let dir = s.models_dir().ok_or_else(|| bad(&cmd, "no settings folder to install into"))?;
if s.models.status(task).busy.is_some() {
return Err(bad(&cmd, "a model is already downloading or loading"));
}
let size = if m.size >= 10_000_000 { format!("{} MB", m.size / 1_000_000) } else { format!("{:.1} MB", m.size as f64 / 1e6) };
let wait = b_p(p, "wait").unwrap_or(false);
spawn(s.models.lane(task), k.label, format!("Downloading {} ({size})…", m.name), wait, move |_| {
std::fs::create_dir_all(&dir).map_err(|e| format!("cannot create {}: {e}", dir.display()))?;
let tmp = dir.join(format!("{}.part", m.file_name));
let out = std::process::Command::new("curl")
.args(["-fsSL", "--retry", "2", "--connect-timeout", "20", "-o"])
.arg(&tmp)
.arg(m.url)
.output()
.map_err(|e| format!("downloading needs curl ({e}); or download {} in a browser and use Install from File", m.url))?;
if !out.status.success() {
let _ = std::fs::remove_file(&tmp);
return Err(format!("download failed: {}", String::from_utf8_lossy(&out.stderr).trim()));
}
let bytes = std::fs::read(&tmp).map_err(|e| format!("cannot read the download: {e}"))?;
let _ = std::fs::remove_file(&tmp);
install_bytes(&dir, m, &bytes)?;
Ok(format!("{}: {} downloaded and verified", k.label, m.name))
});
if wait {
waited(s, task, &cmd)?;
s.apply_model(task, true);
waited(s, task, &cmd)?;
return Ok(json!({"id": m.id, "installed": true}));
}
Ok(json!({"id": m.id, "downloading": true}))
}
fn remove(s: &mut Session, p: &Value, task: Task) -> Result<Value> {
let m = model_p(p, &format!("{}.model.remove", kind(task).prefix), task)?;
let Some(path) = s.model_path(m) else { return Ok(json!({"removed": false})) };
if s.models.status(task).active.as_deref() == Some(m.id) {
s.models.set_active(task, None);
}
std::fs::remove_file(&path).map_err(|e| EngineError::Other(format!("cannot remove {}: {e}", path.display())))?;
if let Some(dir) = path.parent() {
let _ = std::fs::remove_file(notice_path(dir, m));
}
Ok(json!({"removed": true}))
}
pub fn specs() -> Vec<CommandSpec> {
vec![
query!("roto.models", "Roto Brush Models", "{}", |s, _| list(s, Task::Mask)),
cmd!(
"roto.model.select",
"Use Roto Brush Model",
[],
None,
"{id: classical|mobilesam, wait?} (Roto Brush 2.0 / 3.0 use it; wait: loaded before returning)",
always,
|s, p| select(s, p, Task::Mask)
),
cmd!(
"roto.model.install",
"Install Roto Brush Model",
[],
None,
"{path: weights file, id?, wait?} — verified against the registry's SHA-256",
always,
|s, p| install(s, p, Task::Mask)
),
cmd!(
"roto.model.download",
"Download Roto Brush Model",
[],
None,
"{id, wait?} — the official weights, verified (desktop; uses the system curl)",
always,
|s, p| download(s, p, Task::Mask)
),
cmd!("roto.model.remove", "Remove Roto Brush Model", [], None, "{id}", always, |s, p| remove(s, p, Task::Mask)),
query!("face.models", "Face Tracking Models", "{}", |s, _| list(s, Task::Face)),
cmd!(
"face.model.select",
"Use Face Tracking Model",
[],
None,
"{id: classical|mediapipe-face, wait?} (face tracking uses it; wait: loaded before returning)",
always,
|s, p| select(s, p, Task::Face)
),
cmd!(
"face.model.install",
"Install Face Tracking Model",
[],
None,
"{path: model file, id?, wait?} — verified against the registry's SHA-256",
always,
|s, p| install(s, p, Task::Face)
),
cmd!(
"face.model.download",
"Download Face Tracking Model",
[],
None,
"{id, wait?} — the official model, verified (desktop; uses the system curl)",
always,
|s, p| download(s, p, Task::Face)
),
cmd!("face.model.remove", "Remove Face Tracking Model", [], None, "{id}", always, |s, p| remove(s, p, Task::Face)),
]
}

View File

@ -519,6 +519,7 @@ async fn run_job(s: &mut Session, job: WorkerJob, post: Post) -> Result<(), Stri
path,
method,
times,
face_model: if method.is_face() { s.models.face() } else { None },
};
s.start_mask_track(work, mask, direction, true)
})

View File

@ -44,6 +44,8 @@ macro_rules! page {
}
page!(General {
/// Interface language (`en` or `ja`).
language: String = "en".into(),
/// Levels of Undo (1–99).
undo_levels: u32 = 32,
/// Path Point and Handle Size (px).
@ -209,6 +211,24 @@ page!(ThreeD {
realtime_shadows: bool = true,
});
page!(
/// Settings ▸ Roto Brush.
RotoPrefs {
/// The segmentation model Roto Brush 2.0 / 3.0 use: `classical` (built in) or a model id
/// from `effectcraft_segment::MODELS` (once installed).
model: String = "classical".into(),
}
);
page!(
/// Settings ▸ Face Tracking.
FacePrefs {
/// The model Track Mask ▸ Face Tracking uses: `classical` (built in) or a model id from
/// `effectcraft_segment::MODELS` (once installed).
model: String = "classical".into(),
}
);
page!(Scripting {
allow_scripts_write_files: bool = false,
warn_executing_files: bool = true,
@ -242,7 +262,7 @@ fn parse_hex(s: &str) -> Option<[u8; 3]> {
if h.len() != 6 {
return None;
}
let p = |i: usize| u8::from_str_radix(&h[i..i + 2], 16).ok();
let p = |i: usize| u8::from_str_radix(h.get(i..i + 2)?, 16).ok();
Some([p(0)?, p(2)?, p(4)?])
}
@ -272,6 +292,8 @@ pub struct Prefs {
#[serde(rename = "threeD")]
pub three_d: ThreeD,
pub scripting: Scripting,
pub roto: RotoPrefs,
pub face: FacePrefs,
/// File ▸ Open Recent, newest first.
pub recent_projects: Vec<String>,
/// File ▸ Import Recent Footage, newest first.
@ -312,6 +334,8 @@ impl Default for Prefs {
video: Video::default(),
three_d: ThreeD::default(),
scripting: Scripting::default(),
roto: RotoPrefs::default(),
face: FacePrefs::default(),
recent_projects: vec![],
recent_footage: vec![],
recent_presets: vec![],
@ -393,6 +417,9 @@ impl Prefs {
pub fn normalize(&mut self) {
self.version = PREFS_VERSION;
let g = &mut self.general;
if !matches!(g.language.as_str(), "en" | "ja") {
g.language = "en".into();
}
g.undo_levels = g.undo_levels.clamp(1, 99);
g.path_point_size = g.path_point_size.clamp(3, 20);
g.recent_items = g.recent_items.clamp(1, 30);
@ -431,6 +458,9 @@ impl Prefs {
/// Set the value at a dotted key. The key must exist and the value must have its type.
pub fn set(&mut self, key: &str, value: Value) -> Result<(), String> {
if key == "general.language" && !matches!(value.as_str(), Some("en" | "ja")) {
return Err("`general.language` expects `en` or `ja`".into());
}
let mut v = serde_json::to_value(&*self).map_err(|e| e.to_string())?;
let ptr = format!("/{}", key.replace('.', "/"));
let slot = v.pointer_mut(&ptr).ok_or_else(|| format!("unknown setting `{key}`"))?;
@ -709,6 +739,8 @@ pub fn section_key(page: &str) -> Option<&'static str> {
"video" => "video",
"3d" | "threeD" => "threeD",
"scripting" => "scripting",
"roto" => "roto",
"face" => "face",
_ => return None,
})
}
@ -735,6 +767,8 @@ pub fn page_id(name: &str) -> Option<&'static str> {
"video" | "videopreview" => "video",
"3d" | "threed" => "3d",
"scripting" | "scriptingexpressions" => "scripting",
"roto" | "rotobrush" | "models" => "roto",
"face" | "facetracking" => "face",
_ => return None,
};
Some(id)
@ -822,6 +856,9 @@ pub enum Item {
/// The browser's storage manager (web app only, `storage.*`): usage and quota, persistent
/// storage, Clear buttons. Not shown where there is no browser storage.
BrowserStorage,
/// A task's trained models (`roto.models`, `face.models`): authors, licence, size, install,
/// choose.
Models(effectcraft_segment::Task),
}
/// One page of the Settings dialog.
@ -848,6 +885,7 @@ pub fn pages() -> Vec<Page> {
id: "general",
title: "General",
items: vec![
s("general.language", "Language", Kind::Choice(&[("English", "en"), ("日本語", "ja")]), true),
s("general.undoLevels", "Levels of Undo", Kind::Int(1, 99, ""), true),
s("general.pathPointSize", "Path Point and Handle Size", Kind::Int(3, 20, "px"), true),
s("general.recentItems", "Recent Projects Shown", Kind::Int(1, 30, ""), true),
@ -1096,6 +1134,28 @@ pub fn pages() -> Vec<Page> {
Note("Expressions use the JavaScript engine. Script errors report their file and line in the Script Console; there is no step debugger."),
],
},
Page {
id: "roto",
title: "Roto Brush",
items: vec![
Section("Segmentation Model"),
Note(
"Roto Brush 2.0 and 3.0 use the model chosen here (Version 1.0 always uses the classic engine). Models are open source, downloaded only when you ask, and checked against their published SHA-256.",
),
Models(effectcraft_segment::Task::Mask),
],
},
Page {
id: "face",
title: "Face Tracking",
items: vec![
Section("Face Model"),
Note(
"Face tracking in the Tracker panel (Outline Only and Detailed Features) uses the model chosen here. Models are open source, downloaded only when you ask, and checked against their published SHA-256.",
),
Models(effectcraft_segment::Task::Face),
],
},
]
}
@ -1148,6 +1208,10 @@ pub fn pages_json() -> Value {
Item::AudioDevices { key } => json!({"type": "device", "key": key, "label": "Default Output"}),
Item::Note(t) => json!({"note": t}),
Item::BrowserStorage => json!({"browserStorage": "storage.info / storage.persist / storage.clear (web app)"}),
Item::Models(task) => {
let p = if *task == effectcraft_segment::Task::Face { "face" } else { "roto" };
json!({"models": format!("{p}.models / {p}.model.select / {p}.model.download / {p}.model.install / {p}.model.remove"), "key": format!("{p}.model")})
}
})
.collect();
json!({"id": p.id, "title": p.title, "items": items})

View File

@ -636,6 +636,7 @@ impl Session {
/// Frontends call this every frame: finishes jobs and (with `auto`) propagates edited
/// instances in the background.
pub fn poll_roto(&mut self, auto: bool) -> bool {
self.poll_models();
let changed = self.poll_roto_job();
if auto && self.roto_job.is_none() && self.offloaded_roto().is_none() && self.state.roto.auto_propagate {
while let Some((c, l, e)) = self.roto_pending.first().copied() {

View File

@ -69,7 +69,7 @@ fn add_configure_render() {
assert!(s.execute("renderQueue.setOutputModule", json!({"item": id, "format": "jpeg", "channels": "rgba"})).is_err());
let r = s.execute("renderQueue.setOutput", json!({"item": id, "path": "/tmp/rq/out.mov"})).unwrap();
assert_eq!(r["output"]["format"], "ProRes", "Output To with .mov switches to ProRes");
assert_eq!(r["outputPath"], "/tmp/rq/out.mov");
assert_eq!(std::path::Path::new(r["outputPath"].as_str().unwrap()), std::path::absolute("/tmp/rq/out.mov").unwrap());
// A second item that fails, a third that is unqueued.
let b = s.execute("renderQueue.add", json!({"format": "gif", "output": "/tmp/rq/fail.gif"})).unwrap();
let c = s.execute("renderQueue.add", json!({"output": "/tmp/rq/skip.mp4"})).unwrap();
@ -77,11 +77,14 @@ fn add_configure_render() {
assert_eq!(s.project.render_queue[2].status.label(), "Unqueued");
let r = s.execute("renderQueue.render", json!({})).unwrap();
assert_eq!(r["items"].as_array().unwrap().len(), 2);
assert_eq!(*ex.log.lock().unwrap(), vec!["/tmp/rq/out.mov".to_string(), "/tmp/rq/fail.gif".to_string()]);
assert_eq!(
ex.log.lock().unwrap().iter().map(std::path::PathBuf::from).collect::<Vec<_>>(),
[std::path::absolute("/tmp/rq/out.mov").unwrap(), std::path::absolute("/tmp/rq/fail.gif").unwrap()]
);
let q = &s.project.render_queue;
assert_eq!(q[0].status.label(), "Done");
assert!(q[0].started.is_some() && q[0].render_time.is_some());
assert_eq!(q[0].last_output.as_deref(), Some("/tmp/rq/out.mov"));
assert_eq!(std::path::Path::new(q[0].last_output.as_deref().unwrap()), std::path::absolute("/tmp/rq/out.mov").unwrap());
assert_eq!(q[1].status, RenderStatus::Failed("boom".into()));
assert_eq!(q[2].status.label(), "Unqueued");
assert!(s.drain_events().iter().any(|e| matches!(e, Event::Toast { error: true, .. })));

View File

@ -38,6 +38,8 @@ impl Session {
let levels = self.prefs.general.undo_levels.max(1) as usize;
let current = self.project.clone();
self.history.trim(levels, &current);
// Settings ▸ Roto Brush / Face Tracking: load (or drop) the chosen trained models.
self.apply_models();
self.prefs_revision += 1;
}

View File

@ -101,6 +101,16 @@ pub fn parse_vm_stat(text: &str, total: u64) -> Option<SysMemory> {
Some(SysMemory { total, available: (free * page).min(total) })
}
/// Parse `sysctl -n hw.physmem hw.pagesize vm.stats.vm.v_free_count vm.stats.vm.v_inactive_count`
/// (FreeBSD: one number per line, in that order).
pub fn parse_freebsd_sysctl(text: &str) -> Option<SysMemory> {
let mut it = text.split_whitespace().map(|v| v.parse::<u64>().ok());
let total = it.next()??;
let page = it.next()??;
let free = it.next()??.saturating_add(it.next()??);
Some(SysMemory { total, available: free.saturating_mul(page).min(total) })
}
/// Total and available physical memory, when the system reports them.
pub fn memory() -> Option<SysMemory> {
#[cfg(target_os = "linux")]
@ -112,6 +122,10 @@ pub fn memory() -> Option<SysMemory> {
let total: u64 = run("sysctl", &["-n", "hw.memsize"])?.trim().parse().ok()?;
return parse_vm_stat(&run("vm_stat", &[])?, total).or(Some(SysMemory { total, available: total }));
}
#[cfg(target_os = "freebsd")]
{
return parse_freebsd_sysctl(&run("sysctl", &["-n", "hw.physmem", "hw.pagesize", "vm.stats.vm.v_free_count", "vm.stats.vm.v_inactive_count"])?);
}
#[cfg(target_os = "windows")]
{
let text = run(
@ -136,6 +150,10 @@ pub fn cpu_name() -> Option<String> {
{
return run("sysctl", &["-n", "machdep.cpu.brand_string"]).map(|s| s.trim().to_string()).filter(|s| !s.is_empty());
}
#[cfg(target_os = "freebsd")]
{
return run("sysctl", &["-n", "hw.model"]).map(|s| s.trim().to_string()).filter(|s| !s.is_empty());
}
#[cfg(target_os = "windows")]
{
return std::env::var("PROCESSOR_IDENTIFIER").ok();
@ -156,6 +174,10 @@ pub fn os_version() -> String {
{
return n.trim_matches('"').to_string();
}
#[cfg(target_os = "freebsd")]
if let Some(v) = run("freebsd-version", &[]) {
return format!("FreeBSD {}", v.trim());
}
#[cfg(target_os = "windows")]
if let Some(v) = run("cmd", &["/C", "ver"]) {
return v.trim().to_string();
@ -172,6 +194,18 @@ pub fn cpu_cores() -> usize {
mod tests {
use super::*;
#[test]
fn freebsd_sysctl_memory_parses() {
let m = parse_freebsd_sysctl("17179869184\n4096\n1000000\n500000\n").expect("parses");
assert_eq!(m.total, 17_179_869_184);
assert_eq!(m.available, 1_500_000 * 4096);
assert!(parse_freebsd_sysctl("17179869184\n4096\n").is_none());
assert!(parse_freebsd_sysctl("garbage").is_none());
// Available never exceeds total, even with nonsense counts.
let m = parse_freebsd_sysctl("1000\n4096\n999999999\n999999999").expect("parses");
assert_eq!(m.available, 1000);
}
#[test]
fn parses_proc_meminfo_and_vm_stat() {
let m = parse_meminfo("MemTotal: 16000000 kB\nMemFree: 1000 kB\nMemAvailable: 8000000 kB\n").unwrap();

View File

@ -716,15 +716,14 @@ fn footage_dir(s: &Session, name: &str, footage_dir: Option<&str>, project_path:
}
let safe: String = name.chars().map(|c| if c.is_alphanumeric() || " -_()".contains(c) { c } else { '_' }).collect();
let safe = if safe.trim().is_empty() { "Template".to_string() } else { safe.trim().to_string() };
let base = match s.config.as_ref().and_then(|c| c.dir()) {
Some(d) => d.join(FOOTAGE_DIR),
None => PathBuf::from(format!("/{FOOTAGE_DIR}")),
};
let base = s.config.as_ref().and_then(|c| c.dir()).map(|d| d.join(FOOTAGE_DIR));
// A fresh folder per created project.
let mut k = 1;
loop {
let d = base.join(if k == 1 { safe.clone() } else { format!("{safe} {k}") });
let ds = d.to_string_lossy().into_owned();
let folder = if k == 1 { safe.clone() } else { format!("{safe} {k}") };
// Virtual storage keys use '/' independently of the host OS. Native settings
// folders retain their platform path semantics.
let ds = base.as_ref().map(|d| d.join(&folder).to_string_lossy().into_owned()).unwrap_or_else(|| format!("/{FOOTAGE_DIR}/{folder}"));
if !s.services.exists(&ds) || k > 999 {
return ds;
}
@ -955,7 +954,7 @@ mod tests {
assert!(t.width > 0 && t.duration > 0.0, "{t:?}");
let th = decode_thumb(&thumbnail(&s, &t.id).unwrap()).unwrap();
// Not a blank frame: some variety in the pixels.
let distinct: std::collections::BTreeSet<[u8; 3]> = th.chunks_exact(3).map(|c| [c[0] / 16, c[1] / 16, c[2] / 16]).collect();
let distinct: std::collections::BTreeSet<[u8; 3]> = th.as_chunks::<3>().0.iter().map(|c| [c[0] / 16, c[1] / 16, c[2] / 16]).collect();
assert!(distinct.len() > 3, "{}: thumbnail looks blank", t.id);
}
let sq = l.iter().find(|t| t.id == "social-square").unwrap();
@ -1121,7 +1120,7 @@ mod tests {
let dir = root.join("work").join("Promo Footage");
let fp = footage_paths(&s);
let get = |n: &str| fp.iter().find(|f| f.0 == n).unwrap().clone();
assert_eq!(get("Still").1, dir.join("still.png").to_string_lossy());
assert_eq!(std::path::Path::new(&get("Still").1), dir.join("still.png"));
assert_eq!(std::fs::read(dir.join("still.png")).unwrap(), vec![1u8; 1000]);
let shot = get("Shot");
assert_eq!(shot.2.len(), 3);

View File

@ -47,6 +47,76 @@ fn demo_layer_cache_is_transparent() {
}
}
/// `layer.addItem` puts the layer at `index` (1-based) in the stack and starts it at `time`, as a
/// drop in the Timeline does (#89).
#[test]
fn add_item_at_a_stack_index_and_time() {
let mut s = Session::default();
let clip = s.execute("comp.new", json!({"name": "Clip", "width": 16, "height": 16, "frameRate": 30, "duration": 1})).unwrap()["comp"].as_u64().unwrap();
s.execute("comp.new", json!({"name": "Main", "width": 32, "height": 32, "frameRate": 30, "duration": 4})).unwrap();
for name in ["C", "B", "A"] {
s.execute("layer.newSolid", json!({"name": name})).unwrap();
}
let names = |s: &Session| s.active_comp().unwrap().layers.iter().map(|l| l.name.clone()).collect::<Vec<_>>();
s.execute("layer.addItem", json!({"item": clip, "index": 2, "time": 1.01})).unwrap();
assert_eq!(names(&s), ["A", "Clip", "B", "C"]);
let l = &s.active_comp().unwrap().layers[1];
assert_eq!(l.in_point, effectcraft_time::Tick::from_seconds_f64(1.0), "on the nearest frame");
// Past the bottom: the bottom.
s.execute("layer.addItem", json!({"item": clip, "index": 99})).unwrap();
assert_eq!(names(&s).last().map(String::as_str), Some("Clip 2"));
assert!(s.execute("layer.addItem", json!({"item": clip, "index": 0})).is_err(), "1-based");
}
/// Files dropped on the Timeline or the Composition viewer (`file.import {addToComp}`) become
/// layers of the comp that was active, one under the other, where they were dropped (#85, #89).
#[test]
fn imported_files_go_into_the_comp_where_they_were_dropped() {
use effectcraft_project::{Footage, FootageKind};
struct Probe;
impl crate::Importer for Probe {
fn probe(&self, path: &str) -> Result<Footage, String> {
Ok(Footage { path: path.into(), kind: FootageKind::Still, width: 64, height: 32, has_video: true, ..Default::default() })
}
}
let mut s = Session { importer: Some(std::sync::Arc::new(Probe)), ..Default::default() };
s.execute("comp.new", json!({"name": "Main", "width": 320, "height": 180, "frameRate": 30, "duration": 4})).unwrap();
s.execute("layer.newSolid", json!({"name": "Bg"})).unwrap();
let params = json!({"paths": ["/drop/a.png", "/drop/b.png"], "addToComp": true, "position": [40.0, 50.0], "index": 1, "time": 1.0});
s.execute("file.import", params).unwrap();
let comp = s.active_comp().unwrap();
assert_eq!(comp.layers.iter().map(|l| l.name.as_str()).collect::<Vec<_>>(), ["a.png", "b.png", "Bg"]);
for l in &comp.layers[..2] {
assert_eq!(l.props.prop("transform/position").map(|p| p.value.clone()), Some(effectcraft_keyframe::Value::Vec3([40.0, 50.0, 0.0])));
assert_eq!(l.in_point, effectcraft_time::Tick::from_seconds_f64(1.0));
}
// Without `addToComp` they only import.
s.execute("file.import", json!({"paths": ["/drop/c.png"]})).unwrap();
assert_eq!(s.active_comp().unwrap().layers.len(), 3);
assert!(s.execute("layer.addItem", json!({"item": "a.png", "position": [1.0]})).is_err(), "position needs x and y");
}
/// Audio, Lock and Shy don't change pixels: toggling them keeps the cached layers (#103).
#[test]
fn audio_lock_and_shy_keep_cached_layers() {
let mut s = Session::default();
s.execute("comp.new", json!({"name": "C", "width": 160, "height": 90, "frameRate": 30, "duration": 1})).unwrap();
s.execute("layer.newSolid", json!({"color": "#3080ff", "width": 80, "height": 40})).unwrap();
s.execute("effect.apply", json!({"layer": "#1", "effect": "Gaussian Blur"})).unwrap();
s.execute("prop.set", json!({"layer": "#1", "path": "effects/#1/blurriness", "value": 12})).unwrap();
let cid = s.active_comp_id().unwrap();
let t = s.time();
s.render(cid, t, Default::default());
let before = s.layer_cache.stats();
for sw in ["audio", "lock", "shy"] {
s.execute("layer.setSwitch", json!({"layers": ["#1"], "switch": sw, "value": sw != "audio"})).unwrap();
}
s.render(cid, t, Default::default());
let after = s.layer_cache.stats();
assert_eq!(after.misses, before.misses, "no layer rendered again");
assert!(after.hits > before.hits);
}
#[test]
fn every_command_has_unique_id_and_runs_or_reports() {
let mut ids = std::collections::HashSet::new();

View File

@ -30,6 +30,7 @@ fn project(s: &mut Session) -> effectcraft_project::ItemId {
#[test]
fn disk_cache_settings_persistence_and_invalidation() {
crate::tests_roto::hold_roto_cache_test_lock();
let dir = folder("persist");
let mut s = session(&dir);
let st = s.execute_checked("cache.diskStats", json!({})).unwrap();
@ -116,7 +117,22 @@ fn footage_changes_change_the_salt() {
s.execute("prop.set", json!({"layer": "#1", "path": "transform/opacity", "value": 50})).unwrap();
let k2 = effectcraft_render::disk_cache::comp_content_key(&s.project, cid);
assert_ne!(k1, k2);
// Switches that don't change pixels (Audio, Lock, Shy, Hide Shy Layers) don't change it (#103).
for sw in ["audio", "lock", "shy"] {
s.execute("layer.setSwitch", json!({"layers": ["#1"], "switch": sw, "value": sw != "audio"})).unwrap();
}
s.execute("comp.setSwitch", json!({"switch": "hideShy", "value": true})).unwrap();
assert_eq!(effectcraft_render::disk_cache::comp_content_key(&s.project, cid), k2);
// Unrelated comps do not change it.
s.execute("comp.new", json!({"name": "Other", "width": 10, "height": 10, "frameRate": 30, "duration": 1})).unwrap();
assert_eq!(effectcraft_render::disk_cache::comp_content_key(&s.project, cid), k2);
// A proxy, and its Use Proxy switch, change the pixels.
let set_proxy = |s: &mut Session, enabled: bool| {
let it = std::sync::Arc::make_mut(&mut s.project).items.get_mut(&cid).unwrap();
it.proxy = Some(Box::new(effectcraft_project::Proxy { footage: effectcraft_project::Footage::default(), enabled }));
effectcraft_render::disk_cache::comp_content_key(&s.project, cid)
};
let k3 = set_proxy(&mut s, true);
assert_ne!(k3, k2);
assert_ne!(set_proxy(&mut s, false), k3);
}

View File

@ -104,3 +104,37 @@ fn face_tracking_keys_outline_points_and_measurements() {
let layer = s.active_comp().unwrap().layer(clip).unwrap();
assert!(layer.props.find_group(m3).unwrap().get("path").unwrap().keys.is_empty());
}
/// With MediaPipe Face Landmarker installed (`EFFECTCRAFT_FACE_LANDMARKER` = path to
/// `face_landmarker.task`, else skipped): Track Mask ▸ Face Tracking uses it, and the pupils and
/// eye corners it keys sit on the synthetic face's.
#[test]
fn face_tracking_uses_the_chosen_model() {
let Ok(model) = std::env::var("EFFECTCRAFT_FACE_LANDMARKER") else { return };
let (mut s, clip, _) = setup(|f| render_synthetic(W, H, &face(f), 5));
let dir = std::env::temp_dir().join(format!("ec-face-model-{}", std::process::id()));
s.models_dir = Some(dir.clone());
s.execute_checked("face.model.install", json!({"path": model})).unwrap();
// `wait`: loaded before the track starts (as scripts need).
let r = s.execute_checked("face.model.select", json!({"id": "mediapipe-face", "wait": true})).unwrap();
assert_eq!(r["active"], "mediapipe-face");
let c = face(0).center;
let r = s.execute("mask.new", json!({"layer": clip.0, "vertices": ShapePath::rect(c, 120.0, 160.0).vertices, "closed": true})).unwrap();
let mask = r["mask"].as_u64().unwrap();
s.execute("time.set", json!({"time": 0})).unwrap();
let r = s.execute_checked("track.mask", json!({"layer": clip.0, "mask": mask, "method": "faceDetailed", "direction": "forward", "wait": true})).unwrap();
assert_eq!((r["frames"].as_u64(), r["faceModel"].as_str()), (Some(24), Some("mediapipe-face")), "{r}");
let layer = s.active_comp().unwrap().layer(clip).unwrap().clone();
let pts = layer.effects().unwrap().groups().find(|g| g.match_id == effectcraft_effects::face_track::POINTS_ID).expect("Face Track Points");
let mut worst: f64 = 0.0;
for f in [0, 8, 16, 24] {
let truth = synth_landmarks(&face(f));
for id in ["leftPupil", "rightPupil", "leftEyeOuter", "rightEyeOuter"] {
let i = effectcraft_track::face::landmark(id).unwrap();
let KV::Vec2(v) = pts.get(id).unwrap().value_at(frame_time(f)) else { panic!() };
worst = worst.max((v[0] - truth[i][0]).hypot(v[1] - truth[i][1]) / face(f).height);
}
}
assert!(worst < 0.05, "worst eye error {worst} face heights");
let _ = std::fs::remove_dir_all(dir);
}

View File

@ -244,7 +244,7 @@ fn import_export_helpers() {
// Append Bit Depth to File Name.
assert_eq!(p.output_name("/out/Comp 1.png", 16), "/out/Comp 1.png");
p.export.append_bits_to_name = true;
assert_eq!(p.output_name("/out/Comp 1.png", 16), "/out/Comp 1_16bpc.png");
assert_eq!(PathBuf::from(p.output_name("/out/Comp 1.png", 16)), PathBuf::from("/out/Comp 1_16bpc.png"));
assert_eq!(p.output_name("Comp_[#####].exr", 32), "Comp_[#####]_32bpc.exr");
// Segments: sequences by file count, movies by size at the data rate.
assert_eq!(p.segment_frames(100, true, 0.0), None);
@ -299,7 +299,7 @@ fn render_queue_uses_default_folder_bit_depth_names_and_segments() {
assert_eq!(segs.len(), 1, "20 frames × 50 kB fit in 1 MB");
item.output.bitrate_kbps = 8000; // 100 kB per frame: 10 frames per MB
let segs = s.segments(&item, "/o/Main.mp4");
assert_eq!(segs.iter().map(|(_, p)| p.as_str()).collect::<Vec<_>>(), ["/o/Main_001.mp4", "/o/Main_002.mp4"]);
assert_eq!(segs.iter().map(|(_, p)| PathBuf::from(p)).collect::<Vec<_>>(), [PathBuf::from("/o/Main_001.mp4"), PathBuf::from("/o/Main_002.mp4")]);
}
#[test]

View File

@ -147,3 +147,37 @@ fn precompose_leave_and_move_attributes() {
assert_eq!((nl.in_point.seconds(), nl.out_point.seconds()), (1.0, 3.0));
assert_eq!(s.active_comp_id(), Some(inner));
}
/// A precomp layer shows its comp's markers on its bar, mapped through its timing.
#[test]
fn nested_comp_markers_follow_the_precomp_layers_timing() {
let mut s = crate::Session::default();
let pre = s.execute("comp.new", json!({"name": "Pre", "width": 32, "height": 32, "frameRate": 10, "duration": 4})).unwrap()["comp"].as_u64().unwrap();
for (t, c) in [(1.0, "beat"), (3.0, "drop")] {
s.execute("markers.set", json!({"new": true, "time": t, "comment": c})).unwrap();
}
s.execute("comp.new", json!({"name": "Main", "width": 32, "height": 32, "frameRate": 10, "duration": 20})).unwrap();
let l = s.execute("layer.addItem", json!({"item": pre})).unwrap()["layer"].as_u64().unwrap();
s.execute("layer.timing", json!({"layers": [l], "start": 2.0})).unwrap();
let times = |s: &mut crate::Session| -> Vec<(f64, String)> {
let v = s.execute_checked("markers.nested", json!({"layer": l})).unwrap();
v.as_array().unwrap().iter().map(|m| (m["time"].as_f64().unwrap(), m["comment"].as_str().unwrap().to_string())).collect()
};
assert_eq!(times(&mut s), vec![(3.0, "beat".to_string()), (5.0, "drop".to_string())]);
// Stretched to 200 %: twice as far from the start.
s.execute("layer.timeStretch", json!({"layers": [l], "percent": 200})).unwrap();
assert_eq!(times(&mut s), vec![(4.0, "beat".to_string()), (8.0, "drop".to_string())]);
s.execute("layer.timeStretch", json!({"layers": [l], "percent": 100})).unwrap();
// Trimmed: markers outside the In–Out range are not on the bar.
s.execute("layer.timing", json!({"layers": [l], "out": 4.0})).unwrap();
assert_eq!(times(&mut s), vec![(3.0, "beat".to_string())]);
s.execute("layer.timing", json!({"layers": [l], "out": 6.0})).unwrap();
// Time remapping at half speed: nested 1 s is reached 2 s into the layer.
s.execute("layer.enableTimeRemap", json!({"layers": [l]})).unwrap();
s.execute("prop.addKey", json!({"layer": l, "path": "timeRemap", "time": 4.0, "value": 2.0})).unwrap();
let t = times(&mut s);
assert_eq!(t.first(), Some(&(4.0, "beat".to_string())), "{t:?}");
// Not a precomp layer.
let solid = s.execute("layer.newSolid", json!({"color": "#ffffff"})).unwrap()["layer"].as_u64().unwrap();
assert!(s.execute("markers.nested", json!({"layer": solid})).is_err());
}

View File

@ -255,6 +255,7 @@ fn lift_and_extract_work_area() {
#[test]
fn label_group_purge_and_edit_original() {
crate::tests_roto::hold_roto_cache_test_lock();
let mut s = comp();
let a = solid(&mut s, "#ff0000");
let b = solid(&mut s, "#00ff00");

View File

@ -0,0 +1,88 @@
//! Settings ▸ Roto Brush and Settings ▸ Face Tracking: the model registry, choosing, installing
//! (verified, with its notice) and loading trained models.
use std::sync::Arc;
use effectcraft_segment::Task;
use serde_json::json;
use crate::Session;
use crate::config::{ConfigStore, DirConfig};
fn session(tag: &str) -> (Session, std::path::PathBuf) {
let d = std::env::temp_dir().join(format!("ec-models-{tag}-{}", std::process::id()));
let store: Arc<dyn ConfigStore> = Arc::new(DirConfig::new(d.join("config")));
(Session { config: Some(store), ..Default::default() }, d)
}
/// Poll until `task`'s model is in use (or a few seconds pass).
fn wait_loaded(s: &mut Session, task: Task) {
for _ in 0..400 {
s.poll_models();
if s.models.status(task).active.is_some() {
return;
}
std::thread::sleep(std::time::Duration::from_millis(20));
}
}
#[test]
fn models_are_listed_chosen_and_verified() {
crate::tests_roto::hold_roto_cache_test_lock();
let (mut s, d) = session("list");
std::fs::create_dir_all(&d).unwrap();
let bogus = d.join("bogus.bin");
std::fs::write(&bogus, b"not the weights").unwrap();
for (prefix, model, task) in [("roto", "mobilesam", Task::Mask), ("face", "mediapipe-face", Task::Face)] {
let list = s.execute_checked(&format!("{prefix}.models"), json!({})).unwrap();
let ids: Vec<&str> = list["models"].as_array().unwrap().iter().map(|m| m["id"].as_str().unwrap()).collect();
assert_eq!(ids, ["classical", model]);
let m = &list["models"][1];
assert_eq!((m["licence"].as_str(), m["installed"].as_bool()), (Some("Apache-2.0"), Some(false)));
assert!(m["url"].as_str().unwrap().starts_with("https://") && !m["authors"].as_str().unwrap().is_empty());
// Unknown ids, the other task's models and files that aren't the published weights are
// refused.
let other = if task == Task::Mask { "mediapipe-face" } else { "mobilesam" };
for id in ["nope", other] {
assert!(s.execute_checked(&format!("{prefix}.model.select"), json!({"id": id})).is_err(), "{prefix} {id}");
}
let path = bogus.to_string_lossy();
let e = s.execute_checked(&format!("{prefix}.model.install"), json!({"path": path})).unwrap_err().to_string();
assert!(e.contains("SHA-256"), "{e}");
assert!(s.execute_checked(&format!("{prefix}.model.install"), json!({"path": path, "id": model})).is_err());
// Choosing a model that isn't installed keeps the classic engine.
s.execute_checked(&format!("{prefix}.model.select"), json!({"id": model})).unwrap();
assert_eq!(s.chosen_model(task), model);
assert!(s.models.status(task).active.is_none());
assert!(crate::prefs::pages().iter().any(|p| p.id == prefix));
}
assert!(s.models.face().is_none());
assert_eq!(crate::prefs::page_id("Face Tracking"), Some("face"));
// With the real weights (EFFECTCRAFT_MOBILESAM=path/to/mobile_sam.pt): install, load, use.
if let Ok(path) = std::env::var("EFFECTCRAFT_MOBILESAM") {
s.execute_checked("roto.model.install", json!({"path": path})).unwrap();
wait_loaded(&mut s, Task::Mask);
assert_eq!(s.models.status(Task::Mask).active.as_deref(), Some("mobilesam"));
// Roto Brush 3.0 (the default version) now segments with it.
let params = crate::effects::Params { values: [("version".to_string(), effectcraft_keyframe::Value::Enum(2))].into_iter().collect() };
assert_eq!(crate::effects::roto::model_for(&params).map(|m| m.info().id), Some("mobilesam"));
s.execute_checked("roto.model.select", json!({"id": "classical"})).unwrap();
assert!(crate::effects::roto::model_for(&params).is_none());
}
// EFFECTCRAFT_FACE_LANDMARKER=path/to/face_landmarker.task: installed with its notice,
// loaded for face tracking, dropped when the classic tracker is chosen, removed.
if let Ok(path) = std::env::var("EFFECTCRAFT_FACE_LANDMARKER") {
let r = s.execute_checked("face.model.install", json!({"path": path})).unwrap();
assert_eq!(r["id"], "mediapipe-face");
let dir = s.models_dir().unwrap();
let notice = std::fs::read_to_string(dir.join("face_landmarker.task.NOTICE.txt")).unwrap();
assert!(notice.contains("Google LLC") && notice.contains("Apache-2.0"), "{notice}");
wait_loaded(&mut s, Task::Face);
assert_eq!(s.models.face().map(|m| m.info().id), Some("mediapipe-face"));
s.execute_checked("face.model.select", json!({"id": "classical"})).unwrap();
assert!(s.models.face().is_none());
assert_eq!(s.execute_checked("face.model.remove", json!({"id": "mediapipe-face"})).unwrap()["removed"], true);
assert!(!dir.join("face_landmarker.task").exists() && !dir.join("face_landmarker.task.NOTICE.txt").exists());
}
let _ = std::fs::remove_dir_all(d);
}

View File

@ -10,6 +10,21 @@ use serde_json::{Value, json};
use crate::Session;
use crate::tests_track::{H, W, frame_time, setup};
/// Keep cache/model mutation tests from clearing segmentations under another fixture.
/// Libtest runs each test on its own thread; the guard lives until that thread exits.
pub(crate) fn hold_roto_cache_test_lock() {
use std::cell::RefCell;
use std::sync::{Mutex, MutexGuard, PoisonError};
static LOCK: Mutex<()> = Mutex::new(());
thread_local! {
static HELD: RefCell<Option<MutexGuard<'static, ()>>> = const { RefCell::new(None) };
}
HELD.with(|h| {
if h.borrow().is_none() {
*h.borrow_mut() = Some(LOCK.lock().unwrap_or_else(PoisonError::into_inner));
}
});
}
const R: f64 = 40.0;
fn noise(x: f64, y: f64, seed: u32) -> f64 {
@ -74,6 +89,7 @@ fn line(a: [f64; 2], b: [f64; 2]) -> Value {
/// frame 0. `name`: the footage file name (segmentations are cached process-wide by content key,
/// which includes the footage's identity: different clips need different names).
fn painted(path: fn(u32) -> [f64; 2], name: &str) -> (Session, LayerId, u64) {
hold_roto_cache_test_lock();
let (mut s, clip, solid) = setup(move |f| draw(path(f)));
let cid = s.active_comp_id().unwrap();
s.edit("remove solid", None, |p, _| {

View File

@ -460,8 +460,8 @@ fn open_recent_increment_and_revert() {
assert!(s.prefs.recent_projects[0].ends_with("A.ecproj"));
s.execute("file.clearRecent", json!({})).unwrap();
assert!(!s.is_enabled("file.openRecent"));
assert_eq!(autosave::increment_path("/x/Intro 9.ecproj", |_| false), "/x/Intro 10.ecproj");
assert_eq!(autosave::increment_path("/x/v1.ecproj", |p| p.ends_with("v1 2.ecproj")), "/x/v1 3.ecproj");
assert_eq!(std::path::PathBuf::from(autosave::increment_path("/x/Intro 9.ecproj", |_| false)), std::path::PathBuf::from("/x/Intro 10.ecproj"));
assert_eq!(std::path::PathBuf::from(autosave::increment_path("/x/v1.ecproj", |p| p.ends_with("v1 2.ecproj"))), std::path::PathBuf::from("/x/v1 3.ecproj"));
let _ = std::fs::remove_dir_all(d);
}
@ -486,3 +486,38 @@ fn memory_tick_never_waits_for_the_system() {
assert!(got.is_some_and(|m| m.total > 0), "the reading arrives in the background");
}
}
#[test]
fn interface_language_is_validated_persisted_and_backward_compatible() {
let store = Arc::new(MemoryConfig::default());
let mut s = Session { config: Some(store.clone()), ..Default::default() };
assert_eq!(s.prefs.general.language, "en");
s.execute("prefs.set", json!({"key": "general.language", "value": "ja"})).unwrap();
assert_eq!(s.execute("prefs.get", json!({"key": "general.language"})).unwrap(), json!("ja"));
let saved = store.read(PREFS_FILE).unwrap();
for bad in [json!("fr"), json!(""), json!(17), json!(null)] {
assert!(s.execute("prefs.set", json!({"key": "general.language", "value": bad})).is_err());
assert_eq!(s.prefs.general.language, "ja");
assert_eq!(store.read(PREFS_FILE).unwrap(), saved);
}
let mut reloaded = Session { config: Some(store), ..Default::default() };
reloaded.load_settings();
assert_eq!(reloaded.prefs.general.language, "ja");
reloaded.execute("prefs.reset", json!({"page": "general"})).unwrap();
assert_eq!(reloaded.prefs.general.language, "en");
assert_eq!(Prefs::from_json(r#"{"general":{"undoLevels":17}}"#).general.language, "en");
assert_eq!(Prefs::from_json(r#"{"general":{"language":"unknown"}}"#).general.language, "en");
}
#[test]
fn unicode_label_colors_fall_back_without_panicking() {
use effectcraft_color::Label;
let mut s = Session::default();
let default = s.prefs.label_rgb(Label::Red);
for color in ["#€abc", "#a€bc", "#abc€"] {
s.execute("prefs.set", json!({"key": "labels.0.color", "value": color})).unwrap();
assert_eq!(s.prefs.label_rgb(Label::Red), default);
}
s.execute("prefs.set", json!({"key": "labels.0.color", "value": "#aBcD09"})).unwrap();
assert_eq!(s.prefs.label_rgb(Label::Red), [0xab, 0xcd, 9]);
}

View File

@ -284,9 +284,20 @@ fn add_output_module_encodes_twice() {
s.execute("renderQueue.setOutput", json!({"index": 1, "module": 2, "path": "/tmp/om/second.mov"})).unwrap();
assert!(s.execute("renderQueue.setOutput", json!({"index": 1, "module": 3, "path": "/tmp/x.mov"})).is_err());
s.execute("renderQueue.render", json!({})).unwrap();
assert_eq!(*ex.log.lock().unwrap(), vec!["H264:/tmp/om/main.mp4".to_string(), "ProRes:/tmp/om/second.mov".to_string()]);
assert_eq!(
ex.log
.lock()
.unwrap()
.iter()
.map(|entry| {
let (format, path) = entry.split_once(':').unwrap();
(format.to_string(), std::path::PathBuf::from(path))
})
.collect::<Vec<_>>(),
[("H264".to_string(), std::path::absolute("/tmp/om/main.mp4").unwrap()), ("ProRes".to_string(), std::path::absolute("/tmp/om/second.mov").unwrap())]
);
assert_eq!(s.project.render_queue[0].status.label(), "Done");
assert_eq!(s.project.render_queue[0].last_output.as_deref(), Some("/tmp/om/main.mp4"));
assert_eq!(std::path::Path::new(s.project.render_queue[0].last_output.as_deref().unwrap()), std::path::absolute("/tmp/om/main.mp4").unwrap());
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(s.project.render_queue[0].extra_outputs.len(), 1, "undo the Output To edit");
}
@ -307,7 +318,9 @@ fn pre_render_imports_and_replaces() {
assert_eq!(ex.log.lock().unwrap().len(), 1);
let l = &s.project.comp(outer).unwrap().layers[0];
let LayerSource::Footage { item } = l.source else { panic!("not replaced: {:?}", l.source) };
assert!(matches!(&s.project.item(item).unwrap().kind, effectcraft_project::ItemKind::Footage(f) if f.path == "/tmp/pre/inner.mov"));
assert!(
matches!(&s.project.item(item).unwrap().kind, effectcraft_project::ItemKind::Footage(f) if std::path::Path::new(&f.path) == std::path::absolute("/tmp/pre/inner.mov").unwrap())
);
s.execute("edit.undo", json!({})).unwrap();
assert!(matches!(s.project.comp(outer).unwrap().layers[0].source, LayerSource::Comp { .. }));
}
@ -323,13 +336,15 @@ fn post_render_actions_import_and_set_proxy() {
assert_eq!(s.project.render_queue[0].post_render, PostRenderAction::Import);
s.execute("renderQueue.render", json!({})).unwrap();
assert_eq!(s.project.items.len(), items_before + 1);
assert!(s.project.items.values().any(|i| matches!(&i.kind, effectcraft_project::ItemKind::Footage(f) if f.path == "/tmp/post/a.mov")));
assert!(s.project.items.values().any(
|i| matches!(&i.kind, effectcraft_project::ItemKind::Footage(f) if std::path::Path::new(&f.path) == std::path::absolute("/tmp/post/a.mov").unwrap())
));
// Set Proxy: the comp gets the render as its proxy.
let b = s.execute("renderQueue.add", json!({"output": "/tmp/post/b.mov"})).unwrap();
s.execute("renderQueue.setOutputModule", json!({"item": b["item"], "postRenderAction": "Set Proxy"})).unwrap();
s.execute("renderQueue.render", json!({})).unwrap();
let px = s.project.item(cid).unwrap().proxy.clone().expect("proxy set");
assert_eq!(px.footage.path, "/tmp/post/b.mov");
assert_eq!(std::path::Path::new(&px.footage.path), std::path::absolute("/tmp/post/b.mov").unwrap());
// None: nothing happens.
let n = s.project.items.len();
let c = s.execute("renderQueue.add", json!({"output": "/tmp/post/c.mov"})).unwrap();
@ -413,7 +428,7 @@ fn log_notify_and_overflow_settings() {
let (mut s, _) = rq();
let a = s.execute("renderQueue.add", json!({"output": "/tmp/n/a.mp4", "log": "plusPerFrameInfo"})).unwrap();
assert_eq!(a["logLabel"], "Plus Per Frame Info");
assert_eq!(a["logPath"], "/tmp/n/a_RenderLog.txt");
assert_eq!(std::path::Path::new(a["logPath"].as_str().unwrap()), std::path::absolute("/tmp/n/a_RenderLog.txt").unwrap());
s.execute("renderQueue.setLog", json!({"item": a["item"], "log": "plusSettings"})).unwrap();
assert_eq!(s.project.render_queue[0].log, effectcraft_project::render_queue::RenderLog::PlusSettings);
s.execute("renderQueue.setOverflowFolders", json!({"folders": ["/tmp/o1", " ", "/tmp/o2"]})).unwrap();

View File

@ -227,3 +227,35 @@ fn opentype_features_per_range_render_and_query() {
assert!(f["options"]["stylisticSets"].as_array().unwrap().contains(&json!(1)));
assert_eq!(f["options"]["smallCaps"], false);
}
#[test]
fn fonts_lists_bundled_and_installed_families() {
let mut s = Session::default();
let r = s.execute("text.fonts", json!({})).unwrap();
let fams = r["families"].as_array().unwrap();
assert_eq!(r["count"].as_u64().unwrap() as usize, fams.len());
let inter = fams.iter().find(|f| f["family"] == "Inter").unwrap();
assert_eq!(inter["origin"], "bundled");
assert!(inter["styles"].as_array().unwrap().contains(&json!("Bold")));
// Installed fonts are listed without any layer having asked for one.
use effectcraft_text::fonts::FontSource as _;
let installed = effectcraft_text::fonts::DirectorySource::system().faces();
for face in &installed {
assert!(fams.iter().any(|f| f["family"].as_str().unwrap().eq_ignore_ascii_case(&face.family)), "{} not listed", face.family);
}
// Filtered by name; rescanning keeps what was there.
let r = s.execute("text.fonts", json!({"query": "noto ser", "rescan": true})).unwrap();
assert_eq!(r["families"][0]["family"], "Noto Serif");
// The bundled face is Regular; the OS can also provide Bold, Italic, or other faces.
// Compare against the independent directory inventory rather than assuming an empty OS.
let mut expected_styles = std::collections::BTreeSet::from(["Regular".to_string()]);
expected_styles.extend(installed.iter().filter(|face| face.family.eq_ignore_ascii_case("Noto Serif")).map(|face| face.style.clone()));
let styles: Vec<String> = r["families"][0]["styles"].as_array().unwrap().iter().map(|style| style.as_str().unwrap().to_string()).collect();
assert_eq!(styles.iter().cloned().collect::<std::collections::BTreeSet<_>>(), expected_styles);
assert_eq!(styles.len(), expected_styles.len(), "style menu has no duplicates");
assert!(r["families"].as_array().unwrap().iter().all(|f| f["family"].as_str().unwrap().to_lowercase().contains("noto ser")));
// The style menus offer a family's own styles.
assert_eq!(crate::font_styles("noto serif"), styles);
assert_eq!(crate::font_styles("NOTO SERIF"), styles);
assert_eq!(crate::font_styles("No Such Family").len(), 5);
}

View File

@ -162,7 +162,7 @@ impl VideoEncoder for ProRes {
let mut fr = filmcraft_prores::Frame::new(self.w, self.h, chroma, 10, self.alpha);
rgba_to_yuv_10(rgba, self.w as usize, self.h as usize, self.is_444(), &mut fr.y, &mut fr.cb, &mut fr.cr);
if let Some(a) = fr.alpha.as_mut() {
for (d, s) in a.iter_mut().zip(rgba.chunks_exact(4)) {
for (d, s) in a.iter_mut().zip(rgba.as_chunks::<4>().0.iter()) {
*d = ((s[3] as u32 * 1023 + 127) / 255) as u16;
}
}
@ -217,7 +217,7 @@ fn deinterleave(buf: &[f32], channels: usize) -> Vec<Vec<f32>> {
/// (mono = the average of left and right).
pub(crate) fn mix(cx: &Cx, start: effectcraft_time::Tick, n: usize, sr: u32) -> Vec<f32> {
let st = effectcraft_render::audio::mix_comp(&cx.project, cx.footage, cx.expr, cx.comp, start, n, sr);
if cx.output.audio_channels == 1 { st.chunks_exact(2).map(|p| (p[0] + p[1]) * 0.5).collect() } else { st }
if cx.output.audio_channels == 1 { st.as_chunks::<2>().0.iter().map(|p| (p[0] + p[1]) * 0.5).collect() } else { st }
}
/// PCM bytes of samples in `fmt` (little or big endian).

View File

@ -322,7 +322,7 @@ fn write_still<W: std::io::Write + std::io::Seek>(
}
OutputFormat::JpegSequence => {
let px = cx.pixels(img, comp, if channels == Channels::Alpha { Channels::Alpha } else { Channels::Rgb }, w, h);
let rgb: Vec<u8> = px.chunks_exact(4).flat_map(|p| [p[0], p[1], p[2]]).collect();
let rgb: Vec<u8> = px.as_chunks::<4>().0.iter().flat_map(|p| [p[0], p[1], p[2]]).collect();
image::codecs::jpeg::JpegEncoder::new_with_quality(file, quality.clamp(1, 100))
.write_image(&rgb, w, h, ExtendedColorType::Rgb8)
.map_err(|e| ExportError::Encode(e.to_string()))
@ -331,8 +331,8 @@ fn write_still<W: std::io::Write + std::io::Seek>(
let px = cx.pixels(img, comp, channels, w, h);
let (data, ct) = match channels {
Channels::Rgba => (px, ExtendedColorType::Rgba8),
Channels::Alpha => (px.chunks_exact(4).map(|p| p[0]).collect(), ExtendedColorType::L8),
Channels::Rgb => (px.chunks_exact(4).flat_map(|p| [p[0], p[1], p[2]]).collect(), ExtendedColorType::Rgb8),
Channels::Alpha => (px.as_chunks::<4>().0.iter().map(|p| p[0]).collect(), ExtendedColorType::L8),
Channels::Rgb => (px.as_chunks::<4>().0.iter().flat_map(|p| [p[0], p[1], p[2]]).collect(), ExtendedColorType::Rgb8),
};
if fmt == OutputFormat::PngSequence {
image::codecs::png::PngEncoder::new(file).write_image(&data, w, h, ct).map_err(|e| ExportError::Encode(e.to_string()))

View File

@ -352,7 +352,7 @@ fn ffmpeg_oracle_decodes_ogg_opus() {
.expect("run ffmpeg");
let _ = std::fs::remove_file(&path);
assert!(out.status.success(), "ffmpeg failed: {}", String::from_utf8_lossy(&out.stderr));
let dec: Vec<f32> = out.stdout.chunks_exact(4).map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
let dec: Vec<f32> = out.stdout.as_chunks::<4>().0.iter().map(|b| f32::from_le_bytes([b[0], b[1], b[2], b[3]])).collect();
assert_eq!(dec.len(), len * c, "ffmpeg must honour pre-skip and the end granule");
for ch in 0..c {
let d: Vec<f32> = (0..len).map(|i| dec[i * c + ch]).collect();

View File

@ -224,7 +224,7 @@ fn render_logs() {
let r = r.unwrap();
let (path, text) = log_of(&files).expect("log");
assert_eq!(r.log.as_deref(), Some(path.as_str()));
assert_eq!(path, "/l/b_RenderLog.txt");
assert_eq!(std::path::Path::new(&path), std::path::Path::new("/l/b_RenderLog.txt"));
assert!(text.contains("Item: #1 Comp") && text.contains("Result: Done (5 frames"), "{text}");
assert!(text.contains("Field Render: Upper Field First") && text.contains("Template: DV-ish") && text.contains("Format: PNG Sequence"), "{text}");
assert_eq!(text.lines().filter(|l| l.contains("rendered in")).count(), 5, "{text}");
@ -239,7 +239,7 @@ fn render_logs() {
let (r, files) = run_with(&p, ItemId(424_242), &s, &gif, "/l/d.gif", JobOptions::default(), false);
assert!(r.is_err());
let (path, text) = log_of(&files).expect("error log");
assert_eq!(path, "/l/d_RenderLog.txt");
assert_eq!(std::path::Path::new(&path), std::path::Path::new("/l/d_RenderLog.txt"));
assert!(text.contains("Result: Failed") && text.contains("Error: composition not found") && !text.contains("Render Settings:"), "{text}");
// A user stop is not an error: no log.
let (r, files) = run_with(&p, cid, &s, &gif, "/l/e.gif", JobOptions::default(), true);
@ -255,7 +255,7 @@ struct Quota {
impl StorageQuota for Quota {
fn has_room(&self, path: &str, bytes: u64) -> bool {
if !path.starts_with("/primary/") {
if !std::path::Path::new(path).starts_with("/primary") {
return true;
}
let mut u = self.used.lock().unwrap();
@ -278,7 +278,14 @@ fn storage_overflow_moves_files_to_the_overflow_folder() {
let r = r.unwrap();
assert_eq!(r.overflow.len(), 10, "{:?}", r.overflow);
let f = files.lock().unwrap();
assert_eq!(f.iter().filter(|(p, _)| p.starts_with("/overflow/seq_") && p.ends_with(".png")).count(), 10);
assert_eq!(
f.iter()
.filter(|(p, _)| std::path::Path::new(p).parent() == Some(std::path::Path::new("/overflow"))
&& std::path::Path::new(p).file_name().is_some_and(|n| n.to_string_lossy().starts_with("seq_"))
&& p.ends_with(".png"))
.count(),
10
);
let log = f.iter().find(|(p, _)| p.ends_with("_RenderLog.txt")).map(|(_, d)| String::from_utf8_lossy(d).to_string()).unwrap();
assert!(log.contains("Storage overflow: 10 file(s)"), "{log}");
drop(f);
@ -288,7 +295,7 @@ fn storage_overflow_moves_files_to_the_overflow_folder() {
let opts = JobOptions { storage: Some(&q), overflow: vec!["/overflow".into()], ..Default::default() };
let (r, files) = run_with(&p, cid, &off, &om, "/primary/seq_[#####].png", opts, false);
assert!(r.unwrap().overflow.is_empty());
assert!(files.lock().unwrap().iter().all(|(p, _)| p.starts_with("/primary/")));
assert!(files.lock().unwrap().iter().all(|(p, _)| std::path::Path::new(p).starts_with("/primary")));
}
#[test]

View File

@ -482,7 +482,7 @@ fn wav_and_aiff_audio_only() {
} else {
// AIFF: big-endian samples after the SSND chunk's offset/block size.
let at = bytes.windows(4).position(|w| w == b"SSND").expect("SSND") + 16;
let left = bytes[at..].chunks_exact(4).skip(14_400).take(24_000).map(|c| i16::from_be_bytes([c[0], c[1]]) as f32 / 32767.0);
let left = bytes[at..].as_chunks::<4>().0.iter().skip(14_400).take(24_000).map(|c| i16::from_be_bytes([c[0], c[1]]) as f32 / 32767.0);
rms(left)
};
assert!((l - 0.3536).abs() < 0.01, "{name}: left RMS {l}");

View File

@ -21,9 +21,14 @@ effectcraft-keyframe = { workspace = true }
wgpu = { workspace = true }
bytemuck = { workspace = true }
log = { workspace = true }
web-time = { workspace = true }
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
pollster = { workspace = true }
[dev-dependencies]
effectcraft-text = { workspace = true }
[target.'cfg(target_arch = "wasm32")'.dependencies]
wasm-bindgen = "=0.2.129"
js-sys = "0.3"

View File

@ -2,7 +2,7 @@
//! (`Depth32Float`, Greater), the scene's materials, textures, lights and shadow maps in
//! storage buffers, and `advanced3d.wgsl`'s physically based fragment shader. Opaque triangles
//! are drawn with depth writes, then the sorted transparent ones blended over (premultiplied).
//! Colour (`Rgba16Float`) and camera depth (`R32Float`) stay on the GPU for `adv3d.wgsl`'s
//! Colour (`Rgba16Float`) and camera depth (`R32Uint` holding f32 bits) stay on the GPU for `adv3d.wgsl`'s
//! compute kernels, which run the rest of `effectcraft_render::three_d::adv::render_prepared`:
//! the 2×2 supersampling resolve, the motion-blur sub-sample average (nearest depth), the
//! depth-based iris depth of field (the Classic 3D bokeh kernel's row spans and prefix-sum
@ -21,21 +21,194 @@ use crate::context::{Enc, GpuContext, GpuImage};
/// Pipelines (built on first use).
pub(crate) struct Pipes {
bgl: wgpu::BindGroupLayout,
opaque: wgpu::RenderPipeline,
transparent: wgpu::RenderPipeline,
/// (opaque, transparent); `None` when the adapter can't rasterise ([`raster_unsupported`]):
/// scenes then render on the CPU.
raster: Option<(wgpu::BindGroupLayout, wgpu::RenderPipeline, wgpu::RenderPipeline)>,
post: Post,
}
fn pipes(g: &GpuContext) -> &Pipes {
g.adv3d.get_or_init(|| Pipes::new(&g.device))
g.adv3d.get_or_init(|| Pipes::new(&g.device, g.adv3d_raster))
}
const COLOR: wgpu::TextureFormat = wgpu::TextureFormat::Rgba16Float;
const DEPTH_OUT: wgpu::TextureFormat = wgpu::TextureFormat::R32Float;
/// Camera depth as the bits of an f32: integer targets render on every backend, while `R32Float`
/// needs `EXT_color_buffer_float` on GLES (absent on Mesa's llvmpipe, for one).
const DEPTH_OUT: wgpu::TextureFormat = wgpu::TextureFormat::R32Uint;
/// [`DEPTH_OUT`]'s clear value: the bits of −1.0 ("nothing drawn").
const NO_DEPTH: f64 = 0xBF80_0000u32 as f64;
const DEPTH: wgpu::TextureFormat = wgpu::TextureFormat::Depth32Float;
const VERTEX_SIZE: u64 = 52;
/// Why `adapter` can't run the rasteriser, or `None` when it can. Building the pipelines anyway
/// would be a validation error, which release builds only log (and then draw nothing), so the
/// caller keeps the scenes on the CPU instead.
pub(crate) fn raster_unsupported(adapter: &wgpu::Adapter) -> Option<String> {
raster_capability_reason(adapter.get_downlevel_capabilities().flags, |format| adapter.get_texture_format_features(format))
}
fn raster_capability_reason(flags: wgpu::DownlevelFlags, features: impl Fn(wgpu::TextureFormat) -> wgpu::TextureFormatFeatures) -> Option<String> {
use wgpu::TextureFormatFeatureFlags as F;
use wgpu::TextureUsages as U;
if !flags.contains(wgpu::DownlevelFlags::FRAGMENT_STORAGE) {
return Some("no fragment storage buffers".into());
}
let targets = U::RENDER_ATTACHMENT | U::TEXTURE_BINDING | U::COPY_SRC;
let needs = [(COLOR, targets, F::BLENDABLE), (DEPTH_OUT, targets, F::empty()), (DEPTH, U::RENDER_ATTACHMENT, F::empty())];
for (format, usages, required_flags) in needs {
let f = features(format);
if !f.allowed_usages.contains(usages) || !f.flags.contains(required_flags) {
return Some(format!("{format:?} lacks raster usages {usages:?} or flags {required_flags:?}"));
}
}
// The transparent pipeline blends colour and masks the depth target off.
if !flags.contains(wgpu::DownlevelFlags::INDEPENDENT_BLEND) {
return Some("no independent blending".into());
}
None
}
/// Bound one allocation by its actual byte stride, without imposing a global image limit.
fn buffer_bytes(count: u64, stride: u64, limit: u64) -> Option<u64> {
let size = count.checked_mul(stride)?;
(size > 0 && size <= limit && usize::try_from(size).is_ok()).then_some(size)
}
/// Sizes of exactly the arrays serialized by pack, including its empty storage padding.
/// Checking these before serialization bounds host packing as well as the GPU uploads.
fn packed_bytes(counts: [u64; 6], uniform_limit: u64, storage_limit: u64, buffer_limit: u64) -> Option<[u64; 7]> {
let mut sizes = [0; 7];
sizes[0] = buffer_bytes(40, 4, uniform_limit.min(buffer_limit))?;
for (index, (count, stride)) in counts.into_iter().zip([96, 16, 16, 80, 80, 4]).enumerate() {
let size = count.checked_mul(stride)?.max(16);
sizes[index + 1] = buffer_bytes(size, 1, storage_limit.min(buffer_limit))?;
}
Some(sizes)
}
#[cfg(test)]
mod capability_tests {
use super::*;
fn supported(format: wgpu::TextureFormat) -> wgpu::TextureFormatFeatures {
let rt = wgpu::TextureUsages::RENDER_ATTACHMENT;
wgpu::TextureFormatFeatures {
allowed_usages: if format == DEPTH { rt } else { rt | wgpu::TextureUsages::TEXTURE_BINDING | wgpu::TextureUsages::COPY_SRC },
flags: if format == COLOR { wgpu::TextureFormatFeatureFlags::BLENDABLE } else { wgpu::TextureFormatFeatureFlags::empty() },
}
}
fn downlevel() -> wgpu::DownlevelFlags {
wgpu::DownlevelFlags::FRAGMENT_STORAGE | wgpu::DownlevelFlags::INDEPENDENT_BLEND
}
#[test]
fn matching_raster_capabilities_are_accepted_without_optional_features() {
assert!(raster_capability_reason(downlevel(), supported).is_none());
}
#[test]
fn fragment_storage_and_independent_blending_are_required() {
for flag in [wgpu::DownlevelFlags::FRAGMENT_STORAGE, wgpu::DownlevelFlags::INDEPENDENT_BLEND] {
assert!(raster_capability_reason(downlevel() - flag, supported).is_some());
}
}
#[test]
fn color_and_integer_depth_require_their_actual_texture_usages() {
for target in [COLOR, DEPTH_OUT] {
for usage in [wgpu::TextureUsages::RENDER_ATTACHMENT, wgpu::TextureUsages::TEXTURE_BINDING, wgpu::TextureUsages::COPY_SRC] {
assert!(
raster_capability_reason(downlevel(), |format| {
let mut features = supported(format);
if format == target {
features.allowed_usages.remove(usage);
}
features
})
.is_some()
);
}
}
}
#[test]
fn color_blending_and_depth_attachment_are_required() {
for target in [COLOR, DEPTH] {
assert!(
raster_capability_reason(downlevel(), |format| {
let mut features = supported(format);
if format == target {
if target == COLOR {
features.flags.remove(wgpu::TextureFormatFeatureFlags::BLENDABLE);
} else {
features.allowed_usages.remove(wgpu::TextureUsages::RENDER_ATTACHMENT);
}
}
features
})
.is_some()
);
}
}
#[test]
fn byte_guards_use_each_buffer_stride_and_reject_overflow() {
assert_eq!(buffer_bytes(2, VERTEX_SIZE, 104), Some(104));
assert_eq!(buffer_bytes(2, VERTEX_SIZE, 103), None);
assert_eq!(buffer_bytes(26, 4, 104), Some(104));
assert_eq!(buffer_bytes(27, 4, 104), None);
assert_eq!(buffer_bytes(0, 4, 104), None);
assert_eq!(buffer_bytes(u64::MAX, 4, u64::MAX), None);
assert_eq!(buffer_bytes(1, 16, 15), None);
}
#[test]
fn packed_arrays_preflight_exact_strides_and_empty_padding() {
assert_eq!(packed_bytes([0; 6], 160, 16, 160), Some([160, 16, 16, 16, 16, 16, 16]));
assert_eq!(packed_bytes([2; 6], 160, 192, 192), Some([160, 192, 32, 32, 160, 160, 16]));
assert!(packed_bytes([0; 6], 159, 16, 160).is_none());
assert!(packed_bytes([0; 6], 160, 15, 160).is_none());
assert!(packed_bytes([2; 6], 160, 192, 191).is_none());
for index in 0..6 {
let mut counts = [0; 6];
counts[index] = u64::MAX;
assert!(packed_bytes(counts, u64::MAX, u64::MAX, u64::MAX).is_none());
}
}
#[test]
fn predicted_sizes_match_the_real_scene_serializer() {
use effectcraft_render::three_d::adv::{Light, Material, ShadowMap, TexInfo};
let scene = Scene {
materials: vec![Material::default()],
textures: vec![TexInfo { offset: 0, width: 1, height: 1, wrap_u: 0, wrap_v: 0 }],
texels: vec![[0.0; 4]],
lights: vec![Light {
kind: 0,
pos: [0.0; 3],
dir: [0.0; 3],
color: [0.0; 3],
cos_inner: 0.0,
cos_outer: 0.0,
falloff: 0,
radius: 0.0,
falloff_distance: 0.0,
shadow: -1,
shadow_darkness: 0.0,
shadow_diffusion: 0.0,
}],
shadows: vec![ShadowMap { view: [[0.0; 4]; 3], focal: 0.0, ortho: false, size: 1, offset: 0, bias: 0.0, radius: 0.0 }],
shadow_texels: vec![0.0],
..Scene::default()
};
for (scene, counts) in [(Scene::default(), [0; 6]), (scene, [1; 6])] {
let actual = pack(&scene).map(|bytes| bytes.len() as u64);
assert_eq!(Some(actual), packed_bytes(counts, 160, 160, 160));
}
}
}
fn storage(binding: u32) -> wgpu::BindGroupLayoutEntry {
wgpu::BindGroupLayoutEntry {
binding,
@ -46,71 +219,79 @@ fn storage(binding: u32) -> wgpu::BindGroupLayoutEntry {
}
impl Pipes {
fn new(device: &wgpu::Device) -> Pipes {
let module = device.create_shader_module(wgpu::ShaderModuleDescriptor {
label: Some("effectcraft advanced 3d"),
source: wgpu::ShaderSource::Wgsl(include_str!("shaders/advanced3d.wgsl").into()),
fn new(device: &wgpu::Device, raster: bool) -> Pipes {
let raster = raster.then(|| {
let module = device.create_shader_module(wgpu::ShaderModuleDescriptor {
label: Some("effectcraft advanced 3d"),
source: wgpu::ShaderSource::Wgsl(include_str!("shaders/advanced3d.wgsl").into()),
});
let mut entries = vec![wgpu::BindGroupLayoutEntry {
binding: 0,
visibility: wgpu::ShaderStages::VERTEX_FRAGMENT,
ty: wgpu::BindingType::Buffer { ty: wgpu::BufferBindingType::Uniform, has_dynamic_offset: false, min_binding_size: None },
count: None,
}];
entries.extend((1..=6).map(storage));
let bgl = device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor { label: Some("advanced 3d"), entries: &entries });
let layout = device.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor {
label: Some("advanced 3d"),
bind_group_layouts: &[Some(&bgl)],
immediate_size: 0,
});
let attrs = [
wgpu::VertexAttribute { format: wgpu::VertexFormat::Float32x3, offset: 0, shader_location: 0 },
wgpu::VertexAttribute { format: wgpu::VertexFormat::Float32x3, offset: 12, shader_location: 1 },
wgpu::VertexAttribute { format: wgpu::VertexFormat::Float32x2, offset: 24, shader_location: 2 },
wgpu::VertexAttribute { format: wgpu::VertexFormat::Float32x4, offset: 32, shader_location: 3 },
wgpu::VertexAttribute { format: wgpu::VertexFormat::Uint32, offset: 48, shader_location: 4 },
];
let make = |transparent: bool| {
let blend = transparent.then_some(wgpu::BlendState::PREMULTIPLIED_ALPHA_BLENDING);
device.create_render_pipeline(&wgpu::RenderPipelineDescriptor {
label: Some(if transparent { "advanced 3d transparent" } else { "advanced 3d opaque" }),
layout: Some(&layout),
vertex: wgpu::VertexState {
module: &module,
entry_point: Some("vs"),
compilation_options: Default::default(),
buffers: &[Some(wgpu::VertexBufferLayout { array_stride: VERTEX_SIZE, step_mode: wgpu::VertexStepMode::Vertex, attributes: &attrs })],
},
primitive: wgpu::PrimitiveState {
topology: wgpu::PrimitiveTopology::TriangleList,
front_face: wgpu::FrontFace::Ccw,
cull_mode: None,
..Default::default()
},
depth_stencil: Some(wgpu::DepthStencilState {
format: DEPTH,
depth_write_enabled: Some(!transparent),
depth_compare: Some(wgpu::CompareFunction::Greater),
stencil: Default::default(),
bias: Default::default(),
}),
multisample: Default::default(),
fragment: Some(wgpu::FragmentState {
module: &module,
entry_point: Some("fs"),
compilation_options: Default::default(),
targets: &[
Some(wgpu::ColorTargetState { format: COLOR, blend, write_mask: wgpu::ColorWrites::ALL }),
Some(wgpu::ColorTargetState {
format: DEPTH_OUT,
blend: None,
write_mask: if transparent { wgpu::ColorWrites::empty() } else { wgpu::ColorWrites::ALL },
}),
],
}),
multiview_mask: None,
cache: None,
})
};
let opaque = make(false);
let transparent = make(true);
(bgl, opaque, transparent)
});
let mut entries = vec![wgpu::BindGroupLayoutEntry {
binding: 0,
visibility: wgpu::ShaderStages::VERTEX_FRAGMENT,
ty: wgpu::BindingType::Buffer { ty: wgpu::BufferBindingType::Uniform, has_dynamic_offset: false, min_binding_size: None },
count: None,
}];
entries.extend((1..=6).map(storage));
let bgl = device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor { label: Some("advanced 3d"), entries: &entries });
let layout =
device.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor { label: Some("advanced 3d"), bind_group_layouts: &[Some(&bgl)], immediate_size: 0 });
let attrs = [
wgpu::VertexAttribute { format: wgpu::VertexFormat::Float32x3, offset: 0, shader_location: 0 },
wgpu::VertexAttribute { format: wgpu::VertexFormat::Float32x3, offset: 12, shader_location: 1 },
wgpu::VertexAttribute { format: wgpu::VertexFormat::Float32x2, offset: 24, shader_location: 2 },
wgpu::VertexAttribute { format: wgpu::VertexFormat::Float32x4, offset: 32, shader_location: 3 },
wgpu::VertexAttribute { format: wgpu::VertexFormat::Uint32, offset: 48, shader_location: 4 },
];
let make = |transparent: bool| {
let blend = transparent.then_some(wgpu::BlendState::PREMULTIPLIED_ALPHA_BLENDING);
device.create_render_pipeline(&wgpu::RenderPipelineDescriptor {
label: Some(if transparent { "advanced 3d transparent" } else { "advanced 3d opaque" }),
layout: Some(&layout),
vertex: wgpu::VertexState {
module: &module,
entry_point: Some("vs"),
compilation_options: Default::default(),
buffers: &[Some(wgpu::VertexBufferLayout { array_stride: VERTEX_SIZE, step_mode: wgpu::VertexStepMode::Vertex, attributes: &attrs })],
},
primitive: wgpu::PrimitiveState {
topology: wgpu::PrimitiveTopology::TriangleList,
front_face: wgpu::FrontFace::Ccw,
cull_mode: None,
..Default::default()
},
depth_stencil: Some(wgpu::DepthStencilState {
format: DEPTH,
depth_write_enabled: Some(!transparent),
depth_compare: Some(wgpu::CompareFunction::Greater),
stencil: Default::default(),
bias: Default::default(),
}),
multisample: Default::default(),
fragment: Some(wgpu::FragmentState {
module: &module,
entry_point: Some("fs"),
compilation_options: Default::default(),
targets: &[
Some(wgpu::ColorTargetState { format: COLOR, blend, write_mask: wgpu::ColorWrites::ALL }),
Some(wgpu::ColorTargetState {
format: DEPTH_OUT,
blend: None,
write_mask: if transparent { wgpu::ColorWrites::empty() } else { wgpu::ColorWrites::ALL },
}),
],
}),
multiview_mask: None,
cache: None,
})
};
Pipes { opaque: make(false), transparent: make(true), bgl, post: Post::new(device) }
Pipes { raster, post: Post::new(device) }
}
}
@ -222,20 +403,33 @@ fn half_to_f32(h: u16) -> f32 {
}
}
/// Record the render pass of one scene into `e`: (colour `Rgba16Float`, camera depth `R32Float`
/// Record the render pass of one scene into `e`: (colour `Rgba16Float`, camera depth `R32Uint`
/// with −1 where nothing was drawn) at the scene's raster size. `None` when the device can't
/// (size or buffer limits).
/// (render targets, size or buffer limits).
fn raster_into(e: &mut Enc, s: &Scene) -> Option<(wgpu::Texture, wgpu::Texture)> {
let g = e.g;
if !g.fits(s.width, s.height) || s.indices.is_empty() {
if !g.adv3d_raster || !g.fits(s.width, s.height) || s.indices.is_empty() {
return None;
}
let limits = g.device.limits();
buffer_bytes(u64::try_from(s.vertices.len()).ok()?, VERTEX_SIZE, limits.max_buffer_size)?;
buffer_bytes(u64::try_from(s.indices.len()).ok()?, 4, limits.max_buffer_size)?;
let n = u32::try_from(s.indices.len()).ok()?;
if s.opaque_count > n {
return None;
}
let counts = [s.materials.len(), s.textures.len(), s.texels.len(), s.lights.len(), s.shadows.len(), s.shadow_texels.len()];
let mut checked_counts = [0; 6];
for (out, count) in checked_counts.iter_mut().zip(counts) {
*out = u64::try_from(count).ok()?;
}
let expected_sizes = packed_bytes(checked_counts, limits.max_uniform_buffer_binding_size, limits.max_storage_buffer_binding_size, limits.max_buffer_size)?;
let bufs = pack(s);
if bufs.iter().any(|b| b.len() as u64 > limits.max_storage_buffer_binding_size) {
if bufs.iter().zip(expected_sizes).any(|(bytes, expected)| bytes.len() as u64 != expected) {
return None;
}
let p = pipes(g);
let Some((bgl, opaque, transparent)) = &p.raster else { return None };
let dev = &g.device;
let vb: Vec<u8> = s
.vertices
@ -261,7 +455,7 @@ fn raster_into(e: &mut Enc, s: &Scene) -> Option<(wgpu::Texture, wgpu::Texture)>
for (i, b) in sbs.iter().enumerate() {
entries.push(wgpu::BindGroupEntry { binding: i as u32 + 1, resource: b.as_entire_binding() });
}
let bg = dev.create_bind_group(&wgpu::BindGroupDescriptor { label: Some("adv3d"), layout: &p.bgl, entries: &entries });
let bg = dev.create_bind_group(&wgpu::BindGroupDescriptor { label: Some("adv3d"), layout: bgl, entries: &entries });
let tex = |format: wgpu::TextureFormat, usage: wgpu::TextureUsages| {
dev.create_texture(&wgpu::TextureDescriptor {
label: Some("adv3d target"),
@ -294,7 +488,7 @@ fn raster_into(e: &mut Enc, s: &Scene) -> Option<(wgpu::Texture, wgpu::Texture)>
view: &zv,
depth_slice: None,
resolve_target: None,
ops: wgpu::Operations { load: wgpu::LoadOp::Clear(wgpu::Color { r: -1.0, g: 0.0, b: 0.0, a: 0.0 }), store: wgpu::StoreOp::Store },
ops: wgpu::Operations { load: wgpu::LoadOp::Clear(wgpu::Color { r: NO_DEPTH, g: 0.0, b: 0.0, a: 0.0 }), store: wgpu::StoreOp::Store },
}),
],
depth_stencil_attachment: Some(wgpu::RenderPassDepthStencilAttachment {
@ -309,13 +503,12 @@ fn raster_into(e: &mut Enc, s: &Scene) -> Option<(wgpu::Texture, wgpu::Texture)>
pass.set_bind_group(0, &bg, &[]);
pass.set_vertex_buffer(0, vbuf.slice(..));
pass.set_index_buffer(ibuf.slice(..), wgpu::IndexFormat::Uint32);
let n = s.indices.len() as u32;
if s.opaque_count > 0 {
pass.set_pipeline(&p.opaque);
pass.set_pipeline(opaque);
pass.draw_indexed(0..s.opaque_count, 0, 0..1);
}
if n > s.opaque_count {
pass.set_pipeline(&p.transparent);
pass.set_pipeline(transparent);
pass.draw_indexed(s.opaque_count..n, 0, 0..1);
}
}
@ -362,9 +555,11 @@ pub(crate) fn render(g: &GpuContext, s: &Scene) -> Option<Target> {
if cb.len() != (s.width * s.height * 8) as usize || zb.len() != (s.width * s.height * 4) as usize {
return None;
}
let color = cb.chunks_exact(8).map(|c| [0, 1, 2, 3].map(|k| half_to_f32(u16::from_le_bytes([c[2 * k], c[2 * k + 1]])))).collect();
let color = cb.as_chunks::<8>().0.iter().map(|c| [0, 1, 2, 3].map(|k| half_to_f32(u16::from_le_bytes([c[2 * k], c[2 * k + 1]])))).collect();
let depth = zb
.chunks_exact(4)
.as_chunks::<4>()
.0
.iter()
.map(|c| {
let z = f32::from_le_bytes([c[0], c[1], c[2], c[3]]);
if z < 0.0 { f32::INFINITY } else { z }
@ -424,14 +619,10 @@ impl Post {
label: Some("effectcraft advanced 3d post"),
source: wgpu::ShaderSource::Wgsl(include_str!("shaders/adv3d.wgsl").into()),
});
let tex = |binding| wgpu::BindGroupLayoutEntry {
let tex = |binding, sample_type| wgpu::BindGroupLayoutEntry {
binding,
visibility: wgpu::ShaderStages::COMPUTE,
ty: wgpu::BindingType::Texture {
sample_type: wgpu::TextureSampleType::Float { filterable: false },
view_dimension: wgpu::TextureViewDimension::D2,
multisampled: false,
},
ty: wgpu::BindingType::Texture { sample_type, view_dimension: wgpu::TextureViewDimension::D2, multisampled: false },
count: None,
};
let buf = |binding, read_only| wgpu::BindGroupLayoutEntry {
@ -447,8 +638,8 @@ impl Post {
ty: wgpu::BindingType::Buffer { ty: wgpu::BufferBindingType::Uniform, has_dynamic_offset: false, min_binding_size: None },
count: None,
},
tex(1),
tex(2),
tex(1, wgpu::TextureSampleType::Float { filterable: false }),
tex(2, wgpu::TextureSampleType::Uint),
buf(3, false),
buf(4, false),
buf(5, false),
@ -581,6 +772,11 @@ fn read_buffer(e: &mut Enc, buf: &wgpu::Buffer, size: u64) -> Option<Vec<u8>> {
if !e.g.can_wait() {
return None;
}
buffer_bytes(size, 1, e.g.device.limits().max_buffer_size)?;
let row = usize::try_from(size).ok()?;
if size > buf.size() || !size.is_multiple_of(wgpu::COPY_BUFFER_ALIGNMENT) {
return None;
}
let rb = e.g.device.create_buffer(&wgpu::BufferDescriptor {
label: Some("adv3d readback"),
size,
@ -589,14 +785,7 @@ fn read_buffer(e: &mut Enc, buf: &wgpu::Buffer, size: u64) -> Option<Vec<u8>> {
});
e.encoder().copy_buffer_to_buffer(buf, 0, &rb, 0, size);
e.submit();
let (tx, rx) = std::sync::mpsc::channel();
rb.map_async(wgpu::MapMode::Read, .., move |r| {
let _ = tx.send(r);
});
e.g.device.poll(wgpu::PollType::wait_indefinitely()).ok()?;
rx.recv().ok()?.ok()?;
let out = rb.get_mapped_range(..).ok()?.to_vec();
rb.unmap();
let out = e.g.read_buffer(&rb, row, row, 1).map_err(|e| log::error!("gpu 3d readback: {e}")).ok()?;
Some(out)
}
@ -622,12 +811,12 @@ pub(crate) fn resolve_run(e: &mut Enc, prep: &Prepared) -> Option<Option<Resolve
return None;
}
let px = w as u64 * h as u64;
let max = g.device.limits().max_storage_buffer_binding_size;
if px * 16 > max {
return None;
}
let acc = storage_buf(g, px * 16, "adv3d acc");
let depth = storage_buf(g, px * 4, "adv3d depth");
let limits = g.device.limits();
let max = limits.max_storage_buffer_binding_size.min(limits.max_buffer_size);
let acc_bytes = buffer_bytes(px, 16, max)?;
let depth_bytes = buffer_bytes(px, 4, max)?;
let acc = storage_buf(g, acc_bytes, "adv3d acc");
let depth = storage_buf(g, depth_bytes, "adv3d depth");
let n = prep.samples.max(1);
let weight = if n <= 1 { 1.0 } else { 1.0 / n as f32 };
let mut first = true;
@ -636,7 +825,7 @@ pub(crate) fn resolve_run(e: &mut Enc, prep: &Prepared) -> Option<Option<Resolve
if s.indices.is_empty() {
continue;
}
if s.width != w * s.ssaa.max(1) || s.height != h * s.ssaa.max(1) {
if Some(s.width) != w.checked_mul(s.ssaa.max(1)) || Some(s.height) != h.checked_mul(s.ssaa.max(1)) {
return None;
}
let (color, zout) = raster_into(e, &s)?;
@ -661,9 +850,10 @@ pub(crate) fn resolve_run(e: &mut Enc, prep: &Prepared) -> Option<Option<Resolve
fn depth_of_field(e: &mut Enc, acc: &wgpu::Buffer, depth: &wgpu::Buffer, (w, h): (u32, u32), dof: &Dof, scale: f64) -> Option<()> {
let g = e.g;
let px = w as u64 * h as u64;
if (w as u64 + 1) * h as u64 * 16 > g.device.limits().max_storage_buffer_binding_size {
return None;
}
let limits = g.device.limits();
let max = limits.max_storage_buffer_binding_size.min(limits.max_buffer_size);
let prefix_bytes = buffer_bytes((u64::from(w) + 1).checked_mul(u64::from(h))?, 16, max)?;
buffer_bytes(1, 16, max)?;
let radius = storage_buf(g, px * 4, "adv3d radius");
let mut init = Vec::with_capacity(16);
for v in [f32::INFINITY.to_bits(), 0, 0, 0] {
@ -701,12 +891,12 @@ fn depth_of_field(e: &mut Enc, acc: &wgpu::Buffer, depth: &wgpu::Buffer, (w, h):
}
}
let table: Vec<u8> = heads.iter().chain(&spans).flatten().flat_map(|x| x.to_le_bytes()).collect();
if table.len() as u64 > g.device.limits().max_storage_buffer_binding_size {
if table.len() as u64 > max {
return None;
}
let table = e.bytes(table);
let src = storage_buf(g, px * 16, "adv3d dof source");
let prefix = storage_buf(g, (w as u64 + 1) * h as u64 * 16, "adv3d prefix");
let prefix = storage_buf(g, prefix_bytes, "adv3d prefix");
let hl = dof.highlight;
let b = Binds { bufs: [Some(acc), None, Some(&src), None, Some(&radius), None, None], ..Default::default() };
dispatch(e, "boost", [w, h, 0, 0], [hl.gain as f32, hl.threshold as f32, hl.saturation as f32, 0.0], b, pixel_groups(w, h));
@ -744,7 +934,9 @@ pub(crate) fn render_prepared(g: &GpuContext, prep: &Prepared) -> Option<Rendere
let px = r.width as u64 * r.height as u64;
let db = read_buffer(&mut e, &r.depth, px * 4)?;
let depth = db
.chunks_exact(4)
.as_chunks::<4>()
.0
.iter()
.map(|c| {
let z = f32::from_le_bytes([c[0], c[1], c[2], c[3]]);
if z >= BIG * 0.5 { f32::INFINITY } else { z }
@ -763,6 +955,8 @@ fn render_prepared_deferred(g: &GpuContext, d: &std::sync::Arc<crate::deferred::
return None;
}
let (w, h) = prep.out;
let size = buffer_bytes(u64::from(w).checked_mul(u64::from(h))?, 4, g.device.limits().max_buffer_size)?;
let row = usize::try_from(size).ok()?;
let mut k = crate::deferred::Key::new(5);
k.u64(w as u64);
k.u64(h as u64);
@ -791,7 +985,9 @@ fn render_prepared_deferred(g: &GpuContext, d: &std::sync::Arc<crate::deferred::
dst.copy_from_slice(&a.bytes);
let depth = b
.bytes
.chunks_exact(4)
.as_chunks::<4>()
.0
.iter()
.map(|c| {
let z = f32::from_le_bytes([c[0], c[1], c[2], c[3]]);
if z >= BIG * 0.5 { f32::INFINITY } else { z }
@ -809,7 +1005,6 @@ fn render_prepared_deferred(g: &GpuContext, d: &std::sync::Arc<crate::deferred::
let Some(r) = resolve_run(&mut e, prep)? else { return Some(None) };
let img = finish(&mut e, &r, !prep.linear, None);
let _ = e.read_keyed(&img, ki, [0.0; 2], 1.0);
let size = w as u64 * h as u64 * 4;
let rb = g.device.create_buffer(&wgpu::BufferDescriptor {
label: Some("adv3d depth readback"),
size,
@ -819,14 +1014,9 @@ fn render_prepared_deferred(g: &GpuContext, d: &std::sync::Arc<crate::deferred::
e.encoder().copy_buffer_to_buffer(&r.depth, 0, &rb, 0, size);
e.submit();
let done = d.start(kz, w, h, [0.0; 2], 1.0);
let b = rb.clone();
rb.map_async(wgpu::MapMode::Read, .., move |res| {
let out = res.ok().and_then(|_| b.get_mapped_range(..).ok().map(|v| v.to_vec()));
b.unmap();
done(out);
});
g.map_readback(&rb, row, row, 1, move |r| done(r.map_err(|e| log::error!("gpu depth readback: {e}")).ok()));
#[cfg(not(target_arch = "wasm32"))]
let _ = g.device.poll(wgpu::PollType::Poll);
let _ = g.poll_readbacks(wgpu::PollType::Poll);
d.miss();
Some(None)
}

View File

@ -259,11 +259,15 @@ pub struct GpuContext {
zeros: Mutex<HashMap<(u32, u32), GpuImage>>,
/// Advanced 3D render pipelines (built on first use).
pub(crate) adv3d: std::sync::OnceLock<crate::adv3d::Pipes>,
/// The adapter can run the Advanced 3D rasteriser (`adv3d::raster_unsupported`); without
/// it Advanced 3D scenes render on the CPU.
pub(crate) adv3d_raster: bool,
/// GPU particle pipeline (built on first use) and simulation checkpoints.
pub(crate) particles: crate::particles::PipesCell,
pub(crate) particle_states: crate::particles::StatesCell,
/// Deferred readbacks (a browser worker's WebGPU device), see [`crate::deferred`].
pub(crate) deferred: Option<Arc<crate::deferred::Deferred>>,
readbacks: Arc<crate::readback::Readbacks>,
}
fn tex_entry(binding: u32) -> wgpu::BindGroupLayoutEntry {
@ -288,21 +292,75 @@ fn storage_tex_entry(binding: u32, format: wgpu::TextureFormat) -> wgpu::BindGro
}
}
/// Contain synchronous native initialization errors without replacing the device owner's
/// handlers. Scopes are thread-local with wgpu's enabled `std` feature. The pinned native
/// wgpu-core backend returns an immediately-ready future from pop_error_scope, so these
/// drains do not poll or wait for unrelated submitted GPU work.
#[cfg(not(target_arch = "wasm32"))]
pub(crate) fn init_resource<T>(device: &wgpu::Device, label: &str, create: impl FnOnce() -> T) -> Result<T, String> {
let oom = device.push_error_scope(wgpu::ErrorFilter::OutOfMemory);
let internal = device.push_error_scope(wgpu::ErrorFilter::Internal);
let validation = device.push_error_scope(wgpu::ErrorFilter::Validation);
// Some native drivers/wgpu paths can still unwind instead of reporting a scoped error.
// Keep the last-resort guard inside the scopes, then drain every scope even on failure.
let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(create));
let validation_error = pollster::block_on(validation.pop());
let internal_error = pollster::block_on(internal.pop());
let oom_error = pollster::block_on(oom.pop());
if let Some(error) = validation_error.or(internal_error).or(oom_error) {
return Err(format!("GPU initialization ({label}): {error}"));
}
result.map_err(|payload| {
let reason =
payload.downcast_ref::<String>().map(String::as_str).or_else(|| payload.downcast_ref::<&str>().copied()).unwrap_or("native backend panicked");
format!("GPU initialization ({label}): {reason}")
})
}
/// Browser scope completion is asynchronous. Preserve the existing browser constructor;
/// never block its event loop or introduce a guard across an asynchronous boundary here.
#[cfg(target_arch = "wasm32")]
fn init_resource<T>(_device: &wgpu::Device, _label: &str, create: impl FnOnce() -> T) -> Result<T, String> {
Ok(create())
}
impl GpuContext {
/// Build on an existing device (the desktop app shares egui-wgpu's). `Err` when the
/// adapter cannot run the compositor (no compute shaders, e.g. WebGL2; no float storage
/// textures).
/// textures) or the device was created with limits below what its pipelines bind.
/// Preserves the host's device handlers. The host must forward failures using
/// [`Self::device_failure_notifier`] for immediate readback cancellation.
pub fn new(adapter: &wgpu::Adapter, device: wgpu::Device, queue: wgpu::Queue) -> Result<GpuContext, String> {
let info = adapter.get_info();
if !adapter.get_downlevel_capabilities().flags.contains(wgpu::DownlevelFlags::COMPUTE_SHADERS) {
return Err(format!("{} ({:?}): no compute shaders", info.name, info.backend));
}
// egui-wgpu asks for WebGL2's limits on GL (no storage buffers at all): building the
// pipelines there would be a validation error, which release builds only log.
let l = device.limits();
let needs = [
// `adv3d`'s post kernels bind 7 (its rasteriser's fragment stage 6).
("max_storage_buffers_per_shader_stage", l.max_storage_buffers_per_shader_stage, 7),
("max_storage_textures_per_shader_stage", l.max_storage_textures_per_shader_stage, 1),
("max_bind_groups", l.max_bind_groups, 2),
("max_compute_workgroup_size_x", l.max_compute_workgroup_size_x, 64),
("max_compute_workgroup_size_y", l.max_compute_workgroup_size_y, 16),
("max_compute_invocations_per_workgroup", l.max_compute_invocations_per_workgroup, 256),
];
if let Some((limit, have, need)) = needs.iter().find(|(_, have, need)| have < need) {
return Err(format!("{} ({:?}): device limit {limit} is {have}, needs {need}", info.name, info.backend));
}
for f in [FORMAT, wgpu::TextureFormat::Rgba8Unorm] {
if !adapter.get_texture_format_features(f).allowed_usages.contains(wgpu::TextureUsages::STORAGE_BINDING) {
return Err(format!("{} ({:?}): {f:?} storage textures unsupported", info.name, info.backend));
}
}
let readbacks = crate::readback::Readbacks::new()?;
let name = format!("{} ({:?})", info.name, info.backend);
let adv3d_unsupported = crate::adv3d::raster_unsupported(adapter);
if let Some(why) = &adv3d_unsupported {
log::info!("gpu {name}: Advanced 3D renders on the CPU ({why})");
}
let src = [
include_str!("shaders/common.wgsl"),
include_str!("shaders/kernels.wgsl"),
@ -330,52 +388,61 @@ impl GpuContext {
include_str!("shaders/sky.wgsl"),
]
.concat();
let module =
device.create_shader_module(wgpu::ShaderModuleDescriptor { label: Some("effectcraft kernels"), source: wgpu::ShaderSource::Wgsl(src.into()) });
let bgl = device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
label: Some("effectcraft kernels"),
entries: &[
wgpu::BindGroupLayoutEntry {
binding: 0,
visibility: wgpu::ShaderStages::COMPUTE,
ty: wgpu::BindingType::Buffer { ty: wgpu::BufferBindingType::Uniform, has_dynamic_offset: false, min_binding_size: None },
count: None,
},
tex_entry(1),
tex_entry(2),
storage_tex_entry(3, FORMAT),
wgpu::BindGroupLayoutEntry {
binding: 4,
visibility: wgpu::ShaderStages::COMPUTE,
ty: wgpu::BindingType::Buffer {
ty: wgpu::BufferBindingType::Storage { read_only: true },
has_dynamic_offset: false,
min_binding_size: None,
let module = init_resource(&device, "kernels shader module", || {
device.create_shader_module(wgpu::ShaderModuleDescriptor { label: Some("effectcraft kernels"), source: wgpu::ShaderSource::Wgsl(src.into()) })
})?;
let bgl = init_resource(&device, "kernels bind-group layout", || {
device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
label: Some("effectcraft kernels"),
entries: &[
wgpu::BindGroupLayoutEntry {
binding: 0,
visibility: wgpu::ShaderStages::COMPUTE,
ty: wgpu::BindingType::Buffer { ty: wgpu::BufferBindingType::Uniform, has_dynamic_offset: false, min_binding_size: None },
count: None,
},
count: None,
},
],
});
let layout = device.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor {
label: Some("effectcraft kernels"),
bind_group_layouts: &[Some(&bgl)],
immediate_size: 0,
});
tex_entry(1),
tex_entry(2),
storage_tex_entry(3, FORMAT),
wgpu::BindGroupLayoutEntry {
binding: 4,
visibility: wgpu::ShaderStages::COMPUTE,
ty: wgpu::BindingType::Buffer {
ty: wgpu::BufferBindingType::Storage { read_only: true },
has_dynamic_offset: false,
min_binding_size: None,
},
count: None,
},
],
})
})?;
let layout = init_resource(&device, "kernels pipeline layout", || {
device.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor {
label: Some("effectcraft kernels"),
bind_group_layouts: &[Some(&bgl)],
immediate_size: 0,
})
})?;
let storage = |binding: u32| wgpu::BindGroupLayoutEntry {
binding,
visibility: wgpu::ShaderStages::COMPUTE,
ty: wgpu::BindingType::Buffer { ty: wgpu::BufferBindingType::Storage { read_only: true }, has_dynamic_offset: false, min_binding_size: None },
count: None,
};
let bgl_ext = device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
label: Some("effectcraft kernels (ext)"),
entries: &[storage(0), storage(1), storage(2), storage(3)],
});
let layout_ext = device.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor {
label: Some("effectcraft kernels (ext)"),
bind_group_layouts: &[Some(&bgl), Some(&bgl_ext)],
immediate_size: 0,
});
let bgl_ext = init_resource(&device, "extended kernels bind-group layout", || {
device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
label: Some("effectcraft kernels (ext)"),
entries: &[storage(0), storage(1), storage(2), storage(3)],
})
})?;
let layout_ext = init_resource(&device, "extended kernels pipeline layout", || {
device.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor {
label: Some("effectcraft kernels (ext)"),
bind_group_layouts: &[Some(&bgl), Some(&bgl_ext)],
immediate_size: 0,
})
})?;
let pipelines = ENTRIES
.iter()
.chain(crate::fx_color::KERNELS)
@ -402,50 +469,64 @@ impl GpuContext {
.map(|e| (e, &layout))
.chain(EXT_ENTRIES.iter().map(|e| (e, &layout_ext)))
.map(|(e, layout)| {
let p = device.create_compute_pipeline(&wgpu::ComputePipelineDescriptor {
label: Some(e),
layout: Some(layout),
module: &module,
entry_point: Some(e),
compilation_options: Default::default(),
cache: None,
});
(*e, p)
let p = init_resource(&device, e, || {
device.create_compute_pipeline(&wgpu::ComputePipelineDescriptor {
label: Some(e),
layout: Some(layout),
module: &module,
entry_point: Some(e),
compilation_options: Default::default(),
cache: None,
})
})?;
Ok((*e, p))
})
.collect();
let dmodule = device.create_shader_module(wgpu::ShaderModuleDescriptor {
label: Some("effectcraft display"),
source: wgpu::ShaderSource::Wgsl(include_str!("shaders/display.wgsl").into()),
});
let display_bgl = device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
label: Some("effectcraft display"),
entries: &[tex_entry(1), storage_tex_entry(3, wgpu::TextureFormat::Rgba8Unorm)],
});
let dlayout = device.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor {
label: Some("effectcraft display"),
bind_group_layouts: &[Some(&display_bgl)],
immediate_size: 0,
});
let display = device.create_compute_pipeline(&wgpu::ComputePipelineDescriptor {
label: Some("display"),
layout: Some(&dlayout),
module: &dmodule,
entry_point: Some("display"),
compilation_options: Default::default(),
cache: None,
});
let dummy = device.create_texture(&wgpu::TextureDescriptor {
label: Some("dummy"),
size: wgpu::Extent3d { width: 1, height: 1, depth_or_array_layers: 1 },
mip_level_count: 1,
sample_count: 1,
dimension: wgpu::TextureDimension::D2,
format: FORMAT,
usage: wgpu::TextureUsages::TEXTURE_BINDING,
view_formats: &[],
});
let dummy_buf =
device.create_buffer_init(&wgpu::util::BufferInitDescriptor { label: Some("dummy"), contents: &[0u8; 16], usage: wgpu::BufferUsages::STORAGE });
.collect::<Result<HashMap<_, _>, String>>()?;
let dmodule = init_resource(&device, "display shader module", || {
device.create_shader_module(wgpu::ShaderModuleDescriptor {
label: Some("effectcraft display"),
source: wgpu::ShaderSource::Wgsl(include_str!("shaders/display.wgsl").into()),
})
})?;
let display_bgl = init_resource(&device, "display bind-group layout", || {
device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
label: Some("effectcraft display"),
entries: &[tex_entry(1), storage_tex_entry(3, wgpu::TextureFormat::Rgba8Unorm)],
})
})?;
let dlayout = init_resource(&device, "display pipeline layout", || {
device.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor {
label: Some("effectcraft display"),
bind_group_layouts: &[Some(&display_bgl)],
immediate_size: 0,
})
})?;
let display = init_resource(&device, "display compute pipeline", || {
device.create_compute_pipeline(&wgpu::ComputePipelineDescriptor {
label: Some("display"),
layout: Some(&dlayout),
module: &dmodule,
entry_point: Some("display"),
compilation_options: Default::default(),
cache: None,
})
})?;
let dummy = init_resource(&device, "dummy texture", || {
device.create_texture(&wgpu::TextureDescriptor {
label: Some("dummy"),
size: wgpu::Extent3d { width: 1, height: 1, depth_or_array_layers: 1 },
mip_level_count: 1,
sample_count: 1,
dimension: wgpu::TextureDimension::D2,
format: FORMAT,
usage: wgpu::TextureUsages::TEXTURE_BINDING,
view_formats: &[],
})
})?;
let dummy_buf = init_resource(&device, "dummy storage buffer", || {
device.create_buffer_init(&wgpu::util::BufferInitDescriptor { label: Some("dummy"), contents: &[0u8; 16], usage: wgpu::BufferUsages::STORAGE })
})?;
let dummy_tex = init_resource(&device, "dummy texture view", || dummy.create_view(&Default::default()))?;
let max_dim = device.limits().max_texture_dimension_2d;
Ok(GpuContext {
device,
@ -457,7 +538,7 @@ impl GpuContext {
pipelines,
display_bgl,
display,
dummy_tex: dummy.create_view(&Default::default()),
dummy_tex,
dummy,
dummy_buf,
uploads: Mutex::new(Uploads::default()),
@ -466,14 +547,116 @@ impl GpuContext {
transfers: Default::default(),
zeros: Default::default(),
adv3d: std::sync::OnceLock::new(),
adv3d_raster: adv3d_unsupported.is_none(),
particles: Default::default(),
particle_states: Default::default(),
deferred: None,
readbacks,
})
}
/// Whether this context has observed a readback timeout or device failure.
/// A successful check is not a guarantee against faults after it returns.
/// Shared-device hosts must forward their device failures through the notifier.
pub fn check_health(&self) -> Result<(), String> {
self.readbacks.check()
}
/// A notification hook for the owner of a shared device's error/loss handlers.
/// Invoke it with diagnostic context when the host observes a device failure.
/// It retires this context and settles pending readbacks without replacing any
/// host handler or keeping this context/device alive. It is harmless after drop.
///
/// Shared-device construction does not automatically subscribe to device faults:
/// without host forwarding, readbacks rely on poll errors and their deadlines.
pub fn device_failure_notifier(&self) -> impl Fn(String) + Send + Sync + 'static {
crate::readback::failure_notifier(&self.readbacks)
}
/// All device polling routes through this failure bridge.
pub(crate) fn poll_readbacks(&self, kind: wgpu::PollType) -> Result<wgpu::PollStatus, String> {
self.readbacks.check()?;
self.device.poll(kind).map_err(|e| {
let error = format!("GPU readback poll: {e}");
self.readbacks.retire(error.clone());
log::error!("{error}");
error
})
}
/// Start a bounded buffer mapping. Retired contexts reject new requests immediately.
pub(crate) fn map_readback(
&self,
buffer: &wgpu::Buffer,
row: usize,
stride: usize,
height: usize,
done: impl FnOnce(Result<Vec<u8>, String>) + wgpu::WasmNotSend + 'static,
) {
let ticket = match self.readbacks.start(done) {
Ok(t) => t,
Err(e) => {
log::error!("gpu: {e}");
return;
}
};
if !ticket.active() {
return;
}
let b = buffer.clone();
buffer.map_async(wgpu::MapMode::Read, .., move |r| {
if !ticket.active() {
b.unmap();
return;
}
let result = r.map_err(|e| format!("GPU buffer mapping: {e}")).and_then(|_| {
let view = b.get_mapped_range(..).map_err(|e| format!("GPU mapped range: {e}"))?;
crate::readback::packed(&view, row, stride, height)
});
b.unmap();
ticket.complete(result);
});
}
/// A bounded native readback: polling and receiving share one deadline.
#[cfg(not(target_arch = "wasm32"))]
pub(crate) fn read_buffer(&self, buffer: &wgpu::Buffer, row: usize, stride: usize, height: usize) -> Result<Vec<u8>, String> {
let deadline = std::time::Instant::now() + crate::readback::TIMEOUT;
let (tx, rx) = std::sync::mpsc::channel();
self.map_readback(buffer, row, stride, height, move |r| {
let _ = tx.send(r);
});
loop {
self.readbacks.check()?;
let remaining = deadline.saturating_duration_since(std::time::Instant::now());
if remaining.is_zero() {
let error = "GPU readback receive timed out; context retired".to_string();
self.readbacks.retire(error.clone());
return Err(error);
}
// Block until the GPU is done (bounded by the deadline: a timeout retires the
// context) rather than polling and sleeping, which adds latency to every readback.
self.poll_readbacks(wgpu::PollType::Wait { submission_index: None, timeout: Some(remaining) })?;
// The callback may be running on another thread that polled the same device.
match rx.recv_timeout(remaining.min(std::time::Duration::from_millis(1))) {
Ok(result) => return result,
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {}
Err(e) => return Err(format!("GPU readback completion channel: {e}")),
}
}
}
#[cfg(target_arch = "wasm32")]
pub(crate) fn read_buffer(&self, _buffer: &wgpu::Buffer, _row: usize, _stride: usize, _height: usize) -> Result<Vec<u8>, String> {
Err("synchronous GPU readback is unavailable in the browser".into())
}
/// Switch to deferred readbacks (see [`crate::deferred`]): readbacks are never waited for.
pub fn set_deferred(&mut self, on: bool) {
self.readbacks.cancel_pending("GPU readback mode changed".into());
if let Some(d) = self.deferred.take() {
d.cancel();
}
self.deferred = on.then(Default::default);
}
@ -486,6 +669,12 @@ impl GpuContext {
pub async fn request() -> Result<GpuContext, String> {
#[allow(unused_mut)]
let mut desc = wgpu::InstanceDescriptor::new_without_display_handle();
#[cfg(not(target_arch = "wasm32"))]
{
// Match eframe's backend override so CLI and headless acceptance tests
// exercise the backend requested for the native viewer.
desc.backends = wgpu::Backends::from_env().unwrap_or(desc.backends);
}
#[cfg(target_arch = "wasm32")]
{
desc.backends = wgpu::Backends::BROWSER_WEBGPU;
@ -505,17 +694,10 @@ impl GpuContext {
.request_device(&wgpu::DeviceDescriptor { label: Some("effectcraft gpu"), required_limits, ..Default::default() })
.await
.map_err(|e| format!("no device: {e}"))?;
device.on_uncaptured_error(Arc::new(|e| {
// Under test a validation error (a WGSL typo invalidates every kernel) fails loudly.
#[cfg(test)]
#[allow(clippy::panic)] // test builds only
{
panic!("wgpu: {e}")
}
#[cfg(not(test))]
log::error!("wgpu: {e}")
}));
GpuContext::new(&adapter, device, queue)
let context = GpuContext::new(&adapter, device, queue)?;
// This path owns the device; install handlers only after construction succeeds.
crate::readback::install(&context.device, &context.readbacks);
Ok(context)
}
/// A device of its own on the best adapter (CLI, tests, benchmarks). `None` when no
@ -528,13 +710,13 @@ impl GpuContext {
/// Readbacks (GPU → CPU) are possible: waited for natively, or deferred (a browser
/// worker, [`crate::deferred`]). Impossible on the browser's main thread.
pub fn can_readback(&self) -> bool {
self.can_wait() || self.deferred.is_some()
self.readbacks.check().is_ok() && (self.can_wait() || self.deferred.is_some())
}
/// Readbacks can be waited for (natively, unless deferred). Steps that read back without a
/// key ([`crate::deferred`]) need this.
pub fn can_wait(&self) -> bool {
cfg!(not(target_arch = "wasm32")) && self.deferred.is_none()
self.readbacks.check().is_ok() && cfg!(not(target_arch = "wasm32")) && self.deferred.is_none()
}
pub(crate) fn fits(&self, w: u32, h: u32) -> bool {
@ -836,7 +1018,13 @@ impl<'g> Enc<'g> {
/// Read an image back to the CPU (submits and waits). `None` where readback is impossible.
pub fn download(&mut self, img: &GpuImage) -> Option<Image> {
let mut out = Image::new(img.width, img.height);
let layout = crate::readback::Layout::new(img.width, img.height, 16, self.g.device.limits().max_buffer_size)
.map_err(|e| log::error!("gpu image readback: {e}"))
.ok()?;
let mut data = Vec::new();
data.try_reserve_exact(layout.len / 16).map_err(|e| log::error!("gpu image allocation: {e}")).ok()?;
data.resize(layout.len / 16, [0.0; 4]);
let mut out = Image { width: img.width, height: img.height, data };
let dst: &mut [u8] = bytemuck::cast_slice_mut(&mut out.data);
self.read_texture_into(&img.texture, img.width, img.height, 16, dst)?;
Some(out)
@ -844,7 +1032,11 @@ impl<'g> Enc<'g> {
/// Tightly packed texel bytes of a texture (`bpp` bytes per pixel).
pub fn read_texture(&mut self, texture: &wgpu::Texture, w: u32, h: u32, bpp: u32) -> Option<Vec<u8>> {
let mut out = vec![0u8; (w * bpp) as usize * h as usize];
let layout =
crate::readback::Layout::new(w, h, bpp, self.g.device.limits().max_buffer_size).map_err(|e| log::error!("gpu texture readback: {e}")).ok()?;
let mut out = Vec::new();
out.try_reserve_exact(layout.len).map_err(|e| log::error!("gpu readback allocation: {e}")).ok()?;
out.resize(layout.len, 0);
self.read_texture_into(texture, w, h, bpp, &mut out)?;
Some(out)
}
@ -855,9 +1047,15 @@ impl<'g> Enc<'g> {
return None;
}
self.g.transfers.count(false, out.len());
let row = (w * bpp) as usize;
let padded = (w * bpp).div_ceil(wgpu::COPY_BYTES_PER_ROW_ALIGNMENT) * wgpu::COPY_BYTES_PER_ROW_ALIGNMENT;
let size = padded as u64 * h as u64;
let layout =
crate::readback::Layout::new(w, h, bpp, self.g.device.limits().max_buffer_size).map_err(|e| log::error!("gpu texture readback: {e}")).ok()?;
if out.len() != layout.len {
log::error!("gpu readback destination size mismatch");
return None;
}
let row = layout.row as usize;
let padded = layout.stride;
let size = layout.size;
let buf = self.g.staging.lock().ok().and_then(|mut s| s.get_mut(&size).and_then(Vec::pop)).unwrap_or_else(|| {
self.g.device.create_buffer(&wgpu::BufferDescriptor {
label: Some("readback"),
@ -872,27 +1070,12 @@ impl<'g> Enc<'g> {
wgpu::Extent3d { width: w, height: h, depth_or_array_layers: 1 },
);
self.submit();
let (tx, rx) = std::sync::mpsc::channel();
buf.map_async(wgpu::MapMode::Read, .., move |r| {
let _ = tx.send(r);
});
if let Err(e) = self.g.device.poll(wgpu::PollType::wait_indefinitely()) {
log::error!("gpu: poll: {e}");
let bytes = self.g.read_buffer(&buf, row, padded as usize, h as usize).map_err(|e| log::error!("gpu texture readback: {e}")).ok()?;
if bytes.len() != out.len() {
log::error!("gpu texture readback size mismatch");
return None;
}
rx.recv().ok()?.ok()?;
{
let view = buf.get_mapped_range(..).ok()?;
if row == padded as usize {
out.copy_from_slice(&view[..out.len()]);
} else {
for (y, dst) in out.chunks_exact_mut(row).enumerate() {
let s = y * padded as usize;
dst.copy_from_slice(&view[s..s + row]);
}
}
}
buf.unmap();
out.copy_from_slice(&bytes);
// Keep a few staging buffers per size (frames read back at the same size every time).
if let Ok(mut s) = self.g.staging.lock() {
let v = s.entry(size).or_default();
@ -907,11 +1090,24 @@ impl<'g> Enc<'g> {
/// gets the bytes once the copy is mapped (from the browser's event loop on the web; from
/// a later device poll natively).
pub fn read_texture_async(&mut self, texture: &wgpu::Texture, w: u32, h: u32, bpp: u32, done: impl FnOnce(Option<Vec<u8>>) + wgpu::WasmNotSend + 'static) {
let row = w * bpp;
let padded = row.div_ceil(wgpu::COPY_BYTES_PER_ROW_ALIGNMENT) * wgpu::COPY_BYTES_PER_ROW_ALIGNMENT;
if let Err(e) = self.g.readbacks.check() {
log::error!("gpu readback: {e}");
crate::readback::reject(done, e);
return;
}
let layout = match crate::readback::Layout::new(w, h, bpp, self.g.device.limits().max_buffer_size) {
Ok(layout) => layout,
Err(e) => {
log::error!("gpu texture readback: {e}");
crate::readback::reject(done, e);
return;
}
};
let row = layout.row;
let padded = layout.stride;
let buf = self.g.device.create_buffer(&wgpu::BufferDescriptor {
label: Some("readback-async"),
size: padded as u64 * h as u64,
size: layout.size,
usage: wgpu::BufferUsages::MAP_READ | wgpu::BufferUsages::COPY_DST,
mapped_at_creation: false,
});
@ -921,21 +1117,11 @@ impl<'g> Enc<'g> {
wgpu::Extent3d { width: w, height: h, depth_or_array_layers: 1 },
);
self.submit();
let b = buf.clone();
buf.map_async(wgpu::MapMode::Read, .., move |r| {
let out = r.ok().and_then(|_| {
let view = b.get_mapped_range(..).ok()?;
let mut out = Vec::with_capacity((row * h) as usize);
for y in 0..h as usize {
out.extend_from_slice(&view[y * padded as usize..y * padded as usize + row as usize]);
}
Some(out)
});
b.unmap();
done(out);
self.g.map_readback(&buf, row as usize, padded as usize, h as usize, move |r| {
done(r.map_err(|e| log::error!("gpu texture readback: {e}")).ok());
});
#[cfg(not(target_arch = "wasm32"))]
let _ = self.g.device.poll(wgpu::PollType::Poll);
let _ = self.g.poll_readbacks(wgpu::PollType::Poll);
}
/// A deferred readback of `img` (RGBA f32) under `key` (see [`crate::deferred`]): the bytes

View File

@ -14,7 +14,7 @@
//! of its key, and a chain whose kernels read other data through the effect host (other layers,
//! other times) is not started when that data missed while it ran (the miss counter moved).
use std::collections::{HashMap, HashSet};
use std::collections::HashMap;
use std::future::Future;
use std::hash::{Hash, Hasher};
use std::pin::Pin;
@ -48,11 +48,9 @@ pub(crate) enum Lookup {
#[derive(Default)]
struct State {
/// Frame generation: results of an older frame's readbacks are dropped.
generation: u64,
ready: HashMap<u128, Option<Readback>>,
waiting: HashSet<u128>,
in_flight: usize,
/// Each request owns a unique token, even when a key is reused after cancellation.
waiting: HashMap<u128, Arc<()>>,
/// Misses so far in this frame (a counter, so a chain sees misses during its own work).
misses: u64,
wakers: Vec<Waker>,
@ -71,11 +69,15 @@ pub struct Deferred {
impl Deferred {
pub fn frame_begin(&self) {
if let Ok(mut s) = self.st.lock() {
s.generation += 1;
let wakers = {
let mut s = self.st.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
s.ready.clear();
s.waiting.clear();
s.misses = 0;
std::mem::take(&mut s.wakers)
};
for w in wakers {
w.wake();
}
self.gate.store(false, Ordering::Relaxed);
}
@ -111,32 +113,25 @@ impl Deferred {
if let Some(r) = s.ready.get(&key) {
return Lookup::Ready(r.clone());
}
if s.waiting.contains(&key) { Lookup::InFlight } else { Lookup::Absent }
if s.waiting.contains_key(&key) { Lookup::InFlight } else { Lookup::Absent }
}
/// A readback for `key` is being started; returns the callback that stores its bytes.
pub(crate) fn start(self: &Arc<Self>, key: u128, width: u32, height: u32, offset: [f64; 2], scale: f64) -> impl FnOnce(Option<Vec<u8>>) + 'static {
let generation = match self.st.lock() {
Ok(mut s) => {
s.waiting.insert(key);
s.in_flight += 1;
s.generation
}
Err(_) => 0,
};
let token = Arc::new(());
self.st.lock().unwrap_or_else(std::sync::PoisonError::into_inner).waiting.insert(key, token.clone());
self.readbacks.fetch_add(1, Ordering::Relaxed);
let me = self.clone();
move |bytes: Option<Vec<u8>>| {
let wakers = match me.st.lock() {
Ok(mut s) => {
s.in_flight = s.in_flight.saturating_sub(1);
if s.generation == generation {
s.waiting.remove(&key);
s.ready.insert(key, bytes.map(|bytes| Readback { bytes, width, height, offset, scale }));
}
if s.in_flight == 0 { std::mem::take(&mut s.wakers) } else { vec![] }
let wakers = {
let mut s = me.st.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
// Reject stale completion before touching either results or accounting.
if !s.waiting.get(&key).is_some_and(|current| Arc::ptr_eq(current, &token)) {
return;
}
Err(_) => vec![],
s.waiting.remove(&key);
s.ready.insert(key, bytes.map(|bytes| Readback { bytes, width, height, offset, scale }));
if s.waiting.is_empty() { std::mem::take(&mut s.wakers) } else { vec![] }
};
for w in wakers {
w.wake();
@ -144,9 +139,24 @@ impl Deferred {
}
}
/// Cancel pending requests without exposing placeholder pixels as successful results.
/// Existing waiters wake; late callbacks cannot change a newer request's accounting.
pub(crate) fn cancel(&self) {
let wakers = {
let mut s = self.st.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
for (key, _) in std::mem::take(&mut s.waiting) {
s.ready.insert(key, None);
}
std::mem::take(&mut s.wakers)
};
for w in wakers {
w.wake();
}
}
/// Readbacks still in flight.
pub fn in_flight(&self) -> usize {
self.st.lock().map(|s| s.in_flight).unwrap_or(0)
self.st.lock().unwrap_or_else(std::sync::PoisonError::into_inner).waiting.len()
}
/// Resolves once no readback is in flight (on the web the browser's event loop delivers
@ -162,11 +172,13 @@ pub struct Settled(Arc<Deferred>);
impl Future for Settled {
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {
let Ok(mut s) = self.0.st.lock() else { return Poll::Ready(()) };
if s.in_flight == 0 {
let mut s = self.0.st.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
if s.waiting.is_empty() {
return Poll::Ready(());
}
s.wakers.push(cx.waker().clone());
if !s.wakers.iter().any(|w| w.will_wake(cx.waker())) {
s.wakers.push(cx.waker().clone());
}
Poll::Pending
}
}
@ -249,6 +261,56 @@ pub(crate) fn chain_key(chain: &[FxStep], buf: &Buf, levels: Option<f32>) -> u12
mod tests {
use super::*;
struct WakeCount(std::sync::atomic::AtomicUsize);
impl std::task::Wake for WakeCount {
fn wake(self: Arc<Self>) {
self.0.fetch_add(1, Ordering::SeqCst);
}
}
#[test]
fn cancellation_wakes_waiters_and_stale_same_key_does_not_drain_new_work() {
let d = Arc::new(Deferred::default());
let old = d.start(7, 1, 1, [0.0; 2], 1.0);
let wake = Arc::new(WakeCount(std::sync::atomic::AtomicUsize::new(0)));
let waker = Waker::from(wake.clone());
let mut settled = std::pin::pin!(d.settled());
for _ in 0..4 {
assert!(settled.as_mut().poll(&mut Context::from_waker(&waker)).is_pending());
}
d.cancel();
assert_eq!(wake.0.load(Ordering::SeqCst), 1);
assert_eq!(d.in_flight(), 0);
assert!(settled.as_mut().poll(&mut Context::from_waker(&waker)).is_ready());
assert!(matches!(d.lookup(7), Lookup::Ready(None)));
d.frame_begin();
let new = d.start(7, 1, 1, [0.0; 2], 1.0);
old(Some(vec![99; 4]));
assert_eq!(d.in_flight(), 1);
assert!(matches!(d.lookup(7), Lookup::InFlight));
new(None);
assert_eq!(d.in_flight(), 0);
assert!(matches!(d.lookup(7), Lookup::Ready(None)));
}
#[test]
fn device_failure_without_map_callback_drains_and_wakes_deferred() {
let d = Arc::new(Deferred::default());
let registry = crate::readback::Readbacks::new().unwrap();
let done = d.start(11, 1, 1, [0.0; 2], 1.0);
let ticket = registry.start(move |r| done(r.ok())).unwrap();
let wake = Arc::new(WakeCount(std::sync::atomic::AtomicUsize::new(0)));
let waker = Waker::from(wake.clone());
let mut settled = std::pin::pin!(d.settled());
assert!(settled.as_mut().poll(&mut Context::from_waker(&waker)).is_pending());
registry.retire("synthetic device loss".into());
assert_eq!(wake.0.load(Ordering::SeqCst), 1);
assert!(settled.as_mut().poll(&mut Context::from_waker(&waker)).is_ready());
assert_eq!(d.in_flight(), 0);
ticket.complete(Ok(vec![99; 4]));
assert!(matches!(d.lookup(11), Lookup::Ready(None)));
}
#[test]
fn keys_are_stable_and_distinct() {
let mut a = Key::new(1);

View File

@ -81,6 +81,7 @@ mod fx_vr;
mod fx_warp;
mod ops;
mod particles;
mod readback;
mod walk;
use std::sync::Arc;
@ -157,7 +158,7 @@ impl Gpu {
/// Deliver finished readbacks without blocking (native; the browser delivers them itself).
pub fn poll(&self) {
#[cfg(not(target_arch = "wasm32"))]
let _ = self.ctx.device.poll(wgpu::PollType::Poll);
let _ = self.ctx.poll_readbacks(wgpu::PollType::Poll);
}
/// `{passes, readbacks, inFlight}` of deferred rendering (diagnostics).
@ -177,10 +178,14 @@ impl Gpu {
/// Render a top-level comp frame and leave it on the GPU as a display texture (no
/// readback unless a CPU fallback step needs one). `None` = render on the CPU.
pub fn render_display(&self, r: &Renderer, comp: ItemId, t: Tick) -> Option<DisplayFrame> {
self.ctx.check_health().map_err(|e| log::error!("gpu display: {e}")).ok()?;
let mut e = Enc::new(&self.ctx);
let img = walk::render(&mut e, r, comp, t)?;
let texture = e.display(&img);
e.submit();
// Reject failures observed during rendering/submission before publishing the frame.
// A later device fault still requires the presentation host's lifecycle handling.
self.ctx.check_health().map_err(|e| log::error!("gpu display: {e}")).ok()?;
Some(DisplayFrame { texture, width: img.width, height: img.height })
}
@ -192,11 +197,17 @@ impl Gpu {
/// [`Gpu::read_display`] without blocking (the browser's main thread): `done` gets the bytes
/// once the GPU has them.
pub fn read_display_async(&self, f: &DisplayFrame, done: impl FnOnce(Option<Vec<u8>>) + wgpu::WasmNotSend + 'static) {
if let Err(e) = self.ctx.check_health() {
log::error!("gpu display readback: {e}");
crate::readback::reject(done, e);
return;
}
Enc::new(&self.ctx).read_texture_async(&f.texture, f.width, f.height, 4, done);
}
/// Render a frame on the GPU and read it back (`None` = not handled).
pub fn render(&self, r: &Renderer, comp: ItemId, t: Tick) -> Option<Image> {
self.ctx.check_health().ok()?;
if !self.ctx.can_readback() {
return None;
}
@ -244,9 +255,34 @@ impl Gpu {
}
}
/// Wait for all submitted GPU work (benchmarks).
/// Run work in a native thread-local out-of-memory scope. A failed result is discarded
/// and cached allocations are released; callers can retry with a fresh CPU renderer.
/// Browser callers use the host/owned-device failure bridge and bounded readbacks instead
/// of blocking their event loop on an error-scope future.
pub fn within_memory<T>(&self, f: impl FnOnce() -> T) -> Result<T, String> {
self.ctx.check_health()?;
#[cfg(not(target_arch = "wasm32"))]
{
let scope = self.ctx.device.push_error_scope(wgpu::ErrorFilter::OutOfMemory);
let out = f();
match pollster::block_on(scope.pop()) {
Some(e) => {
self.ctx.clear_uploads();
Err(e.to_string())
}
None => self.ctx.check_health().map(|()| out),
}
}
#[cfg(target_arch = "wasm32")]
{
let out = f();
self.ctx.check_health().map(|()| out)
}
}
/// Wait for submitted GPU work with the readback deadline (benchmarks).
pub fn wait(&self) {
let _ = self.ctx.device.poll(wgpu::PollType::wait_indefinitely());
let _ = self.ctx.poll_readbacks(wgpu::PollType::Wait { submission_index: None, timeout: Some(crate::readback::TIMEOUT) });
}
}
@ -260,22 +296,26 @@ impl Accelerator for Gpu {
}
fn supports_effect(&self, id: &str) -> bool {
self.ctx.can_readback() && effects::supports(id)
self.ctx.check_health().is_ok() && self.ctx.can_readback() && effects::supports(id)
}
fn effects(&self, chain: &[FxStep], buf: &Buf, levels: Option<f32>) -> Option<Buf> {
self.ctx.check_health().ok()?;
effects::run_chain(&mut Enc::new(&self.ctx), chain, buf, levels)
}
fn raster_3d(&self, scene: &effectcraft_render::three_d::adv::Scene) -> Option<effectcraft_render::three_d::adv::Target> {
self.ctx.check_health().ok()?;
adv3d::render(&self.ctx, scene)
}
fn render_3d(&self, run: &effectcraft_render::three_d::adv::Prepared) -> Option<effectcraft_render::three_d::adv::Rendered> {
self.ctx.check_health().ok()?;
adv3d::render_prepared(&self.ctx, run)
}
fn particles(&self) -> Option<&dyn effectcraft_effects::psim::ParticleSim> {
self.ctx.check_health().ok()?;
self.ctx.can_readback().then_some(self as &dyn effectcraft_effects::psim::ParticleSim)
}
@ -308,7 +348,7 @@ impl Accelerator for Gpu {
// Natively the device delivers the readbacks when polled: wait for them here.
#[cfg(not(target_arch = "wasm32"))]
if d.in_flight() > 0 {
let _ = self.ctx.device.poll(wgpu::PollType::wait_indefinitely());
let _ = self.ctx.poll_readbacks(wgpu::PollType::Wait { submission_index: None, timeout: Some(crate::readback::TIMEOUT) });
}
Some(Box::pin(async move { d.settled().await }))
}
@ -316,6 +356,7 @@ impl Accelerator for Gpu {
impl effectcraft_effects::psim::ParticleSim for Gpu {
fn simulate(&self, req: &effectcraft_effects::psim::SimRequest) -> Option<Vec<effectcraft_effects::psim::SimParticle>> {
self.ctx.check_health().ok()?;
particles::simulate(&self.ctx, req)
}
}
@ -372,5 +413,7 @@ mod tests_fx_transition;
mod tests_fx_vr;
#[cfg(test)]
mod tests_fx_warp;
#[cfg(all(test, not(target_arch = "wasm32")))]
mod tests_init;
#[cfg(test)]
mod tests_particles;

View File

@ -97,7 +97,7 @@ pub fn warp_q(
p.f[0] = r[0];
p.f[1] = r[1];
p.f[2] = r[2];
p.f[3] = [opacity, accumulate.unwrap_or(1.0), levels.unwrap_or(0.0), 0.0];
p.f[3] = [opacity, accumulate.unwrap_or(1.0), levels.unwrap_or(0.0), levels.map_or(0.0, |v| 1.0 / v)];
p.f[4] = [x0 as f32, y0 as f32, x1 as f32, y1 as f32];
let mut out = e.scratch(dst.width, dst.height);
e.pixels("warp_blend", &p, &img, Some(dst), &out, None);
@ -110,7 +110,7 @@ pub fn warp_q(
pub fn blend_full(e: &mut Enc, dst: &GpuImage, src: &GpuImage, mode: BlendMode, opacity: f32, seed: u32, levels: Option<f32>) -> GpuImage {
let mut p = Params::default();
p.u[0] = [mode_id(mode), 0, seed, 0];
p.f[0] = [opacity, levels.unwrap_or(0.0), 0.0, 0.0];
p.f[0] = [opacity, levels.unwrap_or(0.0), levels.map_or(0.0, |v| 1.0 / v), 0.0];
let mut out = e.scratch(dst.width, dst.height);
e.pixels("blend_full", &p, src, Some(dst), &out, None);
out.levels = levels;
@ -174,6 +174,7 @@ pub fn quantize(e: &mut Enc, img: &GpuImage, levels: f32) -> GpuImage {
}
let mut p = Params::default();
p.f[0][0] = levels;
p.f[0][1] = 1.0 / levels;
let mut out = e.scratch(img.width, img.height);
e.pixels("quantize", &p, img, None, &out, None);
out.levels = Some(levels);

View File

@ -193,29 +193,18 @@ pub(crate) fn simulate(g: &GpuContext, req: &SimRequest) -> Option<Vec<SimPartic
enc.copy_buffer_to_buffer(&next, 0, &read, 0, size);
g.queue.submit([enc.finish()]);
if let (Some(d), Some(k)) = (&deferred, key) {
let row = usize::try_from(size).ok()?;
let done = d.start(k, n, 1, [0.0; 2], 1.0);
let b = read.clone();
read.map_async(wgpu::MapMode::Read, .., move |r| {
let out = r.ok().and_then(|_| b.get_mapped_range(..).ok().map(|v| v.to_vec()));
b.unmap();
done(out);
});
g.map_readback(&read, row, row, 1, move |r| done(r.map_err(|e| log::error!("gpu particle readback: {e}")).ok()));
#[cfg(not(target_arch = "wasm32"))]
let _ = g.device.poll(wgpu::PollType::Poll);
let _ = g.poll_readbacks(wgpu::PollType::Poll);
keep_checkpoint(g, req, n, next);
d.miss();
return Some(vec![]);
}
let (tx, rx) = std::sync::mpsc::channel();
read.map_async(wgpu::MapMode::Read, .., move |r| {
let _ = tx.send(r);
});
g.device.poll(wgpu::PollType::wait_indefinitely()).ok()?;
rx.recv().ok()?.ok()?;
let view = read.get_mapped_range(..).ok()?;
let parts = parse(&view);
drop(view);
read.unmap();
let row = usize::try_from(size).ok()?;
let bytes = g.read_buffer(&read, row, row, 1).map_err(|e| log::error!("gpu particle readback: {e}")).ok()?;
let parts = parse(&bytes);
keep_checkpoint(g, req, n, next);
Some(parts)
}
@ -233,8 +222,10 @@ fn sim_key(req: &SimRequest) -> u128 {
/// Live particles of a state buffer's bytes.
fn parse(bytes: &[u8]) -> Vec<SimParticle> {
let f: Vec<f32> = bytes.chunks_exact(4).map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect();
f.chunks_exact(12)
let f: Vec<f32> = bytes.as_chunks::<4>().0.iter().map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]])).collect();
f.as_chunks::<12>()
.0
.iter()
.enumerate()
.filter(|(_, s)| s[9] != 0.0)
.map(|(id, s)| SimParticle { p: [s[0], s[1], s[2]], v: [s[4], s[5], s[6]], age: s[3], life: s[7], rnd: s[8], id: id as u32 })

529
crates/gpu/src/readback.rs Normal file
View File

@ -0,0 +1,529 @@
//! Bounded, exactly-once readback completion, independent of GPU callbacks.
// Browser callbacks stay on their worker thread. The workspace's wgpu feature still
// requires Send callbacks on non-atomic wasm, and ParticleSim requires Send + Sync.
use std::collections::BTreeMap;
use std::sync::{Arc, Mutex, Weak};
use std::time::Duration;
pub(crate) const TIMEOUT: Duration = Duration::from_secs(10);
pub(crate) type Error = String;
type Done = Box<dyn FnOnce(Result<Vec<u8>, Error>) + Send>;
struct Entry {
deadline: web_time::Instant,
done: Done,
}
#[derive(Default)]
struct State {
next: u64,
pending: BTreeMap<u64, Entry>,
retired: Option<Error>,
#[cfg(target_arch = "wasm32")]
timer_armed: bool,
}
#[derive(Default)]
pub(crate) struct Readbacks(Mutex<State>);
pub(crate) struct Ticket {
owner: Weak<Readbacks>,
id: u64,
}
/// User callbacks run outside locks and are isolated one at a time. On panic=abort
/// (including the usual wasm build), Rust panics cannot be caught: the host must
/// avoid panicking callbacks. This branch does not claim abort-mode recovery.
fn notify(done: Done, result: Result<Vec<u8>, Error>) {
#[cfg(panic = "unwind")]
if std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| done(result))).is_err() {
log::error!("GPU readback callback panicked; remaining callbacks will still settle");
}
#[cfg(not(panic = "unwind"))]
done(result);
}
/// Reject an optional-pixel callback through the same isolation boundary as mapped results.
pub(crate) fn reject(done: impl FnOnce(Option<Vec<u8>>) + wgpu::WasmNotSend + 'static, error: Error) {
notify(Box::new(move |result| done(result.ok())), Err(error));
}
impl Readbacks {
pub(crate) fn new() -> Result<Arc<Self>, Error> {
let me = Arc::new(Self::default());
#[cfg(not(target_arch = "wasm32"))]
{
let weak = Arc::downgrade(&me);
std::thread::Builder::new()
.name("gpu-readback-deadlines".into())
.spawn(move || {
loop {
std::thread::sleep(Duration::from_millis(100));
let Some(me) = weak.upgrade() else { break };
me.expire(web_time::Instant::now());
}
})
.map_err(|e| format!("starting GPU readback deadline worker: {e}"))?;
}
Ok(me)
}
pub(crate) fn check(&self) -> Result<(), Error> {
let s = self.0.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
match &s.retired {
Some(e) => Err(e.clone()),
None => Ok(()),
}
}
pub(crate) fn start(self: &Arc<Self>, done: impl FnOnce(Result<Vec<u8>, Error>) + wgpu::WasmNotSend + 'static) -> Result<Ticket, Error> {
let mut done: Option<Done> = Some(Box::new(done));
let result = {
let mut s = self.0.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
if let Some(e) = &s.retired {
Err(e.clone())
} else if s.pending.len() >= 256 {
Err("too many pending GPU readbacks".into())
} else if let Some(id) = s.next.checked_add(1) {
s.next = id;
if let Some(done) = done.take() {
s.pending.insert(id, Entry { deadline: web_time::Instant::now() + TIMEOUT, done });
}
Ok(Ticket { owner: Arc::downgrade(self), id })
} else {
Err("GPU readback request id exhausted".into())
}
};
if let Err(e) = &result
&& let Some(done) = done
{
notify(done, Err(e.clone()));
}
#[cfg(target_arch = "wasm32")]
if result.is_ok() {
self.arm_timer();
}
result
}
#[cfg(target_arch = "wasm32")]
fn arm_timer(self: &Arc<Self>) {
let delay = {
let mut s = self.0.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
if s.timer_armed || s.retired.is_some() {
return;
}
let Some(deadline) = s.pending.values().map(|e| e.deadline).min() else {
return;
};
s.timer_armed = true;
deadline.saturating_duration_since(web_time::Instant::now())
};
if let Err(e) = schedule(Arc::downgrade(self), delay) {
self.retire(e);
}
}
pub(crate) fn retire(&self, error: Error) {
let pending = {
let mut s = self.0.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
if s.retired.is_some() {
return;
}
s.retired = Some(error.clone());
std::mem::take(&mut s.pending)
};
for (_, entry) in pending {
notify(entry.done, Err(error.clone()));
}
}
pub(crate) fn cancel_pending(&self, error: Error) {
let pending = {
let mut s = self.0.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
std::mem::take(&mut s.pending)
};
for (_, entry) in pending {
notify(entry.done, Err(error.clone()));
}
}
fn expire(&self, now: web_time::Instant) {
let error = "GPU readback timed out; context retired".to_string();
let pending = {
let mut s = self.0.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
// Decide and drain atomically: a cancelled old deadline must not retire
// a newer request inserted between the deadline check and retirement.
if s.retired.is_some() || !s.pending.values().any(|e| now >= e.deadline) {
return;
}
s.retired = Some(error.clone());
std::mem::take(&mut s.pending)
};
for (_, entry) in pending {
notify(entry.done, Err(error.clone()));
}
}
}
impl Drop for Readbacks {
fn drop(&mut self) {
self.retire("GPU readback context dropped".into());
}
}
pub(crate) struct Layout {
pub row: u32,
pub stride: u32,
pub size: u64,
pub len: usize,
}
impl Layout {
pub(crate) fn new(w: u32, h: u32, bpp: u32, limit: u64) -> Result<Self, Error> {
if w == 0 || h == 0 || bpp == 0 {
return Err("empty GPU texture readback".into());
}
let row = w.checked_mul(bpp).ok_or("GPU readback row size overflow")?;
let alignment = wgpu::COPY_BYTES_PER_ROW_ALIGNMENT;
let stride = row.checked_add(alignment - 1).map(|n| n / alignment * alignment).ok_or("GPU readback row alignment overflow")?;
let size = u64::from(stride) * u64::from(h);
// Bound host allocations even when the adapter advertises much larger buffers.
if size > limit.min(1 << 30) {
return Err("GPU readback exceeds buffer budget".into());
}
let len = usize::try_from(u64::from(row) * u64::from(h)).map_err(|e| format!("GPU readback host size: {e}"))?;
Ok(Self { row, stride, size, len })
}
}
impl Ticket {
pub(crate) fn active(&self) -> bool {
self.owner.upgrade().is_some_and(|me| me.0.lock().unwrap_or_else(std::sync::PoisonError::into_inner).pending.contains_key(&self.id))
}
pub(crate) fn complete(self, result: Result<Vec<u8>, Error>) {
let Some(me) = self.owner.upgrade() else { return };
let entry = me.0.lock().unwrap_or_else(std::sync::PoisonError::into_inner).pending.remove(&self.id);
if let Some(entry) = entry {
notify(entry.done, result);
}
}
}
#[cfg(target_arch = "wasm32")]
fn schedule(me: Weak<Readbacks>, delay: Duration) -> Result<(), Error> {
use wasm_bindgen::{JsCast, closure::Closure};
let global = js_sys::global();
let function = js_sys::Reflect::get(&global, &"setTimeout".into())
.map_err(|e| format!("GPU deadline timer lookup: {e:?}"))?
.dyn_into::<js_sys::Function>()
.map_err(|_| "GPU deadline timer unavailable".to_string())?;
let callback = Closure::once(move || {
if let Some(me) = me.upgrade() {
me.0.lock().unwrap_or_else(std::sync::PoisonError::into_inner).timer_armed = false;
me.expire(web_time::Instant::now());
// If the earliest request completed, arm for the next live deadline.
me.arm_timer();
}
});
function
.call2(&global, callback.as_ref(), &wasm_bindgen::JsValue::from_f64(delay.as_secs_f64() * 1000.0 + 1.0))
.map_err(|e| format!("scheduling GPU readback deadline: {e:?}"))?;
// Transfer the wrapper to JS GC rather than leaking a wrapper per readback.
let _ = callback.into_js_value();
Ok(())
}
/// A weak notification bridge. Shared-device hosts keep ownership of their handlers.
pub(crate) fn failure_notifier(me: &Arc<Readbacks>) -> impl Fn(String) + Send + Sync + 'static {
#[cfg(not(target_arch = "wasm32"))]
let fail = {
let me = Arc::downgrade(me);
move |error: String| {
if let Some(me) = me.upgrade() {
me.retire(error);
}
}
};
#[cfg(target_arch = "wasm32")]
let fail = {
// GPU callbacks require Send even on wasm. Capture only the index; the registry
// (which can contain JS callbacks) stays on this worker's event-loop thread.
let index = REGISTRIES.with(|r| {
let mut r = r.borrow_mut();
r.push(Arc::downgrade(me));
r.len() - 1
});
move |error: String| {
let me = REGISTRIES.with(|r| r.borrow().get(index).and_then(Weak::upgrade));
if let Some(me) = me {
me.retire(error);
}
}
};
fail
}
/// Replacement handlers are installed only by the successful owned-device request path.
pub(crate) fn install(device: &wgpu::Device, me: &Arc<Readbacks>) {
let fail = Arc::new(failure_notifier(me));
let lost = fail.clone();
device.set_device_lost_callback(move |reason, message| lost(format!("GPU device lost ({reason:?}): {message}")));
device.on_uncaptured_error(Arc::new(move |e| {
let message = format!("uncaptured GPU error: {e}");
fail(message.clone());
#[cfg(test)]
#[allow(clippy::panic)]
if matches!(e, wgpu::Error::Validation { .. }) {
panic!("{message}");
}
log::error!("{message}");
}));
}
#[cfg(target_arch = "wasm32")]
thread_local! {
static REGISTRIES: std::cell::RefCell<Vec<Weak<Readbacks>>> = const { std::cell::RefCell::new(Vec::new()) };
}
/// Validate every row before publishing tightly packed bytes.
pub(crate) fn packed(view: &[u8], row: usize, stride: usize, height: usize) -> Result<Vec<u8>, Error> {
if row == 0 || height == 0 || stride < row {
return Err("invalid GPU readback layout".into());
}
let size = row.checked_mul(height).ok_or("GPU readback output size overflow")?;
let end = (height - 1).checked_mul(stride).and_then(|s| s.checked_add(row)).ok_or("GPU readback mapped size overflow")?;
if view.len() < end {
return Err(format!("short GPU readback: {} bytes, needs {end}", view.len()));
}
let mut out = Vec::new();
out.try_reserve_exact(size).map_err(|e| format!("allocating GPU readback output: {e}"))?;
for y in 0..height {
let start = y.checked_mul(stride).ok_or("GPU readback row overflow")?;
let bytes = view.get(start..start + row).ok_or("short GPU readback row")?;
out.extend_from_slice(bytes);
}
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::atomic::{AtomicUsize, Ordering};
#[test]
fn early_rejection_reports_no_pixels_exactly_once() {
let calls = Arc::new(AtomicUsize::new(0));
let observed = calls.clone();
reject(
move |pixels| {
assert!(pixels.is_none());
observed.fetch_add(1, Ordering::SeqCst);
},
"synthetic early rejection".into(),
);
assert_eq!(calls.load(Ordering::SeqCst), 1);
}
#[test]
#[cfg(all(not(target_arch = "wasm32"), panic = "unwind"))]
fn early_rejection_contains_callback_panic_and_allows_later_completion() {
let calls = Arc::new(AtomicUsize::new(0));
let first = calls.clone();
reject(
move |pixels| {
assert!(pixels.is_none());
first.fetch_add(1, Ordering::SeqCst);
panic!("synthetic rejected callback panic");
},
"synthetic retired context".into(),
);
let second = calls.clone();
reject(
move |pixels| {
assert!(pixels.is_none());
second.fetch_add(1, Ordering::SeqCst);
},
"synthetic invalid layout".into(),
);
assert_eq!(calls.load(Ordering::SeqCst), 2);
}
#[test]
#[cfg(all(not(target_arch = "wasm32"), panic = "unwind"))]
fn panicking_callback_does_not_strand_waiters_or_kill_deadline_worker() {
use std::future::Future;
use std::task::{Context, Poll, Wake, Waker};
struct WakeCount(AtomicUsize);
impl Wake for WakeCount {
fn wake(self: Arc<Self>) {
self.0.fetch_add(1, Ordering::SeqCst);
}
}
let me = Readbacks::new().unwrap();
let deferred = Arc::new(crate::deferred::Deferred::default());
let (tx, rx) = std::sync::mpsc::channel();
let _first = me.start(|_| panic!("synthetic public callback panic")).unwrap();
for key in 1..=2 {
let done = deferred.start(key, 1, 1, [0.0; 2], 1.0);
let tx = tx.clone();
let _ticket = me
.start(move |r| {
assert!(r.is_err());
done(None);
tx.send(()).unwrap();
})
.unwrap();
}
let wakes = Arc::new(WakeCount(AtomicUsize::new(0)));
let waker = Waker::from(wakes.clone());
let mut settled = std::pin::pin!(deferred.settled());
assert!(settled.as_mut().poll(&mut Context::from_waker(&waker)).is_pending());
me.cancel_pending("synthetic cancellation".into());
assert_eq!(deferred.in_flight(), 0);
assert_eq!(wakes.0.load(Ordering::SeqCst), 1);
assert!(matches!(settled.as_mut().poll(&mut Context::from_waker(&waker)), Poll::Ready(())));
assert!(me.0.lock().unwrap().pending.is_empty());
rx.recv_timeout(Duration::from_secs(2)).unwrap();
rx.recv_timeout(Duration::from_secs(2)).unwrap();
// Drive an actual later watchdog expiry without waiting for the production 10s.
let _first = me.start(|_| panic!("synthetic watchdog callback panic")).unwrap();
let tx2 = tx.clone();
let _second = me
.start(move |r| {
assert!(r.is_err());
tx2.send(()).unwrap();
})
.unwrap();
{
let mut state = me.0.lock().unwrap();
for entry in state.pending.values_mut() {
entry.deadline = web_time::Instant::now();
}
}
rx.recv_timeout(Duration::from_secs(2)).unwrap();
assert!(me.0.lock().unwrap().pending.is_empty());
assert!(me.check().is_err());
}
#[test]
fn missing_callback_times_out_once_and_rejects_late_success() {
let me = Arc::new(Readbacks::default());
let calls = Arc::new(AtomicUsize::new(0));
let c = calls.clone();
let ticket = me
.start(move |r| {
assert!(r.is_err());
c.fetch_add(1, Ordering::SeqCst);
})
.unwrap();
me.expire(web_time::Instant::now() + TIMEOUT + Duration::from_secs(1));
assert!(!ticket.active());
ticket.complete(Ok(vec![42]));
assert_eq!(calls.load(Ordering::SeqCst), 1);
assert!(me.check().is_err());
}
#[test]
fn cancellation_drains_all_and_new_context_is_independent() {
let old = Arc::new(Readbacks::default());
let calls = Arc::new(AtomicUsize::new(0));
let mut tickets = Vec::new();
for _ in 0..3 {
let c = calls.clone();
tickets.push(
old.start(move |r| {
assert!(r.is_err());
c.fetch_add(1, Ordering::SeqCst);
})
.unwrap(),
);
}
old.retire("synthetic device failure".into());
assert_eq!(calls.load(Ordering::SeqCst), 3);
let new = Arc::new(Readbacks::default());
let c = calls.clone();
let fresh = new
.start(move |r| {
assert_eq!(r.unwrap(), vec![7]);
c.fetch_add(1, Ordering::SeqCst);
})
.unwrap();
for t in tickets {
t.complete(Ok(vec![99]));
}
assert!(fresh.active());
fresh.complete(Ok(vec![7]));
assert_eq!(calls.load(Ordering::SeqCst), 4);
}
#[test]
fn cancelled_ticket_cannot_complete_a_new_request_on_same_context() {
let me = Arc::new(Readbacks::default());
let calls = Arc::new(AtomicUsize::new(0));
let c = calls.clone();
let old = me
.start(move |r| {
assert!(r.is_err());
c.fetch_add(1, Ordering::SeqCst);
})
.unwrap();
me.cancel_pending("cancelled".into());
let c = calls.clone();
let new = me
.start(move |r| {
assert_eq!(r.unwrap(), vec![7]);
c.fetch_add(1, Ordering::SeqCst);
})
.unwrap();
old.complete(Ok(vec![99]));
assert!(new.active());
new.complete(Ok(vec![7]));
assert_eq!(calls.load(Ordering::SeqCst), 2);
}
#[test]
#[cfg(not(target_arch = "wasm32"))]
fn completion_and_failure_race_invokes_callback_once() {
for _ in 0..32 {
let me = Arc::new(Readbacks::default());
let calls = Arc::new(AtomicUsize::new(0));
let c = calls.clone();
let ticket = me
.start(move |_| {
c.fetch_add(1, Ordering::SeqCst);
})
.unwrap();
let barrier = Arc::new(std::sync::Barrier::new(2));
let b = barrier.clone();
let t = std::thread::spawn(move || {
b.wait();
ticket.complete(Ok(vec![1]));
});
barrier.wait();
me.retire("concurrent device failure".into());
t.join().unwrap();
assert_eq!(calls.load(Ordering::SeqCst), 1);
}
}
#[test]
fn malformed_layouts_fail_without_allocating() {
assert!(Layout::new(u32::MAX, 2, 16, u64::MAX).is_err());
assert!(Layout::new(1, 0, 4, u64::MAX).is_err());
assert!(Layout::new(8192, 8192, 16, 128).is_err());
}
#[test]
fn completed_request_survives_later_failure_without_second_callback() {
let me = Arc::new(Readbacks::default());
let calls = Arc::new(AtomicUsize::new(0));
let c = calls.clone();
let ticket = me
.start(move |r| {
assert_eq!(r.unwrap(), vec![1]);
c.fetch_add(1, Ordering::SeqCst);
})
.unwrap();
ticket.complete(Ok(vec![1]));
me.retire("later failure".into());
assert_eq!(calls.load(Ordering::SeqCst), 1);
}
#[test]
fn no_partial_rows() {
assert!(packed(&[0; 7], 4, 4, 2).is_err());
assert_eq!(packed(&[1, 2, 0, 0, 3, 4], 2, 4, 2).unwrap(), vec![1, 2, 3, 4]);
}
}

View File

@ -25,7 +25,9 @@ struct Post {
@group(0) @binding(0) var<uniform> P: Post;
@group(0) @binding(1) var color_in: texture_2d<f32>;
@group(0) @binding(2) var z_in: texture_2d<f32>;
// Camera depth as f32 bits (an `R32Uint` target: integer targets render on every backend,
// `R32Float` doesn't on GLES without EXT_color_buffer_float).
@group(0) @binding(2) var z_in: texture_2d<u32>;
@group(0) @binding(3) var<storage, read_write> acc: array<vec4<f32>>;
@group(0) @binding(4) var<storage, read_write> depth: array<f32>;
@group(0) @binding(5) var<storage, read_write> src: array<vec4<f32>>;
@ -55,7 +57,7 @@ fn resolve(@builtin(global_invocation_id) id: vec3<u32>) {
for (var i = 0u; i < k; i = i + 1u) {
let p = vec2<i32>(i32(id.x * k + i), i32(id.y * k + j));
a = a + textureLoad(color_in, p, 0);
let z = textureLoad(z_in, p, 0).x;
let z = bitcast<f32>(textureLoad(z_in, p, 0).x);
if (z >= 0.0) {
dz = dz + z;
dn = dn + 1.0;

View File

@ -75,26 +75,28 @@ fn srgb_to_linear(c: f32) -> f32 {
return pow((c + 0.055) / 1.055, 2.4);
}
// `a` mod `n` in [0, n) for n > 0 (floored). Never `%` on a negative operand: WGSL's `%` truncates,
// but naga's GLSL output is plain `%`, undefined for negative operands (wrong on Mesa's GL).
fn imod(a: i32, n: i32) -> i32 {
if (a >= 0) {
return a % n;
}
return n - 1 - (-(a + 1)) % n;
}
fn wrap_i(i: i32, n: i32, mode: u32) -> i32 {
if (mode == 1u) {
return clamp(i, 0, n - 1);
}
if (mode == 2u) {
let p = 2 * n;
var m = i % p;
if (m < 0) {
m = m + p;
}
let m = imod(i, p);
if (m >= n) {
return p - 1 - m;
}
return m;
}
var m = i % n;
if (m < 0) {
m = m + n;
}
return m;
return imod(i, n);
}
fn texel(t: vec4<u32>, x: i32, y: i32) -> vec4<f32> {
@ -282,7 +284,8 @@ fn attenuation(kind: u32, falloff: u32, radius: f32, falloff_distance: f32, cos_
struct FOut {
@location(0) color: vec4<f32>,
@location(1) depth: f32,
// Camera depth's f32 bits (an `R32Uint` target, see `adv3d.rs`).
@location(1) depth: u32,
};
@fragment
@ -326,7 +329,7 @@ fn fs(i: VOut, @builtin(front_facing) front: bool) -> FOut {
emissive = emissive * sample_tex(tex_em, i.uv).rgb;
}
var out: FOut;
out.depth = dot(g.view_z.xyz, i.pos) + g.view_z.w;
out.depth = bitcast<u32>(dot(g.view_z.xyz, i.pos) + g.view_z.w);
if (unlit || !accepts_lights || g.env2.y < 0.5) {
let c = rgb + emissive;
out.color = vec4<f32>(c * alpha, alpha);

View File

@ -186,7 +186,7 @@ fn attenuation(l: u32, p: vec3<f32>) -> f32 {
}
if (li.ints.x == 3u) {
let cos_a = clamp(dot(-tl.xyz, li.dir.xyz), -1.0, 1.0);
let a = acos(cos_a);
let a = acos_p(cos_a);
let outer = li.cone.x;
let inner = outer * (1.0 - li.cone.y);
if (a <= inner) {

View File

@ -16,6 +16,40 @@ struct Params {
@group(0) @binding(2) var aux: texture_2d<f32>;
@group(0) @binding(4) var<storage, read> data: array<f32>;
// `a` mod `n` in [0, n) for n > 0 (floored). Never `%` on a negative operand: WGSL's `%` truncates,
// but naga's GLSL output is plain `%`, undefined for negative operands (wrong on Mesa's GL).
fn imod(a: i32, n: i32) -> i32 {
if (a >= 0) {
return a % n;
}
return n - 1 - (-(a + 1)) % n;
}
// asin / acos to f32 precision on every backend (Cephes' asinf polynomial). WGSL leaves their
// precision to the implementation, and Mesa's GL drivers use a coarse approximation (errors near
// 1e-4 rad), which shows in every effect that maps through a sphere or a lens.
fn asin_p(x: f32) -> f32 {
let a = abs(x);
let big = a > 0.5;
let z = select(a * a, 0.5 * (1.0 - a), big);
let s = select(a, sqrt(z), big);
var p = ((((4.2163199048e-2 * z + 2.4181311049e-2) * z + 4.5470025998e-2) * z + 7.4953002686e-2) * z + 1.6666752422e-1) * z * s + s;
if (big) {
p = 1.5707963267948966 - 2.0 * p;
}
return select(p, -p, x < 0.0);
}
fn acos_p(x: f32) -> f32 {
if (x > 0.5) {
return 2.0 * asin_p(sqrt(0.5 * (1.0 - x)));
}
if (x < -0.5) {
return 3.141592653589793 - 2.0 * asin_p(sqrt(0.5 * (1.0 + x)));
}
return 1.5707963267948966 - asin_p(x);
}
// ---------------------------------------------------------------- sampling (raster::Image)
fn tex_get(t: texture_2d<f32>, x: i32, y: i32) -> vec4<f32> {

View File

@ -215,7 +215,7 @@ fn dst_cclens(@builtin(global_invocation_id) gid: vec3<u32>) {
}
var s = q;
if (d >= 1e-9) {
let g = powz(asin(min(d / r, 1.0)) / DST_HALF_PI, P.f[0].w);
let g = powz(asin_p(min(d / r, 1.0)) / DST_HALF_PI, P.f[0].w);
let k = r * g / d;
s = vec2<f32>(c.x + dx * k, c.y + dy * k);
}

View File

@ -653,7 +653,7 @@ fn fg2_threads(@builtin(global_invocation_id) gid: vec3<u32>) {
let fv = v / th - f32(cj);
let in_h = abs(fv - 0.5) < coverage * 0.5;
let in_v = abs(fu - 0.5) < coverage * 0.5;
let h_top = ((ci + cj) % (2 * overlaps) + 2 * overlaps) % (2 * overlaps) < overlaps;
let h_top = imod(ci + cj, 2 * overlaps) < overlaps;
var horizontal: bool;
if (in_h && in_v) {
horizontal = h_top;

View File

@ -349,8 +349,8 @@ fn gen_worley(u: f32, v: f32, disperse: f32, evo: f32, seed: u32) -> vec3<f32> {
var tx = gx;
var ty = gy;
if (tile) {
tx = ((gx % nx) + nx) % nx;
ty = ((gy % ny) + ny) % ny;
tx = imod(gx, nx);
ty = imod(gy, ny);
}
let hx = bitcast<u32>(tx);
let hy = bitcast<u32>(ty);
@ -428,7 +428,7 @@ fn gen_axis(q: f32, size: f32, f: f32) -> f32 {
let fr = u - fl;
let d = min(fr, 1.0 - fr) * size;
let n = i32(fl);
let s = select(-1.0, 1.0, ((n % 2) + 2) % 2 == 0);
let s = select(-1.0, 1.0, imod(n, 2) == 0);
return s * min(d / (f * 0.5), 1.0);
}

View File

@ -2,10 +2,17 @@
// the CPU effect in effectcraft-effects operation for operation. Every name here is prefixed
// `fxk_` (the family files share one module).
// Correct the approximate quotient before bitwise/threshold operations.
fn fxk_div3(a: vec3<f32>, b: f32) -> vec3<f32> {
let q = a / b;
let r = fma(-q, vec3<f32>(b), a);
return fma(r, vec3<f32>(1.0 / b), q);
}
// util::unpremul: (straight colour, alpha); colour 0 when alpha ≤ 1e-6.
fn fxk_straight(px: vec4<f32>) -> vec3<f32> {
if (px.w > 1e-6) {
return px.xyz / px.w;
return fxk_div3(px.xyz, px.w);
}
return vec3<f32>(0.0);
}
@ -457,7 +464,7 @@ fn fxk_point(@builtin(global_invocation_id) gid: vec3<u32>) {
textureStore(out, p, px);
return;
}
let s = px.xyz / px.w;
let s = fxk_div3(px.xyz, px.w);
var r = vec3<f32>(0.0);
for (var i = 0u; i < 3u; i++) {
let v = s[i];

View File

@ -15,11 +15,14 @@ fn fxn_pixel(gid: vec3<u32>) -> vec2<i32> {
}
// a / b rounded like the CPU's IEEE division (GPU division may be approximate): one
// residual correction with fma. Used where a threshold follows the quotient.
// residual correction with fma, followed by a second refinement for half-step cases.
// Used where a strict threshold follows the quotient.
fn fxn_div4(a: vec4<f32>, b: vec4<f32>) -> vec4<f32> {
let q = a / b;
let r = fma(-q, b, a);
return fma(r, 1.0 / b, q);
let corrected = fma(r, 1.0 / b, q);
let residual = fma(-corrected, b, a);
return fma(residual, 1.0 / b, corrected);
}
// util::unpremul: (straight colour, alpha).

View File

@ -661,7 +661,7 @@ fn fp2_hextile(@builtin(global_invocation_id) gid: vec3<u32>) {
let hy = r * 1.5 * rz;
var lu = u - hx;
var lv = v - hy;
let odd = ((i32(rx) + i32(rz)) % 2 + 2) % 2 == 1;
let odd = imod(i32(rx) + i32(rz), 2) == 1;
if ((mode == 1u || mode == 2u) && odd) {
lu = -lu;
}

View File

@ -44,7 +44,7 @@ fn fxs_qdiv(x: vec3<f32>, a: f32) -> vec3<f32> {
}
fn fxs_rem2(i: i32) -> i32 {
return ((i % 2) + 2) % 2;
return imod(i, 2);
}
// ---------------------------------------------------------------- per-pixel colour effects
@ -257,7 +257,7 @@ fn fxs_warp(@builtin(global_invocation_id) gid: vec3<u32>) {
let d = sqrt(dx * dx + dy * dy);
if (d < r && d != 0.0) {
let nd = d / r;
let k = (asin(nd) / FXS_HALF_PI) / nd;
let k = (asin_p(nd) / FXS_HALF_PI) / nd;
s = vec2<f32>(c.x + dx * k, c.y + dy * k);
}
}
@ -453,10 +453,15 @@ fn fxs_warp(@builtin(global_invocation_id) gid: vec3<u32>) {
u = bu + (u - bu) * persp;
v = bv + (v - bv) * persp;
}
if (u < -1e-9 || u > 1.0 + 1e-9 || v < -1e-9 || v > 1.0 + 1e-9) {
// The CPU's 1e-9 edge allowance disappears in f32 at 1.0. Allow eight
// f32 ULPs for the inverse solve so exact quad-edge pixels remain covered.
let edge = 0.00000095367431640625;
if (u < -edge || u > 1.0 + edge || v < -edge || v > 1.0 + edge) {
textureStore(out, p, vec4<f32>(0.0));
return;
}
u = clamp(u, 0.0, 1.0);
v = clamp(v, 0.0, 1.0);
let rc = P.f[5];
let lx = rc.x + u * rc.z;
let ly = rc.y + v * rc.w;

View File

@ -681,7 +681,7 @@ fn ftr_cylinder(@builtin(global_invocation_id) gid: vec3<u32>) {
return;
}
let tau = 6.283185307179586;
let th = asin(s);
let th = asin_p(s);
let cz = cos(th);
let spin = P.f[2].x;
let front = ftr_shade(ftr_texel((th + spin) / tau + 0.5, v), vec3<f32>(s, 0.0, cz));
@ -695,7 +695,7 @@ fn ftr_sphere_tex(q: vec3<f32>) -> vec4<f32> {
let m1 = P.f[8].xyz;
let m2 = P.f[9].xyz;
let t = m0 * q.x + m1 * q.y + m2 * q.z;
let lat = asin(clamp(t.y, -1.0, 1.0));
let lat = asin_p(clamp(t.y, -1.0, 1.0));
return ftr_texel(ftr_lon_u(t.x, t.z), lat / 3.141592653589793 + 0.5);
}
@ -775,7 +775,7 @@ fn ftr_env_uv(mapping: u32, r: vec3<f32>) -> vec2<f32> {
}
return vec2<f32>((cr.x + (cr.z + 1.0) * 0.5) / 3.0, (cr.y + (cr.w + 1.0) * 0.5) / 4.0);
}
let lat = asin(clamp(r.y, -1.0, 1.0));
let lat = asin_p(clamp(r.y, -1.0, 1.0));
return vec2<f32>(ftr_lon_u(r.x, r.z), 0.5 + lat / 3.141592653589793);
}

View File

@ -22,7 +22,7 @@ fn fxv_equi_dir(x: f32, y: f32, w: f32, h: f32) -> vec3<f32> {
// vr::equi_px.
fn fxv_equi_px(d: vec3<f32>, w: f32, h: f32) -> vec2<f32> {
let lon = atan2(d.x, d.z);
let lat = asin(clamp(d.y, -1.0, 1.0));
let lat = asin_p(clamp(d.y, -1.0, 1.0));
return vec2<f32>((lon / (2.0 * FXV_PI) + 0.5) * w, (0.5 - lat / FXV_PI) * h);
}
@ -42,7 +42,7 @@ fn fxv_equi_at(xi0: i32, yi0: i32, w: i32, h: i32) -> vec4<f32> {
yi = 2 * h - 1 - yi;
xi += w / 2;
}
let xm = ((xi % w) + w) % w;
let xm = imod(xi, w);
return textureLoad(src, vec2<i32>(xm, clamp(yi, 0, h - 1)), 0);
}
@ -233,7 +233,7 @@ fn fxv_proj_sample(kind: u32, fov: f32, d: vec3<f32>) -> vec4<f32> {
return sample_bilinear_clamped(src, x, y);
}
if (kind == 8u) {
let th = acos(clamp(d.y, -1.0, 1.0));
let th = acos_p(clamp(d.y, -1.0, 1.0));
if (th > FXV_PI / 2.0 + 1e-6) {
return vec4<f32>(0.0);
}
@ -250,7 +250,7 @@ fn fxv_proj_sample(kind: u32, fov: f32, d: vec3<f32>) -> vec4<f32> {
}
if (kind == 4u) {
let half = radians(clamp(fov, 1.0, 360.0) / 2.0);
let th = acos(clamp(d.z, -1.0, 1.0));
let th = acos_p(clamp(d.z, -1.0, 1.0));
if (th > half + 1e-6) {
return vec4<f32>(0.0);
}
@ -346,12 +346,12 @@ fn fxv_box_h(@builtin(global_invocation_id) gid: vec3<u32>) {
let norm = 1.0 / f32(2 * r + 1);
var acc = vec4<f32>(0.0);
for (var i = b0 - r; i <= b0 + r; i++) {
acc += textureLoad(src, vec2<i32>(((i % w) + w) % w, y), 0);
acc += textureLoad(src, vec2<i32>(imod(i, w), y), 0);
}
for (var x = b0; x < b1; x++) {
textureStore(out, vec2<i32>(x, y), acc * norm);
let add = textureLoad(src, vec2<i32>((x + r + 1) % w, y), 0);
let sub = textureLoad(src, vec2<i32>(((x - r) % w + w) % w, y), 0);
let sub = textureLoad(src, vec2<i32>(imod(x - r, w), y), 0);
acc += add - sub;
}
}
@ -362,10 +362,10 @@ fn fxv_pole_at(x0: i32, y0: i32, w: i32, h: i32) -> vec4<f32> {
var y = y0;
if (y < 0) {
y = -1 - y;
x = (x + w / 2) % w;
x = imod(x + w / 2, w);
} else if (y >= h) {
y = 2 * h - 1 - y;
x = (x + w / 2) % w;
x = imod(x + w / 2, w);
}
return textureLoad(src, vec2<i32>(x, clamp(y, 0, h - 1)), 0);
}
@ -413,7 +413,7 @@ fn fxv_wrap_pad(@builtin(global_invocation_id) gid: vec3<u32>) {
sy = 2 * s.y - 1 - sy;
sx += s.x / 2;
}
textureStore(out, p, textureLoad(src, vec2<i32>(((sx % s.x) + s.x) % s.x, clamp(sy, 0, s.y - 1)), 0));
textureStore(out, p, textureLoad(src, vec2<i32>(imod(sx, s.x), clamp(sy, 0, s.y - 1)), 0));
}
// Per-pixel steps. u[0].x = 0: VR Glow's highlights (u[0].y tint; f[0] = (threshold,

View File

@ -237,7 +237,7 @@ fn fxw_pageturn(@builtin(global_invocation_id) gid: vec3<u32>) {
cand[0] = vec4<f32>(FXW_PI * r - d, 1.0, 0.9, 1.0);
cand[3] = vec4<f32>(d, 0.0, 1.0, 1.0);
} else {
let a = asin(clamp(d / r, -1.0, 1.0));
let a = asin_p(clamp(d / r, -1.0, 1.0));
cand[1] = vec4<f32>(FXW_PI * r - r * a, 1.0, 0.6 + 0.4 * cos(a), 1.0);
let psi = a / (0.5 * FXW_PI);
cand[2] = vec4<f32>(r * a, 0.0, 1.0 - 0.4 * psi * (0.5 + 0.5 * ldot), 1.0);

View File

@ -33,7 +33,7 @@ fn warp_blend(@builtin(global_invocation_id) gid: vec3<u32>) {
if ((flags & 2u) != 0u) {
d = vec4<f32>(0.0);
}
store_q(vec2<i32>(x, y), d, P.f[3].z);
store_q(vec2<i32>(x, y), d, P.f[3].z, P.f[3].w);
return;
}
let r0 = P.f[0];
@ -52,7 +52,7 @@ fn warp_blend(@builtin(global_invocation_id) gid: vec3<u32>) {
if (stencil && (flags & 1u) == 0u) {
d = blend_pixel(mode, d, vec4<f32>(0.0), 0.5);
}
store_q(vec2<i32>(x, y), d, P.f[3].z);
store_q(vec2<i32>(x, y), d, P.f[3].z, P.f[3].w);
return;
}
var s = sample(src, sampling, px, py);
@ -60,11 +60,11 @@ fn warp_blend(@builtin(global_invocation_id) gid: vec3<u32>) {
if (s.w > 0.0) {
d = d + s * P.f[3].y;
}
store_q(vec2<i32>(x, y), d, P.f[3].z);
store_q(vec2<i32>(x, y), d, P.f[3].z, P.f[3].w);
return;
}
if (s.w <= 0.0 && !stencil) {
store_q(vec2<i32>(x, y), d, P.f[3].z);
store_q(vec2<i32>(x, y), d, P.f[3].z, P.f[3].w);
return;
}
let op = P.f[3].x;
@ -75,7 +75,7 @@ fn warp_blend(@builtin(global_invocation_id) gid: vec3<u32>) {
if (mode == 1u || mode == 2u) {
n = hash_noise(u32(x), u32(y), seed);
}
store_q(vec2<i32>(x, y), blend_pixel(mode, d, s, n), P.f[3].z);
store_q(vec2<i32>(x, y), blend_pixel(mode, d, s, n), P.f[3].z, P.f[3].w);
}
// Blend `src` (same size) onto `aux` (Image::blend_from).
@ -91,7 +91,7 @@ fn blend_full(@builtin(global_invocation_id) gid: vec3<u32>) {
let d = textureLoad(aux, p, 0);
var s = textureLoad(src, p, 0);
if (s.w <= 0.0 && !is_stencil(mode)) {
store_q(p, d, P.f[0].y);
store_q(p, d, P.f[0].y, P.f[0].z);
return;
}
s = s * P.f[0].x;
@ -99,7 +99,7 @@ fn blend_full(@builtin(global_invocation_id) gid: vec3<u32>) {
if (mode == 1u || mode == 2u) {
n = hash_noise(gid.x, gid.y, P.u[0].z);
}
store_q(p, blend_pixel(mode, d, s, n), P.f[0].y);
store_q(p, blend_pixel(mode, d, s, n), P.f[0].y, P.f[0].z);
}
// Track matte: `src` × matte factor of `aux`. u[0].x = 0 alpha, 1 alpha inverted, 2 luma,
@ -179,24 +179,24 @@ fn quantize(@builtin(global_invocation_id) gid: vec3<u32>) {
if (p.x >= dims.x || p.y >= dims.y) {
return;
}
textureStore(out, p, quantize_px(textureLoad(src, p, 0), P.f[0].x));
textureStore(out, p, quantize_px(textureLoad(src, p, 0), P.f[0].x, P.f[0].y));
}
fn quantize_px(s: vec4<f32>, levels: f32) -> vec4<f32> {
let inv = 1.0 / levels;
let a = round_away(clamp(s.w, 0.0, 1.0) * levels) * inv;
fn quantize_px(s: vec4<f32>, levels: f32, inv: f32) -> vec4<f32> {
// Use the CPU-rounded reciprocal to avoid backend-dependent division.
let a = floor(clamp(s.w, 0.0, 1.0) * levels + 0.5) * inv;
var o = vec4<f32>(0.0, 0.0, 0.0, a);
o.x = round_away(clamp(s.x, 0.0, a) * levels) * inv;
o.y = round_away(clamp(s.y, 0.0, a) * levels) * inv;
o.z = round_away(clamp(s.z, 0.0, a) * levels) * inv;
o.x = floor(clamp(s.x, 0.0, a) * levels + 0.5) * inv;
o.y = floor(clamp(s.y, 0.0, a) * levels + 0.5) * inv;
o.z = floor(clamp(s.z, 0.0, a) * levels + 0.5) * inv;
return o;
}
// Store a compositing result, clamped and quantised to `levels` when > 0 (8/16 bpc: the comp
// after a layer, fused into the layer's composite).
fn store_q(p: vec2<i32>, v: vec4<f32>, levels: f32) {
fn store_q(p: vec2<i32>, v: vec4<f32>, levels: f32, inv: f32) {
if (levels > 0.0) {
textureStore(out, p, quantize_px(v, levels));
textureStore(out, p, quantize_px(v, levels, inv));
} else {
textureStore(out, p, v);
}

View File

@ -21,7 +21,7 @@ fn adv_sky(@builtin(global_invocation_id) gid: vec3<u32>) {
// shade::equirect_uv.
let pi = 3.141592653589793;
var u = 0.5 + (atan2(dir.x, dir.z) + P.f[2].w) / (2.0 * pi);
let v = acos(clamp(-dir.y, -1.0, 1.0)) / pi;
let v = acos_p(clamp(-dir.y, -1.0, 1.0)) / pi;
u = u - floor(u);
let iw = P.f[4].x;
let ih = P.f[4].y;

View File

@ -26,18 +26,57 @@ use effectcraft_time::{FrameRate, Tick};
use crate::Gpu;
/// One GPU test at a time in this process: the calling test thread holds a process-wide lock
/// until it exits (libtest runs every test on a thread of its own). wgpu's OpenGL backend (Mesa
/// llvmpipe on FreeBSD, Linux without Vulkan) has one context lock per adapter and panics when a
/// thread waits on it for more than a few seconds ("Could not lock adapter context"), and
/// concurrent devices on llvmpipe can crash. Every test helper that hands out a device calls this
/// first; calling it again on the same thread is a no-op.
pub(crate) fn hold_gpu_lock() {
use std::cell::RefCell;
use std::sync::{Mutex, MutexGuard, PoisonError};
static GPU_LOCK: Mutex<()> = Mutex::new(());
thread_local! {
static HELD: RefCell<Option<MutexGuard<'static, ()>>> = const { RefCell::new(None) };
}
HELD.with(|h| {
let mut h = h.borrow_mut();
if h.is_none() {
*h = Some(GPU_LOCK.lock().unwrap_or_else(PoisonError::into_inner));
}
});
}
/// The shared test device (the calling test holds [`hold_gpu_lock`]).
pub(crate) fn gpu() -> Option<&'static Gpu> {
static G: OnceLock<Option<Gpu>> = OnceLock::new();
hold_gpu_lock();
G.get_or_init(|| {
let g = Gpu::headless();
if g.is_none() {
eprintln!("effectcraft-gpu tests: no GPU adapter, skipping GPU comparisons");
} else if let Some(g) = &g {
eprintln!("effectcraft-gpu tests: adapter {}", g.ctx.name);
}
g
})
.as_ref()
}
/// The test device is on wgpu's OpenGL backend (Mesa's llvmpipe on the FreeBSD CI job).
pub(crate) fn on_gl() -> bool {
gpu().is_some_and(|g| g.ctx.name.ends_with("(Gl)"))
}
/// On GL, the fraction of pixels allowed past the tolerance: pixels on a decision boundary (a
/// hex cell's edge, a threshold, a corner pin's coverage edge, a median's rank tie) may land on
/// the other side, since GL drivers may divide through a reciprocal and GLSL's `fma` need not be
/// fused (so `fxs_qdiv`-style corrections don't apply). Measured on llvmpipe: at most 0.53 % (CC
/// HexTile at half resolution, many cell edges in a small frame). A wrong wrap (signed `%`) showed
/// in 3–70 % of pixels and still fails; a precision loss may show in fewer (Mesa's `asin`: 0.3–3 %),
/// so the shaders avoid imprecise builtins themselves (`asin_p`) rather than lean on this.
pub(crate) const GL_BOUNDARY_FLIPS: f64 = 0.01;
/// Procedural footage: smooth colour gradients, fine noise and an alpha with soft, opaque and
/// fully transparent regions (different per item).
pub(crate) struct Pattern;
@ -205,6 +244,7 @@ pub(crate) fn check(label: &str, d: Option<Diff>, allow: f64) {
let frac = d.over as f64 / d.total as f64;
// Quantised depths: rounding-boundary flips (see the module docs).
let (allow, cap) = if d.quantized && allow == 0.0 { (0.003, 4.0 / 255.0 + 1e-6) } else { (allow, f32::INFINITY) };
let (allow, cap) = if on_gl() { (allow.max(GL_BOUNDARY_FLIPS), f32::INFINITY) } else { (allow, cap) };
assert!(frac <= allow && d.max <= cap, "{label}: {} of {} pixels over tolerance (max {:.6}); worst {:?}", d.over, d.total, d.max, d.worst);
if d.over > 0 {
eprintln!("{label}: {} of {} pixels over tolerance (max {:.6})", d.over, d.total, d.max);
@ -403,7 +443,15 @@ pub(crate) fn effect_direct(id: &str, vals: &[(&str, Value)], adjustment: bool)
let out = effectcraft_render::Accelerator::effects(g, &[effectcraft_render::FxStep { spec, ctx: ctx() }], &buf, None).expect("the GPU runs the effect");
assert_eq!((out.offset, out.scale), (cpu.offset, cpu.scale), "{id}: geometry");
let d = diff(&cpu.img, &out.img, 1e-3);
assert!(d.over == 0, "{id} {vals:?} adj {adjustment} scale {scale}: {} of {} pixels over 1e-3 (max {}); worst {:?}", d.over, d.total, d.max, d.worst);
let allow = if on_gl() { (d.total as f64 * GL_BOUNDARY_FLIPS) as usize } else { 0 };
assert!(
d.over <= allow,
"{id} {vals:?} adj {adjustment} scale {scale}: {} of {} pixels over 1e-3 (max {}); worst {:?}",
d.over,
d.total,
d.max,
d.worst
);
}
}
@ -534,6 +582,30 @@ fn registry_badges_match_the_gpu_implementation() {
}
}
#[test]
fn quantization_preserves_cpu_levels_and_half_step_neighbors() {
let Some(g) = gpu() else { return };
for levels in [255.0_f32, 32768.0] {
let mut values = vec![0.0, 1.0];
for i in 0..255 {
let half = (i as f32 + 0.5) / levels;
values.extend([half.next_down(), half, half.next_up()]);
}
let mut cpu = Image::new(values.len() as u32, 1);
for (p, v) in cpu.data.iter_mut().zip(values) {
*p = [v, v, v, 1.0];
}
let input = g.ctx.upload_image(&cpu).unwrap();
effectcraft_render::color::quantize(&mut cpu, levels);
let mut e = crate::context::Enc::new(&g.ctx);
let out = crate::ops::quantize(&mut e, &input, levels);
let bytes = e.read_texture(&out.texture, out.width, out.height, 16).unwrap();
for (i, (want, got)) in cpu.data.iter().flatten().zip(bytes.as_chunks::<4>().0).enumerate() {
let actual = f32::from_le_bytes(*got);
assert_eq!(actual, *want, "levels {levels}, component {i}");
}
}
}
#[test]
fn backend_selection_and_display_frames() {
let Some(g) = gpu() else { return };
@ -554,7 +626,7 @@ fn backend_selection_and_display_frames() {
let f = g.render_display(&r, s.cid, Tick::ZERO).expect("display frame");
let bytes = g.read_display(&f).expect("readback");
let cpu = Renderer::new(&s.p, &Pattern, opts()).comp_frame_cpu(s.cid, Tick::ZERO);
for (p, q) in cpu.data.iter().zip(bytes.chunks_exact(4)) {
for (p, q) in cpu.data.iter().zip(bytes.as_chunks::<4>().0.iter()) {
let a = p[3].clamp(0.0, 1.0);
let want = [p[0].clamp(0.0, a), p[1].clamp(0.0, a), p[2].clamp(0.0, a), a].map(|v| (v * 255.0 + 0.5) as u8);
for k in 0..4 {
@ -604,3 +676,25 @@ fn wgsl_float_literals_fit_f32() {
}
assert!(checked > 0);
}
/// A device created with WebGL2's limits (what egui-wgpu asks for on GL, as the desktop app on
/// FreeBSD gets): the compositor declines it instead of building pipelines that fail validation
/// (wgpu's default error handler panics here; release builds would only log it).
#[test]
fn devices_with_webgl2_limits_are_declined() {
hold_gpu_lock();
let instance = wgpu::Instance::new(wgpu::InstanceDescriptor::new_without_display_handle());
let Ok(adapter) = pollster::block_on(instance.request_adapter(&Default::default())) else { return };
let desc = wgpu::DeviceDescriptor { required_limits: wgpu::Limits::downlevel_webgl2_defaults(), ..Default::default() };
let Ok((device, queue)) = pollster::block_on(adapter.request_device(&desc)) else { return };
let err = crate::context::GpuContext::new(&adapter, device, queue).err().expect("declined");
assert!(err.contains("max_storage_buffers_per_shader_stage"), "{err}");
}
/// GPU work inside the video memory guard returns its result when the device has room (#106).
#[test]
fn within_memory_returns_the_work_when_there_is_room() {
let Some(g) = gpu() else { return };
let img = g.within_memory(|| g.ctx.image(64, 32)).expect("room for a 64×32 texture");
assert_eq!((img.width, img.height), (64, 32));
}

View File

@ -14,7 +14,9 @@ use effectcraft_time::{FrameRate, Tick};
use crate::Gpu;
/// A device of the test's own (the test holds [`crate::tests::hold_gpu_lock`]).
fn gpu() -> Option<Gpu> {
crate::tests::hold_gpu_lock();
let g = Gpu::headless();
if g.is_none() {
eprintln!("effectcraft-gpu adv3d tests: no GPU adapter, skipping");
@ -527,3 +529,24 @@ fn gpu_compositor_draws_split_advanced_3d_runs_and_skies() {
assert!(share <= 0.015, "case {case}: {:.3} % of pixels differ from the CPU", share * 100.0);
}
}
/// An adapter whose render targets don't qualify (`R32Float` was not renderable on GL's llvmpipe;
/// the pipelines failed validation, which release builds only log): Advanced 3D renders on the
/// CPU, the GPU compositor's frame matches the CPU's, and no render pipeline is built (a
/// validation error panics under test).
#[test]
fn adapters_that_cannot_rasterise_render_advanced_3d_on_the_cpu() {
crate::tests::hold_gpu_lock();
let Some(mut ctx) = crate::context::GpuContext::headless() else { return };
ctx.adv3d_raster = false;
let g = Gpu::from_context(ctx);
let (p, cid) = scene_project();
let s = scene(&p, cid);
assert!(g.raster_3d(&s).is_none(), "no GPU raster");
let cpu = Renderer::new(&p, &NoFootage, RenderOpts::default()).comp_frame(cid, Tick::ZERO);
let mut r = Renderer::new(&p, &NoFootage, RenderOpts { backend: effectcraft_render::Backend::Gpu, ..Default::default() });
r.accel = Some(&g);
let gimg = r.comp_frame(cid, Tick::ZERO);
let worst = cpu.data.iter().zip(&gimg.data).flat_map(|(a, b)| (0..4).map(move |k| (a[k] - b[k]).abs())).fold(0.0f32, f32::max);
assert!(worst <= 2.0 / 255.0, "GPU compositor with CPU Advanced 3D vs CPU: worst {worst}");
}

View File

@ -19,7 +19,17 @@ use crate::tests::{Pattern, Scene, diff, n, set, tolerance, v3};
pub(crate) fn deferred_gpu() -> Option<(&'static Gpu, std::sync::MutexGuard<'static, ()>)> {
static G: OnceLock<Option<Gpu>> = OnceLock::new();
static LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(());
let g = G.get_or_init(|| pollster::block_on(Gpu::request_deferred()).map_err(|e| eprintln!("deferred gpu tests skipped: {e}")).ok()).as_ref()?;
// Also no other GPU test while it runs (see `hold_gpu_lock`).
crate::tests::hold_gpu_lock();
let g = G
.get_or_init(|| {
let g = pollster::block_on(Gpu::request_deferred()).map_err(|e| eprintln!("deferred gpu tests skipped: {e}")).ok();
if let Some(g) = &g {
eprintln!("deferred gpu tests: adapter {}", g.ctx.name);
}
g
})
.as_ref()?;
Some((g, LOCK.lock().unwrap_or_else(|e| e.into_inner())))
}

View File

@ -228,3 +228,24 @@ fn layer_inputs() {
}
}
}
#[test]
fn arithmetic_preserves_positive_subepsilon_alpha() {
let Some(g) = crate::tests::gpu() else { return };
let spec = effectcraft_effects::find("ec.channel.arithmetic").unwrap();
let size = [1.0, 1.0];
let mut params =
effectcraft_effects::Params { values: spec.params.iter().map(|p| (p.id.to_string(), effectcraft_effects::default_value(p, size))).collect() };
params.values.insert("operator".into(), Value::Enum(3));
for key in ["redValue", "greenValue", "blueValue"] {
params.values.insert(key.into(), n(0.0));
}
params.values.insert("clip".into(), Value::Bool(false));
let ctx = || effectcraft_effects::EffectCtx { params: &params, time: 0.0, layer_size: size, seed: 0, adjustment: false, env: Default::default() };
let img = effectcraft_raster::Image::filled(1, 1, [1.0, 0.0, 0.0, 5e-7]);
let buf = effectcraft_effects::Buf { img, offset: [0.0; 2], scale: 1.0 };
let cpu = (spec.render)(&ctx(), buf.clone());
let out = effectcraft_render::Accelerator::effects(g, &[effectcraft_render::FxStep { spec, ctx: ctx() }], &buf, None).unwrap();
assert_eq!(cpu.img.get(0, 0), [1.0, 0.0, 0.0, 5e-7]);
assert_eq!(out.img.get(0, 0), cpu.img.get(0, 0));
}

View File

@ -202,3 +202,26 @@ fn echo_and_posterize_time() {
}
}
}
#[test]
fn dust_threshold_on_quantized_input() {
let Some(g) = crate::tests::gpu() else { return };
let spec = effectcraft_effects::find("ec.noise.dustscratches").unwrap();
let size = [70.0, 44.0];
let mut params =
effectcraft_effects::Params { values: spec.params.iter().map(|p| (p.id.to_string(), effectcraft_effects::default_value(p, size))).collect() };
params.values.insert("radius".into(), n(3.0));
params.values.insert("threshold".into(), n(10.0));
let ctx = || effectcraft_effects::EffectCtx { params: &params, time: 0.25, layer_size: size, seed: 11, adjustment: false, env: Default::default() };
for seed in 1..32 {
let mut img = crate::tests::pattern(seed, 70, 44);
effectcraft_render::color::quantize(&mut img, 255.0);
let buf = effectcraft_effects::Buf { img, offset: [0.0; 2], scale: 1.0 };
let cpu = (spec.render)(&ctx(), buf.clone());
let gpu = effectcraft_render::Accelerator::effects(g, &[effectcraft_render::FxStep { spec, ctx: ctx() }], &buf, None).unwrap();
for (i, (a, b)) in cpu.img.data.iter().zip(&gpu.img.data).enumerate() {
let d = a.iter().zip(b).map(|(x, y)| (x - y).abs()).fold(0.0_f32, f32::max);
assert!(d < 1e-6, "seed {seed} pixel {i} input {:?} CPU {a:?} GPU {b:?}", buf.img.data[i]);
}
}
}

View File

@ -0,0 +1,161 @@
//! Native constructor containment. Tiny original shaders; no frames or large allocations.
//! Set EFFECTCRAFT_REQUIRE_GPU=1 for acceptance: unavailable adapters/devices then fail
//! instead of taking the explicitly logged headless-CI skip path.
#![cfg(not(target_arch = "wasm32"))]
#![allow(clippy::expect_used)] // Test helpers require successful setup to exercise rejection.
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
#[cfg(target_os = "windows")]
use crate::context::GpuContext;
use crate::context::init_resource;
fn brief(message: impl std::fmt::Display) -> String {
message.to_string().chars().take(1024).collect()
}
fn native_device(desc: wgpu::InstanceDescriptor, required: bool) -> Option<(wgpu::Adapter, wgpu::Device, wgpu::Queue)> {
crate::tests::hold_gpu_lock();
let required = required || std::env::var("EFFECTCRAFT_REQUIRE_GPU").is_ok_and(|value| value == "1");
let instance = wgpu::Instance::new(desc);
let adapter = match pollster::block_on(
instance.request_adapter(&wgpu::RequestAdapterOptions { power_preference: wgpu::PowerPreference::HighPerformance, ..Default::default() }),
) {
Ok(adapter) => adapter,
Err(error) => {
assert!(!required, "required initialization-test adapter unavailable: {}", brief(error));
eprintln!("initialization tests: no adapter, explicitly skipping");
return None;
}
};
let info = adapter.get_info();
eprintln!("initialization tests: adapter {} ({:?})", brief(&info.name), info.backend);
let limits = adapter.limits();
let desc = wgpu::DeviceDescriptor {
label: Some("initialization containment test"),
required_limits: wgpu::Limits {
max_texture_dimension_2d: limits.max_texture_dimension_2d.min(16384),
max_buffer_size: limits.max_buffer_size,
..wgpu::Limits::default()
},
..Default::default()
};
match pollster::block_on(adapter.request_device(&desc)) {
Ok((device, queue)) => Some((adapter, device, queue)),
Err(error) => {
assert!(!required, "required initialization-test device unavailable: {}", brief(error));
eprintln!("initialization tests: no device, explicitly skipping");
None
}
}
}
fn count_host_errors(device: &wgpu::Device) -> Arc<AtomicUsize> {
let calls = Arc::new(AtomicUsize::new(0));
let count = calls.clone();
device.on_uncaptured_error(Arc::new(move |_| {
count.fetch_add(1, Ordering::SeqCst);
}));
calls
}
fn invalid_module(device: &wgpu::Device) -> wgpu::ShaderModule {
device.create_shader_module(wgpu::ShaderModuleDescriptor {
label: Some("original deliberately invalid initialization-test module"),
source: wgpu::ShaderSource::Wgsl("this is deliberately not WGSL".into()),
})
}
fn valid_module(device: &wgpu::Device) -> wgpu::ShaderModule {
init_resource(device, "valid tiny module", || {
device.create_shader_module(wgpu::ShaderModuleDescriptor {
label: Some("original empty initialization-test kernel"),
source: wgpu::ShaderSource::Wgsl("@compute @workgroup_size(1) fn main() {}".into()),
})
})
.expect("a valid tiny module still compiles after rejection")
}
fn pipeline(device: &wgpu::Device, module: &wgpu::ShaderModule, entry: &str) -> wgpu::ComputePipeline {
device.create_compute_pipeline(&wgpu::ComputePipelineDescriptor {
label: Some("initialization-test pipeline"),
layout: None,
module,
entry_point: Some(entry),
compilation_options: Default::default(),
cache: None,
})
}
fn valid_pipeline(device: &wgpu::Device) {
let module = valid_module(device);
init_resource(device, "valid tiny pipeline", || pipeline(device, &module, "main")).expect("a valid pipeline still compiles after rejection");
}
#[test]
fn scoped_module_and_pipeline_errors_preserve_outer_scope_and_host_handler() {
let Some((_, device, _queue)) = native_device(wgpu::InstanceDescriptor::new_without_display_handle().with_env(), false) else { return };
let host = count_host_errors(&device);
let outer = device.push_error_scope(wgpu::ErrorFilter::Validation);
let error = init_resource(&device, "invalid tiny module", || invalid_module(&device)).expect_err("invalid module must return Err");
assert!(error.contains("GPU initialization (invalid tiny module)"), "{}", brief(error));
let module = valid_module(&device);
let error = init_resource(&device, "missing tiny entry", || pipeline(&device, &module, "missing_entry")).expect_err("missing entry point must return Err");
assert!(error.contains("GPU initialization (missing tiny entry)"), "{}", brief(error));
assert_eq!(host.load(Ordering::SeqCst), 0, "contained errors must not reach the host");
valid_pipeline(&device);
// This belongs to our pre-existing caller scope. Its successful pop also proves that
// none of the three nested helper scopes remain on the native thread-local stack.
let _invalid = invalid_module(&device);
assert!(pollster::block_on(outer.pop()).is_some(), "the caller's own scope survives");
assert_eq!(host.load(Ordering::SeqCst), 0);
let _invalid = invalid_module(&device);
assert_eq!(host.load(Ordering::SeqCst), 1, "the original host handler still receives unscoped errors");
}
#[test]
fn initialization_unwind_drains_all_scopes_before_the_next_operation() {
let Some((_, device, _queue)) = native_device(wgpu::InstanceDescriptor::new_without_display_handle().with_env(), false) else { return };
let host = count_host_errors(&device);
let outer = device.push_error_scope(wgpu::ErrorFilter::Validation);
let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
init_resource::<()>(&device, "synthetic native unwind", || panic!("original synthetic initialization failure"))
}));
let error = result.expect("the initialization unwind is contained").expect_err("unwind becomes an error");
assert!(error.contains("synthetic native unwind") && error.contains("original synthetic initialization failure"), "{}", brief(error));
valid_pipeline(&device);
let _invalid = invalid_module(&device);
assert!(pollster::block_on(outer.pop()).is_some(), "all helper scopes must be drained after unwind");
assert_eq!(host.load(Ordering::SeqCst), 0);
let _invalid = invalid_module(&device);
assert_eq!(host.load(Ordering::SeqCst), 1, "unwind must neither replace the host handler nor leak a scope");
}
/// Acceptance for the captured RTX 5090 / FXC failure. Future compiler/shader fixes can
/// legitimately make FXC succeed; this diagnostic is deliberately outside portable CI.
#[test]
#[cfg(target_os = "windows")]
#[ignore = "requires the reproduced Windows DX12/FXC pointwise compiler failure; no adapter skips"]
fn reproduced_fxc_pointwise_failure_returns_error_without_escaping_or_notifying_host() {
let mut desc = wgpu::InstanceDescriptor::new_without_display_handle();
desc.backends = wgpu::Backends::DX12;
desc.backend_options.dx12.shader_compiler = wgpu::Dx12Compiler::Fxc;
let (adapter, device, queue) = native_device(desc, true).expect("required DX12 device");
assert_eq!(adapter.get_info().backend, wgpu::Backend::Dx12);
let host = count_host_errors(&device);
let outer = device.push_error_scope(wgpu::ErrorFilter::Validation);
let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| GpuContext::new(&adapter, device.clone(), queue)));
let error = match result.expect("FXC compiler failure must not escape the constructor") {
Err(error) => error,
Ok(_) => panic!("the platform-specific FXC failure no longer reproduces; reassess this diagnostic"),
};
assert!(error.contains("GPU initialization (pointwise)") && error.contains("FXC D3DCompile"), "{}", brief(error));
assert!(pollster::block_on(outer.pop()).is_none(), "constructor must drain its own scopes and leave the caller's scope clean");
assert_eq!(host.load(Ordering::SeqCst), 0);
valid_pipeline(&device);
let _invalid = invalid_module(&device);
assert_eq!(host.load(Ordering::SeqCst), 1);
}

View File

@ -1 +0,0 @@
{"rustc_fingerprint":9559367331539902066,"outputs":{"4233666815994275565":{"success":true,"status":"","code":0,"stdout":"rustc 1.96.0 (ac68faa20 2026-05-25)\nbinary: rustc\ncommit-hash: ac68faa20c58cbccd01ee7208bf3b6e93a7d7f96\ncommit-date: 2026-05-25\nhost: aarch64-apple-darwin\nrelease: 1.96.0\nLLVM version: 22.1.2\n","stderr":""},"7971740275564407648":{"success":true,"status":"","code":0,"stdout":"___\nlib___.rlib\nlib___.dylib\nlib___.dylib\nlib___.a\nlib___.dylib\n/Users/bt/.rustup/toolchains/stable-aarch64-apple-darwin\noff\npacked\nunpacked\n___\ndebug_assertions\npanic=\"unwind\"\nproc_macro\ntarget_abi=\"\"\ntarget_arch=\"aarch64\"\ntarget_endian=\"little\"\ntarget_env=\"\"\ntarget_family=\"unix\"\ntarget_feature=\"aes\"\ntarget_feature=\"crc\"\ntarget_feature=\"dit\"\ntarget_feature=\"dotprod\"\ntarget_feature=\"dpb\"\ntarget_feature=\"dpb2\"\ntarget_feature=\"fcma\"\ntarget_feature=\"fhm\"\ntarget_feature=\"flagm\"\ntarget_feature=\"fp16\"\ntarget_feature=\"frintts\"\ntarget_feature=\"jsconv\"\ntarget_feature=\"lor\"\ntarget_feature=\"lse\"\ntarget_feature=\"neon\"\ntarget_feature=\"paca\"\ntarget_feature=\"pacg\"\ntarget_feature=\"pan\"\ntarget_feature=\"pmuv3\"\ntarget_feature=\"ras\"\ntarget_feature=\"rcpc\"\ntarget_feature=\"rcpc2\"\ntarget_feature=\"rdm\"\ntarget_feature=\"sb\"\ntarget_feature=\"sha2\"\ntarget_feature=\"sha3\"\ntarget_feature=\"ssbs\"\ntarget_feature=\"vh\"\ntarget_has_atomic=\"128\"\ntarget_has_atomic=\"16\"\ntarget_has_atomic=\"32\"\ntarget_has_atomic=\"64\"\ntarget_has_atomic=\"8\"\ntarget_has_atomic=\"ptr\"\ntarget_os=\"macos\"\ntarget_pointer_width=\"64\"\ntarget_vendor=\"apple\"\nunix\n","stderr":""}},"successes":{}}

View File

@ -1,3 +0,0 @@
Signature: 8a477f597d28d172789f06886806bc55
# This file is a cache directory tag created by cargo.
# For information about cache directory tags see https://bford.info/cachedir/

Some files were not shown because too many files have changed in this diff Show More