filmcraft/docs/control-protocol.md

Control protocol & MCP

Desktop control channel

filmcraft --control 9876 (or FILMCRAFT_CONTROL_PORT) listens on 127.0.0.1:<port> (loopback only). One JSON request per line → one JSON reply per line. The app opts out of macOS App Nap (apps/filmcraft/src/app_nap.rs): a hidden window would otherwise drop the whole process to background priority, and on a busy machine it would stop answering.

{"id": 1, "method": "engine.execute", "params": {"command": "sequence.addEdit", "params": {"seconds": 3}}}
{"id": 1, "ok": true, "result": {"cuts": 2}}

Methods (handlers in crates/ui-egui/src/control.rs):

Method	Params
engine.execute / ui.menu.invoke	{command, params}	any engine or UI command id
engine.commands / ui.menu.list	–	command registry / menu tree
ui.inspect	–	UI state (tool, workspace, dock, timeline view, playback, fps…)
ui.elements	{prefix?}	every on-screen interactive element: id, label, rect
ui.set	{tool, workspace, mode, theme, focused, playbackRes, timeline:{pps,scroll,fit}, program:{…}, source:{…}, clipDialog:{param: value}}	program / source merge fields into the monitor view state (res, paused_res, high_quality, display, zoom, pan, show_rulers, show_guides, lock_guides, snap, guides, compare_ref, …; see monitors.md); clipDialog sets fields of the open Edit / Clip / File dialog (Paste Attributes, Make Subclip, Frame Hold Options, …)
ui.panel.show / ui.panel.close	{panel}
ui.click / ui.move	{id} or {x,y}, button, count, modifiers	synthetic pointer input
ui.drag	{from, to, steps, modifiers}	press–move–release
ui.scroll	`{id	x,y, dx, dy, modifiers}`
ui.key / ui.type	{key} (Cmd+K, Space…) / {text}	keyboard
ui.timeline.hit / ui.timeline.locate	{x,y} / {clip, edge?}	timeline hit-testing
ui.playback	`{action: play	stop
ui.screenshot	{path?, panel?}	PNG of the window or one panel; fails after 10 s when no frame is presented (window hidden, display asleep)
ui.resize, ui.focus, app.quit		ui.focus is the only request that activates the app and takes keyboard focus

Focus. Driving the app never steals the user's keyboard. Started with --control, the app opens without activating itself. UI requests and screenshots that need a rendered frame bring the window forward without making it key (macOS orderFrontRegardless), so the user's typing keeps going to the app they are in. Only an explicit ui.focus activates FilmCraft.

Element ids are stable, e.g. timeline.clip.<id>, timeline.track.V1.lock, tools.Razor, project.item.<id>, effects.item.gaussian_blur, panel.tab.Timeline, program.transport.playback.toggle.

Audio mixing: mixer.* commands (strips "A1", "S1", "Mix" or ids; lanes volume, pan, mute, send.<i>.level, fx.<slot>.<param>): mixer.inspect, mixer.setStrip (volume, pan, mute, solo, record arm, solo safe, mode Off/Read/Latch/Touch/Write, output, input map, channels), mixer.setValue, mixer.touch / mixer.release (a fader gesture; recorded during an automation pass), mixer.recordStart / mixer.recordStop (playback runs them), mixer.addSubmix, mixer.deleteSubmix, mixer.addInsert / removeInsert / setInsert, mixer.addSend / setSend / removeSend, mixer.setKeyframe / deleteKeyframe / moveKeyframe / clearLane, mixer.writeAutomation; clipMixer.set; clip.audioGain {mode: set|adjust|normalizeMax|normalizeAll, db}, clip.audioPeak; effects.setDefaultTransition. UI ids: mixer.<A1|S1|Mix>.<fader|value|pan|panValue|mode|mute|solo| record|soloSafe|output|input|fx.<n>|send.<n>|meter|name> (popup entries below them, e.g. mixer.A1.mode.Touch, mixer.A1.fx.0.studio_reverb), mixer.showEffects, mixer.transport.<cmd>, clipMixer.A1.<fader|pan|mute|solo|keyframe|value>, timeline.track.A1.keyframes[.<lane>|.clip], timeline.track.A1.lane[.kf.<n>], audioGain.<set|adjust|normalizeMax|normalizeAll|ok|cancel|peak>.

Essential Sound: essentialSound.inspect, essentialSound.setType {type: dialogue|music|sfx|ambience}, essentialSound.clearType, essentialSound.set {key, value} or {values: {key: value}} (keys such as repair.noise.on, repair.noise.amount (0–10), repair.humHz, clarity.eqPreset, clarity.enhanceTone, creative.reverbPreset, ducking.against, ducking.reduceDb, ducking.fadeS, pan.value, volume.levelDb, mute, section switches <section>.enabled), essentialSound.applyPreset {preset, type?}, essentialSound.savePreset {name}, essentialSound.deletePreset {name}, essentialSound.autoMatch {target?}, essentialSound.generateDucking; all take clips (default: the selected audio clips). UI ids: essentialSound.tab.<Browse|Edit>, essentialSound.type.<Dialogue|Music|SFX|Ambience>, essentialSound.clearType, essentialSound.preset[.save|.delete|.name|.ok], essentialSound.section.<Name>[.toggle], a row per setting key (essentialSound.repair.noise.on, essentialSound.repair.noise.amount, essentialSound.repair.humHz.<i>), essentialSound.autoMatch, essentialSound.ducking.against.<Type>, essentialSound.generateDucking, essentialSound.volume.on, essentialSound.volume.levelDb, essentialSound.mute, essentialSound.browse.<Type>.<preset>.

Project files, auto-save, crash recovery and preferences commands (file.recover, prefs.set, …) and their automation ids are listed in project-files.md. That file also lists the media management commands and dialog ids: offline media and relinking (media.findMissing, media.relink, media.autoRelink, media.search, media.makeOffline, media.status, linkMedia.*), proxies (media.createProxies, media.attachProxies, media.toggleProxies, proxies.*), ingest (project.ingestSettings) and the Project Manager (file.projectManager, pm.*).

Trim mode (crates/engine/src/trim.rs): trim.selectEditPoint / trim.selectNearest enter trim mode (the Program monitor becomes the Trim Monitor, ids trimMonitor.*); trim.monitor returns what it shows. Dynamic trimming takes an explicit clock (seconds) so it is deterministic: trim.shuttle {direction, slow?, clock} (J/L), trim.tick {clock}, trim.shuttleStop {clock?} (K, one undo step), trim.cancelDynamic (Esc), trim.playAround {clock, loop?} (Space / Shift+K).

Colour (crates/engine/src/color.rs): sequence.colorSettings {workingSpace: rec709|rec2100-pq| rec2100-hlg, wideGamut, autoToneMap}, clip.interpretFootage {items?, colorSpace: auto|<id>}, color.spaces, media.colorInfo {item}; LUTs: lut.import {path, name?}, lut.list, lut.remove {id}, lut.export {lut, path, format?}, lumetri.setInputLut / lumetri.setLook {clip?, lut: lib:<id>|builtin:<id>|"", path?}, lumetri.setSection {clip?, section, on?}, lumetri.applyMatch {clip?, referenceTime|referenceFrame|referenceTimecode, faceDetection?}. Without params the menu entries open dialogs (ids colorDialog.space.<id>, colorDialog.working.<id>, colorDialog.wideGamut, colorDialog.autoToneMap, colorDialog.ok|cancel). Lumetri panel ids: lumetri.input_lut, lumetri.look_lut, lumetri.switch.<section>, lumetri.section.<section>, lumetri.match.*; HDR scopes: scopes.hdrWaveform. file.exportMedia {sdr: true} exports an HDR sequence as tone-mapped SDR.

Keyboard shortcuts (crates/engine/src/shortcuts.rs): shortcuts.list {query?, panel?}, shortcuts.get, shortcuts.set {command, keys, panel?, add?, keepConflicts?}, shortcuts.clear, shortcuts.undo / shortcuts.redo, shortcuts.conflicts {platform?}, shortcuts.forKey {key}, shortcuts.resolve {keys, panel?}, shortcuts.presets, shortcuts.loadPreset / savePreset / deletePreset {name}, shortcuts.export / import {path}, shortcuts.audit. Keys use Cmd (⌘ / Ctrl), Ctrl (macOS ⌃), Alt, Shift. The dialog (Edit ▸ Keyboard Shortcuts…, ⌥⌘K) uses ids shortcuts.* (shortcuts.key.K, shortcuts.cell.<command>, shortcuts.ok, …).

MCP

filmcraft-cli mcp serves MCP on stdio: headless (in-process session; --demo / --project p.fcproj) or --bridge 127.0.0.1:9876 to drive the running app. Tools: command_list, command_run, project_inspect, sequence_inspect, media_import, render_frame, ui_inspect, ui_elements, ui_click, ui_drag, ui_key, ui_type, ui_screenshot, ui_control. .mcp.json registers both.

Time is in ticks (254 016 000 000 per second); commands also accept seconds, frame or timecode.