filmcraft/docs/control-protocol.md

Control protocol & MCP

Desktop control channel

filmcraft --control 9876 (or FILMCRAFT_CONTROL_PORT) listens on 127.0.0.1:<port> (loopback only). One JSON request per line → one JSON reply per line. The app opts out of macOS App Nap (apps/filmcraft/src/app_nap.rs): a hidden window would otherwise drop the whole process to background priority, and on a busy machine it would stop answering.

{"id": 1, "method": "engine.execute", "params": {"command": "sequence.addEdit", "params": {"seconds": 3}}}
{"id": 1, "ok": true, "result": {"cuts": 2}}

Methods (handlers in crates/ui-egui/src/control.rs):

Method	Params
engine.execute / ui.menu.invoke	{command, params}	any engine or UI command id
engine.commands / ui.menu.list	–	command registry / menu tree
ui.inspect	–	UI state (tool, workspace, dock, timeline view, playback, fps…)
ui.elements	{prefix?}	every on-screen interactive element: id, label, rect
ui.set	{tool, workspace, mode, theme, focused, playbackRes, timeline:{pps,scroll,fit}}
ui.panel.show / ui.panel.close	{panel}
ui.click / ui.move	{id} or {x,y}, button, count, modifiers	synthetic pointer input
ui.drag	{from, to, steps, modifiers}	press–move–release
ui.scroll	`{id	x,y, dx, dy, modifiers}`
ui.key / ui.type	{key} (Cmd+K, Space…) / {text}	keyboard
ui.timeline.hit / ui.timeline.locate	{x,y} / {clip, edge?}	timeline hit-testing
ui.playback	`{action: play	stop
ui.screenshot	{path?, panel?}	PNG of the window or one panel; fails after 10 s when no frame is presented (window hidden, display asleep)
ui.resize, ui.focus, app.quit

Element ids are stable, e.g. timeline.clip.<id>, timeline.track.V1.lock, tools.Razor, project.item.<id>, effects.item.gaussian_blur, panel.tab.Timeline, program.transport.playback.toggle.

Audio mixing: mixer.* commands (strips "A1", "S1", "Mix" or ids; lanes volume, pan, mute, send.<i>.level, fx.<slot>.<param>): mixer.inspect, mixer.setStrip (volume, pan, mute, solo, record arm, solo safe, mode Off/Read/Latch/Touch/Write, output, input map, channels), mixer.setValue, mixer.touch / mixer.release (a fader gesture; recorded during an automation pass), mixer.recordStart / mixer.recordStop (playback runs them), mixer.addSubmix, mixer.deleteSubmix, mixer.addInsert / removeInsert / setInsert, mixer.addSend / setSend / removeSend, mixer.setKeyframe / deleteKeyframe / moveKeyframe / clearLane, mixer.writeAutomation; clipMixer.set; clip.audioGain {mode: set|adjust|normalizeMax|normalizeAll, db}, clip.audioPeak; effects.setDefaultTransition. UI ids: mixer.<A1|S1|Mix>.<fader|value|pan|panValue|mode|mute|solo| record|soloSafe|output|input|fx.<n>|send.<n>|meter|name> (popup entries below them, e.g. mixer.A1.mode.Touch, mixer.A1.fx.0.studio_reverb), mixer.showEffects, mixer.transport.<cmd>, clipMixer.A1.<fader|pan|mute|solo|keyframe|value>, timeline.track.A1.keyframes[.<lane>|.clip], timeline.track.A1.lane[.kf.<n>], audioGain.<set|adjust|normalizeMax|normalizeAll|ok|cancel|peak>.

Essential Sound: essentialSound.inspect, essentialSound.setType {type: dialogue|music|sfx|ambience}, essentialSound.clearType, essentialSound.set {key, value} or {values: {key: value}} (keys such as repair.noise.on, repair.noise.amount (0–10), repair.humHz, clarity.eqPreset, clarity.enhanceTone, creative.reverbPreset, ducking.against, ducking.reduceDb, ducking.fadeS, pan.value, volume.levelDb, mute, section switches <section>.enabled), essentialSound.applyPreset {preset, type?}, essentialSound.savePreset {name}, essentialSound.deletePreset {name}, essentialSound.autoMatch {target?}, essentialSound.generateDucking; all take clips (default: the selected audio clips). UI ids: essentialSound.tab.<Browse|Edit>, essentialSound.type.<Dialogue|Music|SFX|Ambience>, essentialSound.clearType, essentialSound.preset[.save|.delete|.name|.ok], essentialSound.section.<Name>[.toggle], a row per setting key (essentialSound.repair.noise.on, essentialSound.repair.noise.amount, essentialSound.repair.humHz.<i>), essentialSound.autoMatch, essentialSound.ducking.against.<Type>, essentialSound.generateDucking, essentialSound.volume.on, essentialSound.volume.levelDb, essentialSound.mute, essentialSound.browse.<Type>.<preset>.

Project files, auto-save, crash recovery and preferences commands (file.recover, prefs.set, …) and their automation ids are listed in project-files.md. That file also lists the media management commands and dialog ids: offline media and relinking (media.findMissing, media.relink, media.autoRelink, media.search, media.makeOffline, media.status, linkMedia.*), proxies (media.createProxies, media.attachProxies, media.toggleProxies, proxies.*), ingest (project.ingestSettings) and the Project Manager (file.projectManager, pm.*).

Trim mode (crates/engine/src/trim.rs): trim.selectEditPoint / trim.selectNearest enter trim mode (the Program monitor becomes the Trim Monitor, ids trimMonitor.*); trim.monitor returns what it shows. Dynamic trimming takes an explicit clock (seconds) so it is deterministic: trim.shuttle {direction, slow?, clock} (J/L), trim.tick {clock}, trim.shuttleStop {clock?} (K, one undo step), trim.cancelDynamic (Esc), trim.playAround {clock, loop?} (Space / Shift+K).

Colour (crates/engine/src/color.rs): sequence.colorSettings {workingSpace: rec709|rec2100-pq| rec2100-hlg, wideGamut, autoToneMap}, clip.interpretFootage {items?, colorSpace: auto|<id>}, color.spaces, media.colorInfo {item}; LUTs: lut.import {path, name?}, lut.list, lut.remove {id}, lut.export {lut, path, format?}, lumetri.setInputLut / lumetri.setLook {clip?, lut: lib:<id>|builtin:<id>|"", path?}, lumetri.setSection {clip?, section, on?}, lumetri.applyMatch {clip?, referenceTime|referenceFrame|referenceTimecode, faceDetection?}. Without params the menu entries open dialogs (ids colorDialog.space.<id>, colorDialog.working.<id>, colorDialog.wideGamut, colorDialog.autoToneMap, colorDialog.ok|cancel). Lumetri panel ids: lumetri.input_lut, lumetri.look_lut, lumetri.switch.<section>, lumetri.section.<section>, lumetri.match.*; HDR scopes: scopes.hdrWaveform. file.exportMedia {sdr: true} exports an HDR sequence as tone-mapped SDR.

Keyboard shortcuts (crates/engine/src/shortcuts.rs): shortcuts.list {query?, panel?}, shortcuts.get, shortcuts.set {command, keys, panel?, add?, keepConflicts?}, shortcuts.clear, shortcuts.undo / shortcuts.redo, shortcuts.conflicts {platform?}, shortcuts.forKey {key}, shortcuts.resolve {keys, panel?}, shortcuts.presets, shortcuts.loadPreset / savePreset / deletePreset {name}, shortcuts.export / import {path}, shortcuts.audit. Keys use Cmd (⌘ / Ctrl), Ctrl (macOS ⌃), Alt, Shift. The dialog (Edit ▸ Keyboard Shortcuts…, ⌥⌘K) uses ids shortcuts.* (shortcuts.key.K, shortcuts.cell.<command>, shortcuts.ok, …).

MCP

filmcraft-cli mcp serves MCP on stdio: headless (in-process session; --demo / --project p.fcproj) or --bridge 127.0.0.1:9876 to drive the running app. Tools: command_list, command_run, project_inspect, sequence_inspect, media_import, render_frame, ui_inspect, ui_elements, ui_click, ui_drag, ui_key, ui_type, ui_screenshot, ui_control. .mcp.json registers both.

Time is in ticks (254 016 000 000 per second); commands also accept seconds, frame or timecode.