soundcraft/docs/control-protocol.md

Control protocol

SoundCraft can be driven completely by other programs: scripts, test harnesses and AI agents. Every menu item, button, mouse gesture and keystroke maps to a command (or to synthetic input), and every command is reachable from:

Surface	How
Desktop app control channel	soundcraft --control <port> (0 = pick a free port; printed as SOUNDCRAFT_CONTROL_PORT=…)
CLI against a running app	soundcraft-cli app --port <port> <method or command id> [JSON]
Headless CLI	soundcraft-cli run --demo --cmd 'mix.volume={"track":"Kick","db":-6}' --bounce out.wav
MCP (headless)	soundcraft-cli mcp [--demo]
MCP (bridged to the app)	soundcraft-cli mcp --connect <port>

Wire format

Newline-delimited JSON over TCP on 127.0.0.1. One request per line, one reply per line:

{"id": 1, "method": "engine.execute", "params": {"command": "track.new", "params": {"count": 2, "format": "Stereo"}}}
{"id": 1, "ok": true, "result": {"tracks": [12, 13]}}

Errors: {"id": 1, "ok": false, "error": "bad parameters for mix.volume: no track named Nope"}.

Methods

Method	Params	What it does
engine.execute (alias command)	{command, params}	Run a command programmatically. Never opens a dialog; empty params use defaults or the current selection.
engine.commands	{filter?}	Every command: id, label, menu path, shortcut, params doc, enabled + reason. Includes UI-layer commands (window.*).
engine.parity	{}	Parity report against the incumbent's menu catalog.
session.inspect	`{detail?: "summary"	"full"}`
ui.inspect	{}	UI state (window, panels, dialogs), window size, playing/position, audio device, and edit_layout (timeline rect and every track row's rect, for clicking).
ui.set	any UiState fields	e.g. {"window": "Mix", "narrow_mix": true}.
ui.menu.list	{}	Every catalog menu path and the command that implements it (or null).
ui.menu.invoke	{path} or {id}	Like clicking the menu item: may open its dialog (e.g. "Track > New...").
ui.click	{x, y, button?, count?, cmd?, shift?, alt?, ctrl?}	Synthetic click (window points).
ui.drag	{x, y, to_x, to_y, steps?}	Synthetic drag with the primary button.
ui.move	{x, y}	Move the pointer.
ui.key	{key, cmd?, shift?, alt?, ctrl?}	Key press (Space, Enter, A, F7, =…).
ui.text	{text}	Type text into the focused field.
ui.screenshot	{path?}	PNG of the window (base64 in png_base64 when no path).
app.quit	{}	Quit.

Positions and references

• Tracks: id (number) or name ("Kick").
• Clips: id (from session.inspect).
• Positions: samples (96000), seconds ({"seconds": 2.5}), or a time string in any timebase, optionally prefixed: "bars_beats:5|1|000", "min_secs:0:12.500", "timecode:00:00:10:00", "samples:48000".
• Ranges: start + end (or start + length); omitted = the current edit selection.

Examples

P=$(soundcraft --demo --control 0 | sed -n 's/SOUNDCRAFT_CONTROL_PORT=//p')   # or read stderr
soundcraft-cli app --port $P mix.solo '{"track":"Bass"}'
soundcraft-cli app --port $P transport.play
soundcraft-cli app --port $P ui.screenshot '{"path":"/tmp/shot.png"}'
soundcraft-cli app --port $P ui.menu.invoke '{"path":"Track > New..."}'

Offscreen (no window): cargo run -p soundcraft-ui-egui --example ui_shot -- out.png script.jsonl, where each script line is a request ({"method": …, "params": …}), {"shot": "path.png"} or {"steps": n}.