photocraft/docs/control-protocol.md

Control protocol

The desktop app (photocraft --control <port>) listens on 127.0.0.1:<port> (loopback only). Each request is one JSON line:

{"id": 1, "method": "ui.inspect", "params": {}}

Each reply is one JSON line with the same id:

{"id": 1, "ok": true, "result": { ... }}
{"id": 2, "ok": false, "error": "unknown tool `foo`"}

The transport is apps/photocraft/src/control_server.rs, and the handlers are in crates/ui-egui/src/control.rs. The MCP server (photocraft-cli mcp --bridge 127.0.0.1:<port>, crate photocraft-automation) wraps this same protocol. See MCP bridge below.

Methods

• engine.execute {command, params}: run any engine or UI command by id. Engine commands run directly with their default params and never open a dialog. Use ui.menu.invoke for menu-click behaviour, which opens a command's dialog when no params are given
• engine.commands: list commands with enablement
• ui.inspect: full UI state (tool, panels, views, dialogs, windows, menu tree, window size). view holds the View/Window/Type preferences: screen_mode, extras, show and snap_to flags, flip_horizontal, arrange (Window › Arrange layout), pixel aspect, font preview size, language options
• ui.set {tool?, panels?, dockTabs?, maskTarget?, zoom?, center?, fit?, theme?, brushSize?, brushSection?, brushTab?}: change UI state (brushSection indexes the Brush Settings sections, brushTab 0 = Brush Settings, 1 = Brushes)
• ui.menu.invoke {id} / ui.menu.list: activate a menu item by id; list the menu tree
• ui.dialog.open {kind, fields?} (kinds newDocument, about, layerStyle {effect?}, colorPicker {target: foreground|background}, command {command}) / ui.dialog.set {dialog, field, value} / ui.dialog.confirm {dialog} / ui.dialog.cancel {dialog}
• ui.window.open {document?} / ui.window.close {window}: extra document windows
• ui.pointer {events: [{kind: down|move|up, x, y, pressure?}], modifiers?}: drive the active tool in document coordinates
• ui.key {key, command?, shift?, alt?, ctrl?} (flags may also be grouped under modifiers): press and release a key, e.g. {"key": "ArrowLeft", "shift": true}
• ui.type {text}: type text (goes to the focused widget, or to the canvas while the Type tool is editing)
• ui.resize {width, height}: resize the main window
• ui.screenshot {path?, focus?}: capture the main window (PNG). Raises the window first (default) because occluded macOS windows stop rendering
• ui.focus: bring the main window to the front
• app.open {path} / app.save {path}: file I/O through the configured services
• app.quit

Engine commands

engine.execute runs any command by id. engine.commands (or the engine command command.list) lists them all, with labels, menu paths, shortcuts, a parameter description, and whether each is currently enabled. Examples:

Command	Params
file.new	{"width":1920,"height":1080,"mode":"rgb","depth":8,"background":"white"}
layer.new.layer	{"name":"Ink"}
layer.select	{"layer":id,"mode":"replace|toggle|range|add"}: ⌘-click = toggle, ⇧-click = range; document.inspect reports selectedLayers and a per-layer selected flag
channel.target / channel.setVisible	Channels panel target and eyes ("composite", colour name, alpha index or name, "quickMask"). document.inspect (and channel.list) report channels: composite/colour/alpha rows with visibility, channel options, quickMask, target. Pixel commands without a target param follow the targeted channel
layer.setProps	{"layer":id?,"name":…,"visible":…,"opacity":0..1,"blend":"Multiply"}
layer.newAdjustmentLayer.hueSaturation	{"hue":30,"saturation":10}
paint.stroke	{"points":[[x,y,pressure],…],"size":20,"color":"#ff0000"}
select.rect	{"x":0,"y":0,"width":100,"height":50,"mode":"add","ellipse":false}
document.inspect	{}: layer tree, history, selection bounds
document.pixel	{"x":10,"y":10}: composite RGBA

UI-level commands (file.open, file.save, view.zoomIn, window.theme.pro, edit.search, …) are also accepted by engine.execute and ui.menu.invoke.

Preferences

Preferences (Edit › Preferences, grouped like Photoshop's dialog sections) live in the engine, so the same commands work in the app, the CLI and headless MCP. Paths are dotted camelCase keys: <section>.<key>, with sections general, interface, workspace, tools, historyLog, fileHandling, export, performance, scratchDisks, cursors, transparencyAndGamut, unitsAndRulers, guidesGridAndSlices, plugIns, type, enhancedControls, rawDefaults, integrations, plus shortcuts.<command id>, menus (hidden, colors.<id>), toolbar (hidden, order) and colorSettings (Edit › Color Settings).

Command	Params
prefs.get	{"path":"performance.historyStates"}; no path returns everything
prefs.set	{"path":"cursors.painting","value":"precise"} or {"values":{"unitsAndRulers.rulers":"cm","guidesGridAndSlices.gridColor":"#ff8800"}}. Values are validated (choices, ranges, #rrggbb colours, shortcut syntax); a batch applies all or nothing. A section path takes an object and merges it key by key
prefs.reset	{"path":"performance"} (a section or key); no path resets everything
edit.preferences.<section>	the section's values; in the app (no params) it opens the Preferences dialog on that section
edit.keyboardShortcuts	{"set":{"edit.fill":"Cmd+Shift+F"},"reset":true|["id",…],"removeConflicts":true,"filter":"blur","list":false}: returns overrides, matching commands and conflicts. A shortcut moved to another command is removed from its old owner unless removeConflicts is false. "" removes a shortcut, null restores the default. In the app it opens Keyboard Shortcuts and Menus
edit.menus / edit.toolbar	{"hide":["edit.fade"],"show":[…],"color":{"edit.fill":"red"},"reset":false} / {"hidden":["Sponge"],"order":[…]}
edit.colorSettings	{"workingRgb":"display-p3","workingCmyk":"coated-cmyk","workingGray":"sgray","policyRgb":"preserve|convert|off",…,"askOnMismatch":true,"intent":"perceptual","bpc":true}; honoured when opening files (file.openAs, the app's File › Open) and by Image › Mode and "working" profile specs. color.profileMismatch {"action":"preserve|convert|discard|assignWorking"} answers the mismatch prompt

Values the app honours live: history states (every open document), effect-cache budget, interface theme, canvas colour and border, checkerboard size and colours (CPU and GPU canvas), gamut warning colour and opacity, guide/grid/smart-guide colours and styles, grid spacing and subdivisions, ruler units (rulers, Info panel, Image Size default unit), image interpolation (Image Size default), painting and other cursors, zoom with scroll wheel, Use Shift Key for Tool Switch, keyboard shortcuts, hidden and coloured menu items, autosave interval and crash recovery, and the history log text file. GPU on/off and the GPU tile size apply at the next launch.

The desktop app stores them in preferences.json in the platform config directory (macOS ~/Library/Application Support/Photocraft, Windows %APPDATA%\Photocraft, Linux $XDG_CONFIG_HOME/photocraft; override with PHOTOCRAFT_CONFIG_DIR); autosaves go to its Recovery folder. The web build keeps them in localStorage.

Snapping

With View › Snap on, tool gestures snap to the View › Snap To targets (guides and grid while they are shown, layer edges and centres, document bounds and centre, selection edges) within 8 screen pixels: Move tool drags (the moved layers' bounds), marquee, crop, shape, type-box and pen points, Free Transform handles and drags inside the box, and guides. Holding Ctrl disables snapping for the drag. With View › Show › Smart Guides on, the Move tool also snaps to other layers and draws magenta alignment lines. ui.pointer drives the same code, so agents get identical results.

MCP bridge

photocraft-automation provides an MCP server built on the official Rust SDK (rmcp). It runs in one of two modes:

• Headless (photocraft-cli mcp): an in-process photocraft_engine::Session. There is no window.
• Bridge (photocraft-cli mcp --bridge 127.0.0.1:7878): every tool is forwarded to a running photocraft --control 7878 over this protocol, so agents see and drive the live app.

The bridge keeps one TCP connection open. It reconnects once if a request fails, and it skips reply lines whose id doesn't match the request (for example, stale replies to requests that timed out). It only accepts loopback addresses, because the app only listens on loopback.

How each MCP tool maps onto control methods in bridge mode:

MCP tool	Control method
command_run {id, params}	engine.execute {command: id, params}
command_list {filter?, enabled_only?}	engine.commands (filtered by the MCP server)
doc_new {…}	engine.execute {command: "file.new", params}
doc_inspect	engine.execute {command: "document.inspect"}
doc_open {path}	app.open {path}
doc_save {path} / doc_export {path}	app.save {path}
doc_render_preview {max_side?}	ui.screenshot {path: <temp>}, returned as PNG image content
session_list, ui_inspect	ui.inspect
ui_screenshot {max_side?}	ui.screenshot, returned as PNG image content
ui_pointer {events, modifiers?}	ui.pointer
ui_menu_invoke {id}	ui.menu.invoke
ui_set {fields}	ui.set
control_call {method, params}	any method, passed through unchanged

doc_select and doc_close work only in headless mode. The ui_* tools and control_call work only in bridge mode; in headless mode they return a tool error that explains how to start bridge mode.

Security note: the control port has no authentication. Any local process can drive the app. Only enable --control when you need it. A token handshake is planned (architecture §12).

Headless server

photocraft-cli serve keeps one headless engine session (no window, no GPU) and answers the same JSON-lines envelope on stdio, or on 127.0.0.1:<port> with --port <port> (loopback only; each connection shares the session). It is the fastest way for a script or agent to make many edits: no MCP framing, no app start-up per command. Implementation: crates/automation/src/rpc.rs.

Method	Params
engine.execute	{command, params?}: any engine command
engine.commands	{filter?}: registry with params docs and enablement
session.list	open documents and the active index
doc.open / doc.new	{path} / file.new params
doc.save	{path?, format?, quality?, index?} (.pcraft native, else export by extension)
doc.inspect	{index?}: same JSON as document.inspect
doc.render	{index?, maxSide? (1024; 0 = full), path?}: PNG to path, else {mime, base64}
doc.select / doc.close	{index} / {index?}
batch	{steps: [{command, params?} | {method, params?}], stopOnError? (true)} → {completed, failed, results}
methods	the list above

printf '%s\n' \
  '{"id":1,"method":"doc.open","params":{"path":"/abs/in.jpg"}}' \
  '{"id":2,"method":"batch","params":{"steps":[{"command":"image.adjustments.invert"},{"command":"filter.blur.gaussianBlur","params":{"radius":3}}]}}' \
  '{"id":3,"method":"doc.save","params":{"path":"/abs/out.png"}}' | photocraft-cli serve

The MCP server has the same batching as the command_batch tool ({steps:[{id, params}], stop_on_error}), in headless and bridge mode.