158 lines
12 KiB
Markdown
158 lines
12 KiB
Markdown
# Control protocol
|
||
|
||
The desktop app (`photocraft --control <port>`) listens on `127.0.0.1:<port>` (loopback only). Each request is one JSON line:
|
||
|
||
```json
|
||
{"id": 1, "method": "ui.inspect", "params": {}}
|
||
```
|
||
|
||
Each reply is one JSON line with the same `id`:
|
||
|
||
```json
|
||
{"id": 1, "ok": true, "result": { ... }}
|
||
{"id": 2, "ok": false, "error": "unknown tool `foo`"}
|
||
```
|
||
|
||
The transport is `apps/photocraft/src/control_server.rs`, and the handlers are in `crates/ui-egui/src/control.rs`. The MCP server (`photocraft-cli mcp --bridge 127.0.0.1:<port>`, crate `photocraft-automation`) wraps this same protocol. See [MCP bridge](#mcp-bridge) below.
|
||
|
||
## Methods
|
||
|
||
- `engine.execute {command, params}`: run any engine or UI command by id. Engine commands run directly with their default params and never open a dialog. Use `ui.menu.invoke` for menu-click behaviour, which opens a command's dialog when no params are given
|
||
- `engine.commands`: list commands with enablement
|
||
- `ui.inspect`: full UI state (tool, panels, views, dialogs, windows, menu tree, window size). `view` holds the View/Window/Type preferences: `screen_mode`, `extras`, `show` and `snap_to` flags, `flip_horizontal`, `arrange` (Window › Arrange layout), pixel aspect, font preview size, language options
|
||
- `ui.set {tool?, panels?, dockTabs?, maskTarget?, zoom?, center?, fit?, theme?, brushSize?, brushSection?, brushTab?}`: change UI state (`brushSection` indexes the Brush Settings sections, `brushTab` 0 = Brush Settings, 1 = Brushes)
|
||
- `ui.menu.invoke {id}` / `ui.menu.list`: activate a menu item by id; list the menu tree
|
||
- `ui.dialog.open {kind, fields?}` (kinds `newDocument`, `about`, `layerStyle {effect?}`, `colorPicker {target: foreground|background}`, `command {command}`) / `ui.dialog.set {dialog, field, value}` / `ui.dialog.confirm {dialog}` / `ui.dialog.cancel {dialog}`
|
||
- `ui.window.open {document?}` / `ui.window.close {window}`: extra document windows
|
||
- `ui.pointer {events: [{kind: down|move|up, x, y, pressure?}], modifiers?}`: drive the active tool in document coordinates
|
||
- `ui.key {key, command?, shift?, alt?, ctrl?}` (flags may also be grouped under `modifiers`): press and release a key, e.g. `{"key": "ArrowLeft", "shift": true}`
|
||
- `ui.type {text}`: type text (goes to the focused widget, or to the canvas while the Type tool is editing)
|
||
- `ui.resize {width, height}`: resize the main window
|
||
- `ui.screenshot {path?, focus?}`: capture the main window (PNG). Raises the window first (default)
|
||
because occluded macOS windows stop rendering
|
||
- `ui.focus`: bring the main window to the front
|
||
- `app.open {path}` / `app.save {path}`: file I/O through the configured services
|
||
- `app.quit`
|
||
|
||
## Engine commands
|
||
|
||
`engine.execute` runs any command by id. `engine.commands` (or the engine command `command.list`) lists them all, with labels, menu paths, shortcuts, a parameter description, and whether each is currently enabled. Examples:
|
||
|
||
| Command | Params |
|
||
|---|---|
|
||
| `file.new` | `{"width":1920,"height":1080,"mode":"rgb","depth":8,"background":"white"}` |
|
||
| `layer.new.layer` | `{"name":"Ink"}` |
|
||
| `layer.select` | `{"layer":id,"mode":"replace\|toggle\|range\|add"}`: ⌘-click = toggle, ⇧-click = range; `document.inspect` reports `selectedLayers` and a per-layer `selected` flag |
|
||
| `channel.target` / `channel.setVisible` | Channels panel target and eyes (`"composite"`, colour name, alpha index or name, `"quickMask"`). `document.inspect` (and `channel.list`) report `channels`: composite/colour/alpha rows with visibility, channel options, `quickMask`, `target`. Pixel commands without a `target` param follow the targeted channel |
|
||
| `layer.setProps` | `{"layer":id?,"name":…,"visible":…,"opacity":0..1,"blend":"Multiply"}` |
|
||
| `layer.newAdjustmentLayer.hueSaturation` | `{"hue":30,"saturation":10}` |
|
||
| `paint.stroke` | `{"points":[[x,y,pressure],…],"size":20,"color":"#ff0000"}` |
|
||
| `select.rect` | `{"x":0,"y":0,"width":100,"height":50,"mode":"add","ellipse":false}` |
|
||
| `document.inspect` | `{}`: layer tree, history, selection bounds |
|
||
| `document.pixel` | `{"x":10,"y":10}`: composite RGBA |
|
||
|
||
UI-level commands (`file.open`, `file.save`, `view.zoomIn`, `window.theme.pro`, `edit.search`, …) are also accepted by `engine.execute` and `ui.menu.invoke`.
|
||
|
||
## Preferences
|
||
|
||
Preferences (Edit › Preferences, grouped like Photoshop's dialog sections) live in the engine, so
|
||
the same commands work in the app, the CLI and headless MCP. Paths are dotted camelCase keys:
|
||
`<section>.<key>`, with sections `general`, `interface`, `workspace`, `tools`, `historyLog`,
|
||
`fileHandling`, `export`, `performance`, `scratchDisks`, `cursors`, `transparencyAndGamut`,
|
||
`unitsAndRulers`, `guidesGridAndSlices`, `plugIns`, `type`, `enhancedControls`, `rawDefaults`,
|
||
`integrations`, plus `shortcuts.<command id>`, `menus` (`hidden`, `colors.<id>`), `toolbar`
|
||
(`hidden`, `order`) and `colorSettings` (Edit › Color Settings).
|
||
|
||
| Command | Params |
|
||
|---|---|
|
||
| `prefs.get` | `{"path":"performance.historyStates"}`; no path returns everything |
|
||
| `prefs.set` | `{"path":"cursors.painting","value":"precise"}` or `{"values":{"unitsAndRulers.rulers":"cm","guidesGridAndSlices.gridColor":"#ff8800"}}`. Values are validated (choices, ranges, `#rrggbb` colours, shortcut syntax); a batch applies all or nothing. A section path takes an object and merges it key by key |
|
||
| `prefs.reset` | `{"path":"performance"}` (a section or key); no path resets everything |
|
||
| `edit.preferences.<section>` | the section's values; in the app (no params) it opens the Preferences dialog on that section |
|
||
| `edit.keyboardShortcuts` | `{"set":{"edit.fill":"Cmd+Shift+F"},"reset":true\|["id",…],"removeConflicts":true,"filter":"blur","list":false}`: returns overrides, matching commands and conflicts. A shortcut moved to another command is removed from its old owner unless `removeConflicts` is false. `""` removes a shortcut, `null` restores the default. In the app it opens Keyboard Shortcuts and Menus |
|
||
| `edit.menus` / `edit.toolbar` | `{"hide":["edit.fade"],"show":[…],"color":{"edit.fill":"red"},"reset":false}` / `{"hidden":["Sponge"],"order":[…]}` |
|
||
| `edit.colorSettings` | `{"workingRgb":"display-p3","workingCmyk":"coated-cmyk","workingGray":"sgray","policyRgb":"preserve\|convert\|off",…,"askOnMismatch":true,"intent":"perceptual","bpc":true}`; honoured when opening files (`file.openAs`, the app's File › Open) and by Image › Mode and "working" profile specs. `color.profileMismatch {"action":"preserve\|convert\|discard\|assignWorking"}` answers the mismatch prompt |
|
||
|
||
Values the app honours live: history states (every open document), effect-cache budget, interface
|
||
theme, canvas colour and border, checkerboard size and colours (CPU and GPU canvas), gamut warning
|
||
colour and opacity, guide/grid/smart-guide colours and styles, grid spacing and subdivisions, ruler
|
||
units (rulers, Info panel, Image Size default unit), image interpolation (Image Size default),
|
||
painting and other cursors, zoom with scroll wheel, Use Shift Key for Tool Switch, keyboard
|
||
shortcuts, hidden and coloured menu items, autosave interval and crash recovery, and the history
|
||
log text file. GPU on/off and the GPU tile size apply at the next launch.
|
||
|
||
The desktop app stores them in `preferences.json` in the platform config directory (macOS
|
||
`~/Library/Application Support/Photocraft`, Windows `%APPDATA%\Photocraft`, Linux
|
||
`$XDG_CONFIG_HOME/photocraft`; override with `PHOTOCRAFT_CONFIG_DIR`); autosaves go to its
|
||
`Recovery` folder. The web build keeps them in `localStorage`.
|
||
|
||
## Snapping
|
||
|
||
With View › Snap on, tool gestures snap to the View › Snap To targets (guides and grid while they
|
||
are shown, layer edges and centres, document bounds and centre, selection edges) within 8 screen
|
||
pixels: Move tool drags (the moved layers' bounds), marquee, crop, shape, type-box and pen points,
|
||
Free Transform handles and drags inside the box, and guides. Holding Ctrl disables snapping for
|
||
the drag. With View › Show › Smart Guides on, the Move tool also snaps to other layers and draws
|
||
magenta alignment lines. `ui.pointer` drives the same code, so agents get identical results.
|
||
|
||
## MCP bridge
|
||
|
||
`photocraft-automation` provides an MCP server built on the official Rust SDK (`rmcp`). It runs in one of two modes:
|
||
|
||
- **Headless** (`photocraft-cli mcp`): an in-process `photocraft_engine::Session`. There is no window.
|
||
- **Bridge** (`photocraft-cli mcp --bridge 127.0.0.1:7878`): every tool is forwarded to a running `photocraft --control 7878` over this protocol, so agents see and drive the live app.
|
||
|
||
The bridge keeps one TCP connection open. It reconnects once if a request fails, and it skips reply lines whose `id` doesn't match the request (for example, stale replies to requests that timed out). It only accepts loopback addresses, because the app only listens on loopback.
|
||
|
||
How each MCP tool maps onto control methods in bridge mode:
|
||
|
||
| MCP tool | Control method |
|
||
|---|---|
|
||
| `command_run {id, params}` | `engine.execute {command: id, params}` |
|
||
| `command_list {filter?, enabled_only?}` | `engine.commands` (filtered by the MCP server) |
|
||
| `doc_new {…}` | `engine.execute {command: "file.new", params}` |
|
||
| `doc_inspect` | `engine.execute {command: "document.inspect"}` |
|
||
| `doc_open {path}` | `app.open {path}` |
|
||
| `doc_save {path}` / `doc_export {path}` | `app.save {path}` |
|
||
| `doc_render_preview {max_side?}` | `ui.screenshot {path: <temp>}`, returned as PNG image content |
|
||
| `session_list`, `ui_inspect` | `ui.inspect` |
|
||
| `ui_screenshot {max_side?}` | `ui.screenshot`, returned as PNG image content |
|
||
| `ui_pointer {events, modifiers?}` | `ui.pointer` |
|
||
| `ui_menu_invoke {id}` | `ui.menu.invoke` |
|
||
| `ui_set {fields}` | `ui.set` |
|
||
| `control_call {method, params}` | any method, passed through unchanged |
|
||
|
||
`doc_select` and `doc_close` work only in headless mode. The `ui_*` tools and `control_call` work only in bridge mode; in headless mode they return a tool error that explains how to start bridge mode.
|
||
|
||
**Security note:** the control port has no authentication. Any local process can drive the app. Only enable `--control` when you need it. A token handshake is planned (architecture §12).
|
||
|
||
## Headless server
|
||
|
||
`photocraft-cli serve` keeps one headless engine session (no window, no GPU) and answers the same
|
||
JSON-lines envelope on stdio, or on `127.0.0.1:<port>` with `--port <port>` (loopback only; each
|
||
connection shares the session). It is the fastest way for a script or agent to make many edits:
|
||
no MCP framing, no app start-up per command. Implementation: `crates/automation/src/rpc.rs`.
|
||
|
||
| Method | Params |
|
||
|---|---|
|
||
| `engine.execute` | `{command, params?}`: any engine command |
|
||
| `engine.commands` | `{filter?}`: registry with params docs and enablement |
|
||
| `session.list` | open documents and the active index |
|
||
| `doc.open` / `doc.new` | `{path}` / `file.new` params |
|
||
| `doc.save` | `{path?, format?, quality?, index?}` (`.pcraft` native, else export by extension) |
|
||
| `doc.inspect` | `{index?}`: same JSON as `document.inspect` |
|
||
| `doc.render` | `{index?, maxSide? (1024; 0 = full), path?}`: PNG to `path`, else `{mime, base64}` |
|
||
| `doc.select` / `doc.close` | `{index}` / `{index?}` |
|
||
| `batch` | `{steps: [{command, params?} \| {method, params?}], stopOnError? (true)}` → `{completed, failed, results}` |
|
||
| `methods` | the list above |
|
||
|
||
```sh
|
||
printf '%s\n' \
|
||
'{"id":1,"method":"doc.open","params":{"path":"/abs/in.jpg"}}' \
|
||
'{"id":2,"method":"batch","params":{"steps":[{"command":"image.adjustments.invert"},{"command":"filter.blur.gaussianBlur","params":{"radius":3}}]}}' \
|
||
'{"id":3,"method":"doc.save","params":{"path":"/abs/out.png"}}' | photocraft-cli serve
|
||
```
|
||
|
||
The MCP server has the same batching as the `command_batch` tool (`{steps:[{id, params}], stop_on_error}`),
|
||
in headless and bridge mode.
|