diff --git a/AGENTS.md b/AGENTS.md index ab9245dd..672a21bb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -83,9 +83,10 @@ If you find an asset that breaks these rules, stop and remove it from the reposi ## 3. Agent control (automation and MCP) - **Everything is reachable headlessly.** Every user-facing feature must be callable without the GUI: an engine API, then a tool in `crates/automation` (`printcraft-cli run` and MCP use the same table). A feature is not done until its tool exists and has an end-to-end test (`crates/automation/README.md`, "Adding a tool"). +- **The UI control channel is opt-in too.** It runs only when the app is launched with `--control `. It binds to loopback, uses a fresh random token per launch, and writes it to an owner-only file. Never enable it by default, widen the bind address, or log the token. - **The MCP server is opt-in.** Never make the app, the CLI or an installer start it, register it with an agent, or listen on a port by default. It runs only when a user (or an agent the user configured) launches `printcraft-cli mcp`. Any future network transport must be off by default, bound to loopback and authenticated with a token. - **Safe by default.** Tools must never discard unsaved work silently, must write atomically, must honour `--root`, and must return errors an agent can act on (say what was wrong and what is allowed). -- **Use it to test.** Agents should drive the engine through `printcraft-cli run --script` or the MCP tools, and look at `page_render` output, to check their own changes. +- **Use it to test.** Agents should drive the engine through `printcraft-cli run --script` or the MCP tools, and look at `page_render` output, to check their own changes. For UI changes, launch the app with `--control` and use `printcraft-cli ui` (`inspect`, `click`, `screenshot`) to check the real interface. ## 4. Everything else diff --git a/CLAUDE.md b/CLAUDE.md index da692058..1c16d931 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -27,7 +27,7 @@ PrintCraft is a clean-room, open-source, Rust-native PDF application targeting A - `cargo run -p printcraft -- ` opens the desktop app. - `cargo run -p printcraft-cli -- run --script steps.json --root DIR` drives the engine headlessly through the automation tools (`printcraft-cli tools` lists them). Use it, together with `page_render`, to check engine changes. `printcraft-cli mcp` is the opt-in MCP server (AGENTS.md §3). - `cargo xtask demo-pdf` builds `dist/demo/printcraft-showcase.pdf` (needs Chrome) for visual checks. -- For UI work, **look at the result**: screenshot the window by id (`plan/acrobat/tools/winlist.swift` + `screencapture -l`) until the UI control channel (M3.9) exists, then use `ui.screenshot`. Compare against `plan/acrobat/02-ui-ux.md`. +- For UI work, **look at the result**. Either launch `printcraft --control FILE doc.pdf` and use `printcraft-cli ui --control FILE screenshot --out x.png` (plus `inspect`, `click`, `key`, `type`, `command`, `set`), or take a headless shot with `cargo run -p printcraft-ui-egui --example shot`. Compare against `plan/acrobat/02-ui-ux.md`. Control-channel tests use kittest (`crates/ui-egui/tests/control.rs`). - Parallel agents: use a separate `CARGO_TARGET_DIR` per agent and separate git worktrees. ## Current bootstrap debt (tracked in STATUS.md) diff --git a/Cargo.lock b/Cargo.lock index c942a94d..40237476 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2885,6 +2885,7 @@ dependencies = [ "eframe", "egui", "printcraft-ui-egui", + "serde_json", ] [[package]] @@ -2903,6 +2904,7 @@ dependencies = [ name = "printcraft-cli" version = "0.1.0" dependencies = [ + "base64", "printcraft-automation", "printcraft-engine", "printcraft-render", @@ -2983,13 +2985,16 @@ dependencies = [ name = "printcraft-ui-egui" version = "0.1.0" dependencies = [ + "base64", "eframe", "egui", "egui_extras", "egui_kittest", + "getrandom 0.4.3", "image", "js-sys", "log", + "png 0.18.1", "printcraft-cos", "printcraft-engine", "printcraft-render", diff --git a/README.md b/README.md index 916f7eca..a51f12b7 100644 --- a/README.md +++ b/README.md @@ -226,7 +226,14 @@ Every engine feature is reachable without the GUI, through one table of JSON-Sch - **The Rust API** (`printcraft_automation::Automation::call`), for embedding. -Edits stay in memory, undoable, until `doc_save`. Saving to the same file appends an incremental update, so the original bytes are preserved, and the write is atomic. Unsaved changes are never discarded silently. A UI control channel, so agents can drive and screenshot the running app, is next (M3.9). +Edits stay in memory, undoable, until `doc_save`. Saving to the same file appends an incremental update, so the original bytes are preserved, and the write is atomic. Unsaved changes are never discarded silently. **Driving the app itself.** Start the desktop app with `printcraft --control /tmp/pc.json` and an agent can see and operate the real interface: the widget tree with labels and positions (from the accessibility tree), clicks, typing, keys, commands, view options and screenshots. This is also off by default. It listens only on loopback, and every connection must present the random token written to that file, which only you can read. + + ```sh + printcraft-cli ui --control /tmp/pc.json inspect query=rotate # find widgets + printcraft-cli ui --control /tmp/pc.json click label="Organize pages" + printcraft-cli ui --control /tmp/pc.json key key=K modifiers='["command"]' + printcraft-cli ui --control /tmp/pc.json screenshot --out window.png + ``` --- diff --git a/apps/printcraft-cli/Cargo.toml b/apps/printcraft-cli/Cargo.toml index 96111880..bf103f25 100644 --- a/apps/printcraft-cli/Cargo.toml +++ b/apps/printcraft-cli/Cargo.toml @@ -17,6 +17,7 @@ printcraft-render = { workspace = true } serde_json = { workspace = true } printcraft-engine = { workspace = true } printcraft-automation = { workspace = true } +base64 = "0.22" [features] default = ["mcp"] diff --git a/apps/printcraft-cli/src/main.rs b/apps/printcraft-cli/src/main.rs index 2eba1df1..74b6659d 100644 --- a/apps/printcraft-cli/src/main.rs +++ b/apps/printcraft-cli/src/main.rs @@ -14,6 +14,8 @@ //! printcraft-cli run [key=value …] [--root DIR] [--out image.png] //! printcraft-cli run --script steps.json [--root DIR] [{"tool": "doc_open", "args": {…}}, …] //! printcraft-cli mcp [--root DIR] MCP server on stdin/stdout (opt-in) +//! printcraft-cli ui --control FILE [key=value …] [--out shot.png] +//! drive a running app started with --control FILE //! ``` //! //! `run` and `mcp` drive the same tool table (`printcraft-automation`). In `run`, values parse as @@ -37,26 +39,29 @@ use printcraft_render::{PageRenderer, RenderConfig, RenderRequest, RequestKind, fn main() -> ExitCode { let args: Vec = std::env::args().skip(1).collect(); - let result = match args.first().map(String::as_str) { - Some("info") => info(&args[1..]), - Some("render") => render(&args[1..]), - Some("text") => text(&args[1..]), - Some("edit") => edit(&args[1..]), - Some("combine") => combine(&args[1..]), - Some("extract") => extract(&args[1..]), - Some("split") => split(&args[1..]), - Some("check") => check(&args[1..]), - Some("check-one") => check_one(&args[1..]), - Some("tools") => tools(), - Some("run") => run(&args[1..]), - #[cfg(feature = "mcp")] - Some("mcp") => mcp(&args[1..]), - Some("--version") => { - println!("printcraft-cli {}", env!("CARGO_PKG_VERSION")); - Ok(()) - } - _ => Err("usage: printcraft-cli … (see source header for options)".into()), - }; + let result = + match args.first().map(String::as_str) { + Some("info") => info(&args[1..]), + Some("render") => render(&args[1..]), + Some("text") => text(&args[1..]), + Some("edit") => edit(&args[1..]), + Some("combine") => combine(&args[1..]), + Some("extract") => extract(&args[1..]), + Some("split") => split(&args[1..]), + Some("check") => check(&args[1..]), + Some("check-one") => check_one(&args[1..]), + Some("tools") => tools(), + Some("run") => run(&args[1..]), + Some("ui") => ui(&args[1..]), + #[cfg(feature = "mcp")] + Some("mcp") => mcp(&args[1..]), + Some("--version") => { + println!("printcraft-cli {}", env!("CARGO_PKG_VERSION")); + Ok(()) + } + _ => Err("usage: printcraft-cli … (see source header for options)" + .into()), + }; match result { Ok(()) => ExitCode::SUCCESS, Err(e) => { @@ -421,3 +426,47 @@ fn mcp(args: &[String]) -> Result<(), String> { eprintln!("printcraft-cli: MCP server on stdio (protocol {}); close stdin to stop", printcraft_automation::mcp::PROTOCOL_VERSIONS[0]); server.serve(std::io::stdin().lock(), std::io::stdout().lock()).map_err(|e| e.to_string()) } + +// ---- UI control channel client ----------------------------------------------------------------- + +/// One request to a running app's control channel (`printcraft --control FILE`). +fn ui(args: &[String]) -> Result<(), String> { + use std::io::{BufRead, BufReader, Write as _}; + let file = flag(args, "--control").ok_or("ui: missing --control FILE (start the app with `printcraft --control FILE`)")?; + let info: serde_json::Value = + serde_json::from_str(&std::fs::read_to_string(file).map_err(|e| format!("{file}: {e}"))?).map_err(|e| format!("{file}: {e}"))?; + let port = info["port"].as_u64().ok_or(format!("{file}: no port"))?; + let token = info["token"].as_str().ok_or(format!("{file}: no token"))?; + let pos = positional(args); + let method = *pos.first().ok_or("ui: missing method (state, inspect, click, type, key, command, commands, set, open, screenshot)")?; + let method = if method.starts_with("ui.") { method.to_string() } else { format!("ui.{method}") }; + let mut params = serde_json::Map::new(); + for kv in pos.iter().skip(1) { + let (k, v) = kv.split_once('=').ok_or(format!("ui: expected key=value, got {kv:?}"))?; + params.insert(k.to_string(), serde_json::from_str(v).unwrap_or_else(|_| serde_json::Value::String(v.to_string()))); + } + let stream = std::net::TcpStream::connect(("127.0.0.1", port as u16)) + .map_err(|e| format!("can't reach the app on port {port}: {e} (is it still running?)"))?; + stream.set_read_timeout(Some(Duration::from_secs(40))).map_err(|e| e.to_string())?; + let mut write = stream.try_clone().map_err(|e| e.to_string())?; + let mut lines = BufReader::new(stream).lines(); + let mut rpc = |id: u64, method: &str, params: serde_json::Value| -> Result { + writeln!(write, "{}", serde_json::json!({ "jsonrpc": "2.0", "id": id, "method": method, "params": params })).map_err(|e| e.to_string())?; + let line = lines.next().ok_or("the app closed the connection")?.map_err(|e| e.to_string())?; + let reply: serde_json::Value = serde_json::from_str(&line).map_err(|e| e.to_string())?; + match reply.get("error") { + Some(e) => Err(e["message"].as_str().unwrap_or("error").to_string()), + None => Ok(reply["result"].clone()), + } + }; + rpc(1, "auth", serde_json::json!({ "token": token }))?; + let mut result = rpc(2, &method, serde_json::Value::Object(params))?; + if let (Some(out), Some(data)) = (flag(args, "--out"), result.get("png_base64").and_then(|d| d.as_str())) { + use base64::Engine as _; + let png = base64::engine::general_purpose::STANDARD.decode(data).map_err(|e| e.to_string())?; + std::fs::write(out, png).map_err(|e| format!("{out}: {e}"))?; + result["png_base64"] = serde_json::json!(format!("written to {out}")); + } + println!("{}", serde_json::to_string_pretty(&result).unwrap_or_default()); + Ok(()) +} diff --git a/apps/printcraft/Cargo.toml b/apps/printcraft/Cargo.toml index 4e126586..5e9762aa 100644 --- a/apps/printcraft/Cargo.toml +++ b/apps/printcraft/Cargo.toml @@ -12,3 +12,4 @@ workspace = true printcraft-ui-egui = { workspace = true } eframe = { workspace = true, features = ["default_fonts", "wgpu", "accesskit", "wayland", "x11", "persistence"] } egui = { workspace = true } +serde_json = { workspace = true } diff --git a/apps/printcraft/src/main.rs b/apps/printcraft/src/main.rs index 189d1309..8c063a57 100644 --- a/apps/printcraft/src/main.rs +++ b/apps/printcraft/src/main.rs @@ -6,12 +6,17 @@ //! `--page N --zoom 150 --layout continuous|two-up|single --panel comments|bookmarks|pages|fields|layers|attachments|none //! --theme light|dark --mode all|read|edit|convert|sign --tool --left open|closed //! --organize on --fields on --dialog properties|shortcuts|about --palette --home on` +//! +//! `--control ` enables the UI control channel (off by default): the app listens on a random +//! loopback port and writes `{"port", "token", "pid"}` to `` (owner-only permissions). +//! Agents then drive it with `printcraft-cli ui --control …`. use printcraft_ui_egui::PrintCraftApp; fn main() -> eframe::Result { let mut files = Vec::new(); let mut options: Vec<(String, String)> = Vec::new(); + let mut control_file: Option = None; let mut args = std::env::args().skip(1); while let Some(a) = args.next() { match a.as_str() { @@ -19,6 +24,7 @@ fn main() -> eframe::Result { println!("printcraft {}", env!("CARGO_PKG_VERSION")); return Ok(()); } + "--control" => control_file = args.next(), flag if flag.starts_with("--") => { let value = args.next().unwrap_or_default(); options.push((flag.trim_start_matches("--").to_string(), value)); @@ -45,6 +51,13 @@ fn main() -> eframe::Result { app.restore(&json); } app.integrated_titlebar = integrated; + if let Some(file) = &control_file { + let client = app.attach_control(&cc.egui_ctx); + match printcraft_ui_egui::control::serve(client).and_then(|ep| write_control_file(file, ep.port, &ep.token).map(|()| ep.port)) { + Ok(port) => eprintln!("printcraft: UI control channel on 127.0.0.1:{port} (connection details in {file})"), + Err(e) => eprintln!("printcraft: --control {file}: {e}"), + } + } // Autosave unsaved changes; offer to recover documents a crashed session left behind. if let Some(dir) = printcraft_ui_egui::RecoveryStore::default_dir() { app.enable_recovery(printcraft_ui_egui::RecoveryStore::new(dir)); @@ -61,3 +74,23 @@ fn main() -> eframe::Result { }), ) } + +/// Write the control endpoint so that only the current user can read the token. +fn write_control_file(path: &str, port: u16, token: &str) -> std::io::Result<()> { + let json = serde_json::json!({ "port": port, "token": token, "pid": std::process::id() }).to_string(); + let mut opts = std::fs::OpenOptions::new(); + opts.write(true).create(true).truncate(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + opts.mode(0o600); + } + use std::io::Write; + let mut f = opts.open(path)?; + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + f.set_permissions(std::fs::Permissions::from_mode(0o600))?; + } + f.write_all(json.as_bytes()) +} diff --git a/crates/ui-egui/Cargo.toml b/crates/ui-egui/Cargo.toml index b9a840a0..fc3ddc6b 100644 --- a/crates/ui-egui/Cargo.toml +++ b/crates/ui-egui/Cargo.toml @@ -19,6 +19,12 @@ serde_json = { workspace = true } log = { workspace = true } rfd = { workspace = true } +png = "0.18" +base64 = "0.22" + +[target.'cfg(not(target_arch = "wasm32"))'.dependencies] +# Control-channel tokens. +getrandom = "0.4" [target.'cfg(target_arch = "wasm32")'.dependencies] wasm-bindgen-futures = "0.4" diff --git a/crates/ui-egui/src/control.rs b/crates/ui-egui/src/control.rs new file mode 100644 index 00000000..326c9216 --- /dev/null +++ b/crates/ui-egui/src/control.rs @@ -0,0 +1,585 @@ +//! The UI control channel (M3.9): lets an agent see and drive the running app. +//! +//! **Opt-in only.** Nothing here runs unless the app is started with `--control ` (or a +//! test attaches a [`ControlClient`]). The transport ([`serve`]) listens on loopback only, on a +//! random port, and every connection must first present the random token written to ``. +//! +//! How it works: +//! - [`ControlPlugin`], an egui plugin, keeps a copy of the AccessKit tree from every frame's +//! output (the widget tree with ids, roles, labels, rects and states) and injects queued input +//! events (AccessKit click actions, pointer events, keys, text) at the start of later frames. +//! - Requests reach the app through a channel; `PrintCraftApp` answers them at the start of a +//! frame, so they see and change exactly what the user would. +//! +//! Methods (JSON in, JSON out): +//! - `ui.state`: open documents, active document, page, zoom, mode, panels, dialog, notice. +//! - `ui.inspect {query?, role?, limit?}`: widgets in tree order with `id` (a string), `role`, `label`, +//! `value`, `rect` (points), `enabled`, `toggled`, `selected`, `clickable`, `depth`. +//! - `ui.click {id}` | `{label}` | `{x, y}`: click a widget (by its AccessKit action) or a point. +//! - `ui.type {text}`, `ui.key {key, modifiers?}`: keyboard input to the focused widget / app. +//! - `ui.command {id}`: run a registry command (as the menu would). `ui.commands` lists them. +//! - `ui.set {key, value}`: the view options of the command line (`--page`, `--zoom`, …). +//! - `ui.open {path}`: open a file. +//! - `ui.screenshot {region?}`: PNG of the window (base64), optionally cropped to a rect. + +use std::collections::{HashMap, VecDeque}; +use std::sync::mpsc::{Receiver, Sender, channel}; +use std::sync::{Arc, Mutex}; + +use egui::accesskit::{self, Action, NodeId}; +use serde_json::{Value, json}; + +pub type Reply = Result; + +/// One request from a client, answered on `reply`. +pub struct ControlRequest { + pub method: String, + pub params: Value, + pub reply: Sender, +} + +/// Sends requests to the app (from the transport thread or a test). +#[derive(Clone)] +pub struct ControlClient { + tx: Sender, + ctx: egui::Context, +} + +impl ControlClient { + /// Queue a request; the reply arrives on the returned receiver once the app has handled it. + pub fn send(&self, method: &str, params: Value) -> Receiver { + let (reply, rx) = channel(); + let req = ControlRequest { method: method.into(), params, reply }; + if let Err(e) = self.tx.send(req) { + let _ = e.0.reply.send(Err("the app has closed".into())); + } + self.ctx.request_repaint(); + rx + } +} + +/// State shared between the plugin (frame hooks) and the app. +#[derive(Default)] +struct Shared { + nodes: HashMap, + root: Option, + focus: Option, + /// Event batches to inject, one batch per frame. + inject: VecDeque>, +} + +/// The egui plugin half of the control channel. +pub struct ControlPlugin { + shared: Arc>, +} + +impl egui::Plugin for ControlPlugin { + fn debug_name(&self) -> &'static str { + "printcraft-control" + } + + fn input_hook(&mut self, ctx: &egui::Context, input: &mut egui::RawInput) { + let Ok(mut s) = self.shared.lock() else { return }; + if let Some(batch) = s.inject.pop_front() { + input.events.extend(batch); + } + if !s.inject.is_empty() { + ctx.request_repaint(); + } + } + + fn output_hook(&mut self, _ctx: &egui::Context, output: &mut egui::FullOutput) { + let Some(update) = &output.platform_output.accesskit_update else { return }; + let Ok(mut s) = self.shared.lock() else { return }; + s.nodes = update.nodes.iter().cloned().collect(); + s.root = update.tree.as_ref().map(|t| t.root); + s.focus = Some(update.focus); + } +} + +enum Pending { + /// Answer after `frames` more frames (the injected input has been handled by then). + Frames { + frames: u32, + reply: Sender, + value: Value, + }, + Screenshot { + tag: u64, + region: Option, + reply: Sender, + }, +} + +/// The app half: the request queue and requests waiting for later frames. +pub struct Control { + rx: Receiver, + shared: Arc>, + pending: Vec, + next_tag: u64, +} + +/// Install the control channel on `ctx`: returns the app half and a client. +pub fn attach(ctx: &egui::Context) -> (Control, ControlClient) { + let shared = Arc::new(Mutex::new(Shared::default())); + ctx.add_plugin(ControlPlugin { shared: shared.clone() }); + ctx.enable_accesskit(); + let (tx, rx) = channel(); + (Control { rx, shared, pending: Vec::new(), next_tag: 1 }, ControlClient { tx, ctx: ctx.clone() }) +} + +/// What a request needs from the app (implemented by `PrintCraftApp`). +pub(crate) trait Host { + fn state(&self) -> Value; + fn command(&mut self, id: &str) -> Reply; + fn commands(&self) -> Value; + fn set(&mut self, key: &str, value: &str) -> Reply; + fn open(&mut self, path: &str) -> Reply; +} + +impl Control { + /// Handle new requests and finish waiting ones. Call at the start of every frame. + pub(crate) fn tick(&mut self, ctx: &egui::Context, host: &mut impl Host) { + self.finish_pending(ctx); + while let Ok(req) = self.rx.try_recv() { + match self.handle(ctx, host, &req.method, &req.params) { + Handled::Now(r) => { + let _ = req.reply.send(r); + } + Handled::AfterFrames(frames, value) => self.pending.push(Pending::Frames { frames, reply: req.reply, value }), + Handled::Screenshot(region) => { + let tag = self.next_tag; + self.next_tag += 1; + ctx.send_viewport_cmd(egui::ViewportCommand::Screenshot(egui::UserData::new(tag))); + self.pending.push(Pending::Screenshot { tag, region, reply: req.reply }); + } + } + } + if !self.pending.is_empty() { + ctx.request_repaint(); + } + } + + fn finish_pending(&mut self, ctx: &egui::Context) { + let shots: Vec<(u64, Arc)> = ctx.input(|i| { + i.raw + .events + .iter() + .filter_map(|e| match e { + egui::Event::Screenshot { user_data, image, .. } => { + user_data.data.as_ref().and_then(|d| d.downcast_ref::()).map(|tag| (*tag, image.clone())) + } + _ => None, + }) + .collect() + }); + let ppp = ctx.pixels_per_point(); + let mut keep = Vec::new(); + for p in self.pending.drain(..) { + match p { + Pending::Frames { frames: 0, reply, value } => { + let _ = reply.send(Ok(value)); + } + Pending::Frames { frames, reply, value } => keep.push(Pending::Frames { frames: frames - 1, reply, value }), + Pending::Screenshot { tag, region, reply } => match shots.iter().find(|(t, _)| *t == tag) { + Some((_, image)) => { + let _ = reply.send(screenshot_png(image, region, ppp)); + } + None => keep.push(Pending::Screenshot { tag, region, reply }), + }, + } + } + self.pending = keep; + } + + fn inject(&self, batches: Vec>) -> u32 { + let n = batches.len() as u32; + if let Ok(mut s) = self.shared.lock() { + s.inject.extend(batches); + } + // The last batch is handled during the frame it is injected into; answer one frame later. + n + 1 + } + + fn handle(&mut self, ctx: &egui::Context, host: &mut impl Host, method: &str, p: &Value) -> Handled { + let str_param = |k: &str| p.get(k).and_then(Value::as_str).ok_or_else(|| format!("{method}: missing string parameter {k}")); + let r: Result = (|| match method { + "ui.state" => Ok(Handled::Now(Ok(host.state()))), + "ui.commands" => Ok(Handled::Now(Ok(host.commands()))), + "ui.command" => Ok(Handled::Now(host.command(str_param("id")?))), + "ui.set" => { + let value = match p.get("value") { + Some(Value::String(s)) => s.clone(), + Some(v) => v.to_string(), + None => return Err("ui.set: missing parameter value".into()), + }; + Ok(Handled::Now(host.set(str_param("key")?, &value))) + } + "ui.open" => Ok(Handled::Now(host.open(str_param("path")?))), + "ui.inspect" => Ok(Handled::Now(Ok(self.inspect(p)))), + "ui.click" => self.click(p), + "ui.type" => { + let text = str_param("text")?.to_string(); + Ok(Handled::AfterFrames(self.inject(vec![vec![egui::Event::Text(text)]]), json!({ "typed": true }))) + } + "ui.key" => { + let name = str_param("key")?; + let key = egui::Key::from_name(name) + .ok_or_else(|| format!("ui.key: unknown key {name:?} (egui key names: A, Enter, Escape, ArrowDown, F5, …)"))?; + let modifiers = modifiers(p.get("modifiers"))?; + let ev = |pressed| egui::Event::Key { key, physical_key: None, pressed, repeat: false, modifiers }; + Ok(Handled::AfterFrames(self.inject(vec![vec![ev(true)], vec![ev(false)]]), json!({ "key": name }))) + } + "ui.screenshot" => { + let region = match p.get("region") { + None | Some(Value::Null) => None, + Some(r) => Some(rect_param(r)?), + }; + ctx.request_repaint(); + Ok(Handled::Screenshot(region)) + } + other => Err(format!( + "unknown method {other:?} (ui.state, ui.inspect, ui.click, ui.type, ui.key, ui.command, ui.commands, ui.set, ui.open, ui.screenshot)" + )), + })(); + r.unwrap_or_else(|e| Handled::Now(Err(e))) + } + + fn inspect(&self, p: &Value) -> Value { + let query = p.get("query").and_then(Value::as_str).map(str::to_lowercase); + let role = p.get("role").and_then(Value::as_str).map(str::to_lowercase); + let limit = p.get("limit").and_then(Value::as_u64).unwrap_or(500) as usize; + let Ok(s) = self.shared.lock() else { return json!({ "widgets": [] }) }; + let mut out = Vec::new(); + let mut total = 0usize; + let mut stack: Vec<(NodeId, usize)> = s.root.map(|r| vec![(r, 0)]).unwrap_or_default(); + while let Some((id, depth)) = stack.pop() { + let Some(node) = s.nodes.get(&id) else { continue }; + for c in node.children().iter().rev() { + stack.push((*c, depth + 1)); + } + let w = widget(id, node, depth, s.focus == Some(id)); + let text = + format!("{} {} {}", w["label"].as_str().unwrap_or(""), w["value"].as_str().unwrap_or(""), w["description"].as_str().unwrap_or("")) + .to_lowercase(); + let role_ok = role.as_ref().is_none_or(|r| w["role"].as_str().is_some_and(|x| x.to_lowercase() == *r)); + let query_ok = query.as_ref().is_none_or(|q| text.contains(q.as_str())); + if role_ok && query_ok && id != s.root.unwrap_or(id) { + total += 1; + if out.len() < limit { + out.push(w); + } + } + } + json!({ "widgets": out, "count": total, "truncated": total > out.len() }) + } + + fn click(&mut self, p: &Value) -> Result { + if let (Some(x), Some(y)) = (p.get("x").and_then(Value::as_f64), p.get("y").and_then(Value::as_f64)) { + let pos = egui::pos2(x as f32, y as f32); + let button = + |pressed| egui::Event::PointerButton { pos, button: egui::PointerButton::Primary, pressed, modifiers: egui::Modifiers::NONE }; + let frames = self.inject(vec![vec![egui::Event::PointerMoved(pos)], vec![button(true)], vec![button(false)]]); + return Ok(Handled::AfterFrames(frames, json!({ "clicked": [x, y] }))); + } + let s = self.shared.lock().map_err(|_| "control state poisoned")?; + let id = match (p.get("id"), p.get("label").and_then(Value::as_str)) { + (Some(id), _) => { + let id: u64 = + id.as_u64().or_else(|| id.as_str().and_then(|s| s.parse().ok())).ok_or("ui.click: id must be a widget id from ui.inspect")?; + let id = NodeId(id); + if !s.nodes.contains_key(&id) { + return Err(format!("ui.click: no widget with id {} on screen (run ui.inspect again)", id.0)); + } + id + } + (None, Some(label)) => { + let want = label.to_lowercase(); + let matches: Vec = s + .nodes + .iter() + .filter(|(_, n)| n.supports_action(Action::Click) && !n.is_disabled() && n.label().is_some_and(|l| l.to_lowercase() == want)) + .map(|(id, _)| *id) + .collect(); + match matches.as_slice() { + [one] => *one, + [] => return Err(format!("ui.click: no enabled clickable widget labelled {label:?} (try ui.inspect with query)")), + many => return Err(format!("ui.click: {} widgets are labelled {label:?}; click one by id", many.len())), + } + } + (None, None) => return Err("ui.click: pass id, label, or x and y".into()), + }; + let node = &s.nodes[&id]; + if node.is_disabled() { + return Err(format!("ui.click: widget {} ({}) is disabled", id.0, node.label().unwrap_or("unlabelled"))); + } + let label = node.label().map(str::to_owned); + let action = accesskit::ActionRequest { action: Action::Click, target_tree: accesskit::TreeId::ROOT, target_node: id, data: None }; + drop(s); + let frames = self.inject(vec![vec![egui::Event::AccessKitActionRequest(action)]]); + Ok(Handled::AfterFrames(frames, json!({ "clicked": id.0.to_string(), "label": label }))) + } +} + +enum Handled { + Now(Reply), + AfterFrames(u32, Value), + Screenshot(Option), +} + +fn widget(id: NodeId, n: &accesskit::Node, depth: usize, focused: bool) -> Value { + let rect = n.bounds().map(|b| [b.x0, b.y0, b.x1, b.y1]); + let mut w = json!({ + // A string: 64-bit ids don't survive JSON parsers that use doubles. + "id": id.0.to_string(), + "role": format!("{:?}", n.role()), + "depth": depth, + "enabled": !n.is_disabled(), + "clickable": n.supports_action(Action::Click), + }); + let obj = w.as_object_mut().expect("object"); + if let Some(l) = n.label() { + obj.insert("label".into(), json!(l)); + } + if let Some(v) = n.value() { + obj.insert("value".into(), json!(v)); + } + if let Some(d) = n.description() { + obj.insert("description".into(), json!(d)); + } + if let Some(r) = rect { + obj.insert("rect".into(), json!(r)); + } + if let Some(t) = n.toggled() { + obj.insert("toggled".into(), json!(format!("{t:?}").to_lowercase())); + } + if let Some(s) = n.is_selected() { + obj.insert("selected".into(), json!(s)); + } + if focused { + obj.insert("focused".into(), json!(true)); + } + w +} + +fn modifiers(v: Option<&Value>) -> Result { + let mut m = egui::Modifiers::NONE; + for name in v.and_then(Value::as_array).into_iter().flatten() { + match name.as_str().unwrap_or("") { + "command" | "cmd" => m.command = true, + "shift" => m.shift = true, + "alt" | "option" => m.alt = true, + "ctrl" | "control" => m.ctrl = true, + "mac_cmd" => m.mac_cmd = true, + other => return Err(format!("ui.key: unknown modifier {other:?} (command, shift, alt, ctrl)")), + } + } + // "command" is ⌘ on macOS and Ctrl elsewhere, as egui expects. + if m.command { + if cfg!(target_os = "macos") { + m.mac_cmd = true; + } else { + m.ctrl = true; + } + } + Ok(m) +} + +fn rect_param(v: &Value) -> Result { + let a: Vec = v.as_array().map(|a| a.iter().filter_map(Value::as_f64).map(|f| f as f32).collect()).unwrap_or_default(); + match a.as_slice() { + [x0, y0, x1, y1] if x1 > x0 && y1 > y0 => Ok(egui::Rect::from_min_max(egui::pos2(*x0, *y0), egui::pos2(*x1, *y1))), + _ => Err("region must be [x0, y0, x1, y1] in points".into()), + } +} + +/// Encode (a region of) a screenshot as PNG; the reply carries base64 data and the size. +fn screenshot_png(image: &egui::ColorImage, region: Option, ppp: f32) -> Reply { + let [w, h] = image.size; + let (x0, y0, x1, y1) = match region { + Some(r) => { + let px = |v: f32, max: usize| ((v * ppp).round().max(0.0) as usize).min(max); + (px(r.min.x, w), px(r.min.y, h), px(r.max.x, w), px(r.max.y, h)) + } + None => (0, 0, w, h), + }; + if x1 <= x0 || y1 <= y0 { + return Err("the region is outside the window".into()); + } + let mut rgba = Vec::with_capacity((x1 - x0) * (y1 - y0) * 4); + for y in y0..y1 { + for px in &image.pixels[y * w + x0..y * w + x1] { + rgba.extend_from_slice(&px.to_srgba_unmultiplied()); + } + } + let (cw, ch) = ((x1 - x0) as u32, (y1 - y0) as u32); + let mut out = Vec::new(); + let mut enc = png::Encoder::new(&mut out, cw, ch); + enc.set_color(png::ColorType::Rgba); + enc.set_depth(png::BitDepth::Eight); + let mut writer = enc.write_header().map_err(|e| e.to_string())?; + writer.write_image_data(&rgba).map_err(|e| e.to_string())?; + writer.finish().map_err(|e| e.to_string())?; + use base64::Engine as _; + Ok(json!({ "png_base64": base64::engine::general_purpose::STANDARD.encode(out), "width": cw, "height": ch, "pixels_per_point": ppp })) +} + +impl Host for crate::PrintCraftApp { + fn state(&self) -> Value { + let active = self.active.and_then(|i| self.views.get(i)); + let docs: Vec = self + .views + .iter() + .enumerate() + .filter_map(|(i, v)| { + let d = self.session.get(v.id)?; + Some(json!({ "tab": i, "doc": v.id.0, "name": d.name, "path": d.path, "pages": d.info.pages.len(), "dirty": d.dirty, "active": self.active == Some(i) })) + }) + .collect(); + json!({ + "documents": docs, + "active": active.map(|v| json!({ + "doc": v.id.0, + "page": v.current + 1, + "zoom_percent": (v.zoom * 100.0).round(), + "fit": format!("{:?}", v.fit), + "layout": format!("{:?}", v.layout), + "organize": v.organize, + "find_open": v.find.is_some(), + })), + "home": self.active.is_none(), + "mode": format!("{:?}", self.mode), + "left_panel": if self.left_open { json!(format!("{:?}", self.left)) } else { Value::Null }, + "right_panel": self.right.map(|r| format!("{r:?}")), + "dialog": self.dialog.map(|d| format!("{d:?}")), + "palette_open": self.palette_open, + "theme": format!("{:?}", self.theme), + "notice": self.toast.as_ref().map(|t| t.0.clone()), + "password_prompt": self.password_prompt.is_some(), + "close_prompt": self.close_request.is_some(), + }) + } + + fn command(&mut self, id: &str) -> Reply { + let spec = printcraft_engine::commands::command(id).ok_or_else(|| format!("unknown command {id:?} (see ui.commands)"))?; + if !self.command_enabled(spec) { + return Err(format!("{id} is disabled right now")); + } + self.execute(id); + Ok(json!({ "ran": id })) + } + + fn commands(&self) -> Value { + let list: Vec = printcraft_engine::commands::COMMANDS + .iter() + .map(|c| json!({ "id": c.id, "label": c.label, "menu": c.menu, "enabled": self.command_enabled(c), "shortcut": c.shortcut.map(|s| s.label(cfg!(target_os = "macos"))) })) + .collect(); + json!({ "commands": list }) + } + + fn set(&mut self, key: &str, value: &str) -> Reply { + self.set_option(key, value).map(|()| json!({ "set": key, "value": value })) + } + + #[cfg(not(target_arch = "wasm32"))] + fn open(&mut self, path: &str) -> Reply { + let before = self.views.len(); + self.open_path(path); + if self.views.len() > before || self.password_prompt.is_some() { + Ok(json!({ "opened": path, "password_prompt": self.password_prompt.is_some() })) + } else { + Err(self.toast.as_ref().map(|t| t.0.clone()).unwrap_or_else(|| format!("couldn't open {path}"))) + } + } + + #[cfg(target_arch = "wasm32")] + fn open(&mut self, _path: &str) -> Reply { + Err("ui.open needs a file system; on the web, drop the file on the page".into()) + } +} + +// ---- transport (native only) ----------------------------------------------------------------- + +/// Where a running app's control channel can be reached (written to the `--control` file). +#[cfg(not(target_arch = "wasm32"))] +pub struct Endpoint { + pub port: u16, + pub token: String, +} + +/// Listen on `127.0.0.1` (random port) and forward authenticated requests to `client`. +/// +/// Protocol: newline-delimited JSON-RPC 2.0. The first request on a connection must be +/// `{"method": "auth", "params": {"token": …}}`; anything else closes the connection. +#[cfg(not(target_arch = "wasm32"))] +pub fn serve(client: ControlClient) -> std::io::Result { + use std::io::{BufRead, BufReader, Write}; + let listener = std::net::TcpListener::bind(("127.0.0.1", 0))?; + let port = listener.local_addr()?.port(); + let token = random_token()?; + let expected = token.clone(); + std::thread::Builder::new().name("printcraft-control".into()).spawn(move || { + for stream in listener.incoming().flatten() { + let client = client.clone(); + let expected = expected.clone(); + let _ = std::thread::Builder::new().name("printcraft-control-conn".into()).spawn(move || { + let Ok(read) = stream.try_clone() else { return }; + let mut write = stream; + let mut authed = false; + for line in BufReader::new(read).lines() { + let Ok(line) = line else { break }; + if line.trim().is_empty() { + continue; + } + let (reply, close) = match serde_json::from_str::(&line) { + Err(e) => (rpc_error(Value::Null, -32700, &format!("parse error: {e}")), false), + Ok(msg) => { + let id = msg.get("id").cloned().unwrap_or(Value::Null); + let method = msg.get("method").and_then(Value::as_str).unwrap_or(""); + let params = msg.get("params").cloned().unwrap_or(Value::Null); + if !authed { + let ok = + method == "auth" && params.get("token").and_then(Value::as_str).is_some_and(|t| constant_time_eq(t, &expected)); + authed = ok; + if ok { + (json!({ "jsonrpc": "2.0", "id": id, "result": { "ok": true } }), false) + } else { + (rpc_error(id, -32001, "authenticate first: auth {token}"), true) + } + } else { + let rx = client.send(method, params); + match rx.recv_timeout(std::time::Duration::from_secs(30)) { + Ok(Ok(v)) => (json!({ "jsonrpc": "2.0", "id": id, "result": v }), false), + Ok(Err(e)) => (rpc_error(id, -32000, &e), false), + Err(_) => (rpc_error(id, -32002, "the app did not answer within 30 s"), false), + } + } + } + }; + if writeln!(write, "{reply}").and_then(|_| write.flush()).is_err() || close { + break; + } + } + }); + } + })?; + Ok(Endpoint { port, token }) +} + +#[cfg(not(target_arch = "wasm32"))] +fn rpc_error(id: Value, code: i64, message: &str) -> Value { + json!({ "jsonrpc": "2.0", "id": id, "error": { "code": code, "message": message } }) +} + +#[cfg(not(target_arch = "wasm32"))] +fn constant_time_eq(a: &str, b: &str) -> bool { + a.len() == b.len() && a.bytes().zip(b.bytes()).fold(0u8, |acc, (x, y)| acc | (x ^ y)) == 0 +} + +/// 128 random bits from the OS, hex-encoded. +#[cfg(not(target_arch = "wasm32"))] +fn random_token() -> std::io::Result { + let mut bytes = [0u8; 16]; + getrandom::fill(&mut bytes).map_err(|e| std::io::Error::other(e.to_string()))?; + Ok(bytes.iter().map(|b| format!("{b:02x}")).collect()) +} diff --git a/crates/ui-egui/src/lib.rs b/crates/ui-egui/src/lib.rs index 93ed14d4..99cb86b1 100644 --- a/crates/ui-egui/src/lib.rs +++ b/crates/ui-egui/src/lib.rs @@ -8,6 +8,7 @@ mod canvas; mod chrome; mod commands; +pub mod control; mod dialogs; mod editing; mod files; @@ -148,6 +149,8 @@ pub struct PrintCraftApp { pending_theme: Option, styled: bool, fonts_ready: bool, + /// The UI control channel, when enabled (`--control`; off by default). + control: Option, } impl Default for PrintCraftApp { @@ -194,6 +197,7 @@ impl PrintCraftApp { pending_theme: None, styled: false, fonts_ready: false, + control: None, } } @@ -370,6 +374,14 @@ impl PrintCraftApp { self.active.and_then(|i| self.views.get(i).map(|v| (i, v.id))) } + /// Enable the UI control channel on `ctx` (opt-in; see [`control`]). Returns a client that + /// sends requests to this app; [`control::serve`] exposes it on loopback. + pub fn attach_control(&mut self, ctx: &egui::Context) -> control::ControlClient { + let (control, client) = control::attach(ctx); + self.control = Some(control); + client + } + pub fn notify(&mut self, msg: impl Into) { self.toast = Some((msg.into(), 0.0)); } @@ -567,6 +579,10 @@ impl eframe::App for PrintCraftApp { self.notify(format!("Couldn't open {name}: {e}")); } } + if let Some(mut control) = self.control.take() { + control.tick(ctx, self); + self.control = Some(control); + } self.guard_quit(ctx); let now = ctx.input(|i| i.time); self.autosave_tick(now); diff --git a/crates/ui-egui/tests/control.rs b/crates/ui-egui/tests/control.rs new file mode 100644 index 00000000..8e02bc54 --- /dev/null +++ b/crates/ui-egui/tests/control.rs @@ -0,0 +1,146 @@ +//! The UI control channel (M3.9): an agent can see the widget tree, click, type, press keys, +//! run commands, change view options and take screenshots of the running app. + +use std::sync::{Arc, Mutex}; + +use egui_kittest::Harness; +use printcraft_ui_egui::PrintCraftApp; +use printcraft_ui_egui::control::{ControlClient, Reply}; +use serde_json::{Value, json}; + +fn fixture(n: usize) -> Vec { + let mut objs: Vec = vec!["<< /Type /Catalog /Pages 2 0 R >>".into()]; + let kids: Vec = (0..n).map(|i| format!("{} 0 R", 4 + 2 * i)).collect(); + objs.push(format!("<< /Type /Pages /Kids [{}] /Count {n} /MediaBox [0 0 200 300] >>", kids.join(" "))); + objs.push("<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>".into()); + for i in 0..n { + objs.push(format!("<< /Type /Page /Parent 2 0 R /Contents {} 0 R /Resources << /Font << /F1 3 0 R >> >> >>", 5 + 2 * i)); + let body = format!("BT /F1 24 Tf 20 150 Td (Page {}) Tj ET", i + 1); + objs.push(format!("<< /Length {} >>\nstream\n{body}\nendstream", body.len())); + } + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offsets.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let xref = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offsets { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objs.len() + 1).as_bytes()); + out +} + +fn harness() -> (Harness<'static, PrintCraftApp>, ControlClient) { + let slot: Arc>> = Arc::default(); + let s = slot.clone(); + let mut h = Harness::builder().with_size(egui::vec2(1400.0, 900.0)).build_eframe(move |cc| { + let mut app = PrintCraftApp::new(); + *s.lock().unwrap() = Some(app.attach_control(&cc.egui_ctx)); + app.open_bytes("doc.pdf", None, fixture(5)).unwrap(); + app + }); + h.run_steps(4); + let client = slot.lock().unwrap().take().unwrap(); + (h, client) +} + +/// Send a request and run frames until it is answered. +fn call(h: &mut Harness<'static, PrintCraftApp>, c: &ControlClient, method: &str, params: Value) -> Reply { + let rx = c.send(method, params); + for _ in 0..30 { + h.step(); + if let Ok(r) = rx.try_recv() { + return r; + } + } + panic!("{method}: no reply after 30 frames"); +} + +fn ok(h: &mut Harness<'static, PrintCraftApp>, c: &ControlClient, method: &str, params: Value) -> Value { + call(h, c, method, params).unwrap_or_else(|e| panic!("{method}: {e}")) +} + +#[test] +fn state_and_view_options() { + let (mut h, c) = harness(); + let s = ok(&mut h, &c, "ui.state", json!({})); + assert_eq!(s["documents"][0]["name"], "doc.pdf"); + assert_eq!(s["documents"][0]["pages"], 5); + assert_eq!(s["active"]["page"], 1); + ok(&mut h, &c, "ui.set", json!({ "key": "page", "value": 4 })); + h.run_steps(3); + assert_eq!(ok(&mut h, &c, "ui.state", json!({}))["active"]["page"], 4); + ok(&mut h, &c, "ui.set", json!({ "key": "panel", "value": "bookmarks" })); + assert_eq!(ok(&mut h, &c, "ui.state", json!({}))["right_panel"], "Bookmarks"); + assert!(call(&mut h, &c, "ui.set", json!({ "key": "panel", "value": "nonsense" })).is_err()); +} + +#[test] +fn inspect_and_click_by_label_and_id() { + let (mut h, c) = harness(); + let found = ok(&mut h, &c, "ui.inspect", json!({ "query": "read" })); + let read = found["widgets"].as_array().unwrap().iter().find(|w| w["label"] == "Read" && w["clickable"] == true).cloned().expect("a Read tab"); + let rect = read["rect"].as_array().unwrap(); + assert!(rect[2].as_f64().unwrap() > rect[0].as_f64().unwrap()); + + ok(&mut h, &c, "ui.click", json!({ "label": "Read" })); + assert_eq!(ok(&mut h, &c, "ui.state", json!({}))["mode"], "Read"); + + let all = ok(&mut h, &c, "ui.inspect", json!({ "query": "all tools" })); + let id = all["widgets"].as_array().unwrap().iter().find(|w| w["label"] == "All tools" && w["clickable"] == true).unwrap()["id"].clone(); + ok(&mut h, &c, "ui.click", json!({ "id": id })); + assert_eq!(ok(&mut h, &c, "ui.state", json!({}))["mode"], "AllTools"); + + // Clicking a point works too (here: the Read tab's centre). + let [x0, y0, x1, y1] = [0, 1, 2, 3].map(|i| rect[i].as_f64().unwrap()); + ok(&mut h, &c, "ui.click", json!({ "x": (x0 + x1) / 2.0, "y": (y0 + y1) / 2.0 })); + assert_eq!(ok(&mut h, &c, "ui.state", json!({}))["mode"], "Read"); + + let err = call(&mut h, &c, "ui.click", json!({ "label": "No such button" })).unwrap_err(); + assert!(err.contains("no enabled clickable widget"), "{err}"); + assert!(call(&mut h, &c, "ui.click", json!({ "id": 12345 })).is_err()); +} + +#[test] +fn commands_keys_and_typing() { + let (mut h, c) = harness(); + let list = ok(&mut h, &c, "ui.commands", json!({})); + let undo = list["commands"].as_array().unwrap().iter().find(|x| x["id"] == "edit.undo").unwrap().clone(); + assert_eq!(undo["enabled"], false); + assert!(call(&mut h, &c, "ui.command", json!({ "id": "edit.undo" })).unwrap_err().contains("disabled")); + assert!(call(&mut h, &c, "ui.command", json!({ "id": "nope" })).is_err()); + + ok(&mut h, &c, "ui.command", json!({ "id": "page.rotate" })); + h.run_steps(2); + let list = ok(&mut h, &c, "ui.commands", json!({})); + assert_eq!(list["commands"].as_array().unwrap().iter().find(|x| x["id"] == "edit.undo").unwrap()["enabled"], true); + + // ⌘K opens the palette; typing filters it; Escape closes it. + ok(&mut h, &c, "ui.key", json!({ "key": "K", "modifiers": ["command"] })); + assert_eq!(ok(&mut h, &c, "ui.state", json!({}))["palette_open"], true); + ok(&mut h, &c, "ui.type", json!({ "text": "split" })); + let hits = ok(&mut h, &c, "ui.inspect", json!({ "query": "split document" })); + assert!(hits["count"].as_u64().unwrap() >= 1, "{hits}"); + ok(&mut h, &c, "ui.key", json!({ "key": "Escape" })); + assert_eq!(ok(&mut h, &c, "ui.state", json!({}))["palette_open"], false); + + assert!(call(&mut h, &c, "ui.key", json!({ "key": "NotAKey" })).is_err()); + assert!(call(&mut h, &c, "ui.frobnicate", json!({})).unwrap_err().contains("unknown method")); +} + +#[test] +fn screenshots_of_window_and_region() { + let (mut h, c) = harness(); + let shot = ok(&mut h, &c, "ui.screenshot", json!({})); + let ppp = shot["pixels_per_point"].as_f64().unwrap(); + assert_eq!(shot["width"].as_f64().unwrap(), (1400.0 * ppp).round()); + use base64::Engine as _; + let png = base64::engine::general_purpose::STANDARD.decode(shot["png_base64"].as_str().unwrap()).unwrap(); + assert_eq!(&png[1..4], b"PNG"); + let region = ok(&mut h, &c, "ui.screenshot", json!({ "region": [10, 20, 110, 70] })); + assert_eq!((region["width"].as_f64().unwrap(), region["height"].as_f64().unwrap()), ((100.0 * ppp).round(), (50.0 * ppp).round())); + assert!(call(&mut h, &c, "ui.screenshot", json!({ "region": [5, 5, 1, 1] })).is_err()); +}