From b1e958cf63a3729b12d04a8ea675a2ff6ee274d0 Mon Sep 17 00:00:00 2001 From: magdale76 Date: Sun, 11 Oct 2026 18:31:23 +0200 Subject: [PATCH] M10: Enhance camera image finds the document's edges; rotate, crop or organize after Enhance page (part 3 of #1008) (#1027) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * M10: enhance camera images: straighten a page from four corners (ocr.camera-perspective) Four corners on a page (any order, a convex quadrilateral) are warped to an upright rectangle that replaces what the page shows, read at the page picture's own resolution (at most 600 dpi) and kept at that physical size. One undoable step through Replace Pages, so comments, links and bookmarks stay. - geom: a pure 3x3 homography solve and corner ordering (no dependencies); collinear, concave and non-finite input gives None. - engine: camera::warp (bilinear, over white, capped at 40 MP) and Session::enhance_camera_page. - automation: page_enhance_camera (corners in top-left-origin points). - parity: ocr.camera-perspective is partial; the UI handles, automatic edge detection and contrast cleanup are still missing. Refs #1008. Co-Authored-By: Claude Opus 5.5 * M10: Enhance camera image: drag four corners on the page, then Enhance page (ocr.camera-perspective) Scan & OCR ▸ Enhance camera image (ocr.enhance_camera) shows four corner handles on the current page, inset 5% from its edges, with what will be cut away dimmed. Enhance page straightens what they enclose through Session::enhance_camera_page (one undoable step); Cancel and Esc leave the page alone. The buttons float at the top right, clear of a centred page's corners, and the instructions are a notice, as for Crop. A command of its own, beside Enhance scanned file (ocr.enhance), which stays for scan correction (#1012). - ui-egui: camera_ui (handles, dimming, bar), QuickTool::EnhanceCamera, quick_tool "enhance-camera" for ui.state and the shot example. - Catalogs: the three new labels in every language with UI coverage (plus es, pl, zh-hant); ATTRIBUTION.toml hashes updated. - automation: page_enhance_camera is linked to the command. - parity: ocr.camera-perspective lists the command and the UI test. Refs #1008. Co-Authored-By: Claude Opus 5.5 * M10: Enhance camera image finds the document's edges (ocr.camera-perspective) The corners now start on the photographed document's edges when they can be found, else inset 5% as before. camera_edges::detect_document, pure Rust and deterministic: shrink to 400 px, blur, Sobel edges thinned and thresholded against the 98th percentile; a Hough transform where each edge votes within 6 degrees of its own normal; pairs of near-horizontal and near-vertical lines make convex quadrilaterals (at least 12% of the picture), kept when every side lies on a matching edge along at least 55% of its length and scored by area times the square of the mean support; each side is then refitted through its edge points (total least squares). None when nothing convinces. - Session::detect_camera_corners (user space); page_enhance_camera finds the corners itself when `corners` is omitted and reports the corners it used and whether they were detected; the UI command starts from them. - Tests on synthetic photos: plain table, text on the sheet, wood grain with a shadow, strong perspective, a 30 degree turn, a sheet running off the picture, no document, white on white, bad input; end to end through the engine and the tool. - Parity notes: what is still missing (turning the result upright, rounded corners, clean-up). Refs #1008. Co-Authored-By: Claude Opus 5.5 * M10: after Enhance page, rotate, crop or organize the page (ocr.camera-perspective) Enhance camera image no longer closes on Enhance page: the bar then offers rotate counterclockwise and clockwise for the enhanced page (one undo step each), Crop pages and Organize pages (which leave the tool for those tools), and Done; Esc leaves too. A photo of a card often comes out a quarter turn off until the page is turned upright automatically. The two icon buttons are named for screen readers. Every label already exists in the catalogs. Refs #1008. Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: magdale76 <16279778+magdale76@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 Co-authored-by: Brandon Thomas --- ATTRIBUTION.toml | 30 +- Cargo.lock | 1 + crates/automation/src/lib.rs | 22 ++ crates/automation/src/tools.rs | 10 + crates/automation/tests/automation.rs | 56 +++ crates/engine/Cargo.toml | 1 + crates/engine/src/camera.rs | 238 ++++++++++++ crates/engine/src/camera_edges.rs | 501 ++++++++++++++++++++++++++ crates/engine/src/catalog.rs | 1 + crates/engine/src/commands.rs | 1 + crates/engine/src/lib.rs | 2 + crates/engine/src/tests.rs | 92 +++++ crates/geom/src/lib.rs | 2 + crates/geom/src/perspective.rs | 171 +++++++++ crates/ui-egui/src/camera_ui.rs | 168 +++++++++ crates/ui-egui/src/canvas.rs | 31 +- crates/ui-egui/src/commands.rs | 17 + crates/ui-egui/src/control.rs | 1 + crates/ui-egui/src/editing.rs | 49 +++ crates/ui-egui/src/i18n/ar.tsv | 3 + crates/ui-egui/src/i18n/bg.tsv | 3 + crates/ui-egui/src/i18n/de.tsv | 3 + crates/ui-egui/src/i18n/es.tsv | 3 + crates/ui-egui/src/i18n/fr.tsv | 3 + crates/ui-egui/src/i18n/hu.tsv | 3 + crates/ui-egui/src/i18n/it.tsv | 3 + crates/ui-egui/src/i18n/ja.tsv | 3 + crates/ui-egui/src/i18n/pl.tsv | 3 + crates/ui-egui/src/i18n/pt-br.tsv | 3 + crates/ui-egui/src/i18n/ru.tsv | 3 + crates/ui-egui/src/i18n/te.tsv | 3 + crates/ui-egui/src/i18n/uk.tsv | 3 + crates/ui-egui/src/i18n/zh-hans.tsv | 3 + crates/ui-egui/src/i18n/zh-hant.tsv | 3 + crates/ui-egui/src/lib.rs | 4 + crates/ui-egui/tests/camera.rs | 125 +++++++ docs/target-app-parity.md | 2 +- parity/acrobat-features.toml | 10 +- 38 files changed, 1558 insertions(+), 22 deletions(-) create mode 100644 crates/engine/src/camera.rs create mode 100644 crates/engine/src/camera_edges.rs create mode 100644 crates/geom/src/perspective.rs create mode 100644 crates/ui-egui/src/camera_ui.rs create mode 100644 crates/ui-egui/tests/camera.rs diff --git a/ATTRIBUTION.toml b/ATTRIBUTION.toml index 60df7267..7835c72e 100644 --- a/ATTRIBUTION.toml +++ b/ATTRIBUTION.toml @@ -19,7 +19,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "Simplified Chinese UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "c1f833a5a61b7d3503d831691deee0e95777f6ec7e9fe12ef0eb68cdcdcd5488" +sha256 = "1494c20c21ac3491d22531c7b05b6e718b361c6e587ae4df8d7fa11a3f5ea484" [[asset]] path = "crates/ui-egui/src/i18n/zh-hant.tsv" @@ -30,7 +30,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "Traditional Chinese UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "c8d2b9d9ef7931296e960799d3e5e07a7c39acf8ecbdc6194086ff3b1d37061e" +sha256 = "7d0770a42541d1c3f1c268450b81ca1d1cf49212f14104ffce25f55fdfbb3d51" [[asset]] path = "crates/ui-egui/src/i18n/cs.tsv" @@ -52,7 +52,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "Brazilian Portuguese UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "21c72b62bd3b0d6a3320e396e13ecc9dc952e76b8c457db5ed4d85f06c94370b" +sha256 = "935be8513694b4bec72b2b998f8061b2c759656d409fa91d7909726d2a2b6252" [[asset]] path = "crates/ui-egui/src/i18n/es.tsv" @@ -63,7 +63,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "Spanish UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "c6bbe72bf89e3642853a8e608386d8a015e31ab6657f508333d8ff666ab75bef" +sha256 = "e78ed8e2cbc06843be00788142d1d14448f51b7fe19e5c264a7757c1e041eda6" [[asset]] path = "crates/ui-egui/src/i18n/fr.tsv" @@ -74,7 +74,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "French UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "c70f2295630e78c936da0f533fa4df33572047eb298e9bdc6b00357e428fa54d" +sha256 = "d72057bf5178e6847303c859b21ef9173f1a0f23ff6f8af4bbe65ffb5d31c5c2" [[asset]] path = "crates/ui-egui/src/i18n/de.tsv" @@ -85,7 +85,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "German UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "c5674c627fe69f8e2f103de58b09b3808b6de927a0c23d629e2aa5e4a9d0662a" +sha256 = "fa4ec97ae6e5c9a21e420db46da3a99e135eea4cf7065c3514e817eb85003ece" [[asset]] path = "crates/ui-egui/src/i18n/te.tsv" @@ -96,7 +96,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "Telugu UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "614ec635e6a18770f5c7c2817dfd26f426239bb1610eab7f24d308882c59f1ba" +sha256 = "a10ac20172d220e4981f122f7c68c5310f7e6e53e6d3c128ea2e0928d58efe84" [[asset]] path = "crates/ui-egui/src/i18n/it.tsv" @@ -107,7 +107,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "Italian UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "08007ca54168afeb35391b2ee6477bf4a034cb285d79b4807a09218e4956d417" +sha256 = "33d7f73c6b8837807ff12451a00a5359546ffef57e4ce75a2ee5fe53578888c8" [[asset]] path = "crates/ui-egui/src/i18n/pl.tsv" @@ -118,7 +118,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "Polish UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "d4f4821c5dfb4ef900d525b70471af487583e2704c841717f52337e9f69a2ebf" +sha256 = "e30e23bafce57e494ab99b8da0658292aa3da656c6d8f4470636490ce1a998f4" [[asset]] path = "crates/ui-egui/src/i18n/ru.tsv" @@ -129,7 +129,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "Russian UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "44f817bf7be5dafbd327c579fe1de7628bc623f060d1b995462b89ec9ced7fa4" +sha256 = "c32312580b8c398cf22ad281dab6cb73acc6d11395e238569e00192254bbf07a" [[asset]] path = "crates/ui-egui/src/i18n/hu.tsv" @@ -140,7 +140,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "Hungarian UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "7c3147a98d190d51d32b97baef3c70ff8e8897ef5daf455ab211fa83e80e89eb" +sha256 = "e82c1076711ad819892a21e9dfc836f27c5545d8acd27a961644477949a3039d" [[asset]] path = "crates/ui-egui/src/i18n/bg.tsv" @@ -151,7 +151,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "Bulgarian UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "a5e7999bd91ebfce11bee791c8c1764ceceac89fd3d3ae841c97df63c1279394" +sha256 = "d92f7f598b61cc2874f75ceb247d6df92c715a0c3bb1cb51574343d2e212fda1" [[asset]] path = "crates/ui-egui/src/i18n/ja.tsv" @@ -162,7 +162,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "Japanese UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "5b1c282abe4891fbe306187b5ff920ea4a943c68ce5f8dbdd95c75f5f42bdb70" +sha256 = "5828c26d65ded4d272bdf1abd8ce56fc71c963953d7c4ea10021fcea02eda6fb" [[asset]] path = "crates/ui-egui/src/i18n/uk.tsv" @@ -173,7 +173,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "Ukrainian UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "d87966fe1ab113fd9560d0c4aab34185fff5b40821db01def22d41a9ff3c9762" +sha256 = "0555902c3f2ec6269121c033050055374cf9afd4de8f64d7857c06cba841c179" [[asset]] path = "crates/ui-egui/src/i18n/ar.tsv" @@ -184,7 +184,7 @@ licence = "MIT OR Apache-2.0" licence_file = "LICENSE-MIT" kind = "translation" usage = "Arabic UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs" -sha256 = "78047120b89a9bc05863888bc7ec8898e3eac8a56cdae7bfa8a436c3080adacc" +sha256 = "cd168b8ca799fd4552b09d791e762b2cf05ec37a6eb7aeb5d304281d812c5411" [[asset]] path = "assets/icons/accessibility.svg" diff --git a/Cargo.lock b/Cargo.lock index 08f95566..60d3a33c 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3704,6 +3704,7 @@ dependencies = [ "pdfcraft-export", "pdfcraft-fonts", "pdfcraft-forms", + "pdfcraft-geom", "pdfcraft-js", "pdfcraft-measure", "pdfcraft-model", diff --git a/crates/automation/src/lib.rs b/crates/automation/src/lib.rs index a98336f4..8d49d66a 100644 --- a/crates/automation/src/lib.rs +++ b/crates/automation/src/lib.rs @@ -196,6 +196,28 @@ impl Automation { out["rotated"] = json!(n); out } + "page_enhance_camera" => { + let page = self.page(&a)?; + let given = a.nums::<8>("corners")?; + let doc = self.doc(&a)?; + let (id, info) = (doc.id, doc.info.pages.get(page).cloned().ok_or_else(|| failed("no such page"))?); + let corners = match given { + Some(c) => [0, 1, 2, 3].map(|i| { + let [x, y] = info.view_to_user(c[2 * i] as f32, c[2 * i + 1] as f32); + [x as f64, y as f64] + }), + None => self.session.detect_camera_corners(id, page).ok_or_else(|| { + failed("the document's edges could not be found on this page: pass `corners` [x1, y1, x2, y2, x3, y3, x4, y4]") + })?, + }; + self.session.enhance_camera_page(id, page, corners).map_err(failed)?; + let r = |v: f32| (v * 100.0).round() / 100.0; + let used: Vec = corners.iter().flat_map(|&[x, y]| info.user_to_view(x as f32, y as f32)).map(r).collect(); + let mut out = summary(self.doc(&a)?); + out["corners"] = json!(used); + out["detected"] = json!(given.is_none()); + out + } "page_delete" => { let pages = self.pages(&a, "pages")?; self.apply(&a, Edit::DeletePages { pages })? diff --git a/crates/automation/src/tools.rs b/crates/automation/src/tools.rs index c99e1d86..60d559f4 100644 --- a/crates/automation/src/tools.rs +++ b/crates/automation/src/tools.rs @@ -182,6 +182,16 @@ pub fn tools() -> Vec { }), &["doc", "degrees"], )), + t("page_enhance_camera", "Enhance camera image", "Straighten a photographed document (Scan & OCR ▸ Enhance ▸ Camera image): the part of the page inside the four corners (x1, y1, … x4, y4 in top-left-origin points, any order, a convex quadrilateral; when omitted, the document's edges are found automatically, or an error says they could not be) is warped to an upright rectangle that replaces what the page shows, at the page picture's own resolution (at most 600 dpi). The page's comments, links and bookmarks stay. Undoable.") + .cmd("ocr.enhance_camera") + .with(schema( + json!({ + "doc": doc(), + "page": { "type": "integer", "minimum": 1 }, + "corners": { "type": "array", "items": { "type": "number" }, "minItems": 8, "maxItems": 8 }, + }), + &["doc", "page"], + )), t("page_delete", "Delete pages", "Delete pages. Undoable until saved.") .destructive() .cmd("page.delete") diff --git a/crates/automation/tests/automation.rs b/crates/automation/tests/automation.rs index 521d903e..b3380bbc 100644 --- a/crates/automation/tests/automation.rs +++ b/crates/automation/tests/automation.rs @@ -1663,6 +1663,62 @@ fn editing_page_images_through_tools() { assert!(matches!(a.call("image_edit", &json!({ "doc": made, "page": 1, "image": 1, "action": "delete" })), Err(ToolError::InvalidArgs(_)))); } +#[test] +fn enhance_camera_image_through_tools() { + let dir = workdir("enhance-camera"); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "a.pdf" }))["doc"].as_u64().unwrap(); + ok(&mut a, "doc_export_images", json!({ "doc": doc, "folder": "src", "dpi": 72, "pages": [1] })); + let photo = ok(&mut a, "doc_create", json!({ "from": "images", "paths": ["src/a_page_1.png"] }))["doc"].as_u64().unwrap(); + // A 200 × 300 pt page at one pixel per point; the corners, in any order, keep a 160 × 240 part. + let r = ok(&mut a, "page_enhance_camera", json!({ "doc": photo, "page": 1, "corners": [180, 270, 20, 30, 20, 270, 180, 30] })); + assert_eq!(r["undo"], "Enhance camera image"); + assert_eq!(r["pages"], 1); + let images = ok(&mut a, "page_images", json!({ "doc": photo, "page": 1 })); + assert_eq!(images["count"], 1); + assert_eq!(images["images"][0]["pixels"], json!([160, 240])); + assert_eq!(images["images"][0]["rect"], json!([0.0, 0.0, 160.0, 240.0])); + for bad in [json!([0, 0, 10, 10, 20, 20, 30, 30]), json!([0, 0, 10]), json!("corners")] { + let e = a.call("page_enhance_camera", &json!({ "doc": photo, "page": 1, "corners": bad })); + assert!(matches!(e, Err(ToolError::InvalidArgs(_) | ToolError::Failed(_))), "{bad}: {e:?}"); + } + // Without corners on a page with no photo, the edges can't be found: an error that says so. + let e = a.call("page_enhance_camera", &json!({ "doc": photo, "page": 1 })); + assert!(matches!(&e, Err(ToolError::Failed(m)) if m.contains("corners")), "{e:?}"); + assert!(matches!( + a.call("page_enhance_camera", &json!({ "doc": photo, "page": 9, "corners": [0, 0, 9, 0, 9, 9, 0, 9] })), + Err(ToolError::InvalidArgs(_)) + )); +} + +#[test] +fn enhance_camera_image_finds_the_edges_by_itself() { + let dir = workdir("enhance-camera-auto"); + // A 600 × 450 photo: a white sheet, tilted, on a dark table. + let sheet = [[90.0, 60.0], [500.0, 30.0], [525.0, 405.0], [60.0, 375.0]]; + let inside = |x: f64, y: f64| { + (0..4).all(|i| { + let (a, b): ([f64; 2], [f64; 2]) = (sheet[i], sheet[(i + 1) % 4]); + (b[0] - a[0]) * (y - a[1]) - (b[1] - a[1]) * (x - a[0]) >= 0.0 + }) + }; + let img = image::RgbImage::from_fn(600, 450, |x, y| { + if inside(x as f64 + 0.5, y as f64 + 0.5) { image::Rgb([240, 238, 232]) } else { image::Rgb([70, 60, 50]) } + }); + img.save(dir.join("photo.jpg")).unwrap(); + let mut a = auto(&dir); + let photo = ok(&mut a, "doc_create", json!({ "from": "images", "paths": ["photo.jpg"] }))["doc"].as_u64().unwrap(); + let r = ok(&mut a, "page_enhance_camera", json!({ "doc": photo, "page": 1 })); + assert_eq!(r["detected"], true); + assert_eq!(r["undo"], "Enhance camera image"); + // One point per pixel: the corners it used are the sheet's. + let used: Vec = r["corners"].as_array().unwrap().iter().map(|v| v.as_f64().unwrap()).collect(); + for (k, p) in sheet.iter().enumerate() { + let (x, y) = (used[2 * k], used[2 * k + 1]); + assert!((x - p[0]).hypot(y - p[1]) < 4.0, "{used:?} vs {sheet:?}"); + } +} + #[test] fn editing_form_artwork_through_tools() { use pdfcraft_cos::{Dict, Document, Object, SaveOptions, Stream, write_incremental}; diff --git a/crates/engine/Cargo.toml b/crates/engine/Cargo.toml index bcf762fc..4e754f7e 100644 --- a/crates/engine/Cargo.toml +++ b/crates/engine/Cargo.toml @@ -32,6 +32,7 @@ pdfcraft-export = { workspace = true } pdfcraft-fonts = { workspace = true } pdfcraft-create = { workspace = true } pdfcraft-model = { workspace = true } +pdfcraft-geom = { workspace = true } thiserror = { workspace = true } png = "0.18" # Already in the graph (url and idna through webbrowser, icu_properties through boa): they show diff --git a/crates/engine/src/camera.rs b/crates/engine/src/camera.rs new file mode 100644 index 00000000..e0457cf2 --- /dev/null +++ b/crates/engine/src/camera.rs @@ -0,0 +1,238 @@ +//! Scan & OCR ▸ Enhance ▸ Camera image: straighten a photographed document. Four corners on the +//! page (PDF user space) are warped to a rectangle, and the page's content becomes that picture +//! (one undoable step, through Replace Pages, so the page's comments, links and bookmarks stay). + +use std::sync::Arc; + +use pdfcraft_geom::perspective::{Homography, Point, order_quad}; +use pdfcraft_render::{RenderConfig, RenderRequest}; + +use crate::{DocId, Edit, EditError, ImageResolution, Session}; + +/// The densest the page is read at: 600 dpi, as for OCR. +const MAX_SCALE: f32 = 600.0 / 72.0; +/// The resolution a page without a picture is read at. +const DEFAULT_SCALE: f32 = 300.0 / 72.0; +/// The most pixels the page is rendered with, and the straightened picture has. +const MAX_PIXELS: f64 = 40_000_000.0; +/// JPEG quality of the straightened picture. +const QUALITY: u8 = 90; + +/// A straightened picture: RGB8, row-major. +#[derive(Clone, Debug, PartialEq)] +pub struct Warped { + pub width: u32, + pub height: u32, + pub rgb: Vec, +} + +/// Warp the quadrilateral `quad` (raster pixels, any corner order) of a premultiplied RGBA +/// raster to an upright rectangle as large as the quadrilateral's longer sides, composited over +/// white. Errors say what was wrong with the corners or the raster. +pub fn warp(rgba: &[u8], width: u32, height: u32, quad: [Point; 4]) -> Result { + let (w, h) = (width as usize, height as usize); + if w == 0 || h == 0 || w.checked_mul(h).and_then(|n| n.checked_mul(4)) != Some(rgba.len()) { + return Err("the page could not be read as a picture".into()); + } + let [tl, tr, br, bl] = order_quad(quad).ok_or("the four corners must enclose an area (a convex quadrilateral)")?; + let dist = |a: Point, b: Point| (a[0] - b[0]).hypot(a[1] - b[1]); + let mut ow = dist(tl, tr).max(dist(bl, br)); + let mut oh = dist(tl, bl).max(dist(tr, br)); + let k = (MAX_PIXELS / (ow * oh).max(1.0)).sqrt().min(1.0); + ow = (ow * k).floor().max(1.0); + oh = (oh * k).floor().max(1.0); + if !(ow.is_finite() && oh.is_finite()) { + return Err("the corners are out of range".into()); + } + // Both are at least 1 and their product is at most MAX_PIXELS, so the casts are exact. + let (ow, oh) = (ow as u32, oh as u32); + let rect = [[0.0, 0.0], [ow as f64, 0.0], [ow as f64, oh as f64], [0.0, oh as f64]]; + let to_src = Homography::from_points(rect, [tl, tr, br, bl]).ok_or("the four corners must not be in a line")?; + + let px = |x: usize, y: usize| -> [f64; 4] { + let i = (y * w + x) * 4; + match rgba.get(i..i + 4) { + Some(p) => [p[0] as f64, p[1] as f64, p[2] as f64, p[3] as f64], + None => [0.0; 4], + } + }; + let mut rgb = Vec::with_capacity(ow as usize * oh as usize * 3); + for y in 0..oh { + for x in 0..ow { + // Sample at the pixel's centre; outside the page is white. + let mut c = [0.0f64; 4]; + if let Some([sx, sy]) = to_src.apply([x as f64 + 0.5, y as f64 + 0.5]) { + let (fx, fy) = (sx - 0.5, sy - 0.5); + if fx > -1.0 && fy > -1.0 && fx < w as f64 && fy < h as f64 { + let (x0, y0) = (fx.floor(), fy.floor()); + let (ax, ay) = (fx - x0, fy - y0); + // Clamped to the raster, so every index below is in range. + let clamp = |v: f64, n: usize| v.clamp(0.0, (n - 1) as f64) as usize; + let (xa, xb) = (clamp(x0, w), clamp(x0 + 1.0, w)); + let (ya, yb) = (clamp(y0, h), clamp(y0 + 1.0, h)); + let (p00, p10, p01, p11) = (px(xa, ya), px(xb, ya), px(xa, yb), px(xb, yb)); + for (i, v) in c.iter_mut().enumerate() { + let top = p00[i] * (1.0 - ax) + p10[i] * ax; + let bottom = p01[i] * (1.0 - ax) + p11[i] * ax; + *v = top * (1.0 - ay) + bottom * ay; + } + } + } + // Premultiplied over white: colour + (1 - alpha) × white. + for v in &c[..3] { + rgb.push((v + 255.0 - c[3]).round().clamp(0.0, 255.0) as u8); + } + } + } + Ok(Warped { width: ow, height: oh, rgb }) +} + +/// The long side, in pixels, a page is rendered at to look for the document's edges. +const DETECT_SIDE: f32 = 1200.0; + +impl Session { + /// Enhance ▸ Camera image: the corners of the photographed document on page `page` + /// (0-based) of document `id`, in PDF user space (top-left, top-right, bottom-right, + /// bottom-left as displayed), or `None` when its edges can't be found with confidence. + pub fn detect_camera_corners(&self, id: DocId, page: usize) -> Option<[Point; 4]> { + let doc = self.get(id)?; + let info = doc.info.pages.get(page)?; + let scale = (DETECT_SIDE / info.width.max(info.height).max(1.0)).clamp(0.05, MAX_SCALE); + let config = RenderConfig { password: doc.password.as_deref().map(Arc::from), ..Default::default() }; + let shot = pdfcraft_render::PageRenderer::new(doc.bytes.clone(), config).render(RenderRequest { page, scale, ..Default::default() }); + if shot.error.is_some() { + return None; + } + let quad = crate::camera_edges::detect_document(&shot.rgba, shot.width, shot.height)?; + let k = shot.width as f64 / (info.width as f64).max(1e-3); + Some(quad.map(|[x, y]| { + let [u, v] = info.view_to_user((x / k) as f32, (y / k) as f32); + [u as f64, v as f64] + })) + } + + /// Enhance ▸ Camera image: replace page `page` (0-based) of document `id` with the part of it + /// inside `corners` (PDF user space, any order), straightened to a rectangle. The page is + /// read at its picture's own resolution (at most 600 dpi) and keeps that physical size. + pub fn enhance_camera_page(&mut self, id: DocId, page: usize, corners: [Point; 4]) -> Result<(), EditError> { + let doc = self.get(id).ok_or(EditError::NoDocument)?; + if let Some(why) = &doc.read_only_reason { + return Err(EditError::ReadOnly(why.clone())); + } + let info = doc.info.pages.get(page).ok_or_else(|| EditError::Invalid(format!("there is no page {}", page.saturating_add(1))))?; + let native = doc + .page_images(page) + .iter() + .filter(|i| i.width > 1 && i.height > 1) + .map(|i| { + let m = i.matrix; + let (w, h) = (m[0].hypot(m[1]), m[2].hypot(m[3])); + (i.width as f64 / w.max(1e-6)).max(i.height as f64 / h.max(1e-6)) as f32 + }) + .filter(|s| s.is_finite()) + .reduce(f32::max); + let area = (info.width.max(1.0) as f64) * (info.height.max(1.0) as f64); + let cap = (MAX_PIXELS / area).sqrt() as f32; + let scale = native.unwrap_or(DEFAULT_SCALE).clamp(1.0, MAX_SCALE).min(cap).max(0.01); + + let config = RenderConfig { password: doc.password.as_deref().map(Arc::from), ..Default::default() }; + let mut r = pdfcraft_render::PageRenderer::new(doc.bytes.clone(), config); + let shot = r.render(RenderRequest { page, scale, ..Default::default() }); + if let Some(e) = shot.error { + return Err(EditError::Invalid(format!("page {} could not be rendered: {e}", page + 1))); + } + let k = shot.width as f64 / (info.width as f64).max(1e-3); + let quad = corners.map(|[x, y]| { + let [vx, vy] = info.user_to_view(x as f32, y as f32); + [vx as f64 * k, vy as f64 * k] + }); + let out = warp(&shot.rgba, shot.width, shot.height, quad).map_err(EditError::Invalid)?; + let mut jpeg = Vec::new(); + image::ImageEncoder::write_image( + image::codecs::jpeg::JpegEncoder::new_with_quality(&mut jpeg, QUALITY), + &out.rgb, + out.width, + out.height, + image::ExtendedColorType::Rgb8, + ) + .map_err(|e| EditError::Invalid(format!("the straightened page could not be encoded: {e}")))?; + // One device pixel per `k` points keeps the page's physical size. + let dpi = (k * 72.0).clamp(1.0, 1200.0); + let bytes = self.create_from_images_with_resolution(&[("camera.jpg".into(), jpeg)], ImageResolution::Dpi(dpi))?; + let replace = Edit::ReplacePages { pages: vec![page], name: "camera.pdf".into(), bytes, src_pages: vec![0] }; + self.apply(id, Edit::Batch { label: "Enhance camera image".into(), edits: vec![replace] }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// An opaque premultiplied raster filled by `f(x, y) -> [r, g, b]`. + fn raster(w: u32, h: u32, f: impl Fn(u32, u32) -> [u8; 3]) -> Vec { + (0..h) + .flat_map(|y| (0..w).map(move |x| (x, y))) + .flat_map(|(x, y)| { + let [r, g, b] = f(x, y); + [r, g, b, 255] + }) + .collect() + } + + #[test] + fn a_tilted_quad_comes_out_upright() { + // A dark quadrilateral on white; warping exactly its corners leaves a dark picture. + let quad = [[20.0, 30.0], [170.0, 10.0], [180.0, 150.0], [10.0, 160.0]]; + let inside = |x: f64, y: f64| { + let q = order_quad(quad).unwrap(); + (0..4).all(|i| { + let (a, b) = (q[i], q[(i + 1) % 4]); + (b[0] - a[0]) * (y - a[1]) - (b[1] - a[1]) * (x - a[0]) >= 0.0 + }) + }; + let src = raster(200, 180, |x, y| if inside(x as f64 + 0.5, y as f64 + 0.5) { [20, 20, 20] } else { [255, 255, 255] }); + let out = warp(&src, 200, 180, quad).unwrap(); + // The longer of each pair of opposite sides: 170 × 140. + assert_eq!((out.width, out.height), (170, 140)); + // Away from the anti-aliased edges, everything is the dark fill. + let at = |x: u32, y: u32| out.rgb[((y * out.width + x) * 3) as usize]; + for (x, y) in [(5, 5), (85, 70), (164, 134), (5, 134), (164, 5)] { + assert!(at(x, y) < 60, "({x}, {y}) = {}", at(x, y)); + } + } + + #[test] + fn the_corner_order_does_not_matter() { + let src = raster(64, 48, |x, y| [(x * 4) as u8, (y * 5) as u8, 128]); + let q = [[4.0, 3.0], [60.0, 6.0], [58.0, 44.0], [2.0, 40.0]]; + let a = warp(&src, 64, 48, q).unwrap(); + let b = warp(&src, 64, 48, [q[2], q[0], q[3], q[1]]).unwrap(); + assert_eq!(a, b); + // Left to right the red channel grows, top to bottom the green one does. + let px = |x: u32, y: u32| &a.rgb[((y * a.width + x) * 3) as usize..][..3]; + assert!(px(a.width - 3, 10)[0] > px(2, 10)[0] + 100); + assert!(px(10, a.height - 3)[1] > px(10, 2)[1] + 100); + } + + #[test] + fn transparency_becomes_white() { + let src = vec![0u8; 8 * 8 * 4]; + let out = warp(&src, 8, 8, [[0.0, 0.0], [8.0, 0.0], [8.0, 8.0], [0.0, 8.0]]).unwrap(); + assert!(out.rgb.iter().all(|&v| v == 255)); + } + + #[test] + fn bad_corners_and_rasters_are_errors_not_panics() { + let src = raster(10, 10, |_, _| [0, 0, 0]); + let line = [[0.0, 0.0], [5.0, 5.0], [10.0, 10.0], [2.0, 2.0]]; + assert!(warp(&src, 10, 10, line).is_err()); + assert!(warp(&src, 10, 10, [[f64::NAN, 0.0], [1.0, 0.0], [1.0, 1.0], [0.0, 1.0]]).is_err()); + let ok = [[0.0, 0.0], [10.0, 0.0], [10.0, 10.0], [0.0, 10.0]]; + assert!(warp(&src, 11, 10, ok).is_err(), "size doesn't match the buffer"); + assert!(warp(&[], 0, 0, ok).is_err()); + // Corners far outside the raster: a capped, white picture, not a huge allocation. + let far = [[-1e9, -1e9], [1e9, -1e9], [1e9, 1e9], [-1e9, 1e9]]; + let out = warp(&src, 10, 10, far).unwrap(); + assert!((out.width as f64) * (out.height as f64) <= MAX_PIXELS); + } +} diff --git a/crates/engine/src/camera_edges.rs b/crates/engine/src/camera_edges.rs new file mode 100644 index 00000000..7084dd5e --- /dev/null +++ b/crates/engine/src/camera_edges.rs @@ -0,0 +1,501 @@ +//! Find a photographed document's corners (Enhance ▸ Camera image): the edges of the sheet are +//! the four strongest straight lines that enclose a large convex area, each backed by an edge +//! along most of its length. Pure Rust and deterministic; `None` when nothing convinces, so the +//! caller falls back to corners the user places. +//! +//! Steps: shrink the picture to [`WORK`] pixels on its long side, blur it, take Sobel gradients +//! and keep thin, strong edges; vote for lines (a Hough transform that only votes along each +//! edge's own gradient direction, give or take a few degrees); pair two near-horizontal with two near-vertical lines; keep +//! the quadrilaterals whose four sides lie on edges, and choose the largest. + +use pdfcraft_geom::perspective::{Point, order_quad}; + +/// The working size: the long side of the shrunken picture, in pixels. +const WORK: usize = 400; +/// The most lines tried (strongest first). +const MAX_LINES: usize = 24; +/// A side counts when at least this share of it lies on a matching edge. +const MIN_SUPPORT: f64 = 0.55; +/// A document fills at least this share of the picture. +const MIN_AREA: f64 = 0.12; +/// How far (share of the picture's size) a corner may lie outside the picture. +const MARGIN: f64 = 0.02; + +/// A grey picture, row-major, read with bounds checks. +struct Grey { + w: usize, + h: usize, + v: Vec, +} + +impl Grey { + fn at(&self, x: isize, y: isize) -> f32 { + // Outside the picture repeats its border. + let x = x.clamp(0, self.w as isize - 1) as usize; + let y = y.clamp(0, self.h as isize - 1) as usize; + self.v.get(y * self.w + x).copied().unwrap_or(0.0) + } +} + +/// An edge pixel: its gradient's strength and direction (radians, the normal of the edge). +#[derive(Clone, Copy)] +struct Edge { + mag: f32, + dir: f32, +} + +/// The document's corners in `rgba` (premultiplied RGBA8, `width` × `height`), in pixels: +/// top-left, top-right, bottom-right, bottom-left. `None` when no quadrilateral convinces. +pub fn detect_document(rgba: &[u8], width: u32, height: u32) -> Option<[Point; 4]> { + let (w, h) = (width as usize, height as usize); + if w < 16 || h < 16 || w.checked_mul(h)?.checked_mul(4)? != rgba.len() { + return None; + } + let grey = shrink(rgba, w, h); + let k = w.max(h) as f64 / grey.w.max(grey.h) as f64; + let edges = edges(&blur(&blur(&grey))); + let lines = hough(&edges, grey.w, grey.h); + let quad = best_quad(&lines, &edges, grey.w, grey.h)?; + let quad = refine(quad, &edges, grey.w, grey.h).unwrap_or(quad); + // Back to the picture's pixels (pixel centres line up at k × (x + 0.5)). + Some(quad.map(|[x, y]| [(x + 0.5) * k, (y + 0.5) * k])) +} + +/// Box-filter `rgba` (premultiplied, over white) down to at most [`WORK`] pixels on its long +/// side, as luminance 0–255. +fn shrink(rgba: &[u8], w: usize, h: usize) -> Grey { + let step = w.max(h).div_ceil(WORK).max(1); + let (sw, sh) = (w.div_ceil(step), h.div_ceil(step)); + let mut v = Vec::with_capacity(sw * sh); + for sy in 0..sh { + for sx in 0..sw { + let (mut sum, mut n) = (0.0f32, 0.0f32); + for y in sy * step..((sy + 1) * step).min(h) { + for x in sx * step..((sx + 1) * step).min(w) { + if let Some(p) = rgba.get((y * w + x) * 4..(y * w + x) * 4 + 4) { + let white = 255.0 - p[3] as f32; + let (r, g, b) = (p[0] as f32 + white, p[1] as f32 + white, p[2] as f32 + white); + sum += 0.299 * r + 0.587 * g + 0.114 * b; + n += 1.0; + } + } + } + v.push(if n > 0.0 { sum / n } else { 255.0 }); + } + } + Grey { w: sw, h: sh, v } +} + +/// A 3×3 binomial blur. +fn blur(g: &Grey) -> Grey { + let mut v = Vec::with_capacity(g.v.len()); + for y in 0..g.h as isize { + for x in 0..g.w as isize { + let mut s = 0.0; + for (dy, wy) in [(-1, 1.0), (0, 2.0), (1, 1.0)] { + for (dx, wx) in [(-1, 1.0), (0, 2.0), (1, 1.0)] { + s += wy * wx * g.at(x + dx, y + dy); + } + } + v.push(s / 16.0); + } + } + Grey { w: g.w, h: g.h, v } +} + +/// Thin, strong edges: Sobel gradients, kept where they are a local maximum across the edge +/// and above a threshold relative to the picture's strongest edges. +fn edges(g: &Grey) -> Vec> { + let mut grad = Vec::with_capacity(g.v.len()); + for y in 0..g.h as isize { + for x in 0..g.w as isize { + let p = |dx, dy| g.at(x + dx, y + dy); + let gx = (p(1, -1) + 2.0 * p(1, 0) + p(1, 1)) - (p(-1, -1) + 2.0 * p(-1, 0) + p(-1, 1)); + let gy = (p(-1, 1) + 2.0 * p(0, 1) + p(1, 1)) - (p(-1, -1) + 2.0 * p(0, -1) + p(1, -1)); + grad.push(Edge { mag: gx.hypot(gy), dir: gy.atan2(gx) }); + } + } + // The threshold: a share of the 98th percentile, never below a faint-edge floor. + let mut mags: Vec = grad.iter().map(|e| e.mag).filter(|m| m.is_finite()).collect(); + mags.sort_by(f32::total_cmp); + let p98 = mags.get(mags.len() * 98 / 100).copied().unwrap_or(0.0); + let threshold = (0.25 * p98).max(24.0); + let mag = |x: isize, y: isize| -> f32 { + if x < 0 || y < 0 || x >= g.w as isize || y >= g.h as isize { + return 0.0; + } + grad.get(y as usize * g.w + x as usize).map_or(0.0, |e| e.mag) + }; + let mut out = Vec::with_capacity(grad.len()); + for (i, e) in grad.iter().enumerate() { + let (x, y) = ((i % g.w) as isize, (i / g.w) as isize); + // Edges touching the picture's border are where blurring repeats it: no information. + let inside = x > 1 && y > 1 && x < g.w as isize - 2 && y < g.h as isize - 2; + let keep = inside && e.mag >= threshold && { + let (dx, dy) = (e.dir.cos().round() as isize, e.dir.sin().round() as isize); + e.mag >= mag(x + dx, y + dy) && e.mag >= mag(x - dx, y - dy) + }; + out.push(keep.then_some(*e)); + } + out +} + +/// A line `x cos θ + y sin θ = ρ` and its votes. +#[derive(Clone, Copy, Debug)] +struct Line { + theta: f64, + rho: f64, +} + +/// The strongest lines through the edges: each edge votes for the lines within 6° of its own +/// normal, weighted by its strength; peaks are taken strongest first, clearing their +/// neighbourhood. +fn hough(edges: &[Option], w: usize, h: usize) -> Vec { + const STEPS: usize = 180; + let diag = ((w * w + h * h) as f64).sqrt().ceil() as usize; + let rows = 2 * diag + 1; + let mut acc = vec![0.0f32; STEPS * rows]; + let trig: Vec<(f64, f64)> = (0..STEPS).map(|t| (t as f64).to_radians()).map(|a| (a.cos(), a.sin())).collect(); + for (i, e) in edges.iter().enumerate() { + let Some(e) = e else { continue }; + let (x, y) = ((i % w) as f64, (i / w) as f64); + let normal = (e.dir as f64).to_degrees().rem_euclid(180.0).round() as isize; + // ±6°: a jagged (aliased) edge's local gradient leans towards the axes. + for t in normal - 6..=normal + 6 { + let t = t.rem_euclid(STEPS as isize) as usize; + let Some(&(c, s)) = trig.get(t) else { continue }; + let r = (x * c + y * s).round() as isize + diag as isize; + if let Some(cell) = usize::try_from(r).ok().and_then(|r| acc.get_mut(t * rows + r)) { + *cell += e.mag; + } + } + } + let mut lines = Vec::new(); + let first = acc.iter().copied().fold(0.0f32, f32::max); + while lines.len() < MAX_LINES { + let Some((best, &v)) = acc.iter().enumerate().max_by(|a, b| a.1.total_cmp(b.1)) else { break }; + if v <= 0.0 || v < 0.08 * first { + break; + } + let (t, r) = (best / rows, best % rows); + lines.push(Line { theta: (t as f64).to_radians(), rho: r as f64 - diag as f64 }); + // Clear ±6° and ±6 px around the peak (θ wraps, flipping ρ's sign). + for dt in -6isize..=6 { + let tt = t as isize + dt; + let (tt, flip) = if tt < 0 { + (tt + STEPS as isize, true) + } else if tt >= STEPS as isize { + (tt - STEPS as isize, true) + } else { + (tt, false) + }; + let rr = if flip { 2 * diag as isize - r as isize } else { r as isize }; + for dr in -6isize..=6 { + if let Some(cell) = usize::try_from(rr + dr).ok().filter(|&q| q < rows).and_then(|q| acc.get_mut(tt as usize * rows + q)) { + *cell = 0.0; + } + } + } + } + lines +} + +fn intersect(a: Line, b: Line) -> Option { + let (c1, s1, c2, s2) = (a.theta.cos(), a.theta.sin(), b.theta.cos(), b.theta.sin()); + let det = c1 * s2 - s1 * c2; + if det.abs() < 1e-6 { + return None; + } + let p = [(a.rho * s2 - b.rho * s1) / det, (c1 * b.rho - c2 * a.rho) / det]; + p.iter().all(|v| v.is_finite()).then_some(p) +} + +/// The share of the side from `a` to `b` that lies on an edge facing across it. +fn support(a: Point, b: Point, edges: &[Option], w: usize, h: usize) -> f64 { + let len = (b[0] - a[0]).hypot(b[1] - a[1]); + if len < 1.0 { + return 0.0; + } + let (ux, uy) = ((b[0] - a[0]) / len, (b[1] - a[1]) / len); + // The side's normal, (-uy, ux), as an angle. + let normal = ux.atan2(-uy) as f32; + let n = (len / 2.0).clamp(8.0, 200.0) as usize; + let mut hits = 0; + for i in 0..n { + let t = 0.05 + 0.9 * (i as f64 + 0.5) / n as f64; + let (px, py) = (a[0] + (b[0] - a[0]) * t, a[1] + (b[1] - a[1]) * t); + let hit = (-2i32..=2).any(|o| { + let (x, y) = ((px - uy * o as f64).round(), (py + ux * o as f64).round()); + if x < 0.0 || y < 0.0 || x >= w as f64 || y >= h as f64 { + return false; + } + edges.get(y as usize * w + x as usize).copied().flatten().is_some_and(|e| { + // Either way across: dark-to-light or light-to-dark. + let d = (e.dir - normal).rem_euclid(std::f32::consts::PI); + d < 0.35 || d > std::f32::consts::PI - 0.35 + }) + }); + hits += hit as usize; + } + hits as f64 / n as f64 +} + +/// Sharpen the quadrilateral found on the coarse line grid: fit each side's line through the +/// edge points along it (total least squares), then intersect neighbouring sides again. +/// `None` when a side has too few points or the result is no longer convex. +fn refine(q: [Point; 4], edges: &[Option], w: usize, h: usize) -> Option<[Point; 4]> { + // Each side as (a point on it, its unit direction). + let mut sides = Vec::with_capacity(4); + for k in 0..4 { + let (a, b) = (q[k], q[(k + 1) % 4]); + let len = (b[0] - a[0]).hypot(b[1] - a[1]); + if len < 4.0 { + return None; + } + let (ux, uy) = ((b[0] - a[0]) / len, (b[1] - a[1]) / len); + let normal = ux.atan2(-uy) as f32; + let mut pts = Vec::new(); + // Pixels near the side, between 5% and 95% of its length (away from the corners). + let (x0, x1) = (a[0].min(b[0]).floor() as isize - 3, a[0].max(b[0]).ceil() as isize + 3); + let (y0, y1) = (a[1].min(b[1]).floor() as isize - 3, a[1].max(b[1]).ceil() as isize + 3); + for y in y0.max(0)..=y1.min(h as isize - 1) { + for x in x0.max(0)..=x1.min(w as isize - 1) { + let Some(e) = edges.get(y as usize * w + x as usize).copied().flatten() else { continue }; + let d = (e.dir - normal).rem_euclid(std::f32::consts::PI); + if d > 0.35 && d < std::f32::consts::PI - 0.35 { + continue; + } + let (dx, dy) = (x as f64 - a[0], y as f64 - a[1]); + let t = (dx * ux + dy * uy) / len; + let off = -dx * uy + dy * ux; + if (0.05..=0.95).contains(&t) && off.abs() <= 3.0 { + pts.push([x as f64, y as f64]); + } + } + } + if pts.len() < 10 { + return None; + } + let n = pts.len() as f64; + let (cx, cy) = (pts.iter().map(|p| p[0]).sum::() / n, pts.iter().map(|p| p[1]).sum::() / n); + let (mut sxx, mut sxy, mut syy) = (0.0, 0.0, 0.0); + for p in &pts { + let (dx, dy) = (p[0] - cx, p[1] - cy); + sxx += dx * dx; + sxy += dx * dy; + syy += dy * dy; + } + // The principal direction of the points. + let angle = 0.5 * (2.0 * sxy).atan2(sxx - syy); + sides.push(([cx, cy], [angle.cos(), angle.sin()])); + } + let meet = |(p, u): (Point, Point), (r, v): (Point, Point)| -> Option { + let det = u[0] * v[1] - u[1] * v[0]; + if det.abs() < 1e-6 { + return None; + } + let t = ((r[0] - p[0]) * v[1] - (r[1] - p[1]) * v[0]) / det; + let out = [p[0] + t * u[0], p[1] + t * u[1]]; + out.iter().all(|c| c.is_finite()).then_some(out) + }; + let side = |k: usize| sides.get(k).copied(); + // Corner k is where side k-1 (ending there) meets side k (starting there). + let corner = |k: usize| meet(side((k + 3) % 4)?, side(k)?); + let refined = [corner(0)?, corner(1)?, corner(2)?, corner(3)?]; + // A fit that moved a corner far is not a refinement. + let moved = refined.iter().zip(q.iter()).any(|(r, o)| (r[0] - o[0]).hypot(r[1] - o[1]) > 6.0); + if moved { + return None; + } + order_quad(refined) +} + +/// The convex quadrilateral of two near-horizontal and two near-vertical lines whose four sides +/// all lie on edges, scored by its area times the square of its sides' mean support. +fn best_quad(lines: &[Line], edges: &[Option], w: usize, h: usize) -> Option<[Point; 4]> { + // A normal within 45° of vertical makes a near-horizontal line. + let flat = |l: &&Line| (l.theta.to_degrees() - 90.0).abs() <= 45.0; + let horizontal: Vec<&Line> = lines.iter().filter(flat).collect(); + let vertical: Vec<&Line> = lines.iter().filter(|l| !flat(l)).collect(); + let (fw, fh) = (w as f64, h as f64); + let (mx, my) = (MARGIN * fw, MARGIN * fh); + let mut best: Option<(f64, [Point; 4])> = None; + for (i, &h1) in horizontal.iter().enumerate() { + for &h2 in horizontal.iter().skip(i + 1) { + for (j, &v1) in vertical.iter().enumerate() { + for &v2 in vertical.iter().skip(j + 1) { + let corners = [intersect(*h1, *v1), intersect(*h1, *v2), intersect(*h2, *v2), intersect(*h2, *v1)]; + let [Some(a), Some(b), Some(c), Some(d)] = corners else { continue }; + let Some(q) = order_quad([a, b, c, d]) else { continue }; + if q.iter().any(|p| p[0] < -mx || p[1] < -my || p[0] > fw + mx || p[1] > fh + my) { + continue; + } + let area = (0..4) + .map(|k| { + let (p, n) = (q[k], q[(k + 1) % 4]); + p[0] * n[1] - n[0] * p[1] + }) + .sum::() + .abs() + / 2.0; + if area < MIN_AREA * fw * fh { + continue; + } + let sides: Vec = (0..4).map(|k| support(q[k], q[(k + 1) % 4], edges, w, h)).collect(); + if sides.iter().any(|&s| s < MIN_SUPPORT) { + continue; + } + // Large, and with every side on its edge: a slightly larger quad with a side + // off the sheet loses to the sheet itself. + let mean = sides.iter().sum::() / 4.0; + let score = area * mean * mean; + if best.is_none_or(|(b, _)| score > b) { + best = Some((score, q)); + } + } + } + } + } + best.map(|(_, q)| q) +} + +#[cfg(test)] +mod tests { + use super::*; + + /// A deterministic pseudo-random generator for noise. + struct Lcg(u64); + impl Lcg { + fn next(&mut self) -> f32 { + self.0 = self.0.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407); + ((self.0 >> 33) as f32) / (u32::MAX >> 1) as f32 + } + } + + fn inside(q: &[Point; 4], x: f64, y: f64) -> bool { + (0..4).all(|i| { + let (a, b) = (q[i], q[(i + 1) % 4]); + (b[0] - a[0]) * (y - a[1]) - (b[1] - a[1]) * (x - a[0]) >= 0.0 + }) + } + + /// A synthetic photo: `sheet` (corners in reading order) in `paper` over `table(x, y)`, with + /// 2×2 supersampled edges, optional text lines on the sheet, and noise of ±`noise`. + fn photo(w: u32, h: u32, sheet: [Point; 4], paper: f32, table: impl Fn(f64, f64) -> f32, text: bool, noise: f32) -> Vec { + let mut rng = Lcg(7); + let at = |u: f64, v: f64| -> Point { + let top = [sheet[0][0] + (sheet[1][0] - sheet[0][0]) * u, sheet[0][1] + (sheet[1][1] - sheet[0][1]) * u]; + let bot = [sheet[3][0] + (sheet[2][0] - sheet[3][0]) * u, sheet[3][1] + (sheet[2][1] - sheet[3][1]) * u]; + [top[0] + (bot[0] - top[0]) * v, top[1] + (bot[1] - top[1]) * v] + }; + let bars: Vec<[Point; 4]> = if text { + (0..12) + .map(|k| { + let v = 0.12 + k as f64 * 0.065; + let u1 = if k % 4 == 3 { 0.5 } else { 0.88 }; + [at(0.1, v), at(u1, v), at(u1, v + 0.022), at(0.1, v + 0.022)] + }) + .collect() + } else { + Vec::new() + }; + let mut out = Vec::with_capacity((w * h * 4) as usize); + for y in 0..h { + for x in 0..w { + let mut s = 0.0; + for (ox, oy) in [(0.25, 0.25), (0.75, 0.25), (0.25, 0.75), (0.75, 0.75)] { + let (fx, fy) = (x as f64 + ox, y as f64 + oy); + s += if bars.iter().any(|b| inside(b, fx, fy)) { + 40.0 + } else if inside(&sheet, fx, fy) { + paper + } else { + table(fx, fy) + }; + } + let v = (s / 4.0 + (rng.next() - 0.5) * 2.0 * noise).clamp(0.0, 255.0) as u8; + out.extend_from_slice(&[v, v, v, 255]); + } + } + out + } + + fn assert_near(found: Option<[Point; 4]>, want: [Point; 4], tol: f64) { + let found = found.expect("the sheet should be found"); + for (f, w) in found.iter().zip(want.iter()) { + assert!((f[0] - w[0]).hypot(f[1] - w[1]) <= tol, "found {found:?}, want {want:?}"); + } + } + + const SHEET: [Point; 4] = [[150.0, 170.0], [760.0, 110.0], [820.0, 1050.0], [100.0, 1090.0]]; + + #[test] + fn finds_a_tilted_sheet_on_a_plain_table() { + let img = photo(900, 1200, SHEET, 245.0, |_, _| 95.0, false, 0.0); + assert_near(detect_document(&img, 900, 1200), SHEET, 6.0); + } + + #[test] + fn text_on_the_sheet_does_not_shrink_the_quad() { + let img = photo(900, 1200, SHEET, 245.0, |_, _| 95.0, true, 6.0); + assert_near(detect_document(&img, 900, 1200), SHEET, 6.0); + } + + #[test] + fn wood_grain_and_a_shadow_do_not_fool_it() { + // Horizontal grain stripes, darker towards the bottom right. + let table = |x: f64, y: f64| 110.0 + 18.0 * ((y / 23.0).sin() as f32) - (0.03 * (x + y)) as f32; + let img = photo(900, 1200, SHEET, 235.0, table, true, 8.0); + assert_near(detect_document(&img, 900, 1200), SHEET, 6.0); + } + + #[test] + fn a_card_photographed_from_an_angle() { + // Strong perspective: the far edge much shorter than the near one. + let card = [[260.0, 300.0], [660.0, 290.0], [800.0, 700.0], [120.0, 720.0]]; + let img = photo(900, 1000, card, 225.0, |_, _| 60.0, false, 5.0); + assert_near(detect_document(&img, 900, 1000), card, 6.0); + } + + #[test] + fn a_sheet_turned_by_thirty_degrees() { + // A 500 × 650 sheet turned 30° about (450, 500), corners in reading order. + let (c, s) = (30f64.to_radians().cos(), 30f64.to_radians().sin()); + let turn = |x: f64, y: f64| [450.0 + x * c - y * s, 500.0 + x * s + y * c]; + let sheet = [turn(-250.0, -325.0), turn(250.0, -325.0), turn(250.0, 325.0), turn(-250.0, 325.0)]; + let img = photo(900, 1000, sheet, 240.0, |_, _| 80.0, true, 5.0); + // Turned 30° clockwise on screen, the top-left corner is the one turned from (-250, -325). + assert_near(detect_document(&img, 900, 1000), sheet, 6.0); + } + + #[test] + fn a_sheet_running_off_the_picture_is_not_invented() { + // The bottom edge is outside the picture: either nothing, or corners on the visible + // edges — never a quadrilateral that cuts the sheet in half. + let sheet = [[150.0, 170.0], [760.0, 110.0], [830.0, 1400.0], [90.0, 1450.0]]; + let img = photo(900, 1200, sheet, 245.0, |_, _| 95.0, true, 4.0); + if let Some(q) = detect_document(&img, 900, 1200) { + let top = (q[0][1] + q[1][1]) / 2.0; + let bottom = (q[2][1] + q[3][1]) / 2.0; + assert!(top < 200.0 && bottom > 1150.0, "{q:?}"); + } + } + + #[test] + fn gives_up_without_a_document() { + let table = |x: f64, y: f64| 110.0 + 18.0 * ((y / 23.0).sin() as f32) - (0.03 * (x + y)) as f32; + let img = photo(900, 1200, [[0.0; 2]; 4], 0.0, table, false, 8.0); + assert_eq!(detect_document(&img, 900, 1200), None); + // White paper on a white desk: no edge to find. + let img = photo(900, 1200, SHEET, 246.0, |_, _| 244.0, false, 2.0); + assert_eq!(detect_document(&img, 900, 1200), None); + } + + #[test] + fn bad_input_is_none_not_a_panic() { + assert_eq!(detect_document(&[], 0, 0), None); + assert_eq!(detect_document(&[0; 40], 10, 1), None); + assert_eq!(detect_document(&[0; 64 * 64 * 4], 64, 63), None); + assert_eq!(detect_document(&vec![255; 64 * 64 * 4], 64, 64), None); + } +} diff --git a/crates/engine/src/catalog.rs b/crates/engine/src/catalog.rs index c553675c..5f2c52bc 100644 --- a/crates/engine/src/catalog.rs +++ b/crates/engine/src/catalog.rs @@ -250,6 +250,7 @@ pub static TOOL_GROUPS: &[ToolGroup] = &[ item("In this file", "scan-text", "ocr.recognize", Ready), item("In multiple files", "files", "ocr.recognize_batch", Ready), item("Enhance scanned file", "sparkles", "ocr.enhance", Planned("M10")), + item("Enhance camera image", "camera", "ocr.enhance_camera", Ready), item("Correct recognized text", "text-select", "ocr.correct", Planned("M10")), ], }], diff --git a/crates/engine/src/commands.rs b/crates/engine/src/commands.rs index 1f68eeb1..2f313bda 100644 --- a/crates/engine/src/commands.rs +++ b/crates/engine/src/commands.rs @@ -309,6 +309,7 @@ pub const COMMANDS: &[CommandSpec] = &[ c("tools.js_console", "JavaScript console…", None, Some(Shortcut::cmd("J")), Document, "square-terminal"), c("tools.document_js", "Document JavaScripts…", None, None, Modification, "file-code"), c("ocr.recognize", "Recognize text…", None, None, Modification, "scan-text"), + c("ocr.enhance_camera", "Enhance camera image", None, None, Modification, "camera"), c("ocr.recognize_batch", "Recognize text in multiple files…", None, None, Nothing, "files"), c("a11y.check", "Check for accessibility…", None, None, Document, "accessibility"), c("a11y.report", "Open accessibility report", None, None, Document, "file-text"), diff --git a/crates/engine/src/lib.rs b/crates/engine/src/lib.rs index 585c79c8..e397ddc0 100644 --- a/crates/engine/src/lib.rs +++ b/crates/engine/src/lib.rs @@ -13,6 +13,8 @@ #![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] pub mod actions; +pub mod camera; +pub mod camera_edges; pub mod catalog; pub mod commands; pub mod compare; diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index 768e0ccb..2971396a 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -3026,3 +3026,95 @@ fn recovered_user_passwords_reach_the_renderer_byte_for_byte() { assert_eq!(crate::renderer_password(&[0x9A, b'l', 0x9A, b'k', 0xA0]), "ılık€"); assert_eq!(crate::renderer_password(&[0xC7, b'o', b'k']), "Çok"); } + +/// A synthetic phone photo, 400 × 300 px: a white sheet, tilted, on a grey table, with a black +/// mark in the sheet's top-left corner. Returns the PNG and the sheet's corners in pixels. +fn camera_photo() -> (Vec, [[f64; 2]; 4]) { + let sheet = [[60.0, 40.0], [330.0, 20.0], [350.0, 270.0], [40.0, 250.0]]; + let inside = |x: f64, y: f64| { + (0..4).all(|i| { + let (a, b) = (sheet[i], sheet[(i + 1) % 4]); + (b[0] - a[0]) * (y - a[1]) - (b[1] - a[1]) * (x - a[0]) >= 0.0 + }) + }; + let img = image::RgbImage::from_fn(400, 300, |x, y| { + let (fx, fy) = (x as f64 + 0.5, y as f64 + 0.5); + if (75.0..95.0).contains(&fx) && (55.0..75.0).contains(&fy) { + image::Rgb([0, 0, 0]) + } else if inside(fx, fy) { + image::Rgb([255, 255, 255]) + } else { + image::Rgb([110, 110, 110]) + } + }); + let mut png = Vec::new(); + img.write_to(&mut std::io::Cursor::new(&mut png), image::ImageFormat::Png).unwrap(); + (png, sheet) +} + +#[test] +fn enhance_camera_page_straightens_the_sheet() { + let mut s = Session::new().with_clock(|| 1_700_000_000); + let (png, sheet) = camera_photo(); + // 72 dpi: one point per pixel, so the page is 400 × 300 pt and y flips to user space. + let bytes = s.create_from_images_with_resolution(&[("photo.png".into(), png)], ImageResolution::Dpi(72.0)).unwrap(); + let id = s.open("photo.pdf", None, bytes, None).unwrap(); + let corners = sheet.map(|[x, y]| [x, 300.0 - y]); + // Corners in any order. + s.enhance_camera_page(id, 0, [corners[2], corners[0], corners[3], corners[1]]).unwrap(); + + let doc = s.get(id).unwrap(); + assert_eq!(doc.info.pages.len(), 1); + assert_eq!(doc.can_undo(), Some("Enhance camera image")); + // The longer of each pair of opposite sides: 310 × 250 pt at the photo's own resolution. + let p = &doc.info.pages[0]; + assert!((p.width - 310.0).abs() <= 1.0 && (p.height - 250.0).abs() <= 1.0, "{} × {}", p.width, p.height); + + let mut r = pdfcraft_render::PageRenderer::new(doc.bytes.clone(), doc.config.clone()); + let shot = r.render(pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() }); + assert!(shot.error.is_none(), "{:?}", shot.error); + let at = |x: u32, y: u32| { + let i = ((y * shot.width + x) * 4) as usize; + shot.rgba[i..i + 3].to_vec() + }; + // No table left in the corners: the sheet fills the page, and its mark is top left. + for (x, y) in [(4, 4), (shot.width - 5, 4), (shot.width - 5, shot.height - 5), (4, shot.height - 5), (shot.width / 2, shot.height / 2)] { + assert!(at(x, y).iter().all(|&v| v > 235), "({x}, {y}) = {:?}", at(x, y)); + } + assert!(at(30, 30).iter().all(|&v| v < 40), "the mark: {:?}", at(30, 30)); + + s.undo(id).unwrap(); + let p = &s.get(id).unwrap().info.pages[0]; + assert_eq!((p.width, p.height), (400.0, 300.0)); +} + +#[test] +fn detect_camera_corners_finds_the_sheet_in_user_space() { + let mut s = Session::new().with_clock(|| 1_700_000_000); + let (png, sheet) = camera_photo(); + let bytes = s.create_from_images_with_resolution(&[("photo.png".into(), png)], ImageResolution::Dpi(72.0)).unwrap(); + let id = s.open("photo.pdf", None, bytes, None).unwrap(); + let found = s.detect_camera_corners(id, 0).expect("the sheet's edges"); + // One point per pixel, y flipped: the sheet's corners in reading order. + for (f, p) in found.iter().zip(sheet.iter()) { + let want = [p[0], 300.0 - p[1]]; + assert!((f[0] - want[0]).hypot(f[1] - want[1]) < 4.0, "{found:?} vs {sheet:?}"); + } + // A page with no photo on it: nothing to find. + let id = s.open("doc.pdf", None, Arc::new(fixture(1)), None).unwrap(); + assert_eq!(s.detect_camera_corners(id, 0), None); + assert_eq!(s.detect_camera_corners(id, 7), None); +} + +#[test] +fn enhance_camera_page_refuses_bad_corners_and_pages() { + let mut s = Session::new().with_clock(|| 1_700_000_000); + let id = s.open("doc.pdf", None, Arc::new(fixture(1)), None).unwrap(); + let line = [[0.0, 0.0], [50.0, 50.0], [100.0, 100.0], [150.0, 150.0]]; + let e = s.enhance_camera_page(id, 0, line).unwrap_err().to_string(); + assert!(e.contains("corners"), "{e}"); + let ok = [[0.0, 0.0], [200.0, 0.0], [200.0, 300.0], [0.0, 300.0]]; + assert!(s.enhance_camera_page(id, 5, ok).unwrap_err().to_string().contains("no page 6")); + assert!(matches!(s.enhance_camera_page(DocId(999), 0, ok), Err(EditError::NoDocument))); + assert_eq!(s.get(id).unwrap().can_undo(), None, "nothing changed"); +} diff --git a/crates/geom/src/lib.rs b/crates/geom/src/lib.rs index b04e47cb..40f38498 100644 --- a/crates/geom/src/lib.rs +++ b/crates/geom/src/lib.rs @@ -5,6 +5,8 @@ #![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] +pub mod perspective; + /// An axis-aligned rectangle in PDF user space (points). #[derive(Clone, Copy, Debug, PartialEq, Default)] pub struct PageRect { diff --git a/crates/geom/src/perspective.rs b/crates/geom/src/perspective.rs new file mode 100644 index 00000000..96aa663e --- /dev/null +++ b/crates/geom/src/perspective.rs @@ -0,0 +1,171 @@ +//! Perspective (projective) transforms between quadrilaterals: what straightening a photographed +//! page needs. Pure maths in `f64`; every input may be hostile, so degenerate or non-finite +//! quadrilaterals give `None` instead of NaNs. + +/// A point `[x, y]`. +pub type Point = [f64; 2]; + +/// A 3×3 projective transform, row-major, normalised so that the last entry is 1. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct Homography(pub [f64; 9]); + +impl Homography { + /// The transform that takes each of `from` to the point at the same index of `to`. `None` + /// when three of either set are collinear or a coordinate is not finite. + pub fn from_points(from: [Point; 4], to: [Point; 4]) -> Option { + if from.iter().chain(to.iter()).flatten().any(|v| !v.is_finite()) { + return None; + } + // x' = (h0 x + h1 y + h2) / (h6 x + h7 y + 1), y' likewise with h3..h5: eight unknowns, + // two equations per point pair. + let mut m = [[0.0f64; 9]; 8]; + for (i, (&[x, y], &[u, v])) in from.iter().zip(to.iter()).enumerate() { + m[2 * i] = [x, y, 1.0, 0.0, 0.0, 0.0, -u * x, -u * y, u]; + m[2 * i + 1] = [0.0, 0.0, 0.0, x, y, 1.0, -v * x, -v * y, v]; + } + let h = solve8(m)?; + let out = Self([h[0], h[1], h[2], h[3], h[4], h[5], h[6], h[7], 1.0]); + // Collinear points can leave a solvable but singular system: check that the transform + // really maps the four points. + let ok = from.iter().zip(to.iter()).all(|(&p, &q)| { + out.apply(p).is_some_and(|r| { + let scale = 1.0 + q[0].abs().max(q[1].abs()); + (r[0] - q[0]).abs() <= 1e-6 * scale && (r[1] - q[1]).abs() <= 1e-6 * scale + }) + }); + ok.then_some(out) + } + + /// The image of `p`, or `None` where the transform sends it to infinity. + pub fn apply(&self, [x, y]: Point) -> Option { + let h = &self.0; + let w = h[6] * x + h[7] * y + h[8]; + if !w.is_finite() || w.abs() < 1e-12 { + return None; + } + let u = (h[0] * x + h[1] * y + h[2]) / w; + let v = (h[3] * x + h[4] * y + h[5]) / w; + (u.is_finite() && v.is_finite()).then_some([u, v]) + } +} + +/// Gaussian elimination with partial pivoting on an 8×8 system (the 9th column is the right-hand +/// side). `None` when the system is singular. +fn solve8(mut m: [[f64; 9]; 8]) -> Option<[f64; 8]> { + for col in 0..8 { + let pivot = (col..8).max_by(|&a, &b| m[a][col].abs().total_cmp(&m[b][col].abs()))?; + if m[pivot][col].abs() < 1e-12 { + return None; + } + m.swap(col, pivot); + let p = m[col]; + for (r, row) in m.iter_mut().enumerate() { + if r == col { + continue; + } + let f = row[col] / p[col]; + if f != 0.0 { + for (a, b) in row.iter_mut().zip(p.iter()) { + *a -= f * b; + } + } + } + } + let mut out = [0.0; 8]; + for (i, o) in out.iter_mut().enumerate() { + *o = m[i][8] / m[i][i]; + if !o.is_finite() { + return None; + } + } + Some(out) +} + +/// The four corners in reading order for a y-down space (view or raster pixels): top-left, +/// top-right, bottom-right, bottom-left, whatever order they were given in. `None` unless they +/// form a convex quadrilateral with a non-negligible area. +pub fn order_quad(points: [Point; 4]) -> Option<[Point; 4]> { + if points.iter().flatten().any(|v| !v.is_finite()) { + return None; + } + let cx = points.iter().map(|p| p[0]).sum::() / 4.0; + let cy = points.iter().map(|p| p[1]).sum::() / 4.0; + let mut q = points; + // Ascending angle about the centre is clockwise on screen (y down): top, right, bottom, left. + q.sort_by(|a, b| (a[1] - cy).atan2(a[0] - cx).total_cmp(&(b[1] - cy).atan2(b[0] - cx))); + let first = (0..4).min_by(|&a, &b| (q[a][0] + q[a][1]).total_cmp(&(q[b][0] + q[b][1])))?; + q.rotate_left(first); + // Convex, with every turn the same way, and not a sliver. + let cross = |o: Point, a: Point, b: Point| (a[0] - o[0]) * (b[1] - o[1]) - (a[1] - o[1]) * (b[0] - o[0]); + let turns: Vec = (0..4).map(|i| cross(q[i], q[(i + 1) % 4], q[(i + 2) % 4])).collect(); + let area = turns.iter().sum::() / 2.0; + let span = q.iter().map(|p| p[0]).fold(f64::MIN, f64::max) - q.iter().map(|p| p[0]).fold(f64::MAX, f64::min); + let rise = q.iter().map(|p| p[1]).fold(f64::MIN, f64::max) - q.iter().map(|p| p[1]).fold(f64::MAX, f64::min); + let size = span.max(rise); + let convex = turns.iter().all(|&t| t > 1e-9 * size * size); + (convex && area > 1e-6 * size * size).then_some(q) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn close(a: Point, b: Point) -> bool { + (a[0] - b[0]).abs() < 1e-6 && (a[1] - b[1]).abs() < 1e-6 + } + + #[test] + fn maps_each_corner_onto_its_target() { + let from = [[12.0, 30.0], [410.0, 5.0], [430.0, 610.0], [-4.0, 590.0]]; + let to = [[0.0, 0.0], [400.0, 0.0], [400.0, 600.0], [0.0, 600.0]]; + let h = Homography::from_points(from, to).unwrap(); + for (p, q) in from.iter().zip(to.iter()) { + assert!(close(h.apply(*p).unwrap(), *q), "{p:?} -> {q:?}"); + } + } + + #[test] + fn a_rectangle_onto_itself_is_the_identity() { + let r = [[0.0, 0.0], [10.0, 0.0], [10.0, 20.0], [0.0, 20.0]]; + let h = Homography::from_points(r, r).unwrap(); + assert!(close(h.apply([3.0, 7.0]).unwrap(), [3.0, 7.0])); + } + + #[test] + fn the_inverse_takes_points_back() { + let from = [[50.0, 40.0], [300.0, 70.0], [320.0, 400.0], [30.0, 380.0]]; + let to = [[0.0, 0.0], [200.0, 0.0], [200.0, 300.0], [0.0, 300.0]]; + let (h, back) = (Homography::from_points(from, to).unwrap(), Homography::from_points(to, from).unwrap()); + let p = [120.0, 210.0]; + assert!(close(back.apply(h.apply(p).unwrap()).unwrap(), p)); + } + + #[test] + fn collinear_or_non_finite_points_have_no_transform() { + let to = [[0.0, 0.0], [1.0, 0.0], [1.0, 1.0], [0.0, 1.0]]; + assert!(Homography::from_points([[0.0, 0.0], [1.0, 1.0], [2.0, 2.0], [3.0, 3.0]], to).is_none()); + assert!(Homography::from_points([[0.0, 0.0]; 4], to).is_none()); + assert!(Homography::from_points([[f64::NAN, 0.0], [1.0, 0.0], [1.0, 1.0], [0.0, 1.0]], to).is_none()); + assert!(Homography::from_points([[f64::INFINITY, 0.0], [1.0, 0.0], [1.0, 1.0], [0.0, 1.0]], to).is_none()); + } + + #[test] + fn corners_come_back_in_reading_order_whatever_the_input_order() { + let tl = [10.0, 12.0]; + let tr = [200.0, 5.0]; + let br = [210.0, 300.0]; + let bl = [0.0, 290.0]; + for q in [[br, tl, bl, tr], [bl, br, tr, tl], [tl, tr, br, bl], [tr, bl, tl, br]] { + assert_eq!(order_quad(q), Some([tl, tr, br, bl]), "{q:?}"); + } + } + + #[test] + fn concave_degenerate_or_non_finite_quads_are_refused() { + // A dart: one corner inside the triangle of the other three. + assert_eq!(order_quad([[0.0, 0.0], [100.0, 0.0], [20.0, 20.0], [0.0, 100.0]]), None); + assert_eq!(order_quad([[0.0, 0.0], [10.0, 0.0], [20.0, 0.0], [30.0, 0.0]]), None); + assert_eq!(order_quad([[5.0, 5.0]; 4]), None); + assert_eq!(order_quad([[f64::NAN, 0.0], [1.0, 0.0], [1.0, 1.0], [0.0, 1.0]]), None); + } +} diff --git a/crates/ui-egui/src/camera_ui.rs b/crates/ui-egui/src/camera_ui.rs new file mode 100644 index 00000000..9b2db74f --- /dev/null +++ b/crates/ui-egui/src/camera_ui.rs @@ -0,0 +1,168 @@ +//! Scan & OCR ▸ Enhance ▸ Camera image: four corner handles on a page, dragged onto the edges of +//! a photographed document; Enhance page straightens what they enclose +//! (`Session::enhance_camera_page`, one undoable step). + +use egui::{Align2, Color32, CornerRadius, Pos2, Rect, Shape, Stroke, vec2}; + +use crate::canvas::{DocView, PageXform, ViewAction}; +use crate::theme::Tokens; + +/// How far in from the page's edges the corners start, as a share of its size. +const INSET: f32 = 0.05; +/// How close (screen points) the pointer must be to pick up a corner. +const GRAB: f32 = 12.0; +const BLUE: Color32 = Color32::from_rgb(0x14, 0x73, 0xE6); + +/// The quadrilateral being placed: its page, its corners in view space (points, y down) and the +/// corner being dragged. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct CameraQuad { + pub page: usize, + pub corners: [[f32; 2]; 4], + pub grab: Option, +} + +impl CameraQuad { + /// The page `page` (view size `w` × `h` points) inset by [`INSET`] on every side. + pub fn inset(page: usize, w: f32, h: f32) -> Self { + let (dx, dy) = (w * INSET, h * INSET); + Self { page, corners: [[dx, dy], [w - dx, dy], [w - dx, h - dy], [dx, h - dy]], grab: None } + } +} + +fn screen(xf: &PageXform, c: [f32; 2]) -> Pos2 { + xf.norm_to_screen(c[0] / xf.pw.max(1e-3), c[1] / xf.ph.max(1e-3)) +} + +/// Corner dragging on one page. A press on a page without the quadrilateral moves it there. +pub(crate) fn page_input(ui: &egui::Ui, resp: &egui::Response, xf: &PageXform, page: usize, view: &mut DocView, allowed: bool) { + let pointer = ui.input(|i| i.pointer.hover_pos()); + let origin = ui.input(|i| i.pointer.press_origin()); + if !allowed { + if pointer.is_some_and(|p| xf.rect.contains(p)) { + ui.ctx().set_cursor_icon(egui::CursorIcon::NotAllowed); + } + return; + } + // Once the page is enhanced, the bar's buttons take over. + if view.camera_done.is_some() { + return; + } + // The tool starts with the corners on the current page. + if view.camera.is_none() && page == view.current { + view.camera = Some(CameraQuad::inset(page, xf.pw, xf.ph)); + } + if resp.drag_started() || resp.clicked() { + let pressed = origin.filter(|o| xf.rect.contains(*o)); + if pressed.is_some() && view.camera.is_none_or(|q| q.page != page) { + view.camera = Some(CameraQuad::inset(page, xf.pw, xf.ph)); + } + if resp.drag_started() + && let (Some(o), Some(q)) = (pressed, view.camera.as_mut()) + { + q.grab = (0..4) + .map(|i| (i, screen(xf, q.corners[i]).distance(o))) + .filter(|(_, d)| *d <= GRAB) + .min_by(|a, b| a.1.total_cmp(&b.1)) + .map(|(i, _)| i); + } + } + let Some(q) = view.camera.as_mut().filter(|q| q.page == page) else { return }; + let near = pointer.is_some_and(|p| q.corners.iter().any(|&c| screen(xf, c).distance(p) <= GRAB)); + if let Some(i) = q.grab { + ui.ctx().set_cursor_icon(egui::CursorIcon::Grabbing); + if let Some(p) = pointer { + let (x, y) = xf.screen_to_view(p); + if let Some(c) = q.corners.get_mut(i) { + *c = [x.clamp(0.0, xf.pw), y.clamp(0.0, xf.ph)]; + } + } + if resp.drag_stopped() { + q.grab = None; + } + } else if near { + ui.ctx().set_cursor_icon(egui::CursorIcon::Grab); + } +} + +/// The quadrilateral and its handles, with what will be cut away dimmed. +pub(crate) fn paint(painter: &egui::Painter, xf: &PageXform, page: usize, view: &DocView) { + let Some(q) = view.camera.filter(|q| q.page == page) else { return }; + let pts: Vec = q.corners.iter().map(|&c| screen(xf, c)).collect(); + // Dim the page outside the quadrilateral: a band from each edge out to the page's border. + let r = xf.rect; + let mut mesh = egui::Mesh::default(); + let dim = Color32::from_black_alpha(70); + let outer = [r.left_top(), r.right_top(), r.right_bottom(), r.left_bottom()]; + // Pair each quad corner with the page corner nearest to it, in order around the page. + let start = (0..4).min_by(|&a, &b| pts[0].distance_sq(outer[a]).total_cmp(&pts[0].distance_sq(outer[b]))).unwrap_or(0); + for i in 0..4 { + let (a, b) = (pts[i], pts[(i + 1) % 4]); + let (oa, ob) = (outer[(start + i) % 4], outer[(start + i + 1) % 4]); + let base = mesh.vertices.len() as u32; + for p in [a, b, ob, oa] { + mesh.colored_vertex(p, dim); + } + mesh.add_triangle(base, base + 1, base + 2); + mesh.add_triangle(base, base + 2, base + 3); + } + painter.add(Shape::mesh(mesh)); + painter.add(Shape::closed_line(pts.clone(), Stroke::new(1.5, BLUE))); + for p in pts { + painter.circle(p, 6.0, Color32::WHITE, Stroke::new(2.0, BLUE)); + } +} + +/// The floating bar at the top right of the document area (where the find bar goes, clear of a +/// centred page's corners): Enhance page and Cancel; once the page is enhanced, rotate it either +/// way, Crop pages, Organize pages, or Done. The instructions are a notice (`ocr.enhance_camera`). +pub(crate) fn bar(ctx: &egui::Context, view: &mut DocView, area: Rect, t: &Tokens, allowed: bool) { + egui::Area::new(egui::Id::new("camera-bar")) + .order(egui::Order::Middle) + .pivot(Align2::RIGHT_TOP) + .fixed_pos(area.right_top() + vec2(-18.0, 12.0)) + .show(ctx, |ui| { + egui::Frame::NONE + .fill(t.card) + .stroke(Stroke::new(1.0, t.border)) + .corner_radius(CornerRadius::same(10)) + .shadow(egui::Shadow { offset: [0, 4], blur: 16, spread: 0, color: Color32::from_black_alpha(if t.dark() { 90 } else { 30 }) }) + .inner_margin(egui::Margin::symmetric(12, 8)) + .show(ui, |ui| { + ui.horizontal(|ui| { + // After Enhance page: put the page the right way up, trim it, or move on. + if view.camera_done.is_some() { + let icon = |ui: &mut egui::Ui, name: &str, tip: &str| { + let r = ui.add_enabled(allowed, egui::Button::image(crate::icons::image(name, 18.0, t.text)).frame(false)); + // Named for screen readers (and tests), not only on hover. + r.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, allowed, tip)); + r.on_hover_text(tip).clicked() + }; + if icon(ui, "rotate-ccw", tl!("Rotate counterclockwise")) { + view.pending_action = Some(ViewAction::CameraRotate { clockwise: false }); + } + if icon(ui, "rotate-cw", tl!("Rotate clockwise")) { + view.pending_action = Some(ViewAction::CameraRotate { clockwise: true }); + } + if crate::widgets::pill_button(ui, tl!("Crop pages"), false).clicked() { + view.pending_action = Some(ViewAction::CameraNext("page.crop")); + } + if crate::widgets::pill_button(ui, tl!("Organize pages"), false).clicked() { + view.pending_action = Some(ViewAction::CameraNext("page.organize")); + } + if crate::widgets::pill_button(ui, tl!("Done"), true).clicked() { + view.pending_action = Some(ViewAction::CancelCamera); + } + return; + } + let ready = allowed && view.camera.is_some(); + if ui.add_enabled_ui(ready, |ui| crate::widgets::pill_button(ui, tl!("Enhance page"), true)).inner.clicked() { + view.pending_action = Some(ViewAction::EnhanceCamera); + } + if crate::widgets::pill_button(ui, tl!("Cancel"), false).clicked() { + view.pending_action = Some(ViewAction::CancelCamera); + } + }); + }); + }); +} diff --git a/crates/ui-egui/src/canvas.rs b/crates/ui-egui/src/canvas.rs index bf66f873..f5e1b00e 100644 --- a/crates/ui-egui/src/canvas.rs +++ b/crates/ui-egui/src/canvas.rs @@ -373,6 +373,11 @@ pub struct DocView { pub pending_redaction: Option<(usize, Vec<[f64; 8]>)>, /// A crop rectangle being dragged (Crop tool). pub crop_drag: crate::crop::CropDrag, + /// Enhance ▸ Camera image: the quadrilateral being placed. + pub camera: Option, + /// Enhance ▸ Camera image: the page just enhanced, while the bar offers rotate, crop and + /// organize. + pub camera_done: Option, /// Use a certificate: a signature rectangle being drawn, or an empty signature field clicked. pub sign: crate::sign_ui::SignView, /// Organize: the pages being dragged to a new place. @@ -446,6 +451,16 @@ pub enum ViewAction { }, /// Paste copied pages after the selection. PastePages, + /// Enhance ▸ Camera image: straighten what the corners enclose. + EnhanceCamera, + /// Leave Enhance ▸ Camera image (Cancel before Enhance page, Done after it). + CancelCamera, + /// After Enhance page: turn the enhanced page a quarter. + CameraRotate { + clockwise: bool, + }, + /// After Enhance page: leave the tool for this command (Crop pages, Organize pages). + CameraNext(&'static str), } impl DocView { @@ -565,6 +580,8 @@ impl DocView { redact_drag: None, pending_redaction: None, crop_drag: None, + camera: None, + camera_done: None, sign: Default::default(), org_drag: None, panel_drag: None, @@ -1677,7 +1694,9 @@ pub fn shortcuts(view: &mut DocView, ctx: &egui::Context) { } let key = |k| ctx.input(|i| i.key_pressed(k)); if key(Key::Escape) { - if view.selection.is_some() { + if view.camera.is_some() || view.camera_done.is_some() { + view.pending_action = Some(ViewAction::CancelCamera); + } else if view.selection.is_some() { view.selection = None; } else { view.find = None; @@ -1925,6 +1944,7 @@ pub fn document_area(app: &mut PdfCraftApp, index: usize, ui: &mut egui::Ui) { QuickTool::Hand | QuickTool::Measure(_) | QuickTool::Crop + | QuickTool::EnhanceCamera | QuickTool::Fill(_) | QuickTool::Field(_) | QuickTool::AddText @@ -2247,6 +2267,9 @@ pub fn document_area(app: &mut PdfCraftApp, index: usize, ui: &mut egui::Ui) { if matches!(tool, QuickTool::MarqueeZoom | QuickTool::Snapshot) { crate::zoom_snap::page_input(ui, &resp, &xf, i, view); } + if tool == QuickTool::EnhanceCamera { + crate::camera_ui::page_input(ui, &resp, &xf, i, view, can_modify); + } if tool == QuickTool::Crop && crate::crop::page_input(ui, &resp, &xf, i, info, view, can_crop) { view.current = i; open_boxes = true; @@ -2438,6 +2461,9 @@ pub fn document_area(app: &mut PdfCraftApp, index: usize, ui: &mut egui::Ui) { if tool == QuickTool::EditText { crate::object_ui::paint(painter, &xf, i, info, &view.objects); } + if tool == QuickTool::EnhanceCamera { + crate::camera_ui::paint(painter, &xf, i, view); + } if tool == QuickTool::Link { crate::link_ui::paint(ui, painter, &xf, i, info, &doc_links, view); } @@ -2649,6 +2675,9 @@ pub fn document_area(app: &mut PdfCraftApp, index: usize, ui: &mut egui::Ui) { }); } find_bar(view, info.pages.len(), avail, ui, &t); + if tool == QuickTool::EnhanceCamera { + crate::camera_ui::bar(ui.ctx(), view, avail, &t, can_modify); + } if let Some(e) = comments::composer(ui.ctx(), view, info, prefs) { view.pending_edit = Some(e); } diff --git a/crates/ui-egui/src/commands.rs b/crates/ui-egui/src/commands.rs index 8ccf1456..f4886f94 100644 --- a/crates/ui-egui/src/commands.rs +++ b/crates/ui-egui/src/commands.rs @@ -570,6 +570,23 @@ impl PdfCraftApp { "page.duplicate" => { self.apply_edit(Edit::DuplicatePages { pages: targets }); } + "ocr.enhance_camera" => { + // The corners start on the document's edges when they can be found, else + // inset on the current page (camera_ui::page_input). + self.quick_tool = crate::QuickTool::EnhanceCamera; + if let Some(i) = self.active + && let Some(view) = self.views.get_mut(i) + && let Some(page) = self.session.get(view.id).and_then(|d| d.info.pages.get(view.current).cloned()) + { + view.camera_done = None; + view.camera = self.session.detect_camera_corners(view.id, view.current).map(|found| crate::camera_ui::CameraQuad { + page: view.current, + corners: found.map(|[x, y]| page.user_to_view(x as f32, y as f32)), + grab: None, + }); + } + self.notify_tr("Drag the corners onto the document's edges, then choose Enhance page."); + } "page.crop" => { self.quick_tool = crate::QuickTool::Crop; self.notify_tr("Drag a rectangle on a page to crop it; double-click a page for Set Page Boxes"); diff --git a/crates/ui-egui/src/control.rs b/crates/ui-egui/src/control.rs index 80bfec29..bba7108c 100644 --- a/crates/ui-egui/src/control.rs +++ b/crates/ui-egui/src/control.rs @@ -678,6 +678,7 @@ impl Host for crate::PdfCraftApp { crate::QuickTool::Select => "select".to_string(), crate::QuickTool::Hand => "hand".to_string(), crate::QuickTool::Crop => "crop".to_string(), + crate::QuickTool::EnhanceCamera => "enhance-camera".to_string(), crate::QuickTool::Redact => "redact".to_string(), crate::QuickTool::AddText => "add-text".to_string(), crate::QuickTool::EditText => "edit-text".to_string(), diff --git a/crates/ui-egui/src/editing.rs b/crates/ui-egui/src/editing.rs index dedb5584..00856237 100644 --- a/crates/ui-egui/src/editing.rs +++ b/crates/ui-egui/src/editing.rs @@ -148,11 +148,60 @@ impl PdfCraftApp { Some(crate::canvas::ViewAction::Split) => self.dialog = Some(crate::Dialog::Split), Some(crate::canvas::ViewAction::CopyPages { cut }) => self.copy_pages(cut), Some(crate::canvas::ViewAction::PastePages) => self.paste_pages(), + Some(crate::canvas::ViewAction::EnhanceCamera) => self.enhance_camera(i), + Some(crate::canvas::ViewAction::CancelCamera) => self.leave_camera(i), + Some(crate::canvas::ViewAction::CameraRotate { clockwise }) => { + if let Some(page) = self.views.get(i).and_then(|v| v.camera_done) { + self.apply_edit(Edit::RotatePages { pages: vec![page], degrees: if clockwise { 90 } else { -90 } }); + } + } + Some(crate::canvas::ViewAction::CameraNext(command)) => { + self.leave_camera(i); + self.execute(command); + } None => {} } applied } + /// Leave Enhance ▸ Camera image on tab `i`. + fn leave_camera(&mut self, i: usize) { + if let Some(view) = self.views.get_mut(i) { + view.camera = None; + view.camera_done = None; + } + if self.quick_tool == crate::QuickTool::EnhanceCamera { + self.quick_tool = crate::QuickTool::Select; + } + } + + /// Enhance ▸ Camera image: straighten what the corners on tab `i` enclose. On success the + /// bar offers rotate, crop and organize for the page; on failure the corners stay so they + /// can be adjusted. + fn enhance_camera(&mut self, i: usize) { + let Some(view) = self.views.get(i) else { return }; + let (id, Some(q)) = (view.id, view.camera) else { return }; + let Some(page) = self.session.get(id).and_then(|d| d.info.pages.get(q.page).cloned()) else { return }; + let corners = q.corners.map(|[x, y]| { + let [u, v] = page.view_to_user(x, y); + [u as f64, v as f64] + }); + match self.session.enhance_camera_page(id, q.page, corners) { + Ok(()) => { + // The tool stays open with rotate, crop and organize for the enhanced page. + if let (Some(doc), Some(view)) = (self.session.get(id), self.views.get_mut(i)) { + view.document_changed(&doc.info); + view.camera = None; + view.camera_done = Some(q.page); + } + } + Err(e) => { + let label = crate::i18n::action_label("Enhance camera image"); + self.notify_fmt("{label} failed: {e}", &[("label", &label), ("e", &e.to_string())]); + } + } + } + /// Organize ▸ Copy / Cut: remember the selected pages (the document as it is now); Cut also /// deletes them (one page always stays). pub fn copy_pages(&mut self, cut: bool) { diff --git a/crates/ui-egui/src/i18n/ar.tsv b/crates/ui-egui/src/i18n/ar.tsv index 66607f03..26c88bdb 100644 --- a/crates/ui-egui/src/i18n/ar.tsv +++ b/crates/ui-egui/src/i18n/ar.tsv @@ -2221,3 +2221,6 @@ key Space مسافة Panels اللوحات Resize panels تغيير حجم اللوحات Resize panel window تغيير حجم نافذة اللوحة + Enhance camera image تحسين صورة الكاميرا + Drag the corners onto the document's edges, then choose Enhance page. اسحب الزوايا إلى حواف المستند، ثم اختر تحسين الصفحة. + Enhance page تحسين الصفحة diff --git a/crates/ui-egui/src/i18n/bg.tsv b/crates/ui-egui/src/i18n/bg.tsv index e78dc818..8cb3d9a8 100644 --- a/crates/ui-egui/src/i18n/bg.tsv +++ b/crates/ui-egui/src/i18n/bg.tsv @@ -2163,3 +2163,6 @@ key Space Интервал Resize panel window Преоразмери прозореца на панела Tools Инструменти Inspector Инспектор + Enhance camera image Подобри снимка от камера + Drag the corners onto the document's edges, then choose Enhance page. Плъзнете ъглите до краищата на документа, след това изберете „Подобри страницата“. + Enhance page Подобри страницата diff --git a/crates/ui-egui/src/i18n/de.tsv b/crates/ui-egui/src/i18n/de.tsv index 00b3568e..46e4f516 100644 --- a/crates/ui-egui/src/i18n/de.tsv +++ b/crates/ui-egui/src/i18n/de.tsv @@ -2201,3 +2201,6 @@ key Space Leertaste Resize panel window Größe des Bereichsfensters ändern Tools Werkzeuge Inspector Inspektor + Enhance camera image Kamerafoto verbessern + Drag the corners onto the document's edges, then choose Enhance page. Ziehen Sie die Ecken auf die Kanten des Dokuments und wählen Sie dann Seite verbessern. + Enhance page Seite verbessern diff --git a/crates/ui-egui/src/i18n/es.tsv b/crates/ui-egui/src/i18n/es.tsv index 148c030e..fd5d6295 100644 --- a/crates/ui-egui/src/i18n/es.tsv +++ b/crates/ui-egui/src/i18n/es.tsv @@ -2116,3 +2116,6 @@ key Space Espacio Resize panel window Cambiar tamaño de la ventana del panel Tools Herramientas Inspector Inspector + Enhance camera image Mejorar foto de cámara + Drag the corners onto the document's edges, then choose Enhance page. Arrastre las esquinas hasta los bordes del documento y elija Mejorar página. + Enhance page Mejorar página diff --git a/crates/ui-egui/src/i18n/fr.tsv b/crates/ui-egui/src/i18n/fr.tsv index c7dc5e51..c334375d 100644 --- a/crates/ui-egui/src/i18n/fr.tsv +++ b/crates/ui-egui/src/i18n/fr.tsv @@ -2201,3 +2201,6 @@ key Space Espace Resize panel window Redimensionner la fenêtre du panneau Tools Outils Inspector Inspecteur + Enhance camera image Améliorer la photo d'un document + Drag the corners onto the document's edges, then choose Enhance page. Faites glisser les coins sur les bords du document, puis choisissez Améliorer la page. + Enhance page Améliorer la page diff --git a/crates/ui-egui/src/i18n/hu.tsv b/crates/ui-egui/src/i18n/hu.tsv index 8a239f87..03e51c58 100644 --- a/crates/ui-egui/src/i18n/hu.tsv +++ b/crates/ui-egui/src/i18n/hu.tsv @@ -2199,3 +2199,6 @@ key Space Szóköz Resize panel window Panelablak átméretezése Tools Eszközök Inspector Tulajdonságvizsgáló + Enhance camera image Kamerakép javítása + Drag the corners onto the document's edges, then choose Enhance page. Húzza a sarkokat a dokumentum széleire, majd válassza az Oldal javítása lehetőséget. + Enhance page Oldal javítása diff --git a/crates/ui-egui/src/i18n/it.tsv b/crates/ui-egui/src/i18n/it.tsv index 5ea6b214..774756df 100644 --- a/crates/ui-egui/src/i18n/it.tsv +++ b/crates/ui-egui/src/i18n/it.tsv @@ -2208,3 +2208,6 @@ key Space Spazio Panels Pannelli Resize panels Ridimensiona pannelli Resize panel window Ridimensiona finestra pannello + Enhance camera image Migliora foto della fotocamera + Drag the corners onto the document's edges, then choose Enhance page. Trascina gli angoli sui bordi del documento, poi scegli Migliora la pagina. + Enhance page Migliora la pagina diff --git a/crates/ui-egui/src/i18n/ja.tsv b/crates/ui-egui/src/i18n/ja.tsv index a4132e7d..f8dead10 100644 --- a/crates/ui-egui/src/i18n/ja.tsv +++ b/crates/ui-egui/src/i18n/ja.tsv @@ -2201,3 +2201,6 @@ key Space スペース Resize panel window パネルウィンドウのサイズを変更 Tools ツール Inspector インスペクター + Enhance camera image カメラ画像を補正 + Drag the corners onto the document's edges, then choose Enhance page. 四隅を文書の端までドラッグしてから、「ページを補正」を選択します。 + Enhance page ページを補正 diff --git a/crates/ui-egui/src/i18n/pl.tsv b/crates/ui-egui/src/i18n/pl.tsv index 2f37f0a6..e4b574c0 100644 --- a/crates/ui-egui/src/i18n/pl.tsv +++ b/crates/ui-egui/src/i18n/pl.tsv @@ -2195,3 +2195,6 @@ key Space Spacja Text used as a clipping path cannot be moved with this selection. Tekstu używanego jako ścieżka przycinania nie można przenieść razem z tym zaznaczeniem. These objects could not be moved. Select them again and try a smaller move. Nie udało się przenieść tych obiektów. Zaznacz je ponownie i spróbuj mniejszego przesunięcia. This page contains objects that cannot be selected together. Ta strona zawiera obiekty, których nie można zaznaczyć razem. + Enhance camera image Popraw zdjęcie z aparatu + Drag the corners onto the document's edges, then choose Enhance page. Przeciągnij narożniki na krawędzie dokumentu, a następnie wybierz Popraw stronę. + Enhance page Popraw stronę diff --git a/crates/ui-egui/src/i18n/pt-br.tsv b/crates/ui-egui/src/i18n/pt-br.tsv index e519585d..a3b7e9d0 100644 --- a/crates/ui-egui/src/i18n/pt-br.tsv +++ b/crates/ui-egui/src/i18n/pt-br.tsv @@ -2202,3 +2202,6 @@ key Space Espaço Resize panel window Redimensionar janela do painel Tools Ferramentas Inspector Inspetor + Enhance camera image Aprimorar foto da câmera + Drag the corners onto the document's edges, then choose Enhance page. Arraste os cantos até as bordas do documento e escolha Aprimorar página. + Enhance page Aprimorar página diff --git a/crates/ui-egui/src/i18n/ru.tsv b/crates/ui-egui/src/i18n/ru.tsv index ec1fc8a8..4358eb3f 100644 --- a/crates/ui-egui/src/i18n/ru.tsv +++ b/crates/ui-egui/src/i18n/ru.tsv @@ -2114,3 +2114,6 @@ key Space Пробел Resize panel window Изменить размер окна панели Tools Инструменты Inspector Инспектор + Enhance camera image Улучшить снимок с камеры + Drag the corners onto the document's edges, then choose Enhance page. Перетащите углы к краям документа, затем выберите «Улучшить страницу». + Enhance page Улучшить страницу diff --git a/crates/ui-egui/src/i18n/te.tsv b/crates/ui-egui/src/i18n/te.tsv index 7ed43414..8eefecea 100644 --- a/crates/ui-egui/src/i18n/te.tsv +++ b/crates/ui-egui/src/i18n/te.tsv @@ -2130,3 +2130,6 @@ key Space స్పేస్ Resize panel window ప్యానెల్ విండో పరిమాణం మార్చు Tools సాధనాలు Inspector పరిశీలకుడు + Enhance camera image కెమెరా చిత్రాన్ని మెరుగుపరచు + Drag the corners onto the document's edges, then choose Enhance page. మూలలను పత్రం అంచుల వరకు లాగి, ఆపై పేజీని మెరుగుపరచు ఎంచుకోండి. + Enhance page పేజీని మెరుగుపరచు diff --git a/crates/ui-egui/src/i18n/uk.tsv b/crates/ui-egui/src/i18n/uk.tsv index 9f7e31b1..018a6ee7 100644 --- a/crates/ui-egui/src/i18n/uk.tsv +++ b/crates/ui-egui/src/i18n/uk.tsv @@ -2222,3 +2222,6 @@ key Space Пробіл Panels Панелі Resize panels Змінити розмір панелей Resize panel window Змінити розмір вікна панелі + Enhance camera image Поліпшити знімок з камери + Drag the corners onto the document's edges, then choose Enhance page. Перетягніть кути до країв документа, потім виберіть «Поліпшити сторінку». + Enhance page Поліпшити сторінку diff --git a/crates/ui-egui/src/i18n/zh-hans.tsv b/crates/ui-egui/src/i18n/zh-hans.tsv index fa9c4f16..b6f83a99 100644 --- a/crates/ui-egui/src/i18n/zh-hans.tsv +++ b/crates/ui-egui/src/i18n/zh-hans.tsv @@ -2165,3 +2165,6 @@ key Space 空格 Resize panel window 调整面板窗口大小 Tools 工具 Inspector 检查器 + Enhance camera image 增强相机图像 + Drag the corners onto the document's edges, then choose Enhance page. 将四个角拖到文档边缘，然后选择“增强页面”。 + Enhance page 增强页面 diff --git a/crates/ui-egui/src/i18n/zh-hant.tsv b/crates/ui-egui/src/i18n/zh-hant.tsv index 7c38b012..b1a97560 100644 --- a/crates/ui-egui/src/i18n/zh-hant.tsv +++ b/crates/ui-egui/src/i18n/zh-hant.tsv @@ -2087,3 +2087,6 @@ key Space 空格 Tools 工具 Inspector 檢查器 Table rows belong directly in a table or its head, body or foot. Fix the table's tags. 表格列應直接位於表格或其表頭、主體、表尾內。請修正表格標記。 + Enhance camera image 增強相機影像 + Drag the corners onto the document's edges, then choose Enhance page. 將四個角拖曳到文件邊緣，然後選擇「增強頁面」。 + Enhance page 增強頁面 diff --git a/crates/ui-egui/src/lib.rs b/crates/ui-egui/src/lib.rs index 7ccd7777..2b9fd04f 100644 --- a/crates/ui-egui/src/lib.rs +++ b/crates/ui-egui/src/lib.rs @@ -31,6 +31,7 @@ pub use combine_grid::ThumbState as CombineThumb; mod combine_ui; pub mod docking; pub use combine_ui::{Columns as CombineColumns, CombineView, Lock as CombineLock, SortKey}; +mod camera_ui; mod commands; mod comment_props; pub mod comments; @@ -186,6 +187,8 @@ pub enum QuickTool { Measure(measure_ui::Tool), /// Crop pages by dragging a rectangle. Crop, + /// Scan & OCR ▸ Enhance ▸ Camera image: drag four corners, then Enhance page. + EnhanceCamera, /// A Fill & Sign tool. Fill(fill_sign::FillTool), /// A Prepare a form field tool. @@ -1787,6 +1790,7 @@ impl PdfCraftApp { ), "hand" => QuickTool::Hand, "crop" => QuickTool::Crop, + "enhance-camera" => QuickTool::EnhanceCamera, "redact" => QuickTool::Redact, "add-text" => QuickTool::AddText, "edit-text" => QuickTool::EditText, diff --git a/crates/ui-egui/tests/camera.rs b/crates/ui-egui/tests/camera.rs new file mode 100644 index 00000000..3555a850 --- /dev/null +++ b/crates/ui-egui/tests/camera.rs @@ -0,0 +1,125 @@ +//! Scan & OCR ▸ Enhance ▸ Camera image in the real shell (egui_kittest): drag a corner, then +//! Enhance page; Cancel and Esc leave the page as it was. + +use egui_kittest::Harness; +use egui_kittest::kittest::Queryable; +use pdfcraft_ui_egui::{PdfCraftApp, QuickTool}; + +const FIXTURE: &[u8] = b"%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 /MediaBox [0 0 300 400] >> endobj +3 0 obj << /Type /Page /Parent 2 0 R >> endobj +4 0 obj << /Type /Page /Parent 2 0 R >> endobj +trailer << /Root 1 0 R >> +%%EOF"; + +fn harness() -> Harness<'static, PdfCraftApp> { + let mut h = Harness::builder().with_size(egui::vec2(1400.0, 900.0)).build_eframe(|_cc| { + let mut app = PdfCraftApp::new(); + app.set_option("language", "en").unwrap(); + app.open_bytes("doc.pdf", None, FIXTURE.to_vec()).unwrap(); + app.set_option("left", "closed").unwrap(); + app.set_option("zoom", "100").unwrap(); + app + }); + h.run_steps(6); + h +} + +fn size(h: &Harness<'static, PdfCraftApp>, page: usize) -> (f32, f32) { + let s = h.state(); + let p = &s.session.get(s.views[0].id).unwrap().info.pages[page]; + (p.width, p.height) +} + +fn undo_label(h: &Harness<'static, PdfCraftApp>) -> Option { + let s = h.state(); + s.session.get(s.views[0].id).unwrap().can_undo().map(str::to_owned) +} + +#[test] +fn dragging_a_corner_then_enhance_page_straightens_the_page() { + let mut h = harness(); + assert!(h.state_mut().execute("ocr.enhance_camera")); + assert_eq!(h.state().quick_tool, QuickTool::EnhanceCamera); + h.run_steps(3); + let quad = h.state().views[0].camera.expect("the corners start on the current page"); + assert_eq!(quad.page, 0); + assert_eq!(quad.corners, [[15.0, 20.0], [285.0, 20.0], [285.0, 380.0], [15.0, 380.0]], "inset by 5%"); + + // Drag the top-left corner to (60, 80) on the page. + let r = h.state().views[0].page_screen_rect(0).unwrap(); + let (a, b) = (r.min + r.size() * 0.05, r.min + r.size() * 0.2); + h.hover_at(a); + h.run_steps(1); + h.drag_at(a); + h.run_steps(1); + for k in 1..=4 { + h.hover_at(a + (b - a) * (k as f32 / 4.0)); + h.run_steps(1); + } + h.drop_at(b); + h.run_steps(2); + let c = h.state().views[0].camera.unwrap().corners[0]; + assert!((c[0] - 60.0).abs() < 1.0 && (c[1] - 80.0).abs() < 1.0, "{c:?}"); + assert_eq!(undo_label(&h), None, "dragging changes nothing yet"); + + h.get_by_label("Enhance page").click(); + h.run_steps(4); + // The longer of each pair of opposite sides: 270 × 360 pt. + let (w, hh) = size(&h, 0); + assert!((w - 270.0).abs() < 2.0 && (hh - 360.0).abs() < 2.0, "{w}×{hh}"); + assert_eq!(size(&h, 1), (300.0, 400.0), "only the page with the corners changes"); + assert_eq!(undo_label(&h).as_deref(), Some("Enhance camera image")); + // The tool stays open for the enhanced page: turn it, then Done. + assert_eq!(h.state().quick_tool, QuickTool::EnhanceCamera); + assert!(h.state().views[0].camera.is_none()); + assert_eq!(h.state().views[0].camera_done, Some(0)); + h.get_by_label("Rotate clockwise").click(); + h.run_steps(4); + let (w, hh) = size(&h, 0); + assert!((w - 360.0).abs() < 2.0 && (hh - 270.0).abs() < 2.0, "{w}×{hh}"); + assert_eq!(undo_label(&h).as_deref(), Some("Rotate page")); + h.get_by_label("Done").click(); + h.run_steps(3); + assert_eq!(h.state().quick_tool, QuickTool::Select); + assert_eq!(h.state().views[0].camera_done, None); +} + +#[test] +fn after_enhance_page_crop_and_organize_take_over() { + for (button, then) in [("Crop pages", QuickTool::Crop), ("Organize pages", QuickTool::Select)] { + let mut h = harness(); + assert!(h.state_mut().execute("ocr.enhance_camera")); + h.run_steps(3); + h.get_by_label("Enhance page").click(); + h.run_steps(4); + assert_eq!(h.state().views[0].camera_done, Some(0)); + h.get_by_label(button).click(); + h.run_steps(3); + assert_eq!(h.state().quick_tool, then, "{button}"); + assert_eq!(h.state().views[0].camera_done, None, "{button}"); + assert_eq!(h.state().views[0].organize, button == "Organize pages"); + } +} + +#[test] +fn cancel_and_escape_leave_the_page_alone() { + let mut h = harness(); + assert!(h.state_mut().execute("ocr.enhance_camera")); + h.run_steps(3); + h.get_by_label("Cancel").click(); + h.run_steps(3); + assert_eq!(h.state().quick_tool, QuickTool::Select); + assert!(h.state().views[0].camera.is_none()); + + assert!(h.state_mut().execute("ocr.enhance_camera")); + h.run_steps(3); + assert!(h.state().views[0].camera.is_some()); + h.key_press(egui::Key::Escape); + h.run_steps(3); + assert_eq!(h.state().quick_tool, QuickTool::Select); + assert!(h.state().views[0].camera.is_none()); + assert_eq!(size(&h, 0), (300.0, 400.0)); + assert_eq!(undo_label(&h), None); +} diff --git a/docs/target-app-parity.md b/docs/target-app-parity.md index 2c46958a..45e5ba61 100644 --- a/docs/target-app-parity.md +++ b/docs/target-app-parity.md @@ -204,7 +204,7 @@ full parity in that area. | F Forms and JavaScript | 79 | 58 | 8 | 78.5% | 65% | 40–75 | Fill and author every AcroForm field type, Acrobat's AF functions in Acrobat's event order, sandboxed JavaScript (boa), data exchange, Fill & Sign; XFA static/dynamic layout, FormCalc and XFA data (partial). Missing: wider JavaScript object model, document actions, submit, debugger, barcode fields, XFA flattening and full dynamic fidelity | | G Protect, redact, sanitize | 41 | 23 | 2 | 58.5% | 55% | 15–30 | Passwords and permissions, Remove security, redaction that removes glyphs, pixels, vectors, annotations and tags, with verification; word lists, patterns, FOIA/Privacy Act codes; sanitize. Missing: certificate security, pattern locales, custom code sets, partial-word redaction, folder redaction, protected view | | H Digital signatures | 45 | 23 | 10 | 62.2% | 45% | 40–70 | PAdES B-B signing and validation (PKCS #12, macOS Keychain, Windows store with PIN prompts), certification, encrypted documents, certificate usage checks; B-T, LTV, OCSP/CRL with caller-supplied evidence. Missing: PKCS #11 and smart cards outside Windows, timestamp-server preferences, online revocation fetching, FieldMDP and field locking, OS trust store, EU Trusted Lists, compare signed version | -| I Scan and OCR | 26 | 5 | 2 | 23.1% | 15% | 40–75 | Searchable image (ocrs, Latin script without accents) on pages, ranges and many files. Missing: every other language (#744 Dutch; accents, CJK, Cyrillic, Arabic), editable-text output, deskew, rotation, despeckle, camera images, MRC compression, scanners | +| I Scan and OCR | 26 | 5 | 2 | 23.1% | 15% | 40–75 | Searchable image (ocrs, Latin script without accents) on pages, ranges and many files. Missing: every other language (#744 Dutch; accents, CJK, Cyrillic, Arabic), editable-text output, deskew, rotation, despeckle, camera images beyond straightening (found or placed corners): orientation, clean-up, MRC compression, scanners | | J Create and export | 34 | 12 | 3 | 39.7% | 25% | 55–100 | Create from images, multipage TIFF, text, clipboard, blank, many files at once; export PNG/JPEG/TIFF, text, RTF; Word and HTML partial. Missing: Office/HTML/web/PostScript to PDF, Excel and PowerPoint export, Word layout fidelity (#773), SVG/XML/EPS/PS/JPEG 2000 export, accessible text | | K Optimize and standards | 58 | 15 | 10 | 34.5% | 20% | 65–115 | Reduce File Size and the Optimizer (images, discards, clean-up, space audit); PDF/A-2b/3b verify and convert (partial). Missing: Preflight (profiles, fixups, reports), PDF/A-1/4 and a/u levels, PDF/X, PDF/UA, PDF/E, PDF/VT, transparency flattening, font unembedding, monochrome JBIG2/CCITT | | L Print | 39 | 15 | 1 | 39.7% | 35% | 30–55 | Acrobat's sizing, n-up, booklet, poster, comments & forms, preview, print-ready PDF, CUPS on macOS/Linux and the Windows spooler (#756). Missing: **printing on the web**, print as image, PostScript, all print-production tools (output preview, separations, ink manager, printer marks, colour conversion) | diff --git a/parity/acrobat-features.toml b/parity/acrobat-features.toml index f4799655..3e312cf0 100644 --- a/parity/acrobat-features.toml +++ b/parity/acrobat-features.toml @@ -6591,11 +6591,11 @@ title = "Enhance camera images (edges, perspective, contrast)" area = "I" tier = "P1" milestone = "M10" -status = "planned" -commands = [] -tools = [] -evidence = [] -notes = "" +status = "partial" +commands = ["ocr.enhance_camera"] +tools = ["page_enhance_camera"] +evidence = ["crates/engine/src/tests.rs::enhance_camera_page_straightens_the_sheet", "crates/automation/tests/automation.rs::enhance_camera_image_through_tools", "crates/ui-egui/tests/camera.rs::dragging_a_corner_then_enhance_page_straightens_the_page", "crates/ui-egui/tests/camera.rs::after_enhance_page_crop_and_organize_take_over", "crates/engine/src/tests.rs::detect_camera_corners_finds_the_sheet_in_user_space", "crates/automation/tests/automation.rs::enhance_camera_image_finds_the_edges_by_itself"] +notes = "Four corners (any order, a convex quadrilateral) are warped to an upright page at the picture's own resolution, in one undoable step; comments, links and bookmarks stay. Scan & OCR ▸ Enhance camera image shows four corner handles on the current page (Enhance page / Cancel / Esc), then offers rotate either way, Crop pages and Organize pages for the enhanced page; page_enhance_camera does the same headlessly. The corners start on the document's edges when they are found (straight lines with an edge along most of each side, refined by a line fit; tested on synthetic photos with text, wood grain, shadows, strong perspective and a 30° turn), else inset 5%. Missing: turning the result upright (text orientation), rounded-corner cards keep their background in the corners, contrast and background cleanup, document types." [[feature]] id = "ocr.camera-document-type"