M10: edit OCR text on scanned pages so the change shows (#1397)
* M10: edit OCR text on scanned pages so the change shows Editing a recognised word on a scan rewrote it as invisible OCR text, so the page still showed the old word, and the whole OCR layer grouped into one paragraph that rewrapped when one word changed. - An edited invisible paragraph is drawn visibly, over a cover in the paper colour sampled around it, in the scan's sampled ink colour (or the chosen colour). - OCR words group into phrases: a table cell or a tab stop splits them and each scanned line stays apart; a phrase grows rightwards. - Styled rewrites measure with the text matrix's horizontal scale, so stretched OCR words keep their widths instead of overlapping. - The inline editor covers the scanned word and grows like the result. * M10: size OCR text from each word's letters A word box from recognition is tight around the ink, but the text layer assumed every box spans Helvetica's ascender-descender height. A capitals-only word such as "DT" spans only the cap height, so its text was set about 23% too small and its baseline floated up; editing it then drew the new word visibly smaller than the scanned one. The em and baseline now come from the word's letters: cap height or x-height on top, descender or baseline at the bottom. On a real scanned form, a rewritten "DT" -> "DSI" now matches the scanned word's height and baseline. The engine OCR-edit tests build their words the way a scan would box them. * Edit OCR text: sample colours only for invisible blocks, from the current bytes Session::apply looked up every edited block with a fresh text_blocks pass and re-serialized the whole document to render the sampling tile. Text blocks are now cached per page on the editor (keyed on the unmodified document's source identity, so any edit, undo or save invalidates them), visible blocks skip sampling entirely, and the tile renders from the document's current bytes. Adds a visible stretched-text regression test and documents that editing any invisible text makes it visible over a painted cover. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Edit OCR text: name the cached text-blocks tuple (clippy), rustfmt Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Brandon Thomas <echelon@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Tangu Achilis
GitHub
parent
a259af95c3
commit
5cce1b247f
|
|
@ -140,6 +140,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
|
|||
|
||||
## Progress log
|
||||
|
||||
- **2026-10-12 (M10, editing OCR text on scans):** Editing a recognised word on a scanned page now changes what the page shows. Before, the rewrite stayed invisible OCR text, so the scan still showed the old word; on a form, the whole OCR layer also grouped into one paragraph that rewrapped when one word changed. OCR words now group into phrases (a table cell or a tab stop splits them, and lines stay apart); an edit draws the phrase visibly in the scan's sampled ink colour over a cover in the sampled paper colour, keeping each word's width and growing rightwards. The editor covers the scanned word while typing. OCR words are now sized from their letters (a capitals-only word's box spans the cap height, not the full ascender-descender span), so recognised and rewritten text is as tall as the scanned word and sits on its baseline. Five engine tests on synthetic scans and a kittest UI test; `ocr.editable-text-output` is partial.
|
||||
- **2026-10-12 (community, M14, recovery metadata):** A recovery snapshot whose metadata is missing, damaged or names another snapshot is no longer deleted when the folder is listed: it is offered for recovery without an original path (Save asks for a destination), and encrypted snapshots still ask for their password. Recovering one writes fresh metadata in place of the damaged file, and clearing the folder also removes metadata left without a snapshot. Before, a failed metadata write meant the next start deleted the finished snapshot. Synthetic regressions cover the failed write, missing, corrupt, mismatched and encrypted cases. Estimates unchanged.
|
||||
- **2026-10-12 (M7, Arabic ActualText):** Add text records each shaped line in typing order with `/ActualText`, so copy and search preserve mixed Arabic, Latin, digits and brackets. Edit text keeps the logical order supplied by `/ActualText` instead of reversing Arabic or Hebrew a second time. Adjacent visual glyph runs retain their own bidi recovery. Font-independent tests cover Arabic marks, numbers, Latin words, reopen and complete replacement; automation checks saved editor text and unchanged rendering. Font fallback and bidi typing remain separate gaps; estimates unchanged.
|
||||
- **2026-10-12 (M5, Fill & Sign text size):** The active Fill & Sign toolbar now chooses 4–300 pt for new text and dates; the in-place editor and saved annotation use that size, which persists in local settings. `fill_sign_add` accepts the same `font_size`. Feature and readiness estimates unchanged.
|
||||
|
|
|
|||
|
|
@ -177,6 +177,8 @@ struct TextState {
|
|||
leading: f64,
|
||||
rise: f64,
|
||||
fill: Vec<Op>,
|
||||
/// The text rendering mode (`Tr`); 3 is invisible text, as in an OCR layer over a scan.
|
||||
render_mode: i64,
|
||||
}
|
||||
|
||||
impl TextState {
|
||||
|
|
@ -189,6 +191,7 @@ impl TextState {
|
|||
leading: ts.leading,
|
||||
rise: ts.rise,
|
||||
fill: ts.fill.clone(),
|
||||
render_mode: ts.render_mode,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -249,6 +252,9 @@ pub struct TextBlock {
|
|||
pub editor_text: String,
|
||||
/// Byte ranges in `editor_text` and their source line indexes.
|
||||
pub spans: Vec<(std::ops::Range<usize>, usize)>,
|
||||
/// Every line is invisible text (rendering mode 3): an OCR layer over a scanned image, whose
|
||||
/// pixels show the words. A rewrite draws the new text visibly (see [`BlockStyle::cover`]).
|
||||
pub invisible: bool,
|
||||
}
|
||||
|
||||
impl TextBlock {
|
||||
|
|
@ -754,6 +760,9 @@ pub fn text_lines(doc: &Document, page: usize) -> Result<Vec<TextLine>, EditErro
|
|||
&& (s.baseline - base).abs() < size * 0.3
|
||||
&& s.start_x > end - size
|
||||
&& s.start_x - end < size * if bt == s.bt { 3.0 } else { 1.5 }
|
||||
// An OCR layer is one text object over the whole scan: its words continue a line
|
||||
// only across a word space, not across a table cell or a tab stop.
|
||||
&& (s.state.render_mode != 3 || s.start_x - end < size * 0.6)
|
||||
});
|
||||
if joins && let Some(l) = lines.last_mut() {
|
||||
let gap = s.start_x - last.map_or(s.start_x, |x| x.3);
|
||||
|
|
@ -1597,6 +1606,9 @@ pub struct BlockStyle {
|
|||
pub offset: Option<[f64; 2]>,
|
||||
/// Rewrap to this width in user space (dragging the box's edge).
|
||||
pub width: Option<f64>,
|
||||
/// For an invisible paragraph ([`TextBlock::invisible`]): fill its old area with this colour
|
||||
/// (RGB 0–1) before the new text, hiding the scanned word beneath it.
|
||||
pub cover: Option<[f64; 3]>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
|
|
@ -1782,6 +1794,7 @@ pub(crate) fn rewrite_block_with_limit(
|
|||
let n = pdfcraft_content::num;
|
||||
// In its own graphics state, so a new colour (or anything else) stops at the paragraph.
|
||||
let mut block_ops = vec![Op::new("q", vec![])];
|
||||
block_ops.extend(reveal_ops(&b, &lines, style, o.ctm)?);
|
||||
// Moved: a translation inside that state.
|
||||
if let Some((ax, ay)) = movement {
|
||||
block_ops.push(Op::new("cm", vec![n(1.0), n(0.0), n(0.0), n(1.0), n(ax), n(ay)]));
|
||||
|
|
@ -1913,6 +1926,43 @@ pub(crate) fn rewrite_block_with_limit(
|
|||
Ok(LineEdit { substituted })
|
||||
}
|
||||
|
||||
/// The cover (when [`BlockStyle::cover`] asks for one) and visible rendering that start a
|
||||
/// rewritten invisible paragraph, drawn in the space whose CTM is `ctm`. Nothing for visible text.
|
||||
/// The paragraph's `q`/`Q` restores the rendering mode for the invisible text after it.
|
||||
pub(crate) fn reveal_ops(block: &TextBlock, lines: &[TextLine], style: &BlockStyle, ctm: [f64; 6]) -> Result<Vec<Op>, EditError> {
|
||||
if !block.invisible {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
let n = pdfcraft_content::num;
|
||||
let mut ops = Vec::new();
|
||||
if let Some(colour) = style.cover {
|
||||
if colour.iter().any(|c| !c.is_finite() || !(0.0..=1.0).contains(c)) {
|
||||
return Err(EditError::Invalid("the cover colour must be RGB values from 0 to 1".into()));
|
||||
}
|
||||
let back = Matrix(ctm).invert().ok_or_else(|| EditError::Invalid("the paragraph is drawn in a space it can't be covered in".into()))?;
|
||||
ops.push(Op::new("q", vec![]));
|
||||
ops.push(Op::new("rg", colour.map(n).to_vec()));
|
||||
for rect in block.lines.iter().filter_map(|i| lines.get(*i)).map(|l| l.rect) {
|
||||
// Scanned ink reaches a little past the recognised box.
|
||||
let pad = ((rect[3] - rect[1]).abs() * 0.1).clamp(0.5, 2.0);
|
||||
let corners =
|
||||
[[rect[0] - pad, rect[1] - pad], [rect[2] + pad, rect[1] - pad], [rect[2] + pad, rect[3] + pad], [rect[0] - pad, rect[3] + pad]];
|
||||
for (i, [x, y]) in corners.into_iter().enumerate() {
|
||||
let (x, y) = back.apply(x, y);
|
||||
if !(x.is_finite() && y.is_finite()) {
|
||||
return Err(EditError::Invalid("the paragraph's area can't be covered".into()));
|
||||
}
|
||||
ops.push(Op::new(if i == 0 { "m" } else { "l" }, vec![n(x), n(y)]));
|
||||
}
|
||||
ops.push(Op::new("h", vec![]));
|
||||
}
|
||||
ops.push(Op::new("f", vec![]));
|
||||
ops.push(Op::new("Q", vec![]));
|
||||
}
|
||||
ops.push(Op::new("Tr", vec![Object::Int(0)]));
|
||||
Ok(ops)
|
||||
}
|
||||
|
||||
/// Existing paragraph indexes, excluding streams owned by an added-content item.
|
||||
pub(crate) fn selectable_blocks(
|
||||
doc: &Document,
|
||||
|
|
|
|||
|
|
@ -35,6 +35,19 @@ pub(super) fn group_reading_blocks(lines: &[TextLine]) -> Vec<TextBlock> {
|
|||
group_layout(lines, false)
|
||||
}
|
||||
|
||||
/// Text-space units → user space along the baseline. It is the vertical scale ([`Origin::k`])
|
||||
/// unless the text matrix stretches the glyphs, as an OCR layer fitting each word to the scan does.
|
||||
fn across(line: &TextLine) -> f64 {
|
||||
let m = Matrix(line.origin.tm).then(&Matrix(line.origin.ctm)).0;
|
||||
let h = m[0].hypot(m[1]);
|
||||
if h.is_finite() && h > 1e-9 { h } else { line.origin.k }
|
||||
}
|
||||
|
||||
/// Invisible text (rendering mode 3), as in an OCR layer over a scanned image.
|
||||
fn invisible(line: &TextLine) -> bool {
|
||||
line.origin.state.render_mode == 3
|
||||
}
|
||||
|
||||
fn same_style(a: &TextLine, b: &TextLine) -> bool {
|
||||
same_font(a, b.font.as_bytes(), b.face.as_deref())
|
||||
&& a.base_font == b.base_font
|
||||
|
|
@ -50,7 +63,11 @@ fn group_layout(lines: &[TextLine], mixed: bool) -> Vec<TextBlock> {
|
|||
for (i, line) in lines.iter().enumerate() {
|
||||
let joins = rows.last().and_then(|r| r.last()).and_then(|i| lines.get(*i)).is_some_and(|prev| {
|
||||
let gap = line.rect[0] - prev.rect[2];
|
||||
(mixed || same_style(prev, line))
|
||||
// OCR words (invisible text over a scan) join only across a word space: a wider gap
|
||||
// is a table cell or a tab stop, which a rewrite would close up.
|
||||
let scanned = !mixed || (invisible(prev) == invisible(line) && (!invisible(line) || gap < prev.size.max(line.size) * 0.6));
|
||||
scanned
|
||||
&& (mixed || same_style(prev, line))
|
||||
&& same_plane(prev, line)
|
||||
&& (prev.origin.baseline - line.origin.baseline).abs() < prev.size.min(line.size) * 0.2
|
||||
&& gap >= -prev.size.min(line.size) * 0.2
|
||||
|
|
@ -81,7 +98,9 @@ fn group_layout(lines: &[TextLine], mixed: bool) -> Vec<TextBlock> {
|
|||
let gap = prev.origin.baseline - first.origin.baseline;
|
||||
let tol = prev.size.max(first.size).max(1.0);
|
||||
let aligned = (pr[0] - rect[0]).abs() < tol || ((pr[0] + pr[2] - rect[0] - rect[2]) / 2.0).abs() < tol || (pr[2] - rect[2]).abs() < tol;
|
||||
(mixed || same_style(prev, first))
|
||||
// OCR lines stay apart: the scan fixes where each one is, so they never rewrap together.
|
||||
(!mixed || !(invisible(prev) || invisible(first)))
|
||||
&& (mixed || same_style(prev, first))
|
||||
&& same_plane(prev, first)
|
||||
&& aligned
|
||||
&& first.size.min(prev.size) >= first.size.max(prev.size) * 0.4
|
||||
|
|
@ -102,6 +121,7 @@ fn group_layout(lines: &[TextLine], mixed: bool) -> Vec<TextBlock> {
|
|||
rotation: first.rotation(),
|
||||
lines: Vec::new(),
|
||||
rows: Vec::new(),
|
||||
invisible: true,
|
||||
});
|
||||
}
|
||||
let Some(block) = blocks.last_mut() else { continue };
|
||||
|
|
@ -124,6 +144,7 @@ fn group_layout(lines: &[TextLine], mixed: bool) -> Vec<TextBlock> {
|
|||
block.editor_text.push_str(&line.text);
|
||||
block.spans.push((start..block.editor_text.len(), index));
|
||||
row_text.push_str(&line.text);
|
||||
block.invisible &= line.origin.state.render_mode == 3;
|
||||
block.lines.push(index);
|
||||
}
|
||||
if !block.text.is_empty() {
|
||||
|
|
@ -310,7 +331,13 @@ pub(super) fn rewrite_styled_block(
|
|||
if !dx.is_finite() || !dy.is_finite() || style.width.is_some_and(|w| !w.is_finite()) {
|
||||
return Err(EditError::Invalid("the paragraph's position and width must be finite".into()));
|
||||
}
|
||||
let width = style.width.unwrap_or(block.rect[2] - block.rect[0]).clamp(1.0, 14_400.0);
|
||||
// A rewritten OCR line grows to the right rather than wrapping onto the scanned line below.
|
||||
let natural = if block.invisible {
|
||||
(p.crop(doc)[2] - 36.0 - (block.rect[0] + dx)).max(block.rect[2] - block.rect[0])
|
||||
} else {
|
||||
block.rect[2] - block.rect[0]
|
||||
};
|
||||
let width = style.width.unwrap_or(natural).clamp(1.0, 14_400.0);
|
||||
// Each tuple is a character, its source style and an encodable resource name.
|
||||
let mut chars = Vec::new();
|
||||
let mut substituted = None;
|
||||
|
|
@ -405,7 +432,7 @@ pub(super) fn rewrite_styled_block(
|
|||
source.origin.state.char_spacing,
|
||||
source.origin.state.word_spacing,
|
||||
) * source.origin.state.scale
|
||||
* source.origin.k;
|
||||
* across(source);
|
||||
let changed_face = style.resource.is_some() || overrides.get(&index).is_some_and(|r| r.family.is_some() || r.resource.is_some());
|
||||
let factor = if nominal > 0.0 && !changed_face { ((source.rect[2] - source.rect[0]) / nominal).clamp(0.5, 2.0) } else { 1.0 };
|
||||
calibration.insert(index, factor);
|
||||
|
|
@ -426,7 +453,7 @@ pub(super) fn rewrite_styled_block(
|
|||
.map_or(source.origin.state.char_spacing, |c| c / source.origin.k.max(1e-6)),
|
||||
source.origin.state.word_spacing,
|
||||
) * overrides.get(&item.1).and_then(|r| r.scale).or(style.scale).map_or(source.origin.state.scale, |s| (s / 100.0).clamp(0.1, 10.0))
|
||||
* source.origin.k
|
||||
* across(source)
|
||||
* calibration.get(&item.1).copied().unwrap_or(1.0)
|
||||
};
|
||||
let measure = |items: &[(char, usize, String)]| -> Result<f64, EditError> {
|
||||
|
|
@ -530,6 +557,7 @@ pub(super) fn rewrite_styled_block(
|
|||
}
|
||||
let n = pdfcraft_content::num;
|
||||
let mut block_ops = vec![Op::new("q", vec![])];
|
||||
block_ops.extend(reveal_ops(block, lines, style, first.origin.ctm)?);
|
||||
let back = Matrix([first.origin.ctm[0], first.origin.ctm[1], first.origin.ctm[2], first.origin.ctm[3], 0.0, 0.0])
|
||||
.invert()
|
||||
.ok_or_else(|| EditError::Invalid("the paragraph is drawn in a space it cannot be edited in".into()))?;
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ pub mod js;
|
|||
pub mod links;
|
||||
pub mod ocr;
|
||||
pub mod optimizer;
|
||||
mod scan_colours;
|
||||
pub mod signature_image;
|
||||
pub mod signature_scan;
|
||||
pub mod text_fonts;
|
||||
|
|
@ -248,6 +249,47 @@ struct Editor {
|
|||
/// runs (content edits leave every annotation dictionary alone). Cleared on any refresh
|
||||
/// that may change annotations; see [`AppearanceCache`].
|
||||
appearances: AppearanceCache,
|
||||
/// The text blocks last computed for one page, reused while the document is unchanged.
|
||||
text_blocks: TextBlocksCache,
|
||||
}
|
||||
|
||||
/// One page's text blocks ([`pdfcraft_edit::text_blocks`]) and the unmodified document state they
|
||||
/// were read from. Opening a paragraph and applying its edit both need them, and grouping a long
|
||||
/// page is not cheap. Only an unmodified document is cached: its identity changes on every edit,
|
||||
/// undo and save (each reopens the working file), so a stale entry never matches.
|
||||
#[derive(Default)]
|
||||
struct TextBlocksCache(std::sync::Mutex<Option<CachedTextBlocks>>);
|
||||
|
||||
/// The document state, the page (0-based) and its text blocks.
|
||||
type CachedTextBlocks = (pdfcraft_cos::SourceIdentity, usize, Arc<Vec<pdfcraft_edit::TextBlock>>);
|
||||
|
||||
impl Clone for TextBlocksCache {
|
||||
fn clone(&self) -> Self {
|
||||
// A poisoned cache only loses the cached blocks.
|
||||
Self(std::sync::Mutex::new(self.0.lock().ok().and_then(|c| c.clone())))
|
||||
}
|
||||
}
|
||||
|
||||
impl Editor {
|
||||
/// The text blocks on `page` (0-based), from the cache when the document hasn't changed.
|
||||
fn text_blocks(&self, page: usize) -> Result<Arc<Vec<pdfcraft_edit::TextBlock>>, EditError> {
|
||||
let source = (!self.cos.is_modified()).then(|| self.cos.source_identity());
|
||||
if let Some(source) = &source
|
||||
&& let Ok(cache) = self.text_blocks.0.lock()
|
||||
&& let Some((s, p, blocks)) = cache.as_ref()
|
||||
&& s == source
|
||||
&& *p == page
|
||||
{
|
||||
return Ok(blocks.clone());
|
||||
}
|
||||
let blocks = Arc::new(pdfcraft_edit::text_blocks(&self.cos, page)?);
|
||||
if let Some(source) = source
|
||||
&& let Ok(mut cache) = self.text_blocks.0.lock()
|
||||
{
|
||||
*cache = Some((source, page, blocks.clone()));
|
||||
}
|
||||
Ok(blocks)
|
||||
}
|
||||
}
|
||||
|
||||
pub struct Document {
|
||||
|
|
@ -441,7 +483,7 @@ impl Document {
|
|||
|
||||
/// Edit a PDF ▸ Edit text: the paragraphs on `page` (0-based).
|
||||
pub fn text_blocks(&self, page: usize) -> Vec<pdfcraft_edit::TextBlock> {
|
||||
self.editor.as_ref().and_then(|e| pdfcraft_edit::text_blocks(&e.cos, page).ok()).unwrap_or_default()
|
||||
self.editor.as_ref().and_then(|e| e.text_blocks(page).ok()).map(|b| b.as_ref().clone()).unwrap_or_default()
|
||||
}
|
||||
|
||||
/// Saved measurement annotations, calculated from their geometry and PDF scales, plus
|
||||
|
|
@ -3025,7 +3067,17 @@ impl Session {
|
|||
let (mut editor, read_only_reason) = match cos {
|
||||
Ok(Ok(cos)) => {
|
||||
let keys = Keys { render: render_password.clone(), reopen: password.map(str::to_owned) };
|
||||
(Some(Editor { cos, undo: Vec::new(), redo: Vec::new(), keys, appearances: AppearanceCache::default() }), None)
|
||||
(
|
||||
Some(Editor {
|
||||
cos,
|
||||
undo: Vec::new(),
|
||||
redo: Vec::new(),
|
||||
keys,
|
||||
appearances: AppearanceCache::default(),
|
||||
text_blocks: TextBlocksCache::default(),
|
||||
}),
|
||||
None,
|
||||
)
|
||||
}
|
||||
Ok(Err(e)) => (None, Some(e.to_string())),
|
||||
Err(_) => (None, Some("the document structure could not be read for editing".into())),
|
||||
|
|
@ -3121,6 +3173,23 @@ impl Session {
|
|||
if let Some(p) = editor.cos.permissions() {
|
||||
check_permission(&edit, &p)?;
|
||||
}
|
||||
// Rewriting an OCR word (invisible text over a scan) also hides the scanned word: cover it
|
||||
// in the paper's colour and draw the new text in the ink's.
|
||||
let edit = match edit {
|
||||
Edit::EditTextBlock { page, block, text, mut style } => {
|
||||
// Only invisible text is sampled; the blocks are usually cached from opening it.
|
||||
if style.cover.is_none()
|
||||
&& let Some(info) = doc.info.pages.get(page)
|
||||
&& let Some(b) = editor.text_blocks(page).ok().and_then(|b| b.get(block).filter(|b| b.invisible).cloned())
|
||||
&& let Some((paper, ink)) = guard(|| scan_colours::sample(&editor.cos, info, page, &b)).ok().flatten()
|
||||
{
|
||||
style.cover = Some(paper);
|
||||
style.color = style.color.or(Some(ink));
|
||||
}
|
||||
Edit::EditTextBlock { page, block, text, style }
|
||||
}
|
||||
other => other,
|
||||
};
|
||||
// Nothing to bake: leave undo history and the dirty flag alone.
|
||||
if matches!(&edit, Edit::FlattenFillSign) && !pdfcraft_annot::has_visible_fill_sign(&editor.cos) {
|
||||
return Ok(());
|
||||
|
|
|
|||
|
|
@ -0,0 +1,84 @@
|
|||
//! Edit text on a scanned page: an OCR word is invisible text over the scan, so rewriting it must
|
||||
//! also hide the scanned word. The paper colour around the word becomes the cover drawn over it
|
||||
//! ([`pdfcraft_edit::BlockStyle::cover`]) and the colour of its ink becomes the new text's.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use pdfcraft_edit::TextBlock;
|
||||
use pdfcraft_render::{PageInfo, PageRenderer, RenderConfig, RenderRequest, Tile};
|
||||
|
||||
/// Device pixels per point the word's surroundings are read at.
|
||||
const SCALE: f32 = 3.0;
|
||||
/// The most pixels read for one paragraph; a larger area is read at a lower resolution.
|
||||
const MAX_PIXELS: f32 = 4_000_000.0;
|
||||
/// How far around the word (points) the paper is read.
|
||||
const RING: f32 = 4.0;
|
||||
|
||||
/// The paper and ink colours (RGB 0–1) of invisible paragraph `b` on `page`, or `None` when the
|
||||
/// paragraph is visible text or the page can't be read.
|
||||
pub(crate) fn sample(cos: &pdfcraft_cos::Document, info: &PageInfo, page: usize, b: &TextBlock) -> Option<([f64; 3], [f64; 3])> {
|
||||
if !b.invisible {
|
||||
return None;
|
||||
}
|
||||
// The word's box in view space (points, y down).
|
||||
let corners = [[b.rect[0], b.rect[1]], [b.rect[2], b.rect[1]], [b.rect[0], b.rect[3]], [b.rect[2], b.rect[3]]]
|
||||
.map(|[x, y]| info.user_to_view(x as f32, y as f32));
|
||||
let (x0, x1) = corners.iter().fold((f32::INFINITY, f32::NEG_INFINITY), |(a, z), c| (a.min(c[0]), z.max(c[0])));
|
||||
let (y0, y1) = corners.iter().fold((f32::INFINITY, f32::NEG_INFINITY), |(a, z), c| (a.min(c[1]), z.max(c[1])));
|
||||
let word = [x0, y0, x1, y1];
|
||||
let area = [x0 - RING, y0 - RING, x1 + RING, y1 + RING];
|
||||
if word.iter().any(|v| !v.is_finite()) || x1 <= x0 || y1 <= y0 {
|
||||
return None;
|
||||
}
|
||||
let pixels = (area[2] - area[0]) * (area[3] - area[1]) * SCALE * SCALE;
|
||||
let scale = if pixels > MAX_PIXELS { SCALE * (MAX_PIXELS / pixels).sqrt() } else { SCALE };
|
||||
let (page_w, page_h) = (info.width * scale, info.height * scale);
|
||||
let clamp = |v: f32, max: f32| v.clamp(0.0, max.max(0.0)).floor() as u32;
|
||||
let (tx0, ty0) = (clamp(area[0] * scale, page_w), clamp(area[1] * scale, page_h));
|
||||
let (tx1, ty1) = (clamp(area[2] * scale, page_w), clamp(area[3] * scale, page_h));
|
||||
if tx1 <= tx0 || ty1 <= ty0 {
|
||||
return None;
|
||||
}
|
||||
let tile = Tile { x: tx0, y: ty0, w: tx1 - tx0, h: ty1 - ty0 };
|
||||
// The working file as it is: written out only if the document holds unsaved changes, which
|
||||
// it doesn't between edits (each one reopens the file it wrote).
|
||||
let bytes = if cos.is_modified() {
|
||||
Arc::new(pdfcraft_cos::write_incremental(cos, &pdfcraft_cos::SaveOptions::default()).ok()?)
|
||||
} else {
|
||||
cos.bytes().clone()
|
||||
};
|
||||
// The document's true colours, whatever the reading colours are set to.
|
||||
let shot = PageRenderer::new(bytes, RenderConfig::default()).render(RenderRequest { page, scale, tile: Some(tile), ..Default::default() });
|
||||
if shot.error.is_some() {
|
||||
return None;
|
||||
}
|
||||
let pad = ((y1 - y0) * 0.1).clamp(0.5, 2.0);
|
||||
let (mut paper, mut ink) = (Vec::new(), Vec::new());
|
||||
for (i, px) in shot.rgba.as_chunks::<4>().0.iter().enumerate() {
|
||||
let (col, row) = (i % shot.width.max(1) as usize, i / shot.width.max(1) as usize);
|
||||
let x = (tx0 as f32 + col as f32 + 0.5) / scale;
|
||||
let y = (ty0 as f32 + row as f32 + 0.5) / scale;
|
||||
let rgb = [px[0], px[1], px[2]].map(|c| f64::from(c) / 255.0);
|
||||
if x >= x0 && x <= x1 && y >= y0 && y <= y1 {
|
||||
ink.push(rgb);
|
||||
} else if x < x0 - pad || x > x1 + pad || y < y0 - pad || y > y1 + pad {
|
||||
paper.push(rgb);
|
||||
}
|
||||
}
|
||||
let luma = |c: &[f64; 3]| 0.299 * c[0] + 0.587 * c[1] + 0.114 * c[2];
|
||||
let by_luma = |a: &[f64; 3], b: &[f64; 3]| luma(a).total_cmp(&luma(b));
|
||||
paper.sort_by(by_luma);
|
||||
ink.sort_by(by_luma);
|
||||
// The paper is the ring's median, so a table rule or a neighbouring letter in it doesn't count.
|
||||
let paper = paper.get(paper.len() / 2).copied().unwrap_or([1.0; 3]);
|
||||
// The ink is the average of the word's darkest pixels; a box without any is printed in black.
|
||||
let darkest = ink.get(..ink.len().div_ceil(8)).unwrap_or_default();
|
||||
let ink = if darkest.is_empty() {
|
||||
[0.0; 3]
|
||||
} else {
|
||||
let sum = darkest.iter().fold([0.0; 3], |s, c| [s[0] + c[0], s[1] + c[1], s[2] + c[2]]);
|
||||
sum.map(|v| v / darkest.len() as f64)
|
||||
};
|
||||
let ink = if luma(&paper) - luma(&ink) < 0.25 { [0.0; 3] } else { ink };
|
||||
Some((paper, ink))
|
||||
}
|
||||
|
|
@ -0,0 +1,209 @@
|
|||
//! Edit text on a scanned page: rewriting an OCR word (invisible text over the scan) shows the
|
||||
//! new text and hides the scanned word. Synthetic image only.
|
||||
|
||||
use pdfcraft_engine::ocr::PlacedWord;
|
||||
use pdfcraft_engine::{Edit, Session, export};
|
||||
use pdfcraft_render::{PageRenderer, RenderRequest};
|
||||
|
||||
const PAPER: [u8; 3] = [230, 225, 200];
|
||||
const INK: [u8; 3] = [20, 30, 90];
|
||||
/// The scanned "word": a block of ink, in image pixels (72 dpi, so points), y down.
|
||||
const WORD: [u32; 4] = [40, 40, 100, 60];
|
||||
|
||||
/// A 200 × 100 pt "scan" on tinted paper, with OCR text over its word.
|
||||
fn scanned() -> (Session, pdfcraft_engine::DocId) {
|
||||
scanned_with(vec![PlacedWord { text: "John".into(), origin: [40.0, 40.0], across: [60.0, 0.0], up: [0.0, 20.0] }])
|
||||
}
|
||||
|
||||
fn scanned_with(words: Vec<PlacedWord>) -> (Session, pdfcraft_engine::DocId) {
|
||||
let (w, h) = (200u32, 100u32);
|
||||
let mut rgba = Vec::new();
|
||||
for y in 0..h {
|
||||
for x in 0..w {
|
||||
let ink = (WORD[0]..WORD[2]).contains(&x) && (WORD[1]..WORD[3]).contains(&y);
|
||||
rgba.extend(if ink { INK } else { PAPER });
|
||||
rgba.push(255);
|
||||
}
|
||||
}
|
||||
let png = export::encode_png(w, h, &rgba).unwrap();
|
||||
let mut s = Session::new();
|
||||
let bytes = s.create_from_images(&[("scan.png".into(), png)]).unwrap();
|
||||
let id = s.open("scan.pdf", None, bytes, None).unwrap();
|
||||
s.apply(id, Edit::AddOcrText { page: 0, words }).unwrap();
|
||||
(s, id)
|
||||
}
|
||||
|
||||
fn word_pixels(s: &Session, id: pdfcraft_engine::DocId) -> Vec<[u8; 3]> {
|
||||
let doc = s.get(id).unwrap();
|
||||
let out = PageRenderer::new(doc.bytes.clone(), Default::default()).render(RenderRequest { page: 0, scale: 1.0, ..Default::default() });
|
||||
assert!(out.error.is_none(), "{:?}", out.error);
|
||||
let mut px = Vec::new();
|
||||
for y in WORD[1] + 2..WORD[3] - 2 {
|
||||
for x in WORD[0] + 2..WORD[2] - 2 {
|
||||
let i = ((y * out.width + x) * 4) as usize;
|
||||
px.push([out.rgba[i], out.rgba[i + 1], out.rgba[i + 2]]);
|
||||
}
|
||||
}
|
||||
px
|
||||
}
|
||||
|
||||
fn near(a: [u8; 3], b: [u8; 3]) -> bool {
|
||||
a.iter().zip(b).all(|(x, y)| x.abs_diff(y) <= 24)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editing_an_ocr_word_covers_the_scan_and_shows_the_new_text() {
|
||||
let (mut s, id) = scanned();
|
||||
let blocks = s.get(id).unwrap().text_blocks(0);
|
||||
assert_eq!(blocks.len(), 1);
|
||||
assert!(blocks[0].invisible, "the OCR layer is invisible text");
|
||||
assert!(word_pixels(&s, id).iter().all(|p| near(*p, INK)), "the scanned word fills its box");
|
||||
|
||||
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "Jane".into(), style: Default::default() }).unwrap();
|
||||
|
||||
let blocks = s.get(id).unwrap().text_blocks(0);
|
||||
assert_eq!(blocks.iter().map(|b| b.text.as_str()).collect::<Vec<_>>(), ["Jane"]);
|
||||
assert!(!blocks[0].invisible, "the new text is drawn");
|
||||
let px = word_pixels(&s, id);
|
||||
let paper = px.iter().filter(|p| near(**p, PAPER)).count();
|
||||
let ink = px.iter().filter(|p| near(**p, INK)).count();
|
||||
// The scanned word is covered in the paper's colour; the new letters are in the ink's.
|
||||
assert!(paper * 10 > px.len() * 4, "mostly paper now: {paper} of {}", px.len());
|
||||
assert!(ink * 100 > px.len() * 3, "the new text shows in the scan's ink colour: {ink} of {}", px.len());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_explicit_colour_is_kept_for_the_new_text() {
|
||||
let (mut s, id) = scanned();
|
||||
let style = pdfcraft_engine::BlockStyle { color: Some([1.0, 0.0, 0.0]), ..Default::default() };
|
||||
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "Jane".into(), style }).unwrap();
|
||||
let px = word_pixels(&s, id);
|
||||
assert!(px.iter().any(|p| p[0] > 200 && p[1] < 60 && p[2] < 60), "red text");
|
||||
assert!(!px.iter().any(|p| near(*p, INK)), "no scanned ink is left");
|
||||
}
|
||||
|
||||
/// A word as OCR boxes it on a scan set at `size` on `baseline`: tight around its ink, from the
|
||||
/// descender line when it has descenders, up to the cap height.
|
||||
fn scanned_word(text: &str, x0: f64, x1: f64, baseline: f64, size: f64) -> PlacedWord {
|
||||
let low = if text.chars().any(|c| "gjpqy".contains(c)) { 0.21 * size } else { 0.0 };
|
||||
PlacedWord { text: text.into(), origin: [x0, baseline - low], across: [x1 - x0, 0.0], up: [0.0, 0.718 * size + low] }
|
||||
}
|
||||
|
||||
fn word(text: &str, x0: f64, x1: f64, baseline: f64) -> PlacedWord {
|
||||
PlacedWord { text: text.into(), origin: [x0, baseline], across: [x1 - x0, 0.0], up: [0.0, 10.0] }
|
||||
}
|
||||
|
||||
/// OCR words keep the scan's layout: a phrase is one paragraph, a value in the next table cell and
|
||||
/// the line below are others, so rewriting one never moves the rest.
|
||||
#[test]
|
||||
fn ocr_words_group_into_phrases_that_edit_in_place() {
|
||||
let (mut s, id) = scanned_with(vec![
|
||||
word("EMPLOYEE", 10.0, 60.0, 70.0),
|
||||
word("NAME", 64.0, 90.0, 70.0),
|
||||
word("John", 120.0, 145.0, 70.0),
|
||||
word("Smith", 120.0, 148.0, 56.0),
|
||||
]);
|
||||
let texts = |s: &Session| s.get(id).unwrap().text_blocks(0).iter().map(|b| b.text.clone()).collect::<Vec<_>>();
|
||||
assert_eq!(texts(&s), ["EMPLOYEE NAME", "John", "Smith"]);
|
||||
let john = s.get(id).unwrap().text_blocks(0)[1].rect;
|
||||
s.apply(id, Edit::EditTextBlock { page: 0, block: 1, text: "Jane".into(), style: Default::default() }).unwrap();
|
||||
assert_eq!(texts(&s), ["EMPLOYEE NAME", "Jane", "Smith"]);
|
||||
let blocks = s.get(id).unwrap().text_blocks(0);
|
||||
assert!(blocks[0].invisible && blocks[2].invisible, "the other OCR words are left alone");
|
||||
assert!(!blocks[1].invisible);
|
||||
assert!((blocks[1].rect[0] - john[0]).abs() < 1.0, "the new word starts where the old one did: {:?} vs {john:?}", blocks[1].rect);
|
||||
}
|
||||
|
||||
/// Several words of one OCR phrase rewrite as one line, without wrapping onto the line below.
|
||||
#[test]
|
||||
fn a_longer_ocr_phrase_grows_rightwards() {
|
||||
let (mut s, id) = scanned_with(vec![
|
||||
scanned_word("Project", 20.0, 55.0, 70.0, 11.0),
|
||||
scanned_word("Lead", 59.0, 80.0, 70.0, 11.0),
|
||||
scanned_word("Testing", 20.0, 56.0, 56.0, 11.0),
|
||||
]);
|
||||
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "Project Coordinator Lead".into(), style: Default::default() }).unwrap();
|
||||
let blocks = s.get(id).unwrap().text_blocks(0);
|
||||
assert_eq!(blocks.iter().map(|b| b.text.as_str()).collect::<Vec<_>>(), ["Project Coordinator Lead", "Testing"]);
|
||||
assert_eq!(blocks[0].rows.len(), 1, "one line");
|
||||
}
|
||||
|
||||
/// OCR words differ in size and are stretched to fit the scan: a rewritten phrase keeps each
|
||||
/// word's width, so the words don't run into one another.
|
||||
#[test]
|
||||
fn a_rewritten_ocr_phrase_keeps_its_word_widths() {
|
||||
let (mut s, id) = scanned_with(vec![
|
||||
scanned_word("John", 20.0, 45.0, 70.0, 10.5),
|
||||
scanned_word("Smith", 49.0, 78.0, 70.0, 11.0),
|
||||
scanned_word("Example", 82.0, 124.0, 70.0, 11.5),
|
||||
]);
|
||||
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "John Smith Exam".into(), style: Default::default() }).unwrap();
|
||||
let lines = s.get(id).unwrap().text_lines(0);
|
||||
assert_eq!(lines.iter().map(|l| l.text.trim()).collect::<Vec<_>>(), ["John", "Smith", "Exam"]);
|
||||
assert!((lines[0].rect[2] - 45.0).abs() < 0.5, "John keeps its width: {:?}", lines[0].rect);
|
||||
for pair in lines.windows(2) {
|
||||
assert!(pair[1].rect[0] >= pair[0].rect[2] - 0.5, "{:?} runs into {:?}", pair[1].text, pair[0].text);
|
||||
}
|
||||
}
|
||||
|
||||
/// Visible text set with a horizontally stretched text matrix (3 × 1), as some generators write.
|
||||
fn stretched_visible() -> std::sync::Arc<Vec<u8>> {
|
||||
use pdfcraft_cos::{Dict, Document, Object, SaveOptions, Stream};
|
||||
let mut d = Document::new_empty();
|
||||
let pages = d.add(Object::Dict(Dict::new()));
|
||||
let contents = d.add(Object::Stream(Stream::flate(Dict::new(), b"BT /F1 12 Tf 3 0 0 1 20 60 Tm (Source) Tj ET")));
|
||||
let mut font = Dict::new();
|
||||
font.set(b"Type".to_vec(), Object::name("Font"));
|
||||
font.set(b"Subtype".to_vec(), Object::name("Type1"));
|
||||
font.set(b"BaseFont".to_vec(), Object::name("Helvetica"));
|
||||
let mut fonts = Dict::new();
|
||||
fonts.set(b"F1".to_vec(), Object::Dict(font));
|
||||
let mut res = Dict::new();
|
||||
res.set(b"Font".to_vec(), Object::Dict(fonts));
|
||||
let mut p = Dict::new();
|
||||
p.set(b"Type".to_vec(), Object::name("Page"));
|
||||
p.set(b"Parent".to_vec(), Object::Ref(pages));
|
||||
p.set(b"Contents".to_vec(), Object::Ref(contents));
|
||||
p.set(b"Resources".to_vec(), Object::Dict(res));
|
||||
p.set(b"MediaBox".to_vec(), Object::Array([0, 0, 400, 120].map(Object::Int).to_vec()));
|
||||
let page = d.add(Object::Dict(p));
|
||||
d.update_dict(pages, |p| {
|
||||
p.set(b"Type".to_vec(), Object::name("Pages"));
|
||||
p.set(b"Kids".to_vec(), Object::Array(vec![Object::Ref(page)]));
|
||||
p.set(b"Count".to_vec(), Object::Int(1));
|
||||
})
|
||||
.unwrap();
|
||||
let mut root = Dict::new();
|
||||
root.set(b"Type".to_vec(), Object::name("Catalog"));
|
||||
root.set(b"Pages".to_vec(), Object::Ref(pages));
|
||||
let root = d.add(Object::Dict(root));
|
||||
d.trailer_mut().set(b"Root".to_vec(), Object::Ref(root));
|
||||
std::sync::Arc::new(pdfcraft_cos::write_full(&d, &SaveOptions::default()).unwrap())
|
||||
}
|
||||
|
||||
/// Visible text is not OCR: a styled rewrite of stretched text keeps its width, draws no cover
|
||||
/// and stays visible.
|
||||
#[test]
|
||||
fn visible_stretched_text_rewrites_unchanged() {
|
||||
let mut s = Session::new();
|
||||
let id = s.open("stretched.pdf", None, stretched_visible(), None).unwrap();
|
||||
let before = s.get(id).unwrap().text_blocks(0);
|
||||
assert_eq!(before.len(), 1);
|
||||
assert!(!before[0].invisible);
|
||||
let style = pdfcraft_engine::BlockStyle { color: Some([1.0, 0.0, 0.0]), ..Default::default() };
|
||||
s.apply(id, Edit::EditTextBlock { page: 0, block: 0, text: "Source".into(), style }).unwrap();
|
||||
let after = s.get(id).unwrap().text_blocks(0);
|
||||
assert_eq!(after.iter().map(|b| b.text.as_str()).collect::<Vec<_>>(), ["Source"]);
|
||||
assert!(!after[0].invisible);
|
||||
let (w0, w1) = (before[0].rect[2] - before[0].rect[0], after[0].rect[2] - after[0].rect[0]);
|
||||
assert!((w0 - w1).abs() < 1.0, "the stretched word keeps its width: {w0} → {w1}");
|
||||
assert!((before[0].rect[0] - after[0].rect[0]).abs() < 0.5);
|
||||
// No paper-coloured cover: outside the glyphs the page is still white.
|
||||
let doc = s.get(id).unwrap();
|
||||
let out = PageRenderer::new(doc.bytes.clone(), Default::default()).render(RenderRequest { page: 0, scale: 1.0, ..Default::default() });
|
||||
assert!(out.error.is_none());
|
||||
let r = after[0].rect;
|
||||
let (x, y) = ((r[0] - 1.0).max(0.0) as u32, (120.0 - (r[1] + r[3]) / 2.0) as u32);
|
||||
let i = ((y * out.width + x) * 4) as usize;
|
||||
assert_eq!(&out.rgba[i..i + 3], &[255, 255, 255]);
|
||||
}
|
||||
|
|
@ -187,6 +187,23 @@ impl PlacedWord {
|
|||
/// Helvetica's ascender and descender (em fractions): a word box spans about this much.
|
||||
const BOX_EM: f64 = 0.93;
|
||||
const DESCENT: f64 = 0.21;
|
||||
/// Helvetica's cap height and x-height (em fractions).
|
||||
const CAP_HEIGHT: f64 = 0.718;
|
||||
const X_HEIGHT: f64 = 0.523;
|
||||
|
||||
/// How far `text`'s ink reaches above and below the baseline, in em fractions: a word box is
|
||||
/// tight around the ink, so "DT" spans the cap height only, "sun" the x-height, and "gap" reaches
|
||||
/// down to the descender. Text without letters or digits keeps the full ascender-descender span.
|
||||
fn ink_extent(text: &str) -> (f64, f64) {
|
||||
if !text.chars().any(char::is_alphanumeric) {
|
||||
return (BOX_EM - DESCENT, -DESCENT);
|
||||
}
|
||||
let tall = text
|
||||
.chars()
|
||||
.any(|c| c.is_uppercase() || c.is_ascii_digit() || "bdfhijklt!?/\\()[]{}|'\"#$%&@".contains(c) || (c.is_alphabetic() && !c.is_ascii()));
|
||||
let low = text.chars().any(|c| "gjpqy()[]{}|_".contains(c));
|
||||
(if tall { CAP_HEIGHT } else { X_HEIGHT }, if low { -DESCENT } else { 0.0 })
|
||||
}
|
||||
|
||||
fn num(v: f64) -> String {
|
||||
let s = format!("{:.4}", if v.abs() < 1e-9 { 0.0 } else { v });
|
||||
|
|
@ -205,10 +222,12 @@ pub fn text_layer(words: &[PlacedWord]) -> Vec<u8> {
|
|||
if width <= 0.0 || height <= 0.0 {
|
||||
continue;
|
||||
}
|
||||
// The em square: its height spans the box; its width is stretched to fit the word.
|
||||
let em = [w.up[0] / BOX_EM, w.up[1] / BOX_EM];
|
||||
// The em square: the word's own ink (cap height or x-height, descenders or not) spans the
|
||||
// box, so the text is as tall as the scanned word; its width is stretched to fit the word.
|
||||
let (top, bottom) = ink_extent(&w.text);
|
||||
let em = [w.up[0] / (top - bottom), w.up[1] / (top - bottom)];
|
||||
let ax = [w.across[0] / width, w.across[1] / width];
|
||||
let o = [w.origin[0] + em[0] * DESCENT, w.origin[1] + em[1] * DESCENT];
|
||||
let o = [w.origin[0] - em[0] * bottom, w.origin[1] - em[1] * bottom];
|
||||
let m = [ax[0], ax[1], em[0], em[1], o[0], o[1]];
|
||||
let nums: Vec<String> = m.iter().map(|v| num(*v)).collect();
|
||||
out.extend_from_slice(nums.join(" ").as_bytes());
|
||||
|
|
@ -248,10 +267,28 @@ mod tests {
|
|||
let s = String::from_utf8(text_layer(&[w])).unwrap();
|
||||
assert!(s.contains("3 Tr") && s.contains("/PCHelv 1 Tf"), "{s}");
|
||||
let width = helvetica_width("Hi", 1.0);
|
||||
assert!(s.contains(&format!("{} 0 0 10 10 22.1 Tm (Hi) Tj", num(30.0 / width))), "{s}");
|
||||
// "Hi" has no descender: its box spans the cap height, from the baseline up.
|
||||
assert!(s.contains(&format!("{} 0 0 {} 10 20 Tm (Hi) Tj", num(30.0 / width), num(9.3 / CAP_HEIGHT))), "{s}");
|
||||
assert!(s.starts_with("/OCR BMC") && s.ends_with("EMC\n"));
|
||||
}
|
||||
|
||||
/// A word box is tight around the ink, so the text's size and baseline come from the letters:
|
||||
/// capitals, lowercase without ascenders, descenders.
|
||||
#[test]
|
||||
fn the_text_is_as_tall_as_the_scanned_word() {
|
||||
assert_eq!(ink_extent("DT"), (CAP_HEIGHT, 0.0));
|
||||
assert_eq!(ink_extent("402337"), (CAP_HEIGHT, 0.0));
|
||||
assert_eq!(ink_extent("sun"), (X_HEIGHT, 0.0));
|
||||
assert_eq!(ink_extent("gap"), (X_HEIGHT, -DESCENT));
|
||||
assert_eq!(ink_extent("Gallus.Ze@camtel.cm"), (CAP_HEIGHT, 0.0));
|
||||
assert_eq!(ink_extent("Corporation"), (CAP_HEIGHT, -DESCENT));
|
||||
assert_eq!(ink_extent("--"), (BOX_EM - DESCENT, -DESCENT));
|
||||
let w = PlacedWord { text: "gap".into(), origin: [0.0, 100.0], across: [20.0, 0.0], up: [0.0, 7.3] };
|
||||
let s = String::from_utf8(text_layer(&[w])).unwrap();
|
||||
let em = 7.3 / (X_HEIGHT + DESCENT);
|
||||
assert!(s.contains(&format!(" 0 0 {} 0 {} Tm (gap) Tj", num(em), num(100.0 + em * DESCENT))), "{s}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rotated_pages_keep_the_reading_direction() {
|
||||
// A page turned 90°: image x runs up the page.
|
||||
|
|
|
|||
|
|
@ -944,7 +944,12 @@ pub(crate) fn overlay(ctx: &egui::Context, view: &mut DocView, info: &DocInfo) -
|
|||
let page = view.line_editor.as_ref()?.page;
|
||||
let xf = view.page_xform(page)?;
|
||||
let hold = view.objects.hold_editor;
|
||||
let background_ready = view.line_editor.as_ref().is_some_and(|ed| view.text_background.ready(ed.page, ed.block));
|
||||
// An OCR word is invisible text over a scan: the textless background still shows the scanned
|
||||
// word, so the editor covers it, as the applied edit will.
|
||||
let background_ready = view
|
||||
.line_editor
|
||||
.as_ref()
|
||||
.is_some_and(|ed| view.text_background.ready(ed.page, ed.block) && !ed.source.as_ref().is_some_and(|(b, _)| b.invisible));
|
||||
let draft_rect = view.text_background.draft_rect.map(|r| xf.user_rect(info, page, r.map(|v| v as f32)));
|
||||
let ed = view.line_editor.as_mut()?;
|
||||
if hold {
|
||||
|
|
@ -971,7 +976,8 @@ pub(crate) fn overlay(ctx: &egui::Context, view: &mut DocView, info: &DocInfo) -
|
|||
// The editor opens at the source size. Once text changes, show the real draft
|
||||
// extent: single rows grow toward the page edge, paragraphs rewrap at their
|
||||
// source width, and the visible frame grows/shrinks vertically with their rows.
|
||||
let single_run = ed.source.as_ref().is_some_and(|(block, _)| block.lines.len() == 1);
|
||||
// An OCR phrase (several stretched words on one scanned line) also grows rightwards.
|
||||
let single_run = ed.source.as_ref().is_some_and(|(block, _)| block.lines.len() == 1 || block.invisible);
|
||||
let width = if !ed.multiline && single_run && !ed.text.contains('\n') && ed.has_changes() {
|
||||
let job = editor_job(ui, ed, &ed.text, scale, f32::INFINITY);
|
||||
let natural = ui.fonts_mut(|f| f.layout_job(job)).size().x;
|
||||
|
|
|
|||
|
|
@ -114,3 +114,51 @@ fn recognize_text_can_read_pictures_on_pages_with_text() {
|
|||
let text = text.text.unwrap().plain_text().to_lowercase();
|
||||
assert_eq!((text.matches("invoice").count(), text.matches("scanned").count()), (1, 1), "{text}");
|
||||
}
|
||||
|
||||
/// Edit text & images on a scanned page: an OCR word (invisible text over the scan) opens for
|
||||
/// editing on a click, stays open when the pointer leaves it, and its editor hides the scanned
|
||||
/// word instead of showing it through.
|
||||
#[test]
|
||||
fn an_ocr_word_opens_for_editing_and_stays_open() {
|
||||
use pdfcraft_engine::ocr::PlacedWord;
|
||||
let mut h = Harness::builder().with_size(egui::vec2(1400.0, 900.0)).build_eframe(|_cc| {
|
||||
let mut app = PdfCraftApp::new();
|
||||
app.set_option("language", "en").unwrap();
|
||||
app.open_bytes("scan.pdf", None, scan()).unwrap();
|
||||
let id = app.views[0].id;
|
||||
let ph = app.session.get(id).unwrap().info.pages[0].height as f64;
|
||||
let words = vec![PlacedWord { text: "Recognize".into(), origin: [72.0, ph - 100.0], across: [80.0, 0.0], up: [0.0, 14.0] }];
|
||||
app.session.apply(id, Edit::AddOcrText { page: 0, words }).unwrap();
|
||||
app.set_option("zoom", "100").unwrap();
|
||||
assert!(app.execute("edit.edit_text"));
|
||||
app
|
||||
});
|
||||
for _ in 0..60 {
|
||||
h.run_steps(2);
|
||||
if !h.state().render_pending() {
|
||||
break;
|
||||
}
|
||||
std::thread::sleep(std::time::Duration::from_millis(5));
|
||||
}
|
||||
let (r, k) = {
|
||||
let app = h.state();
|
||||
let d = app.session.get(app.views[0].id).unwrap();
|
||||
assert!(d.text_blocks(0).first().is_some_and(|b| b.invisible));
|
||||
let r = app.views[0].page_screen_rect(0).unwrap();
|
||||
(r, r.width() / d.info.pages[0].width)
|
||||
};
|
||||
// The word, 100 pt from the top of the page.
|
||||
let p = r.min + egui::vec2(100.0, 93.0) * k;
|
||||
h.hover_at(p);
|
||||
h.run_steps(2);
|
||||
for pressed in [true, false] {
|
||||
h.event(egui::Event::PointerButton { pos: p, button: egui::PointerButton::Primary, pressed, modifiers: Default::default() });
|
||||
}
|
||||
h.run_steps(4);
|
||||
let opened = |h: &Harness<'_, PdfCraftApp>| h.state().views[0].line_editor.as_ref().map(|e| e.text.clone());
|
||||
assert_eq!(opened(&h).as_deref(), Some("Recognize"));
|
||||
assert!(h.state().views[0].image_selection.is_none(), "the scan under the word isn't selected");
|
||||
h.hover_at(r.min + egui::vec2(300.0, 400.0) * k);
|
||||
h.run_steps(4);
|
||||
assert_eq!(opened(&h).as_deref(), Some("Recognize"), "moving the pointer away keeps the word open");
|
||||
}
|
||||
|
|
|
|||
|
|
@ -38,6 +38,17 @@ resource to `text_edit.font_resource`. `text_edit.ranges` holds selected range f
|
|||
`text_edit_layout` accepts resource/ranges too and measures a draft without applying it.
|
||||
All file inputs respect CLI root confinement; importing a font and applying text are undoable.
|
||||
|
||||
On a scanned page, recognised text (Scan & OCR) is an invisible layer over the picture. Edit
|
||||
text groups it into phrases: words joined by a word space, with a table cell, a tab stop and each
|
||||
scanned line kept apart. Applying an edit draws the new text visibly in the scan's ink colour (or
|
||||
the one chosen) and first covers the old words with the paper colour sampled around them. Each
|
||||
word keeps its position and width, and a longer phrase grows rightwards rather than wrapping. The
|
||||
cover is one flat colour, and the text uses the layer's Helvetica, not the scanned typeface.
|
||||
This applies to any invisible text (rendering mode 3), not just a PdfCraft OCR layer: editing
|
||||
invisible text, such as another OCR tool's layer, makes the rewritten paragraph visible over a
|
||||
painted cover in the sampled paper colour. Edit text also groups invisible text differently: its lines never
|
||||
merge into multi-line paragraphs, and words split at gaps of 0.6 em or more.
|
||||
|
||||
Explicit marked paragraph/heading scopes prevent unrelated tagged regions from merging.
|
||||
Compatible geometry groups untagged content, including runs split across content streams
|
||||
or equivalent graphics transforms. Geometry cannot infer author intent in every untagged
|
||||
|
|
|
|||
|
|
@ -6513,11 +6513,11 @@ title = "Editable text and images output"
|
|||
area = "I"
|
||||
tier = "P1"
|
||||
milestone = "M10"
|
||||
status = "planned"
|
||||
commands = []
|
||||
tools = []
|
||||
evidence = []
|
||||
notes = ""
|
||||
status = "partial"
|
||||
commands = ["edit.edit_text"]
|
||||
tools = ["text_edit"]
|
||||
evidence = ["crates/engine/tests/ocr_edit.rs::editing_an_ocr_word_covers_the_scan_and_shows_the_new_text", "crates/engine/tests/ocr_edit.rs::an_explicit_colour_is_kept_for_the_new_text", "crates/engine/tests/ocr_edit.rs::ocr_words_group_into_phrases_that_edit_in_place", "crates/engine/tests/ocr_edit.rs::a_longer_ocr_phrase_grows_rightwards", "crates/engine/tests/ocr_edit.rs::a_rewritten_ocr_phrase_keeps_its_word_widths", "crates/ui-egui/tests/ocr.rs::an_ocr_word_opens_for_editing_and_stays_open"]
|
||||
notes = "On demand, per edit: in Edit text & images, recognised text (the invisible OCR layer over a scan) is grouped into phrases (words across a word space; a wider gap such as a table cell, and each scanned line, stay separate) and opens for editing on a click. Applying draws the new text visibly in the scan's sampled ink colour (or the chosen one) and first covers the old words with the paper colour sampled around them, so the scanned word no longer shows; the phrase keeps each word's position and width and grows rightwards instead of wrapping. Still missing: Acrobat's Recognize Text output that converts a whole page to editable text at once, matching the scanned typeface (the layer's Helvetica is used), and covering on a textured or patterned background (one flat colour is used)."
|
||||
|
||||
[[feature]]
|
||||
id = "ocr.ocr-downsample"
|
||||
|
|
|
|||
Loading…
Reference in New Issue