From e609722b36decde5a09022b84977d3e0c56429ed Mon Sep 17 00:00:00 2001 From: tzagim <2285958+tzagim@users.noreply.github.com> Date: Mon, 12 Oct 2026 00:41:06 +0300 Subject: [PATCH] M7: Hebrew in Edit text (reading order, writing order, Hebrew glyphs) (#521) * Hebrew fix * Tests: the faux-redaction audit gets its own work folder (it shared "audit" with the space audit, so the two raced) * Arabic golden test: compare bytes only in a build without an Arabic face (CI's Linux job builds with craft-fonts) --------- Co-authored-by: Brandon Thomas --- ROADMAP.md | 1 + crates/automation/tests/automation.rs | 2 +- crates/edit/Cargo.toml | 3 +- crates/edit/src/added/arabic_tests.rs | 7 +- crates/edit/src/bidi.rs | 364 ++++++++++++++++++++++++++ crates/edit/src/lib.rs | 1 + crates/edit/src/tests.rs | 142 ++++++++++ crates/edit/src/text.rs | 153 ++++++++--- crates/fonts/README.md | 6 +- crates/fonts/build.rs | 6 +- crates/fonts/src/craft.rs | 36 +++ crates/fonts/src/lib.rs | 4 +- crates/ui-egui/src/theme.rs | 12 +- docs/localization.md | 2 + parity/acrobat-features.toml | 8 +- 15 files changed, 694 insertions(+), 53 deletions(-) create mode 100644 crates/edit/src/bidi.rs diff --git a/ROADMAP.md b/ROADMAP.md index 45f7ec00..bfe8ac0d 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -209,6 +209,7 @@ Newest first: the date, what moved, and the new overall estimate. - **2026-10-09 (M5.7, #434 partial):** Natural-size image stamps placed at a click or through `stamp_custom` remain upright on rotated pages, with a displayed-space-sized rectangle and a counterrotated standard appearance matrix. Move, resize, restyle and undo/redo preserve that orientation. Native and external Poppler save/reopen rendering cover all four quarter-turns; UI tests verify the displayed bounds. Explicit rectangle placement, image signatures, PDF-page stamps and image-file orientation metadata remain outside this repair. #434 stays open; milestone table and effort estimates remain unchanged. - **2026-10-09 (M1, raw-stream decoding limits):** COS checks unfiltered decoded data against the caller and per-stream limits before copying it, including empty filter arrays and streams whose Crypt filter was applied on load. Strict decoding uses the same raw-data guard. Tiny synthetic regressions cover exceeded, exact and zero-byte limits while retaining the filtered decoder behavior. Milestone and overall effort estimates unchanged (about 30–35%). +- **2026-10-09 (community, M7, Hebrew in Edit text):** Editing existing Hebrew text showed it reversed (`בדיוק` read back letter-reversed) and wrote replacement boxes on save. Content streams draw right-to-left lines in visual order; Edit text now reads lines and paragraphs in typing order and writes them back in drawing order (`pdfcraft-edit::bidi`, UAX #9 through `unicode-bidi`, with numbers, Latin and mirrored brackets; where visual order allows several readings, the one that draws back exactly and pairs its brackets wins). Right-to-left paragraphs are right-aligned unless an alignment is chosen, and a single line keeps its right edge as it grows. Letters the page's font lacks are drawn with a craft-fonts `Hebr` face (Noto Sans Hebrew, storytold/craft-fonts PR) as Type 3 glyphs with ToUnicode, so the result is searchable; the interface uses that face too, and the web build keeps it. Without it, a clear error and no change. `edit.rtl-cjk-editing` moves to partial. Not yet: niqqud in replacement text (refused, not misdrawn), Arabic in Edit text (shaping; Add text has it since #403), typing right to left in the box (#437), lines drawn word by word from the right; the release craft-fonts pin must move to a revision with the Hebrew face. Overall estimate unchanged (about 30–35%). - **2026-10-09 (M2, bounded page labels):** Inspector number trees have shared work, depth, and cycle limits; prefixes are bounded before decoding and complete custom labels have per-label and aggregate UTF-8 byte caps. A refused tree leaves physical page numbers intact and records one warning, with custom labels applied only after every range succeeds. Pure COS Roman and repeated-letter formatters are shared with Organize; label queries and edits return errors for unsupported output before changing the stored tree, while ordinary labels and prefix-only ranges remain supported. Synthetic tests cover safe just-over-limit cases, large first numbers, aliases, cycles, wide/deep trees, UTF-16/UTF-8 boundaries, aggregate output, and unchanged refused edits. Milestone and overall effort estimates unchanged (about 30–35%). - **2026-10-09 (M4, page grid zoom):** The Organize page grid zooms from 50% to 300%: toolbar buttons with a percentage that resets, trackpad pinch or Ctrl/⌘ + wheel about the page under the pointer, and ⌘+ / ⌘− / ⌘0. Pages drawn larger than their thumbnails are rendered at the size drawn while in view, so text stays sharp in documents of any length. Not yet: a slider, remembering the size, a size control for the Pages panel. Overall estimate unchanged (about 30–35%). - **2026-10-09 (M7, grouped artwork):** Edit text/images selects page-drawn Form XObjects as whole figures, including nested raster/vector/text content. Their BBox and optional Matrix determine the selection; move/resize/rotate/flip/delete rewrite only the selected page placement, preserving shared Forms and other placements. Engine, automation and UI core regressions cover geometry, rendered movement, save/reopen, undo and selection/drag/Delete. Bitmap replacement/extraction remain raster-only; inline images and individual elements inside a Form are not editable through this tool. Overall estimate unchanged (about 30–35%). diff --git a/crates/automation/tests/automation.rs b/crates/automation/tests/automation.rs index be1c7134..1834d187 100644 --- a/crates/automation/tests/automation.rs +++ b/crates/automation/tests/automation.rs @@ -2399,7 +2399,7 @@ fn faux_redacted_fixture() -> Vec { #[test] fn auditing_faux_redactions_through_tools() { - let dir = workdir("audit"); + let dir = workdir("faux-redaction-audit"); std::fs::write(dir.join("redacted.pdf"), faux_redacted_fixture()).unwrap(); let mut a = auto(&dir); let doc = ok(&mut a, "doc_open", json!({ "path": "redacted.pdf" }))["doc"].as_u64().unwrap(); diff --git a/crates/edit/Cargo.toml b/crates/edit/Cargo.toml index c929bfb4..0c773767 100644 --- a/crates/edit/Cargo.toml +++ b/crates/edit/Cargo.toml @@ -15,7 +15,8 @@ pdfcraft-content = { workspace = true } pdfcraft-fonts = { workspace = true } pdfcraft-model = { workspace = true } thiserror = { workspace = true } -# Display order of right-to-left lines in added text; already in the tree via the interface. +# Display order of right-to-left lines in added text, and reading and writing order in Edit text; +# already in the tree via the interface. unicode-bidi = "0.3" [dev-dependencies] diff --git a/crates/edit/src/added/arabic_tests.rs b/crates/edit/src/added/arabic_tests.rs index 5feb62bb..68743dc7 100644 --- a/crates/edit/src/added/arabic_tests.rs +++ b/crates/edit/src/added/arabic_tests.rs @@ -134,7 +134,12 @@ fn an_item_below_the_old_limit_is_drawn_exactly_as_before() { let mut bytes = out.clone(); pdfcraft_cos::serialize(&Object::Dict(fonts.clone()), &mut bytes); bytes.extend(snapshot(&doc)); - assert_eq!(fnv1a(&bytes), SMALL_ITEM_FINGERPRINT, "output changed; the golden is the code before the sharding change"); + // The golden was recorded without craft-fonts: with an Arabic face built in, the font's + // FontName and FontFamily carry that face's name instead of "Arabic", so only the build + // without it (CI's Windows job) can compare bytes. + if pdfcraft_fonts::document_arabic_font().is_none() { + assert_eq!(fnv1a(&bytes), SMALL_ITEM_FINGERPRINT, "output changed; the golden is the code before the sharding change"); + } } #[test] diff --git a/crates/edit/src/bidi.rs b/crates/edit/src/bidi.rs new file mode 100644 index 00000000..2c66a6ca --- /dev/null +++ b/crates/edit/src/bidi.rs @@ -0,0 +1,364 @@ +//! Right-to-left text in Edit text (Hebrew, Arabic). +//! +//! A content stream draws glyphs left to right, so a producer writes a right-to-left line in +//! *visual* order: `בדיוק` is drawn, and read back from the stream, as `קוידב`. The +//! editor works on *logical* (typing) order instead: [`to_logical`] turns a line read from the +//! page into the order a person types it, and [`to_visual`] turns a line about to be written back +//! into the order it must be drawn in (UAX #9 through `unicode-bidi`). +//! +//! Visual order doesn't always determine one logical order (`abc 12 םולש` is both `abc שלום 12` +//! and `abc 12 שלום`), so [`to_logical`] tries a few readings and keeps the first that draws back +//! exactly as the page did. Lines without right-to-left characters are returned unchanged. +//! +//! Combining marks (Hebrew points, Arabic harakat) stay after their base character in both +//! orders. Brackets drawn in a right-to-left run are mirrored, as UAX #9 L4 mirrors them on +//! display. + +use unicode_bidi::{BidiInfo, Level}; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum Class { + Rtl, + Ltr, + Number, + Neutral, +} + +/// Letters, marks and punctuation of the right-to-left scripts' blocks (digits excepted). +pub(crate) fn is_rtl(c: char) -> bool { + !is_digit(c) && matches!(u32::from(c), 0x0590..=0x08FF | 0xFB1D..=0xFDFF | 0xFE70..=0xFEFC | 0x1_0800..=0x1_0FFF | 0x1_E800..=0x1_EFFF) +} + +/// The Hebrew block and the Hebrew presentation forms. +pub(crate) fn is_hebrew(c: char) -> bool { + matches!(u32::from(c), 0x0590..=0x05FF | 0xFB1D..=0xFB4F) +} + +/// Hebrew points and cantillation marks (niqqud, te'amim): combining, drawn on their letter. +pub(crate) fn is_hebrew_mark(c: char) -> bool { + matches!(u32::from(c), 0x0591..=0x05BD | 0x05BF | 0x05C1..=0x05C2 | 0x05C4..=0x05C5 | 0x05C7 | 0xFB1E) +} + +fn is_digit(c: char) -> bool { + c.is_ascii_digit() || matches!(c, '\u{0660}'..='\u{0669}' | '\u{06F0}'..='\u{06F9}') +} + +/// Combining marks that ride on the character before them. +fn is_mark(c: char) -> bool { + is_hebrew_mark(c) + || matches!( + u32::from(c), + 0x0300..=0x036F | 0x0610..=0x061A | 0x064B..=0x065F | 0x0670 | 0x06D6..=0x06DC | 0x06DF..=0x06E4 | 0x06E7..=0x06E8 | 0x06EA..=0x06ED | 0xFE20..=0xFE2F + ) +} + +/// Direction marks, embeddings, overrides and isolates: they steer the reordering and are not drawn. +fn is_bidi_control(c: char) -> bool { + matches!(c, '\u{061C}' | '\u{200E}' | '\u{200F}' | '\u{202A}'..='\u{202E}' | '\u{2066}'..='\u{2069}') +} + +fn mirrored(c: char) -> char { + match c { + '(' => ')', + ')' => '(', + '[' => ']', + ']' => '[', + '{' => '}', + '}' => '{', + '<' => '>', + '>' => '<', + '«' => '»', + '»' => '«', + other => other, + } +} + +fn class(c: char) -> Class { + if is_digit(c) { + Class::Number + } else if is_rtl(c) { + Class::Rtl + } else if c.is_alphabetic() { + Class::Ltr + } else { + Class::Neutral + } +} + +/// A base character with the combining marks after it. +#[derive(Clone, Debug)] +struct Cluster { + text: String, + class: Class, +} + +impl Cluster { + /// The cluster with its base mirrored (marks stay as they are). + fn mirror(&self) -> Cluster { + let mut chars = self.text.chars(); + let mut text: String = chars.next().map(mirrored).into_iter().collect(); + text.extend(chars); + Cluster { text, class: self.class } + } +} + +fn clusters(s: &str) -> Vec { + let mut out: Vec = Vec::new(); + for c in s.chars() { + match out.last_mut() { + Some(last) if is_mark(c) => last.text.push(c), + _ => out.push(Cluster { text: c.to_string(), class: class(c) }), + } + } + out +} + +/// Whether the line reads right to left as a whole: more right-to-left letters than +/// left-to-right ones. Counting (not the first strong character) gives the same answer for a +/// line in either order, so reading a line and writing it back agree on its direction. +pub(crate) fn rtl_base(text: &str) -> bool { + let (mut r, mut l) = (0usize, 0usize); + for c in text.chars() { + match class(c) { + Class::Rtl if !is_mark(c) => r = r.saturating_add(1), + Class::Ltr if !is_mark(c) => l = l.saturating_add(1), + _ => {} + } + } + r > l +} + +/// Maximal ranges that start and end with a `member` and hold only members and `bridge`s. +fn runs(cl: &[Cluster], member: impl Fn(&Cluster) -> bool, bridge: impl Fn(&[Cluster], usize) -> bool) -> Vec> { + let mut out = Vec::new(); + let mut i = 0; + while i < cl.len() { + if !cl.get(i).is_some_and(&member) { + i += 1; + continue; + } + let mut end = i; + let mut j = i + 1; + while let Some(c) = cl.get(j) { + if member(c) { + end = j; + } else if !bridge(cl, j) { + break; + } + j += 1; + } + out.push(i..end + 1); + i = end + 1; + } + out +} + +/// One separator between two digits (`1,000`, `12:30`, `3.5`, `1/2`, `5-6`) keeps a number whole. +fn number_separator(cl: &[Cluster], j: usize) -> bool { + let digit = |k: Option| k.and_then(|k| cl.get(k)).is_some_and(|c| c.class == Class::Number); + cl.get(j).is_some_and(|c| matches!(c.text.as_str(), "." | "," | ":" | "/" | "-" | "+")) && digit(j.checked_sub(1)) && digit(Some(j + 1)) +} + +fn reverse_mirrored(cl: &mut [Cluster]) { + cl.reverse(); + for c in cl.iter_mut() { + *c = c.mirror(); + } +} + +/// One reading of `visual` in logical order. `wide` lets runs of the other direction swallow the +/// spaces and numbers between their words; narrow keeps numbers and words apart. +fn reading(visual: &[Cluster], rtl: bool, wide: bool) -> String { + let mut cl = visual.to_vec(); + let number = |c: &Cluster| c.class == Class::Number; + if rtl { + // Everything reads right to left, except left-to-right words and numbers. + reverse_mirrored(&mut cl); + let ltr = |c: &Cluster| matches!(c.class, Class::Ltr | Class::Number); + let bridge = + |cl: &[Cluster], j: usize| cl.get(j).is_some_and(|c| c.class == Class::Neutral) && (wide || !neutral_between_word_and_number(cl, j)); + for r in runs(&cl, ltr, bridge) { + if let Some(run) = cl.get_mut(r) { + reverse_mirrored(run); + } + } + } else { + // Left to right, except right-to-left stretches; numbers inside those read left to right. + let rtl_member = |c: &Cluster| c.class == Class::Rtl; + let bridge = |cl: &[Cluster], j: usize| cl.get(j).is_some_and(|c| c.class == Class::Neutral || (wide && c.class == Class::Number)); + for r in runs(&cl, rtl_member, bridge) { + if let Some(run) = cl.get_mut(r) { + reverse_mirrored(run); + for n in runs(run, number, number_separator) { + if let Some(num) = run.get_mut(n) { + reverse_mirrored(num); + } + } + } + } + } + cl.iter().map(|c| c.text.as_str()).collect() +} + +/// Whether the neutral at `j` sits between a left-to-right word on one side and a number on the +/// other (UAX #9 resolves such a space to the line's direction, splitting the run). +fn neutral_between_word_and_number(cl: &[Cluster], j: usize) -> bool { + let strong = |mut k: usize, step: isize| -> Option { + loop { + k = k.checked_add_signed(step)?; + match cl.get(k)?.class { + Class::Neutral => continue, + other => return Some(other), + } + } + }; + matches!((strong(j, -1), strong(j, 1)), (Some(Class::Ltr), Some(Class::Number)) | (Some(Class::Number), Some(Class::Ltr))) +} + +/// A line read from a content stream (drawn left to right), in logical order. +pub fn to_logical(visual: &str) -> String { + if !visual.chars().any(is_rtl) { + return visual.to_owned(); + } + let cl = clusters(visual); + let rtl = rtl_base(visual); + let readings = [reading(&cl, rtl, true), reading(&cl, rtl, false)]; + // Readings that draw back exactly as the page did; of those, one whose brackets pair up + // (`12 (PDF)`, not `PDF) 12)`), else the first. + let exact: Vec<&String> = readings.iter().filter(|r| to_visual(r) == visual).collect(); + let chosen = exact.iter().find(|r| balanced(r)).or(exact.first()).map_or_else(|| readings[0].clone(), |r| (*r).clone()); + // Some producers (LibreOffice) map a bracket drawn in a right-to-left run to the character it + // stands for, not the shape drawn, so mirroring it again turns it inside out: `)טיוטה(`. + let swapped: String = chosen.chars().map(mirrored).collect(); + if !balanced(&chosen) && balanced(&swapped) { swapped } else { chosen } +} + +/// Whether every closing bracket closes an open one. +fn balanced(text: &str) -> bool { + let mut open: Vec = Vec::new(); + for c in text.chars() { + match c { + '(' | '[' | '{' => open.push(c), + ')' | ']' | '}' if open.pop() != Some(mirrored(c)) => return false, + _ => {} + } + } + open.is_empty() +} + +/// A line in logical order, in the order it must be drawn (left to right), without the bidi +/// controls fonts can't draw. +pub fn to_visual(logical: &str) -> String { + let text: String = logical.chars().filter(|c| !is_bidi_control(*c)).collect(); + if !text.chars().any(is_rtl) { + return text; + } + let level = if rtl_base(&text) { Level::rtl() } else { Level::ltr() }; + let info = BidiInfo::new(&text, Some(level)); + let mut out = String::with_capacity(text.len()); + for para in &info.paragraphs { + let (levels, runs) = info.visual_runs(para, para.range.clone()); + for run in runs { + let rtl = levels.get(run.start).is_some_and(|l| l.is_rtl()); + // Runs lie on character boundaries; a missing slice is skipped, never a panic. + let Some(piece) = text.get(run) else { continue }; + if rtl { + for c in clusters(piece).iter().rev() { + out.push_str(&c.mirror().text); + } + } else { + out.push_str(piece); + } + } + } + out +} + +#[cfg(test)] +mod tests { + use super::*; + + /// What a producer writes into the content stream for `logical`. + fn drawn(logical: &str) -> String { + to_visual(logical) + } + + #[test] + fn the_reported_word_reads_back_in_typing_order() { + // The report: Edit text showed "בדיוק" (exactly) reversed. + assert_eq!(drawn("בדיוק"), "קוידב"); + assert_eq!(to_logical("קוידב"), "בדיוק"); + } + + #[test] + fn hebrew_sentences_numbers_latin_and_brackets_round_trip() { + for logical in [ + "שלום עולם", + "עמוד 12 מתוך 40", + "מחיר: 1,250.50 ₪", + "השעה 12:30", + "קובץ PDF חדש", + "שלום (עולם)", + "ראו [1] ו-[2]", + "גרסה 2.0 של PdfCraft נוסחה", + "hello שלום עולם world", + "abc שלום def", + "שלום abc def עולם", + "Report: דוח שנתי 2026", + "שלום עולם, עמוד 12 (PDF)", + "פרק 3 (מבוא)", + "שָׁלוֹם עוֹלָם", + "مرحبا بالعالم", + ] { + let visual = drawn(logical); + assert_ne!(visual, logical, "{logical:?} should be reordered"); + assert_eq!(to_visual(&to_logical(&visual)), visual, "{logical:?} drawn as {visual:?}"); + assert_eq!(to_logical(&visual), logical, "{logical:?} drawn as {visual:?}"); + } + } + + #[test] + fn ambiguous_lines_still_draw_back_the_same() { + // "abc 12 םולש" is both "abc שלום 12" and "abc 12 שלום"; either must draw the same. + for logical in ["abc שלום 12", "שלום 12 abc", "a (ב) c", "דוח Report 2026", "(1) שלום"] { + let visual = drawn(logical); + assert_eq!(to_visual(&to_logical(&visual)), visual, "{logical:?}"); + } + } + + #[test] + fn brackets_mapped_to_their_logical_character_pair_up() { + // LibreOffice draws "(טיוטה)" with ToUnicode giving the logical brackets: `)הטויט(`. + assert_eq!(to_logical(")הטויט("), "(טיוטה)"); + // Brackets mapped to the shape drawn read the same. + assert_eq!(to_logical("(הטויט)"), "(טיוטה)"); + } + + #[test] + fn left_to_right_text_is_untouched() { + for s in ["", "Hello, world (1)", "日本語のテキスト", "Ünïcödé 12:30"] { + assert_eq!(to_logical(s), s); + assert_eq!(to_visual(s), s); + } + } + + #[test] + fn points_stay_on_their_letter() { + // Each point follows its letter in both orders. + let visual = drawn("שָׁלוֹם"); + let first = visual.chars().next(); + assert_eq!(first, Some('ם')); + assert!(visual.contains("לוֹ") || visual.contains("וֹל"), "{visual:?}"); + assert_eq!(to_logical(&visual), "שָׁלוֹם"); + } + + #[test] + fn controls_are_not_drawn_and_hostile_input_does_not_panic() { + assert_eq!(to_visual("\u{200F}שלום\u{202B}"), "םולש"); + for s in ["\u{05B8}", "\u{05B8}\u{05B8}א", "(((", "א\u{0300}", "1-", "-1א", "א\n\tב", "\u{202E}abc"] { + let _ = to_visual(&to_logical(s)); + } + let long = "שלום 12 abc (x) ".repeat(2_000); + assert_eq!(to_visual(&to_logical(&to_visual(&long))), to_visual(&long)); + } +} diff --git a/crates/edit/src/lib.rs b/crates/edit/src/lib.rs index 0e97c63e..2ddb348f 100644 --- a/crates/edit/src/lib.rs +++ b/crates/edit/src/lib.rs @@ -629,6 +629,7 @@ mod flatten; pub use flatten::{flatten, flatten_fill_sign}; pub mod images; pub use images::{ImageChange, PageImage, change_image, page_images, reading_images, rect_to_rect, turn_about_centre}; +pub mod bidi; pub mod text; pub use text::{BlockStyle, LineEdit, TextBlock, TextLine, reading_blocks, replace_block, replace_line, rewrite_block, text_blocks, text_lines}; pub mod added; diff --git a/crates/edit/src/tests.rs b/crates/edit/src/tests.rs index 720b6fcd..4a68edf6 100644 --- a/crates/edit/src/tests.rs +++ b/crates/edit/src/tests.rs @@ -1779,6 +1779,148 @@ fn the_graphics_state_carries_over_between_content_streams() { assert!(close(l.rect[0], 100.0) && close(l.size, 10.0), "{l:?}"); } +/// One page whose font draws a few Hebrew letters (ToUnicode): A ב, B ד, C י, D ו, E ק, F ש, +/// G ל, H ם. Content streams draw Hebrew in visual order, so "בדיוק" is `(EDCBA)`. +fn hebrew_page(content: &str) -> Document { + let cmap = "/CIDInit /ProcSet findresource begin 12 dict begin begincmap /CMapType 2 def \ + 1 begincodespacerange <00> endcodespacerange 9 beginbfchar \ + <20> <0020> <41> <05D1> <42> <05D3> <43> <05D9> <44> <05D5> <45> <05E7> <46> <05E9> <47> <05DC> <48> <05DD> \ + endbfchar endcmap CMapName currentdict /CMap defineresource pop end end"; + // Widths for codes 32..=72: a space, then the eight letters at 65..=72. + let widths: Vec<&str> = (32..=72) + .map(|c| { + if c == 32 { + "250" + } else if c >= 65 { + "500" + } else { + "0" + } + }) + .collect(); + let objs: Vec = vec![ + "<< /Type /Catalog /Pages 2 0 R >>".into(), + "<< /Type /Pages /Kids [3 0 R] /Count 1 >>".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 600 800] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>".into(), + format!("<< /Length {} >>\nstream\n{content}\nendstream", content.len()), + format!( + "<< /Type /Font /Subtype /Type1 /BaseFont /ABCDEF+FakeHebrew /FirstChar 32 /LastChar 72 /Widths [{}] /ToUnicode 6 0 R >>", + widths.join(" ") + ), + format!("<< /Length {} >>\nstream\n{cmap}\nendstream", cmap.len()), + ]; + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offs = Vec::new(); + for (i, o) in objs.iter().enumerate() { + offs.push(out.len()); + out.extend_from_slice(format!("{} 0 obj\n{o}\nendobj\n", i + 1).as_bytes()); + } + let x = out.len(); + out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objs.len() + 1).as_bytes()); + for o in offs { + out.extend_from_slice(format!("{o:010} 00000 n \n").as_bytes()); + } + out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{x}\n%%EOF\n", objs.len() + 1).as_bytes()); + Document::open(Arc::new(out)).unwrap() +} + +/// The report: Edit text showed "בדיוק" (exactly) reversed, because the line was read in the +/// order the page draws it. +#[test] +fn hebrew_lines_read_in_typing_order() { + let doc = hebrew_page("BT /F1 12 Tf 300 700 Td (EDCBA) Tj ET BT /F1 12 Tf 300 600 Td (EDCBA HDGF) Tj ET"); + let lines: Vec = text::text_lines(&doc, 0).unwrap().into_iter().map(|l| l.text).collect(); + assert_eq!(lines, ["בדיוק", "שלום בדיוק"]); + let blocks: Vec = text::text_blocks(&doc, 0).unwrap().into_iter().map(|b| b.text).collect(); + assert_eq!(blocks, ["בדיוק", "שלום בדיוק"]); +} + +/// Text the page's own font can show is written back in drawing order, right-aligned to where +/// the line ended, and reads back as typed. +#[test] +fn hebrew_edits_are_drawn_right_to_left_and_keep_their_right_edge() { + let mut doc = hebrew_page("BT /F1 12 Tf 300 700 Td (EDCBA) Tj ET"); + let before = text::text_lines(&doc, 0).unwrap()[0].rect; + // One line: the bytes are the visual order of the typed text. + assert_eq!(text::replace_line(&mut doc, 0, 0, "שלום בדיוק").unwrap().substituted, None); + let content = String::from_utf8_lossy(&page_content_bytes(&doc, 0)).into_owned(); + assert!(content.contains("(EDCBA HDGF)"), "{content}"); + assert_eq!(text::text_lines(&reopen(&doc), 0).unwrap()[0].text, "שלום בדיוק"); + + // A paragraph: shorter text stays right-aligned at the old right edge. + let mut doc = hebrew_page("BT /F1 12 Tf 300 700 Td (EDCBA) Tj ET"); + assert_eq!(text::replace_block(&mut doc, 0, 0, "דוב").unwrap().substituted, None); + let content = String::from_utf8_lossy(&page_content_bytes(&doc, 0)).into_owned(); + assert!(content.contains("(ADB)"), "{content}"); + let doc = reopen(&doc); + let line = text::text_lines(&doc, 0).unwrap().swap_remove(0); + assert_eq!(line.text, "דוב"); + assert!((line.rect[2] - before[2]).abs() < 0.5, "right edge {} → {}", before[2], line.rect[2]); + assert!(line.rect[0] > before[0] + 1.0, "shorter text starts further right: {:?} vs {before:?}", line.rect); + + // Longer text grows to the left, not off the right edge. + let mut doc = hebrew_page("BT /F1 12 Tf 300 700 Td (EDCBA) Tj ET"); + text::replace_block(&mut doc, 0, 0, "שלום בדיוק שלום").unwrap(); + let doc = reopen(&doc); + let line = text::text_lines(&doc, 0).unwrap().swap_remove(0); + assert_eq!(line.text, "שלום בדיוק שלום"); + assert!((line.rect[2] - before[2]).abs() < 0.5, "right edge {} → {}", before[2], line.rect[2]); + assert!(line.rect[0] < before[0] - 1.0, "longer text grows left: {:?} vs {before:?}", line.rect); + + // An alignment asked for still wins. + let mut doc = hebrew_page("BT /F1 12 Tf 300 700 Td (EDCBA) Tj ET"); + let left = text::BlockStyle { align: Some(Align::Left), ..Default::default() }; + text::rewrite_block(&mut doc, 0, 0, Some("דוב"), &left).unwrap(); + let line = text::text_lines(&reopen(&doc), 0).unwrap().swap_remove(0); + assert!((line.rect[0] - before[0]).abs() < 0.5, "left-aligned: {:?} vs {before:?}", line.rect); +} + +/// Letters the page's font lacks are drawn with the Hebrew face from craft-fonts; a build +/// without it says so and changes nothing. Hebrew points are refused, not misdrawn. +#[test] +fn hebrew_letters_the_page_font_lacks_use_the_hebrew_face() { + let mut doc = hebrew_page("BT /F1 12 Tf 300 700 Td (EDCBA) Tj ET"); + let before = page_content_bytes(&doc, 0); + let pointed = text::replace_line(&mut doc, 0, 0, "שָׁלוֹם עוֹלָם"); + let Err(EditError::Invalid(msg)) = pointed else { panic!("expected a clear error, got {pointed:?}") }; + assert!(msg.contains("niqqud"), "{msg}"); + assert_eq!(page_content_bytes(&doc, 0), before); + + // ע and ה aren't in the page's font. + let typed = "שלום עולם, עמוד 12 (PDF)"; + let line = text::replace_line(&mut doc, 0, 0, typed); + if pdfcraft_fonts::document_hebrew_font(false).is_none() { + let Err(EditError::Invalid(msg)) = line else { panic!("expected a clear error, got {line:?}") }; + assert!(msg.contains("Hebrew fallback font") && msg.contains("CRAFT_FONTS_DIR"), "{msg}"); + assert_eq!(page_content_bytes(&doc, 0), before); + eprintln!("built without a craft-fonts Hebrew face: the Type 3 path is checked with CRAFT_FONTS_DIR"); + return; + } + assert_eq!(line.unwrap().substituted.as_deref(), Some("Noto Sans Hebrew Type3")); + let content = String::from_utf8_lossy(&page_content_bytes(&doc, 0)).into_owned(); + assert!(content.contains("/PCHe"), "{content}"); + let reopened = reopen(&doc); + assert_eq!(text::text_lines(&reopened, 0).unwrap()[0].text, typed); + // Every glyph has an outline, so nothing is drawn as a box. + let p = pdfcraft_model::pages(&reopened).swap_remove(0); + let res = reopened.resolve(p.dict.get(b"Resources").unwrap()); + let fonts = reopened.resolve(res.as_dict().unwrap().get(b"Font").unwrap()); + let font = reopened.resolve(fonts.as_dict().unwrap().get(b"PCHe").unwrap()); + let procs = reopened.resolve(font.as_dict().unwrap().get(b"CharProcs").unwrap()); + let procs = procs.as_dict().unwrap(); + assert_eq!(procs.iter().count(), typed.chars().collect::>().len()); + for (_, glyph) in procs.iter() { + let pdfcraft_cos::Object::Stream(s) = &*reopened.resolve(glyph) else { panic!("glyph stream") }; + let path = String::from_utf8(s.decoded().unwrap()).unwrap(); + assert!(path.starts_with(|c: char| c.is_ascii_digit()) && path.contains(" d1"), "{path}"); + } + + // A whole paragraph too, right-aligned. + let mut doc = hebrew_page("BT /F1 12 Tf 300 700 Td (EDCBA) Tj ET"); + assert_eq!(text::replace_block(&mut doc, 0, 0, "עברית מלאה").unwrap().substituted.as_deref(), Some("Noto Sans Hebrew Type3")); + assert_eq!(text::text_blocks(&reopen(&doc), 0).unwrap()[0].text, "עברית מלאה"); +} + /// #125: page-content tools refuse text the standard fonts can't draw instead of writing `?`, /// and leave the document untouched. #[test] diff --git a/crates/edit/src/text.rs b/crates/edit/src/text.rs index e84ab1ad..3a0d4a10 100644 --- a/crates/edit/src/text.rs +++ b/crates/edit/src/text.rs @@ -38,7 +38,7 @@ pub struct TextLine { /// Whether new text in this font can only be shown by substituting another font (no /// Unicode mapping for the line, so nothing could be reused). pub decodable: bool, - /// The Japanese fallback face the font subsets, when the line is in one (see [`same_font`]). + /// The Japanese or Hebrew fallback face the font subsets, when the line is in one (see [`same_font`]). face: Option, /// The content stream piece the line starts in, and its operators (indices into the page's /// streams parsed joined). @@ -152,7 +152,7 @@ struct Ts { /// The operators that set the current fill colour (`g`, `rg`, `k`, or `cs` + `sc`/`scn`). fill: Vec, font: Option<(Vec, Rc)>, - /// The Japanese fallback face the font subsets, when the font is one (see [`is_japanese_fallback`]). + /// The Japanese or Hebrew fallback face the font subsets, when the font is one (see [`is_type3_fallback`]). face: Option, size: f64, char_spacing: f64, @@ -358,7 +358,7 @@ fn interpret( let (m, face) = cache.entry(name.to_vec()).or_insert_with(|| { let dict = fonts_res.get(name).and_then(|f| doc.resolve(f).as_dict().cloned()); let metrics = dict.as_ref().map_or_else(Metrics::fallback, |d| Metrics::from_dict(doc, d)); - let face = dict.filter(is_japanese_fallback).map(|_| metrics.base_font.clone()); + let face = dict.filter(is_type3_fallback).map(|_| metrics.base_font.clone()); (Rc::new(metrics), face) }); ts.font = Some((name.to_vec(), m.clone())); @@ -555,6 +555,10 @@ pub fn text_lines(doc: &Document, page: usize) -> Result, EditErro } last = Some((si, s.bt, s.baseline, s.end_x, s.size.max(1.0))); } + // Content streams draw right-to-left lines in visual order; the editor works in typing order. + for l in &mut lines { + l.text = crate::bidi::to_logical(&l.text); + } // Lines of only spaces aren't editable text. lines.retain(|l| !l.text.trim().is_empty()); Ok(lines) @@ -741,14 +745,17 @@ fn needs_type3(text: &str) -> bool { /// The `/Name` of the Type 3 fonts generated for Japanese text. const JAPANESE_FONT_NAME: &str = "PCJapanese"; +/// The `/Name` of the Type 3 fonts generated for Hebrew text. +const HEBREW_FONT_NAME: &str = "PCHebrew"; -/// Whether `font` is one of those generated fonts: a replacement longer than one of them is split -/// across several, which read back as one line (see [`same_font`]). -fn is_japanese_fallback(font: &Dict) -> bool { - font.name(b"Subtype") == Some(b"Type3") && font.name(b"Name") == Some(JAPANESE_FONT_NAME.as_bytes()) +/// Whether `font` is one of those generated fonts (Japanese or Hebrew): a replacement longer than +/// one of them is split across several, which read back as one line (see [`same_font`]). +fn is_type3_fallback(font: &Dict) -> bool { + font.name(b"Subtype") == Some(b"Type3") + && font.name(b"Name").is_some_and(|n| n == JAPANESE_FONT_NAME.as_bytes() || n == HEBREW_FONT_NAME.as_bytes()) } -/// Whether a run shown in `font` (a resource name, and the Japanese fallback face it subsets, if +/// Whether a run shown in `font` (a resource name, and the Japanese or Hebrew fallback face it subsets, if /// any) continues `line`: the same font resource, or another subset of the same face. fn same_font(line: &TextLine, font: &[u8], face: Option<&str>) -> bool { line.font.as_bytes() == font || (line.face.is_some() && line.face.as_deref() == face) @@ -777,7 +784,7 @@ const MAX_TYPE3_GLYPHS: usize = 240; struct Type3Fallback { /// The family its glyphs come from ("Shippori Mincho"). family: &'static str, - /// The fonts' resource names in order; the first is `PCJp`. + /// The fonts' resource names in order; the first is `PCJp` (`PCHe` for Hebrew text). names: Vec, /// Every unique character with its advance (em), in order of first appearance. widths: Vec<(char, f64)>, @@ -800,8 +807,8 @@ fn pdf_num(v: f64) -> String { fn type3_path(face: &CraftFont, ch: char) -> Result<(Vec, f64), EditError> { let glyph = japanese_glyph_from(face, ch).map_err(|e| match e { GlyphError::NoFont => no_japanese_font(), - GlyphError::Missing => EditError::Invalid(format!("no fallback font has a glyph for U+{:04X}", ch as u32)), - GlyphError::TooComplex => EditError::Invalid(format!("Japanese fallback glyph U+{:04X} is too complex", ch as u32)), + GlyphError::Missing => EditError::Invalid(format!("the {} fallback font has no glyph for U+{:04X}", face.family, ch as u32)), + GlyphError::TooComplex => EditError::Invalid(format!("the {} fallback glyph U+{:04X} is too complex", face.family, ch as u32)), })?; let (dx, width) = proportional(ch, &glyph); let scale = 1000.0; @@ -865,6 +872,32 @@ fn fallback_face<'a>(faces: &[&'a CraftFont], chars: &[char], has_glyph: impl Fn faces.iter().copied().find(|face| chars.iter().all(|ch| has_glyph(face, *ch))).or_else(|| faces.first().copied()) } +/// This build has no Hebrew face to draw replacement text with. +fn no_hebrew_font() -> EditError { + EditError::Invalid( + "this text needs PdfCraft's Hebrew fallback font (Noto Sans Hebrew), which this build doesn't include \ + (to build it in, set CRAFT_FONTS_DIR to a craft-fonts checkout that has it)" + .into(), + ) +} + +/// The faces a Type 3 replacement may draw `text` with, best first. Text with Hebrew takes the +/// Hebrew face first (its Latin and digits too, so a line keeps one look), then the Japanese +/// faces for anything it lacks; other text takes the Japanese faces, as before. +fn type3_faces(text: &str, family: crate::added::Family, bold: bool) -> Result<(Vec<&'static CraftFont>, bool), EditError> { + let japanese = pdfcraft_fonts::document_japanese_fonts_for_style(family == crate::added::Family::Times, bold); + if !text.chars().any(crate::bidi::is_hebrew) { + return Ok((japanese, false)); + } + // Points are drawn on their letter with the font's positioning, which a one-glyph-per- + // character Type 3 font can't do; refuse rather than print them beside the letter. + if text.chars().any(crate::bidi::is_hebrew_mark) { + return Err(EditError::Invalid("Hebrew points (niqqud) can't be written into a PDF yet; type the text without them".into())); + } + let hebrew = pdfcraft_fonts::document_hebrew_font(bold).ok_or_else(no_hebrew_font)?; + Ok((std::iter::once(hebrew).chain(japanese).collect(), true)) +} + fn type3_font( doc: &mut Document, fonts_res: &mut Dict, @@ -873,7 +906,7 @@ fn type3_font( bold: bool, max_glyphs: usize, ) -> Result { - let faces = pdfcraft_fonts::document_japanese_fonts_for_style(family == crate::added::Family::Times, bold); + let (faces, hebrew) = type3_faces(text, family, bold)?; // Each unique character once, in order of first appearance. let mut chars = Vec::new(); let mut seen = std::collections::HashSet::new(); @@ -886,16 +919,20 @@ fn type3_font( return Err(EditError::Invalid("replacement text is empty".into())); } // The faces differ in coverage (e.g. of Cyrillic), so use the best face that has every - // character. When none has them all, the best face reports the character it lacks. - let face = fallback_face(&faces, &chars, |face, ch| japanese_glyph_from(face, ch).is_ok()).ok_or_else(no_japanese_font)?; + // character. When none has them all, the best face reports the character it lacks. Hebrew + // text keeps the Hebrew face and takes each character it lacks from the next face that has it. + let face = if hebrew { faces.first().copied() } else { fallback_face(&faces, &chars, |face, ch| japanese_glyph_from(face, ch).is_ok()) } + .ok_or_else(no_japanese_font)?; + let others: &[&CraftFont] = if hebrew { faces.get(1..).unwrap_or_default() } else { &[] }; // Every outline is read before the document is touched, so a character that fails leaves it // as it was. Each font takes the next `per_font` characters; codes restart at 1 in each. let per_font = max_glyphs.clamp(1, MAX_TYPE3_GLYPHS); let mut glyphs = Vec::with_capacity(chars.len()); for (i, ch) in chars.into_iter().enumerate() { - let (path, width) = type3_path(face, ch)?; + // Hebrew text: the first face with the glyph, else the Hebrew face's error. + let (path, width) = others.iter().fold(type3_path(face, ch), |found, f| found.or_else(|e| type3_path(f, ch).map_err(|_| e)))?; let code = u8::try_from((i % per_font).saturating_add(1)) - .map_err(|_| EditError::Invalid("Japanese replacement has too many unique characters".into()))?; + .map_err(|_| EditError::Invalid("replacement text has too many unique characters".into()))?; glyphs.push(Type3Glyph { code, ch, path, width }); } let mut codes = HashMap::with_capacity(glyphs.len()); @@ -911,14 +948,15 @@ fn type3_font( codes.insert(glyph.ch, (names.len(), glyph.code)); widths.push((glyph.ch, glyph.width)); } - names.push(add_type3_font(doc, fonts_res, face, group)); + names.push(add_type3_font(doc, fonts_res, face, hebrew, group)); } Ok(Type3Fallback { family: face.family, names, widths, codes }) } /// Adds one generated font whose codes 1 to `glyphs.len()` show `glyphs`, and returns its resource -/// name (`PCJp`, then `PCJp1`, ...). Nothing here can fail: the outlines were read already. -fn add_type3_font(doc: &mut Document, fonts_res: &mut Dict, face: &CraftFont, glyphs: Vec) -> String { +/// name (`PCJp`, then `PCJp1`, ...; `PCHe`, ... for Hebrew text). Nothing here can fail: the +/// outlines were read already. +fn add_type3_font(doc: &mut Document, fonts_res: &mut Dict, face: &CraftFont, hebrew: bool, glyphs: Vec) -> String { let last_char = glyphs.len(); let mut charprocs = Dict::new(); let mut widths = Vec::with_capacity(last_char); @@ -947,7 +985,7 @@ fn add_type3_font(doc: &mut Document, fonts_res: &mut Dict, face: &CraftFont, gl let mut font = Dict::new(); font.set(b"Type".to_vec(), Object::name("Font")); font.set(b"Subtype".to_vec(), Object::name("Type3")); - font.set(b"Name".to_vec(), Object::name(JAPANESE_FONT_NAME)); + font.set(b"Name".to_vec(), Object::name(if hebrew { HEBREW_FONT_NAME } else { JAPANESE_FONT_NAME })); let mut descriptor = Dict::new(); descriptor.set(b"Type".to_vec(), Object::name("FontDescriptor")); descriptor.set(b"FontName".to_vec(), Object::name(&format!("{}-{}", face.family, face.style).replace(' ', ""))); @@ -967,11 +1005,12 @@ fn add_type3_font(doc: &mut Document, fonts_res: &mut Dict, face: &CraftFont, gl font.set(b"Encoding".to_vec(), Object::Dict(encoding)); font.set(b"CharProcs".to_vec(), Object::Dict(charprocs)); font.set(b"ToUnicode".to_vec(), Object::Ref(cmap_ref)); - let mut name = String::from("PCJp"); + let prefix = if hebrew { "PCHe" } else { "PCJp" }; + let mut name = String::from(prefix); let mut suffix = 0usize; while fonts_res.contains(name.as_bytes()) { suffix = suffix.saturating_add(1); - name = format!("PCJp{suffix}"); + name = format!("{prefix}{suffix}"); } fonts_res.set(name.as_bytes().to_vec(), Object::Dict(font)); name @@ -1010,7 +1049,9 @@ pub(crate) fn replace_line_with_limit(doc: &mut Document, page: usize, line: usi let first = target.ops.first().copied().filter(|i| *i < ops.len()).ok_or_else(|| EditError::Invalid("the page's content changed".into()))?; // The line's own font, when it can show every character. let font = fonts_res.get(target.font.as_bytes()).and_then(|f| doc.resolve(f).as_dict().cloned()).map(|d| Metrics::from_dict(doc, &d)); - let reused = font.as_ref().and_then(|m| m.encode(&text)); + // Typed in logical order; drawn left to right (right-to-left runs reversed). + let shown = crate::bidi::to_visual(&text); + let reused = font.as_ref().and_then(|m| m.encode(&shown)); let mut substituted = None; let mut replacement: Vec = Vec::new(); // ' and " also move to the next line; keep that. @@ -1028,8 +1069,8 @@ pub(crate) fn replace_line_with_limit(doc: &mut Document, page: usize, line: usi None => { if needs_type3(&text) { let fallback = type3_font(doc, &mut fonts_res, &text, source_family(&target.base_font), target.bold, max_glyphs)?; - let runs = type3_encode(&fallback, &text) - .ok_or_else(|| EditError::Invalid(format!("\"{text}\" can't be shown by the Japanese fallback")))?; + let runs = type3_encode(&fallback, &shown) + .ok_or_else(|| EditError::Invalid(format!("\"{text}\" can't be shown by the {} fallback", fallback.family)))?; let size = font_size_before(&ops, first).unwrap_or(target.size); for (name, bytes) in runs { replacement.push(Op::new("Tf", vec![Object::name(&name), pdfcraft_content::num(size)])); @@ -1038,10 +1079,10 @@ pub(crate) fn replace_line_with_limit(doc: &mut Document, page: usize, line: usi replacement.push(Op::new("Tf", vec![Object::name(&target.font), pdfcraft_content::num(size)])); substituted = Some(format!("{} Type3", fallback.family)); } else { - let win = pdfcraft_fonts::win_ansi(&text); + let win = pdfcraft_fonts::win_ansi(&shown); // WinAnsi turns what it can't show into '?'; refuse rather than print the wrong thing. let back: String = win.iter().map(|b| char::from_u32(u32::from(*b)).unwrap_or('?')).collect(); - if text.chars().zip(back.chars()).any(|(a, b)| b == '?' && a != '?') { + if shown.chars().zip(back.chars()).any(|(a, b)| b == '?' && a != '?') { return Err(EditError::Invalid(format!("\"{text}\" has characters neither {} nor Helvetica can show", target.base_font))); } // The size in text space: the current Tf's size. @@ -1292,15 +1333,26 @@ pub(crate) fn rewrite_block_with_limit( if !(dx.is_finite() && dy.is_finite()) { return Err(EditError::Invalid("the paragraph can't be moved that far".into())); } - // A paragraph keeps its width (or takes the one asked for); a single line grows to the right, - // up to the page's margin. + // Right-to-left text (more Hebrew or Arabic letters than Latin) is right-aligned unless an + // alignment is asked for, and a single right-to-left line keeps its right edge as it grows. + let rtl = crate::bidi::rtl_base(&text); + let anchored_right = rtl && style.width.is_none() && members.len() == 1; + // A paragraph keeps its width (or takes the one asked for); a single line grows to the right + // (to the left when right-to-left), up to the page's margin. let width = match style.width { // Capped at the largest page PDF allows (14 400 pt). Some(w) if w.is_finite() => w.clamp(size * k, 14_400.0), Some(_) => return Err(EditError::Invalid("the paragraph's width must be a number".into())), + None if anchored_right => (b.rect[2] - b.rect[0]).max(size * k).max(b.rect[2] + dx - (p.crop(doc)[0] + 36.0)), None if members.len() == 1 => (b.rect[2] - b.rect[0]).max(size * k).max(p.crop(doc)[2] - 36.0 - (b.rect[0] + dx)), None => (b.rect[2] - b.rect[0]).max(size * k), }; + if !width.is_finite() { + return Err(EditError::Invalid("the paragraph's width must be a number".into())); + } + let align = style.align.or(rtl.then_some(crate::added::Align::Right)); + // Where a right-aligned line ends, from the paragraph's left edge (user space). + let right_edge = if anchored_right { (b.rect[2] - b.rect[0]).max(0.0) } else { width }; // The move and the width are checked here, before the fallback fonts are written: a refused // paragraph leaves the document as it was. The move is in page space, so it is taken back through // the CTM's linear part into the space the paragraph is drawn in. @@ -1397,32 +1449,38 @@ pub(crate) fn rewrite_block_with_limit( // Alignment: each line's offset from the left edge, in text space. let offset = |line: &str| -> f64 { let free = (width - advance(line)).max(0.0) / k; - match style.align { + match align { Some(crate::added::Align::Center) => free / 2.0, + // A right-to-left line anchored at its right edge may start left of the old box. + Some(crate::added::Align::Right) if anchored_right => (right_edge - advance(line)) / k, Some(crate::added::Align::Right) => free, + // A justified right-to-left paragraph ends its last line at the right edge. + Some(crate::added::Align::Justify) if rtl => free, _ => 0.0, } }; // Justify: word spacing (single-byte code 32 only) so each line but the last fills the width. let single_byte = !reuse || metrics.as_ref().is_some_and(|m| !m.composite); - let justify = style.align == Some(crate::added::Align::Justify) && single_byte; + let justify = align == Some(crate::added::Align::Justify) && single_byte; let mut x = 0.0; let mut tw_set = 0.0; let mut underlines: Vec<(f64, f64, f64)> = Vec::new(); // (x0, x1, y) in text space for (i, line) in wrapped.iter().enumerate() { - let dx = offset(line); + let spaces = line.matches(' ').count(); + let tw = + if justify && i + 1 < wrapped.len() && spaces > 0 { (width - advance(line)).max(0.0) / k / o_state.scale / spaces as f64 } else { 0.0 }; + // A stretched line fills the width from the left edge. + let dx = if tw > 0.0 { 0.0 } else { offset(line) }; if i > 0 || dx != 0.0 { block_ops.push(Op::new("Td", vec![n(dx - x), n(if i > 0 { -lead } else { 0.0 })])); } x = dx; - let spaces = line.matches(' ').count(); - let tw = - if justify && i + 1 < wrapped.len() && spaces > 0 { (width - advance(line)).max(0.0) / k / o_state.scale / spaces as f64 } else { 0.0 }; if tw != tw_set { block_ops.push(Op::new("Tw", vec![n(tw)])); tw_set = tw; } - for (font, bytes) in encode(line).ok_or_else(|| EditError::Invalid(format!("\"{line}\" can't be shown")))? { + // Each wrapped line in typing order; drawn left to right (right-to-left runs reversed). + for (font, bytes) in encode(&crate::bidi::to_visual(line)).ok_or_else(|| EditError::Invalid(format!("\"{line}\" can't be shown")))? { if font != shown_font { block_ops.push(Op::new("Tf", vec![Object::name(&font), n(size)])); shown_font = font; @@ -1695,4 +1753,27 @@ mod tests { assert_eq!(fallback_face(&faces, &['a', 'ф'], |_, ch| ch.is_ascii()).map(|f| f.family), Some("Mincho")); assert!(fallback_face(&[], &['a'], has).is_none()); } + + #[test] + fn hebrew_and_japanese_fallback_shards_are_recognized_as_ours() { + static HEBR: &[&str] = &["Hebr"]; + let face = CraftFont { family: "FakeHebrew", style: "Regular", scripts: HEBR, bytes: b"" }; + let glyph = |code, ch| Type3Glyph { code, ch, path: b"0 0 0 0 0 0 d1\nf\n".to_vec(), width: 0.5 }; + let mut doc = Document::new_empty(); + let mut fonts = Dict::new(); + // A replacement split across two Hebrew fonts, then one Japanese font. + let he = add_type3_font(&mut doc, &mut fonts, &face, true, vec![glyph(1, 'א')]); + let he1 = add_type3_font(&mut doc, &mut fonts, &face, true, vec![glyph(1, 'ב')]); + let jp = add_type3_font(&mut doc, &mut fonts, &face, false, vec![glyph(1, 'あ')]); + assert_eq!([he.as_str(), he1.as_str(), jp.as_str()], ["PCHe", "PCHe1", "PCJp"]); + for name in [&he, &he1, &jp] { + let font = fonts.get(name.as_bytes()).and_then(Object::as_dict).cloned().unwrap_or_default(); + assert!(is_type3_fallback(&font), "{name} isn't recognized as a generated fallback font"); + } + // Someone else's Type 3 font is not ours. + let mut other = Dict::new(); + other.set(b"Subtype".to_vec(), Object::name("Type3")); + other.set(b"Name".to_vec(), Object::name("PCHebrewX")); + assert!(!is_type3_fallback(&other)); + } } diff --git a/crates/fonts/README.md b/crates/fonts/README.md index aa717449..44b617dd 100644 --- a/crates/fonts/README.md +++ b/crates/fonts/README.md @@ -25,6 +25,10 @@ set). Without that variable `CRAFT_FONTS` is empty and everything below copes: titles), in manifest order. - `ui_telugu_fonts`: the `Telu` faces for Telugu interface text (the Telugu catalog, file names, document titles), in manifest order. +- `ui_hebrew_fonts` / `document_hebrew_font(bold)`: the `Hebr` faces (Noto Sans Hebrew) for + Hebrew interface text, and the face whose outlines become the Type 3 fallback for Hebrew text + written by Edit text (Bold when asked for and present, else Regular). `None` without a + craft-fonts input that has one; the editor then reports a clear error. - `document_japanese_font` / `japanese_glyph`: the face (Shippori Mincho, then BIZ UDMincho) whose outlines become the Type 3 fallback font for Japanese text written into PDFs. Without it, `japanese_glyph` returns `GlyphError::NoFont` and the editor reports a clear error. @@ -50,7 +54,7 @@ The descriptor does not change glyph programs or replace the Type 3 `FontBBox` m only Gothic Regular, so it cannot preserve bold. No new font assets or changes to that input are needed. -wasm32 builds embed only BIZ UDPGothic Regular and any `Arab` or `Telu` face, to keep the web build small: the web build +wasm32 builds embed only BIZ UDPGothic Regular and any `Arab`, `Telu` or `Hebr` face, to keep the web build small: the web build currently has no Chinese face, so Chinese there still shows the replacement glyph. Font files are never committed here (`AGENTS.md` §1.4; team members: [craftrules `standards/fonts.md`](https://github.com/storytold/craftrules/blob/main/standards/fonts.md), internal). diff --git a/crates/fonts/build.rs b/crates/fonts/build.rs index 4b98f7d8..e452f9ac 100644 --- a/crates/fonts/build.rs +++ b/crates/fonts/build.rs @@ -1,7 +1,7 @@ //! Embeds the fonts of the optional craft-fonts build input (`CRAFT_FONTS_DIR`) as `CRAFT_FONTS`. //! The recipe is craft-fonts' `docs/integration.md`; unset, `CRAFT_FONTS` is empty. It only reads //! the local checkout (no network). On wasm32 it embeds only the UI faces (BIZ UDPGothic Regular -//! and any `Arab` or `Telu` face), to keep the web build within hosting limits (Cloudflare Pages: +//! and any `Arab`, `Telu` or `Hebr` face), to keep the web build within hosting limits (Cloudflare Pages: //! 25 MiB per file). use std::fmt::Write as _; @@ -53,7 +53,9 @@ fn craft_fonts(dir: &std::path::Path) -> Result { let arabic = scripts.split(',').any(|s| s.trim() == "Arab"); // Telugu faces too, for the Telugu interface. let telugu = scripts.split(',').any(|s| s.trim() == "Telu"); - let web_face = (*family == "BIZ UDPGothic" && *style == "Regular") || arabic || telugu; + // And the Hebrew ones (small), for Hebrew text and Edit text on the web. + let hebrew = scripts.split(',').any(|s| s.trim() == "Hebr"); + let web_face = (*family == "BIZ UDPGothic" && *style == "Regular") || arabic || telugu || hebrew; if wasm && !web_face { continue; } diff --git a/crates/fonts/src/craft.rs b/crates/fonts/src/craft.rs index 163f1dae..d8ab775e 100644 --- a/crates/fonts/src/craft.rs +++ b/crates/fonts/src/craft.rs @@ -66,6 +66,25 @@ pub fn ui_telugu_fonts() -> Vec<&'static CraftFont> { CRAFT_FONTS.iter().filter(|f| f.covers("Telu")).collect() } +/// The `Hebr` craft-fonts faces for Hebrew interface text (file names, document titles, text +/// being edited), in manifest order. Empty when built without craft-fonts or when it has no +/// Hebrew face. +pub fn ui_hebrew_fonts() -> Vec<&'static CraftFont> { + CRAFT_FONTS.iter().filter(|f| f.covers("Hebr")).collect() +} + +/// The face for Hebrew text written into PDFs: the `Hebr` face of the asked weight, else its +/// Regular, else any `Hebr` face. `None` without craft-fonts or without a Hebrew face. +pub fn document_hebrew_font(bold: bool) -> Option<&'static CraftFont> { + hebrew_face(CRAFT_FONTS, bold) +} + +fn hebrew_face(faces: &[CraftFont], bold: bool) -> Option<&CraftFont> { + let hebr = || faces.iter().filter(|f| f.covers("Hebr")); + let style = if bold { "Bold" } else { "Regular" }; + hebr().find(|f| f.style == style).or_else(|| hebr().find(|f| f.style == "Regular")).or_else(|| hebr().next()) +} + /// Interface CJK faces in fallback order for the UI language: Simplified Chinese first when /// `prefer_hans`, otherwise Japanese first (the historical default). /// @@ -251,6 +270,23 @@ mod tests { assert!(!BARRED_FAMILIES.iter().any(|barred| "Noto Sans Arabic".starts_with(barred))); } + #[test] + fn hebrew_document_faces_follow_the_weight() { + static HEBR_LATN: &[&str] = &["Hebr", "Latn"]; + let faces = [ + CraftFont { family: "BIZ UDPGothic", style: "Bold", scripts: &["Jpan"], bytes: b"not-hebrew" }, + CraftFont { family: "FakeHebrew", style: "Regular", scripts: HEBR_LATN, bytes: b"regular" }, + CraftFont { family: "FakeHebrew", style: "Bold", scripts: HEBR_LATN, bytes: b"bold" }, + ]; + assert_eq!(hebrew_face(&faces, false).map(|f| f.bytes), Some(&b"regular"[..])); + assert_eq!(hebrew_face(&faces, true).map(|f| f.bytes), Some(&b"bold"[..])); + // No bold face: Regular, never a face of another script. + assert_eq!(hebrew_face(&faces[..2], true).map(|f| f.bytes), Some(&b"regular"[..])); + assert!(hebrew_face(&faces[..1], false).is_none()); + assert_eq!(ui_hebrew_fonts().len(), CRAFT_FONTS.iter().filter(|f| f.covers("Hebr")).count()); + assert_eq!(document_hebrew_font(false).is_some(), CRAFT_FONTS.iter().any(|f| f.covers("Hebr"))); + } + #[test] fn craft_fonts_are_optional_and_consistent() { // Holds with or without CRAFT_FONTS_DIR. diff --git a/crates/fonts/src/lib.rs b/crates/fonts/src/lib.rs index 75ebe6d3..672e0a6c 100644 --- a/crates/fonts/src/lib.rs +++ b/crates/fonts/src/lib.rs @@ -12,8 +12,8 @@ pub mod pdf; mod script; pub use arabic::{ShapedCluster, arabic_glyph, arabic_has, shape_arabic}; pub use craft::{ - CRAFT_FONTS, CraftFont, SHIPPORI_MINCHO, document_arabic_font, document_japanese_font, document_japanese_font_for_style, - document_japanese_fonts_for_style, ui_arabic_fonts, ui_chinese_fonts, ui_cjk_fonts, ui_japanese_fonts, ui_telugu_fonts, + CRAFT_FONTS, CraftFont, SHIPPORI_MINCHO, document_arabic_font, document_hebrew_font, document_japanese_font, document_japanese_font_for_style, + document_japanese_fonts_for_style, ui_arabic_fonts, ui_chinese_fonts, ui_cjk_fonts, ui_hebrew_fonts, ui_japanese_fonts, ui_telugu_fonts, }; pub use script::{GlyphError, GlyphOutline, INTER_REGULAR, MAX_SIGNATURE_CHARS, ScriptOutline, japanese_glyph, japanese_glyph_from, script_outline}; diff --git a/crates/ui-egui/src/theme.rs b/crates/ui-egui/src/theme.rs index 7d62a6b0..976854fc 100644 --- a/crates/ui-egui/src/theme.rs +++ b/crates/ui-egui/src/theme.rs @@ -204,8 +204,9 @@ pub fn installed_font_definitions(cjk: CjkPreference) -> FontDefinitions { } /// The interface fonts: Inter (and JetBrains Mono for code) first, then egui's defaults, then -/// the CJK, Arabic and Telugu faces of the optional craft-fonts build input as the last fallback -/// in every family. Without craft-fonts there is no Japanese, Chinese, Arabic or Telugu face here. +/// the CJK, Arabic, Telugu and Hebrew faces of the optional craft-fonts build input as the last +/// fallback in every family. Without craft-fonts there is no Japanese, Chinese, Arabic, Telugu or +/// Hebrew face here. pub fn font_definitions() -> FontDefinitions { font_definitions_for(false) } @@ -237,9 +238,10 @@ pub fn font_definitions_for(prefer_hans: bool) -> FontDefinitions { fonts.families.entry(family).or_default().push(name.clone()); } } - // Arabic-script faces (file names, document titles) and Telugu faces (the Telugu interface, - // file names) after the CJK ones; the ranges don't overlap. - for face in pdfcraft_fonts::ui_arabic_fonts().into_iter().chain(pdfcraft_fonts::ui_telugu_fonts()) { + // Arabic-script faces (file names, document titles), Telugu faces (the Telugu interface, + // file names) and Hebrew faces (file names, text being edited) after the CJK ones; the + // ranges don't overlap. + for face in pdfcraft_fonts::ui_arabic_fonts().into_iter().chain(pdfcraft_fonts::ui_telugu_fonts()).chain(pdfcraft_fonts::ui_hebrew_fonts()) { let name = face.name(); if !fonts.font_data.contains_key(&name) { add(&mut fonts, &name, face.bytes); diff --git a/docs/localization.md b/docs/localization.md index 5519e5bb..745ab538 100644 --- a/docs/localization.md +++ b/docs/localization.md @@ -23,6 +23,8 @@ Whole sentences that the layers below word — what the accessibility checker fo The Ukrainian catalog has 2,047 entries, written from the meaning of the English labels using Ukrainian computing vocabulary. Commands use infinitives; fixed-count messages use count-neutral wording. Plural-aware messages use the three [CLDR Ukrainian integer forms](https://www.unicode.org/cldr/charts/48/supplemental/language_plural_rules.html#uk): `1 сторінка`, `2 сторінки`, `5 сторінок`; 11–14 take the third form, while 21 and 22 return to the first and second. Calendar dates use genitive month names (`9 жовтня 2026`), while headings use standalone names (`Жовтень 2026`). Home recommendation cards grow to fit translated text; the toolbar search hint truncates before the shortcut, with the full label available on hover. +Edit text reads Hebrew lines in typing order and writes them back in drawing order (`crates/edit/src/bidi.rs`); letters the page's own font lacks are drawn with the craft-fonts `Hebr` face (Noto Sans Hebrew) as Type 3 glyphs, and a build without one says so. Hebrew points (niqqud) can't be written yet. The craft-fonts revision pinned for CI and releases has no `Hebr` face yet, so until one is added releases report that error instead of drawing the missing letters. + Japanese interface text uses BIZ UDPGothic from [craft-fonts](https://github.com/storytold/craft-fonts), an optional build input that every release includes (`CRAFT_FONTS_DIR`; see the README). A build made without it has no Japanese face, so Japanese labels show replacement boxes. Chinese is drawn with the same craft-fonts CJK faces (in Simplified Chinese the `Hans` faces come first, so a line never mixes faces with different baselines), so a character none of them has shows a replacement box (and every Chinese character does in a build without craft-fonts). Russian, Bulgarian, Czech, Brazilian Portuguese, German, Spanish, French, Hungarian, Italian and Turkish need only the bundled Latin faces (tested in `tests/fonts.rs`). Hungarian uses the same one/other plural rule as English: 1 is singular, and every other count — including 0 — is the other form. After a numeral the noun stays singular (`1 oldal`, `2 oldal`). Telugu is drawn with a craft-fonts `Telu` face (Noto Sans Telugu, kept in the web build too), whose conjuncts egui shapes; a build without one shows Telugu labels as replacement boxes, and the release-pinned craft-fonts input has no Telugu face yet. Japanese interface text uses BIZ UDPGothic from [craft-fonts](https://github.com/storytold/craft-fonts), an optional build input that every release includes (`CRAFT_FONTS_DIR`; see the README). A build made without it has no Japanese face, so Japanese labels show replacement boxes. Chinese is drawn with the same craft-fonts CJK faces (in Simplified Chinese the `Hans` faces come first, so a line never mixes faces with different baselines), so a character none of them has shows a replacement box (and every Chinese character does in a build without craft-fonts). Russian, Bulgarian, Czech, Brazilian Portuguese, German, Spanish, French, Hungarian, Italian and Slovenian need only the bundled Latin faces (tested in `tests/fonts.rs`). Slovenian has four plural forms: one (1, 101, 201…), two (2, 102…), few (3, 4, 103…) and other (everything else, including 0, 5–20 and 21+), so `@plural` entries list four forms (`1 stran`, `2 strani`, `3 strani`, `5 strani`). Hungarian uses the same one/other plural rule as English: 1 is singular, and every other count — including 0 — is the other form. After a numeral the noun stays singular (`1 oldal`, `2 oldal`). Telugu is drawn with a craft-fonts `Telu` face (Noto Sans Telugu, kept in the web build too), whose conjuncts egui shapes; a build without one shows Telugu labels as replacement boxes, and the release-pinned craft-fonts input has no Telugu face yet. Japanese interface text uses BIZ UDPGothic from [craft-fonts](https://github.com/storytold/craft-fonts), an optional build input that every release includes (`CRAFT_FONTS_DIR`; see the README). A build made without it has no Japanese face, so Japanese labels show replacement boxes. Chinese is drawn with the same craft-fonts CJK faces (in Simplified Chinese the `Hans` faces come first, so a line never mixes faces with different baselines), so a character none of them has shows a replacement box (and every Chinese character does in a build without craft-fonts). Until craft-fonts has an allowed `Hans` face, the desktop app draws a Chinese interface with a Chinese face already installed on the machine (Microsoft YaHei or JhengHei on Windows, PingFang, Hiragino Sans GB or STHeiti on macOS, WenQuanYi or Droid Sans Fallback on Linux; never Noto CJK or Source Han), placed before the Japanese faces so a line keeps one face (#826). The web build has no installed fonts to fall back on. Russian, Bulgarian, Czech, Brazilian Portuguese, German, Spanish, French, Hungarian and Italian need only the bundled Latin faces (tested in `tests/fonts.rs`). Hungarian uses the same one/other plural rule as English: 1 is singular, and every other count — including 0 — is the other form. After a numeral the noun stays singular (`1 oldal`, `2 oldal`). Telugu is drawn with a craft-fonts `Telu` face (Noto Sans Telugu, kept in the web build too), whose conjuncts egui shapes; a build without one shows Telugu labels as replacement boxes, and the release-pinned craft-fonts input has no Telugu face yet. diff --git a/parity/acrobat-features.toml b/parity/acrobat-features.toml index 090ba012..bb073ca2 100644 --- a/parity/acrobat-features.toml +++ b/parity/acrobat-features.toml @@ -1995,11 +1995,11 @@ title = "Edit right-to-left and CJK text" area = "C" tier = "P1" milestone = "M7" -status = "planned" +status = "partial" commands = [] -tools = [] -evidence = [] -notes = "" +tools = ["text_lines", "text_paragraphs", "text_edit"] +evidence = ["crates/edit/src/tests.rs::hebrew_lines_read_in_typing_order", "crates/edit/src/tests.rs::hebrew_edits_are_drawn_right_to_left_and_keep_their_right_edge", "crates/edit/src/tests.rs::hebrew_letters_the_page_font_lacks_use_the_hebrew_face", "crates/edit/src/bidi.rs::hebrew_sentences_numbers_latin_and_brackets_round_trip"] +notes = "Hebrew: lines and paragraphs read in typing order and are written back in drawing order, right-aligned by default; letters the page's font lacks use the craft-fonts Noto Sans Hebrew face as Type 3 glyphs. Japanese uses its Type 3 fallback. Missing: Hebrew points (niqqud) in replacement text, writing Arabic (shaping), right-to-left typing in the edit box (misc.rtl-ui), lines a producer draws word by word from the right, vertical CJK." [[feature]] id = "edit.rotated-text-editing"