From 1e53adf9d62f6721165e0d3f0c8b5d98c115a61b Mon Sep 17 00:00:00 2001 From: Eduardo Soares Date: Mon, 12 Oct 2026 08:43:24 -0300 Subject: [PATCH] Portuguese proofing: pt-BR spelling, suggestions, grammar and hyphenation (#100) (#797) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * M15: Polish interface, and Polish spelling, grammar and hyphenation Interface - Polish is the tenth interface language ("Polski", pl.tsv, 1,045 entries); pl, pl-PL and pl_PL.UTF-8 system locales select it automatically. - Right-click menu labels now go through translation in every language. - Review > Language is a menu (English US/UK, Polski, Do not check spelling or grammar); the status bar shows the language at the caret. - With a Polish interface, new blank documents and opened .txt/.md files default to pl-PL. Proofing - Each run is checked in its own language (w:lang, resolved through styles): squiggles, F7, review.issues, suggestions and hyphenation. Text in languages without proofing tools is no longer flagged as English. - Polish spelling from the SJP.PL Hunspell dictionary (Apache-2.0 option), read by a new Rust affix engine (crates/proof/src/affix.rs); 875 KB asset, ~4.5M word forms. Regenerate with the pl_dictionary example. - Polish suggestions: case, missing diacritics, rz/ż, ó/u, ch/h confusions, missing spaces (napewno -> na pewno), edit distance. - Conservative Polish grammar rules: repeated words, spacing, doubled punctuation, sentence capitals, missing comma before że/który/ponieważ/ aby/żeby/jeśli…, we/ze forms. - Polish hyphenation with the hyph-utf8 hyph-pl.tex patterns (MIT option); pattern keys widened to 20 letters, TeX exceptions parsed; the heuristic no longer treats ł, ż, ś… as vowels. review.language validates and normalizes tags, can set noProof alone and the document default; format.state reports lang and noProof. Tests for every part, docs and ATTRIBUTION.md/NOTICE updated; cargo xtask ci passes (fmt, clippy, test, assets, layers, wasm). Co-Authored-By: Claude Opus 5.5 * Layout: rustfmt the test module order left unformatted by #391 cargo fmt --check failed on main after #391 added mod tests_grid. Co-Authored-By: Claude Opus 5.5 * Review fixes for #346: read Polish spelling from the installed Hunspell dictionary - Polish spelling uses the Hunspell pl_PL dictionary installed on the system (OS packages, Homebrew, LibreOffice, ~/.local/share/hunspell, or WORDCRAFT_DICTIONARIES) instead of a bundled 870 KB SJP.PL file; without one, Polish text gets no spelling marks, like Word without the language's proofing tools. .aff/.dic are size-capped and decoded from UTF-8 or ISO 8859-2 (crates/proof/src/hunspell.rs). - The dictionary can be replaced at runtime (install_dictionary) or re-searched (reload_dictionary); an empty one accepts every word. - Tests use small synthetic dictionaries written for them, so they don't depend on what the machine has installed. - The bundled pl.dic, its ATTRIBUTION/NOTICE entries and the generator example are removed; docs describe the installed dictionary. Co-Authored-By: Claude Opus 5.5 * #100: affix — accept FLAG UTF-8 and DEFAULT flag modes Per-character flag modes are what the reader already parses; only the multi-character encodings (long, num) stay errors. VERO pt_BR declares FLAG UTF-8. * #100: affix — widen flag masks to u128 (128 flag capacity) VERO pt_BR needs 106 distinct flags; the u64 bitmask capped at 64. Serialization is unaffected: WCAFF1 stores flag characters and re-derives the masks on load. * #100: affix — support FORBIDDENWORD VERO pt_BR marks spellings it must reject with a forbidden flag (ý). Stems carrying it are listed but never correct and never inflect. * #100: affix — continuation classes in check SFX A 0 able/B: the form derived by A carries flag B, licensing a further suffix of class B (workable -> workables) while the bare stem cannot take one. form_has_flag walks the chain depth-bounded (MAX_CHAIN = 3); with no continuations declared it degenerates to the stem lookup, so existing dictionaries are unaffected. VERO pt_BR uses 26 continuation rules. * #100: affix — continuation classes in forms Enumerate chained suffix forms (workable -> workables) under the same MAX_CHAIN bound the checker walks, so every form the checker accepts is listed and vice versa. * #100: affix — round-trip continuations and FORBIDDENWORD through WCAFF1 The compact format keeps its WCAFF1 magic: continuation classes ride on the rule's appended field (as in the .aff source) and the forbidden flag becomes a FORBIDDENWORD line. Old files read as before. * #100: affix — strip a UTF-8 BOM from .aff/.dic input VERO's files open with a BOM; glued to the count line it parsed as a stem (one garbage entry per load) and to SET it hid the directive. * #782: affix — the module doc still rejected what the code now reads Continuation classes and FORBIDDENWORD are supported; the header said Rejected for both. * #100: pt — Portuguese spelling, following the Polish module ProofLang::Pt resolves pt/pt-BR/pt-PT/por tags; the pt module reads the installed Hunspell pt_BR dictionary (VERO) through the affix engine — FLAG UTF-8 flags, continuation classes and FORBIDDENWORD spellings all exercised by the synthetic test dictionary. Hyphenated compounds are checked part by part (guarda-roupa, ex-marido); without a dictionary installed, Portuguese text gets no marks. Spelling message in Portuguese; suggestions and grammar land next. * #100: pt — Portuguese suggestions Rounds of rising cost, driven by the dictionary's own tables: missing accents from MAP (coraçao -> coração, saude -> saúde), the REP confusions (seje -> seja, excessão -> exceção), a missing space after a short function word (nãoé -> não é, dagua -> da água), one typo over TRY, and an accent-folding stem scan — the Polish rounds, retuned for Portuguese. * #100: pt — Portuguese grammar flags Conservative rules with Portuguese messages: repeated words (except the emphatic doubles Portuguese allows), space before punctuation and double spaces, sentence capitals knowing the Portuguese abbreviations and dialogue dashes, and doubled punctuation. No comma rule: que opens a clause only sometimes, so a wrong flag would cost more than a missed one. * #100: pt — Portuguese hyphenation hyph-utf8's hyph-pt.tex bundled unmodified (BSD 3-clause, ATTRIBUTION row with source and hash): 426 Liang patterns and 2 exception words; falls back to the generic heuristic outside their reach. * #100: pt — Review › Language offers Português (Brasil), and a pt-br interface writes pt-BR The menu's fourth entry sets the run language; the status bar names Portuguese; editing_lang is now allowed for pt-br (wordcraft-proof proofs it), so new documents under a Portuguese interface are Portuguese by default. * #100: pt — engine test: Portuguese proofing through the review commands review.language pt-BR normalizes the tag, review.issues reports Portuguese spelling and grammar with the lang field, F7 and Change apply a suggestion, Add to Dictionary works for Portuguese. * #100: pt — Portuguese in the mixed-language proofing tests and docs check_text proofs a Portuguese span with the Portuguese message; the never-panic loops and the module docs cover the third language. * #100: docs — Portuguese proofing in the gap, parity and roadmap docs Gap 8 narrowed (English, Polish and Portuguese), localization parity gains a Portuguese row and a revision entry, target-app-parity's Proofing row moves 40% -> 42%, and the ROADMAP progress log records what landed. * #100: hunspell — LibreOffice extension folders follow the language The search paths hardcoded LibreOffice's dict-pl; a pt_BR user whose only dictionary is LibreOffice's dict-pt-BR extension — on Windows the only realistic on-disk source — got silently no spelling. Both the primary-language folder and the dashed full name are now tried, so Polish keeps dict-pl and Portuguese finds dict-pt-BR. * #100: pt — review fixes Suggestion ties break toward the change nearest the word's start (saúde before saudê, proven against a fixture that has both); the Language menu's label is the English source "Portuguese (Brazil)" with translations in all twelve catalogs (as Polish is); the attribution counts pt.pat's 2 exception words; the pt fixture drops a duplicate stem; docs/roadmap.md catches up with the landing; the throwaway spike is gone. * Review fixes for #782: two-suffix chain limit, module doc - MAX_CHAIN = 2 counts suffixes in a row, matching Hunspell's two-suffix limit (the old bound of 3 allowed up to four); test pins that a third is rejected. - Module doc: say probes allocate small scratch strings instead of claiming none. Co-Authored-By: Claude Opus 5.5 * Review fixes for #797: NOTICE for pt.pat, no indexing in the LibreOffice folder helper - NOTICE reproduces hyph-pt.tex's BSD 3-Clause copyright notice, conditions and disclaimer, which the licence requires for binary redistributions. - hunspell::lo_ext_dirs builds its list without indexing (never-crash rule). - Catalogs: main already translates "Portuguese (Brazil)" in every language, so the duplicate rows the branch added are dropped. Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: witkowski01 Co-authored-by: Claude Opus 5.5 Co-authored-by: Brandon Thomas --- ATTRIBUTION.md | 1 + NOTICE | 26 ++ ROADMAP.md | 5 +- assets/hyphenation/pt.pat | 503 +++++++++++++++++++++++++++++++ crates/engine/src/tests.rs | 44 +++ crates/proof/src/hunspell.rs | 53 +++- crates/proof/src/hyphen.rs | 60 ++++ crates/proof/src/lang.rs | 15 +- crates/proof/src/lib.rs | 33 +- crates/proof/src/pt/grammar.rs | 245 +++++++++++++++ crates/proof/src/pt/mod.rs | 202 +++++++++++++ crates/proof/src/pt/suggest.rs | 444 +++++++++++++++++++++++++++ crates/proof/src/pt/test_dict.rs | 121 ++++++++ crates/ui-egui/src/i18n/mod.rs | 2 +- crates/ui-egui/src/i18n/tests.rs | 4 +- crates/ui-egui/src/ribbon.rs | 17 +- docs/gaps.md | 3 +- docs/localization-parity.md | 8 +- docs/roadmap.md | 5 +- docs/target-app-parity.md | 5 +- 20 files changed, 1758 insertions(+), 38 deletions(-) create mode 100644 assets/hyphenation/pt.pat create mode 100644 crates/proof/src/pt/grammar.rs create mode 100644 crates/proof/src/pt/mod.rs create mode 100644 crates/proof/src/pt/suggest.rs create mode 100644 crates/proof/src/pt/test_dict.rs diff --git a/ATTRIBUTION.md b/ATTRIBUTION.md index 5856d73..e8a20ad 100644 --- a/ATTRIBUTION.md +++ b/ATTRIBUTION.md @@ -12,6 +12,7 @@ Generated-in-code art is original and has no file to list: the UI icon set (`cra | `assets/hyphenation/en-us.dic` | Grady Ward (Moby Hyphenator II word list); converted by DesignCraft contributors | https://www.gutenberg.org/ebooks/3204 (file `mhyph.txt`, SHA-256 `eeb30474c86b8af3469035ec1a0913e35905325ca885290db9dfed9881e230ac`), via DesignCraft | Public Domain | ~165k words with break points; drives spelling and hyphenation | | `assets/spelling/en-extra.txt` | WordCraft contributors | original word list written for this repository (common modern words the Moby list lacks); no other dictionary copied | MIT OR Apache-2.0 | ~220 lowercase English base forms supplementing spelling (#520) | | `assets/hyphenation/pl.pat` | Hanna Kołodziejska, Bogusław Jackowski, Marek Ryćko (Polish TeX hyphenation patterns, 1987–1995); hyph-utf8 package | https://github.com/hyphenation/tex-hyphen (`hyph-utf8/tex/generic/hyph-utf8/patterns/tex/hyph-pl.tex`, SHA-256 `83ddbbd21bc06a44bc4f4a4c49c38782e809cff5f026b764c044daa29e818843`), bundled unmodified | MIT (the file is offered under MIT or Knuth's public-domain terms; WordCraft uses MIT, whose notice is in the file's header) | 4,053 Liang patterns and 20 exception words; Polish hyphenation | +| `assets/hyphenation/pt.pat` | Pedro J. de Rezende, J. Joao Dias Almeida, Leonardo Araujo, Aline Benevides (Portuguese TeX hyphenation patterns, 1987–2024); hyph-utf8 package | https://github.com/hyphenation/tex-hyphen (`hyph-utf8/tex/generic/hyph-utf8/patterns/tex/hyph-pt.tex`, SHA-256 `c110a39e501db372d1d755e1c64464321a2efa7d8e9617f66001000389b941f2`), bundled unmodified | BSD 3-clause (the file's licence header names it and carries the notice) | 426 Liang patterns and 2 exception words; Portuguese hyphenation | | `assets/app-icon/wordcraft-1024.png` | WordCraft contributors | original artwork drawn in SVG (`assets/app-icon/wordcraft.svg`), rendered by `packaging/icons.sh` | MIT OR Apache-2.0 | Placeholder app icon until the WordCraft mascot drawing lands | | `packaging/macos/dmg/background.svg` | @XusBadia | original work for this repository, derived from the WordCraft app icon (`assets/app-icon/wordcraft.svg`, referenced via ``, not copied); text is outlined Inter / JetBrains Mono glyph paths | MIT OR Apache-2.0, like the code ([`LICENSE-MIT`](LICENSE-MIT), [`LICENSE-APACHE`](LICENSE-APACHE)) | WordCraft macOS DMG window background (2026-10-08); used for the Finder window of the macOS DMG, rendered to `background.tiff` by `packaging/macos/dmg/generate.py` | | `packaging/macos/dmg/background.tiff` | @XusBadia | Rendered from `packaging/macos/dmg/background.svg` by `packaging/macos/dmg/generate.py` | MIT OR Apache-2.0, like the code ([`LICENSE-MIT`](LICENSE-MIT), [`LICENSE-APACHE`](LICENSE-APACHE)) | 1x + 2x HiDPI TIFF for the macOS DMG window; includes the app icon (see its row) and LittleCMS's built-in sRGB profile ("No copyright, use freely") | diff --git a/NOTICE b/NOTICE index 3390e72..8d09e43 100644 --- a/NOTICE +++ b/NOTICE @@ -26,6 +26,32 @@ list (author, source, license) is in ATTRIBUTION.md. - Polish hyphenation patterns (assets/hyphenation/pl.pat) — hyph-pl.tex from the hyph-utf8 package, Copyright (C) 1987-1995 Hanna Kołodziejska, Bogusław Jackowski, Marek Ryćko, bundled unmodified and used under the MIT License; the copyright and permission notice is in the file. +- Portuguese hyphenation patterns (assets/hyphenation/pt.pat) — hyph-pt.tex from the hyph-utf8 + package, bundled unmodified and used under the BSD 3-Clause License: + + Copyright (C) 1987, 1994, 1996, 2015 Pedro J. de Rezende, 1996, 2015 J. Joao Dias Almeida, + 2024 Leonardo Araujo and Aline Benevides + + Redistribution and use in source and binary forms, with or without modification, are + permitted provided that the following conditions are met: + * Redistributions of source code must retain the above copyright notice, this list of + conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright notice, this list of + conditions and the following disclaimer in the documentation and/or other materials + provided with the distribution. + * Neither the name of the University of Campinas, of the University of Minho nor the names + of its contributors may be used to endorse or promote products derived from this software + without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS + OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL PEDRO + J. DE REZENDE OR J.JOAO DIAS ALMEIDA BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, + EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE + GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + OF THE POSSIBILITY OF SUCH DAMAGE. - All UI icons are drawn in code (crates/ui-egui/src/icons.rs) and are original work, MIT OR Apache-2.0. Rust dependencies (egui, eframe, vello_cpu, skrifa, harfrust, krilla, quick-xml, zip, image, …) diff --git a/ROADMAP.md b/ROADMAP.md index 6cfce04..6282657 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,7 +2,7 @@ **Stage: alpha** · next: beta, ~15 points (60% → 75% ready for real work) and ~150–230 h away -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-11 · **Change:** trivial (Turkish interface catalog completed, #774; previously trivial: drag-select by whole units, #549; previously trivial: Insert › SmartArt landed, #497: catalog 429/451 → 437/458 (95.4%); previously trivial: Turkish interface landed; previously trivial: formatting revisions landed, #41; previously trivial: Draw tab ink, #307, and Draw Table and Eraser, #303, landed: catalog 402/434; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; restructured to the craftrules progress-docs standard) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-12 · **Change:** minor (Portuguese proofing; previously minor: Polish interface and Polish proofing; previously trivial: Turkish interface landed; previously trivial: formatting revisions landed, #41; previously trivial: Draw tab ink, #307, and Draw Table and Eraser, #303, landed: catalog 402/434; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; restructured to the craftrules progress-docs standard) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 WordCraft aims for complete parity with Microsoft Word, then goes further on speed, openness and agent control. This page is the summary; the assessment is @@ -113,7 +113,7 @@ Weights and evidence per area: [docs/target-app-parity.md](docs/target-app-parit Also shipped: Traditional Chinese (`zh-hant`), Ukrainian (`uk`), Serbian (`sr`, `sr-latn`, #250), Russian (`ru`) and Polish (`pl`), all partial (~95%). Estonian (`et`), Italian (`it`, #594) and Czech (`cs`) now have complete interface catalogs, and Turkish (`tr`) a complete 2,325-entry one (every key of the Russian catalog), all with system-locale selection and a saved language choice (Italian also has 51 strings none of the others had yet); Estonian, Turkish, Italian and Czech proofing are not included. Hungarian (`hu`) has a complete 2,419-entry catalog, also without proofing. -Proofing: English and Polish (spelling, grammar, hyphenation), chosen per run from its language. Word ships +Proofing: English, Polish and Portuguese (spelling, grammar, hyphenation), chosen per run from its language. Word ships 30 interface languages and proofing for ~50. Detail: [docs/localization-parity.md](docs/localization-parity.md). ## Upcoming @@ -141,6 +141,7 @@ Ranked, to beta (detail and remaining milestones in [docs/roadmap.md](docs/roadm | 2026-10-11 | Reading Mode: existing View ribbon button now opens a read-only, viewport-column layout with animated horizontal paging, transient display controls, heading navigation (Navigation pane) and resize anchoring; document layout and export unchanged | | 2026-10-11 | DOCX open/save keeps inherited complex-script size, bold and italic independent of Latin formatting; explicit values are retained, and font commands still format both scripts. Covered by synthetic round trips and a local GUI save/reopen check. | | 2026-10-11 | Czech (`cs`) interface catalog: 2,021 entries (the full zh-hans key set), Czech locale detection and persisted language choice | +| 2026-10-12 | Portuguese proofing: pt-BR spelling against the installed Hunspell `pt_BR` dictionary (VERO; read at runtime, not bundled) — the affix engine grew `FLAG UTF-8`, 128 flag bits, continuation classes and `FORBIDDENWORD` to read it — suggestions tuned for Portuguese (missing accents first), conservative grammar rules with Portuguese messages, TeX Portuguese hyphenation patterns (BSD 3-clause, attributed); Review › Language offers Português (Brasil); a pt-br interface writes new documents in pt-BR | | 2026-10-11 | Polish: interface (`pl`, the thirteenth language, every catalog key) and proofing — spelling against the installed Hunspell `pl_PL` dictionary (read at runtime, not bundled) through a new Hunspell-style affix engine, suggestions tuned for Polish, Polish grammar rules with Polish messages, TeX Polish hyphenation patterns (MIT option); proofing language per run (`w:lang`) for squiggles, F7, suggestions and hyphenation, languages without tools left unmarked; Review › Language menu and the status bar's language; a Polish interface writes new documents in Polish | | 2026-10-11 | German (`de`) interface (#574): 2,021 translated entries covering the full zh-hans key set, locale and saved-preference tests, bundled-font checks; no German proofing resources | | 2026-10-11 | `.doc` import keeps pictures and drawings (#551): inline and floating pictures (PNG, JPEG, DIB, EMF, WMF), shapes, lines, groups and text boxes with position and wrap, from [MS-ODRAW] | diff --git a/assets/hyphenation/pt.pat b/assets/hyphenation/pt.pat new file mode 100644 index 0000000..cb947c4 --- /dev/null +++ b/assets/hyphenation/pt.pat @@ -0,0 +1,503 @@ +% title: Hyphenation patterns for Portuguese +% copyright: Copyright (C) 1987, 1994, 1996, 2015 Pedro J. de Rezende, 1996, 2015 J. Joao Dias Almeida, 2024 Leonardo Araujo and Aline Benevides +% notice: This file is part of the hyph-utf8 package. +% See http://www.hyphenation.org/tex for more information. +% language: +% name: Portuguese +% tag: pt +% version: 1.4 2024-07-13 +% authors: +% - +% name: Pedro J. de Rezende +% contact: rezende (at) ic.unicamp.br +% - +% name: J. Joao Dias Almeida +% contact: jj (at) di.uminho.pt +% - +% name: Leonardo Araujo +% contact: leolca (at) gmail.com +% - +% name: Aline Benevides +% contact: benevides.aline12 (at) gmail.com +% licence: +% name: BSD 3-clause licence +% url: https://opensource.org/licenses/BSD-3-Clause +% text: > +% Redistribution and use in source and binary forms, with or without +% modification, are permitted provided that the following conditions +% are met: +% * Redistributions of source code must retain the above copyright +% notice, this list of conditions and the following disclaimer. +% * Redistributions in binary form must reproduce the above copyright +% notice, this list of conditions and the following disclaimer in the +% documentation and/or other materials provided with the +% distribution. +% * Neither the name of the University of Campinas, of the University +% of Minho nor the names of its contributors may be used to endorse +% or promote products derived from this software without specific +% prior written permission. +% +% THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +% "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +% LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +% A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL PEDRO J. DE +% REZENDE OR J.JOAO DIAS ALMEIDA BE LIABLE FOR ANY DIRECT, INDIRECT, +% INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, +% BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS +% OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED +% AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +% LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY +% WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +% POSSIBILITY OF SUCH DAMAGE. +% hyphenmins: +% typesetting: +% left: 2 +% right: 3 +% changes: +% - Version 1.4 Release date: 13/07/2024 Leonardo Araujo and Aline Benevides +% - Version 1.3 Release date: 12/08/2015 Pedro J. de Rezende and J. Joao Dias Almeida +% - Version 1.2 Release date: 07/21/1996 Pedro J. de Rezende and J. Joao Dias Almeida +% - Version 1.1 Release date: 04/12/1994 Pedro J. de Rezende +% - Version 1.0 Release date: 02/13/1987 Pedro J. de Rezende +% texlive: +% synonyms: +% - portuges +% encoding: ec +% babelname: portuguese +% legacy_patterns: pthyph.tex +% message: Portuguese hyphenation patterns +% description: Hyphenation patterns for Portuguese in T1/EC and UTF-8 encodings. +% +\patterns{ +1b2l +1b2r +1ba +1be +1bi +1bo +1bu +1bá +1bâ +1bã +1bé +1bí +1bó +1bú +1bê +1bõ +1c2h +1c2l +1c2r +1ca +1ce +1ci +1co +1cu +1cá +1câ +1cã +1cé +1cí +1có +1cú +1cê +1cõ +1ça +1çe +1çi +1ço +1çu +1çá +1çâ +1çã +1çé +1çí +1çó +1çú +1çê +1çõ +1d2l +1d2r +1da +1de +1di +1do +1du +1dá +1dâ +1dã +1dé +1dí +1dó +1dú +1dê +1dõ +1f2l +1f2r +1fa +1fe +1fi +1fo +1fu +1fá +1fâ +1fã +1fé +1fí +1fó +1fú +1fê +1fõ +1g2l +1g2r +1ga +1ge +1gi +1go +1gu +1gu4a +1gu4e +1gu4i +1gu4o +1gá +1gâ +1gã +1gé +1gí +1gó +1gú +1gê +1gõ +1ja +1je +1ji +1jo +1ju +1já +1jâ +1jã +1jé +1jí +1jó +1jú +1jê +1jõ +1k2l +1k2r +1ka +1ke +1ki +1ko +1ku +1ká +1kâ +1kã +1ké +1kí +1kó +1kú +1kê +1kõ +1l2h +1la +1le +1li +1lo +1lu +1lá +1lâ +1lã +1lé +1lí +1ló +1lú +1lê +1lõ +1ma +1me +1mi +1mo +1mu +1má +1mâ +1mã +1mé +1mí +1mó +1mú +1mê +1mõ +1n2h +1na +1ne +1ni +1no +1nu +1ná +1nâ +1nã +1né +1ní +1nó +1nú +1nê +1nõ +1p2l +1p2r +1pa +1pe +1pi +1po +1pu +1pá +1pâ +1pã +1pé +1pí +1pó +1pú +1pê +1põ +1qu4a +1qu4e +1qu4i +1qu4o +1ra +1re +1ri +1ro +1ru +1rá +1râ +1rã +1ré +1rí +1ró +1rú +1rê +1rõ +1sa +1se +1si +1so +1su +1sá +1sâ +1sã +1sé +1sí +1só +1sú +1sê +1sõ +1t2l +1t2r +1ta +1te +1ti +1to +1tu +1tá +1tâ +1tã +1té +1tí +1tó +1tú +1tê +1tõ +1v2l +1v2r +1va +1ve +1vi +1vo +1vu +1vá +1vâ +1vã +1vé +1ví +1vó +1vú +1vê +1võ +1w2l +1w2r +1xa +1xe +1xi +1xo +1xu +1xá +1xâ +1xã +1xé +1xí +1xó +1xú +1xê +1xõ +1za +1ze +1zi +1zo +1zu +1zá +1zâ +1zã +1zé +1zí +1zó +1zú +1zê +1zõ +a3a +a3e +a3o +c3c +e3a +e3e +e3o +i3a +i3e +i3i +i3o +i3â +i3ê +i3ô +o3a +o3e +o3o +r3r +s3s +u3a +u3e +u3o +u3u +.p2si +.p2sí +.g2no +.g2nó +.g2nô +t2c +1p2neu +.t2m +.p2t +su2b3r +su2b3l +.su3b4li +a1ir. +u1ir. +1qu +1vô +1lô +1cô +1gô +1bô +1tô +1rô +1pô +a1é +a1í +a1ó +a1ú +e1á +e1â +e1ã +e1é +e1ê +e1í +e1ó +é1o +e1ú +i1á +i1ã +í1a +i1é +i1í +i1ó +í1o +i1u +i1ú +o1á +o1é +o1í +o1ó +u1á +u1ã +u1â +u1í +ú1o +1qu2á +1qu2â +1qu2í +1gu2í +a1ind +a1i1nh +e1imp +e1inc +e1inf +e1ing +e1ins +e1int +e1inv +u1iz. +a1iz. +4a. +4e. +4o. +1gu2á +1gu2ã +1qu2ã +.m2n +c2za +.s2 +1p2seu1d +1dô +1fô +1mô +1nô +1sô +1zô +tu1i +tu2it +tu2id +bu1i +nu1i +o1in +u1in +su1i +í1e +ju1i +fu1i +du1i +do1im +au1i +u1i1ç +u1ê +1gu2ê +1qu2ê +1çô +u1é +1gu2é +1qu2é +1xô +a1â +a1ã +a1ô +e1ô +.ne4o +o1ã +o1ê +o1im +o2i1na +pro1i1b +co2ima +.t2 +1- +} +\hyphenation{% Do NOT make any alterations to this list! --- PdR +hard-ware +soft-ware +} diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index 26f9ebc..a1c6a54 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -3882,6 +3882,50 @@ fn polish_test_dictionary() { wordcraft_proof::pl::install_dictionary(aff, dic).unwrap(); } +/// A tiny synthetic Portuguese Hunspell dictionary (written for these tests). Real dictionaries +/// are installed on the system, never bundled; every Portuguese test installs this same one. +fn portuguese_test_dictionary() { + let aff = "SET UTF-8\nFLAG UTF-8\nFORBIDDENWORD F\nTRY aeiousrntmdlcpvgqbzfhjxçãõáéíóúâêô\nMAP 2\nMAP aáã\nMAP eé\nREP 1\nREP seje seja\n"; + let dic = "11\nsaudade\ncoração\ne\nele\ndisse\nnão\né\nque\nseja\nfrequencia/F\nmuito\n"; + wordcraft_proof::pt::install_dictionary(aff, dic).unwrap(); +} + +/// Review › Language and the proofing commands work in Portuguese end to end: the language makes +/// the document's words Portuguese, F7 and the issue list use the Portuguese dictionary and +/// grammar, suggestions come from it, and a suggestion fixes the text. +#[test] +fn portuguese_proofing_through_the_review_commands() { + portuguese_test_dictionary(); + let mut s = s(); + run(&mut s, "document.setText", json!({"text": "Saudade coraçao, e ele ele disse."})); + // English by default: the Portuguese words are English misspellings. + assert!(run(&mut s, "review.issues", json!({})).as_array().unwrap().len() >= 5); + run(&mut s, "select.all", json!({})); + run(&mut s, "review.language", json!({"lang": "pt_br", "default": true})); + assert_eq!(s.doc.styles.default_chr.lang.as_deref(), Some("pt-BR"), "the tag is normalized"); + assert_eq!(run(&mut s, "format.state", json!({}))["lang"], "pt-BR"); + let issues = run(&mut s, "review.issues", json!({})); + let arr = issues.as_array().unwrap(); + let has = |text: &str, kind: &str, sugg: &str| { + arr.iter() + .any(|i| i["text"] == text && i["kind"] == kind && i["lang"] == "pt" && i["suggestions"].as_array().unwrap().iter().any(|x| x == sugg)) + }; + assert!(has("coraçao", "spelling", "coração"), "{issues}"); + assert!(has("ele ele", "grammar", "ele"), "{issues}"); + assert_eq!(arr.len(), 2, "{issues}"); + assert!(arr.iter().any(|i| i["message"] == "Palavra repetida: \"ele\""), "{issues}"); + // F7 from the start selects the first issue; Change applies a suggestion. + run(&mut s, "caret.docStart", json!({})); + let first = run(&mut s, "review.spelling", json!({})); + assert_eq!(first["text"], "coraçao"); + run(&mut s, "review.applySuggestion", json!({"text": "coração"})); + assert!(text(&s).starts_with("Saudade coração,"), "{}", text(&s)); + // Add to Dictionary works for Portuguese words too. + run(&mut s, "review.addToDictionary", json!({"word": "coraçao"})); + let left = run(&mut s, "review.issues", json!({})); + assert_eq!(left.as_array().unwrap().len(), 1, "{left}"); +} + /// Review › Language and the proofing commands work in Polish end to end: the language makes /// the document's words Polish, F7 and the issue list use the Polish dictionary and grammar, /// suggestions come from it, and a suggestion fixes the text. diff --git a/crates/proof/src/hunspell.rs b/crates/proof/src/hunspell.rs index 2c09b11..1134208 100644 --- a/crates/proof/src/hunspell.rs +++ b/crates/proof/src/hunspell.rs @@ -26,8 +26,20 @@ pub fn set_dictionary_dir(dir: Option) { *EXTRA_DIR.write().unwrap_or_else(|e| e.into_inner()) = dir; } -/// Directories that may hold Hunspell dictionaries, most specific first. -pub fn search_dirs() -> Vec { +/// The LibreOffice extension directories a Hunspell dictionary name may live in: LibreOffice's +/// IDs are per dictionary — Polish is `dict-pl`, Brazilian Portuguese `dict-pt-BR` — so both +/// the primary-language folder and the dashed full name are tried. +fn lo_ext_dirs(name: &str) -> Vec { + let primary = name.split(['-', '_']).next().unwrap_or(name); + let dashed = name.replace('_', "-"); + let first = format!("dict-{primary}"); + let full = format!("dict-{dashed}"); + if full == first { vec![first] } else { vec![first, full] } +} + +/// Directories that may hold Hunspell dictionaries, most specific first. `lo_exts`, when a +/// language is being searched for, adds LibreOffice's extension folders for it (`dict-pt-BR`). +pub fn search_dirs(lo_exts: Option<&[String]>) -> Vec { let mut dirs: Vec = Vec::new(); if let Some(d) = EXTRA_DIR.read().unwrap_or_else(|e| e.into_inner()).clone() { dirs.push(d); @@ -38,6 +50,9 @@ pub fn search_dirs() -> Vec { if let Some(d) = std::env::var_os("WORDCRAFT_DICTIONARIES").filter(|v| !v.is_empty()) { dirs.push(PathBuf::from(d)); } + // LibreOffice keeps each language under `…/share/extensions/`. + let lo_dirs = + |prefix: &str| -> Vec { lo_exts.map_or_else(Vec::new, |exts| exts.iter().map(|e| PathBuf::from(prefix).join(e)).collect()) }; let home = std::env::var_os("HOME").filter(|v| !v.is_empty()).map(PathBuf::from); if cfg!(windows) { for var in ["LOCALAPPDATA", "APPDATA"] { @@ -48,7 +63,7 @@ pub fn search_dirs() -> Vec { } for var in ["ProgramFiles", "ProgramFiles(x86)"] { if let Some(base) = std::env::var_os(var).filter(|v| !v.is_empty()).map(PathBuf::from) { - dirs.push(base.join("LibreOffice").join("share").join("extensions").join("dict-pl")); + dirs.extend(lo_dirs(&base.join("LibreOffice").join("share").join("extensions").to_string_lossy())); } } } else if cfg!(target_os = "macos") { @@ -58,30 +73,26 @@ pub fn search_dirs() -> Vec { dirs.push(PathBuf::from("/Library/Spelling")); dirs.push(PathBuf::from("/opt/homebrew/share/hunspell")); dirs.push(PathBuf::from("/usr/local/share/hunspell")); - dirs.push(PathBuf::from("/Applications/LibreOffice.app/Contents/Resources/extensions/dict-pl")); + dirs.extend(lo_dirs("/Applications/LibreOffice.app/Contents/Resources/extensions")); } else { if let Some(h) = &home { dirs.push(h.join(".local/share/hunspell")); } - for d in [ - "/usr/share/hunspell", - "/usr/share/myspell", - "/usr/share/myspell/dicts", - "/usr/local/share/hunspell", - "/usr/local/share/myspell", - "/usr/lib/libreoffice/share/extensions/dict-pl", - "/opt/libreoffice/share/extensions/dict-pl", - ] { + for d in ["/usr/share/hunspell", "/usr/share/myspell", "/usr/share/myspell/dicts", "/usr/local/share/hunspell", "/usr/local/share/myspell"] { dirs.push(PathBuf::from(d)); } + for d in ["/usr/lib/libreoffice/share/extensions", "/opt/libreoffice/share/extensions"] { + dirs.extend(lo_dirs(d)); + } } dirs } /// The first `.aff`/`.dic` pair found for any of `names` (`pl_PL`, `pl`), searching -/// [`search_dirs`]. +/// [`search_dirs`] (including LibreOffice's extension folder for the first name). pub fn find(names: &[&str]) -> Option<(PathBuf, PathBuf)> { - search_dirs().into_iter().find_map(|dir| find_in(&dir, names)) + let lo = names.first().copied().map(lo_ext_dirs); + search_dirs(lo.as_deref()).into_iter().find_map(|dir| find_in(&dir, names)) } /// The first `.aff`/`.dic` pair for any of `names` in `dir`. @@ -146,6 +157,18 @@ mod tests { assert!(decode(b"SET UTF-8\n", b"\xff\xfe").is_err()); } + #[test] + fn libreoffice_extension_dirs_follow_the_language() { + // LibreOffice's IDs are per dictionary: Polish is `dict-pl`, Brazilian Portuguese + // `dict-pt-BR`, so both the primary-language folder and the dashed full name are tried. + assert_eq!(super::lo_ext_dirs("pl_PL"), ["dict-pl", "dict-pl-PL"]); + assert_eq!(super::lo_ext_dirs("pt_BR"), ["dict-pt", "dict-pt-BR"]); + assert_eq!(super::lo_ext_dirs("de-DE"), ["dict-de", "dict-de-DE"]); + let dirs = search_dirs(Some(&super::lo_ext_dirs("pt_BR"))); + assert!(dirs.iter().any(|d| d.ends_with("dict-pt-BR")), "{dirs:?}"); + assert!(search_dirs(None).iter().all(|d| !d.to_string_lossy().contains("dict-")), "no language: no LibreOffice guess"); + } + #[test] fn finds_and_loads_a_pair_in_a_directory() { let dir = std::env::temp_dir().join(format!("wordcraft-hunspell-{}", std::process::id())); diff --git a/crates/proof/src/hyphen.rs b/crates/proof/src/hyphen.rs index c244455..9919aa0 100644 --- a/crates/proof/src/hyphen.rs +++ b/crates/proof/src/hyphen.rs @@ -29,6 +29,8 @@ static EN_US_DIC: &[u8] = include_bytes!("../../../assets/hyphenation/en-us.dic" static EN_US_PAT: &str = include_str!("../../../assets/hyphenation/en-us.pat"); /// hyph-utf8's `hyph-pl.tex`, unmodified (its licence header is in the file). static PL_PAT: &str = include_str!("../../../assets/hyphenation/pl.pat"); +/// hyph-utf8's `hyph-pt.tex`, unmodified (its BSD 3-clause header is in the file). +static PT_PAT: &str = include_str!("../../../assets/hyphenation/pt.pat"); impl Dictionary { /// The bundled US English dictionary (decoded on first use). @@ -58,6 +60,12 @@ impl Patterns { static P: OnceLock = OnceLock::new(); P.get_or_init(|| Patterns::parse(PL_PAT)) } + + /// The bundled Portuguese patterns and exception words (parsed on first use). + pub fn pt() -> &'static Patterns { + static P: OnceLock = OnceLock::new(); + P.get_or_init(|| Patterns::parse(PT_PAT)) + } } /// Hyphenation limits from the paragraph's Hyphenation settings. @@ -101,6 +109,9 @@ fn core_breaks(core: &[char], lang: ProofLang) -> (Vec, Source) { if lang == ProofLang::Pl { return core_breaks_pl(core); } + if lang == ProofLang::Pt { + return core_breaks_pt(core); + } let dict = Dictionary::en_us(); let s: String = core.iter().collect(); let from_mask = |m: u64, len: usize| (1..len.min(64)).filter(|&i| m & (1 << i) != 0).collect::>(); @@ -140,6 +151,21 @@ fn core_breaks_pl(core: &[char]) -> (Vec, Source) { (heuristic_points_pl(core), Source::Heuristic) } +/// [`core_breaks_pl`] for Portuguese: the bundled patterns, then the generic heuristic. +fn core_breaks_pt(core: &[char]) -> (Vec, Source) { + let pats = Patterns::pt(); + let s: String = core.iter().collect(); + if let Some(p) = pats.exception(&s) { + return (p.to_vec(), Source::Dictionary); + } + if !pats.is_empty() + && let Some(p) = pats.points(core) + { + return (p, Source::Patterns); + } + (heuristic_points(core), Source::Heuristic) +} + /// Unrestricted break points of an English `word` ([`word_breaks_in`] with [`ProofLang::En`]). pub fn word_breaks(word: &str) -> (Vec, Source) { word_breaks_in(word, ProofLang::En) @@ -667,6 +693,40 @@ mod tests { assert_eq!(p.exception("nigdy"), Some(&[2][..])); } + fn pt(w: &str) -> String { + hyphenate_word_in(w, &Limits::default(), ProofLang::Pt) + } + + #[test] + fn portuguese_patterns_load() { + let p = Patterns::pt(); + assert!(p.len() > 300, "{}", p.len()); + // `hard-ware`, `soft-ware` — and the file's last pattern line, a bare `1-`, which the + // reader counts as a (never-matching) exception because it contains a hyphen. + assert_eq!(p.exception_count(), 3); + assert_eq!(p.exception("hardware"), Some(&[4][..])); + assert_eq!(p.exception("software"), Some(&[4][..])); + } + + #[test] + fn portuguese_words_hyphenate_by_portuguese_rules() { + for (w, want) in [ + ("problema", "pro-ble-ma"), + ("coração", "co-ra-ção"), + ("paralelepípedo", "pa-ra-le-le-pí-pe-do"), + ("computador", "com-pu-ta-dor"), + ("escrever", "es-cre-ver"), + ("saudade", "sau-da-de"), + ("extraordinário", "ex-tra-or-di-ná-rio"), + ("automático", "au-to-má-ti-co"), + // One syllable: no break. + ("flor", "flor"), + ("bem", "bem"), + ] { + assert_eq!(pt(w), want, "{w}"); + } + } + #[test] fn polish_words_hyphenate_by_polish_rules() { for (w, want) in [ diff --git a/crates/proof/src/lang.rs b/crates/proof/src/lang.rs index bfd0bea..344fc52 100644 --- a/crates/proof/src/lang.rs +++ b/crates/proof/src/lang.rs @@ -13,6 +13,10 @@ pub enum ProofLang { /// Polish (`pl`, `pl-PL`): the installed Hunspell dictionary (when there is one), the Polish grammar rules and the Polish /// hyphenation patterns. Pl, + /// Portuguese (`pt`, `pt-BR`, `pt-PT`): the installed Hunspell dictionary (VERO and + /// friends, when there is one), the Portuguese grammar rules and the Portuguese hyphenation + /// patterns. + Pt, /// A language without proofing tools here (`fr-FR`, `ja-JP`, `x-none`…). Like Word without /// the language's proofing tools: no spelling or grammar marks, so its words are never /// flagged as English misspellings. Hyphenation keeps using the English patterns, as before @@ -22,7 +26,7 @@ pub enum ProofLang { impl ProofLang { /// Every language with a spelling dictionary, in display order. - pub const PROOFED: [ProofLang; 2] = [ProofLang::En, ProofLang::Pl]; + pub const PROOFED: [ProofLang; 3] = [ProofLang::En, ProofLang::Pl, ProofLang::Pt]; /// The proofing language of a BCP 47 tag (`pl-PL`, `en_GB`, `PL`…). Only the primary subtag /// counts. No tag (or an empty one) is English, WordCraft's default editing language. @@ -33,6 +37,8 @@ impl ProofLang { ProofLang::En } else if primary.eq_ignore_ascii_case("pl") || primary.eq_ignore_ascii_case("pol") { ProofLang::Pl + } else if primary.eq_ignore_ascii_case("pt") || primary.eq_ignore_ascii_case("por") { + ProofLang::Pt } else { ProofLang::Other } @@ -43,6 +49,7 @@ impl ProofLang { match self { ProofLang::En => "en", ProofLang::Pl => "pl", + ProofLang::Pt => "pt", ProofLang::Other => "", } } @@ -116,6 +123,9 @@ mod tests { for t in ["pl", "pl-PL", "PL-pl", "pl_PL", "pl_PL.UTF-8", " pl-PL ", "pol"] { assert_eq!(ProofLang::from_tag(Some(t)), ProofLang::Pl, "{t}"); } + for t in ["pt", "pt-BR", "PT_br", "pt-BR.UTF-8", " pt-PT ", "por"] { + assert_eq!(ProofLang::from_tag(Some(t)), ProofLang::Pt, "{t}"); + } for t in ["en", "en-US", "en-GB", "EN_au", "en-Latn-US"] { assert_eq!(ProofLang::from_tag(Some(t)), ProofLang::En, "{t}"); } @@ -126,6 +136,9 @@ mod tests { assert_eq!(ProofLang::from_tag(Some("")), ProofLang::En); assert_eq!(ProofLang::from_tag(Some(" ")), ProofLang::En); assert!(ProofLang::Pl.is_proofed() && !ProofLang::Other.is_proofed()); + assert!(ProofLang::Pt.is_proofed() && !ProofLang::Other.is_proofed()); assert_eq!(ProofLang::Pl.code(), "pl"); + assert_eq!(ProofLang::Pt.code(), "pt"); + assert!(ProofLang::PROOFED.contains(&ProofLang::Pt), "every proofed language is listed"); } } diff --git a/crates/proof/src/lib.rs b/crates/proof/src/lib.rs index 32d1aa7..736d216 100644 --- a/crates/proof/src/lib.rs +++ b/crates/proof/src/lib.rs @@ -12,9 +12,13 @@ //! spelling marks), read by the affix engine ([`affix`]), suggestions tuned for Polish //! (diacritics, `ż`/`rz`, `u`/`ó`, missing spaces), and Polish grammar rules with Polish //! messages. +//! - **Portuguese** ([`ProofLang::Pt`], module [`pt`]): spelling against the Hunspell `pt_BR` +//! dictionary installed on the system (VERO and friends; not bundled), suggestions tuned for +//! Portuguese (missing accents, `seje`/`seja` confusions, missing spaces), conservative +//! grammar rules with Portuguese messages, and `hyph-utf8`'s Portuguese patterns. //! - **Other languages** ([`ProofLang::Other`]): no spelling or grammar marks, like Word without //! the language's proofing tools. -//! - **Hyphenation** ([`hyphen`]): dictionary, Liang patterns (English, Polish), heuristic. +//! - **Hyphenation** ([`hyphen`]): dictionary, Liang patterns (English, Polish, Portuguese), heuristic. //! //! [`check_text`] checks a paragraph whose runs are in different languages. #![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] @@ -26,6 +30,7 @@ pub mod hyphen; pub mod lang; pub mod patterns; pub mod pl; +pub mod pt; use std::collections::HashSet; use std::ops::Range; @@ -489,6 +494,7 @@ fn spelling_issue(text: &str, a: usize, b: usize, lang: ProofLang, opts: &Option } let message = match lang { ProofLang::Pl => "Możliwy błąd pisowni", + ProofLang::Pt => "Possível erro de ortografia", _ => "Possible spelling mistake", }; Some(Issue { start: a, end: b, kind: IssueKind::Spelling, message: message.into(), suggestions: Vec::new(), lang }) @@ -504,6 +510,7 @@ pub fn is_correct_in_with(word: &str, lang: ProofLang, opts: &Options) -> bool { match lang { ProofLang::En => is_correct_with(word, opts), ProofLang::Pl => accepted_by_options(word, opts) || pl::is_correct(word), + ProofLang::Pt => accepted_by_options(word, opts) || pt::is_correct(word), ProofLang::Other => true, } } @@ -513,16 +520,19 @@ pub fn suggest_in(word: &str, max: usize, lang: ProofLang) -> Vec { match lang { ProofLang::En => suggest(word, max), ProofLang::Pl => pl::suggest(word, max), + ProofLang::Pt => pt::suggest(word, max), ProofLang::Other => Vec::new(), } } /// Grammar issues in a paragraph of text in `lang`: the English rules, the Polish ones -/// ([`pl::grammar`]), or none for a language WordCraft doesn't proof. +/// ([`pl::grammar`]), the Portuguese ones ([`pt::grammar`]), or none for a language WordCraft +/// doesn't proof. pub fn check_grammar_in(text: &str, lang: ProofLang) -> Vec { match lang { ProofLang::En => check_grammar(text), ProofLang::Pl => pl::grammar::check(text), + ProofLang::Pt => pt::grammar::check(text), ProofLang::Other => Vec::new(), } } @@ -994,6 +1004,12 @@ mod tests { assert_eq!(flagged("helo świat", &check_text("helo świat", &[(5..12, ProofLang::Pl)])), ["helo"]); // A language without tools: nothing flagged there. assert!(check_text("Bonjour mes amis", &[(0..16, ProofLang::Other)]).is_empty()); + // Portuguese checks in their span only, with the Portuguese message. + pt::test_dict::install(); + let t = "Ele disse que coraçao e frequencia existem."; + let v = check_text(t, &[(0..t.len(), ProofLang::Pt)]); + assert_eq!(flagged(t, &v), ["coraçao", "frequencia"], "{v:?}"); + assert!(v.iter().all(|i| i.lang == ProofLang::Pt && i.message == "Possível erro de ortografia"), "{v:?}"); } proptest::proptest! { @@ -1005,7 +1021,7 @@ mod tests { let _ = has_dictionary(Some(&s)); let _ = is_correct_for(&s, Some(&s)); let _ = hyphen::hyphen_points(&s, &hyphen::Limits::default()); - for lang in [ProofLang::En, ProofLang::Pl, ProofLang::Other] { + for lang in [ProofLang::En, ProofLang::Pl, ProofLang::Pt, ProofLang::Other] { let _ = check_spelling_in(&s, lang); let _ = check_grammar_in(&s, lang); let _ = is_correct_in(&s, lang); @@ -1021,9 +1037,14 @@ mod tests { } #[test] - fn mixed_spans_never_panic(s in "\\PC{0,60}", cuts in proptest::collection::vec((0usize..80, 0usize..80, 0u8..3), 0..6)) { - let langs = [ProofLang::En, ProofLang::Pl, ProofLang::Other]; - let spans: Vec<(Range, ProofLang)> = cuts.iter().map(|&(a, b, l)| (a.min(b)..a.max(b), langs[l as usize % 3])).collect(); + fn portuguese_suggestions_never_panic(s in "[a-záàâãéêíóôõúüçA-ZÁÀÂÃÉÊÍÓÔÕÚÜÇ'’ -]{0,14}") { + let _ = suggest_in(&s, 4, ProofLang::Pt); + } + + #[test] + fn mixed_spans_never_panic(s in "\\PC{0,60}", cuts in proptest::collection::vec((0usize..80, 0usize..80, 0u8..4), 0..6)) { + let langs = [ProofLang::En, ProofLang::Pl, ProofLang::Pt, ProofLang::Other]; + let spans: Vec<(Range, ProofLang)> = cuts.iter().map(|&(a, b, l)| (a.min(b)..a.max(b), langs[l as usize % 4])).collect(); for i in check_text(&s, &spans) { proptest::prop_assert!(i.start <= i.end && i.end <= s.len()); } diff --git a/crates/proof/src/pt/grammar.rs b/crates/proof/src/pt/grammar.rs new file mode 100644 index 0000000..c5109de --- /dev/null +++ b/crates/proof/src/pt/grammar.rs @@ -0,0 +1,245 @@ +//! Portuguese grammar and punctuation checks: cheap, rule-based and tuned to stay quiet when +//! unsure (a missed mistake costs less than a wrong mark on correct text). Messages are in +//! Portuguese. +//! +//! - **Repeated word** (`ele ele`), except interjections and words Portuguese doubles on purpose +//! (`sim sim`, `não não`, `ora ora`). +//! - **Space before punctuation** (`bem ,`) and **double spaces**; emoticons (` :)`) and +//! ellipses are left alone. +//! - **Capital letter at the start of a sentence**, knowing Portuguese abbreviations (`etc.`, +//! `pág.`, `Prof.`, `Dr.`), initials, ordinal numbers (`5. colocado`), ellipses and dialogue +//! dashes (`— Vai — disse`), after which lower case is correct. +//! - **Doubled punctuation** (`,,`, `..`, `;;`). +//! +//! No comma rule (`que` opens clauses only sometimes) and no English rule (such as `a`/`an`) +//! applies to Portuguese text. + +use crate::{Issue, IssueKind, ProofLang, words}; + +/// Words Portuguese repeats on purpose: interjections, onomatopoeia and emphatic doubling. +const ALLOWED_REPEATS: &[&str] = &[ + "sim", "não", "ora", "olha", "já", "bem", "muito", "vai", "lá", "cá", "aí", "ha", "he", "hi", "oh", "ah", "ai", "ui", "uf", "uxa", "oba", "viva", + "psiu", "olá", "tchau", "xau", "adeus", "devagar", +]; + +/// Abbreviations written with a final period after which a sentence goes on in lower case +/// (lower case, without the period). +const ABBREVIATIONS: &[&str] = &[ + "etc", "ex", "pág", "págs", "séc", "sécs", "fl", "av", "vol", "p", "pp", "ed", "tel", "min", "max", "art", "par", "prof", "profa", "dr", "dra", + "sr", "sra", "srta", "eng", "arq", "op", "cit", "cf", "ibid", "id", "vide", "seg", "ter", "qua", "qui", "sex", "sáb", "dom", "jan", "fev", "mar", + "abr", "mai", "jun", "jul", "ago", "set", "out", "nov", "dez", "s.a", "d.c", "a.c", "pg", "ref", "cod", "dept", "univ", "inst", "lat", "ing", + "esp", "fr", "it", "al", "obs", "att", "obs", +]; + +fn issue(start: usize, end: usize, message: String, suggestions: Vec) -> Issue { + Issue { start, end, kind: IssueKind::Grammar, message, suggestions, lang: ProofLang::Pt } +} + +fn capitalize(w: &str) -> String { + crate::affix::capitalize(w) +} + +/// Is `prev`, before a period, an abbreviation, an initial, a number or a Roman numeral (after +/// which a lower-case word doesn't start a sentence)? +fn abbreviation_or_number(prev: &str) -> bool { + prev.chars().count() <= 1 + || prev.chars().any(|c| c.is_ascii_digit()) + || (prev.chars().all(|c| matches!(c, 'I' | 'V' | 'X' | 'L' | 'C' | 'D' | 'M'))) + || ABBREVIATIONS.contains(&prev.to_lowercase().as_str()) +} + +const CLOSING: [char; 8] = ['"', '”', '’', '\'', ')', '»', ']', '\u{1}']; +const OPENING: [char; 8] = ['"', '“', '«', '(', '[', '\'', '‚', '—']; + +/// Does the gap between two words end a sentence: a `.`, `!` or `?` (not an ellipsis, not after +/// an abbreviation), then space, with nothing but quotes and brackets around? +fn ends_sentence(gap: &str, prev: &str) -> bool { + let t = gap.trim_start_matches(CLOSING); + let Some(end) = t.chars().next() else { return false }; + if !matches!(end, '.' | '!' | '?') || t.starts_with("..") { + return false; + } + if end == '.' && abbreviation_or_number(prev) { + return false; + } + let rest = t.trim_start_matches(['.', '!', '?']).trim_start_matches(CLOSING); + rest.starts_with(char::is_whitespace) && rest.trim_start().chars().all(|c| OPENING.contains(&c)) +} + +/// Grammar issues in a paragraph of Portuguese text, in order. +pub fn check(text: &str) -> Vec { + let ws: Vec<(usize, usize)> = words(text); + let mut v = Vec::new(); + repeated_words(text, &ws, &mut v); + spaces(text, &mut v); + sentence_capitals(text, &ws, &mut v); + doubled_punctuation(text, &mut v); + v.sort_by_key(|i| (i.start, i.end)); + v +} + +fn repeated_words(text: &str, ws: &[(usize, usize)], v: &mut Vec) { + for pair in ws.windows(2) { + let [(a0, b0), (a1, b1)] = pair else { continue }; + let (Some(w0), Some(w1), Some(gap)) = (text.get(*a0..*b0), text.get(*a1..*b1), text.get(*b0..*a1)) else { continue }; + if !gap.is_empty() + && gap.chars().all(char::is_whitespace) + && w0.chars().all(char::is_alphabetic) + && w0.to_lowercase() == w1.to_lowercase() + && !ALLOWED_REPEATS.contains(&w0.to_lowercase().as_str()) + { + v.push(issue(*a0, *b1, format!("Palavra repetida: \"{w1}\""), vec![w0.to_string()])); + } + } +} + +fn spaces(text: &str, v: &mut Vec) { + for (i, c) in text.char_indices() { + if c != ' ' { + continue; + } + let rest = text.get(i + 1..).unwrap_or(""); + let mut it = rest.chars(); + let Some(p) = it.next() else { continue }; + let after = it.next(); + let flag = match p { + ',' => true, + // `;)` and `;-)` are emoticons. + ';' => !matches!(after, Some(')' | '(' | '-' | 'P' | 'D')), + // ` ...` is an ellipsis, ` .5` a number. + '.' => !matches!(after, Some('.')) && !after.is_some_and(|a| a.is_ascii_digit()), + // ` :)`, ` :-)`, ` :D` and similar are emoticons; ` ?!` is still a stray space. + ':' | '!' | '?' => after.is_none_or(|a| a.is_whitespace() || matches!(a, '"' | '”' | '’' | '!' | '?')), + _ => false, + }; + if flag { + v.push(issue(i, i + 1 + p.len_utf8(), "Espaço antes da pontuação".into(), vec![p.to_string()])); + } + } + for (i, _) in text.match_indices(" ") { + if i > 0 && text.get(..i).is_some_and(|t| t.ends_with(' ')) { + continue; + } + v.push(issue(i, i + 2, "Espaço duplo".into(), vec![" ".into()])); + } +} + +fn sentence_capitals(text: &str, ws: &[(usize, usize)], v: &mut Vec) { + // List items (`– maçãs,`) may start in lower case. + let list_item = text.trim_end().ends_with([',', ';']); + for (k, &(a, b)) in ws.iter().enumerate() { + let Some(w) = text.get(a..b) else { continue }; + let starts = if k == 0 { + let before = text.get(..a).unwrap_or(""); + !list_item && before.chars().all(|c| c.is_whitespace() || OPENING.contains(&c)) + } else { + let Some(&(pa, pb)) = ws.get(k - 1) else { continue }; + ends_sentence(text.get(pb..a).unwrap_or(""), text.get(pa..pb).unwrap_or("")) + }; + if starts && w.chars().next().is_some_and(char::is_lowercase) && w.chars().all(char::is_alphabetic) { + v.push(issue(a, b, "Letra maiúscula no início da frase".into(), vec![capitalize(w)])); + } + } +} + +fn doubled_punctuation(text: &str, v: &mut Vec) { + let bytes = text.as_bytes(); + let mut i = 0; + while let Some(&b) = bytes.get(i) { + if !matches!(b, b',' | b'.' | b';') { + i += 1; + continue; + } + let run = bytes.get(i..).map_or(0, |r| r.iter().take_while(|&&x| x == b).count()); + let flag = match b { + b'.' => run == 2, + _ => run >= 2, + }; + if flag { + let end = i + run; + let suggestions: Vec = match b { + b'.' => vec![".".into(), "…".into()], + _ => vec![char::from(b).to_string()], + }; + v.push(issue(i, end, "Sinal repetido".into(), suggestions)); + } + i += run.max(1); + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn messages(t: &str) -> Vec { + check(t).into_iter().map(|i| format!("{}@{}", i.message, &t[i.start..i.end])).collect() + } + + fn clean(t: &str) { + let m = messages(t); + assert!(m.is_empty(), "{t:?}: {m:?}"); + } + + #[test] + fn repeated_words() { + let v = check("Eu eu vi isso e ele ele saiu."); + assert_eq!(v.len(), 2, "{v:?}"); + assert_eq!(v[0].message, "Palavra repetida: \"eu\""); + assert_eq!(v[0].suggestions, ["Eu"]); + assert!(v.iter().all(|i| i.lang == ProofLang::Pt && i.kind == IssueKind::Grammar)); + clean("Sim, sim! Não, não! Ora ora, que surpresa."); + } + + #[test] + fn spaces_and_punctuation() { + let m = messages("Olá , mundo . Tudo bem ?"); + assert!(m.contains(&"Espaço antes da pontuação@ ,".to_string()), "{m:?}"); + assert!(m.contains(&"Espaço antes da pontuação@ .".to_string()), "{m:?}"); + assert!(m.contains(&"Espaço antes da pontuação@ ?".to_string()), "{m:?}"); + assert!(m.contains(&"Espaço duplo@ ".to_string()), "{m:?}"); + clean("Legal :) Muito bom :-) Ou :D Valor .5 e reticências ..."); + let m = messages("Falou,, e saiu.. depois;; fim."); + assert_eq!(m.iter().filter(|x| x.starts_with("Sinal repetido")).count(), 3, "{m:?}"); + clean("Esperei… e esperei... até ele voltar."); + } + + #[test] + fn sentence_start_capitals() { + let m = messages("isso é um teste. a segunda frase! terceira? ok."); + assert_eq!(m.iter().filter(|x| x.starts_with("Letra maiúscula")).count(), 4, "{m:?}"); + assert_eq!(check("ele tem razão.")[0].suggestions, ["Ele"]); + // Abbreviations, initials, ordinals, ellipses and dialogue dashes. + clean("Comprei frutas, etc. e depois saímos."); + clean("Conheci o Dr. Silva na Av. Paulista."); + clean("Zhou ficou em 5. lugar no séc. XX."); + clean("Esperei... e ele apareceu."); + clean("— Vai — disse ele. — Amanhã eu volto."); + clean("Ele disse: \"Volto amanhã\". E saiu."); + // A list item may start in lower case. + clean("maçãs,"); + } + + #[test] + fn correct_portuguese_text_is_clean() { + clean( + "O rato roeu a roupa do rei de Roma. Quem não tem cão caça com gato, \ + e quem tem paciência colhe o que plantou.", + ); + clean("Brasília, 12 de outubro de 2026 — uma data memorável."); + } + + proptest::proptest! { + #[test] + fn never_panics(s in "\\PC{0,80}") { + let _ = check(&s); + } + + #[test] + fn never_panics_on_portuguese_like_text(s in "[a-záàâãéêíóôõúüçA-Z ,.;:!?\"…\\-—]{0,60}") { + for i in check(&s) { + proptest::prop_assert!(i.start <= i.end && i.end <= s.len()); + proptest::prop_assert!(s.is_char_boundary(i.start) && s.is_char_boundary(i.end)); + } + } + } +} diff --git a/crates/proof/src/pt/mod.rs b/crates/proof/src/pt/mod.rs new file mode 100644 index 0000000..be16c74 --- /dev/null +++ b/crates/proof/src/pt/mod.rs @@ -0,0 +1,202 @@ +//! Portuguese proofing. +//! +//! - **Spelling** against the Hunspell `pt_BR` dictionary installed on the system +//! ([`dictionary`], found by [`crate::hunspell`]: `pt_BR.aff`/`pt_BR.dic` — VERO and its +//! derivatives — from the OS, a package manager or LibreOffice; it isn't bundled). Without it +//! Portuguese text gets no spelling marks. Capitalized stems are names (`Brasília`); a sentence +//! may start with any word and headings may be in capitals. Hyphenated compounds are checked +//! part by part (`guarda-roupa`, `ex-marido`), and the user dictionary applies as in English. +//! - **Suggestions** tuned for Portuguese ([`suggest`]): missing accents first (`coraçao` → +//! `coração`, `saude` → `saúde`), then the dictionary's `REP` confusions (`seje` → `seja`), a +//! missing space, one-letter typos, and a dictionary scan for anything further off. +//! - **Grammar** ([`grammar`]): cheap, reliable rules with Portuguese messages. + +pub mod grammar; +mod suggest; +#[cfg(test)] +pub(crate) mod test_dict; + +use std::sync::{Arc, RwLock}; + +use crate::affix::AffixDict; + +pub use suggest::suggest; + +/// The Portuguese dictionary: `None` until first use, then the installed one (or an empty one). +static DICT: RwLock>> = RwLock::new(None); + +/// File names of a Portuguese Hunspell dictionary. +const NAMES: &[&str] = &["pt_BR", "pt-BR", "pt"]; + +/// The Portuguese dictionary: the installed Hunspell `pt_BR` (read on first use), one given with +/// [`install_dictionary`], or an empty one when none is available (Portuguese spelling is off +/// then; see [`has_dictionary`]). +pub fn dictionary() -> Arc { + if let Some(d) = DICT.read().unwrap_or_else(|e| e.into_inner()).as_ref() { + return d.clone(); + } + let loaded = Arc::new(load_installed()); + let mut slot = DICT.write().unwrap_or_else(|e| e.into_inner()); + // Another thread (or an install) may have filled it meanwhile: that one wins. + slot.get_or_insert(loaded).clone() +} + +fn load_installed() -> AffixDict { + let Some((aff, dic)) = crate::hunspell::find(NAMES) else { + log::info!("Portuguese spelling: no Hunspell pt_BR dictionary installed"); + return AffixDict::default(); + }; + crate::hunspell::load(&aff, &dic).unwrap_or_else(|e| { + log::warn!("Portuguese spelling: {e}"); + AffixDict::default() + }) +} + +/// Use this Portuguese dictionary from now on (Hunspell `.aff` and `.dic` text, decoded to +/// UTF-8), for example one the user picked. On error the current dictionary stays. +pub fn install_dictionary(aff: &str, dic: &str) -> Result<(), String> { + let d = AffixDict::from_hunspell(aff, dic, &[])?; + *DICT.write().unwrap_or_else(|e| e.into_inner()) = Some(Arc::new(d)); + Ok(()) +} + +/// Forget the loaded dictionary, so the next check looks for an installed one again (after +/// [`crate::hunspell::set_dictionary_dir`] or a dictionary was installed). +pub fn reload_dictionary() { + *DICT.write().unwrap_or_else(|e| e.into_inner()) = None; +} + +/// Is a Portuguese dictionary available (spelling is checked)? +pub fn has_dictionary() -> bool { + !dictionary().is_empty() +} + +/// Is `c` a letter of the Latin script (the only one the Portuguese dictionary covers)? +fn is_latin_letter(c: char) -> bool { + c.is_alphabetic() && (c < '\u{0250}' || ('\u{1E00}'..='\u{1EFF}').contains(&c)) +} + +/// Is `word` spelled correctly in Portuguese? Numbers, single letters, short words in capitals +/// (acronyms: `USP`, `PT`), words with digits and words in other scripts are accepted, and so +/// is every word when no Portuguese dictionary is installed. +pub fn is_correct(word: &str) -> bool { + is_correct_with(&dictionary(), word) +} + +/// [`is_correct`] against a given dictionary (an empty one accepts everything). +pub fn is_correct_with(d: &AffixDict, word: &str) -> bool { + if d.is_empty() { + return true; // no Portuguese dictionary installed: no spelling marks + } + let w = word.trim_matches(|c: char| !c.is_alphanumeric() && c != '\'' && c != '’'); + let w = w.trim_matches(['\'', '’']); + let n = w.chars().count(); + if n <= 1 || w.chars().any(|c| c.is_ascii_digit()) || !w.chars().any(char::is_alphabetic) { + return true; + } + if w.chars().all(|c| !c.is_lowercase()) && n <= 6 { + return true; + } + if !w.chars().any(is_latin_letter) { + return true; // other scripts: no dictionary + } + if w.contains('-') { + return w.split('-').all(|p| p.chars().count() <= 1 || known(d, p)); + } + known(d, w) +} + +/// Is one word (no hyphens) in the dictionary or the user dictionary? +fn known(d: &AffixDict, w: &str) -> bool { + d.check(w) || crate::in_user_dictionary(&w.to_lowercase()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_dictionary_loads() { + test_dict::install(); + let d = dictionary(); + assert!(has_dictionary()); + assert!(d.len() > 50, "{}", d.len()); + assert!(d.rule_count() >= 5, "the test dictionary's affix rules"); + assert!(!d.rep().is_empty() && !d.map().is_empty() && !d.try_chars().is_empty()); + } + + #[test] + fn without_a_dictionary_nothing_is_flagged() { + let none = AffixDict::default(); + for w in ["coraçao", "saude", "xqzv", ""] { + assert!(is_correct_with(&none, w), "{w}"); + } + assert!(install_dictionary("FLAG long\n", "1\nkot\n").is_err(), "unsupported files are refused"); + } + + #[test] + fn portuguese_words_are_correct() { + test_dict::install(); + for w in [ + "coração", + "saudade", + "pássaro", + "saúde", + "árvore", + "escrever", + "computador", + "praia", + "conhecer", + "Brasília", + "São", + "autoestima", + "frequência", + "exceção", + "amanhã", + "vocês", + "coragem", + "parabéns", + "cantava", + "cantando", + "provável", + "prováveis", + "descobrir", + "trabalho", + "CORAGEM", + "USP", + "2026", + "e-mail", + "guarda-roupa", + "ex-marido", + "vice-presidente", + "e", + "não", + ] { + assert!(is_correct(w), "{w}"); + } + } + + #[test] + fn misspellings_are_flagged() { + test_dict::install(); + for w in ["coraçao", "saude", "trbalho", "excrever", "prsidente", "frequencia", "amnha", "cantaando"] { + assert!(!is_correct(w), "{w}"); + } + } + + #[test] + fn forbidden_words_are_rejected() { + test_dict::install(); + // `frequencia/F` is listed with FORBIDDENWORD: the old spelling, never correct. + assert!(!is_correct("frequencia") && !is_correct("Frequencia") && !is_correct("FREQUENCIA")); + assert!(is_correct("frequência")); + } + + #[test] + fn user_dictionary_applies() { + test_dict::install(); + assert!(!is_correct("zxqportuguespalavra")); + crate::add_word("Zxqportuguespalavra"); + assert!(is_correct("zxqportuguespalavra")); + } +} diff --git a/crates/proof/src/pt/suggest.rs b/crates/proof/src/pt/suggest.rs new file mode 100644 index 0000000..314ef39 --- /dev/null +++ b/crates/proof/src/pt/suggest.rs @@ -0,0 +1,444 @@ +//! Portuguese spelling suggestions, best first. +//! +//! Candidates come in rounds of rising cost, each checked against the dictionary: +//! 1. letter case (`brasília` → `Brasília`); +//! 2. missing or wrong accents, from the dictionary's `MAP` groups (`coraçao` → `coração`, +//! `saude` → `saúde`), up to four letters changed — Portuguese typed without accents is the +//! most common slip, so adding one ranks first, while taking away an accent the writer typed +//! ranks after the known confusions; +//! 3. the dictionary's `REP` table of common confusions (`seje` → `seja`, `excessão` → +//! `exceção`, `previlégio` → `privilégio`); +//! 4. a missing space (`nãoé` → `não é`, `paraela` → `para ela`), cheaper when one half is a +//! short function word; +//! 5. one typo: a letter left out, added, swapped with the next or mistyped (`TRY` letters); +//! 6. when that finds fewer than asked for: the stems that start like the word (accents +//! folded) and share the most letter pairs with it, whose forms are ranked by an edit distance +//! in which an accent slip costs half (two typos at most). +//! +//! The input is bounded (40 letters), and so is every round, so a call costs ten-odd thousand +//! dictionary lookups at most. + +use std::collections::{HashMap, HashSet}; + +use crate::affix::{AffixDict, Casing, capitalize, casing}; + +/// Short words often run into the next one (`não é`, `da água`, `para isso`). +const LEADING_WORDS: &[&str] = &[ + "a", "o", "e", "é", "de", "do", "da", "dos", "das", "em", "no", "na", "nos", "nas", "por", "para", "com", "sem", "sob", "sobre", "ao", "aos", + "à", "às", "que", "se", "não", "como", "mas", "mais", "já", "até", "entre", "contra", "perante", "desde", "após", "ante", "durante", "mediante", + "segundo", "conforme", "salvo", "fora", "onde", "quando", "bem", "mal", "pois", "logo", "portanto", "porém", "todavia", "contudo", "assim", + "tão", "tanto", "tudo", "nada", "algo", "cada", "outro", "mesmo", "próprio", "ainda", "tampouco", "quase", "talvez", "nunca", "sempre", +]; +/// Short words often run into the one before (`amo-te`, `deu-lhe`; in writing without the +/// hyphen: `amote`, `deulhe`). +const TRAILING_WORDS: &[&str] = &[ + "me", "te", "se", "lhe", "nos", "vos", "lhes", "o", "a", "os", "as", "ele", "ela", "eles", "elas", "isso", "isto", "aquilo", "mim", "ti", "si", + "eu", "tu", "que", "de", "em", "é", +]; +const MAX_LETTERS: usize = 40; +const MAX_MAP_CHANGES: usize = 4; +const MAX_MAP_VARIANTS: usize = 4096; +/// Stems the scan of round 6 expands. +const SCAN_STEMS: usize = 48; + +const COST_CASE: u32 = 0; +const COST_ADD_DIACRITIC: u32 = 4; +const COST_OTHER_MAP: u32 = 7; +const COST_DROP_DIACRITIC: u32 = 11; +const COST_REP: u32 = 10; +const COST_SPLIT: u32 = 12; +const COST_SPLIT_RARE: u32 = 18; +const COST_SWAP: u32 = 14; +const COST_DROP_OR_ADD: u32 = 15; +const COST_MISTYPE: u32 = 16; +const COST_SCAN: u32 = 20; + +struct Collector<'a> { + d: &'a AffixDict, + /// The word, lower case. + input: String, + tested: HashSet, + /// Suggestion → (cost, order found). + found: HashMap, + seq: usize, +} + +impl Collector<'_> { + /// Check a lower-case candidate (a word, or two separated by a space) and keep it if every + /// word is in the dictionary, as written or capitalized (names). + fn offer(&mut self, cand: &str, cost: u32) { + if cand.is_empty() || cand == self.input || !self.tested.insert(cand.to_string()) { + return; + } + let mut words = Vec::new(); + let mut cost = cost; + for part in cand.split(' ') { + if self.d.contains(part) { + words.push(part.to_string()); + } else { + let c = capitalize(part); + if !self.d.contains(&c) { + return; + } + words.push(c); + // A name ranks after a common word at the same distance (`moça`, then `Moça`). + cost += 1; + } + } + self.keep(words.join(" "), cost); + } + + /// Keep a suggestion known to be correct. + fn keep(&mut self, text: String, cost: u32) { + self.seq += 1; + let seq = self.seq; + let e = self.found.entry(text).or_insert((cost, seq)); + if cost < e.0 { + *e = (cost, seq); + } + } + + fn count_within(&self, cost: u32) -> usize { + self.found.values().filter(|(c, _)| *c <= cost).count() + } +} + +/// The MAP group of each letter (its other members are its likely confusions). +fn alternatives(c: char, map: &[Vec]) -> Vec { + map.iter().filter(|g| g.contains(&c)).flat_map(|g| g.iter().copied()).filter(|&x| x != c).collect() +} + +/// The cost of changing letter `from` to its MAP relative `to`. Adding a missing accent is the +/// commonest slip (Portuguese typed without accents: `coracao`); an accent the writer did type +/// is likelier right, so taking one away costs more than a known confusion from the REP table +/// (`seje` → `seja` before `sejé`). +fn map_cost(from: char, to: char) -> u32 { + match (from.is_ascii(), to.is_ascii()) { + (true, false) => COST_ADD_DIACRITIC, + (false, true) => COST_DROP_DIACRITIC, + _ => COST_OTHER_MAP, + } +} + +/// Every way of changing up to [`MAX_MAP_CHANGES`] letters to a MAP relative, with its cost. +fn map_variants(chars: &[char], alts: &[Vec], i: usize, changes: usize, cost: u32, buf: &mut Vec, out: &mut Vec<(String, u32)>) { + if out.len() >= MAX_MAP_VARIANTS { + return; + } + let Some(&c) = chars.get(i) else { + if changes > 0 { + out.push((buf.iter().collect(), cost)); + } + return; + }; + buf.push(c); + map_variants(chars, alts, i + 1, changes, cost, buf, out); + buf.pop(); + if changes < MAX_MAP_CHANGES { + for &a in alts.get(i).map(Vec::as_slice).unwrap_or_default() { + buf.push(a); + map_variants(chars, alts, i + 1, changes + 1, cost + map_cost(c, a), buf, out); + buf.pop(); + } + } +} + +/// A letter's MAP representative (the first of its group), lower case: `ã` → `a`, `ç` → `c`. +fn fold(c: char, map: &[Vec]) -> char { + let l = c.to_lowercase().next().unwrap_or(c); + map.iter().find(|g| g.contains(&l)).and_then(|g| g.first().copied()).unwrap_or(l) +} + +/// Optimal string alignment distance where a MAP slip (`a`/`á`, `c`/`ç`) costs 1 and any other +/// edit 2; `limit + 1` once it is certainly above `limit`. +fn weighted_distance(a: &[char], b: &[char], related: &dyn Fn(char, char) -> bool, limit: u32) -> u32 { + let m = b.len(); + let over = limit + 1; + let mut prev2 = vec![0u32; m + 1]; + let mut prev: Vec = (0..=m as u32).map(|j| j * 2).collect(); + let mut cur = vec![0u32; m + 1]; + for (i, &ai) in a.iter().enumerate() { + if let Some(c) = cur.get_mut(0) { + *c = (i as u32 + 1) * 2; + } + let mut row_min = u32::MAX; + for (j, &bj) in b.iter().enumerate() { + let sub = if ai == bj { + 0 + } else if related(ai, bj) { + 1 + } else { + 2 + }; + let at = |v: &[u32], k: usize| v.get(k).copied().unwrap_or(u32::MAX / 4); + let mut v = (at(&prev, j + 1) + 2).min(at(&cur, j) + 2).min(at(&prev, j) + sub); + if i > 0 && j > 0 && a.get(i - 1) == Some(&bj) && b.get(j - 1) == Some(&ai) { + v = v.min(at(&prev2, j - 1) + 2); + } + if let Some(c) = cur.get_mut(j + 1) { + *c = v; + } + row_min = row_min.min(v); + } + if m > 0 && row_min > limit { + return over; + } + std::mem::swap(&mut prev2, &mut prev); + std::mem::swap(&mut prev, &mut cur); + } + prev.get(m).copied().unwrap_or(over).min(over) +} + +/// Spelling suggestions for a Portuguese word, best first, at most `max`, in the word's letter case. +pub fn suggest(word: &str, max: usize) -> Vec { + suggest_in(&super::dictionary(), word, max) +} + +/// [`suggest`] against a given dictionary. +pub fn suggest_in(d: &AffixDict, word: &str, max: usize) -> Vec { + let word = word.trim(); + let n = word.chars().count(); + if n == 0 || n > MAX_LETTERS || max == 0 || d.is_empty() || word.contains(char::is_whitespace) { + return Vec::new(); + } + let case = casing(word); + let input = word.to_lowercase(); + let chars: Vec = input.chars().collect(); + let mut col = Collector { d, input: input.clone(), tested: HashSet::new(), found: HashMap::new(), seq: 0 }; + // 1. Letter case. + if d.contains(&input) && word != input { + col.keep(input.clone(), COST_CASE); + } else if d.contains(&capitalize(&input)) && word != capitalize(&input) { + col.keep(capitalize(&input), COST_CASE); + } + // 2. Accents and other MAP slips. + let map = d.map(); + let alts: Vec> = chars.iter().map(|&c| alternatives(c, map)).collect(); + let mut variants = Vec::new(); + map_variants(&chars, &alts, 0, 0, 0, &mut Vec::with_capacity(n), &mut variants); + variants.sort_by_key(|v| v.1); + for (v, cost) in &variants { + col.offer(v, *cost); + } + // 3. Common confusions. + for (from, to) in d.rep() { + if from.is_empty() { + continue; + } + for (i, _) in input.match_indices(from.as_str()) { + if let (Some(a), Some(b)) = (input.get(..i), input.get(i + from.len()..)) { + col.offer(&format!("{a}{to}{b}"), COST_REP); + } + } + } + // 4. A missing space: cheap after a short leading word or before a short trailing one, + // dearer between two longer words, never leaving a stray short half. + for (i, _) in input.char_indices().skip(1) { + let (Some(a), Some(b)) = (input.get(..i), input.get(i..)) else { continue }; + let long = |p: &str| p.chars().count() >= 3; + let cost = if (LEADING_WORDS.contains(&a) && long(b)) || (long(a) && TRAILING_WORDS.contains(&b)) { + COST_SPLIT + } else if long(a) && long(b) { + COST_SPLIT_RARE + } else { + continue; + }; + col.offer(&format!("{a} {b}"), cost); + } + // 5. One typo. + let mut letters: Vec = d.try_chars().chars().filter(|c| c.is_lowercase()).collect(); + if letters.is_empty() { + letters = "aeiousrntmdlcpvgqbzfhjxçãõáéíóúâêôkw".chars().collect(); + } + let s = |v: &[char]| v.iter().collect::(); + for i in 0..n.saturating_sub(1) { + if chars.get(i) != chars.get(i + 1) { + let mut v = chars.clone(); + v.swap(i, i + 1); + col.offer(&s(&v), COST_SWAP); + } + } + for i in 0..n { + let mut v = chars.clone(); + v.remove(i); + col.offer(&s(&v), COST_DROP_OR_ADD); + } + for i in 0..=n { + for &c in &letters { + let mut v = chars.clone(); + v.insert(i, c); + col.offer(&s(&v), COST_DROP_OR_ADD); + } + } + for i in 0..n { + for &c in &letters { + if chars.get(i) != Some(&c) { + let mut v = chars.clone(); + if let Some(x) = v.get_mut(i) { + *x = c; + } + col.offer(&s(&v), COST_MISTYPE); + } + } + } + // 6. Further off: scan the stems that start like the word. + if col.count_within(COST_MISTYPE) < max { + scan(&mut col, &chars); + } + let mut found: Vec<(String, (u32, usize))> = col.found.into_iter().collect(); + // Among equal costs, the change nearest the word's start is the likelier intent + // (`saúde` before `saudê`: both add an accent, but at the first difference). + let diff_at = |text: &str| chars.iter().zip(text.chars()).position(|(a, b)| *a != b).unwrap_or(n); + found.sort_by_key(|(text, (cost, seq))| (*cost, diff_at(text), *seq)); + let mut out: Vec = Vec::new(); + for (text, _) in found { + let t = match case { + Casing::Initial => capitalize(&text), + Casing::Upper if n > 1 => text.to_uppercase(), + _ => text, + }; + if t != word && !out.contains(&t) { + out.push(t); + } + if out.len() >= max { + break; + } + } + out +} + +/// Round 6: rank the stems starting with the word's first letter (or a MAP relative of it, either +/// case) by shared letter pairs, then keep the forms of the best within two typos. +fn scan(col: &mut Collector, chars: &[char]) { + let d = col.d; + let map = d.map(); + let Some(&first) = chars.first() else { return }; + let folded: Vec = chars.iter().map(|&c| fold(c, map)).collect(); + let pairs: HashSet<(char, char)> = std::iter::once(' ').chain(folded.iter().copied()).zip(folded.iter().copied()).collect(); + let mut firsts: Vec = std::iter::once(first).chain(alternatives(first, map)).collect(); + let upper: Vec = firsts.iter().flat_map(|c| c.to_uppercase()).collect(); + firsts.extend(upper); + firsts.sort_unstable(); + firsts.dedup(); + let n = folded.len(); + let mut best: Vec<(u32, &str)> = Vec::new(); + for f in firsts { + let prefix = f.to_string(); + for stem in d.stems_with_prefix(&prefix) { + let sf: Vec = stem.chars().map(|c| fold(c, map)).collect(); + let shared = std::iter::once(' ').chain(sf.iter().copied()).zip(sf.iter().copied()).filter(|p| pairs.contains(p)).count(); + if shared < 2 { + continue; + } + // Shared pairs, less the stem's letters the word doesn't explain, less a length gap + // beyond what an ending explains. + let extra = sf.len().saturating_sub(shared); + let gap = sf.len().abs_diff(n).saturating_sub(4); + let score = (shared * 4).saturating_sub(extra * 3 + gap * 2) as u32; + if score == 0 { + continue; + } + if best.len() < SCAN_STEMS { + best.push((score, stem)); + } else if let Some(min) = best.iter_mut().min_by_key(|b| b.0) + && score > min.0 + { + *min = (score, stem); + } + } + } + best.sort_by(|a, b| b.0.cmp(&a.0).then(a.1.cmp(b.1))); + let related = |a: char, b: char| fold(a, map) == fold(b, map); + let limit = 4; + for (_, stem) in best { + for form in d.forms(stem) { + let lf: Vec = form.to_lowercase().chars().collect(); + if lf.len().abs_diff(chars.len()) > 2 { + continue; + } + let dist = weighted_distance(&lf, chars, &related, limit); + if dist <= limit && lf.as_slice() != chars { + col.keep(form, COST_SCAN + dist * 3); + } + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn has(word: &str, want: &str) { + let s = suggest(word, 8); + assert!(s.iter().any(|x| x == want), "{word} → {s:?}, want {want}"); + } + + #[test] + fn portuguese_misspellings_get_the_right_suggestion() { + crate::pt::test_dict::install(); + has("coraçao", "coração"); + has("saude", "saúde"); + has("seje", "seja"); + has("excessão", "exceção"); + has("previlégio", "privilégio"); + has("prsidente", "presidente"); + has("provavel", "provável"); + has("trbalho", "trabalho"); + has("nãoé", "não é"); + has("paraela", "para ela"); + has("brasília", "Brasília"); + has("Coraçao", "Coração"); + } + + #[test] + fn the_likeliest_fix_comes_first() { + crate::pt::test_dict::install(); + assert_eq!(suggest("coraçao", 3).first().map(String::as_str), Some("coração")); + assert_eq!( + suggest("saude", 3).first().map(String::as_str), + Some("saúde"), + "an accent at the first difference beats one further right (saudê is a word too)" + ); + assert_eq!(suggest("seje", 3).first().map(String::as_str), Some("seja")); + } + + #[test] + fn edge_cases_stay_empty_or_bounded() { + crate::pt::test_dict::install(); + assert!(suggest("", 5).is_empty()); + assert!(suggest("bem", 0).is_empty()); + assert!(suggest(&"a".repeat(41), 5).is_empty()); + assert!(suggest("duas palavras", 5).is_empty()); + assert!(suggest_in(&crate::affix::AffixDict::default(), "coraçao", 5).is_empty()); + assert!(suggest("coração", 5).len() <= 5); + assert!(!suggest("coração", 5).contains(&"coração".to_string())); + } + + #[test] + fn distance_weighs_accents_half() { + let map = vec![vec!['a', 'á', 'à', 'â', 'ã'], vec!['c', 'ç']]; + let related = |a: char, b: char| fold(a, &map) == fold(b, &map); + let c = |s: &str| s.chars().collect::>(); + assert_eq!(weighted_distance(&c("coraçao"), &c("coração"), &related, 9), 1); + assert_eq!(weighted_distance(&c("coracao"), &c("coração"), &related, 9), 2, "ç and ã are two accent slips"); + assert_eq!(weighted_distance(&c("coro"), &c("coro"), &related, 9), 0); + assert_eq!(weighted_distance(&c("bem"), &c("bom"), &related, 9), 2); + assert_eq!(weighted_distance(&c("bem"), &c("bemte"), &related, 9), 4); + assert_eq!(weighted_distance(&c("bem"), &c("ver"), &related, 2), 3); + assert_eq!(weighted_distance(&c(""), &c("ab"), &related, 9), 4); + } + + #[test] + fn suggestions_are_fast_enough() { + crate::pt::test_dict::install(); + // Warm the dictionary, then time a worst case (a long word nothing near): the scan round. + let _ = suggest("bem", 1); + let t = std::time::Instant::now(); + for w in ["xqzvweportuguespalavra", "prsidentes", "paralelepípedo"] { + let _ = suggest(w, 6); + } + let ms = t.elapsed().as_secs_f64() * 1000.0; + eprintln!("3 suggestion calls: {ms:.0} ms"); + assert!(ms < 20_000.0, "{ms} ms"); + } +} diff --git a/crates/proof/src/pt/test_dict.rs b/crates/proof/src/pt/test_dict.rs new file mode 100644 index 0000000..e02be3c --- /dev/null +++ b/crates/proof/src/pt/test_dict.rs @@ -0,0 +1,121 @@ +//! A tiny synthetic Portuguese Hunspell dictionary for tests (written for these tests; not taken +//! from any real dictionary). Real dictionaries are installed on the system, never bundled. +//! +//! It exercises what VERO-like dictionaries use: `FLAG UTF-8` (multi-byte flag characters — +//! `F` marks a forbidden spelling), continuation classes (`provável` → `prováveis`), the +//! suggestion tables (`TRY`, `MAP` for accents, `REP` for common confusions) and one prefix. + +pub const AFF: &str = "SET UTF-8 +FLAG UTF-8 +FORBIDDENWORD F +TRY aeiousrntmdlcpvgqbzfhjxçãõáéíóúâêôkw +MAP 6 +MAP aáàâã +MAP eéê +MAP ií +MAP oóôõ +MAP uú +MAP cç +REP 4 +REP seje seja +REP excessão exceção +REP previlégio privilégio +REP analize análise +SFX A Y 2 +SFX A a ava a +SFX A a ando a +SFX B N 1 +SFX B 0 ável/C . +SFX C N 1 +SFX C vel veis vel +PFX X Y 1 +PFX X 0 des . +"; + +/// Stems (the count line first, as in Hunspell files). +pub const DIC: &str = "76 +coração +saudade +saudê +pássaro +saúde +árvore +escrever +computador +praia +conhecer +Brasília +São +Paulo +guarda +roupa +mail +marido +ex +vice +presidente +autoestima +frequência +frequencia/F +exceção +privilégio +análise +seja +trabalho +canta/A +prov/B +cobrir/X +avião +pássaros +amanhã +vocês +coragem +parabéns +está +ele +disse +existem +bem +não +sim +que +com +para +dos +das +pela +pelo +muito +mais +como +porque +aqui +também +já +ser +ter +ir +ver +dar +e +o +é +água +ela +da +do +de +em +no +na +um +uma +"; + +/// Make [`AFF`]/[`DIC`] the Portuguese dictionary (every test that needs one calls this; they all +/// install the same one, so tests running in parallel agree). +pub fn install() { + if let Err(e) = super::install_dictionary(AFF, DIC) { + panic!("test dictionary: {e}"); + } +} diff --git a/crates/ui-egui/src/i18n/mod.rs b/crates/ui-egui/src/i18n/mod.rs index 986fc6e..35b585a 100644 --- a/crates/ui-egui/src/i18n/mod.rs +++ b/crates/ui-egui/src/i18n/mod.rs @@ -85,7 +85,7 @@ pub static LANGUAGES: [LangInfo; 18] = [ name: "Português (Brasil)", source: include_str!("pt-br.tsv"), prefer_hans: false, - editing_lang: None, + editing_lang: Some("pt-BR"), catalog: OnceLock::new(), }, // Serbian, Cyrillic script (the default per BCP 47 when no script is given); `sr`, `sr-RS`, diff --git a/crates/ui-egui/src/i18n/tests.rs b/crates/ui-egui/src/i18n/tests.rs index 0f11fa9..db07c86 100644 --- a/crates/ui-egui/src/i18n/tests.rs +++ b/crates/ui-egui/src/i18n/tests.rs @@ -997,9 +997,9 @@ fn a_polish_interface_writes_new_documents_in_polish() { } #[test] -fn only_polish_has_an_editing_language() { +fn the_editing_languages_are_the_ones_wordcraft_can_proof() { let with: Vec<(&str, &str)> = Lang::all().filter_map(|l| l.editing_lang().map(|e| (l.code(), e))).collect(); - assert_eq!(with, [("pl", "pl-PL")]); + assert_eq!(with, [("pt-br", "pt-BR"), ("pl", "pl-PL")]); } #[test] diff --git a/crates/ui-egui/src/ribbon.rs b/crates/ui-egui/src/ribbon.rs index 967da0e..56dbf32 100644 --- a/crates/ui-egui/src/ribbon.rs +++ b/crates/ui-egui/src/ribbon.rs @@ -409,8 +409,8 @@ fn mi_text(ui: &mut Ui, app: &mut WordApp, text: &str, id: &str, params: Value) } /// The proofing languages the Language menu offers: tag and English name (translated when drawn). -pub(crate) const PROOFING_LANGUAGES: [(&str, &str); 3] = - [("en-US", "English (United States)"), ("en-GB", "English (United Kingdom)"), ("pl-PL", "Polish")]; +pub(crate) const PROOFING_LANGUAGES: [(&str, &str); 4] = + [("en-US", "English (United States)"), ("en-GB", "English (United Kingdom)"), ("pl-PL", "Polish"), ("pt-BR", "Portuguese (Brazil)")]; /// The English name of a proofing language tag, as the status bar shows it (no tag: WordCraft's /// default, US English; a tag without a name here: the tag). @@ -421,6 +421,7 @@ pub(crate) fn language_name(tag: Option<&str>) -> String { } match wordcraft_engine::proof::ProofLang::from_tag(Some(tag)) { wordcraft_engine::proof::ProofLang::Pl => "Polish".into(), + wordcraft_engine::proof::ProofLang::Pt => "Portuguese (Brazil)".into(), wordcraft_engine::proof::ProofLang::En if tag.eq_ignore_ascii_case("en") => "English (United States)".into(), _ => tag.to_string(), } @@ -441,7 +442,9 @@ pub(crate) fn language_menu(ui: &mut Ui, app: &mut WordApp) { let current = st.get("lang").and_then(Value::as_str).unwrap_or("").to_string(); let no_proof = st.get("noProof").and_then(Value::as_bool).unwrap_or(false); for (tag, name) in PROOFING_LANGUAGES { - let on = current.eq_ignore_ascii_case(tag) || (tag == "pl-PL" && current.eq_ignore_ascii_case("pl")); + let on = current.eq_ignore_ascii_case(tag) + || (tag == "pl-PL" && current.eq_ignore_ascii_case("pl")) + || (tag == "pt-BR" && current.eq_ignore_ascii_case("pt")); mi_check(ui, app, name, on, "review.language", json!({"lang": tag})); } ui.separator(); @@ -2435,6 +2438,14 @@ mod tests { } } + #[test] + fn portuguese_has_a_name_in_the_language_menu() { + assert!(PROOFING_LANGUAGES.iter().any(|(t, _)| *t == "pt-BR")); + assert_eq!(language_name(Some("pt-BR")), "Portuguese (Brazil)"); + assert_eq!(language_name(Some("pt")), "Portuguese (Brazil)"); + assert_eq!(language_name(Some("PT-BR".to_string().as_str())), "Portuguese (Brazil)"); + } + #[test] fn picture_format_tab_appears_when_picture_selected() { let mut s = Session::new(wordcraft_doc::Document::new()); diff --git a/docs/gaps.md b/docs/gaps.md index ff66ff8..49d0f27 100644 --- a/docs/gaps.md +++ b/docs/gaps.md @@ -20,7 +20,7 @@ Native save reliability: synthetic cases on macOS reproduced loss through the pr | 5 | **Content controls** landed (#381): `w:sdt` kept, shown and edited (Developer tab, Design Mode, Properties). Move tracking landed (#400): `w:moveFrom`/`w:moveTo` with their named ranges round-trip, a tracked cut and paste of a paragraph or sentence records a move, either end accepts/rejects both, green double strike/underline markup and "Moved from/to" balloons; still missing: drag-and-drop moves (no text drag yet) and move options in Track Changes Options. Formatting revisions landed (#41): `w:rPrChange`/`w:pPrChange`/`w:tblPrChange`/`w:trPrChange`/`w:tcPrChange`/`w:sectPrChange`/`w:numberingChange` round-trip, are recorded while tracking, accepted/rejected and shown in balloons; still missing: tracked style-definition changes and table/section changes made in WordCraft. Opaque custom XML backing stores now survive saves; binding evaluation remains absent. | B | Issues #41, #400; source grep | Legal and editorial review workflows; templates with form controls | 5–8 | [file-format-parity.md](file-format-parity.md) | | 6 | **No native printing** on desktop: File › Print opens the PDF in the system viewer, which prints (#286); File › Print has a preview, page range, odd/even, copies and collate (#641); printer choice, duplex and pages per sheet are still the system's; web prints through the browser (#209) | B | `ui.print` → `apps/wordcraft/src/print.rs`; issue #15 | Everyone who prints | 5–8 | [hardware-parity.md](hardware-parity.md) | | 7 | **Dialog depth**: 54 modal dialogs vs Word's ~100 | B | `Dialog` enum in `crates/ui-egui/src/dialogs.rs` | Power users reach for dialogs the ribbon alone doesn't cover | 11–19 | [ui-parity.md](ui-parity.md) | -| 8 | **Proofing covers English and Polish only**; no dictionaries, grammar or hyphenation for the other ~48 languages Word proofs (text in them is left unmarked) | | `crates/proof`: English and Polish, a language per run (2026-10-10); Word ships 120 proofing tools; issues #25, #40, #100 | Every writer outside English and Polish | 20–40 | [localization-parity.md](localization-parity.md) | +| 8 | **Proofing covers English, Polish and Portuguese only**; no dictionaries, grammar or hyphenation for the other ~47 languages Word proofs (text in them is left unmarked) | | `crates/proof`: English, Polish and Portuguese (pt-BR via the installed VERO dictionary, 2026-10-12), a language per run (2026-10-10); Word ships 120 proofing tools; issues #25, #40, #100 | Every writer outside English, Polish and Portuguese | 20–40 | [localization-parity.md](localization-parity.md) | | 9 | **Password-protected files** can't be opened or saved | | #55; `docbin` detects and refuses RC4/XOR | Corporate and legal users | 6–10 | [file-format-parity.md](file-format-parity.md) | | 10 | **Objects**: no group (#267 open), text in a freely rotated text box (free rotation and flips landed, #332; text direction, alignment and linked text boxes landed, #369; contour wrap and wrap side landed, #354), shape effects, WordArt 3-D effects and connector routing around shapes (WordArt effects and warps and glued connectors landed, #416), dragging members inside a Drawing Canvas (the canvas landed, #344) | | [parity-checklist.md](parity-checklist.md) (Shape Format 6/6, Layout › Group) | Newsletters, flyers, reports with diagrams | 20–30 | [target-app-parity.md](target-app-parity.md) | | 11 | **Interface languages**: 7 of the 12 key languages missing (hi, ar, fr, id, ko, vi, pt-PT); no mirrored RTL interface; ~5% of strings still English in the translated catalogs | | [localization-parity.md](localization-parity.md) | Most of the world's writers | 50–80 | [localization-parity.md](localization-parity.md) | @@ -86,6 +86,7 @@ The same gaps grouped the way the parity documents are, for agents working in on | 2026-10-11 | trivial | Insert › Chart landed (#409); gap #13 narrowed to editing Word's charts, more types and SmartArt, 35–55 h | | 2026-10-11 | trivial | Line Numbers, Hyphenation, Language, Envelopes and Labels dialogs landed (#407); gap #7 at 30 dialogs, 16–24 h; duplicated #7 rows merged | | 2026-10-11 | trivial | Caption, Index, Mark Index Entry, Custom Table of Contents, Source Manager and Create/Edit Source dialogs landed (#402); removed from the dialog-depth list | +| 2026-10-12 | minor | #8: Portuguese proofing (spelling, suggestions, grammar, hyphenation) landed; the gap is now the other languages (20–40 h) | | 2026-10-11 | trivial | Document grid landed (#391); removed from the East Asian gap | | 2026-10-11 | trivial | Move tracking landed (#400); gap #5 narrowed to content controls, 5–8 h | | 2026-10-11 | trivial | #4: note options per section (below text, end of section, restart each section/page) and the Footnote and Endnote dialog (#385) | diff --git a/docs/localization-parity.md b/docs/localization-parity.md index 46603f7..ab4d9f4 100644 --- a/docs/localization-parity.md +++ b/docs/localization-parity.md @@ -1,11 +1,11 @@ # Localization parity with Microsoft Word -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-11 · **Change:** minor (Turkish interface catalog completed, 2,325 entries, #774; previously minor: font list shows fonts by their names in the interface language, #505; previously minor: Turkish interface added; previously major: first version; catalogs measured from `crates/ui-egui/src/i18n`, Word's languages from its bundle listing) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-12 · **Change:** minor (Portuguese proofing: pt-BR spelling, suggestions, grammar and hyphenation from the installed VERO dictionary and TeX patterns; a pt-br interface now writes pt-BR documents; previously minor: Polish interface; Polish proofing — spelling, grammar, hyphenation — and a proofing language per run; previously minor: Turkish interface added; previously major: first version; catalogs measured from `crates/ui-egui/src/i18n`, Word's languages from its bundle listing) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 Interface languages, script support and proofing per language. How translation works (catalog format, adding a language, clean-room rule): `crates/ui-egui/src/i18n/mod.rs`. -**Dimension: ~27% (estimated; 25% before Polish and per-run proofing languages), 75–120 h to full +**Dimension: ~28% (estimated; 27% before Portuguese proofing, 25% before Polish and per-run proofing languages), 70–115 h to full for the twelve key languages** plus ~45 h for the other ~17 languages Word ships, plus native-speaker review (human). @@ -29,7 +29,7 @@ native-speaker review (human). Read Aloud, Backstage greetings) aren't wrapped in `tl!` yet, so they show in English in every language: the true share of visible strings is **~93–95%** (estimated). - Pending PRs: Norwegian Bokmål (#208) and Nynorsk (#210), Hebrew with RTL (#20). -- **Proofing: English and Polish**, chosen per run from its language (`w:lang`, resolved through +- **Proofing: English, Polish and Portuguese**, chosen per run from its language (`w:lang`, resolved through the styles; Review › Language and the status bar set it). Languages without proofing tools get no spelling or grammar marks (as in Word without the language's tools) instead of being flagged as English. Detail in [Proofing per language](#proofing-per-language). @@ -79,6 +79,7 @@ documents in Polish (`pl-PL`). |---|---|---|---|---| | English (`en`, any region; text with no language) | Moby word list (public domain, ~160k words) with inflection rules | edit distance | repeated word, `a`/`an`, sentence capital, space before punctuation, double space | Moby dictionary, our Liang patterns, heuristic | | Polish (`pl`) | The Hunspell `pl_PL` dictionary installed on the system (OS package, Homebrew, LibreOffice, or the `WORDCRAFT_DICTIONARIES` folder; not bundled, so no Polish spelling marks without it), read at runtime by a Hunspell-style affix engine (`crates/proof/src/affix.rs`); names capitalized, compounds part by part, `Kennedy'ego` | missing diacritics first (`zolw` → `żółw`), the dictionary's confusion table (`żeka` → `rzeka`, `chuśtawka` → `huśtawka`), missing space (`napewno` → `na pewno`), one typo, then a stem scan within two typos | Polish messages: repeated word, space before punctuation, double space, sentence capital (knows `np.`, `r.`, initials, ordinals, dialogue dashes), missing comma before `że`/`który`/`ponieważ`/`gdyż`/`aby`/`żeby`/`jeśli`/`jeżeli`/`gdyby`/`lecz` (not after `mimo`, `chyba`, `tak`, prepositions…), `we`/`ze` (`we wtorek`, `ze szkoły`), doubled punctuation | TeX Polish patterns (Kołodziejska, Jackowski, Ryćko; MIT option) with their 20 exception words; a Polish syllable heuristic for letters outside the alphabet | +| Portuguese (`pt`, any region) | The Hunspell `pt_BR` dictionary installed on the system — VERO and its derivatives (same places as Polish; not bundled, LGPL/MPL), read at runtime by the same affix engine, which reads its `FLAG UTF-8`, 106 flags, continuation classes and `FORBIDDENWORD` spellings; names capitalized, compounds part by part (`guarda-roupa`, `ex-marido`) | missing accents first (`coraçao` → `coração`, `saude` → `saúde`), the dictionary's confusion table (`seje` → `seja`, `excessão` → `exceção`), missing space (`nãoé` → `não é`), one typo, then an accent-folding stem scan | Portuguese messages: repeated word (the emphatic doubles are allowed), space before punctuation, double space, sentence capital (knows `etc.`, `pág.`, `Dr.`, initials, ordinals, dialogue dashes), doubled punctuation; no comma rule — `que` opens a clause only sometimes | TeX Portuguese patterns (Rezende, Almeida, Araujo, Benevides; BSD 3-clause) with their 2 exception words | | Any other | — (no marks) | — | — | the English patterns, as before | Measured with the SJP.PL dictionary: every one of the ~4.5 million forms an independent expansion of its affix file @@ -107,6 +108,7 @@ allows); all other text is written from the meaning of the English. No Italian p |---|---|---| | 2026-10-11 | minor | Turkish interface catalog completed (2,325 entries, the same keys as Russian): equation gallery, built-in style names, watermarks, paper sizes and preview tiles; the interface audit checks Turkish too (#774) | | 2026-10-11 | minor | Czech (`cs`) interface catalog added | +| 2026-10-12 | minor | Portuguese proofing: pt-BR spelling from the installed VERO dictionary (the affix engine grew `FLAG UTF-8`, 128 flag bits, continuation classes and `FORBIDDENWORD`), accent-first suggestions, conservative grammar with Portuguese messages, TeX Portuguese hyphenation patterns (BSD 3-clause); Review › Language offers Português (Brasil); a pt-br interface writes pt-BR documents; dimension ~27% → ~28% | | 2026-10-11 | minor | Polish interface (13 languages) and Polish proofing: SJP.PL spelling with suggestions, Polish grammar rules, TeX Polish hyphenation; proofing language per run; dimension ~25% → ~27% | | 2026-10-11 | minor | German interface catalog (#574): 2,021 entries covering the full zh-hans key set; locale, saved-preference and bundled-font tests; no German proofing | | 2026-10-11 | minor | Italian interface catalog (#594), locale selection, saved preference and font coverage checks; count labels put the noun first so they read right at 1 | diff --git a/docs/roadmap.md b/docs/roadmap.md index ef1a7a7..eb3810e 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,6 +1,6 @@ # WordCraft roadmap: milestones and what's next -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-11 · **Change:** minor (Polish interface and Polish proofing; previously trivial: M7: formatting revisions landed, #41; previously trivial: M5: Draw Table and Eraser landed, #303; previously minor: beta distance restated: ~15 points, ~150–230 h; previously: alpha gate checked: six core workflows pass; previously major: milestones moved here from ROADMAP.md; Current focus set from the re-measured gaps) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-12 · **Change:** minor (Portuguese proofing; previously minor: Polish interface and Polish proofing; previously trivial: M7: formatting revisions landed, #41; previously trivial: M5: Draw Table and Eraser landed, #303; previously minor: beta distance restated: ~15 points, ~150–230 h; previously: alpha gate checked: six core workflows pass; previously major: milestones moved here from ROADMAP.md; Current focus set from the re-measured gaps) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 Forward-looking plan. The summary is [`ROADMAP.md`](../ROADMAP.md); the ranked work list is [`gaps.md`](gaps.md); the numbers are in [`target-app-parity.md`](target-app-parity.md). Hours are @@ -61,7 +61,7 @@ partial rows are fidelity gaps (beta work), not broken workflows. WordCraft stay | M12 | Formats breadth (PDF, ODT, RTF, HTML, MD, TXT, LaTeX, `.doc` import) | done (first versions); `.doc` write, encryption, minor formats missing | 60–100 h | | M13 | Performance budgets | on track (1.4 ms relayout); large real-world documents unmeasured | 10–20 h | | M14 | 1.0 polish, packaging, signing | releases v0.1.0–v0.4.0 published (signed macOS, Windows MSI x64/x86/arm64, Linux, FreeBSD, web) | — | -| M15 | Localization (12 key languages, RTL interface, proofing languages) | 15 UI languages; English and Polish proofing, a proofing language per run | 75–120 h | +| M15 | Localization (12 key languages, RTL interface, proofing languages) | 15 UI languages; English, Polish and Portuguese proofing, a proofing language per run | 70–115 h | | M16 | Right-to-left and East Asian typography | RTL paragraphs done (#207) | 30–45 h | ## Next after beta @@ -73,6 +73,7 @@ languages, screen-reader access, ecosystem decisions (VBA, add-ins, cloud), an i | Date | Change | Summary | |---|---|---| +| 2026-10-12 | minor | M15: Portuguese proofing (pt-BR spelling from the installed VERO dictionary, suggestions, grammar, hyphenation; Review › Language gains Português (Brasil)) | | 2026-10-11 | trivial | M15: Polish interface and Polish proofing landed | | 2026-10-11 | trivial | M3: icons landed (#309) | | 2026-10-11 | trivial | M11: contour (Tight/Through) wrap done (#354) | diff --git a/docs/target-app-parity.md b/docs/target-app-parity.md index a82ba7f..a4d8afc 100644 --- a/docs/target-app-parity.md +++ b/docs/target-app-parity.md @@ -1,6 +1,6 @@ # WordCraft parity with Microsoft Word -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-11 · **Change:** minor (Polish interface and Polish proofing; previously trivial: English spelling knows common modern words, #520; previously trivial: all 12 bibliography styles, #383; previously trivial: formatting revisions landed, #41; previously trivial: Draw Table and Eraser landed, #303; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; replaces the parity tables that lived in ROADMAP.md) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-12 · **Change:** minor (Portuguese proofing; previously trivial: English spelling knows common modern words, #520; previously trivial: all 12 bibliography styles, #383; previously trivial: formatting revisions landed, #41; previously trivial: Draw Table and Eraser landed, #303; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; replaces the parity tables that lived in ROADMAP.md) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 The authoritative assessment of how close WordCraft is to Microsoft Word. [`ROADMAP.md`](../ROADMAP.md) summarizes it; [`gaps.md`](gaps.md) itemizes every shortfall; the deep checklists are @@ -66,7 +66,7 @@ full parity for the area. | Footnotes and endnotes | 3 | 60% | 60% | 3–5 | Placed and editable, numbers from Word files (#103); long notes continue onto the next page under the continuation separator (#352). Footnote and Endnote dialog with options per section: below text, end of section, number format, start at, restart each section or page, custom mark, convert (#385). | | References (TOC, citations, bibliography, captions, index, TOA) | 5 | 65% | 60% | 12–20 | Working first versions, Zotero integration (#189), Zotero/Mendeley `ADDIN` fields round-trip. Caption, Index, Mark Entry, Custom TOC and Create/Edit Source dialogs; updates honour the TOC (`\o \u \t \f \n \p \h`), INDEX (`\c \e \h \r`), XE (`\b \i \t \r`) and SEQ/STYLEREF chapter switches (#402). All 12 of Word's bibliography styles (APA, MLA, Chicago, IEEE, Harvard, ISO 690 author-date and numerical, Turabian, GB/T 7714, GOST name and title sort, SIST02; #383); bibliography sources not yet in DOCX (#169 open). | | Review (proofing UI, comments, track changes, compare, protect) | 8 | 72% | 75% | 15–25 | Comment balloons, tracked insert/delete incl. paragraph marks (#244, #125), Compare / Combine with comparison settings (case, white space, formatting, tables, word or character level, #412), Restrict Editing pane with formatting and editing restrictions, exceptions for everyone and a password (#412). Formatting revisions (#41): kept in DOCX, recorded while tracking, accepted/rejected, "Formatted: …" balloons and change bars. Moves (#400): kept in DOCX, recorded on a tracked cut and paste, accepted/rejected at either end, green markup and "Moved from/to" balloons. Show revisions in balloons takes tracked deletions in the body out of the text into "Deleted: …" balloons (insertions stay inline, as in Word; deletions in headers, footers, notes and text boxes, and in print and PDF, stay inline). Missing: drag-and-drop moves, insert/delete balloons, Translate, Block Authors, modern comment threads/mentions. Lowered: these gaps are confirmed by source (no `rPrChange` anywhere). | -| Proofing (spelling, grammar, thesaurus, languages) | 5 | 40% | (in Review) | 20–40 | English and Polish, chosen per run from its language (`w:lang`): English word list (Moby plus our supplement of modern words such as "desktop", #520) and rule-based grammar; Polish against the installed Hunspell dictionary through a Hunspell-style affix engine, Polish grammar rules and TeX hyphenation patterns (2026-10-10). Other languages are left unmarked. The custom dictionary persists across sessions (#40). Word ships 120 proofing tools for ~50 languages plus Editor (style refinements, similarity). Issues #25, #40, #100. | +| Proofing (spelling, grammar, thesaurus, languages) | 5 | 42% | (in Review) | 20–40 | English, Polish and Portuguese, chosen per run from its language (`w:lang`): English word list (Moby plus our supplement of modern words such as "desktop", #520) and rule-based grammar; Polish against the installed Hunspell dictionary through a Hunspell-style affix engine, Polish grammar rules and TeX hyphenation patterns (2026-10-10); Portuguese (pt-BR) against the installed VERO Hunspell dictionary — the affix engine reads its `FLAG UTF-8` flags, continuation classes and forbidden spellings — with accent-first suggestions, conservative grammar rules and TeX patterns (2026-10-12). Other languages are left unmarked. The custom dictionary persists across sessions (#40). Word ships 120 proofing tools for ~50 languages plus Editor (style refinements, similarity). Issues #25, #40, #100. | | Mailings | 2 | 70% | 70% | 6–10 | Mail merge from CSV, fields, rules (IF, SKIPIF, NEXT, NEXTIF, evaluated alike in preview and merge), fields in headers/footers (one section per letter), preview, envelopes, labels. Select Recipients takes a typed list or CSV (#247); no Excel/Outlook/Contacts data sources, no email merge. | | Pictures, shapes, text boxes, WordArt | 6 | 60% | 60% | 20–30 | Insert, crop, recolour, styles, floating placement like Word (#136), wrap (square/top-bottom/behind/front), text boxes edit and overflow (#46), VML text boxes (#242), free rotation and flips with a rotation handle (#332), WordArt: text fill (solid, gradient, none), outline, shadow, glow and reflection with our own styles, and Transform warps along arch, circle, wave, slant, inflate/deflate, chevron and triangle (other presets kept and drawn flat), connectors (straight, elbow, curved, arrowheads) glued to shapes' connection sites and re-routed when those move (#416). Missing: tight/through contour wrap, rotated text-box text, group (#267 open), shape effects, WordArt 3-D/bevel and editing text in its bent form, connector routing around shapes, Drawing Canvas, Icons, online pictures, screenshot. | | Mailings | 2 | 70% | 70% | 6–10 | Mail merge from CSV, fields, rules (IF, SKIPIF, NEXT, NEXTIF, evaluated alike in preview and merge), fields in headers/footers (one section per letter), preview, envelopes, labels. Select Recipients takes a typed list, CSV (#247) or a sheet of an Excel (.xlsx/.xlsm) or OpenDocument (.ods) workbook, with Select Table to pick the sheet (#334); no Outlook/Contacts/Access data sources, no email merge. | @@ -251,6 +251,7 @@ can be used without committing Word output. | Date | Change | Summary | |---|---|---| +| 2026-10-12 | minor | Portuguese proofing (pt-BR spelling from the installed VERO dictionary, accent-first suggestions, grammar, hyphenation; Review › Language gains Português (Brasil)): Proofing 40% → 42%; other dimensions unchanged at this precision | | 2026-10-12 | trivial | Proofing: custom dictionary persists across sessions (#40); no percentage change | | 2026-10-11 | trivial | Proofing: runs in a language without a dictionary are skipped instead of flagged against the English dictionary (#100); no percentage change | | 2026-10-11 | minor | Polish interface (13 UI languages) and Polish proofing with per-run languages: Proofing 35% → 40% (25–45 h → 20–40 h), Localization 25% → 27%; feature depth and readiness unchanged at this precision |