Runs in Bulgarian (`w:lang` bg, bg-BG) are hyphenated by the Bulgarian syllable rules
(crates/proof/src/bg.rs, clean-room from the spelling rules; no pattern file or word list):
vowel-vowel splits, one consonant goes with the next syllable, a consonant cluster splits after
its first consonant except that an obstruent before р/л starts the next syllable (до-бре,
та-бли-ца), й stays with its vowel, дж/дз and ьо are never split.
- ProofLang::Bg (bg, bul): hyphenation only. It isn't "proofed", so like Other there are no
spelling or grammar marks until there is a Bulgarian dictionary.
- Latin-script words in a Bulgarian run are hyphenated as English, as in any other language.
- The layout already takes each word's language from its run (ProofRun), so nothing changes there.
Tests: the rules (срич-ко-пре-на-ся-не, та-бли-ца, шо-фьор, хо-джа, строй-ка), tag resolution,
hyphenate_word_in with the paragraph limits (Бъл-га-рия), and a laid-out bg-BG paragraph whose
hyphenated lines all break at Bulgarian points.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M15: Polish interface, and Polish spelling, grammar and hyphenation
Interface
- Polish is the tenth interface language ("Polski", pl.tsv, 1,045 entries);
pl, pl-PL and pl_PL.UTF-8 system locales select it automatically.
- Right-click menu labels now go through translation in every language.
- Review > Language is a menu (English US/UK, Polski, Do not check spelling
or grammar); the status bar shows the language at the caret.
- With a Polish interface, new blank documents and opened .txt/.md files
default to pl-PL.
Proofing
- Each run is checked in its own language (w:lang, resolved through
styles): squiggles, F7, review.issues, suggestions and hyphenation.
Text in languages without proofing tools is no longer flagged as English.
- Polish spelling from the SJP.PL Hunspell dictionary (Apache-2.0 option),
read by a new Rust affix engine (crates/proof/src/affix.rs); 875 KB asset,
~4.5M word forms. Regenerate with the pl_dictionary example.
- Polish suggestions: case, missing diacritics, rz/ż, ó/u, ch/h confusions,
missing spaces (napewno -> na pewno), edit distance.
- Conservative Polish grammar rules: repeated words, spacing, doubled
punctuation, sentence capitals, missing comma before że/który/ponieważ/
aby/żeby/jeśli…, we/ze forms.
- Polish hyphenation with the hyph-utf8 hyph-pl.tex patterns (MIT option);
pattern keys widened to 20 letters, TeX exceptions parsed; the heuristic
no longer treats ł, ż, ś… as vowels.
review.language validates and normalizes tags, can set noProof alone and
the document default; format.state reports lang and noProof.
Tests for every part, docs and ATTRIBUTION.md/NOTICE updated;
cargo xtask ci passes (fmt, clippy, test, assets, layers, wasm).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout: rustfmt the test module order left unformatted by #391
cargo fmt --check failed on main after #391 added mod tests_grid.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Review fixes for #346: read Polish spelling from the installed Hunspell dictionary
- Polish spelling uses the Hunspell pl_PL dictionary installed on the
system (OS packages, Homebrew, LibreOffice, ~/.local/share/hunspell, or
WORDCRAFT_DICTIONARIES) instead of a bundled 870 KB SJP.PL file;
without one, Polish text gets no spelling marks, like Word without the
language's proofing tools. .aff/.dic are size-capped and decoded from
UTF-8 or ISO 8859-2 (crates/proof/src/hunspell.rs).
- The dictionary can be replaced at runtime (install_dictionary) or
re-searched (reload_dictionary); an empty one accepts every word.
- Tests use small synthetic dictionaries written for them, so they don't
depend on what the machine has installed.
- The bundled pl.dic, its ATTRIBUTION/NOTICE entries and the generator
example are removed; docs describe the installed dictionary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* #100: affix — accept FLAG UTF-8 and DEFAULT flag modes
Per-character flag modes are what the reader already parses; only the
multi-character encodings (long, num) stay errors. VERO pt_BR declares
FLAG UTF-8.
* #100: affix — widen flag masks to u128 (128 flag capacity)
VERO pt_BR needs 106 distinct flags; the u64 bitmask capped at 64.
Serialization is unaffected: WCAFF1 stores flag characters and
re-derives the masks on load.
* #100: affix — support FORBIDDENWORD
VERO pt_BR marks spellings it must reject with a forbidden flag (ý).
Stems carrying it are listed but never correct and never inflect.
* #100: affix — continuation classes in check
SFX A 0 able/B: the form derived by A carries flag B, licensing a
further suffix of class B (workable -> workables) while the bare stem
cannot take one. form_has_flag walks the chain depth-bounded
(MAX_CHAIN = 3); with no continuations declared it degenerates to the
stem lookup, so existing dictionaries are unaffected. VERO pt_BR uses
26 continuation rules.
* #100: affix — continuation classes in forms
Enumerate chained suffix forms (workable -> workables) under the same
MAX_CHAIN bound the checker walks, so every form the checker accepts
is listed and vice versa.
* #100: affix — round-trip continuations and FORBIDDENWORD through WCAFF1
The compact format keeps its WCAFF1 magic: continuation classes ride
on the rule's appended field (as in the .aff source) and the forbidden
flag becomes a FORBIDDENWORD line. Old files read as before.
* #100: affix — strip a UTF-8 BOM from .aff/.dic input
VERO's files open with a BOM; glued to the count line it parsed as a
stem (one garbage entry per load) and to SET it hid the directive.
* #782: affix — the module doc still rejected what the code now reads
Continuation classes and FORBIDDENWORD are supported; the header said
Rejected for both.
* #100: pt — Portuguese spelling, following the Polish module
ProofLang::Pt resolves pt/pt-BR/pt-PT/por tags; the pt module reads the
installed Hunspell pt_BR dictionary (VERO) through the affix engine —
FLAG UTF-8 flags, continuation classes and FORBIDDENWORD spellings all
exercised by the synthetic test dictionary. Hyphenated compounds are
checked part by part (guarda-roupa, ex-marido); without a dictionary
installed, Portuguese text gets no marks. Spelling message in
Portuguese; suggestions and grammar land next.
* #100: pt — Portuguese suggestions
Rounds of rising cost, driven by the dictionary's own tables: missing
accents from MAP (coraçao -> coração, saude -> saúde), the REP
confusions (seje -> seja, excessão -> exceção), a missing space after a
short function word (nãoé -> não é, dagua -> da água), one typo over
TRY, and an accent-folding stem scan — the Polish rounds, retuned for
Portuguese.
* #100: pt — Portuguese grammar flags
Conservative rules with Portuguese messages: repeated words (except
the emphatic doubles Portuguese allows), space before punctuation and
double spaces, sentence capitals knowing the Portuguese abbreviations
and dialogue dashes, and doubled punctuation. No comma rule: que opens
a clause only sometimes, so a wrong flag would cost more than a missed
one.
* #100: pt — Portuguese hyphenation
hyph-utf8's hyph-pt.tex bundled unmodified (BSD 3-clause, ATTRIBUTION
row with source and hash): 426 Liang patterns and 2 exception words;
falls back to the generic heuristic outside their reach.
* #100: pt — Review › Language offers Português (Brasil), and a pt-br interface writes pt-BR
The menu's fourth entry sets the run language; the status bar names
Portuguese; editing_lang is now allowed for pt-br (wordcraft-proof
proofs it), so new documents under a Portuguese interface are
Portuguese by default.
* #100: pt — engine test: Portuguese proofing through the review commands
review.language pt-BR normalizes the tag, review.issues reports
Portuguese spelling and grammar with the lang field, F7 and Change
apply a suggestion, Add to Dictionary works for Portuguese.
* #100: pt — Portuguese in the mixed-language proofing tests and docs
check_text proofs a Portuguese span with the Portuguese message; the
never-panic loops and the module docs cover the third language.
* #100: docs — Portuguese proofing in the gap, parity and roadmap docs
Gap 8 narrowed (English, Polish and Portuguese), localization parity
gains a Portuguese row and a revision entry, target-app-parity's
Proofing row moves 40% -> 42%, and the ROADMAP progress log records
what landed.
* #100: hunspell — LibreOffice extension folders follow the language
The search paths hardcoded LibreOffice's dict-pl; a pt_BR user whose
only dictionary is LibreOffice's dict-pt-BR extension — on Windows the
only realistic on-disk source — got silently no spelling. Both the
primary-language folder and the dashed full name are now tried, so
Polish keeps dict-pl and Portuguese finds dict-pt-BR.
* #100: pt — review fixes
Suggestion ties break toward the change nearest the word's start
(saúde before saudê, proven against a fixture that has both); the
Language menu's label is the English source "Portuguese (Brazil)" with
translations in all twelve catalogs (as Polish is); the attribution
counts pt.pat's 2 exception words; the pt fixture drops a duplicate
stem; docs/roadmap.md catches up with the landing; the throwaway spike
is gone.
* Review fixes for #782: two-suffix chain limit, module doc
- MAX_CHAIN = 2 counts suffixes in a row, matching Hunspell's two-suffix limit
(the old bound of 3 allowed up to four); test pins that a third is rejected.
- Module doc: say probes allocate small scratch strings instead of claiming none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Review fixes for #797: NOTICE for pt.pat, no indexing in the LibreOffice folder helper
- NOTICE reproduces hyph-pt.tex's BSD 3-Clause copyright notice, conditions and
disclaimer, which the licence requires for binary redistributions.
- hunspell::lo_ext_dirs builds its list without indexing (never-crash rule).
- Catalogs: main already translates "Portuguese (Brazil)" in every language, so the
duplicate rows the branch added are dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: witkowski01 <konradwitkowski01@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M15: Polish interface, and Polish spelling, grammar and hyphenation
Interface
- Polish is the tenth interface language ("Polski", pl.tsv, 1,045 entries);
pl, pl-PL and pl_PL.UTF-8 system locales select it automatically.
- Right-click menu labels now go through translation in every language.
- Review > Language is a menu (English US/UK, Polski, Do not check spelling
or grammar); the status bar shows the language at the caret.
- With a Polish interface, new blank documents and opened .txt/.md files
default to pl-PL.
Proofing
- Each run is checked in its own language (w:lang, resolved through
styles): squiggles, F7, review.issues, suggestions and hyphenation.
Text in languages without proofing tools is no longer flagged as English.
- Polish spelling from the SJP.PL Hunspell dictionary (Apache-2.0 option),
read by a new Rust affix engine (crates/proof/src/affix.rs); 875 KB asset,
~4.5M word forms. Regenerate with the pl_dictionary example.
- Polish suggestions: case, missing diacritics, rz/ż, ó/u, ch/h confusions,
missing spaces (napewno -> na pewno), edit distance.
- Conservative Polish grammar rules: repeated words, spacing, doubled
punctuation, sentence capitals, missing comma before że/który/ponieważ/
aby/żeby/jeśli…, we/ze forms.
- Polish hyphenation with the hyph-utf8 hyph-pl.tex patterns (MIT option);
pattern keys widened to 20 letters, TeX exceptions parsed; the heuristic
no longer treats ł, ż, ś… as vowels.
review.language validates and normalizes tags, can set noProof alone and
the document default; format.state reports lang and noProof.
Tests for every part, docs and ATTRIBUTION.md/NOTICE updated;
cargo xtask ci passes (fmt, clippy, test, assets, layers, wasm).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout: rustfmt the test module order left unformatted by #391
cargo fmt --check failed on main after #391 added mod tests_grid.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Review fixes for #346: read Polish spelling from the installed Hunspell dictionary
- Polish spelling uses the Hunspell pl_PL dictionary installed on the
system (OS packages, Homebrew, LibreOffice, ~/.local/share/hunspell, or
WORDCRAFT_DICTIONARIES) instead of a bundled 870 KB SJP.PL file;
without one, Polish text gets no spelling marks, like Word without the
language's proofing tools. .aff/.dic are size-capped and decoded from
UTF-8 or ISO 8859-2 (crates/proof/src/hunspell.rs).
- The dictionary can be replaced at runtime (install_dictionary) or
re-searched (reload_dictionary); an empty one accepts every word.
- Tests use small synthetic dictionaries written for them, so they don't
depend on what the machine has installed.
- The bundled pl.dic, its ATTRIBUTION/NOTICE entries and the generator
example are removed; docs describe the installed dictionary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* #100: affix — accept FLAG UTF-8 and DEFAULT flag modes
Per-character flag modes are what the reader already parses; only the
multi-character encodings (long, num) stay errors. VERO pt_BR declares
FLAG UTF-8.
* #100: affix — widen flag masks to u128 (128 flag capacity)
VERO pt_BR needs 106 distinct flags; the u64 bitmask capped at 64.
Serialization is unaffected: WCAFF1 stores flag characters and
re-derives the masks on load.
* #100: affix — support FORBIDDENWORD
VERO pt_BR marks spellings it must reject with a forbidden flag (ý).
Stems carrying it are listed but never correct and never inflect.
* #100: affix — continuation classes in check
SFX A 0 able/B: the form derived by A carries flag B, licensing a
further suffix of class B (workable -> workables) while the bare stem
cannot take one. form_has_flag walks the chain depth-bounded
(MAX_CHAIN = 3); with no continuations declared it degenerates to the
stem lookup, so existing dictionaries are unaffected. VERO pt_BR uses
26 continuation rules.
* #100: affix — continuation classes in forms
Enumerate chained suffix forms (workable -> workables) under the same
MAX_CHAIN bound the checker walks, so every form the checker accepts
is listed and vice versa.
* #100: affix — round-trip continuations and FORBIDDENWORD through WCAFF1
The compact format keeps its WCAFF1 magic: continuation classes ride
on the rule's appended field (as in the .aff source) and the forbidden
flag becomes a FORBIDDENWORD line. Old files read as before.
* #100: affix — strip a UTF-8 BOM from .aff/.dic input
VERO's files open with a BOM; glued to the count line it parsed as a
stem (one garbage entry per load) and to SET it hid the directive.
* #782: affix — the module doc still rejected what the code now reads
Continuation classes and FORBIDDENWORD are supported; the header said
Rejected for both.
* Review fixes for #782: two-suffix chain limit, module doc
- MAX_CHAIN = 2 counts suffixes in a row, matching Hunspell's two-suffix limit
(the old bound of 3 allowed up to four); test pins that a third is rejected.
- Module doc: say probes allocate small scratch strings instead of claiming none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: witkowski01 <konradwitkowski01@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* Persist the custom dictionary across sessions (#40)
* Review fixes for #796: session-only Ignore All, desktop-only persistence, capped atomic I/O
- review.ignoreAll no longer adds to (and saves) the custom dictionary: ignored words go to a
separate session-only list, as Ignore All does in Word.
- Only the desktop app wires the dictionary file. Headless MCP and `wordcraft-cli run` keep
words in memory, so an agent never reads or changes the person's dictionary (and nothing
writes outside an automation write root, #769).
- Loading reads through `take(1 MB)`, so a huge file is never read whole; lines that aren't a
single word (spaces, control characters, invalid UTF-8, over 64 characters) are skipped.
- review.addToDictionary applies the same word rule (proof::custom_word), so a word added is
never dropped on the next load.
- Saving merges the words on disk first (another process's additions survive; words just
removed stay out), creates the folder, and replaces the file atomically (temp + rename).
The file I/O lives in the proof crate, so the engine doesn't touch std::fs itself.
- Tests for each.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The five tests that assert linear layout/export/hyphenation by comparing
the cost of an n-sized and a larger input (layout: unequal formats, hidden
separators with auto hyphenation, hidden float anchors; pdf: fragmented
hidden headings; proof: hyphenating a long word) timed both with wall
clock, fastest of three. On a loaded machine (CI, shared dev boxes, the
test harness running other tests in parallel) a preempted big run, timer
resolution on the small run, or a tight 4x/8.0 limit (hyphenation measured
~7.2 when linear) made them fail now and then.
Each now uses a shared cost_ratio helper (test-only, no production change):
- CPU time of the test thread from /proc/thread-self/schedstat on Linux, so
time spent waiting for a CPU is left out (wall clock elsewhere);
- the small input runs `scale` times per timing so both timings are long
against the clock's resolution;
- small and big runs alternate, five rounds after a warm-up, cheapest kept;
- 8x the input everywhere: linear ~8x, quadratic ~64x, limit 24x.
Measured under load (load avg ~15-35 on 32 cores): 60 linear ratios in
7.7-8.8. With the old quadratic code injected locally each test fails:
unequal formats (uncapped style bucket scan) ~90-104x, hidden separators
(hyphenator rescan per point) ~36x, float anchors (run walk restarted per
anchor) ~188x, PDF headings (left-out range scan per char) ~37x,
hyphenation 50-65x.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* M15: Polish interface, and Polish spelling, grammar and hyphenation
Interface
- Polish is the tenth interface language ("Polski", pl.tsv, 1,045 entries);
pl, pl-PL and pl_PL.UTF-8 system locales select it automatically.
- Right-click menu labels now go through translation in every language.
- Review > Language is a menu (English US/UK, Polski, Do not check spelling
or grammar); the status bar shows the language at the caret.
- With a Polish interface, new blank documents and opened .txt/.md files
default to pl-PL.
Proofing
- Each run is checked in its own language (w:lang, resolved through
styles): squiggles, F7, review.issues, suggestions and hyphenation.
Text in languages without proofing tools is no longer flagged as English.
- Polish spelling from the SJP.PL Hunspell dictionary (Apache-2.0 option),
read by a new Rust affix engine (crates/proof/src/affix.rs); 875 KB asset,
~4.5M word forms. Regenerate with the pl_dictionary example.
- Polish suggestions: case, missing diacritics, rz/ż, ó/u, ch/h confusions,
missing spaces (napewno -> na pewno), edit distance.
- Conservative Polish grammar rules: repeated words, spacing, doubled
punctuation, sentence capitals, missing comma before że/który/ponieważ/
aby/żeby/jeśli…, we/ze forms.
- Polish hyphenation with the hyph-utf8 hyph-pl.tex patterns (MIT option);
pattern keys widened to 20 letters, TeX exceptions parsed; the heuristic
no longer treats ł, ż, ś… as vowels.
review.language validates and normalizes tags, can set noProof alone and
the document default; format.state reports lang and noProof.
Tests for every part, docs and ATTRIBUTION.md/NOTICE updated;
cargo xtask ci passes (fmt, clippy, test, assets, layers, wasm).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout: rustfmt the test module order left unformatted by #391
cargo fmt --check failed on main after #391 added mod tests_grid.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Review fixes for #346: read Polish spelling from the installed Hunspell dictionary
- Polish spelling uses the Hunspell pl_PL dictionary installed on the
system (OS packages, Homebrew, LibreOffice, ~/.local/share/hunspell, or
WORDCRAFT_DICTIONARIES) instead of a bundled 870 KB SJP.PL file;
without one, Polish text gets no spelling marks, like Word without the
language's proofing tools. .aff/.dic are size-capped and decoded from
UTF-8 or ISO 8859-2 (crates/proof/src/hunspell.rs).
- The dictionary can be replaced at runtime (install_dictionary) or
re-searched (reload_dictionary); an empty one accepts every word.
- Tests use small synthetic dictionaries written for them, so they don't
depend on what the machine has installed.
- The bundled pl.dic, its ATTRIBUTION/NOTICE entries and the generator
example are removed; docs describe the installed dictionary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* M15: Polish catalog covers the interface text merged from main
The merge of main (98456ac) brought ~575 new interface strings (charts,
SmartArt, content controls, Developer tab, AutoCorrect, outline view,
protection, compare, print, accessibility, references and Language/
Hyphenation dialogs, Draw additions). pl.tsv lacked them, so
polish_covers_every_key_of_every_other_catalog and
every_tab_and_command_is_translated failed. All are now translated.
cargo xtask ci: all 6 steps pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
File › Options is now a pane list (General, Display, Proofing, Save,
Language, Accessibility, Advanced, Agents), one function per pane in
crates/ui-egui/src/options.rs. Every setting shown is wired:
- General: theme, interface size (#475), mini toolbar on selection, dark page, community
button, user name and initials (comments use the initials).
- Display: tab characters, spaces, paragraph marks and hidden text shown
with Show/Hide ¶ off; rulers; print the page colour; update fields
before printing.
- Proofing: ignore words in capitals / with numbers / web and file
addresses (proof::Options, passed through layout), check spelling and
mark grammar as you type, custom dictionary add/remove (kept in ui.json).
- Save: AutoSave. Language: interface language. Accessibility: Read Aloud
speed and citations. Advanced: typing replaces selection, Insert key
overtype, overtype (text.overtype), recent documents count, measurement
units (dialogs, Layout ribbon indents and rulers).
file.options reads and sets the engine-side settings headlessly and only
opens the page when given no setting. ui.json loads leniently: a bad value
falls back to its default instead of dropping every setting.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
The English speller only knew the 1990s Moby word list plus a small
hard-coded EXTRA array, so everyday modern words (desktop, laptop,
download, upload, spreadsheet, database, screenshot, inbox, offline,
workplace…) were flagged. Inflection, doubled-consonant and prefix rules
also only looked stems up in Moby, so even supplemented words failed in
inflected forms (blogging, uploader).
- New supplement assets/spelling/en-extra.txt: ~220 lowercase base forms,
our own list (MIT OR Apache-2.0), replacing the EXTRA array.
- All stem lookups (inflections, doubled consonant, prefixes) consult the
dictionary and the supplement; suggestions include supplement words.
- Regression tests: common modern words and inflections accepted,
misspellings still flagged, "dekstop" suggests "desktop"; the list
stays sorted and lowercase.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* Tables: stop repeating a header row that leaves no room for the next row
A repeating header row plus a row that could not fit below it on an empty
page made place_table loop: after each page break the header was placed
again, so the page was never "at top" when the row was retried, split_row
had nothing to split, and the loop ran its 1,000 iterations per row. A
20-row table with a 640 pt header came out as 19,002 pages.
After a page break the header now repeats only when the row (or its first
split) still fits below it; otherwise the row starts at the top of the
page, as Word does when a header row would fill the page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout: give runs with different links or decorations their own style
Builder::style memoised StyleRuns by a key that folded strike, double
strike and "has a link" into one boolean and left out the underline
colour and baseline position. The display list reads all of these from
the shared StyleRun, so in a paragraph of like-formatted links every link
pointed at the first one's URL in the PDF (and an external link after an
internal one became a bookmark jump), double strike drew as single strike
after a struck run, and underline colours were copied.
The key is now the whole resolved format, so runs share a StyleRun only
when nothing the display list reads differs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* PDF and No Markup: leave tracked deletions out of the final text
PDF export defaults to no markup, but it laid the document out with every
deleted run in place and only switched off the strike and colour, so the
PDF printed deleted text as ordinary text ("Keep DELETEDTEXT INSERTED"):
neither the original nor the final document, and a leak of text the
author removed. The No Markup view did the same on screen.
A new LayoutOptions::hide_deleted lays deleted runs out like hidden text
(zero-width markers, so offsets, carets and hit testing keep working). PDF
export sets it whenever markup is off, the session layout sets it when
markup is hidden, and the markup toggles relayout as Show/Hide marks does.
The display list also never draws a deletion as plain text when markup is
off, whatever layout it is given.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Tables: repeat the header on every page and put the next row under it, as Word does
The earlier fix for the runaway loop dropped the repeated header row on any
page where the header and the next row did not fit together. Word never
does that: measured on Word for Mac, it moves the row to a new page, repeats
the header there and places the row under it even when the row then runs
into the bottom margin (a 300 pt header with 400 pt rows gives one header +
row per page; a 640 pt header with 100 pt rows does the same).
place_table now remembers when the page holds only the repeated header
rows. A row that cannot split is placed there anyway instead of breaking
the page again, which is both Word's behaviour and what keeps the loop from
spinning: every pass now places something. Rows of text taller than the
room under the header still split across pages.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* PDF: keep tracked deletions out of bookmarks and heading tags
Without markup the PDF pages show the final text, but bookmark titles and
the titles of heading structure tags were still built from plain_text(),
which includes deleted runs. A heading "Intro SECRET Part" with "SECRET "
deleted printed as "Intro Part" yet bookmarked as "Intro SECRET Part", and
a fully deleted heading still got a bookmark: text the author removed
leaked into the outline and the tag tree.
Paragraph::final_text() gives the text with deletions left out, and the
exporter titles headings with it unless markup is included. A heading
whose text is all deleted has an empty title and so no bookmark.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* No Markup: deleted note references take no number and print no endnote
With deletions hidden, a tracked-deleted footnote or endnote reference
disappears from the text, but note_numbers() still counted it. Deleting
the first of two footnote references left the surviving one numbered 2,
and a deleted endnote reference still printed its endnote after the last
paragraph, because the endnote list is taken from the same map.
note_numbers() now skips references inside tracked deletions when the
layout hides them, so the remaining notes number from 1 and endnotes whose
reference was deleted are not emitted. With markup shown nothing changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* No Markup: a deleted floating object leaves no wrap area behind
With deletions hidden, a tracked-deleted floating picture or shape was
not drawn, but place_para had already added its wrap exclusion from
p.objects without looking at the anchor, so the final text still flowed
around an invisible object (indented beside a square-wrapped shape).
Floating objects whose anchor is a tracked deletion are now skipped when
the layout hides deletions; deleted_objects() is shared with note
numbering, which needs the same test. With markup shown nothing changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* No Markup: a deleted soft hyphen no longer breaks the word with a hyphen
With deletions hidden, a tracked-deleted soft hyphen became a zero-width
marker, but hyphenation_points() reads the paragraph's raw text and
accepted marker clusters, so the line could still break there and draw
"ab-" in a narrow column although the final text has no soft hyphen.
When the layout hides deletions, clusters inside a tracked deletion are
no longer hyphenation points (soft hyphens or dictionary points alike).
With markup shown nothing changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* No Markup: tracked deletions add no line-break opportunities
Break opportunities came from running UAX #14 over the paragraph's raw
text, deleted runs included. With deletions hidden, a deleted soft hyphen
or space still let the line break there: "xx ab<del>SHY</del>cdef" in a
narrow column came out as "xx ab" / "cdef" instead of moving the word
"abcdef" to the next line whole.
When the layout hides deletions, the opportunities are computed over the
text without them and mapped back to offsets in the paragraph. With
markup shown, or with no deletions, the computation is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Hidden text: no hyphenation points or line breaks in text that is not shown
Hidden text that is not shown (the default, and always in print and PDF)
is laid out as zero-width markers, but break opportunities came from UAX #14
over the paragraph's raw text and hyphenation_points() accepted marker
clusters, so a hidden space or soft hyphen still let the line break there:
"xx ab<hidden SHY>cdef" in a narrow column came out "xx ab-" / "cdef".
Measured in Word for Mac, the hidden soft hyphen and the hidden space are
no place to break: "xx" / "abcdef".
left_out() is now the one test for text the layout leaves out (hidden text
not shown, tracked deletions without markup). The run loop records the
ranges it leaves out, and both the break opportunities and the hyphenation
points come from that, so deletions behave exactly as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Hidden text: a floating object anchored in it leaves no wrap area behind
With hidden text not shown, a floating picture or shape anchored in a
hidden run was not drawn (the run is a zero-width marker, so no object
cluster places it), but place_para had already added its wrap exclusion,
so the text still flowed around an invisible object. Measured in Word for
Mac: a square-wrapped picture anchored in hidden text is not printed and
the text runs the full width.
deleted_objects() becomes left_out_objects(), which resolves each anchor's
formatting through the styles (hidden text often comes from a character
style) and applies the same left_out() test as the paragraph layout. Note
numbering keeps counting hidden reference marks, as Word does, so it asks
only about deletions; its behaviour is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Hidden text: print the footnote of a hidden reference mark, as Word does
A footnote whose reference mark is hidden text was numbered but never
printed: the hidden run became zero-width markers without recording the
note, so place_para never put the footnote at the foot of the page.
Measured in Word for Mac with hidden text not printed, a hidden footnote or
endnote reference still takes its number (the next visible footnote is 2)
and both notes are printed. Endnotes already were, and the numbering
already matched.
A hidden reference mark now records its note like a visible one, so the
footnote goes to the foot of the page holding the line it sits on. A
deleted reference in the final text still records nothing, as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* PDF: keep hidden text out of bookmarks and heading tags
The PDF never prints hidden text (its layout does not show it), but the
bookmark and heading-tag titles were built from final_text(), which drops
tracked deletions only. A heading "Intro SECRET Part" with "SECRET " hidden
printed as "Intro Part" yet bookmarked as "Intro SECRET Part", and a heading
that is all hidden text still got a bookmark for a heading nobody can see.
Paragraph::text_where() generalises final_text() to any per-run test, and
the exporter titles headings with the runs that are neither hidden (resolved
through the styles, since a character style can hide text) nor, without
markup, deleted. A heading that is all hidden has an empty title and so no
bookmark. Word's PDF export from AppleScript writes no outline, so there is
no Word answer to match; this follows what the pages show, as for deletions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* PDF: titles leave out what the layout left out, table formatting included
Bookmark and heading-tag titles resolved hidden text from the paragraph
and run formatting alone, but the layout also applies the table style's
conditional formatting (header row, first column, total row) beneath the
runs. A Heading 1 in a table header row whose style hides it printed
nothing yet still got a bookmark, and a partly hidden heading leaked the
hidden part into its tag.
ParaLayout now records the byte ranges it laid out as nothing (left_out),
resolved exactly as laid out, and the exporter titles a heading from the
text outside them (plus tracked deletions without markup, as before), so
the titles cannot disagree with the pages. text_where() gives way to
text_without() over byte ranges; final_text() is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Hyphenation: hyphenate words as laid out, without hidden or deleted text
Automatic hyphenation looked up the words of the paragraph's raw text and
only dropped points that fell inside text the layout leaves out. Words and
the minimum-letters limits still came from the raw text: with "accep"
hidden in "acceptable", the visible "table" could break as "t-able" (the
point after "accept"), and a hidden space split "foot<space>ball" into two
words, so the visible "football" never hyphenated.
Soft hyphens and dictionary points are now found in the same text as laid
out that the UAX #14 break opportunities already use, and mapped back to
paragraph offsets. With hidden text shown and markup on, that text is the
paragraph's own and nothing changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout, PDF: skip left-out text in one pass, not a scan per char
Building the text as laid out tested every char against every left-out
range, and Paragraph::text_without did the same for PDF titles. Hidden
text in many runs (formatting that alternates, a common result of imports)
made both quadratic: a 100,000-char hidden heading in one-char runs took
13.9 s to convert with the release CLI, 400,000 chars minutes.
The layout now merges adjacent left-out runs as it records them and walks
the ordered ranges with a cursor; text_without sorts its ranges by start
(they may come in any order and overlap) and walks them the same way. Both
are linear: 0.20 s for that heading, 0.81 s at 400,000 chars, with the same
output. No left-out text still means no filtering at all.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Hyphenation: check the limits of each point in constant time
hyphen_points() found each candidate point's letter run by scanning out
from the point and counted the letters on each side by rescanning the run,
and the syllable heuristic rebuilt the rest of the word as a string for
every position. Both are quadratic in the word's length. Ordinary words are
short, but with hidden separators the layout now hyphenates the visible
word, so "hyphenation " repeated 8,192 times with every space hidden is one
90,112-letter word: 2.0 s to convert with the release CLI.
Letter and uppercase counts before each char, the letter run of each char
and the heuristic's next vowel and byte offsets are now computed once, so
every point is checked in constant time: that document converts in 0.09 s.
The output is unchanged: the old functions stay in the tests as the
reference, compared over awkward words and every combination of limits, and
a hyphenated document's PDF text is identical. No maximum word length is
imposed: the work is linear, so a cap would only change output.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout: find the formatting of every object anchor in one walk
left_out_objects() asked props_of_char() for each floating object's anchor,
and props_of_char() scans the runs from the start, so a paragraph with many
anchors in their own runs cost a scan per anchor, quadratic, and paid it on
every layout because it runs before the paragraph cache is consulted:
40,000 hidden anchors took 0.69 s, against 0.05 s for 10,000.
The runs and the object offsets are both in order, so one walk now finds
each anchor's run, resolving a run's formatting once; past the last run it
is the paragraph mark, as props_of_char() gives. With hidden text shown and
markup on nothing can be left out and no work is done at all.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout: find a run's style without formatting it as a string
#152 keyed each run's style by its whole format's Debug string, built for every
shaped piece; on a long document with many runs that doubled layout time (3000
paragraphs x 40 linked runs: 12.7 s vs 6.4 s for the old key). Key by a hash of
the commonly varying fields and confirm by comparing formats: 3.8 s. A capped
bucket scan keeps formats that never compare equal (NaN) linear.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout: floats hidden in table cells take no room, table style included
Test the layout_box path the merge extended (a float anchored in hidden text in a
table cell is not placed and leaves no wrap area), and resolve the anchors'
formatting with the table style's character formatting under the runs' own, as
the cell's text is laid out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Tests: linear-time checks compare 8x the input against a 24x limit
At 4x with an 8x limit a busy machine could push a linear layout over the line (8.8x seen once in a full workspace run); at 8x a linear layout takes about 8x and a quadratic one about 64x, so 24x stays far from both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Dimitar
Eduardo Soares
RegalDragoon200
ripdog
Brandon Thomas
witkowski01
Valentin Klein
AiBizzApps