* D0+D1: open Word 97-2003 .doc — container, FIB, piece table text (#28)
New wordcraft-docbin crate (L2) reads the OLE2 container via the cfb
crate (MIT) and parses the FIB and piece table from the [MS-DOC]
specification: text pieces decode as UTF-16LE or Windows-1252 (table
moved to wordcraft-doc::encoding), subdocument CP ranges, and Word's
special characters map to the model's (tabs, breaks, hyphens). .doc/.dot
registered in OPEN_EXTS and the I/O dispatch. Encrypted and Word 6/95
files give clear errors; every length is checked and proptest
truncation/flips never panic.
* D2: .doc formatting — stylesheet, fonts, CHPX/PAPX FKPs, sprms (#28)
Sprm decoder with operand-size rules (incl. sprmTDefTable/sprmPChgTabs
exceptions and the Prm0 isprm table), bin tables + ChpxFkp/PapxFkp page
lookup, STSH styles (StdfBase layout validated against real Word files:
bchUpe == cbStd, sti 0 = Normal) and the SttbfFfn font table in both
extended and self-delimiting flavours. The CP walk now splits runs on
formatting changes and resolves paragraph styles: bold/italic/strike/
underline/caps/hidden, colours (Ico palette + COLORREF), half-point
sizes, super/subscript, fonts, highlights; alignment, indents, spacing,
keep options, outline levels. Unknown sprms are skipped leniently;
an env-gated corpus test opens real files (styles + text asserted).
* D3: .doc structure — sections, headers/footers, tables, lists (#28)
- Sections: PlcfSed/Sepx with 16-bit measurement sprms (sprmSXaPage & co),
explicit sprmSBOrientation with a width/height fallback; even/odd/first
header and footer stories from Plcfhdd become document parts.
- Tables: sprmTDefTable grids (operand length fixed to cb+1 per the spec),
horizontal merges (sprmTMerge), vertical merges (sprmTVertMerge), cell
shading (sprmTSetShd / sprmTDefTableShd80), cell widths, valign, row
height/header/cant-split, and table borders (sprmTTableBorders80 Brc80s).
- Lists: PlfLst (LSTF + appended LVLs) and PlfLfo (LFO + LFOLVL overrides)
resolved into the numbering model; sprmPIlvl/sprmPIlfo mark paragraphs.
- Combined fixture (intro + table + list + 2 sections + header) round-trips
through Document JSON; hostile proptests now mutate the structured fixture.
* D4: .doc notes, fields, bookmarks and inline pictures (#28)
- Footnotes/endnotes: PlcffndRef/PlcfEndRef reference CPs paired with the
PlcffndTxt/PlcfEndTxt story ranges; each story walks into a Footnote or
Endnote part and the 0x02 anchor becomes a NoteRef object.
- Fields: 0x13/0x14/0x15 state machine in the walker — instruction text is
hidden, the cached result kept; HYPERLINK fields keep their result as
linked runs, everything else becomes one Field object.
- Bookmarks: SttbfBkmk names + Plcfbkf/Plcfbkl CPs become BookmarkStart/
BookmarkEnd objects at their anchors.
- Pictures: sprmCPicLocation points into the Data stream; the PICF header
gives the display size (dxaGoal/dyaGoal scaled by mx/my) and the payload
is format-sniffed (PNG/JPEG/GIF/BMP; metafiles are dropped with a warn).
- Hostile proptests now truncate/flip a fixture with all of the above.
* D5: .doc import hardening, caps and roadmap (#28)
- Cap inline pictures at 5 000 per document.
- Engine test: .doc/.dot bytes dispatch through the Word 97-2003 reader
(garbage must error, never panic or open empty).
- ROADMAP: Word 97-2003 import row (60% — text, formatting, styles,
sections, headers/footers, tables, lists, notes, fields, bookmarks,
raster pictures).
Real-corpus validation stays opt-in: WORDCRAFT_DOCBIN_FIXTURES=<dir>
cargo test -p wordcraft-docbin corpus_files_open.
* Corpus fixes: compressed-piece FC space, PAPX query point, none-level labels (#28)
Validated against two real files (kept locally under plan/word/fixtures,
gitignored): a 1 MB Lorem-ipsum sample and a 2003 styles/lists/table
sample.
- FKP bin tables and pages address the WordDocument stream in true byte
offsets; a compressed piece stores its base fc doubled (byte = fc/2),
so fc_of_cp now unwinds the doubling. Before, every PAPX/CHPX lookup in
compressed (cp1252) files landed on the wrong entry — tables collapsed
to one row, lists and heading styles were lost.
- The paragraph-properties query now uses the paragraph mark's own FC
(inside the paragraph) instead of the FC after it, which picked up the
next paragraph's FKP entry.
- Fixture papx entries re-keyed to paragraph-start CPs, the real rgfc
convention.
- Counters::next_label returns nothing for NumFormat::None levels (Word's
numFmt none shows no label; also fixes a latent docx-reader case).
- Symbol/Wingdings bullet chars map to Unicode glyphs (• ▪ ○ ●).
With this, both corpus files render headings, styles, nested and numbered
lists, the tables and the PAGE field correctly; metafile-wrapped images
and OLE objects stay dropped by policy (logged).
* docbin: bound the text walk by real text; dedupe and cap pictures (#28)
Hostile CP numbers no longer drive the work:
- header/footnote/endnote story ranges from the Table stream are clamped to their
subdocument (ccpHdd/ccpFtn/ccpEdn) with checked base additions, and every walk stops at
the last piece's CP; a story spanning [0, 4e9) used to push billions of U+FFFD;
- characters are found by binary search over the pieces (PieceTable::char_at), and an
unreadable stretch (gap, truncated or out-of-stream piece) is skipped in one step rather
than one U+FFFD per CP; the old per-character scan was O(chars x pieces);
- the piece CPs are made non-decreasing at parse, so inverted pieces are empty and binary
search stays sound; surrogate pairs now decode;
- note references are looked up by binary search; the style's CharProps is borrowed, not
cloned, per character;
- pictures are interned by fc_pic (5000 anchors on one picture share one media entry and
one copy) and the bytes copied per document are capped at 256 MB.
Each fix has a crafted-bytes regression test; against the previous code the story,
piece-claim, last-piece and many-pieces tests hang (killed at 60 s) and the shared
picture test copies the picture once per anchor (80 GB).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docbin: checked offset arithmetic in lists, sections and table rows (#28)
- list.rs: PlfLst/PlfLfo/LVL offsets (at + 2 + cLst*28, at + 4 + lfoMac*16, off += len,
...) saturate, which matters on 32-bit targets (wasm32) where fc + n can overflow usize;
the LFO override path no longer indexes abstracts directly.
- sections.rs: the Sepx range (fc_sepx + 2 + cb) is checked; Plcfhdd is read only as far
as the stories a capped section count can use (it used to reserve lcb/4 entries).
- table.rs: sprmTDyaRowHeight = i16::MIN negated in i16 (overflow panic in debug builds);
now -(v as f32).
Regression tests: the row height test failed before (overflow panic with debug assertions, a 0 pt height without); the list test pins the saturating
offsets (it only overflowed on 32-bit targets).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Format the merged OPEN_EXTS list
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Zotero 1: range fields and custom document properties round-trip
Citation managers' ADDIN fields (Zotero, Mendeley, EndNote) are now kept as
range fields: FieldStart/FieldEnd markers with the result as ordinary,
formatted content between them, possibly spanning paragraphs (a Zotero
bibliography). Before, the result was flattened to plain text and a
multi-paragraph field lost its end.
- doc: InlineObject::FieldStart/FieldEnd, Document::field_ranges,
balance_field_ranges; CustomProp and custom_props with get/set/remove.
- docx: read ADDIN fields as ranges (field codes up to 2 MiB), write them
back as begin/instr/separate…end; orphan markers are dropped on save so
files always hold whole fields. docProps/custom.xml is parsed and
regenerated (simple types as text, vectors/blobs verbatim) instead of
passed through, so Zotero's ZOTERO_PREF_n settings are editable.
- engine: file.properties reads and sets custom properties.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Zotero 2: wire-protocol client and document bridge
New crate wordcraft-zotero (L5) speaks Zotero's published word-processor
protocol (port 23116; 32-bit transaction id + big-endian length frames,
JSON payloads, ERR: replies):
- client: sends an integration command (addEditCitation,
addEditBibliography, addNote, refresh, removeCodes, setDocPrefs) and
answers Zotero's calls until Document_complete or the connection
closes (cancelled dialog). Answers go through a callback so the app can
apply them on its UI thread; run_on_session answers directly.
- Bridge: implements Zotero's Document_*/Field_* calls on a Session.
Fields are stored as Word's plugin stores them (ADDIN ZOTERO_ range
fields; prefs in ZOTERO_PREF_n, with Zotero's ReferenceMark field type
kept as Word's Field in the file), so documents stay Word-compatible.
Rich text (RTF or HTML) keeps citation formatting; bibliographies get
a Bibliography style from setBibliographyStyle; note styles create
footnotes/endnotes, and deleting a lone note citation removes its note;
placeholder links convert to fields. One transaction = one undo step.
Field ids stay stable through inserts and deletes.
- Tests: a fake Zotero over a real socket, every call, docx round trip,
hostile arguments for every method, random frames (proptest).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Zotero 3: Zotero in the app and the CLI, tested with the real Zotero
- ui.zotero.<command> (addEditCitation, addEditBibliography, addNote,
refresh, removeCodes, setDocPrefs), ui.zotero.status and
ui.zotero.answer. The socket runs on a background thread; each of
Zotero's calls is answered on the UI thread against the session, and
Zotero's questions become a dialog the session waits on. Web builds
report that Zotero needs the desktop app.
- wordcraft-cli zotero <command> <file> [--save] [--trace] [--port].
- Bridge: a selection Zotero makes to show a citation is put back when
the session ends.
- ui_shot: {"wait": ms} keeps frames running for background work.
- docs/zotero.md.
Checked against Zotero 10.0.6: refresh of a citation and bibliography
(APA), its "modified citation" question answered in the app's dialog,
RTF results with italics, Bibliography style, prefs round trip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Zotero 4: switching between in-text and note styles moves citations
Document_convert and Field_convert now move a citation between the
text, footnotes and endnotes, as Zotero asks when the document's style
changes between an in-text and a note style: into a new note at the
same place, or back into the text where the note mark was (removing a
note that held only the citation; keeping one with other text). Field
ids survive the move, formatting is kept, and each Zotero session stays
one undo step.
Checked against Zotero 10.0.6: unlink citations (removeCodes, with its
confirmation question) and a refresh after switching the document to
Chicago notes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Zotero 5: Zotero tab in the ribbon
A Zotero tab (before Help) with Add/Edit Citation, Add Note, Add/Edit
Bibliography, Refresh, Document Preferences and Unlink Citations. The
buttons are disabled while a Zotero command is running and in the web
build. Three new icons drawn in code (document preferences, unlink,
add note); the others reuse the References tab's. Labels and status
messages translated (zh-hans, zh-hant, ja).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Zotero: Add/Edit Bibliography inserted nothing
Zotero reads a newly inserted bibliography field back before filling
it, and unlinks a field whose text is empty, taking it for a
bibliography the user deleted. Our new fields were empty, so Add/Edit
Bibliography ended with Field_removeCode and nothing in the document.
New fields now hold "{Citation}" until Zotero writes the result.
Found by tracing the session with Zotero 10.0.6; regression test added.
wordcraft-cli zotero takes --cmd (run before the Zotero command, e.g.
caret.docEnd) to reproduce sessions from a given caret position.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Zotero: log each command's outcome and Zotero's calls to the app log
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Cargo.lock: wordcraft-zotero at the workspace version (0.4.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* MCP bridge: send the control key, to loopback addresses only
`wordcraft-cli mcp --connect` now sends the app's control key with every
request. It takes the key from WORDCRAFT_CONTROL_KEY, or else from the
key file the app writes for that port, and reads it again on every new
connection, also after an `unauthorized` reply (the app restarted with
a new key).
The key goes only to loopback addresses: 127.0.0.0/8, ::1 and
`localhost`, and only when the connection really leads to a loopback
address. For any other host the bridge sends no key, prints why at
start, and turns the app's `unauthorized` into an error that says the
key was not sent and how to reach the app (through 127.0.0.1, for
example an SSH tunnel). Other host names are never resolved for this.
The new `wordcraft_mcp::control_key` module holds the rule the app and
the bridge share: the settings folder (the one with ui.json), the
Flatpak instance id and `<settings>/control-key.<instance>`
(`local-<port>` outside Flatpak). No new dependency; wasm-safe.
* Control port: every request needs the window's key
Any local program, and possibly a web page, could drive the app through
its loopback control port. Each window started with --control now has
a key:
- once the port is bound, the app writes a random key (32 bytes from
the OS, as hex) to <settings>/control-key.<instance>, mode 0600 on
Unix, replacing a stale file; it removes the file on a clean exit. A
failed bind writes nothing. <instance> is the Flatpak instance id, or
local-<port> outside Flatpak (wordcraft_mcp::control_key, the rule
the MCP bridge reads it with). The settings folder is the one that
holds ui.json; prefs_path now uses the same helper.
- every request needs the key: top-level "key", or
{"method":"auth","key":...} once per connection. Only auth reads
params.key; elsewhere it stays an ordinary parameter (ui.key). The key
is checked on every request, in time that does not depend on where a
wrong key differs. A missing or wrong key gets
{"ok":false,"error":"unauthorized"} and the connection is closed.
- a line over 1 MiB gets "line too long" and the connection is closed
(the rest of the line is drained first, so the client reads the
error).
- a connection that sends no authorised request within 30 s of being
accepted is closed; blank lines and bad JSON do not extend that
limit. After that it may stay idle.
* Docs: control-port keys
docs/control-protocol.md gets a Keys section: where the key file is
(per platform and per instance), top-level "key" or auth once per
connection, `unauthorized` and the closed connection, the 1 MiB line
limit, the 30 s limit for the first authorised request, and how the MCP
bridge finds the key and sends it only to loopback addresses (an SSH
tunnel for an app on another machine). docs/mcp.md points there, and
the agent guides (AGENTS.md, CLAUDE.md) show the key in their control
channel example.
* Control key: the key-file helpers in their own small crate
wordcraft-control-key (standard library only, Standalone in the layering
table) holds where the key file lives and how to read it. The app and the
MCP bridge both use it, so the app no longer depends on wordcraft-mcp.
* Cargo.lock: wordcraft-control-key follows the workspace version (0.4.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: lpgn <2556951+lpgn@users.noreply.github.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Clean-room foundations for WordCraft: Arc-shared document model with
paragraph/run editing, styles, lists, sections and tables; Word-style
font substitution; first-fit line breaking with tabs, lists, alignment,
pagination, tables, headers/footers and hit testing; vello_cpu page
rasteriser.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eduardo Soares
Dr. Gled Guri
Lucio
Brandon Thomas