M13: scan each directory once in CLI checks (#1501)
Co-authored-by: MMexicann <MMexicann@users.noreply.github.com> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: Brandon Thomas <echelon@gmail.com>
This commit is contained in:
Mex
GitHub
parent
5ebb53a4a4
commit
e88dd728cb
|
|
@ -2,6 +2,7 @@
|
|||
|
||||
**Stage: alpha** · next: beta, ~26% (ready for real work 49% → 75%) and ~400–700 h away
|
||||
|
||||
> **Last reviewed:** 2026-10-12 · **Last updated:** 2026-10-12 · **Change:** minor (M13: CLI check visits each directory once through aliases and cycles; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
|
||||
> **Last reviewed:** 2026-10-12 · **Last updated:** 2026-10-12 · **Change:** minor (M4: Extract and Split preserve shared Type 3 glyph resources; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
|
||||
> **Last reviewed:** 2026-10-12 · **Last updated:** 2026-10-12 · **Change:** minor (M2: packed TIFF-predictor RGB rows retain their byte boundaries; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
|
||||
> **Last reviewed:** 2026-10-12 · **Last updated:** 2026-10-12 · **Change:** minor (M3 #1364 candidate: consent before absolute local GoToR access; estimates unchanged); earlier: minor (M1 #1339: an unchanged incremental save of a classic cross-reference table writes a zero-length subsection; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
|
||||
|
|
@ -147,6 +148,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
|
|||
|
||||
## Progress log
|
||||
|
||||
- **2026-10-12 (M13, CLI robustness scans):** `pdfcraft-cli check` visits a directory once even when input roots overlap or directory symlinks lead back into the tree. PDFs reached through file symlinks remain accepted. Three real CLI regressions cover overlapping roots, a directory cycle and directory/file aliases. Estimates unchanged.
|
||||
- **2026-10-12 (M4, Type 3 extraction fidelity):** Extract and Split preserve shared image/resource dictionaries reached through Type 3 glyphs, including nested forms, patterns and soft masks. A bounded object-graph walk protects unknown glyph content; unreadable pages/forms retain their glyph resources, and shared explicit fonts are visited once. Separate page dictionaries still prune unused images. Synthetic save/reopen regressions and an opt-in Poppler pixel oracle cover resource ownership, nested forms, cycles, limits and repeated fonts. Estimates unchanged.
|
||||
- **2026-10-12 (M2, TIFF predictor rows):** Packed RGB images using TIFF horizontal differencing keep every row when samples do not fill the last byte. The renderer aligns each decoded row before moving to the next. Synthetic decoding tests cover 1/2/4-bit samples, horizontal differencing and an 8-bit control; a red/green page-render regression verifies visible output. Estimates unchanged.
|
||||
- **2026-10-12 (M3, #1364 candidate):** Absolute local GoToR links wait for Allow/Cancel before target-file access. Pending requests preserve their source and page/view destination; relative links retain their existing behavior. Headless `link_open` requires explicit `allow_absolute: true`, preserving existing root, symlink, file and password validation. Estimates unchanged.
|
||||
|
|
|
|||
|
|
@ -637,8 +637,20 @@ fn short(s: &str) -> String {
|
|||
fn collect(paths: &[&str]) -> Vec<PathBuf> {
|
||||
let mut out = Vec::new();
|
||||
let mut stack: Vec<PathBuf> = paths.iter().map(PathBuf::from).collect();
|
||||
let mut visited = std::collections::HashSet::new();
|
||||
while let Some(p) = stack.pop() {
|
||||
if p.is_dir() {
|
||||
// Directory aliases and symlink cycles must not scan the same tree repeatedly.
|
||||
let identity = match p.canonicalize() {
|
||||
Ok(identity) => identity,
|
||||
Err(error) => {
|
||||
eprintln!("{}: could not resolve directory: {error}", p.display());
|
||||
continue;
|
||||
}
|
||||
};
|
||||
if !visited.insert(identity) {
|
||||
continue;
|
||||
}
|
||||
if let Ok(rd) = std::fs::read_dir(&p) {
|
||||
stack.extend(rd.flatten().map(|e| e.path()));
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,111 @@
|
|||
//! The robustness harness visits each directory once, including symlink aliases and cycles.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::{Command, Stdio};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
const BIN: &str = env!("CARGO_BIN_EXE_pdfcraft-cli");
|
||||
|
||||
struct Workdir(PathBuf);
|
||||
|
||||
impl Workdir {
|
||||
fn new(test: &str) -> Self {
|
||||
let dir = std::env::temp_dir().join(format!("pdfcraft-cli-check-{test}-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
std::fs::create_dir_all(dir.join("input/nested")).unwrap();
|
||||
let objects = [
|
||||
"<< /Type /Catalog /Pages 2 0 R >>",
|
||||
"<< /Type /Pages /Kids [3 0 R] /Count 1 /MediaBox [0 0 20 30] >>",
|
||||
"<< /Type /Page /Parent 2 0 R >>",
|
||||
];
|
||||
let mut bytes = b"%PDF-1.7\n".to_vec();
|
||||
let mut offsets = Vec::new();
|
||||
for (i, object) in objects.iter().enumerate() {
|
||||
offsets.push(bytes.len());
|
||||
bytes.extend_from_slice(format!("{} 0 obj\n{object}\nendobj\n", i + 1).as_bytes());
|
||||
}
|
||||
let xref = bytes.len();
|
||||
bytes.extend_from_slice(b"xref\n0 4\n0000000000 65535 f \n");
|
||||
for offset in offsets {
|
||||
bytes.extend_from_slice(format!("{offset:010} 00000 n \n").as_bytes());
|
||||
}
|
||||
bytes.extend_from_slice(format!("trailer\n<< /Size 4 /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n").as_bytes());
|
||||
std::fs::write(dir.join("input/a.pdf"), &bytes).unwrap();
|
||||
std::fs::write(dir.join("input/nested/b.PDF"), &bytes).unwrap();
|
||||
Self(dir)
|
||||
}
|
||||
|
||||
fn check(&self, inputs: &[&Path]) -> Vec<serde_json::Value> {
|
||||
let report = self.0.join("report.json");
|
||||
let mut child = Command::new(BIN)
|
||||
.arg("check")
|
||||
.args(inputs)
|
||||
.args(["--dpi", "36", "--timeout", "2", "--json"])
|
||||
.arg(&report)
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::piped())
|
||||
.spawn()
|
||||
.unwrap();
|
||||
let deadline = Instant::now() + Duration::from_secs(10);
|
||||
while child.try_wait().unwrap().is_none() {
|
||||
if Instant::now() >= deadline {
|
||||
child.kill().unwrap();
|
||||
let output = child.wait_with_output().unwrap();
|
||||
panic!("check did not finish scanning directories: {}", String::from_utf8_lossy(&output.stderr));
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(10));
|
||||
}
|
||||
let output = child.wait_with_output().unwrap();
|
||||
assert!(output.status.success(), "{}", String::from_utf8_lossy(&output.stderr));
|
||||
serde_json::from_slice(&std::fs::read(report).unwrap()).unwrap()
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for Workdir {
|
||||
fn drop(&mut self) {
|
||||
let _ = std::fs::remove_dir_all(&self.0);
|
||||
}
|
||||
}
|
||||
|
||||
fn assert_two_valid_pdfs(results: &[serde_json::Value]) {
|
||||
assert_eq!(results.len(), 2, "each PDF should be checked once: {results:?}");
|
||||
for result in results {
|
||||
assert_eq!(result["status"], "ok", "{result}");
|
||||
assert_eq!(result["pages"], 1, "{result}");
|
||||
}
|
||||
assert!(results[0]["file"].as_str().unwrap().ends_with("a.pdf"));
|
||||
assert!(results[1]["file"].as_str().unwrap().ends_with("b.PDF"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn check_scans_overlapping_directories_once() {
|
||||
let dir = Workdir::new("overlap");
|
||||
let input = dir.0.join("input");
|
||||
assert_two_valid_pdfs(&dir.check(&[&input, &input, &input.join("nested")]));
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[test]
|
||||
fn check_scans_a_directory_symlink_cycle_once() {
|
||||
let dir = Workdir::new("cycle");
|
||||
let input = dir.0.join("input");
|
||||
std::os::unix::fs::symlink(".", input.join("again")).unwrap();
|
||||
assert_two_valid_pdfs(&dir.check(&[&input]));
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[test]
|
||||
fn check_follows_directory_and_file_symlinks() {
|
||||
let dir = Workdir::new("aliases");
|
||||
let input = dir.0.join("input");
|
||||
let alias = dir.0.join("alias");
|
||||
std::os::unix::fs::symlink(&input, &alias).unwrap();
|
||||
assert_two_valid_pdfs(&dir.check(&[&input, &alias]));
|
||||
|
||||
let file_alias = dir.0.join("page.pdf");
|
||||
std::os::unix::fs::symlink(input.join("a.pdf"), &file_alias).unwrap();
|
||||
let results = dir.check(&[&file_alias]);
|
||||
assert_eq!(results.len(), 1);
|
||||
assert_eq!(results[0]["status"], "ok");
|
||||
assert_eq!(results[0]["file"], file_alias.to_string_lossy().as_ref());
|
||||
}
|
||||
|
|
@ -1,5 +1,6 @@
|
|||
# PdfCraft roadmap: milestones and what's next
|
||||
|
||||
> **Last reviewed:** 2026-10-12 · **Last updated:** 2026-10-12 · **Change:** minor (M13: CLI check visits each directory once through aliases and cycles; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
|
||||
> **Last reviewed:** 2026-10-12 · **Last updated:** 2026-10-12 · **Change:** minor (M4: Extract and Split preserve shared Type 3 glyph resources; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
|
||||
> **Last reviewed:** 2026-10-12 · **Last updated:** 2026-10-12 · **Change:** minor (M2: packed TIFF-predictor RGB rows retain their byte boundaries; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
|
||||
> **Last reviewed:** 2026-10-12 · **Last updated:** 2026-10-12 · **Change:** minor (M3 #1364 candidate: consent before absolute local GoToR access; estimates unchanged); earlier: minor (M1 #1339: unchanged incremental saves of a classic cross-reference table write a zero-length subsection; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
|
||||
|
|
@ -110,6 +111,8 @@ total matches the parity estimate.
|
|||
| M9 | Signatures (PAdES, validation) | 160–280 | 60% | 40–70 | Done: new `sign` crate (DER, X.509, CMS, PKCS #12 on RustCrypto; aws-lc-rs for RSA private keys); PAdES B-B signing (visible, invisible, existing fields, certification with DocMDP); validation with trust store and changes-after-signing classification; self-signed digital IDs; Signatures panel, message bar, sign dialogs; agent tools. Checked with pdfsig and OpenSSL.<br>Done since: macOS Keychain and Windows Current User Personal store signing (software-backed CNG keys), certificate viewer, signed documents protected from rewrites. Missing: timestamps (B-T), LTV (DSS, OCSP, CRL), FieldMDP, smart cards, PKCS #11 |
|
||||
| M10 | OCR, create, export, print | 200–350 | 33% | 120–215 | Done: create from blank/text/PNG/JPEG/TIFF (multi-page)/GIF/BMP; export PNG/JPEG/TIFF and text; Print (Acrobat's sizing, n-up, booklet, poster, comments & forms, preview, the CUPS and Windows spoolers, print-ready PDF).<br>Done since: embedded/72/custom DPI choices for image imports; export all images; OCR (searchable image for pages, ranges and multiple files); single-sided cut-and-stack imposition with cut marks, through Print and doc_print; architectural/ANSI/large ISO sheet presets, Match page at 100% and PDF-scale preview (#599 follow-up); Create PDF from multiple files (PDFs, images and text, edited as pages in the grid); OCR can also read pictures on pages that have text, without adding that text twice (#1028). Done since: Word invoice export uses borderless inferred tables, floating positioned text/artwork only where text overlaps an image or is rotated, and per-page sizes, covered by synthetic invoice automation regressions (#773); printing on the web through the browser's print dialog, sheets drawn by PdfCraft (gap 4). Missing: OCR languages beyond Latin, editable-text OCR output, full Office layout fidelity/import |
|
||||
| M11 | Optimize, preflight, PDF/A/X/UA, print production | 200–350 | 18% | 80–140 | Done: new `optimize` crate: Reduce File Size and the PDF Optimizer (images measured where drawn, bicubic downsampling, JPEG/ZIP recompression only when smaller, discard objects and user data, Flate clean-up, resource merging, object streams). Missing: fonts and transparency panels, space audit, preflight, PDF/A/X/UA |
|
||||
| M12 | Accessibility, compare, measure, search, XFA | 200–380 | 30% | 90–160 | Done: new `a11y` crate with the Accessibility Checker (all 32 rules, report, Fix/Skip/Explain, options dialog and results panel, agent tools). Locatable figure, annotation and form findings highlight their page bounds.<br>Done since: 2D distance, perimeter and area measurements with persistent viewport calibration, snapping, live information and CSV export; a bounded, text-only auto-tag proposal tool for one page. Missing: writing proposed tags and parent tree, field autotag, visual Tags/Order/Content panels, Reading Order tool, compare, geospatial/3D measurement, search index, XFA |
|
||||
| M13 | Automation (MCP, Action Wizard, CLI) + AI providers | 60–120 | 45% | 25–45 | Done: MCP resources (document info, text, page images); headless tool table (123 tools incl. signing, optimizing, initial view, links, stamps, data exchange, comment review, forms authoring and scripts, redaction, sanitize, print, add content), opt-in MCP server over stdio, CLI `run`/`tools` (closed stdout pipes exit cleanly), UI control channel with drag. Missing: Action Wizard, AI providers<br>Done since: CLI run and MCP reject missing, empty, repeated and option-valued --root arguments before any tool/script execution or MCP initialization. A single-hyphen directory name remains valid; synthetic CLI regressions preserve a would-be overwritten file. `text_find` also caps the per-page match ranges it materializes at the remaining result limit plus one continuation match. CLI robustness scans visit each directory once through overlapping roots and symlink aliases. |
|
||||
| M12 | Accessibility, compare, measure, search, XFA | 200–380 | 30% | 90–160 | Done: new `a11y` crate with the Accessibility Checker (all 32 rules, report, Fix/Skip/Explain, options dialog and results panel, agent tools). Locatable figure, annotation and form findings highlight their page bounds.<br>Done since: 2D distance, perimeter and area measurements with persistent viewport calibration, snapping, live information and CSV export; a bounded text-only auto-tag proposal tool; text-only Apply writes marked content, paragraph tags and a parent tree, then opens the Tags panel for review. Missing: mixed-content and field autotag, heading/list/table/figure classification, Order/Content panels, Reading Order tool, compare, geospatial/3D measurement, search index, XFA |
|
||||
| M13 | Automation (MCP, Action Wizard, CLI) + AI providers | 60–120 | 45% | 25–45 | Done: MCP resources (document info, text, page images); headless tool table (123 tools incl. signing, optimizing, initial view, links, stamps, data exchange, comment review, forms authoring and scripts, redaction, sanitize, print, add content), opt-in MCP server over stdio, CLI `run`/`tools` (closed stdout pipes exit cleanly), UI control channel with drag. Missing: Action Wizard, AI providers<br>Done since: CLI run and MCP reject missing, empty, repeated and option-valued --root arguments before any tool/script execution or MCP initialization. A single-hyphen directory name remains valid; synthetic CLI regressions preserve a would-be overwritten file. `text_find` also caps the per-page match ranges it materializes at the remaining result limit plus one continuation match. |
|
||||
| M14 | 1.0 polish: performance, localization, installers | 120–250 | 15% | 80–150 | Done: PhotoCraft's translation system (`i18n/`: TSV catalogs, `tl!`, command-id and plural entries, system-language detection, strict catalog tests); every dialog, panel and notice goes through `tl!`. Catalogs of ≈ 2,000–2,140 entries for Japanese, Simplified and Traditional Chinese, Russian, Bulgarian, German, Spanish, French, Telugu, Hungarian, Ukrainian, Italian, Brazilian Portuguese and Arabic (logical-order catalog put into display order; layout not yet mirrored); Czech covers the menus. Signed and notarized macOS builds, signed Windows installers (x64, x86, ARM64), AppImage, deb, rpm, Flatpak and FreeBSD packages. #307 (bounded large-PDF memory): shared resource indexes and parser metadata, lazy inspection, bounded caches; a 9,156-page file inspects in ≈ 1.8 s at ≈ 500 MiB instead of exceeding 10 GiB. Edits let go of the previous working file before writing the next (a 1 GiB file peaks at two copies while editing, not three; gap 13). Shortcut display uses localized modifier/key names across menus, toolbar, palette, Help and tooltips (#496). Lithuanian adds a 2,578-entry catalog and date names, reviewed by a native-speaking contributor. Validated candidate (macOS raster probe; Windows runtime unverified): the Windows policy changes only embedded JetBrains Mono; horizontal alpha-edge energy per ink improves 2.78%, 3.87% and 1.59% at 100%, 125% and 150%, and is down 0.79% at 200%. Inter is pixel-identical; layout hashes, glyph coverage and the 8 MiB atlas are unchanged. Timing and RSS ranges overlap, with no improvement claimed. Estimates unchanged. Missing: a right-to-left layout, a Chinese UI face in releases, Hindi/Indonesian/Korean/Vietnamese, file-backed input and global memory budgets, performance budgets, keyboard-only operation |
|
||||
|
|
@ -158,6 +161,7 @@ M2 (renderer), M7 (content editing), M10 (OCR and Office), M11 (Preflight and st
|
|||
|
||||
| Date | Change | Summary |
|
||||
|---|---|---|
|
||||
| 2026-10-12 | minor | M13: CLI check directory identity, with overlap and symlink regressions; estimates unchanged |
|
||||
| 2026-10-12 | minor | M4: shared Type 3 glyph resources survive extraction, with save/reopen and independent raster regressions; estimates unchanged |
|
||||
| 2026-10-12 | minor | M2: packed TIFF-predictor RGB row boundaries, with decoding and page-render regressions; estimates unchanged |
|
||||
| 2026-10-12 | minor | M3/#1364 candidate: absolute local GoToR links require desktop consent or headless allow_absolute; existing relative navigation and validation retained. Estimates unchanged. |
|
||||
|
|
|
|||
Loading…
Reference in New Issue