Skip to content

Optional mineru extra, pymupdf backend, and a Pages/Read the Docs site (issues #2, #3, #4) - #5

Merged
mgraffg merged 3 commits into
mainfrom
issues-2-3-4
Sep 14, 2026
Merged

mgraffg merged 3 commits into
mainfrom
issues-2-3-4

Conversation

@mgraffg

@mgraffg mgraffg commented Sep 13, 2026

Copy link
Copy Markdown
Member

Implements issues #2, #3 and #4 in that order (#3 builds on #2, #4 builds on
#2 and #3), as one run against main at 9956c0b.

#2 — Make mineru an optional extra and add the backend selector

Makes mineru the document2md[mineru] extra instead of a hard dependency,
and adds the BatchConverter(backend=...) / CLI --backend selector
(auto, mineru, pymupdf), with auto resolving to mineru when its CLI
is on PATH and to pymupdf otherwise. __version__ bumped to 0.4.0.

Commit: 470de5e

#3 — Add a pymupdf backend for PDFs with an embedded text layer

Adds document2md/pymupdf_backend.py, reading a born-digital PDF's own
embedded text layer via pymupdf4llm instead of OCR-ing it. Neither backend
silently falls back to the other: a list of page images, or a PDF without
enough of an embedded text layer, under pymupdf raises RuntimeError
naming the mineru extra. Imports are lazy inside functions so the docs job
(installed with --no-deps) can still autodoc the module.

Commit: e612ad7

#4 — Publish a Quarto user site on Pages and developer docs on Read the Docs

Adds a Quarto user site under website/ (install, CLI, Python, backends;
published to ingeotec.github.io/document2md via website.yml on push to
main) and a Sphinx developer site under docs/ (architecture, backends,
development, full API reference; published to
document2md.readthedocs.io). The two sites have two audiences and no
overlapping page.

Commit: ace1751

Verification run on this branch before opening this PR

  • python -m pytest tests — 90 passed
  • python -m pytest dof2md/tests — 2 passed
  • python scripts/check_package_versions.pydocument2md 0.3.0 (pypi) → 0.4.0 (local): OK; dof2md: OK: tombstone: final release, not maintained
  • python -m sphinx -n -W --keep-going -b html docs/source docs/build/html
    — build succeeded, no warnings
  • python -m sphinx -b doctest docs/source docs/build/doctest — 4 tests,
    0 failures
  • quarto render website (Quarto 1.9.38) — all 5 pages rendered,
    _site/index.html created

Follow-ups that are not this PR's job

  • Read the Docs: nothing in this repository can import the project at
    readthedocs.org — a human has to do that once. Until then
    document2md.readthedocs.io answers 404.
  • GitHub Pages: website.yml's render job runs (and passes) on this
    pull request, but the publish job — and therefore the gh-pages branch
    and the live site at ingeotec.github.io/document2md — only exists after
    the first push to main, i.e. after this PR merges. Check the site after
    merging.
  • LegalIA's website/pages/document2md.ipynb template still links to the
    old legalia.readthedocs.io API page rather than
    document2md.readthedocs.io; worth a follow-up fix in LegalIA once Read
    the Docs is imported.
  • nota2md's ocr extra needs to require document2md[mineru]>=0.4.0
    (this release) before nota2md's next release, or
    pip install nota2md[ocr] breaks for everyone outside the LegalIA repo.

Closes #2
Closes #3
Closes #4

🤖 Generated with Claude Code

mgraffg and others added 3 commits September 13, 2026 17:45
Moves mineru[pipeline] from a hard dependency to the document2md[mineru]
extra, and gives BatchConverter/the CLI a backend= / --backend selector
(auto, mineru) so the follow-up text-layer backend has a seam to plug
into. Resolution and the missing-dependency check now happen in
BatchConverter.__enter__, before any document is converted, and the
CLI turns that RuntimeError into a clean exit message instead of a
traceback. Bumps __version__ to 0.4.0 and adds a document2md-light CI
job that installs with no mineru extra.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds document2md/pymupdf_backend.py, reading a born-digital PDF's
embedded text layer with pymupdf4llm instead of running mineru's OCR.
BatchConverter/CLI backend selector grows a "pymupdf" choice; "auto"
now resolves to mineru when its CLI is on PATH, otherwise pymupdf,
keeping mineru as the reference backend where installed. Neither
backend silently falls back to the other: a list of page images or a
PDF without enough of an embedded text layer under "pymupdf" raises
RuntimeError naming the mineru extra. pymupdf4llm joins the core
dependencies (its wheels are small, unlike mineru[pipeline]).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…cs (issue #4)

Adds website/, a Quarto site in LegalIA's house style (litera theme,
style.scss copied verbatim, same navbar/footer shape) covering install,
CLI, Python and backend guidance for users, published to GitHub Pages
by .github/workflows/website.yml (render on PRs, publish to gh-pages on
push to main). Splits docs/source/index.rst into index, architecture,
backends and development pages plus one API reference page per module
under api/, so Read the Docs becomes the developer site: architecture
walkthrough, how backend resolution/dispatch works and how to add a
backend, repository conventions, and the full private+public API
reference. Usage examples move to the Pages site, so Read the Docs
doctest keeps only cutter's real example.

Also: docs/Makefile, conf.py's dangling templates_path removed and
html_title added, test.yml's docs-doctest job renamed to docs and
gated by a strict Sphinx HTML build, tests/test_docs.py guarding every
document2md/*.py module against zero or duplicate automodule pages,
README's Documentation section, CLAUDE.md's Layout/Commands/Publishing
updated for both sites, and the repository homepage set to the Pages
URL.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mgraffg
mgraffg merged commit cc6f165 into main Sep 14, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant