feat(core): ingest docx, pptx, and csv into document sidecar notes with markitdown

[phernandez]

Summary

Extend document ingestion beyond PDF so a .docx, .pptx, or .csv dropped into a project gets the same sidecar Markdown note a PDF gets today (report.docx.md next to report.docx), using microsoft/markitdown for the Office formats and the stdlib for CSV. PDF stays on pdf-inspector.

Evaluation (2026-09-07, markitdown 0.1.7)

PDF head-to-head on an 87-page two-column paper:

	pdf-inspector 1.17	markitdown 0.1.7
Time	0.36s	5.55s
Page markers	87	0
Title	extracted	none
Text	clean headings, spaced words	words glued together, title mangled into a fake table

markitdown's PDF path is pdfminer plus a word-position table heuristic. It has no OCR detection, no confidence, and no page markers, so the page citations from #1490 would have nothing to anchor to. PDF stays on pdf-inspector.

Real files (24 from a personal iCloud vault, structure only):

Type	Result
docx	Good for letters and prose. Form-style layout tables collapse into stray pipes.
pptx	Good. <!-- Slide number: N --> markers, speaker notes, tables. Images become dangling  references.
xlsx / xls	Noisy. Title rows produce Unnamed: N headers and NaN in every empty cell (pandas header=0). Deferred.
csv	Good tables. One UTF-8 file came out as mojibake from markitdown's charset guess. Do it ourselves.
html	Good, title extracted. Not in scope yet.
json	Crashes: magika sniffs the prefix as ASCII, then IpynbConverter.accepts() decodes the whole stream and raises UnicodeDecodeError, which escapes the conversion loop.
rtf	Falls through to PlainTextConverter and dumps raw RTF markup.
odt, doc, pages, numbers, key	Unsupported.

Design

Dispatch on the media type we already index and call the specific markitdown converter directly. Never use the sniffing MarkItDown.convert() loop. That avoids magika, the requests.Session, ZipConverter recursion, the URL converters, and the json/rtf/charset failures above.

• Extractor protocol. RawPdfDocumentRuntime.extractor is typed as the concrete PdfInspector. Introduce a neutral extraction result and a DocumentExtractor protocol keyed by media type; pdf-inspector becomes one implementation, markitdown (docx, pptx) another, CSV a stdlib one.
• Bounded worker. Run markitdown in the same rlimit-capped, byte-capped, killable subprocess pattern as pdf_inspector_worker.py. markitdown is synchronous, in-process, and unbounded; its README tells hosted users to sanitize inputs themselves.
• Run identity. Engine microsoft/markitdown plus the installed version from importlib.metadata, one profile per converter. Upgrades yield new runs rather than rewriting provenance, same as pdf-inspector. CSV gets its own engine so a markitdown upgrade does not re-run every CSV.
• CSV. Decode as UTF-8 and fail fast. Header plus the first N rows plus a row count; a 475 KB export became a 7,400-row table with no semantic value.
• Extra. basic-memory[documents] = markitdown[docx,pptx]. New install weight is roughly 40 to 50 MB (lxml, Pillow, mammoth, python-pptx). onnxruntime and numpy are already present via fastembed, so magika adds only its 4 MB model. No pandas.
• Local entry point. bm import document <path> over the project directory: resolve the indexed source entity, extract, build artifacts through the existing contract, write <file>.<ext>.md and the document-ingestion-runs/<run-id>.md note, index them. This is the same local source-reader and writer work still pending for PDF from Parse PDFs into searchable LiteParse sidecar notes #1006.

Contract touch points (schemas/document.py, document_ingestion/raw_document.py)

Already neutral: derive_document_note_path yields x.docx.md; DocumentSourceV1.media_type accepts any MIME type; DocumentMetadataV1.kind is free text; run identity hashes engine, version, profile, and options.

To generalize: the hardcoded ("document", "pdf", "generated") tags and kind="pdf"; the pdf-specific pipeline and profile constants; DocumentExtractionV1 requires page fields (pageless formats pass with page_count=0, which should be documented or made optional); cloud's application/pdf gate becomes the dispatch table. Page citations stay PDF-only by design.

Cloud imports RawPdfDocumentRuntime, build_raw_document_artifacts, and the identity helpers, so the PDF signatures stay intact.

Known rough edges to accept for now

• pptx image references point at internal picture names; strip to alt text.
• docx form-style tables lose structure.
• xlsx waits on a header strategy for title rows.

POC scope

• Neutral extraction result + DocumentExtractor protocol; pdf-inspector adapted, existing tests green
• markitdown bounded worker for docx and pptx behind basic-memory[documents]
• stdlib CSV extractor with row cap
• bm import document <path> local runtime (source reader, sidecar writer, run note)
• Tests for mapping, CSV rendering, and worker dispatch

[claude]

Claude finished @phernandez's task in 35s —— View job

---

Triage

Type: Enhancement — extends the existing document-ingestion pipeline (introduced for PDF in #1006) to .docx, .pptx, and .csv via markitdown for Office formats and a stdlib extractor for CSV.

Component: Core / document ingestion (document_ingestion/, schemas/document.py) plus a new local CLI entry point (bm import document <path>). Not cloud-specific — the design deliberately keeps RawPdfDocumentRuntime, build_raw_document_artifacts, and the identity helpers stable so cloud's existing imports of them don't break, but no cloud-only behavior is being added here.

Priority: Medium — this is a capability extension, not a regression or breakage; no user-facing functionality is currently broken.

Complexity: Complex — the design spans several non-trivial pieces: a new DocumentExtractor protocol to de-couple RawPdfDocumentRuntime from the concrete PdfInspector, a bounded/killable subprocess worker for markitdown (mirroring pdf_inspector_worker.py's rlimit/byte-cap/kill pattern), per-converter run-identity/versioning via importlib.metadata, a stdlib CSV extractor with row capping, and the still-pending local source-reader/sidecar-writer runtime for bm import document (shared groundwork with #1006). Each POC checkbox is realistically its own PR-sized unit of work.

Notes:

• The issue includes a well-researched evaluation (markitdown 0.1.7 vs pdf-inspector, real-file survey across docx/pptx/xlsx/csv/html/json/rtf) and a clear design already, including named rough edges (pptx image refs, docx form tables, xlsx header strategy deferred). This looks ready for implementation planning/PR breakdown rather than needing more clarifying questions.
• Related, not duplicate: Parse PDFs into searchable LiteParse sidecar notes #1006 (PDF sidecar note parsing, referenced as the pattern being extended and the source of the pending local source-reader/writer work) and Optional source locator on document observations (page → context, resolvable against the extraction page map) #1366 (page-anchored source locators, referenced as the reason PDF stays on pdf-inspector rather than markitdown).

[phernandez]

POC is on branch feat/1512-document-ingestion-office (one signed commit on top of main).

What it does

• ExtractedDocument + DocumentExtractor protocol in raw_document.py; RawDocumentRuntime dispatches on the indexed media type. RawPdfDocumentRuntime and build_raw_document_artifacts keep their signatures and delegate, so Cloud's wiring is untouched.
• bounded_process.py holds the killable, byte-capped child-process mechanics extracted from the pdf-inspector adapter. markitdown_extractor.py / markitdown_worker.py reuse them and call DocxConverter / PptxConverter directly (no sniffing loop, no magika, no plain-text fallthrough).
• csv_extractor.py renders a stdlib preview (header, first 200 rows, row count) with strict UTF-8.
• local_runtime.py is the project-directory runtime: API-backed source resolver, checksum-as-generation reader, sidecar + run-note writer that refuses to overwrite hand-written or enriched notes and no-ops on an identical raw projection.
• bm import document <path> [--project] indexes the project, then runs the pipeline. PDF works through the same command, which closes the local bm import pdf gap from Parse PDFs into searchable LiteParse sidecar notes #1006.
• basic-memory[documents] extra = markitdown[docx,pptx].

Verified

• just fast-check clean; 108 tests in tests/document_ingestion + tests/cli/test_import_document.py pass with 100% line coverage on every new module.
• End to end in an isolated home: docx, pptx, csv, and an 87-page PDF each produced <file>.<ext>.md plus document-ingestion-runs/<run-id>.md, both indexed and found by bm tool search-notes. A second run reported both notes as already current.

Known gaps / follow-ups

• The import_document() body (project resolution + wiring) is only covered by the manual e2e run; the command wrapper has unit tests.
• The indexer appends a permalink to the sidecar after the write, so the run note's output.raw.checksum records the pre-permalink bytes. Derived state; converges on the next run.
• The pdf-inspector worker still carries its own rlimit helpers; the markitdown worker duplicates them (tests monkeypatch the pdf ones by name). Shared worker_limits module is a small follow-up.
• pptx image refs stay as ![alt](PictureN.jpg); docx form-style tables lose structure; xlsx deferred.