Convert article documents to KPUB in the pipeline (retire the dead microwave service)

[rtibbles]

❌ This issue is not open for contribution. Visit Contributing guidelines to learn about the contributing process and how to find suitable issues.

Overview

Retire the defunct microwave service and give article-style documents a first-class path to KPUB (self-contained HTML article). A chef references a source document and it becomes a validated KPUB automatically:

ContentNode(uri="article.docx")   # → validated KPUB in the channel

Complexity: Medium
Target branch: main

Context

• microwave (http://35.185.105.222:8989/unoconv/pdf) is dead.
• It is referenced only in docs/examples/document_conversion.ipynb — no runtime code uses it.
• It documented a manual workflow: convert an office doc to PDF out-of-band, then add the PDF.
• KPUB is already supported end-to-end (KPUBConversionHandler + KPUBMetadataExtractor); no path exists to produce one from a source document.
• The CONVERT stage rewrites the working file's extension, so a handler emitting .kpub feeds the existing KPUBMetadataExtractor — net new code is one handler plus the sanitizer.
• Kolibri's KPUB renderer strips all CSS at runtime; ricecooker's validator only forbids external .css.
• Selector-bearing <style> can leak into Kolibri chrome; element-scoped style= is safe when bounded to an allowlist.
• The Studio rich text editor (TipTap v3) emits exactly one inline style — text-align — on <p>, <h1>–<h3>, <small>, <img>; every other format is a semantic tag.

The Change

• Add a CONVERT-stage handler converting .docx, .odt, .rtf, .md, .markdown → KPUB via pandoc.
• Emit semantic-only HTML: images as sibling files (--extract-media), math as presentation MathML.
• Tighten KPUBConversionHandler into a sanitizer applied to all KPUB (converted and hand-authored).
• Sanitizer: strip every <style> block.
• Sanitizer: on style=, keep only allowlisted properties, drop the rest.
• Sanitizer: strip offending CSS rather than reject the file; keep the existing <script>/.js/.css bans.
• CSS style= allowlist v1 = {text-align, color, background-color}, a ricecooker-local constant. text-align from the Studio RTE; color/background-color for source-doc text and highlight colours. Widening further requires extending the RTE first.
• Change GoogleDriveHandler to export Google Docs as .docx (currently PDF); Google Slides stay PDF.
• Rewrite docs/examples/document_conversion.ipynb around the new workflow; remove the dead service and IP.
• Document pandoc as a system dependency (alongside ffmpeg/poppler) and install it in CI on all supported OSes.
• Render math as MathML (MathJax needs no LaTeX present); a LaTeX <annotation> for source preservation is a verify-then-implement enhancement, not a blocker.

Out of Scope

• Presentations (.pptx/.ppt) and spreadsheets (.xlsx/.xls) — not article content.
• Legacy .doc — pandoc can't read the binary format.
• .epub (native EPUB) and .html/zipped HTML (HTML5) — not claimed by the new handler.
• Widening the allowlist beyond {text-align, color, background-color}.
• The coupled Kolibri renderer change to honor the allowlist instead of stripping all CSS — tracked as a separate Kolibri issue.
• Any hosted or self-hosted conversion service.

Acceptance Criteria

• ContentNode(uri="article.docx") (and .odt, .rtf, .md, .markdown) produces a valid KPUB through the pipeline with no manual pre-conversion.
• Converted output has index.html with a non-empty body and no <script>/.js/.css/<style>.
• Converted images land under media/ and are referenced by relative path.
• Math in a source document appears as presentation MathML (<math>) in the output.
• KPUBConversionHandler strips all <style> blocks from any KPUB (converted and hand-authored).
• style= with text-align, color, or background-color survives sanitization.
• A non-allowlisted style= property (e.g. position) is dropped; a mixed attribute keeps only the allowlisted properties.
• A hand-authored KPUB with disallowed CSS still ingests, with a log line naming what was removed.
• Google Docs downloaded via GoogleDriveHandler export as .docx and convert to KPUB.
• Google Slides still export as PDF.
• pandoc is documented as a system dependency and installed in CI on all supported OSes.
• A clear, actionable error is raised when pandoc is absent.
• docs/examples/document_conversion.ipynb no longer references microwave or the dead IP.
• Tests cover each source format converting to a valid KPUB.
• Tests cover the sanitizer keep/drop cases.
• Tests cover the pandoc-missing error and the GDrive export-format change.

References

• ricecooker/utils/pipeline/convert.py — KPUBConversionHandler, ConversionStageHandler
• ricecooker/utils/pipeline/transfer.py — GoogleDriveHandler.GOOGLE_WORKSPACE_FORMATS
• Studio RTE (allowlist source of truth): contentcuration/.../frontend/shared/views/TipTapEditor/

AI usage

Used Claude Code (Opus) to draft this issue and map the codebase from a live brainstorming dialogue that I directed. Codebase claims (microwave is docs-only; the RTE emits only text-align) were verified against the ricecooker and studio repos.