Skip to content

feat: add automatic model fallback chain and page-level resume cache - #36

Open
AhmedCoolProjects wants to merge 4 commits into
MarkPDFdown:masterfrom
AhmedCoolProjects:feat/fallback-models-and-page-cache
Open

feat: add automatic model fallback chain and page-level resume cache#36
AhmedCoolProjects wants to merge 4 commits into
MarkPDFdown:masterfrom
AhmedCoolProjects:feat/fallback-models-and-page-cache

Conversation

@AhmedCoolProjects

Copy link
Copy Markdown

Description

This PR introduces two features to improve robustness and efficiency during PDF-to-Markdown conversions, especially when handling long documents or encountering model rate limits / quota limits:

1. Automatic Model Fallback Chain

  • Added support for fallback_models in LLMClient and the main conversion logic.
  • When an API request fails due to rate limits (HTTP 429), quota exhaustion, or after exhausting retries on the primary model, LLMClient automatically falls back to the next model in the chain.
  • The working fallback model is retained as the active model for subsequent pages.

2. Deterministic Page-Level Checkpoint & Resume Cache

  • Added SHA-256 document hashing and page-level markdown checkpointing in src/markpdfdown/main.py.
  • When resume=True and a cache_dir is configured (or via CLI / env), previously completed pages are reused directly from disk.
  • If a run fails or is interrupted midway on Model A, users can immediately resume with Model B without re-processing or re-spending tokens on already completed pages.

3. CLI and Configuration Updates

  • Added CLI arguments:
    • --model / -m: Primary model override.
    • --fallback-models: Comma-separated list of fallback models.
    • --cache-dir: Cache directory path.
    • --no-resume: Flag to disable cache reuse.
  • Added environment variable support: FALLBACK_MODELS, CACHE_DIR, RESUME.
  • Updated .env.sample and README.md documentation.

4. Tests and Verification

  • Added unit tests covering:
    • Fallback model switching and multi-model failure handling in tests/test_llm_client.py.
    • Cache reuse and checkpoint generation in tests/test_main.py.
    • SHA-256 hashing in tests/test_utils.py.
    • New CLI arguments and parameter parsing in tests/test_cli.py.
    • Config parsing from environment variables in tests/test_config.py.
  • All 124 tests pass with full pre-commit validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant