fix(pdf): pad table rows to prevent silent cell truncation in PDF tables#2241
Open
hsusul wants to merge 1 commit into
Open
fix(pdf): pad table rows to prevent silent cell truncation in PDF tables#2241hsusul wants to merge 1 commit into
hsusul wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_to_markdown_tableto match the maximum row length in PDF table extraction.Problem
When converting PDF tables to Markdown tables using
_to_markdown_table, column widths and row formatting were calculated usingzip(*table). Because Python's built-inzip()truncates iteration at the length of the shortest row in the sequence, any table row containing more columns than the shortest row would have its extra cells silently dropped and discarded from the resulting Markdown table output.Root cause
_to_markdown_tablecomputedcol_widths = [max(...) for col in zip(*table)]directly on raw input rows without ensuring all rows had uniform length. Whenzip(*table)ran on a 2D list with unequal row lengths, it stopped at the shortest row, reducingcol_widthsto that shorter column count. Subsequent calls tozip(row, col_widths)insidefmt_rowthen truncated all cells beyond that index.Fix
Pad every row in
tablewith empty strings up tomax_cols = max(len(row) for row in table)prior to calculatingcol_widthsand building table lines. This ensures all columns across all rows are preserved in the Markdown table layout.Testing
pytest tests/test_pdf_tables.py— PASSED (23/23 tests pass, including new regression test)black --check src/markitdown/converters/_pdf_converter.py tests/test_pdf_tables.py— PASSEDgit diff --check— PASSED (no whitespace or formatting issues)Compatibility