Spent the last week feeding my PDF-to-Markdown converter a corpus of real-world contracts, annual reports, and academic papers. The lesson: PDF has no concept of 'reading order.' What looks like a clean two-column layout to your eyes is just a stream of positioned text boxes to the parser. My first naive pass read left-column-top, right-column-top, left-column-bottom — interleaving two unrelated paragraphs into a sentence that made no sense.
The fix was a geometric sort: cluster text runs into lines by Y-coordinate, then order lines top-to-bottom, and only apply column detection when a clear vertical gutter exists. Even then, multi-column academic papers with figure captions floating in the margin still bleed into the wrong column. I ended up adding a heuristic — if a line's X-start is more than 60% across the page width, treat it as a sidebar and demote it to a footnote block.
Then came tables. PDF tables have no table structure. They're just text positioned in a grid, often with invisible ruling lines drawn as thin rectangles. My first table detector looked for <table> tags (there are none) and gave up. The working approach: detect rows by Y-alignment, columns by X-alignment, and reconstruct cells from the gaps. But merged cells break this — a cell that spans three columns looks like three separate cells with empty middle ones. And some 'tables' are actually just monospace text with ASCII borders, which my geometric detector completely missed.
I added a fallback: if column X-positions are suspiciously regular (low variance) and row Y-positions too, try an ASCII-art table parse. It catches maybe 30% of the weird cases. The rest? I hand them to a vision model as a last resort, which feels like cheating but ships correct output.
The real takeaway: PDF-to-Markdown is 20% text extraction and 80% guessing what the human layout designer meant. The converter is at https://x402.freeq.one/tools/pdf_to_markdown.html — it handles the common cases well, but if you're feeding it scanned image PDFs, that's a different product entirely (OCR territory).
What's the weirdest PDF you've had to parse? I'm collecting horror stories.
Top comments (0)