Document parsing for RAG: PDFs, tables and scanned pages

Why parsing decides RAG quality

Most RAG tutorials start with clean text files. Real company documents are PDFs exported from Word, scanned contracts, slide decks, spreadsheets, emails with attachments, and web pages full of navigation menus.

If a table of prices comes out as a stream of numbers with no column names, no embedding model and no LLM can recover which price belongs to which product. If a two-column page is read straight across, sentences from different columns are mixed together. If a scanned contract has no text layer, the document simply doesn't exist for your search.

These failures are silent. The pipeline runs, the vector database fills up, and the answers are wrong for reasons nobody sees until someone reads the parsed text.

What makes documents hard

• Layout: multiple columns, sidebars, headers and footers on every page, footnotes.
• Tables: rows and columns that lose their meaning when flattened into text, merged cells, tables split across pages.
• Scans and images: pages with no text layer, photos of documents, handwriting.
• Visual content: charts, diagrams and architecture drawings where the meaning is in the picture.
• Structure: headings, numbered clauses and nested lists that tell you what a paragraph belongs to.
• Noise: page numbers, repeated disclaimers, cookie banners and navigation text.

Kinds of parsers

Basic text extractors

Libraries like pypdf, PyMuPDF and pdfplumber read the text layer of a PDF. They are fast and free, and they work well for simple, single-column, text-based documents. pdfplumber is also good at extracting tables from PDFs with clear lines.

Layout-aware document parsers

Tools like Docling, Unstructured and Marker detect the structure of a page, headings, paragraphs, lists, tables and figures, and output clean Markdown or structured elements. They are slower, and much better for complex documents. Markdown output is especially useful for RAG, because it keeps headings and tables in a form LLMs read well.

OCR

For scanned pages, OCR (optical character recognition) converts images of text into text. Tesseract is the common open-source engine, and cloud services from AWS, Google and Azure handle forms, tables and handwriting better.

Vision language models

Multimodal models can read a page image directly and return structured text, including tables and descriptions of charts. They handle messy layouts better than rules-based tools, at a higher cost per page, and like any model, they can make mistakes, so check them on your documents. See multimodal models.

Handling tables

Tables need special care, because their meaning comes from the relationship between rows and columns.

• Convert tables to Markdown or HTML, so each value stays with its column header.
• Keep a table and its caption together in one chunk, and don't split tables in the middle if you can avoid it.
• For large tables, store each row as a sentence that repeats the headers ("Product: X200, Price: 49999, Warranty: 2 years"), so a single row can be retrieved and understood on its own.
• For data people will filter or calculate on, put it in a database and let the model query it with a tool, instead of retrieving it as text.

Handling images and diagrams

Text extraction ignores images. If your documents contain diagrams, charts or screenshots that matter, generate a text description of each one with a vision model at ingestion time, and index the description alongside the text, with a link back to the image.

Keep structure as metadata

While parsing, capture the information that later makes retrieval and citations work:

• Document title, file name and source location.
• Page number for every piece of text.
• The heading path ("Leave policy > Sick leave > Documentation required").
• Document date and version.
• Element type: paragraph, table, list, figure description.

You will use these for chunking, filtering, ranking fresher documents higher, and showing users exactly where an answer came from.

Clean the text

• Remove repeated headers and footers, page numbers and boilerplate.
• Fix broken words from hyphenation at line ends.
• Normalise whitespace and Unicode characters.
• Remove navigation, menus and cookie notices from web pages.
• Detect and drop duplicate documents and near-duplicates, like five versions of the same policy.

Check parsing quality

Before building retrieval, look at the output.

1. Pick 10 to 20 representative documents, including the hardest ones: scanned pages, big tables, multi-column layouts.
2. Parse them and read the output next to the original.
3. Check tables, reading order, headings and whether any page came out empty.
4. Compare two or three parsers on the same sample and pick per document type if needed.

It takes an afternoon, and it prevents weeks of debugging retrieval problems that were really parsing problems.

Choosing a parser

Documents	Good starting point
Simple text PDFs, Word files	pypdf or PyMuPDF, python-docx
PDFs with tables and complex layout	Docling, Unstructured, Marker
Scanned documents	OCR (Tesseract or a cloud OCR service), or a vision model
Forms, invoices, receipts	Cloud document AI services or a vision model with structured output
Charts and diagrams	A vision model to describe them
Sensitive documents that can't leave your servers	Open-source parsers and local OCR or local vision models

From the floor

In the Build30 discussion on chunking, a builder asked how to handle a collection of educational material that mixed text, PDFs, interview questions and architecture diagrams. Because diagrams carry little text, we talked about keeping one or two images with about 100 words of surrounding text in each chunk, so the model gets the diagram and its explanation together, and an extracted or generated description of the image becomes searchable.

For Neural Vault, the offline RAG app we built in AI Engineer HQ for sensitive documents, every stage had to work without internet, including parsing, which rules out cloud OCR and parsing services and means choosing local tools from the start.

Related

• Chunking strategies
• Multimodal models
• Data for AI
• RAG failure modes

Sources

• Docling, IBM Research
• Unstructured documentation
• PyMuPDF documentation
• pdfplumber
• Tesseract OCR