Parsing & extraction
How does a document become usable evidence?
Parsing converts a page or file into usable text and structure. It may recover headings, tables, reading order, and text from scanned images. The aim is to preserve meaning and source locations so downstream search can find a passage and the answer can point back to it.
HomeArchiveSubscribe
Field notesMeasuring the river
The team measured water depth at three sites every morning for seven days.
SiteMean depthNorth bank1.2 m
Newsletter signup
Original text and table, surrounded by page furniture
A useful parser separates the document’s evidence from navigation and other page furniture.
What to understand
- Fetching obtains the file; parsing interprets its contents. A successful download does not guarantee a useful extraction.
- Test difficult layouts directly. Columns, table headers, footnotes, and scanned pages can change the meaning when extracted incorrectly.
- Keep document identifiers and available page or section metadata alongside extracted text so a result remains traceable.
Go to the source
Primary documentation for the ideas in this explainer.
Follow the next part of the system.
Chunking