Chunking
How much context belongs in one result?
Chunking divides extracted content into units that can be indexed and retrieved. Small chunks can isolate a precise fact but lose surrounding context; large chunks retain context but may mix unrelated material. Boundaries, overlap, and document structure influence which evidence reaches the agent.
Theteammeasuredwaterdepthatthreesiteseverymorningforsevendaysbeforecomparingtheresults.
1
The team measured water depth at three sites
2
every morning for seven days before comparing the
3
results.
Fixed boundaries are simple, but can split a thought. This word-based illustration uses a fictional sentence, not a tokenizer.
What to understand
- Prefer meaningful boundaries where practical. Keep a heading with its section and preserve the context needed to interpret a table.
- Overlap repeats material across neighboring chunks. It can protect boundary context, but also adds storage and duplicate retrieved text.
- Choose chunk sizes using representative questions and document types. There is no universally best token count.
Go to the source
Primary documentation for the ideas in this explainer.
Follow the next part of the system.
Embeddings