Chunking

Before a RAG system can retrieve your content, it has to split it into chunks: passages typically a paragraph or a few hundred tokens long, each embedded and indexed independently. When a query comes in, vector search matches against these chunks, not whole documents. The chunk is the real unit of retrieval.

This has a direct editorial consequence that most writers miss. A chunk is retrieved and quoted on its own, stripped of the surrounding page. A paragraph that depends on the previous three paragraphs for context retrieves poorly and quotes worse, because the engine pulls it in isolation and finds it incomplete. A paragraph built around one self-contained, specific claim retrieves cleanly and survives extraction.

The practical rule that falls out of chunking: structure paragraphs around discrete claims, make each one stand on its own, and put the citable sentence where it does not need the rest of the page to make sense. This is the mechanical reason behind the self-contained factual sentence pattern. Long-context engines like Moonshot Kimi partly relax this by ingesting whole documents, but for the majority of engines that chunk, paragraph-level self-containment is what gets you retrieved.