llms-full.txt

Where llms.txt is an index (a map of authoritative URLs for an engine to crawl individually), llms-full.txt is the corpus itself: the full Markdown content of priority pages concatenated into one file. The idea: instead of an engine making N HTTP requests to crawl your site, it makes one request and ingests your entire authoritative corpus.

The use case is asymmetric. Most production AI engines today still crawl page-by-page; they don't preferentially consume llms-full.txt. But the cost of publishing one is low (a build step that concatenates Markdown), and the file becomes useful for: (1) AI engines that may adopt the pattern as it matures, (2) RAG ingestion by enterprise customers and partners who want to load your content into their own knowledge bases, and (3) AI agents that prefer single-fetch corpus loading over multi-request crawling.

Practical considerations: keep llms-full.txt under a few megabytes (some engines reject larger files); update it on every content change (typically via the same build pipeline that produces the static site); structure each section with clear URL, title, and last-updated metadata so an ingesting engine can attribute correctly. Most leading content publishers (Stripe, Anthropic, Cloudflare, Vercel) now ship both files; the cost is small and the asymmetric upside is real.