llms.txt

How it works

llms.txt is a proposed convention: a markdown file at the root of a domain that tells large language models what the site contains and where the authoritative pages are. It is not a permission file. Unlike robots.txt, which grants or denies crawling, llms.txt is a curated map, typically a short site description followed by annotated links grouped by section, optionally paired with an llms-full.txt carrying the full text of the corpus.

When it matters

It matters as a low-cost hedge rather than a proven lever. Adoption by model providers is uneven and no major engine has committed to reading it as a ranking or retrieval input, so treat it as inexpensive insurance and not as a GEO strategy. The measurable benefit today is internal: writing one forces you to state what your site is actually about and which pages are canonical, which is the same clarity work that helps retrieval anyway. See Writing for AI Citations and AI Crawler Setup.

Common misconceptions

• "It controls whether AI can use my content." It does not. Access control is robots.txt, crawler tokens, and your CDN. llms.txt is descriptive.
• "Publishing one improves citation rates." There is no reliable public evidence of that. Anyone selling it as a citation lever is ahead of the data.
• "It replaces a sitemap." A sitemap enumerates everything for crawlers. llms.txt is a curated subset written for a reader that will only look at a handful of pages.