Learn/What is llms.txt?
What is llms.txt?
Last updated 2026-10-15
llms.txt is a markdown file served at the root of your domain (https://your-domain.com/llms.txt) that gives AI agents a curated, structured summary of your site: what it is, what is on it, and which pages matter most. It follows a convention described at llmstxt.org: one H1 title, a blockquote summary, then H2 sections containing markdown links.
Why llms.txt exists
AI agents do not browse your site the way a person does. They fetch pages, keep only what fits in a limited context window, and guess at what matters. A homepage clogged with navigation, cookie banners, and marketing copy wastes that budget. llms.txt hands the agent a clean table of contents so its first fetch already points at your best pages.
The format
The file is plain markdown with three required parts. First, an H1 line with the name of your project or site. Second, a blockquote (a line starting with >) summarizing what the site is in one paragraph. Then any number of H2 sections, each containing markdown links with optional descriptions after a colon. The last section is conventionally called "Optional" for material agents can skip.
Where it goes
Serve it at /llms.txt on your apex domain. No special headers are required — text/plain is fine. A companion file, /llms-full.txt, can hold the full text of your documentation in one file for agents with larger context windows.
How to write a good one
Lead with what the site does, not who you are. Link to the pages an agent would actually need: docs, pricing, API reference, contact. Write the link descriptions as if the agent will never click through — because sometimes it will not. Keep the whole file under a few kilobytes; it is a map, not a dump.
FAQ
Is llms.txt a standard?
It is a widely adopted convention rather than an official web standard. The format is documented at llmstxt.org, and major AI products increasingly check for the file when crawling.
Does llms.txt replace robots.txt?
No. robots.txt controls what crawlers may fetch. llms.txt describes what your content means and where the important pages are. Sites that want to be agent-friendly publish both.
Does adding llms.txt affect SEO?
It does not replace SEO work, but it helps AI search and answer engines cite your pages correctly by giving them a clear summary and structure.