Glossary

What is llms.txt?

llms.txt is a proposed plain-text Markdown file at a site's root (/llms.txt) that gives language models a short summary of the site and a curated list of links to its most useful pages. It's an invitation about what to read, not an access rule. Major AI companies haven't committed to reading it.

By the AgentLaunch team · Updated November 2026 · Also called llms.txt file, llms-full.txt

The format

The proposal at llmstxt.org describes a Markdown file with:

  1. An H1 with the site or project name.
  2. A blockquote with a short summary.
  3. Optional paragraphs with more context.
  4. H2 sections listing links, each with a short note.
  5. An optional "Optional" section for links that can be skipped.

A companion file, llms-full.txt, includes the full text of the key pages in one file, so a model can read everything without following links.

llms.txt vs robots.txt vs sitemap

File Purpose Audience
robots.txt Allow or block crawling All crawlers
sitemap.xml List every URL to index Search engines
llms.txt Curated summary and key links Language models and agents

Does anyone use it?

Many developer-focused companies publish one, including Anthropic, Stripe and Cloudflare, mostly for their docs, so coding agents can load the right pages. Google's Search team has said Google doesn't use llms.txt for Search or AI Overviews. Other AI companies haven't said they read it during crawling, though agents and tools sometimes fetch it directly when pointed at a site. Treat it as cheap, useful documentation for agents rather than a ranking lever.

How to write one

Keep it short and factual: what the product is, who it's for, and links to the pages an agent would need, such as pricing, docs, key guides and a FAQ. Use the same facts as everywhere else. Our own llms.txt lists our main pages, the directory profiles and this glossary.

Common mistake

Treating it like robots.txt. Adding "Disallow" lines or trying to block AI crawlers in llms.txt does nothing. Access is controlled in robots.txt and at your server or CDN.

Example

A developer tool publishes llms.txt with links to its quick start, API reference and pricing, plus llms-full.txt with the docs. Developers who point their coding agent at the site get correct setup steps instead of an outdated blog post.

FAQ

Does Google use llms.txt?

Google has said it doesn't use llms.txt for Search or AI Overviews.

Is llms.txt required for AI visibility?

No. Crawlability, content and mentions matter far more.

Where does llms.txt go?

At the root of your domain: https://yoursite.com/llms.txt.

What is llms-full.txt?

A companion file with the full text of your key pages in one Markdown file.

Sources

  1. llms.txt proposal (llmstxt.org)
  2. Anthropic docs llms.txt
  3. Google Search Central: AI features and your website

Data

All glossary terms · Search and AI search terms