TL;DR
- llms.txt is a proposed convention — a Markdown file at your site root (/llms.txt) that points AI systems to your most important, canonical content.
- It is not robots.txt: robots.txt controls crawler access, while llms.txt curates and summarizes content for large language models.
- The format is plain Markdown: an H1 with your site name, an optional summary, then sections of curated links with short descriptions.
- Adoption is voluntary and not yet universally honoured, so treat llms.txt as a low-cost guide, not a guarantee.
- Keep it accurate and current — a stale llms.txt is worse than none because it misdirects models to outdated pages.
llms.txt is one of the newest ideas in AI visibility, and one of the most misunderstood. It is not a magic switch that gets you cited, and it is not a replacement for robots.txt. It is a curated map of your best content, written in Markdown, designed to make a model's job easier. Here is what it is and how to write one well.
What is llms.txt?
llms.txt is a proposed convention for a Markdown file placed at the root of your site (at /llms.txt) that points AI systems to your most important and canonical content. The idea is to give models a clean, curated entry point — your key pages, summarized — instead of forcing them to infer your site's structure from raw HTML. It is a guide, authored by you, to what matters on your site.
How is llms.txt different from robots.txt?
robots.txt controls whether crawlers may access your pages; llms.txt curates and summarizes content for language models. They solve different problems and complement each other rather than overlap:
- robots.txt — an access-control file: it tells crawlers which paths they may or may not fetch.
- llms.txt — a curation file: it tells AI systems which pages matter and what they contain, in Markdown.
- robots.txt is widely honoured and long-established; llms.txt is an emerging convention with partial, voluntary adoption.
- You can and often should have both — robots.txt to manage access, llms.txt to guide understanding.
llms.txt won't get you cited by itself
Adding the file does not guarantee inclusion in any AI answer, and not every system reads it yet. Think of llms.txt as helpful signposting that complements crawlable, well-structured content — not a shortcut around it. The content still has to be good.
What does an llms.txt file look like?
It is plain Markdown with a simple, predictable shape: a title, an optional summary, and curated link sections. The structure is intentionally easy for a model to parse:
- An H1 heading with your project or site name.
- An optional short blockquote or paragraph summarizing what your site is and who it's for.
- H2 sections grouping related pages (e.g. Docs, Guides, Product).
- Under each section, a Markdown list of links — each as [Title](URL): one-line description.
- Optionally, an 'Optional' section for secondary links a model can skip when context is limited.
How do you write a good one?
Curate ruthlessly and describe accurately. The value of llms.txt is editorial — you are choosing the handful of pages that best represent your site and summarizing each so a model knows what it'll find. Link to canonical URLs only, write honest one-line descriptions, and order sections by importance. Keep it short enough to be useful and current enough to be trusted.
- Link only to canonical, indexable URLs — never to redirects or staging pages.
- Write a one-line description per link that accurately states what the page covers.
- Group logically (product, docs, pricing, guides) and lead with what matters most.
- Review it on a schedule so it never points to removed or outdated pages.
llms.txt is editorial, not technical. Its power isn't in the syntax — it's in the discipline of deciding, in your own words, what on your site is actually worth a model's attention.
What is llms.txt?
llms.txt is a proposed convention for a Markdown file at your site root (/llms.txt) that points AI systems to your most important, canonical content. It gives models a curated, summarized entry point to your site rather than making them infer structure from raw HTML. Adoption is voluntary and not yet universal.
Is llms.txt the same as robots.txt?
No. robots.txt is an access-control file that tells crawlers which paths they may fetch, while llms.txt is a curation file that tells AI systems which pages matter and what they contain. They complement each other — robots.txt manages access, llms.txt guides understanding — and a site can have both.
Will adding llms.txt get me cited by AI?
Not on its own. llms.txt is helpful signposting that complements crawlable, well-structured, specific content, but it does not guarantee inclusion in any AI answer, and not every system reads it yet. Treat it as a low-cost guide layered on top of genuinely citable content, not a shortcut.