Playbook · AI search and GEO

Updated

What is llms.txt, and does your site need one?

llms.txt is a Markdown file at the root of your site that gives AI models a short summary and a list of links to your most useful pages. It is cheap to add and does no harm, but no major AI provider’s crawler docs say they read it, and Google says Search ignores it.

Proposed by Jeremy Howard on 3 September 2024. An H1 with the site or project name is the only required section.
Source: llmstxt.org, modified 10 Aug 2026

How this site makes its own

My /llms.txt is not a file I edit. It is built from the same data as the pages: the prices and every offer, then every live service page and guide with its description, grouped by topic. Drafts are left out, so it never links a page that does not exist. Terms, refunds and privacy go under the Optional heading the proposal suggests.

Because it is generated, it changes the moment a price or a page changes. A hand-written llms.txt goes out of date the first time someone forgets it.

What the llms.txt proposal says

  • The file lives at /llms.txt, or under a path to cover only that part of a site.
  • An H1 with the name of the site. This is the only required part, so the bar is low.
  • A blockquote with a short summary.
  • Optional paragraphs or lists with more detail, without headings.
  • H2 sections that list links in Markdown, each with an optional note after a colon.
  • A section called Optional for secondary links an AI can skip when it needs less text.
  • Pages can also offer a clean Markdown copy at the same URL with .md added.

Who actually reads llms.txt

Here the evidence gets thin. OpenAI and Perplexity publish an llms.txt for their own developer docs, and llmstxt.org says Anthropic and Google’s Gemini docs do too. Publishing one is not the same as reading yours.

I opened the crawler pages of OpenAI, Anthropic and Perplexity again on 17 November 2026. None of them says their bots read llms.txt on other people’s sites. Google is the most direct: its AI optimization guide says you do not need machine-readable or AI text files to appear in Google Search, and that keeping them for other services will neither harm nor help, because Google Search ignores them.

One tool does check for it. llmstxt.org says Chrome’s Lighthouse audits sites for an llms.txt in its agentic browsing checks, which test how well AI agents can use a site.

When an llms.txt is worth it

  • Developer docs and APIs, where people paste your docs into a coding assistant.
  • Sites with many pages, where a short map helps an agent find the right one.
  • When it can be generated, so nobody has to remember it exists.

When not to spend time on it

If your site cannot be crawled, renders its content only with JavaScript, or blocks AI search bots in robots.txt, fix those first. They decide whether assistants can see you at all, and no text file will talk a bot past a firewall.

To make one by hand, the free llms.txt generator at /tools/llms-txt-generator builds a file in the format above.

Read next

Sources

Start

Tell me where the product is stuck.

In 30 minutes we look at where things stand, what you need from a CTO right now, and which option fits. Either way, you leave with a clear next step.

How the call goes

  1. 01You tell me about the product, the team and the next 6 months.
  2. 02I ask about the code, the stack and what keeps breaking.
  3. 03We pick the option that fits, or agree that none does.

09:00 to 20:00 CET

Founding rate: the first 3 clients get 20% off, in return for a named testimonial and a case study. On a monthly plan, the 20% applies to the first month.