If you have read anything about AI search this year, you have probably seen someone claim that every website needs an llms.txt file. Others say it does nothing at all. Both camps are partly right, and the truth matters if you are deciding where to spend your marketing team’s time.

Here is a plain-English explanation of what llms.txt is, who actually uses it, what Google has said about it, and how to add one in a few minutes if you decide it is worth it.

What llms.txt is

llms.txt is a proposed convention for helping large language models and AI agents understand a website. The idea was published by Jeremy Howard on September 3, 2024, at llmstxt.org, and the proposal has been revised since.

The file is written in Markdown and lives at a predictable address, usually yoursite.com/llms.txt. It gives an AI tool a short, clean summary of your site and a curated list of links to your most important pages, ideally pointing at clean Markdown versions of those pages.

Why would AI tools need that? Web pages are full of navigation menus, cookie banners, scripts and ads. When an AI agent reads a page to answer a question, all that clutter wastes its limited context window. A short Markdown file that says “here is who we are, and here are the pages that matter” is easier to read. If you want background on context windows, our explainer on AI tokens and context windows covers it.

How llms.txt differs from robots.txt and sitemaps

People often confuse these 3 files. They do very different jobs.

The key point: llms.txt does not block or allow anything. If you want to control AI crawlers, use robots.txt. OpenAI, for example, documents separate crawlers in its bots documentation: OAI-SearchBot for surfacing sites in ChatGPT search, GPTBot for collecting content that may be used for model training, and ChatGPT-User for certain user-initiated actions. You manage those in robots.txt, not llms.txt.

What goes in the file

The format in the proposal is simple. In order:

1. An H1 with your site or project name. This is the only required part.
2. A blockquote summary of what the site is and who it serves.
3. Optional detail paragraphs with anything an AI tool should know, such as product names or key terms.
4. H2 sections with lists of links, each written as [Page name](URL): short note.
5. An “Optional” section for secondary links an AI tool can skip if it needs a shorter context.

Here is a small example for a local accounting firm:

```

Example Accounting Co.

Example Accounting Co. is a 6-person accounting firm in Austin, Texas, serving small businesses and self-employed clients with tax prep, bookkeeping and payroll.

We do not offer investment advice or legal services.

Services

• Small business tax prep: Pricing, deadlines and what to bring
• Monthly bookkeeping: Plans and software we support

Help

• Client FAQ: Common questions about engagement and fees

Optional

• About the team ```

The proposal also suggests offering a clean Markdown version of important pages at the same URL with .md added, for example /pricing.md or /pricing.html.md. Some documentation sites also publish a much longer llms-full.txt that contains the full text of their docs in one file. That longer file is a common practice among documentation platforms rather than part of the core proposal.

Who actually uses llms.txt

Adoption is strongest among developer documentation sites, because developers increasingly ask AI coding assistants to read docs for them. As of October 2026, examples include:

• Anthropic’s developer documentation, which publishes an llms.txt index and a longer llms-full.txt.
• Cloudflare’s developer docs, which link to a separate llms.txt for each product.
• Stripe’s documentation, whose llms.txt includes instructions aimed at AI coding tools.
• OpenAI’s API docs, which point readers to an llms.txt index and offer Markdown versions of pages by adding .md to the URL.

Tooling has followed. The llmstxt.org site lists documentation platforms such as Mintlify and GitBook, website builders such as Wix, and WordPress SEO plugins such as Yoast SEO and AIOSEO that can generate the file. Yoast’s llms.txt feature can build the file automatically from your recent content and refresh it weekly, or let you pick pages manually.

What is missing is clear confirmation from the big AI search products that they use llms.txt to choose which sources to cite. Treat any claim that llms.txt “gets you into ChatGPT answers” with skepticism unless the provider itself says so.

What Google says about llms.txt

Google has been direct. Its guide to optimizing for generative AI features in Google Search, last updated in July 2026, says you don’t need to create machine-readable files, AI text files, markup or Markdown to appear in Google Search, because Google Search itself doesn’t use them. It also says it is completely fine to create and maintain llms.txt files for other services.

Google’s AI features documentation makes the same point for AI Overviews and AI Mode: there are no additional technical requirements or special optimizations needed beyond normal Search eligibility.

So, for Google visibility, llms.txt is neutral. It will not help and it will not hurt. What does help is the usual work: crawlable pages, helpful content and clear answers. Our guides on showing up in Google AI Overviews and answer engine optimization cover that in detail.

Key takeaway: llms.txt is a low-cost courtesy file for AI tools and agents, not a ranking factor. Add it if it takes minutes, but spend most of your AI search effort on clear, well-structured content that search engines already index.

Should your website have one?

Use this quick checklist to decide.

The real cost of llms.txt is not creating it. It is keeping it accurate. A file that points to old prices, retired services or broken links can do more harm than having none, because any AI tool that reads it gets outdated information.

How to add llms.txt in minutes

If you decide to add one, here is the fastest path.

1. Check your platform first. In WordPress, look in your SEO plugin’s settings for an llms.txt option. Some site builders and documentation platforms also generate it automatically.
2. If you write it by hand, start with 10 to 20 links. Pick the pages that answer the most common customer questions: services, pricing, policies, FAQs and contact details.
3. Write a 1 to 2 sentence summary. Say who you are, where you operate and who you serve, in plain language.
4. Save it as plain text named llms.txt and upload it to your site’s root folder so it loads at yoursite.com/llms.txt.
5. Test it. Open the URL in a browser and confirm it shows plain text, not a 404 page or your homepage.
6. Add a review date to your calendar. Update the file whenever you change prices, services or key URLs.

While you are in your site files, review your robots.txt rules for AI crawlers too. Our technical SEO checklist covers crawlability basics, and our guide to schema markup explains structured data, which is still useful for regular search results.

How to tell if it is doing anything

Most small sites will never know for sure whether an AI tool read their llms.txt. You can still look for signals:

• Server logs. Check whether requests to /llms.txt appear, and which user agents make them.
• Referral traffic. In your analytics, watch for visits from AI assistants and AI search products.
• Answer accuracy. Ask AI assistants common questions about your business and see whether the details match your current pages.

We cover these methods in more depth in our guide on how to track your visibility in AI search.

The bottom line

llms.txt is a simple, voluntary file that gives AI tools a tidy map of your site. It is genuinely useful for documentation and content-heavy sites, it costs almost nothing to add, and Google has said plainly that it neither helps nor hurts Search. Add it if your platform makes it easy, keep it accurate, and put your real effort into clear, helpful pages that both people and AI systems can trust.

Sources

• llmstxt.org: The /llms.txt file proposal — origin, date, file format and list of tools that generate llms.txt
• Google Search Central: Optimizing your website for generative AI features on Google Search — Google Search does not use llms.txt; creating one for other services is fine
• Google Search Central: AI features and your website — no special files or markup needed for AI Overviews and AI Mode
• OpenAI: Overview of OpenAI crawlers — OAI-SearchBot, GPTBot and ChatGPT-User roles; docs index offered via llms.txt
• Yoast SEO: llms.txt feature — automatic and manual llms.txt generation in a WordPress SEO plugin