What llms.txt is – and where it comes from
An llms.txt is a text file that sits at the very top of a domain, at your-domain.com/llms.txt. It summarises the website in a few lines and links the pages that matter to a language model. Language models, or large language models, are the programs behind ChatGPT, Gemini or Claude.
Jeremy Howard of Answer.AI proposed the file on 3 September 2024. His reasoning is on llmstxt.org. Websites are built for people, and a page wraps its information in navigation, ads and scripts. The context window, meaning the amount of text a model can process at once, stays too small for most websites.
The format is Markdown, that is, text with a few characters for headings and lists. The only requirement is a heading with the name of the website. After that, a one-sentence summary and lists of links are customary, often with a short description per link.
The llms.txt is a 2024 proposal for a summary of your website, written for machines. It is not a binding standard. It replaces neither the robots.txt, which controls crawler access, nor the sitemap, which lists pages for search engines.
Who publishes llms.txt today and who requests it
Providers with their own llms.txt include Anthropic, Stripe, Cloudflare and Vercel. On its page about its own crawlers, OpenAI links the index of its documentation as llms.txt. A crawler is a program that reads websites automatically.
Whether programs also request the file is shown by the analysis by the SEO provider Ahrefs. Among AI tools, OpenAI's GPTBot and the coding assistant Claude Code lead, ahead of all AI search engines and chat assistants. GPTBot collects training data in the process. That proves requests, but not use in answers. According to Ahrefs, AI agents, that is, programs that carry out tasks on their own, only request the file when they are pointed to it. They do not look for it on spec. Without a link, an index or an instruction, the file mostly goes unread.
On Search, Google is unambiguous. Its guide to AI features in Google Search states that Search ignores llms.txt files. Google adds that creating one is still perfectly fine. Lighthouse, Chrome's auditing tool, treats the file as voluntary. Nowhere in the crawler documentation of OpenAI and Anthropic does it say that their services read the file. Both pages deal with the robots.txt.
The key statements at a glance:
| What it states | Date of the source | |
|---|---|---|
| Google Search Central | Google Search ignores llms.txt files. They neither help nor harm visibility. | July 2026 |
| Chrome Lighthouse | Treats llms.txt as voluntary. If the file is missing, the test counts as not applicable. | May 2026 |
| Ahrefs, 137,210 websites | 28 percent have a file, a figure Ahrefs itself calls an upper bound. 97 percent of these files received not a single request in May 2026. | June 2026 |
| SE Ranking, around 300,000 domains | 10.13 percent have a file. According to SE Ranking, there is no link to AI citations. | November 2025 |
| Common Crawl, 584,107 files | 68.27 percent come from templates, 22.56 percent contain not a single link. | August 2026 |
Sources: Google Search Central, Chrome for Developers, Ahrefs, SE Ranking, Common Crawl.
The studies measure different things. Ahrefs and SE Ranking count requests and citations, while Common Crawl, a non-profit web archive, reads the files themselves. Three caveats come with the numbers. None of the studies says what a provider does internally with the file. The rates of 28 and 10.13 percent come from different samples and periods. And Ahrefs itself calls its 28 percent an upper bound, because its own customers work more technically than the average.
The studies in the original: Ahrefs, SE Ranking and Common Crawl.
What llms.txt means for an SME website
For Google Search, the file has no effect, as Google itself says. For answers from ChatGPT and the like, there is so far no reliable evidence that it brings citations. Creating it takes little time and does not harm Google Search. Effort and risk lie in maintenance.
A sensible rule of thumb is to create the file where a lot of factual information comes together, such as price lists, product data, manuals or a knowledge base. If a program is pointed to the file, a short overview should help it find the right page faster. That is also the reasoning on llmstxt.org. For a website with ten pages, the benefit stays small.
Ahrefs warns that an outdated file misleads every agent that reads it. If the file is read, old prices or a former phone number do more harm than a missing file. Plan maintenance for every change, as you would for the legal notice.
Template: llms.txt for your company
The template leads you to your own llms.txt in four blocks. With fill-in hints and six checks, the file is ready in about 30 minutes.
Thank you. Your download will start shortly.
How to create an llms.txt
A short file is enough to get started: name, one sentence and the pages visitors need first. The noevu.ch file runs to 245 lines and also lists services, references and notes for language models. Here is the start of the file, shortened. It is written in English:
# Noevu GmbH
> Swiss web agency born in Zürich, building custom websites, modern content platforms, and AI solutions for Swiss SMEs.
## Key Pages
- [Homepage](https://noevu.ch/de/)
- [About the agency](https://noevu.ch/de/ueber-uns)
- [Tips & Insights (blog)](https://noevu.ch/de/tipps/)
- [Contact](https://noevu.ch/de/kontakt)Structure and link lists follow the proposal on llmstxt.org, where a description after the link is optional. If you create the file yourself, you proceed in six steps:
- 1. Write the name and a summary A heading with the company name, then one sentence on what you do and for whom.
- 2. Choose the important pages Services, process, contact and privacy are enough for a start. Each link can get a one-line description.
- 3. Save it as a text file File name llms.txt, plain text. Word and Pages are not suitable; a simple text editor is enough.
- 4. Place it at /llms.txt The file must be reachable directly at your-domain.com/llms.txt. Plugins such as Yoast SEO generate it at the push of a button; you check the choice of pages yourself afterwards. For other systems, the provider's guide explains how to put a file in the root directory.
- 5. Link it and check it in the browser A link on the website makes the file discoverable; noevu.ch puts it in the footer. Then open the address and check that the text appears and every link leads to an existing page.
- 6. Plan for maintenance Whenever services, prices or contact details change, the change goes into the file.
What to check besides the llms.txt
AI crawlers are controlled through a different file, the robots.txt. It tells crawlers which areas of a website they may read. OpenAI distinguishes three kinds of access. OAI-SearchBot belongs to ChatGPT search, GPTBot to training models, and ChatGPT-User to actions that users trigger themselves. OpenAI recommends allowing OAI-SearchBot if you want your pages to appear in ChatGPT's search answers. Anthropic writes that its programs follow blocks in the robots.txt.
If you want to keep the crawlers of the big AI providers out, add that to the robots.txt. For ChatGPT-User, which fetches on behalf of individual users, robots.txt rules can come to nothing, according to OpenAI. In the llms.txt, such an entry has no effect. Common Crawl analysed 584,107 files and notes that a block there blocks nothing.
For Google Search, Google writes that no new machine-readable files, no special markup and no Markdown are needed. The guide expressly lists the llms.txt among the measures you can skip. Google points to effective SEO work instead.
Regardless of the llms.txt, it is worth looking at pages that are easy to read and at the structure of the HTML source. Google does not need structured data for this. They name your company data unambiguously for machines; whether AI answers benefit from that is not proven. You can check by spot test how often your domain actually appears in AI answers. The page on AI visibility for Swiss SMEs shows how that works. Whether an llms.txt suits your website is mostly decided by the rule of thumb above. In doubtful cases, a conversation about AI visibility helps.
A short conversation shows whether and how your website appears in AI answers and which steps make sense for you. An llms.txt is not necessarily part of it.
Frequently Asked Questions
What is an llms.txt?
An llms.txt is a text file at your-domain.com/llms.txt. It summarises your website in Markdown and links the most important pages, usually with a short description. Jeremy Howard of Answer.AI proposed it in September 2024, but it is not a binding internet standard. The only requirement is a heading with the name of the website.
Does Google read the llms.txt?
Google Search does not, says Google. Its guide to AI features states that Search ignores such files. So the file neither improves nor harms your ranking. Chrome checks for it in Lighthouse as a voluntary test. If the file is missing, the test counts as not applicable.
Do ChatGPT and Claude read the llms.txt?
There is no evidence. Nowhere in the crawler documentation of OpenAI and Anthropic does it say that the services read the file. Ahrefs sees requests from GPTBot and the coding assistant Claude Code, but that does not prove use in answers. SE Ranking found no link between the file and citations in AI answers.
What is the difference between llms.txt, robots.txt and sitemap.xml?
The robots.txt tells programs which areas they may read. The sitemap.xml lists all pages for search engines. The llms.txt picks a few important pages and describes them in text form. Many crawlers follow the robots.txt voluntarily, and it is the established way to control access. For the llms.txt, according to Common Crawl, nothing obliges anyone to read it.
Can you use the llms.txt to keep AI programs out?
No, that is what the robots.txt is for. Common Crawl analysed 584,107 llms.txt files and found that blocks in them have no effect. If you want to keep GPTBot or ClaudeBot out, add that to the robots.txt.
Where does the llms.txt go, and how often does it need maintenance?
The file must be reachable directly at your-domain.com/llms.txt. Plugins such as Yoast SEO can generate it at the push of a button; for other systems, the provider's guide helps. Update it whenever your content changes. An outdated file misleads programs.
