The short answer
llms.txt is a plain-text Markdown file placed at the root of a website (for example, example.com/llms.txt) that gives AI tools a short summary of the site and links to its most useful pages. It is a proposed convention, suggested in 2024 by Jeremy Howard of Answer.AI, not an official web standard, and no AI tool is obliged to read it. It is cheap to add and harmless, but it does not replace good SEO, clear pages or structured data.
Key takeaways
- llms.txt is a proposed convention, not an official standard or a ranking factor.
- It is a Markdown file at your site root: a title, a summary and curated links.
- It does not control AI crawler access; robots.txt does that.
- Worth adding as a low-cost extra once your pages and SEO basics are solid.
What is llms.txt?
llms.txt is a proposal for a standard file that tells large language models what a website is about and where its most useful content lives. It sits at the root of the domain, in the same place as robots.txt, and is written in Markdown, the simple formatting language that AI models read easily.
The reasoning behind it is practical. Web pages are full of navigation, scripts, cookie banners and layout code that make it harder for an AI tool to find the substance. A short, curated Markdown file gives a tool a clean overview in a few hundred words, plus links to the pages that matter.
The proposal also suggests that sites can offer Markdown versions of individual pages, and some sites publish a longer companion file, often called llms-full.txt, containing the full text of their key pages. Both ideas are optional extras. The core of the convention is the single llms.txt file.
What goes in an llms.txt file?
The proposal defines a simple structure, in a fixed order. Only the first part is required:
| Part | Markdown used | What it contains |
|---|---|---|
| Title | An H1 heading (a line starting with #) | The name of the business or site. This is the only required section. |
| Summary | A blockquote (a line starting with >) | A short description with the key facts needed to understand the rest of the file. |
| Details | Plain paragraphs or lists | Optional extra context, such as who you serve, languages and how to book. |
| Link sections | H2 headings (##), each followed by a list of links | Groups of important pages, each written as a link with a short note after a colon. |
| Optional | An H2 heading titled “Optional” | Secondary links that a tool can skip if it needs a shorter overview. |
Keep it short. The point is to summarise and curate, not to list every URL on your site; that is what an XML sitemap is for.
What does an llms.txt file look like for a small business?
Here is a complete example for a fictional clinic, shown line by line. Each row is one line of the file:
| Line | Text in the file |
|---|---|
| 1 | # Example Dental Clinic |
| 2 | > Doctor-led dental clinic in central Bangkok offering check-ups, whitening, aligners and implants, with treatment in English and Thai. Open seven days; book online. |
| 3 | ## Services |
| 4 | - [Dental implants](https://example.com/implants): what is included, price range, number of visits and recovery |
| 5 | - [Teeth whitening](https://example.com/whitening): in-clinic and take-home options, prices and aftercare |
| 6 | - [Clear aligners](https://example.com/aligners): who they suit, treatment length and consultation process |
| 7 | ## Booking and contact |
| 8 | - [Book an appointment](https://example.com/book): live availability and online booking |
| 9 | - [Location and hours](https://example.com/contact): address, map, parking, nearest station and opening hours |
| 10 | ## Optional |
| 11 | - [Patient guides](https://example.com/blog): articles answering common treatment questions |
Notice that the summary states plain facts (location, services, languages, how to book) and that every link note tells a reader what they will find. That is what makes the file useful.
Do ChatGPT, Google and Perplexity actually use llms.txt?
There is no public confirmation that the major AI search tools use llms.txt to decide what to cite. Google representatives have said publicly that Google Search does not use it, and the other large AI providers have not announced that their answer engines rely on it.
Adoption so far has been strongest among software and documentation sites, where developers point AI coding assistants at a project’s llms.txt to get a clean overview. An AI agent or tool that is sent to your site may read it, and anyone can paste it into an assistant as a summary of your business, but you should not expect it to change your visibility on its own.
So why bother? Because it costs very little, it does no harm, and writing it forces you to state your key facts clearly, which improves your website too. Treat it as a small bet on where AI tools may go, not as a ranking tactic.
It is also worth being clear about priorities. If your service pages are vague, your contact details inconsistent or your site blocked to crawlers, an llms.txt file will not fix any of that. Spend your first hours on the pages people and AI tools actually read, then add the file as a finishing touch. For a business with a small site and a clear offer, that finishing touch is a short, satisfying job; for a large site with hundreds of pages, the curation step is where the real thinking happens.
llms.txt vs robots.txt vs sitemap.xml: what is the difference?
These three files live side by side but do different jobs. Only robots.txt controls crawler access.
| File | Job | Status |
|---|---|---|
| robots.txt | Tells crawlers which parts of the site they may visit, rule by rule for each user-agent | Formal internet standard (RFC 9309), respected by major search and AI crawlers |
| sitemap.xml | Lists the URLs you want search engines to discover and index | Widely supported protocol used by Google, Bing and others |
| llms.txt | Summarises the site and curates key pages for AI tools | Proposed convention; use is voluntary |
If you want to allow or block AI crawlers, use robots.txt with their user-agent names: for example GPTBot and OAI-SearchBot for OpenAI, ClaudeBot for Anthropic and PerplexityBot for Perplexity, plus Google-Extended, a token that controls whether Google may use your content for its Gemini models without affecting Google Search. An llms.txt file has no effect on any of this.
How do you create and maintain an llms.txt file?
Most small businesses can write a good llms.txt in under an hour:
- Pick your 5 to 15 most important pages: core services, pricing, booking, contact, about and your best guides.
- Write a one or two sentence summary with concrete facts: what you do, where, for whom, in which languages and how to get started.
- Group the links under clear H2 headings, such as Services, Booking and contact, and Guides.
- Add a short note after each link saying what the page contains.
- Save it as plain UTF-8 text named llms.txt and upload it to the root of your domain.
- Open yoursite.com/llms.txt in a browser to check it loads as text.
- Update it whenever prices, services or key pages change, so it never contradicts your website.
Avoid stuffing it with keywords, listing private or unpublished pages, or copying your whole site into it. A file that disagrees with your pages does more harm than no file at all.
At PJW Partners, we add an llms.txt file to every website we launch, alongside the things that matter more: clear answer-first pages, schema markup and an AI-friendly robots.txt. It is one item on our new website SEO checklist and part of the wider approach we describe in what is generative engine optimisation? If you would like one written for your site, send us your website address.
Questions
Is llms.txt required? +
No. It is a voluntary, proposed convention. Your site can be crawled, indexed and cited by AI tools without one.
Will llms.txt improve my Google rankings? +
There is no evidence that it does, and Google representatives have said Google Search does not use it. Treat it as a helpful extra, not a ranking tactic.
Can llms.txt stop AI from training on my content? +
No. To control AI crawlers, add rules to robots.txt for user-agents such as GPTBot, ClaudeBot and Google-Extended.
Where exactly should the file go? +
At the root of your domain, so it loads at yourdomain.com/llms.txt, in the same place as robots.txt.
How long should an llms.txt file be? +
Short enough to read in a minute or two. A title, a summary and a curated list of key pages is enough for most small businesses.
Work with PJW Partners
Websites and AI agents that bring in customers.
Websites from $1,500 and AI agents from $1,000. Tell us about your business and we’ll reply within one business day.