TL;DR — Quick answer
llms.txt is a proposed plain-text file that lists a website's key content for AI crawlers, written in simple Markdown and placed at the site root. It is easy to add and harmless, but it is a community proposal, not an official standard, and Google has publicly said it does not use llms.txt for rankings or AI features. No major AI platform has confirmed it relies on the file. The honest verdict: treat llms.txt as optional and low priority. Real AI visibility comes from solid SEO, clean HTML and structured data, not from a curated text file most crawlers ignore.
Key takeaways
- llms.txt is a proposed Markdown file at your site root that summarises and links your key content for AI models
- It is different from robots.txt (access control) and sitemap.xml (discovery of all URLs), and unlike them it is not an adopted standard
- Google has stated it does not use llms.txt, comparing it to the abandoned keywords meta tag, and other AI platforms have not confirmed support
- Adding the file is quick and low risk, but do not expect it to change rankings or AI citations
- AI assistants mostly read normal HTML, so the content that ranks well in search is what gets cited in AI answers
- For real AI visibility, invest in solid SEO and structured data rather than chasing an unverified file
Every few months a new file or tag is held up as the secret to ranking in AI tools like ChatGPT, Google AI Overviews and Perplexity. llms.txt is the current example. It is a genuinely interesting idea, and adding it costs you almost nothing, but the marketing around it has run well ahead of the evidence. This article gives you the honest picture so you can decide whether it is worth your time, using the same approach we take with every client at Juicy Designs: solid SEO and structured data first, hype later.
What llms.txt is
llms.txt is a proposed plain-text file, written in Markdown and placed at the root of your domain, that lists and summarises your most important content so AI crawlers and large language models can find it quickly. The idea, put forward by developer Jeremy Howard in 2024, is that AI systems have limited context windows and struggle with cluttered HTML, so a clean, curated index of your key pages could help them understand and represent your site accurately.
A typical file lives at a path like example.co.za/llms.txt. It opens with an H1 title for the site, a short blockquote summary, and then groups your important pages under headings as a tidy list of Markdown links with brief descriptions. In spirit it is a human-readable, AI-friendly map of what matters on your website. It is worth being clear up front: llms.txt is a community proposal, not an official standard agreed by search engines or AI vendors, and adoption on the crawler side is unconfirmed.
How llms.txt differs from robots.txt and sitemap.xml
The three files sound similar but do very different jobs, and only two of them are established standards. Mixing them up is the most common confusion we hear, so here is the plain-English distinction.
| File | Purpose | Format | Status |
|---|---|---|---|
| robots.txt | Tells crawlers which URLs they may or may not access | Plain text, directives | Established, widely supported |
| sitemap.xml | Lists all indexable URLs to help search engines discover pages | XML | Established, widely supported |
| llms.txt | Curates and summarises key content for AI models to read | Markdown, human-readable | Proposed, adoption unconfirmed |
The key point: robots.txt controls access and sitemap.xml aids discovery of everything, while llms.txt tries to highlight and explain a curated subset for AI. The first two are part of how the web has worked for years. The third is an optimistic addition that the major players have not committed to.
llms.txt is a proposed Markdown file at a website's root that lists and summarises key content for AI crawlers and large language models. It differs from robots.txt, which controls crawler access, and sitemap.xml, which lists all indexable URLs for search engines. robots.txt and sitemap.xml are established standards; llms.txt is a 2024 community proposal with unconfirmed adoption by major AI platforms. Source: Juicy Designs, June 2026.
What the llms.txt spec proposes
The proposal defines a simple, predictable structure so any AI system could parse it the same way. The format is deliberately minimal:
- An H1 with the name of the site or project (the only required element).
- An optional blockquote summary giving the key context in a sentence or two.
- Optional sections under H2 headings, each containing a Markdown list of links to important pages, with a short note on what each page covers.
- An optional “Optional” section for secondary links an AI can skip if it is short on context.
Some advocates also suggest publishing clean Markdown versions of individual pages (for example a page.html.md companion) so models receive content without navigation, scripts and styling. None of this is technically difficult. The harder question is whether anything on the receiving end actually uses it.
Does llms.txt actually help? What the evidence says
As of mid-2026, there is no strong public evidence that llms.txt improves rankings, AI citations or traffic. This is the part the hype tends to skip, so it is worth being direct about it.
Google has been the clearest. John Mueller of Google publicly compared llms.txt to the keywords meta tag, a signal Google abandoned years ago because it was abused and unreliable, and indicated Google does not use llms.txt for Search or its AI features. Google has also said its crawlers do not look for the file. On the AI vendor side, the major assistants have not confirmed that they read llms.txt, and server-log analyses shared by the SEO community have generally found little or no crawler traffic requesting the file.
“We test these things before we recommend them. With llms.txt the picture is simple: Google has said it does not use it, the AI platforms have not confirmed they do, and our logs barely show it being fetched. So we tell clients the truth. Add it if you like, it does no harm, but do not expect it to change a single ranking. The work that actually gets you cited in AI answers is the same work that gets you ranked: clear, well-structured content and clean technical SEO.”
— Cobus van der Westhuizen, Founder & Digital Strategist, Juicy Designs — reviewed and verified June 2026
To be fair to the idea, llms.txt is new, standards take time, and it is possible some tools will adopt it in future. It is also genuinely useful as internal documentation of your most important pages. But basing an AI visibility strategy on it today would be a mistake. The honest verdict stands: optional, low priority, no proven benefit.
How to create an llms.txt file if you want one
If you want to add llms.txt for completeness or future-proofing, it takes about fifteen minutes. There is no downside as long as you keep it accurate and do not let it distract from real SEO work.
- Create a plain-text file named
llms.txtin Markdown. - Start with an H1 of your business or site name, followed by a one or two sentence blockquote summary of what you do.
- Add H2 sections such as Services, Guides or About, and under each list your most important pages as Markdown links with a short description of each.
- Keep it to your genuinely key pages. This is a curated map, not a full sitemap.
- Upload it to your domain root so it resolves at
yourdomain.co.za/llms.txt, and review it whenever your key pages change.
That is the whole job. If a client asks us to add one, we will, but we are upfront that it sits alongside the real work rather than replacing any of it.
What actually moves the needle for AI visibility
AI assistants overwhelmingly read normal HTML pages, so the content that ranks well in Google is the content that gets surfaced and cited in AI answers. That is good news, because it means you do not need a separate AI strategy built on unproven files. You need your SEO done properly. At Juicy Designs we focus AI visibility on the fundamentals that demonstrably work:
- Solid technical and on-page SEO: clean, fast, crawlable HTML, sensible URL structure, correct heading hierarchy and internal linking. See our SEO services for South African businesses.
- Structured data: Schema.org markup (Article, FAQ, Organisation, Breadcrumb) that tells machines exactly what your content means, which is far better understood than a curated text file.
- Genuinely helpful, answer-first content: clear definitions, direct answers and well-organised sections that both readers and models can quote.
- Authority and consistency: accurate business information, reviews and citations across the web so AI systems trust what they find.
If your goal is to be found and cited by AI tools, this is where the effort belongs. Our AI search optimisation service and our guide to generative engine optimisation (GEO) go deeper on the techniques that actually earn AI mentions, with no reliance on speculative files.
Frequently asked questions
What is llms.txt?
llms.txt is a proposed plain-text file placed at the root of a website that lists the site's most important content in a clean, structured way so that AI crawlers and large language models can find and read it more easily. It uses simple Markdown and is read by humans and machines. It is a community proposal, not an official standard, and no major AI company has confirmed that it uses the file.
Does my website need an llms.txt file?
Most websites do not need an llms.txt file. It is easy to add and harmless, but Google has publicly said it does not use llms.txt for rankings or AI features, and other major AI platforms have not confirmed support either. Treat it as optional and low priority. Solid SEO, clean HTML and structured data do far more for AI visibility.
Does llms.txt help with AI visibility or SEO?
There is no strong evidence that llms.txt improves AI visibility or SEO. Google's John Mueller has compared it to the long-abandoned keywords meta tag and stated it does not influence Google's systems. AI assistants generally crawl normal HTML pages, so the same content that ranks well in search is what gets cited in AI answers.
How is llms.txt different from robots.txt and sitemap.xml?
robots.txt tells crawlers which URLs they may or may not access, and sitemap.xml lists all indexable URLs to help discovery. Both are official, widely supported standards. llms.txt is different: it is a proposed file that curates and summarises your key content for AI models, rather than controlling access or listing every page. It is not yet an adopted standard.
How do I create an llms.txt file?
Create a plain-text file named llms.txt in Markdown, place it at your domain root (for example example.co.za/llms.txt), start with an H1 title and short summary of the site, then list your most important pages as Markdown links grouped under headings. Keep it accurate and concise. Remember it is optional and unverified, so do not let it replace proper SEO work.