Executive Summary
llms.txt evolved from niche proposal to near-mainstream convention over the 24 months since its 2024 introduction. By April 2026, partial support exists across every major Western AI platform and the file is a routine piece of the AI access stack for technically sophisticated brands. Adoption remains uneven across sectors, and the spec itself is still community-managed rather than formally standardized. This report documents where the convention stands and where it is headed.
What llms.txt Is and How We Got Here
llms.txt is a plain text file served at a domain root that gives AI systems a curated view of a site's most important content, often with a one-paragraph authoritative brand summary. The original proposal came from Jeremy Howard in 2024 and gained traction through community adoption rather than formal standardization. By late 2024, Anthropic had publicly confirmed support. By mid-2025, Perplexity had followed. Other platforms were less explicit but increasingly treated llms.txt as a meaningful signal.
The convention fills a gap robots.txt cannot address. robots.txt grants or denies crawler access at the URL level but says nothing about which pages matter most or how a brand should be described. llms.txt adds editorial curation on top of access control, which is why the two coexist rather than competing.
Adoption Trajectory
Adoption growth followed a classic technology-diffusion curve. Early adopters were technically sophisticated sites in technology, cybersecurity, and blockchain. By mid-2025 adoption was routine among developer-facing SaaS. Through 2026 Q1, adoption expanded into mainstream SaaS, publishing, and some consumer sectors. Sectors with legal or compliance conservatism (financial services, healthcare, legal) remained slow adopters, with publication rates often under 10 percent among top-100 domains.
The top-100 adoption gap by sector is revealing. In every industry measured, the largest brands adopted llms.txt at rates multiples of the general population. The gap indicates that llms.txt remains a sophistication signal among marketing and engineering teams, not yet a default practice. That gap is also an opportunity for mid-market brands to move before their sector as a whole.
Platform Support Status
Support status at the time of this report:
Anthropic: publicly confirmed. Claude Desktop and Claude.ai both respect llms.txt directives in retrieval workflows.
Perplexity: publicly confirmed. Perplexity retrieves llms.txt and uses it to prioritize page selection.
OpenAI: unconfirmed explicitly but observable in retrieval patterns. Teams that publish llms.txt see correlated changes in ChatGPT SearchGPT citation patterns.
Google: no explicit confirmation. Google-Extended and Gemini behavior does not visibly change in response to llms.txt. Google's preferred signal stack remains traditional: robots.txt, sitemap, structured data.
Mistral: confirmed with caveats. Mistral reads llms.txt where present but the spec support is still maturing.
Open-weight LLM deployments: varies by deployment. Private fine-tunes and enterprise RAG pipelines can be configured to respect llms.txt. Out-of-the-box behavior depends entirely on the deployment.
Common Practice Patterns
Best practice emerged through 2025 and solidified in 2026. Well-structured llms.txt files share several common patterns. They are short, typically under 5 kilobytes. They lead with an authoritative one-paragraph brand summary in a blockquote. They organize sections by content function (core pages, documentation, research, comparisons, policies). They include contact information for licensing and press.
Common mistakes also solidified. The most frequent: treating llms.txt as a sitemap rather than a curation. A file with 200 URLs defeats its purpose. The second most common: using template copy without customization. A generic llms.txt signals low investment to the AI systems that read it. The third: contradicting robots.txt, which happens when teams edit the two files in isolation rather than treating them as a coordinated stack.
The Standardization Question
llms.txt remains a community convention rather than a formal standard. The spec is maintained through the llmstxt.org project and through contributions from Anthropic, Perplexity, and various open-source contributors. A formal IETF RFC or similar standardization effort has been discussed but has not materialized as of April 2026. The practical effect is that different platforms interpret edge cases slightly differently, but core syntax and semantics are stable enough that a well-formed llms.txt works across all supporting platforms.
Formal standardization would raise adoption by giving legal and compliance teams firmer ground to approve publication. For now, the community-convention status has been good enough for mainstream adoption but creates a long tail of uncertainty that holds back some enterprise adopters.
Enterprise Adoption
Enterprise adoption patterns lag consumer-facing brand adoption, but the direction of travel is clear. Enterprises increasingly publish llms.txt files through their marketing domains. Technical documentation sites often publish separate llms.txt files tailored to developer-focused AI clients. A small but growing number of enterprises run internal llms.txt-style files on intranet search pipelines, effectively extending the convention inside the firewall.
The enterprise use case that grew most was brand governance. Large brands concerned about AI hallucination risk used llms.txt to point AI systems at canonical descriptions of their products, reducing the rate at which AI systems invented or distorted product claims. This is a content-authority play rather than an access-control play.
Outlook Through 2027
Three developments are likely in the next 12 months. First, formal standardization discussions will accelerate as regulatory pressure on AI training and retrieval grows. A formal spec would clear up ambiguity and raise adoption especially in compliance-heavy sectors. Second, tooling will mature. Dedicated llms.txt generators, validators, and publishing tools will reach a level of polish similar to sitemap tools today. Third, platform support will expand. Google's position is the largest outstanding question. If Google formally supports llms.txt in 2026 or 2027, the convention becomes effectively universal for Western AI.
How Presenc AI Helps
Presenc AI audits llms.txt for every domain we monitor, scoring structure, coverage, alignment with schema and robots.txt, and respect by AI platforms. The platform detects adoption gaps relative to industry peers, flags common mistakes, and correlates llms.txt quality with measured AI citation outcomes. For brands publishing their first llms.txt, Presenc generates a recommended starter file based on your content map and visibility priorities.