What is llms.txt, and how do you write one?
llms.txt is a community proposal for a Markdown file, usually at /llms.txt, that links to selected site content for tools that choose to read it. It is not an IETF or W3C web standard, access control, or a sitemap replacement. Google says it ignores llms.txt for Search, including its AI features; the proposal does not establish a ranking signal.
Is llms.txt a standard?
No. The project describes llms.txt as a proposal and emerging convention. It has a published format and some voluntary tool adoption, but that is different from a formal web standard or a provider commitment to use the file for discovery, ranking, or citation selection.
The safe interpretation is narrow: a well-maintained file can be an optional navigation surface for agents and documentation tools that explicitly support it. Test that use with the target tool instead of assuming every search or answer engine reads it.
Where does llms.txt go?
The proposal places a Markdown document at /llms.txt and version 2 also describes scoped files for subpaths. Serve a normal text response that is publicly reachable and keep every link canonical. The current proposal does not define a special expanded-file requirement.
What should an llms.txt file contain?
Use an H1 for the project or site, a short summary, then grouped Markdown links to accurate documentation or reference pages. Describe what each destination contains. Exclude private, duplicate, redirected, stale, or noindex URLs, and keep the file synchronized with canonical changes.
- # Example product
- > One sentence describing the product and documentation scope.
- ## Documentation
- - [API reference](https://example.com/docs/api): Current endpoints and authentication.
- - [Changelog](https://example.com/changelog): Dated product changes.
How is llms.txt different from robots.txt and a sitemap?
Robots.txt communicates crawler preferences under the Robots Exclusion Protocol. An XML sitemap is a supported discovery feed for search engines. llms.txt is a voluntary content index for tools that adopt its proposal. Publishing one does not grant access that robots, authentication, a CDN, or a WAF denies, and it does not submit URLs to Google or Bing.
Does llms.txt actually help?
The dated sources reviewed here do not establish that llms.txt improves search rankings, AI impressions, or citations. Google explicitly says it ignores the file for Search and its AI features. Other tools may use it for navigation or context, but support must be verified per tool.
Publishing also has maintenance cost and disclosure risk: stale links misdirect tools, and a curated list can reveal routes you did not intend to advertise. Add the file only for a defined audience, keep private URLs out, and measure that audience's actual use.
How should I test and maintain llms.txt?
Validate the Markdown, follow every link, confirm each destination is canonical and public, and record which supported agent or documentation tool consumes the file. Check server logs without treating a fetch as a ranking or citation signal. Review after documentation moves, redirects, or major content releases.
Should llms.txt be part of an AI-readiness score?
Only as an optional convention, not as an engine requirement. A score can report whether the file exists and whether its links are valid, but it should not infer Search eligibility, agent reachability, or citation likelihood from that presence. RankEcho's public readiness check is a heuristic and its llms.txt component is not a provider-defined standard.
Sources reviewed
Material crawler-role and control claims below were checked against primary provider documentation and the robots standard. Access settings affect eligibility and reachability; they do not guarantee indexing, ranking, an AI impression, or a citation.
2 claim-level source records
| Claim reviewed | Official source | Review record |
|---|---|---|
| The llms.txt project describes llms.txt as a community proposal for a root or scoped-path Markdown index that participating tools may use, not as an IETF or W3C standard. | llms.txt proposal | Checked 2026-09-01 · Proposal version 2 modified 2026-08-10 · Primary proposal review; voluntary tool adoption is not treated as a universal search or citation signal. · Confidence: High |
| Google says it ignores llms.txt for Search and that the file neither helps nor harms Google Search rankings or AI-feature visibility. | Google AI optimization guide | Checked 2026-09-01 · Google Search guidance updated 2026-07-10 · Primary-source documentation review; the statement is limited to Google Search and its AI features. · Confidence: High |
Frequently asked questions
No. Robots.txt communicates crawl preferences. llms.txt is a proposed Markdown index for tools that voluntarily support it. Neither file is authentication or authorization.
Google says it ignores llms.txt for Search and its AI features. The community proposal describes an optional content index; it does not establish a ranking signal.
The proposal describes voluntary adoption, and Google says it ignores the file for Search and its AI features. Treat consumption as tool-specific and verify it with each target tool's documentation or observable requests.
Neither the proposal nor Google's reviewed Search guidance establishes citation impact. A supporting tool may use the file to find documentation, but that is not the same as an AI impression or citation.