llms.txt File

Rule Severity and Score

Rule id

indexing.llms_txt

Group

Indexing

Severity of a failure

Info

Counts in the score

Yes, whenever it applies to the page

Required for a score

No

Condition and Exceptions

Condition

Probe llms_txt (/llms.txt of the final URL's scheme and host, once per audit or crawl, first 512 KiB, same-host redirects only), judged as the Lighthouse llms-txt audit (Q102, owner decision 2026-10-18): 5xx or 429 → fail ("Failed with HTTP status N"); 404/410 → not_applicable ("The site has no llms.txt file"); any other 4xx → not_applicable; 1xx/3xx final answer (no content) → fail; 2xx body (content type ignored, as in Lighthouse) → fail with each reason that applies: no line matching /^\s*#\s+.+/m (no H1), no /\[.+\]\(.+\)/ (no Markdown link), JavaScript length under 50 (UTF-16 code units of the whole body, untrimmed); otherwise pass, observed size, length, H1, link count, H2 sections, summary blockquote. An HTML body is named in the message. A community proposal for AI agents; Google Search does not use it (Q104). Host-level: one issue per host (IssueTracker::HOST_RULES), one result per crawl

Exceptions

HTML mode → not_checked (html_mode); no probe (non-HTML page) → not_checked; no answer (timeout, DNS, connection) → not_checked (probe_failed; Lighthouse fails it); a 2xx bot challenge with no body kept → not_checked (probe_failed); a redirect to another host is not followed → not_checked

Sources and Project Decision

Sources

The /llms.txt file (proposal by Jeremy Howard, v2: a Markdown file whose only required section is an H1 with the name of the project or site) (llmstxt.org, Q101)
Lighthouse llms-txt audit, core/audits/agentic/llms-txt.js (Markdown with an H1, links, at least 50 characters; a 4xx is not applicable, a 5xx or 429 fails) (GoogleChrome/lighthouse (GitHub), Q102)
llms.txt | Lighthouse | Chrome for Developers (an emerging convention; providing the file is optional) (Chrome for Developers, Q103)
Google's Guide to Optimizing for Generative AI Features on Google Search (Google Search does not use llms.txt; it neither helps nor harms rankings there) (Google Search Central, Q104)

Project decision

Owner decision 2026-10-18: the verdict follows the Lighthouse llms-txt audit (Q102, Q103): a 4xx other than 429 (no file) is not applicable; 5xx, 429 or no content fails; a file passes with an H1, a Markdown link and at least 50 characters. A failed request, a redirect to another host and a bot challenge stay not checked. Only the file at the root of the host is checked. Severity info: the file is optional (Q103) and Google Search ignores it (Q104).