--- title: "What Is llms.txt? Definition, Format and Who Reads It" description: "llms.txt is a Markdown file at /llms.txt that lists a site's key pages for AI tools. See the format, an example, and who reads it and who does not." url: https://seenlens.com/glossary/llms-txt last_updated: 2026-11-06 --- # What is llms.txt? llms.txt is a Markdown file at /llms.txt that lists the key pages of a site for AI tools. Jeremy Howard proposed it on 3 September 2024. It is a community proposal, not a ratified standard. Google Search says it does not use the file, and no answer engine is shown to rely on it. By SeenLens. Last updated: 2026-11-06 ## Definition An llms.txt file is a curated index of a site, written in Markdown and served at `/llms.txt`. It gives an AI agent a short entry point with links to the pages that matter, usually Markdown versions of those pages. The proposal says the file should stay small enough to fit in a context window. The name looks like robots.txt, but the two files differ. The robots.txt file tells crawlers which paths they may fetch. The llms.txt file lists pages worth reading. It controls nothing and blocks nothing. See [llms.txt vs robots.txt](https://seenlens.com/guides/llms-txt-vs-robots-txt). ## How does llms.txt work? The file has a fixed order. Only the H1 is required: 1. An H1 with the name of the project or site. 2. A blockquote with a short summary. 3. Optional notes in paragraphs or lists, without headings. 4. Zero or more H2 sections. Each one is a list of links, and each link can have notes after a colon. 5. By convention, an H2 section named Optional holds links an agent can skip when it needs a shorter context. ```text # Example Co > Example Co makes scheduling software for dental clinics. ## Product - [How it works](https://example.com/how-it-works.md): The booking flow in four steps - [Pricing](https://example.com/pricing.md): Plans and limits ## Optional - [Company history](https://example.com/about.md) ``` The file can sit at the site root or at a subpath such as `/docs/llms.txt`. A file at a subpath covers the URLs under it. When several files apply, an agent should use the most specific one. ## Who uses llms.txt and who does not? - **Google Search does not use it.** Google's guide to generative AI features says Google Search itself does not use these files. It adds that making them is fine and neither helps nor harms Search visibility. - **OpenAI does not document it.** The OpenAI crawler page lists GPTBot, OAI-SearchBot and ChatGPT-User and does not mention llms.txt. - **Few files get read.** An Ahrefs study of 137,210 domains (May 2026) found that 28% publish an llms.txt file and 97% of those files received no request in the month. Ahrefs treats the 28% as an upper bound. This is a vendor study. - **Some tools do fetch it.** In the same study, GPTBot was the most frequent named AI fetcher of llms.txt files and Claude Code was second. Chrome's experimental Lighthouse Agentic Browsing audit also checks for the file. The evidence supports one conclusion: publishing llms.txt is cheap, and no study shows it improves AI citations. Fix crawler access and page content first. The guide [does llms.txt work?](https://seenlens.com/guides/does-llms-txt-work) weighs the evidence. ## How do I check an llms.txt file? Use the [llms.txt checker](https://seenlens.com/tools/llms-txt-checker). It fetches `/llms.txt`, validates the format, tests up to 10 linked URLs and offers a generator. If your site has no file, follow [add an llms.txt file](https://seenlens.com/fixes/add-llms-txt). The full SeenLens scan also includes an llms.txt check in its agent readiness dimension. ## Related terms - [Markdown content negotiation](https://seenlens.com/glossary/markdown-content-negotiation): serving a Markdown version of each page at its own URL. - [Content signals](https://seenlens.com/glossary/content-signals): a usage preference in robots.txt. - [AI visibility](https://seenlens.com/glossary/ai-visibility): the outcome that these files try to support. - [llms.txt vs agents.md](https://seenlens.com/guides/llms-txt-vs-agents-md) and [GPTBot](https://seenlens.com/bots/gptbot). ## Frequently asked questions ### What is the llms.txt file? It is a Markdown index of a site's key pages, served at `/llms.txt`. It starts with an H1 and usually has a one-sentence summary and lists of links. ### Who created llms.txt? Jeremy Howard proposed it on 3 September 2024. The proposal lives at llmstxt.org, whose current page is version 2, modified on 10 August 2026. ### Is llms.txt an official standard? No. It is a proposed convention published at llmstxt.org. Google Search says it does not use the file. ### Does llms.txt help me rank in Google? No. Google says Google Search does not use llms.txt, and the file does not help or harm Search visibility. ### What is the difference between llms.txt and llms-full.txt? The llmstxt.org format defines llms.txt only. Many sites also publish llms-full.txt, which holds the text of their pages in one file. The [llms.txt checker](https://seenlens.com/tools/llms-txt-checker) notes whether that file exists. ## Sources - [llmstxt.org](https://llmstxt.org/) (proposal, format, location and Optional section; version 2 modified 2026-08-10; checked 2026-11-06) - [Google: Optimizing your website for generative AI features](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide) (Google Search does not use llms.txt, checked 2026-11-06) - [OpenAI: Overview of OpenAI crawlers](https://developers.openai.com/api/docs/bots) (no mention of llms.txt, checked 2026-11-06) - [Ahrefs: llms.txt study](https://ahrefs.com/blog/llmstxt-study/) (137,210 domains, May 2026, vendor study, checked 2026-11-06)