Just as robots.txt controls crawl access and sitemap.xml lists pages, llms.txt is a Markdown-formatted file that provides AI language models with a curated overview of a website. It typically includes:
- A brief description of what the site and organisation do
- Links to the most important pages (documentation, products, about, contact)
- Links to content the site considers authoritative
Why it matters: AI answer engines like ChatGPT, Perplexity, and Claude retrieve information from the web when answering questions. A well-structured llms.txt helps these systems understand what a site is about and which content is most relevant — increasing the likelihood of accurate citation.
One prerequisite is worth checking before writing the file: an llms.txt only helps if the engines reading it are allowed to fetch your site at all. A quick crawler checker run will confirm that before you invest in the content.
Format: The file lives at https://example.com/llms.txt, written in plain Markdown, kept under a few hundred lines. An optional llms-full.txt can include more complete documentation.
What SearchVitals checks: The GEO / AI Visibility audit verifies whether an llms.txt file is present, reachable, and correctly formatted. Missing or broken llms.txt is flagged as a warning-level finding.
Related: robots.txt, AI crawlers, AI Overviews