llms.txt

While standard documentation is built for human eyes, AI agents work best when they have a "map" to navigate your content. An llms.txt file is that map.

What is an llms.txt File?

Think of llms.txt as the robots.txt for the AI era. Instead of telling a crawler what not to index, it tells an LLM what it should prioritize.

Key Features

  • Markdown Format: Human-readable and machine-parsable.

  • Structured Overview: Contains a high-level summary and a list of categorized links.

  • Context-First: Points to "full" versions of documentation (llms-full.txt) that are optimized for LLM context windows, stripped of ads and navigation menus.

How to Work with llms.txt

The file typically lives at the root of your project or website (e.g., yourproject.com/llms.txt).

Step #1: Downloading the File

Step #2: Using it in Development

When you point an AI agent to a repository or a URL, the agent checks for this file. It uses the links provided to "pull" only the relevant Markdown files into its memory, rather than trying to scrape a complex HTML website.

Last updated

Was this helpful?