L.IDX
—
2026
Index · Learn
Articles on llms.txt and AI search.
The long-form library.
Background reading for site owners and small teams who want to ship llms.txt, understand what AI assistants actually do with it, and tell whether the work is paying off. New articles are added when something on the spec or the AI search landscape shifts in a way that’s actionable.
Complete guide · 15 min read
What is llms.txt? The complete guide for site owners
From the GEO/AEO terminology landscape to step-by-step install instructions and the misconceptions that trip site owners up. Written so a non-developer can ship it in an afternoon.
Read →
Explainer · 8 min
What goes in the llms.txt Optional section (and what doesn't)
The section written to be skipped: what the spec says, what the 2026 revision changed, how 28 real files use it, and a one-question rule for deciding what belongs there.
Read →
Tutorial · 10 min
Generating llms.txt at build time in Next.js, Astro, and Hugo
The smallest working code for each framework: a force-static Route Handler, an Astro endpoint, and a Hugo custom output format. Plus the pattern that keeps the file honest: one routes list shared with the sitemap.
Read →
Platform guide · 11 min
llms.txt for documentation sites: Docusaurus, Mintlify, GitBook, VitePress
Mintlify and GitBook generate the file for you; Docusaurus and VitePress need a build plugin. What each produces, what real outputs look like on the Vite, Vue, Mintlify, and GitBook docs, and the five editorial choices no generator makes.
Read →
Platform guide · 10 min
llms.txt for Shopify stores: what to put in it
Every Shopify store has served /llms.txt since May 2026, but the default only describes checkout protocols. What the file contains, the six things worth adding, a complete template, and the robots.txt detail that makes inline shipping rules necessary.
Read →
Reference · 11 min
AI crawlers list: which bots to allow in robots.txt (2026)
Every documented AI user agent sorted by job: training crawlers, search indexers, and user-initiated fetchers. Three robots.txt recipes, the two tokens that never appear in logs, and how to verify a bot is real.
Read →
Reference · 10 min
The llms.txt spec, explained line by line
The v2 spec (August 2026) read clause by clause: the one required element, the optional ones, what v2 changed, and the things the spec never says that people assume it does.
Read →
Explainer · 9 min
Does Google use llms.txt? Search, AI Overviews, and what to expect
Google's own guide says Search ignores llms.txt and AI Overviews need no extra files. Every Google statement in date order, the two places the file does appear at Google, and what to do instead.
Read →
Explainer · 10 min
Does ChatGPT actually use llms.txt? What we know in 2026
OpenAI's crawler docs never mention llms.txt, and 2026 log studies see almost no fetches from its bots. What OpenAI has actually said, what the logs show, and why the file is still worth publishing.
Read →
Walkthrough · 11 min
How to write llms.txt by hand: a 30-minute walkthrough
From a blank text file to a validated llms.txt in 30 minutes: what goes in the title, the summary, the notes for the AI, each section and each description, with a complete worked example.
Read →
Annotated examples · 12 min
llms.txt examples: 8 real files, annotated
Eight real, public llms.txt files — from the ten-line one on llmstxt.org to Anthropic's 629-link index — fetched in September 2026 and annotated: what each does well, what to copy, and what to skip.
Read →
Long read · 8 min
llms.txt vs robots.txt vs sitemap.xml
A side-by-side guide to the three small files at the root of your site. What each does, who reads them, and how they work together for AI search.
Read →
Reference · 7 min
llms.txt vs llms-full.txt — when to publish each
llms.txt is the curated table of contents. llms-full.txt is the full-text bundle. Here's how the two differ, when each is worth publishing, and what large sites like Anthropic and Stripe do.
Read →
How-to · 9 min
How to measure the impact of llms.txt and AI search
There is no Google Search Console for ChatGPT yet. Here are the four signals that actually move — server logs, referral traffic, manual citation audits, and on-site behaviour — and how to track each.
Read →
How-to · 6 min
How to check whether AI search has picked up your site
A practical month-one checklist for confirming that ChatGPT, Claude, Perplexity, and Google AI Overviews can actually find and cite your site — using only the tools every site owner already has.
Read →