L.IDX
—
2026
Index · Learn
Articles on llms.txt and AI search.
The long-form library.
Background reading for site owners and small teams who want to ship llms.txt, understand what AI assistants actually do with it, and tell whether the work is paying off. New articles are added when something on the spec or the AI search landscape shifts in a way that’s actionable.
Complete guide · 15 min read
What is llms.txt? The complete guide for site owners
From the GEO/AEO terminology landscape to step-by-step install instructions and the misconceptions that trip site owners up. Written so a non-developer can ship it in an afternoon.
Read →
Reference · 9 min
llms.txt HTTP headers: Content-Type, caching, and redirects
The file is Markdown; the headers decide whether a fetcher can read it. The two good Content-Type values and the three bad ones, charset and the byte-order mark, short public caching with revalidation, the redirects that lose files, the optional Link header from spec v2, what six public files sent on 18 October 2026, config for nginx, Netlify, and Next.js, and the curl commands to check your own.
Read →
Guide · 9 min
llms.txt when you can't upload to the root (Wix, Notion Sites, Studio)
The spec wants a file at /llms.txt and hosted builders do not always give you the root. What Wix, Shopify, STUDIO, Framer, Webflow, Squarespace, and Notion Sites support as of October 2026 (two of them changed this year), how to use Wix's generated file and STUDIO's upload, and the fallbacks for the rest in order: a proxy in front of the domain, a redirect, a plain page.
Read →
Guide · 8 min
llms.txt for local businesses: hours, location, and booking
A shop, clinic, or practice is asked three things: where, when, and how to book. Why local sites keep exactly those facts where a fetch cannot see them (map embeds, images, widgets, PDFs), a structure that puts the answers in dated text, a complete clinic example, how to keep hours identical in the file, the page, LocalBusiness structured data, and your Business Profile, and what the file will not do.
Read →
Guide · 8 min
llms.txt for portfolio and freelancer sites
A portfolio is built to be looked at; an assistant asked about you reads text. The four questions a client puts to an assistant about a person, why galleries, PDF résumés and hero images fail a plain fetch, the structure of a file that answers them, a complete example to copy, what goes stale, what stays private, and the page fixes to make alongside it.
Read →
Guide · 9 min
Google AI Overviews and small sites: what actually helps
Google says there are no additional requirements to appear in AI Overviews or AI Mode, and that the features fan out into subtopic searches and show a wider set of links. What its two pages actually say, four steps for a small site (rule out exclusions, own a subtopic, keep structured data honest, measure in Search Console), and a table of things sold as tactics that do nothing.
Read →
Explainer · 9 min
ClaudeBot, Claude-User, Claude-SearchBot: how Anthropic crawls and what llms.txt changes
Anthropic documents three bots with three jobs: ClaudeBot for training, Claude-SearchBot for the search index, Claude-User for fetches a person asked for. What disallowing each one does, the robots.txt and Crawl-delay commitments, the single IP list, how it differs from OpenAI and Perplexity, two robots.txt recipes, and what llms.txt does and does not change for Claude.
Read →
Guide · 9 min
How to get cited in ChatGPT search results
OpenAI says any public site can appear in ChatGPT search if OAI-SearchBot is allowed, and finds pages through search providers it does not name; the evidence points at Bing. The three OpenAI agents and their jobs, where Bing fits as of October 2026, five steps (allow the bots, get into Bing with IndexNow, make pages quotable, keep facts consistent, measure with utm_source=chatgpt.com), and a diagnostic table.
Read →
Guide · 9 min
How to get your site cited by Perplexity
Perplexity searches the web at question time and cites what it reads. What its bot documentation and help center say, the five steps that follow (let PerplexityBot in, make pages quotable, look like a reviewed source, keep facts consistent, measure), and a diagnostic table for when you are missing from the source list.
Read →
Glossary · 8 min
AEO vs GEO vs SEO (and AIO, LLMO): a plain-language glossary
Five acronyms for two ideas. What each term expands to, who uses it, and where it came from; what is identical under every label; the seven ways the AI side genuinely differs from classic SEO; and which word to use in which room.
Read →
Guide · 9 min
Generative Engine Optimization (GEO) for small sites: the 5 things that matter
What the GEO paper measured, what Google says it rewards, and the five things a small site can do in an afternoon: be fetchable as text, lead with the answer, give assistants something to cite, keep every copy of a fact consistent, and measure. Plus the list of things sold as GEO that you can skip.
Read →
Explainer · 9 min
How AI crawlers differ from Googlebot
Googlebot renders JavaScript, keeps an index, and reads robots.txt by published rules. AI agents are three different jobs (training, search index, user fetch) that read the HTML as served, some at answer time, and two of which say robots.txt may not apply. What the vendors document, compared.
Read →
Checklist · 9 min
12 common llms.txt mistakes (and how to fix each)
What a validator sees over and over, in three groups: delivery (HTML fallbacks, Content-Type, encoding, blocked bots), structure (H1, blockquote, link syntax, relative URLs), and content (dead links, sitemap dumps, empty annotations, nobody maintaining it).
Read →
Guide · 9 min
Stop AI from quoting your old pricing: llms.txt for SaaS pricing pages
An assistant quoting a retired price is reading an old copy. The four places it survives, how to make the current pricing page the copy every reader prefers, what structured data and a dated llms.txt line add, and how to sweep the stale copies.
Read →
Guide · 9 min
llms.txt for e-commerce: summarizing a product catalog for AI
A store's file is a table of contents, not a feed. Categories instead of products, shipping and returns written out in the file, prices and stock left to Product structured data, and a complete example for a fictional bike shop.
Read →
Explainer · 9 min
llms.txt vs schema.org structured data: what each does for AI
One describes a page to a crawler before anyone asks; the other describes a site to an assistant at answer time. What Google's docs and the spec promise, where the two overlap, and how to keep them from contradicting each other.
Read →
Explainer · 9 min
llms.txt for multilingual sites: one file or one per language?
The spec never mentions languages. The convention that works: one file per language at the prefix its pages use, lined up with hreflang's x-default, plus the generated setup behind this site's nine language versions.
Read →
Explainer · 8 min
How big should llms.txt be? Size, link count, and token budgets
The spec sets no limit, so here are the numbers: sizes and link counts from 24 real files, what a kilobyte costs in tokens, where validators and platforms draw lines, and five cutting rules.
Read →
Explainer · 8 min
What goes in the llms.txt Optional section (and what doesn't)
The section written to be skipped: what the spec says, what the 2026 revision changed, how 28 real files use it, and a one-question rule for deciding what belongs there.
Read →
Tutorial · 10 min
Generating llms.txt at build time in Next.js, Astro, and Hugo
The smallest working code for each framework: a force-static Route Handler, an Astro endpoint, and a Hugo custom output format. Plus the pattern that keeps the file honest: one routes list shared with the sitemap.
Read →
Platform guide · 11 min
llms.txt for documentation sites: Docusaurus, Mintlify, GitBook, VitePress
Mintlify and GitBook generate the file for you; Docusaurus and VitePress need a build plugin. What each produces, what real outputs look like on the Vite, Vue, Mintlify, and GitBook docs, and the five editorial choices no generator makes.
Read →
Platform guide · 10 min
llms.txt for Shopify stores: what to put in it
Every Shopify store has served /llms.txt since May 2026, but the default only describes checkout protocols. What the file contains, the six things worth adding, a complete template, and the robots.txt detail that makes inline shipping rules necessary.
Read →
Reference · 11 min
AI crawlers list: which bots to allow in robots.txt (2026)
Every documented AI user agent sorted by job: training crawlers, search indexers, and user-initiated fetchers. Three robots.txt recipes, the two tokens that never appear in logs, and how to verify a bot is real.
Read →
Reference · 10 min
The llms.txt spec, explained line by line
The v2 spec (August 2026) read clause by clause: the one required element, the optional ones, what v2 changed, and the things the spec never says that people assume it does.
Read →
Explainer · 9 min
Does Google use llms.txt? Search, AI Overviews, and what to expect
Google's own guide says Search ignores llms.txt and AI Overviews need no extra files. Every Google statement in date order, the two places the file does appear at Google, and what to do instead.
Read →
Explainer · 10 min
Does ChatGPT actually use llms.txt? What we know in 2026
OpenAI's crawler docs never mention llms.txt, and 2026 log studies see almost no fetches from its bots. What OpenAI has actually said, what the logs show, and why the file is still worth publishing.
Read →
Walkthrough · 11 min
How to write llms.txt by hand: a 30-minute walkthrough
From a blank text file to a validated llms.txt in 30 minutes: what goes in the title, the summary, the notes for the AI, each section and each description, with a complete worked example.
Read →
Annotated examples · 12 min
llms.txt examples: 8 real files, annotated
Eight real, public llms.txt files — from the ten-line one on llmstxt.org to Anthropic's 629-link index — fetched in September 2026 and annotated: what each does well, what to copy, and what to skip.
Read →
Long read · 8 min
llms.txt vs robots.txt vs sitemap.xml
A side-by-side guide to the three small files at the root of your site. What each does, who reads them, and how they work together for AI search.
Read →
Reference · 7 min
llms.txt vs llms-full.txt — when to publish each
llms.txt is the curated table of contents. llms-full.txt is the full-text bundle. Here's how the two differ, when each is worth publishing, and what large sites like Anthropic and Stripe do.
Read →
How-to · 9 min
How to measure the impact of llms.txt and AI search
There is no Google Search Console for ChatGPT yet. Here are the four signals that actually move — server logs, referral traffic, manual citation audits, and on-site behaviour — and how to track each.
Read →
How-to · 6 min
How to check whether AI search has picked up your site
A practical month-one checklist for confirming that ChatGPT, Claude, Perplexity, and Google AI Overviews can actually find and cite your site — using only the tools every site owner already has.
Read →