KnownByLLM

Explainer · 10 min read

Does ChatGPT actually use llms.txt?

What OpenAI has documented, what server logs show, and what that means for your site, as of September 2026.

It is the question behind every llms.txt decision: if the biggest assistant does not read the file, why bother? This article separates what OpenAI has actually said from what people infer, and looks at the public evidence about whether ChatGPT’s crawlers ever request the file.

Everything here was checked against OpenAI’s own documentation and public sources on 21 September 2026. Where the answer is “nobody knows”, the article says so rather than guessing.

The 30-second answer: does ChatGPT use llms.txt?

There is no evidence that it does, and OpenAI has never said it does. OpenAI’s crawler documentation describes four bots and how each treats robots.txt; it does not mention reading llms.txt from other sites at all. The independent server-log write-ups published in 2026 find OpenAI’s bots requesting robots.txt thousands of times and llms.txt almost never.

Two things are still true. OpenAI publishes an llms.txt for its own developer documentation, so it clearly considers the format useful for agents that read docs. And ChatGPT visits ordinary web pages a great deal, through OAI-SearchBot and ChatGPT-User. What gets you into ChatGPT answers today is being fetchable as plain HTML with those bots allowed, not the presence of a special file.

The practical conclusion: publish llms.txt because it is cheap and other readers use it, but do not expect it to change what ChatGPT says about you, and do not build a forecast on it.

What OpenAI has actually documented

OpenAI maintains one page describing its crawlers (as of September 2026 it lives under the developer documentation, at developers.openai.com/api/docs/bots). It lists four user agents.

BotStated purposerobots.txtWhat it means for you
OAI-SearchBotSurface websites in ChatGPT's search resultsRespected; must be allowed to appear in searchThe one that decides whether ChatGPT search can show you
ChatGPT-UserFetch pages when a user's action in ChatGPT or a custom GPT needs themMay not apply, because the fetch is user-initiatedAnswer-time visits; not used for crawling or search inclusion
GPTBotCrawl content that may be used to train foundation modelsRespected; disallow to opt out of trainingA training crawler; search inclusion is governed by OAI-SearchBot
OAI-AdsBotValidate the safety of pages submitted as ads on ChatGPTNot the point; only touches advertised pagesIrrelevant unless you buy ChatGPT ads

Read that page for the word llms.txt and you find it once, in the opening line that points at OpenAI’s own documentation index. There is no sentence saying any of these bots looks for the file on your site, and no instruction to publish one. OpenAI also publishes machine-readable IP ranges for each bot (openai.com/searchbot.json, chatgpt-user.json, gptbot.json), which is the reliable way to confirm a log entry really came from them.

Why the distinction between the bots matters

Most advice online treats “ChatGPT” as one crawler. It is three with different jobs. Blocking GPTBot to keep your content out of training is a legitimate choice; OpenAI describes it as the training crawler and ties search inclusion to OAI-SearchBot, so blocking the latter is what removes you from ChatGPT search. And ChatGPT-User may ignore robots.txt entirely because a person asked for the page. None of that has anything to do with llms.txt, which is the point.

What OpenAI does with llms.txt on its own sites

OpenAI is itself an adopter. Fetched on 21 September 2026, developers.openai.com/llms.txt is a hub file of about 6 KB and 38 links that points to separate llms.txt indexes for the API, Codex, plugins, the Ads API, the Cookbook and more, plus a short list of common tasks.

# OpenAI Developers

> Complete documentation hub for OpenAI API, Ads, Plugins, Workspace Agents, Codex, Agentic Commerce, the developer blog, Cookbook, learning resources, and learning tracks.

Start with the product index that matches your task. Product indexes route to detailed pages; documentation indexes also provide Markdown pages and full-document exports.

## Documentation sets
- [OpenAI API guides and reference](https://developers.openai.com/api/llms.txt): Quickstart, Responses API, ...
- [Codex developer tools](https://developers.openai.com/codex/llms.txt): Codex CLI, IDE, cloud, config.toml, ...
- [ChatGPT product documentation](https://learn.chatgpt.com/llms.txt): ChatGPT Work, administration, ...

## Common tasks
- [OpenAI API quickstart](https://developers.openai.com/api/docs/quickstart.md): Make your first API request with the OpenAI SDK and Responses API.

This is easy to misread. It shows that OpenAI thinks llms.txt is a good way for agents and coding tools to navigate documentation, and it is consistent with the spec’s own note that AI labs publish the file for their developer docs. It is not a statement that the ChatGPT product reads the file from your domain. A company can serve a format without consuming it.

What server logs show

OpenAI does not publish crawl statistics, so the only evidence about what its bots request is what site owners see in their own logs. Two public write-ups from 2026 are worth reading because they state their method and their numbers.

  • A 12-week panel study (published 27 July 2026). EZY.ai monitored 83 sites that served llms.txt between 27 April and 19 July 2026. It counted 3,990 robots.txt requests from OpenAI against 7 llms.txt fetches. Anthropic’s numbers were similar (3,120 against 9), Perplexity fetched it zero times, and the one crawler that requested it routinely was Meta’s, at 193.
  • A 14-day single-site log (published 3 August 2026). Saaslinks logged 151 AI-crawler requests to its pages from 18 named bots between 20 July and 3 August 2026, with ChatGPT-User and OAI-SearchBot making up about 90% of them. Requests for /llms.txt from any AI crawler: zero, although the file had been live since June.

Both samples are small and identify bots by user agent, so treat the exact counts loosely. But they point the same way, and nobody has published the opposite: a log where OpenAI’s bots fetch llms.txt as a matter of course.

The second study also shows the other half of the picture. ChatGPT is not ignoring the site; it is the biggest AI visitor there. It is reading the pages, not the index.

How ChatGPT actually finds and cites your pages

Given the above, the levers that matter for ChatGPT are ordinary ones. As of September 2026, based on OpenAI’s documentation and observed behaviour:

  • Allow OAI-SearchBot in robots.txt. OpenAI states sites must allow it to appear in ChatGPT search. A blanket block on unknown bots, common in WAF defaults, will exclude you.
  • Do not block ChatGPT-User. It fetches the page a user is asking about at answer time. If it cannot load the page, the assistant answers from memory or cites someone else.
  • Serve readable HTML. The fetch is a plain HTTP request. Content that only exists after heavy client-side rendering, behind a consent wall, or in a PDF is easy to lose.
  • Say plain, quotable things. A page with a clear first paragraph, real numbers and a date is what an assistant can lift into an answer. This is the same advice as for Perplexity or Google AI Overviews.

Notice that llms.txt is not on that list. It does not hurt, and the discipline of writing one usually improves the pages themselves, but it is not what OpenAI’s bots are looking at.

So should you still publish it?

Yes, with the right expectations. The case for publishing has not changed; only the claim that ChatGPT reads it has.

  • It costs almost nothing. Thirty minutes to write, no maintenance beyond keeping links alive.
  • Other readers exist. Coding agents and documentation tools follow the file, the spec notes that Chrome’s Lighthouse now audits for it, and the log study above found Meta’s crawler requesting it regularly.
  • It is ready if OpenAI changes course. Support could be added without announcement, and the log evidence would show it within weeks.
  • It forces clarity. Writing a one-sentence description of each important page is a useful exercise whether or not any bot reads the result.

What you should stop doing is treating the file as a ChatGPT visibility tactic, promising it to clients as one, or reading a week without an llms.txt fetch as a sign something is broken. Nothing is broken; that is the normal state in 2026.

See what an AI crawler gets from your site

Enter your URL and the checker fetches your pages the way a bot does, reports what is reachable, and drafts an llms.txt from what it found. Free, no account.

Run the check →

How to check your own logs

You do not have to take anyone’s word for this. Any access log answers the question for your site in a minute.

# Requests from OpenAI's bots, all paths
grep -Ei "OAI-SearchBot|ChatGPT-User|GPTBot" access.log | wc -l

# Of those, how many asked for llms.txt?
grep -Ei "OAI-SearchBot|ChatGPT-User|GPTBot" access.log | grep -c "/llms.txt"

# Which paths does ChatGPT actually fetch?
grep -Ei "OAI-SearchBot|ChatGPT-User" access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head

If the first number is healthy and the second is zero, your site matches the public studies. Confirm a suspicious hit by checking its IP against OpenAI’s published ranges; user-agent strings are trivially faked.

What would change this answer

This article will be updated if any of the following happens: OpenAI documents llms.txt handling on its crawler page; a published log study shows OAI-SearchBot or ChatGPT-User requesting the file routinely; or ChatGPT begins citing llms.txt URLs directly in answers. Until then, the honest answer to “does ChatGPT use llms.txt?” is: not that anyone can show.

FAQ

Has OpenAI ever said that ChatGPT reads llms.txt?

Not as of September 2026. OpenAI's crawler documentation describes OAI-SearchBot, GPTBot, ChatGPT-User and OAI-AdsBot, explains how each treats robots.txt, and says nothing about reading llms.txt from other sites. The only llms.txt OpenAI talks about is the one it publishes for its own developer documentation.

Then why does OpenAI publish an llms.txt itself?

Because the file is useful to agents and coding tools that read documentation, and OpenAI's docs are heavily used that way. developers.openai.com/llms.txt is a hub that links to separate llms.txt files for the API, Codex, plugins and other doc sets. That tells you OpenAI thinks the format is worth serving; it does not tell you ChatGPT consumes it from your site.

Will publishing llms.txt get my site into ChatGPT search?

No. Inclusion in ChatGPT search is governed by OAI-SearchBot, which must be allowed in robots.txt, and by whether your pages can be fetched and read as ordinary HTML. llms.txt does not replace either. If OAI-SearchBot is blocked, no llms.txt will help; if it is allowed, llms.txt is at most a small extra.

Do the OpenAI bots ever fetch llms.txt at all?

Rarely, according to the public server-log write-ups available in 2026. One 12-week study of 83 sites counted 7 llms.txt fetches from OpenAI against about 4,000 robots.txt requests; a separate 14-day study of one site saw none. Both are small samples, but they agree, and neither found OpenAI treating the file as a regular crawl target.

Should I still publish llms.txt if ChatGPT ignores it?

Yes, as long as you treat it as cheap insurance rather than a traffic lever. It costs a few minutes, other consumers do read it (the same log study found Meta's crawler fetching it regularly, and coding agents and documentation tools use it), Chrome's Lighthouse now audits for it, and if OpenAI adds support later you are already done. Just do not put it in a forecast.

How can I check whether ChatGPT is visiting my site?

Search your access logs for the user agents OAI-SearchBot, ChatGPT-User and GPTBot, and filter by path to see whether /llms.txt is among the requests. To confirm a hit really came from OpenAI, compare the IP against the published ranges at openai.com/searchbot.json, openai.com/chatgpt-user.json and openai.com/gptbot.json.

Next steps