What is OAI-SearchBot?
OAI-SearchBot is the crawler OpenAI uses to surface websites in ChatGPT's search features. Its robots.txt token is OAI-SearchBot. OpenAI says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links. OpenAI recommends allowing it. Verified 2026-11-06.
By SeenLens · Last updated
Free, no sign-up. We read robots.txt and request the page with a browser and four crawler user agents. We do not store your page text.
OAI-SearchBot fact sheet
| Fact | Value |
|---|---|
| Operator | OpenAI |
| Purpose | Search index. It surfaces websites in search results in ChatGPT's search features. |
| User agent string | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot (OpenAI's example; the version number may change) |
| User agent string for robots.txt fetches | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; robots.txt; +https://openai.com/searchbot |
| robots.txt token | OAI-SearchBot |
| Published IP list | openai.com/searchbot.json |
| Obeys robots.txt | Yes, as OpenAI documents it. OpenAI lists OAI-SearchBot as a robots.txt tag, and says a search change takes about 24 hours (OpenAI). |
| Related bots | GPTBot (training), ChatGPT-User (user requests), OAI-AdsBot (ad landing page checks) |
| Last verified | 2026-11-06 |
What does OAI-SearchBot do?
OAI-SearchBot crawls pages so that ChatGPT can find them when it answers with search. It is a search crawler, and OpenAI keeps it separate from the training crawler GPTBot. OpenAI says each setting is independent of the others.
OpenAI also lists OAI-AdsBot. That bot validates landing pages submitted as ads on ChatGPT. OpenAI says OAI-AdsBot visits only pages submitted as ads and that its data is not used to train generative AI foundation models. The OpenAI page lists no robots.txt token for OAI-AdsBot.
OpenAI says OAI-SearchBot may add a robots.txt marker to its user agent string when it fetches robots.txt files. To confirm that a request is real, compare its IP address with searchbot.json.
How do I allow or block OAI-SearchBot?
OpenAI recommends allowing OAI-SearchBot in robots.txt and allowing requests from its published IP ranges. An explicit allow group looks like this:
User-agent: OAI-SearchBot
Allow: /To block OAI-SearchBot from the whole site:
User-agent: OAI-SearchBot
Disallow: /To stay in ChatGPT search and opt out of training, allow OAI-SearchBot and block GPTBot:
User-agent: OAI-SearchBot
Allow: /
User-agent: GPTBot
Disallow: /OpenAI says it can take about 24 hours after a robots.txt update for its search systems to adjust.
Does blocking OAI-SearchBot affect ChatGPT search and citations?
Yes. OpenAI says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers. Such sites can still appear as navigational links.
Blocking GPTBot alone does not remove you from ChatGPT search. OpenAI documents the two bots as independent settings. Allowing OAI-SearchBot does not guarantee that ChatGPT cites a page. It only removes one reason for exclusion.
How do I check whether my site blocks OAI-SearchBot?
Run the AI crawler checker above, which is preselected for OAI-SearchBot. It shows whether your robots.txt allows the bot for your page path and which rule matched. It also requests the page with the OAI-SearchBot user agent and compares the result with a browser request. A 403 or a much smaller response means a firewall or CDN rule blocks the bot even if robots.txt allows it.
In your own logs, search for OAI-SearchBot and check the source IPs against searchbot.json.
What are the common OAI-SearchBot mistakes?
- robots.txt allows the bot but the firewall returns 403. The firewall decides first. Allow the OAI-SearchBot user agent together with OpenAI's published IP ranges. See AI crawlers blocked by a firewall.
- **A
User-agent: *group withDisallow: /.** Any bot without its own group falls back to this rule. Add a namedOAI-SearchBotgroup. A crawler uses the group that matches its token and uses*only when nothing matches (RFC 9309). - Blocking GPTBot and expecting search to stop, or the reverse. The two bots have separate tokens. Set each one on purpose.
- Managed AI bot blocking at the CDN. Cloudflare's AI Crawler and Agent access policy has separate actions for search, agent and training behavior. A Block action for search crawlers can stop OAI-SearchBot at Cloudflare's edge. See robots.txt blocks AI crawlers.
- Expecting an instant effect. OpenAI says a robots.txt update can take about 24 hours to reach its search systems.
OAI-SearchBot FAQ
What is OAI-SearchBot?
OAI-SearchBot is OpenAI's crawler for ChatGPT's search features. It surfaces websites in ChatGPT search results.
What is the OAI-SearchBot user agent?
OpenAI's example string ends with compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot. The version number may change, so match on OAI-SearchBot.
Does blocking GPTBot affect ChatGPT search?
No, according to OpenAI. GPTBot is for training. OAI-SearchBot controls search, and each setting is independent.
Should I allow OAI-SearchBot?
OpenAI recommends allowing OAI-SearchBot in robots.txt and allowing its published IP ranges if you want your site to appear in ChatGPT search.
Does OAI-SearchBot use my content for training?
OpenAI assigns training to GPTBot and search to OAI-SearchBot. OpenAI also says that if you allow both, it may use the results of one crawl for both purposes.
Sources
- OpenAI: Overview of OpenAI crawlers (user agent strings, token, IP list URL, 24 hour note, OAI-AdsBot; checked 2026-11-06)
- OpenAI: searchbot.json (published IP prefixes; the file loaded on 2026-11-06)
- Cloudflare: AI Crawler and Agent access policies (Search, Agent and Training actions; checked 2026-11-06)
- IETF RFC 9309: Robots Exclusion Protocol (group selection and the
*fallback; checked 2026-11-06)