OAI-SearchBot: the Crawler Behind ChatGPT Search

OAI-SearchBot is the OpenAI crawler that surfaces websites in ChatGPT's search features. If it cannot read your site, your pages cannot be cited in ChatGPT search answers, whatever you do about GPTBot.

Updated · 2 min read · By the VisibleReady team

OperatorOpenAI
PurposeChatGPT search
robots.txt tokenOAI-SearchBot
Follows robots.txtYes
Used for trainingNo (that is GPTBot)

What OAI-SearchBot does

OpenAI describes OAI-SearchBot as the bot that surfaces websites in ChatGPT's search features. When ChatGPT searches the web to answer a question, the pages it can show and link come from what this crawler has been allowed to read. OpenAI's guidance for publishers who want to appear in ChatGPT search summaries is simply to make sure they are not blocking OAI-SearchBot.

It is separate from GPTBot, which collects training data, and from ChatGPT-User, which opens a page when a person asks ChatGPT to. OpenAI says each setting "is independent of the others".

OAI-SearchBot user agent

OpenAI documents the full string like this (the version numbers may change):

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot

In robots.txt, use the token OAI-SearchBot. To verify a request, check its IP against openai.com/searchbot.json.

How to allow OAI-SearchBot

It is allowed by default unless a rule or a firewall blocks it. To be explicit:

User-agent: OAI-SearchBot
Allow: /

Then check for the blocks robots.txt cannot show: firewall rules, bot-protection services and CDN settings that refuse AI crawlers. The AI visibility checker requests your homepage as OAI-SearchBot and tells you if it gets through.

How to block OAI-SearchBot (and what happens)

User-agent: OAI-SearchBot
Disallow: /

Blocking it keeps your pages out of ChatGPT search answers. If what you want is to stay out of search for a single page, OpenAI points publishers to the noindex meta tag instead, and notes that the crawler must be allowed to read the page to see that tag. OpenAI says robots.txt changes take about 24 hours to apply.

How to see its visits

In your server or CDN logs, filter the user agent by OAI-SearchBot. Pages it fetches are pages ChatGPT search can use. Visitors who then click through from ChatGPT arrive with utm_source=chatgpt.com in the link, so they show up in your analytics as a separate source.

For the full process of earning those citations, see How to get cited by ChatGPT.

Source: OpenAI, "Overview of OpenAI Crawlers".

Can AI assistants read your site?

Check your homepage as ten AI crawlers, including OAI-SearchBot, ClaudeBot and PerplexityBot. Free, about 30 seconds, no signup.

https://
VisibleReady

Create your free account

One click with Google. No card and no password to remember.

  1. 1Sign in with Google
  2. 2Answer 4 quick questions
  3. 3See what your bot found
or
Continue with GitHub

  • Free plan, no card
  • Nothing goes live without your OK
  • We never see your Google password

By continuing you agree to our privacy policy.