Guides: AI Visibility and Automated SEO
How AI assistants and search engines find, read and cite websites, and what to change so yours is one of them.
Getting into AI answers
How to Get Cited by ChatGPT (and Rank in Its Answers)How ChatGPT picks the sources it cites, and 8 steps to get your site into its answers: OAI-SearchBot access, indexable pages, answer-first writing, tracking.LLM SEO: How to Get Your Site Into ChatGPT, Claude and Perplexity AnswersLLM SEO is how a site gets found and cited by ChatGPT, Claude, Perplexity, Gemini and Copilot: which crawlers matter, robots.txt rules and a 7-step checklist.
AI crawlers
AI Crawlers List: Every AI Bot That Visits Your WebsiteEvery major AI crawler in one table: GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, Google-Extended and more, with robots.txt names and a ready robots.txt.ClaudeBot: Anthropic's Crawler, User Agent and robots.txt RulesClaudeBot is Anthropic's training crawler. Its user agent, robots.txt rules to allow or block it, Crawl-delay, IP list and how it differs from Claude-SearchBot.GPTBot: OpenAI's Training Crawler, User Agent and robots.txtGPTBot is OpenAI's training crawler. Its user agent, the robots.txt rule to block it, its IP list, and why blocking it does not remove you from ChatGPT search.OAI-SearchBot: the Crawler Behind ChatGPT SearchOAI-SearchBot decides whether your pages can appear in ChatGPT search answers. Its user agent, robots.txt rules, IP list, and how it differs from GPTBot.PerplexityBot: Perplexity's Search Crawler, User Agent and robots.txtPerplexityBot surfaces and links websites in Perplexity answers. Its user agent, robots.txt rules, IP list, and how it differs from Perplexity-User.