The canonical docs, tools, and reading we rely on when auditing small business websites. External links go to original sources, so you can verify everything for yourself.
Google's own help centre page explaining what AI Overviews are, when they appear, and how sources are selected. The reference point for any argument about what Google says it does.
The developer-facing documentation on AI features in Google Search, including how Google selects and cites sources in generative experiences. Check here before you rewrite any copy for AI Overviews.
OpenAI's page listing every bot it operates, GPTBot, OAI-SearchBot, and ChatGPT-User, with user agent strings and robots.txt rules. The source of truth when you are deciding what to allow.
The launch announcement for ChatGPT Search, with notes on how live web results are surfaced inside ChatGPT. Useful context when you are explaining to a client why Bing indexing now matters.
Anthropic's support article on whether Claude has access to the internet, with notes on citations. For the current ClaudeBot user agent behaviour, see Anthropic support centre, as the specific page moves from time to time.
The main entry point to Anthropic's developer docs. Use it as a jumping-off point to find current information on ClaudeBot, tool use, and how Claude handles web content inside the API.
Perplexity's guide to PerplexityBot and Perplexity-User, with user agents, IP ranges, and the robots.txt rules they honour. The first page to read before allowing or blocking Perplexity at the edge.
Bing's hub for IndexNow, the open protocol for instant URL submission that Bing, Yandex, and several smaller engines support. Since ChatGPT Search uses Bing, this is a fast way to nudge citations.
The official product page for Gemini, the model family behind Google's generative answers. Useful when you need a primary source for capability claims or a pointer to the latest Gemini release notes.
The shared vocabulary for structured data on the web, maintained by Google, Microsoft, Yahoo, and Yandex. Every JSON-LD block you write references types defined here, so bookmark it.
The open specification for instant URL submission, so search engines know the moment a page is added, updated, or removed. Free to implement, and supported by Bing and several other engines.
The proposed text file at the root of a site that tells AI crawlers which pages to read and how to interpret them. Not yet a formal standard, but cheap to publish if you want to signal intent.
The formal specification for the Robots Exclusion Protocol, published by the IETF in 2022. The reference when you are arguing about what a directive should mean, or why a crawler is misbehaving.
The original XML sitemap specification, still used as the baseline by every major search engine. Check here when you are deciding what belongs in a sitemap and how to encode last-modified dates.
Paste a URL or a block of HTML and Google tells you which rich result types it detected, with warnings and errors. The first tool to run after shipping any JSON-LD change.
The official schema.org validator, maintained by the community that defines the vocabulary. It catches structural errors that Google's Rich Results Test ignores, so run both on any important page.
Bing's equivalent of Google Search Console, with URL inspection, sitemap submission, and IndexNow integration. Since ChatGPT Search relies on Bing, a site not indexed here is invisible to ChatGPT.
Google's performance and Core Web Vitals tool, run on real-world Chrome data. Slow pages are deprioritised by every crawler, so fix the red numbers before you argue about schema.
Our free tool. Drop in a URL, get a quick snapshot of schema, sitemap, robots rules, and which AI crawlers are allowed. Use it as a first pass before booking an audit.
Rand Fishkin's write-ups on zero-click search, audience research, and AI search analysis. Less shouting than most SEO blogs, more data, and usually a sharper read on where clicks actually go.
Practical, implementation-focused SEO writing from Aleyda Solis. Her recent posts on AI search and structured data are a useful counterweight to vendor content, with real examples and clear caveats.
Google's official blog for search updates, from ranking changes to new schema support. Subscribe if you want primary sources rather than second-hand takes, and check it before trusting a hot-take post.
Daily news coverage of search, AI search, and the wider marketing stack. Not every story is essential, but it is the fastest way to spot a change at Google, Bing, or OpenAI worth a closer look.
Another daily news and how-to site, with a broader practitioner audience. Use it as a second source when a story breaks, and to find practical walk-throughs for tools and platforms.
Ann Handley, Wiley, 2022. A working writer's guide to clear marketing prose. Not an AI search book, but the habits it teaches, short sentences, specific nouns, no jargon, are what AI models quote.
Eric Enge, Stephan Spencer, Jessie Stricchiola, O'Reilly, 2024 edition. The long, textbook treatment of classical SEO. The fundamentals still underpin everything an AI search product does on top.
Steve Krug, New Riders. A short book on web usability that still sets the bar. If your page is hard for a human to skim, an AI model will struggle to summarise it cleanly too.
If you want to go deeper on anything covered above, start here.
That is the short list. It gets pruned as tools close and added to when something new proves its worth. If you think a resource belongs here, or one of these has gone downhill, tell us.
Happy to answer anything, Bob. [email protected]