Bots are classed by what they do: ads crawlers, SEO tools and Applebot as search
Ads crawlers (AdsBot-Google, Microsoft's AdIdxBot, OAI-AdsBot) are their own class and no longer inflate Googlebot figures, SEO tools are a class rather than scrapers, Applebot counts as search, and Googlebot-Image is named. Older rows are corrected by bot name so a window that straddles the change reads consistently.
A bot taxonomy is only useful if it matches how the bots are used. Google's ads crawler fetches landing pages for ad quality, not for the organic index, yet it was counted under search; Microsoft's AdIdxBot was folded into Bingbot; Ahrefs and Semrush sat with anonymous scrapers; Applebot, which indexes for Siri, Spotlight and Safari, was counted as an AI crawler.
What changed
- Ads crawlers are their own class: AdsBot-Google, AdsBot-Google-Mobile, Mediapartners-Google, Microsoft's AdIdxBot and OpenAI's OAI-AdsBot. Each verifies against its operator's published ranges (Google's special-crawlers list, Bing's ranges, OpenAI's adsbot list). The SEO pages' default crawler scope no longer includes them, so Googlebot figures are organic only and AdsBot can be compared against Googlebot on the same URLs.
- SEO tools are a class: AhrefsBot, SemrushBot, Majestic, DotBot, Rogerbot, Screaming Frog, SEOkicks, DataForSEO, Serpstat and BLEXBot. They are named, documented and mostly verifiable by reverse DNS; "scraper" now means a bot without a documented identity or purpose.
- Applebot is search, with a note: Apple's AI training opt-out is the robots.txt token Applebot-Extended, and the requests still arrive as Applebot. PetalBot (Huawei's Petal Search) is search too.
- Google's named crawlers are named: Googlebot-Image, Googlebot-Video and Googlebot-News appear as themselves instead of under Googlebot; plain Googlebot is the desktop crawler and Googlebot Smartphone the mobile one, on every feed.
- History reads consistently. Categories are stamped as requests arrive, so older rows for a bot that moved class still carry the old one. The bots overview, its category filter and the category timelines correct them by bot name, on the Athena and ClickHouse paths alike, and the daily rollups have been recomputed. Rows an ads crawler left before it had a name of its own stay under Other.
- The API's bots response carries the new
adsandseocategories incategoriesandhourly,category=adsandcategory=seofilter the list, andGET /public/v1/definitionsexplains each class (bot_category_ads,bot_category_seo,bot_category_search,bot_category_scraper,googlebot_smartphone).
See bot categories in the help.