How long does a new page wait for Googlebot? Time to first verified crawl, by template
Sitemap Coverage gains a last verified fetch column, and a new panel shows how long new sitemap URLs wait for their first verified Googlebot crawl, by template, with the URLs still waiting. The crawl record now counts verified and unverified fetches apart.
"Crawled" used to mean "a request wearing Googlebot's name reached this URL". That can be an impersonator, and it cannot tell you whether a page you published last week has been fetched yet.
What changed
- Last verified fetch on Sitemap Coverage: the latest fetch whose identity verified against the operator's published addresses or reverse DNS, next to the old last-crawled column. A dash means no verified fetch is recorded. Sortable.
- Time to first crawl: for sitemap URLs that appeared in the last 30 days (the later of their first sitemap sighting and their
lastmod), by template (first path segment): how many have a verified Googlebot fetch, the median and 90th-percentile hours to it, the URLs still waiting and how old the oldest is, and how many were fetched only by an unverified address wearing Googlebot's name, which never counts as crawled. Google's own rough figures (discovery about 20 hours, sitemap processing about 24 hours) sit alongside for scale. - The crawl record is verification-aware. The nightly aggregator now stores verified counts, the last and first verified fetch and the first fetch of any kind per URL and bot, and no longer sweeps AdsBot-Google or the URL Inspection tool into Googlebot's crawl record.
GET /public/v1/websites/{id}/seo/sitemap/time-to-crawlcarries the same figures (days,bot); the MCP server addsget_sitemap_time_to_crawl(1.26.0) and the CLIseo time-to-crawl(2.6.0).
See time to first crawl in the help.