Site Checks: standing safety checks for crawler access and security hygiene
A new Site Checks page continuously watches your logs for the problems that silently hurt search visibility: Googlebot being blocked or rate-limited, unreliable robots.txt serving, sensitive files exposed, and alerts you the moment a check starts failing.
Some of the most damaging site problems never show up as errors you'd notice. A rate limiter that catches Googlebot. A robots.txt that intermittently returns errors, causing Google to back off crawling your entire site. These issues can cost months of organic traffic before anyone connects the symptoms to the cause, we've seen a real case where exactly this took a team months of external investigation to pin down.
Salience now checks for these automatically. The new Site Checks page (under Analytics) evaluates your last 7 days of traffic every night against a set of standing safety checks:
Crawler access
- Googlebot isn't being blocked or rate-limited
- Search crawlers aren't receiving 429 responses
- robots.txt is served reliably to crawlers (no 4xx/5xx)
- robots.txt fetch cadence looks healthy (excessive refetching suggests it's being served uncacheably)
- Your sitemap is being fetched and returning 200s
- AI crawler access, which AI bots are allowed, partially blocked, or fully blocked (with no judgement: if blocking is intentional, that's fine)
Security hygiene
- No sensitive files (
.env,.git, backups, config) returning 200 to probes - Bot impersonation levels, how much claimed-Googlebot traffic fails IP verification
- Unidentified bot share, how much of your bot traffic can't be classified
Every verdict comes with the evidence behind it: expand any check to see the actual numbers from your logs. Checks that lack enough traffic to judge are marked N/A rather than guessed at.
When a check starts failing: Googlebot suddenly blocked, robots.txt suddenly erroring, an alert goes out through your email streams immediately. Recoveries are announced too. Steady state stays quiet: the page shows standing status, alerts fire only on change.