How often does each AI crawler actually re-visit a site?

IEO Engine Research · Published 2026-07-11 · Measured from production server logs
The cadences differ by more than an order of magnitude between agents, so a single figure is meaningless. Measured across a 24-hour window in July 2026: ClaudeBot polled the sitemap roughly every two hours. PerplexityBot returned about every three hours. GPTBot visited the domain twice in the entire day. YandexBot, once triggered via IndexNow, swept 33 pages in 14 seconds and kept returning.

Measured cadences, 10–11 July 2026

ClaudeBot — sitemap poll approximately every 2 hours, consistently. Observed at 19:40, 21:47, 23:35, 01:35, 03:37.

PerplexityBot — full crawl roughly every 3 hours (00:21 and 03:25), with a light trickle of individual page fetches between.

GPTBot — 2 hits in 24 hours on the domain observed. Extremely low frequency.

YandexBot — near-zero before an IndexNow submission; afterwards the most active agent on the property, sweeping 33 pages in 14 seconds.

Why an average would mislead you

These are not variations around a mean. They are different behaviours. A crawler that visits every two hours and one that visits twice a day are not doing the same job at different speeds — they are running different discovery strategies. Any figure that averages them describes no real agent.

The practical consequence

Judging a change by 'have the AI crawlers picked it up' requires knowing which crawler you are waiting on. A fix that ClaudeBot sees within two hours may take GPTBot a day or more to notice — and there is no mechanism to hurry it.

Primary source: this answer is drawn from production access logs across four live deployments, not from vendor documentation or third-party tooling.
Full data and method: read the underlying field note →
All field notes: IEO Engine Research →