On 11 July 2026 a reference domain published a research series and updated its sitemap. ClaudeBot fetched sitemap.xml at 03:37:05, and fetched a brand-new page from that series at 04:23:06 — 46 minutes from map to page. Across the same 24-hour window, PerplexityBot crawled the domain three separate times, hitting 35+ pages in one twenty-second burst across eight parallel workers, and never fetched sitemap.xml once. Its pattern, every visit: fetch robots.txt, then walk links.
Sitemap-driven and link-driven crawlers do not fail in the same way. A sitemap-driven crawler will find an orphaned page — one with no inbound links — because the sitemap hands it the URL. A link-driven crawler cannot. It has no path. On the domain observed, a research directory was listed in the sitemap and linked from only 27 of 284 pages. ClaudeBot found the entire series within minutes. Perplexity crawled straight past it, three times, because every section it entered was a dead end.
Stop treating 'it's in the sitemap' as distribution. It is distribution for exactly one class of crawler. For the other class, only the link graph exists. If a page matters, it needs both: a sitemap entry and a real inbound link path from pages that get crawled. The failure is silent — the page returns 200 to anyone who asks, so nothing in your logs looks broken. It simply never gets asked.