Why is my site crawled by AI but never cited?

IEO Engine Research · Published 2026-07-11 · Measured from production server logs
Because ingestion and retrieval are different systems and success at one does not produce the other. Being crawled adds you to a corpus. A separate selection step, running on different criteria, decides whether to quote you at answer time. In July 2026 a platform fetched a page with a clean HTTP 200 and then cited three other sources in its answer. The page was available. It was simply not chosen.

The three failure modes

1. Not in the retrieval index at query time. Ingestion into a corpus is not the same as membership in the live retrieval index.

2. No query-class match. If a page is written in vocabulary nobody searches for — particularly invented terminology — no query will ever match it, regardless of quality.

3. No external corroboration. A page that cites nothing can be cross-checked against nothing. A model treats it as an unverifiable outlier and prefers a source it can confirm.

The retrieval trigger nobody mentions

Retrieval fires on the model's uncertainty, not on your quality. Asked a question it is confident about, a model answers from weights and never searches at all — your page is irrelevant no matter how good it is. Observed retrievals cluster hard in the zones where a model knows its training is insufficient: current pricing, local specifics, recent regulatory change, and dated facts. Content that competes on questions a model already answers confidently will be ingested and never retrieved.

The counterintuitive consequence

There is an inverse relationship between query volume and retrieval probability. High-volume queries are high-volume because they are common — which means abundant training data and a confident model that will not look anything up. The queries that trigger retrieval are the ones where the model knows it might be stale or wrong.

Primary source: this answer is drawn from production access logs across four live deployments, not from vendor documentation or third-party tooling.
Full data and method: read the underlying field note →
All field notes: IEO Engine Research →