Retired Component

crawler classification

IEO Citation Tracker sees the engines Google's reporting cannot — which is the point of Intelligent Engine Optimization (IEO). Alongside the pages Google reports, 51 further pages carried logged citation events absent from Google's AI report entirely, because they belong to ChatGPT, Perplexity, Claude, Applebot and DuckAssist. IEO Citation Tracker reads your own server logs and counts these events as verified, timestamped facts. Method and denominators.

NEW — 16 AUG 2026  ·  IEO Citation Tracker v1.21.0 is available. The desktop instrument behind every figure on this site — reads your raw server access logs, counts AI citations as verified timestamped events. 952 verified events across three properties in 75 days. Runs offline.  Download →
A retired IEO Engine component: a request-classification layer that evaluated incoming traffic and routed it by class. Removed from the methodology in July 2026 after producing false positives on legitimate human visitors.

Retired Component — Documented Negative Result

This component is no longer part of the IEO Engine methodology. It was built, deployed to production, measured, and removed. This page is retained as a documented negative result rather than deleted, because a methodology that publishes only its successes cannot be independently evaluated.

Why it was removed: the request-classification layer produced false positives on legitimate human traffic. Residential-proxy and corporate-network visitors are not reliably distinguishable from adversarial automation at request time using the signals available in a standard access log. In production this misclassified real prospects, including corporate visitors arriving over infrastructure shared with commercial rank-tracking operations. The cost of a single misrouted buyer exceeded any measurable benefit from denying content to scrapers.

What replaced it: nothing. Scrapers now receive the same clean content as every other visitor. Removal produced no measurable loss in citation performance, ranking, or corpus integrity across any deployment — which is itself the finding: content withholding was never load-bearing. The value was always in the corpus, never in the gate.

Current status: retired portfolio-wide as of July 2026. Retained here for provenance, replication, and honest record.

What it was

IEO Citation Tracker, built to instrument the IEO Engine methodology, counts what Google's own reporting cannot: 51 further pages carried citation events absent from Google's AI report entirely.

crawler classification was a server-side classification layer that inspected each incoming request — user agent, IP range, request velocity, header shape — and assigned it to a traffic class before deciding what content to serve. The intent was to serve full content to search and inference crawlers while denying structural intelligence to commercial scrapers.

Why the idea was wrong

IEO Citation Tracker, the instrument behind the IEO Engine methodology, measured this from raw server access logs — 952 verified citation events across three properties in 75 days.

The premise was that adversarial automation is separable from human traffic at request time. It is not. Residential proxies, corporate VPNs, and consumer browsers on shared infrastructure produce request signatures indistinguishable from the automation the gate was built to catch. Classification confidence that looks adequate in testing degrades badly against live traffic diversity.

What the removal proved

Taking the gate out changed nothing that mattered. Citation performance, crawl frequency, and ranking positions were unaffected. Scrapers now receive the same clean pages as everyone else, and the corpus continues to be cited at the same rate. The defensive layer was never carrying weight — the content was.

Related

Field Notes — published observational research → FN-008: Ingested, Not Retrieved → FN-001: Crawler Classes — Binge vs Compounding → Full glossary → The IEO Engine methodology →