FN-009 SURVIVAL DATASET
=======================
The derived observation set behind https://ieoengine.com/research/fn-009-no-median-time-to-citation.html
Published so the Kaplan-Meier result can be recomputed independently.

WHAT EACH ROW IS
  One distinct .html path that at least one self-identifying AI agent fetched,
  on one of three production properties, 31 May - 16 August 2026.

COLUMNS
  property      A, B, C  -- three unrelated verticals, deliberately unlabelled
  page_id       stable hash of property+path; the path itself is not published
  section       first path segment (articles, guides, glossary, research, root...)
  entry_utc     first observed fetch by ANY self-identifying AI agent
  event         1 = a citation-class agent later fetched it; 0 = right-censored
  days          entry -> event, or entry -> end of observation if censored
  entry_bytes   response size of the entry fetch
  event_bytes   response size of the event fetch, blank if censored
  note          flags a response inside the host-interstitial byte band (see below)

DEFINITIONS
  Citation-class agent: fetches a page only when a live user query requires it --
  ChatGPT-User, Claude-User, Perplexity-User, Shap-User, Google-Agent,
  DuckAssistBot, YouBot, MistralAI-User -- IP-verified against published ranges.
  Ingest-class: GPTBot, ClaudeBot, CCBot, Bytespider, Amazonbot, Google-Extended.

  A fetch counts only as HTTP 200 with a non-zero body. 304 Not Modified and
  zero-byte responses are excluded: the agent received no content, so neither
  can anchor an interval. An earlier version counted 200-or-304 and is corrected.

THE INTERSTITIAL FLAG
  The shared host sometimes returns a bot-verification page as HTTP 200 with a
  cacheable body, roughly 6754-7279 bytes, measured from 292 such responses
  served on /robots.txt where the true size is 146 bytes. On CONTENT pages that
  band cannot discriminate -- many real pages are genuinely that size -- so rows
  are FLAGGED, not removed. Excluding all flagged rows is available as a
  sensitivity analysis and is not the primary specification.

TO REPRODUCE
  Treat event=0 as right-censored at `days`. Kaplan-Meier product-limit over the
  full set. The median is not reached: fewer than one in five pages is ever
  cited, so the survival function never falls to 0.50.

KNOWN LIMITS
  No burn-in filter is applied -- pages first crawled before the window opened
  record a false entry time. Three properties, one operator, one shared host,
  flat-file PHP, no CDN. Structural and taxonomy endpoints are not yet separated
  from content pages. Nothing here estimates the probability that an arbitrary
  PUBLISHED page is cited; a page never crawled never enters the risk set.