Policy Radar

The search engines and AI companies change their rules, crawlers and guidance quietly — a doc edited here, a bot token appearing there. We watch 84 first-party sources every hour and publish the ledger below, dated and hash-verified. Nobody should have to take a vendor's word for "what changed". Not even ours.

Last sweep: 2026-10-14 19:01 UTC · Verified 77/84 · Unverified 7 (a failed fetch is never reported as "unchanged") · Raw ledger: JSON · change log

official docs

official blog

crawler registry

open spec

industry feed

Methodology, honestly

Each source is fetched hourly (browser-grade fetch, one vantage point). HTML is tag-stripped, JSON is key-sorted, whitespace is normalized into lines, and the result is SHA-256 hashed per line-set. A hash change = "changed" — we report the fact, the size delta, and since 26 Sep 2026 also the changed lines themselves, quoted from our own snapshots (a digit-stripped twin-hash marks pure timestamp churn as cosmetic). A human reads the diff before saying what it means. Registry and IP files (OpenAI's gptbot.json, Googlebot and Bing IP JSONs, robots.txt files across AI labs) are where new crawler tokens appear first; schema.org and the llms.txt spec are watched because they version; the Gmail and Yahoo sender-policy pages are watched because they govern deliverability. Industry feeds are labeled as such — they are commentary, not first-party truth. Promoted 2026-09-26: Perplexity — its pages return 403 to non-JavaScript readers (excluded 2026-08-29), but its robots.txt is fetchable, so that artifact is now watched directly. Coverage update 2026-09-28: Yandex and Brave are now watched directly — their English documentation URLs were verified fetchable today and wired in. Exclusion remains policy, not oversight: if any source is ever excluded, the reason is stated here and re-tested quarterly.

Cite this freely. Journalists and researchers: the raw ledger is public, hash prefixes let you verify any snapshot, and we share data on request — hello@synapsein.com. Corrections are logged and dated, never silently edited.