72% of the independent web has no AI policy at all.
We fetched the real robots.txt of 110 well-known independent sites.
Most have never made a single decision about AI access. Is your site one of them?
Scan it live →
🟠 Open — no AI rules at all🔵 Managing — some AI crawlers restricted⚫ Blocking — most AI crawlers blocked
The sites
Who's exposed, who's in control.
Site
Category
Posture
AI crawlers blocked
Method: live robots.txt fetch, parsed for 16 known AI crawler user-agents.
Posture reflects declared policy only — not actual AI traffic. Site owners can
claim and verify their listing.