ResearchAI search
How the FTSE 100 handles AI crawlers
We read the robots.txt file on every FTSE 100 company's corporate website. Almost none have an AI crawler policy, and not one blocks the bots behind ChatGPT, Claude or Perplexity search.
Insights
AI search and marketing insights.
What we're learning about how people and AI systems find, understand and choose businesses. Practical guides, our own research and notes from client work, written by the people doing the work.
Featured
All insights.
GuideMeasurement
What is the best AI visibility tool? 10 trackers compared
Ten AI search visibility tools scored on the same eight criteria, with prices and G2 ratings checked on 2 November 2026. Profound comes out on top, and Ahrefs is the sensible first step for freelancers and small agencies.