I Logged Every AI Crawler for 34 Days. ChatGPT Outreads Googlebot
In mid-July, my Google clicks in my home market (Israel) dropped by almost half. Buyer-intent queries that used to bring steady leads just evaporated from Search Console. While I was staring at GSC dashboards trying to figure out what broke, I finally did the thing I should have done months earlier: I stopped looking at dashboards and started reading raw server logs. What I found there was a parallel universe. Google Search was sending me less than ever — but AI systems were reading my site constantly . Not "someday this will matter" constantly. Right-now constantly: an AI assistant was fetching a page of mine roughly every 26 minutes, around the clock, because a real human had just asked it a question. So I built a small log analyzer and let it run. Here's what 34 days of complete Caddy logs from a small business site (about 70 real human visitors a day) actually look like. The numbers All counts are HTTP 200 responses only (more on why below), over 34 days: Bot Requests Per day What it is bingbot 5,444 158.2 Bing's index — which feeds ChatGPT ChatGPT-User 1,388 40.3 Live fetch while a human asks ChatGPT Googlebot 1,233 35.8 Classic Google crawl GPTBot 547 15.9 OpenAI training crawler Claude-User 519 15.1 Live fetch while a human asks Claude OAI-SearchBot 281 8.2 ChatGPT search indexing Applebot 268 7.8 Apple (Siri / Apple Intelligence) ClaudeBot 214 6.2 Anthropic training crawler Amazonbot 136 4.0 Amazon (Alexa & co.) PerplexityBot 103 3.0 Perplexity indexing Three things in that table genuinely surprised me. ChatGPT-User outreads Googlebot. 40.3 fetches a day versus 35.8. This isn't a crawler building an index for later — ChatGPT-User is the user-agent OpenAI sends when a human is mid-conversation and ChatGPT decides to pull a live page to answer them. On my site, that now happens more often than Googlebot visits. For a tiny business site in a niche market, I did not expect that. Bing crawls 4.4x harder than Google. 158 requests a day versus 36. Nobody optimizes