AI crawler behavior observed on production mirror sites
Insights
28,918
Total LLM Bot Requests
12,092
Citation Events
108
Days of Data
12
Log Entries
The Crawler Logs document real AI crawler behavior on production AI sites built by ROZZ. Using CloudFront access logs and User-Agent classification, we track how GPTBot, BingBot, ClaudeBot, ChatGPT-User, PerplexityBot, and other LLM bots discover, crawl, and cite GEO-optimized content. Across 108 days of observation on rozz.genymotion.com, we recorded 28,918 total LLM bot requests and 12,092 citation events—real users receiving Genymotion content in AI conversations.
Genymotion’s AI site has 16 topic pages. In one week of logs, AI platforms asked for 61 more that no longer exist—1,001 requests to retired topic URLs. Clustering-algorithm iterations kept improving the taxonomy but broke every external cache that learned the old map.
People evaluate products through ChatGPT, Claude, and Perplexity now—and the company whose product is being discussed usually has no idea it happened. 667 chatbot conversations and 2,500+ reconstructed ChatGPT sessions show who’s actually asking, what they want, and where the conversations lead.
rozz.genymotion.com + Genymotion.com chatbot · Mar 1 – Apr 14, 2026 · 3,830 March AI site fetches, 1,323 this week
Two data sources reveal invisible AI-mediated evaluations: chatbot intent on the site + ChatGPT session reconstruction via CloudFront logs
95% of chatbot conversations are hobbyists/individual users; enterprise-buyer goals represent <5%—not the audience marketing targets
Reconstructed sessions show live pricing evaluations, cross-continent macOS checks, and recurring VirtualBox bug reports no one else sees
Three AI platforms—ChatGPT-User, Claude-User, Perplexity-User—now retrieve content from the same AI site during live user sessions. 9,250 citation requests across 90 days. Three months ago there were zero.
rozz.genymotion.com · Jan 8 – Apr 8, 2026 · 22,679 total LLM bot requests (90-day cumulative)
Perplexity-User first appearance (Apr 5)—all 3 major citation pipelines now active
376 commits across 90 days; structural fixes (sitemaps, robots.txt, topic taxonomy) drove every breakthrough
Q&A pages drive 66–75% of citations; CLI runbooks open a developer-tool sales channel
A developer asked Claude Code how much Genymotion costs. Ten seconds later, it was showing them how to set it up. Both answers came from our AI site. 14 Claude-User requests in 6 days, 12 from Claude Code.
rozz.genymotion.com · Mar 24–31, 2026 · 1,612 total LLM bot requests
Claude-User: first-ever appearance—14 requests, 12 from Claude Code terminal sessions
10 seconds from pricing Q&A to CLI runbook—evaluation and implementation in one session
Claude pipeline complete: ClaudeBot crawl → 5 days → Claude-User live retrieval
ClaudeBot made 958 requests in one week, up from 123 the week before. 503 GEO pages and 162 Q&A pages—the largest ClaudeBot crawl since December. Six hours after deploying per-topic sitemaps, it came back.
rozz.genymotion.com · Mar 17–24, 2026 · 2,446 total LLM bot requests
ClaudeBot 8x increase (123 → 958) triggered the day per-topic sitemapindex was deployed
March 20: 577 ClaudeBot requests in a single day—largest since December
Every major AI crawler has now completed a deep indexing event on the AI site
PerplexityBot made 511 requests in one week, up from 42 the week before. It crawled 172 Q&A pages and 256 GEO pages—more content in 7 days than in its entire prior history on the site combined.
rozz.genymotion.com · Mar 10–17, 2026 · 2,532 total LLM bot requests
PerplexityBot 12x increase (42 → 511) triggered the day after index page redesign
84% of PerplexityBot requests hit content pages—Q&A and GEO pages, no homepage
ClaudeBot: 0 Q&A pages, 0 Claude-SearchBot traffic—still in discovery mode after 3 weeks
ChatGPT-User made 681 visits in one week. By grouping visits into sessions using IP hashes and timing, we reconstructed 168 sessions showing how users navigate AI-mediated discovery.
rozz.genymotion.com · Mar 3–10, 2026 · 681 ChatGPT-User visits
4.6 pages fetched per turn—ChatGPT-User verifies across multiple sources, not just one
28% of sessions hit only the index page and stopped—led to index redesign
30% of sessions are multi-turn: we can reconstruct actual ChatGPT conversations
We tested 24 queries across four AI platforms. ChatGPT cites Genymotion 83% of the time. Claude, 21%. Perplexity, 17%. Gemini, 4%. The first three track with crawl volume. Gemini doesn’t crawl the AI site at all.
rozz.genymotion.com · Feb 24 – Mar 3, 2026 · 24 queries × 4 platforms
ChatGPT: 83% citation rate, up from 14% before the AI site launched
ClaudeBot jumped 24x (21 to 505 requests)—reads topic taxonomy, not individual pages
Gemini recommends Genymotion in most queries but doesn’t link—different pipeline entirely
BingBot made 1,556 requests this week—more than any other bot, including ChatGPT-User. What started as a ChatGPT story is now happening across six platforms.
rozz.genymotion.com · Feb 17–24, 2026 · 3,188 total requests
BingBot: 1,556 requests—largest single bot category, surpassing ChatGPT-User
ChatGPT-User citations: 1,329, up from 1,077 the prior week (+23%)
Six platforms now crawling: OpenAI, Microsoft, Anthropic, Meta, ByteDance, Perplexity
ChatGPT citations grew from 7 to 116 in one week. 345 citation events, 161 unique sessions, and 75% of requests hitting Q&A pages—not traditional content.
rozz.genymotion.com · Feb 2–9, 2026 · 2,195 total requests
16x daily citation growth: from 7 on Feb 2 to 116 on Feb 9
75% of ChatGPT requests landed on Q&A pages, not traditional content
Q&A pages cited 10x more than traditional GEO content pages
GPTBot made 547 requests in a single day—47% of all training bot activity in 30 days. Three weeks later, ChatGPT users were receiving content in their conversations.
rozz.genymotion.com · Jan 3–Feb 2, 2026 · 1,280 total requests
GPTBot made 547 requests on January 7—47% of 30-day training activity in one day
42 citation events recorded; concentrated on high-intent pages (requirements, compatibility)
~3 weeks from major crawl to first ChatGPT citations
Methodology
Data source: CloudFront access logs for rozz.genymotion.com. Bot classification based on User-Agent strings. Training bots (GPTBot, ClaudeBot) are distinguished from search index bots (OAI-SearchBot) and citation events (ChatGPT-User). Citation events represent real user conversations where ChatGPT retrieved and cited mirror site content. All data is from a single production mirror site; results may vary by domain, content volume, and vertical.
Get Your Own Crawler Logs
ROZZ builds mirror sites with full CloudFront logging. See exactly which AI bots crawl your content, when they crawl it, and when citations begin.