The site had a 91.4% bounce rate. I was about to redesign it.
uxxray.com is a small UX tool of mine. Every month it reported a bounce rate that would get a page condemned: 91.4%, near enough everybody arriving, looking at one page and leaving. That is the number you take to a redesign. Wrong headline, wrong hero, wrong offer, start again.
Before touching anything I looked at who was bouncing.
In the twenty days I measured, uxxray recorded 533 sessions. 461 of them were a single crawler. Meta’s, specifically, at 86.5% of everything. The overwhelming majority of that terrible bounce rate was a robot arriving, taking one page and leaving, which is precisely what a crawler is supposed to do.
There was nothing wrong with the page. There was almost nobody on it.
If your site is small and your engagement metrics are inexplicably terrible, check what fraction of them is machines before you redesign anything. That is the whole practical point of this post, and if you stop reading here you have the useful part.
Why you cannot just look up what is normal
The obvious next move is to find out what a normal crawler mix looks like and compare. I have bad news about that.
Across 29 domains on one tracker, same 20 days, same definitions, split into three groups:
| Group | Crawler sessions | Meta | ClaudeBot | OpenAI |
|---|---|---|---|---|
| 7 sites I own | 472 | 470 (99.6%) | 2 (0.4%) | 0 |
| 2 client sites | 1,385 | 937 (67.7%) | 441 (31.8%) | 7 (0.5%) |
| Other askbowtie users | 82 | 8 (9.8%) | 74 (90.2%) | 0 |
| All 29 domains | 1,939 | 1,415 (73.0%) | 517 (26.7%) | 7 (0.36%) |
Crawler sessions only. Human traffic totals are withheld for the client and customer groups. Window 2026-08-04 to 2026-08-23 inclusive.
My sites are 99.6% Meta. Other people’s sites, on the same instrument, in the same twenty days, are 90.2% ClaudeBot. A near inversion, measured identically.
Cloudflare’s published figures, which are the ones everyone quotes, had GPTBot at 30% of AI crawler requests, ClaudeBot 21% and Meta-ExternalAgent 19% in May 2025. That distribution predicted none of my three groups. It is a real measurement of a different population, taken at a different layer, and it is not a baseline you can hold your own site against.
And your own count is a floor, not a total
Here is the part I got wrong in public, which is the most useful thing in this post.
The first version of this piece ran with GPTBot’s zero as a finding about OpenAI. Zero sessions, on any domain, in any window. I checked robots.txt before concluding anything: GPTBot is explicitly allowed on three of my seven sites and blocked on none of them. So I published it as a measured absence.
It is not a measured absence. It was guaranteed before I collected a single row.
My tracker records a session when a page runs its JavaScript and posts a beacon. The Worker reads the user-agent off that POST. No JavaScript execution, no session, no row, ever. And current testing consistently finds the major AI crawlers fetch JavaScript files and never execute them (Vercel/MERJ via SearchOptimo, searchviu, Lantern).
Edge and server logs answer: who requested my pages?
JavaScript analytics answers: who requested my pages and ran my code?
GPTBot’s zero in my analytics is what that instrument returns under any crawl volume, including a heavy one. I had confused membership in my detection list with the ability to observe. The post was rewritten within the hour.
Which means the 1,939 sessions above are not AI crawler traffic. They are the thin slice of it that runs a real browser and voluntarily appends a crawler token to an otherwise ordinary user-agent string. On uxxray, all 461 Meta sessions carry full Chrome, Edge or Firefox strings with the token appended. Every operator name here is self-declared and unverifiable at this layer. I store the token, not proof of who sent it. Read every name above as “claimed to be”.
The practical consequence is small and specific: your bot share is at least what your analytics says, and probably more. For the bounce-rate question that is fine, because it only ever makes the case stronger.
Nothing came back
Cloudflare publishes a crawl-to-refer ratio, pages taken per visitor sent back. In its August 2025 cut, Anthropic sat at 50,000:1, OpenAI at 887:1, Perplexity at 118:1.
Mine, across all 29 domains: 1,939 crawler sessions, and 0 sessions referred from an AI product. Zero, on every site, over the whole window.
This half is not affected by the rendering problem, which is why I trust it more than the rest. A human arriving from an AI assistant is a real browser running my JavaScript by definition. If those clicks existed, I would see them.
How to check your own site
What this does not say
The instrument sees renderers, not crawlers. Everything above describes a slice, and the biggest operators sit outside it. This post cannot tell you how much GPTBot crawls anyone, mine included.
Sessions, not requests. A crawler taking 400 pages in one session counts once here and 400 times in a log study. Not comparable to Cloudflare’s figures, which is why I have not compared them directly.
20 days, 2026-08-04 to 2026-08-23 inclusive, because that is when the classification column started writing. A longer query returns identical rows across an empty window and looks like a stable trend.
n=82 on the customer group. Enough for “here is a counterexample”, nowhere near enough for “askbowtie customers skew ClaudeBot”.
I am not claiming that Meta crawls more than OpenAI, that any of this generalises past the sites measured, or that a 91.4% bounce rate is usually robots. On a site with meaningful human traffic it usually is not. The claim is narrower and more useful: on a small site, check before you conclude.
Method
Both numerator and denominator come from the same table over the same window. My first pass divided 20 days of crawler sessions by 30 days of total sessions and understated uxxray as 79.3% instead of 86.5%.
-- window 2026-08-04 00:00 .. 2026-08-24 00:00 UTC (exclusive end = 20 days), per site
SELECT
(SELECT COUNT(DISTINCT session_id) FROM events
WHERE occurred_at >= 1785801600 AND occurred_at < 1787529600) AS total,
(SELECT COUNT(DISTINCT session_id) FROM events
WHERE traffic_type = 'ai_agent'
AND occurred_at >= 1785801600 AND occurred_at < 1787529600) AS crawler;
SELECT agent_name, COUNT(DISTINCT session_id) AS sessions
FROM events
WHERE traffic_type = 'ai_agent' AND agent_name IS NOT NULL
AND occurred_at >= 1785801600 AND occurred_at < 1787529600
GROUP BY agent_name ORDER BY sessions DESC;Detection covers 12 known agents including gptbot, chatgpt-user, oai-searchbot, claudebot, perplexitybot, ccbot and bytespider. As established above, list membership is necessary and nowhere near sufficient.
That the published version of this post was wrong for an hour is the reason it now leads with the bounce rate instead of the crawler league table. The number I could stand behind was never the one I found most exciting, which is a pattern worth watching for and the same failure that turned up in my own agent memory ledger and again in a scroll depth chart my own tool drew.
Sources
- Cloudflare, From Googlebot to GPTBot: who’s crawling your site in 2025 (May 2025) and AI crawler traffic by purpose and industry (August 2025), for the published mix and the crawl-to-refer ratios.
- On AI crawlers not executing JavaScript: Vercel/MERJ via SearchOptimo, searchviu, Lantern.
Related:
- I Build Analytics Tools. Mine Drew Me a Backwards Chart.
- You Are Invisible to Machines
- Claude Code Memory: I Measured a 24.6% Delivery Rate
Measured 2026-08-24, corrected the same day. Re-check 2026-11-24: crawler behaviour moves fast.