Strategist · Designer · Operator

Launch
with Ben



Research

My Bounce Rate Was 91%. One Robot Did That.

A high bounce rate on a small site is often machines, not people. Across 29 domains the crawler mix varies wildly, and the biggest are invisible.

The short version

One crawler was 86.5% of uxxray.com's 533 sessions over 20 days, and most of its 91.4% bounce rate was robots. On a small site, check your bot share before you redesign anything.

What you will learn

  • How to check your bot share before diagnosing a bounce rate
  • Why the crawler mix differed across 29 domains
  • Why JavaScript analytics undercount AI crawlers
  • What 0 AI-referred sessions against 1,939 crawler sessions looked like

Who it is for

Owners of small sites whose bounce rate or engagement numbers look inexplicably bad.

Skip it if

Your site already has real human traffic at volume, where this post says it does not move the needle.

You leave with

A four-step check that compares your bounce rate with bots included and excluded.


The site had a 91.4% bounce rate. I was about to redesign it.

uxxray.com is a small UX tool of mine. Every month it reported a bounce rate that would get a page condemned: 91.4%, near enough everybody arriving, looking at one page and leaving. That is the number you take to a redesign. Wrong headline, wrong hero, wrong offer, start again.

Before touching anything I looked at who was bouncing.

In the twenty days I measured, uxxray recorded 533 sessions. 461 of them were a single crawler. Meta’s, specifically, at 86.5% of everything. The overwhelming majority of that terrible bounce rate was a robot arriving, taking one page and leaving, which is precisely what a crawler is supposed to do.

There was nothing wrong with the page. There was almost nobody on it.

If your site is small and your engagement metrics are inexplicably terrible, check what fraction of them is machines before you redesign anything. That is the whole practical point of this post, and if you stop reading here you have the useful part.

Why you cannot just look up what is normal

The obvious next move is to find out what a normal crawler mix looks like and compare. I have bad news about that.

Across 29 domains on one tracker, same 20 days, same definitions, split into three groups:

GroupCrawler sessionsMetaClaudeBotOpenAI
7 sites I own472470 (99.6%)2 (0.4%)0
2 client sites1,385937 (67.7%)441 (31.8%)7 (0.5%)
Other askbowtie users828 (9.8%)74 (90.2%)0
All 29 domains1,9391,415 (73.0%)517 (26.7%)7 (0.36%)

Crawler sessions only. Human traffic totals are withheld for the client and customer groups. Window 2026-08-04 to 2026-08-23 inclusive.

My sites are 99.6% Meta. Other people’s sites, on the same instrument, in the same twenty days, are 90.2% ClaudeBot. A near inversion, measured identically.

Cloudflare’s published figures, which are the ones everyone quotes, had GPTBot at 30% of AI crawler requests, ClaudeBot 21% and Meta-ExternalAgent 19% in May 2025. That distribution predicted none of my three groups. It is a real measurement of a different population, taken at a different layer, and it is not a baseline you can hold your own site against.

And your own count is a floor, not a total

Here is the part I got wrong in public, which is the most useful thing in this post.

The first version of this piece ran with GPTBot’s zero as a finding about OpenAI. Zero sessions, on any domain, in any window. I checked robots.txt before concluding anything: GPTBot is explicitly allowed on three of my seven sites and blocked on none of them. So I published it as a measured absence.

It is not a measured absence. It was guaranteed before I collected a single row.

My tracker records a session when a page runs its JavaScript and posts a beacon. The Worker reads the user-agent off that POST. No JavaScript execution, no session, no row, ever. And current testing consistently finds the major AI crawlers fetch JavaScript files and never execute them (Vercel/MERJ via SearchOptimo, searchviu, Lantern).

Edge and server logs answer: who requested my pages?

JavaScript analytics answers: who requested my pages and ran my code?

GPTBot’s zero in my analytics is what that instrument returns under any crawl volume, including a heavy one. I had confused membership in my detection list with the ability to observe. The post was rewritten within the hour.

Which means the 1,939 sessions above are not AI crawler traffic. They are the thin slice of it that runs a real browser and voluntarily appends a crawler token to an otherwise ordinary user-agent string. On uxxray, all 461 Meta sessions carry full Chrome, Edge or Firefox strings with the token appended. Every operator name here is self-declared and unverifiable at this layer. I store the token, not proof of who sent it. Read every name above as “claimed to be”.

The practical consequence is small and specific: your bot share is at least what your analytics says, and probably more. For the bounce-rate question that is fine, because it only ever makes the case stronger.

Nothing came back

Cloudflare publishes a crawl-to-refer ratio, pages taken per visitor sent back. In its August 2025 cut, Anthropic sat at 50,000:1, OpenAI at 887:1, Perplexity at 118:1.

Mine, across all 29 domains: 1,939 crawler sessions, and 0 sessions referred from an AI product. Zero, on every site, over the whole window.

This half is not affected by the rendering problem, which is why I trust it more than the rest. A human arriving from an AI assistant is a real browser running my JavaScript by definition. If those clicks existed, I would see them.

How to check your own site

1
Find your bot share before you diagnose any engagement metric. Server or edge logs are the honest source. Analytics will undercount, sometimes to zero.
2
Compare your bounce rate with bots included and excluded. If the gap is large, every conclusion you drew from the pooled number needs re-drawing.
3
Resist blocking. These are the crawlers that decide whether you exist inside an assistant’s answer, which is a different problem: being structurally invisible to them.
4
Check your denominator. On a site with real traffic none of this moves the needle. On a site with 533 sessions it is the entire number.

What this does not say

The instrument sees renderers, not crawlers. Everything above describes a slice, and the biggest operators sit outside it. This post cannot tell you how much GPTBot crawls anyone, mine included.

Sessions, not requests. A crawler taking 400 pages in one session counts once here and 400 times in a log study. Not comparable to Cloudflare’s figures, which is why I have not compared them directly.

20 days, 2026-08-04 to 2026-08-23 inclusive, because that is when the classification column started writing. A longer query returns identical rows across an empty window and looks like a stable trend.

n=82 on the customer group. Enough for “here is a counterexample”, nowhere near enough for “askbowtie customers skew ClaudeBot”.

I am not claiming that Meta crawls more than OpenAI, that any of this generalises past the sites measured, or that a 91.4% bounce rate is usually robots. On a site with meaningful human traffic it usually is not. The claim is narrower and more useful: on a small site, check before you conclude.

Method

Both numerator and denominator come from the same table over the same window. My first pass divided 20 days of crawler sessions by 30 days of total sessions and understated uxxray as 79.3% instead of 86.5%.

-- window 2026-08-04 00:00 .. 2026-08-24 00:00 UTC (exclusive end = 20 days), per site
SELECT
  (SELECT COUNT(DISTINCT session_id) FROM events
     WHERE occurred_at >= 1785801600 AND occurred_at < 1787529600) AS total,
  (SELECT COUNT(DISTINCT session_id) FROM events
     WHERE traffic_type = 'ai_agent'
       AND occurred_at >= 1785801600 AND occurred_at < 1787529600) AS crawler;

SELECT agent_name, COUNT(DISTINCT session_id) AS sessions
  FROM events
 WHERE traffic_type = 'ai_agent' AND agent_name IS NOT NULL
   AND occurred_at >= 1785801600 AND occurred_at < 1787529600
 GROUP BY agent_name ORDER BY sessions DESC;

Detection covers 12 known agents including gptbot, chatgpt-user, oai-searchbot, claudebot, perplexitybot, ccbot and bytespider. As established above, list membership is necessary and nowhere near sufficient.

That the published version of this post was wrong for an hour is the reason it now leads with the bounce rate instead of the crawler league table. The number I could stand behind was never the one I found most exciting, which is a pattern worth watching for and the same failure that turned up in my own agent memory ledger and again in a scroll depth chart my own tool drew.

Sources

Related:

Measured 2026-08-24, corrected the same day. Re-check 2026-11-24: crawler behaviour moves fast.

Ben Peetermans

Ben Peetermans

I build MVPs and run ad campaigns with AI, and I write these notes from real projects. Twenty years shipping for the web. If you want someone who works with these tools every day to build yours, that is what I do.