Strategist · Designer · Operator

Launch
with Ben

Research

Measured findings from running AI coding agents in production. Every post here has a denominator, a published dataset, and its own errata.

Measurements, not claims.

Most writing about AI agents describes what was built. Almost none of it reports what happened next, because reporting that means publishing a number that might be unflattering, and the number usually is.

Three rules decide whether something belongs here.

It has a denominator. A count of successes is not a measurement. “The memory fired 357 times” tells you nothing without the number of times it could have. Anything published here logs the attempts, not just the wins, which is what makes a rate possible at all.

It ships its data. Every post links the aggregates and the script that produced them, so the figures can be recomputed rather than believed. Where raw records can’t be published — usually because they carry private paths or client detail — the post says so and explains what was withheld.

It corrects itself in public. Instruments break quietly. When a number here turns out to be wrong, the correction goes into the piece it belongs to, with the evidence that exposed it, rather than into a quiet edit. Your Coding Agent Has Memory. Does It Actually Arrive? does that twice, and carries the erratum that convicted its own instrument.

That is a deliberately high bar and it means this section grows slowly. A short shelf of numbers that survived scrutiny is worth more than a long one that never faced any.

The shelf