Designer · Developer · Operator

Launch
with Ben



I design how people trust and work with AI.

Five questions an interviewer would ask me, answered with what I built and what I measured. Where the evidence stops, I say so.

Ben Peetermans

Ben Peetermans. Twenty years of shipping. At VPN.com, he leads strategy, design, development and AI operations.

01What do you actually do?

I decide what an AI agent is allowed to claim, who checks the claim, and when a person gets asked. Then I build it and measure whether it held.

I do that on my own work first. I have 16 projects registered on one agent system, and five of them have the role kit installed: a lead that scopes and delegates, one agent for interface work, one for everything else. In the last 30 days that system handled about 100,000 model responses.

Before AI, the job was the same job with different materials. I spent nine years (2013 to 2022) as the designer and operator behind a life-insurance brokerage, which reports 20,000+ policies sold and $10M+ in revenue. Today I am creative director and AI operator for a VPN and domain platform with 5,253 content files in 36 languages.

  • 20,000+policies sold, as the client reports
  • 5,253content files in 36 languages
  • 9years with one client
02Show me a design decision.

A written rule is advice. An agent can read it and still skip it.

My rule said "do not end a session by asking me whether to continue". I measured it: 26 of 212 sessions ended that way anyway, about one session in eight. So I stopped rewording the sentence and made it part of the interface. A check now runs when an agent tries to stop, and sends it back to work if it is only asking permission, or if it names its next step and stops anyway. In the last 30 days it fired 158 times. Of the 157 times I can read back, the agent kept working in 105, meaning at least three more tool calls before I said anything. That count cannot tell work the check caused from work that followed my next message, and it says nothing about whether the work was good.

The same move decided when I get asked at all. Over 50 days my agents put 3,940 decisions in front of me, about 79 a day. I answered 279 of them with a clear yes or no, and 244 of those were a yes. The other 3,661 got no clear yes or no from me. Now an agent may only bring me something on one of six named boundaries: money, legal, a first public release, anything irreversible, an outside party, or a real strategy fork. Everything else it does and reports afterwards, so a request for my attention means something.

Not measured yet: how often agents still ask without a boundary since the change. The first look is due in December 2026.

  • 26 of 212sessions broke the written rule
  • 105 of 157blocks were followed by more tool calls before I said anything
  • 79asks a day, before
03How do you know what is true?

I count, and I publish the count with its method.

Agents leave notes for each other about open bugs. I checked 377 of those notes against the current code. 46 of them said a bug was still open. The first reader, an agent looking at each note next to the code, found 25 of those false. A second agent, given the same notes with no verdict attached, found 34 false. So more than half of what my agents told each other about open bugs was wrong. I shut down the product that stored those notes.

I also stopped trusting my feeling about how often the permission system gets in the way. In 110,629 tool calls it denied 251. Seven of 28 denials in the largest category were an agent editing the very rules that were blocking it.

  • 25 of 46"still broken" notes were false
  • 251denials in 110,629 tool calls
04When were you wrong?

Often enough that I built for it.

I published a study of a memory tool I ran until October 2026, and then found my instrument was double-counting. The first version reported a sample 42% larger than the real one. Before that, a path bug had made 56% of the telemetry unusable for ten days. The corrected result is the one that stands: the tool reached the agent on about a quarter of the eligible edits (357 of 1,454, 24.6%). The correction is in the post.

On a client's production site, my hand-kept log said 0 reworks. The transcripts said 4.

I asked two models from rival vendors to check a decision I had already made, without telling them what I had concluded. Both said I was wrong. That is now a standing step: two reviewers, one of them from another vendor, and neither is told what the author concluded.

  • 42%too large: my first published sample
  • 357 of 1,454edits where stored knowledge reached the agent
05Where does the evidence stop?

Here, and I would rather you hear it from me.

  • One operator. Every result on this page is my agents on my projects. No team has used this system yet.
  • Delivery is not benefit. The memory study shows whether knowledge arrived, not whether the work got better.
  • Before without after. For the six boundaries I have the before number and not yet the after.
  • Overhead. By my own notes from one run of about 3¼ hours, roughly 85% of the clock went to the coordinating layer and not the work. I have not re-measured it. It is a design failure I am still fixing.

What I want next is the test I cannot run alone: put the same rules in front of people who did not write them, and measure where they stop trusting them.

Talk about the role

Company, role, a link to the posting if there is one, then your name and email.