Strategist · Designer · Operator

Launch
with Ben

Build Notes

Don't Assume AI Fixes Things Properly

AI coding tools are good at text manipulation. They're bad at knowing when they're wrong. Here's the verification step most people skip.

Ben Peetermans 3 min read

The short version

Claude Code fixed one Laravel service four times, and each fix looked right and passed static analysis. It never ran the code. A 30 second tinker run found the error that four deployments missed.

What you will learn

  • Why an AI fix can pass tests and static analysis and still be broken
  • How one missing method cost me 4 deployments
  • The three-step check to run after any AI fix

Who it is for

Anyone using an AI coding tool to fix backend code, especially Laravel services, models or controllers.

Skip it if

You already run every AI change through the real code path before you trust it.

You leave with

A three-step check (tinker, log, route) to run before accepting an AI fix.

AI coding tools are good at text manipulation. They’re bad at knowing when they’re wrong.

I watched Claude Code “fix” a Laravel service four times in a row. Each fix looked correct. Each fix passed static analysis. Each fix broke something else.

The root cause was simple: the AI was calling a method that didn’t exist on the model. $domain->database() instead of DomainDatabaseService::getConnection($domain).

How did this slip through?

The AI never ran the code.

It edited files. It read documentation. It updated tests. It declared the fix complete. But it never actually executed BounceRateService::getForPages() to see if it worked.

When I finally ran it in tinker, the error was immediate: BadMethodCallException: database() doesn't exist.

30 seconds to find. 4 deployments wasted chasing symptoms.

Why this happens

AI tools treat code as text. They pattern-match against what they’ve seen. They assume methods exist because similar patterns exist elsewhere.

Static analysis didn’t catch it either. The error was already in PHPStan’s baseline, explicitly suppressed. This is the same failure mode as trusting docs that were never verified against the actual system: the tool reports green, the bug runs fine.

The verification step that should have happened:

php artisan tinker --execute="BounceRateService::getForPages(\$domain, \$cutoff);"

One command. Immediate failure. Obvious fix.

The rule

Don’t trust AI fixes until you’ve run the actual code path.

Not “the tests pass.” Not “it looks right.” Not “static analysis is clean.”

AI can write code fast. But it can’t tell you if that code actually works. That’s still your job.

The checklist

After any AI-assisted fix to a service, model, or controller:

1
Run the method in tinker: with real arguments, not mocks
2
Check laravel.log: tail -20 storage/logs/laravel.log | grep -i error
3
Hit the route: if it’s exposed via HTTP, curl it

If you skip these, you’re not using AI to code faster. You’re using AI to create bugs faster.

The takeaway

AI tools are fast. They’re also confident, and confidence without verification is how four deployments disappear chasing symptoms instead of causes. The same trap catches a reviewer: a rebuttal that reads well is not evidence, which is why a second opinion has to come from outside your own context.

Instruments have the same problem. When I measured whether my agent’s memory actually arrives, two separate bugs in the logging each produced a number that looked perfectly publishable. I published one of them before I caught it.

The fix isn’t better prompts. It’s your own verification habit, applied every time, regardless of how plausible the fix looks. Judging whether a result is correct is the one part of the job that never moved over to the machine, which is really a question about what you delegate and what you keep. Build that into your process, not your memory. That’s the difference between a skill that guides judgment and a script that enforces it.

Related:

Ben Peetermans

Ben Peetermans

I build MVPs and run ad campaigns with AI, and I write these notes from real projects. Twenty years shipping for the web. If you want someone who works with these tools every day to build yours, that is what I do.

Work with me See the work

Related reading