An AI read one of my commits in August and told me the redirect was correct by design. It was wrong. A scripted check caught what the read-through missed. A rendering bug had left the destination page empty of the documentation the redirect promised, and had done since the page was written. Nine of my own URLs pointed into that gap, and Google had already sent 74 impressions at an average position of 8.2 into a page that did not contain the answer.
The AI had not been careless. It had done what a competent reviewer does: read the change, understood the intent, judged it sound. The intent was sound. The fact was broken, and reading cannot tell you that.
That is the whole problem in one incident. AI can often arrive at the same answer a person would. What is interesting is that it takes a very different route to get there, and the routes fail in different places.
Humans tend to shrink the search space quickly. AI is unusually powerful at exploring the search space quickly.
Those are not the same strength. Knowing which one owns which part of a job is, as far as I can tell after a year of building this way, the actual skill.
Humans shrink the problem before they solve it
Say a site stops getting traffic. When I look at that, I am not wondering whether the font caused it. I do not consciously evaluate every possible explanation and assign each one a probability. Experience has already taught me: these things matter, those things probably don’t.
If the traffic collapsed right after a migration, the list I actually check is short and ordered.
Five things, in that order. The possible search space became very small before I opened a single tab. That is not intelligence in the impressive sense. It is compression, learned by having been wrong before.
It also has a documented shelf life. Kahneman and Klein spent years on opposite sides of the expert-intuition fight, and when they finally wrote it up together they mostly failed to disagree. The paper says so in its own subtitle: Conditions for Intuitive Expertise: A Failure to Disagree (American Psychologist, 2009). What they landed on is a test. Intuition earns trust when the environment is regular enough to contain learnable patterns, and when you have had prolonged practice with fast, unambiguous feedback. Their line worth keeping is blunter than the finding: subjective experience is not a reliable indicator of judgment accuracy. How certain you feel tells you nothing about whether you are right.
SEO after a migration passes that test for me. I have done it enough times, and the feedback is fast. Plenty of the other calls I make in a day do not pass it, and I am no better at telling the difference from the inside than anyone else.
The machine does the opposite
A language model starts somewhere else entirely. At its foundation it learns by predicting what comes next across enormous amounts of text. OpenAI describes GPT-4’s base training in exactly those terms: the base model was trained to predict the next word in a document.
That sounds absurdly simple considering what these things do. But the patterns learned through that process get extremely sophisticated. Ask about a traffic drop and the model has encountered enormous numbers of relationships involving analytics failures, indexing problems, migrations, ranking changes, server problems, robots.txt, tracking scripts and redirects. Add one sentence, “we redesigned the site three days ago”, and the weighting shifts hard toward migration errors, canonicals and redirect chains.
The AI has not remembered your redesign the way a coworker who sat through it remembers. It has recognized a relationship: redesign plus immediate traffic loss points at a particular family of problems. It is not searching a giant database for an identical problem someone solved before. It is constructing an answer from learned relationships between concepts. That is why it can handle a problem it has never seen word for word.
The human move: cut the space fast, using experience about what never matters, then go deep on what is left.
The machine move: hold the whole space open, weight it by relevance, and go wide on everything that scores.
What experience does
5 of 24 get looked at. The other 19 never enter the problem, and I could not tell you what most of them were.
What the model does
24 of 24 carry a weight. Nothing is ruled out early, which is the strength and the cost in the same move.
The same problem, two shapes. Narrowing is cheap and lossy. Weighting everything is thorough and expensive. Neither is the better one.
“It’s just predicting the next word” is a bad dismissal
This is where the popular description becomes technically true and practically useless.
A calculator ultimately manipulates electrical states. That doesn’t mean “moving electricity around” is a particularly useful explanation of what Excel does.
Next-token prediction describes how these models are trained. It does not describe everything that falls out of training them at scale. In 2022 a Google team showed that simply asking a model to write out intermediate reasoning steps changed what it could solve: on a set of grade-school math problems, a 540-billion-parameter model went from 17.9% to 56.9% accuracy with eight worked examples in the prompt (Wei et al., NeurIPS 2022). Same model, same weights. The difference was being asked to show its work.
So from where I sit, running these things all day, the useful description is not “autocomplete” and not “it thinks like me”. It is closer to: identify the relevant concepts, explore possible approaches, compare them, discard the weak ones, answer.
Breadth is worth paying for
Here is what that buys, in numbers from my own projects.
A dead-code scan across a Laravel app I had been growing for months surfaced 733 candidates and narrowed to 9 safe deletes. Over 90% of the first pass was noise. I would never have opened 733 files by hand, and the 9 were real.
Before a redesign went live I swept all 47 pages rather than spot-checking the ones I had touched. One of the five defects it found: the /hire quiz was recording twelve form_submit events for a single visitor reaching the last step, because every one of the six Continue presses fired the form’s submit event. Nothing was ever posted. Analytics counted each one anyway. /hire is the only page on this site that converts, so its conversion data would have been unreadable from launch day.
Worth being precise about what is doing the work here, because it is not the thing I said it was not. The model is not recalling 733 dead files from its training data. It is driving a scan and reading the output, which is a different act from generating an answer. The breadth comes from tools and patience. The relevance ranking comes from the model.
A machine doesn’t become impatient because it has diagnosed the same problem twenty times. It can compare twenty possibilities when I only feel like checking three. It can read hundreds of files, evaluate thousands of rows, and test the same assumption repeatedly without getting bored. That is a real advantage and it is the one I underuse most.
The best move is often deleting the question
Breadth has a blind spot, and it is the expensive one.
Ask a model “we need more leads, should we redesign the site?” and it will give you a genuinely good list. Conversion rate, confusing UX, weak messaging, slow pages, weak offer, insufficient trust, wrong audience, poor traffic quality. Every one of those is a legitimate thing to investigate. It will investigate them well.
What it is much less likely to do, unprompted, is tell you the site is fine.
I know that one from the inside. I shipped four SaaS products and made $279 across all of them, and $250 of that was a trading bot getting lucky on a hallucinated signal. The products worked. The code was clean, the UIs were good, the few people who used them asked when they could keep using them. I spent a long time assuming I had built the wrong things.
The apparent task was: build a better product. The actual task was: get anyone to see it. I did not solve the build problem better. The build problem shouldn’t have existed.
That reframe is available to a model, and it becomes dramatically more likely to reach it once it has the objective, the measurements and the constraints. Without them it will faithfully answer the question you asked, which is not always the question you needed answered.
Both sides fail, and they fail differently
The counterpoint matters, because human intuition is not magic. Experience creates shortcuts. Sometimes those shortcuts are excellent. Sometimes they become assumptions, habits and blind spots.
Two of mine, both from this year, both caught only by going wider.
I researched a post by searching for the phrase I had already decided was the topic. github project management came back at 480 searches a month, in a cluster full of people trying to connect GitHub to Jira and Trello, which is a different question entirely. I wrote it off as a weak topic. That was wrong. I had seeded the research with my own answer, so I measured only the people who had already reached it. Re-seeding on what the reader actually wants surfaced the real conversation at 6,600 a month, 14 times the volume, under a phrase I had not considered.
The other one cost more. I titled a post with a coined term, on the theory that a distinctive phrase is an ownable one. Nobody types it. The search results for it are FDA guidance and enterprise change-management software. That post earned zero impressions while three siblings published the same day earned 300, 47 and 21.
The machine’s failures look different. It rarely runs out of patience. It runs out of grounding.
The redirect it called correct by design is one shape of that. Here is the other, and it is worse because it looks like success: a gate I wrote to catch invented numbers printed a green all 0 dollar figure(s) traceable to source while checking nothing at all. It matched only dollar signs, and the post it was guarding made its case in counts and percentages. Once fixed, the same check compared 37 figures on the same post.
The first of those two failures, the confident wrong verdict, has a name in the research now. OpenAI published Why Language Models Hallucinate in September 2025, arguing that training and evaluation reward guessing over acknowledging uncertainty. The mechanism is mundane. Most benchmarks grade an answer right or wrong and give zero credit for abstaining, so a confident guess scores better than “I don’t know”, exactly as it would on a multiple-choice exam with no penalty. The willingness to produce an answer is useful. It is also something you have to control, because without the right context, measurements and objective, the model may faithfully solve the wrong problem.
Bolting them together is not the answer
This is the part I got wrong for most of a year, and the research is unkind about it.
A 2024 meta-analysis in Nature Human Behaviour pooled 106 studies and 370 effect sizes on human-AI teams (Vaccaro, Almaatouq & Malone). On average the combinations performed worse than the better of human alone or AI alone, at a Hedges’ g of -0.23. Split by task type, decision-making got worse (-0.27) and open-ended content creation got better (+0.19).
Hedges' g for human-AI teams against the better of human alone or AI alone, across 106 studies. Left of the line means the team lost. Only open-ended creation sits on the right.
Read that carefully, because the obvious reading is “human-AI collaboration doesn’t work” and that is not what it says. It says the naive version does not work. Putting a person and a model on the same decision, both weighing in, tends to produce something worse than whichever one was better on its own.
Which is precisely why allocation is the skill. If putting a person and a model on the same problem were automatically additive, none of this would need thinking about. It is not additive, so it needs thinking about.
What each side actually owns
The split I now work to, and the reason I frame this as a question of ownership rather than a claim about capability:
The human owns: what we are actually trying to accomplish, what matters, what can safely be ignored, what “done” looks like, and whether the answer makes sense in the real world.
The machine owns: breadth, recall, speed, iteration and consistency. In practice that means the scans, the sweeps, the enumerations, the cross-checks and the first drafts.
Neither owns alone: deciding which of the two just ran, and whether to believe it.
The gap between those two columns is where the operator lives. Take a genuinely simple instruction, “improve this landing page”. Someone casually using AI sends exactly that. What I send is the objective, the traffic source, the device split, the current conversion rate, the constraint that aesthetics don’t count unless they plausibly move the number, and an instruction to identify the three most likely bottlenecks and say what evidence would validate each before recommending anything.
That is not a better prompt. It is a better model of the work. An ambiguous human objective converted into something a machine can actually investigate.
And once it is running, the moving-between-modes never stops.
- Me
- The model
- I set the objective
- it expands the possibilities
- I supply the context
- it finds the relationships
- I decide what needs evidence
- it goes and gets it
- I name the assumption
- it tests the alternatives
- I judge whether it makes sense
- I decide it is finished
Ten passes, and the last two do not cross. Judging the result and calling it finished stay on the left, which is the only part of this loop I have never successfully handed over.
Notice what is missing from that loop: nowhere does the model rule on whether its own output is correct. That job stayed with me, and every incident in this post is a case of it being handed over by accident. It is the same discipline as not trusting an AI fix until you have run the actual code path, and the same reason I stopped letting one model instruct another without a gate underneath.
The real skill isn’t prompting
Not prompting. Not memorizing which model scores highest on a benchmark. Not finding clever ways to remove humans from every workflow.
Understanding where each form of intelligence belongs.
Humans contribute context, objectives, judgment, taste, responsibility and a sense of what actually matters. Machines contribute breadth, recall, speed, iteration and the ability to work across quantities of information people simply cannot. Put those together deliberately and you don’t just do the same work faster. More possibilities get explored, more assumptions get challenged, more evidence gets gathered before anyone commits.
Put them together carelessly and you get -0.23.
That is the job. Not someone who asks AI to think like a human. Someone who can answer three questions:
What should the human own?
What should the machine own?
And how should the work move between them?
The quality of that orchestration increasingly determines the quality of the result.
Related:
- AI Can Create Convincing Justification for Expensive Bad Decisions: why the premise and company context must be settled before broad investigation begins
- Don’t Assume AI Fixes Things Properly: the verification step that catches “looks right but isn’t”
- From 733 False Positives to 9 Safe Deletes: what machine breadth plus human judgment produces in practice
- Fable 5 as the Driver: when to let one AI instruct another, and where that breaks
Sources
- OpenAI, GPT-4 research: base-model training described as next-word prediction
- Wei et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, NeurIPS 2022
- Kalai, Nachum, Vempala & Zhang, Why Language Models Hallucinate, 2025
- Kahneman & Klein, Conditions for Intuitive Expertise: A Failure to Disagree, American Psychologist 64(6), 2009
- Vaccaro, Almaatouq & Malone, When combinations of humans and AI are useful, Nature Human Behaviour 8, 2024