AI Can Create Convincing Justification for Expensive Bad Decisions

AI can create convincing justification for expensive bad decisions.

The dangerous answer is not always fabricated. Sometimes the facts are correct, every step is logical, and the presentation is excellent. The problem is that the assignment began from a premise someone inside the company could have killed in one sentence.

I find AI easier to understand when I stop treating it as magic or slop and treat it as an employee. A remarkably capable employee. A very unusual one.

Human-in-the-loop AI starts with a capable employee

Imagine hiring someone who can write, research, analyze data, build software, translate documents and learn new tools almost instantly.

There is a catch.

They do not sit in your meetings. They did not overhear the conversation where you decided to stop a campaign. They do not know that sales complained about a certain type of customer six months ago. They do not remember why a strange-looking page exists unless someone wrote it down.

They start with the assignment in front of them.

That difference sounds small. It is not.

Imagine traffic dropped 20% this week

You ask AI:

“Traffic is down 20% week over week. Find out why.”

That is a reasonable assignment. The AI confirms the decline, finds where the traffic disappeared and starts testing explanations. Organic search, paid traffic, direct, referrals, broken tracking, lost rankings, removed pages, indexing problems. Given enough access, it could investigate dozens of possibilities and do an excellent job.

Now ask Sarah. She has worked with the company for three years.

Her answer might be:

“That makes sense. We removed Post X last week. It generated a lot of traffic, but almost none of those visitors were potential customers.”

Investigation over.

1. Sarah
2. AI
3. We intentionally removed Post X last week
4. Confirm the 20% decline
5. It attracted visitors who never became customers
6. Segment the loss by channel
7. The drop is expected
8. Check tracking and attribution
9. Compare rankings and landing pages
10. Inspect indexing and technical SEO

Sarah closes the question with company history. AI opens five reasonable lines of investigation because nobody gave it that history.

The AI was not stupid. It did exactly what we asked. It simply did not know what Sarah knew.

Humans accumulate context without trying

A person who has been inside a company for years carries an enormous amount of context. Some of it is documented. Most of it is not.

Sarah remembers the campaign that looked successful until sales explained why the leads were terrible. She knows which metric the CEO still cares about and which one the company stopped using. She remembers that engineering already tried the obvious fix. She knows when an unusual result is expected.

AI can retrieve context. It can search documents, analytics, meeting notes, code, tickets and databases. But first it has to know that the context exists and that it matters.

This is the hidden asymmetry in the traffic example. The employee passively collected six pieces of context. The model received none of them.

That creates a dangerous combination: AI can think very deeply inside a very narrow understanding of the problem.

This is a different question from what to delegate to AI. Delegation decides who should do the work. Context decides whether the work should exist at all.

Correct evidence can still support the wrong decision

Go back to the traffic investigation.

Suppose the AI discovers traffic did not fall 20%. It fell 10%. Good catch.

It notices that many pages have mediocre titles and descriptions. True.

It finds a page with a 100% exit rate. Also true. It recommends redesigning that page.

Now imagine the page contains a form that completes the conversion without sending the visitor anywhere else. A 100% exit rate may be perfectly normal.

Nothing here needed to be fabricated. The traffic fell. The metadata could be better. The page has a 100% exit rate. The recommendations can be logical, detailed and professionally presented.

And almost completely unnecessary.

This is the same failure behind an AI that trusts a planning document instead of tracing what the system actually does. The model is excellent at working inside the frame it receives. That does not make the frame true.

A wrong premise gets expensive quickly

Cheap execution makes this problem worse. Once the AI has built a persuasive case, the recommendation can turn into a project before anyone asks whether the original decline was intentional.

The audit creates a metadata project. The project creates design and development work. The changes require QA and new reporting. Then someone needs another analysis to explain what the changes did.

Meanwhile Sarah knew the answer on Monday morning.

The cost is not the model’s two hours. It is the work that follows a polished recommendation nobody needed.

The research found persuasive wrong answers

This is not an argument that AI makes knowledge workers worse. The useful result is more uncomfortable than that.

In a preregistered experiment with 758 Boston Consulting Group consultants, researchers gave participants realistic consulting tasks. On 18 tasks inside GPT-4’s capability frontier, people using AI completed 12.2% more tasks and worked 25.1% faster. Their work quality improved by about 32% on average.

On a complex task deliberately placed outside that frontier, AI users were 19 percentage points less likely to reach the correct answer.

Then the researchers measured how the recommendations read. AI-supported participants produced more coherent and persuasive recommendations even when their underlying answers were wrong. The published paper calls the uneven boundary the “jagged technological frontier”.

The same tool improved output and speed on fitting tasks, then reduced correctness by 19 percentage points on the task outside its frontier. The wrong recommendations still sounded more coherent.

That last finding is the one businesses should remember. A bad answer does not have to look bad. AI may improve the argument wrapped around it.

Expertise did not become less valuable

A second study followed more than 5,000 customer-support agents after an AI assistant was introduced. The 2023 NBER working paper reported a 14% productivity increase overall and a 34% improvement for novice and lower-skilled workers. The peer-reviewed version published in 2025 reports a 15% average gain across 5,172 agents, with the gains still concentrated among less-experienced and lower-skilled workers. The most experienced workers saw small gains in speed and small declines in quality.

The researchers found suggestive evidence that the system was spreading the practices of stronger workers to less-experienced ones.

At first glance, that sounds like expertise matters less. I read it differently.

AI made expert knowledge easier to reproduce. Someone still had to know what good looked like. Someone still had to create the practices worth copying. When the situation falls outside those patterns, someone still needs enough expertise to notice.

That changes the economics of expertise. The number of experts required to produce the same amount of work may fall. The value of the person deciding what work matters may rise.

One strong marketer can oversee research and production that previously required several people. One experienced developer can direct multiple AI workers. One analyst can test far more explanations than they could manually.

The expert moves away from producing every artifact and toward deciding what should happen, what looks wrong and when the machine is solving the wrong problem.

Three ways companies respond

Companies tend to reach for one of three answers.

The third option is human-in-the-loop AI in a useful sense. The human is not there to approve whatever the machine writes. The human owns the premise, supplies company context and can cancel the investigation before it turns into work.

Then AI does what it is unusually good at: pull reports, inspect hundreds of pages, compare periods, find anomalies, test theories and complete the work that used to consume Sarah’s week.

A pre-investigation brief for human-in-the-loop AI

“Give the model more context” is not a procedure. Before asking AI to investigate a business problem, fill this out. The first four fields are designed to let an experienced person delete the assignment before the model expands it.

Objective:
What decision will this investigation change?

Known changes:
What did the company intentionally launch, stop, remove or redefine?

Expected signals:
Which unusual numbers might be a normal result of those changes?

Prior decisions:
What has already been tried, rejected or accepted as a tradeoff?

Customer context:
What do sales, support or account teams know that the data cannot show?

Stop condition:
What fact would make this investigation unnecessary?

Evidence rule:
Separate verified facts from plausible explanations. Do not recommend work
until the original premise survives the checks above.

This will not give AI everything Sarah knows. It does something more realistic: it makes the missing context visible before execution gets cheap.

It also gives you a clean verification boundary. A recommendation should survive the same discipline as an AI-generated code change: run the real path before accepting that the fix works.

The advantage is not access to AI

If everyone can access the same models, where does the advantage come from?

Everyone has access to employees too. Companies still perform very differently.

The advantage was never simply having workers. It was knowing what to ask them to do, giving them the right information, building good processes and recognizing when the work was not good enough.

AI does not remove that problem. It magnifies it.

1. One expert
2. AI workers
3. Defines the decision
4. Pulls the reports
5. Supplies company context
6. Compares every segment
7. Names the assumptions
8. Tests the alternatives
9. Judges the recommendation
10. Drafts the execution plan
11. Stops unnecessary work

The economic change is not eight humans becoming one AI. It is one expert directing more execution without handing over the judgment that makes the execution useful.

Put AI in the hands of someone who deeply understands the work and you give that person enormous reach. Put the same AI in the hands of someone who does not understand the work and you give them the ability to make questionable decisions faster, at greater scale, with remarkably convincing reasoning behind them.

AI makes execution cheaper. It does not make judgment cheaper.

If anything, the cheaper execution becomes, the more expensive bad judgment becomes.

Related:

• How AI Tackles Tasks Differently Than Humans: the allocation rule for human judgment and machine breadth
• AI Trusts Your Docs. That’s the Problem.: how a false premise survives a thorough audit
• AI Second Opinion: Why Your Own Agent Can’t Give One: how shared context changes what a reviewer can see

Sources

• Dell’Acqua et al., Navigating the Jagged Technological Frontier, Organization Science, published online March 11, 2026
• Brynjolfsson, Li and Raymond, Generative AI at Work, NBER Working Paper 31161, revised November 2023
• Brynjolfsson, Li and Raymond, Generative AI at Work, The Quarterly Journal of Economics, May 2025