When a Good AI Answer Leads to a Bad Decision

# When a Good AI Answer Leads to a Bad Decision

These days, in many meetings where people talk about Artificial Intelligence, I often hear one sentence:

“We asked AI, and this was its recommendation.”

There is nothing wrong with this sentence by itself. I also use AI almost every day for analysis, comparison, scenario review, and even to challenge some of my own ideas.

The problem starts when the distance between “we asked AI” and “so we should do this” disappears.

This is more serious when the person who receives the answer does not have enough knowledge or experience in that area.

Today, AI models can give answers about almost any subject. These answers may look professional, well structured, and convincing. They can explain advantages and disadvantages and even give a clear recommendation.

But a well-written answer is not always a correct answer.

And even a correct answer does not always lead to the right decision.

---

A few years ago, if a manager wanted to make a decision about network architecture, cybersecurity, ERP, or Cloud, they usually needed to speak with someone who had real experience in that field.

Today, they can simply enter the question into ChatGPT, Copilot, Gemini, or another AI tool.

A few seconds later, they may receive a complete analysis.

This is very useful because access to specialized knowledge has become much easier.

But access to knowledge is not the same as having expert judgment.

Imagine that someone asks AI about designing a Disaster Recovery solution.

The AI may explain RTO, RPO, Replication, Backup, and Failover very well.

For a non-expert, this answer may look complete.

But an experienced infrastructure professional will probably ask more questions:

Were the dependencies between services considered?

What type of failure is the Replication protecting us from?

What happens if the data is corrupted or encrypted?

When was the last real Restore test?

How long will Failover really take?

And more importantly, is Disaster Recovery really the main problem of the organization?

A non-expert often focuses on what is written in the answer.

An expert also notices what is missing.

This difference is very important.

---

We often talk about Hallucination, when AI produces incorrect information.

This is an important issue.

But the risk is not only that AI may give a wrong answer.

Sometimes the bigger risk is that the user cannot recognize that the answer is wrong.

AI can present incorrect information with the same confidence and professional tone as correct information.

In this situation, the quality of writing can easily be confused with the quality of analysis.

This is where non-expert users can become more vulnerable.

---

AI has also made it possible for many people to speak about specialized subjects even when they are not experts in those areas.

From one side, this is very positive.

A CEO can learn about Zero Trust before meeting the CIO.

A CFO can ask questions about FinOps.

An HR manager can learn about People Analytics.

This can improve the quality of management discussions.

But there is also a risk.

Knowing the language of a subject is not the same as having real expertise in that subject.

Someone may work with AI for a few hours and then speak confidently about Kubernetes, API Gateway, Data Lakehouse, or Zero Trust.

But speaking well about a topic is not the same as being able to make the right decision about it.

Expertise is not only a collection of information.

A large part of expertise comes from seeing failures, exceptions, limitations, and real-world consequences.

Someone who has experienced several ERP migrations usually asks questions that may never come to the mind of someone who has only read about ERP.

---

For example, imagine that a company manager asks AI:

“Should we replace our current ERP system?”

AI can explain the advantages and disadvantages, migration risks, a possible Roadmap, and even criteria for selecting a new product.

All of this can be useful.

But does the AI know how many Customizations have been added to the current ERP during the last ten years?

Does it know the quality of the Master Data?

Has it reviewed the current vendor contract?

Does it know how much change the internal team can handle?

Does it know what other transformation projects are happening at the same time?

Does it know if business managers are really ready to change their processes?

Probably not.

So the AI analysis may be logically correct, but the final decision may still be wrong for that specific organization.

Strategic decisions are not based only on information.

Context, experience, limitations, timing, organizational politics, risk tolerance, and many other factors are also important.

A large part of this information may never be included in the Prompt.

---

Some people believe that Prompt Engineering can solve this problem.

A better Prompt can certainly create a better answer.

But a good Prompt cannot replace expertise.

If you do not know what is important, how can you make sure that AI reviews all the important areas?

If you do not understand the Failure Modes of an architecture, how can you ask about them?

If you do not know the risks inside Cloud contracts, how can you properly evaluate Vendor Lock-in?

In many decisions, the problem is not the lack of answers.

The problem is that the real problem was not defined correctly from the beginning.

Problem definition is still one of the most valuable human skills.

---

Another important topic is Human in the Loop.

Many AI Governance frameworks say that a human should remain involved in important decisions.

This is reasonable.

But there is another important question:

Which human?

If an AI output about cybersecurity is reviewed by someone who does not have enough cybersecurity knowledge, this human review may not create real control.

Human in the Loop is useful when the person can challenge the AI output.

For example, they should be able to say:

“This assumption is not correct.”

“This data is incomplete.”

“This option cannot work in our environment.”

Or even:

“We defined the problem incorrectly from the beginning.”

Otherwise, the process may look human-controlled, but in reality the decision may still be shaped mostly by the machine.

---

Will this problem disappear when AI models become better?

Part of the problem may become smaller.

AI models are becoming more accurate, their reasoning is improving, and they can now work with documents and real data.

But organizations should not design their Governance based on the idea that the next AI model will no longer make mistakes.

A better model can reduce risk.

It cannot remove risk completely.

Also, when AI answers become more professional and more convincing, it may become even harder for non-experts to recognize mistakes.

So the main question should not only be:

“How much can we trust AI?”

A more important question is:

**“If AI is wrong, can our process detect the error before it becomes a real decision?”**

From a management point of view, this question is much more important.

---

For many professional activities, I see AI more as a Copilot than an Autopilot.

We can use AI to generate ideas.

We can compare different options.

We can review different scenarios.

We can challenge an architecture.

We can even ask AI to argue against our own opinion to find weaknesses in our thinking.

These are very valuable uses.

But when the impact of a decision becomes higher, expert review should also become stronger.

Writing an email is not the same as designing the security architecture of an organization.

Summarizing a report is not the same as migrating the Core System of a bank or a large company.

The level of Governance should match the level of Impact.

---

For this reason, organizations need more than AI Access.

They also need AI Literacy.

Giving ChatGPT or Copilot access to one thousand employees does not mean the organization is ready to use AI correctly.

Employees should know when an AI answer can be used and when they should stop and ask for expert review.

They should learn to ask for sources.

They should check assumptions.

They should not enter sensitive information into AI tools without proper control.

They should compare AI output with real organizational data.

And most importantly, they should know which decisions still need an expert.

Maybe one of the most important skills in the future will not be who can communicate with AI better.

It may be who knows **when not to accept the AI answer.**

---

I use a simple model when working with AI:

**AI → Analyse → Verify → Decide**

First, use AI to make the analysis faster.

Then review the answer in the real Context of the organization.

Verify the assumptions, data, and critical points.

And finally, the person who is responsible for the decision should make the final decision.

This process may sound simple.

But when we see how many decisions today start with the sentence “we asked ChatGPT,” we understand that it is not always followed in practice.

---

I do not believe that AI will remove the importance of experts.

Some parts of expert work will probably become automated.

But the value of expertise may move from “knowing the answer” to “being able to judge the answer.”

The expert of the future may spend less time collecting information and more time defining the problem, checking assumptions, understanding Trade-offs, evaluating risk, and reviewing AI output.

AI has reduced the cost of producing answers.

But the cost of a wrong decision is still real.

And maybe when producing answers becomes almost free, good judgment will become even more valuable.

In the end, the main question for managers is no longer whether they should use AI or not.

AI is already part of our work, and its role will continue to grow.

The more important questions are:

Who is using it?

For what type of decision?

With what level of knowledge?

Based on what data?

And who is reviewing the final output?

One of the biggest risks of the AI era may not be that AI models make mistakes.

They will make mistakes.

The bigger risk is when we do not recognize those mistakes.

In this situation, the important skill is not only to ask AI a good question.

**We also need to learn how to question AI’s answers.**