AI Can Give You an Answer. Experience Tells You Whether to Trust It.

Our Perspective

Over the past few years, AI has become more and more popular throughout the world, industries and all scopes of social class and educational level. In organizations, we’ve seen just that. At first, the IT department was using it for their tasks and planning, then accounting started using it for analyzing and organizing, sales used it for forecasting and marketing used it for content. Customer service developed auto responses and AI agents to filter out any requests and then AI creeped into manager’s, leader’s and CEO’s computers and before we knew it, it was now being used up and down the organizational structure of companies. 

A lot of team members in an organization, including CEOs, leadership teams, sales reps, analysts, HR managers among others, are asking language models what they should do. They use it for anything from growth strategy, hiring, sales, marketing, financial decisions, M&As, operation, IT or even organizational changes. It’s a long list, and I bet we missed a couple. It usually goes like this:

• A manager, leader or CEO asks a language model something

• They ask a few follow up questions

• They receive a convincing recommendation

• They execute accordingly

The thin line runs along leaders using AI to improve their thinking, or letting it replace their thinking. This is a dangerous line to cross with great risk.

What is the Risk?

The risk is not using AI. It is surrendering your judgement while using it. There needs to be an adequate balance. It is extremely useful for comparing possibilities, organizing your ideas, creating a summary of information, and identifying questions. Explore different scenarios, draft out a plan, support during research or even challenge assumptions. Whatever you are using it for, and whatever the response, AI should be used as an input into a decision, not as the actual decision maker. Think of it as another mind sitting at the table, it can have an opinion and insight, but not the final vote. 

A collection of those ideas, factors, responses and experience must be taken into account for the final decision. Your use of judgement taking the rest into account should drive a decision in your organization.

AI is Great at Sounding Convincing

Language models always give you a convincing answer, but it is not necessarily an accurate one. They provide plausible responses, and with them, they can still invent facts, stats, sources, or unreal examples. Sometimes they will rely on outdated information, misinterpret a question, assume things and present them as facts, or miss very important context. Also, when you reframe a question, different recommendations can be produced and all this while sounding extremely confident and possibly incorrect.

Usually, the answer looks spot on. That is what it is trained to do. The most dangerous and risky AI response is not the one that sounds 100% bogus and is obviously wrong. It is the one that sounds completely reasonable that can get you into trouble. AI does not need to get everything wrong in their response. One faulty assumption or misinterpretation can be just enough to compromise a decision.

AI Depends on Your Prompt

A language model cannot see a complete organization. It sees what you share in the prompt. If you say, “Our salespeople are underperforming. Should we change our sales manager?” AI will probably recommend restructuring, looking at compensation, hiring new people or demanding more sales activity, but there are things it does not know. The sales process may be undocumented, CRM information may not be reliable, leads are not qualified correctly, managers have forgotten about coaching, there is not a clear value proposition, lack of training, changes in pricing, operations are under delivering or maybe leadership changes direction too often.

With experience, an advisor can investigate beyond the problem stated in the prompt. AI only sees the question, but experience allows you to examine the system surrounding it.

Usually, AI Reinforces What You Want to Believe

If a leader introduces a bias prompt or question, the language model will more than likely confirm it. If the prompt says: “Why should I replace my sales manager?” You will get reasons that are very general that will simply confirm replacing the sales manager, simply because it resonates with the CEO. If you follow up with other questions, you could get a better recommendation. Following up with “What evidence would suggest replacing the sales manager is a wrong decision?” could get you a totally different response, but will give you another perspective.

If you frame a question in a way that produces the preferred recommendation, then the AI validates with their response. It is essentially agreeing with the leader in a professional way with their recommendation, rather than helping them think about the factors that could lead to a decision on the matter. When you challenge the initial response or even the preferred decision, you can identify missing information and construct a stronger argument to fuel a change.

A Modern Example: Let’s Build It with AI

One common example we see is companies replacing a well known CRM or other type of software with an inhouse alternative. The reasoning sounds fantastic. “Why should we pay thousands of dollars in subscriptions for something when we can have someone  internally build it with AI?” This is sometimes a good option, especially if what you have is not exactly what you need, but a prototype should never take the place of a mature system. 

Visually, AI can easily recreate contact records, pipelines, dashboards, assign leads, automate reminders and help with follow up, but there are certain things leadership may overlook, especially when blinded by the “save a few thousand a month without a subscription” reasoning. For example, data security, permissions, audit trails, back ups, recovery from disasters, maintaining integrations, handling errors, being able to scale, having the proper documentation, compliance with third parties, having proper technical support, long term development or even the dependence on the employee who created it.

The company may stop paying that big subscription, but is then responsible for everything that comes with owning a software and that has a cost as well. AI can make something possible very quickly, but experience is still important and needed to evaluate what the company is getting into. 

The Value of Experience

With all the information and recommendations that AI has made available, judgement is becoming a scarce resource in the world. This comes with experience. It helps advisors and leaders and people as a whole identify missing information, what is real or not, if a stated problem is actually the problem, or when numbers don’t match to what people say. AI does not understand company culture, so something technically viable may fail culturally. Sometimes, a company is not ready for the proposed growth, or short term savings may cause a liability long term. Experience is what lets you recognize what is likely to happen next. 

AI bases everything on information and it recognizes patterns in said information, but experience is what allows an advisor or leader to identify what information matters, what's missing or when something simply does not add up. Experienced leaders are able to identify patterns in the real world.

Are We More Productive Now, or Are We Thinking Less?

Usually, the most valuable work is the most uncomfortable one, and AI can remove that. When you sit in uncertainty, your brain kicks in to find solutions. Investigating conflicting information is quite the task, but this can only really be done by a human. There are certain things you can only uncover when you speak to a customer or an employee. With AI, you don’t need to ask difficult questions, test assumptions, learn from previous failures, challenge leadership or even assume responsibility for a recommendation or decision. All these things are difficult and cause friction, but that is exactly where judgement is developed. The efficiency we achieve through AI becomes dangerous when the friction that forces us to think is removed. Language models are designed to eliminate repetitive work, not the intellectual responsibility of being a leader.

How Can Leaders Use AI Responsibly?

Verification is a great tool when you include it in a framework. Before taking action on a recommendation provided by AI, there are a few questions a leader should ask themselves, not AI.

What parts of the answer are actually facts and not just assumptions, estimates or general information? Where did the information come from? Check and read the sources cited by the language model. Is there something the AI did not provide like people, culture, customers, capacity, cash flow or history? Did my wording influence the response and should I reframe the question?

Another thing a leader can do is bring in someone with experience to challenge the recommendation. Someone who understands the area in question and the organization. Also, they must take into account what would happen if the recommendation is wrong. A wrong decision can have financial, legal, operational, human and even reputational consequences.

Testing the change in a controlled setting and defining certain checkpoints to evaluate before committing fully to a decision is also smart. This should help make the decision, based on the information and experience. Also, the final decision must be tied to a person who will assume responsibility whether it’s a fail or a win.

So Where Does That Leave Leadership?

AI can help analyze information faster, discover possibilities, ask better questions and even create tools that would otherwise take more time and resources. But a language model cannot read the room, visit a customer, understand the organizational history of a company or recognize what is missing or be responsible for what happens after a decision. 

Just because you have a confident answer, you cannot accept it as proof. A prototype is not a solution. A leader will be more successful with AI not by following most closely, but by being the one who knows when to question it. AI can accelerate the work, but it takes a human with experience to protect the decision.