The Erosion of Expertise: Why AI’s Authority Bias is Making Us Lazy

(Part 1 of the series “The Human Downgrade”)

A few months ago, during a live product demo, a client started asking questions to better understand our ERP solution. After I gave my answer, the client challenged it, claiming they read that a specific feature was available and even cited the exact session code.

Surprised but curious, I asked for the session code so we could check it together live. The moment I entered the code, the ERP returned a Session not found error.

I even checked the official online help for the software, but the session did not exist. When I asked the client where they found that information, they admitted they had queried ChatGPT, which had provided the solution, and they assumed it was correct.

Without consulting any official manual, ChatGPT had probably mixed up references from multiple different ERP systems, hallucinated a session, and generated a response presenting a non-existent feature as a fact.

This is a trend I’m observing more and more among project teams, system integrators, and consultants. When faced with a complex technical problem, an undocumented error code, or a gap in domain knowledge, the reflexive reaction has become to ask a Large Language Model, take the generated answer at face value, and move on to the next task.

We’re stopping our research. We’re outsourcing our knowledge acquisition to algorithms. And the data tells us, in no uncertain terms, that we’re doing so at our own peril.

A recent study by Oumi, systematically analyzed the reliability of AI-generated results in modern search engines. They examined thousands of queries to understand how often these quick summaries actually provide factual information.

The result: 50% of AI Overviews contain claims completely unsupported by the cited sources. Only 39% can be considered fully trustworthy.

Think about what this means. Half the time, the assumption you’re basing your next critical business decision on, or the parameter you’re adjusting in a live system configuration, might be a complete hallucination disguised as absolute truth.

The Consumerization of IT Consulting

To understand the severity of this, we need to look at how the daily habits of IT professionals have shifted. Today, highly paid consultants and enterprise architects are searching for complex system information exactly like an average internet user searches for a quick pancake recipe.

In the past, consulting required a deep, methodical approach to information gathering. If you needed to understand how a specific routing rule worked in an ERP system like Infor LN or SAP, you’d open the official documentation portal, read the release notes, consult whitepapers, or search through vetted community forums. You were forced to understand the context.

Now, under the immense pressure of tight project deadlines, consultants are adopting consumer behaviors in a professional setting. They type a complex enterprise query into a search engine and stop reading the moment the AI Overview box appears at the top of the screen.

They treat the LLM not as a tool, but as an infallible senior colleague who has supposedly read all the manuals for them. They rely entirely on an AI-generated summary without ever clicking a link, reading a primary source, or verifying if the information applies to their specific software version.

This consumerization of professional research is actively degrading the quality of consulting output.

The Silent Damage of Authority Bias

Why do we fall for it so easily? The machine learning community calls it Authority Bias, and AI has amplified it beyond anything we’ve seen before.

Authority Bias is our innate human tendency to attribute greater accuracy to the opinion of an authority figure, regardless of the actual factual content. In today’s digitized world, the “machine” has become the ultimate supreme authority.

When an LLM returns a perfectly formatted text, written in an assertive tone, grammatically flawless, and elegantly laid out with convincing bullet points and bold text, our brain lowers its critical defenses. Subconsciously, we think: “If the system states it so confidently and presents it so beautifully, it must be true.”

This bias creates devastating long-term damage for professionals in consulting and IT. True expertise isn’t built by reading a three-line AI-generated summary. Expertise is forged through the mental friction of studying. It’s built through the painful process of comparing different, sometimes conflicting sources, and through deeply understanding the “why” and the “how” behind technical data.

When we completely delegate our information retrieval to AI without ever challenging it, we stop exercising our analytical muscles. We transform from thinking, problem-solving expert consultants into paper pushers of outputs generated by probabilistic algorithms.

In the long run, this cognitive flattening will render our skills superficial, obsolete, and easily replaceable by the very algorithms we’re blindly trusting.

Always Demand and Verify Sources

Our entire approach to LLMs must fundamentally change. These models aren’t infallible oracles possessing true comprehension. They’re mathematical tools designed to predict the next logical word in a sequence. They’re formidable accelerators for drafting and structuring data, but they can’t replace our critical human judgment.

To combat Authority Bias and protect your expertise, you must adopt a non-negotiable rule: always demand sources, and verify the quality of those sources.

The way we implement this rule depends on the tool we’re using. We must treat search engine summaries and conversational LLMs differently, applying specific safeguards for each.

Navigating AI Overviews. When using search engines that provide AI Overviews, you must fight the urge to read the summary. Actually, you can totally avoid this risk by typing “-ai” before you hit the Enter Button on your keyboard. Google will show you the results without any AI box in between.

In case you don’t use this small trick, remember that overview provides tiny citation links embedded in the text. Click them. If the AI summary doesn’t provide a clear source for a specific claim, discard that claim entirely. Your professional reputation is worth far more than the three minutes you save by not clicking a link.

Setting Rules for Conversational LLMs. For conversational tools like ChatGPT, Claude, or corporate AI assistants, you have much more control. Don’t just passively hope the AI provides a source; engineer it to do so. I strongly suggest implementing a future-proof rule in your daily workflows by using “Custom Instructions” or system prompts.

Configure your LLM with a strict directive: “For every technical, factual, or business-related question I ask, you must always append a dedicated ‘Sources Used’ section at the end of your response. Provide the specific URLs, document names, or data points you referenced to generate your answer.” You can place this prompt directly inside the “Instructions” layer if you use the webapp, or in specific Markup files (CLAUDE.md, AGENTS.md …) based on the software you are using (Claude Code, Codex, Opencode, Antigravity, …).

This practice is essential not just to prove that a source exists, but to allow you to evaluate the quality of the source itself. Not all information is created equal. An LLM might give you a perfectly confident answer, but if you look at the sources and see it pulled the information from an outdated 2014 Reddit thread or an unverified personal blog, you immediately know the data is suspect. Conversely, if the sources point to official vendor documentation or updated academic papers, you can proceed with much higher confidence.

Protecting Your Professional Value

Technology should help us find information faster, but the hard work of understanding, validating, and internalizing that information must remain ours. The moment we stop questioning the source of our knowledge is the moment we stop being experts.

If we stop doubting and verifying the quality of the data we consume, we lose exactly what our clients and employers pay us for. They pay us for our capacity for nuanced judgment, our ability to navigate ambiguity, and our assurance that the solutions we implement are built on a foundation of verified truth, not probabilistic hallucinations

Written by Andrea Guaccio 

August 13, 2026