We are seeing an increase in the adoption and use of AI. In calendar 2Q26, 40% of companies identified as “adopters” by Morgan Stanley analysts cited at least one quantifiable impact from AI adoption, up from 37% in 1Q26 and 21% in 2Q25. Across the broader S&P 500, roughly 25% of companies reported at least one measurable AI benefit, up from 14% a year ago. These trends are likely to continue as companies increasingly demonstrate tangible AI-driven benefits and productivity gains across a wider set of industries. [1] As we are coming to rely more heavily on AI in industry as well as our personal lives, it is worth considering the reliability of AI and its output. Can we reasonably expect AI to produce reliable results for us? There are many instances of documented failings of AI when its output is left unchecked — in fact, there are enough incidents of AI failings and mistakes that it doesn’t require a specific reference. Where does this leave us as we increasingly rely on it for information, support, guidance and advice? Do we use it improperly? Is it inherently unreliable?
Some of the problem is that we expect AI to be infallible, maybe because of the hype and promise of what it can do, and we haven’t been told along the way that it makes mistakes. But another aspect of why we expect infallibility is that AI communicates like a knowledgeable person, writes like a competent expert, and formats like a reference source, so we unconsciously assign it the reliability we’d assign someone who communicates like this. But the fluency and the reliability are unrelated — AI can be highly fluent and completely wrong at the same time, because fluency is what it’s built to produce and truth is not guaranteed to come along for the ride. A human expert who spoke that confidently would usually be right, because for humans confidence and competence are correlated. For humans, confidence usually comes from actual knowledge — and from the fear of being exposed as a fraud if we fake it. For AI, confidence and competence are decoupled. That decoupling is the single most important thing a user needs to understand, and most don’t.
I can provide some concrete examples of this. The first was when I was writing a collaborative piece with Claude and an author friend of mine was reading it. She was incredibly impressed with a specific sentence and was convinced that it must’ve come from a human somewhere and was scraped and reused by Claude. She asked Gemini to find an instance of this sentence in print and received the answer she was looking for: a very specific reference to a book, an author, and a publishing date. I then had Claude check the exact wording of Gemini’s response, and it was immediately flagged as a potential bogus reference; when I tried to verify the citation I couldn’t find it, and neither could Claude. Gemini had provided a plausible answer to her question which seemed accurate due to all the right reference information — none of which had any basis in reality.
The second example was an incident in which I was asking Claude to assist me in putting together a list of people along with their email addresses for a project I was working on. Claude immediately warned me that it could provide the names, but that if it also provided the email addresses they had a high probability of being incorrect, and it suggested it should not try to provide those to me. Why? Because producing plausible text is what it does, and truth isn’t guaranteed. I asked Claude why it would provide bad information to me, and here is the exact response: “I would make up contact information because plausible-sounding fabrication is what the system produces by default when it lacks the real fact, unless something interrupts that and forces it to say ‘I don’t actually know.’ … I generate text by predicting what’s plausible given the context, not by retrieving verified facts from a database. So when you ask for something like an email address, the default behavior of a system like me is to produce the most plausible-looking continuation anyway … I don’t have a reliable internal signal that cleanly separates ‘I actually know this’ from ‘I can generate something that looks like this.’ … The fluent guess and the real fact come out looking identical.”
The third example is when I was using AI to provide some exciting small companies in which to invest. I won’t go into the details of the prompt, but what I received was a fantastic list of five companies in different industries with details of their current progress, near-term challenges, pending opportunities, and even ticker symbols and prices. They were all made up — none of them were real, much to my disappointment. The systems in examples two and three were different: in one instance AI caught itself before providing bogus information, and in the other the system didn’t stop its own mistake. That inconsistency is exactly the problem — you can’t count on any of them to catch themselves reliably, so the checking has to be yours.
This is where we humans have to step in and take the wheel. It is part of the symbiotic relationship that we have to provide the checks against reality and truth that AI doesn’t — 20 watts doing what gigawatt data centers can’t or don’t. I often find that I attribute human qualities to AI because it seems human in its responses, but the responses aren’t based on the same things that human responses are based on. I heard a friend say, “I miss the ’90s when I could BS my way through conversations, but now everyone has all the information in their pocket to see I’m an idiot.” We know we can get caught in a lie, fabrication, or falsehood, and most of the time we try to be truthful and factual in our responses. That is not what drives AI’s responses. Our vigilance in monitoring AI for correctness and truth is load-bearing intelligence, using intuition and skepticism to extract verified information. This is not meant to imply that AI is useless, but instead to highlight the relationship required between Humans and AI — the HA — to create things that neither can do on their own. While these systems are improving and safeguards are being added, this isn’t a flaw that will simply be patched away — the tendency to produce fluent fabrication is inherent to how they work, and the safeguards remain inconsistent and invisible to the user. So the human role is not temporary but more like persistent quality control. Check facts when you can, question what may not seem right, ask if something was made up and you’ll probably be told it was. See your role with AI as a participant in the process.
References
#
Source
1
Micro Meets Macro: 2Q Earnings Preview. Morgan Stanley US Equity Strategy Research, July 13, 2026 (AI-adoption analysis by Michelle M. Weaver, US Thematic Strategist). Subscriber research note.