Why Do LLMs Hallucinate? Exploring the Causes & Mitigation Efforts

Why do LLMs hallucinate? What causes these sophisticated language models to tell us things that aren’t true?

Large language models (LLMs) such as ChatGPT have revolutionized how we interact with artificial intelligence, generating impressively human-like responses. Yet these models frequently produce false information—known as hallucinations.

Keep reading to discover the fascinating reasons behind these mistakes and discover what researchers are doing to create more reliable systems.

Why LLMs Hallucinate

If you’ve ever had a conversation with a chatbot such as ChatGPT, you’ve seen that AI models can do an amazing job of sounding like a person. But they’re also prone to mistakes—often wildly wrong information—called hallucinations. Hallucinations are difficult to eliminate, and they might be with us for the foreseeable future. So, why do LLMs hallucinate? We’ll look at how LLMs work, what hallucinations are and what causes them, and the efforts to reduce these mistakes.

How LLMs Work

AI researchers have been developing large language models (LLMs) for years. But LLMs have taken off thanks to the most recent models’ impressive capabilities in generating human-like text. You can see this skill on display when you interact with a chatbot such as ChatGPT: In their training, they learn complex patterns and relationships that are embedded in language. Their ability to recognize and use these patterns and relationships enables them to answer questions and participate in a conversation. When you ask an LLM a question, it can often produce text that not only sounds like a human could have written it but is also accurate and relevant.

LLM Hallucinations

LLM hallucinations occur when an AI model generates information that sounds plausible but proves to be factually incorrect or even nonsensical once you dig deeper. They do this all the time—so often that researchers can’t definitively say how often LLMs say something inaccurate.

Experts say that hallucinations are inherent in the way AI is built. They come about because of the many constraints on what a model can do when you ask it to answer a question or complete a task. While these models are “trained” on vast amounts of data, they can’t capture every nuance and detail of human knowledge, or understand information as we do. Instead, they rely on statistical patterns and approximations to read and write text. That means that often, LLMs can produce outputs that seem coherent and believable but aren’t actually true. In this article, we’ll take a closer look at how LLMs work, why they hallucinate, and whether that’s a problem researchers can solve.

Causes of Hallucinations

Depending on whom you ask, experts blame LLM’s tendency to hallucinate on different aspects of the way they’re trained with token limits, the data they use, and the way they make inferences about the patterns in that data. There’s a different argument for each explanation—though ultimately, many experts agree that what makes LLMs so good at generating text is also what makes them so good at generating inaccurate text.

Cause #1: Bad Data

The quality and diversity of the data a model is trained on have a major impact. If the training data contains biases, inconsistencies, or factual errors (such as the claim that Berlin is the capital of France), then the model can learn these flaws and perpetuate them. Then, it can produce hallucinated content.

For example, if a model’s training data includes inaccurate information about a historical event, such as the first moonwalk—perhaps the wrong date or an incorrect explanation of who was involved—the LLM can incorporate these inaccuracies into its generated text. It can do this even when it’s seen the right fact but has also seen other data that causes it to mix things up and make incorrect associations. It presents these inaccuracies as facts. That’s because the model doesn’t know that these “facts” are wrong, and it doesn’t know that it’s saying something that isn’t true. If asked about the first person to walk on the moon, a model might respond, “Michael Jackson was the first person to walk on the moon in 1969,” incorrectly conflating Michael Jackson’s signature dance move with Neil Armstrong’s historic lunar steps.

Cause #2: Token Limits

LLMs operate within computational constraints, which can lead to hallucinations. A key limitation is the number of “tokens”—data units that can be words, characters, or word fragments—that models can process at once. Because data processing is resource-intensive, LLMs must work within the hardware’s memory, energy, and time limitations.

Some researchers argue that hallucinations are unavoidable due to these token limits. When answering a question such as “What is the capital of France?” the LLM generates its response by predicting the most likely next token based on previous tokens. To work within their token limits, models often generate responses one token at a time rather than planning ahead—essentially making up answers as they go along.

This approach often enables LLMs to generate text that’s accurate and makes sense. But that isn’t always the case. Depending on how well the model can find relevant facts and summarize them, it may produce the wrong answer instead of the right one.  For instance, an LLM might generate the answer, “The capital of France is Berlin.” That statement sounds plausible if you’re just looking at the sentence structure. But it’s factually incorrect. The model could come to an answer like this one because it’s made an incorrect prediction and is choosing an answer that might sound right, but is wrong.

The Current Inevitability of Hallucinations

While tools such as ChatGPT are known to produce some questionable text—which can be a problem if you’re depending on a chatbot to do your homework or draft that report your boss asked for—it’s an open question whether future versions of the models that underlie these chatbots are going to be as prone to hallucination as current versions. Here’s the bad news: Researchers contend that hallucination is inevitable with LLMs. They suggest that, despite sophisticated efforts to improve the accuracy and reliability of LLMs, some degree of hallucination is going to persist in models that are built and trained this way.

That said, the extent and impact of hallucination can vary significantly, depending on what model you’re using and what you’re using it to do. Recent studies have focused on classifying and quantifying the rate of hallucination for many models and figuring out the best ways to detect hallucinations. The goal is for researchers to better understand the nature and prevalence of hallucinations. With that knowledge, they can work toward reducing their effects. That might lead to LLMs that are less likely to hallucinate.

Strategies to Reduce Hallucinations

Researchers have already identified ways to make some models less prone to hallucination. One approach is to try to better ground LLMs in factual information by giving them the ability to check their own answers. If they can determine whether their answer is accurate, using math, then they can correct their own errors and avoid hallucinations. (However, math can’t make up for LLM’s lack of real-world experience, a weakness that’s on full display when models produce an answer such as, “The human body contains exactly 206 toes, which are crucial for maintaining balance while walking on our hands.”  

Another promising approach to reducing hallucinations is to provide models with external knowledge sources and fact-checking mechanisms that use information outside of the model and its training data. These tools can help models and their users identify and correct hallucinations in real time. A model might first produce the hallucination, “The Great Wall of China is visible from the moon.” You could prompt it to fact-check itself and get the correction, “Actually, that’s a myth. The wall isn’t visible to the naked eye from lunar orbit. The Great Wall of China is impressive, but not that impressive.” 

These strategies may not wholly eliminate hallucinations. But they could significantly reduce the occurrence (and impact) of the kinds of inaccuracies that users currently see with these models. In the meantime, it’s essential to remain aware of the potential for hallucinations and approach the information generated by LLMs with a critical eye—especially when your boss or your professor is watching.

Want to fast-track your learning? With Shortform, you’ll gain insights you won't find anywhere else .

Here's what you’ll get when you sign up for Shortform :

• Complicated ideas explained in simple and concise ways
• Smart analysis that connects what you’re reading to other key concepts
• Writing with zero fluff because we know how important your time is

Start your free trial today!