AI Hallucinations: Understanding the Phenomenon and Its Implications

Key takeaways

• AI hallucinations are incorrect, misleading, or fabricated outputs from generative AI or computer vision systems that arise when models misinterpret patterns in their training data.

• Hallucinations in AI can have serious implications across industries, including incorrect health care diagnoses, poor financial advice, and misleading marketing that can damage a company’s brand and customer trust.

• Causes of AI hallucinations include inaccurate, biased, or incomplete training data; model complexity; data poisoning by bad actors; and overfitting.

Examples of AI hallucinations include outputs containing historical or scientific inaccuracies, wrong financial data, or irrelevant legal advice. Learn what causes these hallucinations, how to reduce their impact, and how to use AI responsibly and effectively in the following article.

If you’re ready to start building expertise in AI, enroll in the IBM AI Foundations for Everyone Specialization. You’ll have the opportunity to gain a fundamental understanding of AI, its applications, and use cases across various industries and become acquainted with terms like machine learning, deep learning, and neural networks in as little as four weeks. Upon completion, you’ll have earned a career certificate for your resume.

What are AI hallucinations?

AI hallucinations are instances in which a generative AI chatbot or computer vision system outputs incorrect, misleading, or completely made-up responses due to the model’s misunderstanding of patterns in its training data. This data may contain factual errors and biases.

Examples of AI hallucinations

AI hallucinations vary from simple incorrect query responses to downright surrealistic outputs. For example, an image generator model might add an extra finger to a hand, a legal chatbot might make up a citation, or an AI model could contradict itself by calling a medicine “widely available” when it previously said it was regulated.

Common AI hallucinations include:

• Historical inaccuracies

• Geographical errors

• Incorrect financial data

• Inept legal advice

• Scientific inaccuracies

Read more: What Is ChatGPT? Meaning, Uses, Features, and More

What causes AI hallucinations?

To understand the causes of AI hallucinations, such as flawed training data or model complexity, keep in mind that AI models can’t “think” in a truly human sense. Instead, their algorithms work probabilistically. For example, some AI models predict which word is most likely to follow another word based on how often that combination occurs in their training data.

Underlying reasons for AI hallucinations include:

Training data limitations

One problem with AI training is input bias, which is present in the wide range of data programmers use to train AI models. The training data AI models learn from may contain inaccuracies, biases, or gaps, leading them to draw incorrect conclusions. Without proper grounding in real-world knowledge and facts, AI models can’t distinguish between true and false information, producing inaccurate and biased hallucinations as if they were reliable information.

Model complexity

If an AI model is so complex that it lacks constraints limiting the kind of outputs it could produce, you may see AI hallucinations more frequently. To address hallucinations directly, you can take measures to limit the probabilistic range of an AI model’s learning capacity.

Data poisoning

Data poisoning occurs when bad actors, such as black hat hackers, input false, misleading, or biased data into an AI model’s training data sets. For example, faulty data in an image can cause the AI model to misclassify the image, which may create a security issue or even lead to a cyberattack.

Overfitting

An AI model displaying overfitting tendencies can accurately predict on the training data, but can’t generalize what it learned from that data to predict new data. Overfit AI models learn irrelevant noise in data sets without being capable of differentiating between noise and what you meant for them to learn.

For example, imagine you’re training an AI model to recognize humans and feed it photos of people. If many of those images show people standing next to lamps, the model might mistakenly learn that lamps are a feature of people and eventually start identifying lamps as people.

Why does ChatGPT hallucinate?

OpenAI explains that hallucinations are a fundamental issue in large language models (LLMs) like ChatGPT [1]. Hallucinations result from the current evaluation methods’ incentive system, which rewards LLM models’ guessing rather than admitting uncertainty. Despite newer models having lower rates of hallucinations, OpenAI asserts that having primary evaluation metrics that reward an “I don’t know” response when uncertain and penalize guessing an answer through pattern-based next-word prediction is key to mitigating AI models’ hallucinations.

Implications of hallucinations in AI

A wide variety of industries and sectors use AI, and its use in the business world is likely to expand. Occupations that utilize AI include retail, travel, education, customer service, health care, finance, and marketing. As AI adoption grows across industries, the risk of AI hallucinations presents a real challenge for businesses.

Regardless of the business in which you work or plan to work, understanding AI hallucinations can help you be aware of and mitigate their effects when using AI. Take a closer look at how AI hallucinations might affect the health care, finance, and marketing industries.

Health care

While AI can help detect issues doctors might miss, it may also hallucinate health problems, like cancerous growths, leading to unnecessary or even harmful treatment of healthy patients.

This can happen when a programmer trains an AI model on data that doesn’t distinguish between healthy and diseased human examples. In this instance, an AI model doesn’t learn to distinguish differences that naturally occur in healthy people, for example, benign spots on the lungs, from images that suggest disease.

According to OpenAI, over 230 million people around the world seek health advice from ChatGPT every week [2]. This increases the surface area for damage that AI hallucinations can cause, since AI chatbots occasionally offer reassurance without considering patient history, leave out important safety considerations, or provide unsafe advice due to a lack of medical context awareness and clinical training.

Hallucinations also pose a risk with AI-generated medical documentation. Where medical professionals would originally create and verify clinical patient records themselves, using AI to generate patient documentation increases the risk of false information being included in reports. This could include fabricated medical histories, symptoms, summaries, and timelines, all of which can influence treatment decisions and patient outcomes.

Finance

AI hallucinations occurring within the financial sector can also present problems. Many large banks utilize AI models for:

• Making investments

• Analyzing securities

• Predicting stock prices

• Selecting stocks

• Assessing risk

AI hallucinations in this context can result in bad financial advice regarding investments and debt management. Because some companies aren’t transparent about whether or not they use AI to make recommendations to consumers, some consumers unwittingly place their trust in technology they assume is a trained expert with sophisticated critical-thinking skills. AI-based financial forecasting and decision-making, although beneficial in the short term, require the AI model to understand how factors such as structural policy changes, economic volatility, and geopolitical shifts affect financial decisions to provide more reliable long-term predictions.

Marketing

In terms of marketing, you might have worked for years to develop a specific tone and style to represent your business, but if AI hallucinations produce information that is false, misleading, or does not align with how you typically interact with your customers, you might face the erosion of your brand’s identity. Consequently, this might also disrupt the connection you worked to establish with your customers.

Essentially, AI could generate messages that distribute false information about your products while also making promises your company cannot fulfill, which may present your brand as untrustworthy.

Mitigating AI hallucinations

Fortunately, when dealing with AI hallucinations, strategies like data quality and user education can help mitigate their impact. Explore a few of the strategies for reducing the occurrence of AI hallucinations:

Data quality improvement

One way to reduce the possibility of AI hallucinations is for programmers to train AI models on high-quality data. Data should be diverse, balanced, and well-structured. Simply put, AI output quality correlates with input quality. You’d be just as likely to give faulty information if you read a historical book with factual inaccuracies.

Model evaluation and validation

You can implement rigorous testing and validation processes to identify and correct hallucinations. Your business can also work with vendors committed to ethical practices in AI development. Doing this allows for more transparency when updating a model as issues arise.

You can also decrease the possibility of AI hallucinations by giving your AI model strong prompts, which can improve its output. Another option is to use pre-defined data templates, which can help your AI model output more accurate content.

Also, consider using filters and predefined probabilistic thresholds for your AI model. If you limit an AI’s capability to predict quite so broadly, you may reduce hallucinations.

User education

Educating yourself, colleagues, and others you interact with about AI hallucinations is important to reduce the assumption that technology is objective and accurate. To combat this, for example, become educated about the limits and capabilities of a large language model (LLM). Doing so will help you better understand what an LLM can and can’t do and be more equipped to identify a hallucination.

Human oversight

It's also essential to implement human oversight. This might mean refraining from fully automating your system with AI and assigning a competent professional to review outputs for any signs of hallucinations. Having a trained reviewer evaluate responses before they reach the end user can not only reduce the risk of hallucinations but also contribute to improving prompts and systems over time.

It’s also advantageous to work closely with subject matter experts (SME). They can correct factually incorrect data in specialized fields.

Explore structured resources to deepen your knowledge

Explore career paths, assess your skills, and connect with resume guidance while browsing our Career Resources Hub. Or if you want to learn more about AI, check out these free resources:

• Read our Career Chat issue: AI Ethics: What It Is, Why It Matters + Challenges to Know

• Hear from an expert: AI Problem-Solving Tactics: Expert Insights from Vanderbilt’s Dr. Jules White

• Watch on YouTube: The Dark Side of AI | Challenges, Risks, & Ethical Considerations

Whether you want to develop a new skill, get comfortable with an in-demand technology, or advance your abilities, keep growing with a Coursera Plus subscription. You’ll get access to over 10,000 flexible courses.

Build job-ready skills with Coursera Plus

Start 7-day free trial

• 
• 
• 
• 

Start 7-day free trial