AI Hallucinations in Production: A Damage Control Playbook
A customer asks your AI support agent about a refund. The agent confidently quotes a 60-day return window. Your actual policy is 30 days. The customer screenshots the response, files a complaint, and a tribunal sides with them. The business honors the 60-day window. Welcome to AI procurement in 2026.
This is not a hypothetical. Air Canada lost exactly this case in 2024, when a Canadian civil tribunal ruled the airline had to honor a bereavement fare its chatbot invented. [1] The tribunal rejected the defense that the chatbot was "a separate legal entity." DPD's chatbot called itself "useless" and wrote a poem criticizing the company. [2] New York City's "MyCity" small business chatbot advised users to break federal labor law. [3] In 2025, Cursor's AI support agent invented a single-device login policy that did not exist, triggering visible user cancellations before the company corrected it. [4]
The pattern is consistent: customer-facing AI invents something, the customer relies on it, and the business owns the consequences. The fix is not "use a better model." It is a documented playbook for prevention, detection, and response.
Recent Cases Worth Studying
These are the incidents that shaped current legal and operational thinking on AI hallucinations. Each one ended differently, and the differences are instructive.
Three lessons stand out. First, speed of acknowledgment determines reputational outcome more than the original error. Second, the businesses that defended the AI in court lost; the businesses that fixed the issue and refunded the customer survived the news cycle. Third, every one of these incidents was preventable with the controls below.
The 5-Stage Damage Control Playbook
The playbook is organized into two halves: before the incident (prevention and detection) and after (response and prevention). Build all five stages before any customer-facing AI agent goes live.
Make the AI unable to invent answers in the first place
Use retrieval-augmented generation (RAG) backed by a curated, version-controlled knowledge base. Instruct the model explicitly to refuse questions it cannot answer from retrieved sources. Add a fixed list of "do not answer" categories: pricing, refunds, legal terms, medical advice, anything that creates business or regulatory exposure. The agent should escalate, not improvise, on those topics.
Tell every user they are talking to an AI
This is now legally required in the EU under the AI Act, and increasingly required at the state level in the US (California, Utah, Colorado). Beyond compliance, disclosure changes user behavior: people who know they are talking to an AI are less likely to treat an AI-stated commitment as binding. The disclosure should be visible at the start of every conversation, not buried in a privacy policy.
Log everything and run automated quality checks
Every AI response in production should be logged with the prompt, retrieved context, model output, and any tools called. Run two passes on the logs: an automated classifier looking for high-risk topics (pricing, policy, legal), and a daily sample for human review. The companies that catch hallucinations early have observability. The companies that hear about them on Twitter do not.
Have a documented incident response plan before you need it
Define in writing: who owns the first response, what authority they have to take the AI offline, how customers are notified, and what the public statement template looks like. The 24-hour acknowledgment, 48-hour containment, 7-day root cause, 30-day prevention timeline is a reasonable default. The exact numbers matter less than the fact that they exist before the incident.
When the AI made the promise, honor it
This is the hardest stage culturally and the one that protects the business most. Once you decide to litigate an AI commitment, you have lost the news cycle regardless of the legal outcome. The Air Canada decision made this explicit, but it was always true commercially. Honor the commitment, refund the customer, document the fix, and use the case as the internal evidence that the prevention work is worth funding.
The Prevention Stack vs. The Response Stack
Most teams overinvest in one of these two and underinvest in the other. The prevention stack is what runs every day. The response stack is what runs the day something goes wrong. Both are required, and neither substitutes for the other.
Prevention Stack
- RAG over curated knowledge base
- Refusal instructions for high-risk topics
- Source citation required in responses
- Confidence thresholds with auto-escalation
- Pre-launch red-team testing
- Versioned prompts and knowledge base
Response Stack
- Full conversation logging with retention
- Automated high-risk topic classifier
- Daily human sample review
- Documented incident response plan
- Public response template ready to fill
- Refund and resolution authority defined
The Question Every Deployment Should Answer
Before any customer-facing AI agent ships, the team should be able to answer one question in writing: if this agent tells a customer something wrong tomorrow, what happens in the next 24 hours?
If the answer is "we'd find out from a screenshot on social media and figure it out from there," the deployment is not ready. If the answer is "the logging system flags it, the on-call owner takes the topic offline, the customer is contacted, and the public response goes out within four hours," the team has built the right thing.
The 2024 to 2025 incidents are not anomalies. They are the visible cases. The unreported cases, the small ones absorbed by support teams without making headlines, are larger by an order of magnitude. The companies treating AI hallucinations as an inevitable operational risk, not a model bug to wait out, are the ones whose deployments will still be running customer conversations in 2027.
Key Takeaways
- The business owns what the AI says. The Air Canada precedent eliminated the "the chatbot is a separate entity" defense. Plan for the AI to be treated, legally and reputationally, like a human employee.
- Constrain before you ship. RAG with strict refusal instructions on high-risk topics prevents the most common and most damaging hallucinations. Most production AI failures are deployment failures, not model failures.
- Disclose every time. Tell users they are talking to an AI. It is increasingly required by law and consistently improves business outcomes when something goes wrong.
- Speed of acknowledgment beats perfection of response. Companies that confirmed an issue within 24 hours kept the story short. The longer the silence, the longer the news cycle.
- Honor the commitment, then fix it. Once an AI has made a promise to a customer, defending against it costs more in trust than the commitment costs in dollars.
Frequently Asked Questions
Who is legally responsible when an AI chatbot gives wrong information to a customer?
The business that deploys the AI is responsible, not the vendor that built it. The Air Canada decision in 2024 set the precedent: a Canadian tribunal ruled the airline had to honor a bereavement fare invented by its chatbot, rejecting the argument that the chatbot was a separate legal entity. Subsequent cases in the United States and Europe have followed similar reasoning. If your AI tells a customer something, the business owns that statement the same way it would own a statement from a human employee.
What is the most effective way to prevent AI hallucinations in customer-facing systems?
Retrieval-augmented generation (RAG) combined with a strict source-citation requirement is the most effective single intervention. Instead of relying on what the model learned during training, the system retrieves answers from a controlled knowledge base and is instructed to refuse any question it cannot ground in a retrieved source. This does not eliminate hallucinations, but it cuts them substantially and gives the business a clear audit trail when an answer is challenged.
How quickly should a business respond when an AI hallucination is reported?
The first response should happen within 24 hours, even if the resolution takes longer. The standard sequence is: acknowledge the issue publicly within 24 hours, take the affected functionality offline or constrain it within 48 hours, post a root-cause statement within 7 days, and document the prevention fix within 30 days. The companies that have handled this well, including Cursor in 2025, moved fast on the acknowledgment and the constraint, then took the time they needed on the fix.
Should I disclose to customers that they are talking to an AI agent?
Yes, and increasingly the law requires it. The EU AI Act, in force since 2026, mandates disclosure when a person is interacting with an AI system. California, Utah, and Colorado have passed similar requirements at the state level. Beyond compliance, disclosure protects the business: a customer who knows they are talking to an AI is less likely to assume an AI-stated policy is a verified commitment, which reduces both legal exposure and reputational damage when an error occurs.
Sources & References
- Moffatt v. Air Canada, 2024 BCCRT 149. British Columbia Civil Resolution Tribunal, February 2024. Decision establishing that an airline is liable for representations made by its customer service chatbot.
- The Guardian. "DPD AI chatbot swears at customer and calls itself the 'worst delivery firm in the world.'" January 2024. Reporting on the viral DPD chatbot incident and the company's response.
- The Markup / AP News. "NYC's AI Chatbot Tells Businesses to Break the Law." April 2024. Investigation of the MyCity small business chatbot's advice on labor law, housing, and consumer protection.
- Reuters / TechCrunch. "Cursor's AI support agent invented a login policy that didn't exist." April 2025. Coverage of the Cursor incident and the company's response timeline.
- Vectara. "Hallucination Leaderboard for Large Language Models." 2025 to 2026 benchmarks tracking measured hallucination rates across foundation models on grounded summarization tasks.
- Edelman Trust Barometer. "Trust and Technology Report 2026." Annual report covering business trust impacts of AI incidents and recovery time correlations.
- European Commission. "Regulation (EU) 2024/1689 on Artificial Intelligence (AI Act)." Entered into force August 2024, with transparency requirements for AI systems applying through 2026.