Customer service chatbots have gained popularity for their always-on ability to streamline the customer experience for a wide variety of tasks — assisting with purchases, offering product support, providing recommendations, answering questions, and more. 

Chatbots have also earned a reputation for being less-than-helpful in certain situations. But thanks in large part to recent advances in AI, many of today’s chatbots are subverting expectations. Platforms like OpenAI’s ChatGPT, Anthropoic's Claude, Google's Gemini, and Microsoft’s Copilot are setting new standards for AI chatbot functionality.

As these chatbots become increasingly more mainstream and accessible, CX departments are often tempted to employ them as cost-effective quick fixes for problems like insufficient support channels, limited service hours, and rapid growth. However, when AI-enabled chatbots are used as a band-aid solution, they can lead to bigger problems — and even PR disasters. 

This raises a few important questions: What challenges do AI chatbots present? Are they worth the potential risk? And if they are, how can effective chatbot implementation strategy  help enhance the customer experience while avoiding rogue AI?

How AI Chatbots Work — and Why They Go Rogue

AI chatbots leverage technology like machine learning (ML) and natural language processing (NLP) to try and understand what customers are saying and respond with relevant information, based on a predefined dataset. They're designed to mirror the basic experience of human conversation and grow more accurate with each interaction, continuously improving over time.

An AI chatbot’s reliability depends greatly on the data it uses. This can range from open-source platforms like OpenAI's ChatGPT, which pulls information from the internet, to closed systems that work from a limited dataset — for example your own FAQs. 

While open-source tools are generally affordable (or even free), they often require extensive customization and are more likely to “go rogue” — providing inaccurate, unintended responses. On the other hand, closed systems give you more control, but might be more expensive.

Rogue AI can be attributed to several factors. First, chatbots with too much creative freedom often fabricate information. It’s challenging to create generative AI that can successfully mimic human conversation (which requires creativity) and not hallucinate, or make things up and present them as fact,(which is often a symptom of creativity).

Lack of quality assurance (QA) and human input is another serious concern. Without regular monitoring and updates, rogue chatbots may continue to repeat mistakes or offer outdated information, and a chatbot left to operate without human feedback is unlikely to improve — in fact, it’s likely to get worse. 

Finally, bad actors, especially those with coding knowledge, can manipulate chatbot vulnerabilities to elicit inappropriate responses. Consequences range from relatively harmless pranks to serious data breaches. 

In the Spotlight: AI Gone Wrong

Recently, chatbots have made headlines for behaving in unexpected ways. Some of these incidents were due to bad actors, while others were primarily due to issues with the design and setup of the chatbots in question. Here are just a few examples of newsworthy chatbots gone wrong:

• A user manipulated a ChatGPT-powered chatbot on a car dealership website into offering him a 2024 Chevy Tahoe for one dollar, even convincing the bot to falsely state that the offer was legally binding. The chatbot provider quickly patched the system to prevent similar issues..

• Air Canada's chatbot gave false information to a passenger booking a flight to attend a funeral. The chatbot assured the passenger that he could book a full-fare flight, then apply for a bereavement fare later. But this went against the airline’s true policy, and they did not honor the chatbot’s promise. Although Air Canada argued that the chatbot was a "separate legal entity” that was “responsible for its own actions,” a civil court ruled that the airline was responsible for everything on their website — including the chatbot — and ordered them to pay the passenger for damages and fees. 

•  International shipper Dynamic Parcel Distribution (DPD) faced a PR nightmare when a frustrated customer manipulated their chatbot into swearing; the bot even wrote a haiku criticizing DPD. The company quickly disabled the update it blamed for the chatbot gone rogue, but the damage was already done — the exchange went viral on X (formerly Twitter).

While these examples are certainly extreme, they should serve as cautionary tales — AI-powered CX tools require careful consideration, training, testing, and maintenance to keep them on the straight and narrow.

Navigating The Benefits and Risks of Chatbots

Although AI is making large strides, it isn’t perfect. Therefore, integrating AI-powered tools into your CX strategy inherently carries some risks. Whether or not it makes sense to take on those risks will depend on your organization’s unique situation, and what advantages an AI chatbot might deliver for your business.

Benefits of Chatbots to Consider

Increased Efficiency and Cost-Effectiveness — Automating routine interactions via AI can help reduce operational costs and increase efficiency.

Enhanced Customer Experience — AI chatbots can improve CX by providing 24/7 support and offering quick solutions to customer queries, improving KPIs like time to resolution. 

Increased Scalability — Chatbots can effectively manage high volumes of repetitive inquiries and free up human agents to tackle complex issues, reducing friction during times of rapid growth.

Chatbot Challenges to Consider

Misalignment with Customer Needs and Preferences — While some bots offer quick transfer to a human representative, others don’t. Not all customers are comfortable interacting with AI, and not having access to human support can exacerbate frustrations.

When AI Is the Problem — The discrepancy between customer needs and what AI chatbots can provide is especially stark when another AI system — one unrelated to customer service — has caused the very issue the customer needs to solve; the last thing they want to do in that situation is interact with another AI-based tool.

For example, Etsy has recently come under fire for a faulty AI-powered account review system that flags and freezes seller accounts that have no legitimate violations. The affected sellers report that they’re forced to go in circles with an unhelpful AI chatbot and are unable to reach a real person to address the issue. This issue likely contributed to a recent mass exodus of sellers from Etsy. 

Glitches and Errors — Chatbots that go rogue due to errors or poor design can confuse customers and potentially damage your business’ reputation.

Choosing the Right Tools — AI technology is evolving fast, and it can be difficult to keep up with all the latest tools and features. The saturated market can make it hard to select a provider, and security and data privacy concerns can add even more complexity to the decision-making process.

Mitigating Risks

Implementing AI-powered CX solutions can streamline operations, and with some common-sense measures in place, you can reduce AI chatbot risks.

If you decide that implementing an AI-powered chatbot solution makes sense for your business, we recommend following these best practices to mitigate risk:

• Create and deploy closed data sets and limit the AI creativity to prevent generating incorrect or irrelevant results. Keep datasets within the confines of provided documentation for control and accuracy.
• Direct the AI on how to proceed when it lacks necessary information to respond to customer inquiries.
• Strike a balance between human and AI capabilities by implementing a human-in-the-loop (HITL) approach.

• Choose technology solutions that allow for customization and control over the AI’s learning process and enable seamless transitions between AI and human customer support.

• Choose a vendor or provider that offers your desired level of support for implementation, continuous improvement, and maintenance. 
• Stay informed about advancements in AI and emerging conversational AI challenges to anticipate and address evolving chatbot security issues and potential vulnerabilities.

Want a Chatbot You Can Feel Confident In?

When implemented thoughtfully, AI chatbots can be worth the risk — and with the right partner by your side, the journey is a lot smoother. 

With SupportNinja, you can utilize AI to improve experiences for both your customers and CX teams. 

We’ll set up a chatbot that pulls only from your carefully-selected knowledge bases, leveraging HITL and delivering regular feedback and iterations to ensure consistent improvement. 

Together, we can turn the challenges of AI chatbots into opportunities.

Contact us to learn more about our CX support offerings.

‍