How to Stop Bad Actors Hijacking AI

With all the threats companies face in the AI landscape, both old and new, we tend to think big about risk management. There’s no doubt that stopping massive data breaches and ransomware attacks is important, but what about smaller, less serious transgressions? Well, it turns out those can add up too. With AI agents and chatbots, some users have resorted to “hijacking” AI systems. This is when a user prompts an AI to act in a way other than its intended purpose. This can be as simple as a high schooler convincing a retailer’s chatbot to do their math homework, or as serious as hackers prompting their way into company records. A recent Korn Ferry article addresses the issue of bad actors hijacking AI:

“A car dealer’s AI-driven program sells a car… for $1. An AI agent gets its user a spot in a Pilates class by hacking the gym’s website (and kicking out another member). And then there’s the fast-food outlet’s chatbot that customers are using not to order hamburgers but to debug sophisticated computer code… experts say that people’s ability to evade safeguards is frustrating the non-technology organizations that are on pace to lay out $280 billion on AI this year alone. Low-level though they may be, these snafus can still be damaging to both a firm’s reputation and its bottom line. Worse, they could become an even bigger problem as AI tools become increasingly powerful and ubiquitous.”

Bad actors hijacking AI do so through prompts. This means it doesn’t take much time, effort, or technical knowledge. Companies should ensure that public-facing AI integrations have proper guardrails in place. AI systems should not be able to do important tasks like adjusting pricing without human approval.

Rate limiting your AI tools can also be an effective way to ensure they stay on task. This stops one user from requesting responses from the AI too frequently. A customer may have a handful of questions; a hijacker might be making full-time use of your computing power.

Additionally, updating your terms and conditions is an important backstop. Make certain that any unauthorized deals offered by chatbots are not binding. It is also important to include language prohibiting users from utilizing AI tools for unapproved purposes. This makes it easier to ban users who abuse your tools and can form an important element of a courtroom strategy if the transgression leads to litigation.