Agentic AI: What CIOs & CISOs Need to Know About Agents’ Vulnerabilities

Boston Consulting Group recently issued a report discussing the unique risks posted by Agentic AI tools and provides insights on what companies must do to build a risk framework specifically tailored to address the unique cyber vulnerabilities posed by Agentic AI. That starts with the CIO & CISO understanding the nature of those risks, which this excerpt indicates are beyond those that information technology professionals are accustomed to addressing:

CIOs and CISOs need an extra level of understanding on the specific cyber vulnerabilities of AI agents. Each enables malicious actors to exploit one of the four components mentioned above. Unfortunately, these new threats come with a whole new vocabulary.

State Representation Risks. These are the risks that come from AI agents having “memory” and include:

• Context Poisoning and State Corruption: A hacker or malicious insider can corrupt the agent’s internal “world model” through manipulated inputs or logs, leading to persistent misperception of reality and altering the agent’s behavior, to the hacker’s advantage.
• Adversarial Prompt Injection at Scale: GenAI models have difficulties separating data inputs from new prompts/instructions. Capitalizing on this, attackers hide prompts in emails, chat messages, or websites that agents crawl. Again, this allows them to alter how the AI agent acts.

Reasoning and Decision Risks. These exploit the greater decision-making skills and are typically more direct attempts to control the agent or its ecosystem, and include:

• Agent Hijacking: While prompt injection explained above subtly hijacks an agent through manipulated input, this is more direct, directly accessing its capabilities or processes.
• Goal Manipulation: An attacker who can alter an AI agent’s goals can dramatically change its behavior. For instance, a customer service bot may have its goal changed to issue as many refunds as possible.

Action and Influence Risks. Here, malicious actors aim to exploit the connection between the AI agent and the environment it inhabits. Attacks include:

• Toolchain Exploitation: An attacker may insert themselves between the AI agent and the systems it interacts with, for instance, replacing bank details in a payment with their own.
• Unauthorized Autonomy Escalation: This is a way of enhancing the illicit gains of other types of attacks, allowing a compromised agent to access actions or data that should be outside its reach.

Iterative Loop Risks. Here, attackers are capitalizing on a key capability in AI agents: their ability to evolve, iterate, and cooperate, but turning it to malevolent ends. Attacks include:

• Cross-Agent Contagion: This is a subtle, sophisticated attack in which a single corrupted agent influences the behavior of many others. Attackers may now have control of an entire ecosystem.
• Data Leakage Through Emergent Behavior: This strange yet very real vulnerability stems from AI’s ability to infer information it has never been explicitly told. For instance, based on purchasing data, an AI agent may be able to infer a customer’s age, income, and medical conditions. It may then be persuaded to divulge it to unauthorized outsiders.

BCG’s report goes on to say that an effective risk management framework for Agentic AI’s vulnerabilities requires construction of an agent-specific risk taxonomy that maps risks across technical, operational, and user risks, conducting simulations of real-world conditions prior to deployment, implementation of real-time behavior monitoring, and designing-in resilience and escalation protocols.

Our blog will be off for the holidays and will return in the new year. Thanks to everyone for reading and Happy Holidays!