Cybersecurity: When Agentic AI Attacks

Last month, Anthropic released a report on its investigation of an unprecedented cyberattack by a Chinese state sponsored group.  This excerpt from the report explains what made the attack unique:

The actor achieved what we believe is the first documented case of a cyberattack largely executed without human intervention at scale—the AI autonomously discovered vulnerabilities in targets selected by human operators and successfully exploited them in live operations, then performed a wide range of post-exploitation activities from analysis, lateral movement, privilege escalation, data access, to data exfiltration. Most significantly, this marks the first documented case of agentic AI successfully obtaining access to confirmed high-value targets for intelligence collection, including major technology corporations and government agencies.

This attack occurred around the same time as the “ShadowLeak” vulnerability on ChatGPT was exposed by Radware. This excerpt from Radware’s report explains the implications of ShadowLeak:

ShadowLeak is a newly discovered zero‑click indirect prompt injection (IPI) vulnerability that occurs when OpenAI’s ChatGPT is connected to enterprise Gmail and allowed to browse the web. An attack takes advantage of the vulnerability by sending a legitimate‑looking email that quietly embeds malicious instructions in invisible or non‑obvious HTML. When an employee asks the assistant to “summarize today’s emails” or “research my inbox about a topic,” the agent ingests the booby‑trapped message and, without further user interaction, exfiltrates sensitive data by calling an attacker‑controlled URL with private parameters (e.g., names, addresses, and internal and sensitive information).

A recent Cleary blog addresses the implications of these two vulnerabilities and notes that they represent a frightening milestone in cyberthreats: “AI is no longer merely a tool that aids attackers, in some cases, it has become the attacker itself.”  The blog goes on to discuss the strategic implications of agentic AI’s ability to independently execute complex offensive campaigns and offers some key takeaways for integrating AI into everyday workflows. These include:

– Treat AI assistants and agents like privileged system users. As noted above, organizations should consider separating “read‑only” from “action” permissions, using distinct service accounts and requiring auditable controls for tool use, browsing and API calls.

– Contract for upstream safeguards. Require vendors to (a) sanitize inputs (including stripping risky HTML), (b) validate systems against prompt injection and natural language attack vectors (i.e., by implementing advanced controls such as judge LLM evaluation, spotlighting, and security-focused prompt-engineering patterns) and (c) provide action logs you can audit and use in incidents.

– Build telemetry that captures agent behavior. Insist on provider‑side logs that record who did what, when and why for every agent action, and align those logs to your incident response and reporting needs.

Other recommendations including updating security documentation and protocols to address these evolving agentic AI threats, prioritizing secure AI development, and addressing the interplay of these new threats with mandatory cybersecurity rules such as those imposed by the EU.