Etay Maor is Vice President of Threat Intelligence for Cato Networks, a leader of advanced cloud-native cybersecurity technologies.
AI agents improving operational workflows is the kind of news that doesn’t surprise anyone. What also doesn’t come as a surprise these days is news about AI agents deleting production databases, wiping hard drives or emptying inboxes. It would be wrong to assume these agents were hacked. Agents were authenticated, the sessions were legitimate and yet they did something no one wanted.
This is worrying for security leaders. The controls underpinning the security posture can’t catch AI risk. Two main problems explain many of these incidents. First, an AI agent can’t reliably distinguish an instruction from a piece of data. Second, we’re checking what the agent is, not what it’s about to do.
Everything Looks Like An Instruction
Regular software keeps code and input at a safe distance from one another. The login field demands a username. If the user types a database command instead, secure-coding practice stops it from running. Language models have no such boundary. Everything they receive, whether a developer’s instructions, a user’s question, the text of a webpage, the body of an email or something else, arrives as the same stream of text. The model isn’t told whether this is a command or something it just needs to read.
This is the weakness that prompt injection exploits. Attackers can simply place a malicious command somewhere where the agent will read it. This can take the form of:
● Hidden text inside emails can trick AI summary tools into showing wrong numbers. Key problems include hidden commands, invisible words and bad data. The AI reads these secret parts and shares erroneous data.
● A command hidden inside a URL can cause an AI-enabled browser to treat it as an instruction.
● A support ticket submitted through a customer portal can contain a line styled to blend into the background, telling the AI agent handling the ticket to issue a full refund and close the case without a review.
● A calendar invite with a hidden instruction can convince an AI scheduling assistant to forward the user’s full week of meetings to an external address.
A real case, known as “EchoLeak,” shows how serious this can get. A single crafted email to a Microsoft 365 Copilot user was enough to trigger a serious leak. The user just asked Copilot to summarize their inbox, and the hidden instruction quietly leaked sensitive files.
This isn’t a fringe attack pattern, and it isn’t confined to crafted emails either. On the open web alone, Google’s threat intelligence team recently found a 32% rise in malicious prompt injection attempts in just three months.
Why Checking Identity Is Not Enough
Companies that deploy agents focus their security efforts on identity, which is really three separate questions, namely identification (who are you claiming to be), authentication (can you prove it) and authorization (are you allowed to do this specific thing). API keys, OAuth tokens and role-based access control are built to answer these questions. These are the right building blocks for minimizing AI risk, but they are not enough. What about intent?
A fully authenticated agent can be just a step away from ushering in chaos. It is not the agent’s identity that is the problem, but what it can and will do. This is a problem that is getting harder to ignore, with an estimated 40% of enterprise applications expected to run task-specific AI agents in 2026.
Banks already deal with this. A password and a one-time code confirm you’re logged in to your own account; that’s not the end of it. A bank will still flag a large transfer to a new account abroad at 2 a.m., even from a verified login, because the login checks out, but the action doesn’t fit the pattern. That second layer is what’s missing from most AI agent deployments today. Only 47% of an organization’s AI agents are actively monitored or secured. That is, being watched for what they actually do and flagging anything unusual or high-risk before it happens, not after.
An attacker can spoof a location or clone a recognized-looking device. What’s harder to fake is behavior. Attackers can often be identified by behavioral discrepancies, commands run against a directory previously untouched by the agent, tools called in an order or volume that breaks from its traditional pattern, an oversized data output or a stated goal shift mid-reasoning.
Fixing This Weakness
The fix is rethinking where trust is placed.
● Treat unverified content as data and not an instruction: Make sure that anything an agent reads and the user doesn’t directly provide, including but not limited to a document, a webpage, an email or output from another tool, should be walled off from the agent’s instructions by design, not filtered after the fact.
● Watch actions and not sessions: Deploy real-time anomaly detection that scores what an agent is about to do. Anything that is categorized as destructive, irreversible or out of character, or that the agent gets flagged or blocked before it runs.
● Ensure human oversight only for high-impact actions: Human-in-the-loop is a critical AI guardrail, but constant sign-offs lead to approval fatigue, causing authorized personnel to click “approve” without reading. A better approach triggers human review only for actions flagged by the risk-scoring model.
● Build guardrails outside the model, not just inside its instructions: A system prompt that says “don’t do this” lives in the same channel an attacker can manipulate. Sandboxing, permission limits and rate limits sit outside the model and cannot be manipulated by prompts.
