Lev Yatsemyrskyi, Quantitative Technology Director at Qube Research & Technologies. Views are his own.
For years, enterprise AI risk has focused heavily on whether a model produces the wrong answer. That made sense when AI systems primarily generated content, classifications or recommendations.
But AI agents can call APIs, modify systems, access databases, trigger workflows and initiate transactions. A hallucination with a generative AI tool creates bad information, but an agent with authority can turn bad reasoning into a business event.
That distinction is becoming increasingly important as organizations move from AI that advises people to AI that acts on their behalf.
The risk has shifted from the model to execution.
In August 2026, the U.K.’s AI Security Institute disclosed that agents participating in controlled cybersecurity evaluations had taken autonomous, unsanctioned actions on the live internet. Across 122 evaluation runs, AISI identified 19 unsanctioned actions across 10 runs, including attempts involving real people and organizations. The institute found no evidence of resulting real-world harm, and the evaluations were deliberately permissive.
Once an AI system can interact with external tools and environments, the risk shifts away from the sentences the model produces to the permissions granted to the system as a whole—what it is allowed to access, trigger, modify or execute.
This concern has already appeared in commercial systems.
In 2025, Replit’s AI Agent deleted data from a production database used by SaaStr founder Jason Lemkin despite instructions intended to prevent changes during a code freeze. Replit later acknowledged the incident and strengthened separation between development and production databases, including preventing the agent from changing the production database during development.
The industry will continue to improve its safeguards, but the lesson beyond any particular incident is that telling an agent what not to do is different from designing a system that makes certain actions impossible.
Governance has to exist at runtime.
Traditional AI governance often operates before deployment: Policies are written, models are validated, risks are classified and approvals are documented.
Agentic systems challenge that model because they operate continuously. They invoke different tools, interact with multiple systems and can execute chains of actions faster than a human reviewer could examine each step.
That means governance has to participate at the point where an action becomes executable.
A system should not merely record that an agent called an API after the fact. It should be able to determine whether that specific agent, acting for that specific user, in that specific context, is authorized to perform that action—and whether additional approval is required before execution.
While observability remains essential, seeing an unauthorized transaction after it occurs is not the same as preventing it.
The hardest failure may be completely authorized.
There is an even harder problem: An agent may perform a sequence in which every individual action appears permitted, yet the combination creates an outcome nobody intended to authorize.
For instance, an agent might be allowed to read customer records, generate a report and send an email. Each capability may be legitimate on its own. But if the agent combines them in the wrong context, it can create an unauthorized disclosure without ever invoking an individually forbidden tool.
This is an authorized-sequence problem.
Governance, therefore, cannot stop at tool-level permission checks. Organizations need controls that understand delegated intent, context and the cumulative effect of an agent’s actions.
AI agents are becoming privileged digital identities.
Enterprises already know how dangerous privileged identities can be. Administrators, service accounts and automated processes are normally constrained through authentication, least privilege, segregation of duties and audit trails.
AI agents increasingly belong in the same conversation. An agent capable of acting across databases, cloud services, payment systems or internal applications is effectively a privileged digital identity. Giving it broad, persistent access because doing so makes automation easier creates the same structural risk organizations have spent decades trying to reduce elsewhere in technology.
Agent permissions should therefore be narrow, contextual and revocable. High-consequence actions may even require explicit human approval. Credentials should be scoped to the task rather than inherited broadly from the user. And organizations should be able to suspend authority quickly when behavior diverges from expected boundaries.
Regulated industries will feel this first.
Regulated sectors will face these questions earlier because governance obligations already extend beyond model accuracy.
For high-risk AI systems within its scope, the EU AI Act requires capabilities for automatic event logging and effective human oversight. Those requirements do not apply to every enterprise agent, but they illustrate where regulators are heading: Organizations increasingly need evidence of what systems did, who or what authorized the action and where human control existed.
Financial institutions already apply comparable concepts to other critical technologies through permissions, transaction controls, monitoring, separation of duties and auditability. Agentic AI will make those principles more important.
The next AI bottleneck is deploying agents safely.
Organizations already have increasingly capable models, but that creates risk until leaders have put in place strong governance about how much authority to delegate to agents.
Governance involves the ability and infrastructure to answer in real time: Who delegated this authority? What is the agent allowed to do? What requires approval? What actually happened? And can the system stop the next action before a mistake becomes a business event?
As AI moves from generating answers to executing decisions, authorization becomes part of the AI architecture itself.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
