Rishi Katdare, Senior Leader in Networking and Edge for Global Financial Services at Amazon Web Services.
When I talk with leaders about AI governance, the conversation still centers on model behavior. Hallucinations, bias, data leakage, safety testing and regulatory exposure all matter. Yet as AI systems act across the enterprise, another question becomes harder to avoid. Who or what acted, under whose authority, against which system, and how long will the proof remain defensible?
The next consequential AI failure may not come from a model producing an unreliable answer. It may come from an enterprise being unable to prove that an AI-enabled action was authorized, attributable, properly constrained and auditable.
The Trust Layer Leaders Miss
When a system routes a request to an approved model, the organization assumes the artifact in production is the one teams reviewed. When a model uses enterprise data, leaders assume the lineage is understood, licenses are respected and sensitive information stays where it belongs. When an incident occurs, they assume logs, access records and deployment history can reconstruct what happened.
In my experience, governance reviews often examine model behavior while treating service identities, agent permissions and evidence retention as implementation details. That is where the exposure begins. Those assumptions were easier to manage when enterprise systems were built around human users and static applications. They become fragile when nonhuman actors can initiate transactions, update records, change configuration, recommend approvals and operate across system boundaries. If you give an AI agent authority to act, you have not merely deployed another feature. You have introduced an actor into the operating model.
When Agents Become Actors
This is where AI governance stops being abstract. A support assistant may no longer only suggest a response. It may open, update, route and close cases. A developer assistant may no longer only draft code. It may propose changes, modify pipelines or interact with shared assets. An operations workflow may assemble evidence, trigger escalation or recommend action within defined limits.
Once a system can change a system of record, alter infrastructure, move money or affect a regulated process, it deserves to be governed as an identity with authority. That means it can be over-entitled, impersonated, misused or left running under permissions no one has reviewed closely enough. I have seen too many organizations treat these actors as conveniences rather than principals with concentrated power.
If you want to know whether your AI governance model is mature, do not start with approved use cases. Start by asking how many nonhuman actors can write into systems of record, change access controls, move sensitive data or trigger business outcomes. Then ask whether you can show what identity each actor uses, what authority it holds, who approved it and when it was last reviewed. If you cannot answer that, deployment has already outrun governance.
When Proof Begins To Age
AI governance depends on evidence. You need to know which model was used, which data informed it, which identity acted, which approval path applied and which logs prove the chain of events. The harder reality is that proof is not timeless. Signatures, certificates, keys, identity assertions and evidence trails all depend on trust mechanisms that can weaken as standards, algorithms, threats and regulatory expectations evolve.
You do not need to predict the exact year when post-quantum risk becomes urgent to recognize the management issue. Some evidence created today may need to defend a material decision later. If the trust basis behind that evidence becomes inadequate, the record may still exist while its authority has weakened. A dashboard may show a complete evidence trail, while a lawyer, regulator or risk leader may no longer be comfortable with the assumptions beneath it.
That is why evidence cannot be treated as a static checklist. It has to be treated as a living system with strength that changes over time. Some proofs remain acceptable. Some become weaker. Some may no longer be sufficient for high-consequence decisions. It’s critical to understand whether the evidence will still carry weight when the enterprise needs to defend it.
What Leaders Should Do Now
First, inventory every nonhuman actor with material authority. That includes agents, models, orchestration services, automation scripts, privileged service accounts and workflow engines that can touch sensitive systems or influence business outcomes. For each one, leaders should know what credentials it uses, what systems it can reach, what data it can access, what action it can take, who approved that authority and who owns its behavior.
Second, test whether your trust mechanisms can change before urgency forces the issue. Rotate keys in a bounded environment. Change a signing mechanism. Strengthen an identity pattern. Move one workflow to a stricter trust model and watch what breaks. The goal is not to complete a post-quantum migration in one motion, but to reveal where your AI operating environment assumes the trust foundation will never change.
Third, set evidentiary thresholds before approval. If an AI use case touches revenue, regulated data, production infrastructure or customer outcomes, leaders should define what proof must exist before launch, how that proof will be retained and what would cause the organization to withdraw trust from the system. If you cannot unwind trust, you have not governed it.
The Leadership Test
Leaders must be willing to examine the authority structures that allow AI to act. Intelligence may capture the headlines, but identity will determine whether enterprises can trust, constrain, audit and defend what their AI systems do.
Beyond what the model produces, it must be known who or what acted in the enterprise’s name, under what authority, with what proof and for how long that proof remains defensible. Leaders who answer now can build AI governance on design rather than hope.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
