Heather Ceylan is the Chief Information Security Officer at Box, where she leads the global information security program and strategy.
In security, the metric I distrust most is a low incident count.
A quiet quarter can mean the controls are working. Or it can mean that we weren’t looking in the right places. Telling those two apart has always been part of the job, but with AI moving into every corner of the business, it has become more important than ever.
Earlier this year, my team at Box, working with Harris Poll, surveyed 1,640 IT decision-makers about how they’re deploying AI. Nearly half said an AI tool had already exposed information someone wasn’t supposed to see, and one in six called the incident significant. But the more interesting detail is which companies reported it.
Maturity Doesn’t Prevent Incidents; It Reveals Them
The organizations furthest along with AI—the ones running agents across the business—reported more incidents, not fewer. Sixty percent of the most mature reported an exposure event, compared with 46% of the least mature. That’s easy to misread as the leaders having weaker controls than everyone else. A more careful read is that they’re running more agents across more systems, so there’s more that can go wrong and they’re far better at spotting it when it does.
The difference is visibility. More than a quarter of the least mature organizations had never audited for an AI incident; only 17% had a clear view of how their employees use AI with company data. At the leading edge, that figure was 73%. A clean report from a company that can’t see its own AI usage isn’t evidence of safety; it’s a blind spot written up as a clean bill of health.
The capability that separates the two groups is detection. Our own AI rollout showed me where the real challenge is. We already had a clear view of which AI tools people were using; blocking the unsanctioned ones by default was easy. The harder layer, and the one that keeps most incident reports quiet, is what tools and agents do once they have legitimate access; whether anyone is watching for unauthorized data access and exposure incidents that don’t announce themselves. Blocking an unsanctioned tool is straightforward; detecting misuse of an approved one is where the real work is, and where most organizations have the least coverage.
Other high-stakes industries settled this question long ago. The airlines and nuclear operators with the strongest safety records also keep the most detailed incident and near-miss logs. They surface small failures aggressively, so those failures never compound into large ones. In those fields, a thin incident log is read as a warning, not a trophy.
You Can’t Govern What You Can’t See
None of this makes visibility a substitute for control. Seeing an agent misbehave and being able to stop it are different jobs, and you need both. But they run in order: you can’t govern what you can’t see. The organizations that are getting this right built the visibility first. But most are still putting that foundation in place; in our survey, only about a third had a formal standard for how agents access company data—the groundwork that the rest of governance depends on.
The same principle reaches past day-to-day detection. Leadership has to see the organization’s real risk, and that depends on what you choose to measure. This is why we spent much of the past year evolving how we measure security risk. Our earlier metrics leaned on operational signals like service levels and remediation timelines, which show how well the security program runs more than how much exposure we carry. We’ve shifted the program toward what a misconfigured agent or an attacker could reach, and whether we’d catch it. Risk you can’t measure is risk you can’t govern—it’s the same problem, just one level up.
Change The Question You Ask Your Security Team
For executives, the shift here is small but meaningful. Most leadership teams ask how many incidents occurred last quarter. The better question is whether the organization would know if one had. Asking only the first quietly rewards teams for staying silent, which raises real risk while improving the optics—the worst combination a business can buy without realizing it.
So if your AI incident numbers went up the year you got serious about AI, that might be your detection doing its job. The number I’d push on is the one that holds flat while the AI footprint around it keeps growing.
Visibility Is What Lets You Move Fast
Too often, oversight gets treated as a tax you pay for speed. The reality is closer to the opposite. In our survey, 76% of organizations said their current governance was slowing AI deployment, and 93% said better governance would enable them to move faster over time. Both are true. The companies pulling ahead built visibility into their AI operations early, and that’s precisely what allows them to run as much of it as they do.
So when the next report lands and the incident count is up, slow down before you call it a failure. In an enterprise environment increasingly run by AI agents, the organizations that can see what’s happening hold the advantage. The ones operating on a clean dashboard they can’t fully trust are the ones I’d worry about.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
