There’s a harsh truth most cybersecurity professionals know but rarely admit: the majority of our metrics are little more than theater. For years, organizations have celebrated patch rates, compliance certificates and clean audit checklists as evidence of their security posture. But the Target breach and countless others taught us that being “compliant” is not the same as being secure.
It’s time we confront a hard reality—impressive numbers can be dangerously misleading, and in the world of cyber, the illusion of progress is sometimes worse than no progress at all.
Defining Vanity Metrics and the Compliance Trap
So what is a vanity metric in cybersecurity?
I asked Jason Fruge, CISO in resident at XM Cyber to define it. He explained, “A vanity metric looks like it displays something—it looks like it’s tangible and shows progress—but really it doesn’t have any real value.”
The classic example is the “95% of high vulnerabilities patched in 30 days” badge. Sounds fantastic, until you realize the 5% left unpatched are likely the most critical. Percentages are relative; patching 95% of ten vulnerabilities is not the same as 95% of ten thousand. Leadership often interprets these activity metrics as risk reduction, but that’s a leap of logic—a confusion between effort and impact.
The compliance trap is even more insidious. Compliance standards are built to be broadly applicable, but broadness is their weakness. Passing PCI or HIPAA checks may satisfy auditors but doesn’t guarantee security for the unique contours of your business. I have emphasized for years that compliance is not the pinnacle of security. On the contrary, it is the “minimum payment” or lowest common denominator—it keeps you legal, but it doesn’t necessarily make you safe.
The Shift from Vulnerability Management to Exposure Management
Fruge described how traditional metrics fail because they measure activity in silos—patches applied, devices scanned, boxes checked. They don’t capture how attackers chain exposures across domains. Exposure management, and frameworks like Continuous Threat Exposure Management, break these silos by mapping how vulnerabilities, identities, assets and network exposures combine to create real attack paths.
Picture the Cybersecurity Defense Matrix, a model that overlays NIST’s functions (Identify, Protect, Detect, Respond, Recover) across asset types—devices, applications, identities, data. Siloed teams focus on their own column, missing how a low-priority device vulnerability and a stale admin credential together open a path for lateral movement.
According to Fruge, exposure management tears down these walls, showing you not just the “what,” but the “so what.”
The Role—and Limitations—of Tools and Frameworks
Can you “buy” CTEM or exposure management? Not really.
As Fruge notes, “Gartner is absolutely adamant—CTEM is not a tool.” Technology helps, but without the right culture and processes, the shiniest dashboard will just become the next set of vanity metrics. Digital twins, for instance, can simulate attack paths and overlay business context—showing not just where the exposures are, but which matter most to critical systems. But if all you report is “number of exposures found,” you’re back to square one.
Choke Points, Blast Radius and Metrics That Matter
So what should we measure? Fruge points to “choke points”—critical nodes where multiple exposures converge, creating a large “blast radius” if compromised. Fixing one choke point may eliminate dozens of potential attack paths. Tracking how many choke points you identify and remediate—and how quickly you do it—directly measures risk reduction, not just activity.
Fruge believes these are metrics that move the needle.
Organizational Dysfunction: The Silent Exposure
But here’s the dirty secret: the biggest exposure isn’t always technical. “Organizational dysfunction,” Fruge observes, is often the largest and least acknowledged risk. Fragmented teams, siloed data and poor communication create blind spots attackers can exploit. True risk reduction requires cross-team, cross-domain collaboration and metrics that reflect the whole—not just the sum of the parts.
Changing the Metrics Mindset: Practical Steps for Security Leaders
If you’re a CISO or security leader, here’s where to start:
- Stop Borrowing IT KPIs: Uptime, ticket closure rates and patch percentages don’t measure risk.
- Educate Upwards: Boards and executives need context-rich metrics—risk to revenue, customer trust, or core operations—not technical statistics.
- Emphasize Business Context: Tie exposures to what the business values—payment systems, IP, customer data.
- Prioritize Continuous Correlation: Don’t settle for periodic snapshots. Exposure management is about ongoing vigilance and dynamic measurement.
Letting Go of Old Myths
The cybersecurity industry is addicted to numbers that look good on a dashboard but mean little in the real world. As threats grow more sophisticated and interconnected, so too must the approach to measuring success. It’s time to weed out the metrics that don’t matter and double down on those that do: measurable, business-impactful reductions in exposure.
If we want real security progress—not just the appearance of it—cyber leaders must lead the charge. The organizations that move beyond vanity metrics—measuring what truly matters, ruthlessly prioritizing real risk reduction and relentlessly translating security into business terms—won’t just be more secure; they’ll define cybersecurity success.
