In the world of product management—especially in domains like security, abuse prevention, and fraud—success is often invisible. The best outcome is when nothing bad happens. But how do you measure progress when your goal is to prevent harm that ideally never occurs? Traditional metrics like the number of attacks prevented or detected can be misleading. Instead, a growing movement in security and risk management is advocating for “damage” as the ultimate metric.
Why Prevention and Detection Aren’t Enough
Most teams track prevention (how many bad things you stop) and detection (how many bad things you find). But these metrics can pull teams in opposite directions: prevention teams want the numbers to go down, detection teams want them to go up. More importantly, neither metric captures the true impact of what slips through the cracks. You might block 99% of attacks, but the 1% that gets through could cause catastrophic damage—like service outages, financial loss, or reputational harm. Conversely, thousands of minor attacks might have little real impact.
Measuring Damage: The Real Impact
Damage is a more meaningful metric because it focuses on the intensity and consequences of attacks, not just their volume. For example, a single account compromise could lead to doxing, financial theft, or even threats to physical safety. On a company level, damage is often measured in dollar value, but it can also include loss of reputation, regulatory penalties, or even the ability to continue doing business.
Subjectivity and Complexity
Damage isn’t always easy to quantify. It’s subjective and can vary depending on context. For instance, a fake YouTube view might seem harmless, but if it leads to a chain of events that undermines trust or enables fraud, the real damage could be much greater. Companies must consider not only direct harm to their own platforms but also indirect harm to third parties and the broader ecosystem.
Strategies for Reducing Damage
To effectively reduce damage, organizations should:
- Identify and prioritize the types of harm that matter most—financial, reputational, operational, or regulatory.
- Empower cross-functional teams to address damage at every stage of the attacker journey, not just at the point of entry.
- Deploy targeted interventions like blocking, throttling, suspending accounts, and monitoring attacker ecosystems, all while minimizing impact on legitimate users.
- Continuously revise damage calculations as new threats emerge and attackers adapt.
Communicating Damage to Leadership
Bringing the concept of damage into the boardroom requires clear examples and stories. Regulatory fines, high-profile breaches, and real-world incidents can help leadership understand the stakes and motivate investment in security.
Getting Started with Damage Metrics
For organizations new to this approach, start by analyzing past incidents: where did harm occur, what was the cost, and how could it have been prevented? Use these examples to advocate for prioritizing damage reduction and to guide future investments.
Conclusion
Shifting your focus from prevention and detection to damage can unify teams, clarify priorities, and maximize impact. While measuring damage is complex and often subjective, it’s the metric that best reflects the real-world consequences of security failures—and the value of getting security right.
This article is AI-generated based on the transcript of the episode #4: Damage as the Ultimate Metric in Security Product Management of the Inverted Podcast.
Leave a Reply