Security Metrics as a Process

Recently, I was presented with an interesting challenge within my organization. Quite honestly, the challenge is not anything new or specific to this organization; it is a systematic problem within the Information Security function. The concept of Information Security metrics is, in my opinion, largely based on snake oil sales. Unfortunately, it perpetuates as a nebulous science, complicated further by inconsistency and contention on resources that fail to see the value of the art.

If you had to read that last sentence multiple times to catch the meaning, well then you are at the same wavelength that our decision makers are at when it comes to the metrics that we define as a horizontal function. The statement is based on opinion, contains fancy words, and yet somehow dos not address the challenge in quantitative terms.

This is the crux of my challenge, and one that I hope can translate into my dissertation in organizational management. Metrics can be defined by nearly everyone; effective metrics cannot. The practice of arriving at effective security metrics will take many resources: human, financial, temporal, and technological.

Like the Information Security function itself, the effective metrics process is a process, not a product. Borrowing from Andrew Jaquith's book Security Metrics: Replacing Fear, Uncertainty, and Doubt, I believe he is correct when he defines the criteria for an effective metric:

• Consistently Measured


• Cheap to Gather


• Expressed as a Cardinal Number or Percentage


• Expressed using at least One Unit of Measure


• Contexually specific

Surprisingly to me at this point, is the noticable lack of material on the subject of Information Security Metrics. Outside of Andrew Jaquith's book and the works of ISO 27004, which is yet to be published, I have yet to find good material on the topic.

What I would be interested to see, is what practioners in the field use to measure their effectiveness in Information Security as a process.