Please wait while the page is being loaded Skip this advertisement >
Saturday | 22 November, 2008
CIO
A Few Good Metrics
Mention metrics to a CIO or infosecurity executive and immediately their thoughts may well turn to sigmas, standard deviations and, probably, probability. To many, metrics equals statistics.
Scott Berinato 08 September, 2005 11:15:21

Information security metrics don't have to rely on heavy-duty maths to be effective, but they also don't have to be dumbed down to red, yellow, green. Here are six smart measurements - and effective ways to present them.

Metrics have a bad rep. Mention metrics to a CIO or infosecurity executive and immediately their thoughts may well turn to sigmas, standard deviations and, probably, probability. To many, metrics equals statistics.

There's no denying that proven economic principles can - and should - be applied to information security investments. At the same time, a bumper crop of valuable metrics exist that don't require classes on Nobel Prize-winning theories or a working knowledge of the Greek alphabet. You've actually already sowed the seeds of these less dense but equally valuable metrics. They're sitting in your log files, on your network, in the brains of your business unit managers, just waiting to be harvested. You won't need computational prowess to exploit this crop's value, just some legwork and - this is key - the most effective presentation tools.

Here we discuss six such metrics, along with some ways to present them visually, as imagined by Andrew Jaquith. Jaquith is a co-founder of the consultancy @stake (which was bought in 2004 by Symantec) and a protege of infosecurity guru Dan Geer. At @stake he invented a popular analytic methodology that is used to evaluate a client's risk in its application portfolio. He's since left Symantec and joined The Yankee Group. More recently he started Securitymetrics.org, a Web site open to all security professionals for sharing, contributing and advancing the use of metrics in information security. He's also writing a book, Security Metrics, due out later this year.

Jaquith has sharp, sometimes contrarian opinions on what makes a good metric and what makes for good presentation of metrics. For example, he thinks annual loss expectancy (ALE), a tool used to measure potential losses against probability of losses occurring over time, is useless, because in infosecurity, the L and the E in ALE are wild guesses. Quoting Geer, he says: "The numbers are too poor even to lie with."

He also thinks CIOs and CISOs are too apt to dumb down visual representations of metrics for their executive counterparts, mistaking simplicity for clarity. He holds a particular grudge against the overuse of the "red, yellow, green" representation of metrics to signify high, medium and low numbers. "A CEO's favourite visualization of metrics is a stock chart, a 2.5cm square that contains a month's worth of opening and closing prices, a trend line and several other indicators. Maybe 50 or more data points right there. Don't tell me they can't handle complex data. They can, as long as it's presented well."

By no means does Jaquith (or CIO for that matter) think these five metrics are the final word on infosecurity. Quite the contrary, they're a starting point, relatively easy to ascertain and hopefully smart enough to get CIOs thinking about finding other metrics like these, out in the vast fields of data, waiting to be reaped.

Related Features
  • +

    Ticked Off at Tick the Box Mentality 04 February, 2008 13:01:15

    Does your executive search firm know the difference between an MIS manager and a CIO, and if it does, can it explain that difference to its corporate clients?
    Does your executive search firm know its MIS managers from its elbow? Does it even know the difference between an MIS manager and a CIO, and if it does, can it explain that difference to its corporate clients?
Additional Resources
Executive Guides
Whitepapers
Newsletter Subscription
Sign up for our CIO newsletters!
RSS Feeds
Featured Whitepaper Sponsors
Market Place
 
Featured Whitepapers

Smart SOA World Tour

Discover how SOA can create smarter outcomes for your business.

Attend and learn:

  • How SOA is helping leading companies to become more agile
  • Where you should be applying SOA processes in your company
  • The top SOA implementation mistakes to avoid

Click here for more information.
  • +

    CIO Live Podcast #79: Brent D Taylor, author of The Outsider's Edge: The Making of Self-Made Billionaires Part II 05 October, 2007 06:00:00

    For his new book, The Outsider's Edge: The Making of Self-Made Billionaires, social researcher Brent D Taylor spent four years of intensive research investigating the psychological make-up and backgrounds of some of the world's richest men and women, including IT luminaries Bill Gates, Larry Ellison and Steve Jobs. Taylor discovered that, despite working in different industries and coming from different upbringings, they all have one thing in common -- they are all outsiders.
  • +

    CIO Live Podcast #78: Brent D Taylor, author of The Outsider's Edge: The Making of Self-Made Billionaires 28 September, 2007 17:34:25

    For his new book, The Outsider's Edge: The Making of Self-Made Billionaires, social researcher Brent D Taylor spent four years of intensive research investigating the psychological make-up and backgrounds of some of the world's richest men and women, including IT luminaries Bill Gates, Larry Ellison and Steve Jobs. Taylor discovered that, despite working in different industries and coming from different upbringings, they all have one thing in common -- they are all outsiders.
  • +

    CIO Live Podcast #77: Panasonic Speeds Up Trans-Pacific File Transfers, Part III 21 September, 2007 07:00:00

    Part three in our three-part special report from CIO's sister publication Network World in the US, as Paul Desmond reports from the Network World IT Roadmap Conference in Santa Clara, California. With development teams in the US and Japan, Panasonic needed a more efficient way to move very large files between the two locations. Iben Rodriguez, IT consultant for Panasonic Research and Development, explains how a storage-area network and virtual server technology helped speed up WAN performance.
  • +

    CIO Live Podcast #76: Panasonic Speeds Up Trans-Pacific File Transfers, Part II 14 September, 2007 07:00:00

    Part two in our three-part special report from CIO's sister publication Network World in the US, as Paul Desmond reports from the Network World IT Roadmap Conference in Santa Clara, California. With development teams in the US and Japan, Panasonic needed a more efficient way to move very large files between the two locations. Iben Rodriguez, IT consultant for Panasonic Research and Development, explains how a storage-area network and virtual server technology helped speed up WAN performance.
  • +

    CIO Live Podcast #75: Panasonic Speeds Up Trans-Pacific File Transfers, Part I 07 September, 2007 07:00:05

    Part one in our three-part special report from CIO's sister publication Network World in the US, as Paul Desmond reports from the Network World IT Roadmap Conference in Santa Clara, California. With development teams in the US and Japan, Panasonic needed a more efficient way to move very large files between the two locations. Iben Rodriguez, IT consultant for Panasonic Research and Development, explains how a storage-area network and virtual server technology helped speed up WAN performance.
  • +

    Chris Hoff on Virtualization and Cloud Computing 20 November, 2008 10:55:00

    Chris Hoff, chief security architect for the systems and technology division at Unisys and an advisor on the Skybox Security customer advisory board, is one of the biggest critics of virtualization security out there. Not because it isn't important - but rather because it is vital and needs to mature rapidly.
  • +

    Cybersecurity is focus of new start-up incubator 20 November, 2008 07:19:00

    Texas uni announces the Institute for Cyber Security.
    The University of Texas at San Antonio Tuesday announced a technology incubator aimed at fostering IT security-based start-ups within the state.
  • +

    Dilip Sarangan on Physical Security M&A 20 November, 2008 11:18:00

    Dilip Sarangan tracks physical security companies for Frost & Sullivan. He expects the industry's "need to have" products to weather the economic storm well, with the big players (now including IBM and Cisco) looking for value-priced acquisitions.
  • +

    International Challenges in PCI Security 20 November, 2008 09:15:00

    In a country that's seen many regulatory compliance challenges this decade, the headaches of PCI security tend to be analyzed from a largely American perspective.
  • +

    PCI council sharpens oversight of security auditors 19 November, 2008 10:53:00

    Quality assurance plan targets security assessors and scanning vendors
    The PCI Security Standards Council Monday unveiled a plan to sharpen oversight of the hundreds of security-service providers now authorized to evaluate merchant networks under the organization's Payment Card Industry data standards.
CIO Webcast Innovation #8 - What are the biggest roadblocks to IT's involvement in innovation at your company?
Watch the latest latest edition of CIO Innovation which is now available for download.
Watch the webcast
Sign up to the CIO Innovation update email


CIO Live Podcast #79: Brent D Taylor, author of The Outsider's Edge: The Making of Self-Made Billionaires Part II
Listen to the latest edition of CIO Live which is now available for download.
Listen to the podcast
Sign up to the CIO Live email
Whitepaper

Email Archiving 101—Customer Case Study

Join Lee Benjamin, a Microsoft Exchange MVP and Ryan Shipkowski, network administrator for Matthews, to discuss the process and ROI of implementing an email archiving solution, with emphasis on a case study from Matthews International.