How do you measure IT security? These metrics offer a standardized way.

Public/private group updates and expands its consensus tools for quantifying elements of security

set of consensus metrics for measuring IT security status in a standardized way has been updated and expanded by the Center for Internet Security.

The non-profit group, whose membership includes federal, state and local government agencies along with companies and academic organizations, released the new guidelines on Monday, 18 months after the original version was released in May 2009. CIS describes the metrics as “unambiguous definitions for security professionals to measure some of the most important aspects of the information security status,” and they are available for free download.

The goal is to give an organization the ability to repeatedly evaluate security in a standardized way, allowing it to identify trends, understand the impact of activities and make responses to improve the security status.

Measuring IT security in a meaningful way is challenging because of the difficulty of quantifying negatives, such as security breaches and impacts that do not occur. Standardized metrics can help by allowing comparisons between organizations and within an organization over time.

The metrics were decided upon by a group of more than 150 security experts from the commercial, government and academic worlds. Federal members of the CIS include the Federal Aviation Administration, Federal Reserve Board, Library of Congress, NASA, the National Institute of Standards and Technology, National Institutes of Health, Census Bureau, Patent and Trademark Office, the Energy and Interior deepartments, and one anonymous agency.


Related coverage:

NIST suggests areas for further security metrics research


In addition to some updates to make the original 20 metrics more useful, eight new metrics have been added in the latest release. They address basic questions about the outcomes of activities in seven business functions:

  • Incident management: How well do we detect, accurately identify, handle and recover from security incidents?
  • Vulnerability management: How well do we manage the exposure of the organization to vulnerabilities by identifying and mitigating known vulnerabilities?
  • Patch management: How well are we able to maintain the patch state of our systems?
  • Configuration management: What is the configuration state of the systems in the organization?
  • Change management: How do changes to system configuration affect the security of the organization?
  • Application security: Can we rely on the security model of business applications to operate as intended?
  • Financial metrics: What is the level and purpose of spending on information security?

Most new metrics were added in the areas of incident and configuration management, said CIS chief security officer Steven Piliero.

“One of the key goals of incident management is to reduce the impact of security incidents over time,” he said. Providing a common taxonomy for scoring impact gives a way to see if security controls are having an effect. Many organizations are already using some type of automated configuration assessment tool. The new metrics help standardize reporting across platforms.

CIS also has released a Quick Start Guide for its metrics to give users a basic understanding of them and help with implementation. Piliero said that organizations should phase in a metrics program.

“We recommend they not try to implement all of the metrics at once but pick a few key ones to begin with,” Piliero said.

He suggested starting with financial measurements and incident and configuration management, because most organizations already have some programs for assessing and reporting in these areas. “There is not a lot of heavy lifting” in implementing these metrics, he said.

Piliero said government involvement in the program is crucial.

“The federal government represents a major base of users,” he said, and helps to balance representation from the private sector and from other countries.

CIS also is developing electronic schemas for sharing metric data and also is working to align its metrics with the Consensus Audit Guidelines, the top 20 critical security controls agreed on by private and government security experts. The goal is to produce a standardized, actionable reporting format for the consensus guidelines.

 The Consensus Audit Guidelines were developed by a consortium headed by the SANS Institute and the Center for Strategic and International Studies, along with a number of federal agencies including the NSA, US-CERT, several Defense Department agencies, the Energy Department nuclear laboratories and the State Department. The guidelines have gotten considerable traction in government, including the State Department, which has used them as the basis for a real-time security monitoring program that has been held up as a model for improving government IT security.

 Piliero said work on the alignment is expected to continue through next year.

About the Author

William Jackson is freelance writer and the author of the CyberEye blog.

Reader Comments

Thu, Nov 4, 2010 Jeffrey A. Williams

The problem with this schema of metrics is that it has one huge ommision, that being there was no consideration given to Network Security considerations.

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above