A Cyber Security Metrics Tool for Utilities

Mark McGranaghan, VP, Distribution & Utilization, EPRI

As cyber threats grow and become more sophisticated, utilities must constantly evaluate and improve the effectiveness of security programs. Moreover, utilities must manage their security programs across both traditional information technology (IT) and operational technology (OT) systems.

Utilities must also be able to quantify the investments and technologies needed to satisfy rate case requirements. Determining a utility’s existing cyber security posture is a necessary first step. Lord Kelvin famously observed, “When you can measure what you are speaking about, and express it in numbers, you know something about it.”

As part of the Electric Power Research Institute’s (EPRI) research program in Cyber Security, we initiated a collaborative effort with the Edison Electric Institute, the American Public Power Association, the National Rural Electric Cooperative Association, the Utilities Telecom Council, and the SANS Institute to examine security metrics for the electric sector. In 2015 the collaboration created a security metrics methodology and a framework for creating security metrics. In 2016 the group revised the methodology and developed specific metrics for utilities to use as a starting point in evaluating their own posture and path forward.

A Practical Methodology for Cyber Security Metrics Development

EPRI’s research approach incorporated five common-sense rules to our metrics development:

• Utility cyber security metrics must be based on quantitative and repeatable data
• Metrics must be independent of compliance to mandatory standards
• Metrics must allow for tailoring across the utility’s business units, functions, and ownership structures
• Metrics must take into account difference between IT and OT architectures
• Metrics must be able to clearly communicate the utility’s state of cyber security to different stakeholders

EPRI’s approach, as shown in the metrics “pyramid” (Figure 1) organizes data points, then rolls them up and assigns a weight of importance to either an operational, tactical, or strategic metric. The resulting tiers of data will help a broad range of utility stakeholders gain improved knowledge about cyber security postures and thus inform decision-making about policies, investments, and action plans.

"Determining a utility’s existing cyber security posture is a necessary first step"

More than 100 data points provide the quantitative foundation for the metrics, consisting of various operational statistics collected from different points in utility operations. The availability and quality of these data are important factors in metrics calculations.

Operational metrics measure real-time, day-to-day operations such as logs, rule sets, and signatures. Tactical metrics address programmatic health and progress in the organization. Strategic metrics measure corporate risk and alignment of the metrics to the direction of the business.

A Cyber Security Scorecard for Utilities

Each succeeding layer of metrics is based on rolling up the lower level metrics to the higher level ones. As shown in the figure, the top-level, three strategic metrics are calculated from 11 tactical metrics; and each tactical metric is calculated by summarizing relevant operational metrics. As data points shift, the impacts are reflected in metrics calculations and scorecards.

A Path Forward

As a relatively new field, security metrics is not as mature or robust as metrics in finance, reliability operations, or safety. However, EPRI’s collaborative research and practical methodology offers an optimal, standardized and complementary approach utilities can use to evaluate their own postures and resulting action plans.