OUR PROJECTS

EPRI DNP Security Interoperability Testing

The Distributed Network Protocol (DNP3) is the most widely used utility communications protocol in North America. Ensuring that DNP3 communications are secure is an important goal for the power industry. The previous DNP Security Testing and Evaluation product ensured that version 3.0 of the DNP Secure Authentication Specification was released, including a method for remotely changing cryptographic keys and a review by cryptographic experts. The next step in ensuring the successful deployment this technology is to develop procedures for testing vendor implementations.

TVA Bradley Sub

EnerNex supported TVA in developing a next-generation Substation Automation System (SAS) at its Bradley, Tennessee 500kV Substation, using IEC 61850, an IP-based, self-describing, object-oriented protocol suite implementing a common data model agreed upon by major substation vendors. EnerNex’ involvement on the project included strategic guidance during the engineering and integration phases of the project, creation of a System Specification Document, creation of test plans covering all phases of the system development and deployment, as well as on-site testing support.

Additionally, EnerNex assisted TVA in developing specifications for its next generation of substation gateway, Remote Terminal Unit (RTU) and standalone I/O equipment to support Supervisory Control And Data Acquisition (SCADA) applications associated with its transmission and generation resources. These specifications emphasize the use of open industry standards with IP based communications and include functional requirements covering reliability, communications, electrical, environmental, testing, maintenance, configuration management, and cyber security.

PacifiCorp Satellite

PacifiCorp is interested in exploring the use of non-conventional technologies to facilitate communications to substations and other facilities not served by more conventional means such as leased lines, fiber network, or microwave. Primarily, these communications are desired to access “”non-operational”" data in various devices. Such devices provide useful information related to the performance of PacifiCorp facilities but these devices or the data that is desired is not used to facilitate the real-time control and operation of the power system itself. Such devices include power quality monitors, fault recorders, digital relays, equipment status monitors, and other metering equipment.

Some of these devices (e.g. digital relays) are of course critical to the safe and reliable operation of the power system, but the data they contain after they operate falls into the non-operational category.

There are several means available today to provide connectivity to these devices given the explosion in communications technology and the resulting reduction in cost of the equipment and services that implement those technologies. These technologies are primarily related to providing TCP/IP protocol based connectivity. The technologies are both wired and wireless infrastructure related (e.g. WiFi 802.11a,b,g, DSL, ISDN, SONET ATM, etc.), application related (e.g. VPN, secure shell, web, etc.), and network management related (routers, switches, gateways, protocol converters, etc.).

Of these technologies, PacifiCorp has determined that some stations can be most cost effectively served by satellite based IP connectivity. To that end, EnerNex is proposes to evaluate the use of this technology for a few well defined applications by installing satellite receivers at three PacifiCorp substations. Satellite based TCP/IP communications technology has several unique characteristics that require careful review of the applications that will make use of it. These issues are primarily related to bandwidth, latency, and security and will be the core issues investigated during the project.

There are several means available today to provide connectivity to these devices given the explosion in communications technology and the resulting reduction in cost of the equipment and services that implement those technologies. These technologies are primarily related to providing TCP/IP protocol based connectivity. The technologies are both wired and wireless infrastructure related,application related,and network management related.

Of these technologies, PacifiCorp has determined that some stations can be most cost effectively served by satellite based IP connectivity. To that end, EnerNex proposes to evaluate the use of this technology for a few well defined applications by installing satellite receivers at three PacifiCorp substations. Satellite based TCP/IP communications technology has several unique characteristics that require careful review of the applications that will make use of it. These issues are primarily related to bandwidth, latency, and security and will be the core issues investigated during the project.

GE Client Loader Test Plan

This project resulted in EnerNex developing detailed test procedures for the GE IEC 61850 Loader software based on the test plan “NTEK-A027-0TP D400 IEC 61850 Client/DCA Test Procedures v7” previously developed for GE by EnerNex.

The test procedures covered testing of the Loader software and its associated GUI, but did not cover the following sections of the test plan:

DCA Features Testing
Protocol Testing
D400 Features Testing
Performance Testing

It should be noted that one goal of the sections listed above was to verify that the Loader had successfully created a configuration for the DCA. However, as requested by GE, the development of these procedures was not included in this project. Therefore testing of the Loader software in these procedures consisted of verifying that:

The behavior of the software was consistent with the functional specifications, e.g. It is possible to save a configuration.
The behavior of the software was consistent with itself, e.g. when a configuration is saved, the changes that were made are visible when the configuration is is reloaded.
Failure indications such as error messages occur only when intended.
Exceptions such as crashes, etc. do not occur.

Additionally, EnerNex developed a set of business rules for the configuration of the D400 based on the functional specifications and design documentation available for the Loader. These rules consist of text entries in a spreadsheet similar to the example already provided by GE. The purpose of the business rules shall be to define the filtering, processing and checking that must be performed by the Loader and Loader Web Services on the configuration information entered by the user.

GE – 61850

EnerNex developed a configuration tool for the IEC 61850 Data Processing Application (DPA) software running on the GE Energy D20/D25/D200 family of products and compatible with the latest version of the GE Energy Config ProTM configuration software. The software was developed as a Microsoft ExcelTM spreadsheet augmented with executable code. The project included the following tasks:

Task 1 – Functional Specification

EnerNex developed, reviewed with GE staff, and revised a functional specification for the tool according to GE Energy software documentation standards. . The functional specification lists the IEC 61850 configuration rules that will be checked by the tool.

Task 2 – Design Specification

EnerNex wrote, reviewed with GE staff, and revised a design specification for the tool according to GE Energy software documentation standards. Since this tool is a spreadsheet, the documentation consisted primarily of the following information:

Description of external interfaces, primarily the Comma-Separated Value (CSV) files
List of macros, their triggers and their functions
List of Visual Basic functions written, their triggers and parameters
Software libraries used in the design

Areva

The purpose of this project was to assist AREVA in the upgrade of their IED’s to be compatable with IEC 61850. Their product capabilities include:

Measurement of three-phase electrical system parameters
Computation of integral, derivative, and thermal demand characteristics
Recording of time-series data (oscillography, disturbance, and trending)
Time management (lRIG-B client, UCA client/server, backup local time source)
Binary status monitoring and control (opto input, relay output)

The instrument presently supports the protocols Modbus (RTU and TCP/IP), DNP3 (serial and TCP/IP and UDPIIP), and UCA2 (server, GOOSE publisher/subscriber).

Some members of the 70-series family are modular. This requires that the instrument configuration appear different depending upon the results of power-on self-discovery.

Present configurations allow for optionality of the analog/DSP section, optionality of the Ethernet module, and incorporation of 0 to 7 digital 110 cards (each with 4 outputs and 4/8116 inputs).

AREVA has become aware that the trade association, UCA International Users Group, endorses only IEC 61850, not UCA2. AREV A also recognizes that existing customers with UCA2 systems need an upgrade path for their systems which does not require conversion to 61850. For these reasons, AREVA wishes to add the 61850 series of standards to the existing 70-series instrument while continuing support for the existing protocols.

The IEC 61850 standards were derived from the EPRI-sponsored UCA2 projects and it inherits many ofUCA2′s characteristics. At the highest level, 61850 can be considered a remapping of the UCA2 objects and services. However, 61850 encompasses much more functionality than UCA2. It adds configuration management, continuous quality tests, conformance testing, and other concepts, as well as the configuration management component, specifically the Substation Configuration Files, or SCL.

EPRI Living Lab Tactical Support

In 2007, EPRI proposed to establish a laboratory to evaluate equipment in support of their Energy Efficiency Initiative. One objective of the lab is evaluate products that are intended to be integrated through emerging key open standars that have been identified in the IntelliGrid Architecture and in various user groups working toward interoperable systems for the industry.

EnerNex supported this effort by assisting in the tactical elements of technology. EnerNex worked with EPRI Staff and Hypertek to jointly develop the strategic elements of the testing laboratory. EnerNex’s role was to:

Develop a plan and strategic approach for EPRI’s Living Lab
Provide strategic input and direction into the specification, construction and management of the lab.

Deliverables included:

Technical Update document “Initial Setup and Commissioning of EPRI Living Laboratory”
Strategy papers and responses to inquiries

SWPA DOE DNP Consulting

This project resulted from a request from the Southwestn Power Administration (SWPA) to assist in the resolution of an ongoing performance issues with DNP3 Master software at SWPA. The effort involved both on-site and off-site consulting in close cooperation with the SWPA Lead IT, Technical Point of Contact, and COTR. The “Key Project Deliverables” from the SWPA “Statement of Work clause 1.5 was utilized. In addition, a final written report was provided.

The work consisted of observation of the system instabilities, as well as, conducting “experiments” on the development and test systems to attempt to narrow-down the possible root causes of problems. Detailed discussions with the SWPA IT Lead and Technical Lead were initiated and a general plan of action was developed. Source code for relevant portions of the system was provided for off-site analysis.

SWPA deliverable 1
This information gathered during the on-site visit was examined preliminary report on the present implementation was prepared Deliverable 1 will include recommendations, intended effects and potential impacts to the system along with enough detail that allowed SWPA to approve or reject any recommendation(s).

SWPA deliverable 2
Upon acceptance of the preliminary report, a second report was written detailing the Plan of Action & Milestones (POA&M) for implementing SWPA approved changes detailed in the first report.

SWPA deliverable 3
EnerNex then analyzed the system for conversion from polled to an event-driven system and prepare a POA&M for this effort Deliverable 3 will included recommendations, intended effects and potential impacts to the system along with enough detail that allows SWPA to approve or reject any recommendation(s).

SWPA deliverable 4
Upon acceptance of Deliverable 3, a formal test plan was created in order to determine whether the resulting system meets the needs of SWPA. EnerNex then performed the conversion in a developmental environment with strong assistance from the SWPA IT person. The final task was the generation of a brief report on the changes made to the system along with their intended effects. The report also detailed possible unintended impacts to the system.

Lemnos

This project address cyber security risk to critical infrastructure by evaluating performance and interoperability among control system environment network security products. The project will research, develop, test, and ultimately foster commercialization and energy community standards acceptance for security interoperability. The project will bridge directly to the broader industry through a community leadership project and an example transition to Schweitzer Engineering Laboratories line of products to drive the industry. In Phase I: Specifications and Methodologies, all aspects of utility communications security will be examined, analyzed, and decomposed to define the problem space. In Phase II: Build and Test, a specific subset of the problem space defined in Phase I will be selected and applicable reference designs, product prototypes, and testing tools developed. In Phase III: Exhibition, project focus will shift to communicating the results of Phase I and II to the utility industry.

Bitronics

The purpose of this project was to assist Bitronic’s with integration of a IEC 61850 protocol stack into their measurement unit product.

The work consisted of customizing the 61850 protocol stack to their unique system architecture. Main work items included the handling of the SCL configuration files in a manner common to the Alstom Grid products, integration of the event-driven architecture with stack components, object modeling issues, generation of conformance various Implementation Conformance Statements (PICS, MICS, TICS, PIXIT), content creation for device user manuals, and pre-conformance testing.

UCA Testing Report

UCAIug Work Includes:

Certificate reviews: verification of contents of the certificates provided by the testing organizations against UCA requirements.
Initial accreditation of testing laboratories: review contents of an entire test report (including thousands of detailed Ethernet frames) to ensure that testing organization is qualified to perform testing on behalf of UCAIug.
Annual testing laboratory re-accreditation: audit for continued laboratory competency.
Continual review of testing procedures: verify that actual testing by the testing organizations matches the intent of the tests.
Continual review of Quality Assurance Program and Laboratory Accreditation Procedure to ensure they still meet needs of UCAIug.
Continual review of individual test procedures suites for errors, omissions, and new edge cases in need of immediate resolution.
Write the requirements for certificate content, initial tester audit procedures, annual audit procedures.
Maintain UCAIug web site with respect to testing activities.
Lead UCAIug testing subcommittee meetings, write/post minutes.
Lead testing acceleration meetings in instances where normal IEC/UCA process would delay tested product introduction.
Work with UCA to adapt UCA organization from an oversight organization into a NIST TCC ITCA organization .
Direct other contractors in tester auditing activities.
Help coordinate completion of IEC test procedure documents.

Arbiter

Arbiter Systems manufactures a wide variety of equipment including GPS satellite-controlled time sources. The primary output of these sources is the 1 pulse-per-second signal, from which all other outputs are derived. Many of the clocks include an optional NTP server for use with Ethernet-based NTP/SNTP clients such as those incorporating IEC 61850. Because the NTP output is derived from the 1PPS output, it is known to be (much) less accurate than ± 1.0 microsecond. Preliminary testing by EnerNex personnel challenged the assertion of Arbiter that the NPT output met 1 milllisecond accuracy. Subsequent investigation by Arbiter uncovered a flaw in the clock firmware which caused an approximate 2.5 millisecond offset in the NTP output. Arbiter produced a firmware update which EnerNex tested to determine that the error was reduced below 500 microseconds.

IEC 61850 specifies a minimum IED synchronization accuracy of ± 1.0 millisecond (T1). In order to meet this requirement, the time source must be “much better” than ± 1.0 millisecond. Most customers expecting to meet IEC 61850 T1 requirements insist on NTP accuracies at least 3 to 5 times better than ± 1.0 millisecond (specifically, 200 to 330 microseconds). Verifying accuracies in this realm is challenging because it requires equipment capable of launching and receiving packets with an accuracy of around 40 microseconds. EnerNex now has this capability.

EnerNex provided the following services to Arbiter Systems to address the issues identified above:

Calibration type-test certification providing proof that the NTP server can meet the IEC 61850 T1 synchronization
Composition of accuracy text that customers can insert into procurement specifications
An analysis of the fixed offsets within the NTP server system (removal of this offset would dramatically increase the NTP accuracy)

TED 5000 Testing

EnerNex provides electric utilities and their device manufacturers with custom and repeatable test services via its Smart Grid Labs service offering. In this project EnerNex provided a series of tests for TXU’s TED-5000 home energy display. TXU has requested testing for three specific parts of the TED-5000 system. The first part is testing of an unspecified power line communications testing between the MTU and the gateway. The second part is ZigBee range, interference, and materials testing for gateway to IHD communications. The third part is to characterize system power measurement accuracy. For this project EnerNex specifically excluded device usability, software usability, software & platform testing, and any functional testing of the TED-5000 system beyond those specified by TXU.

Duke

EnerNex is currently engaged in a multiyear project with Duke Energy to evaluate Smart Grid and Advanced Metering Infrastructure Security. This engagement has included penetration testing of the field deployed devices including the meter, communication nodes, and web-based head-end system. The security testing evaluates cyber-physical weaknesses, communication, protocol, software and firmware vulnerabilities. EnerNex works closely with Duke to understand risk, impact and develop mitigation strategies. Testing includes various network architectures, manufacturers and models. As new versions of the hardware and software are released, EnerNex validates and reports that previous issues are corrected or improved and that new vulnerabilities are not introduced through an iterative process.

Cisco Lab Test

AHAM

The purpose of this project is to provide the Association of Home Appliance Manufacturers (AHAM) a technical study of communication standards and protocols as they relate to the development and interoperability of the Smart Grid,and what standards and protocols would be most effective for the appliance industry. EnerNex has previously helped many utilities conduct similar technology surveys for most Smart Grid domains.Particular relevance to smart appliance manufacturers is our ongoing work in meter or gateway served home area network integration. Balancing maximum premise network facing systems interoperability with security, long term deployment life cycles, and minimum future application requirements can be a challenge. The approach to the project began with the premise customer’s needs first, the situation’s needs second, and the technical requirements third will pay off over the long term. Appliance manufacturers should be armed with the right information to build a product now that can make customers happy with smart appliance features for the foreseeable future.

As detailed in the 2009 AHAM Smart Grid Whitepaper, a big part of the new need in appliance communications competencies is Smart Grid integration. Advanced Metering Infrastructure (AMI) meters and Internet connected premise gateways are two priority premise edge devices that will serve the customer premise with energy services and information. Various application standards under development such as the Smart Energy Profile 2.0 can address the needs of edge energy service layers. Since these are generally software standards, they are often held to be ‘just a firmware upgrade away’ with respect to platform hardware. Even still, they do not address all the market requirements that must also be placed on core electromechanical, physical, link, and network layer components selected for product integration. The purpose of this project is to aid AHAM in identification of priority Smart Grid communications technologies that are correct for the Appliance market.

The following deliverables were provided:
Deliverable 1: Smart Appliance Communications Score Card
EnerNex worked with AHAM to define the ‘scorecard’ for Smart Appliance Communications. It is assumed that standards basis and standardization potential take precedence in the analysis given AHAMs role in the present NIST effort, but technologies are rightly differentiated along many axes. The project team recommend selecting or expanding focuses and requirements that can be assured or verified in an interoperability test environment such as EnerNex’s family of Smart Grid Labs.

Deliverable 2: Survey of Wired and Wireless Smart Appliance Communications Technology
The project included gathering standards, marketing data, and other information sufficient to populate the score card according to the members guidance and with best of breed communications technologies that are potentially useable for Smart Appliance adoption. The main output was a survey and summary of each technology considered with details expanded for each item agreed upon in the score card.

Deliverable 3: AHAM Study Telecon Support
EnerNex facilitated the initial meeting on the study and subsequent telecons to assure member input was taken into account. Output from this deliverable were meeting minutes input to AHAM. EnerNex continued to provide weekly telecon logistics support for the duration of the project.

EPRI CIM Testing Procedures

WG 13′s IEC 61970 and WG 14′s IEC 61968 series of standards are intended to facilitate integration of various distributed software application systems supporting the operation and management of utility electrical networks. The standards specify the interfaces that an application should implement to exchange information with other applications and/or to access publicly available data in a standard way. The application interfaces describe the content and format of the data exchanged as well as the mechanism. For the development of new applications, the standards enable knowing what and how information is available for processing from other applications as well as what and how information is expected by other applications. For the integration of an existing application, the standards enable a single adapter to be supplied off the shelf for a given infrastructure Technology Platform independent of who developed the application.

The IEC CIM standards define requirements, integration architecture, and interfaces for the major utility applications.The CIM standards can be applied in any utility domain where a common model is needed to facilitate interoperability and plug and play compatibility between applications and systems independent of any particular implementation.

The objective of this work is to develop standardized reusable test procedures, best practices and abstract test cases for conformity testing and interoperability testing for the 61968 series of standards. The test procedures shall be in conformance with the NIST TCC IPRM and CPRM. The development of said procedures shall be in coordination with the various testing working groups in the UCA and WG 14 as well as NIST TCC. The resulting standardized CIM test procedures and test cases will be designed to enable testing in multiple labs at multiple arbitrary times throughout the year.

EPRI CIM Testing 2

Introduction & Background

The primary purpose for the development of the Common Information Model (CIM) and companion IEE Technical Committee (TC) 57 Working Group (WG) 13 Energy Management System API and WG 14 System Interfaces for Distribution Management standards is to:

Reduce the cost and time required to add new electric utility applications.
Protect the investment in existing electric utility applications by enabling interoperability between native CIM and non-CIM applications.
Enable electric utility applications to communicate using standardized extensible messages which are both forward and backwardly compatible.

WG 13′s IEC 61970 and WG 14′s IEe 61968 series of standards are intended to facilitate integration of various distributed software application systems supporting the operation and management of utility electrical networks. The standards specify the interfaces that an application should implement to exchange information with other applications and/or to access publicly available data in a standard way. The application interfaces describe the content and format of the data exchanged as well as the mechanism. For the development of new applications, the standards enable knowing what and how information is available for processing from other applications as well as what and how information is expected by other applications. For the integration of an existing application, the standards enable a single adapter to be supplied off the shelf for a given infrastructure Technology Platform independent of who developed the application.

The IEE ClM standards define requirements, integration architecture, and interfaces for the major utility applications including, but not limited to:

Energy-Distribution Management Systems
Transmission-Distribution Planning Systems
Work and Asset Management Systems
Outage Management Systems
Customer Information and Meter Data Management Systems
Geographic Information Systems

The CIM standards can be applied in any utility domain where a common model is needed to facilitate interoperability and plug and play compatibility between applications and systems independent of any particular implementation.

Objectives
The objective of this work is to test standardized reusable test procedures, best practices and abstract test cases for conformity testing and interoperability testing for the 61968 series of standards developed under a previous SOW.

PECO Ami System Functionality and Performance Test Service

EnerNex supplies PECO Energy with test requirements, design, implementation, and execution support for AMI systems and meter test under their ARRA funded AMI Deployment program. PECO is preparing to replace 600,000 of their existing 1.8 million AMR meters in Philadelphia and the surrounding area. In preparation, EnerNex has supported PECO in provision of testing and best practice guidance for the following areas in advanced metering:

Standards Guidance (Sampling, Accuracy, Functionality, Interoperability)
AMI System and Meter Understanding
Meter Program Design Support
Urban System Pilot Test
3rd Party Meter Comparisons
Security / Encryption Functional Testing
Meter and AMI Functional Testing
Test Fixture Requirements, Design, and Verfication

EnerNex has placed on-site personnel with PECO’s business units and meter shop to develop detailed test plans, set-up test software and hardware, execute tests, analyze results and prepare preliminary and final test reports. EnerNex helps PECO to identify test requirements and refine the proposed test protocols to ensure PECO business needs are satisfactorily met prior to the start of physical testing. EnerNex tests the Exelon AMI System and associated communication components according to standard where possible, and to best practice otherwise.

General workflow includes:

Identify tested business values (e.g. Act 129, internal docs, etc)
Identify test requirements
Create tests and expected outcomes
Test fixture design / build / identify
Map tests to physical test fixtures and metering platforms
Schedule and deployment of tests and resources
Execution and documentation of tests
Interpretation of test results

PacifiCorp, TVA, Southern Co.
Flicker Testing

This objective of this project was to assist Utilities in dealing with Flicker problems ensure the measurements they make are accurate and complete. The IEC Standard 61000-4-15 was designed to ensure that the power industry has a single methodology for evaluating voltage fluctuations that result in light flicker and other end use equipment impacts. This standardized measurement methodology provides a series of outputs that can be used to objectively reflect the instantaneous, short, and long term statistical levels of flicker at a point in the utility network.

In principal, all flicker meters built according to the standard should provide the same results when connected to the same source. However, the flicker meter specifications were originally written for an analog design, the acceptance test in the standard is not comprehensive and instrument characteristics and other attribute are not specified in the standard.

In 2005, it was documented that flicker meters from different manufactures give widely varying results when measuring the same signal. As a result EnerNex performed tests on a variety of Power Quality meters provided by PacifiCorp, TVA, and Southern Company, providing the following:

Comprehensive testing of commercial flicker meters according to the draft Cigre’ C4.1.01 test protocol
Detailed report of test findings on each individual instrument
Public report with summary findings
Report for each meter vendor who contributed hardware containing individual evaluation and recommendations
Report on the test protocol itself

ASAP-SG

In 2009, the utilities of the Open Smart Grid Subcommittee of the UCAIug collaborated with the DOE, NIST and other interested parties to form the ASAP for the Smart Grid (ASAP-SG). This project is a follow-on activity to ASAP, completed in 2008 by many of the same entities. The EnerNex Director of Communication and Information Systems Security is the Program Lead for this effort. ASAP-SG has two primary objectives: to utilize the AMI-SEC System Security Requirements to develop a baseline Smart Grid Security Specification, supplemented by tailored Risk Assessments and Security Profiles for individual smart grid applications such as AMI, Distribution Automation, Outage Management, Substation Automation, etc.; and to improve the ability for users to approach, understand, and implement the guidance for securing smart grid systems. ASAP-SG will provide all utilities uniform access to a greater body of knowledge and expertise than would otherwise be available to any one utility. All contributing utilities will have transparency and input opportunities into the ongoing work efforts and access to the project artifacts through an access-controlled web site and regular meetings. The project will also contribute work to other collaborative and/or standards bodies to facilitate and accelerate community security efforts.

PacifiCorp Flicker Analysis Study

Erich Gunther led this study of flicker undertaken by PacifiCorp in response to customer complaints about lamp flicker. These complaints are geographically diverse, indicating a transmission-propagated cause. PacifiCorp conducted a series of measurements, which validated the customer complaints. These measurements also indicated that a large steel mill with an arc furnace and a Static VAR Compensator (SVC) in the area could be the primary cause. There was also the possibility of another customer facility contributing to the problem. PacifiCorp retained Erich to develop an accurate electrical model of the northern Utah area that can predict IEC flicker and harmonic distortion at PacifiCorp’s major load serving substations in the northern Utah area. Once developed the model helped in providing an assessment of the relative contributions of the sources of the flicker. EnerNex staff ran cases with this model to test various solutions to mitigate flicker to acceptable levels.