Digital Twin for Cyber Testing Michael J. OConnor Chief - - PDF document

IT2EC 2020 IT2EC Extended Abstract Template Presentation/Panel

Digital Twin for Cyber Testing

Michael J. O’Connor Chief Technologist, Trideum, Huntsville, United States

Abstract — This paper addresses both the Digital Twin Approach and Synthetic Environments topics for performing cyber testing on weapons systems. The use of digital twins is a critical part of cyber testing because this type of testing cannot be performed on operational systems. Weapon systems have cyber vulnerabilities in addition to the ones found

• n IP networks. These vulnerabilities can only be found by creating a synthetic wrap-around to fully stimulate the

system with an operationally relevant environment. The use of the cyber table top process to implement a digital twin and synthetic environments as part of weapons testing is described. The methodology used to apply the concepts of digital twin and synthetic environments along with lessons learned in the performance to cyber testing of weapons systems will be presented.

1 Introduction

Performing cyber testing on complex systems presents a number of challenges. One of the biggest is access to the systems to perform potentially destructive cyber testing. Another issue is providing the operationally relevant wrap around environment to the system under test (SUT) without compromising any of the systems with a cyber

• threat. One approach for this is to create a digital twin of

the system. This is the only approach for single copy national systems, but is also useful for weapon systems.

2 Approach

The goal is to develop a digital twin that can be tested for cyber vulnerabilities in an operationally relevant

• environment. This approach allows potentially destructive

cyber testing without damaging an operational system or expense weapon system. There are three parts to this approach:  Perform the Cyber Table Top  Develop the Digital Twin  Develop the Operational Wraparound This approach creates an operationally relevant test environment for the system under test. This will allow the performance of cyber testing in an operationally realistic environment without exposing an operational system to risk. 2.1 Cyber Table Top The development of a digital twin begins with a Cyber Table Top (CTT). CTT is a process that brings operators, developers, and threat teams together to review the SUT for likely cyber threat vectors. The threat vectors identified in the CTT informs the development of the digital twin. While it preferable to create a complete hardware and software digital twin, this is not always possible due to availability or cost reasons. The CTT results can be used to determine which parts of the system have to be implemented with the real hardware and software and which parts can be emulated. The threat teams propose potential threat vectors to the

• system. This is based on the threat teams experience and

their review of the system. The developers respond to the by assessing the likely hood of the system being vulnerable to the potential threats. If the developers believe the system may be vulnerable to the threat, they determine what the effect on the system from the threat would be. Finally, the operators evaluate impact on operations of the threats effect on the system. The operators may determine that the impact to operations is minimum because there are redundant systems in place or there are existing workarounds to the impact. They may also determine the impact is significant because the threat effect would prevent their mission. The impact may also be between these two extremes. All of the participants of the CTT are involved in the

• discussion. The goal is to reach consensus of the CTT

participants on the threats and the impact on the mission of the system. A risk matrix is produced for each of the threats which are determined to have a mission impact. The research for the CTT and the discussions will also establish the requirements for an operationally realistic test

• environment. The developers and operators will document

the inputs and outputs of the system as part of the process. The focus will be on the interfaces which are vulnerable to the validated threats. 2.2 Development of the Digital Twin It seems obvious the best test environment would always include the actual system in an operational environment and this is true for certain types of tests. Some tests cannot be performed on the actual system for human safety reasons or the inability to create the operational

IT2EC 2020 IT2EC Extended Abstract Template Presentation/Panel

• environment. Performing cyber on an aircraft in-flight is

not possible because of the risk to the pilot. Creating all

• f the live communications links for a command and

control (C2) system is not practical. In these cases, a digital twin becomes the solution to testing. For a digital twin to be a solution its development must be cost efficient and developable in a limited timeline. If the cost for the digital twin becomes too high, it may be more cost efficient to risk destroying an actual system. If the development takes too long, it will not be ready for testing in time to support the deployment or sustainment of the system under test. The results of the CTT are critical in driving the development of the digital twin. The scope for developing a digital twin varies greatly based on the nature of the system under test. A large enterprise Information Technology (IT) system could be composed of hundreds of networking devices and

• computers. A vehicle will have many fewer networking

devices and computers, but large number of computer controlled mechanical systems. The results of the CTT will determine the level of fidelity of each of the components in the digital twin. In a large enterprise IT system, a potential threat vector identified in the CTT may depend on the hardware of a particular router. It may be cheaper to procure this router for the digital twin than not try to virtualize the hardware to the level required for the threat vector to work. If the threat vector destroys the router instead of just making it

• perable creating a virtualization the can be reset for each

test may be the correct decision. If a threat vector only depends on the software of the router, then solution could be to just run the router software on commodity hardware as part of the digital twin. For vehicle systems the results of the CTT are even more critical in developing the digital twin. For IT systems most

• f the components are readily available the simulation is

mostly message based. Vehicles have many more systems and types of interfaces. For example, how much the drive system of the vehicle needs to be in the digital twin and to what fidelity. The risk matrixes from the CTT will drive these decisions. It is critical that digital twin not only provide the threat surfaces identified in the CTT, but the systems that will be impacted. The results from the CTT allows the developers of the digital twin to focus on the critical aspects of the system and the required fidelity for the components. This results in a cost effective digital twin that meets of the cyber test requirements. 2.3 Development Operational Wraparound The next step in the development of the digital twin is to provide all of the inputs to the SUT and have consumers for all the outputs. The CTT helps define the required inputs and outputs to create the operationally realistic

• environment. Modeling and simulation is used to create

the synthetic environment to wrap the system under test in. While creating the inputs is obvious, the need to consume the outputs may be less so. It is critical to have systems consume the outputs so that any mission impacts can be determined based on the cyber threat. The CTT identified all of the significant inputs required to provide the operationally relevant environment for the cyber threat vectors. These should be reviewed and simulations found to provide the necessary inputs. Sometimes additional inputs will be required for the system to preform correctly. Sometimes data replay can be used to provide the environment, however this does not provide flexibility in creating scenarios. Providing systems to consume the outputs of the SUT is also critical to providing the operationally relevant

• environment. In some cases, the SUT will not create
• utputs unless there is a consumer of the data. The data

consumers are also required because the mission impact of the cyber threat may not be apparent in the SUT, and will

• nly be shown in consumers of the SUT data.

3 Testing

Once the digital twin of the SUT and the operational wraparound has been developed, cyber testing can be

• conducted. The testing may proceed as the testing would

with the actual system. Test plans should be developed to provide the correct conditions for the cyber threat vectors that were developed in the CTT. The exact number of tests will depend on the threat vectors. As with all testing data collection is critical to cyber

• testing. Systems should be in place to collect data

provided to the SUT as part of the operational wrap-around and to collect all of the data produced by the SUT. Based

• n the type of threat vectors, additional collection methods

maybe required to collect threat actor actions.

4 Conclusions

The goal of using a digital twin for cyber testing is to determine the mission impacts that result from the cyber

• threat. Providing the full operational wrap-around to the

SUT will show the mission impacts to the cyber threats. This is critical to provide actionable information to the system owners to address the impacts of the cyber threat. Experience in applying the digital twin approach with synthetic environments has provided lessons learned in cyber weapon testing. These include the importance of the CCT, selection of components for the digital twin, and how to provide the synthetic environment wrap around.

Author Biography

Michael J. O’Connor is Chief Technologist at Trideum Corporation with more than 25 years’ experience in Modeling and Simulation (M&S). He is currently the Chairman of the SISO Executive Committee. He has

IT2EC 2020 IT2EC Extended Abstract Template Presentation/Panel served as the chair of the I/ITSEC Simulation Subcommittee and the I/ITSEC Training Subcommittee.