IT 2 EC 2020 Cyber Training Architecture, Enabling Digital Twin - - PDF document

IT2EC 2020 IT2EC Extended Abstract Template Presentation/Panel

IT2EC 2020 – Cyber Training Architecture, Enabling Digital Twin Environments

Amit Kapadia1, Rick Osborne2, Brian Vermillion3

1Chief Engineer, U.S. Army PEO STRI, Orlando, United States 2 Simulation Engineer, The MITRE Corporation, Orlando, United States 3 Simulation Engineer, The MITRE Corporation, Orlando, United States

Abstract — In 2019, The U.S. Army Program Executive Officer for Simulation, Training and Instrumentation (PEO STRI) released an initial cyberspace operations training platform prototype called the Persistent Cyber Training Environment (PCTE). The PCTE platform includes tools to rapidly create ‘Digital Twins’ that replicate cyberspace

• perational environments in a virtualized platform for the Cyber Mission Force (CMF) to execute realistic training and

mission rehearsals. PCTE is laying the groundwork for these virtualized assets to connect to real world physical security assets such as industrial controls systems (ICS) that are otherwise not practical to emulate. To address this challenge, PEO STRI and the larger DoD cyber training community are utilizing an evolutionary architecture to rapidly integrate PCTE with real world physical security assets. This paper introduces evolutionary architecture and discusses the cyber training community’s approach for evolving a cyber training architecture over time while simultaneously delivering capability to the CMF.

1 Introduction

In 2019, The U.S. Army Program Executive Officer for Simulation, Training and Instrumentation (PEO STRI) released an initial cyberspace operations training platform prototype called the Persistent Cyber Training Environment (PCTE). The PCTE mission is to solve significant U.S. Department of Defense (DoD) gaps— specifically, the capability to effectively plan, prepare, and execute Cyber Mission Force (CMF) training. Today, CMF training scenarios are manually deployed on a variety of cyber training range resources using varying technologies that often lack fidelity, interoperability, reusability, and the ability to scale to support projected CMF demands. The PCTE platform addresses these gaps by delivering tools to rapidly create ‘Digital Twins’ that replicate cyberspace operational environments in a virtualized platform for the CMF to execute realistic training and mission rehearsals. PCTE provides virtualized cyber assets, tools, and environments to manage and deploy them as a service through a web application accessible anywhere for members of the CMF. Additionally, PCTE is laying the groundwork for these virtualized assets to connect to real world physical security assets such as industrial controls systems (ICS) that are

• therwise not practical to emulate. To address this

challenge, PEO STRI and the larger DoD cyber training community are utilizing an evolutionary architecture to rapidly integrate PCTE with real world physical security

• assets. This paper introduces evolutionary architecture

and discusses the cyber training community’s approach for evolving a cyber training architecture over time while simultaneously delivering capability to the CMF. The approach allowed the CMF to utilize an ICS asset in a cyber training event a mere 7 months after agreement on the approach, enabling the realistic replication of a ‘Digital Twin’ environment.

2 Evolutionary Architecture

In the US Department of Defense (DoD), the failures of following waterfall / big design up front (BDUF) development processes are well known [1]. According to the Standish Group 2018 Chaos Report: The results for all projects show that agile projects enjoy a 60% greater chance of success than non-agile

• projects. Looking deeper, we find that “waterfall”

projects are three times more likely to fail than agile projects. Eliminating BDUF on an agile project does not mean no architecture at all. To align with agile, the architecture of a system should evolve continuously over time, while simultaneously supporting the needs of current users. An evolutionary architecture supports incremental, guided change as a first principle across multiple dimensions [2]. Here are some of the characteristics of an evolutionary architecture [2]:  Modularity and Coupling: Support for modularity, enables separating components along well-defined boundaries.  Organized Around Business Capabilities: Components / services implement a single business domain capability, increasing modularity

IT2EC 2020 IT2EC Extended Abstract Template Presentation/Panel  Experimentation: Allows for several versions of the same service to run at the same time, enabling A/B testing and Canary releases. PCTE is rapidly adopting a microservice architecture [3] which is an evolutionary architecture that allows for incremental change. The microservices architecture is a design approach that enables rapid releasing of software by developing an application from a collection of loosely coupled services. Each service provides a single business

• capability. Figure 1 depicts the PCTE business

capabilities which are provided by microservices. For example, PCTE has a content repository microservice that allows the end users to Discover cyber training content. Fig 1. PCTE Business Capabilities The remainder of this section describes how PCTE has met the characteristics of an evolutionary architecture.  Modularity and Coupling: Each service provides a Well-defined RESTful [4] API to facilitate third party integration.  Organized Around Business Capabilities: As mentioned earlier, most services implement a single business capability such as Content Discovery and Scheduling.  Experimentation: The PCTE architecture could support running multiple versions of a service. PEO STRI is currently adopting OpenShift [5] (i.e. Kubernetes) to better support A/B testing [6] and canary releases.

3 Approach

Prior to elaborating the approach for evolving the cyber training architecture to create ‘Digital Twins’, we need to establish two key assumptions.

• 1. PCTE is the single platform for CMF to conduct all

cyber training. External range assets are leveraged in training via PCTE.

• 2. Cyber ranges provide high fidelity persistent

environments and remote physical assets such as the Virtualized Joint Regional Security Stacks, and ICSs, respectively. Given the key assumptions, here are the steps for evolving the cyber training architecture to integrate PCTE with external cyber assets.

• 1. Select Use Case(s): Define and select a use case (s)

that describes the expected user interaction with remote cyber asset via PCTE web interface. This activity includes storyboarding as well as writing epics / user stories that articulate the desired

• functionality. It is important to only select 1-2 use

cases to avoid a BDUF and evolve the architecture

• vertime.

Figure 2 depicts the high-level use cases for leveraging an external range asset in a cyber training

• event. It also depicts the touch points between PCTE

and cyber ranges that will likely result in the development of an API to achieve the use case. The cyber training community is using this diagram to identify and select use cases for agile prototyping (i.e. Step 2). Fig 2. PCTE Use Cases with External Range Assets

• 2. Agile

Prototyping: PCTE leverages agile development process to pilot interoperability efforts with a crawl, walk, run approach. This activity includes implementing the use cases and supporting Architecture / APIs. Figure 3 is a high-level depiction of the PCTE agile development process, which is based on Scrum [7]. Scrum is a widely used framework for iterative product development. The use cases selected in step 2 serve as the requirements backlog for building interoperability with remote external range assets. The requirements will be validated and implemented iteratively through an agile integration process led by PEO STRI for PCTE. As shown in Figure 3, this process focuses on quickly producing demonstrable products between PCTE and external ranges through a sprint, validating the usefulness with the user

IT2EC 2020 IT2EC Extended Abstract Template Presentation/Panel community (Step 3), then building on that demonstrable product over future sprints. As mentioned earlier, PCTE only iterates on 1-2 use cases concurrently. Additionally, the use cases can be implemented using a crawl, walk, run, approach allowing the developers to collect user feedback quickly to prevent squandering resources on a solution that potentially provides no value. Fig 3. PCTE Agile Development Process

• 3. CMF End User Feedback: Capture CMF feedback
• f new features that leverage remote cyber assets.

PCTE currently uses cyber training events that the CMF conducts as an opportunity to collect feedback from operational use of the platform. During these events, the CMF trainees’ complete surveys and participate in ‘hot washes’ to provide PEO STRI feedback on PCTE features. In the near future, PCTE will also support canary releases using OpenShift (i.e. Kubernetes). These will allow PEO STRI to rollout new features to a small set of users prior to releasing to the entire CMF. This is a best practice for reducing the risk of introducing new software into production.

4 Results and Future Work

In this section, the author will discuss the process and results of one iteration of the architecture evolution

• approach. For Step 1, Select Use Case(s), Discovery and

Hardware in the Loop (HWIL) was selected. For Discovery, the crawl step was for the CMF to be able to discover remote cyber assets via content card in the content repository shown in Figure 4. The content repository is the PCTE capability that allows the CMF to search for and discover content such as training packages and the atomic components that make up training packages (e.g. network specifications, virtual machine templates, etc.). Fig 4. Content Repository For the crawl phase, the content card would need to list enough information for the CMF to determine whether they want to use the range asset in a training event and information about the point of contact (POC) for manually networking a PCTE virtual environment with the remote range asset. Networking the virtual environment with a remote physical asset calls for HWIL

• solution. HWIL is not a use case but necessary

infrastructure required to provide the CMF value early

• n. For Step 2, Agile Prototyping, PEO STRI iterated for

six months on the HWIL infrastructure and Discovery use

• case. During that time, there were many discussions on

metadata need for range assets as well HWIL solutions. The PCTE integration team used NSX software-defined networking [8] to accomplish the integration with external range assets. In November 2019, the CMF successfully utilized an ICS device connected to PCTE within a cyber training event. For Step 3, CMF Feedback, participants in the training event provided comments regarding the issues networking subnets in the PCTE virtual environment to the external ICS. Those issues were captured and will help with automating the networking in a future iteration. In the next iteration, the Discovery capability will be enhanced to include more metadata about the range

• assets. This metadata will provide information to support

automating the networking of a PCTE virtual environment with a remote range asset. Additionally, future iterations will enable CMF users to schedule the use of external range assets within PCTE.

5 Conclusions

The US DoD and allies cannot afford to continue the waterfall / BDUF development approach that often fails

• r takes years to field a capability to the warfighter.

Leveraging an evolutionary architecture allows the acquisition community to evolve architectures overtime and simultaneously field incremental capabilities to the

• warfighter. Continued iterations and improvements to the

Cyber Training Architecture will allow PCTE to achieve full interoperability with real world cyber assets, enabling the realistic replication of ‘Digital Twin’ environments.

IT2EC 2020 IT2EC Extended Abstract Template Presentation/Panel

References

[1] R. N. Charettem, "U.S. Air Force Blows $1 Billion

• n Failed ERP Project," 15 November 2012.

[Online]. Available: https://spectrum.ieee.org/riskfactor/aerospace/milita ry/us-air-force-blows-1-billion-on-failed-erp- project. [2] N. Ford, "nealford.com," [Online]. Available: http://nealford.com/downloads/Evolutionary_Archit ectures_by_Neal_Ford.pdf. [Accessed 2 December 2019]. [3] S. Newman, Building Microservices: Designing Fine-Grained Systems, O'Reilly Media, 2015. [4] R. T. Fielding, "Architectural Styles and the Design

• f Network-based Software Architectures,"

University of California, Irvine, 2000. [5] Red Hat, "Red Hat OpenShift," Red Hat, 2019. [Online]. Available: https://www.openshift.com/. [Accessed 4 December 2019]. [6] J. F. Box, " Guinness, Gosset, Fisher, and Small Samples," Statistical Science, vol. 2, no. 1, pp. 45- 52, 1987. [7] J. V. Sutherland and K. Schwaber, "Business object design and implementation:," in OOPSLA '95 workshop proceedings, Michigan, 1995. [8] VMWare, "VMWare NSX Data Center," VMWare,

• 2019. [Online]. Available:

https://www.vmware.com/products/nsx.html. [Accessed 4 December 2019]. [9] M. Fowler, "CanaryRelease," 25 June 2014. [Online]. Available: https://martinfowler.com/bliki/CanaryRelease.html. [Accessed 4 December 2019].

Author/Speaker Biographies

Amit Kapadia is the Product Manager Cyber Resiliency and Training (PdM CRT) Chief Engineer at the U.S. Army Program Executive Office for Simulation, Training and Instrumentation (PEO STRI). He provides technical direction and oversight for the PdM CRT portfolio that features the Persistent Cyber Training Environment (PCTE), Army Acquisition Blue Team, and National Cyber Range Complex (NCRC). Amit has worked in a variety of domains supporting the acquisition of test instrumentation, live-virtual-constructive simulations, and Mission Command Systems. He received his Bachelor and Master of Science degrees in Electrical Engineering from the University of Central Florida. Rick Osborne is a Lead Simulation Engineer at The MITRE Corporation. He is currently the chief architect for the Persistent Cyber Training Environment (PCTE) program at U.S. Army PEO STRI. He earned his B.S. in Computer Engineering from Christopher Newport University and M.S. in Modeling and Simulation from The University of Central Florida. Brian Vermillion is a Simulations and Training Engineer at The MITRE Corporation. He is currently working as an architect and DevOps engineer for the Persistent Cyber Training Environment (PCTE) program at U.S. Army PEO

• STRI. He earned his B.S. in Computer Engineering from

the University of Central Florida.