Raising a Digital Twin, Avoiding the Terrible Twos John Meyers, - - PDF document

ITEC 2020 ITEC Extended Abstract Presentation

NAWCTSD Public Release 20-ORL001 Distribution Statement A - Approved for public release; distribution is unlimited.

Raising a Digital Twin, Avoiding the “Terrible Twos”

John Meyers, Naval Air Warfare Center Training Systems Division, 32826, USA Michael Merritt, Naval Air Warfare Center Training Systems Division, 32826, USA

Abstract - Data analytics and Digital Twins are becoming popular tools in many organizations, this includes government organizations. However, the realities and processes to bring these new tools into an organization is not always well understood and shared. This paper captures lessons learned specific to the use of data analytics and creating a digital twin to better understand a very complex and high priority Fleet issue (in this case aircraft physiological episodes). These lessons learned begin with understanding the "problem at hand", standing up a functional team, enablers (e.g. software tools, computing power), barriers (e.g. data availability, data quality, business and regulatory issues), successes and best practices. Current capabilities will be presented as well as consideration of the potential benefit of creating human digital twins (both physical and cognitive). Data analytics and digital twin technology is invaluable to better understanding complex problems with the potential to greatly improve human performance in complex systems.

Introduction

Data analytics is a powerful tool in modern businesses. For this paper following, data analytics is the science of analysing raw data to make conclusions to guide decisions. A digital twin is a virtual model of a physical object, process or system. The problem, or use case, requiring the

• rganization to develop and use these tools is referred to

as an aircraft physiological episode or PE. The Navy's definition of a physiological episode is when a pilot experiences loss in performance related to insufficient

• xygen, depressurization, or other factors during flight. It

is not the purpose of this paper to cover specifics related to the issues, suspected causes or corrective actions resulting from efforts to better understand PEs. Information is available in the public domain regarding the Navy’s work

• n PEs. This paper covers how data analytics and digital

twin technology was used in understanding the problem and lessons learned as an organization.

Background

PEs are dangerous and effect safety of flight. Reported PEs were increasing over historical rates. There are many causes that can result in an official PE. Different aircraft were affected and responsibilities crossed multiple

• rganizational boundaries. Aircrews were concerned and

Senior Leadership wanted answers. Several potential causes were hotly debated; however, no single root cause was clearly identifiable at the time. The result was a need for a data driven approach to provide a sound foundation to make safety of flight and fiscal decisions.

Creating the Team

In the early phases, data visualization tools were utilized in an effort to link disparate data sets to identify PE casual

• factors. It was then decided to contract out some of the data

analytics tasking to get an independent perspective and glean insight into data analytic technics applied by

• industry. Because of the magnitude, the complexity, the

need for in-depth domain expertise, and the desire to establish a strong data analytics capability within NAVAIR, the decision was made to use in-house resources to continue the data analytic efforts. Building a solid data analytics team is not an easy task.

• The Team Lead requires excellent leadership skills in

addition to knowing the organization and being recognized as having strong technical credibility from the

• rganization.
• Individual team member skills are critical. Performance

requires computer science, statistics and domain knowledge in the area that you are applying data analytics

• to. See Figure 1. Realistically, hiring the right academic

background is the most practical approach. For this effort, since the effort required logisticians and engineers, the initial thought was to train personnel currently in these disciplines to be good data scientists. What was found to be the best approach was to teach data scientists to understand the engineering and logistics involved in the issues at hand. Collaboration with subject matter experts for domain experience was necessary (specifically aircraft systems and maintenance processes).

• Fig. 1: Team Skills
• Recruiting talent was not just a Human Resource

Department effort, it took required senior leadership engagement to build the team.

• Enterprise level support is required to remove
• rganizational and bureaucratic barriers, including the

team being able to have direct access to senior leaders. A

ITEC 2019 ITEC Extended Abstract Template Presentation/Panel

NAWCTSD Public Release 20-ORL001 Distribution Statement A - Approved for public release; distribution is unlimited.

collaborative work environment is required both internal and externally.

• An appropriate level of autonomy and trust is required.

In one case we had a new hire briefing senior leadership within weeks because they best understood their work product.

• Retention of talent requires attention. Feeling valued and

challenged with helping to solve a real world difficult problem is important. Also, exposure to the operational units and aircraft are invaluable morale builders. As a government organization, competing on salary is not a viable retention strategy. Our team now consist of 14 full time employees, most with advanced technical degrees. This team routinely collaborates with other data analytics teams within the enterprise and various fleet activities to ensure successful

• utcomes.

Data – Harder than you think

If data analytics is the science of analysing raw data, then clearly you need access to relevant data (preferably lots of data). A common mental model of data analytics focuses

• n the Internet of Things and the “ocean of data” available.

However, this is not the situation in our use case.

• There needs to be a clear question statement covering the

data analytics tasking. In this example two questions were

• critical. Is there a root-cause resulting in the increase of

PEs? Can specific categories of causes capable of creating PEs be eliminated as causal to specific aircraft under investigation?

• Data availability must be considered. Understanding the

data available and how to get access to it was not a trivial

• exercise. For example, the aircraft had a great deal of

sensor data that was available, however that data was only available in binary form. A data decoder was required to convert the binary data to a usable format. This was further complicated by having a different data decoder required for each different lot of aircraft.

• Even more challenging was that other relevant data was

scattered around the enterprise. Instead of an ocean of data we had “small lakes” in many different formats. Organizational “ownership” on data was across the Enterprise resulting is complex approvals to release information.

• Data quality must be considered. Inconsistent formats,

intellectual property issues, gaps in data all result in manual intervention to “smooth” the data before any analysis can begin. In addition, modelling may be required to “fill in gaps” as necessary, given of course appropriate models exist. After various sources of data started to be compiled, it was clear that there were data that was needed that was not being collected. For example, it was theorized that pressure fluctuations in the cockpit was a major contributor of PEs, however the pressure in the cockpit was not being measured. This led to issuing wearable sensors that pilots would put in their flight equipment pocket and data from those sensors would be downloaded after each flight. Time syncing this data with the aircraft data for every flight for each aircraft was no trivial task.

• Moving data around the enterprise was also very
• challenging. Data at rest in various enterprise systems

cross many IT boundaries. The Authority To Connect can not only be time consuming, but in some cases impossible. Bandwidth issues for the amount of data that needed to be transferred was also a major obstacle. It was common for teammates to hand carry hard drives of significant amounts

• f data between sites (e.g. “sneaker net”) just to move data

to where it was required for use.

• Choice of data analytics tools requires planning and
• flexibility. Many tools are available, some more optimal

than others. It is important to not be limited to enterprise level tools because as the world of work evolves, needs can change quickly. In one instance, the volume of the data necessitated a change in the analysis software being used.

• The quantity of data can become very large, plan for it.

Every day a few GBs of new data is added to keep the analysis and reports up-to-date. Currently the data set contains about 300TB of data and adds approximately 5GB a day. Data analytics proved to be a powerful tool in understanding PEs. Data driven decisions greatly aided the way forward and several suspected PE causes were eliminated allowing the team and leadership to better focus

• n fewer issues.

Computing power and IT

As the amount of data increased and the demand for timely answers became more acute, upgraded hardware became a necessity.

• A high performance computer cluster was required. The

computer cluster contains 14 nodes with four NVidia V100 GPUS per node and 12 CPU nodes with two INTEL XEON E5-2620V4 per node. Peak performance is in the 7 peta FLOPS range.

• IT constraints must be a consideration through the project

lifecycle, currently the computer cluster CPU nodes are being upgraded to 52 nodes.

• Leadership intervention was required to overcome the

normal bureaucratic process to procure Information Processing equipment. In this case from the identification

• f need to delivery was twelve weeks. Very quick for a

government operation!

Digital Twin

The data analytical effort eventually evolved into creating a digital twin of an aircraft Environment Control System (ECS). A digital twin was needed to provide a prescriptive maintenance response for ECS components prior to failure, reducing the risk of a PE. Figure 2 shows a generic visual representation of the “health” of a specific aircraft’s ECS system after each flight and the change to that health as various maintenance actions were performed.

• The ECS system is very complex. Success requires

reaching out to domain experts, in this case requiring extensive fleet and aircraft engineering support to create effective models.

ITEC 2019 ITEC Extended Abstract Template Presentation/Panel

NAWCTSD Public Release 20-ORL001 Distribution Statement A - Approved for public release; distribution is unlimited.

• The need was to identify predictive maintenance actions

before conditions resulted in a component failure and a potential PE (e.g. 3-5 days prior notice before component failure).

• Fig. 2: Data Features and Visualization
• Understanding features (data attributes) is key to

performance outcomes. Features can be indicators of condition changes critical for performing prescriptive

• maintenance. However, routine flight profiles and routine

maintenance actions affect these same features. The power

• f data visualization allows insight into causes allowing

actionable recommendations. For our use case, recommended maintenance actions on specific parts prior to failure or undesired performance outcomes (e.g. PEs).

• Given the need to push specific data products directly to

the fleet at the operational level, the team created specific software tools compatible with our enterprise network solutions, in this case the Navy Marine Corps Intranet (NMCI) Network. Understanding the data needed, where it’s needed and effective methods to move the data within enterprise constraints is critical for positive results.

What’s Next? A digital twin of the human!

Now that we have a high performing data analytics team, what is the next challenge?” One answer – the application

• f data analytics to human performance measurement with

a long term goal of a digital twin of the human. Typically, modern weapons systems are optimized by focusing on optimizing the hardware and software of that weapon system. Additionally, each system must be linked to other systems to create a capability which quickly grows into a fairly complicated systems of systems environment. However, optimizing hardware and software alone will not yield the most optimized systems capability. Figure 3 shows that the third, and most critical component, especially in a system of systems environment, is the Human Performance of that system of systems. Optimizing human performance is critical as systems become more complex to include increased human/machine interaction. Much needs to be accomplished before a digital twin of the human can become a reality. Given that integration of human capabilities with hardware and software yields system capabilities, how to you optimize the human performance and what is the role of data analytics? For capturing human performance, data can be binned into two buckets, the physical and the cognitive data of the human.

• Fig. 3: The human impact to system capability

There are two fundamental issues with developing a human digital twin, source data and digital models (both physical and cognitive human performance models). In the context of data analytics, human performance data will be challenging. Data sources are well known. For example education records, training records, mission performance and exercise data are collected today. However, these records distributed in various locations, lack common structures (e.g. data taxonomies) and currently do have the requirement of metatags. Recent advancements in wearable technologies will be invaluable in understanding human performance and health while performing tasks. However, each person is unique and defining and understanding physiological performance features will be challenging. Collecting, storing and using human performance data requires ethical and regulatory requirements be considered from the beginning. The exciting opportunity, if the proper approach is taken, is that the operator or maintainer of any Navy system, or system

• f systems can be optimized for that system or system of
• systems. In an era where the technological playing field is

now relatively even, human performance is now the key discriminator.

Conclusion

Data analytics and the creation of a digital twin of a specific aircraft subsystem were critical elements in understanding a very complex human performance issue related to PEs. However, the organizational challenges and lessons learned to develop this capability within our

• rganization are not unique to this one use case. The

purpose of this paper is not only to share our experiences, but to foster an environment of collaboration and sharing to advance human performance measurement and analysis

ITEC 2019 ITEC Extended Abstract Template Presentation/Panel

NAWCTSD Public Release 20-ORL001 Distribution Statement A - Approved for public release; distribution is unlimited.

given recent advancements in the extremely powerful capabilities of data analytics and digital twin technologies.

Acknowledgements

The author would like to thank the many Integrated Product Team members performing the work and analysis used for the various projects used as a basis for this paper. The views expressed in this paper are solely those of the author and do not reflect any Department of Defense agency or service.

References

[1] "Naval teams narrow factors in physiological episodes

• n jets", Defence News, 2 Apr 2019, Allen Cone

[2] "Physiological episodes", Navy.mil > secnav. Hot Topics [3] "Navy rules out Suspected Physiological Episodes Cause While Super Hornet Rates Grow in 2019", USNI News, 4 Apr 2019, Megan Eckstein Author Biography John Meyers is the Executive Director at the Naval Air Warfare Center Training Systems Division tasked with the responsibility for ensuring that business and financial

• bjectives are met and the overall mission is executed in a

safe and efficient manner. While in the honors program at the Pennsylvania State University, he obtained a Bachelor’s degree in Engineering Science and Mechanics in 1990 and a Master’s of Science in Mechanical Engineering in 1992. Michael Merritt is the Acquisition Director at the Naval Air Warfare Center Training Systems Division tasked with the responsibility for workload planning, mission execution and collaboration with Government and Industry partners to effectively execute NAWCTSD mission tasking. Mr. Merritt earned a Bachelor’s degree in Engineering from the University of Central Florida in

• 1982. He received his Master’s Degree in Electrical

Engineering from the Air Force Institute of Technology in 1984.