IT 2 EC 2020 Train, Reflect, Learn, and Train again Chris Duncalfe 1 - - PDF document

IT2EC 2020 IT2EC Extended Abstract Template Presentation/Panel

IT2EC 2020 – Train, Reflect, Learn, and Train again

Chris Duncalfe1, and John Gardner2

1SO2 Training Capability Systems, Training Capability Branch, Army Headquarters, United Kingdom 2SO2 Training Measurement & Evaluation, Training Capability Branch, Army Headquarters, United Kingdom

Abstract — The British Army has embarked on an ambitious programme to transform collective

• training. The Future Collective Training System will prepare soldiers for complex future operations,

both above and below the threshold of conflict. The system will exploit data and information to empower individuals and commanders, and to maximise effectiveness. This paper focuses on providing insight into ongoing research identifying technologies which offer novel ways of capturing

• bjective data to support analysis of individual and team performance, increasing the potential for the

training audience to reflect and learn before they train again.

1 Introduction

In response to the 2017 Collective Training Review, the Executive Committee of the Army Board approved the Collective Training Transformation Programme (CTTP)

• Mandate. The CTTP was directed to design, develop and

deliver a Future Collective Training System (FCTS) that will, through a continuous evolution, transform collective training by 2025. A significant component of the FCTS is enhanced use of synthetic, and training measurement and evaluation capabilities. These will provide the synthetic

• perating environment with the stressors, cues, and

stimulus required to train skills, and capture and analyse

• bjective data to support evaluation of team and task
• performance. To support this ambition, the CTTP has

commissioned research into team skills and technologies that capture objective data to support evaluation of team performance.

2 Background

The CTTP vision is: “From now to 2025, Collective Training will be transformed to prepare the Army (critically, through delivering trained force elements at readiness, but also through contributing to maintenance of a depth of institutional capability and capacity) for the dynamic and complex future operating environments faced in an era of constant confrontation. Collective Training will become surrogate for warfare; driving adaptation, generating combat ethos, empowering commanders, and delivering tactical innovation. In order so to do, it will be measurable, enabled by simulation and technology, and form part of an assured and responsive lessons loop and wider Warfare Development process. In terms of delivery, the British Army will train in regions of the world that cement our Joint and International partnerships and reassure our friends and deter potential adversaries. Underpinning all of this, an alliance with the best of industry will deliver technological innovation and

• ptimise outputs.” There are six projects which will

deliver the capabilities the FCTS will required. Pj Synthetics will provide the capabilities which generate the synthetic

• perating

environment. Pj Training Measurement & Evaluation (TME) will deliver capability to capture and evaluate data to develop the training audiences , enhance the training system and support Force

• Development. Both projects are currently in the concept
• phase. They have been supported by a study into the

knowledge, skills, and attitudes required for effective team performance, and the Virtual Reality in Land Training (VRLT) Pilot which is investigating technology which might aid capture of data to support TME.

3 Knowledge, Skills, and Attitudes

This research looked to identify the knowledge, skills, and attitudes (KSA) necessary for effective teamwork. It suggested a teamwork taxonomy of five components: leadership, situational awareness, decision making, communication, and co-operation. The study identified the need to understand how technology could assist with automated capture of objective data which could support measurement and evaluation of the five component areas. The Army HQ Training Capability Branch are working with the Collective Training Group to exploit the data capture potential of in-service capabilities in support of

• this. These efforts will be enhanced when the FCTS

delivers new technology focused on measuring the KSA of individuals within teams and their interactions. For example, there already exists the capability to capture voice communications. However, analysis of this communication is mandraulic, ad-hoc and generally

• subjective. The future vision is to provide technology

which automates such processes, providing both the training audience and training delivery agent with consistent capture and analysis of objective data . This will be integrated with subjective evaluation to create a more comprehensive understanding of team performance, which will, in turn help us to diagnose task performance

IT2EC 2020 IT2EC Extended Abstract Template Presentation/Panel

• utcomes. This approach is considered critical to ensuring

teams are provided with the information they need to reflect and learn before they train again. The VRLT Pilot was chosen as the route to exploring technologic aides.

4 – Technology support for measuring team performance

The first VRLT Pilot was delivered in 2019. The project explored a number of innovations which could support transformation of training. Many related to use of Virtual Reality and Mixed Reality. However, several innovations explored technology to support measuring situational awareness, team communication and co-operation skills. The project explored these innovations through three

• sprints. Each sprint used an actual training exercise with

real soldiers as the construct. This approach was considered highly beneficial as it provided the opportunity to integrate and evaluate the technical innovations in the hands of the user within a formal training context. The training scenario was set around a combined arms Armoured Infantry platoon. During the second sprint, voice analytics translated voice to text and interpreted the semantic meaning of the content. The communications were captured from the soldiers’ radios using an Audio Recording Unit. The third sprint conducted further communication analysis using software to produce charts that show the factors contributing towards communication

• effectiveness. Additionally, an automated system was

employed to monitor an individual’s performance during a high workload event by rating it according to the NATO recognised Bedford Workload rating scale.

5 – Findings

The VRLT innovations were separated into three tiers. Tier 1 was considered to have the most benefit to a combined arms sub-unit group. Tier 2 innovations were considered beneficial but not a priority and Tier 3 were seen as beneficial within a wider context but not explicitly at the combined arms sub-unit level. The communication analysis, workload monitoring, and automated voice analysis were all considered as Tier 1 following a stakeholder review which consisted of military, academic, and industry subject matter experts. The communication analysis identified that standards of communication performance during high workload tasks could be measured, supported with objective data. This was achieved by processing speech signals and calculating levels of accommodation from values like pitch, rhythm, tone, and energy in the voice. Workload performance monitoring demonstrated the potential benefit from confirming and understanding cognitive workload, providing the means to recognise when individuals were subjected to greater levels of stress by points in time. Combining this information with communication analysis, decisions made, and task outcomes, it was shown this data would support evaluation of the team performance allowing provision of actionable feedback to the training audience and enabling future training challenge to be set at appropriate levels.

6 – Next steps

The success of the first VRLT Pilot secured 3* approval and direction to conduct further research using a similar construct to ensure that soldiers were benefiting immediately from additional training opportunities in parallel to developing understanding of technology which could be used to support the FCTS. VRLT 2 commenced in January 2020 and is due to run until 2021. In addition to numerous investigations into VR and MR technology, it will explore a Remote Observation Package (ROP) which builds upon the findings from the first VRLT Pilot. Newman Spur Consultancy (NSC) are working with a team led by Cervus. A variety of technologies will be used to integrate analysis of voice communications, workload analysis, and eye motion tracking. The data captured will be evaluated to provide measurement of stress and cognitive burden, command and control performance, team and task performance, which are then blended to provide a dashboard displaying objective data driven analysis which can complement subjective analysis of the team performance.

7 – Summary

The CTTP is running to an ambitious timeline of delivering capabilities from 2023-2025. The research conducted thus far has provided confirmation concepts envisaged for the FCTS are viable. It has also proved that the ambition to collate a far greater level of objective data is possible via automated means and this data can be displayed in an easily interpretable manner. This will ensure that the training audience can easily understand the evaluation of their performance and, through collaboration with the training delivery organisation, can focus future training with appropriate levels of challenge on the areas most likely to enhance the team’s performance. The research approach creates more training opportunities for soldiers, providing immediate benefit but also enabling gradual introduction to new methods of analysing and evaluating training which will assist with the change management required as the FCTS comes into service.

Author/Speaker Biographies

Chris Duncalfe is an infantry officer currently employed in a Capability and Acquisition role at Army Headquarters. He is working for the Collective Training Transformation Programme and is the lead for Project Synthetics which will deliver the Future Collective Training System synthetic capabilities. John Gardner is an Educational and Training Services

• fficer currently working on the Collective Training

IT2EC 2020 IT2EC Extended Abstract Template Presentation/Panel Transformation Programme and is the lead for Project Training Measurement & Evaluation.