Data Mining Combat Simulations: Data Mining Combat Simulations: an - - PowerPoint PPT Presentation

Computational and Information Sciences Directorate

Army Research Laboratory (ARL)

The U.S. Army’s Corporate Laboratory

Barry A. Bodt

babodt@arl.army.mil (410) 278-6659

Data Mining Combat Simulations: Data Mining Combat Simulations: an Emerging Opportunity an Emerging Opportunity

Motivation Motivation Motivation

• Simulation and statistical analysis are underutilized

in helping the commander’s staff to analyze courses of action.

• Battle results are infinite in scope, yet the outcome
• f any one battle is defined by a unique set of

battlefield interactions.

• Key is to recognizing those interactions through

development of more informative performance measures unique to the scenario at hand.

Approach Approach Approach

Use statistical methods Use statistical methods and combat models to and combat models to create a methodology create a methodology that identifies non that identifies non-

• traditional metrics for

traditional metrics for plan evaluation. plan evaluation.

COA COA

Background Background Background

• Focus on

Focus on wargame wargame

• Disciplined rules

Disciplined rules

• Synchronization matrix

Synchronization matrix

Military Decision Making Process Military Decision Making Process Military Decision Making Process

Joint Tactical Operation Center, Qatar Joint Tactical Operation Center, Qatar Joint Tactical Operation Center, Qatar

Communicate… Communicate… Smart Logistics Smart Logistics On On-

• board Diagnostics

board Diagnostics Soldier Health Soldier Health Sensor information Sensor information … …

Network Centric Warfare Network Centric Warfare Network Centric Warfare

The key to any analysis is the set of measures used to The key to any analysis is the set of measures used to represent the performance and effectiveness of the represent the performance and effectiveness of the alternatives being considered. We are relatively good at alternatives being considered. We are relatively good at measuring the performance of sensors and actors, but measuring the performance of sensors and actors, but less adept at measuring command and control. less adept at measuring command and control. Command and control, to be fully understood, cannot Command and control, to be fully understood, cannot be analyzed in isolation, but only in the context of the be analyzed in isolation, but only in the context of the entire chain of events that close the sensor entire chain of events that close the sensor-

• to

to-

• actor

actor

• loop. To make this even more challenging, we cannot
• loop. To make this even more challenging, we cannot

isolate on one target, or even a set of targets but need isolate on one target, or even a set of targets but need to consider the entire target set. Furthermore, network to consider the entire target set. Furthermore, network centric warfare is not limited to attrition warfare … It is centric warfare is not limited to attrition warfare … It is not sufficient to know how many targets are killed, but not sufficient to know how many targets are killed, but exactly which ones and when… exactly which ones and when…

Ref: Network Centric Warfare, 2002 Ref: Network Centric Warfare, 2002

Information Requirements in NCW Information Requirements in NCW Information Requirements in NCW

Simulation Data Simulation Data Simulation Data

• Scenario development
• OneSAF lay down of forces
• OneSAF modified output
• Data supporting modeling

Scenario Scenario Scenario

Company Objective Town

BMP-2 BMP-2 BMP-2 T-80 T-80 T-80 T-80 T-72M T-72M T-72M T-72M T-72M

OneSAF Screen Dump OneSAF OneSAF Screen Dump Screen Dump

Automated Data Collection Automated Data Collection Automated Data Collection

• OneSAF Modifications

OBJECT_ID: 100A31 X = 24396.82 Y = 25828.75 Z = 755.72 Vehicle Authorized Undamaged Catastrophic Firepower Mobility Damage Damage Damage M2 1 0 1 0 0 Equip/Supplies: Current Lvl Resupply Lvl Avg Per Veh 25mm HE (M792) 625.00 625.00 625.00 25mm APFSDS-T (M919) 325.00 325.00 325.00 TOW (TOW) 0.00 5.00 0.00 7.62mm MG (M240) 2340.00 2340.00 2340.00 Fuel (Fuel) (gallons) 171.00 174.00 171.00

OneSAF Modification Killer/Victim Scoreboard OneSAF OneSAF Modification Modification Killer/Victim Scoreboard Killer/Victim Scoreboard

Time Stamp 1010070890 Vehicle ID 1076 Firer ID 1087 Projectile 1143670848 Firer Position: X = 220217.00 Y = 146765.00 Z = 12.37 Target Position: X = 222454.38 Y = 149117.80 Z = 9.99 Vehicle 1076: Hit with 1 "munition_USSR_Spandrel" (0x442b0840) Comp DFDAM_EXPOSURE_HULL, angle 19.53 deg Disp 0.889701 ft Kill Thermometer is: Pk:1.00, Pmf:1.00, Pf:0.90, Pm:0.80 Pn:0.80 RANGE 3246.773576 r = 0.990835 kill_type = MF

• Firer and Target Identity and Location

Firer and Target Identity and Location

• Type of Ammo

Type of Ammo

• Range

Range

• Outcome

Outcome

1076 100A41 vehicle_US_M1 1087 100A23 vehicle_USSR_BMP2

Data Supporting Classification Models Data Supporting Classification Models Data Supporting Classification Models

• 228 OneSAF runs
• 3 situational snapshots per

run

– 10% blue ammo expended – 25% blue ammo expended – 40% blue ammo expended

• 429 data points per run (143

per stopping time)

– Number of K, M/F, F, and M kills – Ammunition levels – Number of hits delivered – Range of hits – Number of side hits delivered – Distance to objective – Number of Blue on objective

Response Response – – mission mission accomplished (success) accomplished (success) if an undamaged platoon if an undamaged platoon

• ccupies objective at
• ccupies objective at

battle end (MA) battle end (MA)

• other responses include
• ther responses include

MBT and “Eric” strength MBT and “Eric” strength and forces on objective and forces on objective

Data Matrix Data Matrix 228 x 434 228 x 434

Company Objective

74 35

1

34 85

1

Pred Obs

Correctly Classified Correctly Classified Loss: 71% Loss: 71% Win: 68% Win: 68% Overall: 70% Overall: 70% Slice 1 ~ 2000m Slice 1 ~ 2000m Or ~ 5 ½ minutes Or ~ 5 ½ minutes

Company Objective

84 25

1

21 98

1

Pred Obs

Correctly Classified Correctly Classified Loss: 82% Loss: 82% Win: 77% Win: 77% Overall: 80% Overall: 80% Slice 2 ~ 4000m Slice 2 ~ 4000m Or ~ 10 minutes Or ~ 10 minutes

Company Objective

89 20

1

14

105

1

Pred Obs

Correctly Classified Correctly Classified Loss: 88% Loss: 88% Win: 82% Win: 82% Overall: 85% Overall: 85% Slice 3 ~ 5800m Slice 3 ~ 5800m Or ~ 20 minutes Or ~ 20 minutes

Model Performance Model Performance Model Performance

Method Comparison Method Comparison Method Comparison 85% 82% 85% 20 74% 75% 80% 10 69% 70% 70% 5 ½ Logistic Regression CART Discriminant Analysis Stopping Time (min)

Percent Correct Classification Percent Correct Classification by Stopping Time and Method by Stopping Time and Method

Advantages Advantages Advantages

– Support prediction for COA performance evaluation – Provide models identifying key battle parameters for a given engagement, influencing both COA development and commander’s critical information requirements – Input to CCIRs – Input to contingency plans – Input to tolerances for synchronization

Implementation Models Implementation Models Implementation Models

Reach back Reach back Advantages Advantages

• computational power (ARL 9

computational power (ARL 9th

th)

)

• more complex analyses

more complex analyses Disadvantages Disadvantages

• latency

latency

• can’t smell gunpowder

can’t smell gunpowder Distributed Distributed Advantages Advantages

• cheaper boxes (250

cheaper boxes (250 OneSAF OneSAF boxes used at Ft. Leavenworth) boxes used at Ft. Leavenworth)

• closer to action

closer to action Disadvantages Disadvantages

• depth of a field analysis

depth of a field analysis

• automation required

automation required

Why Aren’t We Already Doing This? Why Aren’t We Already Doing This? Why Aren’t We Already Doing This?

• Computer simulation focus has been mainly strategic or
• riented toward acquisition. Tactical application has been

limited.

• Simulations did not have high enough fidelity for tactical

application.

• Simulations were unstable.
• Computing resources were inadequate.
• Necessary communication of inputs had not been

imagined.

• Simulation creators do not always talk to statisticians.

A few reasons … A few reasons …

Improvements Here and On the Way Improvements Here and On the Way Improvements Here and On the Way

• Stability
• Power Point force laydown of forces
• MS Word OPORD
• Terrain, weather wizzards
• Composable simulations
• After Action Report data
• Man-in-loop allowed
• Sensor advances
• Communication advances
• Computation speed and cost

Catching On? Catching On? Catching On?

20

After Action Review

• Situation awareness during the execution of the exercise and

afterwards during exercise playback: – PVD & 3D Stealth display – Statistical charts, tables – OPORD paragraphs – Task Organizations Summaries – Radio/audio playback (Future)

• Mining of collected data to construct MOPs/MOEs
• Automatically build AAR presentations & Take Home Package

using COTS Office Automation

PURPOSE: The OneSAF After Action Review component provides the capability to correlate, roll- up, and analyze simulation outputs and visualize the results of the simulation exercise. The toolset allows the analyst to preplan the AAR prior to exercise execution.

Wei Wei-

• Yin

Yin Loh Loh, Regression Tree Analysis of Battle Simulation Data , Regression Tree Analysis of Battle Simulation Data David Kim, Robust Modeling Based on L2E Applied to Combat David Kim, Robust Modeling Based on L2E Applied to Combat Simulation Data Simulation Data Warren Warren Liao Liao, Discovery of Battle States Knowledge from Multi , Discovery of Battle States Knowledge from Multi-

• Dimensional Time Series Data

Dimensional Time Series Data

Next Up Next Up Next Up