Situation recognition as a step to an intelligent Situation - PowerPoint PPT Presentation

Situation recognition as a step to an intelligent Situation recognition as a step to an intelligent situation- -aware crew assistant system aware crew assistant system situation Quint Mouthaan Quint Mouthaan Patrick Ehlert Patrick Ehlert Leon Rothkrantz Leon Rothkrantz October 23, 2003 - - BNAIC, BNAIC, Nijmegen Nijmegen October 23, 2003 The ICE project Faculty of Electrical Engineering, Mathematics, and Computer Science Delft University of Technology

Contents Contents ! The ICE project The ICE project ! ! Design Design ! ! Implementation Implementation ! ! Results Results ! ! Conclusions and future work Conclusions and future work ! 2 2

ICE: the project ICE: the project ! Ultimate goal: artificial co Ultimate goal: artificial co- -pilot pilot ! " Take over tasks Take over tasks " " Provide useful information Provide useful information " ! Main focus on reasoning Main focus on reasoning ! 3 3

ICE: architecture ICE: architecture ICE system System system crew Crew management settings commands Decision- interface Interface making commands environment state Situation assessment actuator commands Aircraft systems avionics Data fusion information 4 4

Design: situation recognition Design: situation recognition ! Detect current situation based on data Detect current situation based on data ! from simulator from simulator " Aircraft status Aircraft status " " Pilot actions Pilot actions " " Environment Environment " ! Probabilistic approach Probabilistic approach ! 5 5

Design: architecture Design: architecture Situation Flight Input Rule Overall module base controller simulator Knowledge Flight plan converter interpreter Knowledge Flight base plan 6 6

Design: overall controller Design: overall controller 7 7

Design: start probabilities Design: start probabilities Startconditions Rule1 Rulen Action Action 1 n Pcan Pcr1 Pca1 Pcrn P sc noisy-OR P Situation cond started Previous P P s end situation ended noisy-AND Situation occurring P start 8 8

Design: end probabilities Design: end probabilities Time window Rule1 Rulen Endconditions Pcr1 Pt Pcrn P c P noisy-OR cond Situation ended P P Situation e s started noisy-AND Situation finished P end 9 9

Implementation Implementation ! Java Java ! ! MSFS 2002 Simulator MSFS 2002 Simulator ! ! Rule Rule- -based system JESS based system JESS ! ! Knowledge base for Cessna and F Knowledge base for Cessna and F- -16 16 ! ! Hard Hard- -coded probabilistic reasoning coded probabilistic reasoning ! 10 10

Results: example experiment (1) Results: example experiment (1) An attack on a ground target: An attack on a ground target: Situation Time Time Situation Time Time started (s) detected (s) started (s) detected (s) Startup Startup 0 0 0 0 Taxiing to runway 10 12 Taxiing to runway 10 12 Taking off Taking off 14 14 15 15 Normal flight 43 34 Normal flight 43 34 Navigating 83 83 Navigating 83 83 Normal flight 91 91 Normal flight 91 91 Navigating 123 123 Navigating 123 123 11 11

Results: example experiment (2) Results: example experiment (2) An attack on a ground target: An attack on a ground target: Situation Time Time Situation Time Time started (s) detected (s) started (s) detected (s) Normal flight Normal flight 128 128 128 128 Visual attack 186 193 Visual attack 186 193 Normal flight 221 221 Normal flight 221 221 Landing 361 366 Landing 361 366 Aborting landing Aborting landing - - 410 410 Taxiing from runway 409 410 Taxiing from runway 409 410 Shutdown 427 427 Shutdown 427 427 12 12

Conclusions Conclusions ! Probabilistic model performs fairly well Probabilistic model performs fairly well ! ! Few mistakes, able to correct Few mistakes, able to correct ! ! Recognize situations in real Recognize situations in real- -time for time for ! Cessna and F- -16 16 Cessna and F ! First step towards situation First step towards situation- -awareness awareness ! system system 13 13

Future work Future work ! Compare with other approaches Compare with other approaches ! ! Feedback from flight plan Feedback from flight plan ! ! Add other causal relationships Add other causal relationships ! 14 14

The F- -16 cockpit 16 cockpit The F 16 16

Design: probabilistic network Design: probabilistic network gear throttle P(approach|gear) P(approach|throttle) P noisy-OR: 1 – (1 – P1) * (1 – P2) radio radio approach P approach gear throttle P(approach approach) ) gear throttle P( F F 0 F F 0 noisy-AND: landing F T 0.2 F T 0.2 P = P1 * P2 T T F F 0.8 0.8 P landing T T T T 0.84 0.84 17 17

Implementation: XML file F- -16 16 Implementation: XML file F <situation name="Taxiing to runway" timewindow timewindow="5"> ="5"> <situation name="Taxiing to runway" <nextsituations < nextsituations> > < <nextsituation nextsituation>Taking off</ >Taking off</nextsituation nextsituation> > </ </nextsituations nextsituations> > <actions> <actions> <phase name="taxiing"> <phase name="taxiing"> <action name="parking brakes" priority="1" probability="MP">&OFF;</action> ;</action> <action name="parking brakes" priority="1" probability="MP">&OFF <action name="throttle" priority="1" probability="SP">>&IDLE;</action> ction> <action name="throttle" priority="1" probability="SP">>&IDLE;</a <action name="throttle" priority="0.4" probability="SP">&IDLE;</ <action name="throttle" priority="0.4" probability="SP">&IDLE;</action> action> </phase> </phase> <phase name="time independent"> <phase name="time independent"> <action name="wheel brakes" priority="0.6" probability="MP">&ON;</action> <action name="wheel brakes" priority="0.6" probability="MP">&ON; </action> <action name="MPO" priority="1" probability="MP">&NORMAL;</actio <action name="MPO" priority="1" probability="MP">&NORMAL;</action> n> </phase> </phase> </actions> </actions> < <visualChecks visualChecks> > <instrument name="speed brakes" repetitive="no" priority="0.3"/> <instrument name="speed brakes" repetitive="no" priority="0.3"/> <instrument name="caution panel" repetitive="no" priority="0.8"/> <instrument name="caution panel" repetitive="no" priority="0.8"/ > …… …… </visualChecks visualChecks> > </ <constraints startProbability startProbability="MP" ="MP" endProbability endProbability="BP"> ="BP"> <constraints <constraint name="parking brakes" start="&ON;" end="&OFF;"/> <constraint name="parking brakes" start="&ON;" end="&OFF;"/> <constraint name="wheel brakes" start="&OFF;"/> <constraint name="wheel brakes" start="&OFF;"/> …… …… </constraints> </constraints> <additionalrules additionalrules> > < <rule name="altitude" end=">0" probability="1" /> <rule name="altitude" end=">0" probability="1" /> <rule name="roll" end=">0" probability="1" /> <rule name="roll" end=">0" probability="1" /> …… …… 18 18 </ </additionalrules additionalrules> > </situation> </situation>

Results: overview Results: overview ! Error rate: time wrong/total time Error rate: time wrong/total time ! ! 4 flights 4 flights ! " Average: error rate 0.08 Average: error rate 0.08 " " Lowest: error rate 0.05 Lowest: error rate 0.05 " " Highest: error rate 0.11 Highest: error rate 0.11 " 19 19

20 20