Situation recognition as a step to an intelligent Situation - - PowerPoint PPT Presentation

Faculty of Electrical Engineering, Mathematics, and Computer Science Delft University of Technology The ICE project

Situation recognition as a step to an intelligent Situation recognition as a step to an intelligent situation situation-

• aware crew assistant system

aware crew assistant system

Quint Mouthaan Quint Mouthaan Patrick Ehlert Patrick Ehlert Leon Rothkrantz Leon Rothkrantz October 23, 2003 October 23, 2003 -

• BNAIC,

BNAIC, Nijmegen Nijmegen

2 2

Contents Contents

! ! The ICE project

The ICE project

! ! Design

Design

! ! Implementation

Implementation

! ! Results

Results

! ! Conclusions and future work

Conclusions and future work

3 3

ICE: the project ICE: the project

! ! Ultimate goal: artificial co

Ultimate goal: artificial co-

• pilot

pilot

" " Take over tasks

Take over tasks

" " Provide useful information

Provide useful information

! ! Main focus on reasoning

Main focus on reasoning

4 4

ICE: architecture ICE: architecture

ICE system Aircraft avionics Crew Interface

actuator commands

Decision- making

Situation assessment

environment state

System management

crew commands system settings systems information

Data fusion

interface commands

5 5

Design: situation recognition Design: situation recognition

! ! Detect current situation based on data

Detect current situation based on data from simulator from simulator

" " Aircraft status

Aircraft status

" " Pilot actions

Pilot actions

" " Environment

Environment

! ! Probabilistic approach

Probabilistic approach

6 6

Design: architecture Design: architecture

Flight simulator Knowledge base Flight plan

Input module Knowledge converter Flight plan interpreter Rule base Overall controller

Situation

7 7

Design: overall controller Design: overall controller

8 8

Design: start probabilities Design: start probabilities

Situation

• ccurring

noisy-OR noisy-AND P sc Startconditions Rule1 Pcr1 Rulen Pcrn Situation started P s Previous situation ended P end P start P cond Action Action Pca1 Pcan 1 n

9 9

Design: end probabilities Design: end probabilities

Situation finished noisy-OR noisy-AND Endconditions P c Rule1 Pcr1 Rulen Pcrn Situation ended P e Situation started P s P end P cond Time window Pt

10 10

Implementation Implementation

! ! Java

Java

! ! MSFS 2002 Simulator

MSFS 2002 Simulator

! ! Rule

Rule-

• based system JESS

based system JESS

! ! Knowledge base for Cessna and F

Knowledge base for Cessna and F-

• 16

16

! ! Hard

Hard-

• coded probabilistic reasoning

coded probabilistic reasoning

11 11

Results: example experiment (1) Results: example experiment (1)

An attack on a ground target: An attack on a ground target:

123 123 123 123 Navigating Navigating 91 91 91 91 Normal flight Normal flight 83 83 83 83 Navigating Navigating 34 34 43 43 Normal flight Normal flight 15 15 14 14 Taking off Taking off 12 12 10 10 Taxiing to runway Taxiing to runway Startup Startup

Time Time detected (s) detected (s) Time Time started (s) started (s) Situation Situation

12 12

Results: example experiment (2) Results: example experiment (2)

An attack on a ground target: An attack on a ground target:

Time Time detected (s) detected (s) Time Time started (s) started (s) Situation Situation

427 427 427 427 Shutdown Shutdown 410 410 409 409 Taxiing from runway Taxiing from runway 410 410

• Aborting landing

Aborting landing 366 366 361 361 Landing Landing 221 221 221 221 Normal flight Normal flight 193 193 186 186 Visual attack Visual attack 128 128 128 128 Normal flight Normal flight

13 13

Conclusions Conclusions

! ! Probabilistic model performs fairly well

Probabilistic model performs fairly well

! ! Few mistakes, able to correct

Few mistakes, able to correct

! ! Recognize situations in real

Recognize situations in real-

• time for

time for Cessna and F Cessna and F-

• 16

16

! ! First step towards situation

First step towards situation-

• awareness

awareness system system

14 14

Future work Future work

! ! Compare with other approaches

Compare with other approaches

! ! Feedback from flight plan

Feedback from flight plan

! ! Add other causal relationships

Add other causal relationships

16 16

The F The F-

• 16 cockpit

16 cockpit

17 17

Design: probabilistic network Design: probabilistic network

gear throttle approach radio noisy-OR: 1 – (1 – P1) * (1 – P2) P approach landing

Planding

P(approach|gear) P(approach|throttle) P radio noisy-AND: P = P1 * P2 P( P(approach approach) ) throttle throttle gear gear 0.84 0.84 T T T T 0.8 0.8 F F T T 0.2 0.2 T T F F F F F F

18 18

Implementation: XML file F Implementation: XML file F-

• 16

16

<situation name="Taxiing to runway" <situation name="Taxiing to runway" timewindow timewindow="5"> ="5"> < <nextsituations nextsituations> > < <nextsituation nextsituation>Taking off</ >Taking off</nextsituation nextsituation> > </ </nextsituations nextsituations> > <actions> <actions> <phase name="taxiing"> <phase name="taxiing"> <action name="parking brakes" priority="1" probability="MP">&OFF <action name="parking brakes" priority="1" probability="MP">&OFF;</action> ;</action> <action name="throttle" priority="1" probability="SP">>&IDLE;</a <action name="throttle" priority="1" probability="SP">>&IDLE;</action> ction> <action name="throttle" priority="0.4" probability="SP">&IDLE;</ <action name="throttle" priority="0.4" probability="SP">&IDLE;</action> action> </phase> </phase> <phase name="time independent"> <phase name="time independent"> <action name="wheel brakes" priority="0.6" probability="MP">&ON; <action name="wheel brakes" priority="0.6" probability="MP">&ON;</action> </action> <action name="MPO" priority="1" probability="MP">&NORMAL;</actio <action name="MPO" priority="1" probability="MP">&NORMAL;</action> n> </phase> </phase> </actions> </actions> < <visualChecks visualChecks> > <instrument name="speed brakes" repetitive="no" priority="0.3"/> <instrument name="speed brakes" repetitive="no" priority="0.3"/> <instrument name="caution panel" repetitive="no" priority="0.8"/ <instrument name="caution panel" repetitive="no" priority="0.8"/> > …… …… </ </visualChecks visualChecks> > <constraints <constraints startProbability startProbability="MP" ="MP" endProbability endProbability="BP"> ="BP"> <constraint name="parking brakes" start="&ON;" end="&OFF;"/> <constraint name="parking brakes" start="&ON;" end="&OFF;"/> <constraint name="wheel brakes" start="&OFF;"/> <constraint name="wheel brakes" start="&OFF;"/> …… …… </constraints> </constraints> < <additionalrules additionalrules> > <rule name="altitude" end=">0" probability="1" /> <rule name="altitude" end=">0" probability="1" /> <rule name="roll" end=">0" probability="1" /> <rule name="roll" end=">0" probability="1" /> …… …… </ </additionalrules additionalrules> > </situation> </situation>

19 19

Results: overview Results: overview

! ! Error rate: time wrong/total time

Error rate: time wrong/total time

! ! 4 flights

4 flights

" " Average: error rate 0.08

Average: error rate 0.08

" " Lowest: error rate 0.05

Lowest: error rate 0.05

" " Highest: error rate 0.11

Highest: error rate 0.11

20 20