Attention detection in driver simulator project #4 INPG : L. - - PowerPoint PPT Presentation

▶
attention detection in driver simulator
SMART_READER_LITE
LIVE PREVIEW

Attention detection in driver simulator project #4 INPG : L. - - PowerPoint PPT Presentation

Feb 20, 2024 406 likes •615 views

Attention detection in driver simulator project #4 INPG : L. Bonnaud, A. Caplier UCL : D. Trevisan, B. Macq Participants : A. Benoit, G. Chanel, P. Ngo, V. Levacic, C. Thillou Guest star : L. Lawson, Burak Introduction Heart

slide-1
SLIDE 1

Attention detection in driver simulator

project #4

  • INPG : L. Bonnaud, A. Caplier
  • UCL : D. Trevisan, B. Macq
  • Participants : A. Benoit, G. Chanel, P. Ngo,
  • V. Levacic, C. Thillou
  • Guest star : L. Lawson, Burak
slide-2
SLIDE 2

8/19/2005 2

Introduction

Heart rate EEG

slide-3
SLIDE 3

8/19/2005 3 Head motion detector EEG Eye Blinking detector Mouth motion detector ECG Redundancy / Equivalence Head inclination ? Yawn ? frequency

Physiological sensors Camera

Loss of watchfulness EEG interpreter ECG interpreter Dialog controller Screen Speakers Redundancy / Equivalence Sonic representation Graphical epresentation Alarm (presentation task) EEG: electroencephalogram ECG: electrocardiogram

Provided To be developped

slide-4
SLIDE 4

8/19/2005 4

Architecture

slide-5
SLIDE 5

8/19/2005 5

5 Challenges

  • Driver Simulator
  • Attention detection

– Biological signals – Stress detection – Video-based information – Fatigue detection

  • Fusion
  • Fission
  • Integration

– Distributed architecture – OpenInterface

slide-6
SLIDE 6

8/19/2005 6

Driver Simulator

slide-7
SLIDE 7

8/19/2005 7

Driver Simulator

  • TORCS - GPL program well built with source code

well structured (C++ and OpneGl)

  • Force Feedback with controlled level of wheel

vibration

  • Message (color)
  • Button click (user’s interaction)
  • Multi-thread server
  • The network protocol used is TCP/IP. We used a

“GPL” library called Openthreads to allow threads access global variables with a Mutual Exception implementation

slide-8
SLIDE 8

8/19/2005 8

Attention Detection

  • Video-based system
  • Fatigue detection

– Eyes – Yawn – Head movement

slide-9
SLIDE 9

8/19/2005 9

Attention Detection

  • Biological-based system (stress detection)
  • ECG and GSR
  • 3 situations :

– rest / relaxation – Stress stimuli while reading

  • Hand clapping
  • Light in eyes
  • Answering simple question
  • Telephone call, “your dead” (killer game)

– Playing with the driving simulator (difficult tracks)

  • GSR acquisition and analysis can be integrated in real

time

  • New experiments for detecting relax situation
slide-10
SLIDE 10

8/19/2005 10

Fusion

Video image

Contextual Information Physical fitness Sleep history Time of day Temperature

Face-based detection

Head, eyelid and yawn movement

Data Fusion

Bayesian Network

Fatigue ?

Activate Data Fission component No Yes Video image

Contextual Information Physical fitness Sleep history Time of day Temperature

Face-based detection

Head, eyelid and yawn movement

Data Fusion

Bayesian Network

Fatigue ? Fatigue ?

Activate Data Fission component No Yes

Bayesian Network for detecting Fatigue State

slide-11
SLIDE 11

8/19/2005 11

Fusion

Bayesian Network for detecting Fatigue State

Contextual information Prior probability video-based information

Qiang Ji, Zhiwei Zhu and Peilin Lan, Real-Time Nonintrusive Monitoring and Prediction of Driver Fatigue, IEEE Transactions on Vehicular Technology, Vol. 53, No. 4, July, 2004, p1052-1068].

slide-12
SLIDE 12

8/19/2005 12

Data fission responsibility is to collect the data from data fusion and to generate a XML message that is sent to the driver simulator « 100 » « 0 » « 0 » Shaking power « Red » « Green » « » Message color « A sleep » « Tired » « » Message [66,100] [33,66] [0,33] Fatigue range

Fission

slide-13
SLIDE 13

8/19/2005 13

Architecture

slide-14
SLIDE 14

8/19/2005 14

OpenInterface Integration

OpenInterface Kernel C++ C++ C++ C++ CIDL XML CIDL XML CIDL XML Mouse Component C++ Speech Component Java Image Viewer Component Java Graphic Editor (Java) Execution pipeline Components properties Components properties OpenInterface Kernel C++ C++ C++ C++ CIDL XML CIDL XML CIDL XML Mouse Component C++ Speech Component Java Image Viewer Component Java Graphic Editor (Java) Execution pipeline Components properties Components properties

Each component is registered into OpenInterface Platform using the Component Interface Description Language (CIDL) and described in XML The registered components properties are retrieved by the Graphic Editor (Java).

Camera Driver

C++

Face detection

MatLab

Start Fusion for fatigue Bayes Network

MatLab

F i s s i

  • n

O P E N I N T E R F A C E O P E N I N T E R F A C E

MatLab

Camera Driver

C++

Face detection

MatLab

Start Fusion for fatigue Bayes Network

MatLab

F i s s i

  • n

O P E N I N T E R F A C E O P E N I N T E R F A C E

MatLab

slide-15
SLIDE 15

8/19/2005 15

Future Works

  • Integrate biological signals for fatigue

detection

  • Usability tests to assess interface interactions.
  • Improve the Bayesian Network to take account

more specialized information about head

  • rientation once this information is available in

the head detection code.

  • Transform the face detection component into 3

OpenInterface components.

slide-16
SLIDE 16

8/19/2005 16

Conclusion

1st goal: real time distributed system based on video data, integrated under OpenInterface, for driver attention level analysis with feed-back to the user 2nd goal: multimodal system taking into account the biological signals - Stress

slide-17
SLIDE 17

8/19/2005 17

TEAM

slide-18
SLIDE 18

8/19/2005 18

TEAM

slide-19
SLIDE 19

8/19/2005 19

DEMO

  • Have Fun and

come to play during our Demo session!

slide-20
SLIDE 20

8/19/2005 20

Questions?