and Electrooculography Features Ruofei Du 1 , Renjie Liu 1 , - - PowerPoint PPT Presentation

▶
and electrooculography features
SMART_READER_LITE
LIVE PREVIEW

and Electrooculography Features Ruofei Du 1 , Renjie Liu 1 , - - PowerPoint PPT Presentation

Oct 21, 2023 253 likes •461 views

Online Vigilance Analysis Combining Video and Electrooculography Features Ruofei Du 1 , Renjie Liu 1 , Tianxiang Wu 1 , Baoliang Lu 1234 1 Center for Brain-like Computing and Machine Intelligence Department of Computer Science and Engineering 2

slide-1
SLIDE 1

Online Vigilance Analysis Combining Video and Electrooculography Features

Ruofei Du1, Renjie Liu1, Tianxiang Wu1, Baoliang Lu1234

1 Center for Brain-like Computing and Machine Intelligence

Department of Computer Science and Engineering

2 MOE-Microsoft Key Lab. for Intelligent Computing and Intelligent Systems 3 Shanghai Key Laboratory of Scalable Computing and Systems 4 MOE Key Laboratory of Systems Biomedicine

Shanghai Jiao T

  • ng University

800 Dongchuan Road, Shanghai 200240, China

ICONIP 2012 Shanghai Jiao T

  • ng University
slide-2
SLIDE 2

Outline

  • Motivation
  • Introduction
  • System Overview
  • Video Features
  • Electrooculography
  • Conclusion and Future Work

ICONIP 2012 Shanghai Jiao T

  • ng University
slide-3
SLIDE 3

Motivation

  • 60

600, 0, 00 000 0 people die from traffic accidents every year, and

  • 10

10,0 ,000 00,0 ,000 00 people get injured throughout the world.

  • 60

60% % of adult drivers – about 16 168 million people – say they have

driven a vehicle while feeling drowsy in 2004 in the U.S. Drowsy driving results in 550

550 deaths, 71 71,0 ,000 00 injuries, and $1 $12. 2.5 billion

in monetary losses.

  • In China, 45

45.7 .7% % accidents on the highway are caused by fatigued

driving.

ICONIP 2012 Shanghai Jiao T

  • ng University
slide-4
SLIDE 4

Introduction

ICONIP 2012 Shanghai Jiao T

  • ng University

Vid Video eo EO EOG EEG EEG Intrusive

Least Moderate Most

Accuracy

Moderate, influenced by luminance Most accurate Moderate, need to denoise.

Features

Eye movement, yawn state and facial orientation. Eye blinks, movement and energy. Delta waves (Slow Wave Sleep) and theta waves (drowsiness)

slide-5
SLIDE 5

System Overview

ICONIP 2012 Shanghai Jiao T

  • ng University
slide-6
SLIDE 6

System Overview

ICONIP 2012 Shanghai Jiao T

  • ng University

Train T est

slide-7
SLIDE 7

System Overview

ICONIP 2012 Shanghai Jiao T

  • ng University

Black screen 5~7s stimulus 500ms One trial

slide-8
SLIDE 8

System Overview

ICONIP 2012 Shanghai Jiao T

  • ng University
slide-9
SLIDE 9

Visual Features

ICONIP 2012 Shanghai Jiao T

  • ng University
  • Video signals: By infrared cameras, 640×480, 30 frames/s
  • Face Detection: Haar-like cascade Adaboost classifier.
  • Active Shape Model: Locate the landmarks on the face.
slide-10
SLIDE 10

Visual Features

ICONIP 2012 Shanghai Jiao T

  • ng University
  • PERCLOS (percentage of closure):
  • Blink frequency, etc.:
  • Yawn frequency:
  • Body Posture: (By ASM)
slide-11
SLIDE 11

Linear Dynamic System

ICONIP 2012 Shanghai Jiao T

  • ng University

𝑄 𝑦𝑢 𝑨𝑢 = 𝑂 𝑦𝑢 𝑨𝑢 + ഥ 𝑥, 𝑅 𝑄 𝑨𝑢 𝑨𝑢−1 = 𝑂 𝑨𝑢 𝐵𝑨 𝑢−1 + ҧ 𝑤, 𝑆

slide-12
SLIDE 12

Electrooculography

ICONIP 2012 Shanghai Jiao T

  • ng University
slide-13
SLIDE 13

Forehead Signals Separated by ICA

HEO VEO

ICONIP 2012 Shanghai Jiao T

  • ng University
slide-14
SLIDE 14

Electrooculography

ICONIP 2012 Shanghai Jiao T

  • ng University
  • Filter the vertical EOG signal by a low-pass filter with a frequency of 10Hz.
  • Adjust the amplitude of the signals.
  • Computer the difference of signals for the extraction of eye blinks.
  • 𝐸 𝑗 = 𝑊 𝑗 + 𝑗 − 𝑊 𝑗

× 𝑆

  • where V denotes the signal, R as the sampling rate
  • Slow Eye Movement (SEM) and Rapid Eye Movement (REM) are extracted

according to different kinds of time threshold.

  • Fourier transformation: 0.5Hz and 2Hz to process the horizontal EOG.
  • The sampling rate: 125Hz, time window: 8 seconds.
slide-15
SLIDE 15

Electrooculography

ICONIP 2012 Shanghai Jiao T

  • ng University
slide-16
SLIDE 16

Conclusion

ICONIP 2012 Shanghai Jiao T

  • ng University
slide-17
SLIDE 17

Conclusion

ICONIP 2012 Shanghai Jiao T

  • ng University
slide-18
SLIDE 18

Future Work

ICONIP 2012 Shanghai Jiao T

  • ng University
  • Smaller EOG chip: to
  • Comprehensive feature: depth information and grip power.
  • Robustness and stability:

Various luminance, moving car, actual environment...

slide-19
SLIDE 19

Thank you

BCMI: We are family! http://bcmi.sjtu.edu.cn

ICONIP 2012 Shanghai Jiao T

  • ng University
slide-20
SLIDE 20

Reference

  • 1. May,

, J., , Bald ldwin, in, C.: Driver r fa fatig igue ue: : The imp mportance tance of i f ide dentif tifyin ying g ca causa sal l fa fact ctor

  • rs

s of f fa fatig igue ue when co consi side dering ing de detect ctio ion n and co d counte terme measu asure tech chnolo

  • logi

gies.

  • s. Trans

nspo portation tation Re Rese search ch Part t F: Traffi fic Psychology and Behaviour 12(3) (November 2009) 218–224 224

  • 2. Stutt

tts, J., , Wilk lkins ins, J., Vaugh ghn, n, B.: Why do do pe peopl ple have dr drowsy sy dr driving ing cr crash shes.

  • s. A Fo

Foundat dation ion fo for Traffi fic Safety (202/638)

  • 3. W

Wang ang, Q., Yang, g, J., Re Ren, M., Zheng, g, Y.: Driver r fa fatigu tigue e de detect ctio ion: n: a su

  • survey. Intellig

ligent nt Control

  • l and

d Automat tomation ion, 2006. WCICA A 2006. The Sixth th World rld Congr gress ss on 2 (2006) 8587–8591 8591

  • 4. J

Ji, , Q., Yang, g, X.: Re Real-time time eye, ga gaze, and fa d face ce po pose se track acking ing fo for mo monito itoring ring dr driver vigi gilanc lance. . Re Real-Time Time Ima magi ging ng 8(5) ( (2002) 357–377

  • 5. Dinge

ges, D., Malli llis, M., Mais islin lin, G., Powell, ll, I., et al.: : Fi Final al repo port: t: Evaluation aluation of f tech chniq nique ues s fo for ocu cular ar me measu surem ement nt as s an inde dex of fa f fatig igue ue and as d as the ba basi sis s fo for alertnes tness s ma manage ageme ment nt. . National tional Highway Traffi fic Safety Administration (HS 808762) (1998)

  • 6. F

Fletche tcher r LAp Apostol stoloffn ffnPete terso son L, L, e.a e.a.: : Visi sion n in and o d out of v f vehicl cles.

  • s. IEEE Trans

ns on Intelli llige gent nt Trans anspo porta tation tion Syst stems ms 18(3)

  • 7. Ma, J., Shi, L.,

, Lu, , B.: Vigi gilanc ance est stimation mation by by u usi sing g elect ctroo

  • ocu

culog lograph raphic ic fe featur ures.

  • s. Proce

ceedi ding ngs s of 3 f 32nd I d Internat rnation ional al Confe ferenc nce of th f the IEEE E Engi gine neerin ering g in Medi dici cine ne and d Biolo logy gy Soci ciety ty (2010) 6591–6594 6594

  • 8. W

Wei, i, Z., Lu, , B.: On Online ine vigi gilanc lance anay aysi sis ba base sed d on elect ctroo

  • ocu

culog lograph raphy. . Internat rnatio iona nal l Joint int Confe ferenc nce on Neural ral Network works ( s (2012)

  • 9. Cootes

es, T., , T aylor lor, , C., Coope per, , D., Graham, aham, J., et al.: : Activ ctive e sh shape pe mo mode dels s - their r traini ining ng and ap d appl plicat cation ion. . Comp mputer r Visi sion and Im d Image ge Unde derst stand anding ng 61(1) (1995) 38–59 59 10

  • 10. Viola,

la, P ., Jone nes, s, M.: : Ra Rapi pid d obj bject ct de detect ction

  • n usi

sing g a bo boost sted ca d casc scade de of si f simp mple fe featur tures

  • es. Proce

ceedi ding ngs s of C f Comp mputer r Visi sion and P d Patt ttern rn Re Reco cogn gnition ition 1 ( (2001) I–511 511 – I–51810 51810 Ru Ruo-Fe Fei i Du, Re Ren-Ji Jie Liu, u, Tian an-Xiang Xiang Wu, , Bao-Lian Liang g Lu

  • 11. Delorme

rme, A., , Make keig ig, , S.: Eegl glab ab: an ope pen so source ce toolb lbox x fo for analy alysi sis o s of si f singl gle-tria trial l eeg eeg dy dynami mics cs incl clud uding ing inde depe pende dent nt co comp mponent t analys alysis. s. Journ urnal l of n f neuros

  • sci

cienc nce me methods ds 134(1) (2004) 9–21 21

  • 12. Dinge

ges, D., Grace ce, , R. R.: Percl clos

  • s: A valid

lid ps psych choph physi siolo

  • logi

gical cal me measu sure of a f alertnes tness as s as ass ssess ssed d by by ps psych chomo motor tor vigi gilanc lance. . Fe Fede deral l High ghway way

  • Administration. Offi

fice of motor carriers, T ech

  • ch. Re
  • Rep. MCRT

RT-98 98-00 006 (1998)

  • 13. A. Bullin

ling, g, J. Ward, d, H.G., , Trost ster er, , G.: Eye mo moveme ment nt analys alysis s fo for act ctivity vity reco cogn gnition ition usi sing g elect ctroo

  • ocu

culog lograph raphy. . Pattern ttern Analys alysis s and d Mach chine ine Intellige ligenc nce, , IEEE E Trans ansactio ctions ns (99) ( (2011) 1–1

  • 14. Shi, L., Lu, B.: Of

Off-line ine and o d on-lin line vigi gilanc lance est stimation mation ba base sed o d on linear ear dy dynamical mical sy syst stem an m and d ma manifold fold learnin rning.

  • g. Engi

gine neering ring in Medi dici cine ne and d Biolog logy Soci ciety ty (EMBC BC), , 2010 Annu nnual al Internat rnation ional al Confe ferenc nce of th f the IEEE E (Sept ptemb mber 2010) 6 6587–6590 6590

  • 15. Chang

ng, C., Lin, n, C.: Libs bsvm: : a libr brar ary y fo for su supp pport vect ctor r ma mach chines

  • nes. ACM Trans

nsactio actions ns on Intellig ligent nt Syst stems ms and d T ech chnolog logy y (TIST) 2(3) ( (2011) 27

ICONIP 2012 Shanghai Jiao T

  • ng University