Wearable Technology: the wave of the future Omid Dehzangi Computer - - PowerPoint PPT Presentation

Omid Dehzangi

Computer and Information Science University of Michigan - Dearborn Wearable Sensing and Signal Processing Lab

Wearable Technology: the wave of the future

Outline

Introduction to wearable technology Vision and mission

• Application and high level model design
• Wearable platform design and development

My research contributions

• Brain-computer interface
• Activity of daily living (ADL) monitoring

My current research plans

2

Technology Trends

 Transistors Digital Processing Brought to homes

3

Analog computer Personal Computers Wearable Computers Today’s Computers  Smaller Slimmer Faster Hand held  Smarter Hands free Natural Interface

Wearable Technology

 ABI Research has projected that by 2016, wearable wireless device sales will reach more than 100 million devices annually.  The market for wearable sports, fitness, and healthcare monitoring devices cover 80% of it.  The market for wearable technologies in healthcare "is projected to exceed $2.9 billion in 2016 (at least half of all wearable technology)

4

Photo courtesy of http://www.phonearena.com/

Vibrotactile Modules

Wearable Computers

5

Phantom

Photo Courtesy of SenseGraphics

Haptics Deep Brain Stimulation

Photo Courtesy of mindmodulation.com

GUI-based feedback

Feedback Sensors

Dry-contact EEG Inertial Sensors Galvanic Skin Response Flex, Pressure and Piezo-electric Sensors

Wearable Computers

6

Sensors Processing Unit Communication Information Fusion Prediction/Detection

Data Analytics

Big data analysis Data mining Machine learning Predictive modeling Statistical analysis

Signal Processing Feedback

Outline

Introduction to wearable technology

 Vision and mission

• Application and high level model design
• Wearable platform design and development

7

Research Vision

Applications Model design Algorithms and analytics System integration Technologies  Demonstrate the linkage between discovery and societal benefit  Validate real pains and necessities and identify effective high level solutions  Design and develop in multiple technical levels  Resolve upcoming challenges in practice  Generate transitioning technologies

Wearable platform

9

Wireless Health

Ubiquitous monitoring and intervention for the applications of health-care and wellness

9 Courtesy of Misha Pavel, Program Director, National Science Foundation

Outline

Introduction to wearable technology Vision and mission

• Application and high level model design
• Wearable platform design and development

 My current research contributions

• Brain-computer interface
• Activity monitoring and motion detection

10

WEARABLE BRAIN COMPUTER INTERFACE

Application Case Study

11

Applications

Brain Computer Interface

• Brain Computer Interface

– Provide a non-muscular avenue for the user to communicate with

• thers and to control external

devices – Infer user’s intentions using brain activities

• Applications

– Assist locked-in individuals to interact with cyber and physical system – Gaming – Diagnosis and treatment for neurological disorder

12

Wearable EEG Systems

• Smaller form factor (size
• f a credit card vs. bulky

amps)

• Quicker setup time

(seconds vs. 30 mins)

• Faster software training

(5 mins vs. 30 mins)

• Quicker EEG signal

detection (seconds vs. minutes)

• No need for EEG tech

13

 Custom-designed mobile EEG-based BCI

• Dry-contact electrodes
• Low-noise front-end (ADS1299)
• Low power processing (MSP430)
• Low component count
• Bluetooth low energy (TI BLE)

communication module

Wearable EEG-Based BCI

14

Wearable BCI Units

15

Canonical Correlation Analysis(CCA)

16

Picture taken from ref.1

Video

17

USING GAIT AND SWAY BIOFEEDBACK TO REDUCE FALLS IN THE ELDERLY

17

Applications

ACTIVITY OF DAILY LIVING MONITORING

• Dehzangi, Omid, Biggan, John, Birjandtalab Golkhatmi, Javad, Ray, Christopher, Jafari,

Roozbeh, “An Inertial Sensor-Based Method for Early Detection and Prevention of Excessive Sway in Older Adults via Gait Analysis and Vibrotactile Biofeedback”, Gait & Posture journal.

• Dehzangi, Omid, Zhao, Zheng, Biggan, John, Ray, Christofer, Jafari, Roozbeh, “The

Impact of Vibrotactile Biofeedback on the Excessive Walking Sway and the Postural Control in Elderly”, Wireless Health 2013, November 1-3, Baltimore, Maryland, 2013.

Application Case Study

Postural control and gait analysis

18

Sway Biofeedback for Fall Prevention

 Fall is a considerable health concern in the elderly  Wearable kinematic biofeedback system to detect pre-cursors of falls based on the sway of the upper body and other gait parameters, and activate biofeedback

19

20

Hardware Architecture

Laptop UART BLE transceiver Microprocessor Motion Sensor: Gyro & Accelerometer BLE I2C

21

Software Architecture

Accelerometer X,Y,Z

Sensor Calibration Drift Detection PI controller DCM Self-adaptive Angle Threshold Setting

Calibrated Accelerometer X,Y,Z Calibrated Gyroscope X,Y,Z

Feedback loop

Euler Angles (roll, pitch) X,Y,Z X,Y,Z Accelerometer X,Y,Z

Gyroscope

The developed system

22

(a) (b) (a) Our designed wearable low-power motion sensor board, (b) Our biofeedback system, consisting of two motion sensor boards for the chest and the ankle along with the vibratory feedback modules.

 Subjects:

24 older adults (age: M = 75.5, SD = 4.32 years; 10 females)

Procedure:

23

Experiments

24

Mean difference in the sway range.

The test Control Experimental P-value Difference in the sway range 0.59±1.77

• 0.60±0.63

0.04

The results of the statistical test on the sway range

Results and Analysis

25

Identification of the gait phases on the ACC X readings

Gait phase analysis

Initial sway Mid sway Terminal sway Mid stance

The selected phases of a gait cycle.

26

DTW extracted strides based on ACC X readings (Experimental)

Gait phase analysis

5 10 15 20 25

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 x 10

4

Sample ACC X reading

5 10 15 20 25 30

• 2
• 1.5
• 1
• 0.5

0.5 1 1.5 2 x 10

4

ACC X reading Sample

DTW extracted strides based on ACC X readings (Control)

27

Mean difference in the variance of the gait phases between pre- and post- training.

Gait phase analysis

The gait phases Control Experimental P-value Initial sway 0.17±0.62 0.40±0.15 0.09 Mid sway

• 1.54±1.72
• 0.03±0.48

0.08 Terminal sway

• 1.29±1.04

0.038±1.01 0.05 Mid stance

• 0.18±0.89
• 0.07±1.14

0.27

The results of the Chi-square test on the gait phases

Initial sway Mid sway Terminal sway Mid stance

Outline

Introduction to wearable technology Vision and mission

• Application and high level model design
• Wearable platform design and development

My current research contributions

• Brain-computer interface
• Activity of daily living (ADL) monitoring

My Current Research Plans

28

WEARABLE DRIVER MONITORING

Application Case Study

29

Applications

Goal

• To form relationships between biological state
• f the driver with his/her driving behavior

37

31

Motivation:

Multi-Modal Driver Monitoring and Modeling via Heterogeneous Wearable Body Sensor Network

System photograph:

a) b) c) Integration of heterogeneous wearable monitoring technology, on-board sensing units, and wireless networking capabilities : a) The full body sensor network, b) the portable EEG system, c) the OBD-II device

Body sensor networks are capable of generating a reliable human state model

Hypotheses:

• 1. Minimally intrusive: Driver behavior is not affected by

the devices that are used to acquire the necessary biomedical markers

• 2. Comprehensive: the system will extract the data

collected from a large number of heterogeneous sensors and correlate the various readings for earlier detection

• 3. Ubiquitous and remotely available: The collected

measurements will be transmitted to a remote location for longitudinal analysis and discover association in a long term

• 4. Real--time responsive: The information will be

accessible in an online fashion to enable real--time processing and decision- making

• 5. User- friendly: Suitable user interface and

visualization tools will be in place for a human user to be able to interpret the acquired information Hypothesis 1: Specific driver mental and physical states can generate abnormal driving behaviors and a high level of driving impairment. Hypothesis 2: Driver biological states will have an impact on his/her biometric measures while

• driving. Biometric markers that correspond to

changes in performance of the impaired driver subjects will aid in explaining the underlying impact on driving outcome. Hypothesis 3: There are signature patterns in the biometric readings from the normal behavior

• f the driver that can be non-invasively extracted

and employed for control, identification & authentication, and interaction with other smart infrastructures.

The Proposed Platform:

Multi-Modal Driver Monitoring and Modeling via Heterogeneous Wearable Body Sensor Network

39

Multi-Modal Driver Monitoring and Modeling via Heterogeneous Wearable Body Sensor Network

Real-time Processing EEG ECG OBD-II

DAQ-FrontEnd

Data Mining Sensors In-Vehicle Mobile Device Long-term analysis Data Base Big Data Analysis Backend Processing Bio-feedback Data Visualization GSR Data Matrix User Interface Driver

DAQ-BackEnd

GPS Traffic BlueTooth Mobile Network

40

Platform User Interface

34

Characterizing Driver Distraction

30

Characterizing Driver Distraction:

Two rounds of driving:

• 1. Non-peak traffic period,
• 2. Peak traffic period

Objective: Investigate the effect of the road condition

• n the driver distraction

Hypothesis: Theta and beta power increase in the EEG spectrum is related to distraction effects

45

Characterizing Driver Distraction

30

Comparison of total theta and beta power (dB×103) in the subjects averaged EEG in frontal component between round 1 and round 2 Subjects Round 1 Round 2 theta beta theta beta Sbj 1 1.8 1.2 1.5 1.2 Sbj 2 2.8 1.6 1.7 1.3 Sbj 3 2.5 1.5 1.3 1.1 Avg 2.33 1.5 1.48 1.27

47

Characterizing Driver Distraction

48

• 0.1
• 0.05

0.05 0.1 0.15

• 0.1
• 0.05

0.05 0.1 0.15 x y pdf(obj,[x,y])

• 0.1
• 0.05

0.05 0.1 0.15

• 0.1
• 0.05

0.05 0.1 0.15 x pdf(obj,[x,y])

the effect of the driving condition on the driver distraction

• 0.1
• 0.05

0.05 0.1 0.15

• 0.1
• 0.05

0.05 0.1 0.15 x pdf(obj,[x,y])

• 0.1
• 0.05

0.05 0.1 0.15

• 0.1
• 0.05

0.05 0.1 0.15 x pdf(obj,[x,y])

38

Ideas to Pursue

 Towards proactive driver monitoring and safety platform as advancement in automated passenger vehicle infrastructure.  Connect the vehicle occupants to the loop via development of D2V & D2I in automated vehicles to improve occupant safety, performance, health & wellness.  The connected vehicle infrastructure will associate the driver with the smart city and/or smart home infrastructure to optimize his/her daily

• perations.

 Driver identification and authentication is an important outcome which will be performed non-invasively via extracting the signature biometrics from the normal behavior of the driver.

51

Thanks for your attention

39