Mobile Health Monitoring health and well-being using mobile devices, - - PowerPoint PPT Presentation

▶
mobile health
SMART_READER_LITE
LIVE PREVIEW

Mobile Health Monitoring health and well-being using mobile devices, - - PowerPoint PPT Presentation

Oct 12, 2022 345 likes •1.1k views

6.808: Mobile and Sensor Computing aka IoT Systems Lecture 11: Mobile Health Mobile Health Monitoring health and well-being using mobile devices, wearable sensors, and smart environments Applications: What do we want to measure? And why?

slide-1
SLIDE 1

Lecture 11: Mobile Health

6.808: Mobile and Sensor Computing

aka IoT Systems

slide-2
SLIDE 2

Mobile Health

Monitoring health and well-being using mobile devices, wearable sensors, and smart environments

slide-3
SLIDE 3

Applications: What do we want to measure? And why?

Calories Sleep Mental & emotional well-being Steps Gait & activity Vitals (HR, breathing) Many others: Hb, skin, etc.

slide-4
SLIDE 4

How do we measure?

Digital pill: beyond measurement Voice Accelerometer Cameras (food, diseases) Log it Wireless reflections

slide-5
SLIDE 5

Background

slide-6
SLIDE 6

Personal Health

Elderly Health

6

Baby Sleep Adapt Lighting and Music to Mood

Can smart homes monitor and adapt to our breathing and heart rates?

slide-7
SLIDE 7

But: today’s technologies for monitoring vital signs are cumbersome

Breath Monitoring

Heart Rate Monitoring

Not suitable for elderly & babies

slide-8
SLIDE 8

Can we monitor breathing and heart rate from a distance?

8

slide-9
SLIDE 9

Vital-Radio

  • Technology that monitors breathing and

heart rate remotely with 97% accuracy

  • Can monitor multiple users

simultaneously

  • Operates through walls and can cover

multiple rooms

9

slide-10
SLIDE 10

Idea: Use wireless reflections off the human body

slide-11
SLIDE 11

Idea: Use wireless reflections off the human body

Wireless device

dexhale

slide-12
SLIDE 12

Wireless device

dexhale dinhale

Device analyzes the wireless reflections to compute distance to the body Problem: Localization accuracy is only 12cm and cannot capture vital signs

Why? How did we compute the resolution?

slide-13
SLIDE 13

Wireless device

dexhale dinhale

Device analyzes the wireless reflections to compute distance to the body Problem: Localization accuracy is only 12cm and cannot capture vital signs

Why does phase allow us to get the distance at higher granularity?

Solution: Use the phase of the wireless reflection

slide-14
SLIDE 14

Wireless device

dexhale dinhale φ = 2π distance wavelength

Wireless wave has a phase:

  • Chest Motion changes distance
  • Heartbeats also change distance

Device analyzes the wireless reflections to compute distance to the body Problem: Localization accuracy is only 12cm and cannot capture vital signs Solution: Use the phase of the wireless reflection Why did we need FMCW if phase is so accurate?

slide-15
SLIDE 15
slide-16
SLIDE 16

Let’s zoom in on these signals

slide-17
SLIDE 17

Exhale Inhale Heartbeats

How do we get from here to extracting breathing rate and heart rate?

slide-18
SLIDE 18

What happens when a person moves his limb?

slide-19
SLIDE 19

What happens when a person moves his limb?

Breathing Limb Motion Periodic Not periodic

Use periodicity test to eliminate variations that are not due to breathing/heartbeats Band-pass filter the cleaned signals to extract breathing and heart rate

slide-20
SLIDE 20

What happens with multiple users in the environment?

20

slide-21
SLIDE 21

Reflections from different objects collide

Reflection 1 Reflection 2

Problem: Phase becomes meaningless!

21

slide-22
SLIDE 22

Reflection 1 Reflection 2

22

Solution: Use WiTrack as a filter to isolate reflections from different positions

slide-23
SLIDE 23

Solution: Use WiTrack as a filter to isolate reflections from different positions

Bucket1 Bucket2 Bucket3

23

Reflection 1 Reflection 2

slide-24
SLIDE 24

24

Solution: Use WiTrack as a filter to isolate reflections from different positions

Bucket1 Bucket2 Bucket3

Analyze reflections in each bucket to

Reflection 1 Reflection 2

slide-25
SLIDE 25

Recall Formulation with FMCW

  • Output of FFT with reflectors
  • Looked at the amplitude only
  • Now will also look at phase

How do we deal with multipath?

slide-26
SLIDE 26

Putting It Together

Step 1: Transmit a wireless signal and capture its reflections Step 2: Isolate reflections from different

  • bjects based on their positions

Step 3: Zoom in on each object’s reflection to obtain phase variations due to vital signs

26

slide-27
SLIDE 27

Vital-Radio Evaluation

Vital-Radio’s antennas

slide-28
SLIDE 28

Vital-Radio Evaluation

Vital-Radio’s antennas

Pulse Oximite r Baseline:

  • FDA-approved breathing

and heart rate monitor Chest Strap Experiments:

  • 200 experiments
  • 14 participants
  • 1 million measurements
slide-29
SLIDE 29

Accuracy vs. Orientation

User is 4m from device, with different orientations

Forward Right Backward Left

Accuracy (%) 36.6667 73.3333 110 97.6 96.6 97.1 98.7 97.7 96.7 97.4 99.1 Breathing Rate Heart Rate

Why does it work when facing away?

slide-30
SLIDE 30

Accuracy for Multi-User Scenario

Multiple users sit at different distances

Nearest (at 2m) Middle (at 4m) Furthest (at 6m)

Accuracy (%) 36.6667 73.3333 110 98.7 98.9 98.7 98.2 97.3 99.4 Breathing Rate Heart Rate

slide-31
SLIDE 31

Accuracy for Tracking Heart Rate

Measured Heart Rate (beats/ minute) 50 65 80 95 110 Time (seconds) 20 40 60 80 100 120 Reference Vital-Radio

Vital-Radio accurately tracks changes in vital signs Measure user’s heart rate after exercising

slide-32
SLIDE 32

Vital-Radio Limitations

  • Minimum separation between users: 1-2m
  • Monitoring range: 8m
  • Collects measurements when users are

quasi-static

32

slide-33
SLIDE 33

Baby Monitoring

Works for multiple people and through walls

slide-34
SLIDE 34

Breathing & Heart Rate Want Emotions

slide-35
SLIDE 35

Recognizing Human Emotions

slide-36
SLIDE 36

Key challenge: Inter-Beat Interval (IBI)

  • Emotion recognition needs accurate measurements of

the length of every single heartbeat

  • We need to extract IBI with accuracy over 99%
slide-37
SLIDE 37

Input signal

Wireless reflection of the human body

slide-38
SLIDE 38

Step 1: Remove breathing signal

  • We use acceleration filter
  • Heartbeat involves rapid contraction of muscle
  • Breathing is slow and steady
  • Breathing masks heartbeats
slide-39
SLIDE 39

Heartbeat signal

  • Output of acceleration filter
  • ECG signal
slide-40
SLIDE 40

Heartbeat signal

  • Other typical examples:

How to segment the signal into individual heartbeats?

slide-41
SLIDE 41

Step 2: Heartbeat segmentation

  • Intuition: heartbeat repeats with certain shape (template)
  • If we can somehow discover the template, then we can

segment into individual heartbeats

slide-42
SLIDE 42

Step 2: Heartbeat segmentation

Template Update Segmentation Update Random template:

  • Intuition: heartbeat repeats with certain shape (template)
slide-43
SLIDE 43

Step 2: Heartbeat segmentation

Template Update Segmentation Update Random template:

  • Intuition: heartbeat repeats with certain shape (template)
slide-44
SLIDE 44

Step 2: Heartbeat segmentation

Template Update Segmentation Update Random template:

  • Intuition: heartbeat repeats with certain shape (template)
slide-45
SLIDE 45

Step 2: Heartbeat segmentation

Template Update Segmentation Update Random template:

  • Intuition: heartbeat repeats with certain shape (template)
slide-46
SLIDE 46

Step 2: Heartbeat segmentation

Template Update Segmentation Update Random template:

  • Intuition: heartbeat repeats with certain shape (template)
slide-47
SLIDE 47

Caveat: Shrinking & Expanding

  • IBI are not always the same
  • Template subject to shrink and expanding
  • Linear warping
slide-48
SLIDE 48

Algorithm

  • Joint optimization:

minimize

S,µ

X

si∈S

ksi ω(µ, |si|)k2

segmentation template warping

Segmentation Update

Sl+1 = arg min

S

X

si∈S

ksi ω(µl, |si|)k2

(dynamic programming) Template Update (weighted least squares)

µl+1 = arg min

µ

X

si∈Sl+1

ksi ω(µ, |si|)k2

Need to recover both segmentation and template

slide-49
SLIDE 49

Algorithm

  • Joint optimization:

minimize

S,µ

X

si∈S

ksi ω(µ, |si|)k2

segmentation template warping

Segmentation Update

Sl+1 = arg min

S

X

si∈S

ksi ω(µl, |si|)k2

(dynamic programming) Template Update (weighted least squares)

µl+1 = arg min

µ

X

si∈Sl+1

ksi ω(µ, |si|)k2

Need to recover both segmentation and template

  • Both updates have linear complexity
  • Each update achieves global optimum
  • Iterative algorithm is guaranteed to converge
slide-50
SLIDE 50

Example run

slide-51
SLIDE 51

Example run

Iteration 1:

Segmentation Template

slide-52
SLIDE 52

Example run

Iteration 2:

Segmentation Template

slide-53
SLIDE 53

Example run

Iteration 2:

Segmentation Template

slide-54
SLIDE 54

Example run

Iteration 3:

Segmentation Template

slide-55
SLIDE 55

Example run

Iteration 3:

Segmentation Template

slide-56
SLIDE 56

Example run

Iteration 7:

Segmentation Template

slide-57
SLIDE 57

Example run

Iteration 7:

Segmentation Template

ECG

slide-58
SLIDE 58

From vital signs to emotions

slide-59
SLIDE 59

Physiological Features for Emotion Recognition

  • 37 Features similar to ECG-based methods
  • Variability of IBI
  • Irregularity of breathing
slide-60
SLIDE 60

Emotion Classification

  • Recognize emotion using physiological features
  • Used L1-SVM classifier
  • select features and train classifier at the same time
slide-61
SLIDE 61

Emotion Model

Positivity Negativity High Excitement Low Excitement

Joy Pleasure Anger Sadness

  • Standard 2D emotion model
  • Classify into anger, sadness, pleasure and joy
slide-62
SLIDE 62

Evaluation

slide-63
SLIDE 63

Implementation

  • FMCW radio
  • 5.5 GHz to 7.2 GHz
  • sub-mW power
slide-64
SLIDE 64

0.00 0.25 0.50 0.75 1.00 1 2 3

Error of IBI (%) CDF

Can we capture IBI accurately?

  • Ground truth: ECG
  • 30 subjects, over 130,000 heartbeats

Median IBI estimation error: 0.4% 90th percentile error: 0.8%

slide-65
SLIDE 65

Can we detect emotions accurately?

  • Experiment:
  • 12 subjects (6 female and 6 male)
  • Prepare personal memories for each emotion
  • Elicit certain emotion with prepared memories
  • classify every 2 minutes to an emotional state
  • Ground truth: self-reported for each 2-min period
slide-66
SLIDE 66

Positivity Negativity High Excitement Low Excitement

Joy Pleasure Anger Sadness

Can we detect emotions accurately?

slide-67
SLIDE 67

Positivity Negativity High Excitement Low Excitement

Joy Pleasure Anger Sadness

Can we detect emotions accurately?

slide-68
SLIDE 68

Positivity Negativity High Excitement Low Excitement

Joy Pleasure Anger Sadness

Person-dependent Classification

  • ●
  • ●
  • Accuracy: 82.5%
  • Train and test on the same person
slide-69
SLIDE 69

Positivity Negativity High Excitement Low Excitement

Joy Pleasure Anger Sadness

Person-dependent Classification

  • ●
  • ●
  • Accuracy: 92.5%
  • Train and test on the same person
slide-70
SLIDE 70

Person-independent Classification

  • Positivity

Negativity High Excitement Low Excitement

Joy Pleasure Anger Sadness Accuracy: 72.3% We can recognize a person’s emotions without having ever trained on him/her before

  • Train and test on the different person
slide-71
SLIDE 71

Comparison with ECG-based system

87 88.2 72.3 73.2

25 50 75 100 Person−dependent Person−independent

Task Classification Accuracy (%)

EQ−Radio ECG−based system

slide-72
SLIDE 72
slide-73
SLIDE 73

HemApp