Streaming Sensor Data from the Home What does it all mean? How can - PowerPoint PPT Presentation

Streaming Sensor Data from the Home What does it all mean? How can it help you? ! Holly%Jimison,%Misha%Pavel,%% Xuan%“Sean”%Li,%Krissy%Mainello% College%of%Computer%&%InformaAon%Science% Bouve%College%of%Health%Sciences% ConsorAum%on%Technology%for%ProacAve%Care% Northeastern%University% % ! ! !

• Funding' – Na&onal!Science!Founda&on! – Na&onal!Ins&tute!on!Aging! – Alzheimer’s!Associa&on!/!Intel!Company! – Na&onal!Ins&tute!on!Standards!&!Technology! – TEKES!(Finland!Government)! • No'conflicts'of'interest' ' • Collabora5ve'work'with' – Oregon!Health!&!Science!University! – University!of!California!at!Berkeley!

Scalable Approach to Delivering Health Interventions to the Home ! Sensors, algorithms, mobile communications for lifestyle interventions ! Remote, just-in-time, continuous care ! Incorporate principles of health behavior change ! Optimal use of lower cost personnel ! Integrate family & informal caregivers into the health care team (untapped resource) ! Platform for testing sustained cognitive interventions in the home

Modular Software for Multiple Protocols ! Cognitive Exercise (computer game format) ! Novelty exercise ! Physical Exercise ! Sleep Management ! Socialization ! Medication Management ! Mood Management (depression)

Behavioral Markers = Continuous Monitoring & Computational Models Home health based on unobtrusive, continuous monitoring

Models!to!Infer!Ac&vi&es!of!Daily!Living ! Pavel et al., IEEE Special Issue, in press 6

Sensor Events Activity Monitoring in the Home Private Home Bedroom Bathroom Living Rm Front Door Kitchen Hayes et al., www.orcatech.org Hayes, ORCATECH 2007

Sensor Events Activity Monitoring in the Home Residential Facility Bedroom Bathroom Living Rm Front Door Kitchen Hayes et al., www.orcatech.org Hayes, ORCATECH 2007

Measuring!Gait!in!the!Home ! • Unobtrusive'gait'measurement'in<home'with'passive' infrared'(PIR)'sensors '<' Hagler,%et%al.,%IEEE%Trans%Biomed%Eng,%2010% – Four!restricted!view!PIR!sensors! – Measure!gait!velocity!whenever!a! – ! subjects!passes!through!the!! – ! “sensorVline”! – Deployed!for!the!Intelligent!! – !!!!Systems!for!Assessing!! – !!!!Aging!Changes!(ISAAC)!study! – 200+!subjects!monitored!for!>!4!years! 9

Subject!1 ! 0.035 90 0.03 Stroke 80 0.025 Velocity (cm/s) 70 0.02 60 0.015 50 0.01 40 0.005 30 12/07 08/08 11/09 12/10 Time Aus&n!et!al,!Sept!2011!V!EMBC!(Gait)! 10

Subject!2 ! 0.05 CDR=0.5 and MCI 0.045 90 diagnosis 0.04 80 0.035 Velocity (cm/s) 0.03 70 0.025 0.02 60 0.015 0.01 50 0.005 07/07 02/09 09/10 Time Aus&n!et!al,!Sept!2011!V!EMBC!(Gait)! 11

Health!Coaching!Pla_orm ! Family Interface • Safety monitoring • Soft alerts • Team-based care • Socialization

Automated!Coaching!for!Physical!Exercise ! • Collabora5on'with'' – Oregon!Health!and!Science!University! – University!California!Berkeley! • Pre<recorded'video'clips'for' tailored'exercise'and'Kinect' Camera' • Real<5me'feedback'based'on' image'interpreta5on'from'Kinect' skeleton'representa5on' • Monitoring'of'balance,'flexibility,' strength,'endurance' • Poten5al'for'remote'interac5on''

Sleep Module Assessment • Sleep Hygiene • Anxiety • Circadian Rhythm Tailored Intervention 14

Socialization Protocols for Cognitive Health ! Web cams and Skype software given to participants and their remote family partner ! Frequent spontaneous use among participants

Cogni&on!V!Monitoring!&!Interven&on ! 16

! Computer!Game!to!Measure!Execu&ve!Func&on! 17

Model!the!&ming!of!the!mouse!clicks ! Recall Search for Move to Next Target Next Target Next Target t ( ) + t n d , t + R M S S. Hagler et al., www.ORCATECH.org 18

Es&mates!from!Game!Predict!TMT!Scores ! 2 R = 0.78 p < 0.0001 S. Hagler et al., www.ORCATECH.org 19

Cognitive Modeling Example: Memory B B A C D A E B C F B G H D G E I A C D D E B B F F G H D D D B A C C D E E B E F G H H H B A B C D D E D E F G F F B C D D E F E E Characterize Memory Capacity • Intervening number of events Subject 1020, N = 8687 Probability of Correct 1 • Intervening time 0.5 • Memory load Simple Memory Model: Discrete 0 0 5 10 15 Intervening Number of Events Buffer Probability of Correct 1 0.5 0 0 5 10 15 20 25 Intervening Time [sec] Characterize Memory Capacity with a Single Parameter M Pavel, et al., www.ORCATECH.org

Dynamic!User!Model!to!Support!Tailored!Messaging ! Family Interface • Safety monitoring • Soft alerts • Team-based care • Socialization

Family Caregiver Interface Link to Demo

!!!!!!!!!!!!!!!!!!!!!!Monitoring!V>!Interven&on! • Ac5vity'Monitoring'in'the'Home' • Cogni5ve'Monitoring' – Adap&ve!Computer!Games!–!Divided!Aaen&on,!Planning,!Memory,! Verbal!Fluency,!+++! – Linguis&c!Complexity!–!Emails,!phone! • Motor'Speed' – Speed!of!Walking,!Computer!Typing,!Mouse!Movements! • Sleep'Monitoring' • Depression'–'affect'on'phone,'linguis5c'analysis' • Medica5on'Management'–'Context'aware'reminding' • Socializa5on'–'Skype,'phone,'emails' • Physical'Exercise'–'Interac5ve'video' 23