Advanc e s in Me asur ing Ca sc a de o f me a sure me nt Gra - - PDF document

9/25/2011 1

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 1

Advanc e s in Me asur ing Be havior

PHT H 5228 2nd Class

Pro f. Ste phe n I ntille Offic e : 450 WVH s.intille @ ne u.e du

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 2

Booth ar tic le

• Ca sc a de o f me a sure me nt
• Gra phic me tho d o f re c o rding da ta
• Cle ve r thinking a b o ut wha t to me a sure

– “Co unte r pre ssure whic h wo uld b e ne c e ssa ry to c a use the pulsa tio n in a n a rte ry to c e a se ”

• Co mple xity impo rta nt – simplic ity o f

instrume nt pa ra mo unt

• Surg e o ns – no tic e thing s re se a rc he rs miss
• Va lue in e sta b lishing a no rma l ra ng e

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 3

Booth ar tic le

• As to o l g e ts g o o d e no ug h, sta rt se e ing

pa tte rns

• Co mpre sse d o n a ll side s a nd width: “sma ll

c ha ng e ” le a ds to b ig pe rfo rma nc e g a in

• Sma ll o b se rva tio ns

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 4

Stone ar tic le

• Re po rte d c o mplia nc e : 90%
• Ac tua l c o mplia nc e 11% (20% 90 min)
• Ho a rding : 32% o f da ys no o pe ning s b ut

re po rte d c o mplia nc e 92%

• 75% o f 40 pe o ple ha d 1+ da y o f ho a rding

(“pa rking lo t c o mplia nc e ”)

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 5

Smyth ar tic le

• Be ha vio ra l me dic ine : inte g ra tive a nd wide

a rra y o f o utc o me type s a nd mo de ls

• T

ime ! Ne e d to unde rsta nd inte ra c tio ns

• ve r time
• E

xpe rie nc e sa mpling vs E MA

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 6

Smyth ar tic le

• Re c a ll o f pa st e ve nts

– Be lie fs a b o ut b e ha vio r o r the wa y the wo rld func tio ns (“e ffo rt a fte r me a ning ”) – Outc o me o f the e ve nt (re tro a c tive re c o nstruc tio n) – Curre nt sta te , pa rtic ula rly mo o d (c o ng rue nt mo o d sta te ) – Sa lie nt e ve nts

9/25/2011 2

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 7

Smyth ar tic le

• E

xte rna l va lidity/ g e ne ra liza b ility o f finding s

– L a b vs. re a l wo rld – “White -c o a t hype rte nsio n” – Missing so c ia l e nviro nme nt – Surprising ly we a k c o rre spo nde nc e (e .g . HRV)

• Dyna mic pro c e sse s

– Re pe a te d me a sure s – Sug g e stive o f c a usa l a sso c ia tio ns

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 8

Smyth ar tic le

• E

xte rna l va lidity/ g e ne ra liza b ility o f finding s

– L a b vs. re a l wo rld – “White -c o a t hype rte nsio n” – Missing so c ia l e nviro nme nt – Surprising ly we a k c o rre spo nde nc e (e .g . HRV)

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 9

Smyth ar tic le

• Dyna mic pro c e sse s

– T hink a b o ut yo ur o wn life ! (mo o d/ stre ss) – Va ria tio ns in time o f da y (e .g ., diurna l pa tte rns

• f mo o d; “o b sc ure d a t b e st, a nd c o ntrib ute

to e rro r o r b ia s a t wo rst”) – Re pe a te d me a sure s – Sug g e stive o f c a usa l a sso c ia tio ns

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 10

Smyth ar tic le

• E

MA

– Multiple time s pe r da y – We e ks o r mo nths – Re duc e pe rio d o f re c a ll – I n the mo me nt (re duc e summa ry) – Na tura l e nviro nme nt – Na tura l e ve nts – Da te / time sta mpe d (+ timing info ) – Pro mpt o r “e ve nt drive n” a c tio ns

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 11

Smyth ar tic le

• “Re c a ll the mo st stre ssful e ve nt o f the la st

mo nth”

• Co ping : muc h o f wha t wa s re po rte d in

re a l time wa s fo rg o tte n a t re c a ll a nd tha t c o ping e ffo rts tha t we re no t re po rte d in re a l time we re re po rte d a t re c a ll

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 12

Smyth ar tic le

• E

MA c ha lle ng e s

– Que stio nna ire de sig n – T ra ining o f pa rtic ipa nts – E xtra de vic e – F ie ld mo nito ring – Mo tiva tio n to fo llo w pro to c o l (e .g ., stylus) – T e c h g litc he s – E xpe nsive – Re a c tivity

9/25/2011 3

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 13

Smyth ar tic le

• E

MA c ha lle ng e s (c o ntinue d)

– “Ma ssive ” a mo unts o f da ta – Sta ts c ha lle ng e s

• Ag g re g a tio n
• Multi-le ve l
• Missing da ta (unb a la nc e d)

Technological innovations

Circa 2003

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Take away: 4 new opportunities

 Continuous, rich recording from a variety of

sensors

 Algorithms to process data to reduce coding

time

 Context-sensitive data collection to collect data

and prompt for self-report at desired times and places

 Context-sensitive, personalized interventions

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Your task…

What are the possibilities for your research?

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Relevance to health research (1)

 Ability to better study how context

(people, places, things) impacts behavior

 Examples

Measurement of moderate intensity or greater

physical activity

Dietary decision making Making every interruption count Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Relevance to health research (2)

 Ability to create and measure impact of

“just-in-time” interventions

 Example: physical activity

Measurement is important, but we already know

people don’t get enough physical activity!

Just-in-time detection of activity for positive

reinforcement

9/25/2011 4

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Overview

 New developments  Examples

Context-sensitive experience sampling Portable kit of “tape on” environmental sensors PlaceLab

 Emerging opportunities  Challenges

New developments

• New developments
• Examples
• Emerging opportunities
• Challenges

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Data collection in the (not-so-distant) future

 Record and save everything from subjects:

24/7 video stream (160x120 resolution,10fps,MPEG-4) [1.56 GB/day] 24/7 audio stream (24kHz mp3) [.57 GB/day] 24/7 1 photo per minute or other data [.57 GB/day] 16/7 One 3MB data file per hour [72MB/day]

 A year of data: 990MB  2007: Terabyte of data < $300

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Sensors in the (not-so-distant) future

 Example:

Video/photos from miniature pocket/cap camera Continuous audio recording, keyword detection Real-time HR data Real-time motion data all limbs, hip Real-time indoor/outdoor position Real-time position relative to other people Real-time data from home: objects touched/used Data on use of communication devices No encumbering or nerdy-looking devices Context-sensitive self report Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Data analysis in the (not-so-distant) future

 Computers pre-process data:

Translate noisy sensor data into meaningful labels

E.G. Cooking, socializing, running, smoking, …

 Computer helps researcher search data:

“find all the moments when the subject might have

been cooking”

“query the subject whenever the subject is near

another subject”

“show me video clips of moments when the subject

was with other people”

“indicate where the subject spent the most time” Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Personalized mobile computing device

Powerful, inexpensive, sensor-enabled mobile computing device carried nearly everywhere

Take your pick…

9/25/2011 5

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

The mobile computing device…

 Color touch screen  Light, comfortable to

carry everywhere

 1GB+ disk space  Sound player (MP3 and

• ther)

 Sound recorder  Camera  Fingerprint recognizer  400+ MHz processor  Always on wireless

connection

 Battery life  Cost (not for long)  Barcode scanner  Handwritten input  Speech input  Video game player  GPS / location detection  Accelerometers  Biomonitors Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

New developments in pattern recognition

 Innovation:

Real-time recognition of activities

(e.g. walking, running, posture, cooking …)

Recognition of affect

(e.g. frustration, stress, anger)

Speech recognition Recognition of socialization activity

 Remaining challenges:

Real-time recognition of many activities Unencumbering recognition of many emotional states Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Technologist’s interest

 Want to design technology for real-world

environments and to test technology in context, but…

 Vast majority of homes and workplaces do not

look anything like our labs and prototype environments!

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

The future of computing?

New York Times Magazine

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Motivation for sensing/measurement tools

 Behavior is “situated”, i.e. influenced by

environment

 Simulating natural setting in lab difficult

(impossible?)

 Real environments are terribly complex  Need sensors to measure reaction to

interventions in context of everyday life

Examples

• New developments
• Examples
• Emerging opportunities
• Challenges

9/25/2011 6

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

House_n: tools to study natural settings

Portable data collection and intervention toolkit PlaceLab residential research facility

Context-aware experience sampling

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Electronic experience sampling

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

MIT version: new data collection capabilities

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

E.g.: trigger sample based on position

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

E.g.: trigger sample based on HR

9/25/2011 7

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)  Scheduling options

 Fixed  Random within intervals  User-initiated  Triggered by context

 PDA plug-in sensors and sampling devices

 GPS  Heart rate  Bar code scanner  Camera  Accelerometers  Future: Bluetooth

Context-aware experience sampling

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Context-aware experience sampling tool

 Uses at MIT:

Machine learning algorithm

development

Physical activity interventions Studying interruptions

(using biometric data)

Planned: workplace studies

 Available to researchers

http://caes.sourceforge.net

Mobile activity recognition

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Multiple, wire-free accelerometers

 Placement

5 points

 Collect data up to 24 hours  2 axis, 85Hz sampling  No wires  Next version (Fall):

watch size, comfortable, real-time wireless

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Mobile activity recognition

 Ling

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Mobile activity recognition

 Features

9/25/2011 8

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Mobile activity recognition

 Aggregate confusion matrix for fast C4.5

classifier based on leave-one-subject out validation for 20 subjects using laboratory and

• bstacle course data.

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Current work

 Development of comfortable, 24 hour wireless,

2-3 axis mobile accelerometers

Smaller than CSA actigraph Real-time data streaming High sampling rate

 Real-time mobile activity recognition for

context-sensitive data collection

Tape-on environmental sensors

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Environmental sensor kit

 Data collection board with swappable sensor  Small, robust  Relatively inexpensive ($27 each at qty of 150)  Collect state change data 4+ weeks  + /- 2 second timestamp synchronization  Tape-on install  Non stigmatizing  Relatively non-invasive

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Environmental sensor kit

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

One subject’s home

 3 hours with

small team

 Install: tape-on  Approx. 85-100

sensors in small 1 bedroom

 On | Off  Open | Closed  Position | Identity

9/25/2011 9

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Studying behavior in context

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

9/25/2011 10

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

9/25/2011 11

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Cooking breakfast 3/27

9/25/2011 12

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Cooking breakfast 4/01

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Current work: detecting activities automatically using probabilistic algorithms

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T) Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Collaborative development of interventions

Challenges

• New developments
• Examples
• Emerging opportunities
• Challenges

9/25/2011 13

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Volume of data / ethical collection

 Terrabytes possible  Annotation can be time consuming, costly, and

challenging

 Ethical issues may be raised by data collection

Technology I nnovations for Real-Time Data Capture (S. I ntille – MI T)

Data analysis techniques

 New types of multi-modal data  Sensor algorithms noisy/probabilistic  Desired contextual cues can be ill-defined:

E.g. “Cooking” E.g. “Jittery” E.g. “Getting dressed” E.g. “Busy” Adva nc e s in Me a sur ing Be ha vior F a ll 2011

No rthe a ste rn Unive rsity 75

Rappapor t ar tic le

• 70-90% o f dise a se risks pro b a b ly due to

diffe re nc e s in e nviro nme nt

• Ne e d b e tte r e nviro nme nta l to o ls fo r GWAS
• “Slic ing o f the dise a se pie a lo ng pa ro c hia l

line s”

• E

nviro nme nta l e xpo sure : b o dy’ s inte rna l c he mic a l e nviro nme nt

• Missing : g o ing b e yo nd c o rre la tio n: ne e d

to kno w mo re to c ha ng e e xpo sure !

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 76

Kix ar tic le

• Cre a tive thinking . Ye s!
• I

nitia l re sista nc e ... c o mmo n (K uhn)

• Ma ny po sitive s

– Sma ll numb e r o f me a sure me nt site s – E a sy to instrume nt – Co uld me a sure b y the ho ur – Priva c y pre se rve d

Adva nc e s in Me a sur ing Be ha vior F a ll 2011 No rthe a ste rn Unive rsity 77

Kix ar tic le

• No thing pe rfe c t

– Se ptic ta nks in rura l a re a s – Ac c ura c y a nd pre c isio n – L e g a l issue s? – So me o f the sa me c ha lle ng e s a s E MA

• T

hink c re a tive ly! L

• o k fo r a diffe re nt po int
• f me a sure me nt

(e .g . e a ting )