Some Markov models for direct observation of behavior James E. - - PowerPoint PPT Presentation

Some Markov models for direct observation of behavior

James E. Pustejovsky Northwestern University May 29, 2013

Direct observation of behavior

• Quantities of interest
• Prevalence: proportion of time that a behavior occurs
• Incidence: rate at which new behavioral events begin
• Intensity, contingency, others
• Applications in psychology and education research
• Measurement of teaching practice
• Measurement of student behavior
• Evaluating interventions for individuals with disabilities
• Other examples in animal behavior, organizational psychology, social

work, exercise physiology

2

Observation recording methods

• How to turn direct observation of a “behavior stream” into

data?

• Continuous recording methods
• Produce rich data, amenable to sophisticated modeling
• Effort-intensive
• Discontinuous recording methods
• Less demanding methods needed in field settings
• Momentary time sampling
• Interval recording
• Other possibilities?

3

Outline

• Model for behavior stream as observed
• Momentary time sampling
• Interval recording
• Some novel proposals
• Efficiency considerations

4

A model for the behavior stream

5

Session time Interim times Event durations D1 D2 D3 E0 E1 E2 E3

 

1 1

is not occuring at time is occu 0 behavior ( ) ring at tim 1beh e avior

j i i j j i

t t Y t I D t E D

   

              

 

Behavior stream MTS Interval recording Novel proposals Efficiency

Assumptions

1.

Event durations:

2.

Interim times:

3.

Event durations and interim times are all mutually independent.

4.

Process is in equilibrium. Under this model:

• Prevalence ϕ = μ / (μ + λ)
• Incidence ζ = 1 / (μ + λ)

Alternating Poisson Process

6

Exp(1/ ), 1,2, , .. ~ 3 .

j

D j   Exp(1/ ), 1,2, , .. ~ 3 .

j

E j  

Behavior stream MTS Interval recording Novel proposals Efficiency

Behavior stream MTS Interval recording Novel proposals Efficiency

Momentary time sampling (MTS)

7

• (K + 1) moments, equally spaced at intervals of length L.
• Observer records the presence or absence of a behavior at

each moment

• Recorded data are

( ), 0,...,

k

Y kL k K X  

X0 = 0 X1 = 1 X2 = 0 X3 = 0 X4 = 1 X5 = 1 X6 = 1 X7 = 1 X8 = 0

Session time

Model for MTS data

• Under the alternating Poisson process, X1,…,XK follow a

discrete-time Markov chain (DTMC) with two states

(see e.g., Kulkarni, 2010).

8

X=0 X=1 p0(L) 1 - p0(L) p1(L) 1 - p1(L)

1

( ) Pr( ( ) 1| (0) 0) 1 exp (1 ) ( ) Pr( ( ) 1| (0) 1) (1 )exp (1 ) t t Y t Y t p t p Y t Y                                          

Behavior stream MTS Interval recording Novel proposals Efficiency

MTS model, continued

• Maximum likelihood estimators of ϕ and ζ have closed form

expressions (Brown, Solomon, & Stephens, 1975).

• But

under more general models.

• Extensive literature, lots of generalizations
• stopping rules for observation time

(Brown, Solomon, & Stephens, 1977, 1979; Griffin & Adams, 1983)

• Irregular observation times (e.g., Cook, 1999)
• Random effects to describe variation across subjects

(e.g., Cook et al., 1999)

9

 

E X  

Behavior stream MTS Interval recording Novel proposals Efficiency

Partial interval recording (PIR)

10

• Divide period into K intervals, each of length L.
• For each interval, observer records whether behavior
• ccurred at any point during the interval.
• Recorded data are

U1 = 1 U2 = 1 U3 = 0 U4 = 1 U5 = 1 U6 = 1 U7 = 1 U8 = 1

Session time

 

 

[0, )

1 , 1,..., .

L k

I U Y k L t dt k K           



Behavior stream MTS Interval recording Novel proposals Efficiency

PIR, continued

• Unlike MTS, the mean of PIR data is not readily

interpretable:

11

 

(1 ) 1 exp 1 L E U                      

Behavior stream MTS Interval recording Novel proposals Efficiency

Model for PIR data

• Define Vk as the number of consecutive intervals where

behavior is present:

• Under the alternating Poisson process, V1,…,VK follow a

DTMC on the space {0,1,2,3,…}.

12

 

max 0 : U 0 .

k j

k j k V     

V=0 V=1 π01 1 - π01 V=2 V=3 … 1 – π12 1 – π23 1 – π34 π12 π23 π34

Behavior stream MTS Interval recording Novel proposals Efficiency

Whole interval recording (WIR)

13

• Divide period into K intervals, each of length L.
• For each interval, observer records whether behavior
• ccurred for the duration of the interval.
• Recorded data are
• Equivalent to PIR for absence of event.

W1 = 0 W2 = 0 W3 = 0 W4 = 0 W5 = 1 W6 = 1 W7 = 0 W8 = 0

Session time

 

 

[0, )

1 , 1,..., .

L k

I W L Y k L t dt k K           



Behavior stream MTS Interval recording Novel proposals Efficiency

Augmented interval recording (AIR)

14

• Divide period into K/2 intervals, each of length 2L.
• Use MTS at the beginning of each interval, to record Xk-1.
• If Xk-1 = 0, use PIR for the remainder of the interval.
• If Xk-1 = 1, use WIR for the remainder of the interval.

X0 = 0 X1 = 0 X2 = 1 X3 = 1 X4 = 0 U1 = 1 U2 = 1 W3 = 1 W4 = 0

Session time

Behavior stream MTS Interval recording Novel proposals Efficiency

Model for AIR data

15

• Define Zk = Uk + Wk + Xk .
• Under the alternating Poisson process, Z1,…,ZK/2 follow a DTMC on

{0,1,2,3}, with transition probabilities πab= Pr(Zk = b | Xk-1 = a)

Z=0

U=0, W=0, X=0

Z=1

U=1, W=0, X=0

Z=2

U=1, W=0, X=1

Z=3

U=1, W=1, X=1

π01 π00 π00 π01 π02 π11 π11 π02 π12 π13 π12 π13

Behavior stream MTS Interval recording Novel proposals Efficiency

Intermittent transition recording (ITR)

16

• Divide period into K/2 intervals, each of length 2L.
• Use MTS at the beginning of each interval, to record Xk-1.
• Record time until next transition as Tk.

X0 = 0 X1 = 0 X2 = 1 X3 = 1 X4 = 0 T1 T2 T3 >2L T4

Session time

Behavior stream MTS Interval recording Novel proposals Efficiency

• Under the alternating Poisson process, T1,X1, …,TK/2,XK/2

have the property that F(Tk,Xk | T1,X1,…,Tk-1,Xk-1) = F(Tk,Xk|Xk-1)

Model for ITR data

17

Behavior stream MTS Interval recording Novel proposals Efficiency

X=0 X=1 p0(L-t) 1 - p0(L-t) p1(L-t) 1 - p1(L-t) T | X=1 T | X=0 fμ(t) fλ(t)

Asymptotic relative efficiency

18

Behavior stream MTS Interval recording Novel proposals Efficiency

• Procedure p, q ∈ {MTS, PIR, AIR, ITR}
• are maximum likelihood estimators based on

procedure p

• are asymptotic variances based on inverse of

expected information matrix.

• Asymptotic relative efficiency of p versus q

     

ˆ V ˆ ˆ , R ˆ E V A

q p q p

    

     

ˆ V ˆ ˆ , R ˆ E V A

q p q p

     ˆ ˆ ,

p p

 

   

, ˆ ˆ

p p

V V  

PIR AIR ITR 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 MTS PIR AIR 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8

True prevalence True incidence (frequency per interval)

0.5 1.0 2.0 asymptotic relative efficiency

Asymptotic relative efficiency: Prevalence

19

Behavior stream MTS Interval recording Novel proposals Efficiency

PIR AIR ITR 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 MTS PIR AIR 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8

True prevalence True incidence (frequency per interval)

0.95 1.00 1.05 asymptotic relative efficiency

blue = column is more efficient red = row is more efficient

PIR AIR ITR 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 MTS PIR AIR 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8

True prevalence True incidence (frequency per interval)

0.5 1.0 2.0 asymptotic relative efficiency

Asymptotic relative efficiency: Incidence

20

Behavior stream MTS Interval recording Novel proposals Efficiency

blue = column is more efficient red = row is more efficient

• Evaluating these models & methods
• Field testing
• When is it okay to treat ML estimates from individual sessions as

“pre-processing”?

• Lots still to do
• Build data-collection software
• Extensions to between-period regression models
• Random period/subject effects
• PIR, AIR, ITR under other distributional assumptions?

Future work

21

Questions? Comments?

pusto@u.northwestern.edu

PIR model

• Transition probabilities are uglier than MTS:

where and f(j) is the j-fold recursion of f.

23

 

/ ) , ( 1 1

1 ( Pr 1| 0) 1

L j j k k j

V j V j f e





  

          

 

/ (1 ) /(1 )

( ) 1 ( ) (1 )

L L

q f q q e e

    

 

   

    

AIR, continued

24

Event occurring at time (k-1)L? Event ends before time kL? Event begins before time kL? Xk-1=1, Uk=1 Xk-1=0, Wk=0 Yes No Wk=1 Wk=0 Yes No Vk=1 Vk=0 Yes No

Behavior stream MTS Interval recording Novel proposals Efficiency

AIR model

25

Extras

• Transition probabilities are given by

2 / 00 10 L

e



 



 

2 / 01 11

(2 ) ( ) 1 2

L

L p L e p



   



   

02 2 / 12

1 (2 ) (2 )

L

p e p L L



   



   

03 2 / 13 L

e



 



 

• Under alternating Poisson process,

ITR model

26

     

1 1 /2 /2 /2 1 1 1

, ,..., , | ; , , ; | , |

K K K k k k k k k

f t x f t x x t x x f x t    

  



Behavior stream MTS Interval recording Novel proposals Efficiency