+ Research Question When providing In Home Behavioral Services to - - PowerPoint PPT Presentation

+ EVALUATING

CAREGIVER COMPLIANCE WITH DIFFERENT DATA COLLECTION METHODOLOGIES

+ Research Question When providing In Home Behavioral Services to families, how do we get them to collect data when we are not present in the home?

+ Participants

• Three families receiving home

consultation services.

• These families were receiving

consultation and parent training with the goal of reducing dangerous and disruptive behaviors.

+ Setting

• All data collection took place in the

home of the families.

+ Dependent Variable

• At the end of each day, families were

asked to complete a brief data sheet that described the frequency and intensity of multiple behaviors.

• The data sheets contained 4-7

multiple choice questions describing the frequency and intensity of behaviors the family is currently focusing on.

+

Independent variable

• First Condition - Each family was

asked to complete a pen and paper data sheet at the end of the day.

• Second Condition - Each family was

asked to complete an online data sheet at the end of each day.

• In both conditions, the data sheets

contained identical questions with identical choices.

+

Pen & Paper Sheet Sample

Sun Mon Tue Wed Thu Fri Sat Did Not Happen Once Twice Three Times Four or More Did Not Happen Once Twice Three Times Four or More Did Not Happen Once Twice Three Times Four or More Did Not Happen Once Twice Three Times Four or More Great Really Good Ok Not So Good Very Rough A Lot A Couple Times Once Not At All Unsure Overall, This Morning Went… Was He Up During The Night?

*write any notes on the back of this data sheet*

Week Of _____ To _____

Telling Mom What To Do Name Calling Innapropriate Language Inappropriate Touching

+ Google docs

• The online data sheets were created

using Google Docs and using the “Forms” tool.

• This tool allows you to easily create a

free online form that can be filled out by any user and data filled out are compiled into an online spreadsheet.

+ Google docs (cont.)

• Data entry options include…
• Multiple Choice
• Drop Down List
• Date
• Time
• Duration
• Check Boxes
• Text
• Paragraph Text
• Grid
• Scale

+ Online Data Demo

+ Online Data Demo

+ procedure

• All families participated in both

conditions of the study in a multiple baseline design.

• All families were first asked to complete

pen and paper sheets. Later they were asked to complete online data sheets asking identical questions.

+ Data collection

• Pen and Paper Phase - Data were

collected by having the consultant collect the data sheets during meetings with the family.

• Online Data Collection Phase - Data

were collected by having the consultant log onto the website that compiles the data on a daily basis.

+ Data collection

(cont.)

• Data were collected on the percentage
• f days the data sheets were

completed.

• Each day was scored as “Yes” data

were collected or “No” data were not collected

• Note: No data were taken on the

accuracy of the data collected by the family.

+ Interobserver

Agreement

• Interobserver agreement was

conducted on 83% of the data collected and agreement between

• bservers was calculated at 97%.

+

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 2 3 4 5 6 7 8 9 10 11 12 Percentage Of Days Data Completed Week

Family A

Pen & Paper Online

+

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 2 3 4 5 6 7 8 9 10 11 12 Percentage of Days Data Completed Week

Family B

Pen & Paper Online

+

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 2 3 4 5 6 7 8 9 10 11 12 Percentage of Days Data Completed Week

Family C

Pen & Paper Online

+

Percentage of Days Data Sheets Completed

0% 20% 40% 60% 80% 100%

1 2 3 4 5 6 7 8 9 10 11 12

Week

Family 3

0% 20% 40% 60% 80% 100%

Online Data Collection vs Pen & Paper Data Collection

Pen and Paper Data Collection Online Data Collection Family 1

0% 20% 40% 60% 80% 100%

Family 2

+ Latency to

view data

• Another questioned answered in the

study was how long will it take for the consultant to see the data.

• These data were collected by

comparing the date the behaviors

• ccurred and the date the consultant

was presented with the data (either by checking the online spreadsheet or being handed a paper data sheet)

+ Time Between Behavior

Occurring and Data Seen By Consultant

Sunday Tuesday Monday Wednesday

Pen & Paper

3.9 Days

Online

0.9 Days

+ Social validity

• Parents were asked to complete a brief

3 question social validity questionnaire at the completion of the study.

• The questions involved preference,

ease of use, and privacy concerns.

• The questionnaires were (of course)

filled out online.

+ which data collection

method did you prefer?

Very Strongly Preferred Online Strongly Preferred Online Preferred Online No Preference Preferred Pen & Paper Strongly Preferred Pen & Paper Very Strongly Preferred Pen & Paper

✔ ✔ ✔

+ Which data collection

method do you think was easier?

Online Was A Lot Easier Online Was A Little Easier No Difference Pen & Paper Was A Little Easier Pen & Paper Was A Lot Easier

✔ ✔ ✔

+ How Would You Rate

Your Level Of Concern With Placing Information Online?

Not Concerned At All Somewhat Concerned Very Concerned

✔ ✔ ✔

+ Limitations

• No data were taken on the accuracy of

the data collected.

• All participants were required to have

reliable access to the internet.

+ Future research

• Future research should build on this

study by examining the use on online data collection while…

• Examining the accuracy of the data

collected.

• Applying these methodologies to
• ther data collection types, such as

skills acquisition data.

+ conclusion

• Using online data collection such as

the one described in this study is a useful tool to…

• Increase the frequency of data

collected by families

• Decrease the latency between the
• ccurrence of the behavior and the

consultant viewing the data

• Increase parent satisfaction