REForm: A Data Capture Framework for Large-scale Interventional - - PowerPoint PPT Presentation

REForm: A Data Capture Framework for Large-scale Interventional Studies with Survey Workflow Management

Shiqiang Tao, PhD

• N. Zeng, X. Wu, X. Li, W. Zhu, L. Cui, GQ Zhang

Institute for Biomedical Informatics University of Kentucky shiqiang.tao@uky.edu

March 30, 2017

me March 30, 2017 1 / 20

Disclosure

Authors disclose that they have no relationship with commercial interests.

me March 30, 2017 2 / 20

Background

Survey since 1980s REDCap – 2298 institutions, 108 countries, 399k projects, 510k users Interventional study IMPACT

me March 30, 2017 3 / 20

Background Contd.

Oncology nurses Improve knowledge, attitudes, and communication skills about clinical trials Improve the patient recruitment for clinical trials

3/29/2017 IMPACT workflow.svg file:///Users/strongtsq/Downloads/IMPACT%20workflow.svg 1/1

Registration Screening Baseline Intervention Optional Post Intervention Followup EMA1 EMA2 EMA3 EMA4 EMA5 EMA6 END Randomization

Figure: Workflow of IMPACT

me March 30, 2017 4 / 20

Challenges

Survey and intervention separation Lack of workflow management Insufficient support for large scale surveys

Interruption and return Intuitive progress indication Friendly navigation

me March 30, 2017 5 / 20

Methods

Survey authoring Workflow management Randomization Intervention assignment Large-scale survey support

me March 30, 2017 6 / 20

Methods – Survey Authoring

Figure: Left: The form to create a new survey; Right: Default page 1 of a survey and the button to add a new question.

me March 30, 2017 7 / 20

Methods – Survey Authoring Contd.

Question Content – What is asking for? Options – What are the answers? Question Type Data validation rules Branching logic: internal vs. external

me March 30, 2017 8 / 20

Methods – Workflow Management

LEFT RIGHT

Figure: Left: Workflow management interface of REForm; Right: The form to

create reminders.

me March 30, 2017 9 / 20

Methods – Randomization

3/30/2017 Randomization.svg file:///Users/strongtsq/Downloads/Randomization.svg 1/1

White, Outpatient, Academic White, Outpatient, Non-Academic White, Inpatient, Academic

...

Stratum 1 Stratum 2 Stratum 3 N Slots

Figure: Stratified Randomization

me March 30, 2017 10 / 20

Methods – Personalized Intervention Assignment

3/30/2017 Randomization-2.svg file:///Users/strongtsq/Downloads/Randomization-2.svg 1/1 Baseline Survey 1 Question 1 4 Question 4 2 Question 2 3 Question 3 5 Question 5 ... 2 1 3 4 5 Mapping Rules Video 1 Video 4 Video 2 Video 3 Video 5 ... Answers Interventions Rank & Assign Score Formula = k*(m/n) + p

Figure: Personalized Intervention Assignment

me March 30, 2017 11 / 20

Methods – Large-scale Survey Support

Real time saving; Interruption and return; Realtime progress indication; Fast page navigation.

Figure: Progress bar for REForm: it combines page-wise progress indication and

fast page navigation functions.

me March 30, 2017 12 / 20

Results

3000 invitations sent in July 2016; 390 users registered, 358 consented, and 264 are eligible (within 8 weeks).

Table: Surveys of IMPACT. Survey Name Questions Options Completions Baseline 95 662 238 Post Intervention 82 574 132 Followup 15 125 NA EMA 1 4 26 75 EMA 2 4 26 62 EMA 3 4 26 6 EMA 4 4 26 3 EMA 5 4 26 NA EMA 6 4 26 NA

me March 30, 2017 13 / 20

Evaluation

Survey authoring usability Survey participation usability

me March 30, 2017 14 / 20

Evaluation – Survey Authoring

Table: Setting of tests for comparative evaluation between REForm and REDCap.

Name Question Type Questions Options Test 1 Single Choice 5 4 Test 2 Multiple Choice 5 4 Test 3 Text Field 5 1 Test 4 True/False 5 2 Test 5 Matrix 8 8 Test 6 Mixed types 22 1 to 11

me March 30, 2017 15 / 20

Evaluation – Survey Authoring Contd.

3" 4" 3" 3" 15" 49" 5" 5" 4" 3" 2" 53" 0" 10" 20" 30" 40" 50" 60" Test"1" Test"2" Test"3" Test"4" Test"5" Test"6" Cost"Time"(mins) REForm" REDCap" 3" 3" 3" 2" 13" 46" 2" 2" 1" 2" 1" 49" 0" 10" 20" 30" 40" 50" 60" Test"1" Test"2" Test"3" Test"4" Test"5" Test"6" Cost"Time"(mins) 6" 5" 2" 2" 12" 64" 3" 4" 2" 2" 2" 52" 0" 10" 20" 30" 40" 50" 60" 70" Test"1" Test"2" Test"3" Test"4" Test"5" Test"6" Cost"Time"(mins) Evaluator-­‑1 Evaluator-­‑2 Evaluator-­‑3

Figure: Comparative evaluation between REForm and REDCap by time cost to

build survey questions in six tests.

me March 30, 2017 16 / 20

Evaluation – Survey Participation

(a) How satisfied are you with the information presented to you?

• 1. Not at all satisfied; 2. A little satisfied; 3. Moderately satisfied; 4.

Very satisfied; 5. Extremely satisfied (b) The web site is easy to navigate?

• 1. Not at all; 2. Very little; 3. Some what; 4. Quite a bit; 5. A great

deal

me March 30, 2017 17 / 20

Evaluation – Survey Participation Contd.

27# 70# 32# 3# 0# 0# 30# 60# 90# 5# 4# 3# 2# 1# Numbers 74# 50# 8# 0# 0# 0# 30# 60# 90# 5# 4# 3# 2# 1# Numbers Ques%on ¡a Ques%on ¡b

Figure: Answers to evaluation questions (a) and (b) from IMPACT participants.

X-axis shows the selected options for question (a) and (b) and Y-axis shows the number of times those options are selected.

me March 30, 2017 18 / 20

Discussion

Integration of randomization and intervention; Comprehensive workflow management; Large survey support; Future work.

me March 30, 2017 19 / 20

Conclusion

We created a framework REForm to solve the data capture challenges

• f large-scale interventional studies;

REForm provides survey authoring interface and supports integration

• f online surveys with interventions in a comprehensive workflow;

REForm also provides sufficient support for large-scale surveys by implementing features such as real-time input data saving, page-wise progress indication, and fast page navigation.

me March 30, 2017 20 / 20

Thank You!

Questions?

me March 30, 2017 21 / 20