REForm: A Data Capture Framework for Large-scale Interventional - PowerPoint PPT Presentation

REForm: A Data Capture Framework for Large-scale Interventional Studies with Survey Workflow Management Shiqiang Tao, PhD N. Zeng, X. Wu, X. Li, W. Zhu, L. Cui, GQ Zhang Institute for Biomedical Informatics University of Kentucky shiqiang.tao@uky.edu March 30, 2017 me March 30, 2017 1 / 20

Disclosure Authors disclose that they have no relationship with commercial interests. me March 30, 2017 2 / 20

Background Survey since 1980s REDCap – 2298 institutions, 108 countries, 399k projects, 510k users Interventional study IMPACT me March 30, 2017 3 / 20

Background Contd. Oncology nurses Improve knowledge, attitudes, and communication skills about clinical trials Improve the patient recruitment for clinical trials 3/29/2017 IMPACT workflow.svg Registration Post Screening Baseline Randomization Intervention Followup END Intervention Optional EMA1 EMA2 EMA3 EMA4 EMA5 EMA6 Figure: Workflow of IMPACT me March 30, 2017 4 / 20 file:///Users/strongtsq/Downloads/IMPACT%20workflow.svg 1/1

Challenges Survey and intervention separation Lack of workflow management Insufficient support for large scale surveys Interruption and return Intuitive progress indication Friendly navigation me March 30, 2017 5 / 20

Methods Survey authoring Workflow management Randomization Intervention assignment Large-scale survey support me March 30, 2017 6 / 20

Methods – Survey Authoring Figure: Left: The form to create a new survey; Right: Default page 1 of a survey and the button to add a new question. me March 30, 2017 7 / 20

Methods – Survey Authoring Contd. Question Content – What is asking for? Options – What are the answers? Question Type Data validation rules Branching logic: internal vs. external me March 30, 2017 8 / 20

Methods – Workflow Management LEFT RIGHT Figure: Left: Workflow management interface of REForm; Right: The form to create reminders. me March 30, 2017 9 / 20

Methods – Randomization 3/30/2017 Randomization.svg N Slots White, Outpatient, Academic Stratum 1 Stratum 2 White, Outpatient, Non-Academic Stratum 3 ... White, Inpatient, Academic Figure: Stratified Randomization me March 30, 2017 10 / 20 file:///Users/strongtsq/Downloads/Randomization.svg 1/1

Methods – Personalized Intervention Assignment 3/30/2017 Randomization-2.svg Interventions Answers Video 1 Question 1 1 Question 2 Video 2 1 2 2 Video 3 Question 3 3 3 Rank & Assign Baseline Survey Score Formula = k*(m/n) + p 4 4 Video 4 Question 4 5 5 Mapping Rules Video 5 Question 5 ... ... Figure: Personalized Intervention Assignment me March 30, 2017 11 / 20 file:///Users/strongtsq/Downloads/Randomization-2.svg 1/1

Methods – Large-scale Survey Support Real time saving; Interruption and return; Realtime progress indication; Fast page navigation. Figure: Progress bar for REForm: it combines page-wise progress indication and fast page navigation functions. me March 30, 2017 12 / 20

Results 3000 invitations sent in July 2016; 390 users registered, 358 consented, and 264 are eligible (within 8 weeks). Table: Surveys of IMPACT. Survey Name Questions Options Completions Baseline 95 662 238 Post Intervention 82 574 132 Followup 15 125 NA EMA 1 4 26 75 EMA 2 4 26 62 EMA 3 4 26 6 EMA 4 4 26 3 EMA 5 4 26 NA EMA 6 4 26 NA me March 30, 2017 13 / 20

Evaluation Survey authoring usability Survey participation usability me March 30, 2017 14 / 20

Evaluation – Survey Authoring Table: Setting of tests for comparative evaluation between REForm and REDCap. Name Question Type Questions Options Test 1 Single Choice 5 4 Test 2 Multiple Choice 5 4 Test 3 Text Field 5 1 Test 4 True/False 5 2 Test 5 Matrix 8 8 Test 6 Mixed types 22 1 to 11 me March 30, 2017 15 / 20

Evaluation – Survey Authoring Contd. Evaluator-­‑1 60" 49" Cost"Time"(mins) � 46" 50" 40" 30" 20" 13" 10" 3" 3" 3" 2" 2" 2" 2" 1" 1" 0" Test"1" Test"2" Test"3" Test"4" Test"5" Test"6" Evaluator-­‑2 60" 53" 49" Cost"Time"(mins) � 50" 40" REForm" 30" 15" REDCap" 20" 5" 5" 10" 3" 4" 3" 4" 3" 3" 2" 0" Test"1" Test"2" Test"3" Test"4" Test"5" Test"6" Evaluator-­‑3 64" 70" Cost"Time"(mins) � 60" 52" 50" 40" 30" 20" 12" 6" 5" 10" 4" 3" 2" 2" 2" 2" 2" 0" Test"1" Test"2" Test"3" Test"4" Test"5" Test"6" Figure: Comparative evaluation between REForm and REDCap by time cost to build survey questions in six tests. me March 30, 2017 16 / 20

Evaluation – Survey Participation (a) How satisfied are you with the information presented to you? 1. Not at all satisfied; 2. A little satisfied; 3. Moderately satisfied; 4. Very satisfied; 5. Extremely satisfied (b) The web site is easy to navigate? 1. Not at all; 2. Very little; 3. Some what; 4. Quite a bit; 5. A great deal me March 30, 2017 17 / 20

Evaluation – Survey Participation Contd. Ques%on ¡a Ques%on ¡b 90# 90# 74# 70# Numbers � Numbers � 60# 60# 50# 32# 27# 30# 30# 8# 3# 0# 0# 0# 0# 0# 5# 4# 3# 2# 1# 5# 4# 3# 2# 1# Figure: Answers to evaluation questions (a) and (b) from IMPACT participants. X-axis shows the selected options for question (a) and (b) and Y-axis shows the number of times those options are selected. me March 30, 2017 18 / 20

Discussion Integration of randomization and intervention; Comprehensive workflow management; Large survey support; Future work. me March 30, 2017 19 / 20

Conclusion We created a framework REForm to solve the data capture challenges of large-scale interventional studies; REForm provides survey authoring interface and supports integration of online surveys with interventions in a comprehensive workflow; REForm also provides sufficient support for large-scale surveys by implementing features such as real-time input data saving, page-wise progress indication, and fast page navigation. me March 30, 2017 20 / 20

Thank You! Questions? me March 30, 2017 21 / 20