Learning from Applicants and Reviewers about Engagement Resources - PowerPoint PPT Presentation

Learning from Applicants and Reviewers about Engagement Resources Sana N. Vieux, MPH Program Associate, Research Integration and Evaluation

Applicant and Reviewer Surveys Spring 2014 Fall 2014 Spring 2014 Pragmatic Trials (May 2014) (November 2014) (August 2014) Total N = 791 Applicant Surveys Response rates = 44 – 74% Total N = 363 Reviewer Surveys --- Response rates = 86 – 88%

Applicant Survey Results

Helpfulness of Sample Engagement Plans 100% 90% 82% 80% 75% 71% 70% 60% 50% 40% 30% 20% 14% 11% 10% 10% 9% 9% 10% 5% 5% 0% 0% Spring 2014 Spring 2014 Pragmatic Trials Fall 2014 Did Not Use Unhelpful Neutral Helpful

Helpfulness of Engagement Rubric 100% 90% 82% 80% 75% 75% 70% 60% 50% 40% 30% 20% 12% 11% 9% 9% 7% 10% 6% 5% 5% 5% 0% Spring 2014 Spring 2014 Pragmatic Trials Fall 2014 Did Not Use Unhelpful Neutral Helpful

Engagement Rubric Helped Identify and Fill Gaps in the Engagement Plan 100% 90% 82% 80% 69% 67% 70% 60% 50% 40% 30% 18% 20% 15% 13% 11% 10% 8% 10% 5% N/A 0% 0% Spring 2014 Spring 2014 Pragmatic Trials Fall 2014 Did Not Use Disagree Neutral Agree

“I Understand PCORI’s Requirements for Patient and Other Stakeholder Engagement.” 100% 91% 90% 85% 84% 80% 70% 60% 50% 40% 30% 20% 9% 9% 7% 10% 6% 5% 5% 0% Spring 2014 Spring 2014 Pragmatic Trials Fall 2014 Disagree Neutral Agree 95

Duration of Partnership Prior to Application >85% of applicants established a partnership before submitting the application. 100% 90% 80% 70% 60% 53% 50% 42% 37% 40% 29% 26% 30% 25% 24% 21% 16% 20% 12% 10% 5% 10% 0% Spring 2014 Spring 2014 Pragmatic Trials Fall 2014 0-6 Months 7 Months - 1 Year 2-5 Years More than 5 Years Before 96

Summary and Discussion: Applicant Survey Findings • Engagement rubric and sample engagement plans were perceived as helpful and as facilitating plans for engagement. • Most applicants reported understanding PCORI’s requirements for patient and stakeholder engagement. • Most applicants established a stakeholder partnership(s) prior to applying to PCORI. 97

Reviewer Survey Results

“The Engagement Rubric Helped me Identify Gaps in my Assigned Applications” Spring 2014 100% 90% 80% 75% 72% 70% 57% 60% 50% 40% 35% 30% 19% 16% 20% 9% 8% 8% 10% 0% Patients Stakeholders Scientists Disagree Neutral Agree

“The Engagement Rubric Helped me Evaluate Patient and Stakeholder Engagement in my Assigned Applications” Spring Pragmatic Trials 100% 100% 100% 90% 90% 80% 70% 60% 50% 40% 30% 20% 10% 5% 0% 0% 0% 0% 0% 0% Patients Stakeholders Scientists Disagree Neutral Agree

“The Engagement Rubric was Helpful in Framing the Discussion about Engagement During the In-person Merit Review Panel” 100% 89% 90% 85% 84% 80% 74% 69% 70% 61% 60% % Agree 50% 40% 30% 20% 10% 0% Spring 2014 Spring 2014 Spring 2014 Spring 2014 Spring 2014 Spring 2014 Pragmatic Pragmatic Pragmatic Trials Trials Trials Patients Stakeholders Scientists

Reviewer Survey Summary • Most reviewers agree that the Engagement Rubric was helpful for: – Evaluating assigned applications – Framing the discussion about engagement 102

Thank You! 103