Dashboard Review First Quarter of FY 2015 Joe Selby, MD, MPH - - PowerPoint PPT Presentation

▶
dashboard review first quarter of fy 2015
SMART_READER_LITE
LIVE PREVIEW

Dashboard Review First Quarter of FY 2015 Joe Selby, MD, MPH - - PowerPoint PPT Presentation

Sep 14, 2022 340 likes •611 views

Dashboard Review First Quarter of FY 2015 Joe Selby, MD, MPH Executive Director Michele Orza, ScD Senior Advisor to the Executive Director Presentation Overview Evolution of our Board of Governors Dashboard from 2013 Latest

slide-1
SLIDE 1

Dashboard Review First Quarter of FY 2015

Joe Selby, MD, MPH

Executive Director

Michele Orza, ScD

Senior Advisor to the Executive Director

slide-2
SLIDE 2

Presentation Overview

  • Evolution of our Board of Governors’ Dashboard from 2013
  • Latest revisions
  • Overall organization and types of graphics
  • Noteworthy and Yellow-flagged items
  • Examples of background slides
  • Examples of metrics proposed for Committee dashboards
slide-3
SLIDE 3

Discussion Questions

  • Are there metrics that should be revised or retired or

moved to the Committee Dashboards?

  • Are there new or different metrics suggested by today’s

presentation and discussion? If so, are they appropriate for the Board or one of the Committees?

  • What further improvements should we consider so that our

Dashboard and accompanying materials effectively convey the status of our work and progress toward our goals?

slide-4
SLIDE 4

Our End-of-CY 2013 Dashboard

slide-5
SLIDE 5

Our End-of-FY 2014 Dashboard

slide-6
SLIDE 6

10 20 30

Q1 Q2 Q3 Q4 Number of Projects

Targeted Pragmatic Broad PCORnet Engagement

NA

Needs Attention On Target Off Target

Legend

Funds Committed to Research – Budget=$640M Percent of Projects Meeting All Milestones

Board of Governors FY2015 Dashboard – Q1 (As of 12/31/2014) Our Goals: Increase Information, Speed Implementation, and Influence Research

*Influencing Research* PCORI inspired the American Heart Association to pilot the Citizen Scientist Think Tank, intended to foster patient engagement in the design of heart disease and stroke research Journal Articles Published Projects Awarded

NA=Not Applicable

10 20 30 By Awardees About or By PCORI Number of Articles

Q2 Q3 Q4 Q1

1,000 2,000 3,000 4,000 5,000 6,000 7,000 Web Views 5 10 15 20 25 30 Citations

Q2 Q3 Q4 Q1

5 10 15 20 25 30 35

Q1 Q2 Q3 Q4

Number of Projects Expected Actual

Uptake of Methodology Standards Completion of Projects Expenditures – Total Budget=$362M Progress of PCORnet

Q2=Q2 2014 Q3=Q3 2014 Q4=Q4 2014 Q1=Q1 2015

Phase II PFA Released Governance policies approved Version 2.0 of CDM Complete Aspirin Trial Awarded Obesity Cohort Project Awarded Health Systems Projects Awarded Phase II awarded

100 200 300 400 500 600

Actual Budget

$ Millions Q1 Q2 Q3 Q4 50 100 150 200 250

Other Actual Other Budget Research Actual Research Budget

$ Millions

Q1 Q2 Q3 Q4

Q1

Q1 Q2 Q3

10 20 30 40 50 60 70 80 90 100 Research Engagement PPRNs CDRNs Percent Meeting All

Q2 Q3 Q4 Q1

NA

=Target =Actual

slide-7
SLIDE 7

Concerns with FY 2014 Dashboard and Revisions in FY 2015 Dashboard

Concern with FY 2014 Dashboard Revision in FY 2015 Dashboard Cluttered – too many items Cut from 17 boxes to 9 Hard to read Remaining items enlarged Arrow graphics difficult to interpret Replaced with horizontal bar charts PCORnet arrow provided no information PCORnet timeline with key milestones Many metrics not right level for Board

  • Focused more on key Outputs and Goals
  • Others moved to Committee Dashboards
slide-8
SLIDE 8

10 20 30

Q1 Q2 Q3 Q4 Number of Projects

Targeted Pragmatic Broad PCORnet Engagement

NA

Needs Attention On Target Off Target

Legend

Funds Committed to Research – Budget=$640M Percent of Projects Meeting All Milestones

Board of Governors FY2015 Dashboard – Q1 (As of 12/31/2014) Our Goals: Increase Information, Speed Implementation, and Influence Research

*Influencing Research* PCORI inspired the American Heart Association to pilot the Citizen Scientist Think Tank, intended to foster patient engagement in the design of heart disease and stroke research Journal Articles Published Projects Awarded

NA=Not Applicable

10 20 30 By Awardees About or By PCORI Number of Articles

Q2 Q3 Q4 Q1

1,000 2,000 3,000 4,000 5,000 6,000 7,000 Web Views 5 10 15 20 25 30 Citations

Q2 Q3 Q4 Q1

5 10 15 20 25 30 35

Q1 Q2 Q3 Q4

Number of Projects Expected Actual

Uptake of Methodology Standards Completion of Projects Expenditures – Total Budget=$362M Progress of PCORnet

Q2=Q2 2014 Q3=Q3 2014 Q4=Q4 2014 Q1=Q1 2015

Phase II PFA Released Governance policies approved Version 2.0 of CDM Complete Aspirin Trial Awarded Obesity Cohort Project Awarded Health Systems Projects Awarded Phase II awarded

100 200 300 400 500 600

Actual Budget

$ Millions Q1 Q2 Q3 Q4 50 100 150 200 250

Other Actual Other Budget Research Actual Research Budget

$ Millions

Q1 Q2 Q3 Q4

Q1

Q1 Q2 Q3

10 20 30 40 50 60 70 80 90 100 Research Engagement PPRNs CDRNs Percent Meeting All

Q2 Q3 Q4 Q1

NA

=Target =Actual

slide-9
SLIDE 9

10 20 30

Q1 Q2 Q3 Q4 Number of Projects

Targeted Pragmatic Broad PCORnet Engagement

NA

Needs Attention On Target Off Target

Legend

Funds Committed to Research – Budget=$640M Percent of Projects Meeting All Milestones

Board of Governors FY2015 Dashboard – Q1 (As of 12/31/2014) Our Goals: Increase Information, Speed Implementation, and Influence Research

*Influencing Research* PCORI inspired the American Heart Association to pilot the Citizen Scientist Think Tank, intended to foster patient engagement in the design of heart disease and stroke research Journal Articles Published Projects Awarded

NA=Not Applicable

10 20 30 By Awardees About or By PCORI Number of Articles

Q2 Q3 Q4 Q1

1,000 2,000 3,000 4,000 5,000 6,000 7,000 Web Views 5 10 15 20 25 30 Citations

Q2 Q3 Q4 Q1

5 10 15 20 25 30 35

Q1 Q2 Q3 Q4

Number of Projects Expected Actual

Uptake of Methodology Standards Completion of Projects Expenditures – Total Budget=$362M Progress of PCORnet

Q2=Q2 2014 Q3=Q3 2014 Q4=Q4 2014 Q1=Q1 2015

Phase II PFA Released Governance policies approved Version 2.0 of CDM Complete Aspirin Trial Awarded Obesity Cohort Project Awarded Health Systems Projects Awarded Phase II awarded

100 200 300 400 500 600

Actual Budget

$ Millions Q1 Q2 Q3 Q4 50 100 150 200 250

Other Actual Other Budget Research Actual Research Budget

$ Millions

Q1 Q2 Q3 Q4

Q1

Q1 Q2 Q3

10 20 30 40 50 60 70 80 90 100 Research Engagement PPRNs CDRNs Percent Meeting All

Q2 Q3 Q4 Q1

NA

=Target =Actual

slide-10
SLIDE 10

10 20 30

Q1 Q2 Q3 Q4 Number of Projects

Targeted Pragmatic Broad PCORnet Engagement

NA

Needs Attention On Target Off Target

Legend

Funds Committed to Research – Budget=$640M Percent of Projects Meeting All Milestones

Board of Governors FY2015 Dashboard – Q1 (As of 12/31/2014) Our Goals: Increase Information, Speed Implementation, and Influence Research

*Influencing Research* PCORI inspired the American Heart Association to pilot the Citizen Scientist Think Tank, intended to foster patient engagement in the design of heart disease and stroke research Journal Articles Published Projects Awarded

NA=Not Applicable

10 20 30 By Awardees About or By PCORI Number of Articles

Q2 Q3 Q4 Q1

1,000 2,000 3,000 4,000 5,000 6,000 7,000 Web Views 5 10 15 20 25 30 Citations

Q2 Q3 Q4 Q1

5 10 15 20 25 30 35

Q1 Q2 Q3 Q4

Number of Projects Expected Actual

Uptake of Methodology Standards Completion of Projects Expenditures – Total Budget=$362M Progress of PCORnet

Q2=Q2 2014 Q3=Q3 2014 Q4=Q4 2014 Q1=Q1 2015

Phase II PFA Released Governance policies approved Version 2.0 of CDM Complete Aspirin Trial Awarded Obesity Cohort Project Awarded Health Systems Projects Awarded Phase II awarded

100 200 300 400 500 600

Actual Budget

$ Millions Q1 Q2 Q3 Q4 50 100 150 200 250

Other Actual Other Budget Research Actual Research Budget

$ Millions

Q1 Q2 Q3 Q4

Q1

Q1 Q2 Q3

10 20 30 40 50 60 70 80 90 100 Research Engagement PPRNs CDRNs Percent Meeting All

Q2 Q3 Q4 Q1

NA

=Target =Actual

slide-11
SLIDE 11

Early Signs of Influence on Research

“As PCORI has evolved, the notion of patient engagement has changed dramatically, and the American Heart Association (AHA) has sought more ways we can make the patient the focal point.”* The AHA was an early adopter of PCORI’s efforts to get people and organizations to focus

  • n patient-centeredness. For example, they developed the

Citizen Scientist Think Tank (CSTT) Pilot

  • A Community of patient and caregiver volunteers from AHA with availability and

training to help design research studies for heart disease and stroke

  • The pilot involved 30 volunteers, was successful, and AHA’s goal now is to obtain

funding and scale up to 300+

  • The AHA explained that, although they are a patient advocacy organization, most of

the research they fund is designed by clinicians and researchers, and the CSTT is intended to ensure that patients have a voice and the appropriate resources to engage in the research process

*Marko Mijic, Government Relations Manager, American Heart Association

*Influencing Research* PCORI inspired the American Heart Association to pilot the Citizen Scientist Think Tank, intended to foster patient engagement in the design of heart disease and stroke research

slide-12
SLIDE 12

Funds Committed by Type of Project and Fiscal Year

31 9 96 1 244 196 50 58 25 50 100 150 200 250 300 350 FY 2012 FY 2013 FY 2014 FY 2015 Q1 Pragmatic PFAs Targeted PFAs Broad PFAs PCORnet Pilot Projects $Millions

100 200 300 400 500 600

Actual Budget

$ Millions Q1 Q2 Q3 Q4

slide-13
SLIDE 13

Funds Committed by Type of Project: Cumulative by Fiscal Year

$31M $440M $106M $133M

As of Q1 FY 2015 Total=$710M

Pilots Broad PCORnet Targeted Pragmatic $31M $440M $105M $108M

End of FY 2014 Total=$684

$31M $244M $9M $50M

End of FY 2013 Total=$334M

100 200 300 400 500 600

Actual Budget

$ Millions Q1 Q2 Q3 Q4

slide-14
SLIDE 14

Number and Types

  • f Projects Awarded

147 121 29 13 3 7 25 50 75 100 125 150 2013 2014 Q1 2015 Q2 2015 Q3 2015 Q4 2015 Number of Projects Broad PCORnet Targeted Pragmatic Engagement

10 20 30

Q1 Q2 Q3 Q4 Number of Projects

Targeted Pragmatic Broad PCORnet Engagement

slide-15
SLIDE 15

Engagement Awards

32 22 3 36 18 7 5 10 15 20 25 30 35 40 LOIs Received* Applications Received Awards Made** Number

Q4 FY 2014 Q1 FY 2015

*The January 2015 submission cycle closed on 1/2/15. For this cycle, we received 36 LOIs by the close of Q1 FY 2015 (12/31/14), and an additional 28 LOIs by the close of the cycle (1/2/15). **Awards include only contracts that were fully executed during the quarter

10 20 30

Q1 Q2 Q3 Q4 Number of Projects

Targeted Pragmatic Broad PCORnet Engagement

slide-16
SLIDE 16

2014 2015 Q2 Q3 Q4 Q1 Number of Projects with a Progress Report Due

  • 62 with 6-month reports
  • 87 with 12-month report
  • 46 with 18-month reports

57 140 186 195 Percent of Projects Meeting All Milestones Due 63% 51% 45% 40% Average Percent of Milestones Due that Were Met 87% 85% 84% 83%

Progress of Research Projects: Milestones

We continue to refine our approach to measuring and reporting on the progress of projects and will present additional analyses as we have sufficient data for them.

10 20 30 40 50 60 70 80 90 100 Research Engagement PPRNs CDRNs Percent Meeting All

Q2 Q3 Q4 Q1

NA

slide-17
SLIDE 17

Progress of Research Projects: Additional Measures

63 76 3 51 80 4 4 45 57 91 1 4 40 56 87 1 4 10 20 30 40 50 60 70 80 90 100

Meeting All Milestones Meeting Recruitment Milestones Obtained IRB Approval on Schedule Payment Hold for Programmatic Reasons Contract Modification for Milestones Terminated

Percent

Q2 Q3 Q4 Q1

2014 2015

NA

10 20 30 40 50 60 70 80 90 100 Research Engagement PPRNs CDRNs Percent Meeting All

Q2 Q3 Q4 Q1

NA

slide-18
SLIDE 18

Actual (2014) and Expected (2015 >>) Number of Contracts Ending per Quarter

(Includes estimates for some contract modifications in process)

3 25 4 18 3 1 12 55 17 36 14 21 21 4 2 1 2 11 13 8 4 4 14

10 20 30 40 50 60 2014 Q1 2015 Q2 Q3 2015 Q4 Q1 2016 Q2 Q3 2016 Q4 Q1 2017 Q2 Q3 2017 Q4 Q1 2018 Q2 Q3 2018 Q4 Q1 2019 Q2 Q3 2019 Q4 Q1 2020 Q2 Q3 2020 Q4

Number of Projects

Fiscal Year Pilot Expected Pilot Actual CER Expected CER Actual Methods Expected Methods Actual

0 0

5 10 15 20 25 30 35

Q1 Q2 Q3 Q4

Number of Projects Expected Actual

slide-19
SLIDE 19

Number of Journal Articles Concerning PCORI

5 3 3 8 11 23 22 12 56 21 5 116 10 20 30 40 50 60 70 80 90 100 110 120 Articles Resulting from PCORI-funded Projects Articles by or about PCORI Articles that Cite or Mention PCORI Work Number of Articles

Q2 Q3 Q4 Q1

10 20 30 By Awardees About or By PCORI Number of Articles

Q2 Q3 Q4 Q1

slide-20
SLIDE 20

Uptake of Methodology Standards

6139 4768 4603 3009

1,000 2,000 3,000 4,000 5,000 6,000 7,000 Web Views 24 26 26 14 5 10 15 20 25 30 Citations Q2 2014 Q3 2014 Q4 2014 Q1 2015

We are also tracking:

  • Adoption
  • Endorsements
  • CE/CME (Q3)
  • Uptake into Curriculum
  • Use of PCORI-developed

Curriculum (2016)

slide-21
SLIDE 21

Communications: Website, E-mail, Social Media, and Media Coverage Metrics

Communications Website E-mail Social Media

Because our early Twitter stats were low, year-to-year comparisons against targets aren’t useful. We’re tracking follower growth and impressions and working on a more sophisticated reach analysis. Our Q4 unique website visitors numbers continue to beat our target. We continue to exceed industry standards for open and click-through rates by a wide margin.

Media Coverage

We continue to grow the number of mentions of our work in general/trade media.

2013 2014 2015

Q1 45 150 157 Q2 48 212 Q3 89 149 Q4 117 166 20000 40000 60000 Q2 2014 Q3 2014 Q4 2014 Q1 2015

Website Unique Traffic

Target Actual 1,000 2,000 3,000 4,000 5,000 6,000 7,000

Monthly Twitter Follower Growth

0% 10% 20% 30% Q2 2014 Q3 2014 Q4 2014 Q1 2015

Email Open Rate

Target Actual 0% 5% 10% 15% 20% Q2 2014 Q3 2014 Q4 2014 Q1 2015

Email Click-Thru Rate

10,000 20,000 30,000 40,000 50,000 60,000

Monthly Unique Traffic

2012 2013 2014

slide-22
SLIDE 22

Ambassadors Enrolled by Stakeholder Group

(Cumulative)

5 10 15 20 25 30 35 40 Q2 Q3 Q4 Q1

Ambassadors Enrolled by Geographic Region

(Cumulative)

Cumulative Total as of Q1 2015 = 165

PCORI Ambassadors Program

West: 20 Southwest: 25 Midwest: 42 Southeast: 29 Northeast: 49

slide-23
SLIDE 23

Operational Excellence

37 93 90 94 91 88 93 90 69* 96 92 20 40 60 80 100 Award to Contract Contracts HelpDesk Science HelpDesk Percent of Target w/in 90 days 100% Customer Satisfaction

Q2 Q3 Q4 Q1

*100% for Targeted (3/3) and 67% for Broad (29/43) NA

slide-24
SLIDE 24

Discussion Questions

  • Are there metrics that should be revised or retired or

moved to the Committee Dashboards?

  • Are there new or different metrics suggested by today’s

presentation and discussion? If so, are they appropriate for the Board or one of the Committees?

  • What further improvements should we consider so that our

Dashboard and accompanying materials effectively convey the status of our work and progress toward our goals?

slide-25
SLIDE 25

10 20 30

Q1 Q2 Q3 Q4 Number of Projects

Targeted Pragmatic Broad PCORnet Engagement

NA

Needs Attention On Target Off Target

Legend

Funds Committed to Research – Budget=$640M Percent of Projects Meeting All Milestones

Board of Governors FY2015 Dashboard – Q1 (As of 12/31/2014) Our Goals: Increase Information, Speed Implementation, and Influence Research

*Influencing Research* PCORI inspired the American Heart Association to pilot the Citizen Scientist Think Tank, intended to foster patient engagement in the design of heart disease and stroke research Journal Articles Published Projects Awarded

NA=Not Applicable

10 20 30 By Awardees About or By PCORI Number of Articles

Q2 Q3 Q4 Q1

1,000 2,000 3,000 4,000 5,000 6,000 7,000 Web Views 5 10 15 20 25 30 Citations

Q2 Q3 Q4 Q1

5 10 15 20 25 30 35

Q1 Q2 Q3 Q4

Number of Projects Expected Actual

Uptake of Methodology Standards Completion of Projects Expenditures – Total Budget=$362M Progress of PCORnet

Q2=Q2 2014 Q3=Q3 2014 Q4=Q4 2014 Q1=Q1 2015

Phase II PFA Released Governance policies approved Version 2.0 of CDM Complete Aspirin Trial Awarded Obesity Cohort Project Awarded Health Systems Projects Awarded Phase II awarded

100 200 300 400 500 600

Actual Budget

$ Millions Q1 Q2 Q3 Q4 50 100 150 200 250

Other Actual Other Budget Research Actual Research Budget

$ Millions

Q1 Q2 Q3 Q4

Q1

Q1 Q2 Q3

10 20 30 40 50 60 70 80 90 100 Research Engagement PPRNs CDRNs Percent Meeting All

Q2 Q3 Q4 Q1

NA

=Target =Actual