Board of Governors Meeting Monday, December 8, 2014 10:15 a.m. - PowerPoint PPT Presentation

New and Noteworthy Items Legend� � *Influencing� Research*� � On� Target� Working� with� PCORI� influenced� Geisinger� Health� System� to� incorporate� pa ent� Target� Board� of� Governors� Off� Target� NA� engagement� into� its� strategic� plan� for� research� and� require� it� for� all� projects� FY2014� Dashboard� – � Q4� � � � Previous� Actual� Needs� A en on� Our� Goals:� Increase� Informa on,� Speed� Implementa on,� and� Influence� Research� (As� of� 9/30/2014)� X� Current� Actual� NA=Not� Applicable� Success� Rates� for� Broad� Awards� Composi on� of� Funded� Research� Por olio� Dissemina on� and� Uptake� Indicators� 100 40 % of Applications Awarded 50 7,000 25 % of Applications that were Resubmissions Q1 Other 80 6,000 % of Resubmissions Awarded Percent 40 20 30 Percent Q2 60 Treatment 5,000 Number Q3 30 15 40 4,000 Diagnosis 20 Q4 3,000 20 20 10 Prevention 10 2,000 0 10 5 FY 2013 FY 2014 All All by $ 1,000 NA� NA� 0 Percent� of� Projects� Mee ng� All� Milestones� 0 0 0 Cycle I Cycle II Cycle III August Winter Spring Journal� Ar cles� Web Views and� Citations 100 Percent Meeting All Q1 2013 2014 2014 by� Awardees� (of� Methodology� Standards)� 80 Q2 Priority� Topics� to� Be� Awarded� =� 5 Pipeline� to� Proposal� Awards� to� Be� Made� =� 65 60 Q3 40 5 Q4 � 65 5 30� 20 Q4 Q4 NA� NA� NA� 0 Research P2P PPRNs CDRNs Ambassadors� to� Be� Fully� Trained� =� 100 Funds� to� Be� Commi ed� to� Research� =� $528M Comple on� of� Research� Projects� � 100 82� Number of Projects $528M 350� 4 Q4 Expected Actual Q4 Por on� of� PCORnet� I� to� Be� Completed� � =� 1/3 Funds� to� Be� Expended� on� Research� Awards=$94M 2 � 1/3� of� $94M 113� Q4 Phase� I 0 Q4 Q1 Q2 Q3 Q4 Expense� Categories� External� Research� Commitments� Responsiveness� Other� Expenditures� Planned� =� $89M 350� 100� 100 Percent of Target Q1 300� 80� 80 � $89M Targeted� 45� Program� 250� Percent� Q2 60 60� Q4� Broad� $M� 200� 40 Program� Q3 PCORnet� 40� 150� Support� Planned� Staffing� Level� =� 164� People 20 Pilots� 100� Q4 20� NA� Admin� 0 153� � 164 50� Award to Contracts Science 0� Contract HelpDesk HelpDesk 0� FY� 2014�FY� 2015� Q4 Actual� � � � Budget� w/in 90 days 100% Customer Satisfaction FY� 2012�FY� 2013�FY� 2014� Board of Governors Meeting, December 8, 2014 17

Goal 3: Influence Research *Influencing Research* Early Signs Working with PCORI influenced Geisinger Health System to incorporate patient engagement into its strategic plan for research and require it for all projects Geisinger Health System (GHS) leadership met to evaluate their progress toward achieving goals in their 10 year research strategic plan. They discussed changes which had occurred in the research environment over those 6 years, to which they needed to adapt. They identified patient engagement in research as key area of emphasis.  GHS had received 2 PCORI awards over that period of time Patient Engagement Work Group was formed to explore the place of patient engagement in GHS research going forward  Key recommendation is that patient engagement be the default at GHS, not the exception  Work Group’s goal is to create an atmosphere conducive to engaging patients in GHS research by raising awareness and making resources available to GHS research community  Several PCORI engagement concepts were central to the work group’s thinking and subsequently appeared in their final report and recommendation to leadership Board of Governors Meeting, December 8, 2014 18

GHS Patient Engagement Framework “.. as we navigate our way toward enhancing patient engagement in research and discovery at Geisinger, PCORI has been and will continue to be both guide and anchor for that effort”* GHS attributes their new focus on patient engagement in research to: • Internal culture that has always valued a patient centered approach (readily adaptable) • External forces, such as PCORI, which have advanced the national conversation around patient engagement and provided invaluable resources (both financial and intellectual) for making it possible PCORI resources have served as a “lead impetus,”* and heavily informed GHS’s conceptualization of their own patient engagement framework Patient Engagement Goals have been built into GHS Strategic Plan and personnel goals of GHS leadership *Dan Davis, PhD, Director, GHS Bioethics and Patient Engagement Work Group Lead Board of Governors Meeting, December 8, 2014 19

Goal 2: Speed Implementation Number of Journal Articles Concerning PCORI 60 56 Q1 50 Q2 Q3 40 Number of Articles Q4 30 23 22 20 12 11 8 10 5 3 3 0 Articles by Awardees Articles by or about PCORI Articles that Cite or Mention PCORI Work Journal Articles Published by Quarter Board of Governors Meeting, December 8, 2014 20

Goal 1: Increase Information Expected Number of Contracts Ending per Quarter (Includes estimates for some contract modifications in process) 60 Pilot 56 CER 50 Methods Number of Projects 40 35 32 30 30 21 20 16 15 13 13 11 10 8 6 5 5 4 3 2 2 1 1 1 1 1 1 0 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 2014 2014 2015 2015 2016 2016 2017 2017 2018 Fiscal Year Board of Governors Meeting, December 8, 2014 21

Goal 1: Increase Information Contracts’ Help Desk Response Time 99% 99% 99% 100% 3,500 96% 3200 Tickets 3200 Tickets 3,000 80% 2,500 Percent within target Volume of tickets 60% Target: 2,000 2100 Tickets 3 1700 Tickets Business Days 1,500 40% 1,000 20% 500 0% 0 Q1 Q2 Q3 Q4 Board of Governors Meeting, December 8, 2014 22

Goal 1: Increase Information Satisfaction with Contracts Help Desk Response Data from software that handles help desk tickets Staff follow-up with all respondents who indicated that they were dissatisfied 100% 800 94% 93% 93% 93% 699 (22%) Rated 700 609 (19%) Rated 80% Number (percent) of tickets rated Percent satisfied with response 600 500 60% 400 40% 284 (17%) Rated 335 (16%) Rated 300 200 20% 100 0% 0 Q1 Q2 Q3 Q4 Industry average = 85% of respondents satisfied Board of Governors Meeting, December 8, 2014 23

Goal 1: Increase Information Science Help Desk Response Time 99% 99% 100% 700 81% 600 80% 599 Tickets 500 Percent within target 63% Volume of tickets 60% Target: 384 Tickets 400 3 Business Days 300 40% 292 Tickets 273 Tickets 200 20% 100 0% 0 Q1 Q2 Q3 Q4 Board of Governors Meeting, December 8, 2014 24

Goal 1: Increase Information Satisfaction with Science Help Desk Response Data from software that handles help desk tickets Staff follow-up with all respondents who indicate that they were dissatisfied 100% 100 91% 90% 90% 85% 80% 80 Number (percent) of tickets rated Percent satisfied with response 70 (12%) Rated 58 (15%) Rated 60% 60 53 (19%) Rated 40% 40 40 (14%) Rated 20% 20 0% 0 Q1 Q2 Q3 Q4 Industry average = 85% of respondents satisfied Board of Governors Meeting, December 8, 2014 25

Goal 1: Increase Information Applicant Satisfaction with Support Information about PCORI’s Applicant Survey Completion Rates of Applicant Surveys The survey is administered upon submission of application and is anonymous Cycle August 2013 Winter 2014 Spring 2014 Completion Rate 27% 61% 43% Number of Respondents 626 355 519 Our Applicant Survey asks about the following sources of support for applicants: pfa@pcori.org – “Contracts” sciencequestions@pcori.org – “Science” Science Program Telephone Help Line Board of Governors Meeting, December 8, 2014 26

Goal 1: Increase Information Ratings of Helpfulness of Email Support by Cycle Data from PCORI’s Applicant Survey (Contracts & Science combined) Survey Item: The email response I Survey Item: The answer I received by received answered my question email was helpful 100% Percent Agreeing or Disagreeing with Statement 151 160 Responses 122 140 80% Responses 128 Responses Number of Responses 120 60% 56% 100 80 42% 40% 36% 33% 60 27% 40 18% 20% 16% 14% 13% 9% 8% 8% 8% 20 7% 5% 0% 0 August 2013 Winter 2014 Spring 2014 Strongly Disagree Somewhat Disagree Neither Agree nor Disagree Somewhat Agree Strongly Agree Number of respondents Board of Governors Meeting, December 8, 2014 27

Goal 1: Increase Information Ratings of Helpfulness of Email Support from Contracts Compared with that from Science for Spring Cycle (only) (Note small number of respondents for Science) 100% Percent Agreeing or Disagreeing with Statement Survey Item: The email response(s) I received answered my question(s) 80% 60% 43% 42% 40% 32% 25% 17% 17% 20% 9% 9% 7% 0% 0% pfa@pcori.org (N=151, 66%) sciencequestions@pcori.org (N=12, 5%) Spring 2014 Strongly Disagree Somewhat Disagree Neither Agree nor Disagree Somewhat Agree Strongly Agree Board of Governors Meeting, December 8, 2014 28

Goal 1: Increase Information Ratings of Helpfulness of Telephone Support to Applicants Requesting a Phone Call by Cycle Data from PCORI’s Applicant Survey Survey Item: The answer(s) I received during the telephone call with a PCORI representative was helpful 100% 60 58 Percent Agreeing or Disagreeing with Statement Number who responded that they requested a call responses 90% 46 50 responses 80% 70% 40 59% 59% 60% 23 50% 30 43% responses 40% 20 30% 26% 22% 22% 22% 20% 10 11% 10% 9% 10% 5% 4% 4% 4% 0% 0% 0 August 2013 Winter 2014 Spring 2014 Strongly Disagree or No Call Occurred Somewhat Disagree Neither Agree nor Disagree Somewhat Agree Strongly Agree Number who requested a phone call Board of Governors Meeting, December 8, 2014 29

Yellow-Flagged Items – More than 10% Off Target Legend� � *Influencing� Research*� � On� Target� Working� with� PCORI� influenced� Geisinger� Health� System� to� incorporate� pa ent� Target� Board� of� Governors� Off� Target� NA� engagement� into� its� strategic� plan� for� research� and� require� it� for� all� projects� FY2014� Dashboard� – � Q4� � � � Previous� Actual� Needs� A en on� Our� Goals:� Increase� Informa on,� Speed� Implementa on,� and� Influence� Research� (As� of� 9/30/2014)� NA=Not� Applicable� X� Current� Actual� Success� Rates� for� Broad� Awards� Composi on� of� Funded� Research� Por olio� Dissemina on� and� Uptake� Indicators� 40 % of Applications Awarded 100 50 7,000 25 % of Applications that were Resubmissions Q1 Other 80 6,000 % of Resubmissions Awarded Percent 40 20 30 Percent Q2 Treatment 60 5,000 Number Q3 30 15 40 4,000 Diagnosis 20 Q4 3,000 20 20 10 Prevention 10 2,000 0 10 5 FY 2013 FY 2014 All All by $ 1,000 NA� NA� 0 Percent� of� Projects� Mee ng� All� Milestones� 0 0 0 Cycle I Cycle II Cycle III August Winter Spring Journal� Ar cles� Web Views and� Citations 100 Q1 2013 2014 2014 Percent Meeting All by� Awardees� (of� Methodology� Standards)� 80 Q2 Priority� Topics� to� Be� Awarded� =� 5 Pipeline� to� Proposal� Awards� to� Be� Made� =� 65 60 Q3 40 5 Q4 � 65 5 30� 20 Q4 Q4 NA� NA� NA� 0 Research P2P PPRNs CDRNs Ambassadors� to� Be� Fully� Trained� =� 100 Funds� to� Be� Commi ed� to� Research� =� $528M Comple on� of� Research� Projects� � 100 82� Number of Projects $528M 350� 4 Q4 Expected Actual Q4 Por on� of� PCORnet� I� to� Be� Completed� � =� 1/3 Funds� to� Be� Expended� on� Research� Awards=$94M 2 � 1/3� of� $94M 113� Q4 Phase� I 0 Q4 Q1 Q2 Q3 Q4 External� Research� Commitments� Expense� Categories� Responsiveness� Other� Expenditures� Planned� =� $89M 350� 100� 100 Percent of Target Q1 300� 80� 80 � $89M Targeted� 45� Program� 250� Percent� Q2 60 Broad� 60� Q4� $M� 200� 40 Program� Q3 PCORnet� 40� 150� Support� Planned� Staffing� Level� =� 164� People 20 Pilots� 100� 20� Q4 NA� Admin� 0 � 164 50� 153� Award to Contracts Science 0� Contract HelpDesk HelpDesk 0� FY� 2014�FY� 2015� Q4 Actual� � � � Budget� w/in 90 days 100% Customer Satisfaction FY� 2012�FY� 2013�FY� 2014� Board of Governors Meeting, December 8, 2014 30

Commitments and Expenditures Board of Governors Meeting, December 8, 2014 31

Goal 1: Increase Information Progress of Research Projects – Milestones Q1 Q2 Q3 Q4 Number of Projects with a Progress Report Due • Predominantly 6-month report only 23 57 140 186 • 80 with 12-month report also • A few with18-month report due next Q Percent of Projects Meeting All 67% 63% 51% 45% Milestones Due Average Percent of Milestones Due that 88% 87% 85% 84% Were Met Percent of Projects Meeting Recruitment Milestones (47 projects with a NA NA NA 57% recruitment milestone due by Q4 2014) We continue to refine our approach to measuring and reporting on the progress of projects and will present additional analyses as we have sufficient data for them Board of Governors Meeting, December 8, 2014 32

Goal 1: Increase Information Additional Measures of Progress of Projects 100 93% 91% Q1 Q2 Q3 Q4 90 80% 80 76% 67% 70 63% 57% 60 Percent 51% 50 45% 40 30 20 10 4% 4% 4% 3% 2% 1% 0 0 0 0 NA 0 Meeting All Meeting Obtained IRB Payment Hold for Contract Terminated Milestones Recruitment Approval on Programmatic Modification for Milestones Schedule Reasons Milestones Board of Governors Meeting, December 8, 2014 33

Discussion and Questions Are there new metrics suggested by today’s presentation and discussion? Are there metrics that should be retired? Board of Governors Meeting, December 8, 2014 34

Legend *Influencing Research* On Target Working with PCORI influenced Geisinger Health System to incorporate patient Target Board of Governors Off Target NA engagement into its strategic plan for research and require it for all projects FY2014 Dashboard – Q4 Previous Actual Needs Attention Our Goals: Increase Information, Speed Implementation, and Influence Research (As of 9/30/2014) NA=Not Applicable X Current Actual Success Rates for Broad Awards Composition of Funded Research Portfolio Dissemination and Uptake Indicators 40 % of Applications Awarded 100 50 7,000 25 % of Applications that were Resubmissions Q1 Other 80 6,000 % of Resubmissions Awarded Percent 40 20 30 Q2 Percent Treatment 60 5,000 Number Q3 30 15 40 4,000 Diagnosis 20 Q4 3,000 20 20 10 Prevention 2,000 10 0 10 5 FY 2013 FY 2014 All All by $ 1,000 NA NA Percent of Projects Meeting All Milestones 0 0 0 0 Journal Articles Cycle I Cycle II Cycle III August Winter Spring Web Views and Citations 100 Q1 2013 2014 2014 by Awardees (of Methodology Standards) Percent Meeting All 80 Q2 Priority Topics to Be Awarded = 5 Pipeline to Proposal Awards to Be Made = 65 60 Q3 40 5 5 Q4 65 30 20 Q4 Q4 NA NA NA 0 Research P2P PPRNs CDRNs Ambassadors to Be Fully Trained = 100 Funds to Be Committed to Research = $528M Completion of Research Projects 100 82 $528M 350 Number of Projects Q4 4 Expected Actual Q4 Portion of PCORnet I to Be Completed = 1/3 Funds to Be Expended on Research Awards=$94M 2 1/3 of $94M 113 Q4 Phase I 0 Q4 Q1 Q2 Q3 Q4 Expense Categories External Research Commitments Responsiveness Other Expenditures Planned = $89M 350 100 100 Percent of Target Q1 300 80 80 Targeted $89M 45 Program 250 Q2 60 Percent Broad 60 Q4 200 $M 40 Program PCORnet Q3 40 150 Planned Staffing Level = 164 People Support 20 Pilots 100 20 Q4 NA Admin 0 164 50 153 0 Award to Contracts Science 0 Contract HelpDesk HelpDesk FY 2014 FY 2015 Q4 Actual Budget FY 2012 FY 2013 FY 2014 w/in 90 days 100% Customer Satisfaction

Appendix Additional Background Slides to Respond to Questions Board of Governors Meeting, December 8, 2014 36

Goal 1: Increase Information Number of LOIs, Applications, and Awards for Broad PFAs by Cycle 1302 1200 1099 1135 1050 Letters of Intent 1038 970 1000 Letters of Intent Accepted 850 Applications 800 Applications that were Resubmissions Awards 600 557 508 Awards that were 482 461 Resubmissions 434 400 355 200 95 79 76 71 54 51 45 43 33 25 22 13 13 12 0 Cycle I Cycle II Cycle III August 2013 Winter 2014 Spring 2014 Board of Governors Meeting, December 8, 2014 37

Goal 1: Increase Information Status of Targeted Topics as of Q4 2014 Topic Status FY 2014 FY 2015  Awarded Asthma (Q1): 8 Projects, $23M  Falls (Q3): 1 Project, $30M  Care Transitions (Q4): 1 Project, $15M  Fibroids (Q4): 1 Project, $20M  Obesity (Q4): 2 Projects, $20M Year-to-Date 5 Topics, 13 Projects, $108M Total • To Be Awarded Hypertension: up to 2 Projects, $25M • 19 Pragmatic: up to 27 Projects, $270M • • Posted 19 on Pragmatic Studies List 19 on Pragmatic Studies List PFA in Development • Nearing SOC Perinatal • and/or Board Vote Hepatitis C Under 95 Topics 95 Topics Consideration Cumulative Total 5 Topics, 13 Projects, $108M Target for Year 5 Topics, ~14 Projects, $108M Up to 20 Topics, 27 Projects, $337M Board of Governors Meeting, December 8, 2014 38

Goal 1: Increase Information Funds Announced for Solicitations Compared with Funds Awarded Funding Level Announced 120 120 114 Funding Level Awarded 108 96 100 94 90 90 89 81 80 80 74 72 $Millions 68 67 60 55 41 40 23 20 20 17 20 15 15 0 Board of Governors Meeting, December 8, 2014 39

Goal 1: Increase Information Funds Committed for External Research 700 Targeted PFAs 108 600 Broad PFAs 500 PCORnet 400 $Millions Pilot Projects 440 58 300 50 200 196 244 100 105 96 31 31 9 0 FY 2012 FY 2013 FY 2014 Cumulative Board of Governors Meeting, December 8, 2014 40

Goal 1: Increase Information Composition of Current CER Portfolio (by number of projects and amount of funding) Diagnosis Prevention Other 3% Other 10% 10% Prevention 11% 15% Diagnosis 2% Treatment Treatment $492M in Total 223 Studies 77% 72% (All funded cycles to in Total date, including (All funded cycles to targeted, not including date, including Methods) targeted, not including Methods) Prevention Diagnosis Treatment Other “Other” projects include screening, systems, and those overlapping categories Board of Governors Meeting, December 8, 2014 41

Goal 3: Influence Research Progress of PCORnet at 6 months – Overall Status of PCORnet Major Deliverables at Q4  Common data model  Version 2 drafted and sent out for comment  Proposed data domains: Prescriptions Dispensed, Patient-Reported Outcomes Common Measures, Lab Results, Conditions Governance policies – 13 in development (up from 8 last quarter)   Two draft policies are at step 8 of 10 total steps (Introduction, Decision-making and Leadership)  Four draft policies are at step 7 (Distributed Research Networks, Privacy and Confidentiality, Data Security, Confidential Business Information)  Clinical trial  Issued a limited PCORI Funding Announcement (PFA) soliciting a multi-network research plan for Aspirin Trial  Principles of PCORnet Clinical Trials has been drafted and circulated to PCORI and PCORnet PIs for comment  Clinical Trials Transformative Imitative (CTTI) Quality by Design Document is drafted and out for comment  Steering Committee and Task Force – meetings occur regularly  Communications  IOM-PCORI Workshop Summary Published on Integrating Research and Practice: Health System Leaders Working Toward High-Value Care  JAMIA Theme Issue published on PCORnet, including 11 articles on CDRNs and 1 PPRN summary article  Health Affairs Big Data theme issue published, including 5 PCORnet-related articles  PCORnet staff presentations at major conferences (Health Datapalooza, Association of Health Care Journalists) Board of Governors Meeting, December 8, 2014 42

Goal 3: Influence Research Progress of PCORnet: CDRNs at 6 months Task Number of CDRNs Completing Task  PopMedNet capability established (DataMartClient Install Complete) 11/11  Significant disease-specific collaborations across 2 or more CDRNs 2/11  Submission of ETL/ADD form* for review by DSSNI operations team 5/11  Performed initial, simple query on real data 1/11  Developed plans for streamlined Institutional Review Board 11/11  Patients currently engaged in leadership and/or advisory roles 11/11  Plans for conducting clinical trials without disrupting healthcare operations 11/11  Exploring approaches to complete data 11/11 Meeting 100% of Milestones Due in Q4 – 2/11 Average Percent of Milestones Met – 85% *ETL/ADD = Extract, Transform and Load / Annotated Data Dictionary Board of Governors Meeting, December 8, 2014 43

Goal 3: Influence Research Progress of PCORnet: PPRNs at 6 months 8 RARE CONDITION PPRNs + CENA 9 COMMON CONDITION PPRNs  Data domains :  Data domains :  demographic 9/9  Demographic 5/9  vital signs 6/9  vital signs 5/9  enrollment, diagnosis data, and encounter data:  enrollment, diagnosis data, and encounter data 4/9 8/9  Patient portals launched and enrolling patients:  Patient portals :  PPRNs 5/8  Developing 9/9  CENA organizations 5/9  Beta launch 6/9  IRB Approval:  Launched 6/9  Full: 6/9  Partial: 3/9  IRB Approval: 8/9  Governance Structures Developed: 9/9  Patient Engagement : patients in  Patient Engagement : patients in governance 9/9 governance 9/9 Meeting 100% of Milestones Due Q4 : 4/9 Meeting 100% of Milestones Due Q4 : 2/9 Average Percent of Milestones Met – All PPRNs – 92% Board of Governors Meeting, December 8, 2014 44

Goal 3: Influence Research Communications: Website, E-mail, Social Media, and Media Coverage Metrics Communications Website E-mail Our Q4 unique website visitors numbers continue to beat our target. We continue to exceed industry standards for open and click-through rates by a wide margin. Monthly Unique Traffic Website Unique Traffic 60,000 Email Click-Thru Rate Email Open Rate Target Actual 60,000 50,000 30% 20% 40,000 15% 40,000 20% 30,000 10% 20,000 10% 20,000 5% 10,000 0% 0% 0 0 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 2014 2014 2014 2014 2014 2014 2014 2014 2014 2014 2014 2014 2012 2013 2014 Target Actual Social Media Media Coverage Because our early Twitter stats were low, year-to-year comparisons We continue to grow the number of mentions of our work in against targets aren’t useful. We’re tracking follower growth and general/trade media. impressions and working on a more sophisticated reach analysis. 2012 2013 2014 Monthly Twitter Follower Growth 6,000 Q1 - 45 150 5,000 Q2 47 48 212 4,000 Q3 46 89 149 3,000 2,000 Q4 26 117 166 1,000 Board of Governors Meeting, December 8, 2014 45

Goal 3: Influence Research Adherence of Awarded CER* Applications to PCORI’s Methodology Standards at Time of Award Adherence by Standard Category (average across 3 cycles – 101 applications) Standard Category % Adherence Research Questions 77% Patient-Centeredness 96% Adherence by Cycle Data Integrity/ Rigorous 82% (average across all standards) Analyses August 2013 Winter 2014 Spring 2014 Missing Data 100% 53 Applications 23 Applications 25 Applications Heterogeneity of 80% Treatment Effects Data Registries 67% 89% 92% 87% Data Networks n/a Causal Inference 95% *Does not include Methods Studies Adaptive/Bayesian Trials n/a Diagnostic Tests 80% Systematic Reviews 0% Board of Governors Meeting, December 8, 2014 46

Committee Reports Governance Committee Science Overview Committee (SOC) Engagement, Dissemination and Implementation Committee (EDIC) Research Transformation Committee (RTC) Finance and Administration Committee (FAC) Board of Governors Meeting, December 8, 2014 47

Governance Committee Report Steven Lipstein, MHA Chair Board of Governors Meeting, December 8, 2014 48 48

Background: PCORI Board Evaluation In 2013, external consultants Marla Bobowick and Debbie Hechinger conducted an evaluation of the Board’s policies, procedures and practices, and presented recommendations for enhancement. The Board endorsed almost all recommendations. Recommendations were implemented by the end of the first quarter of 2014. We continue to evaluate our progress and consider whether our governing policies, procedures and practices require further assessment and/or attention. Board of Governors Meeting, December 8, 2014 49

Key Findings and Actions Taken in 2013 Finding: Actions taken : 1. Strategic Plan approved by the There is strong value in having clarity and consensus on PCORI’s Board on November 18, 2013. strategy, focus and priorities to 2. Dashboard to measure support planning and development progress against the Strategic of metrics. Plan implemented. 3. Annual review of the Strategic Plan. Board of Governors Meeting, December 8, 2014 50

Key Findings and Actions Taken in 2013 Findings: Actions taken: The need for PCORI’s early 4. Board Meetings and Planning Sessions: governance structures and processes • Number of in-person meetings of Board to focus on operational issues should reduced. evolve to support a Board that • Number of teleconference/webinar focuses on governing. meetings of the Board increased. • Number of planning sessions reduced. Effective meeting preparation and 5. Board Meeting agendas: practices can facilitate Board • Organized around goals of the Strategic decision-making. Plan and structured to bring focus and discipline. The 21 Board members respect each • At each meeting, there is time for the other, appreciate their diverse Board to discuss issues, share new ideas backgrounds, and work together and take up additional topics. collegially. The Board generally seeks consensus, which sometimes presents challenges. Board of Governors Meeting, December 8, 2014 51

Key Findings and Actions Taken in 2013 Finding: Action taken: 6. The Chairperson, Vice- Strategies can be Chairperson, and Executive Director implemented to support the have standing weekly meetings to interaction between the discuss Board/management issues Executive Director and the Board to enhance PCORI’s and interactions, approaches to resolving problems, and to support governance and effective communications. management. Board of Governors Meeting, December 8, 2014 52

Key Findings and Actions Taken in 2013 Findings: Actions taken: 7. Committees were re-constituted and re- Board members bring strong chartered. expertise that benefits PCORI. Staff members also have strong Board oversight committees include: expertise but some are relatively • Finance and Administration new to PCORI. • Governance • Audit and Conflict of Interest Sub- Committees created during the committee start-up phase (when PCORI had • Executive Evaluation and no staff) should continue efforts to Compensation Sub-committee re-define their roles and Strategy Committees include: relationships. • Science Oversight Committee • Engagement, Dissemination and Implementation Committee • Research Transformation Committee 53 Board of Governors Meeting, December 8, 2014

PCORI Strategic Goals and Relationship to Strategy Committees Goal Generally aligns with Chair and Vice Chair Substantially increase the Science Oversight Christine Goertz, DC, PhD, Committee Chair quantity, quality, and timeliness of useful, trustworthy information available to support health decisions. Engagement, Dissemination, Debra Barksdale, PhD, RN, Speed the implementation and use of patient-centered and Implementation Chair outcomes research Committee evidence. Robert Jesse, MD, PhD, Vice Chair Influence clinical and Research Transformation Freda Lewis-Hall, MD, Chair health care research Committee funded by others to be more patient-centered. 54 Board of Governors Meeting, December 8, 2014

Science Overview Committee Report Christine Goertz, DC, PhD Chair Board of Governors Meeting, December 8, 2014 55 55

SOC’s Progress on the Objectives Objective 2014 Deliverable/Action Status • Research Portfolio Impact – Establish a plan with Developed metrics and a plan to monitor  key metrics to describe and assess the significance and evaluate the research strategy, the and potential impact of PCORI’s research strategy, overall research portfolio, merit review and and the research portfolio. patient involvement • Planning and Process Standardization – Develop and institute a plan, standard operating Established plans and SOPs for all procedures (SOPs) and metrics for: (A) topic processes (A-D). Proposed evaluation  selection, (B) merit review, (C) selection committee metrics for each process. Once metrics are process, and (D) monitoring and evaluating the agreed upon, staff will propose PCORI research portfolio (1) fiduciary aspects and (2) benchmarks. identifying scientific gaps and opportunities. Approved 3 targeted announcements (obesity, transitional care and aspirin). • Targeted Funding Announcements – Develop  priority topics for consideration including approval Recommended 1 targeted announcement of two new targeted funding announcements for for FY2014 Board approval; 2 additional large priority-driven CER trials. topics are near-term, including perinatal and a PCORnet observational study. Board of Governors Meeting, December 8, 2014 56

Engagement, Dissemination, and Implementation Committee Report Debra Barksdale, PhD, RN Chair Board of Governors Meeting, December 8, 2014 57 57

EDIC’s Progress on the Objectives Objective 2014 Deliverable/Action Status • Engagement Awards Program - Enhance our Establish close oversight to build-on a series  primary research efforts by cultivating a PCOR-ready of awards that are strategically related to community, and encourage dissemination and help advance engagement and science implementation of patient engagement practices. objectives. Complete the development of the PCORI • Dissemination and Implementation (D&I) - Dissemination and Implementation Action Facilitation of D&I through partnerships and In Progress Plan. infrastructure to engage patients and other stakeholder Engage a wide and inclusive range of key communities in D&I of PCOR. Coordinate with the patient and non-patient stakeholder groups Agency for Healthcare Research and Quality (AHRQ). such as nurses, payers, physicians, etc. • Active Portfolio Management with the PCORI Integrate rubric into PFAs, Merit Review  Engagement Rubri c – Promote the rubric to advance training, and applicant training. engagement in research. Provide options for incorporating engagement into the research process Hire, train, and implement the role of the for applicants, merit reviewers, awardees and Engagement Officers in monitoring research Engagement Officers. projects. • Non-Patient Stakeholders - Develop strategy for  Provide input on the strategy for stakeholder outreach to key non-patient stakeholder groups. outreach. • D&I Evaluation - Develop targets and milestones to Provide guidance on the development of speed implementation. policies and practices to speed In Progress implementation.

Research Transformation Committee Report Freda Lewis-Hall, MD Chair Board of Governors Meeting, December 8, 2014 59

RTC’s Progress on 2014 Objectives Objective 2014 Deliverable/Action Status •  Provide direction for the Draft plan for Phase II • development of PCORnet Draft strategy for sustainability • In Promote open science Develop PCORI strategy and Progress implementation plan for open science •  Propose PCORI strategy for co- Develop strategy for PCORI funding partnerships/alternative collaborations • funding models, and evaluate Assess collaborations under way the success of such models Board of Governors Meeting, December 8, 2014 60

Finance and Administration Committee Report Kerry Barnett, JD Chair Board of Governors Meeting, December 8, 2014 61 61

FAC’s Progress on Objectives Objective FY2014 Deliverable (s) Status Monitor financial reports and budget FY2015 approved budget Completed. Budget approved on variance to ensure timely time implementation of programs and operations within budget and provide oversight to the creation of the proposed annual operating budget Guide and monitor key issues related Meet FY2014 staffing goal of 164; Completed and ongoing. Grew from to PCORI’s key position recruitment, Improvements in employee 77 to 153 employees at end of attrition, job satisfaction and effectiveness survey results FY2014 with balance of 164 pending onboarding of new employees start date as of 10/1/2014; saw significant improvement in annual survey results Provide oversight to the development Key policies monitored and reviewed. Ongoing. Continue to monitor and revisions to key policies to policies to ensure they capture needs ensure PCORI has effectiveness, of organization. Revised pre- and efficiency and transparency in its post-award policies will be operations proceeding to the full Board this winter. Guide and support the staff in Key financial issues identified and Ongoing. Plans being developed to developing its strategy specific to the plans developed review future funding. financial issues related to PCORI’s governing legislation

PUBLIC COMMENT Join the conversation on Twitter via #PCORI 63 Board of Governors Meeting, December 8, 2014

BREAK Join the conversation on Twitter via #PCORI 64 Board of Governors Meeting, December 8, 2014

Science Oversight Committee Presentation: A Proposed 5-Year Research Funding Strategy Christine Goertz, DC, PhD Chair, Science Oversight Committee Board of Governors Meeting, December 8, 2014 65

Discussion Question How well does this 5-year research funding strategy reflect the Board’s vision? Board of Governors Meeting, December 8, 2014 66

Funding Vision: Key Recommendations and Strategic Rationale Continue but taper broad research program  Continue to evaluate and build upon existing portfolio to identify and strategically manage clusters of research projects Continue Large Pragmatic Clinical Studies program Continue to target key topics, with increasing emphasis on comprehensive studies of selected Priority Topics Encourage the use of PCORnet, when appropriate Board of Governors Meeting, December 8, 2014 67

Laying the Groundwork for an Increasingly Focused Research Portfolio • Learning how to engage patients and stakeholders in PCOR • No restrictions on clinical areas 2011 Pilots • Up to $500,000, two years • CER, patient-centeredness, stakeholder engagement required • Any clinical area in which practice could be changed 2012 Broad • Up to 1.5 million, three years • CER, patient-centeredness, more robust engagement expected • Single clinical area, with narrow question(s) Targeted • Much larger, variable funding amounts, 3-5 years 2013 • CER, patient-centeredness, rigorous engagement required • Set of high priority topics, narrow research questions Pragmatic • Up to $10 million, five years 2014 • CER, wider impact, implementation potential, prospective for partnering • Fewer, larger studies, increased emphasis on RCT’s that directly compare two High or more health care options, specific research questions, fill evidence gaps, 2015 Priority cluster of related studies, larger funding amounts Board of Governors Meeting, December 8, 2014 68

Continue to Fund Research Through Broad PFAs, but Taper Funding This line of funding solicits the research community’s best ideas under PCORI’s initial National Priorities for Research We have funded more than 250 studies with a maximum of $1.5M over up to 3 years The broad PFA remains the best way to tap into the creative cutting edge and to fund relatively young investigators Strategy going forward: because large clinical studies may be best suited to head-to-head comparisons in representative populations, funding will shift away from the broad PFA Board of Governors Meeting, December 8, 2014 69

Total Dollar Amount Committed to Current Broad Research Portfolio Broad PCORI Funding Total Number of Total Funding % of Total Announcements (PFAs) Research Awards (in $M) Funding 83 $148 34% Assessment of Options Improving Healthcare Systems 53 $105 24% Addressing Disparities 41 $71 16% Methods 58 $54 13% Communication and Dissemination 33 $57 13% Broad PFA Total: 268 $437 100% Note: This includes the Spring 2014 Cycle awards, which were announced on September 30, 2014. Board of Governors Meeting, December 8, 2014 70

Continue to Evaluate and Build Existing Research Portfolio PCORI is working to identify clusters of research in our existing portfolio as a way to actively manage our portfolio This is a key part of our detailed research strategy; the committee is working on developing the plan and timeline Provides opportunities to engage our stakeholders to help us identify gaps within a cluster Board of Governors Meeting, December 8, 2014 71

Continue PFAs for Large Pragmatic Studies Funding announcements stipulate:  Research topics that reflect national priorities for PCOR (PCORI, IOM, AHRQ)  Head-to-head comparisons in large, representative study populations We anticipate 5 funding cycles, up to $450M total commitment, and ~45 studies under way by late 2016  The first awards will be announced in February 2015 Strategic purpose: Large Pragmatic Studies may be an effective way to pursue head-to-head comparisons of active interventions in representative populations Board of Governors Meeting, December 8, 2014 72

Large Pragmatic Studies Cycle # of LOIs # of LOIs # of Awards to be Received Accepted Applications Announced Received Spring 2014 231 35 35 February 2015 Fall 2014 168 24 16 Late April 2015 Winter 2015 132 TBD TBD July/August 2015 Board of Governors Meeting, December 8, 2014 73

Targeted PFA Initiatives Funding announcements target a single research topic A Targeted PFA requires broad stakeholder support through the topic generation and prioritization process Strategic purpose: Targeted PFAs are important because they enable PCORI to move quickly to address topics of urgent national concern (e.g., Hepatitis C)  The number of projects in this program will likely increase but at a slower rate Board of Governors Meeting, December 8, 2014 74

Targeted Funding Announcement Topics Approved by the Board Targeted Funding Announcement Topics Status Funding Total Clinical Trial of a Multifactorial Fall Injury Prevention $30M Awarded Strategy in Older Persons (Administered by NIA) Clinical Interventions to Reduce Hypertension Disparities $25M Award in (Administered through NHBLI) 2015 Treatment Options for African Americans and $24M Awarded Hispanics/Latinos with Uncontrolled Asthma Obesity Treatment Options Set in Primary Care for $20M Awarded Underserved Populations PCOR Treatment Options in Uterine Fibroids (Administered $20M Awarded by AHRQ) Effectiveness of Transitional Care $15M Awarded Optimal Maintenance Aspirin Dose for Patients with $10M Award in FY Coronary Artery Disease (PCORnet demonstration trial) 2015 Targeted PFA Total: $144M Board of Governors Meeting, December 8, 2014 75

Dedicate New Resources to a Few Targeted Priority Topics Strategic purpose: to make a lasting difference, focus a portion of our resources on key topics The Priority Topic Funding Announcements will:  Be limited to a small number of high priority topics (~10)  Make an extended commitment to each selected topic  Topics identified in variety of ways (e.g. key PCORI, ARRA funded CER studies and clusters of PCORI studies within the portfolio)  Fund clusters of studies around each Priority Topic  Enhance partnership opportunities (e.g. with federal agencies, patient, caregiver and clinician organizations; payers and the life sciences sector) Board of Governors Meeting, December 8, 2014 76

Proposed Percent Distribution Annual Funding Commitment Commitment Period ($ in Millions, unless noted) Fiscal Period Research Broad Pragmatic Targeted* FY 2012-2013 $331 FY 2014 $300 69% 0% 31% FY 2015 $475 25% 55% 20% FY 2016 $400 FY 2017 $300 FY 2018 $300 FY 2019 $200 ~30% ~20% ~50% Cumulative Total ~ $2.3 B (100%) ~ $917 (40%) ~ $655 (28%) ~ $734 (32%) (%) *This will be a blend of approaches to targeted funding; some may be single targeted topics and some may be Priority Topics (clusters of studies). Board of Governors Meeting, December 8, 2014 77

Encourage the Use of PCORnet, When Appropriate PCORnet to be “research ready” late 2015 We can expect some applicants to the Broad, Large Pragmatic Studies, and Targeted PFAs will propose to use PCORnet Research in PCORnet will:  Be conducted in ‘real world’ settings  Take less time  Be less expensive Board of Governors Meeting, December 8, 2014 78

Funding Vision Recap Continue but taper Broad research program  Continue to evaluate and build upon existing portfolio to identify and strategically manage clusters of research projects Continue Large Pragmatic Clinical Studies program Continue to target key topics, with increasing emphasis on comprehensive studies of selected Priority Topics Encourage the use of PCORnet, when appropriate Board of Governors Meeting, December 8, 2014 79

Advantages of This Proposed 5 Year Funding Strategy It is evolutionary and consistent with strategy to date, building on existing investments Continues commitment to research (albeit at reduced level) Leverages our multi-stakeholder Advisory Panels prioritization efforts Engages our stakeholder community and builds off momentum created by its programs Maintains flexibility to respond to opportunity Enhances our opportunities for partnering with federal agencies and private institutions Provides for concentration of future resources to maximize adoption into practice and improvement of patient outcomes Board of Governors Meeting, December 8, 2014 80

Caveats of this Proposed 5 Year Funding Strategy While the Broad PFAs will continue, funding will be at a lower level The Large Pragmatic Studies PFA strategy depends on a continuing supply of good questions and experienced researchers Both the PFAs for Priority Topics and PCORnet research funding strategies are untried Board of Governors Meeting, December 8, 2014 81

Final Comment This proposed funding strategy is the continuing evolution of our shift to larger, targeted, more stakeholder-based patient- centered outcomes research. Board of Governors Meeting, December 8, 2014 82

Discussion Question How well does this 5-year research funding strategy reflect the Board’s vision ? Board of Governors Meeting, December 8, 2014 83

Proposal for a Hepatitis C Targeted PFA Bryan Luce, PhD, MBA Chief Science Officer, PCORI Board of Governors Meeting, December 8, 2014 84

Clinical Impact of Hepatitis C An estimated 2.7-3.9 million cases in US, majority undiagnosed Peak incidence 29,000 in 2002; incidence now ~18,000 per year over the past decade About 1/3 will develop chronic liver disease if untreated  ~ 20 years to develop liver cirrhosis  If SVR achieved on interferon regimens, 2% develop cirrhosis Accounts for about ¼ of all US cases of cirrhosis and liver cancer No evidence about the effect of new direct-acting antivirals (DAAs) on long-term outcomes – relapse, progression of liver disease or in ‘real world’ settings  Reinfection rates not known SVR = Sustained viral response Board of Governors Meeting, December 8, 2014 85

Current and Pending Treatment Options 2011: FDA approves telaprevir (Incivek) and boceprevir (Victrelis) for use in “triple therapy” with interferon  SVR rates increased from 50% to 70% 2013: FDA approves simeprevir (Olysio) and sofosbuvir (Sovaldi)  Both are used in triple therapy for genotype 1  Sofosbuvir/ribavirin double therapy for genotypes 2 and 3 October 2014: FDA approves combination drug sofosbuvir/ledipasvir (Harvoni) for genotype 1 Early 2015: Several additional agents anticipated for approval Board of Governors Meeting, December 8, 2014 86

Why a Targeted PFA on this Topic? Hepatitis C constitutes a substantial burden of illness  Prevalence, suffering, and mortality (including advanced liver disease); also patient and societal treatment costs Because of new, potentially transformational medications and as yet no long- term, ‘real world’ effectiveness or comparative effectiveness data, PCORI has an opportunity to inform practice through research A large and diverse number of stakeholders identified important clinical questions that address practical clinical issues and outcomes important to patients Board of Governors Meeting, December 8, 2014 87

Stakeholder Input PCORI’s Assessment of Options Advisory Panel met September 19 th  Panelists ranked hepatitis C #1 Multi-Stakeholder workshop held October 17 th  Recommendation: Four research topic areas received strong support in a multi-round voting process Board of Governors Meeting, December 8, 2014 88

Stakeholder Meeting Attendees Stakeholder Hepatitis C Advisory Panel on Group Workshop APDTO Clinicians 7 4 Coalition 1 0 Federal 5 0 Industry 6 3 Patients 8 7 Payers 4 1 Purchasers 1 1 Researchers 1 5 Systems 2 0 Total 35 21 Board of Governors Meeting, December 8, 2014 89

Organizations Represented at the Hepatitis C Workshop AbbVie Academy of Managed Care Pharmacy Agency for Healthcare Research and Quality American Academy of Family Physicians American Association for the Study of Liver Diseases American Association of Nurse Practitioners American College of Physicians American Gastroenterological Association American Public Health Association America's Health Insurance Plans Blue Cross and Blue Shield Association Bristol-Myers Squibb Caring Ambassadors Program Centers for Disease Control and Prevention Centers for Medicare and Medicaid Services Food and Drug Administration Gilead Sciences Board of Governors Meeting, December 8, 2014 90

Organizations Represented at the Hepatitis C Workshop (continued) Global Liver Institute Hepatitis C Association Hepatitis C Mentor and Support Group Hepatitis Education Project Hepatitis Foundation International Infectious Diseases Society of America Janssen Medicaid Medical Directors Network Merck & Co. National Business Group on Health National Institutes of Health National Pharmaceutical Council National Viral Hepatitis Roundtable OHSU/Drug Effectiveness Review Program Partnership to Improve Patient Care UnitedHealth Group Veterans Health Administration Veterans Health Council Board of Governors Meeting, December 8, 2014 91

Four Key Questions Emerged 1. Screening and Diagnosis Which screening methods and testing strategies in which settings lead to the best detection rates? 2. Care Delivery What is the comparative effectiveness of intermediate care management versus more intensive, integrated care management strategies on treatment medication adherence and sustained viral response in patients with chronic hepatitis C infection who are at high risk of non-adherence? Board of Governors Meeting, December 8, 2014 92

Four Key Questions Emerged 3. Head-to-Head Treatment Comparisons of New Therapies What are the trade-offs between long-term virologic response and adverse effects for different new regimens of oral antiviral medications, including sofosbuvir/ledipasvir and other emerging therapies for treatment of hepatitis C infection in various patient populations? Note: Because of the rapidly changing treatment landscape, PCORI will fund only adaptive head-to-head designed trials in order to integrate newly approved drugs to treat HCV. Board of Governors Meeting, December 8, 2014 93

Four Key Questions Emerged 4. Studies of the Effect of Delayed Treatment in Newly Diagnosed Hepatitis C without Serious Liver Disease What are the comparative benefits and harms of starting treatment immediately after a diagnosis of HCV infection (in the absence of serious liver disease) versus active surveillance (starting treatment only if the patient shows progression of liver disease or other manifestations of hepatitis C infection)?  The study population would be all newly diagnosed patients found to be at relatively low risk of progressive liver disease. Note: Rather than proposing a randomized comparison, PCORI will encourage rigorous, prospective observational outcomes studies based on practice and coverage variation (e.g., insurance coverage policies that differ from state to state). Board of Governors Meeting, December 8, 2014 94

Recommendation for Targeted PFA Up to four large clinical studies to test the comparative effectiveness of alternative approaches to:  screening in various patient populations  care delivery choices for Hepatitis C treatment  timing of treatment (observational)  active treatment choices Recommended by staff, endorsed by the SOC for recommendation to the Board of Governors Board of Governors Meeting, December 8, 2014 95

Budget and Project Period PCORI proposes to commit up to $50 million in total costs to fund up to four studies Maximum for any single study: $20 million total costs It is expected that project budgets and duration will vary substantially, depending on the study approach, needs for recruitment and/or primary data collection, required length of follow-up, and analytic complexity Board of Governors Meeting, December 8, 2014 96

Patient Centered and Clinical Outcomes (not exhaustive) Screening performance score Sustained viral response (SVR), time to SVR, adherence, tolerability, long term outcomes (cirrhosis, decompensated cirrhosis, utilization) Patient-reported outcomes (e.g., quality of life, cognitive function, depression, satisfaction, fatigue) Utilization of health care Evidence of liver fibrosis or cirrhosis Long term relapse Risk of reinfection Board of Governors Meeting, December 8, 2014 97

Target Populations Adult populations that may include: Persons defined solely by their birth cohort and without signs of active liver disease – (approximately 75% of all infected persons born during 1945- 1965) Hard-to-treat patients as defined by CDC (e.g., genotype 1a with cirrhosis patients, those with significant co-morbidities, those likely to be re-infected) Hard-to-reach populations (e.g., homeless, people with risky behaviors) and other target populations including incarcerated adults and pregnant women Board of Governors Meeting, December 8, 2014 98

Proposed Timeline Action Date Release Date February 5, 2015 PCORI Online System Opens February 5, 2015 Applicant Town Hall Session February 15, 2015 (Webinar) Letter of Intent Due March 6, 2015 by 5:00 PM (EST) Application Deadline May 5, 2015 by 5:00 PM (EST) Merit Review Week of August 4, 2015 Awards Announced September 2015 Board of Governors Meeting, December 8, 2014 99

Board Vote: Hepatitis C PFA Call for a Motion • Approve $50M for Hepatitis C PFA to: • Second the Motion Call for the Motion • If further discussion, may propose an Amendment to the Motion or an to Be Seconded: Alternative Motion • Vote to Approve the Final Motion Roll Call: • Ask for votes in favor, opposed, and abstentions Board of Governors Meeting, December 8, 2014 100