Broad PCORI Funding Announcement Applicant Town Hall Cycle 3 2016 - - PowerPoint PPT Presentation

November 2, 2016

Broad PCORI Funding Announcement

Applicant Town Hall Cycle 3 2016

Agenda

Research Strategy Patient Engagement and Engagement Plan Additional sections required for your application The Merit Review Process Questions

Submitting Questions:

Submit questions via the chat function in Meeting Bridge. Ask a question via phone (an operator will standby to take your questions).

Introductions

Mira Greiser, MHS Program Officer Addressing Disparities Gyasi-Moscou- Jackson, PhD, MHS, RN Program Officer Improving Healthcare Systems Mari Kimura, PhD Merit Review Officer Merit Review Lauren Massey Contracts Associate Contracts Management and Administration William Lawrence, MD, MS Senior Program Officer, Communication and Dissemination Research Greg Martin Deputy, Chief Engagement and Dissemination Officer

Your letter of intent (LOI) was reviewed and you have been invited to submit a full application

CONGRATULATIONS!

Letter of Intent and Application

You were invited to submit a full application based

• n the information provided in the LOI; changes

after the LOI require PCORI approval.

Show stoppers include:

Changes to the PI Changes to the Institution Changes to the Study Design Changes to the Budget/ Period of Performance Changes to Research Question Changes to Specific Aim Changes to Comparators

Application Requirements

Preparing Your Application

• To begin, all applicants should:
• Read the funding announcement and review the PCORI

Application Guidelines Document

• Review the PCORI Research Plan Template
• Have a copy of your approved LOI readily accessible
• Review PCORI feedback on your LOI

7

Tips for Success

• Clearly describe comparators for the study
• Document evidence of efficacy/effectiveness for the

intervention and comparator(s) and/or demonstrate that they are in widespread use

• Provide a sound rationale for why the research question of

interest is important to patients

• Clearly demonstrate the feasibility of the study

– Show that you have the team to do this and you are the right team – Define and support your recruitment and retention plan – Document that sites are already committed to participating – Include realistic timelines for site start-up, IRB approval, and recruitment

Research Strategy: Overview

Use the Research Plan Template as your guide

• Background
• Significance
• Study Design or Approach
• Specific Aims
• Methods
• Patient Population
• Recruitment Plan
• Estimated Racial/Ethnic and Gender Enrollment Table
• Research Team and Environment
• Engagement Plan

PLEASE provide all the information requested, as

• utlined in the template.

9

Page Limit 20

Research Strategy: Study Design/Approach

Document how proposed study will fill the evidence gap Include a clear conceptual framework, theory or model that anchors the background, significance, and informs the design, key variables, and relationships being tested Clearly describe and justify comparators, including usual care. Document efficacy/effectiveness and/or frequency of proposed comparator(s) in clinical practice Provide clear justification for the study design and outcome measures. Include clear rationale for sample size, power estimates, effect sizes and support rationale with evidence or pilot work Demonstrate that your study is feasible by outlining realistic strategies for participant recruitment and retention or by providing preliminary data analysis for observational approaches

Research Strategy: Methodology Standards

In any study, methods are critical. PCORI’s Methodology Committee developed Methodology Standards to which patient- centered CER must adhere

Methodology Standards: 11 Specific Categories

• Formulating Research

Questions

• Patient-Centeredness
• Data Integrity and Rigorous

Analyses

• Preventing/Handling Missing

Data

• Heterogeneity of Treatment

Effects

• Data Networks
• Data Registries
• Adaptive and Bayesian

Trial Designs

• Causal Inference
• Studies of Diagnostic

Tests

• Systematic Reviews

Research Strategy: Patient Population and Recruitment

Provide numbers for the pool of potential participants, those estimated to be eligible, and the expected participation rate Discuss past experience with recruitment of the target population Provide preliminary evidence of the potential for successful recruitment and document rationale for proposed retention rate Discuss barriers to recruitment and retention and how you plan to overcome them Discuss how and why study sites were selected Complete Recruitment Plan for Prospective Studies table Complete Estimated Racial/Ethnic and Gender Enrollment table

Research Strategy: Engagement Plan

Describe the plan to engage patients and stakeholders meaningfully in the various phases of the proposed research Planning the study Designing the study Conducting the study Evaluating the study

Research Plan Template: Dissemination & Implementation Potential

Describe the potential for disseminating and implementing the results of this research in other settings Describe possible barriers to disseminating and implementing the results of this research in other settings Describe how you will make study results available to study participants after you complete your analyses PCORI does not expect applicants to disseminate and implement findings at this time

Page Limit 2

Research Plan Template: Protection of Human Subjects

Page Limit 5 Provide a Data and Safety Monitoring Plan that operates under the auspices of your institution Ensure that key personnel are trained on human subjects protections Ensure appropriate informed consent Establish procedures to minimize risks to participants Establish procedures to protect privacy and maintain confidentiality If you anticipate seeking waiver of individual informed consent, provide the rationale Refer to NIH standards for research involving human subjects

Describe the proposed research projects that will be performed by subcontracted organizations Explain the strengths that these partners bring to the overall project

Page Limit

5

Research Plan Template: Consortium Contractual Arrangement

Research Plan Template: References Cited

Following scholarly citation practice, list the source material cited in the Research Plan.

Page Limit 10

Research Plan Template: Appendix

• PCORI applications may include an appendix for

additional materials the investigators think may be useful – Reviewers will not be required to include the appendices in the review and assessment of the project

• Examples of additional materials are:

– Survey instruments – Papers and publications

Page Limit 10

Milestones/Deliverables Template

Deliverables: Measurable and verifiable

• utcomes or objects that a

project team must create and deliver according to the contract terms Milestones: Significant events or accomplishments within the project; may have deliverables associated with them

Example of Milestone Schedule

People and Places Template: Biosketch

You may use the NIH biosketch or PCORI’s format Biosketches are required for all key personnel List all partners within the Key Personnel section Patient/Stakeholder Biosketch

Page Limit 5 Per person

People and Places Template: Project / Performance Site(s)

Demonstrate that the proposed facilities have the appropriate resources required to conduct the project to plan, within budget, and on time. Provide a description of the facilities that will be used during the project, including capacity, capability, characteristics, proximity, and availability to the project

Page Limit 15

Professional Profile/Biosketch

Leadership Plan Template ( Dual PI application)

Describe the governance and organizational structure of the leadership team and the research project Delineate the administrative, technical, scientific, and engagement responsibilities for each PI and the rationale for submitting a dual-PI application Discuss communication plans and the process for making decisions on scientific and engagement direction Describe the procedure for resolving conflicts

Page Limit 5

Letters of Support

• Save all letters of support as single PDF file prior to uploading

to the PCORI Online System

• All letters of support should be addressed to the PI and

demonstrate the commitment of key personnel and supporting

• rganizations to your proposed project
• Letters of support should clearly reflect the substantive

involvement and material contribution to be provided by the signatory parties, and are meant to substantiate the commitment of collaboration of all forms

• Each letter of support should be unique

Reminders: Tips for Success

• Clearly describe comparators for the study including a

comprehensive description of the established efficacy/effectiveness and/or widespread use

• Clearly demonstrate the feasibility of the study

– You have the team to do this and you are the right team – Recruitment goals are realistic

• Time allotted for recruitment should be realistic
• Successful recruitment strategies must be highlighted

– Research sites are already committed to participating

• Realistic about length of time it will take to get IRB at all participating sites
• Provide a sound rationale for why the research question of

interest is important to patients

Questions related to the research plan

Engagement

Patients and Stakeholders

PCORI Community

Patient/ Consumer Caregiver/ Family Member

• f Patient

Clinician

Patient/ Caregiver Advocacy Org

Hospital/ Health System

Training Institution

Policy Maker Industry Payer

Purchaser

Merit Review Criteria

Potential for study to fill critical gaps in evidence Potential for the study findings to be adopted into clinical practice and improve delivery of care Scientific merit (research, design, analysis,

• utcomes)

NEW: Investigator(s) and Environment Patient-centeredness Patient and stakeholder engagement

Patient-Centeredness

Is the research focused on questions that affect

• utcomes of interest to patient and their

caregivers?

• Does the research question address choices that are

important to - and faced frequently by - patients, their caregivers, or clinicians?

• Is the study powered on outcomes that are important to

patients?

Are the interventions being compared in the study available to patients now?

Addressing Patient-Centeredness

Provide evidence that the research question(s) and

• utcomes are important to patients (and/or their

caregivers) Describe your strategy for measuring outcomes that are important to patients Remember that a study can be patient-centered even if the end-user is not the patient, as long as patients will benefit from its information

If the end-user is not the patient, be sure to carefully describe how your study is still patient-centered

Selecting Important Patient-Centered Outcomes

Do proposed outcomes include outcomes based

• n input from patients and other stakeholders?

What is the validity of this outcome measure? What is a minimal important difference in this

• utcome measure?

Have you considered the use of a previously validated patient reported outcome (PRO)?

Patient and Stakeholder Engagement

Evidence that patients, caregivers, clinicians, and other stakeholders have been and will be engaged in:

• Formulating the research questions
• Defining the characteristics of study participants, comparators and
• utcomes
• Selecting the important outcomes to be assessed
• Monitoring study conduct and progress
• Designing plans for dissemination of study results

Clear statement of the roles and the decision-making authority of all patient and stakeholder research partners.

The Engagement Rubric

The rubric is intended to provide guidance to applicants, merit reviewers, awardees, and engagement/program

• fficers (for creating milestones and monitoring projects)

regarding engagement in the conduct of research. It is divided into three segments:

Planning the Study Conducting the Study Disseminating the Study Results

Budgeting

Financial compensation of partners Expenses of partners (transportation, childcare, caregiver) Budgeting for program staff dedicated to engagement tasks Costs of engagement meetings and events (travel, food, AV) Additional time and resource to incorporate partner feedback into various project process

Public Posting of Partner Names

Many members of the patient and stakeholder community have requested that PCORI make the names of partnering individuals and

• rganizations available to credit the contributions of the full research

team adequately You should provide PCORI only those names of patient or stakeholder partners for whom you have obtained appropriate permission to disclose their identity to PCORI and for PCORI to use their names in public communications If partners wish to remain anonymous, you may use pseudonyms or categorical descriptors (e.g., caregiver to husband with COPD, breast cancer survivor 20 years) If you are selected for funding, the individuals and organizations you provided (including those described by pseudonym or categorical descriptor) will be listed on the project description page along with the

• ther information about your project (such as abstract and PI)

Engagement Resources

PCORI’s “Engagement Rubric”

http://www.pcori.org/sites/default/files/PCORI-Engagement-Rubric- with-Table.pdf

Sample Engagement Plans

http://www.pcori.org/sites/default/files/PCORI-Sample-Engagement- Plans.pdf

PCORI Compensation Framework

http://www.pcori.org/sites/default/files/PCORI-Compensation- Framework-for-Engaged-Research-Partners.pdf

Engagement in Research website page

http://www.pcori.org/content/engagement-research

PCORI’s Methodology Standards PC-1 to PC-4

http://www.pcori.org/assets/PCORI-Methodology-Standards1.pdf

Questions related to engagement

Budget Information

Budget Templates: Overview Three budget sections must be submitted as part of the online application process:

Detailed Budget Budget Summary Budget Justification NOTE:

A detailed budget is needed for each year of the

• program. Complete each budget section for the

prime applicant and any/each subcontractor.

Budget Justification

Narrative that fully supports and explains the basis for the information in the Budget Detail – Provide sufficient detail to understand the basis for costs, the reason that the costs are necessary, and an explanation for major cost variances – Use the budget template to tell PCORI why the costs are reasonable for the work to be performed Breakdown of costs proposed for each consortia or contractor Must specify any other sources of funding that are anticipated to support the proposed research project Provide quotes, indirect cost rate letter, fringe benefit policy

Costs of Interventions

PCORI will not cover costs for clinical care alternatives that are being compared in the project. PCORI will consider covering costs for ancillary tasks necessary in the implementation or monitoring of a clinical intervention or strategy as part of the research program.

• Examples include costs for obtaining consent,

collecting data, or monitoring that would not normally be performed in routine care Support for the study by the involved healthcare delivery systems must be documented.

Guidelines for Intervention Cost/Coverage

Costs for study interventions must be covered by delivery system, payer, manufacturer or developer of the intervention. The willingness of one or more of the stakeholder groups to cover treatment costs, even when one of the proposed intervention arms is not currently covered by insurance, will be taken as strong endorsement

• f the study by the health system or payer and of the likelihood that they

will implement or use the study‘s findings if definitive. This material support for the study by host delivery system, payer or developer should therefore be discussed in the application. In exceptional cases, PCORI may consider coverage of the co-payment

• r coinsurance costs of participating patients when that is necessary to

preserve blinding in a study or to assure access to the study for vulnerable populations. Contact PCORI with questions.

Common Application Errors

Using the wrong browser, access PCORI Online via Chrome

• r Safari browsers

Not entering information into all required fields in the system Not clicking the ‘Save’ button Having multiple people working on the application at the same time Having the incorrect file extension, only PDF files can be uploaded Not choosing the correct document type from the drop-down menu AO is unable to view the application

Tips for Success

• Start and submit early
• Download PCORI’s Online User Manual for Submitting an

Online Application

• Ensure that all team members can see the application in the

system (check during the LOI stage)

• Inform your AO of your intent to submit
• Submit the completed application before the due date or on

it by 5:00 PM ET

Questions related to project budgets

What happens to your application after you submit it?

Administrative Screening

Applicants must follow administrative requirements set in PCORI’s Application Guidelines.

► Exceeding page limits, budget, or time

limitations

► Not using PCORI’s required templates ► Submitting incomplete sections or applications

Missing the Mark

Programmatic Screening

► Study deviates from approved LOI ► Study includes cost-effectiveness analysis ► Study is not responsive to PFA and/or does NOT

address the PFA’s “Research Areas of Interest”

► Study is not comparative

Missing the Mark

PFA Responsiveness PCORI Mission Advance Through to Merit Review

Merit Review

Merit Review

• Multistep process:

– Full applications screened by committee of PCORI staff for responsiveness to PFA and consistency with LOI – Preliminary (online) review – In-Person review – Post-Panel review (PCORI program staff)

• Merit review officers assign applications to reviewers based on

reviewer expertise.

Merit Review Criteria

Crosswalk of PCORI Merit Review Criteria with NIH Criteria

SIGNIFICANCE

• 1. Potential for the study to fill critical gaps in evidence
• 2. Potential for the study findings to be adopted into

clinical practice and improve delivery of care

APPROACH

• 3. Scientific merit (research design, analysis, and
• utcomes)

NEW 4. Investigator(s) and environment

PCORI-only Merit Review Criteria

PATIENT-CENTEREDNESS/ ENGAGEMENT

• 5. Patient-centeredness
• 6. Patient and stakeholder engagement

Merit Review

• PCORI guides reviewers to use the bullet points under each

merit review criterion to evaluate their assigned applications (see Merit Review section in PFA).

• Resubmissions:

– Reviewers will see applicant resubmission letters, previous critiques and previous summary statements.

Building an Inclusive Merit Review

• Panels include 3 reviewer types to bring diverse perspectives

to the merit review process.

• Each application is reviewed by 3 scientists, 1 patient and 1
• ther stakeholder

Patients Other Stakeholders Scientists Including 1 methodologist

Submission and Key Dates

What When Application Deadline December 19, 2016 by 5:00pm (ET) Merit Review Dates March 2017 Awards Announced June 2017 Earliest Start Date August 2017

Resources Available to Applicants

Visit pcori.org/apply

• Application Guidelines
• FAQs
• PCORI Online User Manuals
• Sample Engagement Plans

Schedule a Call with a Program Officer

• Submit a request at pcori.org/content/research-inquiry
• Call 202-627-1884 (programmatic inquiries)
• E-mail sciencequestions@pcori.org

Contact our Helpdesk

• E-mail pfa@pcori.org
• Call 202-627-1885 (administrative and

technical inquiries)

Q&A

Ask a question via the chat function in Meeting Bridge. Ask a question via phone (an operator will standby to take your questions).

If we are unable to address your question during this time, e-mail the Helpdesk at pfa@pcori.org.