Broad PCORI Funding Announcement Applicant Town Hall Cycle 2 2015 - - PowerPoint PPT Presentation

September 17, 2015

Broad PCORI Funding Announcement

Applicant Town Hall Cycle 2 2015

Agenda

Research Strategy Patient Engagement and Engagement Plan Additional sections required for your application The Merit Review Process Questions

Submitting Questions:

Submit questions via the chat function in Meeting Bridge. Ask a question via phone (an operator will standby to take your questions).

Introductions

Lauren Azar, MHA Senior Program Associate Improving Healthcare Systems Suzanne Schrandt, JD Deputy Director Patient Engagement Cathy Gurgol, MS Program Officer Addressing Disparities Mari Kimura, PhD Merit Review Officer Merit Review Maricon Gardner, CRA Contracts Associate, Pre-Award Contracts Management and Administration

Your letter of intent (LOI) was reviewed and you have been invited to submit a full application……..

CONGRATULATIONS!

Letter of Intent and Application

You were invited to submit a full application based

• n the information provided in the LOI; changes

after the LOI require PCORI approval.

Show stoppers include:

Changes to the PI Changes to the Institution Changes to the Study Design Changes to the Budget/ Period of Performance Changes to Research Question Changes to Specific Aim Changes to Comparators

Preparing your application for submission – Research Strategy

First things first……. Get prepared

• Read and review the PCORI Application Guidelines

Document http://www.pcori.org/sites/default/files/PCORI-PFA- Cycle 2 2015 Application-Guidelines.pdf

• Re-read the funding announcement for which you

are applying http://www.pcori.org/funding/opportunities

• Review the PCORI Research Plan Template
• Have a copy of your approved LOI readily

accessible

►

Research Strategy

►

Dissemination & Implementation Potential

►

Replication & Reproducibility of Research and Data Sharing

►

Protection of Human Subjects

►

Consortium Contractual Arrangements

►

References Cited

►

Appendix

Research Plan Template

Research Strategy

Maximum 20 pages in length Use the Research Plan Template as your guide

• Background
• Significance
• Study Design/Approach
• Project Milestones/Timeline
• Patient Population
• Recruitment Plan
• Estimated Racial/Ethnic and Gender Enrollment Table
• Research Team and Environment
• Engagement Plan

PLEASE provide all the information requested, as outlined in the template.

Page Limit

20

Research Strategy

Two items to note:

• Research questions and specific aims MUST be

the same as in the approved LOI unless the applicant has explicit and documented approval from the program to which they will submit.

• While completing the study design/approach

section, applicants should cite PCORI’s Methodology Standards.

http://www.pcori.org/assets/2013/11/PCORI-Methodology- Report.pdf

Methodology Standards

In any study, methods are critical. PCORI’s Methodology Committee developed Methodology Standards to which patient- centered CER must adhere

Methodology Standards: 11 Broad Categories

• Formulating Research

Questions

• Patient-Centeredness
• Data Integrity and Rigorous

Analyses

• Preventing/Handling Missing

Data

• Heterogeneity of Treatment

Effects

• Data Networks
• Data Registries
• Adaptive and Bayesian

Trial Designs

• Causal Inference
• Studies of Diagnostic

Tests

• Systematic Reviews

PCORI Monitors Projects

Deliverables: Measurable and verifiable

• utcomes or objects that a

project team must create and deliver according to the contract terms Milestones: Significant events or accomplishments within the project; may have deliverables associated with them

Example of Milestone Schedule

Dissemination & Implementation

Describe the potential for disseminating and implementing the results of this research in other settings. Describe possible barriers to disseminating and implementing the results of this research in other settings. Describe how you will make study results available to study participants after you complete your analyses. PCORI does not expect applicants to disseminate and implement findings at this time.

Page Limit

2

Replication & Reproducibility of Research and Data Sharing

Describe the ability to reproduce potentially important findings from this research in other data sets and populations. Describe how you will make a complete, cleaned, de- identified copy of the final data set used in conducting the final analyses available within 90 days of the end of the final year of funding, or your data-sharing plan, including the method by which you will make this data set available, if requested. Propose a budget to cover costs of your data-sharing plan, if requested.

Page Limit 2

Describe the protection of human subjects who will be involved in your research. Refer to NIH standards for research involving human subjects Page Limit

5

Protection of Human Subjects

Study Participant Protection

Provide a Data and Safety Monitoring Plan that operates under the auspices of your institution Assure that key personnel are educated on human subjects protections Assure appropriate informed consent Establish procedures to minimize risks to participants Establish procedures to protect privacy and maintain confidentiality If you anticipate seeking waiver of individual informed consent, provide the rationale Refer to NIH standards for research involving human subjects

Describe the proposed research projects that will be performed by subcontracted organizations; explain the strengths that these partners bring to the overall project. Page Limit

5

Consortium Contractual Arrangement

References Cited

Following scholarly citation practice, list the source material cited in the Research Plan. Page Limit

10

Appendix

PCORI applications may include an appendix for additional materials the investigators think may be useful Examples of additional materials are:

– Survey instruments – Papers and publications from members of the

research team; however, reviewers will not be required to include the appendices in the review and assessment of the project

Page Limit

10

Recruitment

Discuss past experiences with recruitment of the target population Provide preliminary evidence of the potential for successful recruitment Consider barriers to recruitment – and how you plan to

• vercome them

Strategies for successful recruitment – Engaged clinical sites – Clinical advocates – Proactive, experienced research coordinator

Application Common Mistakes

Inadequate support for the research question based on systematic reviews or identified gaps in clinical guidelines Studies of efficacy, not comparative effectiveness Low likelihood of study impact Limited, inappropriate, or non-specific patient-centered

• utcomes

Inappropriate use or explanation of usual care as a comparator

Missing key aspects of scientific design, such as well- justified power calculations Overly optimistic timeframe for enrollment -- failure to think through complexities of recruitment and enrollment Lack of support for key statements in the Research Plan Inadequately justified budgets

Application Common Mistakes

Addressing engagement in your application

Patient-Centeredness; do the question(s) and outcomes matter and are they important to patients and their caregivers? Patient and Stakeholder Engagement; are patients and

• ther stakeholders involved in meaningful ways

throughout the research process? Remember; patient partners are not the same as patient subjects or participants Remember: there are eight stakeholder categories in addition to patients and caregivers

Patient-Centeredness vs. Patient and Stakeholder Engagement

The Engagement Rubric

Planning the Study Conducting the Study Disseminating the Study Results PCOR Engagement Principles

Potential activities include but are not limited to;

Identifying the research question and outcomes Drafting study protocol

Examples;

“Mental Health” study name and materials amended by

partners Diabetes study; clinician partners amended study design to more accurately reflect real-world practice settings

Planning the Study

Potential activities include but are not limited to;

Recruitment and retention of study participants Data collection Data analysis

Examples;

Patient partners in cancer imaging studies amended inclusion criteria to be more reflective realistic patient actions Parent partners in pediatric psychiatry study are delivering components of the intervention

Conducting the study

Potential activities include but are not limited to;

Co-authoring publications such as journal articles Co-presenting at conferences Using social media outlets

Examples;

Neurology study results are being shared at a neurology patient advocacy organization conference Patient partners are co-authoring journal articles on a pain study

Disseminating study results

Six Engagement Principles

• Partnership
• Co-learning
• Reciprocal relationships
• Trust
• Transparency
• Honesty

Embodying the Engagement Principles

Real-World Examples

• Providing financial compensation to patient and

stakeholder partners

• Providing training opportunities for patient,

researcher, and stakeholder partners

• Scheduling and locating meetings at mutually

accessible times and locations

Embodying the Engagement Principles (Cont’d)

Addressing engagement

“Show your work” Provide letters of support Budget appropriately New this cycle

Public posting of patient and stakeholder partner names

If a proposal is selected for funding, these names will be included with the name of the PI and the public abstract on our website By providing names of individuals or organizations, you are indicating that you have received any appropriate authorizations or permission to do so If you have patient or stakeholder partners who wish to remain anonymous, please contact pfa@pcori.org for guidance on how to proceed

Compensation Framework

Engagement Resources

PCORI’s “Engagement Rubric”

http://www.pcori.org/sites/default/files/PCORI-Engagement-Rubric- with-Table.pdf

Sample Engagement Plans

http://www.pcori.org/sites/default/files/PCORI-Sample-Engagement- Plans.pdf

PCORI Compensation Framework

http://www.pcori.org/sites/default/files/PCORI-Compensation- Framework-for-Engaged-Research-Partners.pdf

Engagement in Research website page

http://www.pcori.org/content/engagement-research

PCORI’s Methodology Standards PC-1 to PC-4

http://www.pcori.org/assets/PCORI-Methodology-Standards1.pdf

Other sections required for your application

pcori.org/apply

PCORI Online: Application

► PI and Contact Information ► Project Information ► Key Personnel ► Milestones ► Templates and Uploads ► Save

Public Abstract

Project Information

Technical Abstract Project Narratives

Key Personnel

► PI and Contact Information ► Project Information ► Key Personnel ► Milestones ► Templates and Uploads ► Save and Review

Milestones

► PI and Contact Information ► Project Information ► Key Personnel ► Milestones ► Templates and Uploads ► Save and Review

People and Places Template - Biosketch

You may use the NIH biosketch or PCORI’s format Biosketches are required for all key personnel List all partners within the Key Personnel section Patient/Stakeholder Biosketch Page Limit

5

Per person

People and Places Template – Project / Performance Site(s)

Demonstrate that the proposed facilities have the appropriate resources required to conduct the project to plan, within budget, and on time. Provide a description of the facilities that will be used during the project, including capacity, capability, characteristics, proximity, and availability to the project Page Limit

15

Professional Profile/Biosketch

Letters of Support

Save all letters of support as single PDF file prior to uploading to the PCORI Online System. All letters of support should be addressed to the PI and demonstrate the commitment of key personnel and supporting

• rganizations to your proposed project.

Letters of support should clearly reflect the substantive involvement and material contribution to be provided by the signatory parties, and are meant to substantiate the commitment of collaboration of all forms. Letters of support should be organized in the following manner:

• Letters of organizational support
• Letters of collaboration
• Letters confirming access to patient populations, data sets, and

additional resources

Budget Templates: Overview Three budget sections must be submitted as part of the online application process:

Detailed Budget Budget Summary Budget Justification NOTE:

A detailed budget is needed for each year of the

• program. Complete each budget section for the

prime applicant and any/each subcontractor.

Detailed Budget

• Personnel
• Consultant
• Equipment
• Supplies
• Travel
• Other Expenses
• Consortium/Contractual Direct Costs
• Consortium/Contractual Indirect Costs
• Prime Indirect Costs

Budget Justification

Narrative that fully supports and explains the basis for the information in the Budget Detail – Provide sufficient detail to understand the basis for costs, the reason that the costs are necessary, and an explanation for major cost variances – Use the budget template to tell PCORI why the costs are reasonable for the work to be performed Breakdown of costs proposed for each consortia or contractor Must specify any other sources of funding that are anticipated to support the proposed research project Provide quotes, indirect cost rate letter, fringe benefit policy

Allowable and Unallowable Costs

44

NOTE

Review Appendix 3 of Application Guidelines for updates

• n allowability of patient and clinical care costs.

Justifying Commonly Denied Items

You must do the following:

• Provide a detailed explanation for why the purchase of

the requested item(s) is required for project completion.

• Include a statement verifying the requested item(s) is

not currently available for PI use.

• Include a statement that the requested item(s) will be

purchased in accordance to applicable cost principles.

NOTE

Items purchased under PCORI funded projects are not to be used as incentives to recruit or retain graduate students or other personnel.

Templates and Uploads

► PI and Contact Information ► Project Information ► Key Personnel ► Milestones ► Templates and Uploads ► Save and Review

Common Application Errors

Using the wrong browser, access PCORI Online via Chrome

• r Safari browsers

Not entering information into all required fields in the system Having multiple people working on the application at the same time Having the incorrect file extension, only PDF files can be uploaded Not choosing the correct document type from the drop-down menu AO is unable to view the application

Tips for Success

• Adhere to the PFA and Application Guidelines for the funding

cycle you are applying to

• Talk to a Program Officer if you have questions
• Start and submit early
• Download PCORI’s Online User Manual for Submitting an

Online Application

• Ensure that all team members can see the application in the

system (check during the LOI stage)

• Inform your AO of your intent to submit
• Submit the completed application before the due date or on

it by 5:00 PM ET

Resources

Refer to the funding opportunities page in our Funding Center (http://www.pcori.org/funding/opportunities) for the following resources:

PFA and Application Guidelines PCORI Online User Manuals Sample Engagement Plans General Applicant FAQs: http://bit.ly/applicant_faqs PCORI Online: https://pcori.fluxx.io/ Research Methodology: http://www.pcori.org/node/4020

What happens to your application after you submit it?

Administrative Screening

Applicants must follow administrative requirements set in PCORI’s Application Guidelines.

► Exceeding page limits, budget, or time

limitations

► Not using PCORI’s required templates ► Submitting incomplete sections or applications

Missing the Mark

Programmatic Screening

► Study deviates from approved LOI ► Study includes cost-effectiveness analysis ► Study is not responsive to PFA and/or does NOT

address the PFA’s “Research Areas of Interest”

► Study is not comparative

Missing the Mark

PFA Responsiveness PCORI Mission Advance Through to Merit Review

Merit Review

Building an Inclusive Merit Review

• Panels include 3 reviewer types to bring diverse

perspectives to the merit review process.

• Each application is reviewed by 2 scientists,1 patient,

and 1 other stakeholder.

• The panel chair facilitates discussion and promotes a

culture of mutual respect and understanding among reviewer types. Patients Other Stakeholders Scientists

Merit Review Criteria

Criterion #1: Impact of the condition on the health of individuals and population Criterion #2: Potential for the study to improve healthcare and outcomes Criterion #3: Technical merit Criterion #4: Patient-centeredness Criterion #5: Patient and other stakeholder engagement Patient and Other Stakeholder Reviewers Scientist Reviewers

       

Scoring Range

Range Score Descriptor Characteristics High 1 Exceptional Exceptionally strong with essentially no weaknesses 2 Outstanding Extremely strong with negligible weaknesses 3 Excellent Very strong with only some minor weaknesses Medium 4 Very Good Strong but with numerous minor weaknesses 5 Good Strong but with at least one moderate weakness 6 Satisfactory Some strengths but also some moderate weakness Low 7 Fair Some strengths but with at least one major weakness 8 Marginal A few strengths and a few major weaknesses 9 Poor Very few strengths and numerous major weaknesses

A score of 1 indicates an exceptionally strong application. A score of 9 indicates an application with serious and substantive weaknesses.

How Are Applications Reviewed?

Impact of the condition on the health of individuals/populations Potential for the study to improve healthcare and outcomes Technical merit Patient-centeredness Patient and stakeholder engagement

Reviewers evaluate applications using five criteria:

• PCORI’s Board of

Governors makes funding decisions based on merit review and staff recommendations.

Impact of the condition on the health of individuals and populations

Is the condition or disease associated with a significant burden in the US population, in terms of prevalence, mortality, morbidity, individual suffering, or loss of productivity? Alternatively, does the condition or disease impose a significant burden on a smaller number of people who have a rare disease? Does the proposal include a particular emphasis on patients with one or more chronic condition?

Potential for the study to improve healthcare and outcomes

Does the research question address a critical gap in current knowledge as noted in systematic reviews, guideline development efforts, or previous research prioritizations? Has it been identified as important by patient, caregiver, or clinician groups? Do wide variations in practice patterns suggest current clinical uncertainty? Is the research novel or innovative in its methods or approach, in the population being studied, or in the intervention being evaluated in ways that make it likely to improve care? Do preliminary studies indicate potential for a sizeable benefit of the intervention relative to current practice? How likely is it that positive findings could be disseminated and implemented quickly, resulting in improvements in practice and patient

• utcomes?

Technical Merit

Does the proposal delineate a clear conceptual framework/theory/model that anchors the background literature and informs the design, key variables, and relationships being tested? Are the comparison interventions realistic options that exist in current practice? Are sample size and power estimates presented that are based on realistic and careful evaluations of the anticipated effect size? Is the project timeline realistic, including specific scientific and engagement milestones? Does the research team have the necessary expertise to conduct the project? Is the organizational structure and are the described resources appropriate to carry out the project? Is there a diverse study population with respect to age, gender, race, ethnicity, and clinical status, appropriate for the proposed research?

Patient-centeredness

Is the research focused on questions that affect outcomes of interest to patients and their caregivers? Does the research address one or more of the key questions mentioned in PCORI’s definition of patient-centered

• utcomes research?

– “Given my personal characteristics, conditions, and preferences, what should I expect will happen to me?” – “What are my options, and what are the potential benefits and harms of those options?” – “What can I do to improve the outcomes that are most important to me?” – “How can clinicians and the care delivery systems they work in help me make the best decisions about my health and health care?”

Patient and stakeholder engagement

Are patients and stakeholders engaged in:

• Formulating research questions
• Defining essential characteristics of study participants, comparators, and
• utcomes
• Identifying and selecting outcomes that the population of interest notices

and cares about (e.g., survival function, symptoms, health-related quality of life) and that inform decision making relevant to the research topic

• Monitoring study conduct and progress
• Designing/suggesting plans for dissemination and implementation activities

Are the roles and the decision making authority of all research partners clearly stated? Does the proposal demonstrate the principles of reciprocal relationships, co-learning, partnership, trust, transparency, and honesty?

Multi-phase process for review Preliminary (online) review In-Person review Post-Panel review (PCORI program staff)

Submission and Key Dates

What When Application Deadline November 3, 2015 by 5:00pm ET Merit Review Dates February 2016 Awards Announced April 2016 Earliest Start Date June 2016

Where can I find help?

Visit pcori.org/apply

• Application Guidelines
• FAQs
• PCORI Online User Manuals
• Sample Engagement Plans

Schedule a Call with a Program Officer

• Submit a request at pcori.org/content/research-inquiry
• Call 202-627-1884 ( programmatic inquiries)
• E-mail sciencequestions@pcori.org

Contact our Helpdesk

• E-mail pfa@pcori.org
• Call 202-627-1885 ( administrative and

technical inquiries)

Q&A

Ask a question via the chat function in Meeting Bridge. Ask a question via phone (an operator will standby to take your questions).

If we are unable to address your question during this time, e-mail the Helpdesk at pfa@pcori.org.