Spring 2015 Cycle: Large Pragmatic Studies to Evaluate - - PowerPoint PPT Presentation

Spring 2015 Cycle: Large Pragmatic Studies to Evaluate Patient-Centered Outcomes

Applicant Town Hall June 10, 2015

Your letter of intent (LOI) was reviewed and you have been invited to submit a full application……..

CONGRATULATIONS!

Agenda

Welcome Overview Merit Review Criteria Other topics What to think about before you apply Questions?

Submit questions via the chat function in Meeting Bridge. Ask a question via phone (an operator will standby to take your questions).

Overview Overview

David Hickam, MD, MPH Program Director Clinical Effectiveness Research

Purpose of the Pragmatic Studies PFA

To assist applicants to prepare strong proposals in response to this funding announcement: Applicants proposing “clinical comparative effectiveness research (CER)” Applicants proposing “improving healthcare systems (IHS) CER” Applicants proposing “CER to reduce or eliminate health and health care disparities”

What is a Pragmatic CER Study?

Answers a practical, real world comparative effectiveness research question. Assesses whether two or more options differ in effectiveness when administered as they are in real life Project is conducted in a clinical setting that is as close as possible to a real world setting. The methodological approach (including study design, outcome measures, and follow-up) is as simple as possible without sacrificing scientific rigor.

What is a Pragmatic CER Study? (Cont.)

Pragmatic CER trials addressing populations at risk for disparities may:

• Require higher levels of outcome data collection.
• Require tailoring of intervention (e.g., language, culture,

access).

• Consist of multi-component, multi-level interventions

(e.g., targeting the patient, provider, and systems), as evidenced by disparities literature.

Justification for the Design Elements of a Large Pragmatic Study

Suggest reviewing pragmatic–explanatory continuum indicator summary (PRECIS) tool Consider tradeoffs

• Eligibility criteria
• Flexibility of intervention
• Range and types of outcomes
• Follow up intensity
• Adherence
• Etc.

Source: A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers. Thorpe, et al. CMAJ 2009; 180:E47-E57.

Research Activities Not Supported in this PFA

Studies of decision aids Efficacy trials Evidence syntheses Cost-effectiveness analysis Research that aims to compare the overall costs of care between two or more alternatives and use the results to determine the preferred alternative

Merit Review Criteria

Impact and Potential to Improve Care Steven Clauser, PhD, MPA Program Director Improving Healthcare Systems

Merit Review Criteria

Impact of the condition on health of individuals and populations Potential for the study’s results to improve care and

• utcomes

Technical merit Patient-centeredness Engagement with key patient (including caregiver), clinician, and/or other stakeholder organizations

Impact of the Condition

In the US population, or in underserved sub- populations: Is the condition or disease associated with a significant burden - in terms of prevalence, mortality, morbidity, costs to society, individual suffering, or loss of productivity?

When the Condition is a Healthcare System Problem

In the US population, or in underserved sub- populations: Is the healthcare system problem associated with poor access to care, suboptimal care quality, high burden to society or to healthcare systems, individual suffering, mortality, morbidity, and/or loss

• f productivity?

Addressing Impact of the Condition or System Problem

Clinical CER

• State the prevalence of the condition
• Discuss the impact of the condition on the individual

Improving Healthcare Systems CER

• Describe the magnitude of the healthcare system

problem

• Discuss the impact of the system problem on healthcare

access and quality – and on individual and population suffering, morbidity, mortality, productivity and costs

Potential for the Study’s Findings to Improve Care and Outcomes

What knowledge gap underlies the problem? How does that gap manifest in practice? Would positive study findings close the gap and improve practice and outcomes? Would such findings be adopted widely?

Addressing Potential for Study Findings to Improve Care and Outcomes

Provide evidence of an important knowledge gap:

• A recent systematic review or clinical guidelines pointing to a

research need. An update of previous systematic review is also acceptable.

Provide evidence of substantial variation in practice, resulting from the knowledge gap, if known. Provide indications that the proposed CER has a substantial potential to improve practice and patient

• utcomes?
• Should refer to existing efficacy or effectiveness studies
• IHS CER should refer to prior evidence of efficacy or

effectiveness of components of current intervention, or of effectiveness of the interventions in smaller studies or other settings

Addressing Potential (cont.)

Summarize ongoing studies of the intervention Discuss the likelihood that positive study findings related to improvements in practice and patient outcomes would be implemented widely and quickly

• Breadth of current use in existing practice
• Extent of patient, practice or organizational barriers and facilitators to

rapid adoption by patients, clinicians and relevant health

• rganizations
• Applicability to diverse delivery systems
• Endorsement of the current study by key patient, physician and other

stakeholder groups.

Describe how the partners in your proposal (e.g., national and/or regional stakeholder organizations) would help disseminate the study’s findings to potential adopters

Merit Review Criteria

Technical Merit and Methodology Standards David Hickam, MD, MPH Program Director, Clinical Effectiveness Research

Technical Merit

A clear research plan with rigorous methods that adhere to PCORI’s Methodology Standards and accepted best practices A clear justification for the study design A carefully constructed and realistic timeline that includes specific scientific and engagement milestones A research team with documented appropriate expertise and experience - and a rational organizational structure An excellent research environment, including the appropriate delivery system(s) to host the study, adequate resources, and strong support of the proposed research

Addressing Technical Merit

Describe how all relevant PCORI Methodology Standards are addressed Justify the proposed sample size estimates, including the anticipated effect size and standard deviation, and all other relevant parameters Describe anticipated rates of and feasibility of recruitment Justify the proposed study design Describe the validity of the proposed measures of patient

• utcomes

Discuss proposed analytic methods, including subgroup analyses Discuss proposed approach to consent and protection of human participants

PCORI Methodology Standards

47 standards in 11 groups. The Methodology Standards do not address all issues related to study designs and methods. Note that PCORI is not using a specific set of methodological standards for “pragmatic studies.”

• Consider design tradeoffs (e.g., blinding vs not blinding)
• Refer to other respected sources for additional guidance.
• View here: http://www.pcori.org/assets/2013/11/PCORI-

Methodology-Report.pdf

Sample Size Estimate and Subgroups

Document previous relevant studies to justify use

• f the proposed effect size.

Assure appropriate adjustments based on the particular study circumstances, including the study population, expected dropout and crossover, and

• ther parameters.

Demonstrate sufficient power to analyze important pre-specified subgroups; these subgroups should be based on patient attributes with strong a priori pathophysiological or empirical justification.

Cluster Randomized Trial Design

Need justification for the use of this design

• State the benefits and disadvantages of a cluster design
• vs. individual randomization, based on the clinical issues

and the interventions being studied.

Sample size estimates need to account for reasonable estimates of intraclass correlation. Maximize the number of clusters that are randomized.

Merit Review Criteria

Patient-Centeredness and Patient Engagement Sue Sheridan, MIM, MBA, DHL Program Director, Patient Engagement

Patient-Centeredness

Is the research focused on questions that affect

• utcomes of interest to patient and their

caregivers?

• Does the research question address choices that are

important to - and faced frequently by - patients, their caregivers, or clinicians?

• Is the study powered on outcomes that are important to

patients?

Does the research address one or more of the key questions mentioned in PCORI’s definition of patient-centered outcomes research?

Addressing Patient-Centeredness

Provide evidence that the research question(s) and

• utcomes are important to patients (and/or their

caregivers)

• Have you obtained the endorsement and participation of

major patient/caregiver organizations that would use the research?

Describe your strategy for measuring outcomes that are important to patients.

Selecting Important Patient-Centered Outcomes

Do proposed outcomes include outcomes based

• n input from patients and other stakeholders?

What is the validity of this outcome measure? What is a minimal important difference in this

• utcome measure?

Have you considered the use of a previously validated patient reported outcome (PRO)?

Patient and Stakeholder Engagement

Evidence that patients, caregivers, patient and caregiver

• rganizations, clinician organizations, and other stakeholders have

been and will be engaged in:

• Formulating the research questions
• Defining the characteristics of study participants, comparators and
• utcomes
• Selecting the important outcomes to be assessed
• Monitoring study conduct and progress
• Designing plans for dissemination of study results

Clear statement of the roles and the decision-making authority of all patient and stakeholder research partners Demonstration of the principles of reciprocal relationships, co- learning, partnership, trust, transparency, and honesty

Patient-Centeredness vs. Patient Engagement

Patient-centeredness is about whether the project aims to answer questions or examine outcomes that matter to patients/caregivers. Patient engagement is about having patients/caregivers as partners in research, as

• pposed to merely being recruited as study

participants.

The Engagement Rubric

The rubric is intended to provide guidance to applicants, merit reviewers, awardees, and engagement/program

• fficers (for creating milestones and monitoring projects)

regarding engagement in the conduct of research. It is divided into four segments:

Planning the Study Conducting the Study Disseminating the Study Results PCOR Engagement Principles

Addressing Engagement

Several approaches to engagement can succeed PCORI provides many engagement resources:

• PCORI’s “The Engagement Rubric”
• http://www.pcori.org/sites/default/files/PCORI-Engagement-Rubric-with-

Table.pdf

• Sample Engagement Plans
• http://www.pcori.org/sites/default/files/announcement-

resources/PCORI-Sample-Engagement-Plans.pdf

• Engagement in Research website page
• http://www.pcori.org/content/engagement-research
• PCORI’s Methodology Standards PC-1 to PC-4
• http://www.pcori.org/assets/PCORI-Methodology-Standards1.pdf

Other Topics

(EHRs, Recruitment, Research Protections, & Dissemination) Anne Trontell, MD, MPH Senior Program Officer, Clinical Effectiveness Research

Use of Electronic Health Records and Other Computerized Data Sources

Pragmatic studies should take advantage of existing electronic clinical and demographic data whenever possible. Discuss any proposed uses of existing electronic data in the approach section:

• Cohort identification and recruitment
• Collection of covariate and outcome data

Provide evidence of the validity and completeness of available data (e.g., that all follow-up and outcomes of interest are captured)

• PCORI Standards on Data Networks and Registries

Recruitment

Discuss past experiences with recruitment Provide best available evidence to support your estimated recruitment success Consider barriers to recruitment – and how you plan to

• vercome them

Some strategies for successful recruitment

• Engaged clinical sites
• Clinician advocates for the study
• Proactive, experienced research coordinator
• Minimize disruptions in clinical care processes
• Protocol flexibility, within reason

Study Participant Protection

• Provide a Data and Safety Monitoring Plan that operates

under the auspices of your institution

• Assure that key personnel are educated on human

subjects protections

• Assure appropriate informed consent
• Establish procedures to minimize risks to participants
• Establish procedures to protect privacy and maintain

confidentiality

• If you anticipate seeking waiver of individual informed

consent, provide the rationale

• Refer to NIH standards for research involving human

subjects

Use of FDA-Regulated Medical Products

All drugs, biologics, devices, and diagnostics used in your study must be FDA-approved

• Indicate within your application whether investigational

new drug (IND) or other regulatory approval will be needed to conduct your study For example, if your study administers an approved medical product in an off-label manner (e.g. a dose, route, frequency, indication, or patient population that is unapproved by FDA) you may need an IND

• If uncertain, seek guidance from FDA or your legal

counsel

Disseminating and Implementing Research Findings

PCORI does not expect applicants to disseminate and implement findings at this time. Applicants should describe the potential for dissemination and implementation.

Describe the potential for disseminating and implementing the study results in other settings. Describe possible barriers to dissemination and implementation.

What To Think About While Completing Your Application

Iris Giggetts, MSW, CRA Contracts Associate, Pre-Award

Letter of Intent and Application

You were invited to submit a full application based

• n the information provided in the LOI, changes

after the LOI require PCORI approval.

Show stoppers include:

Changes to the PI Changes to the Institution Changes to the Study Design Changes in Engagement

PCORI Online: Application

► PI and Contact Information ► Project Information ► Key Personnel ► Milestones ► Templates and Uploads ► Save and Review

Public Abstract

Project Information

Technical Abstract Project Narratives

Milestones

► PI and Contact Information ► Project Information ► Key Personnel ► Milestones ► Templates and Uploads ► Save and Review

PCORI Monitors Projects

Deliverables: Measurable and verifiable

• utcomes or objects that a project

team must create and deliver according to the contract terms Milestones: Significant events or accomplishments within the project; may have deliverables associated with them

Example of Milestone Schedule

Templates and Uploads

► PI and Contact Information ► Project Information ► Key Personnel ► Milestones ► Templates and Uploads ► Save and Review

People and Places Template: Biosketch

You may use the NIH biosketch or PCORI’s format Biosketches are required for all key personnel List all partners within the Key Personnel section Patient/Stakeholder Biosketch Page Limit

5

Per person

People and Places Template: Project / Performance Site(s)

Demonstrate that the proposed facilities have the appropriate resources required to conduct the project to plan, within budget, and on time. Provide a description of the facilities that will be used during the project, including capacity, capability, characteristics, proximity, and availability to the project. Page Limit

15

Professional Profile/Biosketch

Budget Templates Three budget sections must be submitted as part of the online application process:

Detailed Budget Budget Summary Budget Justification NOTE:

A detailed budget is needed for each year of the

• project. Complete each budget section for the

prime applicant and any/each subcontractor.

Budget and Period Limitations

Funds & Budget Direct costs up to $10 million over the life of the project Indirect costs: up to 40% capped for the prime and sub contracts Prime organization can only charge up to 40% IDC on first $25,000 for all sub contracts combined Institutional base salary up to $200,000 The limit for Scientific Travel is $10,000 over the duration of the

• project. There is no cap on Programmatic Travel.

Period of Performance Maximum of 5 years Requests to extend are not permitted during any stage Do not anticipate receiving a cost OR no-cost time extension. Propose realistic timelines accounting for the burdens associated with obtaining IRB approval

Budget Justification

Provide a narrative that fully supports and explains the basis for the information found in the Detailed Budget Provide sufficient detail to understand the basis for costs, the reason that the costs are necessary, and an explanation for major cost variances – Provide justification as to why the costs are reasonable for the work to be performed Provide a breakdown of costs proposed for each consortia or contractor Must specify any other sources of funding that are anticipated to support the proposed research project Provide quotes, indirect cost rate letter, and fringe benefit policy

Costs of Interventions

PCORI will not cover costs for clinical care alternatives that are being compared in the project. PCORI will consider covering costs for ancillary tasks necessary in the implementation or monitoring of a clinical intervention or strategy as part of the research program.

• Examples include costs for obtaining consent,

collecting data, or monitoring that would not normally be performed in routine care Support for the study by the involved healthcare delivery systems must be documented.

Guidelines for Intervention Cost/Coverage

Costs for study interventions must be covered by delivery system, payer, manufacturer or developer of the intervention. The willingness of one or more of the stakeholder groups to cover treatment costs, even when one of the proposed intervention arms is not currently covered by insurance, will be taken as strong endorsement

• f the study by the health system or payer and of the likelihood that they

will implement or use the study‘s findings if definitive. This material support for the study by host delivery system, payer or developer should therefore be discussed in the application. In exceptional cases, PCORI may consider coverage of the co-payment

• r coinsurance costs of participating patients when that is necessary to

preserve blinding in a study or to assure access to the study for vulnerable populations. Contact PCORI with questions.

Letters of Support

All letters of support should be addressed to the PI and institution to demonstrate the commitment of key personnel and supporting

• rganizations to your proposed project.

Letters of support should clearly reflect the substantive involvement and material contribution to be provided by the signatory parties, and are meant to substantiate the commitment

• f collaboration of all forms.

Please be sure to use the Letters of Support Table provided in the funding center.

What happens to your application after you submit it?

Administrative Screening

Applicants must follow administrative requirements set in PCORI’s Application Guidelines.

► Exceeding page limits, budget, or time

limitations

► Not using PCORI’s required templates ► Submitting incomplete sections or applications

Missing the Mark

General Guidance

Refer to the Pragmatic Studies page in our Funding Center: http://www.pcori.org/funding-opportunities/announcement/large- pragmatic-studies-evaluate-patient-centered-outcomes-spring Use the applicant checklist. Adhere to the formatting, page limit, and template requirements. Convert all documents to PDF and upload to PCORI Online using the appropriate filing convention. Use Biosketches for all key personnel (scientists and patient/stakeholder team members). Address Letters of Support to the PI’s institution.

Common Application Errors

Using the wrong browser, access PCORI Online via Chrome or Safari browsers Not entering information into all required fields in the system Not clicking the ‘Save and Review’ button Having multiple people working on the application at the same time Having the incorrect file extension, only PDF files can be uploaded Not choosing the correct document type from the drop-down menu AO is unable to view the application

Resources

Refer to the Pragmatic Studies page in our Funding Center for the following resources PFA and Application Guidelines PCORI Online User Manuals Sample Engagement Plans General Applicant FAQs: Large Pragmatic Studies Applicant FAQs: PCORI Online: https://pcori.fluxx.io/ Research Methodology: http://www.pcori.org/node/4020

Submission and Key Dates

What When Application Deadline (by invitation only) July 31, 2015 by 5:00pm ET Merit Review Dates October 29-30, 2015 Awards Announced January 2016 Earliest Start Date March 2016

Learn More

www.pcori.org info@pcori.org

Questions and Answers

Submit questions via the chat function in Meeting Bridge Ask a question via phone (press 7) Contact Us:

• Programmatic Questions: E-mail sciencequestions@pcori.org
• Schedule a call at http://bit.ly/programmatic_inquiry
• Call 202-627-1884
• Administrative Questions: E-mail pfa@pcori.org or call 202-627-1885

Thank You