Cycle 1 2017: Pragmatic Clinical Studies to Evaluate - - PowerPoint PPT Presentation

Cycle 1 2017: Pragmatic Clinical Studies to Evaluate Patient-Centered Outcomes

Applicant Town Hall Date April 13, 2017

Your letter of intent (LOI) was reviewed and you have been invited to submit a full application……..

CONGRATULATIONS!

Agenda

Welcome Overview Merit Review Criteria Other topics What to think about before you apply Questions?

Submit questions via the chat function in Meeting Bridge. Ask a question via phone (an operator will standby to take your questions).

Overview Overview Anne Trontell, MD. MPH

Associate Director Clinical Effectiveness and Decision Science

Purpose of Town Hall

To assist applicants to prepare strong proposals in response to this funding announcement: Applicants proposing “clinical comparative effectiveness research (CER)” Applicants proposing “improving healthcare systems (IHS) CER” Applicants proposing “CER to reduce or eliminate health and health care disparities” Applicants proposing “CER to communicate or disseminate research results to patients, caregivers,

• r clinicians”

What is a Pragmatic CER Study?

Answers a practical, real world comparative effectiveness research question – a decisional dilemma Assesses whether two or more options differ in effectiveness when administered as they are in real life (vs. tightly controlled research situations) The project is conducted in a clinical setting that is as close as possible to a real world setting. The methodological approach (including study design,

• utcome measures, and follow-up) is as simple as

possible without sacrificing scientific rigor.

Goals of the PCS PFA

• Helps address important evidence gaps for

which the lack of sufficient evidence is a barrier to clinical and/or health care system decision making. The evidence gap must be meaningful and central to the decision making context.

• Provides insight on heterogeneity of

treatment effects by funding studies with sufficiently large sample sizes.

Justify the Design Elements in Your Large Pragmatic Study

Suggest reviewing pragmatic–explanatory continuum indicator summary (PRECIS) tool Consider tradeoffs in relation to your research question

• Eligibility criteria
• Flexibility of intervention
• Range and types of outcomes
• Follow up intensity
• Adherence
• Etc.

Source: A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers. Thorpe, et al. CMAJ 2009; 180:E47-E57.

Research Activities Not Supported in this PFA

Studies to develop, validate, or test the efficacy of new or existing decision support tools Efficacy trials Evidence syntheses Cost-effectiveness analysis Research that aims to compare the overall costs of care between two or more alternatives and use the results to determine the preferred alternative

Healthcare Delivery and Disparities Research

Parag Aggarwal, PhD Senior Program Officer

Studies Specifically Addressing Disparities

We seek pragmatic clinical trials to test the comparative effectiveness of evidence-based interventions set in real world settings for patients who are at risk for experiencing disparities in outcomes Study should test the ability of interventions to improve

• utcomes and reduce disparities for at risk populations

Target populations include:

• Racial and ethnic minorities
• Low-income groups
• Residents of rural areas
• Individuals with disabilities
• Patients with low health literacy/numeracy and limited English

proficiency

• Lesbian, gay, bisexual, transsexual

Studies Specifically Addressing Disparities (cont.)

Studies should measure a broad range of patient-centered, clinical, and structural outcomes that contribute to reducing disparities in care We encourage applications from organizations that serve majority underserved populations (e.g., safety net settings) Studies addressing populations at risk for disparities may:

• Require a more frequent, comprehensive and diverse set of
• utcomes data to adequately capture the impact of the intervention

(e.g., follow-up that takes place outside of clinical setting, additional validated measures to assess outcomes).

• Require tailoring of intervention (e.g., language, culture, access).
• Consist of multi-component, multi-level interventions (e.g., targeting

the patient, provider, and systems), as evidenced by disparities literature.

Merit Review Criteria

Potential to Fill Evidence Gaps and Improve Care

Steve Clauser, PhD, MPA Program Director Healthcare Delivery and Disparities Research

Merit Review Criteria

Crosswalk of PCORI Merit Review Criteria with NIH Criteria SIGNIFICANCE

• 1. Potential for the study to fill critical gaps in

evidence

• 2. Potential for the study findings to be adopted

into clinical practice and improve delivery of care APPROACH

• 3. Scientific merit (research design, analysis, and
• utcomes)
• 4. Investigator(s) and environment

PCORI-only Merit Review Criteria PATIENT-CENTEREDNESS/ ENGAGEMENT

• 5. Patient-centeredness
• 6. Patient and stakeholder engagement

Potential to fill critical gaps

A clear clinical burden A critical gap in current knowledge based on systematic reviews, clinical practice guidelines, or previous research prioritizations Variations in practice patterns that suggest clinical uncertainty Decisional dilemmas experienced by patients and

• ther stakeholders

The application should clearly define and describe: Does the study have the potential to fill these gaps and inform decision making for key stakeholders

Addressing Potential to Fill Critical Gaps

Clinical CER

• Describe the decisional dilemma from information gathered

directly from stakeholders and patients, or based on evidence synthesis efforts and prioritization

Improving Healthcare Systems CER

• Discuss the impact of the system problem on healthcare

access and quality – and on individual and population suffering, morbidity, mortality, productivity and costs

CER to Address Disparities

• Describe the critical gaps in evidence related to health, health

care, and health outcomes for sub-populations with known health disparities

Potential for the Study’s Findings to be Adopted into Clinical Practice and Improve Delivery of Care

Who are the end-users and how would the information from this study support a demand for information from end-users? How likely are the findings to be reproduced by others? What are the barriers? What is the dissemination plan for study results, beyond traditional publication and presentation?

Describe how evidence generated from this study could be adopted into clinical practice and care delivery

Addressing Potential for Study Findings to be Adopted into Clinical Practice

Provide indications that the proposed CER has a substantial potential to improve practice and patient

• utcomes
• Should refer to existing efficacy or effectiveness studies
• IHS CER should refer to prior evidence of efficacy or

effectiveness of components of current intervention, or of effectiveness of the interventions in smaller studies or other settings

• CER to Address Disparities should refer to existing efficacy
• r effectiveness studies in the target population where

possible, or highlight efficacy or effectiveness studies in the general population that may be promising for the target population

Addressing Potential (cont.)

Discuss the likelihood that positive study findings related to improvements in practice and patient outcomes would be implemented widely and quickly

• Breadth of current use in existing practice
• Extent of patient, practice or organizational barriers and facilitators

to rapid adoption by patients, clinicians and relevant health

• rganizations
• Applicability to diverse delivery systems
• Endorsement of the current study by key patient, physician and
• ther stakeholder groups.

Describe how the partners in your proposal (e.g., national and/or regional stakeholder organizations) would help disseminate the study’s findings to potential adopters

Merit Review Criteria

Scientific Merit and Methodology Standards; Investigators and Environment Mari Kimura, PhD Merit Review Officer

Scientific Merit

A clear research plan with rigorous methods that adhere to PCORI’s Methodology Standards and accepted best practices A clear justification for the study design and outcome measures Clearly described and justified comparators Sample sizes and power estimates based on careful evaluations of the anticipated effect size Feasibility

A carefully constructed and realistic timeline that includes specific scientific and engagement milestones Realistic strategies for participant recruitment and retention

PCORI Methodology Standards

47 standards in 11 groups. The Methodology Standards do not address all issues related to study designs and methods. Note that PCORI is not using a specific set of methodological standards for “pragmatic studies.”

• Consider design tradeoffs (e.g., blinding vs not blinding)
• Refer to other respected sources for additional guidance.
• View here: http://www.pcori.org/assets/2013/11/PCORI-

Methodology-Report.pdf

Sample Size Estimate and Subgroups

Document previous relevant studies to justify use

• f the proposed effect size.

Assure appropriate adjustments based on the particular study circumstances, including the study population, expected dropout and crossover, and

• ther parameters.

Demonstrate sufficient power to analyze important pre-specified subgroups; these subgroups should be based on patient attributes with strong a priori pathophysiological or empirical justification.

Cluster Randomized Trial Design

Need justification for the use of this design

• State the benefits and disadvantages of a cluster design
• vs. individual randomization, based on the clinical issues

and the interventions being studied.

Sample size estimates need to account for reasonable estimates of intraclass correlation. Maximize the number of clusters that are randomized.

Investigators and Environment

Need to demonstrate team’s experience, leadership approach, governance, and

• rganizational structure are appropriate to

achieving project’s goals Need to demonstrate resources and institutional support to conduct project as planned, within budget, and on time Dual PIs must submit Leadership Plan

Merit Review tips

Address the bulleted points under each merit review criterion in the PFA

• We refer reviewers to these bulleted points.

Remember that panels consist of scientist, patient and stakeholder reviewers. Resubmissions

• Reviewers see resubmission letter and previous summary

statement

• Will sometimes but not always be read by the same reviewers
• We guide reviewers to use the merit review criteria but also to

note if applications have improved based on previous feedback.

Merit Review Criteria

Patient-Centeredness and Patient & Stakeholder Engagement Kristin Carman, MA, PhD Director of Public and Patient Engagement Public and Patient Engagement

Criterion 5: Patient-Centeredness

The application should demonstrate that the study focuses

• n improving patient-centered outcomes and employs a

patient-centered research design (i.e., a design informed or endorsed by patients).

• Does the application include a thorough description about which
• utcomes (both benefits and harms) are important to patients, and

are those outcomes included in the study plan?

• Does the application provide information that indicates that closing

the evidence gap is important to patients and other stakeholders?

• Are the interventions being compared in the study available to

patients now, and are they the best options for comparison (including whether they would be chosen by patients and their healthcare providers for managing the condition being studied)?

Criterion 5: Patient Centeredness

Remember that a study can be patient-centered even if the end-user is not the patient, as long as patients will benefit from its information.

If the end-user is not the patient, be sure to carefully describe how your study is still patient-centered.

Selecting Important Patient-Centered Outcomes

Do proposed outcomes include outcomes based

• n input from patients and other stakeholders?

What is the validity of this outcome measure? What is a minimal important difference in this

• utcome measure?

Have you considered the use of a previously validated patient reported outcome (PRO)?

Criterion 6: Patient and Stakeholder Engagement

Evidence that patients, caregivers, clinicians, and other stakeholders have been and will be engaged in:

• Formulating the research questions
• Defining the characteristics of study participants, comparators and
• utcomes
• Selecting the important outcomes to be assessed
• Monitoring study conduct and progress
• Designing plans for dissemination of study results

Clear statement of the roles and the decision-making authority of all patient and stakeholder research partners An organizational structure that takes into account the Study Advisory Committee which will bring together national patient and stakeholder groups to further the goals of the study

Patient-Centeredness vs. Patient Engagement

Patient-centeredness is about whether the project aims to answer questions or examine outcomes that matter to patients/caregivers. Patient engagement is about having patients/caregivers as partners in research, as

• pposed to merely being recruited as study

participants.

The Engagement Rubric

The rubric is intended to provide guidance to applicants, merit reviewers, awardees, and engagement/program

• fficers (for creating milestones and monitoring projects)

regarding engagement in the conduct of research. It is divided into three segments:

Planning the Study Conducting the Study Disseminating the Study Results

Addressing Engagement

Several approaches to engagement can succeed PCORI provides many engagement resources:

• PCORI’s “The Engagement Rubric”
• http://www.pcori.org/sites/default/files/PCORI-Engagement-Rubric-with-

Table.pdf

• Sample Engagement Plans
• http://www.pcori.org/sites/default/files/announcement-

resources/PCORI-Sample-Engagement-Plans.pdf

• Engagement in Research website page
• http://www.pcori.org/content/engagement-research
• PCORI’s Methodology Standards PC-1 to PC-4
• http://www.pcori.org/assets/PCORI-Methodology-Standards1.pdf

Other Topics

(EHRs, Recruitment, Research Protections, & Dissemination) Anne Trontell, MD, MPH Associate Director Clinical Effectiveness and Decision Science

Use of Electronic Health Records and Other Computerized Data Sources

Pragmatic studies should take advantage of existing electronic clinical and demographic data whenever possible. Discuss any proposed uses of existing electronic data in the approach section:

• Cohort identification and recruitment planning
• Collection of covariate and outcome data

Provide evidence of the validity and completeness of available data (e.g., that all follow-up and outcomes of interest are captured)

• PCORI Methodology Standards on Data Networks and

Registries

Recruitment – Provide a convincing plan

Discuss past experiences with recruitment of the target population Provide preliminary evidence of the potential for successful recruitment Provide numbers for the pool of potential participants, those estimated to be eligible, and the expected participation rate. Discuss barriers to recruitment and how you plan to overcome them. Strategies for successful recruitment – Engaged clinical sites – Clinical advocates – Proactive, experienced research coordinator – Protocol flexibility, within reason – Alignment and integration of recruitment activities with clinical workflow

Institutional Responsibilities for Study Participant Protection

• Provide a Data and Safety Monitoring Plan that operates

under the auspices of your institution

• Assure that key personnel are educated on human

subjects protections

• Assure appropriate informed consent
• Establish procedures to minimize risks to participants
• Establish procedures to protect privacy and maintain

confidentiality

• If you anticipate seeking waiver of individual informed

consent, provide the rationale

• Refer to NIH standards for research involving human

subjects

Use of FDA-Regulated Medical Products

All drugs, biologics, devices, and diagnostics used in your study must be FDA-approved

• Indicate within your application whether investigational

new drug (IND) or other regulatory approval will be needed to conduct your study For example, if your study administers an approved medical product in an off-label manner (e.g., a dose, route, frequency, indication, or patient population that is unapproved by FDA) you may need an IND

• If uncertain, seek guidance from FDA or your legal

counsel

Disseminating and Implementing Research Findings

PCORI does not expect applicants to disseminate and implement findings at this time. Applicants should describe the potential for dissemination and implementation.

Describe the potential for disseminating and implementing the study results in other settings. Describe possible barriers to dissemination and implementation.

What To Think About While Completing Your Application

Iris Giggetts, MSW, CRA Administrator, Contracts Operations

Letter of Intent and Application

You were invited to submit a full application based

• n the information provided in the LOI, changes

after the LOI require PCORI approval.

Show stoppers include:

Changes to the PI/Institution Changes to the Specific Aim/Comparator Changes to the Study Design/Research Question Changes in Engagement

PCORI Online: Application

43

• Contact Information
• Project Information
• Project Personnel
• Milestones
• Budget
• Templates and Uploads
• Review and Submit

PCORI Online – Application Tabs

44

• Contact Information
• Project Information
• Project Personnel
• Milestones
• Budget
• Templates and Uploads
• Review and Submit

NOTE If funded, the public abstract will be posted on the PCORI website.

Application Tabs

Contact Information (LOI) Add basic contact information for the Organization/ Institution, including the PI, AO, and any Designees. Pre Screen Questionnaire (LOI) Respond to initial screening questions. Resubmission (LOI) Indicate whether the organization is resubmitting a previous Application, or has been invited to resubmit an Application from a previous cycle. PI Information (LOI) Enter detailed information about the PI’s experience with research and grants management. Project Information (LOI) Provide relevant project information, including the focus disease or condition, projected costs, target populations, and analytic methods. Project Personnel (LOI) Add the names of Project Personnel who will make up the Awardee Project team. Budget Provide details related to the budget of the project. Project Narratives Provide more detail regarding your proposed research, including abstracts, narratives, and goals. Milestones List concrete, specific events or accomplishments that will be documented as contractual tasks of your project. Templates & Uploads Download any applicable templates, and upload attachments to include with the Application.

Milestone and Deliverable Schedule

Examples of milestones:

• When will you have IRB approval?
• When will enrollment begin?
• Key meetings
• 25% / 50% / 75% / 100% enrolled
• Completion of data analysis

45

When Proposing a Dual PI Application…

• One PI must be designated as the

“Contact PI”

• The second PI is listed as the “Dual PI”

within the PCORI Online System – Only two PIs may be named – Can be from the same institution – Can be from another institution – Can have different focuses (e.g., engagement vs. scientific)

• Follow instructions when resubmitting an

application with changes to the original dual PI team

46

Leadership Plan

• Describe the governance and organizational

structure of the leadership team and the research project.

• Delineate the administrative, technical,

scientific, and engagement responsibilities for each Principal Investigator (PI) and the rationale for submitting a dual-PI application.

• Discuss communication plans and the process

for making decisions on scientific and engagement direction.

• Describe the procedure for resolving conflicts.

47

Page Limit

5

People and Places Template: Biosketch

You may use the NIH biosketch or PCORI’s format Biosketches are required for all key personnel List all partners within the Key Personnel section Patient/Stakeholder Biosketch Page Limit

5

Per person

People and Places Template: Project / Performance Site(s)

Demonstrate that the proposed facilities have the appropriate resources required to conduct the project to plan, within budget, and on time. Provide a description of the facilities that will be used during the project, including capacity, capability, characteristics, proximity, and availability to the project. Page Limit

15

Professional Profile/Biosketch

Inputting Your Budget into the System

50

• Streamlined process – yearly budget data copied to remaining

project years

Budget and Period Limitations

Funds & Budget Direct costs up to $10 million over the life of the project Indirect costs: up to 40% capped for the prime and sub contracts Prime organization can charge up to 40% IDC on first $25,000 for each sub contract Institutional base salary up to $200,000 The limit for Scientific Travel is $10,000 over the duration of the

• project. There is no cap on Programmatic Travel.

Period of Performance Maximum of 5 years Requests to extend are not permitted during any stage Do not anticipate receiving a cost OR no-cost time extension. Propose realistic timelines accounting for the burdens associated with obtaining IRB approval

Budget Justification

Provide a narrative that fully supports and explains the basis for the information found in the Detailed Budget Provide sufficient detail to understand the basis for costs, the reason that the costs are necessary, and an explanation for major cost variances – Provide justification as to why the costs are reasonable for the work to be performed Provide a breakdown of costs proposed for each consortia or contractor Must specify any other sources of funding that are anticipated to support the proposed research project Provide quotes Provide indirect cost rate letter, and fringe benefit policy for the prime

• rganization

Costs of Interventions

PCORI will not cover costs for clinical care alternatives that are being compared in the project. PCORI will consider covering costs for ancillary tasks necessary in the implementation or monitoring of a clinical intervention or strategy as part of the research program.

• Examples include costs for obtaining consent,

collecting data, or monitoring that would not normally be performed in routine care Support for the study by the involved healthcare delivery systems must be documented.

Guidelines for Intervention Cost/Coverage

Costs for study interventions must be covered by delivery system, payer, manufacturer or developer of the intervention. The willingness of one or more of the stakeholder groups to cover treatment costs, even when one of the proposed intervention arms is not currently covered by insurance, will be taken as strong endorsement

• f the study by the health system or payer and of the likelihood that they

will implement or use the study‘s findings if definitive. This material support for the study by host delivery system, payer or developer should therefore be discussed in the application. In exceptional cases, PCORI may consider coverage of the co-payment

• r coinsurance costs of participating patients when that is necessary to

preserve blinding in a study or to assure access to the study for vulnerable populations. Contact PCORI with questions.

Letters of Support

All letters of support should be addressed to the PI and institution to demonstrate the commitment of key personnel and supporting

• rganizations to your proposed project.

Letters of support should clearly reflect the substantive involvement and material contribution to be provided by the signatory parties, and are meant to substantiate the commitment

• f collaboration of all forms.

Letters of support should be organized in the following manner:

• Letters of organizational support
• Letters of collaboration
• Letters confirming access to patient populations, data sets, and additional

resources

What Constitutes a Resubmission?

• Same Topic, PFA
• Same PI
• Received Summary Statement

56

What happens to your application after you submit it?

Administrative Screening

Applicants must follow administrative requirements set in PCORI’s Application Guidelines.

► Exceeding page limits, budget, or time

limitations

► Not using PCORI’s required templates ► Submitting incomplete sections or applications

Missing the Mark

General Guidance

Refer to the Pragmatic Clinical Studies page in our Funding Center: http://www.pcori.org/funding-opportunities/announcement/pragmatic- clinical-studies-evaluate-patient-centered-outcomes-1 Use the applicant checklist. Adhere to the formatting, page limit, and template requirements. Convert all documents to PDF and upload to PCORI Online using the appropriate filing convention. Use Biosketches for all key personnel (scientists and patient/stakeholder team members). Address Letters of Support to the PI’s institution.

Common Application Errors

Not entering information into all required fields in the system Not clicking the ‘Save and Review’ button AO is unable to view the application

Resources

Refer to the Pragmatic Clinical Studies page in our Funding Center for the following resources PFA and Application Guidelines PCORI Online User Manuals Sample Engagement Plans General Applicant FAQs: Pragmatic Clinical Studies Applicant FAQs: PCORI Online: https://pcori.my.salesforce.com/ Research Methodology: http://www.pcori.org/node/4020

Submission and Key Dates

What When Application Deadline (by invitation only) May 17, 2017 by 5:00pm ET Merit Review Dates August 2017 Awards Announced November 2017 Earliest Start Date January 2018

Learn More

www.pcori.org info@pcori.org

Questions and Answers

Submit questions via the chat function in Meeting Bridge Ask a question via phone (press 7) Contact Us:

• Programmatic Questions: E-mail sciencequestions@pcori.org
• Schedule a call at http://bit.ly/programmatic_inquiry
• Call 202-627-1884
• Administrative Questions: E-mail pfa@pcori.org or call 202-627-1885

Thank You