Cycle 2 2016: Pragmatic Studies to Evaluate Patient-Centered - - PowerPoint PPT Presentation

Cycle 2 2016: Pragmatic Studies to Evaluate Patient-Centered Outcomes

Applicant Town Hall June 20, 2016

Your letter of intent (LOI) was reviewed and you have been invited to submit a full application……..

CONGRATULATIONS!

Agenda

Welcome Overview Merit Review Criteria Other topics What to think about before you apply Questions?

Submit questions via the chat function in Meeting Bridge. Ask a question via phone (an operator will standby to take your questions).

Overview Overview Anne Trontell, MD. MPH

Associate Director Clinical Effectiveness Research

Purpose of Town Hall

To assist applicants to prepare strong proposals in response to this funding announcement: Applicants proposing “clinical comparative effectiveness research (CER)” Applicants proposing “improving healthcare systems (IHS) CER” Applicants proposing “CER to reduce or eliminate health and health care disparities”

What is a Pragmatic CER Study?

Answers a practical, real world comparative effectiveness research question. Assesses whether two or more options differ in effectiveness when administered as they are in real life Project is conducted in a clinical setting that is as close as possible to a real world setting. The methodological approach (including study design, outcome measures, and follow-up) is as simple as possible without sacrificing scientific rigor.

Justification for the Design Elements of a Large Pragmatic Study

Suggest reviewing pragmatic–explanatory continuum indicator summary (PRECIS) tool Consider tradeoffs

• Eligibility criteria
• Flexibility of intervention
• Range and types of outcomes
• Follow up intensity
• Adherence
• Etc.

Source: A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers. Thorpe, et al. CMAJ 2009; 180:E47-E57.

Research Activities Not Supported in this PFA

Studies of decision aids Efficacy trials Evidence syntheses Cost-effectiveness analysis Research that aims to compare the overall costs of care between two or more alternatives and use the results to determine the preferred alternative

Addressing Disparities

Ayodola Anise, MHS Program Officer

Studies Specifically Addressing Disparities

We seek pragmatic clinical trials to test the comparative effectiveness of evidence-based interventions set in real world settings for patients who are at risk for experiencing disparities in outcomes Study should test the ability of interventions to improve

• utcomes and reduce disparities for at risk populations

Target populations include:

• Racial and ethnic minorities
• Low-income groups
• Residents of rural areas
• Individuals with disabilities
• Patients with low health literacy/numeracy and limited English

proficiency

• Lesbian, gay, bisexual, transsexual

Studies Specifically Addressing Disparities (cont.)

Studies should measure a broad range of patient-centered, clinical, and structural outcomes that contribute to reducing disparities in care We encourage applications from organizations that serve majority underserved populations (e.g., safety net settings) Studies addressing populations at risk for disparities may:

• Require a more frequent, comprehensive and diverse set of
• utcomes data to adequately capture the impact of the intervention

(e.g., follow-up that takes place outside of clinical setting, additional validated measures to assess outcomes).

• Require tailoring of intervention (e.g., language, culture, access).
• Consist of multi-component, multi-level interventions (e.g., targeting

the patient, provider, and systems), as evidenced by disparities literature.

Merit Review Criteria

Potential to Fill Evidence Gaps and Improve Care

Steven Clauser, PhD, MPA Program Director Improving Healthcare Systems

Merit Review Criteria

Potential for the study to fill critical gaps and generate actionable evidence Potential for the study findings to be adopted into clinical practice and improve delivery of care Scientific merit (research design, analysis, and

• utcomes)

Patient-centeredness Patient and stakeholder engagement

Potential to fill critical gaps

A clear clinical burden A critical gap in current knowledge based on systematic reviews, clinical practice guidelines, or previous research prioritizations Variations in practice patterns that suggest clinical uncertainty Decisional dilemmas experienced by patients and

• ther stakeholders

The application should clearly define and describe: Does the study have the potential to fill these gaps and inform decision making for key stakeholders

Addressing Potential to Fill Critical Gaps

Clinical CER

• Describe the decisional dilemma from information gathered

directly from stakeholders and patients, or based on evidence synthesis efforts and prioritization

Improving Healthcare Systems CER

• Discuss the impact of the system problem on healthcare

access and quality – and on individual and population suffering, morbidity, mortality, productivity and costs

CER to Address Disparities

• Describe the critical gaps in evidence related to health care

and health outcomes for sub-populations with known health disparities

Potential for the Study’s Findings to be Adopted into Clinical Practice and Improve Delivery of Care

Who are the end-users and how would the information from this study support a demand for information from end-users? How likely are the findings to be reproduced by others? What are the barriers? What is the dissemination plan for study results, beyond traditional publication and presentation? Can the study be readily adopted in other settings with minimal adaptations or complexities?

Describe how evidence generated from this study could be adopted into clinical practice and care delivery

Addressing Potential for Study Findings to be Adopted into Clinical Practice

Provide indications that the proposed CER has a substantial potential to improve practice and patient

• utcomes
• Should refer to existing efficacy or effectiveness studies
• IHS CER should refer to prior evidence of efficacy or

effectiveness of components of current intervention, or of effectiveness of the interventions in smaller studies or other settings

• CER to Address Disparities should refer to existing efficacy
• r effectiveness studies in the target population where

possible, or highlight efficacy or effectiveness studies in the general population that may be promising for the target population

Addressing Potential (cont.)

Discuss the likelihood that positive study findings related to improvements in practice and patient outcomes would be implemented widely and quickly

• Breadth of current use in existing practice
• Extent of patient, practice or organizational barriers and facilitators

to rapid adoption by patients, clinicians and relevant health

• rganizations
• Applicability to diverse delivery systems
• Endorsement of the current study by key patient, physician and
• ther stakeholder groups.

Describe how the partners in your proposal (e.g., national and/or regional stakeholder organizations) would help disseminate the study’s findings to potential adopters

Merit Review Criteria

Scientific Merit and Methodology Standards Marina Broitman, PhD Senior Merit Review Officer

Scientific Merit

A clear research plan with rigorous methods that adhere to PCORI’s Methodology Standards and accepted best practices A clear justification for the study design and outcome measures Clearly described and justified comparators Sample sizes and power estimates based on careful evaluations of the anticipated effect size Feasibility

A carefully constructed and realistic timeline that includes specific scientific and engagement milestones Realistic strategies for participant recruitment and retention

PCORI Methodology Standards

47 standards in 11 groups. The Methodology Standards do not address all issues related to study designs and methods. Note that PCORI is not using a specific set of methodological standards for “pragmatic studies.”

• Consider design tradeoffs (e.g., blinding vs not blinding)
• Refer to other respected sources for additional guidance.
• View here: http://www.pcori.org/assets/2013/11/PCORI-

Methodology-Report.pdf

Sample Size Estimate and Subgroups

Document previous relevant studies to justify use

• f the proposed effect size.

Assure appropriate adjustments based on the particular study circumstances, including the study population, expected dropout and crossover, and

• ther parameters.

Demonstrate sufficient power to analyze important pre-specified subgroups; these subgroups should be based on patient attributes with strong a priori pathophysiological or empirical justification.

Cluster Randomized Trial Design

Need justification for the use of this design

• State the benefits and disadvantages of a cluster design
• vs. individual randomization, based on the clinical issues

and the interventions being studied.

Sample size estimates need to account for reasonable estimates of intraclass correlation. Maximize the number of clusters that are randomized.

Merit Review Criteria

Patient-Centeredness and Patient & Stakeholder Engagement Jean Slutsky, PA, MSPH Chief Engagement and Dissemination Officer Program Director for Communication and Dissemination Research

Patient-Centeredness

Is the research focused on questions that affect

• utcomes of interest to patient and their

caregivers?

• Does the research question address choices that are

important to - and faced frequently by - patients, their caregivers, or clinicians?

• Is the study powered on outcomes that are important to

patients?

Are the interventions being compared in the study available to patients now?

Addressing Patient-Centeredness

Provide evidence that the research question(s) and

• utcomes are important to patients (and/or their

caregivers) Describe your strategy for measuring outcomes that are important to patients. Remember that a study can be patient-centered even if the end-user is not the patient, as long as patients will benefit from its information.

If the end-user is not the patient, be sure to carefully describe how your study is still patient-centered.

Selecting Important Patient-Centered Outcomes

Do proposed outcomes include outcomes based

• n input from patients and other stakeholders?

What is the validity of this outcome measure? What is a minimal important difference in this

• utcome measure?

Have you considered the use of a previously validated patient reported outcome (PRO)?

Patient and Stakeholder Engagement

Evidence that patients, caregivers, clinicians, and other stakeholders have been and will be engaged in:

• Formulating the research questions
• Defining the characteristics of study participants, comparators and
• utcomes
• Selecting the important outcomes to be assessed
• Monitoring study conduct and progress
• Designing plans for dissemination of study results

Clear statement of the roles and the decision-making authority of all patient and stakeholder research partners An organizational structure that takes into account the Study Advisory Committee which will bring together national patient and stakeholder groups to further the goals of the study

Patient-Centeredness vs. Patient Engagement

Patient-centeredness is about whether the project aims to answer questions or examine outcomes that matter to patients/caregivers. Patient engagement is about having patients/caregivers as partners in research, as

• pposed to merely being recruited as study

participants.

The Engagement Rubric

The rubric is intended to provide guidance to applicants, merit reviewers, awardees, and engagement/program

• fficers (for creating milestones and monitoring projects)

regarding engagement in the conduct of research. It is divided into three segments:

Planning the Study Conducting the Study Disseminating the Study Results

Addressing Engagement

Several approaches to engagement can succeed PCORI provides many engagement resources:

• PCORI’s “The Engagement Rubric”
• http://www.pcori.org/sites/default/files/PCORI-Engagement-Rubric-with-

Table.pdf

• Sample Engagement Plans
• http://www.pcori.org/sites/default/files/announcement-

resources/PCORI-Sample-Engagement-Plans.pdf

• Engagement in Research website page
• http://www.pcori.org/content/engagement-research
• PCORI’s Methodology Standards PC-1 to PC-4
• http://www.pcori.org/assets/PCORI-Methodology-Standards1.pdf

Other Topics

(EHRs, Recruitment, Research Protections, & Dissemination) Anne Trontell, MD, MPH Associate Director Clinical Effectiveness Research

Use of Electronic Health Records and Other Computerized Data Sources

Pragmatic studies should take advantage of existing electronic clinical and demographic data whenever possible. Discuss any proposed uses of existing electronic data in the approach section:

• Cohort identification and recruitment
• Collection of covariate and outcome data

Provide evidence of the validity and completeness of available data (e.g., that all follow-up and outcomes of interest are captured)

• PCORI Standards on Data Networks and Registries

Recruitment – Provide a convincing plan

Discuss past experiences with recruitment of the target population Provide preliminary evidence of the potential for successful recruitment Provide numbers for the pool of potential participants, those estimated to be eligible, and the expected participation rate. Discuss barriers to recruitment and how you plan to overcome them. Strategies for successful recruitment – Engaged clinical sites – Clinical advocates – Proactive, experienced research coordinator – Protocol flexibility, within reason – Alignment and integration of recruitment activities with clinical workflow

Study Participant Protection

• Provide a Data and Safety Monitoring Plan that operates

under the auspices of your institution

• Assure that key personnel are educated on human

subjects protections

• Assure appropriate informed consent
• Establish procedures to minimize risks to participants
• Establish procedures to protect privacy and maintain

confidentiality

• If you anticipate seeking waiver of individual informed

consent, provide the rationale

• Refer to NIH standards for research involving human

subjects

Use of FDA-Regulated Medical Products

All drugs, biologics, devices, and diagnostics used in your study must be FDA-approved

• Indicate within your application whether investigational

new drug (IND) or other regulatory approval will be needed to conduct your study For example, if your study administers an approved medical product in an off-label manner (e.g., a dose, route, frequency, indication, or patient population that is unapproved by FDA) you may need an IND

• If uncertain, seek guidance from FDA or your legal

counsel

Disseminating and Implementing Research Findings

PCORI does not expect applicants to disseminate and implement findings at this time. Applicants should describe the potential for dissemination and implementation.

Describe the potential for disseminating and implementing the study results in other settings. Describe possible barriers to dissemination and implementation.

What To Think About While Completing Your Application

Iris Giggetts, MSW, CRA Senior Associate, Contracts Operations

Letter of Intent and Application

You were invited to submit a full application based

• n the information provided in the LOI, changes

after the LOI require PCORI approval.

Show stoppers include:

Changes to the PI/Institution Changes to the Specific Aim/Comparator Changes to the Study Design/Research Question Changes in Engagement

PCORI Online: Application

► PI and Contact Information ► Project Information ► Key Personnel ► Milestones (upload) ► Templates and Uploads ► Save and Review

Public Abstract

Project Information

Technical Abstract Project Narratives

Key Personnel

Key Personnel

Milestones

► PI and Contact Information ► Project Information ► Key Personnel ► Milestones ► Templates and Uploads ► Save and Review

• Please note that milestones were previously entered

into the PCORI Online system.

• It is now a template that will be completed and

uploaded into the PCORI Online system.

Templates and Uploads

► PI and Contact Information ► Project Information ► Key Personnel ► Milestones ► Templates and Uploads ► Save and Review

People and Places Template: Biosketch

You may use the NIH biosketch or PCORI’s format Biosketches are required for all key personnel List all partners within the Key Personnel section Patient/Stakeholder Biosketch Page Limit

5

Per person

People and Places Template: Project / Performance Site(s)

Demonstrate that the proposed facilities have the appropriate resources required to conduct the project to plan, within budget, and on time. Provide a description of the facilities that will be used during the project, including capacity, capability, characteristics, proximity, and availability to the project. Page Limit

15

Professional Profile/Biosketch

Budget Templates Three budget sections must be submitted as part of the online application process:

Detailed Budget Budget Summary Budget Justification NOTE:

A detailed budget is needed for each year of the

• project. Complete each budget section for the

prime applicant and any/each subcontractor.

Budget and Period Limitations

Funds & Budget Direct costs up to $10 million over the life of the project Indirect costs: up to 40% capped for the prime and sub contracts Prime organization can only charge up to 40% IDC on first $25,000 for all sub contracts combined Institutional base salary up to $200,000 The limit for Scientific Travel is $10,000 over the duration of the

• project. There is no cap on Programmatic Travel.

Period of Performance Maximum of 5 years Requests to extend are not permitted during any stage Do not anticipate receiving a cost OR no-cost time extension. Propose realistic timelines accounting for the burdens associated with obtaining IRB approval

Budget Justification

Provide a narrative that fully supports and explains the basis for the information found in the Detailed Budget Provide sufficient detail to understand the basis for costs, the reason that the costs are necessary, and an explanation for major cost variances – Provide justification as to why the costs are reasonable for the work to be performed Provide a breakdown of costs proposed for each consortia or contractor Must specify any other sources of funding that are anticipated to support the proposed research project Provide quotes, indirect cost rate letter, and fringe benefit policy

Costs of Interventions

PCORI will not cover costs for clinical care alternatives that are being compared in the project. PCORI will consider covering costs for ancillary tasks necessary in the implementation or monitoring of a clinical intervention or strategy as part of the research program.

• Examples include costs for obtaining consent,

collecting data, or monitoring that would not normally be performed in routine care Support for the study by the involved healthcare delivery systems must be documented.

Guidelines for Intervention Cost/Coverage

Costs for study interventions must be covered by delivery system, payer, manufacturer or developer of the intervention. The willingness of one or more of the stakeholder groups to cover treatment costs, even when one of the proposed intervention arms is not currently covered by insurance, will be taken as strong endorsement

• f the study by the health system or payer and of the likelihood that they

will implement or use the study‘s findings if definitive. This material support for the study by host delivery system, payer or developer should therefore be discussed in the application. In exceptional cases, PCORI may consider coverage of the co-payment

• r coinsurance costs of participating patients when that is necessary to

preserve blinding in a study or to assure access to the study for vulnerable populations. Contact PCORI with questions.

Letters of Support

All letters of support should be addressed to the PI and institution to demonstrate the commitment of key personnel and supporting

• rganizations to your proposed project.

Letters of support should clearly reflect the substantive involvement and material contribution to be provided by the signatory parties, and are meant to substantiate the commitment

• f collaboration of all forms.

Letters of support should be organized in the following manner:

• Letters of organizational support
• Letters of collaboration
• Letters confirming access to patient populations, data sets, and additional

resources

What happens to your application after you submit it?

Administrative Screening

Applicants must follow administrative requirements set in PCORI’s Application Guidelines.

► Exceeding page limits, budget, or time

limitations

► Not using PCORI’s required templates ► Submitting incomplete sections or applications

Missing the Mark

General Guidance

Refer to the Pragmatic Studies page in our Funding Center: http://www.pcori.org/funding-opportunities/announcement/pragmatic- clinical-studies-Cycle-2-2016 Use the applicant checklist. Adhere to the formatting, page limit, and template requirements. Convert all documents to PDF and upload to PCORI Online using the appropriate filing convention. Use Biosketches for all key personnel (scientists and patient/stakeholder team members). Address Letters of Support to the PI’s institution.

Common Application Errors

Using the wrong browser, access PCORI Online via Chrome or Safari browsers Not entering information into all required fields in the system Not clicking the ‘Save and Review’ button Having multiple people working on the application at the same time Having the incorrect file extension, only PDF files can be uploaded Not choosing the correct document type from the drop-down menu AO is unable to view the application

Resources

Refer to the Pragmatic Studies page in our Funding Center for the following resources PFA and Application Guidelines PCORI Online User Manuals Sample Engagement Plans General Applicant FAQs: Pragmatic Clinical Studies Applicant FAQs: PCORI Online: https://pcori.fluxx.io/ Research Methodology: http://www.pcori.org/node/4020

Submission and Key Dates

What When Application Deadline (by invitation only) August 8, 2016 by 5:00pm ET Merit Review Dates November 14-15, 2016 Awards Announced January 2017 Earliest Start Date March 2017

Learn More

www.pcori.org info@pcori.org

Questions and Answers

Submit questions via the chat function in Meeting Bridge Ask a question via phone (press 7) Contact Us:

• Programmatic Questions: E-mail sciencequestions@pcori.org
• Schedule a call at http://bit.ly/programmatic_inquiry
• Call 202-627-1884
• Administrative Questions: E-mail pfa@pcori.org or call 202-627-1885

Thank You