High Hopes for CER Better information about the costs and benefits - - PowerPoint PPT Presentation

EVIDENTIARY STANDARDS FOR DETERMINING

THE CLINICAL UTILITY OF

GENOMICS-BASED DIAGNOSTIC TESTS GENOMICS-BASED DIAGNOSTIC TESTS

Sean Tunis MD, MSc May 23, 2012

High Hopes for CER

 “Better information about the costs and benefits of

different treatment options, combined with new incentive structures reflecting the information….is essential to putting the country on a sounder long- term fiscal path.” essential to putting the country on a sounder long- term fiscal path.”

Peter Orszag, CBO (later OMB) Congressional Testimony, June 2007

2

CER Defining Characteristics (IOM)

 Objective of directly informing clinical or health

policy decision

 Study in “real world” patients and settings  Compares at least 2 alternative, each with  Compares at least 2 alternative, each with

potential to be best practice

 Measures outcomes important to patients  Results at population and subgroup level  Methods and data sources appropriate for the

decision of interest

3

SACGHS Recommendation

 “Information on clinical utility is critical for

managing patients, developing professional guidelines, and making coverage decisions.”

 “HHS should create a public private entity of

“HHS should create a public private entity of stakeholders to….establish evidentiary standards and levels of certainty required for different situations”

4

The Fundamental Trade-off

 There is an inherent tension between level of

certainty about risk-benefit and early access to new technologies (innovation, ROI)

 Not easily determined what is the optimal

Not easily determined what is the optimal balance to maximize long-term public health

 Varies by stakeholder interest and perspectives

 Clarity, consistency and predictability of evidence

expectations are essential

5

Effectiveness Guidance Documents

 Specific recommendations for study design

reflecting information needs of patients, clinicians, payers

 Targeted to public/private sector clinical

researchers researchers

 Describe study designs that provide “reasonable

confidence of improved health outcomes”

 Balance internal validity with generalizability,

feasibility, timeliness and cost

6

Contact Info

 sean.tunis@cmtpnet.org  www.cmtpnet.org  410 547 2687 x120 (W)  410 547 2687 x120 (W)  410 963 8876 (M)

7

Example: PROs in off-label studies of oncology drugs

 Include the following 14 patient-reported

symptoms (“core symptom set”) in all research designs for post-market cancer clinical trials: anorexia, anxiety, constipation, depression, diarrhea, dyspnea, fatigue, insomnia, mucositis- diarrhea, dyspnea, fatigue, insomnia, mucositis-

• ral, nausea, pain, sensory neuropathy, rash, and

vomiting.

8

PCORI Methods Committee View

• JAMA. 2012;307(15):1636-1640

 “Engagement of patients at every step of the

research process is viewed as essential, including in the selection of research questions, study design, conduct, analysis, and implementation of findings.” findings.”

 “As such, the Methodology Committee is engaged

in developing standards to support the validity and generalizability of research, as well as patient-centeredness.”

9

Methodological Guidance for Molecular Diagnostics Research in Oncology

May 23, 2012 Washington, DC

Diagnostics Research in Oncology

Project Goals

 Use a stakeholder-driven process to develop

specific methodological recommendations to address the evidence gaps regarding the clinical validity and clinical utility of actionable MDx tests in oncology (solid tumors). in oncology (solid tumors).

 Improve the relevance, timeliness and quality of

the evidence available to patients, clinicians and payers for clinical and policy decision-making.

11

Welcome & Introductions Sean Tunis 4:00 PM – 4:15 PM Molecular Diagnostics EGD Initiative Pat Deverka 4:15 PM – 4:45 PM Methods Panel: Establishing Clinical Utility of Molecular Diagnostics Pat Deverka, Donna Messner (Moderators), Gary Lyman, Robert McCormack, David Nelson, Margaret Piper, Richard Simon, Mary Lou Smith 4:45 PM – 6:00 PM

Meeting Overview

Break for Dinner 6:00 PM – 6:30 PM Dinner Panel: Payer Perspectives

• n Clinical Utility

Sean Tunis (Moderator), Michael Doherty, Elaine Jeter, Lee Newcomer, Ed Pezella , Jeff Roche, Alan Rosenberg 6:30 PM – 7:30 PM Progressive Validation of Molecular Diagnostics to establish Clinical Utility Sean Tunis & Steve Phurrough 7:30 PM – 8:15 PM Wrap up and Next Steps Sean Tunis & Pat Deverka 8:15 PM – 8:30 PM

Meeting Objectives

 Discuss key issues and methodological

recommendations

 Make recommendations more specific & actionable  Evaluate feasibility of implementation  Discuss study design features relevant to establishing clinical

Discuss study design features relevant to establishing clinical utility

 Obtain payer perspectives on establishing clinical utility of MDx*

 Discuss the existing policy framework &

alternative mechanisms to establish Clinical Utility of MDx*

 Progressive validation of MDx* to establish clinical utility  Evaluate potential study designs  Assess the feasibility of implementation

13

 Provide specific recommendations on the

design of prospective studies intended to inform decisions by patients, clinicians and payers

 Balance between internal validity, relevance

and feasibility to provide decision-makers with

Effectiveness Guidance Document

(EGD) Aims

and feasibility to provide decision-makers with a reasonable level of confidence that the intervention improves net health outcomes

 Target audience: clinical researchers working

in industry or academic settings

 Analogous

and complementary to FDA guidance, focused

• n

design elements

particularly relevant to clinical and health policy decision-making

14

Predisposition/ Risk Assessment

• BRCA1/2
• Lynch Syndrome

testing

Staging

• Various

IHC/FISH

Prognosis

• Oncotype

Dx

Predictive

• EGFR
• K-RAS
• BRAF
• HER-2

Monitoring

• PSA
• BCR-ABL

Spectrum of Molecular Diagnostic Tests

testing

• HER-2

15

Conduct Literature Review **Define Project Scope Review interviews to identify key issues and themes Institutional Review Board (IRB) Review Protocol Development for Qualitative Interviews *Conduct Key Informant Interviews Identify additional experts (Snowball Sampling) Identify Key Methodological Issues Related to Clinical Validity & Clinical Utility

1 2 3

MDx EGD Process Overview

Project Initiation: July 2010 Draft major issues and recommendations Develop Delphi survey for TWG members TWG meetings to develop consensus on issues and draft recommendations Final Effectiveness Guidance Document (EGD) on Molecular Diagnostics for Solid Tumors

4 5 6

Target Completion : December 2012 MDAG & IOM workshops to obtain stakeholder input & refine recommendations

7

POSSIBLE BIOMARKERS TYPES OF CANCER BIOMARKER EVALUATION

Prognostic Predictive Analytic Validity Clinical Validity

Project Scope

Prognostic & Predictive Focus on Clinical Validity & Clinical Utility Focus on Solid Tumors

Early Detection Predisposition Clinical Utility ELSI Solid Tumors Hematologic Tumors 17

Actionable Tests

Scope of MDX EGD

IN OUT Type of Test

Actionable (Companion Diagnostics) High risk Low/Moderate risk New test/existing drug Stand-alone test

18

Stand-alone test Single gene “omics”

Target Condition

Solid Tumors Hematologic Malignancies Circulating Tumor Cells

Recommendations

Clinical Validity Analytic Validity Clinical Utility Economic considerations (e.g. cost- effectiveness) Methodological Implementation barriers

Rules of Engagement for Discussions

Participant’s Role

 Ensure recommendations

are actionable and specific

 Ensure recommendations

CMTP’s Role

 Listen carefully  Elicit viewpoints

Ensure all viewpoints are

 Ensure recommendations

are comprehensive

 Ensure recommendations

are grounded in reality

 Please

speak up- the meeting is being recorded

 Ensure all viewpoints are

articulated and encouraged

19

General Questions for Discussions

Discussion Focus:

 Is the recommendation actionable?  Is the language specific enough?  Is the recommendation

designed to reflect the

 Is the recommendation

designed to reflect the information needs of patients, clinicians and payers?

 Can the recommendation be implemented in a study

that is likely to be timely and feasible?

 What are the barriers/enablers?

 Also, are there additional recommendations you

would make?

20

Recommendation #2

Rationale

 By itself, associational evidence is not sufficient to

establish the clinical validity of an MDx test.

 Odds ratios, hazard ratios, regression coefficients,

and area under the ROC curve are inadequate as and area under the ROC curve are inadequate as standalone tools for demonstration of clinical validity.

 Developers need to ensure that the target goal is

clinically relevant (e.g., enrich for a particular level of response, and discriminate for avoidance

• f particular level of adverse events)

21

Recommendation #2

Appropriate Metrics for CV

 Appropriate metrics to demonstrate the strength of

an association between a molecular diagnostic test and a disease state (clinical validity) include clinical sensitivity, clinical specificity, PPV (positive-predictive value) and NPV (negative-predictive value). value) and NPV (negative-predictive value).

 Developers need a good understanding of MDx

biomarker prevalence in the study population to use PPV and NPV.

 To

avoid ambiguity when reporting results, developers should specifically define the measurement terms and concepts used.

22

Recommendation #5

Rationale

 A disconnect exists between regulatory outcomes

and outcomes needed to establish clinical utility (CU).

 Relevant outcomes for CU include:

 Avoiding an ineffective therapy

Avoiding an ineffective therapy

 Switching more quickly to an effective therapy  Helping to choose among seemingly equal treatment

• ptions

 Patient-reported outcomes (e.g., Quality of Life)  Survival and progression-free survival.

23

Recommendation #5

Generate Relevant Outcomes Recommendation

 CU studies should include the assessment of

proven outcomes that measure both benefits and harms, recognizing that these outcomes may harms, recognizing that these outcomes may

• ccur at different time points and are the result
• f management decisions guided by the test
• results. For example, measures of benefits and

harms could include survival, toxicity, health- related quality of life (HRQoL), etc., and should use validated measures whenever possible.

24

Recommendation #7

Rationale

 When new markers are developed for therapies for which

high quality definitive trials have been conducted to establish patient benefit, archived tissues from the definitive RCTs might be used to evaluate the clinical utility of the new marker if:

 1) sufficient archived tissue is available for an appropriately

powered study and to assure that patients included in the biomarker evaluation are representative of the patients in the source RCT

 2) the analytic validity of the test is well established  3) the analysis plan for the biomarker study is completely

pre-specified;

 4) the results are validated in one or more similarly

designed studies using the same assay techniques.

25

Recommendation #7

Studies w/Tissue from Previous RCTs Recommendation

 If

an appropriately designed, powered and conducted clinical trial with banked biospecimens exists, then a properly conducted prospective- exists, then a properly conducted prospective- retrospective study is adequate evidence of clinical utility.

26

Recommendation #8

Rationale

 When direct evidence of CU is not available, it

may be possible to model the clinical utility of a biomarker through the construction of an indirect evidence chain.

 To do so, there must exist:  To do so, there must exist:

 1) adequate information that the test identifies a clear

surrogate marker and

 2) a separate body of clinical evidence linking the surrogate

to health outcomes (e.g., smoking cessation and lung cancer).

 e.g. EGAPP assessment of the clinical utility of

mismatch repair (MMR) mutation testing in Lynch syndrome.

27

Recommendation #8

Indirect Evidence Chain Recommendation

 The demonstration of the clinical utility of a test

may take advantage of existing evidence to form an indirect evidence chain. an indirect evidence chain.

28

Recommendation #9

Rationale

 When developing a new test to serve as a companion

diagnostic to an existing approved drug, the clinical benefit of the drug is already established.

 A successful companion diagnostic product improves

the clinical benefit of the associated therapy by the clinical benefit of the associated therapy by providing information about which patients benefit most (or least) from the therapy.

 In this case, all that is needed for CU is proof that the

companion diagnostic test can satisfactorily distinguish between responders and non-responders in the applicable clinical setting.

 A single-arm study can serve that purpose.  e.g. EGFR with erlotinib treatment for NSCLC.

29

Recommendation #9

Single-Arm Studies w/Approved Drugs Recommendation

 When MDx test is a companion to an approved

drug with established efficacy, and archived tissues are lacking for conducting a prospective- tissues are lacking for conducting a prospective- retrospective study, a single-arm, well-conducted prospective study demonstrating that a new MDx test can distinguish responders from non- responders would be considered adequate evidence of the CU of the test.

30

What’s Missing?

 Post-reimbursement utilization data collection

(defer discussion)

 What about tumor heterogeneity?

 How to address methodologically?

 How much evidence is enough given variability of

effectiveness of available therapies across disease

 How much evidence is enough given variability of

effectiveness of available therapies across disease states?

 Tried to scope EGD to address this question (e.g., across

solid tumors)

 “Need for an objective function that values the

clinical utility in terms of the cost/benefits to patients, payers, and society in general.”

 Economic cost currently outside of EGD scope.

31

Policy Suggestions

 NIH data warehouse development for all genetic,

genomic, proteomic data from NIH trials/studies…banked biospecimens for hypotheses to be validated in cooperative group trials. trials.

 Only NIH is identified. Is there a role for private sector

to allow access to banked biospecimens?

 “Ongoing expectation of validated, well done

clinical trials with clinically significant endpoints to show improvement in net health outcome base

• n use of test.”

 How to operationalize?

32

EGD Position Statement on Managed Entry Schemes Entry Schemes

EGD Position Statement

Managed Entry Schemes

 We support the development and use of novel

policy approaches to promote clinical utility evidence generation for molecular diagnostic tests and other medical devices and drugs. Managed entry schemes encompass a broad Managed entry schemes encompass a broad range of policy tools that provide the flexibility to payers to cover innovative, emerging molecular diagnostic tests while generating valid evidence

• n the relative benefits and risks of these tests

while these are in use in clinical practice

(cont’d on next slide)…

34

EGD Position Statement

Managed Entry Schemes

 Among the possible tools to be considered on a

case-by-case basis are FDA-CMS parallel review and adaptive licensing (for companion diagnostics and in vitro diagnostic tests undergoing FDA review) and performance-based risk-sharing review) and performance-based risk-sharing arrangements (potentially applicable to both LDTs and in vitro diagnostic tests undergoing FDA review), including the provision of coverage for patients in well-designed clinical trials to gather CU evidence for clinically promising MDx tests.

35

Managed Entry Schemes

Managed Entry Schemes

 Provide early access to technologies while

generating evidence that meets the basic regulatory or reimbursement standards

37

Managed Entry Schemes

 Variation in process  Variation in evidentiary requirements

38

Managed Entry Schemes

 Adaptive licensing  Performance-based risk-sharing arrangements  Outcomes-based schemes  Risk-sharing agreements  Coverage with evidence development  Coverage with evidence development  Access with evidence development  Patient access schemes  Conditional licensing  Pay-for-performance programs  “Parallel review”

39

Evidentiary Threshold to Invoke CED

Medicare CED Experience to Date

 Several

encouraging successes, but some challenges with past implementation

 Proposed approach based on review of Medicare

CED for MedPAC and multiple stakeholder CED for MedPAC and multiple stakeholder interviews and workshops

41

Policy Objectives of CED

 Rapid access to promising technologies; promote

innovation for Medicare population

 Generate

evidence to address important uncertainties uncertainties

 Aligned with Medicare’s programmatic aims to

improve health and use resources wisely

42

Minimum Requirements for CED

 Technology to diagnose or treat a serious disease

• r condition, or for important unmet health need

in the Medicare population

 Intervention can be plausibly anticipated to:

Intervention can be plausibly anticipated to:

 Substantially improve health outcomes with same or

lower level of aggregate health spending

 Produce comparable health outcomes at substantially

reduced aggregate spending

 Quality improving, cost increasing technologies

addressed through binary coverage process

43

Binary vs. Non-Binary Coverage

ge, investment

Binary Coverage (“adequate evidence”)

Time (years) Knowledge

Non-Binary Coverage (CED)

Evidence threshold to initiate CED

 A

moderate level

• f

confidence based

• n

available evidence that the item or service will improve health outcomes.

 Benefits considered more likely than not to exceed

risks risks

 Higher confidence should not apply CED

 Threshold met through two mechanism

 “Deemed” categories (primary mechanism)  CMS determination

Moderate Confidence - Deemed

 Interventions

being evaluated in prospective, CER/PCOR studies funded by NIH, AHRQ or PCORI

 Drugs, devices, diagnostics, surgical procedures, etc.

 Drugs and biologics granted accelerated approval by

the FDA the FDA

 Medical devices approved through 510(k) process,

when additional clinical data has been provided

 In-vitro diagnostics approved or cleared by the FDA

based on clinical validity, but not clinical utility

 Deemed categories could be refined and augmented

through NCD process (as with Clinical Trials NCD)

Moderate Confidence – CMS Determination

 For all interventions, one high quality study shows

improvement in intermediate or surrogate outcomes

 High quality studies demonstrating effectiveness but

for which patient population, setting, etc not representative of the Medicare population representative of the Medicare population

 Drugs

and devices approved by the FDA with significant post-approval study requirements

 For diagnostic tests, clinical validity has been clearly

demonstrated, but evidence of clinical utility limited

 Surgical procedures for which one high quality

• bservational study demonstrates effectiveness

Additional Requirements

 Study

protocol approved that will address specified key uncertainties approved in advance

 Sufficient funds have been identified to cover all

clinical and research costs clinical and research costs

 Study results available within reasonable time

frame (e.g. less the 5 years)

 CED process must be transparent, predictable and

consistent

CED as Primary Emphasis for Medicare National Coverage Process

 Limited impact from the current NCD process

 Over past decade, most product developers avoid NCD

process, work through regional contractors

 Number of NCDs per year decreasing over time

 Most binary coverage policy can continue to be

developed and updated by regional contractors developed and updated by regional contractors

 National coverage process should be primarily

focused on implementation of CED to promote access to and evidence for promising technologies

 Proposed framework would selectively promote new

technologies that improve health and/or reduce costs

Next Steps

 Circulate meeting summary, mid-June  MDAG/TWG Teleconference, July 2012  In-person MDAG/TWG meeting, September 2012  Publish final EGD online, December 2012  Publish final EGD online, December 2012