Clinical trials and the impact Clinical trials and the impact of - - PowerPoint PPT Presentation

Clinical trials and the impact Clinical trials and the impact

• f regulations
• f regulations
• f regulations
• f regulations

Cytel Conference Cytel Conference

October 12, 2012 October 12, 2012 Octobe , Octobe , David L DeMets, PhD Department of Biostatistics & Department of Biostatistics & Medical Informatics U i it f Wi i M di University of Wisconsin-Madison

Topics Topics

• Proliferation of multinational trials
• Regulatory Guidelines & their
• Regulatory Guidelines & their

interpretation Ad t ti f f t

• Adverse event reporting for safety
• One site monitoring
• SOPs & SAPs
• DMCs

DMCs

• Adaptive Designs

N G id li f Di b t D

• New Guidelines for Diabetes Drug

Approval

Regulatory Guidelines Regulatory Guidelines Regulatory Guidelines Regulatory Guidelines

• Over the past 20 years, there are more

large phase III trials done by industry

• As a result, there are more regulatory

, g y guidelines on the design, conduct and analysis of clinical trials (Eg ICH, FDA, y ( g , , EMA)

• Are the recent regulatory guidelines on

Are the recent regulatory guidelines on the design, conduct and analysis of industry sponsored trials making trials industry sponsored trials making trials better?

Opinion Opinion

• Clinical trials are not necessarily better

today than 20 years ago today than 20 years ago

• The guidelines are generally consistent

ith li i l t i l f d t l with clinical trial fundamentals

• The problem is in the interpretation of

those guidelines and practice

• Industries have been built up on the

p interpretation and implementation of regulatory guidelines g y g

• Not getting a better bang for the cost!

Globalization of Investigators Globalization of Investigators

Percent of Total 1572s Filed Average Number of Countries Average Number of Countries Where Clinical Trials Conducted (2010) Phase I Studies 2 Phase II Studies 13

2001 1997 2005 2009

Phase III Studies 34

5

Source: Tufts CSDD

2001 1997 2005 2009

Adult Non Adult Non-

• US Trials

US Trials

• More than doubled in previous 10 years
• One-third of clinical trials by 20 largest

One third of clinical trials by 20 largest US-based companies are conducted solely outside the US solely outside the US

• US Inspector General Report

80% of drugs approved had non US sites – 80% of drugs approved had non-US sites – 80% of subjects from non-US sites T i l f d lit ll – Trials of good quality generally

R f Gli k t l NEJM 2009

• Ref: Glickman et al, NEJM 2009

US Situation US Situation

• Sponsors often view US sites as

p

– Too slow to get studies approved – Too expensive p – Too slow to enroll

• US investigators often view trials as
• US investigators often view trials as

– Competing with pressures to see patients Regulatory overhead more costly than – Regulatory overhead more costly than actually taking care of the patient Not enough academic currency – Not enough academic currency

Estimated Costs of Drug Estimated Costs of Drug Estimated Costs of Drug Estimated Costs of Drug Development ($Millions) Development ($Millions)

Fee R: The cost of clinical trials. Drug Discovery and Development Webcast. March 1, 2007 DiMasi JA et al: The price of innovation: new estimates of drug development costs. J Health Economics 2003; 22:151-185 p g p ; DiMasi JA: New drug development in the United States from 1963 to 1999. Clin Pharmacol Ther 2001; 69:286-96

Multiple Countries / Regulatory Multiple Countries / Regulatory Multiple Countries / Regulatory Multiple Countries / Regulatory Agencies / regulations Agencies / regulations

• Sponsors want their produce approved

in multiple countries – especially large p p y g markets

• Want to abide by each country’s

Want to abide by each country s regulations for approval

• Not all countries have same regulations
• Not all countries have same regulations
• Sponsors feel the need to meet lowest

d i t common denominator

International Conference on International Conference on Harmonization (ICH) Harmonization (ICH)

Att t t t d i l 1990’ t k

• Attempt started in early 1990’s to make

regulatory guidance between US and E i t t l ith Europe more consistent, also with Japan

• Many guidelines developed on design,

conduct, analysis, reporting for submissions

• Guidelines generally support good

g y pp g principles (E.g. ICH-E9 Statistical )

Regulatory Side Business Regulatory Side Business

• Many companies (CROs) have been

developed to help sponsors & developed to help sponsors & investigators interpret the various regulations avoid problems & pass regulations, avoid problems & pass audits Similarly companies developed to

• Similarly, companies developed to

conduct audits for regulatory compliance compliance

• Sometimes the same company
• All of this activity adds substantial cost

Clinical Trials Transformation Clinical Trials Transformation I iti ti (CTTI) I iti ti (CTTI) Initiative (CTTI) Initiative (CTTI)

• Collaborative effort between US FDA,

Collaborative effort between US FDA, EMEA, academia and drug/device industry industry

• Centered at Duke University

Governed by an Executive Committee

• Governed by an Executive Committee,

& operated by a Steering Committee Id ifi d i h

• Identified some practices that were

costly, perhaps not effective, and areas f li i for streamlining

CTTI Projects CTTI Projects Two examples Two examples Two examples Two examples

• SAE Reports

Copies from a trial shipped to all participating sites – Copies from a trial shipped to all participating sites – Not unblinded & so not useful to sites – Not required by US federal regulations A l di i i f i – A costly tradition, not very informative – Use DMC process, either internal or external – FDA has new IND safety reporting guideline y p g g

• CRF Complete Auditing

– Line by line on site auditing by CRA’s Very costly to sponsor and investigators – Very costly to sponsor and investigators – Two natural experiments suggest errors in NIH (academic) type approach are small

GUSTO

• GUSTO
• Breast Cancer Trial / Montreal Site Fraud

Clinical Safety Clinical Safety

• Many possible AEs and SAEs
• Some can be prespecified, e.g.

– LFTs – QT Interval

• Unexpected events challenging

Unexpected events challenging

• Rare events challenging

S d t d l b k

• Some adverse events are deal breakers
• Multiple comparison problem
• Short term vs long term surveillance

Coronary Drug Project Coronary Drug Project Mortality Results Mortality Results Mortality Results Mortality Results

Life table cumulative mortality rates Life-table cumulative mortality rates, Coronary Drug Research Project Group

Coronary Drug Project Coronary Drug Project “ f “ f “Unreliability of small numbers” “Unreliability of small numbers”

Placebo Superior Clofibrate Superior

z values for clofibrate-placebo differences in proportion of deaths by calendar month since beginning of study by calendar month since beginning of study (Month 0 = March 1966, Month 100 = July 1974)

AE Standard Coding Systems AE Standard Coding Systems AE Standard Coding Systems AE Standard Coding Systems

• AE’s often collected from patient

complaints, text recorded & coded

• Several adverse event coding systems

g y

• Often organized by body systems
• Subcategorized into events as reported
• Subcategorized into events as reported

by investigator/patient S f t itt ft i th i

• Safety committees often review these in

tabular form by treatment arms

• (AE listings not helpful after awhile)

AE Standard Coding Systems AE Standard Coding Systems g y g y

• Generally these tables not helpful

At l l t l d t bl fill d

• At one level, too granular and tables filled

with small number of events – difficult to interpret interpret

• At higher level, too many critical and non

critical events pooled together – also difficult to interpret

• DMCs often need to select critical events and

create new safety variable(s) best not to be create new safety variable(s), best not to be done post hoc

• Need to sharpen our safety focus on Adverse

Need to sharpen our safety focus on Adverse Events of Special Interest (AESIs)

CTTI/FDA AE Reporting CTTI/FDA AE Reporting I iti ti I iti ti Initiative Initiative

• As a result, FDA has issued updated

As a result, FDA has issued updated guidelines on AE reporting: IND Rule

• Focus on AE’s that matter
• Focus on AE s that matter
• However,

Gl b l l t i t – Global regulatory agencies may not agree – Tradition is hard to break – Many jobs depend on the current practice – What trial wants to be the “first” ?

• Industry struggling to deal with new rule

Site Monitoring Site Monitoring Site Monitoring Site Monitoring

• Regulations & good practice suggest

that CTs need to establish that trial had

– Real patients – Relevant disease – No allocation bias / randomization – Intervention applied without bias – Unbiased Outcome ascertainment – All relevant outcomes reported – Consent forms signed Consent forms signed

Site Monitoring: Site Monitoring: C t P ti C t P ti Current Practice Current Practice

• Need for site performance validation

p has led to industry practice of

– On site monitoring, multiple times g, p – 100% Case Report Form (CRF) line by line validation – Costly to both sponsor and sites – Perhaps as much as 30-35% of trial cost p

• CTTI On Site Monitoring Survey

– Industry/CROs: > 80% of trials – Industry/CROs: > 80% of trials – Academic networks: < 30% of trials

Is 100% CRF validation cost Is 100% CRF validation cost ff ti ? ff ti ? effective? effective?

• Two natural experiments

Two natural experiments

– GUSTO-I trial of tpa in cardiac patients

• NEJM, 1993

NEJM, 1993

– Breast cancer trial

• NEJM, 1995

,

• Some errors detected but no influence
• r even noticeable affect on analysis
• r even noticeable affect on analysis
• But public often expects perfection but

we can’t afford it we can t afford it

– Need to minimize bias

CTTI & Site Monitoring CTTI & Site Monitoring

• CTTI Recommendations

– Focus on what matters – Develop on-line, not post-hoc, quality p , p , q y management – Assess quality in key parameters q y y p

• Improve training of investigators

– 1000 sites with 1 patient vs 10 sites each 1000 sites with 1 patient vs 10 sites each with 100 patients?

• Share experiences work together to

Share experiences, work together to improve QC without adding burden

Multiple IRB Review Multiple IRB Review p

• Multi-center trials will typically have

lti l IRB i multiple IRB reviews

• Multi-country trials will have multiple

IRBs operating under different “regulatory” rules or guidelines

• Each protocol may get different review

& requests for different changes q g

• Every amendment must go through the

process across all IRBs process across all IRBs

• Need to promote more central IRBs

Standard Operating Standard Operating Procedures (SOPs) Mania Procedures (SOPs) Mania Procedures (SOPs) Mania Procedures (SOPs) Mania

• Good practice to document certain key

Good practice to document certain key aspects for a trial

– Definitions Definitions – Procedures & equipment – Training / certification – Training / certification – Data collection

• Current practice is to document
• Current practice is to document

everything imaginable, whether critical

• r not

it’s now a business

• r not – it s now a business
• Audit teams demand your SOPs

Statistical Analysis Plans (SAPs) Statistical Analysis Plans (SAPs) Mania Mania Mania Mania

• Many statistical methods exist for any

single question or data single question or data

• Multiple comparison, repeated testing,

endless subgroup issues endless subgroup issues

• Good to write down before trial starts an

analysis plan for major question(s)

• BUT not all contingencies can be

anticipated and planned for

• Need to use statistics as a tool to help

Need to use statistics as a tool to help us think, not an excuse to stop thinking

Greenberg Report Greenberg Report

Recommendations for CT Conduct Recommendations for CT Conduct Recommendations for CT Conduct Recommendations for CT Conduct

• NIH Report 1967, CCT 1988

NIH Report 1967, CCT 1988

• Develop a mechanism to terminate early if

Question has been answered – Question has been answered – Trial can’t achieve its goals Unusual circumstances – Unusual circumstances – Hypothesis no longer relevant

S h ld i i l

• Sponsor should not terminate a trial

without outside consultants

• Led NIH to use of external DMCs

DMCs in Industry DMCs in Industry

• Increased use of DMCs since 1990
• FDA 1989 guidelines very brief mention of data

FDA 1989 guidelines very brief mention of data monitoring and DMCs

• International Conference on Harmonization

(ICH)

– ICH/E9

• Section 4.5

Interim Analyses

• Section 4.6

Independent DMCs ICH/E6 – ICH/E6

• Gene therapy patient death (2000)

29

• FDA DMC Guidelines (2001, 2005)

FDA: When External DMCs Are FDA: When External DMCs Are FDA: When External DMCs Are FDA: When External DMCs Are Probably Needed Probably Needed

• Trials with mortality or major morbidity

endpoints p

• Trials for which assessment of serious

toxicity requires comparison of rates toxicity requires comparison of rates

• Trials of novel, potentially high-risk

treatments treatments

• Trials with high risk or vulnerable subjects

DMC Guideline Impact DMC Guideline Impact DMC Guideline Impact DMC Guideline Impact

• DMCs formed for Phase III and some

Phase II trials, more & more

• More DMCs than available experienced

More DMCs than available experienced members

• No sustained training programs for
• No sustained training programs for

DMCs

Despite books and numerous papers – Despite books and numerous papers – No regulatory science training

FDA till ibl f d t

• FDA still responsible for adequate

DMCs implementation

Industry-Modified NIH Model

(PROMISE 1991 NEJM)

Pharmaceutical Industry Sponsor Steering Committee Regulatory Agencies

(PROMISE, 1991, NEJM)

Sponsor Independent Data Monitoring Committee Agencies Central Units Data Monitoring Committee (IDMC) Statistical Data Central Units (Labs, …) Data Coordinating Center (Sponsor or CRO) Statistical Data Analysis Center Clinical Centers Institutional Review Board Patients

Statistical Data Analysis Statistical Data Analysis Center (SDAC) Center (SDAC)

• FDA DMC guidelines suggest a

preference for an SDAC independent of the sponsor (industry or NIH)

• Not enough such centers
• Most CROs not experienced at DMC

support support

– DMC Reports often not focused or presented effectively p y – Often lack flexibility via contracting mechanism

DMC Charters DMC Charters DMC Charters DMC Charters

• Early charters were written to provide

guidance to DMC

– Membership – Responsibilities – Guidelines for early termination y

• FDA guidance supports this approach
• Recent charters becoming more
• Recent charters becoming more

contractual, suggest legal liability

DMCs expected to live by the charter – DMCs expected to live by the charter – Rules rather than guidelines

DMC SAPs DMC SAPs

• Difficult to develop a total SAP for the

final analysis of a trial

• Impossible to develop an SAP for the

p p DMC of a trial

– Too many unexpected scenarios y p – “Expect the unexpected” – Attempts to have detailed DMC SAP have Attempts to have detailed DMC SAP have failed

• Plan for primary outcome a leading

Plan for primary outcome, a leading secondary outcome, let DMC do it’s job

Indemnification of DMCs Indemnification of DMCs

• DMCs or members have been

subpoenaed and become defendants in p litigation.

– Experience indicates investors most likely

• DMCs must be indemnified by the sponsor
• r through some other defined process
• Indemnification language should be part of

the DMC Charter as well as contracts

• Should not be compromised by the

insertion of “negligence” or similar terms as an exclusion for protection.

• Ref: DeMets, Fleming et al (2004, CCT)

Erosion of DMC Independence Erosion of DMC Independence Erosion of DMC Independence Erosion of DMC Independence

• Charters read like a legal document, not a

Charters read like a legal document, not a set of principles to protect patients

• Charters may limit the number of interim
• Charters may limit the number of interim

analyses, or endpoint data to be reviewed (don’t spend any alpha!) (don t spend any alpha!)

• SDACs not always experienced in DMCs

T i i f DMC b l ki

• Training of DMC members lacking
• Threat of litigation against DMCs may

influence their judgment negatively

DMCs and Adaptive Designs DMCs and Adaptive Designs

• Adaptive designs popular but not new

Adaptation of sample si e based on

• Adaptation of sample size based on

interim results still problematic

St ti ti l th d i t t t l T I – Statistical methods exist to control Type I – Logistics still not fully resolved

• DMCs being asked to make sample size

adjustment recommendations

• May not be a good idea

– DMCs know totality of data y – Sample size adjustment may be contrary

Some Common Myths Some Common Myths Some Common Myths Some Common Myths

• DMCs should be blinded

DMCs should be blinded

• DMCs must follow the SAP exactly
• DMC meetings must be scheduled
• DMC meetings must be scheduled

precisely & limited in number

• DMC reports must be on cleaned
• DMC reports must be on cleaned

adjudicated data

• DMC reports can be totally
• DMC reports can be totally

preprogrammed – following SAP

• DMCs review each AE or SAE

39

• DMCs review each AE or SAE

New Guidelines for Diabetes New Guidelines for Diabetes D A l D A l Drug Approval Drug Approval

• Recent guidelines require a ruling out of

Recent guidelines require a ruling out of cardiovascular risk for drug approval

• For initial approval rule out risk of > 1 8
• For initial approval, rule out risk of > 1.8
• Must follow with further data to rule out

i k f 1 3 a risk of > 1.3

• OK if there are two consecutive trials
• Problematic if done within a single trial

– Filing “1.8 results” NDA may compromise Filing 1.8 results NDA may compromise trial continuation to get “1.3 results”

Summary Summary

• RCTs our best methodology if correctly

designed, conducted & analyzed

• Unnecessary burdens growing with narrow

regulatory interpretation

• Burdens increasing costs for sponsors &

investigators dramatically

• US role diminishing
• US role diminishing
• Must streamline and focus more on real

critical issues critical issues

• If we fail, other external forces will play a

major role & not be as well informed

• In the end, our patients lose