A P A P A Perspective o A Perspective o ti ti on the Current - - PowerPoint PPT Presentation

A P ti A P ti A Perspective o A Perspective o Environment for A Environment for A Tri Tri

Robert T. O Robert T. O Senior Statist Senior Statist Office of Translational Office of Translational

For presentation at the 2nd Annual East U MA ; October 12, 2012

th C t th C t

• n the Current
• n the Current

Adaptive Clinical Adaptive Clinical p ials ials

’Neill, PhD ’Neill, PhD tical Advisor tical Advisor l Sciences, CDER, FDA l Sciences, CDER, FDA

ser Group Meeting, Cambridge,

Outline o Outline o Outline o Outline o

 How did the adaptive d How did the adaptive d h did it d l ith h did it d l ith how did it develop with how did it develop with environment environment  P i d idi P i d idi  Promoting and guiding Promoting and guiding  Some concerns and curr Some concerns and curr  What have we learned What have we learned  Where are we going Where are we going  Where are we going Where are we going

• f the talk
• f the talk
• f the talk
• f the talk

design initiative begin and design initiative begin and hi th l t hi th l t hin the regulatory hin the regulatory i i i i g innovation g innovation rent climate rent climate

http://www.fda.gov/oc/initiatives/criticalpath/

Main m Main m Main m Main m

 Failure rate of phase 3 confir Failure rate of phase 3 confir  Few industry case stud Few industry case stud  Desire to streamline clinical Desire to streamline clinical  Desire to take advantage of n Desire to take advantage of n are not impacting success rat are not impacting success rat  Concern for a non sustainab Concern for a non sustainab  Concern for a non sustainab Concern for a non sustainab  This is a problem for multip This is a problem for multip academics, regulators, patien academics, regulators, patien  regulatory science is n regulatory science is n recognized as a discipl recognized as a discipl

messages messages messages messages

rmatory trials rmatory trials dies diagnosing the causes dies diagnosing the causes trials trials new science and discoveries that new science and discoveries that tes for new treatments tes for new treatments ble model for drug development ble model for drug development ble model for drug development ble model for drug development ple stakeholders (industry, ple stakeholders (industry, nts) nts) not well understood or not well understood or line line

The PhRMA The PhRMA discussion discussion discussion discussion A WG with A WG with (JBS 2006) (JBS 2006) (JBS 2006) (JBS 2006)

Impact of PDU Impact of PDU FDA negotiated and FDA negotiated and

• n Adapt
• n Adapt

FA IV (Sept. 2007) FA IV (Sept. 2007) d promised a guidance d promised a guidance tive Designs tive Designs

Two Regulato Two Regulato Issued (20 Issued (20 Issued (20 Issued (20

• ry Guidances
• ry Guidances

006 2010) 006 2010) 006, 2010) 006, 2010)

Philosophy of F Philosophy of F Philosophy of F Philosophy of F

 Understandable to a wi Understandable to a wi clinicians and statisticia clinicians and statisticia clinicians and statisticia clinicians and statisticia  Cautionary but encoura Cautionary but encoura studies and for confirm studies and for confirm d warranted warranted  Terminology and defini Terminology and defini Seamless Phase 2 /Phase Seamless Phase 2 /Phase Seamless Phase 2 /Phase Seamless Phase 2 /Phase exploration with confirm exploration with confirm  We are still learning, ex We are still learning, ex l d l d g evolving , as is methodo evolving , as is methodo

FDA Guidance FDA Guidance FDA Guidance FDA Guidance

de audience, including de audience, including ans and others ans and others ans, and others ans, and others age use for exploratory age use for exploratory matory studies, if and when matory studies, if and when itions are important: itions are important: e 3 terminology confuses e 3 terminology confuses e 3 terminology confuses e 3 terminology confuses mation mation xamples are increasing and xamples are increasing and l p g p g

• logy
• logy

Definition and Definition and d i d i adaptive adaptive

(FDA Gu (FDA Gu

For the purposes of this gu For the purposes of this gu clinical study is defined as clinical study is defined as ti l l d d ti l l d d prospectively planned and s prospectively planned and s potential for modification o potential for modification o aspects of the study design aspects of the study design l i f d t f bj l i f d t f bj analysis of data from subjec analysis of data from subjec

• f the accumulating study d
• f the accumulating study d

prospectively planned poin prospectively planned poin b f d i b f d i f ll bli f ll bli be performed in a be performed in a fully blin fully blin blinded manner blinded manner, and may, at , and may, at

• ccur with or without form
• ccur with or without form

t ti ( d i i t l ) t ti ( d i i t l ) testing (eg. decision tools). testing (eg. decision tools).

d concept of an d concept of an d i d i e design e design

uidance) uidance)

idance, an adaptive design idance, an adaptive design a study that includes a a study that includes a ifi d difi ti ifi d difi ti specified modification specified modification or

• r
• f one or more specified
• f one or more specified

and hypotheses, and hypotheses, based on based on t i th t d t i th t d A l A l cts in the study cts in the study. Analyses . Analyses data are often performed at data are often performed at nts within the study, may nts within the study, may d d i d d i nded manner or in a non nded manner or in a non-

• t an interim time point,

t an interim time point, mal statistical hypothesis mal statistical hypothesis

Some Clar Some Clar

(FDA G (FDA G (FDA Gu (FDA Gu

h “ i ” “ i ” h The The term “prospective” term “prospective” here here was planned (and details spe was planned (and details spe examined in an unblinded fo examined in an unblinded fo involved in planning for the involved in planning for the involved in planning for the involved in planning for the plans that are introduced or plans that are introduced or has commenced if confidenc has commenced if confidenc the personnel involved is un the personnel involved is un the personnel involved is un the personnel involved is un and documented when the m and documented when the m

• proposed. The documentatio
• proposed. The documentatio

unequivocal assurance of bli unequivocal assurance of bli l hil d i l hil d i personnel while a study is on personnel while a study is on to discuss with FDA. to discuss with FDA. Chang Chang

• ccurring after an interim an
• ccurring after an interim an

data and that were not prosp data and that were not prosp data and that were not prosp data and that were not prosp within the scope of this guid within the scope of this guid

rifications rifications

id ) id ) uidance) uidance)

h h d i h h d i e means that the adaptation e means that the adaptation ecified) before data were ecified) before data were

• rm by any personnel
• rm by any personnel

e revision This can include e revision This can include e revision. This can include e revision. This can include finalized after the study finalized after the study ce in the blinded state of ce in the blinded state of nequivocally maintained nequivocally maintained nequivocally maintained nequivocally maintained modification plan is modification plan is

• n that can support
• n that can support

inding for the necessary inding for the necessary i b i i b i ngoing may be important ngoing may be important ges in study design ges in study design nalysis of unblinded nalysis of unblinded study study pectively planned are not pectively planned are not pectively planned are not pectively planned are not dance. dance.

Some Clar Some Clar

(FDA G (FDA G (FDA Gu (FDA Gu

R i i d d R i i d d Revisions made or proposed Revisions made or proposed analysis analysis raise major concern raise major concern potential introduction of bia potential introduction of bia d fi d d f ll i l d fi d d f ll i l defined and carefully imple defined and carefully imple irresolvable uncertainty in t irresolvable uncertainty in t In contrast, In contrast, revisions based revisions based f d t ( t f d t ( t

• f data (e.g., aggregate even
• f data (e.g., aggregate even

rates, baseline characteristic rates, baseline characteristic bias to the study or into sub bias to the study or into sub th l C t i th l C t i the same personnel. Certain the same personnel. Certain such as sample size revision such as sample size revision

• r variance, are advisable pr
• r variance, are advisable pr

b i l l b i l l can be prospectively planne can be prospectively planne

rifications rifications

id ) id ) uidance) uidance)

d ft bli d d i t i d ft bli d d i t i d after an unblinded interim d after an unblinded interim ns about study integrity (i.e., ns about study integrity (i.e., as) and need to be prospectively as) and need to be prospectively t d t id i ki t d t id i ki emented to avoid risking emented to avoid risking the interpretation of study results. the interpretation of study results.

• n blinded interim evaluations
• n blinded interim evaluations

t t i di ti ti t t i di ti ti t rates, variance, discontinuation t rates, variance, discontinuation cs, etc.) usually do not introduce cs, etc.) usually do not introduce bsequent study revisions made by bsequent study revisions made by bli d d bli d d d t b d h d t b d h n blinded n blinded-data based changes, data based changes, ns based on aggregate event rates ns based on aggregate event rates rocedures for most studies, and rocedures for most studies, and d d ed. ed.

Confirmatory Confirmatory Explorator Explorator Explorator Explorator

 Confirmatory studie Confirmatory studie d t d ll d t d ll adequate and well adequate and well-c phase clinical studie phase clinical studie hypotheses hypotheses hypotheses hypotheses  Exploratory studies Exploratory studies primary basis for eff primary basis for eff primary basis for eff primary basis for eff well designed to add well designed to add explore dose, popula explore dose, popula conditions, PD endp conditions, PD endp

y studies and y studies and ry studies ry studies ry studies ry studies

es : otherwise known as es : otherwise known as t ll d ll l t t ll d ll l t controlled, usually later controlled, usually later es that have pre es that have pre-

• stated

stated : not intended as : not intended as ficacy evaluations yet ficacy evaluations yet ficacy evaluations yet ficacy evaluations yet dress dose selection or dress dose selection or ations, optimal study ations, optimal study points, markers, etc. points, markers, etc.

I t f th d f I t f th d f Impact of the draf Impact of the draf

 Alot of comments and specific Alot of comments and specific  Generally well received Generally well received  S i l i ‘l S i l i ‘l  Some terminology issues: ‘less Some terminology issues: ‘less with their actual use with their actual use  Set out FDA’s expectations and Set out FDA’s expectations and  Communication and exp Communication and exp FDA, investigators, NIH FDA, investigators, NIH  CDER is receiving protocols for CDER is receiving protocols for medical areas medical areas medical areas medical areas  Internal education of both clini Internal education of both clini  Discussions of the operational Discussions of the operational

ft id t d t ft id t d t ft guidance to date ft guidance to date

suggestions for edits suggestions for edits ll d d’ l i ll d d’ l i well understood’ vs. less experience well understood’ vs. less experience d a framework to move forward d a framework to move forward perience is needed, among industry, perience is needed, among industry, and others and others r adaptive designs across many r adaptive designs across many icians and statisticians icians and statisticians model model

The intersection o The intersection o b i h b i h bayesian approaches bayesian approaches

 We are not there yet, bu We are not there yet, bu in progress in progress in progress in progress  The NIH/FDA AD The NIH/FDA AD  F th di i F th di i  Further discussion Further discussion demonstrate family demonstrate family control for multipl control for multipl designs designs designs designs  Promotion to all when o Promotion to all when o issues issues – – a potential to ra a potential to ra d p contribute to disappoin contribute to disappoin

• f frequentist and
• f frequentist and

t d ti d i t d ti d i s to adaptive designs s to adaptive designs

ut interesting initiatives are ut interesting initiatives are APT APT-

• IT project

IT project i l ti t i l ti t n on simulations to n on simulations to y wise strong type 1 y wise strong type 1 le hypotheses in K stage le hypotheses in K stage

• nly a few understand the
• nly a few understand the

aise expectations and aise expectations and l d l d p ntments ntments – – it can be solved it can be solved

The Bayesian / Frequ The Bayesian / Frequ Enrichment Enrichment Enrichment Enrichment uentist Intersection uentist Intersection t strategies t strategies t strategies t strategies

ADAPT‐IT ‐ Obje ADAPT IT Obje

• “To illustrate and explore

To illustrate and explore clinical trial designs to imp drugs and medical devices drugs and medical devices methods to characterize a

• pinions and concerns of
• pinions, and concerns of

and after the developmen

ective ective

how best to use adaptive how best to use adaptive prove the evaluation of s and to use mixed s and to use mixed and understand the beliefs, f key stakeholders during f key stakeholders during nt process.”

21

ADAPT‐IT ‐ Obje ADAPT IT Obje

• “To illustrate and explore

To illustrate and explore clinical trial designs to imp drugs and medical devices drugs and medical devices methods to characterize a

• pinions and concerns of
• pinions, and concerns of

and after the developmen

ective ective

how best to use adaptive how best to use adaptive prove the evaluation of s and to use mixed s and to use mixed and understand the beliefs, f key stakeholders during f key stakeholders during nt process.”

22

UNIQUE PUBLIC/PRIVA COLLABORATION

NETT—Neurological Emerg network ‐NINDS funded clinical t

B C lt t Berry Consultants ‐Statistical Consulting com innovative/adaptive designs innovative/adaptive designs

ATE

gencies Treatment Trials trials network (NS056975)

mpany specializing in (Bayesian) (Bayesian)

23

Specific Task Specific Task

• Design four, five clinical

g , trials

– Status Epilepticus (Refractory) (Refractory) – Glycemic control in stroke – Spinal cord trauma p – Post cardiac arrest hypothermia Progesterone for ischemic – Progesterone for ischemic stroke

ks ks

• Learn about process

p

– Surveys – Focus Groups – Observation – Key Stakeholder Interviews – Thematic analysis Thematic analysis

Ed t

• Educate

– Clinicians – Statisticians

24

Statisticians

ADAPT‐IT Proc ADAPT IT Proc

• Investigators and statis

Di li i l bl

FTF ‐ 1

• Discuss clinical problem
• Berry Consultants pres

CTC

y p

• Clinical & data teams p

Si l ti t d

Perf WG

• Simulations presented
• Several iterations

FTF – 2

• Near final design prese
• Work out final details f

cess cess

sticians meet d i l d i m and potential designs ent concept p provides feedback ith f db k with feedback entation for grant / IND submission

25

Wh t i FDA’ i t Wh t i FDA’ i t What is FDA’s inter What is FDA’s inter

 P d i l i i P d i l i i  Promote and stimulate innovation Promote and stimulate innovation  Contribute feedback to a novel project Contribute feedback to a novel project  Involve multiple centers (CDRH, CBER Involve multiple centers (CDRH, CBER multiple disciplines (clinical and statis multiple disciplines (clinical and statis multiple disciplines (clinical and statis multiple disciplines (clinical and statis  Expand the experience base (actual exam Expand the experience base (actual exam  Operationally, disciplines, cons Operationally, disciplines, cons  Most of FDA’s experience is wit Most of FDA’s experience is wit network, funding streams, study network, funding streams, study  These clinical studies are dealing with These clinical studies are dealing with solutions to date solutions to date – – ie. Study failures

• ie. Study failures -
• it

it y contribution to refining the issues, stud contribution to refining the issues, stud  The project has an evaluation of the pro The project has an evaluation of the pro

t i ADAPT t i ADAPT IT IT rest in ADAPT rest in ADAPT-IT IT

R, CDER ) in the collaboration and R, CDER ) in the collaboration and tical) in the experience tical) in the experience tical) in the experience tical) in the experience mples) with adaptive designs mples) with adaptive designs stituencies (NIH, academics, industry) stituencies (NIH, academics, industry) th the industry and not the NIH th the industry and not the NIH y sections y sections complex issues that have not had good complex issues that have not had good t is an opportunity to determine if new t is an opportunity to determine if new pp y pp y dy designs, questions, etc. dy designs, questions, etc.

• cess and an assessment of participants
• cess and an assessment of participants

A lot of moving A lot of moving d i d i E designs designs – Ex Ex Confirm Confirm

 Response or outcome adaptive r Response or outcome adaptive r prognostic covariates vs. both a prognostic covariates vs. both a  Longitudinal modeling of inter Longitudinal modeling of inter

• utcomes to aid in random alloc
• utcomes to aid in random alloc

 Assumed models and eva Assumed models and eva

• f early date to later outc
• f early date to later outc

 Type 1 error control for multipl Type 1 error control for multipl

• f simulation vs closed form tes
• f simulation vs closed form tes

 Dose response estimation vs. pi Dose response estimation vs. pi

• ne study or two studies in seq
• ne study or two studies in seq

 Currently, no sample size re Currently, no sample size re-

• est

est

g parts to these g parts to these l t l t xploratory or xploratory or matory matory

randomization vs. balancing on randomization vs. balancing on and the proper analysis of such and the proper analysis of such rmediate outcome and final clinical rmediate outcome and final clinical cation or in recruitment strategies cation or in recruitment strategies aluation of strength of association aluation of strength of association g come come le hypotheses le hypotheses – – not that easy not that easy – – role role sting sting icking the right dose or doses icking the right dose or doses – – in in quence quence timation timation

Enrichment Enrichment strategies strategies – – the the with adaptiv with adaptiv with adaptiv with adaptiv

 A guidance on enric A guidance on enric soon be out for comm soon be out for comm soon be out for comm soon be out for comm commitment from P commitment from P

designs and designs and eir relationship eir relationship ve approaches ve approaches ve approaches ve approaches

hment designs will hment designs will ment ment – another another ment ment another another DUFA IV DUFA IV

Genomics, biom Genomics, biom subsets and tar subsets and tar subsets and tar subsets and tar markers, patient markers, patient rgeted therapy rgeted therapy rgeted therapy rgeted therapy

Nested sub-populatio p p

• ns

What are some What are some concerns a concerns a concerns a concerns a

 Trust in the firewall Trust in the firewall implementing the ad implementing the ad unblinding unblinding  The new role for dat The new role for dat committees, if they t committees, if they t y responsibility responsibility

e of the current e of the current and issues and issues and issues and issues

l process for l process for daptations that require daptations that require ta monitoring ta monitoring take on this take on this

Early reporting Early reporting Early reporting Early reporting unblinding, preserv unblinding, preserv critical for adaptive critical for adaptive critical for adaptive critical for adaptive g of trial results g of trial results g of trial results, g of trial results, ving trial integrity ving trial integrity – – e designs to succeed e designs to succeed e designs to succeed e designs to succeed

May 7 May 7 May 7 May 7 7 2012 7 2012 7, 2012 7, 2012

Is the Data Monit Is the Data Monit d l i d l i model appropriate model appropriate adaptive adaptive

 Independence Independence  Firewalls Firewalls  Al ith Al ith  Algorithms vs more Algorithms vs more choices choices  Concern for back cal Concern for back cal size re size re-

• estimation an

estimation an  Who represents the s Who represents the s

toring Committee toring Committee f i l i f i l i e for implementing e for implementing designs ? designs ?

fl ibilit i d i i fl ibilit i d i i flexibility in decision flexibility in decision lculation in sample lculation in sample nd inducing bias nd inducing bias sponsor, if at all sponsor, if at all

A B A B P d t f t P d t f t A By A By-

• Product of t

Product of t

 Better up front planning and Better up front planning and clinical trials, fixed or adapti clinical trials, fixed or adapti clinical trials, fixed or adapti clinical trials, fixed or adapti  Consideration of routine use Consideration of routine use advance how trial assumptio advance how trial assumptio unexpected features will imp unexpected features will imp unexpected features will imp unexpected features will imp  My concern ! My concern !  Not enough time to do Not enough time to do  Not enough time to do Not enough time to do fast fast – – this is a return to this is a return to

th AD i iti ti th AD i iti ti the AD initiative the AD initiative

d design choices for all late phase d design choices for all late phase ive ive – the choice is not obvious the choice is not obvious ive ive the choice is not obvious the choice is not obvious e of simulations to play out in e of simulations to play out in

• ns as well as unknown or
• ns as well as unknown or

pact results pact results NOT THAT EASY NOT THAT EASY pact results pact results – NOT THAT EASY NOT THAT EASY

• the planning
• the planning – desire to move too

desire to move too

• the planning
• the planning – desire to move too

desire to move too

• failure in my mind
• failure in my mind

Adaptive Adaptive Have stimulated a Have stimulated a modeling and simu modeling and simu among clinical t among clinical t g

 Designs for confirm Designs for confirm  Designs for explorat Designs for explorat  Designs for explorat Designs for explorat  Designs to accommo Designs to accommo  Software availability Software availability

e designs e designs renewed interest in renewed interest in ulation, particularly ulation, particularly trial statisticians trial statisticians

mation mation tion tion tion tion

• date both
• date both

y ? y ?

Models ca Models ca Models ca Models ca

 Mechanistic: assuming Mechanistic: assuming  Disease progression bas Disease progression bas follow follow-

• up data: clinical

up data: clinical d d i i l d l f i i l d l f data data - empirical model f empirical model f  Models for the study de Models for the study de  Multi Multi-

• center, adap

center, adap  Stochastic probabilistic Stochastic probabilistic  Stochastic, probabilistic Stochastic, probabilistic

an include an include an include an include

• ne knows enough
• ne knows enough

sed upon longitudinal sed upon longitudinal trials or observational trials or observational fi i fi i fitting fitting esign itself esign itself tive, group sequential tive, group sequential c sources of variability c sources of variability c, sources of variability c, sources of variability

Simul Simul Many options a Many options a Many options a Many options a Used for planni Used for planni

 To evaluate what you ob To evaluate what you ob suspected suspected - sensitivity a sensitivity a suspected suspected sensitivity a sensitivity a  To evaluate what you h To evaluate what you h data approaches data approaches -

• fill in

fill in i h i b i h i b points where it was obs points where it was obs  To predict what you did To predict what you did approaches with proper approaches with proper approaches with proper approaches with proper uncertainty associated w uncertainty associated w prediction itself prediction itself  l f l f  To evaluate performanc To evaluate performanc bias, achieved dose leve bias, achieved dose leve

lations lations and applications and applications and applications and applications ing and analysis ing and analysis

bserved, but where bias is bserved, but where bias is analysis analysis analysis analysis have not observed have not observed -

• missing

missing n data between design n data between design d served served d not observe d not observe -

• prediction

prediction r accounting for stochastic r accounting for stochastic r accounting for stochastic r accounting for stochastic with model selection and with model selection and ce characteristics; power, ce characteristics; power, els els

The current e The current e for protoco for protoco for protoco for protoco

Simulate the performance cha Simulate the performance cha designs , fixed or adaptive, befor designs , fixed or adaptive, befor exercise to engage the entire deve exercise to engage the entire deve made and the expectations such made and the expectations such made and the expectations such made and the expectations such everyone understand variability, everyone understand variability, uncertainties involved uncertainties involved – – that shou that shou

environment environment

• l planning
• l planning
• l planning
• l planning

aracteristics of candidate study aracteristics of candidate study re they are carried out and use the re they are carried out and use the lopment team in the choices to be lopment team in the choices to be h choices present h choices present it will help it will help h choices present h choices present – it will help it will help sources of heterogeneity, and the sources of heterogeneity, and the uld reduce failure rates hopefully uld reduce failure rates hopefully

Interactions with FDA Interactions with FDA

• f an adaptive
• f an adaptive

p possibly du possibly du

 Early and Middle Perio Early and Middle Perio  Late stages of drug deve Late stages of drug deve  Special Protocol Assess Special Protocol Assess p  FDA does not wish to b FDA does not wish to b design decisions, as wit design decisions, as wit decisions decisions  However, it may be tim However, it may be tim i t i ht d i t i ht d inspects, oversights, aud inspects, oversights, aud compliance with SOP’s compliance with SOP’s

A during the planning A during the planning e design study and e design study and g y g y uring its conduct uring its conduct

d of Drug development d of Drug development elopment elopment ments ments be involved in any adaptive be involved in any adaptive th not being aware of DMC th not being aware of DMC me to consider how one me to consider how one dit d dit d dits, and assures dits, and assures for firewalls for firewalls

Wh t d th Wh t d th What do oth What do oth

 A lot of literature coming ou A lot of literature coming ou blinded access to interim dat blinded access to interim dat  Others suggest new ways to Others suggest new ways to designs designs  V l t i i t i l t V l t i i t i l t  Value to up sizing a trial to m Value to up sizing a trial to m zone’ zone’  Interesting to see and hear N Interesting to see and hear N response and/or reactions response and/or reactions n response and/or reactions response and/or reactions – n this this  Also interesting in ADAPT Also interesting in ADAPT-

• I

interactions and iterative fee interactions and iterative fee interactions and iterative fee interactions and iterative fee experience experience

h thi k ? h thi k ? hers think ? hers think ?

ut ut – – some do not even agree that some do not even agree that ta protects type 1 error ta protects type 1 error p yp p yp analyze response adaptive analyze response adaptive i t i i t i th ‘ i i th ‘ i i maintain power maintain power – the ‘promising the ‘promising NIH network statisticians NIH network statisticians not clear where they are with not clear where they are with not clear where they are with not clear where they are with IT to observe the clinical team IT to observe the clinical team edback edback – part of the learning part of the learning edback edback – part of the learning part of the learning

So what have w So what have w the last 5 y the last 5 y the last 5 y the last 5 y

 There has been an extensive literatu There has been an extensive literatu  Regulators have helped focus the p Regulators have helped focus the p  Regulators have helped focus the p Regulators have helped focus the p future of these designs (Europe and future of these designs (Europe and  A lot of education remains A lot of education remains – – but it i but it i  Several companies now specializin Several companies now specializin  p p p p  FDA feedback and constructive adv FDA feedback and constructive adv are viewed and used are viewed and used  Methodology is not the issue Methodology is not the issue – – imp imp  Over promotion; naïve unbl Over promotion; naïve unbl  Continued need for communication Continued need for communication

we learned over we learned over years or so years or so years or so years or so

ure developed on AD’s with good ideas ure developed on AD’s with good ideas purpose utility and framework for the purpose utility and framework for the purpose, utility and framework for the purpose, utility and framework for the d US) d US) is happening; conferences, ADAPT is happening; conferences, ADAPT-

• IT

IT ng in advice and design ng in advice and design g g g g vice is very important to how the designs vice is very important to how the designs plementation is plementation is linding can be a problem linding can be a problem n among disciplines and experience n among disciplines and experience