Alternatives Assessment 123 Webinar: Mechanistic Data in a - - PowerPoint PPT Presentation
Alternatives Assessment 123 Webinar: Mechanistic Data in a Systematic Review Framework: Developing Confidence in Bodies of Evidence JANUARY 21, 2015 FACILITATED BY: JOEL TICKNER, SCD JOEL_TICKNER@UML.EDU LOWELL CENTER FOR SUSTAINABLE
JANUARY 21, 2015 FACILITATED BY: JOEL TICKNER, SCD JOEL_TICKNER@UML.EDU LOWELL CENTER FOR SUSTAINABLE PRODUCTION, UMASS LOWELL
* If you would like to ask a question or comment during this webinar please type your question in the Q&A box located in the control panel.
Continuing education and dialog To advance the practice of alternatives assessment for
informed substitution across federal, state, and local agencies through networking, sharing of experiences, development of common approaches, tools, datasets and frameworks, and creation of a community of practice.
significant gaps in toxicological data
data gaps and serve as primary data
collaborative effort, is to shift the assessment of chemical hazards from traditional experimental animal toxicology studies to one based on target-specific, mechanism- based, biological observations largely obtained using in vitro assays, with the ultimate aim of improving risk assessment for humans and the environment and the design of safer chemicals
To view program and to register visit: www.saferalternatives.org
Due to the number of participants on the Webinar, all lines
will be muted.
If you wish to ask a question, please type your question in
the Q&A box located in the drop down control panel at the top of the screen.
All questions will be answered at the end of the
presentations.
Andrew Rooney, PhD Office of Health Assessment and Translation National Institute of Environmental Health Sciences
January 21, 2015
– Office of Health Assessment and Translation – Systematic review, mechanistic data, and environmental health questions
– Planning – Identifying the evidence – Evaluating the evidence – Integrating the evidence
– SR on OHAT Website (http://ntp.niehs.nih.gov/go/38673)
to assess the evidence that environmental substances cause adverse health effects
– Hazard or State-of-science evaluations – Provide opinions on whether substances may be of concern given current human exposure
– Systematic review – Increasing integration of mechanistic data – Approaches to assess confidence in mechanistic data
NTP NTP Monograph
Developmental Effects and Pregnancy Outcomes Associated With Cancer Chemotherapy Use During Pregnancy Month 2013OHAT Evaluations
– Wide range of human study designs – Animal studies – Mechanistic studies (in vitro and other relevant data)
– 1) Integrate with human/animal evidence – 2) Potential to support decisions in absence of human or animal data
Human studies Animal studies Mechanistic studies Alternatives often have small, primarily mechanistic data sets
– The process for reaching conclusions on the NTP’s confidence across a body of studies within an evidence stream (i.e., human and animal data separately) and then integrating those conclusions across the evidence streams with consideration of other relevant data such as supporting evidence from mechanistic studies – Lack of consensus on term “Weight of Evidence”? (Weed et al., 2005)
Problem Formulation and Prepare Protocol Search for and Select Studies for Inclusion Extract Data from Studies Assess Quality of Individual Studies Rate Confidence in Body of Evidence Translate Confidence into Level of Evidence for Health Effects Integrate Evidence to Develop Hazard ID Conclusions
Systematic Review Evidence Integration
Integrate Evidence
Increased transparency and objectivity
− Individual study quality − Confidence in bodies of evidence − Hazard ID conclusions
Human Animal Mechanistic
Systematic Review
Evaluate Evidence
Human Animal Mechanistic Human Animal Mechanistic
Evidence Integration
Develop Protocol Identify Evidence
Search for Studies Extract Data Select Studies Assess Individual Study Quality Develop Hazard Identification Conclusions Rate Confidence in Bodies Of Evidence
Integrate Evidence
Human Animal Mechanistic
Systematic Review
Evaluate Evidence
Human Animal Mechanistic
Evidence Integration
Identify Evidence
Search for Studies Extract Data Select Studies Assess Individual Study Quality Develop Hazard Identification Conclusions Rate Confidence in Bodies Of Evidence
Planning
Scoping Problem Formulation Broad Literature Search
Human Animal Mechanistic
Develop Protocol
Planning step develops
– Population: Humans or animals without restriction on sex or life stage – Exposure: PFOA (CAS# 335-67-1) or PFOS (CAS# 1763-23-1) or their salts – Comparator: Humans or animals exposed to lower levels or vehicle – Outcomes:
– Outcomes:
– For narrow, well-defined outcomes
– For multiple outcomes or general health effects reviews
– After health effects are identified, additional search may be warranted – All changes are documented
Relevant outcomes may not be clear until after human and animal data are collected
Integrate Evidence
Human Animal Mechanistic
Systematic Review
Evaluate Evidence
Human Animal Mechanistic Human Animal Mechanistic
Evidence Integration
Develop Protocol Identify Evidence
Search for Studies Extract Data Select Studies Assess Individual Study Quality Develop Hazard Identification Conclusions Rate Confidence in Bodies Of Evidence
Challenge for Mechanistic Data
How narrowly do you define “relevant” studies? Or Which mechanistic data are relevant?
References identified through database searches (n=5,534) References identified through other sources (n=4)
Identification Screening Included
References excluded for criteria established in protocol (n=2,364) # of full-text articles excluded for pre-established criteria, with reasons
Studies included for data extraction in step 3, and risk of bias assessment in step 4 (n=114) Full-text articles assessed for relevance and eligibility (n= 315) References after duplicate removal Title-abstract screened for relevance and eligibility (n=2,675) Animal studies (n=80) Human studies (n=18) Mechanistic studies (n=19)
n=3
– Risk of bias or internal validity How credible are findings based on study design and conduct? – Reporting quality How well was the study reported? – Directness and applicability How well does the study address the topic under review?
Integrate Evidence
Human Animal Mechanistic
Systematic Review
Evaluate Evidence
Human Animal Mechanistic Human Animal Mechanistic
Evidence Integration
Develop Protocol Identify Evidence
Search for Studies Extract Data Select Studies Assess Individual Study Quality Develop Hazard Identification Conclusions Rate Confidence in Bodies Of Evidence
– Observational human studies – Animal studies – Mechanistic studies?
EBPCs
– Human studies – Animal toxicology studies – In vitro/mechanistic studies
– Answers equate to risk of bias rating for each question – Answers on 4-point scale from Clarity Group – Evaluation is endpoint specific
++ + − −−
Definitely Low risk of bias Probably Low risk of bias Probably High risk of bias Definitely High risk of bias
11: Were there other potential threats to internal validity?
Specific Guidance
Guidance defines all 4 ratings for each question
animals were allocated to any study group including controls using a method with a random component. Restricted randomization (e.g., blocked) will be considered low bias …
animals were allocated to any study group including controls using a method with a random component (i.e., authors state that allocation was random …
Was administered dose or exposure adequately randomized?
Support for final rating: “mice were randomly divided by weight”
weight into six groups of 12/group. Once distributed into groups, the mice were acclimated to cage conditions and
randomized?
– Helps to assure that treatment is not given selectively based on potential differences in human subjects, animals, cells, or tissues – Requires each human subject, animal, or cell had an equal chance
– In vitro /mechanistic applicability
depend on study design
– Finite cell strains with document number of population doublings – Primary cells from multiple donors – Homogeneous cell suspension
In vitro-specific Guidance
Experimental Animal Studies
– Direct evidence that animals were allocated to any study group including controls using a method with a random component. – Restricted randomization (e.g., blocked randomization) is considered acceptable – Requires concurrent controls
In vitro studies
– Direct evidence that cells were allocated to any study group including controls using a method with a random component. – OR all cells in culture come from a homogenous cell suspension recently collected from cell culture vessels following appropriate techniques – Requires concurrent control
Integrate Evidence
Human Animal Mechanistic
Systematic Review
Evaluate Evidence
Human Animal Mechanistic Human Animal Mechanistic
Evidence Integration
Develop Protocol Identify Evidence
Search for Studies Extract Data Select Studies Assess Individual Study Quality Develop Hazard Identification Conclusions Rate Confidence in Bodies Of Evidence
– How confident are you that findings from a group of studies reflect the true relationship between exposure to a substance and effect?
– Widely accepted method for rating confidence in a body of evidence on healthcare interventions
– Guidance for human and animal studies – Parallel approach for mechanistic studies – Initial confidence stratified based on study design features
Moderate (+++) 3 Features Low (++) 2 Features Very Low (+) 1≤ Features High (++++)
4 Features
Initial Confidence Cohort Case series Case report Ecologic Case-control Cross-sectional Human controlled trial Experimental animal
Each Body of Evidence
(as specified in protocol)
Factors Decreasing Confidence
Factors Increasing Confidence
Moderate (+++)
3 Features
Low (++)
2 Features
Very Low (+)
1≤ Features
High (++++)
4 Features
Initial Confidence
Cohort 3-features Case-Control 3-features
– Factors increasing/decreasing confidence
– magnitude of effect ≈ potency – dose-response – consistency – risk of bias – directness/applicability ≈ relevance
– publication bias
– Similarity profiles – Exploring utility of pathway approach (AOP)
Factors Decreasing Confidence
Factors Increasing Confidence
Factors Considered for Human and Animal Evidence
– Consider human and animal evidence together – Consider impact of mechanistic data
– Consider human and animal evidence together – Consider impact of mechanistic data
– strong support? – strong opposition?
Level of Evidence for Health Effects in Human Studies Level of Evidence for Health Effects in Animal Studies
Level of Evidence Reflects Confidence in Data
High Moderate Low / High Moderate Low/Inadequate Inadequate
– Low / Inadequate Level of Evidence
body of evidence for an association between exposure and health outcome
Alternatives may have no human data
“Presumed”
Level of Evidence for Health Effects in Human Studies Level of Evidence for Health Effects in Animal Studies
Level of Evidence Reflects Confidence in Data
High Moderate Low / High Moderate Low/Inadequate Inadequate
“Suspected” – Low / Inadequate Level of Evidence
body of evidence for an association between exposure and health outcome
Alternatives may have no human data
Level of Evidence for Health Effects in Human Studies Level of Evidence for Health Effects in Animal Studies
Level of Evidence Reflects Confidence in Data
High Moderate Low / High Moderate Low/Inadequate Inadequate
“Not classifiable”
– Low / Inadequate Level of Evidence
body of evidence for an association between exposure and health outcome
Alternatives may have no human data and no animal data
Level of Evidence for Health Effects in Human Studies Level of Evidence for Health Effects in Animal Studies
Challenging decisions in absence of human or animal data
High Moderate Low / High Moderate Low/Inadequate Inadequate
“Not classifiable”
– Mechanistic data
– It is envisioned that strong evidence for a relevant biological process from mechanistic data could result in a conclusion of “suspected” in the absence of human or animal data
“Suspected”
Rate Confidence in Body of Evidence Assess Individual Study Quality Search for and Select Studies, Extract Data Rate Confidence in Body of Evidence Assess Individual Study Quality Search for and Select Studies, Extract Data Search for and Select Studies, Extract Data ??? ??? Assess Individual Study Quality
Draft OHAT risk-of-bias tools for in vitro studies No established framework for rating confidence in bodies of evidence for diverse mechanistic data Established methods
Human Studies Animal Studies Mechanistic Data
Identify Evidence Evaluate Evidence Integrate Evidence OHAT Considers Parallel Factors
Rate Confidence in Body of Evidence
toxicology and environmental health
streams (human, animal, mechanistic studies)
support decision making
– Risk of bias tool for in vitro/mechanistic studies – Developing confidence ratings in mechanistic studies for integrating with human/animal effects – Developing confidence ratings in mechanistic studies for use as stand-alone evidence
– Abee Boyles – Kembra Howdeshell – Katie Pelch – Kyla Taylor – Kristina Thayer, Director – Vickie Walker
Development of OHAT Framework
– Lisa Bero, Director, San Francisco Branch, United States Cochrane Center at UC San Francisco – Gordon Guyatt, Co-chair, GRADE Working Group, McMaster University – Malcolm Macleod, CAMARADES Centre, University of Edinburgh – Karen Robinson, Co-Director, Evidence-Based Practice Center, Johns Hopkins Bloomberg School of Public Health – Holger Schünemann, Co-chair, GRADE Working Group, McMaster University – Tracey Woodruff, Director, Program on Reproductive Health and the Environment, UCSF
– Jamie Dewitt, ECU Department Pharmacology and Toxicology – Tony Fletcher, London School of Hygiene and Tropical Med. – Dori Germolec, NIEHS/NTP – Christopher Lau, US EPA, ORD/NHEERL – Roberta Scherer, Johns Hopkins, Bloomberg School Pub.H.
Management
– Stephanie Holmgren
– John Bucher, Associate Director – Mary Wolfe, Director OLPR – Lori White (OLPR)
– ATSDR – EPA – DoD – FDA – NIOSH
Presenters: Dr. Jay Tunkel and Ms. Cathy Rudisill, SRC, Inc.
Presenters: Thierry Nicot and Denis Mottet, European Chemicals Agency Tatiana Santos, European Environment Bureau Julius Waller EPPA Inc.