bets the dangers of selection bias in early analyses of
play

BETS: The dangers of selection bias in early analyses of the - PowerPoint PPT Presentation

Dec 20, 2023 •119 likes •555 views

BETS: The dangers of selection bias in early analyses of the coronavirus disease (COVID-19) pandemic Qingyuan Zhao Statistical Laboratory, University of Cambridge May 5, 2020 @ YSPH Biostatistics Seminar Manuscript: arXiv:2004.07743 Slides:

  1. BETS: The dangers of selection bias in early analyses of the coronavirus disease (COVID-19) pandemic Qingyuan Zhao Statistical Laboratory, University of Cambridge May 5, 2020 @ YSPH Biostatistics Seminar Manuscript: arXiv:2004.07743 Slides: http://www.statslab.cam.ac.uk/~qz280/ .

  2. Collaborators Nianqiao (Phyllis) Ju Sergio Bacallado Rajen Shah PhD student at Harvard Stats Lab, Cambridge Stats Lab, Cambridge And many thanks to... Cindy Chen, Yang Chen, Yunjin Choi, Hera He, Michael Levy, Marc Lipsitch, James Robins, Andrew Rosenfeld, Dylan Small, Yachong Yang, Zilu Zhou, and many other who have provided helpful suggestions. Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 1 / 53

  3. COVID-19 is personal for everyone Me and my parents, all grew up in in Wuhan, China. (September 7, 2019) Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 2 / 53

  4. Wuhan Lockdown (January 23, 2020) Before the lockdown After the lockdown Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 3 / 53

  5. The beginning of this project On January 29, I heard from my parents that a close relative was just diagnosed with “viral pneumonia”. This prompted me to start looking into the data available at the time. However, epidemiological data from Wuhan are very unreliable! Some anecdotal evidence Inadequate testing: The relative of mine could not get a RT-PCR test till mid-February, when she was already recovering. False negative test: Her first test was negative. A few days later she was tested again and the result came back positive. Insufficient contact tracing: Her husband who also showed COVID symptoms quickly recovered and was never tested. Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 4 / 53

  6. Insufficient testing in Wuhan A change of diagnostic criterion on February 12 led to a huge spike of cases. Solution: Using cases “exported” from Wuhan This has two benefits: Testing and contact tracing were intensive in other locations. 1 Detailed case reports (instead of mere case counts) are often available. 2 This design was first used by Neil Ferguson’s team in Imperial College, who estimated on January 17 that there might be already over 1,700 cases in Wuhan. Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 5 / 53

  7. Our first analysis Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 6 / 53

  8. A puzzling comparison Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 7 / 53

  9. Which one is correct? 1,000,000 United States United States Italy Spain France United Kingdom 10,000 Spain Italy Germany France United Kingdom 100,000 Iran Belgium Iran Turkey Germany Netherlands Belgium Canada Netherlands Brazil Switzerland Brazil Turkey Russia Portugal Sweden Total deaths Total cases Austria 1,000 Canada Switzerland Israel India Ireland Sweden Peru South Korea 10,000 Portugal Japan Ecuador Chile Poland Romania Norway Indonesia Czech Republic Denmark Australia Pakistan MexicoAustria Ireland Mexico Saudi Arabia India Philippines Malaysia Romania Ecuador United Arab Emirates Indonesia Algeria Philippines Denmark Serbia Poland Panama Qatar Belarus Dominican Republic UkraineLuxembourg Finland Singapore Peru South Korea Colombia Thailand Dominican Republic South Africa Argentina Egypt Russia Egypt Greece Czech Republic Algeria Moldova Morocco Hungary Croatia Iceland Colombia Morocco Norway Hungary Bahrain Israel Japan Estonia Iraq Kuwait Pakistan Argentina Kazakhstan Ukraine Greece Uzbekistan Armenia Azerbaijan Slovenia 100 Chile Panama Bosnia and Herzegovina New Zealand Lithuania Bangladesh Serbia MalaysiaIraq 1,000 Saudi Arabia Luxembourg Finland Australia Slovenia Singapore 100 10 0 20 40 60 0 20 40 Days since 100 cases Days since 10 deaths In countries most hard hit by COVID-19, the total cases and deaths grew about 100 times in the first 20 days (doubling time: 20 / log 2 (100) = 3 . 01 days). Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 8 / 53

  10. How can the results be so different? Spoilers... Similar data and model were used in these two studies, with one crucial difference: The Lancet study did not take into account the travel ban. Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 9 / 53

  11. Rest of the talk Overview of selection bias 1 Dataset 2 Model 3 Why some early analyses were severely biased? 4 Bayesian nonparametric inference 5 Conclusions 6 Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 10 / 53

  12. Bias (i): Under-ascertainment This may occur if symptomatic patients did not seek healthcare or could not be diagnosed. Susceptible studies: All studies using cases confirmed when testing is insufficient. Direction of bias: Varied, depending on the pattern of under-ascertainment and parameter of interest. Solution: Use carefully considered and planned study designs. Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 12 / 53

  13. Bias (ii): Non-random sample selection Cases included in the study are not representative of the population. Susceptible studies: All studies, as detailed information of COVID-19 cases is sparse, but especially those without clear inclusion criteria. Direction of bias: Varied. Solution: Follow a protocol for data collection and exclude data that do not meet the sample inclusion criterion. Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 13 / 53

  14. Bias (iii): Travel ban Outbound travel from Wuhan was banned from January 23, 2020 to April 8, 2020. Susceptible studies: Studies that analyze cases exported from Wuhan. Direction of bias: Under-estimation of epidemic growth and infection-to-recovery time. Solution: Derive tailored likelihood functions to account for travel restrictions. Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 14 / 53

  15. Bias (iv): Epidemic growth Patients were more likely to be infected towards the end of their exposure period. Susceptible studies: Studies that treat infections as uniformly distributed over the exposure period. Direction of bias: Over-estimation of the incubation period. Solution: Derive tailored likelihood functions to account for epidemic growth. Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 15 / 53

  16. Bias (v): Right-truncation Cases confirmed after a certain time are excluded from the dataset. Susceptible studies: Studies that only use cases detected early in an epidemic. Direction of bias: Under-estimation of the incubation period. Solution: Collect all cases that meet a selection criterion, do not end data collection 1 prematurely; Derive tailored likelihood functions to correct for right-truncation. 2 Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 16 / 53

  17. Recap Types of bias in COVID-19 analyses (i) Under-ascertainment. (ii) Non-random sample selection. (iii) Travel ban. (iv) Epidemic growth. (v) Right-truncation. Keys to avoid the selection bias Carefully design the study and adhere to the sample inclusion criterion. 1 Start from a generative model and derive likelihood functions that adjust for 2 sample selection. Qingyuan Zhao (Stats Lab, Cambridge) BETS on COVID-19 May 5, 2020 17 / 53

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

BETS: The dangers of selection bias in early analyses of the coronavirus disease (COVID-19)

BETS: The dangers of selection bias in early analyses of the coronavirus disease (COVID-19)

BETS: The dangers of selection bias in early analyses of the coronavirus disease (COVID-19) pandemic Qingyuan Zhao Statistical Laboratory, University of Cambridge September 24, 2020 @ Ohio State University Manuscript: arXiv:2004.07743

629 views • 40 slides
Making Generative Classifiers Robust to Selection Bias Andrew Smith Charles Elkan November

Making Generative Classifiers Robust to Selection Bias Andrew Smith Charles Elkan November

Making Generative Classifiers Robust to Selection Bias Andrew Smith Charles Elkan November 30th, 2007 Outline What is selection bias? Types of selection bias. Overcoming learnable bias with weighting. Overcoming bias with

736 views • 59 slides
Review Selection bias, overfitting Bias v. variance v. residual Bias-variance tradeoff

Review Selection bias, overfitting Bias v. variance v. residual Bias-variance tradeoff

Review Selection bias, overfitting Bias v. variance v. residual Bias-variance tradeoff 1 n=1 Cramr-Rao bound n=4 0.8 n=30 0.6 CDF of max of n samples of 0.4 N( =2, 2 =1) 0.2 [representing error estimates for n

517 views • 28 slides
Variable selection bias Bias in Ensemble Bias in Ensemble Methods Methods Variable selection

Variable selection bias Bias in Ensemble Bias in Ensemble Methods Methods Variable selection

Variable Selection Variable Selection Variable selection bias Bias in Ensemble Bias in Ensemble Methods Methods Variable selection Variable selection Variable Selection Bias in Classification bias bias rpart Trees and Ensemble Methods

183 views • 6 slides
Early SUSY analyses with ATLAS Giacomo Polesello INFN, Sezione di Pavia Early analyses at the

Early SUSY analyses with ATLAS Giacomo Polesello INFN, Sezione di Pavia Early analyses at the

. Early SUSY analyses with ATLAS Giacomo Polesello INFN, Sezione di Pavia Early analyses at the LHC The LHC will start producing high-energy collisions in the next months Large uncertainties on the data-taking parameters: Energy (ies)

634 views • 40 slides
Using Cost-Benefit Analyses to Promote Using Cost-Benefit Analyses to Promote y y the Early

Using Cost-Benefit Analyses to Promote Using Cost-Benefit Analyses to Promote y y the Early

Using Cost-Benefit Analyses to Promote Using Cost-Benefit Analyses to Promote y y the Early Childhood Agenda the Early Childhood Agenda Rob Bradham Senior Vice President of Business Development p and Government Affairs Greater Richmond

521 views • 16 slides
Dose selection Dose selection in early paediatric development in early paediatric development

Dose selection Dose selection in early paediatric development in early paediatric development

Dose selection Dose selection in early paediatric development in early paediatric development Oscar Della Pasqua Clinical Pharmacology Modelling & Simulation Leiden/Amsterdam Center for Drug Research Division of Pharmacology Outline

578 views • 29 slides
How Non Ignorable is the Selection Bias in Non Probability Samples? An Illustration of New

How Non Ignorable is the Selection Bias in Non Probability Samples? An Illustration of New

11/8/2018 How Non Ignorable is the Selection Bias in Non Probability Samples? An Illustration of New Measures using a Large Genetic Study on Facebook Brady T. West 1 Phil Boonstra 2 Roderick J.A. Little 1,2 Jingwei Hu 1 Fernanda Alvarado

382 views • 14 slides
So you think you have all the data? Causes and consequences of selection bias David J. Hand

So you think you have all the data? Causes and consequences of selection bias David J. Hand

So you think you have all the data? Causes and consequences of selection bias David J. Hand Imperial College London and Winton Capital Management 16 th September 2016 1 BACKGROUND The promise of big data: Out with every theory of human behavior,

402 views • 38 slides
Identification of Causal Effect in the Presence of Selection Bias Juan D. Correa Jin Tian Elias

Identification of Causal Effect in the Presence of Selection Bias Juan D. Correa Jin Tian Elias

Identification of Causal Effect in the Presence of Selection Bias Juan D. Correa Jin Tian Elias Bareinboim AAAI Honolulu, 2019 Challenge 1: Confounding Bias Age Whats the causal effect of Exercise on Cholesterol ? What about

297 views • 15 slides
Selection and estimation in exploratory subgroup analyses a proposal Gerd Rosenkranz, Novartis

Selection and estimation in exploratory subgroup analyses a proposal Gerd Rosenkranz, Novartis

Selection and estimation in exploratory subgroup analyses a proposal Gerd Rosenkranz, Novartis Pharma AG, Basel, Switzerland EMA Workshop, London, 07-Nov-2014 Purpose of this presentation Proposal for exploratory subgroup analyses

290 views • 12 slides
BIAS What Is Bias? Bias can be defined as favoring one side, position, or belief being

BIAS What Is Bias? Bias can be defined as favoring one side, position, or belief being

BIAS What Is Bias? Bias can be defined as favoring one side, position, or belief being partial or prejudiced Where have you seen bias? Bias vs. Propaganda Bias is prejudice; a preconceived judgment or an opinion formed

717 views • 29 slides
An Interval Estimation Approach to Selection Bias in Observational Studies Matt Tudball 1 with

An Interval Estimation Approach to Selection Bias in Observational Studies Matt Tudball 1 with

An Interval Estimation Approach to Selection Bias in Observational Studies Matt Tudball 1 with Rachael Hughes 1 , Kate Tilling 1 , Qingyuan Zhao 2 and Jack Bowden 1 1 MRC Integrative Epidemiology Unit, University of Bristol 2 Department of

282 views • 11 slides
ECON 626: Applied Microeconomics Lecture 1: Selection Bias and the Experimental Ideal

ECON 626: Applied Microeconomics Lecture 1: Selection Bias and the Experimental Ideal

ECON 626: Applied Microeconomics Lecture 1: Selection Bias and the Experimental Ideal Professors: Pamela Jakiela and Owen Ozier Potential Outcomes Do Hospitals Make People Healthier? Your health status is: excellent, very good, good, fair, or

1.27k views • 94 slides
A Modeling Approach to Compensate for Nonresponse and Selection Bias in Surveys? Tien-Huan (Amy)

A Modeling Approach to Compensate for Nonresponse and Selection Bias in Surveys? Tien-Huan (Amy)

A Modeling Approach to Compensate for Nonresponse and Selection Bias in Surveys? Tien-Huan (Amy) Lin, Ismael Flores Cervantes Westat JSM 2019 Denver, Colorado July 27-August 1 The views presented in this paper are those of the

646 views • 13 slides
Using Hierarchical Models to Calibrate Selection Bias Douglas Rivers Stanford University and

Using Hierarchical Models to Calibrate Selection Bias Douglas Rivers Stanford University and

Using Hierarchical Models to Calibrate Selection Bias Douglas Rivers Stanford University and YouGov February 26, 2016 Margins of error We agree that margin of sampling error in surveys has an accepted meaning and that this measure is not

564 views • 20 slides
Human Mobility Restrictions and the Spread of the Novel Coronavirus (2019-nCoV) in China Hanming

Human Mobility Restrictions and the Spread of the Novel Coronavirus (2019-nCoV) in China Hanming

Human Mobility Restrictions and the Spread of the Novel Coronavirus (2019-nCoV) in China Hanming Fang 1 , 2 , Long Wang 2 , Yang (Zoe) Yang 3 1 University of Pennsylvania & NBER 2 ShanghaiTech University 3 The Chinese University of Hong Kong

369 views • 26 slides
The presentation will start shortly COVID-19 Health Care Provider Briefing Middlesex and London

The presentation will start shortly COVID-19 Health Care Provider Briefing Middlesex and London

The presentation will start shortly COVID-19 Health Care Provider Briefing Middlesex and London Region May 8, 2020 Welcome Presenter: Dr. Alex Summers Associate Medical Officer of Health Middlesex-London Health Unit Surveillance Report As

599 views • 26 slides
Key Concept Definitions Staying home allows individuals who may have been exposed to COVID-19 to

Key Concept Definitions Staying home allows individuals who may have been exposed to COVID-19 to

Key Concept Definitions Staying home allows individuals who may have been exposed to COVID-19 to monitor their symptoms during the period in which they Staying home may be infectious. These individuals should separate themselves from others

468 views • 4 slides
Childbearing Postponement, its Option Value, and the Biological Clock David de la Croix 1 Aude

Childbearing Postponement, its Option Value, and the Biological Clock David de la Croix 1 Aude

Childbearing Postponement, its Option Value, and the Biological Clock David de la Croix 1 Aude Pommeret 2 1 Universit e catholique de Louvain and CEPR 2 City University of Hong Kong and Universit e Savoie Mont Blanc October, 2018, Mannheim

813 views • 42 slides
Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University About

Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University About

Temporal Graph Analytics with GRADOOP Christopher Rost and Kevin Gomez Leipzig University About the speakers and the team Kevin, Christopher, Prof. Dr. Erhard Rahm, PhD Student PhD Student Database Chair Timo, Lucas, M.Sc. Student

501 views • 31 slides
Guidelines, and Testing August 12, 2020 Presented by Ashley Wegner, MPH, CIC Kim Mawby, MSN,

Guidelines, and Testing August 12, 2020 Presented by Ashley Wegner, MPH, CIC Kim Mawby, MSN,

COVID-19 Updates, Guidelines, and Testing August 12, 2020 Presented by Ashley Wegner, MPH, CIC Kim Mawby, MSN, RN, CBS Corrie Courtney, MPH, BSN, RN COVID-19 February 11, 2020 the World Health Organization announced an official name

520 views • 23 slides
Novel 2019 Coronavirus Update (COVID-19) HCPA March 4 th 2020 Prof Jim McManus Director of

Novel 2019 Coronavirus Update (COVID-19) HCPA March 4 th 2020 Prof Jim McManus Director of

Novel 2019 Coronavirus Update (COVID-19) HCPA March 4 th 2020 Prof Jim McManus Director of Public Health www.hertfordshire.gov.uk www.hertfordshire.gov.uk Note All this information is in the Public Domain All has been taken from

530 views • 36 slides
The Coronavirus: What Insurance Will and Wont Do. What We Know Today -1,300 Known Cases in

The Coronavirus: What Insurance Will and Wont Do. What We Know Today -1,300 Known Cases in

The Coronavirus: What Insurance Will and Wont Do. What We Know Today -1,300 Known Cases in the US https://informationisbeautiful.net/visualizations/covid- 19-coronavirus-infographic-datapack/ (Epicenters: Seattle, Bay Area, and New York)

479 views • 10 slides

More recommend