Gov 51: Missing Data Matthew Blackwell Harvard University 1 / 7

Civilian attitudes and war against insurgency • War in Afghanistan: counter-insurgency war • Military against insurgents • Key to victory: winning hearts and minds of civilians • Aid provision, information campaign, minimizing civilian casualties • How does exposure to violence afgect support for Taliban, coalition? 2 / 7

Afghan study 0 Logar Baraki Barak 80 18 10 ## employed income violent.exp.ISAF ## 1 0 2,001-10,000 0 ## 2 1 2,001-10,000 ## 3 12 1 2,001-10,000 1 ## 4 1 2,001-10,000 0 ## 5 1 2,001-10,000 0 ## 6 1 <NA> 0 ## 6 21 afghan <- read.csv(”data/afghan.csv”) 49 head(afghan[, 1:8]) ## province district village.id age educ.years ## 1 Logar Baraki Barak 80 26 10 ## 2 Logar Baraki Barak 80 3 80 ## 3 Logar Baraki Barak 80 60 0 ## 4 Logar Baraki Barak 80 34 14 ## 5 Logar Baraki Barak 3 / 7

• Missing data in R: a special value NA Missing data • Nonresponse : respondent can’t or won’t answer question. • Sensitive questions social desirability bias • Some countries lack offjcial statistics like unemployment. • Leads to missing data. • Causes problems with calculating statistics: ## prop. of those who got hurt by ISAF mean(afghan$violent.exp.ISAF) ## [1] NA 4 / 7

• Missing data in R: a special value NA Missing data • Nonresponse : respondent can’t or won’t answer question. • Some countries lack offjcial statistics like unemployment. • Leads to missing data. • Causes problems with calculating statistics: ## prop. of those who got hurt by ISAF mean(afghan$violent.exp.ISAF) ## [1] NA 4 / 7 • Sensitive questions ⇝ social desirability bias

Missing data • Nonresponse : respondent can’t or won’t answer question. • Some countries lack offjcial statistics like unemployment. • Leads to missing data. • Causes problems with calculating statistics: ## prop. of those who got hurt by ISAF mean(afghan$violent.exp.ISAF) ## [1] NA 4 / 7 • Sensitive questions ⇝ social desirability bias • Missing data in R: a special value NA

Handling missing data in R • Adding na.rm = TRUE to some functions removes missing data. mean(afghan$violent.exp.ISAF, na.rm = TRUE) ## [1] 0.375 • Or, you can explicitly remove missing values using na.omit() function: mean(na.omit(afghan$violent.exp.ISAF)) ## [1] 0.375 • Add NA to table() with exclude = NULL : table(ISAF = afghan$violent.exp.ISAF, exclude = NULL) ## ISAF ## 0 1 <NA> ## 1706 1023 25 5 / 7

• Or, you can explicitly remove missing values using na.omit() function: Handling missing data in R • Adding na.rm = TRUE to some functions removes missing data. mean(afghan$violent.exp.ISAF, na.rm = TRUE) ## [1] 0.375 mean(na.omit(afghan$violent.exp.ISAF)) ## [1] 0.375 • Add NA to table() with exclude = NULL : table(ISAF = afghan$violent.exp.ISAF, exclude = NULL) ## ISAF ## 0 1 <NA> ## 1706 1023 25 5 / 7

Handling missing data in R • Adding na.rm = TRUE to some functions removes missing data. mean(afghan$violent.exp.ISAF, na.rm = TRUE) ## [1] 0.375 • Or, you can explicitly remove missing values using na.omit() function: mean(na.omit(afghan$violent.exp.ISAF)) ## [1] 0.375 • Add NA to table() with exclude = NULL : table(ISAF = afghan$violent.exp.ISAF, exclude = NULL) ## ISAF ## 0 1 <NA> ## 1706 1023 25 5 / 7

• Add NA to table() with exclude = NULL : Handling missing data in R • Adding na.rm = TRUE to some functions removes missing data. mean(afghan$violent.exp.ISAF, na.rm = TRUE) ## [1] 0.375 • Or, you can explicitly remove missing values using na.omit() function: mean(na.omit(afghan$violent.exp.ISAF)) ## [1] 0.375 table(ISAF = afghan$violent.exp.ISAF, exclude = NULL) ## ISAF ## 0 1 <NA> ## 1706 1023 25 5 / 7

Handling missing data in R • Adding na.rm = TRUE to some functions removes missing data. mean(afghan$violent.exp.ISAF, na.rm = TRUE) ## [1] 0.375 • Or, you can explicitly remove missing values using na.omit() function: mean(na.omit(afghan$violent.exp.ISAF)) ## [1] 0.375 table(ISAF = afghan$violent.exp.ISAF, exclude = NULL) ## ISAF ## 0 1 <NA> ## 1706 1023 25 5 / 7 • Add NA to table() with exclude = NULL :

Gov 51: Missing Data Matthew Blackwell Harvard University 1 / 7 - PowerPoint PPT Presentation

Gov 51: Missing Data Matthew Blackwell Harvard University 1 / 7 Civilian attitudes and war against insurgency War in Afghanistan: counter-insurgency war Military against insurgents Key to victory: winning hearts and minds of

Missing Data and Imputation NINA ORWITZ OCTOBER 30 TH , 2017 Outline Types of missing data

Missing data and data imputation with the Swiss Household Panel Andr Berchtold LIVES, LINES,

Is the data missing at random? DEALIN G W ITH MIS S IN G DATA IN P YTH ON Suraj Donthi Deep

Wh y do missing v al u es e x ist ? FE ATU R E E N G IN E E R IN G FOR MAC H IN E L E AR N IN

Bayesian Generalized linear mixed models with data missing not at random Overview: Two simple

N 1 N 1 IV x i y i s i z i s i z i (2) 3. Imputation i 1

Lecture #12: kNN Classification and Missing Data Data Science 1 CS 109A, STAT 121A, AC 209A,

Multiple Imputation for Missing Data in KLoSA Juwon Song Korea University and UCLA Contents 1.

Estimating Gaussian Mixture Models from Data with Missing Features by Daniel McMichael CSSIP

Performing and tracking imputation Nicholas Tierney Statistician DataCamp Dealing With Missing

Missing Data in Machine Learning Guy Van den Broeck Emerging Challenges in Databases and AI

Big data: What are you missing? the risks of assuming data equals all David J. Hand Imperial

Effective Missing Data Prediction for Collaborative Filtering Hao Ma, Irwin King, and Michael R.

Attention-based Learning for Missing Data Imputation in HoloClean Richard Wu 1 , A oqian Zhang 1 ,

Approaches to imputing missing data in complex survey data Christine Wells, Ph.D. IDRE UCLA

Searching for and replacing missing values Nicholas Tierney Statistician DataCamp Dealing With

Semester projects The Plan Principles of Complex Systems Suggestions for CSYS/MATH 300, Fall,

AEC POP & JAZZ AEC : the leading voice for Platform Meeting 2015 Higher Music Education in

language aesthetics Sam Aaron "The aesthetic is at once, as I try to show, the very secret

Coda Mechanical Sheet Music Page Turner pink b pink b Need: Current Workarounds

Modeling conflict, with examples from terrorism Kristian Skrede Gleditsch Based on joint work

2017 changing Environment Airline Economics Growth Frontiers Dublin Presented by Abdol Moabery,

NADA Implementation in Mozilla Browser and Draft Update draft-ietf-rmcat-nada-11 Xiaoqing Zhu,

Hyperintensionality and Impossible Worlds: An Introduction David Chalmers One Guiding Idea