working with missing data
play

Working with Missing Data Steve Borgatti LINKS Center Workshop on - PDF document

Jun 20, 2023 •16 likes •116 views

2009 LINKS Center Workshop on Social Network Analysis Slide 1 Working with Missing Data Steve Borgatti LINKS Center Workshop on Social Network Analysis _____________________________________________________________________________________

  1. 2009 LINKS Center Workshop on Social Network Analysis Slide 1 Working with Missing Data Steve Borgatti LINKS Center Workshop on Social Network Analysis _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ __________ FRIDAY (c) 2009 LINKS Center ADVANCED Session

  2. 2009 LINKS Center Workshop on Social Network Analysis Slide 2 The problem: 1 1 1 1 1 1 1 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 Some respondents did • A A B B C G G L M P P P R S S T not participate in the - - - - - - - - - - - - - - - - survey, leaving blank 1 ACCIAIUOL 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 2 ALBIZZI 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0 rows in the network 3 BARBADORI 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 data matrix 4 BISCHERI Important note: if you 5 CASTELLAN 0 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 • 6 GINORI 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 enter data as edgelist 7 GUADAGNI 0 1 0 1 0 0 0 1 0 0 0 0 0 0 0 1 or nodelist using dl file, 8 LAMBERTES 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 missing values are 9 MEDICI 1 1 1 0 0 0 0 0 0 0 0 0 1 1 0 1 automatically 10 PAZZI 11 PERUZZI 0 0 0 1 1 0 0 0 0 0 0 0 0 0 1 0 converted to zeros. 12 PUCCI 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 How would you convert – 13 RIDOLFI 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 1 them back? 14 SALVIATI 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 15 STROZZI 0 0 0 1 1 0 0 0 0 0 1 0 1 0 0 0 16 TORNABUON 0 0 0 0 0 0 1 0 1 0 0 0 1 0 0 0 _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ __________ FRIDAY (c) 2009 LINKS Center ADVANCED Session

  3. 2009 LINKS Center Workshop on Social Network Analysis Slide 3 Size of the problem • Counting the number of missing values with Tools | Freq. – Select “matrices” 1 1 1 1 1 1 1 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 A A B B C G G L M P P P R S S T - - - - - - - - - - - - - - - - 1 ACCIAIUOL 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 2 ALBIZZI 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0 3 BARBADORI 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 4 BISCHERI 5 CASTELLAN 0 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 6 GINORI 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 GUADAGNI 0 1 0 1 0 0 0 1 0 0 0 0 0 0 0 1 8 LAMBERTES 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 9 MEDICI 1 1 1 0 0 0 0 0 0 0 0 0 1 1 0 1 10 PAZZI 11 PERUZZI 0 0 0 1 1 0 0 0 0 0 0 0 0 0 1 0 12 PUCCI 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 13 RIDOLFI 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 1 14 SALVIATI 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 15 STROZZI 0 0 0 1 1 0 0 0 0 0 1 0 1 0 0 0 16 TORNABUON 0 0 0 0 0 0 1 0 1 0 0 0 1 0 0 0 Output: 1 ----- 0.000 0.725 1.000 0.150 blank 0.125 _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ __________ FRIDAY (c) 2009 LINKS Center ADVANCED Session

  4. 2009 LINKS Center Workshop on Social Network Analysis Slide 4 Standard Solutions • Convert missings to zeros (since you did NOT observe a tie) – Re ‐ run having converted missings to ones, to see how different the results could be • Convert missings to zeros and ones at random, using density of the matrix as guide • Impute the missing values using other information – Symmetricity – QAP regression _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ __________ FRIDAY (c) 2009 LINKS Center ADVANCED Session

  5. 2009 LINKS Center Workshop on Social Network Analysis Slide 5 One solution 1 1 1 1 1 1 1 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 Suppose the data are • A A B B C G G L M P P P R S S T largely symmetric, and - - - - - - - - - - - - - - - - the social relation is 1 ACCIAIUOL 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 logically symmetric 2 ALBIZZI 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0 3 BARBADORI 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 Marriage to – 4 BISCHERI Saw movie with – 5 CASTELLAN 0 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 Then we can impute the • 6 GINORI 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 missing data from the 7 GUADAGNI 0 1 0 1 0 0 0 1 0 0 0 0 0 0 0 1 transpose of the matrix 8 LAMBERTES 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 9 MEDICI 1 1 1 0 0 0 0 0 0 0 0 0 1 1 0 1 i.e., assume that if A says – 10 PAZZI B is a friend, then if B had 11 PERUZZI 0 0 0 1 1 0 0 0 0 0 0 0 0 0 1 0 participated, s/he would 12 PUCCI 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 have said A was a a friend too 13 RIDOLFI 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 1 14 SALVIATI 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 So, fill in missing row with – 15 STROZZI 0 0 0 1 1 0 0 0 0 0 1 0 1 0 0 0 the corresponding column 16 TORNABUON 0 0 0 0 0 0 1 0 1 0 0 0 1 0 0 0 _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ __________ FRIDAY (c) 2009 LINKS Center ADVANCED Session

  6. 2009 LINKS Center Workshop on Social Network Analysis Slide 6 Doing it … • In matrix algebra, you can do it with the REPLACENA command: – newdata = replacena(olddata transp(olddata)) • Syntax – > <newds> = replacena(<ds1> <ds2>) – Where ds1 is the dataset that contains missing values and ds2 is the dataset from which to draw the correct values _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ __________ FRIDAY (c) 2009 LINKS Center ADVANCED Session

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Missing Data and Imputation NINA ORWITZ OCTOBER 30 TH , 2017 Outline Types of missing data

Missing Data and Imputation NINA ORWITZ OCTOBER 30 TH , 2017 Outline Types of missing data

Missing Data and Imputation NINA ORWITZ OCTOBER 30 TH , 2017 Outline Types of missing data Simple methods for dealing with missing data Single and multiple imputation R example Missing data is a complex problem We must consider:

377 views • 22 slides
Estimands and Missing Data Working Group ISCTM Spring Meeting Feb 19, 2019 Outline

Estimands and Missing Data Working Group ISCTM Spring Meeting Feb 19, 2019 Outline

Estimands and Missing Data Working Group ISCTM Spring Meeting Feb 19, 2019 Outline Objectives of this working group Background: ICH E9(R1) Addendum Defining estimands for a trial Components of an estimand Examples of

435 views • 20 slides
Missing data and data imputation with the Swiss Household Panel Andr Berchtold LIVES, LINES,

Missing data and data imputation with the Swiss Household Panel Andr Berchtold LIVES, LINES,

WHAT ARE MISSING DATA ? HOW TO TREAT MISSING DATA ? LONGITUDINAL DATA, CAUSALITY, &amp; ETHICS Missing data and data imputation with the Swiss Household Panel Andr Berchtold LIVES, LINES, Universit de Lausanne FORS SHP workshop June

1.2k views • 108 slides
Is the data missing at random? DEALIN G W ITH MIS S IN G DATA IN P YTH ON Suraj Donthi Deep

Is the data missing at random? DEALIN G W ITH MIS S IN G DATA IN P YTH ON Suraj Donthi Deep

Is the data missing at random? DEALIN G W ITH MIS S IN G DATA IN P YTH ON Suraj Donthi Deep Learning &amp; Computer Vision Consultant Possible reasons for missing data Note (variable data eld or column in a DataFrame) Values

650 views • 44 slides
Wh y do missing v al u es e x ist ? FE ATU R E E N G IN E E R IN G FOR MAC H IN E L E AR N IN

Wh y do missing v al u es e x ist ? FE ATU R E E N G IN E E R IN G FOR MAC H IN E L E AR N IN

Wh y do missing v al u es e x ist ? FE ATU R E E N G IN E E R IN G FOR MAC H IN E L E AR N IN G IN P YTH ON Robert O ' Callaghan Director of Data Science , Ordergroo v e Ho w gaps in data occ u r Data not being collected properl y Collection

602 views • 32 slides
Bayesian Generalized linear mixed models with data missing not at random Overview: Two simple

Bayesian Generalized linear mixed models with data missing not at random Overview: Two simple

Bayesian Generalized linear mixed models with data missing not at random Overview: Two simple introductory examples of data missing not at random (MNAR) Missing mechanism and likelihood in the case of missing at random (MAR) as defined by

722 views • 40 slides
Eliciting and using expert opinions about informatively missing outcome data in clinical trials

Eliciting and using expert opinions about informatively missing outcome data in clinical trials

Eliciting and using expert opinions about informatively missing outcome data in clinical trials Ian White MRC Biostatistics Unit, Cambridge, UK Bayes working group German Biometric Society Kln, 3 December 2004 1 Why do Bayesian analyses?

1.13k views • 73 slides
N 1 N 1 IV x i y i s i z i s i z i (2) 3. Imputation i 1

N 1 N 1 IV x i y i s i z i s i z i (2) 3. Imputation i 1

A Course in Applied Econometrics 1 . When Can Missing Data be Ignored ? Lecture 18 : Missing Data Linear model with IVs: y i x i u i , (1) Jeff Wooldridge IRP Lectures, UW Madison, August 2008 where x i is 1 K , instruments z

236 views • 10 slides
Lecture #12: kNN Classification and Missing Data Data Science 1 CS 109A, STAT 121A, AC 209A,

Lecture #12: kNN Classification and Missing Data Data Science 1 CS 109A, STAT 121A, AC 209A,

Lecture #12: kNN Classification and Missing Data Data Science 1 CS 109A, STAT 121A, AC 209A, E-109A Pavlos Protopapas Kevin Rader Margo Levine Rahul Dave Lecture Outline ROC Curves k -NN Revisited Dealing with Missing Data Types of

795 views • 67 slides
Multiple Imputation for Missing Data in KLoSA Juwon Song Korea University and UCLA Contents 1.

Multiple Imputation for Missing Data in KLoSA Juwon Song Korea University and UCLA Contents 1.

Multiple Imputation for Missing Data in KLoSA Juwon Song Korea University and UCLA Contents 1. Missing Data and Missing Data Mechanisms 2. Imputation 3. Missing Data and Multiple Imputation in Baseline KLoSA Data Missing Data and Multiple

634 views • 35 slides
Estimating Gaussian Mixture Models from Data with Missing Features by Daniel McMichael CSSIP

Estimating Gaussian Mixture Models from Data with Missing Features by Daniel McMichael CSSIP

Estimating Gaussian Mixture Models from Data with Missing Features by Daniel McMichael CSSIP Missing Data In classification we frequently see to classify objects using vectors of measured features. Sometimes these features are missing: [ 0

478 views • 10 slides
Performing and tracking imputation Nicholas Tierney Statistician DataCamp Dealing With Missing

Performing and tracking imputation Nicholas Tierney Statistician DataCamp Dealing With Missing

DataCamp Dealing With Missing Data in R DEALING WITH MISSING DATA IN R Performing and tracking imputation Nicholas Tierney Statistician DataCamp Dealing With Missing Data in R Lesson overview Using imputations to understand data structure

647 views • 53 slides
Missing Data in Machine Learning Guy Van den Broeck Emerging Challenges in Databases and AI

Missing Data in Machine Learning Guy Van den Broeck Emerging Challenges in Databases and AI

Computer Science Reasoning about Missing Data in Machine Learning Guy Van den Broeck Emerging Challenges in Databases and AI Research (DBAI) Nov 12 2019 Outline 1. Missing data at prediction time a. Reasoning about expectations b.

1.91k views • 46 slides
Big data: What are you missing? the risks of assuming data equals all David J. Hand Imperial

Big data: What are you missing? the risks of assuming data equals all David J. Hand Imperial

Big data: What are you missing? the risks of assuming data equals all David J. Hand Imperial College London and Winton Capital Management 6 th January 2016 Theory of Big Data, UCL 1 BACKGROUND The promise of big data: McKinseys big data

679 views • 48 slides
Effective Missing Data Prediction for Collaborative Filtering Hao Ma, Irwin King, and Michael R.

Effective Missing Data Prediction for Collaborative Filtering Hao Ma, Irwin King, and Michael R.

Outline Introduction Missing Data Prediction Empirical Analysis Conclusions and Future Work Effective Missing Data Prediction for Collaborative Filtering Hao Ma, Irwin King, and Michael R. Lyu Department of Computer Science and Engineering

1.2k views • 118 slides
Attention-based Learning for Missing Data Imputation in HoloClean Richard Wu 1 , A oqian Zhang 1 ,

Attention-based Learning for Missing Data Imputation in HoloClean Richard Wu 1 , A oqian Zhang 1 ,

Attention-based Learning for Missing Data Imputation in HoloClean Richard Wu 1 , A oqian Zhang 1 , Ihab F. Ilyas 1 Theodoros Rekatsinas 2 1 2 Problem Missing data is a persistent problem in many fields Sciences Data mining

322 views • 28 slides
INDIRECT DARK MATTER SEARCHES IN ICECUBE WIN2017, UC IRVINE 2 DM SEARCHES IN ICECUBE

INDIRECT DARK MATTER SEARCHES IN ICECUBE WIN2017, UC IRVINE 2 DM SEARCHES IN ICECUBE

Photo: NSF/Hans Boenish MORTEN MEDICI INDIRECT DARK MATTER SEARCHES IN ICECUBE WIN2017, UC IRVINE 2 DM SEARCHES IN ICECUBE MORTEN MEDICI INDIRECT DETECTION WITH NEUTRINOS Indirect dark

716 views • 27 slides
Medici October 6-7, 2017 Laurel Heights Conference Center San Francisco, California Course

Medici October 6-7, 2017 Laurel Heights Conference Center San Francisco, California Course

Asian Health Institute, Department of Medicine University of California, San Francisco Presents 6 th Annual Asian Health Symposium Medici October 6-7, 2017 Laurel Heights Conference Center San Francisco, California Course Directors Gordon

403 views • 13 slides
7th ANNUAL Asian Health Symposium Medici October 11-12, 2018 Laurel Heights Conference Center

7th ANNUAL Asian Health Symposium Medici October 11-12, 2018 Laurel Heights Conference Center

Asian Health Institute Department of Medicine University of California, San Francisco University of Washington Kaiser Permanente presents 7th ANNUAL Asian Health Symposium Medici October 11-12, 2018 Laurel Heights Conference Center San

352 views • 11 slides
Spectra, dynamical systems, and geometry Fields Medal Symposium, Fields Institute, Toronto

Spectra, dynamical systems, and geometry Fields Medal Symposium, Fields Institute, Toronto

Spectra, dynamical systems, and geometry Fields Medal Symposium, Fields Institute, Toronto Tuesday, October 1, 2013 Dmitry Jakobson April 16, 2014 M = S 1 = R / ( 2 Z ) - a circle. f ( x ) - a periodic function, f ( x + 2 ) = f ( x ) .

360 views • 33 slides
Real-World Networks And their common properties 1. Macro-level (graph-level) 2. Micro-level

Real-World Networks And their common properties 1. Macro-level (graph-level) 2. Micro-level

9/22/20 DCS/CSCI 2350 Social &amp; Economic Networks What does a real-world network look like? Reading: Ch 2 of EK, Ch 2 &amp; 3 of Jackson Graph visualization using Gephi Mohammad T . Irfan Email: mirfan@bowdoin.edu 1 Real-World Networks

749 views • 26 slides
BootcampR AN INTRODUCTION TO R Jason A. Heppler, PhD University of Nebraska at Omaha March 10,

BootcampR AN INTRODUCTION TO R Jason A. Heppler, PhD University of Nebraska at Omaha March 10,

BootcampR AN INTRODUCTION TO R Jason A. Heppler, PhD University of Nebraska at Omaha March 10, 2020 @jaheppler Hi. I'm Jason. I like to gesture at screens. Digital Engagement Librarian , University of Nebraska at Omaha Mentor, Mozilla Open

492 views • 46 slides
THE DEEP HEAT MINING PROJECT Status February 2006 by Robert J. Hopkirk 1 &amp; Markus O. Hring

THE DEEP HEAT MINING PROJECT Status February 2006 by Robert J. Hopkirk 1 &amp; Markus O. Hring

THE DEEP HEAT MINING PROJECT Status February 2006 by Robert J. Hopkirk 1 &amp; Markus O. Hring 2 DEEP HEAT MINING Association (CH-8162 Steinmaur) 1 Polydynamics Engineering, CH-8708 Mnnedorf 2 Hring Geo-Project. CH-8162 Steinmaur M.O.

423 views • 21 slides
Variational wave functions for multiband Hubbard models Federico Becca CNR IOM-DEMOCRITOS and

Variational wave functions for multiband Hubbard models Federico Becca CNR IOM-DEMOCRITOS and

Variational wave functions for multiband Hubbard models Federico Becca CNR IOM-DEMOCRITOS and International School for Advanced Studies (SISSA) ICTP Trieste, November, 2017 C. de Franco (SISSA), L.F. Tocchio (Torino) R. Kaneko (Tokyo), R.

581 views • 30 slides

More recommend