introd u ction to e x plorator y data anal y sis
play

Introd u ction to E x plorator y Data Anal y sis STATISTIC AL TH IN - PowerPoint PPT Presentation

Jul 06, 2023 •269 likes •734 views

Introd u ction to E x plorator y Data Anal y sis STATISTIC AL TH IN K IN G IN P YTH ON ( PAR T 1 ) J u stin Bois Lect u rer at the California Instit u te of Technolog y E x plorator y data anal y sis The process of organi z ing , plo ing ,

  1. Introd u ction to E x plorator y Data Anal y sis STATISTIC AL TH IN K IN G IN P YTH ON ( PAR T 1 ) J u stin Bois Lect u rer at the California Instit u te of Technolog y

  2. E x plorator y data anal y sis The process of organi z ing , plo � ing , and s u mmari z ing a data set STATISTICAL THINKING IN PYTHON ( PART 1)

  3. “ E x plorator y data anal y sis can ne v er be the w hole stor y, b u t nothing else can ser v e as the fo u ndation stone .” — John T u ke y STATISTICAL THINKING IN PYTHON ( PART 1)

  4. 2008 US s w ing state election res u lts 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  5. 2008 US s w ing state election res u lts import pandas as pd df_swing = pd.read_csv('2008_swing_states.csv') df_swing[['state', 'county', 'dem_share']] state county dem_share 0 PA Erie County 60.08 1 PA Bradford County 40.64 2 PA Tioga County 36.07 3 PA McKean County 41.21 4 PA Potter County 31.04 5 PA Wayne County 43.78 6 PA Susquehanna County 44.08 7 PA Warren County 46.85 8 OH Ashtabula County 56.94 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  6. 2008 US s w ing state election res u lts 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  7. Let ' s practice ! STATISTIC AL TH IN K IN G IN P YTH ON ( PAR T 1 )

  8. Plotting a histogram STATISTIC AL TH IN K IN G IN P YTH ON ( PAR T 1 ) J u stin Bois Lect u rer at the California Instit u te of Technolog y

  9. 2008 US s w ing state election res u lts Data retrie v ed from Data . go v ( h � ps ://www. data . go v/ ) STATISTICAL THINKING IN PYTHON ( PART 1)

  10. Generating a histogram import matplotlib.pyplot as plt _ = plt.hist(df_swing['dem_share']) _ = plt.xlabel('percent of vote for Obama') _ = plt.ylabel('number of counties') plt.show() STATISTICAL THINKING IN PYTHON ( PART 1)

  11. Al w a y s label y o u r a x es STATISTICAL THINKING IN PYTHON ( PART 1)

  12. 2008 US s w ing state election res u lts Data retrie v ed from Data . go v ( h � ps ://www. data . go v/ ) STATISTICAL THINKING IN PYTHON ( PART 1)

  13. Histograms w ith different binning Data retrie v ed from Data . go v ( h � ps ://www. data . go v/ ) STATISTICAL THINKING IN PYTHON ( PART 1)

  14. Setting the bins of a histogram bin_edges = [0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100] _ = plt.hist(df_swing['dem_share'], bins=bin_edges) plt.show() STATISTICAL THINKING IN PYTHON ( PART 1)

  15. Setting the bins of a histogram _ = plt.hist(df_swing['dem_share'], bins=20) plt.show() STATISTICAL THINKING IN PYTHON ( PART 1)

  16. Seaborn An e x cellent Matplotlib - based statistical data v is u ali z ation package w ri � en b y Michael Waskom STATISTICAL THINKING IN PYTHON ( PART 1)

  17. Setting Seaborn st y ling import seaborn as sns sns.set() _ = plt.hist(df_swing['dem_share']) _ = plt.xlabel('percent of vote for Obama') _ = plt.ylabel('number of counties') plt.show() STATISTICAL THINKING IN PYTHON ( PART 1)

  18. A Seaborn - st y led histogram 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  19. Let ' s practice ! STATISTIC AL TH IN K IN G IN P YTH ON ( PAR T 1 )

  20. Plot all of y o u r data : Bee s w arm plots STATISTIC AL TH IN K IN G IN P YTH ON ( PAR T 1 ) J u stin Bois Lect u rer at the California Instit u te of Technolog y

  21. 2008 US s w ing state election res u lts 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  22. 2008 US s w ing state election res u lts 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  23. Binning bias The same data ma y be interpreted di � erentl y depending on choice of bins STATISTICAL THINKING IN PYTHON ( PART 1)

  24. Bee s w arm plot 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  25. Organi z ation of the data frame 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  26. Organi z ation of the data frame STATISTICAL THINKING IN PYTHON ( PART 1)

  27. Organi z ation of the data frame STATISTICAL THINKING IN PYTHON ( PART 1)

  28. Generating a bee s w arm plot _ = sns.swarmplot(x='state', y='dem_share', data=df_swing) _ = plt.xlabel('state') _ = plt.ylabel('percent of vote for Obama') plt.show() STATISTICAL THINKING IN PYTHON ( PART 1)

  29. 2008 US s w ing state election res u lts 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  30. Let ' s practice ! STATISTIC AL TH IN K IN G IN P YTH ON ( PAR T 1 )

  31. Plot all of y o u r data : ECDFs STATISTIC AL TH IN K IN G IN P YTH ON ( PAR T 1 ) J u stin Bois Lect u rer at the California Instit u te of Technolog y

  32. 2008 US s w ing state election res u lts 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  33. 2008 US election res u lts : East and West 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  34. Empirical c u m u lati v e distrib u tion f u nction ( ECDF ) 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  35. Empirical c u m u lati v e distrib u tion f u nction ( ECDF ) 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  36. Empirical c u m u lati v e distrib u tion f u nction ( ECDF ) 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  37. Making an ECDF import numpy as np x = np.sort(df_swing['dem_share']) y = np.arange(1, len(x)+1) / len(x) _ = plt.plot(x, y, marker='.', linestyle='none') _ = plt.xlabel('percent of vote for Obama') _ = plt.ylabel('ECDF') plt.margins(0.02) # Keeps data off plot edges plt.show() STATISTICAL THINKING IN PYTHON ( PART 1)

  38. 2008 US s w ing state election ECDF 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  39. 2008 US s w ing state election ECDFs 1 Data retrie v ed from Data . go v ( h � ps ://www. data . go v/) STATISTICAL THINKING IN PYTHON ( PART 1)

  40. Let ' s practice ! STATISTIC AL TH IN K IN G IN P YTH ON ( PAR T 1 )

  41. On w ard to w ard the w hole stor y! STATISTIC AL TH IN K IN G IN P YTH ON ( PAR T 1 ) J u stin Bois Lect u rer at the California Instit u te of Technolog y

  42. STATISTICAL THINKING IN PYTHON ( PART 1)

  43. “ E x plorator y data anal y sis can ne v er be the w hole stor y, b u t nothing else can ser v e as the fo u ndation stone .” — John T u ke y STATISTICAL THINKING IN PYTHON ( PART 1)

  44. Coming u p … Thinking probabilisticall y Discrete and contin u o u s distrib u tions The po w er of hacker statistics u sing np.random STATISTICAL THINKING IN PYTHON ( PART 1)

  45. Let ' s practice ! STATISTIC AL TH IN K IN G IN P YTH ON ( PAR T 1 )

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

E x plorator y data anal y sis P R E D IC TIN G C TR W ITH MAC H IN E L E AR N IN G IN P YTH

E x plorator y data anal y sis P R E D IC TIN G C TR W ITH MAC H IN E L E AR N IN G IN P YTH

E x plorator y data anal y sis P R E D IC TIN G C TR W ITH MAC H IN E L E AR N IN G IN P YTH ON Ke v in H u o Instr u ctor A closer look at feat u res int : an integer : 1 , 2 , etc . print(df.columns) float : decimals : 3.02 , 4.56 , etc .

189 views • 18 slides
Introd u ction : Working With Web Data in R W OR K IN G W ITH W E B DATA IN R Oli v er Ke y

Introd u ction : Working With Web Data in R W OR K IN G W ITH W E B DATA IN R Oli v er Ke y

Introd u ction : Working With Web Data in R W OR K IN G W ITH W E B DATA IN R Oli v er Ke y es & Charlo e Wickham Instr u ctors Working w ith Web Data in R Do w nloading les and u sing speciali z ed packages to get data from w eb

554 views • 16 slides
Introd u ction to signals FIN AN C IAL TR AD IN G IN R Il y a Kipnis Professional Q u antitati

Introd u ction to signals FIN AN C IAL TR AD IN G IN R Il y a Kipnis Professional Q u antitati

Introd u ction to signals FIN AN C IAL TR AD IN G IN R Il y a Kipnis Professional Q u antitati v e Anal y st and R programmer What are signals ? Signals are the interactions of : Market data w ith indicators Indicators w ith other indicators E

523 views • 24 slides
Introd u ction to Te x t Encoding FE ATU R E E N G IN E E R IN G FOR MAC H IN E L E AR N IN G

Introd u ction to Te x t Encoding FE ATU R E E N G IN E E R IN G FOR MAC H IN E L E AR N IN G

Introd u ction to Te x t Encoding FE ATU R E E N G IN E E R IN G FOR MAC H IN E L E AR N IN G IN P YTH ON Robert O ' Callaghan Director of Data Science , Ordergroo v e Standardi z ing y o u r te x t E x ample of free te x t : Fello w- Citi z

808 views • 46 slides
Introd u ction VISU AL IZIN G G E OSPATIAL DATA IN P YTH ON Mar y v an Valkenb u rg Data

Introd u ction VISU AL IZIN G G E OSPATIAL DATA IN P YTH ON Mar y v an Valkenb u rg Data

Introd u ction VISU AL IZIN G G E OSPATIAL DATA IN P YTH ON Mar y v an Valkenb u rg Data Science Program Manager , Nash v ille So w are School Location 1854 cholera o u tbreak in London 600+ deaths VISUALIZING GEOSPATIAL DATA IN PYTHON Sno

525 views • 26 slides
Introd u ction to iterators P YTH ON DATA SC IE N C E TOOL BOX ( PAR T 2 ) H u go Bo w ne -

Introd u ction to iterators P YTH ON DATA SC IE N C E TOOL BOX ( PAR T 2 ) H u go Bo w ne -

Introd u ction to iterators P YTH ON DATA SC IE N C E TOOL BOX ( PAR T 2 ) H u go Bo w ne - Anderson Data Scientist at DataCamp Iterating w ith a for loop We can iterate o v er a list u sing a for loop employees = ['Nick', 'Lore', 'Hugo'] for

379 views • 25 slides
Introd u ction to a u dio data in P y thon SP OK E N L AN G U AG E P R OC E SSIN G IN P YTH ON

Introd u ction to a u dio data in P y thon SP OK E N L AN G U AG E P R OC E SSIN G IN P YTH ON

Introd u ction to a u dio data in P y thon SP OK E N L AN G U AG E P R OC E SSIN G IN P YTH ON Daniel Bo u rke Machine Learning Engineer / Yo u T u be Creator Dealing w ith a u dio files in P y thon Di erent kinds all of a u dio les mp

463 views • 17 slides
Introd u ction to APIs and JSONs IN TE R ME D IATE IMP OR TIN G DATA IN P YTH ON H u go Bo w

Introd u ction to APIs and JSONs IN TE R ME D IATE IMP OR TIN G DATA IN P YTH ON H u go Bo w

Introd u ction to APIs and JSONs IN TE R ME D IATE IMP OR TIN G DATA IN P YTH ON H u go Bo w ne - Anderson Data Scientist at DataCamp APIs Application Programming Interface Protocols and ro u tines B u ilding and interacting w ith so w

426 views • 24 slides
Introd u ction to the dataset W OR K IN G W ITH G E OSPATIAL DATA IN P YTH ON Joris Van den

Introd u ction to the dataset W OR K IN G W ITH G E OSPATIAL DATA IN P YTH ON Joris Van den

Introd u ction to the dataset W OR K IN G W ITH G E OSPATIAL DATA IN P YTH ON Joris Van den Bossche Open so u rce so w are de v eloper and teacher , GeoPandas maintainer Artisanal mining site data from IPIS IPIS : International Peace

531 views • 38 slides
Introd u ction to s w imming data C ASE STU D IE S IN STATISTIC AL TH IN K IN G J u stin Bois

Introd u ction to s w imming data C ASE STU D IE S IN STATISTIC AL TH IN K IN G J u stin Bois

Introd u ction to s w imming data C ASE STU D IE S IN STATISTIC AL TH IN K IN G J u stin Bois Lect u rer , Caltech The 2015 FINA World Championships 1 Photo b y Chan - Fan , CC - BY - SA -4.0 CASE STUDIES IN STATISTICAL THINKING Strokes at

439 views • 27 slides
Introd u ction to statistical seismolog y C ASE STU D IE S IN STATISTIC AL TH IN K IN G J u

Introd u ction to statistical seismolog y C ASE STU D IE S IN STATISTIC AL TH IN K IN G J u

Introd u ction to statistical seismolog y C ASE STU D IE S IN STATISTIC AL TH IN K IN G J u stin Bois Lect u rer , Caltech California mo v es and shakes 1 Fa u lt data : USGS Q u aternar y Fa u lt and Fold Database of the United States CASE

970 views • 39 slides
Introd u ction to relational databases IN TR OD U C TION TO IMP OR TIN G DATA IN P YTH ON H u

Introd u ction to relational databases IN TR OD U C TION TO IMP OR TIN G DATA IN P YTH ON H u

Introd u ction to relational databases IN TR OD U C TION TO IMP OR TIN G DATA IN P YTH ON H u go Bo w ne - Anderson Data Scientist at DataCamp What is a relational database ? Based on relational model of data First described b y Edgar Ted

634 views • 50 slides
Introd u ction to databases STR E AML IN E D DATA IN G E STION W ITH PAN DAS Aman y Mahfo uz

Introd u ction to databases STR E AML IN E D DATA IN G E STION W ITH PAN DAS Aman y Mahfo uz

Introd u ction to databases STR E AML IN E D DATA IN G E STION W ITH PAN DAS Aman y Mahfo uz Instr u ctor Relational Databases Data abo u t entities is organi z ed into tables Each ro w or record is an instance of an entit y Each col u mn has

718 views • 43 slides
Introd u ction to spreadsheets STR E AML IN E D DATA IN G E STION W ITH PAN DAS Aman y Mahfo

Introd u ction to spreadsheets STR E AML IN E D DATA IN G E STION W ITH PAN DAS Aman y Mahfo

Introd u ction to spreadsheets STR E AML IN E D DATA IN G E STION W ITH PAN DAS Aman y Mahfo uz Instr u ctor Spreadsheets Also kno w n as E x cel les Data stored in tab u lar form , w ith cells arranged in ro w s and col u mns Unlike

974 views • 51 slides
Introd u ction IN TE R ME D IATE IN TE R AC TIVE DATA VISU AL IZATION W ITH P L OTLY IN R

Introd u ction IN TE R ME D IATE IN TE R AC TIVE DATA VISU AL IZATION W ITH P L OTLY IN R

Introd u ction IN TE R ME D IATE IN TE R AC TIVE DATA VISU AL IZATION W ITH P L OTLY IN R Adam Lo y Statistician , Carleton College Moti v ation Is it easier to see the changes o v er time based on the animation ? Or the faceted v ie w s ?

474 views • 31 slides
Introd u ction to Tid y Data W OR K IN G W ITH DATA IN TH E TIDYVE R SE Alison Hill

Introd u ction to Tid y Data W OR K IN G W ITH DATA IN TH E TIDYVE R SE Alison Hill

Introd u ction to Tid y Data W OR K IN G W ITH DATA IN TH E TIDYVE R SE Alison Hill Professor & Data Scientist WORKING WITH DATA IN THE TIDYVERSE WORKING WITH DATA IN THE TIDYVERSE The Great British Bake Off Series 8 WORKING WITH DATA

786 views • 60 slides
Regular Lossy Functions and Applications in Leakage-Resilient Cryptography CT-RSA 2018 April

Regular Lossy Functions and Applications in Leakage-Resilient Cryptography CT-RSA 2018 April

Regular Lossy Functions and Applications in Leakage-Resilient Cryptography CT-RSA 2018 April 20th, 2018 1 / 41 Yu Chen 1 Baodong Qin 2 Haiyang Xue 1 1 SKLOIS, IIE, Chinese Academy of Sciences 2 Xian University of Posts and Telecommunication

1.92k views • 115 slides
Leakage-Resilient Chosen-Ciphertext Secure Public-Key Encryption from Hash Proof System and

Leakage-Resilient Chosen-Ciphertext Secure Public-Key Encryption from Hash Proof System and

Leakage-Resilient Chosen-Ciphertext Secure Public-Key Encryption from Hash Proof System and One-Time Lossy Filter Baodong Qin and Shengli Liu Shanghai Jiao Tong University ASIACRYPT 2013 Dec 5, Bangalore, India B. Qin and S. Liu LR-CCA Secure

580 views • 47 slides
Reconstruction and identification of hadronic tau at 13 TeV Aniello Spiezia, Taozhe YU CLHCP

Reconstruction and identification of hadronic tau at 13 TeV Aniello Spiezia, Taozhe YU CLHCP

Reconstruction and identification of hadronic tau at 13 TeV Aniello Spiezia, Taozhe YU CLHCP 2018.12.20-22, WuHan Performance of reconstruction and identification of leptons decaying to hadrons and v in pp collisions at s=13

226 views • 10 slides
Sinking Fund Sinking Fund Millage In 2016 Voters Approved 5-Year Sinking Fund Millage of 1.6

Sinking Fund Sinking Fund Millage In 2016 Voters Approved 5-Year Sinking Fund Millage of 1.6

Hopkins Public Schools Sinking Fund Sinking Fund Millage In 2016 Voters Approved 5-Year Sinking Fund Millage of 1.6 Mills This generated about $450,000 per year High School Kiln Room High School Kiln Room Corridor High School

382 views • 22 slides
Uranium Medical Research Center Washington, New York, Toronto, London www.umrc.net The

Uranium Medical Research Center Washington, New York, Toronto, London www.umrc.net The

Uranium Medical Research Center Washington, New York, Toronto, London www.umrc.net The Quantitative Analysis of Uranium Isotopes in the Urine of Civilians after Operation Enduring Freedom in Jalalabad, Afghanistan A. Durakovic, R. Parrish,

427 views • 19 slides
Performance analysis with Periscope M. Gerndt, V. Petkov, Y. Oleynik, S. Benedict Technische

Performance analysis with Periscope M. Gerndt, V. Petkov, Y. Oleynik, S. Benedict Technische

Technische Universitt Mnchen Performance analysis with Periscope M. Gerndt, V. Petkov, Y. Oleynik, S. Benedict Technische Universitt Mnchen September 2010 Technische Universitt Mnchen Outline Motivation Periscope architecture

337 views • 22 slides
Interval-Based Memory Reclamation Haosen Wen , Joseph Izraelevitz, Wentao Cai, H. Alan Beadle and

Interval-Based Memory Reclamation Haosen Wen , Joseph Izraelevitz, Wentao Cai, H. Alan Beadle and

Interval-Based Memory Reclamation Haosen Wen , Joseph Izraelevitz, Wentao Cai, H. Alan Beadle and Michael L. Scott University of Rochester PPoPP18 Background Unlike lock-based concurrent data Thread 2 structures, non-blocking ones

355 views • 19 slides
Questions of Reality Gone Wrong: The Philosophical Importance of Force Composition Corey Dethier

Questions of Reality Gone Wrong: The Philosophical Importance of Force Composition Corey Dethier

Introduction The Principle of Composition A Role for HPS HPS and Realism Questions of Reality Gone Wrong: The Philosophical Importance of Force Composition Corey Dethier University of Notre Dame Philosophy Department corey.dethier@gmail.com

351 views • 17 slides

More recommend