merging dataframes
play

Merging DataFrames Merging DataFrames with pandas Population - PowerPoint PPT Presentation

Aug 24, 2022 •12 likes •412 views

MERGING DATAFRAMES WITH PANDAS Merging DataFrames Merging DataFrames with pandas Population DataFrame In [1]: import pandas as pd In [2]: population = pd.read_csv('pa_zipcode_population.csv') In [3]: print(population) Zipcode 2010 Census

  1. MERGING DATAFRAMES WITH PANDAS Merging DataFrames

  2. Merging DataFrames with pandas Population DataFrame In [1]: import pandas as pd In [2]: population = pd.read_csv('pa_zipcode_population.csv') In [3]: print(population) Zipcode 2010 Census Population 0 16855 282 1 15681 5241 2 18657 11985 3 17307 5899 4 15635 220

  3. Merging DataFrames with pandas Cities DataFrame In [4]: cities = pd.read_csv('pa_zipcode_city.csv') In [5]: print(cities) Zipcode City State 0 17545 MANHEIM PA 1 18455 PRESTON PARK PA 2 17307 BIGLERVILLE PA 3 15705 INDIANA PA 4 16833 CURWENSVILLE PA 5 16220 CROWN PA 6 18618 HARVEYS LAKE PA 7 16855 MINERAL SPRINGS PA 8 16623 CASSVILLE PA 9 15635 HANNASTOWN PA 10 15681 SALTSBURG PA 11 18657 TUNKHANNOCK PA 12 15279 PITTSBURGH PA 13 17231 LEMASTERS PA 14 18821 GREAT BEND PA

  4. Merging DataFrames with pandas Merging In [6]: pd.merge(population, cities) Out[6]: Zipcode 2010 Census Population City State 0 16855 282 MINERAL SPRINGS PA 1 15681 5241 SALTSBURG PA 2 18657 11985 TUNKHANNOCK PA 3 17307 5899 BIGLERVILLE PA 4 15635 220 HANNASTOWN PA

  5. Merging DataFrames with pandas Medal DataFrames In [7]: bronze = pd.read_csv('bronze_sorted.csv') In [8]: gold = pd.read_csv('gold_sorted.csv') In [9]: print(bronze) NOC Country Total 0 USA United States 1052.0 1 URS Soviet Union 584.0 2 GBR United Kingdom 505.0 3 FRA France 475.0 4 GER Germany 454.0 In [10]: print(gold) NOC Country Total 0 USA United States 2088.0 1 URS Soviet Union 838.0 2 GBR United Kingdom 498.0 3 ITA Italy 460.0 4 GER Germany 407.0

  6. Merging DataFrames with pandas Merging all columns In [11]: pd.merge(bronze, gold) Out[11]: Empty DataFrame Columns: [NOC, Country, Total] Index: []

  7. Merging DataFrames with pandas Merging on In [12]: pd.merge(bronze, gold, on='NOC') Out[12]: NOC Country_x Total_x Country_y Total_y 0 USA United States 1052.0 United States 2088.0 1 URS Soviet Union 584.0 Soviet Union 838.0 2 GBR United Kingdom 505.0 United Kingdom 498.0 3 GER Germany 454.0 Germany 407.0

  8. Merging DataFrames with pandas Merging on multiple columns In [13]: pd.merge(bronze, gold, on=['NOC', 'Country']) Out[13]: NOC Country Total_x Total_y 0 USA United States 1052.0 2088.0 1 URS Soviet Union 584.0 838.0 2 GBR United Kingdom 505.0 498.0 3 GER Germany 454.0 407.0

  9. Merging DataFrames with pandas Using su ffi xes In [14]: pd.merge(bronze, gold, on=['NOC', 'Country'], suffixes=['_bronze', '_gold']) Out[14]: NOC Country Total_bronze Total_gold 0 USA United States 1052.0 2088.0 1 URS Soviet Union 584.0 838.0 2 GBR United Kingdom 505.0 498.0 3 GER Germany 454.0 407.0

  10. Merging DataFrames with pandas Counties DataFrame In [15]: counties = pd.read_csv('pa_counties.csv') In [16]: print(counties) CITY NAME COUNTY NAME 0 SALTSBURG INDIANA 1 MINERAL SPRINGS CLEARFIELD 2 BIGLERVILLE ADAMS 3 HANNASTOWN WESTMORELAND 4 TUNKHANNOCK WYOMING In [17]: print(cities.tail()) Zipcode City State 10 15681 SALTSBURG PA 11 18657 TUNKHANNOCK PA 12 15279 PITTSBURGH PA 13 17231 LEMASTERS PA 14 18821 GREAT BEND PA

  11. Merging DataFrames with pandas Specifying columns to merge In [18]: pd.merge(counties, cities, left_on='CITY NAME', right_on='City') Out[18]: CITY NAME COUNTY NAME Zipcode City State 0 SALTSBURG INDIANA 15681 SALTSBURG PA 1 MINERAL SPRINGS CLEARFIELD 16855 MINERAL SPRINGS PA 2 BIGLERVILLE ADAMS 17307 BIGLERVILLE PA 3 HANNASTOWN WESTMORELAND 15635 HANNASTOWN PA 4 TUNKHANNOCK WYOMING 18657 TUNKHANNOCK PA

  12. Merging DataFrames with pandas Switching le � /right DataFrames In [19]: pd.merge(cities, counties, left_on='City', right_on='CITY NAME') Out[19]: Zipcode City State CITY NAME COUNTY NAME 0 17307 BIGLERVILLE PA BIGLERVILLE ADAMS 1 16855 MINERAL SPRINGS PA MINERAL SPRINGS CLEARFIELD 2 15635 HANNASTOWN PA HANNASTOWN WESTMORELAND 3 15681 SALTSBURG PA SALTSBURG INDIANA 4 18657 TUNKHANNOCK PA TUNKHANNOCK WYOMING

  13. MERGING DATAFRAMES WITH PANDAS Let’s practice!

  14. MERGING DATAFRAMES WITH PANDAS Joining DataFrames

  15. Merging DataFrames with pandas Medal DataFrames In [1]: import pandas as pd In [2]: bronze = pd.read_csv('bronze_sorted.csv') In [3]: gold = pd.read_csv('gold_sorted.csv') In [4]: print(bronze) NOC Country Total 0 USA United States 1052.0 1 URS Soviet Union 584.0 2 GBR United Kingdom 505.0 3 FRA France 475.0 4 GER Germany 454.0 In [5]: print(gold) NOC Country Total 0 USA United States 2088.0 1 URS Soviet Union 838.0 2 GBR United Kingdom 498.0 3 ITA Italy 460.0 4 GER Germany 407.0

  16. Merging DataFrames with pandas Merging with inner join In [6]: pd.merge(bronze, gold, on=['NOC', 'Country'], ...: suffixes=['_bronze', '_gold'], how='inner') Out[6]: NOC Country Total_bronze Total_gold 0 USA United States 1052.0 2088.0 1 URS Soviet Union 584.0 838.0 2 GBR United Kingdom 505.0 498.0 3 GER Germany 454.0 407.0

  17. Merging DataFrames with pandas Merging with le � join ● Keeps all rows of the le � DF in the merged DF ● For rows in the le � DF with matches in the right DF: ● Non-joining columns of right DF are appended to le � DF ● For rows in the le � DF with no matches in the right DF: ● Non-joining columns are filled with nulls

  18. Merging DataFrames with pandas Merging with le � join In [7]: pd.merge(bronze, gold, on=['NOC', 'Country'], ...: suffixes=['_bronze', '_gold'], how='left') Out[7]: NOC Country Total_bronze Total_gold 0 USA United States 1052.0 2088.0 1 URS Soviet Union 584.0 838.0 2 GBR United Kingdom 505.0 498.0 3 FRA France 475.0 NaN 4 GER Germany 454.0 407.0

  19. Merging DataFrames with pandas Merging with right join In [8]: pd.merge(bronze, gold, on=['NOC', 'Country'], ...: suffixes=['_bronze', '_gold'], how='right') Out[8]: NOC Country Total_bronze Total_gold 0 USA United States 1052.0 2088.0 1 URS Soviet Union 584.0 838.0 2 GBR United Kingdom 505.0 498.0 3 GER Germany 454.0 407.0 4 ITA Italy NaN 460.0

  20. Merging DataFrames with pandas Merging with outer join In [9]: pd.merge(bronze, gold, on=['NOC', 'Country'], ...: suffixes=['_bronze', '_gold'], how='outer') Out[9]: NOC Country Total_bronze Total_gold 0 USA United States 1052.0 2088.0 1 URS Soviet Union 584.0 838.0 2 GBR United Kingdom 505.0 498.0 3 FRA France 475.0 NaN 4 GER Germany 454.0 407.0 5 ITA Italy NaN 460.0

  21. Merging DataFrames with pandas Population & unemployment data In [10]: population = pd.read_csv('population_00.csv', index_col=0) In [11]: unemployment = pd.read_csv('unemployment_00.csv', index_col=0) In [12]: print(population) 2010 Census Population Zip Code ZCTA 57538 322 59916 130 37660 40038 2860 45199 In [13]: print(unemployment) unemployment participants Zip 2860 0.11 34447 46167 0.02 4800 1097 0.33 42 80808 0.07 4310

  22. Merging DataFrames with pandas Using .join(how=‘le � ’) In [16]: population.join(unemployment) Out[16]: 2010 Census Population unemployment participants Zip Code ZCTA 57538 322 NaN NaN 59916 130 NaN NaN 37660 40038 NaN NaN 2860 45199 0.11 34447.0

  23. Merging DataFrames with pandas Using .join(how=‘right’) In [17]: population.join(unemployment, how= 'right') Out[17]: 2010 Census Population unemployment participants Zip 2860 45199.0 0.11 34447 46167 NaN 0.02 4800 1097 NaN 0.33 42 80808 NaN 0.07 4310

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Appending & concatenating Series Merging DataFrames with pandas append() .append():

Appending & concatenating Series Merging DataFrames with pandas append() .append():

MERGING DATAFRAMES WITH PANDAS Appending & concatenating Series Merging DataFrames with pandas append() .append(): Series & DataFrame method Invocation: s1.append(s2) Stacks rows of s2 below s1 Method for

697 views • 44 slides
Review of pandas DataFrames PAN DAS F OUN DATION S Dhavide Aruliah Director of Training,

Review of pandas DataFrames PAN DAS F OUN DATION S Dhavide Aruliah Director of Training,

Review of pandas DataFrames PAN DAS F OUN DATION S Dhavide Aruliah Director of Training, Anaconda pandas DataFrames Example: DataFrame of Apple Stock data PANDAS FOUNDATIONS Indexes and columns import pandas as pd type(AAPL)

1.03k views • 56 slides
Parton Showers and Matching/Merging Lecture 2 of 2: Matching/Merging & Non-Perturbative

Parton Showers and Matching/Merging Lecture 2 of 2: Matching/Merging & Non-Perturbative

Parton Showers and Matching/Merging Lecture 2 of 2: Matching/Merging & Non-Perturbative Corrections (Hadron Decays) Hadronisation Parton Shower Matching & Merging Hard process Peter Skands (Monash University) Feynrules/Madgraph

698 views • 27 slides
Optimal Merging in Quantum k -xor and k -sum Algorithms Mara Naya-Plasencia, Andr

Optimal Merging in Quantum k -xor and k -sum Algorithms Mara Naya-Plasencia, Andr

Optimal Merging in Quantum k -xor and k -sum Algorithms Mara Naya-Plasencia, Andr Schrottenloher Inria, France Merging with many Solutions Quantum Merging With a Single Solution Outline Merging with many Solutions 1 Quantum Merging 2

727 views • 34 slides
Introduction to PySpark DataFrames BIG DATA F UN DAMEN TALS W ITH P YS PARK Upendra Devisetty

Introduction to PySpark DataFrames BIG DATA F UN DAMEN TALS W ITH P YS PARK Upendra Devisetty

Introduction to PySpark DataFrames BIG DATA F UN DAMEN TALS W ITH P YS PARK Upendra Devisetty Science Analyst, CyVerse What are PySpark DataFrames? PySpark SQL is a Spark library for structured data. It provides more information about the

1.01k views • 33 slides
Introducing DataFrames DATA MAN IP ULATION W ITH PAN DAS Richie Cotton Curriculum Architect

Introducing DataFrames DATA MAN IP ULATION W ITH PAN DAS Richie Cotton Curriculum Architect

Introducing DataFrames DATA MAN IP ULATION W ITH PAN DAS Richie Cotton Curriculum Architect at DataCamp What's the point of pandas? DATA MANIPULATION WITH PANDAS Course outline Chapter 1: DataFrames Chapter 3: Slicing and Indexing Data

659 views • 34 slides
Working w ith a DataSet to Create DataFrames W OR K IN G W ITH TH E C L ASS SYSTE M IN P YTH

Working w ith a DataSet to Create DataFrames W OR K IN G W ITH TH E C L ASS SYSTE M IN P YTH

Working w ith a DataSet to Create DataFrames W OR K IN G W ITH TH E C L ASS SYSTE M IN P YTH ON Vicki Bo y kis Senior Data Scientist MTCars A data frame with 32 observations on 11 (numeric) variables. [, 1] mpg Miles/(US) gallon [,

489 views • 22 slides
Track Filtering/Quality/Merging A proposal for data format of track quality and track merging in

Track Filtering/Quality/Merging A proposal for data format of track quality and track merging in

Track Filtering/Quality/Merging A proposal for data format of track quality and track merging in CMSSW K.Burkett, L.Lista, B.Mangano, S.Rahatlou, A.Rizzi, J.-R.Vlimant 2/4/08 Track Quality/Filtering/Merging Jean-Roch Vlimant 1 Requirements

432 views • 7 slides
Comparison Based Merging Upper and Lower bounds EMADS Fall 2003: Comparison Based Merging Page 1

Comparison Based Merging Upper and Lower bounds EMADS Fall 2003: Comparison Based Merging Page 1

Comparison Based Merging Upper and Lower bounds EMADS Fall 2003: Comparison Based Merging Page 1 Merging Input: Two sorted lists X and Y of length n and m . We may assume n m . n X : Y : m Theorem: In a comparison based model, the

435 views • 10 slides
Buying, Selling, Merging Buying, Selling, Merging and Valuation and Valuation Sponsored by: US

Buying, Selling, Merging Buying, Selling, Merging and Valuation and Valuation Sponsored by: US

Buying, Selling, Merging Buying, Selling, Merging and Valuation and Valuation Sponsored by: US Oncology Sponsored by: US Oncology Daryl Johnson Chris E. Rossman Principal, HealthCare Appraisers, Inc. Partner, Foley & Lardner LLP

718 views • 20 slides
Lecture 7 Rebasing Sign in on the attendance sheet! Today Review of merging Rebasing

Lecture 7 Rebasing Sign in on the attendance sheet! Today Review of merging Rebasing

Lecture 7 Rebasing Sign in on the attendance sheet! Today Review of merging Rebasing instead of merging Interactive rebases Review: Merging To combine two branches, with a merge, we do something like branchA, branchB HEAD

242 views • 22 slides
Inde x ing DataFrames MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u ctor A

Inde x ing DataFrames MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u ctor A

Inde x ing DataFrames MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u ctor A simple DataFrame import pandas as pd df = pd.read_csv('sales.csv', index_col='month') df eggs salt spam month Jan 47 12.0 17 Feb

669 views • 44 slides
Pi v oting DataFrames MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u ctor

Pi v oting DataFrames MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u ctor

Pi v oting DataFrames MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u ctor Clinical trials data import pandas as pd trials = pd.read_csv('trials_01.csv') print(trials) id treatment gender response 0 1 A F

669 views • 29 slides
Chapter 1 : Informatics Practices Advance operations Class XII ( As per on dataframes CBSE

Chapter 1 : Informatics Practices Advance operations Class XII ( As per on dataframes CBSE

Chapter 1 : Informatics Practices Advance operations Class XII ( As per on dataframes CBSE Board) (pivoting, sorting & aggregation/Descri- ptive statistics) New Syllabus 2019-20 Visit : python.mykvs.in for regular updates Pivoting

483 views • 20 slides
Quantum Merging Algorithms Mara Naya-Plasencia 2 , Andr Schrottenloher 2 Joint work with Andr

Quantum Merging Algorithms Mara Naya-Plasencia 2 , Andr Schrottenloher 2 Joint work with Andr

Quantum (Generalized) Collisions Quantum Merging Extended Quantum Merging Quantum Merging Algorithms Mara Naya-Plasencia 2 , Andr Schrottenloher 2 Joint work with Andr Chailloux 2 and Lorenzo Grassi 1 1 IAIK, Graz University of Technology,

727 views • 50 slides
Automatic Merging of Automatic Merging of Pedigree Information Pedigree Information Annual

Automatic Merging of Automatic Merging of Pedigree Information Pedigree Information Annual

Automatic Merging of Automatic Merging of Pedigree Information Pedigree Information Annual Workshop on Family History Technology April 3, 2003 Sue Dintelman and Tim Maness Pleiades Software Development, Inc. Pleiades Software Development,

340 views • 17 slides
1 Who Am I? Gabrielle Loring 2018-2019 IABC Awards Committee Chair 12-Time Gold Quill Award

1 Who Am I? Gabrielle Loring 2018-2019 IABC Awards Committee Chair 12-Time Gold Quill Award

1 Who Am I? Gabrielle Loring 2018-2019 IABC Awards Committee Chair 12-Time Gold Quill Award Winner Senior Consultant ROC Group 2 What Well Cover Today Divisions, categories and special awards Evaluation Key elements of a

641 views • 43 slides
Sec ectio ion I I: Health I th Insuranc ance 2020 Sec ectio ion I II: Pen ension &

Sec ectio ion I I: Health I th Insuranc ance 2020 Sec ectio ion I II: Pen ension &

Sec ectio ion I I: Health I th Insuranc ance 2020 Sec ectio ion I II: Pen ension & & Ret etiremen ement R Rev eview ew Sec ectio ion I III: Gen ener eral C l Confer eren ence & ce & Futur ure P e Pen

1.49k views • 76 slides
Physical gold outlook 2015: higher prices increasingly likely Intraday gold price per troy ounce:

Physical gold outlook 2015: higher prices increasingly likely Intraday gold price per troy ounce:

Physical gold outlook 2015: higher prices increasingly likely Intraday gold price per troy ounce: $1,183; date: May 7 th , 2015 Strategic topic: Scarce gold valuation amidst unprecedented global QE, extensive negative yields, rising solvency

497 views • 26 slides
Latent class analysis Daniel Oberski Dept of Methodology & Statistics Tilburg University,

Latent class analysis Daniel Oberski Dept of Methodology & Statistics Tilburg University,

Latent class analysis Daniel Oberski Dept of Methodology & Statistics Tilburg University, The Netherlands (with material from Margot Sijssens-Bennink & Jeroen Vermunt) About Tilburg University Methodology & Statistics About

892 views • 85 slides
2 nd Annual Strategic and Critical Minerals Summit Alaska Department of Natural Resources

2 nd Annual Strategic and Critical Minerals Summit Alaska Department of Natural Resources

2 nd Annual Strategic and Critical Minerals Summit Alaska Department of Natural Resources Commissioner Daniel S. Sullivan www.dnr.alaska.gov Photo by Colby Wright 2 nd Annual Strategic & Critical Minerals Summit Welcome to Alaskas

274 views • 11 slides
An algorithm selection approach for QF FP Solvers Joseph Scott, Pascal Poupart, Vijay Ganesh {

An algorithm selection approach for QF FP Solvers Joseph Scott, Pascal Poupart, Vijay Ganesh {

An algorithm selection approach for QF FP Solvers Joseph Scott, Pascal Poupart, Vijay Ganesh { joseph.scott,ppoupart,vganesh } @uwaterloo.ca University of Waterloo, Ontario, Canada July 11, 2019 Joseph Scott, Pascal Poupart, Vijay Ganesh An

765 views • 27 slides
Choueiry AIMA: Chapter 7 (Setions 7.1, 7.2, and 7.3) In tro dution to

Choueiry AIMA: Chapter 7 (Setions 7.1, 7.2, and 7.3) In tro dution to

B.Y. Title: Logial Agen ts Choueiry AIMA: Chapter 7 (Setions 7.1, 7.2, and 7.3) In tro dution to Artiial In telligene CSCE 476-876, Spring 2016 URL: www.se.unl.edu/~ ho uei ry/ S1 6-4 76- 87 6 1

707 views • 23 slides
Deep Neural Networks Machine Learning Organizational info All up-to-date info is on piazza.

Deep Neural Networks Machine Learning Organizational info All up-to-date info is on piazza.

10-701: Introduction to Deep Neural Networks Machine Learning Organizational info All up-to-date info is on piazza. Instructors - Ziv Bar-Joseph - Eric Xing TAs: See info on piazza for recitations, office hours etc. See also

1.12k views • 53 slides

More recommend