Joining data: a real- world necessity PAN DAS JOIN S F OR S P - - PowerPoint PPT Presentation

▶

Dec 10, 2023 367 likes •624 views

Joining data: a real- world necessity PAN DAS JOIN S F OR S P READS H EET US ERS John Miller Principal Data Scientist Pandas for spreadsheet users Learn based on similarities to spreadsheets Understand the power and exibility of pandas

SLIDE 1

Joining data: a real- world necessity

PAN DAS JOIN S F OR S P READS H EET US ERS

John Miller

Principal Data Scientist

SLIDE 2

PANDAS JOINS FOR SPREADSHEET USERS

Pandas for spreadsheet users

Learn based on similarities to spreadsheets Understand the power and exibility of pandas Use data from the National Football League (NFL)

SLIDE 3

PANDAS JOINS FOR SPREADSHEET USERS

Common situations

Datasets split by time or other factor Datasets with related factors

SLIDE 4

PANDAS JOINS FOR SPREADSHEET USERS

Split data

Inuenced by reporting cycle Common splits Time Geography Business unit

SLIDE 5

PANDAS JOINS FOR SPREADSHEET USERS

Split data example

SLIDE 6

PANDAS JOINS FOR SPREADSHEET USERS

Split data example

SLIDE 7

PANDAS JOINS FOR SPREADSHEET USERS

Split data example

SLIDE 8

PANDAS JOINS FOR SPREADSHEET USERS

Complementary data

Results from collecting data for different purposes Department-specic data Storage in separate les or database tables

SLIDE 9

PANDAS JOINS FOR SPREADSHEET USERS

Complementary data example

SLIDE 10

PANDAS JOINS FOR SPREADSHEET USERS

Complementary data example

SLIDE 11

PANDAS JOINS FOR SPREADSHEET USERS

Complementary data example

SLIDE 12

Let's practice!

PAN DAS JOIN S F OR S P READS H EET US ERS

SLIDE 13

Concatenation

PAN DAS JOIN S F OR S P READS H EET US ERS

John Miller

Principal Data Scientist

SLIDE 14

PANDAS JOINS FOR SPREADSHEET USERS

Concatenation basics

Similar to spreadsheet CONCATENATE Mimics copy-paste of cells

pd.concat() along rows or columns

SLIDE 15

PANDAS JOINS FOR SPREADSHEET USERS

Concatenating rows

Useful when working with split data

pd.concat([df1, df2, ...])

Uses unique key(s) as data frame index Includes all rows by default

SLIDE 16

PANDAS JOINS FOR SPREADSHEET USERS

Concatenating rows with overlapping indices

Data frame indices may overlap Don't worry!

pd.concat([df1, df2, ...], ignore_index=True)

SLIDE 17

PANDAS JOINS FOR SPREADSHEET USERS

Concatenating columns

Like pasting tables side by side Across columns: axis=1

pd.concat([df1, df2, ...], axis=1)

Includes all columns by default

SLIDE 18

Let's practice!

PAN DAS JOIN S F OR S P READS H EET US ERS

SLIDE 19

Power and exibility

PAN DAS JOIN S F OR S P READS H EET US ERS

John Miller

Principal Data Scientist

SLIDE 20

PANDAS JOINS FOR SPREADSHEET USERS

Scalability

No hard limits on data frame size Built-in ways to "chunk" data Use distributed/parallel computing

SLIDE 21

PANDAS JOINS FOR SPREADSHEET USERS

Efciency

Join on multiple columns Preference for simple code

joined_df = left_df.merge(right_df)

SLIDE 22

PANDAS JOINS FOR SPREADSHEET USERS

Integration

Improved speed and scale Data visualization Machine learning

SLIDE 23

PANDAS JOINS FOR SPREADSHEET USERS

A word on advanced spreadsheet usage

Data models and query tools Programming languages Advanced formulas

SLIDE 24

Let's practice!

PAN DAS JOIN S F OR S P READS H EET US ERS