Digital Medicine I: Introduction to Programming Pandas Autumn 2019 - - PowerPoint PPT Presentation

▶
digital medicine i introduction to programming
SMART_READER_LITE
LIVE PREVIEW

Digital Medicine I: Introduction to Programming Pandas Autumn 2019 - - PowerPoint PPT Presentation

Sep 25, 2022 190 likes •651 views

Hans-Joachim Bckenhauer and Dennis Komm Digital Medicine I: Introduction to Programming Pandas Autumn 2019 December 19, 2019 So far. . . numpy and matplotlib The Modules numpy and matplotlib numpy Calculations with vectors and matrices

slide-1
SLIDE 1

Hans-Joachim Böckenhauer and Dennis Komm

Digital Medicine I: Introduction to Programming

Pandas

Autumn 2019 – December 19, 2019

slide-2
SLIDE 2

So far. . .

numpy and matplotlib

slide-3
SLIDE 3

The Modules numpy and matplotlib

numpy

Calculations with vectors and matrices Numerical methods Documentation: https://numpy.org/doc/

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 1 / 12

slide-4
SLIDE 4

The Modules numpy and matplotlib

numpy

Calculations with vectors and matrices Numerical methods Documentation: https://numpy.org/doc/

matplotlib

Data visualization (Plots) Documentation: https://matplotlib.org/contents.html

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 1 / 12

slide-5
SLIDE 5
  • Now. . .

pandas

slide-6
SLIDE 6

The Module pandas

pandas

Processing of large sets of data Allows a functionality similar to Excel Documentation: https://pandas.pydata.org/pandas-docs/stable/

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 2 / 12

slide-7
SLIDE 7

The Module pandas

pandas

Processing of large sets of data Allows a functionality similar to Excel Documentation: https://pandas.pydata.org/pandas-docs/stable/ Project 3: Reading in and processing CSV file “manually”

pandas contains data structures and functions for this

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 2 / 12

slide-8
SLIDE 8

The Module pandas

Import pandas analogously to numpy and matplotlib

import pandas as pd

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 3 / 12

slide-9
SLIDE 9

The Module pandas

Import pandas analogously to numpy and matplotlib

import pandas as pd

Read in CSV file and store it in a special data type pandas dataframe (instead

  • f Python list or numpy array)

data = pd.read_csv("daten.csv")

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 3 / 12

slide-10
SLIDE 10

The Module pandas

Import pandas analogously to numpy and matplotlib

import pandas as pd

Read in CSV file and store it in a special data type pandas dataframe (instead

  • f Python list or numpy array)

data = pd.read_csv("daten.csv")

Files in Excel format can be read in analogously

data = pd.read_excel("daten.xlsx")

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 3 / 12

slide-11
SLIDE 11

Air Measurements using pandas

slide-12
SLIDE 12

Exercise – Air Measurements

Air measurements Copy the data file from project 3

ugz_luftqualitaetsmessung_seit-2012.csv

Read in the CSV file and output its content To this end, use read_csv() and print()

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 4 / 12

slide-13
SLIDE 13

Air Measurements

import pandas as pd data = pd.read_csv("ugz_luftqualitaetsmessung_seit-2012.csv") print(data)

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 5 / 12

slide-14
SLIDE 14

Air Measurements

import pandas as pd data = pd.read_csv("ugz_luftqualitaetsmessung_seit-2012.csv") print(data)

Accessing individually cells using data.iloc Same functionality as lists

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 5 / 12

slide-15
SLIDE 15

Air Measurements

import pandas as pd data = pd.read_csv("ugz_luftqualitaetsmessung_seit-2012.csv") print(data)

Accessing individually cells using data.iloc Same functionality as lists

print(data.iloc[5]) print(data.iloc[0:10]) print(data.head(3)) print(data.iloc[8, 0])

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 5 / 12

slide-16
SLIDE 16

Air Measurements

import pandas as pd data = pd.read_csv("ugz_luftqualitaetsmessung_seit-2012.csv") print(data)

Accessing individually cells using data.iloc Same functionality as lists

print(data.iloc[5]) Output line 5 print(data.iloc[0:10]) print(data.head(3)) print(data.iloc[8, 0])

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 5 / 12

slide-17
SLIDE 17

Air Measurements

import pandas as pd data = pd.read_csv("ugz_luftqualitaetsmessung_seit-2012.csv") print(data)

Accessing individually cells using data.iloc Same functionality as lists

print(data.iloc[5]) print(data.iloc[0:10]) Output lines 0 to 9 print(data.head(3)) print(data.iloc[8, 0])

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 5 / 12

slide-18
SLIDE 18

Air Measurements

import pandas as pd data = pd.read_csv("ugz_luftqualitaetsmessung_seit-2012.csv") print(data)

Accessing individually cells using data.iloc Same functionality as lists

print(data.iloc[5]) print(data.iloc[0:10]) print(data.head(3)) Output lines 0 to 2 print(data.iloc[8, 0])

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 5 / 12

slide-19
SLIDE 19

Air Measurements

import pandas as pd data = pd.read_csv("ugz_luftqualitaetsmessung_seit-2012.csv") print(data)

Accessing individually cells using data.iloc Same functionality as lists

print(data.iloc[5]) print(data.iloc[0:10]) print(data.head(3)) print(data.iloc[8, 0]) Output line 8, column 0

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 5 / 12

slide-20
SLIDE 20

Reading in and Processing CSV Files

Extract data

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 6 / 12

slide-21
SLIDE 21

Reading in and Processing CSV Files

Extract data Numerical data starts from line 5 We are only interested in the first 3 columns We want to change column names

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 6 / 12

slide-22
SLIDE 22

Reading in and Processing CSV Files

Extract data Numerical data starts from line 5 We are only interested in the first 3 columns We want to change column names

import pandas as pd data = pd.read_csv("ugz_luftqualitaetsmessung_seit-2012.csv") newdata = data.iloc[5:, 0:3] newdata = newdata.rename(columns={"Zürich Stampfenbachstrasse": "SO2", \ "Zürich Stampfenbachstrasse.1": "CO"}) newdata.to_csv("messungen.csv")

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 6 / 12

slide-23
SLIDE 23

Reading in and Processing CSV Files

Extract data Numerical data starts from line 5 We are only interested in the first 3 columns We want to change column names

import pandas as pd data = pd.read_csv("ugz_luftqualitaetsmessung_seit-2012.csv") newdata = data.iloc[5:, 0:3] newdata = newdata.rename(columns={"Zürich Stampfenbachstrasse": "SO2", \ "Zürich Stampfenbachstrasse.1": "CO"}) newdata.to_csv("messungen.csv")

Selection from line 5 and columns 0 to 2

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 6 / 12

slide-24
SLIDE 24

Reading in and Processing CSV Files

Extract data Numerical data starts from line 5 We are only interested in the first 3 columns We want to change column names

import pandas as pd data = pd.read_csv("ugz_luftqualitaetsmessung_seit-2012.csv") newdata = data.iloc[5:, 0:3] newdata = newdata.rename(columns={"Zürich Stampfenbachstrasse": "SO2", \ "Zürich Stampfenbachstrasse.1": "CO"}) newdata.to_csv("messungen.csv")

Rename columns

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 6 / 12

slide-25
SLIDE 25

Reading in and Processing CSV Files

Accessing data using the column names

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 7 / 12

slide-26
SLIDE 26

Reading in and Processing CSV Files

Accessing data using the column names Output all column names as list

print(data.columns)

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 7 / 12

slide-27
SLIDE 27

Reading in and Processing CSV Files

Accessing data using the column names Output all column names as list

print(data.columns)

Output column “Datum”

print(data["Datum"])

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 7 / 12

slide-28
SLIDE 28

Reading in and Processing CSV Files

Accessing data using the column names Output all column names as list

print(data.columns)

Output column “Datum”

print(data["Datum"])

Output column “Zürich Stampfenbachstrasse – Kohlenmonoxid”

print(data["Zürich Stampfenbachstrasse.1"])

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 7 / 12

slide-29
SLIDE 29

Reading in and Processing CSV Files

Filtering data

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 8 / 12

slide-30
SLIDE 30

Reading in and Processing CSV Files

Filtering data Use loc instead of iloc in order to specify conditions

print(data.loc[data["Datum"] == "2014-12-19"])

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 8 / 12

slide-31
SLIDE 31

Reading in and Processing CSV Files

Filtering data Use loc instead of iloc in order to specify conditions

print(data.loc[data["Datum"] == "2014-12-19"])

Combination of different Boolean expressions

Parentheses around single expressions

& instead of and | instead of or ~ instead of not

print(data.loc[(data["Datum"] == "2014-12-19") \ | (data["Datum"] == "2014-12-20")])

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 8 / 12

slide-32
SLIDE 32

Reading in and Processing CSV Files

Filtering data

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 9 / 12

slide-33
SLIDE 33

Reading in and Processing CSV Files

Filtering data Convert strings to rational numbers (float)

newdata["SO2"] = newdata["SO2"].astype(float) newdata["CO"] = newdata["CO"].astype(float)

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 9 / 12

slide-34
SLIDE 34

Reading in and Processing CSV Files

Filtering data Convert strings to rational numbers (float)

newdata["SO2"] = newdata["SO2"].astype(float) newdata["CO"] = newdata["CO"].astype(float)

Use relation operators to filter

print(newdata.loc[newdata["SO2"] > 0.1])

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 9 / 12

slide-35
SLIDE 35

Reading in and Processing CSV Files

Filtering data Convert strings to rational numbers (float)

newdata["SO2"] = newdata["SO2"].astype(float) newdata["CO"] = newdata["CO"].astype(float)

Use relation operators to filter

print(newdata.loc[newdata["SO2"] > 0.1])

Combine different Boolean expressions

print(newdata.loc[(newdata["SO2"] > 0.1) & (newdata["SO2"] < 0.4)])

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 9 / 12

slide-36
SLIDE 36

Reading in and Processing CSV Files

Filtering data Convert strings to rational numbers (float)

newdata["SO2"] = newdata["SO2"].astype(float) newdata["CO"] = newdata["CO"].astype(float)

Use relation operators to filter

print(newdata.loc[newdata["SO2"] > 0.1])

Combine different Boolean expressions

print(newdata.loc[(newdata["SO2"] > 0.1) & (newdata["SO2"] < 0.4)])

Choose columns with second argument

print(newdata.loc[newdata["SO2"] > 0.2, "Datum"])

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 9 / 12

slide-37
SLIDE 37

Exercise – Air Measurements

Air measurements Extract all CO entries from newdata for which the SO2 value is smaller than 0.1 or larger than 0.25 Convert the CO entries into a Python list using list() Plot the values using matplotlib

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 10 / 12

slide-38
SLIDE 38

Reading in CSV File

import pandas as pd import matplotlib.pyplot as plt data = pd.read_csv("ugz_luftqualitaetsmessung_seit-2012.csv") newdata = data.iloc[5:, 0:3] newdata = newdata.rename(columns={"Zürich Stampfenbachstrasse": "SO2", \ "Zürich Stampfenbachstrasse.1": "CO"}) newdata["SO2"] = newdata["SO2"].astype(float) newdata["CO"] = newdata["CO"].astype(float) newdata = newdata.loc[(newdata["SO2"] < 0.1) | (newdata["SO2"] > 0.25), "CO"] datalist = list(newdata) plt.plot(datalist) plt.show()

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 11 / 12

slide-39
SLIDE 39

Pandas

Further Functionality

slide-40
SLIDE 40

Further Functionality

Delete columns

del data["Column"]

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 12 / 12

slide-41
SLIDE 41

Further Functionality

Delete columns

del data["Column"]

Add columns

data["Sum"] = data["Column 1"] + data["Column 2"]

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 12 / 12

slide-42
SLIDE 42

Further Functionality

Delete columns

del data["Column"]

Add columns

data["Sum"] = data["Column 1"] + data["Column 2"]

Sort data

data = data.sort_values("Column")

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 12 / 12

slide-43
SLIDE 43

Further Functionality

Delete columns

del data["Column"]

Add columns

data["Sum"] = data["Column 1"] + data["Column 2"]

Sort data

data = data.sort_values("Column")

. . .

Digital Medicine I: Introduction to Programming – Pandas Autumn 2019 Böckenhauer, Komm 12 / 12

slide-44
SLIDE 44

Thanks for your attention