inde x objects and labeled data
play

Inde x objects and labeled data MAN IP U L ATIN G DATAFR AME S W - PowerPoint PPT Presentation

Sep 20, 2023 •245 likes •525 views

Inde x objects and labeled data MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u ctor pandas data str u ct u res Ke y b u ilding blocks Index es : Seq u ence of labels Imm u table ( Like dictionar y ke y s ) Homogeneo u s in data t

  1. Inde x objects and labeled data MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u ctor

  2. pandas data str u ct u res Ke y b u ilding blocks Index es : Seq u ence of labels Imm u table ( Like dictionar y ke y s ) Homogeneo u s in data t y pe ( Like N u mP y arra y s ) Series : 1 D arra y w ith Inde x DataFrame s : 2 D arra y w ith Series as col u mns MANIPULATING DATAFRAMES WITH PANDAS

  3. Creating a Series import pandas as pd prices = [10.70, 10.86, 10.74, 10.71, 10.79] shares = pd.Series(prices) print(shares) 0 10.70 1 10.86 2 10.74 3 10.71 4 10.79 dtype: float64 MANIPULATING DATAFRAMES WITH PANDAS

  4. Creating an inde x days = ['Mon', 'Tue', 'Wed', 'Thur', 'Fri'] shares = pd.Series(prices, index=days) print(shares) Mon 10.70 Tue 10.86 Wed 10.74 Thur 10.71 Fri 10.79 dtype: float64 MANIPULATING DATAFRAMES WITH PANDAS

  5. E x amining an inde x print(shares.index) print(shares.index[-2:]) Index(['Mon', 'Tue', 'Wed', Index(['Thur', 'Fri'], 'Thur', 'Fri'], dtype='object') dtype='object') print(shares.index.name) print(shares.index[2]) None Wed print(shares.index[:2]) Index(['Mon', 'Tue'], dtype='object') MANIPULATING DATAFRAMES WITH PANDAS

  6. Modif y ing inde x name shares.index.name = 'weekday' print(shares) weekday Monday 10.70 Tuesday 10.86 Wednesday 10.74 Thursday 10.71 Friday 10.79 dtype: float64 MANIPULATING DATAFRAMES WITH PANDAS

  7. Modif y ing inde x entries shares.index[2] = 'Wednesday' TypeError: Index does not support mutable operations shares.index[:4] = ['Monday', 'Tuesday', 'Wednesday', 'Thursday'] TypeError: Index does not support mutable operations MANIPULATING DATAFRAMES WITH PANDAS

  8. Modif y ing all inde x entries shares.index = ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday'] print(shares) Monday 10.70 Tuesday 10.86 Wednesday 10.74 Thursday 10.71 Friday 10.79 dtype: float64 MANIPULATING DATAFRAMES WITH PANDAS

  9. Unemplo y ment data unemployment = pd.read_csv('Unemployment.csv') unemployment.head() Zip unemployment participants 0 1001 0.06 13801 1 1002 0.09 24551 2 1003 0.17 11477 3 1005 0.10 4086 4 1007 0.05 11362 MANIPULATING DATAFRAMES WITH PANDAS

  10. Unemplo y ment data unemployment.info() <class 'pandas.core.frame.DataFrame'> RangeIndex: 33120 entries, 0 to 33119 Data columns (total 3 columns): Zip 33120 non-null int64 unemployment 32556 non-null float64 particpants 33120 non-null int64 dtypes: float64(1), int64(2) memory usage: 776.3 KB MANIPULATING DATAFRAMES WITH PANDAS

  11. Assigning the inde x unemployment.index = unemployment['Zip'] unemployment.head() Zip unemployment participants Zip 1001 1001 0.06 13801 1002 1002 0.09 24551 1003 1003 0.17 11477 1005 1005 0.10 4086 1007 1007 0.05 11362 MANIPULATING DATAFRAMES WITH PANDAS

  12. Remo v ing e x tra col u mn unemployment.head(3) Zip unemployment participants Zip 1001 1001 0.06 13801 1002 1002 0.09 24551 1003 1003 0.17 11477 del unemployment['Zip'] unemployment.head(3) unemployment participants Zip 1001 0.06 13801 1002 0.09 24551 1003 0.17 11477 MANIPULATING DATAFRAMES WITH PANDAS

  13. E x amining inde x and col u mns print(unemployment.index) print(type(unemployment.index)) Int64Index([1001, 1002, 1003, ...], <class dtype='int64', 'pandas.indexes.numeric.Int64Index'> name='Zip', length=33120) print(unemployment.index.name) print(unemployment.columns) Zip Index(['unemployment', 'participants'], dtype='object') MANIPULATING DATAFRAMES WITH PANDAS

  14. read _ cs v() w ith inde x_ col unemployment = pd.read_csv('Unemployment.csv', index_col='Zip') unemployment.head() unemployment participants Zip 1001 0.06 13801 1002 0.09 24551 1003 0.17 11477 1005 0.10 4086 1007 0.05 11362 MANIPULATING DATAFRAMES WITH PANDAS

  15. Let ' s practice ! MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS

  16. Hierarchical Inde x ing MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u ctor

  17. Stock data import pandas as pd stocks = pd.read_csv('datasets/stocks.csv') print(stocks) Date Close Volume Symbol 0 2016-10-03 31.50 14070500 CSCO 1 2016-10-03 112.52 21701800 AAPL 2 2016-10-03 57.42 19189500 MSFT 3 2016-10-04 113.00 29736800 AAPL 4 2016-10-04 57.24 20085900 MSFT 5 2016-10-04 31.35 18460400 CSCO 6 2016-10-05 57.64 16726400 MSFT 7 2016-10-05 31.59 11808600 CSCO 8 2016-10-05 113.05 21453100 AAPL MANIPULATING DATAFRAMES WITH PANDAS

  18. Setting inde x stocks = stocks.set_index(['Symbol', 'Date']) print(stocks) Close Volume Symbol Date CSCO 2016-10-03 31.50 14070500 AAPL 2016-10-03 112.52 21701800 MSFT 2016-10-03 57.42 19189500 AAPL 2016-10-04 113.00 29736800 MSFT 2016-10-04 57.24 20085900 CSCO 2016-10-04 31.35 18460400 MSFT 2016-10-05 57.64 16726400 CSCO 2016-10-05 31.59 11808600 AAPL 2016-10-05 113.05 21453100 MANIPULATING DATAFRAMES WITH PANDAS

  19. print(stocks.index) MultiIndex(levels=[['AAPL', 'CSCO', 'MSFT'], ['2016-10-03', '2016-10-04', ‘2016-10-05']], labels=[[1, 0, 2, 0, 2, 1, 2, 1, 0], [0, 0, 0, 1, 1, 1, 2, 2, 2]], names=['Symbol', 'Date']) print(stocks.index.name) None print(stocks.index.names) ['Symbol', 'Date'] MANIPULATING DATAFRAMES WITH PANDAS

  20. Sorting inde x stocks = stocks.sort_index() print(stocks) Close Volume Symbol Date AAPL 2016-10-03 112.52 21701800 2016-10-04 113.00 29736800 2016-10-05 113.05 21453100 CSCO 2016-10-03 31.50 14070500 2016-10-04 31.35 18460400 2016-10-05 31.59 11808600 MSFT 2016-10-03 57.42 19189500 2016-10-04 57.24 20085900 2016-10-05 57.64 16726400 MANIPULATING DATAFRAMES WITH PANDAS

  21. Inde x ing ( indi v id u al ro w) stocks.loc[('CSCO', '2016-10-04')] Close 31.35 Volume 18460400.00 Name: (CSCO, 2016-10-04), dtype: float64 stocks.loc[('CSCO', '2016-10-04'), 'Volume'] 18460400.0 MANIPULATING DATAFRAMES WITH PANDAS

  22. Slicing ( o u termost inde x) stocks.loc['AAPL'] Close Volume Date 2016-10-03 112.52 21701800 2016-10-04 113.00 29736800 2016-10-05 113.05 21453100 MANIPULATING DATAFRAMES WITH PANDAS

  23. Slicing ( o u termost inde x) stocks.loc['CSCO':'MSFT'] Close Volume Symbol Date CSCO 2016-10-03 31.50 14070500 2016-10-04 31.35 18460400 2016-10-05 31.59 11808600 MSFT 2016-10-03 57.42 19189500 2016-10-04 57.24 20085900 2016-10-05 57.64 16726400 MANIPULATING DATAFRAMES WITH PANDAS

  24. Fanc y inde x ing ( o u termost inde x) stocks.loc[(['AAPL', 'MSFT'], '2016-10-05'), :] Close Volume Symbol Date AAPL 2016-10-05 113.05 21453100 MSFT 2016-10-05 57.64 16726400 stocks.loc[(['AAPL', 'MSFT'], '2016-10-05'), 'Close'] Symbol Date AAPL 2016-10-05 113.05 MSFT 2016-10-05 57.64 Name: Close, dtype: float64 MANIPULATING DATAFRAMES WITH PANDAS

  25. Fanc y inde x ing ( innermost inde x) stocks.loc[('CSCO', ['2016-10-05', '2016-10-03']), :] Close Volume Symbol Date CSCO 2016-10-03 31.50 14070500 2016-10-05 31.59 11808600 MANIPULATING DATAFRAMES WITH PANDAS

  26. Slicing ( both inde x es ) stocks.loc[(slice(None), slice('2016-10-03', '2016-10-04')),:] Close Volume Symbol Date AAPL 2016-10-03 112.52 21701800 2016-10-04 113.00 29736800 CSCO 2016-10-03 31.50 14070500 2016-10-04 31.35 18460400 MSFT 2016-10-03 57.42 19189500 2016-10-04 57.24 20085900 MANIPULATING DATAFRAMES WITH PANDAS

  27. Let ' s practice ! MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Learning to Rank with Learning to Rank with Partially-Labeled Data Partially-Labeled Data Kevin

Learning to Rank with Learning to Rank with Partially-Labeled Data Partially-Labeled Data Kevin

Learning to Rank with Learning to Rank with Partially-Labeled Data Partially-Labeled Data Kevin Duh University of Washington 1 The Ranking Problem The Ranking Problem Definition: Given a set of objects, sort them by preference. objectA

716 views • 59 slides
Meas u ring Segregation : The Inde x of Dissimilarit y AN ALYZIN G U S C E N SU S DATA IN P YTH

Meas u ring Segregation : The Inde x of Dissimilarit y AN ALYZIN G U S C E N SU S DATA IN P YTH

Meas u ring Segregation : The Inde x of Dissimilarit y AN ALYZIN G U S C E N SU S DATA IN P YTH ON Lee Hachadoorian Asst . Professor of Instr u ction , Temple Uni v ersit y What is Segregation ? ANALYZING US CENSUS DATA IN PYTHON Inde x of

496 views • 46 slides
Learning to Rank Learning to Rank with Partially-Labeled Data with Partially-Labeled Data Kevin

Learning to Rank Learning to Rank with Partially-Labeled Data with Partially-Labeled Data Kevin

Learning to Rank Learning to Rank with Partially-Labeled Data with Partially-Labeled Data Kevin Duh University of Washington (Joint work with Katrin Kirchhoff) 1 Motivation Motivation Machine learning can be an effective solution for

472 views • 29 slides
Objects (Demo1) Objects Objects represent information They consist of data and behavior,

Objects (Demo1) Objects Objects represent information They consist of data and behavior,

Objects (Demo1) Objects Objects represent information They consist of data and behavior, bundled together to create abstractions Objects can represent things, but also properties, interactions, &amp; processes A type of object is

591 views • 30 slides
61A Lecture 12 Announcements Objects (Demo) Objects 4 Objects Objects represent

61A Lecture 12 Announcements Objects (Demo) Objects 4 Objects Objects represent

61A Lecture 12 Announcements Objects (Demo) Objects 4 Objects Objects represent information 4 Objects Objects represent information They consist of data and behavior, bundled together to create abstractions 4 Objects

1.66k views • 132 slides
Learning from Limited Labeled Data (but a lot of unlabeled data) NELL as a case study Tom M.

Learning from Limited Labeled Data (but a lot of unlabeled data) NELL as a case study Tom M.

Learning from Limited Labeled Data (but a lot of unlabeled data) NELL as a case study Tom M. Mitchell Carnegie Mellon University Thesis: We will never really understand learning until we build machines that learn many different things,

582 views • 43 slides
Mutable Values Announcements Objects (Demo) Objects 4 Objects Objects represent

Mutable Values Announcements Objects (Demo) Objects 4 Objects Objects represent

Mutable Values Announcements Objects (Demo) Objects 4 Objects Objects represent information 4 Objects Objects represent information They consist of data and behavior, bundled together to create abstractions 4 Objects

2.23k views • 197 slides
Ne w Me xic o s K inde rg a rte n Ob se rva tio n T o o l (K OT ) E ARL Y L E ARNI

Ne w Me xic o s K inde rg a rte n Ob se rva tio n T o o l (K OT ) E ARL Y L E ARNI

Ne w Me xic o s K inde rg a rte n Ob se rva tio n T o o l (K OT ) E ARL Y L E ARNI NG ADVI SORY COUNCI L MARCH 26, 2015 Ra c e to the T o p E a rly L e a rning Cha lle ng e RT T -E L C g ra nt a wa rde d to

500 views • 25 slides
Snorkel + Data Programming: Beyond Hand-Labeled Training Data Alex Ratner Stanford University,

Snorkel + Data Programming: Beyond Hand-Labeled Training Data Alex Ratner Stanford University,

AAAI DeLBP Workshop 2/3 / 2018 Snorkel + Data Programming: Beyond Hand-Labeled Training Data Alex Ratner Stanford University, InfoLab MOTIVATION: In practice, training data is often: The bottleneck The practical injection point for

671 views • 38 slides
Ma Margi ginal l Inde depe pende dence and d Co Condi diti tion onal l Inde depe

Ma Margi ginal l Inde depe pende dence and d Co Condi diti tion onal l Inde depe

Ma Margi ginal l Inde depe pende dence and d Co Condi diti tion onal l Inde depe pende dence Com omputer Sc Science c cpsc sc322, L , Lecture 2 26 (Te Text xtboo ook k Chpt 6.1-2) 2) June 1 13, 2017 Lecture Overview

251 views • 24 slides
An Intr oduc tion to the F utur e Re ady PA Inde x What is the F utur e Re ady PA Inde

An Intr oduc tion to the F utur e Re ady PA Inde x What is the F utur e Re ady PA Inde

An Intr oduc tion to the F utur e Re ady PA Inde x What is the F utur e Re ady PA Inde x? T he F uture Re a dy PA I nde x is the ne w sc ho o l me a sure me nt to o l fo r c o mmunitie s to e va lua te ho w sc ho o ls a re do

354 views • 9 slides
X:Y XandYmatch X::Y XandYareidentical sult goal, for example: X \:: Y X and Y are not identical

X:Y XandYmatch X::Y XandYareidentical sult goal, for example: X \:: Y X and Y are not identical

2.1 Data objects 33 data objects ,/\ simple objects structures /\ of Prolog constants variables /\ atoms numbers Figure 2.1 Data objects in Prolog. Atoms and numbers 2.1.1 44 In chapter 1 we have seen some examples of atoms and variables.

365 views • 11 slides
Co-Training Based on Combining Labeled and Unlabeled Data with Co-Training by A. Blum

Co-Training Based on Combining Labeled and Unlabeled Data with Co-Training by A. Blum

0. Co-Training Based on Combining Labeled and Unlabeled Data with Co-Training by A. Blum &amp; T. Mitchell, 1998 1. Problem: Learning to classify data (ex: web pages) when the description of each example can be partitioned in 2

560 views • 9 slides
/\ This method for data structuring is simple and powerful. It is one of the reasens 4'. why

/\ This method for data structuring is simple and powerful. It is one of the reasens 4'. why

2.1 Data objects 33 data objects ,/\ simple objects structures /\ of Prolog constants variables /\ atoms numbers Figure 2.1 Data objects in Prolog. Atoms and numbers 2.1.1 44 In chapter 1 we have seen some examples of atoms and variables.

711 views • 49 slides
Getting To Know Your Data Road Map 1. Data Objects and Attribute Types 2. Descriptive Data

Getting To Know Your Data Road Map 1. Data Objects and Attribute Types 2. Descriptive Data

Getting To Know Your Data Road Map 1. Data Objects and Attribute Types 2. Descriptive Data Summarization 3. Measuring Data Similarity and Dissimilarity Data Objects and Attribute Types Types of data sets Data objects

767 views • 42 slides
61A Lecture 12 Announcements Objects (Demo) Objects Objects represent information They

61A Lecture 12 Announcements Objects (Demo) Objects Objects represent information They

61A Lecture 12 Announcements Objects (Demo) Objects Objects represent information They consist of data and behavior, bundled together to create abstractions Objects can represent things, but also properties, interactions, &amp;

521 views • 16 slides
13 Reasons Why Not: A Suicide Prevention Student Voice Project Pamela Fine, Riley Juntti, &amp;

13 Reasons Why Not: A Suicide Prevention Student Voice Project Pamela Fine, Riley Juntti, &amp;

13 Reasons Why Not: A Suicide Prevention Student Voice Project Pamela Fine, Riley Juntti, &amp; Andrea Wittenborn Suicide is the leading cause Someone dies by suicide every 40 seconds, and suicide attempts of death for are 20 times more

128 views • 10 slides
IRF Signature Study - Voice of the Market, Part 2: Engaging Program Owners in the Incentives

IRF Signature Study - Voice of the Market, Part 2: Engaging Program Owners in the Incentives

IRF Signature Study - Voice of the Market, Part 2: Engaging Program Owners in the Incentives Marketplace MELISSA VAN DYKE, PRESIDENT MEGG WITHINTON, CHIEF RESEARCH ADVISOR LEARNING CURVE LOW INDUSTRY AWARENESS MENTORS MAKE AN IMPACT

485 views • 12 slides
Voice &amp; &amp; Touch Con Controlled Che Checkl klists Fernando Salvetti - J.D., P.P.E.,

Voice &amp; &amp; Touch Con Controlled Che Checkl klists Fernando Salvetti - J.D., P.P.E.,

Voice &amp; &amp; Touch Con Controlled Che Checkl klists Fernando Salvetti - J.D., P.P.E., M.Phil., Ph.D. Centro Studi Logos, Turin, Italy Logosnet, Lugano, Switzerland and Houston, TX, USA Roxane Gardner - M.D., M.H.P.E., M.P.H., Ph.D.

635 views • 11 slides
ByMarcellusE.Parker MorehouseCollege DualDegreeProgram

ByMarcellusE.Parker MorehouseCollege DualDegreeProgram

ByMarcellusE.Parker MorehouseCollege DualDegreeProgram Advisor:JamesSantucci Outline APhotoinjector(Background) Functionofdipolemagnets Aboutdoglegsetup

446 views • 27 slides
NDN DeLorean: An Authentication System for Data Archives in Named Data Networking Yingdi Yu

NDN DeLorean: An Authentication System for Data Archives in Named Data Networking Yingdi Yu

NDN DeLorean: An Authentication System for Data Archives in Named Data Networking Yingdi Yu (UCLA), Alexander Afanasyev (Florida International University), Jan Seedorf (HFT Stuttgart), Zhiyi Zhang (UCLA), Lixia Zhang (UCLA), ACM Information

455 views • 18 slides
Haojun Liao Jizhong Han Jinyun Fang {liaohaojun, hjz, fangjy}@ict.ac.cn 2010-7-15

Haojun Liao Jizhong Han Jinyun Fang {liaohaojun, hjz, fangjy}@ict.ac.cn 2010-7-15

Haojun Liao Jizhong Han Jinyun Fang {liaohaojun, hjz, fangjy}@ict.ac.cn 2010-7-15 Introduction Design Overview Implementation Details Performance Evaluation Conclusion &amp; Future work Data Volume Data volumes

405 views • 29 slides
Chapter 7: Modifying Samples in a Range Knowing where we are in the sound More complex

Chapter 7: Modifying Samples in a Range Knowing where we are in the sound More complex

Chapter 7: Modifying Samples in a Range Knowing where we are in the sound More complex operations require us to know where we are in the sound, which sample Not just process all the samples exactly the same Examples: Reversing a

386 views • 25 slides
March 6, 2014 1 Mitigation of Covert Channels 2 About Voting and Computers March 6, 2014 ECS 235B

March 6, 2014 1 Mitigation of Covert Channels 2 About Voting and Computers March 6, 2014 ECS 235B

Mitigation of Covert Channels About Voting and Computers March 6, 2014 1 Mitigation of Covert Channels 2 About Voting and Computers March 6, 2014 ECS 235B Winter Quarter 2014 Slide 1 Mitigation of Covert Channels About Voting and Computers

416 views • 26 slides

More recommend