introduction to the course
play

Introduction to the course James Lamb Instructor DataCamp Time - PowerPoint PPT Presentation

Sep 06, 2023 •321 likes •590 views

DataCamp Time Series with data.table in R TIME SERIES WITH DATA . TABLE IN R Introduction to the course James Lamb Instructor DataCamp Time Series with data.table in R A data frame is a general-purpose data structure A data frame is not

  1. DataCamp Time Series with data.table in R TIME SERIES WITH DATA . TABLE IN R Introduction to the course James Lamb Instructor

  2. DataCamp Time Series with data.table in R A data frame is a general-purpose data structure A data frame is not something unique to R! It's a common data structure that meets these properties: List of lists All lists are of equal length Value type must be the same within each list (column) Value types can be different across columns someDF <- data.frame(x = rnorm(10), y = rep(TRUE, 100)) str(someDF) 'data.frame': 100 obs. of 2 variables: $ x: num -1.5456 -1.1905 0.6055 0.9489 0.0023 ... $ y: logi TRUE TRUE TRUE TRUE TRUE TRUE ...

  3. DataCamp Time Series with data.table in R data.table is an extension on data.frame data.frame = R's default data frame implementation data.table = extension of that base class data.table improvements: more expressive syntax more efficient memory use via pass-by-reference operators library(data.table) someDT <- data.table(x = rnorm(100), y = rep(TRUE, 100)) str(someDT) Classes ‘data.table’ and 'data.frame': 100 obs. of 2 variables: $ x: num -0.474 -0.944 0.382 -0.505 -1.128 ... $ y: logi TRUE TRUE TRUE TRUE TRUE TRUE ...

  4. DataCamp Time Series with data.table in R Selecting columns with .() You can select columns from a data.table with .() : baseballDT[, .(timestamp, winning_team)] timestamp winning_team 1: 2018-01-01 00:00:00 BOS 2: 2018-01-01 00:00:36 CWS 3: 2018-01-01 00:01:12 MIL

  5. DataCamp Time Series with data.table in R Column selection with .SD Use .SD ( S ubset of D ata) to reference a subset of columns. cols <- c("timestamp", "winning_team") baseballDT[, .SD, .SDcols = cols] This is identical: baseballDT[, .SD, .SDcols = c("timestamp", "winning_team")] "new data.table with specific columns" timestamp winning_team 1: 2018-01-01 00:00:00 BOS 2: 2018-01-01 00:00:36 CWS 3: 2018-01-01 00:01:12 MIL

  6. DataCamp Time Series with data.table in R Brief review of grep() grep() returns indexes of strings matching a pattern. grep(pattern = 'art', c('artistic', 'colorful')) [1] 1 Use value = TRUE to get values instead of indexes. grep(pattern = 'art', c('artistic', 'colorful'), value = TRUE) [1] "artistic" `

  7. DataCamp Time Series with data.table in R Using column suffixes and grep() Use column suffixes to group columns. innings_pitched_COUNT runs_allowed_COUNT era_AVERAGE 1: 10 8 7.2 2: 20 4 1.8 3: 30 22 6.6 Get just the count data count_cols <- grep('COUNT$', names(baseballDT), value = TRUE) countDT <- baseballDT[, .SD, .SDcols = count_cols] countDT innings_pitched_COUNT runs_allowed_COUNT 1: 10 8 2: 20 4 3: 30 22

  8. DataCamp Time Series with data.table in R Combining row and column selection Expressive subset statements with row selectors cols <- c("timestamp", "winning_team") baseballDT[ which.max(timestamp), .SD, .SDcols = cols ] "Get the most recent observation" timestamp winning_team 1: 2018-01-01 01:00:00 BOS

  9. DataCamp Time Series with data.table in R TIME SERIES WITH DATA . TABLE IN R Let's practice!

  10. DataCamp Time Series with data.table in R TIME SERIES WITH DATA . TABLE IN R Flexible data selection James Lamb Instructor

  11. DataCamp Time Series with data.table in R Explicit references Use direct name references in [] locDT <- data.table( cities = c("Chicago", "Boston", "Milwaukee"), ppl_mil = c(2.7, 0.673, 0.595) ) locDT[, cities] [1] "Chicago" "Boston" "Milwaukee"

  12. DataCamp Time Series with data.table in R Calling functions Functions in the i block to select rows locDT[which.max(ppl_mil)] cities ppl_mil 1: Chicago 2.7

  13. DataCamp Time Series with data.table in R Using get() get() : evaluate a string as a column reference locDT <- data.table( cities = c("Chicago", "Boston", "Milwaukee"), ppl_mil = c(2.7, 0.673, 0.595) ) city_col <- "cities" locDT[, get(city_col)] [1] "Chicago" "Boston" "Milwaukee"

  14. DataCamp Time Series with data.table in R get() is great when writing functions Write reusable functions without hard-coded column names: square_col <- function(DT, col_name){ return(DT[, get(col_name) ^ 2]) } square_col(locDT, "ppl_mil") [1] 7.290000 0.452929 0.354025

  15. DataCamp Time Series with data.table in R Using () Problem: get people in thousands from the ppl_mil column. locDT[, ppl_bil := ppl_mil * 1000] locDT[, ppl_bil] [1] 2700 673 595 But what if you want to parameterize the new column name? add_bil_ppl <- function(DT, new_name){ DT[, (new_name) := ppl_mil * 1000 } add_bil_ppl(locDT, "some_rand_name") print(locDT) cities ppl_mil some_rand_name 1: Chicago 2.700 2700 2: Boston 0.673 673 3: Milwaukee 0.595 595

  16. DataCamp Time Series with data.table in R Combining () and get() Function to create features by adding 10 to existing columns add10 <- function(DT, cols){ for (col in cols){ new_name <- paste0(col, "_plus10") DT[, (new_name) := get(col) + 10] } } add10(locDT, cols = "ppl_mil") locDT cities ppl_mil ppl_mil_plus10 1: Chicago 2.700 12.700 2: Boston 0.673 10.673 3: Milwaukee 0.595 10.595

  17. DataCamp Time Series with data.table in R Changing names with setnames() Change a single column's name: locDT <- data.table( cities = c("Chicago", "Boston", "Milwaukee"), ppl_mil = c(2.7, 0.673, 0.595) ) setnames(locDT, old = "cities", new = "city_names") names(locDT) [1] "city_names" "ppl_mil"

  18. DataCamp Time Series with data.table in R setnames() in functions Using setnames() in a function tag_important_columns <- function(DT, cols){ setnames(DT, old = cols, new = paste0(cols, "_important")) } Calling this function is efficient and doesn't copy the data! tag_important_columns(locDT, "ppl_mil") locDT cities ppl_mil_important 1: Chicago 2.700 2: Boston 0.673 3: Milwaukee 0.595

  19. DataCamp Time Series with data.table in R TIME SERIES WITH DATA . TABLE IN R Let's practice!

  20. DataCamp Time Series with data.table in R TIME SERIES WITH DATA . TABLE IN R Executing functions inside data.tables James Lamb Instructor

  21. DataCamp Time Series with data.table in R Use functions in the "i" block to select rows stockDT <- data.table( close_date = seq.POSIXt(as.POSIXct("2017-01-01"), as.POSIXct("2017-01-30"), MSFT = runif(100, 70, 80), AAPL = runif(100, 140, 180) ) Best day for Microsoft: stockDT[which.max(MSFT)] close_date MSFT AAPL 1: 2017-01-08 07:45:27 79.9235 159.9928 Final 8 hours of the dataset: stockDT[close_date > max(close_date) - 60 * 60 * 8] close_date MSFT AAPL 1: 2017-01-29 16:58:10 73.78340 157.9154 2: 2017-01-30 00:00:00 71.51727 141.8897

  22. DataCamp Time Series with data.table in R Using functions in the "j" block to summarize data cor() creates a correlation matrix between columns cor(stockDT[, .SD, .SDcols = c('AAPL', 'MSFT')]) AAPL MSFT AAPL 1.00000000 0.05680504 MSFT 0.05680504 1.00000000 You can call this directly inside a data.table ! corr_mat <- stockDT[, cor(.SD), .SDcols = c('AAPL', 'MSFT')] print(corr_mat) AAPL MSFT AAPL 1.00000000 0.05680504 MSFT 0.05680504 1.00000000

  23. DataCamp Time Series with data.table in R Use functions in the "j" block to generate new columns Add a new column: stockDT[, rand_noise := AAPL + rnorm(100)] close_date MSFT AAPL rand_noise 1: 2017-01-01 00:00:00 76.46907 163.6131 162.4594 2: 2017-01-01 07:01:49 78.68001 174.1177 174.9193

  24. DataCamp Time Series with data.table in R Using functions in the "by" block to dynamically group data Two-step process to generate "mean price by hour of the day": stockDT[, hour_of_day := as.integer(strftime(close_date, "%H"))] stockDT[, mean(AAPL), by = hour_of_day][order(hour_of_day)] hour_of_day V1 1: 0 155.4853 2: 1 163.5479 3: 2 152.5203 1-step process to generate "mean price by hour of day": stockDT[, mean(AAPL), by = .( hour_of_day = as.integer(strftime(close_date, "%H")) )][order(hour_of_day)] hour_of_day V1 1: 0 155.4853 2: 1 163.5479 3: 2 152.5203

  25. DataCamp Time Series with data.table in R Applying a function over every column with .SD Use lapply() if you want a data.table back Use sapply() if you want a vector or list back Count percent missing values by column: stockDT[, lapply(.SD, function(x){mean(is.na(x))})] close_date MSFT AAPL 1: 0 0.1 0.26 Count non-NA values: num_obs <- stockDT[, sapply(.SD, function(x){sum(!is.na(x), na.rm = TRUE)})] print(num_obs) close_date MSFT AAPL 100 90 74

  26. DataCamp Time Series with data.table in R TIME SERIES WITH DATA . TABLE IN R Let's practice!

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION

INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION

TRAINING COURSE ON SOCIAL PROTECTION &amp; FORMALIZATION TRINIDAD AND TOBAGO MARCH 15, 2017 INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION Design of the NIS Assistance from the

547 views • 23 slides
INTRODUCTION Introduction 2/42 INTRODUCTION Alternations I am giving bread to the

INTRODUCTION Introduction 2/42 INTRODUCTION Alternations I am giving bread to the

Patterns of variation in the expression of case and agreement Andrs Brny Leiden University Centre for Linguistics 9 November 2019, BaSIS Workshop a.barany@hum.leidenuniv.nl INTRODUCTION Introduction 2/42 INTRODUCTION

1.37k views • 42 slides
Introduction Introduction Introduction Introduction Outline Motivation Failures

Introduction Introduction Introduction Introduction Outline Motivation Failures

Introduction Introduction Introduction Introduction Outline Motivation Failures Definition and concepts Process and product properties Principles SE Approaches Motivation Software and the economy The economies of

1.03k views • 49 slides
Introduction Introduction Introduction Nationwide Cause for Concern 1

Introduction Introduction Introduction Nationwide Cause for Concern 1

Introduction Introduction Introduction Nationwide Cause for Concern 1 https://www.cdc.gov/std/stats17/infographic.htm Introduction Epidemiology in Western Colorado CPHE Surveillance Data Report, May 2018 Chlamydia Epidemiology in Western

977 views • 39 slides
DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every

DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every

DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every meeting (2) Some ideas of diagrams for section 7.1 (Introduction) of TP. Giles Barr 25 Jan 2018, Pembroke College On DUNE, the reality is there is a

474 views • 10 slides
INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT

INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT

ADVANCED COMPUTER SECURITY INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT IS SECURITY? A systems security policies describe What the system is supposed to do Store and provide access

479 views • 16 slides
Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL

Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL

Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL Writing a SMIL file Adding Media Objects Martin Halvey Clipping media files January 2008 Timing and Synchronising Layout World Wide

387 views • 14 slides
14 Introduction Introduction Bad guys can put malware into Example: hosts

14 Introduction Introduction Bad guys can put malware into Example: hosts

Introduction Introduction source Encapsulation ISO/OSI reference model message M application segment H t H t M transport network datagram H n H n H t M presentation: allow applications to frame link H l H n H t M interpret

583 views • 4 slides
Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL

Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL

Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL Writing a SMIL file Adding Media Objects Martin Halvey Clipping media files January 2008 Timing and Synchronising Layout

322 views • 8 slides
Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL

Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL

Introduction Zynq Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL Processing System Processor Peripherals Data Buses AXI Bus Conclusion BTE5380 - Embedded Systems Octobre 2014 Andreas Habegger

1.81k views • 30 slides
2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04

2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04

2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04 Project References S E 5 PART.1 SMILE5 Introduction M I L Introduction STORION-SMILE5 Residential Energy Storage System Inverter 5kW Output Input

1.06k views • 25 slides
Introduction to CICS Course introduction Course introduction What is CICS? What is an

Introduction to CICS Course introduction Course introduction What is CICS? What is an

Introduction to CICS Course introduction Course introduction What is CICS? What is an application server? Why use an application server? Course introduction Services provided by CICS How CICS applications are defined Scaling CICS to meet

1.84k views • 146 slides
Network Visualization Introduction Presented by Shahed Introduction Introduction Basic

Network Visualization Introduction Presented by Shahed Introduction Introduction Basic

Network Visualization Introduction Presented by Shahed Introduction Introduction Basic building blocks Node Links (relationship between nodes) Spatial information Network data

584 views • 9 slides
Introduction Introduction (1) Main argument of the paper: universities are not only

Introduction Introduction (1) Main argument of the paper: universities are not only

Le jugement des pairs comme outil de management des universits Peer-review as a management tool Christine Musselin (CSO, Sciences Po et CNRS) Avril 2013, Lirrsistible ascension du capitalisme acadmique Introduction Introduction (1)

590 views • 28 slides
Introduction Introduction to storage and to storage and filesystems filesystems Introduction

Introduction Introduction to storage and to storage and filesystems filesystems Introduction

Moreno Baricevic Gilberto Daz Axel Kohlmeyer Stefano Cozzini ULA ICTP CNR-IOM DEMOCRITOS Merida, VENEZUELA Trieste, ITALY Trieste, ITALY Introduction Introduction to storage and to storage and filesystems filesystems Introduction

1.19k views • 51 slides
JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of

JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of

JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of the brand new JA-100 alarm system Visions Basic architecture System components Setting Help us with final validations You are the FIRST ones

1.69k views • 111 slides
Nonadiaba(c cavity QED effects with superconduc(ng qubit-resonator

Nonadiaba(c cavity QED effects with superconduc(ng qubit-resonator

Workshop on the Theory &amp; Prac2ce of Adiaba2c Quantum Computers and Quantum Simula2on, Trieste, Italy, 2016 Nonadiaba(c cavity QED effects with

579 views • 31 slides
1 ChronologyofMartyrdom Jesus death 30 C.E. Occasional, localized Revelation 95-110

1 ChronologyofMartyrdom Jesus death 30 C.E. Occasional, localized Revelation 95-110

Class7b Outline ChronologyofMartyrdom TheSpectacleofMartyrdom Typesofspectacle Archaeologicalevidence ThePiercedLambofRevelation

339 views • 10 slides
DEEP LEARNING WITH COTS HPC SYSTEMS Adam Coates, Brody Huval, Tao Wang, David Wu, Bryan

DEEP LEARNING WITH COTS HPC SYSTEMS Adam Coates, Brody Huval, Tao Wang, David Wu, Bryan

DEEP LEARNING WITH COTS HPC SYSTEMS Adam Coates, Brody Huval, Tao Wang, David Wu, Bryan Catanzaro, Ng Andrew ; Proceedings of the 30th International Conference on Machine Learning, PMLR 28(3):1337-1345, 2013. MODELS NOW Larger and larger

732 views • 46 slides
Speaker: Pastor Gilbert van Bueren REVELATION 5 Series The Future has already Begun #5 IBC

Speaker: Pastor Gilbert van Bueren REVELATION 5 Series The Future has already Begun #5 IBC

Series The future has already begun #5 Revelation 5: The Lion of Judah is the Lamb slain Revelation 5 Speaker: Pastor Gilbert van Bueren REVELATION 5 Series The Future has already Begun #5 IBC Eindhoven, 26 January 2020 THRONE THE

472 views • 6 slides
Ttulo do captulo Luis Lamb 8 May 2017 Dagstuhl, DE Summary Dies ist im Wesentlichen die

Ttulo do captulo Luis Lamb 8 May 2017 Dagstuhl, DE Summary Dies ist im Wesentlichen die

From Turing to deep learning: explaining AI through neurons and symbols Ttulo do captulo Luis Lamb 8 May 2017 Dagstuhl, DE Summary Dies ist im Wesentlichen die gleiche Vorlesung, die ich auf der City University in London im

703 views • 68 slides
27 September 2020 Access service sheets at thecrossing.com.sg/im-new/sunday-services/ Family

27 September 2020 Access service sheets at thecrossing.com.sg/im-new/sunday-services/ Family

27 September 2020 Access service sheets at thecrossing.com.sg/im-new/sunday-services/ Family Declaration 4 But Gods mercy is great, and he loved us very much. 5 We were spiritually dead because of the things we did wrong against God. But

978 views • 55 slides
The Lord is not slow in keeping his promise, as some understand slowness. Instead he is patient

The Lord is not slow in keeping his promise, as some understand slowness. Instead he is patient

The Lord is not slow in keeping his promise, as some understand slowness. Instead he is patient with you, not wanting anyone to perish, but everyone to come to repentance. 2 Peter 3:9 After this I looked, and there

304 views • 7 slides
Requirements Analysis - Ambiguity R. Kuehl/J. Scott Hawker p. 1 R I T Lecture 4-1 Software

Requirements Analysis - Ambiguity R. Kuehl/J. Scott Hawker p. 1 R I T Lecture 4-1 Software

Requirements Analysis - Ambiguity R. Kuehl/J. Scott Hawker p. 1 R I T Lecture 4-1 Software Engineering Reduce Ambiguity from Start to Finish Ambiguity uncertainty because of obscurity or indistinctness ; capable of being understood in two

760 views • 17 slides

More recommend