csc 411 lecture 02 linear regression
play

CSC 411: Lecture 02: Linear Regression Class based on Raquel Urtasun - PowerPoint PPT Presentation

Apr 08, 2024 •175 likes •1.15k views

CSC 411: Lecture 02: Linear Regression Class based on Raquel Urtasun & Rich Zemels lectures Sanja Fidler University of Toronto Jan 13, 2016 (Most plots in this lecture are from Bishops book) Urtasun, Zemel, Fidler (UofT) CSC 411:

  1. CSC 411: Lecture 02: Linear Regression Class based on Raquel Urtasun & Rich Zemel’s lectures Sanja Fidler University of Toronto Jan 13, 2016 (Most plots in this lecture are from Bishop’s book) Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 1 / 22

  2. Problems for Today What should I watch this Friday? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 2 / 22

  3. Problems for Today What should I watch this Friday? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 2 / 22

  4. Problems for Today Goal : Predict movie rating automatically! Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 2 / 22

  5. Problems for Today Goal: How many followers will I get? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 2 / 22

  6. Problems for Today Goal: Predict the price of the house Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 2 / 22

  7. Regression What do all these problems have in common? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  8. Regression What do all these problems have in common? ◮ Continuous outputs, we’ll call these t (eg, a rating: a real number between 0-10, # of followers, house price) Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  9. Regression What do all these problems have in common? ◮ Continuous outputs, we’ll call these t (eg, a rating: a real number between 0-10, # of followers, house price) What do I need in order to predict these outputs? Predicting continuous outputs is called regression Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  10. Regression What do all these problems have in common? ◮ Continuous outputs, we’ll call these t (eg, a rating: a real number between 0-10, # of followers, house price) What do I need in order to predict these outputs? Predicting continuous outputs is called regression ◮ Features (inputs), we’ll call these x (or x if vectors) Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  11. Regression What do all these problems have in common? ◮ Continuous outputs, we’ll call these t (eg, a rating: a real number between 0-10, # of followers, house price) What do I need in order to predict these outputs? Predicting continuous outputs is called regression ◮ Features (inputs), we’ll call these x (or x if vectors) ◮ Training examples, many x ( i ) for which t ( i ) is known (eg, many movies for which we know the rating) Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  12. Regression What do all these problems have in common? ◮ Continuous outputs, we’ll call these t (eg, a rating: a real number between 0-10, # of followers, house price) What do I need in order to predict these outputs? Predicting continuous outputs is called regression ◮ Features (inputs), we’ll call these x (or x if vectors) ◮ Training examples, many x ( i ) for which t ( i ) is known (eg, many movies for which we know the rating) ◮ A model, a function that represents the relationship between x and t Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  13. Regression What do all these problems have in common? ◮ Continuous outputs, we’ll call these t (eg, a rating: a real number between 0-10, # of followers, house price) What do I need in order to predict these outputs? Predicting continuous outputs is called regression ◮ Features (inputs), we’ll call these x (or x if vectors) ◮ Training examples, many x ( i ) for which t ( i ) is known (eg, many movies for which we know the rating) ◮ A model, a function that represents the relationship between x and t ◮ A loss or a cost or an objective function, which tells us how well our model approximates the training examples Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  14. Regression What do all these problems have in common? ◮ Continuous outputs, we’ll call these t (eg, a rating: a real number between 0-10, # of followers, house price) What do I need in order to predict these outputs? Predicting continuous outputs is called regression ◮ Features (inputs), we’ll call these x (or x if vectors) ◮ Training examples, many x ( i ) for which t ( i ) is known (eg, many movies for which we know the rating) ◮ A model, a function that represents the relationship between x and t ◮ A loss or a cost or an objective function, which tells us how well our model approximates the training examples ◮ Optimization, a way of finding the parameters of our model that minimizes the loss function Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 3 / 22

  15. Today: Linear Regression Linear regression ◮ continuous outputs ◮ simple model (linear) Introduce key concepts: ◮ loss functions ◮ generalization ◮ optimization ◮ model complexity ◮ regularization Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 4 / 22

  16. Simple 1-D regression Circles are data points (i.e., training examples) that are given to us Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 5 / 22

  17. Simple 1-D regression Circles are data points (i.e., training examples) that are given to us The data points are uniform in x , but may be displaced in y t ( x ) = f ( x ) + ǫ with ǫ some noise Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 5 / 22

  18. Simple 1-D regression Circles are data points (i.e., training examples) that are given to us The data points are uniform in x , but may be displaced in y t ( x ) = f ( x ) + ǫ with ǫ some noise In green is the ”true” curve that we don’t know Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 5 / 22

  19. Simple 1-D regression Circles are data points (i.e., training examples) that are given to us The data points are uniform in x , but may be displaced in y t ( x ) = f ( x ) + ǫ with ǫ some noise In green is the ”true” curve that we don’t know Goal: We want to fit a curve to these points Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 5 / 22

  20. Simple 1-D regression Key Questions: Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 6 / 22

  21. Simple 1-D regression Key Questions: ◮ How do we parametrize the model? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 6 / 22

  22. Simple 1-D regression Key Questions: ◮ How do we parametrize the model? ◮ What loss (objective) function should we use to judge the fit? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 6 / 22

  23. Simple 1-D regression Key Questions: ◮ How do we parametrize the model? ◮ What loss (objective) function should we use to judge the fit? ◮ How do we optimize fit to unseen test data (generalization)? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 6 / 22

  24. Example: Boston Housing data Estimate median house price in a neighborhood based on neighborhood statistics Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 7 / 22

  25. Example: Boston Housing data Estimate median house price in a neighborhood based on neighborhood statistics Look at first possible attribute (feature): per capita crime rate Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 7 / 22

  26. Example: Boston Housing data Estimate median house price in a neighborhood based on neighborhood statistics Look at first possible attribute (feature): per capita crime rate Use this to predict house prices in other neighborhoods Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 7 / 22

  27. Example: Boston Housing data Estimate median house price in a neighborhood based on neighborhood statistics Look at first possible attribute (feature): per capita crime rate Use this to predict house prices in other neighborhoods Is this a good input (attribute) to predict house prices? Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 7 / 22

  28. Represent the Data Data is described as pairs D = { ( x (1) , t (1) ) , · · · , ( x ( N ) , t ( N ) ) } ◮ x ∈ R is the input feature (per capita crime rate) ◮ t ∈ R is the target output (median house price) ◮ ( i ) simply indicates the training examples (we have N in this case) Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 8 / 22

  29. Represent the Data Data is described as pairs D = { ( x (1) , t (1) ) , · · · , ( x ( N ) , t ( N ) ) } ◮ x ∈ R is the input feature (per capita crime rate) ◮ t ∈ R is the target output (median house price) ◮ ( i ) simply indicates the training examples (we have N in this case) Here t is continuous, so this is a regression problem Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 8 / 22

  30. Represent the Data Data is described as pairs D = { ( x (1) , t (1) ) , · · · , ( x ( N ) , t ( N ) ) } ◮ x ∈ R is the input feature (per capita crime rate) ◮ t ∈ R is the target output (median house price) ◮ ( i ) simply indicates the training examples (we have N in this case) Here t is continuous, so this is a regression problem Model outputs y , an estimate of t y ( x ) = w 0 + w 1 x Urtasun, Zemel, Fidler (UofT) CSC 411: 02-Regression Jan 13, 2016 8 / 22

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

CS70: Lecture 35. Regression (contd.): Linear and Beyond CS70: Lecture 35. Regression (contd.):

CS70: Lecture 35. Regression (contd.): Linear and Beyond CS70: Lecture 35. Regression (contd.):

CS70: Lecture 35. Regression (contd.): Linear and Beyond CS70: Lecture 35. Regression (contd.): Linear and Beyond 1. Review: Linear Regression (LR), LLSE 2. LR: Examples 3. Beyond LR: Quadratic Regression 4. Conditional Expectation (CE) and

2.12k views • 132 slides
CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear

CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear

CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear Regression 1 / 30 Overview First learning algorithm of the course: linear regression Task: predict scalar-valued targets, e.g. stock prices (hence

506 views • 34 slides
CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear

CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear

CSC321 Lecture 2: Linear Regression Roger Grosse Roger Grosse CSC321 Lecture 2: Linear Regression 1 / 26 Overview First learning algorithm of the course: linear regression Task: predict scalar-valued targets, e.g. stock prices (hence

328 views • 29 slides
Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear

Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear

Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear Regression M. Magdon-Ismail CSCI 4100/6100 recap: Approximation Versus Generalization VC Analysis Bias-Variance Analysis E out E in + ( d

390 views • 23 slides
Outline The Simple Linear Regression Model (12.1) Fitting the Regression Line (12.2)

Outline The Simple Linear Regression Model (12.1) Fitting the Regression Line (12.2)

2/22/2007 219323 Probability and Statistics for Software Statistics for Software and Knowledge Engineers Lecture 13: Simple Linear Regression Simple Linear Regression and Correlation Monchai Sopitkamon, Ph.D. Outline The Simple Linear

968 views • 19 slides
CSC 411 Lecture 6: Linear Regression Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla

CSC 411 Lecture 6: Linear Regression Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla

CSC 411 Lecture 6: Linear Regression Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla University of Toronto UofT CSC 411: 06-Linear Regression 1 / 37 A Timely XKCD UofT CSC 411: 06-Linear Regression 2 / 37 Overview So far,

482 views • 37 slides
Learning From Data Lecture 9 Logistic Regression and Gradient Descent Logistic Regression

Learning From Data Lecture 9 Logistic Regression and Gradient Descent Logistic Regression

Learning From Data Lecture 9 Logistic Regression and Gradient Descent Logistic Regression Gradient Descent M. Magdon-Ismail CSCI 4100/6100 recap: Linear Classification and Regression The linear signal: s = w t x Good Features are Important

272 views • 23 slides
Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common

Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common

0. Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common property CMU, 2004 fall, Andrew Moore, HW2, pr. 4 2. Linear Regression and Logistic Regression: Definitions Given an input vector X , linear

1.07k views • 22 slides
Lecture 1. From Linear Regression Nan Ye School of Mathematics and Physics University of

Lecture 1. From Linear Regression Nan Ye School of Mathematics and Physics University of

Lecture 1. From Linear Regression Nan Ye School of Mathematics and Physics University of Queensland 1 / 20 Quiz Q1. Which dataset is linear regression of y against x suitable for? y y 15 1

596 views • 21 slides
Notes on the Non-linear Regression The model Non-linear regression models, like ordinary linear

Notes on the Non-linear Regression The model Non-linear regression models, like ordinary linear

Notes on the Non-linear Regression The model Non-linear regression models, like ordinary linear models, assume that the response, Y , has a normal distribution with constant variance. Unlike linear models, however, the mean function may depend

159 views • 4 slides
LINEAR REGRESSION LINEAR REGRESSION - FROM A MACHINE LEARNING POINT OF VIEW 25 SIMPLE LINEAR

LINEAR REGRESSION LINEAR REGRESSION - FROM A MACHINE LEARNING POINT OF VIEW 25 SIMPLE LINEAR

LINEAR REGRESSION LINEAR REGRESSION - FROM A MACHINE LEARNING POINT OF VIEW 25 SIMPLE LINEAR REGRESSION 20 Sales 15 Starting point 10 Simplest parametric function 5 Easy to interpret the parameters: 0 50 100 150 200 250

1.12k views • 13 slides
Lecture 6: Multiple Linear Regression, Polynomial Regression and Model Selection CS109A

Lecture 6: Multiple Linear Regression, Polynomial Regression and Model Selection CS109A

Lecture 6: Multiple Linear Regression, Polynomial Regression and Model Selection CS109A Introduction to Data Science Pavlos Protopapas and Kevin Rader Announcements Section : Friday 1:30-2:45pm : @ MD 123 (only this Friday) A-section: Today:

921 views • 65 slides
CS70: Jean Walrand: Lecture 30. Linear Regression: Preamble Linear Regression: Preamble Thus, if

CS70: Jean Walrand: Lecture 30. Linear Regression: Preamble Linear Regression: Preamble Thus, if

X Y CS70: Jean Walrand: Lecture 30. Linear Regression: Preamble Linear Regression: Preamble Thus, if we want to guess the value of Y , we choose E [ Y ] . The best guess about Y , if we know only the distribution of Y , is E [ Y ] . Now assume

53 views • 4 slides
CS480/680 Machine Learning Lecture 3: May 13, 2019 Linear Regression [RN] Sec. 18.6.1, [HTF]

CS480/680 Machine Learning Lecture 3: May 13, 2019 Linear Regression [RN] Sec. 18.6.1, [HTF]

CS480/680 Machine Learning Lecture 3: May 13, 2019 Linear Regression [RN] Sec. 18.6.1, [HTF] Sec. 2.3.1, [D] Sec. 7.6, [B] Sec. 3.1, [M] Sec. 1.4.5 University of Waterloo CS480/680 Spring 2019 Pascal Poupart 1 Linear model for regression

582 views • 14 slides
Lecture #5: Multiple Linear Regression Data Science 1 CS 109A, STAT 121A, AC 209A, E-109A Pavlos

Lecture #5: Multiple Linear Regression Data Science 1 CS 109A, STAT 121A, AC 209A, E-109A Pavlos

Lecture #5: Multiple Linear Regression Data Science 1 CS 109A, STAT 121A, AC 209A, E-109A Pavlos Protopapas Kevin Rader Margo Levine Rahul Dave Lecture Outline Review More on Model Evaluation Multiple Linear Regression Evaluating

585 views • 32 slides
Lecture 5: Multiple Linear Regression CS109A Introduction to Data Science Pavlos Protopapas and

Lecture 5: Multiple Linear Regression CS109A Introduction to Data Science Pavlos Protopapas and

Lecture 5: Multiple Linear Regression CS109A Introduction to Data Science Pavlos Protopapas and Kevin Rader Lecture Outline Simple Regression: Predictor variables Standard Errors Evaluating Significance of Predictors Hypothesis

540 views • 50 slides
FCOT Portfolio detail as at 31 March 2014 Portfolio details Singapore & Australia focus

FCOT Portfolio detail as at 31 March 2014 Portfolio details Singapore & Australia focus

FCOT Portfolio detail as at 31 March 2014 Portfolio details Singapore & Australia focus 2 Focus on key markets of Singapore and Australia Asset values 1 Net property income TOTAL: S$1,796.5 million TOTAL: S$21.7 million for

437 views • 8 slides
In Praise of Snapshots Ravi Kanbur www.Kanbur.Dyson.cornell.edu UNU-WIDER, Helsinki, 5

In Praise of Snapshots Ravi Kanbur www.Kanbur.Dyson.cornell.edu UNU-WIDER, Helsinki, 5

In Praise of Snapshots Ravi Kanbur www.Kanbur.Dyson.cornell.edu UNU-WIDER, Helsinki, 5 September, 2019 Outline Introduction Positive Analysis: From Mobility to Distribution and Back Normative Assessment: Snapshots and Movies

354 views • 31 slides
[3] The Vector Space Linear Combinations An expression 1 v 1 + + n v n is a linear

[3] The Vector Space Linear Combinations An expression 1 v 1 + + n v n is a linear

The Vector Space [3] The Vector Space Linear Combinations An expression 1 v 1 + + n v n is a linear combination of the vectors v 1 , . . . , v n . The scalars 1 , . . . , n are the coefficients of the linear combination.

212 views • 18 slides
CS 161: Computer Security Prof. Vern Paxson TAs: Jethro Beekman, Mobin Javed, Antonio Lupher,

CS 161: Computer Security Prof. Vern Paxson TAs: Jethro Beekman, Mobin Javed, Antonio Lupher,

CS 161: Computer Security Prof. Vern Paxson TAs: Jethro Beekman, Mobin Javed, Antonio Lupher, Paul Pearce & Matthias Vallentin http://inst.eecs.berkeley.edu/~cs161/ January 22, 2013 Course Size The course has a capacity (= room, TAs)

676 views • 50 slides
60% are racial or ethnic minorities Over 50% of have symptoms of serious mental illness More

60% are racial or ethnic minorities Over 50% of have symptoms of serious mental illness More

The United States locks up more people, per capita, than any other nation. 60% are racial or ethnic minorities Over 50% of have symptoms of serious mental illness More than 68 percent have substance abuse disorders 1 A large body of

299 views • 8 slides
Spiritual Inspiration Introduction Beauty - Mklppen SW tip of Sweden v 2.5 Normal book

Spiritual Inspiration Introduction Beauty - Mklppen SW tip of Sweden v 2.5 Normal book

Spiritual Inspiration Introduction Beauty - Mklppen SW tip of Sweden v 2.5 Normal book Ink + Paper = Holy Message Bible Living Word Living Gospel Divine Principle Living Word for a New Age Living Word

759 views • 41 slides
LNG IN DEVENTER JOOST DE RUIJTER 10 oktober 2019 CLEANER AND MORE SUSTAINABLE Transition from

LNG IN DEVENTER JOOST DE RUIJTER 10 oktober 2019 CLEANER AND MORE SUSTAINABLE Transition from

LNG IN DEVENTER JOOST DE RUIJTER 10 oktober 2019 CLEANER AND MORE SUSTAINABLE Transition from fossil to sustainable Air pollution Global warming Scarcity of resources 2 GLOBAL ENERGY DEMAND BY 2035, THE WORLD WILL USE ENERGY AT A RATE OF

561 views • 38 slides
Challenges in Antimicrobial Clinical Development Axel Dalhoff & Heino Sta Institut fr

Challenges in Antimicrobial Clinical Development Axel Dalhoff & Heino Sta Institut fr

Challenges in Antimicrobial Clinical Development Axel Dalhoff & Heino Sta Institut fr Klinische Pharmakologie Bayer AG, Wuppertal, D New antibacterial agents approved in the United States 1983 - 2002 20 No. of New Antibacterial

610 views • 24 slides

More recommend