lecture 3 linear classifiers
play

Lecture 3: Linear Classifiers Justin Johnson Lecture 3 - 1 - PowerPoint PPT Presentation

Sep 10, 2022 •231 likes •1.19k views

Lecture 3: Linear Classifiers Justin Johnson Lecture 3 - 1 September 11, 2019 Reminder: Assignment 1 http://web.eecs.umich.edu/~justincj/teaching/eecs498/assignment1.html Due Sunday September 15, 11:59pm EST We have written a

  1. Lecture 3: Linear Classifiers Justin Johnson Lecture 3 - 1 September 11, 2019

  2. Reminder: Assignment 1 • http://web.eecs.umich.edu/~justincj/teaching/eecs498/assignment1.html • Due Sunday September 15, 11:59pm EST • We have written a homework validation script to check the format of your .zip file before you submit to Canvas: • https://github.com/deepvision-class/tools#homework- validation • This script ensures that your .zip and .ipynb files are properly structured; they do not check correctness • It is your responsibility to make sure your submitted .zip file passes the validation script Justin Johnson Lecture 3 - 2 September 11, 2019

  3. Last time: Image Classification Input : image Output : Assign image to one of a fixed set of categories cat bird deer dog truck This image by Nikita is licensed under CC-BY 2.0 Justin Johnson Lecture 3 - 3 September 11, 2019

  4. Last Time: Challenges of Recognition Illumination Deformation Occlusion Viewpoint This image by Umberto Salvagnin is This image is CC0 1.0 public domain This image by jonsson is licensed licensed under CC-BY 2.0 under CC-BY 2.0 Clutter Intraclass Variation This image is CC0 1.0 public domain This image is CC0 1.0 public domain Justin Johnson Lecture 3 - 4 September 11, 2019

  5. Last time: Data-Drive Approach, kNN 1-NN classifier 5-NN classifier train test train validation test Justin Johnson Lecture 3 - 5 September 11, 2019

  6. Today: Linear Classifiers Justin Johnson Lecture 3 - 6 September 11, 2019

  7. Neural Network Linear classifiers This image is CC0 1.0 public domain Justin Johnson Lecture 3 - 7 September 11, 2019

  8. Recall CIFAR10 50,000 training images each image is 32x32x3 10,000 test images. Justin Johnson Lecture 3 - 8 September 11, 2019

  9. Parametric Approach Image 10 numbers giving f( x , W ) class scores Array of 32x32x3 numbers W (3072 numbers total) parameters or weights Justin Johnson Lecture 3 - 9 September 11, 2019

  10. Parametric Approach: Linear Classifier f(x,W) = Wx Image 10 numbers giving f( x , W ) class scores Array of 32x32x3 numbers W (3072 numbers total) parameters or weights Justin Johnson Lecture 3 - 10 September 11, 2019

  11. Parametric Approach: Linear Classifier (3072,) f(x,W) = Wx Image (10,) (10, 3072) 10 numbers giving f( x , W ) class scores Array of 32x32x3 numbers W (3072 numbers total) parameters or weights Justin Johnson Lecture 3 - 11 September 11, 2019

  12. Parametric Approach: Linear Classifier (3072,) f(x,W) = Wx + b (10,) Image (10,) (10, 3072) 10 numbers giving f( x , W ) class scores Array of 32x32x3 numbers W (3072 numbers total) parameters or weights Justin Johnson Lecture 3 - 12 September 11, 2019

  13. Example for 2x2 image, 3 classes (cat/dog/ship) Stretch pixels into column f(x,W) = Wx + b 56 56 231 231 24 2 24 Input image 2 (2, 2) (4,) Justin Johnson Lecture 3 - 13 September 11, 2019

  14. Example for 2x2 image, 3 classes (cat/dog/ship) Stretch pixels into column f(x,W) = Wx + b 56 0.2 -0.5 0.1 2.0 1.1 -96.8 56 231 231 + = 1.5 1.3 2.1 0.0 3.2 437.9 24 2 24 0 0.25 0.2 -0.3 -1.2 61.95 Input image 2 (2, 2) W b (3,) (3, 4) (4,) (3,) Justin Johnson Lecture 3 - 14 September 11, 2019

  15. Linear Classifier: Algebraic Viewpoint Stretch pixels into column f(x,W) = Wx + b 56 0.2 -0.5 0.1 2.0 1.1 -96.8 56 231 231 + = 1.5 1.3 2.1 0.0 3.2 437.9 24 2 24 0 0.25 0.2 -0.3 -1.2 61.95 Input image 2 (2, 2) W b (3,) (3, 4) (4,) (3,) Justin Johnson Lecture 3 - 15 September 11, 2019

  16. Add extra one to data vector; Linear Classifier: Bias Trick bias is absorbed into last column of weight matrix Stretch pixels into column 56 0.2 -0.5 0.1 2.0 1.1 -96.8 56 231 231 = 1.5 1.3 2.1 0.0 3.2 437.9 24 2 24 0 0.25 0.2 -0.3 -1.2 61.95 Input image 2 (2, 2) W (3,) (3, 5) (5,) 1 Justin Johnson Lecture 3 - 16 September 11, 2019

  17. Linear Classifier: Predictions are Linear! f(x, W) = Wx (ignore bias) f(cx, W) = W(cx) = c * f(x, W) Justin Johnson Lecture 3 - 17 September 11, 2019

  18. Linear Classifier: Predictions are Linear! f(x, W) = Wx (ignore bias) f(cx, W) = W(cx) = c * f(x, W) Image Scores 0.5 * Scores 0.5 * Image -96.8 -48.4 218.9 437.8 62.0 31.0 Justin Johnson Lecture 3 - 18 September 11, 2019

  19. Interpreting a Linear Classifier Algebraic Viewpoint f(x,W) = Wx + b Stretch pixels into column 56 0.2 -0.5 0.1 2.0 1.1 -96.8 56 231 231 + = 1.5 1.3 2.1 0.0 3.2 437.9 24 2 24 0 0.25 0.2 -0.3 -1.2 61.95 Input image 2 (2, 2) W b (3,) (3, 4) (4,) (3,) Justin Johnson Lecture 3 - 19 September 11, 2019

  20. Interpreting a Linear Classifier Algebraic Viewpoint f(x,W) = Wx + b Stretch pixels into column 0.2 -0.5 1.5 1.3 0 .25 W 56 0.2 -0.5 0.1 2.0 1.1 -96.8 56 231 0.1 2.0 2.1 0.0 0.2 -0.3 231 + = 1.5 1.3 2.1 0.0 3.2 437.9 24 2 24 0 0.25 0.2 -0.3 -1.2 61.95 Input image 2 b (2, 2) W 1.1 3.2 b -1.2 (3,) (3, 4) (4,) (3,) -96.8 437.9 61.95 Justin Johnson Lecture 3 - 20 September 11, 2019

  21. Interpreting an Linear Classifier 0.2 -0.5 1.5 1.3 0 .25 W 0.1 2.0 2.1 0.0 0.2 -0.3 b 1.1 3.2 -1.2 -96.8 437.9 61.95 Justin Johnson Lecture 3 - 21 September 11, 2019

  22. Interpreting an Linear Classifier: Visual Viewpoint 0.2 -0.5 1.5 1.3 0 .25 W 0.1 2.0 2.1 0.0 0.2 -0.3 b 1.1 3.2 -1.2 -96.8 437.9 61.95 Justin Johnson Lecture 3 - 22 September 11, 2019

  23. Interpreting an Linear Classifier: Visual Viewpoint Linear classifier has one “template” per category 0.2 -0.5 1.5 1.3 0 .25 W 0.1 2.0 2.1 0.0 0.2 -0.3 b 1.1 3.2 -1.2 -96.8 437.9 61.95 Justin Johnson Lecture 3 - 23 September 11, 2019

  24. Interpreting an Linear Classifier: Visual Viewpoint Linear classifier has one “template” per category 0.2 -0.5 1.5 1.3 0 .25 W A single template cannot capture 0.1 2.0 2.1 0.0 0.2 -0.3 multiple modes of the data b 1.1 3.2 -1.2 e.g. horse template has 2 heads! -96.8 437.9 61.95 Justin Johnson Lecture 3 - 24 September 11, 2019

  25. Interpreting a Linear Classifier: Geometric Viewpoint f(x,W) = Wx + b Airplane Score Deer Score Classifier score Car Score Array of 32x32x3 numbers (3072 numbers total) Value of pixel (15, 8, 0) Justin Johnson Lecture 3 - 25 September 11, 2019

  26. Interpreting a Linear Classifier: Geometric Viewpoint Pixel f(x,W) = Wx + b (11, 11, 0) Car score increases this way Pixel (15, 8, 0) Array of 32x32x3 numbers Car Score (3072 numbers total) = 0 Justin Johnson Lecture 3 - 26 September 11, 2019

  27. Interpreting a Linear Classifier: Geometric Viewpoint Car template Pixel on this line f(x,W) = Wx + b (11, 11, 0) Car score increases this way Pixel (15, 8, 0) Array of 32x32x3 numbers Car Score (3072 numbers total) = 0 Justin Johnson Lecture 3 - 27 September 11, 2019

  28. Interpreting a Linear Classifier: Geometric Viewpoint Car template Airplane Pixel on this line f(x,W) = Wx + b Score (11, 11, 0) Car score increases this way Pixel (15, 8, 0) Array of 32x32x3 numbers Car Score (3072 numbers total) = 0 Deer Score Justin Johnson Lecture 3 - 28 September 11, 2019

  29. Interpreting a Linear Classifier: Geometric Viewpoint Car template Airplane Pixel Hyperplanes carving up a on this line Score (11, 11, 0) high-dimensional space Car score increases this way Pixel (15, 8, 0) Car Score = 0 Deer Score Plot created using Wolfram Cloud Justin Johnson Lecture 3 - 29 September 11, 2019

  30. Hard Cases for a Linear Classifier Class 1 : Class 1 : Class 1 : First and third quadrants 1 <= L2 norm <= 2 Three modes Class 2 : Class 2 : Class 2 : Everything else Everything else Second and fourth quadrants Justin Johnson Lecture 3 - 30 September 11, 2019

  31. Recall: Perceptron couldn’t learn XOR y X Y F(x,y) 0 0 0 0 1 1 1 0 1 1 1 0 x Justin Johnson Lecture 3 - 31 September 11, 2019

  32. Linear Classifier: Three Viewpoints Visual Viewpoint Algebraic Viewpoint Geometric Viewpoint One template Hyperplanes f(x,W) = Wx per class cutting up space Justin Johnson Lecture 3 - 32 September 11, 2019

  33. f(x,W) = Wx + b So Far: Defined a linear score funct ction Given a W, we can compute class scores for an image x. -3.45 -0.51 3.42 -8.87 6.04 4.64 0.09 5.31 2.65 But how can we 2.9 -4.22 5.1 actually choose a 4.48 -4.19 2.64 8.02 3.58 5.55 good W? 3.78 4.49 -4.34 1.06 -4.37 -1.5 -0.36 -2.09 -4.79 -0.72 -2.93 6.14 Cat image by Nikita is licensed under CC-BY 2.0; Car image is CC0 1.0 public domain; Frog image is in the public domain Justin Johnson Lecture 3 - 33 September 11, 2019

  34. f(x,W) = Wx + b Choosing a good W TODO: 1. Use a loss function to -3.45 -0.51 3.42 quantify how good a -8.87 6.04 4.64 value of W is 0.09 5.31 2.65 2.9 -4.22 5.1 4.48 -4.19 2. Find a W that minimizes 2.64 8.02 3.58 5.55 the loss function 3.78 4.49 -4.34 ( optimization ) 1.06 -4.37 -1.5 -0.36 -2.09 -4.79 -0.72 -2.93 6.14 Justin Johnson Lecture 3 - 34 September 11, 2019

  35. Loss Function A loss function tells how good our current classifier is Low loss = good classifier High loss = bad classifier (Also called: objective function ; cost function ) Justin Johnson Lecture 3 - 35 September 11, 2019

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

CS440/ECE448 Lecture 22: Including Slides by Svetlana Lazebnik, 10/2016 Linear Classifiers

CS440/ECE448 Lecture 22: Including Slides by Svetlana Lazebnik, 10/2016 Linear Classifiers

Mark Hasegawa-Johnson, 3/2020 CS440/ECE448 Lecture 22: Including Slides by Svetlana Lazebnik, 10/2016 Linear Classifiers License: CC-BY 4.0 Linear Classifiers Classifiers Perceptron Linear classifiers in general Logistic

581 views • 45 slides
Nonlinear Classifiers II 2 Nonlinear Classifiers: Introduction Classifiers Supervised

Nonlinear Classifiers II 2 Nonlinear Classifiers: Introduction Classifiers Supervised

1 Nonlinear Classifiers II 2 Nonlinear Classifiers: Introduction Classifiers Supervised Classifiers XOR problem Linear Classifiers Perceptron Least Squares Methods Linear Support Vector Machine Nonlinear

355 views • 20 slides
Linear Classifiers: Expressiveness Machine Learning 1 Lecture outline Linear models:

Linear Classifiers: Expressiveness Machine Learning 1 Lecture outline Linear models:

Linear Classifiers: Expressiveness Machine Learning 1 Lecture outline Linear models: Introduction What functions do linear classifiers express? 2 Where are we? Linear models: Introduction What functions do linear classifiers

459 views • 27 slides
Linear Classifiers and the Perceptron William Cohen February 4, 2008 1 Linear classifiers

Linear Classifiers and the Perceptron William Cohen February 4, 2008 1 Linear classifiers

Linear Classifiers and the Perceptron William Cohen February 4, 2008 1 Linear classifiers Lets assume that every instance is an n -dimensional vector of real numbers x R n , and there are only two possible classes, y = (+1) and y = (

269 views • 9 slides
Lecture 14: Linear Classifiers Justin Johnson EECS 442 WI 2020: Lecture 14 - 1 February 25,

Lecture 14: Linear Classifiers Justin Johnson EECS 442 WI 2020: Lecture 14 - 1 February 25,

Lecture 14: Linear Classifiers Justin Johnson EECS 442 WI 2020: Lecture 14 - 1 February 25, 2020 Administrative HW3 due Wednesday, March 4 11:59pm TAs will not be checking Piazza over Spring Break. You are strongly encouraged to finish

869 views • 63 slides
CSC 311: Introduction to Machine Learning Lecture 3 - Linear Classifiers, Logistic Regression,

CSC 311: Introduction to Machine Learning Lecture 3 - Linear Classifiers, Logistic Regression,

CSC 311: Introduction to Machine Learning Lecture 3 - Linear Classifiers, Logistic Regression, Multiclass Classification Roger Grosse Chris Maddison Juhan Bae Silviu Pitis University of Toronto, Fall 2020 Intro ML (UofT) CSC311-Lec3 1 /

699 views • 48 slides
Extending Binary Linear Classification One-Versus-All Classification (OVA) } In the presence of

Extending Binary Linear Classification One-Versus-All Classification (OVA) } In the presence of

Binary and Other Classification } We will generally discuss binary classifiers, which divide data into one of two classes } Linear classifiers as presented in last lecture are inherently binary, defining the classes based on two regions,

336 views • 7 slides
Linear Classifiers and Perceptron E.D. And F. MAN TO BUY INTO

Linear Classifiers and Perceptron E.D. And F. MAN TO BUY INTO

Text Classification Linear Classifiers and Perceptron E.D. And F. MAN TO BUY INTO HONG KONG FIRM CS678 Advanced Topics in Machine Learning The U.K. Based commodity

157 views • 3 slides
Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall

Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall

Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Topics Discriminant functions Linear classifiers Perceptron SVM will be covered in the later lectures Fisher Multi-class

616 views • 48 slides
Sketching Linear Classifiers Over Data Streams Kai Sheng Tai Vatsal Sharan, Peter Bailis,

Sketching Linear Classifiers Over Data Streams Kai Sheng Tai Vatsal Sharan, Peter Bailis,

Sketching Linear Classifiers Over Data Streams Kai Sheng Tai Vatsal Sharan, Peter Bailis, Gregory Valiant Stanford University High-dimensional linear classifiers on streams Ubiquitous: spam detection, ad click prediction, network traffic

604 views • 28 slides
Lecture #18: Support Vector Classifiers Data Science 1 CS 109A, STAT 121A, AC 209A, E-109A

Lecture #18: Support Vector Classifiers Data Science 1 CS 109A, STAT 121A, AC 209A, E-109A

Lecture #18: Support Vector Classifiers Data Science 1 CS 109A, STAT 121A, AC 209A, E-109A Pavlos Protopapas Kevin Rader Rahul Dave Margo Levine Lecture Outline Classifying Linear Separable Data Classifying Linear Non-Separable Data 2

671 views • 22 slides
Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall

Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall

Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2019 Topics } Linear classifiers } Perceptron SVM will be covered in the later lectures } Fisher } Multi-class classification 2 Classification

881 views • 55 slides
Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall

Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall

Linear classifiers CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2018 Topics } Linear classifiers } Perceptron SVM will be covered in the later lectures } Fisher } Multi-class classification 2 Classification

786 views • 50 slides
Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied

Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied

Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied linear regression ; the output/label space Y was R . 90 80 delay 70 60 50 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 duration 1 / 68 Linear

607 views • 38 slides
Linear, Binary SVM Classifiers COMPSCI 371D Machine Learning COMPSCI 371D Machine

Linear, Binary SVM Classifiers COMPSCI 371D Machine Learning COMPSCI 371D Machine

Linear, Binary SVM Classifiers COMPSCI 371D Machine Learning COMPSCI 371D Machine Learning Linear, Binary SVM Classifiers 1 / 17 Outline 1 What Linear, Binary SVM Classifiers Do 2 Margin 3 Loss and Regularized Risk 4 Training an SVM is

510 views • 17 slides
Linear Classifiers and Regressors Borrowed with permission from Andrew Moore (CMU)

Linear Classifiers and Regressors Borrowed with permission from Andrew Moore (CMU)

Linear Classifiers and Regressors Borrowed with permission from Andrew Moore (CMU) Single-Parameter Linear Regression Linear: Slide 2 Regression vs Classification Input Prediction of Classifier Attributes categorical output Input

1.08k views • 48 slides
Chapter 4: Variability O Overview i In statistics, our goal is to measure the amount of

Chapter 4: Variability O Overview i In statistics, our goal is to measure the amount of

Chapter 4: Variability O Overview i In statistics, our goal is to measure the amount of variability for a particular set of t f i bilit f ti l t f scores, a distribution. In simple terms, if the scores in a distribution are

586 views • 37 slides
Seven types of community participation (adapted from Pretty 1994 and Cornwall 1996) 1.

Seven types of community participation (adapted from Pretty 1994 and Cornwall 1996) 1.

Seven types of community participation (adapted from Pretty 1994 and Cornwall 1996) 1. Manipulative participation (Co-option): Community participation is simply a pretence, with people's representatives on official boards who are unelected and

412 views • 7 slides
supporting international students in the classroom BALEAP. The Janus Moment in EAP: Revisiting

supporting international students in the classroom BALEAP. The Janus Moment in EAP: Revisiting

Teach for success: supporting international students in the classroom BALEAP. The Janus Moment in EAP: Revisiting the Past and Building the Future, University of Nottingham, 19-21 April 2013 Jill Doubleday, Modern Languages &amp; Centre for

832 views • 30 slides
SETTING U G UP A TELE-PSYCHIATRY S Y SERVICE CE F FOR R RU RURAL E EMERGENCY D NCY DEPART

SETTING U G UP A TELE-PSYCHIATRY S Y SERVICE CE F FOR R RU RURAL E EMERGENCY D NCY DEPART

SETTING U G UP A TELE-PSYCHIATRY S Y SERVICE CE F FOR R RU RURAL E EMERGENCY D NCY DEPART RTMENT NTS DR. R RAH AHUL G GUPTA FRANZCP CLINICAL LEAD NMHEC-RAP HUNTER NEW ENGLAND MENTAL HEALTH SERVICE 1 NMHE MHEC-RAP RAP NORT

774 views • 51 slides
Educator Licensure and Preparation Subcommittee APRIL 30, 2020 Agenda edTPA Research and

Educator Licensure and Preparation Subcommittee APRIL 30, 2020 Agenda edTPA Research and

Educator Licensure and Preparation Subcommittee APRIL 30, 2020 Agenda edTPA Research and Trends in Tennessee 2019 Educator Preparation Report Card TN Department of Education Annual Reports TN Department of Education Updates

737 views • 36 slides
DESIGN, SIMULATION, TESTING CSSE 120Rose Hulman Institute of Technology

DESIGN, SIMULATION, TESTING CSSE 120Rose Hulman Institute of Technology

DESIGN, SIMULATION, TESTING CSSE 120Rose Hulman Institute of Technology Designing/implementing a larger program Until now, our programs have been small and simple Possible exceptions: dayOfYear, fileAverage, TicTacToe. For larger

751 views • 30 slides
L ea rn i n g D ee p K e rn el s L ea rn i n g D ee p K e rn el s f or E xpon e nt ial F a m il y D

L ea rn i n g D ee p K e rn el s L ea rn i n g D ee p K e rn el s f or E xpon e nt ial F a m il y D

L ea rn i n g D ee p K e rn el s L ea rn i n g D ee p K e rn el s f or E xpon e nt ial F a m il y D e ns i t ie s f or E xpon e nt ial F a m il y D e ns i t ie s Li K. Wenliang D. J. Sutherland H. Strathmann A. Gretton Gatsby unit, University

721 views • 21 slides
Part 6: Scoring in a Complete Search System Francesco Ricci Most of these slides comes from the

Part 6: Scoring in a Complete Search System Francesco Ricci Most of these slides comes from the

Part 6: Scoring in a Complete Search System Francesco Ricci Most of these slides comes from the course: Information Retrieval and Web Search, Christopher Manning and Prabhakar Raghavan 1 Content p Vector space scoring p Speeding up

850 views • 37 slides

More recommend