lecture 3 linear classification
play

Lecture 3: Linear Classification Fei-Fei Li & Andrej Karpathy - PowerPoint PPT Presentation

Jan 01, 2023 •719 likes •1.2k views

Lecture 3: Linear Classification Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 1 Last time: Image Classification assume given set of discrete labels {dog, cat, truck,

  1. Lecture 3: Linear Classification Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 1

  2. Last time: Image Classification assume given set of discrete labels {dog, cat, truck, plane, ...} cat Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 2

  3. k-Nearest Neighbor test images training set Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 3

  4. Linear Classification 1. define a score function class scores Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 4

  5. Linear Classification 1. define a score function data (image) “bias vector” “weights” class scores “parameters” Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 5

  6. Linear Classification data (image) 1. define a score function [3072 x 1] (assume CIFAR-10 example so 32 x 32 x 3 images, 10 classes) bias vector weights class scores Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 6

  7. Linear Classification data (image) 1. define a score function [3072 x 1] (assume CIFAR-10 example so 32 x 32 x 3 images, 10 classes) bias vector weights class scores [10 x 1] [10 x 3072] [10 x 1] Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 7

  8. Linear Classification Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 8

  9. Interpreting a Linear Classifier Question: what can a linear classifier do? Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 9

  10. Interpreting a Linear Classifier Example training classifiers on CIFAR-10: Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 10

  11. Interpreting a Linear Classifier Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 11

  12. Bias trick Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 12

  13. So far: We defined a (linear) score function: Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 13

  14. 2. Define a loss function (or cost function, or objective) Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 14

  15. 2. Define a loss function (or cost function, or objective) - scores, label loss . Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 15

  16. 2. Define a loss function (or cost function, or objective) - scores, label loss . Example: Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 16

  17. 2. Define a loss function (or cost function, or objective) - scores, label loss . Question: if you were to Example: assign a single number to how “unhappy” you are with these scores, what would you do? Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 17

  18. 2. Define a loss function (or cost function, or objective) One (of many ways) to do it: Multiclass SVM Loss Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 18

  19. 2. Define a loss function (or cost function, or objective) One (of many ways) to do it: Multiclass SVM Loss (One possible generalization of Binary Support Vector Machine to multiple classes) Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 19

  20. 2. Define a loss function (or cost function, or objective) One (of many ways) to do it: Multiclass SVM Loss loss due to difference between the correct class sum over all example i score and incorrect class score incorrect labels Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 20

  21. loss due to difference between the correct class sum over all example i score and incorrect class score incorrect labels Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 21

  22. Example: e.g. 10 loss = ? Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 22

  23. Example: e.g. 10 Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 23

  24. Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 24

  25. There is a bug with the objective… Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 25

  26. L2 Regularization Regularization strength Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 26

  27. L2 regularization: motivation Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 27

  28. Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 28

  29. Do we have to cross-validate both and ? Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 29

  30. Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 30

  31. So far… 1. Score function 2. Loss function Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 31

  32. Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 32

  33. score function Softmax Classifier is the same (extension of Logistic Regression to multiple classes) Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 33

  34. score function Softmax Classifier is the same Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 34

  35. score function Softmax Classifier is the same softmax function i.e. we’re minimizing the negative log likelihood. Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 35

  36. score function Softmax Classifier is the same softmax function Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 36

  37. score function Softmax Classifier is the same i.e. we’re minimizing the negative log likelihood. Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 37

  38. Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 38

  39. Softmax vs. SVM - Interpreting the probabilities from the Softmax Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 39

  40. Softmax vs. SVM - Interpreting the probabilities from the Softmax suppose the weights W were only half as large (we use a higher regularization strength) Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 40

  41. Softmax vs. SVM - Interpreting the probabilities from the Softmax suppose the weights W were only half as large: Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 41

  42. Softmax vs. SVM - Interpreting the probabilities from the Softmax suppose the weights W were only half as large: What happens in the limit, as the regularization strength goes to infinity? Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 42

  43. Softmax vs. SVM 1 scores: [10, -2, 3] [10, 9, 9] [10, -100, -100] Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 43

  44. Softmax vs. SVM 1 Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 44

  45. Interactive Web Demo time.... http://vision.stanford.edu/teaching/cs231n/linear-classify-demo/ Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 45

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear

Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear

Learning From Data Lecture 8 Linear Classification and Regression Linear Classification Linear Regression M. Magdon-Ismail CSCI 4100/6100 recap: Approximation Versus Generalization VC Analysis Bias-Variance Analysis E out E in + ( d

390 views • 23 slides
Homework Homework Lecture 7: Linear Classification Methods Final projects? Groups Topics

Homework Homework Lecture 7: Linear Classification Methods Final projects? Groups Topics

Homework Homework Lecture 7: Linear Classification Methods Final projects? Groups Topics Proposal week 5 Lecture 20 is poster session, Jacobs Hall Lobby, snacks Final report 15 June. What is linear classification? Classification is

524 views • 26 slides
Web Information Retrieval Lecture 14 Text classification Sec. 13.1 Text Classification

Web Information Retrieval Lecture 14 Text classification Sec. 13.1 Text Classification

Web Information Retrieval Lecture 14 Text classification Sec. 13.1 Text Classification Nave Bayes Classification Vector space methods for Text Classification K Nearest Neighbors Decision boundaries Linear Classifiers

744 views • 47 slides
CSC 411 Lecture 8: Linear Classification II Roger Grosse, Amir-massoud Farahmand, and Juan

CSC 411 Lecture 8: Linear Classification II Roger Grosse, Amir-massoud Farahmand, and Juan

CSC 411 Lecture 8: Linear Classification II Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla University of Toronto UofT CSC 411: 08-Linear Classification 1 / 34 Todays Agenda Todays agenda: Gradient checking with finite

597 views • 34 slides
Data Mining Lecture 03: Introduction to classification Linear classifier Theses

Data Mining Lecture 03: Introduction to classification Linear classifier Theses

CISC 4631 Data Mining Lecture 03: Introduction to classification Linear classifier Theses slides are based on the slides by Tan, Steinbach and Kumar (textbook authors) Eamonn Koegh (UC Riverside) 1 Classification:

650 views • 32 slides
Lecture 9: Logistic Regression Discriminative vs. Generative Classification Linear

Lecture 9: Logistic Regression Discriminative vs. Generative Classification Linear

Lecture 9: Logistic Regression Discriminative vs. Generative Classification Linear Discriminant Functions Aykut Erdem October 2017 Hacettepe University Administrative Midterm exam will be held on November 6. Project proposals

562 views • 52 slides
Extending Binary Linear Classification One-Versus-All Classification (OVA) } In the presence of

Extending Binary Linear Classification One-Versus-All Classification (OVA) } In the presence of

Binary and Other Classification } We will generally discuss binary classifiers, which divide data into one of two classes } Linear classifiers as presented in last lecture are inherently binary, defining the classes based on two regions,

336 views • 7 slides
Logistic Regression Lecture 6 Logistic Regression Classification Model CS 335

Logistic Regression Lecture 6 Logistic Regression Classification Model CS 335

Logistic Regression Lecture 6 Logistic Regression Classification Model CS 335 Cost function Gradient descent Linear classifiers and decision boundaries Dan Sheldon Classification Example: Hand-Written Digits Input: 20

541 views • 5 slides
Linear Regression 1 / 10 The Linear Model So far weve dealt with classification, where the

Linear Regression 1 / 10 The Linear Model So far weve dealt with classification, where the

Linear Regression 1 / 10 The Linear Model So far weve dealt with classification, where the target function maps feature vectors to discrete classes, but the linear model is more versatile. Consider the credit analysis problem: We can use

566 views • 10 slides
Multiclass Classification Machine Learning So far: Binary Classification We have seen linear

Multiclass Classification Machine Learning So far: Binary Classification We have seen linear

Multiclass Classification Machine Learning So far: Binary Classification We have seen linear models Learning algorithms for linear models Perceptron, Winnow, Adaboost, SVM We will see more soon: Nave Bayes, Logistic Regression

635 views • 24 slides
Lecture #10: Classification & Logistic Regression Data Science 1 CS 109A, STAT 121A, AC

Lecture #10: Classification & Logistic Regression Data Science 1 CS 109A, STAT 121A, AC

Lecture #10: Classification & Logistic Regression Data Science 1 CS 109A, STAT 121A, AC 209A, E-109A Pavlos Protopapas Kevin Rader Margo Levine Rahul Dave Lecture Outline Module 2: Classification Why not Linear Regression? Binary

1.1k views • 77 slides
CS480/680 Lecture 8: June 3, 2019 Classification by Logistic Regression, Generalized linear

CS480/680 Lecture 8: June 3, 2019 Classification by Logistic Regression, Generalized linear

CS480/680 Lecture 8: June 3, 2019 Classification by Logistic Regression, Generalized linear models [RN] Sec 18.6.4, [B] Sec. 4.3, [M] Chapt. 8, [HTF] Sec. 4.4 University of Waterloo CS480/680 Spring 2019 Pascal Poupart 1 Beyond Mixtures of

756 views • 18 slides
Algorithms for NLP CS 11-711 Fall 2020 Lecture 2: Linear text classification Emma Strubell

Algorithms for NLP CS 11-711 Fall 2020 Lecture 2: Linear text classification Emma Strubell

Algorithms for NLP CS 11-711 Fall 2020 Lecture 2: Linear text classification Emma Strubell Lets try this again Emma Yulia Bob Sanket Han Jiateng she/her she/her he/him he/him he/him he/him 2 Outline Basic

534 views • 37 slides
E9 205 Machine Learning for Signal Processing Probablistic Linear Models 30-09-2019 Linear

E9 205 Machine Learning for Signal Processing Probablistic Linear Models 30-09-2019 Linear

E9 205 Machine Learning for Signal Processing Probablistic Linear Models 30-09-2019 Linear Models for Classification Optimize a modified cost function Bishop - PRML book (Chap 3) Least Squares for Classification K-class classification

423 views • 6 slides
Linear Classification 1 / 14 The Linear Model In the next few lectures we will extend the

Linear Classification 1 / 14 The Linear Model In the next few lectures we will extend the

Linear Classification 1 / 14 The Linear Model In the next few lectures we will extend the perceptron learning algorithm to handle non-linearly separable data, explore online versis batch learning, learn three different learning

466 views • 14 slides
CSE 258 Lecture 3 Web Mining and Recommender Systems Supervised learning Classification

CSE 258 Lecture 3 Web Mining and Recommender Systems Supervised learning Classification

CSE 258 Lecture 3 Web Mining and Recommender Systems Supervised learning Classification Last week Last week we started looking at supervised learning problems Last week We studied linear regression , in order to learn linear

987 views • 67 slides
Computer Graphics III Spherical integrals, Light & Radiometry Exercises Jaroslav

Computer Graphics III Spherical integrals, Light & Radiometry Exercises Jaroslav

Computer Graphics III Spherical integrals, Light & Radiometry Exercises Jaroslav Kivnek, MFF UK Jaroslav.Krivanek@mff.cuni.cz Reminders & org Renderings due next week Upload to google drive, show on the big screen, 5

396 views • 20 slides
Agents that Reason Logically by Chris Horn Jiansui Yang Xiaojing Wu 3/30/00 Logic

Agents that Reason Logically by Chris Horn Jiansui Yang Xiaojing Wu 3/30/00 Logic

Agents that Reason Logically by Chris Horn Jiansui Yang Xiaojing Wu 3/30/00 Logic Prerequisites: Computer Science 313, 331 and Philosophy 279 or 377. 3/30/00 Presentation Outline 1 A Knowledge-Based Agent 2 Representation,

689 views • 40 slides
Inference engine Knowledge base Breeze Stench 4 PIT Breeze Breeze 3 PIT Stench Gold

Inference engine Knowledge base Breeze Stench 4 PIT Breeze Breeze 3 PIT Stench Gold

Inference engine Knowledge base Breeze Stench 4 PIT Breeze Breeze 3 PIT Stench Gold Breeze Stench 2 Breeze Breeze 1 PIT START 1 2 3 4 OK OK OK A B OK A OK OK A P? B OK P? A OK OK A P? B OK P? A S OK

582 views • 30 slides
Google Slides Lots of options to start Lots of template options to start with. Try to keep it

Google Slides Lots of options to start Lots of template options to start with. Try to keep it

Google Slides Lots of options to start Lots of template options to start with. Try to keep it clean and simple 1 G OOGLE S LIDES Google Slides Basic formatting Choose your theme, add some content and name your presentation. Presentations are

553 views • 11 slides
Linear classifiers : prediction eq u ations L IN E AR C L ASSIFIE R S IN P YTH ON Michael ( Mike

Linear classifiers : prediction eq u ations L IN E AR C L ASSIFIE R S IN P YTH ON Michael ( Mike

Linear classifiers : prediction eq u ations L IN E AR C L ASSIFIE R S IN P YTH ON Michael ( Mike ) Gelbart Instr u ctor , The Uni v ersit y of British Col u mbia Dot Prod u cts x = np.arange(3) np.sum(x*y) x 14 array([0, 1, 2]) x@y y =

513 views • 22 slides
EXPLOITING AND REMODELLING SEMANTIC RELATIONSHIPS Fran Ale lexande der, , Taxonomy Manager,

EXPLOITING AND REMODELLING SEMANTIC RELATIONSHIPS Fran Ale lexande der, , Taxonomy Manager,

TRANSFORMATION OF A LEGACY UDC- BASED CLASSIFICATION SYSTEM: EXPLOITING AND REMODELLING SEMANTIC RELATIONSHIPS Fran Ale lexande der, , Taxonomy Manager, BBC Information and Archives, London, UK Andy y Heather er, , Chief Technical

165 views • 14 slides
Error Link Detection and Correction in Wikipedia Chengyu Wang, Rong Zhang, Xiaofeng He, Aoying

Error Link Detection and Correction in Wikipedia Chengyu Wang, Rong Zhang, Xiaofeng He, Aoying

Error Link Detection and Correction in Wikipedia Chengyu Wang, Rong Zhang, Xiaofeng He, Aoying Zhou School of Computer Science and Software Engineering East China Normal University Shanghai, China Outline Introduction Related Work

353 views • 24 slides
Introduction Karl Stratos Rutgers University Karl Stratos CS 533: Natural Language Processing

Introduction Karl Stratos Rutgers University Karl Stratos CS 533: Natural Language Processing

CS 533: Natural Language Processing Introduction Karl Stratos Rutgers University Karl Stratos CS 533: Natural Language Processing 1/10 Modern Natural Language Processing (NLP) NLP is everywhere Other examples? Karl Stratos CS 533: Natural

115 views • 10 slides

More recommend