perceptrons
play

Perceptrons Jonathan Mugan jonathanwilliammugan@gmail.com - PowerPoint PPT Presentation

Aug 21, 2023 •323 likes •530 views

Perceptrons Jonathan Mugan jonathanwilliammugan@gmail.com www.jonathanmugan.com @jmugan April 10, 2014 (Slides taken from Dan Klein) Classification: Feature Vectors Hello, # free : 2 SPAM YOUR_NAME : 0 Do you want free printr

  1. Perceptrons Jonathan Mugan jonathanwilliammugan@gmail.com www.jonathanmugan.com @jmugan April 10, 2014 (Slides taken from Dan Klein)

  2. Classification: Feature Vectors Hello, # free : 2 SPAM YOUR_NAME : 0 Do you want free printr MISSPELLED : 2 or cartriges? Why pay more FROM_FRIEND : 0 when you can get them ... + ABSOLUTELY FREE! Just PIXEL-7,12 : 1 “2” PIXEL-7,13 : 0 ... NUM_LOOPS : 1 ... This slide deck courtesy of Dan Klein at UC Berkeley

  3. Some (Simplified) Biology  Very loose inspiration: human neurons 2

  4. Linear Classifiers  Inputs are feature values  Each feature has a weight  Sum is the activation  If the activation is: w 1 f 1 Σ  Positive, output +1 w 2 >0? f 2 w 3  Negative, output -1 f 3 3

  5. Example: Spam  Imagine 4 features (spam is “positive” class):  free (number of occurrences of “free”)  money (occurrences of “money”)  BIAS (intercept, always has value 1) BIAS : 1 BIAS : -3 free : 1 free : 4 “free money” money : 1 money : 2 ... ...

  6. Classification: Weights  Binary case: compare features to a weight vector  Learning: figure out the weight vector from examples # free : 4 YOUR_NAME :-1 # free : 2 MISSPELLED : 1 YOUR_NAME : 0 FROM_FRIEND :-3 MISSPELLED : 2 ... FROM_FRIEND : 0 ... # free : 0 YOUR_NAME : 1 MISSPELLED : 1 Dot product positive FROM_FRIEND : 1 means the positive class ...

  7. Binary Decision Rule  In the space of feature vectors  Examples are points  Any weight vector is a hyperplane  One side corresponds to Y=+1  Other corresponds to Y=-1 money 2 +1 = SPAM 1 BIAS : -3 free : 4 money : 2 0 ... -1 = HAM 0 1 free

  8. Mistake-Driven Classification  For Naïve Bayes:  Parameters from data statistics  Parameters: causal interpretation Training Data  Training: one pass through the data  For the perceptron:  Parameters from reactions to mistakes Held-Out Data  Prameters: discriminative interpretation  Training: go through the data until held- Test out accuracy maxes out Data 7

  9. Learning: Binary Perceptron  Start with weights = 0  For each training instance:  Classify with current weights  If correct (i.e., y=y*), no change!  If wrong: adjust the weight vector by adding or subtracting the feature vector. Subtract if y* is -1. 8

  10. Multiclass Decision Rule  If we have more than two classes:  Have a weight vector for each class:  Calculate an activation for each class  Highest activation wins 9

  11. Multiclass Decision Rule  If we have multiple classes:  A weight vector for each class:  Score (activation) of a class y:  Prediction highest score wins Binary = multiclass where the negative class has weight zero

  12. Example BIAS : 1 win : 1 “win the vote” game : 0 vote : 1 the : 1 ... BIAS : -2 BIAS : 1 BIAS : 2 win : 4 win : 2 win : 0 game : 4 game : 0 game : 2 vote : 0 vote : 4 vote : 0 the : 0 the : 0 the : 0 ... ... ...

  13. Learning: Multiclass Perceptron  Start with all weights = 0  Pick up training examples one by one  Predict with current weights  If correct, no change!  If wrong: lower score of wrong answer, raise score of right answer 12

  14. Example: Multiclass Perceptron “win the vote” “win the election” “win the game” BIAS : 1 BIAS : 0 BIAS : 0 win : 0 win : 0 win : 0 game : 0 game : 0 game : 0 vote : 0 vote : 0 vote : 0 the : 0 the : 0 the : 0 ... ... ...

  15. Examples: Perceptron  Separable Case 14

  16. Examples: Perceptron  Separable Case 15

  17. Properties of Perceptrons  Separability: some parameters get Separable the training set perfectly correct  Convergence: if the training is separable, perceptron will eventually converge (binary case)  Mistake Bound: the maximum Non-Separable number of mistakes (binary case) related to the margin or degree of separability 16

  18. Examples: Perceptron  Non-Separable Case 17

  19. Examples: Perceptron  Non-Separable Case 18

  20. Problems with the Perceptron  Noise: if the data isn’t separable, weights might thrash  Averaging weight vectors over time can help (averaged perceptron)  Mediocre generalization: finds a “barely” separating solution  Overtraining: test / held-out accuracy usually rises, then falls  Overtraining is a kind of overfitting

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Perceptrons Sven Koenig, USC Russell and Norvig, 3 rd Edition, Sections 18.7.1-18.7.4 These

Perceptrons Sven Koenig, USC Russell and Norvig, 3 rd Edition, Sections 18.7.1-18.7.4 These

12/18/2019 Perceptrons Sven Koenig, USC Russell and Norvig, 3 rd Edition, Sections 18.7.1-18.7.4 These slides are new and can contain mistakes and typos. Please report them to Sven (skoenig@usc.edu). 1 Perceptrons We now study how to

622 views • 12 slides
CS7015 (Deep Learning) : Lecture 2 McCulloch Pitts Neuron, Thresholding Logic, Perceptrons,

CS7015 (Deep Learning) : Lecture 2 McCulloch Pitts Neuron, Thresholding Logic, Perceptrons,

CS7015 (Deep Learning) : Lecture 2 McCulloch Pitts Neuron, Thresholding Logic, Perceptrons, Perceptron Learning Algorithm and Convergence, Multilayer Perceptrons (MLPs), Representation Power of MLPs Mitesh M. Khapra Department of Computer

859 views • 69 slides
Perceptrons Introduction: Neural Networks 1 The Perceptron 2 Using Perceptrons Perceptrons

Perceptrons Introduction: Neural Networks 1 The Perceptron 2 Using Perceptrons Perceptrons

Introduction: Neural Networks The Perceptron Multilayer Perceptrons Training MLPs Applying MLPs Introduction: Neural Networks The Perceptron Multilayer Perceptrons Training MLPs Applying MLPs Perceptrons Introduction: Neural Networks 1

761 views • 11 slides
CSC321 Lecture 5: Multilayer Perceptrons Roger Grosse Roger Grosse CSC321 Lecture 5: Multilayer

CSC321 Lecture 5: Multilayer Perceptrons Roger Grosse Roger Grosse CSC321 Lecture 5: Multilayer

CSC321 Lecture 5: Multilayer Perceptrons Roger Grosse Roger Grosse CSC321 Lecture 5: Multilayer Perceptrons 1 / 21 Overview Recall the simple neuron-like unit: y i'th weight bias output output w 1 w 2 weights w 3 y = g b +

240 views • 23 slides
Perceptrons Steven J Zeil Old Dominion Univ. Fall 2010 1 Introduction: Neural Networks The

Perceptrons Steven J Zeil Old Dominion Univ. Fall 2010 1 Introduction: Neural Networks The

Introduction: Neural Networks The Perceptron Multilayer Perceptrons Training MLPs Applying MLPs Perceptrons Steven J Zeil Old Dominion Univ. Fall 2010 1 Introduction: Neural Networks The Perceptron Multilayer Perceptrons Training MLPs

589 views • 47 slides
Neural Net Backpropagation 3/20/17 Recall: Limitations of Perceptrons vs. AND and OR are

Neural Net Backpropagation 3/20/17 Recall: Limitations of Perceptrons vs. AND and OR are

Neural Net Backpropagation 3/20/17 Recall: Limitations of Perceptrons vs. AND and OR are linearly separable. XOR isnt What is the output of the network? ( 0 x < 0 f ( x ) = 1 x 0 1 f ( x ) = 1 + e x ( 0 x < 0 f ( x

714 views • 13 slides
Machine Learning and Data Mining Multi-layer Perceptrons & Neural Networks: Basics Kalev

Machine Learning and Data Mining Multi-layer Perceptrons & Neural Networks: Basics Kalev

+ Machine Learning and Data Mining Multi-layer Perceptrons & Neural Networks: Basics Kalev Kask Linear classifiers (perceptrons) Linear Classifiers a linear classifier is a mapping which partitions feature space using a linear

511 views • 39 slides
Neural Network Backpropagation 3-2-16 Recall from Monday... Perceptrons can only classify

Neural Network Backpropagation 3-2-16 Recall from Monday... Perceptrons can only classify

Neural Network Backpropagation 3-2-16 Recall from Monday... Perceptrons can only classify linearly separable data. Multi-layer networks Can represent any boolean function. We dont want to build them by hand, so we need a way to

570 views • 11 slides
Perceptrons 2-29-16 What is a neural network? activation connection functions A NN is a

Perceptrons 2-29-16 What is a neural network? activation connection functions A NN is a

Perceptrons 2-29-16 What is a neural network? activation connection functions A NN is a directed acyclic graph. weights Nodes are organized into layers. -0.5 Consecutive layers are fully connected. 1.5 0.2 Edges have

147 views • 13 slides
Perceptrons From the heights of error, To the valleys of Truth Piyush Kumar Advanced

Perceptrons From the heights of error, To the valleys of Truth Piyush Kumar Advanced

Perceptrons From the heights of error, To the valleys of Truth Piyush Kumar Advanced Computational Geometry Reading Material Duda/Hart/Stork : 5.4/5.5/9.6.8 Any neural network book (Haykin, Anderson) Look at papers of

906 views • 46 slides
Linear Classifiers CS 188: Artificial Intelligence Perceptrons and Logistic Regression Pieter

Linear Classifiers CS 188: Artificial Intelligence Perceptrons and Logistic Regression Pieter

Linear Classifiers CS 188: Artificial Intelligence Perceptrons and Logistic Regression Pieter Abbeel & Dan Klein University of California, Berkeley Feature Vectors Some (Simplified) Biology Very loose inspiration: human neurons Hello,

86 views • 7 slides
CS 188: Artificial Intelligence Perceptrons and Logistic Regression Anca Dragan University of

CS 188: Artificial Intelligence Perceptrons and Logistic Regression Anca Dragan University of

CS 188: Artificial Intelligence Perceptrons and Logistic Regression Anca Dragan University of California, Berkeley Last Time Classification: given inputs x, Y predict labels (classes) y Nave Bayes F 1 F 2 F n Parameter estimation:

538 views • 32 slides
CSC421/2516 Lecture 3: Multilayer Perceptrons Roger Grosse and Jimmy Ba Roger Grosse and Jimmy

CSC421/2516 Lecture 3: Multilayer Perceptrons Roger Grosse and Jimmy Ba Roger Grosse and Jimmy

CSC421/2516 Lecture 3: Multilayer Perceptrons Roger Grosse and Jimmy Ba Roger Grosse and Jimmy Ba CSC421/2516 Lecture 3: Multilayer Perceptrons 1 / 25 Overview Recall the simple neuron-like unit: Linear regression and logistic regression

646 views • 29 slides
A NEW IDENTIFICATION SCHEME BASED ON THE PERCEPTRONS PROBLEM David POINTCHEVAL L.I.E.N.S.

A NEW IDENTIFICATION SCHEME BASED ON THE PERCEPTRONS PROBLEM David POINTCHEVAL L.I.E.N.S.

A NEW IDENTIFICATION SCHEME BASED ON THE PERCEPTRONS PROBLEM David POINTCHEVAL L.I.E.N.S. Ecole Normale Sup erieure 45, rue dUlm F-75230 Paris cedex 05 available on anonymous ftp: ftp.ens.fr in /pub/reports/liens/liens-95-2 WWW:

578 views • 8 slides
CSE 473: Artificial Intelligence Perceptrons Steve Tanimoto --- University of Washington [These

CSE 473: Artificial Intelligence Perceptrons Steve Tanimoto --- University of Washington [These

CSE 473: Artificial Intelligence Perceptrons Steve Tanimoto --- University of Washington [These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials are available at

545 views • 36 slides
Autoencoders David Dohan So far: supervised models Multilayer perceptrons (MLP)

Autoencoders David Dohan So far: supervised models Multilayer perceptrons (MLP)

Autoencoders David Dohan So far: supervised models Multilayer perceptrons (MLP) Convolutional NN (CNN) Up next: unsupervised models Autoencoders (AE) Deep Boltzmann Machines (DBM) Build high-level representations

1.05k views • 68 slides
Learning a Distance Metric for Structured Network Prediction Stuart Andrews and Tony Jebara

Learning a Distance Metric for Structured Network Prediction Stuart Andrews and Tony Jebara

Learning a Distance Metric for Structured Network Prediction Stuart Andrews and Tony Jebara Columbia University Learning to Compare Examples Workshop, December 8, 2006 Outline Introduction Context, motivation & problem

622 views • 45 slides
MIRA, SVM, k-NN Lirong Xia Linear Classifiers (perceptrons) Inputs are feature values

MIRA, SVM, k-NN Lirong Xia Linear Classifiers (perceptrons) Inputs are feature values

MIRA, SVM, k-NN Lirong Xia Linear Classifiers (perceptrons) Inputs are feature values Each feature has a weight Sum is the activation ( ) = ( ) = w i f x ( ) activation w x w i i f i x i If the activation is:

799 views • 25 slides
Does Training Affect Match Performance? A Study Using Data Mining And Tracking Devices

Does Training Affect Match Performance? A Study Using Data Mining And Tracking Devices

Does Training Affect Match Performance? A Study Using Data Mining And Tracking Devices Fernndez, J. Medina, D., Gmez A., Arias M., Gavald R. From Training Variation to Match Performance From Training Variation to Match Performance

530 views • 21 slides
CSE 573: Artificial Intelligence Autumn 2010 Lecture 16: Machine Learning Topics 12/7/2010 Luke

CSE 573: Artificial Intelligence Autumn 2010 Lecture 16: Machine Learning Topics 12/7/2010 Luke

CSE 573: Artificial Intelligence Autumn 2010 Lecture 16: Machine Learning Topics 12/7/2010 Luke Zettlemoyer Most slides over the course adapted from Dan Klein. 1 Announcements Syllabus revised Machine learning focus We will do

1.01k views • 49 slides
CS 4100: Artificial Intelligence Perceptrons and Logistic Regression Jan-Willem van de Meent,

CS 4100: Artificial Intelligence Perceptrons and Logistic Regression Jan-Willem van de Meent,

CS 4100: Artificial Intelligence Perceptrons and Logistic Regression Jan-Willem van de Meent, Northeastern University [These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials are

272 views • 16 slides
Search for top Squarks Using Multivariate Methods Jonas Graw Max Planck Institute for Physics

Search for top Squarks Using Multivariate Methods Jonas Graw Max Planck Institute for Physics

Search for top Squarks Using Multivariate Methods Jonas Graw Max Planck Institute for Physics (Werner-Heisenberg-Institut) Thursday 20 th July, 2017 Motivation Standard Strategy: Cut-based analysis ATLAS Work in progress ATLAS Work in

427 views • 16 slides
x 2 > b w T x > 0 SPAM!! x ( x , 1) w 3 x 3 w T x + b ( w , b ) T ( x , 1)

x 2 > b w T x > 0 SPAM!! x ( x , 1) w 3 x 3 w T x + b ( w , b ) T ( x , 1)

A neuron (or how our brains work) Linear models Subhransu Maji CMPSCI 670: Computer Vision November 3, 2016 Neuroscience 101 CMPSCI 670 Subhransu Maji (UMASS) 2 Perceptron Example: Spam Input are feature values Imagine 3 features (spam

682 views • 8 slides
First Observation of Single Top Quark Production at D Monica Pangilinan Brown University on

First Observation of Single Top Quark Production at D Monica Pangilinan Brown University on

First Observation of Single Top Quark Production at D Monica Pangilinan Brown University on behalf of Pheno Symposium 2009 May 12, 2009 M. Pangilinan Pheno Symposium 2009 Motivation t channel s channel Single top cross sections:

507 views • 21 slides

More recommend