BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning - PowerPoint PPT Presentation

50 Image Classification : a core task in Computer Vision slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson

51 The problem : semantic gap slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson

52 Challenges: Viewpoint Variation slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson

53 Challenges: Illumination slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson

54 Challenges: Deformation slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson

55 Challenges: Occlusion slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson

56 Challenges: Background clutter slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson

57 Challenges: Intraclass variation slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson

An image classifier slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson Unlike e.g. sorting a list of numbers, no obvious way to hard-code the algorithm for recognizing a cat, or other classes. 58

59 Attempts have been made slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson

Data-driven approach: 1.Collect a dataset of images and labels 2.Use Machine Learning to train an image classifier 3.Evaluate the classifier on a withheld set of test images slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson 60

First classifier: Nearest Neighbor Classifier Remember all training images and their labels slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson Predict the label of the most similar training image 61

62 slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson

63 slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson

How do we compare the images? What is the distance metric ? slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson 64

Nearest Neighbor classifier slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson 65 Lecture 2 - Lecture 2 - 6 Jan 2016 6 Jan 2016 65

Nearest Neighbor classifier remember the training data 66 Lecture 2 - Lecture 2 - 6 Jan 2016 6 Jan 2016 66

Nearest Neighbor classifier slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson for every test image: - find nearest train image with L1 distance - predict the label of nearest training 67 Lecture 2 - Lecture 2 - 6 Jan 2016 6 Jan 2016 image 67

Nearest Neighbor classifier Q: how does the classification speed depend on the size of slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson the training data? 68 Lecture 2 - Lecture 2 - 6 Jan 2016 6 Jan 2016 68

Nearest Neighbor classifier Q: how does the classification speed depend on the size of the slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson training data? linearly :( 69 Lecture 2 - Lecture 2 - 6 Jan 2016 6 Jan 2016 69

Aside: Approximate Nearest Neighbor find approximate nearest neighbors quickly slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson 70 Lecture 2 - Lecture 2 - 6 Jan 2016 6 Jan 2016 70

71 slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson

k-Nearest Neighbor find the k nearest images, have them vote on the label slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson 72

K-Nearest Neighbor (kNN) • Given: Training data {( 𝑦 1 , 𝑧 1 ),…, ( 𝑦 n , 𝑧 n )} 
 – Attribute vectors: 𝑦 𝑗 ∈ 𝑌 
 – Labels: 𝑧 𝑗 ∈ 𝑍 ( 𝑦 ⃗ � , 𝑧 � , … , x � , 𝑧 � ) • 𝑦 ⃗ � ∈ 𝑌 – • Parameter: 
 𝑧 � ∈ 𝑍 – – Similarity function: 𝐿 ∶ 𝑌 × 𝑌 → R 
 • – Number of nearest neighbors to consider: k 𝐿 ∶ 𝑌 × 𝑌 ¡ → ¡ℜ – – • Prediction rule 
 • – New example 𝑦′ 
 – x’ – K-nearest neighbors: k train examples with largest 𝐿 ( 𝑦 𝑗 , 𝑦′ ) ⃗ � ) 𝐿(𝑦 ⃗ � , 𝑦 – slide by Thorsten Joachims 73

74 1-Nearest Neighbor slide by Thorsten Joachims

75 4-Nearest Neighbors slide by Thorsten Joachims

4-Nearest Neighbors Sign slide by Thorsten Joachims 76

4-Nearest Neighbors Sign For binary classification problems, 
 why is it a good idea to use an odd slide by Thorsten Joachims number of K ? 77

78 slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson

79 We will talk about this later! slide by Fei-Fei Li & Andrej Karpathy & Justin Johnson

If we get more data • 1 Nearest Neighbor - Converges to perfect solution if clear separation - Twice the minimal error rate 2 p (1- p ) for noisy problems • k-Nearest Neighbor - Converges to perfect solution if clear separation ( but needs more data ) - Converges to minimal error min( p , 1- p ) for noisy problems if k increases 80

Demo 81

Weighted K-Nearest Neighbor • Given: Training data {( 𝑦 1 , 𝑧 1 ),…, ( 𝑦 n , 𝑧 n )} 
 – Attribute vectors: 𝑦 𝑗 ∈ 𝑌 
 𝑦 ⃗ � , 𝑧 � , … , 𝑦 ⃗ � , 𝑧 � • – Target attribute 𝑧 𝑗 ∈ 𝑍 𝑦 ⃗ � ∈ 𝑌 – 𝑧 � ∈ 𝑍 – • Parameter: 
 • – Similarity function: 𝐿 ∶ 𝑌 × 𝑌 → R 
 𝐿 ∶ 𝑌 × 𝑌 ¡ → ¡ℜ – – Number of nearest neighbors to consider: k – • • Prediction rule 
 – x’ – New example 𝑦′ 
 ⃗ � 𝐿 𝑦 ⃗ � , 𝑦 – – K-nearest neighbors: k train examples with largest 𝐿 ( 𝑦 𝑗 , 𝑦′ ) 82

More Nearest Neighbors 
 in Visual Data 83

Where in the World? [Hays & Efros, CVPR 2008] A nearest neighbor 
 recognition example slide by James Hays 84

Where in the World? [Hays & Efros, CVPR 2008] slide by James Hays 85

Where in the World? [Hays & Efros, CVPR 2008] slide by James Hays 86

6+ million geotagged photos 
 by 109,788 photographers slide by James Hays Annotated by Flickr users 87

6+ million geotagged photos 
 by 109,788 photographers slide by James Hays Annotated by Flickr users 88

89 89 slide by James Hays

90 Scene Matches slide by James Hays

91 slide by James Hays

92 Scene Matches slide by James Hays

93 slide by James Hays

94 Scene Matches slide by James Hays

95 slide by James Hays

The Importance of Data slide by James Hays 96

Scene Completion [Hays & Efros, SIGGRAPH07] slide by James Hays 97

slide by James Hays … 200 total 98 Hays and Efros, SIGGRAPH 2007

Context Matching slide by James Hays 99 Hays and Efros, SIGGRAPH 2007

slide by James Hays Graph cut + Poisson blending 100 100 Hays and Efros, SIGGRAPH 2007