SLIDE 1
Basics Machine Learning
Blood Pressure Sugar Hyper Tension 10 15 No 5 5 No 25 25 Yes 30 36 Yes
5 10 15 20 25 30 35 5 10 15 20 25 30 35 40
Y-Values
Learning Structural SVMs with Latent Variables Presented By- - - PowerPoint PPT Presentation
Learning Structural SVMs with Latent Variables Presented By- - - PowerPoint PPT Presentation
Learning Structural SVMs with Latent Variables Presented By- Subhabrata Debnath(Roll- 13111063) Anjan Banerjee(Roll-13111008) Basics Machine Learning Blood Sugar Hyper Pressure Tension 10 15 No 5 5 No 25 25 Yes 30 36
SLIDE 2
SLIDE 3
Basics Machine Learning
w Support Vectors
Y1 Y2
SLIDE 4
Objective Function of SVM
s.t.
SLIDE 5
Objective Function of SVM
s.t.
SLIDE 6
Score
w
Higher the value of WTx,higher the chance of belonging to this class
SLIDE 7
SLIDE 8
Latent Structured SVM
Final Objective Function:
Non-Convex Objective Function Can be solved by CCCP
SLIDE 9
Soft-Margin SVM
SLIDE 10
Soft-Margin SVM
s. t.
SLIDE 11
Soft-Margin SVM
SLIDE 12
Multiclass SVM
wY1 w
Y 2
wY3
Predicted Class:
Y1 Y3 Y2
SLIDE 13
Multi-Class SVM
s.t.
here,
SLIDE 14
Multi-class SVM
- What if we don’t want the same
amount of margin for all the classes?
- E.g.: Given age, sex of an user and the movie
genre, predict the rating(1-5) that the user will give.
- Highly Incorrect Class and Lesser Incorrect Class
Actual Rating Predicted Rating Loss 5 4 Less 5 1 High
SLIDE 15
Multi-Class SVM
s.t.
SLIDE 16
Multi-Class SVM
s.t.
SLIDE 17
Structured SVM
SLIDE 18
Structured SVM
SLIDE 19
Structured SVM
SLIDE 20
Structured SVM
SLIDE 21
Structured SVM
SLIDE 22
Structured SVM
Could have been solved using any convex solver The only problem is the number of classes, hence the number of constraints are exponentially large. e.g. Number of possible parse trees for a given sentence is exponential in the number of words.
SLIDE 23
Cutting Plane Method
However, this method gives a solution of the given convex
- ptimization problem with
precision ε. Our Convex Objective Function W*, ξi Cutting Plane Method
SLIDE 24
Latent Information
Hidden Information present in the training set that can improve our learning Let us denote these hidden/latent information as hi. xi yi hi (given/observed)
(hidden/unobserved)
SLIDE 25
Latent Information
Noun Phrase Coreference Problem:
Input x: Noun Phrases with edge features Labels y: Clusters Of Noun Phrases Latent Variable h: ‘Strong’ links as trees
SLIDE 26
Latent Information
Noun Phrase Coreference Problem:
SLIDE 27
Latent Structured SVM
Objective function:
SLIDE 28
Latent Structured SVM
Objective function:
SLIDE 29
Latent Structured SVM
Objective function:
SLIDE 30
Latent Structured SVM
Objective function:
SLIDE 31
Latent Structured SVM
Final Objective Function:
Non-Convex Objective Function Can’t be solved using Cutting plane
SLIDE 32
Property Of Convex Function
x1 x2 f x f(x1) f(x2) f ’ ( x 1 )
f ( x2 ) >= f ( x1 ) + ( x2 - x1 ) * f’ ( x1 )
x1 x2 f(x1) f(x2)
f ( x1 ) <= f ( x2 ) + ( x1 – x2 ) * f’ ( x2 )
f ’ ( x 2 )
Property Of Concave Function
SLIDE 33
Concave-Convex Procedure
SLIDE 34
Concave-Convex Procedure
SLIDE 35
Concave-Convex Procedure
SLIDE 36
Concave-Convex Procedure
Minimize the resulting sum
Cutting plane Algorithm
Wt+1 Iterate till desired precision
SLIDE 37
Overview of the CCCP
- Initialize w0
repeat
–Find h* using the wi –Obtain wi+1 by optimizing the convex function using cutting plane. –Set wi=wi+1
till objective function improves by at least ε
SLIDE 38