S u pport Vectors L IN E AR C L ASSIFIE R S IN P YTH ON Michael ( Mike ) Gelbart Instr u ctor , The Uni v ersit y of British Col u mbia
What is an SVM ? Linear classi � ers ( so far ) Trained u sing the hinge loss and L 2 reg u lari z ation LINEAR CLASSIFIERS IN PYTHON
S u pport v ector : a training e x ample not in the � at part of the loss diagram S u pport v ector : an e x ample that is incorrectl y classi � ed or close to the bo u ndar y If an e x ample is not a s u pport v ector , remo v ing it has no e � ect on the model Ha v ing a small n u mber of s u pport v ectors makes kernel SVMs reall y fast LINEAR CLASSIFIERS IN PYTHON
Ma x- margin v ie w point The SVM ma x imi z es the " margin " for linearl y separable datasets Margin : distance from the bo u ndar y to the closest points LINEAR CLASSIFIERS IN PYTHON
Ma x- margin v ie w point The SVM ma x imi z es the " margin " for linearl y separable datasets Margin : distance from the bo u ndar y to the closest points LINEAR CLASSIFIERS IN PYTHON
Let ' s practice ! L IN E AR C L ASSIFIE R S IN P YTH ON
Kernel SVMs L IN E AR C L ASSIFIE R S IN P YTH ON Michael ( Mike ) Gelbart Instr u ctor , The Uni v ersit y of British Col u mbia
Transforming y o u r feat u res LINEAR CLASSIFIERS IN PYTHON
Transforming y o u r feat u res LINEAR CLASSIFIERS IN PYTHON
Transforming y o u r feat u res transformed feature = (original feature) 2 LINEAR CLASSIFIERS IN PYTHON
Transforming y o u r feat u res transformed feature = (original feature) 2 LINEAR CLASSIFIERS IN PYTHON
Transforming y o u r feat u res transformed feature = (original feature) 2 LINEAR CLASSIFIERS IN PYTHON
Kernel SVMs from sklearn.svm import SVC svm = SVC(gamma=1) # default is kernel="rbf" LINEAR CLASSIFIERS IN PYTHON
Kernel SVMs from sklearn.svm import SVC svm = SVC(gamma=0.01) # default is kernel="rbf" smaller gamma leads to smoother bo u ndaries LINEAR CLASSIFIERS IN PYTHON
Kernel SVMs from sklearn.svm import SVC svm = SVC(gamma=2) # default is kernel="rbf" larger gamma leads to more comple x bo u ndaries LINEAR CLASSIFIERS IN PYTHON
Let ' s practice ! L IN E AR C L ASSIFIE R S IN P YTH ON
Comparing logistic regression and SVM L IN E AR C L ASSIFIE R S IN P YTH ON Michael ( Mike ) Gelbart Instr u ctor , The Uni v ersit y of British Col u mbia
Logistic regression : S u pport v ector machine ( SVM ): Is a linear classi � er Is a linear classi � er Can u se w ith kernels , b u t slo w Can u se w ith kernels , and fast O u tp u ts meaningf u l probabilities Does not nat u rall y o u tp u t probabilities Can be e x tended to m u lti - class Can be e x tended to m u lti - class All data points a � ect � t Onl y " s u pport v ectors " L 2 or L 1 reg u lari z ation a � ect � t Con v entionall y j u st L 2 reg u lari z ation LINEAR CLASSIFIERS IN PYTHON
Use in scikit - learn Logistic regression in sklearn : linear_model.LogisticRegression Ke y h y perparameters in sklearn : C ( in v erse reg u lari z ation strength ) penalty ( t y pe of reg u lari z ation ) multi_class ( t y pe of m u lti - class ) SVM in sklearn : svm.LinearSVC and svm.SVC LINEAR CLASSIFIERS IN PYTHON
Use in scikit - learn ( cont .) Ke y h y perparameters in sklearn : C ( in v erse reg u lari z ation strength ) kernel ( t y pe of kernel ) gamma ( in v erse RBF smoothness ) LINEAR CLASSIFIERS IN PYTHON
SGDClassifier SGDClassifier : scales w ell to large datasets from sklearn.linear_model import SGDClassifier logreg = SGDClassifier(loss='log') linsvm = SGDClassifier(loss='hinge') SGDClassifier h y perparameter alpha is like 1/C LINEAR CLASSIFIERS IN PYTHON
Let ' s practice ! L IN E AR C L ASSIFIE R S IN P YTH ON
Concl u sion L IN E AR C L ASSIFIE R S IN P YTH ON Michael ( Mike ) Gelbart Instr u ctor , The Uni v ersit y of British Col u mbia
Ho w does this co u rse fit into data science ? Data science → Machine learning →→ S u per v ised learning →→→ Classi � cation →→→→ Linear classi � ers ( this co u rse ) LINEAR CLASSIFIERS IN PYTHON
Congrat u lations & thanks ! L IN E AR C L ASSIFIE R S IN P YTH ON
Recommend
More recommend
