SLIDE 1
Support Vector Machines
Alex Leblang and Sam Birch
ML Framework
Data projected into feature space, each feature is a dimension.
Via Wikipedia, scikit-learn examples
Support Vector Machines Alex Leblang and Sam Birch ML Framework - - PowerPoint PPT Presentation
Support Vector Machines Alex Leblang and Sam Birch ML Framework - - PowerPoint PPT Presentation
Support Vector Machines Alex Leblang and Sam Birch ML Framework Data projected into feature space, each feature is a dimension. Via Wikipedia, scikit-learn examples SVMs Supervised learning model: train / test Binary, discriminative
SLIDE 2
SLIDE 3
SVMs
- Supervised learning model: train / test
- Binary, discriminative classifier
○ Models the boundary, not the data
- Linear classification boundary defined with
support vectors
- Non-linear decision boundaries can be
introduced via the kernel trick
SLIDE 4
Where SVMs fit (Hays)
SLIDE 5
Linear separators (Hays)
SLIDE 6
Non-linearity (Andrew Moore via Hays)
SLIDE 7
Kernel Trick
SLIDE 8
Demo
MLDemo
SLIDE 9
Challenges
- Non-linear classification takes > O(n) time
○ Have to compute entire kernel matrix ○ Most large-scale classification tasks use linear SVMs
- Validation
- Feature choice is challenging in many
domains
SLIDE 10
LibSVM
An efficient implementation of SVM routines Functions in LibSVM:
- C-Support vector classification
- V-Support vector classification
- Distribution Estimation
- v-Support vector regression
SLIDE 11
Optimization
Moving training and classification to the GPU can provide 1-2 orders of magnitude performance advantage “Fast Support Vector Machine Training and Classification on Graphics Processors” (Catanzaro, Sundaram, Keutzer)
SLIDE 12
Conclusion
- Solid go-to classification algorithm
- Features > specifics
- Tuning and validation make a big difference