Lab 09
1. LDA: pset5
2. SVM: kernel trick
Linear Discriminant Analysis Problem set 5 example
Linear Discriminant Analysis
from Lectures
Linear Discriminant Analysis
Linear Discriminant Analysis
Slide Credit: Jia Li http://www.stat.psu.edu/∼jiali
Linear Discriminant Analysis
https://en.wikipedia.org/wiki/Pooled_variance
Slide Credit: Jia Li http://www.stat.psu.edu/∼jiali
Linear Discriminant Analysis
scikit-learn.org
Support Vector Machine
Key points:
• Difference between logistic regression and SVMs • Maximum margin principle
• Target function for SVMs
• Slack variables for misclassified points
• Kernel trick allows non-linear generalizations
slide credit: R. Urtasun
Support Vector Machine
https://cs.stanford.edu/people/karpathy/svmjs/demo/
CVXOPT User’s Guide
slide credit: R. Urtasun
slide credit: R. Urtasun
Kernel Trick
Kernel Trick
Kernel Trick
Kernel Trick
K – Gram matrix X – Design matrix
Kernel Trick
K – Gram matrix X – Design matrix
Mercer’s Theorem
Bishop, p. 296
Example Kernels
LDA – linearity
from Lectures
Unequal Priors
from Lectures
HW5 hints