CS 570: Machine Learning
Problem Set 3: Due March 6
Consider Figure 6.10 in the book on page 141. Begin by generating sample points similar to the distribution shown in the figure. (You won’t be exact since there are no printed coordinates, but do your best to get fairly close). Use numpy’s array class to represent your data points. If you follow the book’s notation, then an individual example will be a column vector (here with two attributes). Thus, your array of data points should have 2 rows, and 13 columns.
1. (5 points) Plot your data set with two colors, one for each class along with the line in the direction of wguess = [2, 1].
2. (5 points) Project the points from your dataset onto the line wguess and rebuild the plot. Use colored X’s for the projected points. Your life may be easier if you work with the unit vector wˆguess.
3. (5 points) Compute Fisher’s Linear Discriminant for your two class set as described in the book on page 141 with equation 6.41:
J(w) = (m1 − m2)2 s 21 + s 2 2
4. (5 points) Using your vector wguess, perform a search (grid search will do) to find the unit vector that maximizes J(), plot this vector w∗, with the points projected on it as well as the original wguess (as a dashed line). Additionally, print out the vector w∗ and the value J(w∗).
5. (5 points) Use the scikit learn LinearDiscriminantAnalysis object with n components set to 1. Use the fit transform method to generate transformed points for the data. Note that scikit learn expects that an individual example will be a row vector, not a column vector as the book expects. Compute J on the transformed points produced by LDA. Does it match the value you obtained?
6. (5 points) Use scikit learn’s TruncatedSVD object with n components set to 1, to run PCA on the data. Again, calculate J for the new projected points and compare to your previous values. Use the object’s components instance variable to print the direction of the first principal component.
Submit your code via Blackboard.
Problem Set 3: Due March 6 – page 1 of 1