CSCI 5521: Introduction to Machine Learning (Spring 2022)1
Homework 1
Due date: Feb 16, 2022 11:59pm
1. (30 points) Find the Maximum Likelihood Estimation (MLE) of θ in the following probabilistic density functions. In each case, consider a random sample of size n. Show your calculation:
Copyright By PowCoder代写 加微信 powcoder
(a) f(x|θ) = x exp{−x2 },x ≥ 0 θ2 2θ2
(b) f(x|α,β,θ)=αθ−αβxβexp{−(xθ)β},x≥0,α>0,β>0,θ>0
(c) f(x|θ) = 1θ,0 ≤ x ≤ θ,θ > 0 (Hint: You can draw the likelihood function)
2. (30 points) We want to build a pattern classifier with continuous attribute using Bayes’ Theorem. The object to be classified has one feature, x in the range 0 ≤ x < 6. The conditional probability density functions for each class are listed below:
16 if0≤x<6 P (x|C1) = 0 otherwise
4(x−1) if1≤x<3
P(x|C2)= 14(5−x) if 3 ≤ x <5 0 otherwise
(a) Assuming equal priors, P (C1) = P (C2) = 0.5, classify an object with the attribute value x = 2.5.
(b) Assuming unequal priors, P(C1) = 0.7,P(C2) = 0.3, classify an object with the attribute value x = 4.
1Instructor: Zhao. TA: Botcha, , , , . Email:
P(x|C2) P(x|C1)
(c) Consider a decision function φ(x) of the form φ(x) = (|x − 3|) − α with one free parameter α in the range 0 ≤ α ≤ 2. You classify a given input x as class 2 if and only if φ(x) < 0, or equivalently 3−α < x < 3+α, otherwise you choose x as class 1. Assume equal priors, P(C1) = P(C2) = 0.5, what is the optimal decision boundary - that is, what is the value of α which minimizes the probability of misclassification? What is the resulting probability of misclassification with this optimal value for α? (Hint: take advantage of the symmetry around x = 3.)
3. (40 points) In this programming exercise you will implement three multivariate Gaus- sian classifiers, with different assumptions as follows:
• Assume S1 and S2 are learned independently (learned from the data from each class).
• Assume S1 = S2 (learned from the data from both classes).
• Assume S1 = S2 (learned from the data from both classes), and the covariance is
a diagonal matrix.
What is the discriminant function in each case? Show in your report and briefly explain.
For each assumption, your program should fit two Gaussian distributions to the 2-class training data in training data.txt to learn m1, m2, S1 and S2 (S1 and S2 refer to the same variable for the second assumption). Then, you use this model to classify the test data in test data.txt by comparing log P(Ci|x) for each class Ci, with P(C1) = 0.3 and P(C2) = 0.7. Each of the data files contains a matrix M ∈ RN×9 with N samples, the first 8 columns include the features (i.e. x ∈ R8) used for classifying the samples while the last column stores the corresponding class labels (i.e. r ∈ {1, 2}).
Report the confusion matrix on the test set for each assumption. Briefly explain the results.
We have provided the skeleton code MyDiscriminant.py for implementing the clas- sifiers. It is written in a scikit-learn convention, where you have a fit function for model training and a predict function for generating predictions on given samples. Use Python class GaussianDiscriminant for implementing the multivariate Gaussian classifiers under the first two assumptions, and GaussianDiscriminant Diagonal for the third one. To verify your implementation, call the main function hw1.py, which automatically generates the confusion matrix for each classifier. Note that you do not need to modify this file.
Submission
• Things to submit:
1. hw1 sol.pdf: a document containing all your answers for the written questions (including those in problem 3).
2. MyDiscriminant.py: a Python source file containing two python classes for Prob- lem 3, i.e., GaussianDiscriminant
and GaussianDiscriminant Diagonal. Use the skeleton file MyDiscriminant.py found with the data on the class web site, and fill in the missing parts. For each class object, the fit function should take the training features and labels as in- puts, and update the model parameters. The predict function should take the test features as inputs and return the predictions.
• Submit: All material must be submitted electronically via Gradescope. Note that There are two entries for the assignment, i.e., Hw1-Written (for hw1 sol.pdf) and Hw1-Programming (for a zipped file containing the Python code), please submit your files accordingly. We will grade the assignment with vanilla Python, and code submitted as iPython notebooks will not be graded.
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com