CSE 6363 – Machine Learning Homework 1: MLE, MAP, and Basic Supervised Learning
MLE and MAP
The Gamma distribution is:
βα α−1 −βλ pα,β(λ) = Γ(α)λ e
CSE 6363 – Machine Learning Homework 1- Spring 2019
Due Date: Feb. 8 2019, 11:59 pm
1. In class we covered the derivation of basic learning algorithms to derive a model for a coin flip task. Consider a similar problems where we monitor the time of the occurrence of a severe computer failure (which requires a system reboot) and which occurs according to a Poisson process (i.e. it is equally likely to happen at any point in time with an arrival rate of λ ). For a Poisson process the probability of the first event to occur at time x after a restart is described by an exponential distribution:
pλ(x) = λe−λx
We are assuming here that the different data points we measured are independent, i.e. nothing changes
between reboots.
a) Derive the performance function and the optimization result for analytic MLE optimization for a model learning algorithm that returns the MLE for the parameter λ of the model given a data set D = {k1,…kn}. Make sure you show your steps.
b) Apply the learning algorithm from a) to the following dataset:
D = {1.5, 3, 2.5, 2.75, 2.9, 3} .
c) Derive the optimization for a MAP approach using the conjugate prior, the Gamma distribution.
Note that α and β are constants and that there still is only one parameter, λ, to be learned. Show your derivation and the result for the data in part b) and values for α and β of 5 and 10, respectively.
K Nearest Neighbor
2. Consider the problem where we want to predict the gender of a person from a set of input parameters, namely height, weight, and age. Assume our training data is given as follows:
2019 Manfred Huber Page 1
CSE 6363 – Machine Learning Homework 1: MLE, MAP, and Basic Supervised Learning
D={ ((170, 57, 32), W), ((192, 95, 28), M), ((150, 45, 30), W), ((170, 65, 29), M), ((175, 78, 35), M), ((185, 90, 32), M), ((170, 65, 28), W), ((155, 48, 31), W), ((160, 55, 30), W), ((182, 80, 30), M), ((175, 69, 28), W), ((180, 80, 27), M), ((160, 50, 31), W),
((175, 72, 30), M), }
a) Using Cartesian distance as the similarity measurements show the results of the gender prediction for the following data items for values of K of 1, 3, and 5. Include the intermedia steps (i.e. distance calculation, neighbor selection, prediction).
(155, 40, 35), (170, 70, 32), (175, 70, 35), (180, 90, 20)
b) Implement the KNN algorithm for this problem. Your implementation should work with different
training data sets and allow to input a data point for the prediction.
c) Repeat the prediction using KNN when the age data is removed. Try to determine (using multiple target values) which data gives you better predictions. Show your intermediate results.
Gaussian Na ̈ıve Bayes Classification
3. Using the data from Problem 2, build a Gaussian Na ̈ıve Bayes classifier for this problem. For this you have to learn Gaussian distribution parameters for each input data feature, i.e. for p(height|W ), p(height|M ), p(weight|W ), p(weight|M ), p(age|W ), p(age|M ).
a) Learn/derive the parameters for the Gaussian Na ̈ıve Bayes Classifier and apply them to the same target as in problem 2b). Show your intermediate steps.
b) Implement the Gaussian Na ̈ıve Bayes Classifier for this problem.
c) Repeat the experiment in part 2c) with the Gaussian Na ̈ıve Bayes Classifier.
d) Compare the results of the two classifiers and discuss reasons why one might perform better than the other.
2019 Manfred Huber Page 2