QF5208 – Project 1
• The project is due on 14 March 2021. Submit your report on LumiNUS.
• Choose Project 1a or Project 1b. Only one of the two projects (1a, 1b) is required.
• This is an individual project. Each student needs to work on his/her own and write a report.
• The report should include the method and details of the numerical result. At- tach the computer program after the report.
Project 1a
Download the dataset (“image-data”) from LumiNUS. This dataset consists of 1000 images of two hand-written digits: the first 500 images (from “id 1.png” to “id 500.png”) are for the digit “3” and the second 500 images are for another digit “4”. Each image has 28 × 28 pixels (an array of length 784). You may arrange the images in a data matrix using the following MATLAB code:
Note: After the loop, X should be a 784 × 1000 matrix with the first 500 columns for the digit “3” and the second 500 columns for the digit “4”.
(a) Apply the principal component analysis (PCA) to project the data to the space of the first two principal components.
(b) UsetheGaussianmixturemodeltoseparatetheprojecteddataintotwoclusters.
Plot the projected data and mark the two clusters obtained in (b) using different colors or markers. In another figure, plot the same projected data but mark the data points based on the true label (e.g. blue for one digit and red for the other digit).
Compute the error of the clustering, defined as
1 n
e(h) = n
where n = 1000, h is the classifier based on the Gaussian mixture model, yi is the true label of xi, 1h(xi)̸=yi equals 1 if h(xi) ̸= yi and 0 otherwise.
1h(xi)̸=yi, (1)
i=1
Project 1b
A gambler has the opportunity to make bets on the outcomes of a sequence of coin flips. If the coin comes up the head, he wins as many dollars as he has staked on that flip; if it is the tail, he loses his stake. The game ends when the gambler wins by reaching his goal of $100 or loses by running out of money. On each flip, the gambler must decide what portion of his capital to stake, in integer numbers of dollars.
The state is the gambler’s capital, s ∈ {0, 1, 2, . . . , 100}, where 0 and 100 are terminal states. The actions are stakes, a ∈ {1, . . . , min(s, 100 − s)}. The reward is zero on all transitions except those on which the gambler reaches his goal, when it is +1. The discount factor is one. The state-value function gives the probability of winning from each state. The optimal policy maximizes the probability of reaching the goal.
Let ph denote the probability of the coin coming up the head. Use value iteration to find the optimal policy for different cases of ph:
(i) ph =0.3, (ii) ph =0.4, (iii) ph =0.6
For each case, plot the final value function V∗(s) and the optimal policy π∗(s) as functions of the states s. If there are ties in the argmax action selection with respect to the optimal value function, arbitrarily choose one of those actions.
In the value iteration, you may fix values of the terminal states s = 0 and s = 100 at 0 and 1, respectively. The iteration is terminated when the change in the value function over successive sweep is less than a small number (e.g. 10−3).