2020/11/25 COMP9444 Exercises 3
COMP9444 Neural Networks and Deep Learning Term 3, 2020
Exercises 3: Probability This page was last updated: 09/30/2020 07:15:12
1. Bayes’ Rule
One bag contains 2 red balls and 3 white balls. Another bag contains 3 red balls and 2 green balls. One of these bags is chosen at random, and two balls are drawn randomly from that bag, without replacement. Both of the balls turn out to be red. What is the probability that the first bag is the one that was chosen?
2. Entropy and Kullback-Leibler Divergence
Consider these two probability distributions on the same space Ω = {A, B, C, D}
p = ⟨ 1⁄2, 1⁄4, 1⁄8, 1⁄8 ⟩ q = ⟨ 1⁄4, 1⁄8, 1⁄8, 1⁄2 ⟩
a. Construct a Huffmann tree for each distribution p and q
b. Compute the entropy H(p)
c. Compute the KL-Divergence in each direction DKL(p || q) and DKL(q || p). Which one is larger? Why?
Make sure you attempt the questions yourself, before looking at the Sample Solutions.
https://www.cse.unsw.edu.au/~cs9444/20T3/tut/Ex3_Probability.html 1/1