COMP2610/COMP6261 – Information Theory Tutorial 4: Entropy and Information
Young Lee and
Tutors: and
Week 5 (21st – 25th August), Semester 2, 2017
Copyright By PowCoder代写 加微信 powcoder
1. Suppose Y is a geometric random variable, Y ~ Geom(y) . i.e., Y has probability function P(Y =y)=p(1−p)y−1, y=1,2,…
Determine the mean and variance of the geometric random variable.
2. A standard deck of cards contains 4 suits — ♥, ♦, ♣, ♠ (“hearts”, “diamonds”, “clubs”, “spades”) — each with 13 values — A, 2, 3, 4, 5, 6, 7, 8, 9, 10, J, Q, K (The A, J, Q, K are called “Ace”, “Jack”, “Queen”, “King”). Each card has a colour: hearts and diamonds are coloured red; clubs and spades are black. Cards with values J, Q, K are called face cards.
Each of the 52 cards in a deck is identified by its value v and suit s and denoted vs. For example, 2♥, J♣, and 7♠ are the “two of hearts”, “Jack of clubs”, and “7 of spades”, respectively. The variable c will be used to denote acard’scolour. Letf =1ifacardisafacecardandf =0otherwise.
A card is drawn at random from a thoroughly shuffled deck. Calculate:
(a) The information h(c = red, v = K) in observing a red King
(b) The conditional information h(v = K|f = 1) in observing a King given a face card was drawn.
(c) The entropies H(S) and H(V, S).
(d) The mutual information I(V ; S) between V and S.
(e) The mutual information I(V ; C) between the value and colour of a card using the last result and the data processing inequality.
3. Recall that for a random variable X, its variance is
Var[X] = E[X2] − (E[X])2.
Using Jensen’s inequality, show that the variance must always be nonnegative.
4. LetXandY beindependentrandomvariableswithpossibleoutcomes{0,1},eachhavingaBernoullidistribution
with parameter 21 , i.e.
p(X =0)=p(X =1)= 21 p(Y = 0) = p(Y = 1) = 12.
(a) ComputeI(X;Y).
(b) LetZ=X+Y.ComputeI(X;Y|Z).
(c) Do the above quantities contradict the data-processing inequality? Explain your answer.
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com