CS代考 Kullback–Leibler divergence

Kullback–Leibler divergence
A measure of how a probability distribution differs from another probability distribution
Consider two probability distributions P and Q, the relative entropy from Q to P (KL divergence) is often denoted DKL(P ∥ Q)
DKL(P∥Q) = 􏰭 x∈X

Copyright By PowCoder代写 加微信 powcoder

P(x)log􏰚P(x)􏰛 for discrete cases Q(x)
March 6, 2020 1/5

Stochastic Neighbor Embedding
High-dimensional Euclidean distance between datapoints xj and xi → conditional probability pj|i that represent similarity:
exp􏰚−∥xi −xj∥2/2σ2􏰛
p= 􏰚 􏰛 (1)
j|i 􏰭k̸=iexp −∥xi−xk∥2/2σ2
Low-dimensional Euclidean distance between representation yj and yi → conditional probability qj|i that represent similarity:
􏰚1+∥yi −yj∥2􏰛−1
exp􏰚−∥yi −yj∥2􏰛
j|i 􏰭 ij k̸=i exp −∥yi −yk∥2
􏰭k̸=l 1+∥yk −yl∥2 Minimize cost function C with gradient descent:
C = 􏰦KL(Pi||Qi) = 􏰦􏰦pji log pji i ijqji
TA March 6, 2020 2/5

Student t-distribution
ν+1 􏰚 􏰛−ν+1
Γ( 2 ) t2 2 2 −1
f(t)=√νπΓ(ν2) 1+ν →f(t,ν=1)=(1+t)
Figure 1: heavy tailed t-distribution VS Normal distribution
TA March 6, 2020 3/5

t-SNE algorithm
TA March 6, 2020 4/5

Figure 2: Compare t-SNE with other algorithms for MNIST dataset
Interactive demo for MNIST dataset
TA March 6, 2020 5/5

程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com