Kullback–Leibler divergence
A measure of how a probability distribution differs from another probability distribution
Consider two probability distributions P and Q, the relative entropy from Q to P (KL divergence) is often denoted DKL(P ∥ Q)
DKL(P∥Q) = x∈X
Copyright By PowCoder代写 加微信 powcoder
P(x)logP(x) for discrete cases Q(x)
March 6, 2020 1/5
Stochastic Neighbor Embedding
High-dimensional Euclidean distance between datapoints xj and xi → conditional probability pj|i that represent similarity:
exp−∥xi −xj∥2/2σ2
p= (1)
j|i k̸=iexp −∥xi−xk∥2/2σ2
Low-dimensional Euclidean distance between representation yj and yi → conditional probability qj|i that represent similarity:
1+∥yi −yj∥2−1
exp−∥yi −yj∥2
j|i ij k̸=i exp −∥yi −yk∥2
k̸=l 1+∥yk −yl∥2 Minimize cost function C with gradient descent:
C = KL(Pi||Qi) = pji log pji i ijqji
TA March 6, 2020 2/5
Student t-distribution
ν+1 −ν+1
Γ( 2 ) t2 2 2 −1
f(t)=√νπΓ(ν2) 1+ν →f(t,ν=1)=(1+t)
Figure 1: heavy tailed t-distribution VS Normal distribution
TA March 6, 2020 3/5
t-SNE algorithm
TA March 6, 2020 4/5
Figure 2: Compare t-SNE with other algorithms for MNIST dataset
Interactive demo for MNIST dataset
TA March 6, 2020 5/5
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com