Npplieationsafcmd Finish Em properties
Factor Analysis EmAlyouthm
fog
RE
CtX CHD LAST TIME
2 4 04
Lto’t
fight
TACTORANn.ly GAUSSIAN MIXTURE MODEL AS CM
LIQ
Ig log
QiL2
QiHP
Qicz log pcxcil.z.jo
cute
Qiu
4Lot
WE SHOWED Property 1 I O 7 4 Q
log
key STEP IS JENSEN
of 3 Ii Qicz log Plx jo1
TO SET Property 2 we pick Qiczy o.us
Since
PCZ O ow 2 Logc
log C Property 2 holds
Qicz Plz
xD
Qicz
Qicts ZQ
ELBOW
P Zlx iF o does not depend
VERIFY
LHS ELBOWS
P Zj0 Qila
Qicz c Qitel lose
log
Drs z Qi ell se s p Y RESTATEEM
o
CESTEP for eI n set Qicz Plz IX 0t
m step ft
WARMUp_ Mixtureof GaussANS EM RECOVERS OUR AD40CAlgorithm
WE
could compute Via idk
Bayes
Rule PIX
j D
M
Yo
S
Qilz7
SAW that
Muey more likely for than But if WE KNEW 0 227
t 2
Got
Leto
DATA Input Params
Argnfax
Argifax IfELBOCX Q
Q
O 1 Inclusterj cluster means
PCxci za PCx 20 pcz.ci
2
2 is our
W4ATISEMHE.EE
latent VARIABLE
PCZ’i
n Multinomial E di o X I z j N N try of
j xci7jo
I 2
MAYBE WE’dthinklikely from ROLE AUTOMATES thisREASONING
not
Bayes
0 i Compete Derivatives
Eci Qitz tlogpcxcis.ec f a Qilz’D
c WRITE0 fornonanonAbove
Rsee WjQil2j
Pex2go pXliIzciPza
Gaussian CaE
fico Way logfzittqikexpftzlxcis ig.IE jlxli u 3 d
way Teej Efile Teej why I Cx a F
I w EjCx u Skiingto 0 AND usingEj is fullrank
x ie
IEj Cx nD
w x Mj o
Kj
constrained
w age’s
dj 7
CAS Beeore
dy 20 NEED LAGRANGIAN
dy is 00J
Bince i
MEISACLE
NB If Z Is continuous youCAN Reecnee SUMSW Integrals
5
II 1 dj I
logdjttg.ME i
w
log
EM Recovers GMM
Automatically
was
I
I
do IEwin Wj
dog
o
fs gRunWltgs FaetorANalysis
need Ef Gmms nod Lydsof Raccoons fewsources
MANYfewer pants 7hAM dimensions Howdoes this happen
PLACE SENSORS All OVER CAMPUS RECORD 1000s allocations d N 10005
But Only record for 30 days Ncd WANT TO FIT A Density but seems hopeless
KEYI DEA Assume there IS SOME LATENT V.V that IS NOT too complex ANI Explains behavior
1Stlet’s
IX
see Problems w GMMs Even 1 Gaussian
¦Ì
this is Ok IIIx aCxest
RANK Ei ENL d Not Fole RANK Probed IN Gaussianlikelihood
Phx ¦ÌE III ik exp k
IS NOTDEFINED exustE x ers
0
WE will fix these Issues by Examining three Models
that Are Simpler Speke WE’ll COMBINEthese intheEND Recall ME for Gaussian
MIE
mining
f
exrEtzlxuTE.com EI exuTECx u x log181
IE dog Equivalent
If isfullrankOnIIExie o ¦ÌIn
we’ll
BuildngB.la c1
Suppose INDEPENDENT And identical covariance
COVARIANCE ARE circles
io
USE this AS PLUGIN Below
let
202 myIC tdlog2 ifEC’tn o ten
02 IT4
Subtraet MEAN AND SQUARE All Entries
x ¦Ì5cx ie
I
Bwedngblock2
z.fi
O
SET Zi
of SAME IDEA
Axis Alegwerselapse AS ABOVE
this is
d problems for EAe4 l dimension
T nmodeTPARAMETER.SU
Mz z
II II
ZI’t A5
logz
II zj
of
o
estt lo z
Xi
Cx
a5
C Rd N CRd’s
OI e RdW MODEL
DiagonalMATRIX
P HZ PLZ
X ¦ÌtNZtC on xnNieNzET
P
latent
x z z
2 Is
N o I E RS fore Sed smaeedim
MAPS From small latentSPACE to large SPACE H OI
MEAN IN
the nee
l GENERATE 2l 2 fromNCOD
C N LO Noisy D251n5 X¦ÌtNZtE
I1
2 Suppose N
3 Add¦Ì 4 Add c
BigSPACE
X
Cs
is
zoo
u Nz
21 En
zu zest za
WE would OBSERVE ARE PurpleDots
So Small LATENT SPACE PRODUCES DATA IN high dim SPACE
TECHNieal.TO
Block GAUSSIANS
DATA
7 zoo
8
cRd de de Idi
69L EEL wz cRdixdii.jc.si23
I
x c Rd xcR
IS widelyOSED AND helpful For GaussIANS PLA HIM
NOTATION
FAET 1 PCx PlexXz
marginalization Guotsurprising
conditioning
Fact PIX Xz HIMk
42 iz¦Ìtzzz Uz
th e Xz
112 z z Proofsowline 4APPY to Add
matrixInversionlemma
Hang
BACK.to ncTOR ANalqss
E n Wheat is
N IL E EzzT I
EEZ
Gaussian Another GAUSSIAN CLOSED
Marginalization Conditioning
WE HAVE formula for PARAMETERS
t NZ
u
ElzztnTJtE.fzfEJ
Eu E Lxa Lxust
E IN2 te Nztef
II Nzzth t E EE
NRT t Intel
t.EC C.xei5I
AT
X ¦Ì SINCE
t E
_o Ex
0
E
I
IE
QIilZ Summary
P Z IIX o USE conditional MSTEI WE HAVE CLOSED forms
E.si
WE SAW that EM CAPTURES GMM
WE LEARNED ABOUT FACTORAnalysis flattenslowdim STRUCTURE
WE SAW How to ESTIMATE PARAMETERS OF FA Using EM