CSC311 Tutorial 4: Optimization for Machine Learning
University of Toronto
September 30, 2021
1based on slides by , , , , Shenlong Wang and others
Copyright By PowCoder代写 加微信 powcoder
!
! !
!
!
! !
θ∗ = θ(θ) θ ! θ ∈ R
! : R → R
(θ) −(θ)
! θ
! θ
!
θ
! (, ) (|, θ)
! −log(|,θ)
!
∂(θ∗) =
! θ !
! ∇θ=(∂ , ∂ ,…, ∂ ) ∂θ ∂θ ∂θ
η ! θ
! =:
! δ ← −η∇θ−
! θ←θ−+δ
η
! θ ! =:
! η (θ − η∇θ− ) < (θ) ! δ←−η∇θ−
! θ←θ−+δ
α ∈ [, )
! θ
! δ ! =:
! δ ← −η∇θ− +αδ− ! θ←θ−+δ
α
η ! θ
! δ ← −η∇θ−
! θ←θ−+δ !
! |(θ+) − (θ)| < ε
! ∥∇θ∥ < ε
!
! ∇
!
∂ ≈ ((θ,...,θ +ε,...,θ))−((θ,...,θ −ε,...,θ))
!
!
!
!
!
!
!
!
!
!
!
!
θ θ ∈ [, ]
(θ + ( − )θ) ≤ (θ) + ( − )(θ)
! αα≥
! +
!
!
!
! () () () ()
(θ)=− log(=| ,θ)+(− )log(=| ,θ) − log σ(θ)
!
!
!
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com