PowerPoint Presentation
Lecture 5: Gradient Descent
Basic algorithm for gradient descent:
BASIC-GRADIENT-DESCENT(f)
i = 0
x0 = arbitrary point
while f′(xi) !≈ 0 do
if f′(xi) > 0 : move left
if f′(xi) < 0 : move right
i = i + 1
if f′(xi) ≈ 0 then x is a local extremum
f′(xi) ≈ 0 is local optimality condition.
Gradient Descent: Tangent lines (red) of f at successive points xi (green) (MathWorks image)
Nonconvex objective function: different starting points can give different local minima.
Convex function on an interval.
/docProps/thumbnail.jpeg