1/24/2021 https://ilykei.com/course/5655560d598e6616003f1986/assignments/5696a41b3e7daf1800b711f6/assignment
Home (/) / Linear and Nonlinear Models (MScA, 31010) (/course/5655560d598e6616003f1986) / Week 2 Assignment
Week 2 Assignment
Submissions
Due 1/27/21 9:59 PM
iLykei Lecture Series
University of Chicago
Linear and Non-Linear Models (MScA, 31010)
Week 2 Assignment: Optimization © Yuri Balasanov, iLykei 2014-2021
1 Code Your Own Optimizer in R 1.1 Writing your optimizer 1.2 Checking optimizer
1.3 Running your optimizer
2 Test
This project helps understanding how non-linear optimization of log-likelihood function by the Newton-Raphson method works
This assignment is individual
1 Code Your Own Optimizer in R
Use the formulas for the steps of Newton-Raphson method from the lecture notes to code a simple version of optimizer.
The goal of your optimizer is to find a root (crossing of X-axis) of a given function.
When applied to maximum likelihood estimation this function is going to be derivative of log likelihood function with respect to the estimated parameter: in order to find maximum of a function find root of its derivative.
1.1 Writing your optimizer
Optimizer is a function that takes initial guess and an arbitrary objective function name and returns a point on X-axis where the objective functions equals zero.
Let the declaration of the optimizer function be myOptimizer<- function(startValue,Function_To_Optimize,Epsilon,projectID) where
startValue is the initial guess for the optimizer
Function_To_Optimize is the name of your test function that needs to be optimized
Epsilon is the stopping criterion, a small number, for example 0.0001
projectID is the reserved argument for individual test function in the test below. Parameter projectID will be
passed to the test function.
The function myOptimizer should repeat iterations of the Newton-Raphson algorithm while , where is the approximation obtained during the recent iteration and is the approximation obtained during the previous iteration.
You can use any of the loops in R language, for example, while(cond) expr where cond is the condition of moving to the next iteration ( ) and expr is the sequence of the commands that need to be performed at each iteration.
1.2 Checking optimizer
To check your optimizer create a test function that needs to be optimized.
In this project we use one-dimensional optimization, i.e. optimization with respect to only one variable.
Add one more argument to the function, called projectID . The meaning of it will become clear in section Test.
Let the declaration of the function be my_Function<-function(my_X, projectID) , where my_X is a scalar parameter with respect to which the optimization is done. Argument projectID is not going to be used until next section.
The function should cross x-axis at least in one point. For example, you can use
Documents in Week 2 Assignment
Assignment
my_Function<-function(my_X,projectID) { my_X^2*3-my_X*5-6
}
https://ilykei.com/course/5655560d598e6616003f1986/assignments/5696a41b3e7daf1800b711f6/assignment 1/3
1+ix ε => |ix − 1+ix|
ix
ε => |ix − 1+ix|
1/24/2021 https://ilykei.com/course/5655560d598e6616003f1986/assignments/5696a41b3e7daf1800b711f6/assignment
Plot this function.
X<-seq(from=-5,to=5,by=.1) Y<-my_Function(X) plot(X,Y,type="l") abline(h=0)
Note that for test you will be given different function to optimize. That function will use your personal project identificator projectID as an argument.
Be prepared to use an unfamiliar function with your optimizer.
1.3 Running your optimizer
Use your optimizer with the test function and initial approximation.
For example, myOptimizer(-3,my_Function,Epsilon=.0001) . Make sure you calculate the answer manually to verify the answer.
You can also test the optimizer by running uniroot() . For example,
uniroot(my_Function,lower=-5,upper=+1)
## $root
## [1] -0.8081429 ##
## $f.root
## [1] -1.138463e-06 ##
## $iter
## [1] 9 ##
## $init.it
## [1] NA ##
## $estim.prec
## [1] 6.103516e-05
Try using your optimizer with different start values.
The root returned by your optimizer should be almost the same as the output $root of the object returned by uniroot() .
Try also to run optim() . Explain the difference between the results of two functions: uniroot() and optim() . How can we reconsile these results?
2 Test
Download test function (http://ilykei.com/api/fileProxy/documents%2FMScA%20Linear%20and%20Non- Linear%20Models%2031010%2FMScA%2031010%20Lecture%202%2FMScA_Nonlinear_Models_Week2_TestFunction.rds) that will be used to test your optimizer.
https://ilykei.com/course/5655560d598e6616003f1986/assignments/5696a41b3e7daf1800b711f6/assignment 2/3
1/24/2021 https://ilykei.com/course/5655560d598e6616003f1986/assignments/5696a41b3e7daf1800b711f6/assignment
Contact admin@ilykei.com (mailto:admin@ilykei.com) with any questions or problems.
Test 2
© Yuri Balasanov, Alexander Doynikov, iLykei 2017
Email:
DataStream Password:
Sign In
https://ilykei.com/course/5655560d598e6616003f1986/assignments/5696a41b3e7daf1800b711f6/assignment 3/3