Exercise – LR and Out of Sample Prediction
• Generate 99 independent variables uniformly distributed between -100 and 100 of size 100 observations each.
• Generate the dependent variable y = 3 + 10*V99, where V99 is the last covariate and add some noise
• Construct 3 models: one linear model with no variables, one with all the variables and one with only the variable V99
• Compute the MSE of each model
Hint: for the first two points code is provided below.
• Pick from your data only 1/5th random observations
• Use the remaining 4/5th observations to rebuild the three models
• Make prediction on the 1/5th observations
• What do you observe now?
Hint: for the first point code is provided.
Exercise – Part 2
Now:
• Build 99 different models including from 1 to 99 input variables on training data (4/5th observations)
• For each model compute the out-of-sample MSE on the remaining 1/5th (test data)
• Plot the out-of-sample MSE as a function of the number of variables
Hint: you may prefer to use a for-loop.
Ridge Regression
We are interested in predicting the level of alchol consumption during the weekend for students, controlling for many social and academic indicators. Some of them are the average grades for three years, the income of the family, the age, etc. In total we have 32 variables, but we want to find just the ones most correlated with alchol consumption.
We will explore the linear mode, the ridge regression and lasso.
Do the following:
• Download the student txt file
Note: the dependent variable is Walc (Week-end alchol consumption)
• Explore the variables and construct two different linear models. You can use any specification you think is most appropriate. Provide justifications.
• Report the interpretation of the coefficients
Ridge Regression:
• Construct a sequence of lambda from to
• Use cross validation to find the best lambda to be used for estimating ridge regression (use the skeleton provided in the hints of the previous exercises)
• Construct a ridge regression with the lambda with minimum error
Hint: Code for the first two points is provided.
Model comparison:
• Use cross validation to compare the linear models that you choose and the ridge regression.
• Do you think it is the correct way to compare the models?