LECTURE 2 TERM 2:
MSIN0097
Predictive Analytics Video 6: Regression
A P MOORE
CL ASSIFIC ATION
A. ClAssification B. Regression
C. Clustering D. Decomposition
Supervised
Unsuper vised
END- TO-END
— Discover — Explore — Visualize
— Clean
— Sample — Impute — Encode — Transform — Modeling
– Overfitting
– ModelSelection
— Documentation — Presentation
— Launch — Monitor — Maintain
– Learning curves – Regularization
– Degrees of freedom – Generalization
B. REGRESSION REAL VALUED VARIABLE
LINEAR REGRESSION REAL VALUED VARIABLE
LINEAR REGRESSION
Measured data
Features Inferred/Predicted/Estimated value
Trueinitialvalue𝑥 →𝑥#→𝑓 𝑥 =𝑦# →𝑦
(world state)
predicted value
Learned/Fitted function From n observations
True target value (world state)
hypothesis function
feature vector
T
parameter vector
COST FUNCTION
Closed-form solution— Normal Equation
POLYNOMIAL REGRESSION
POLYNOMIAL REGRESSION
DEGREES OF FREEDOM
LEARNING CURVES
10TH DEG POLYNOMIAL
LEARNING CURVES
BIAS-VARIANCE TRADEOFF
Bias
— due to wrong assumptions e.g. data is linear when it is actually quadratic. — A high-bias model is most likely to underfit the training data.
Variance
— data dependency
— model’s excessive sensitivity to small variations in the training data. — A model with many degrees of freedom will overfit the training data.
— Irreducible error
— noisiness of the data
— Change, improve the measurement process
REGUL ARIZ ATION
REGUL ARIZ ATION
— Reduce the number of parameters
— Ridge Regression — Lasso Regression — Elastic Net
RIDGE (REGULARIZED) REGRESSION
LASSO (REGULARIZED) REGRESSION
LECTURE 2 TERM 2:
MSIN0097
Predictive Analytics Video 6: Regression
A P MOORE