LECTURE 2 TERM 2:
MSIN0097
Predictive Analytics Video 4: Optimisation
A P MOORE
MACHINE LEARNING
Data + modelàprediction
MACHINE LEARNING DATA DRIVEN AI
Assume there is enough data to find statistical associations to solve specific tasks
Data + modelàprediction Define how
to maximize performance
adapt the parameters
well the model solves the task and
A – B – C- D
A TAXONOMY OF PROBLEMS
A. ClAssification B. Regression Week 2 – Classification and Regression
Week 3 – Trees and Ensembles
C. Clustering D. Decomposition
Week 5 – Clustering
Week 4 – Kernel spaces and Decomposition
END- TO-END
— Discover — Explore — Visualize
— Clean
— Sample — Impute — Encode — Transform
– Scale — Features — Pipelines
— Documentation — Presentation
— Launch — Monitor — Maintain
— Training/Validation splits — Modeling
— Tuning
— Error Analysis
END- TO-END
— Discover — Explore — Visualize
— Clean
— Sample — Impute — Encode — Transform
– Scale
— Modeling
– Overfitting
– Optimization
– ModelSelection – Regularization
– Generalization
— Documentation — Presentation
— Launch — Monitor — Maintain
LEARNING A FUNCTION
𝑥→𝑦
𝑥 →𝑓(𝑥)→𝑦
LEARNING A FUNCTION
𝑥→𝑦
𝑥 →𝑓(𝑥)→𝑦
Measured data
Features Inferred/Predicted/Estimated value
Trueinitialvalue𝑥 →𝑥’→𝑓 𝑥 =𝑦’ →𝑦
(world state) True target value
Learned/Fitted function (world state) From n observations
COMPONENTS OF A
MACHINE LEARNING SOLUTION
B. REGRESSION REAL VALUED VARIABLE
array([[4.21509616], [2.77011339]])
MAKING PREDICTIONS
ERROR SURFACE
GRADIENT DESCENT
LEARNING RATE TOO SMALL
LEARNING RATE TOO LARGE
MINIMA AND PLATEAUS
ERROR SURFACES
(NON) CONVEX
PARTIAL DERIVATIVES IN PARAMETER SPACE
FEATURE SCALING
LEARNING RATES
STOCHASTIC GRADIENT DECENT
PATHS IN THE SEARCH SPACE
LEARNING SCHEDULE
OP TIMIZERS
OPTIMIZERS A2GR ADEXP
OP TIMIZERS ADABELIEF
HYPER-PARAMETERS
FINE- TUNING
GRID SEARCH
RANDOM SEARCH
TRAINING EPOCHS
ERROR SURFACES
EARLY STOPPING
PROGRAM SPACE
Source: https://medium.com/@karpathy/software-2-0-a64152b37c35
CRISP CYCLE
DATA DEVELOPMENT LIFECYCLES
LECTURE 1 TERM 2:
MSIN0097
Predictive Analytics
A P MOORE