https://xkcd.com/2048/
Announcements
Assignment 1
Copyright By PowCoder代写 加微信 powcoder
out, we’ll talk about
future session.
new tutors joined
Tutorials start this week
Hope everyone is doing okay!
Linear Regression (Linear models for regression)
linear models?
Input data, features, basis functions
Maximum likelihood and least squares
Geometric intuition
Regularised least squares
Multiple outputs
Bias-variance decomposition
The relation between MLE and linear models.
least squares, Lagrange multipliers, multiple
ways of looking at
Why linear models?
– it saves lives
Image and table from wikipedia https://www.pregnancybirthbaby.org.au/apgar-score
Everything should be made as simple as possible, but not simpler. —
Occam’s razor, also spelled Ockham’s razor, also called law of economy or law of
parsimony, principle stated by the Scholastic philosopher William of Ockham (1285–1347/49)
that pluralitas non est ponenda sine necessitate, “plurality should not be posited without
necessity.” The principle gives precedence to simplicity: of two competing theories, the simpler
explanation of an entity is to be preferred. The principle is also expressed as “Entities are not to be
multiplied beyond necessity.”
https://www.britannica.com/topic/Occams-razor
The Lancet, 2003
https://sfar.org/scores2/snap22.php https://www.sciencedirect.com/science/article/abs/pii/S0140673603133971
Linear models and likelihood
Why linear
regression?
But what if ?
Conventions in [Bishop 2006]
Linear models and likelihood
also (3.8)
Computing the likelihood
= … (see next page)
Computing the likelihood
… first rewrite
the error function
stationary
The normal equation:
Obtaining and interpreting MLE results
The normal equation:
Numerical difficulties when
SVD or regularisation will help.
is ill-conditioned.
Cool geometric derivation of the normal equation
Sum-of-squares error
linear models?
Input data, features, basis functions
Maximum likelihood and least squares
Geometric intuition, sequential
Regularised least squares
Multiple outputs
Bias-variance decomposition
Regularised least squares
also (1.4)
Regularisation by
multipliers (appendix E)
The first encounter in SML – we’ll see it again
objective function equality constraint
maximize f(x) subject to g(x)=0
in kernel methods.
Multiple regression outputs
linear models?
Input data, features, basis functions
Maximum likelihood and least squares
Geometric intuition
Regularised least squares
Multiple outputs
Bias-variance decomposition
Bias-variance: the
What are the sources of different dots
What remains constant: learning target, model specification, learning algorithm.
What changes:
Data (subsets), randomness in learning (including but are not limited to initialisations), …
https://www.kdnuggets.com/2016/08/bias-variance-tradeoff-overview.html
Expected square loss
(See Sec 1.5.5)
Generalising squared loss
to Mikowski distances
introduce data D
Q: how does one know to add+subtract E_D[y(x; D)] ?
Unbiased estimators
The bias-variance
decomposition
“explainable models”? Interpretable models
Dawes, .. “The robust beauty of improper linear models in decision making.”
American Psychologist
34 (1979): 571-582.
Ustun, Berk and . “Supersparse linear integer models for optimized medical scoring systems.”
(2015): 349-391.
Rudin, Cynthia. “Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.”
Machine Learning 102
Nature Machine Intelligence
1 (2019): 206-215.
2022 Squirrel Prize in AI
https://aaai.org/Awards/squirrel-ai-award.php
for Humanity
For pioneering scientific work in the area of interpretable and
transparent AI systems in real-world deployments, the advocacy
for these features in highly sensitive areas such as social justice
and medical diagnosis, and serving as a role model for researchers
and practitioners.
Video: https://www.youtube.com/watch?v=PwLN5irdMT8
super-sparse linear models
Linear models for
linear models?
Input data, features, basis functions
Maximum likelihood and least squares
Geometric intuition
Regularised least squares
Multiple outputs
Bias-variance decomposition
The relation between MLE and linear models.
regression 1
least squares, Lagrange multipliers, multiple
ways of looking at
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com