Lecture 1: Introduction and Overview
COMP90049
Introduction to Machine Learning
Semester 2, 2021
Lida Rashidi, CIS
Copyright @ University of Melbourne 2021. All rights reserved. No part of the publication
may be reproduced in any form by print, photoprint, microfilm or any other means without
written permission from the author.
Acknowledgement: Lea Frermann
1
Roadmap
This lecture
• Introduction and Warm-up
• Housekeeping COMP90049
• Machine Learning
2
Intros & Warm-up
Introductions
About Lida
• Lecturer in CIS since 2019
• Research in graph mining and information retrieval
• PhD from the University of Melbourne
• 4 years of research in academia
About Qiuhong
• Lecturer in CIS since 2020
• Research in machine learning and computer vision
• PhD from the University of Western Australia
• 1.5 years of research in Max Planck Institute for Informatics, Germany
About you
Please go to: pollev.com/comp90049
3
pollev.com/comp90049
Introductions
About Lida
• Lecturer in CIS since 2019
• Research in graph mining and information retrieval
• PhD from the University of Melbourne
• 4 years of research in academia
About Qiuhong
• Lecturer in CIS since 2020
• Research in machine learning and computer vision
• PhD from the University of Western Australia
• 1.5 years of research in Max Planck Institute for Informatics, Germany
About you
Please go to: pollev.com/comp90049
3
pollev.com/comp90049
Brainstorm / Discuss
What is Learning?
4
Brainstorm / Discuss
What is Machine Learning?
4
Definitions of Machine Learning
Some proposed definitions…
“The computer automatically learns something”
“Statistics, plus marketing”
“ … how to construct computer programs that automatically improve
with experience …. A computer program is said to learn from expe-
rience … if its performance … improves with experience… ”
Mitchell [1997, pp. xv-17]
5
Definitions of Machine Learning
“We are drowning in information, but we are starved for knowledge”
John Naisbitt, Megatrends
Our definition of Machine Learning
automatic extraction of valid, novel, useful and comprehensible
knowledge (rules, regularities, patterns, constraints, models, …) from
arbitrary sets of data
5
Definitions of Machine Learning
Learning what?
• Task to accomplish a goal, e.g.,
– Assign continuous values to inputs (essay→ grade)
– Group inputs into known classes (email→ {spam, no-spam})
– Understand regularities in the data
Learning from what?
• Data
• Where do the data come from? Is it reliable? Representative?
How do we learn?
• define a model that explains how to get from input to output
• derive a learning algorithm to find the best model parameters
How do we know learning is happening?
• The algorithm improves at its task with exposure to more data
• We need to be able to evaluate performance objectively
5
About COMP90049
COMP90049 – Teaching Staff
Coordinator
& Lecturer1
Lida Rashidi .au
Lecturer 2 Qiuhong Ke quihong. .au
Tutors Tahrima Hashem .au
Pei-Yun Sun .au
Ella Alipourchavary ella. .au
Kazi Adnan kazi. .au
Hasti Samadi hasti. .au
Zenan Zhai zenan. .au
6
COMP90049 – Organisation
• The lectures will be delivered fully online
• I’ll aim for as much interaction as possible (and desired)
• All live lectures will be recorded. All recordings and other materials will
be made available online through Canvas
• Live lectures via Zoom for the first couple of weeks
• Afterwards possibly pre-recorded with live Q&A sessions
• Live and in-person workshops throughout the semester dual delivery
mode
7
COMP90049 – Lectures
Lectures
Lecture 1 Wed 16:15-17:15
Online; Zoom
Lecture 2 Thu 14:15-15:15
Online; Zoom
Lecture content
• Theory
• Derivation of ML algorithms from scratch
• Motivation and context
• Some coding demos in Python
8
COMP90049 – Workshops
Workshops
• start from week 2
• 1 hour per week
• ∼ 14 slots, please sign up and stick to one
• Online workshops are live via zoom and In-person workshops will be on
campus
• At the moment, due to restrictions the on campus workshops are
converted to online workshops
• Return to campus will be announced on LMS
Workshop Content
• Practical exercises
• Working through numerical examples
• Revising theoretical concepts from the lectures
9
Other Support
Coding drop-in sessions
Session 1 Tuesday 12:00–13:00 (link via Canvas Zoom)
Session 2 Friday 15:00–16:00 (link via Canvas Zoom)
• start from week 2 and run until week 5
• you can ask questions around Python / the weekly code snippets
• Not an assignment consultation
10
COMP90049 – Subject Communication
Materials and announcements
• All materials will be made available through LMS (Canvas)
• Important news will be shared via Canvas Announcements (expect
about 1 per week)
General inquiries: Piazza forum on LMS
• We encourage all students to join in discussions answering other
students questions is one of the best ways to improve your own
understanding
• Please do not post sections of your code or reports publicly on Piazza! If
you must include these, private-message the instructors
Personal/private concerns: Email your tutor or lecturer
• If you email us about a general inquiry, we may ask you to re-post your
question in the forum
• Please include COMP90049 in email subject
11
COMP90049 – Subject Communication
I am looking for 2-3 Student Representatives
• Communication channel between class and teaching team
• Collect and pass on (anonymous) feedback or complaints
• Attend a student-staff meeting during the semester (TBD)
• Represent the diversity of the class
Interested? Send me an email with a short paragraph on why you want
this role.
11
COMP90049 – Lectures / Engagement / Cameras
Interaction and Engagement
• We’ll experiment with breakout rooms, polls, shared whiteboards…
please engage!
• Feel free to ask questions / use the chat / raise your hands (I’ll do my
best to monitor)
• Feedback surveys
• You are encouraged to switch on your camera in lectures and
(particularly) workshops to maximize engagement. Please see the
recent announcement / post on the subject Home page for
acknowledgment of and details on privacy concerns.
12
COMP90049 – Subject Content
• Topics include: classification, clustering, optimization, unsupervised
learning, semi-supervised learning, neural networks
• All from a theoretical and practical perspective
• Refreshers on maths and programming basics
• Theory in the lectures (some live-coding and demo-ing of libraries and
toolkits)
• Hands-on experience in workshops and projects
• Guest lecture 1: academic writing skills
• Guest lecture 2: Industry talk with focus on bias and fairness in
machine learning
13
Expected Background
Programming concepts
• We will be using Python and Jupyter Notebooks
• Basic familiarity with libraries (numpy, scikit-learn, scipy)
• You need to be able to write code to process your data, apply different
algorithms, and evaluate the output
• Optional practice / demo Jupyter notebooks (most weeks)
• Optional coding consultation sessions in the first weeks of semester
Mathematical concepts
• formal maths notation
• basic probability, statistics, calculus, geometry, linear algebra
• (why?)
14
Expected Background
Programming concepts
• We will be using Python and Jupyter Notebooks
• Basic familiarity with libraries (numpy, scikit-learn, scipy)
• You need to be able to write code to process your data, apply different
algorithms, and evaluate the output
• Optional practice / demo Jupyter notebooks (most weeks)
• Optional coding consultation sessions in the first weeks of semester
Mathematical concepts
• formal maths notation
• basic probability, statistics, calculus, geometry, linear algebra
• (why?)
14
What Level of Maths are we Talking?
ln
P(y = true|x)
1− P(y = true|x)
=w · f
P(y = true|x)
1− P(y = true|x)
=ew·f
P(y = true|x) =ew·f − ew·f P(y = true|x)
P(y = true|x) + ew·f P(y = true|x) =ew·f
P(y = true|x) = h(x) =
ew·f
1 + ew·f
=
1
1 + e−w·f
P(y = false|x) =
1
1 + ew·f
=
e−w·f
1 + e−w·f
15
What Level of Maths are we Talking?
P(y = 1|x ;β) = hβ(x)
P(y = 0|x ;β) = 1− hβ(x)
→P(y |x ;β) = (hβ(x))y ∗ (1− hβ(x))1−y
argmax
β
n∏
i=1
P(yi |xi ;β)
= argmax
β
n∏
i=1
(hβ(xi))
yi ∗ (1− hβ(xi))
1−yi
=argmax
β
n∑
i=1
yi log hβ(xi) + (1− yi) log(1− hβ(xi))
15
Assessment
Two small coding projects (30%)
• Project 1: release week 2, due week 3
• Project 2: release week 5, due week 6
• Read in data, apply ML algorithm(s), evaluate.
Open-ended research project (30%)
• Release week 7, due week 10
• You will be given a data set and will formulate a research question and
write a short research paper on your findings. You will be graded based
on the quality of your report.
Final exam (40%)
• during exam period
• 2 hours; closed-book
• Hurdle requirement: you have to pass the exam (≥ 50%).
16
Academic Honesty
• Videos & Quiz
• Linked from Canvas ’Home’ page (or in Modules)
• CIS-specific scenarios
17
What and Why of Machine Learning?
What is Machine Learning?
18
What is Machine Learning?
https://xkcd.com/1838/
18
https://xkcd.com/1838/
Relevance
(you’re sitting in the right class!)
Source: https://www.springboard.com/blog/machine-learning-engineer-salary-guide/
19
Three ingredients for machine learning
… and related questions
20
Three ingredients for machine learning
… and related questions
1. Data
• Discrete vs continuous vs …
• Big data vs small data
• Labeled data vs unlabeled data
• Public vs sensitive data
20
Three ingredients for machine learning
… and related questions
Models
• function mapping from inputs to outputs
• motivated by a data generating hypothesis
• probabilistic machine learning models
• geometric machine learning models
• parameters of the function are unknown
20
Three ingredients for machine learning
… and related questions
Learning
• Improving (on a task) after data is taken into account
• Finding the best model parameters (for a given task)
• Supervised vs. unsupervised learning
20
ML Example Problem
ML Example Problem
• Scenario 1
You are an archaeologist in charge of classifying a mountain of fossilized
bones, and want to quickly identify any “finds of the century” before
sending the bones off to a museum
• Solution:
Identify bones which are of different size/dimensions/characteristics to
others in the sample and/or pre-identified bones
CLUSTERING/OUTLIER DETECTION
21
ML Example Problem
• Scenario 1
You are an archaeologist in charge of classifying a mountain of fossilized
bones, and want to quickly identify any “finds of the century” before
sending the bones off to a museum
• Solution:
Identify bones which are of different size/dimensions/characteristics to
others in the sample and/or pre-identified bones
CLUSTERING/OUTLIER DETECTION
21
ML Example Problem
• Scenario 2:
You are an archaeologist in charge of classifying a mountain of fossilized
bones, and want to come up with a consistent way of determining the
species and type of each bone which doesn’t require specialist skills
• Solution:
Identify some easily measurable properties of bones (size, shape,
number of “lumps”, …) and compare any new bones to a pre-classified
database of bones
SUPERVISED CLASSIFICATION ;
22
ML Example Problem
• Scenario 2:
You are an archaeologist in charge of classifying a mountain of fossilized
bones, and want to come up with a consistent way of determining the
species and type of each bone which doesn’t require specialist skills
• Solution:
Identify some easily measurable properties of bones (size, shape,
number of “lumps”, …) and compare any new bones to a pre-classified
database of bones
SUPERVISED CLASSIFICATION ;
22
ML Example Problem
• Scenario 3:
You are in charge of developing the next “release” of Coca Cola, and
want to be able to estimate how well received a given recipe will be
• Solution:
Carry out taste tests over various “recipes” with varying proportions of
sugar, caramel, caffeine, phosphoric acid, coca leaf extract, … (and any
number of “secret” new ingredients), and estimate the function which
predicts customer satisfaction from these numbers
REGRESSION
23
ML Example Problem
• Scenario 3:
You are in charge of developing the next “release” of Coca Cola, and
want to be able to estimate how well received a given recipe will be
• Solution:
Carry out taste tests over various “recipes” with varying proportions of
sugar, caramel, caffeine, phosphoric acid, coca leaf extract, … (and any
number of “secret” new ingredients), and estimate the function which
predicts customer satisfaction from these numbers
REGRESSION
23
More Applications
• natural language processing
• image classification
• stock market prediction
• movie recommendation
• web search
• medical diagnoses
• spam / malware detection
• …
24
Machine Learning, Ethics, and Transparency
commons.wikimedia.org/wiki/File:Pseudo-
algorithm comparison for my slides on machine learning ethics.svg
Def 1. Discrimination= To make distinctions.
For example, in supervised ML, for a given instance, we might try to
discriminate between the various possible classes.
25
Machine Learning, Ethics, and Transparency
commons.wikimedia.org/wiki/File:Pseudo-
algorithm comparison for my slides on machine learning ethics.svg
Def 2. Discrimination= To make decisions based on prejudice.
Digital computers have no volition, and consequently cannot be prejudiced.
However, the data may contain information which leads to an application
where the ensuing behavior is prejudicial, intentionally or otherwise.
25
Machine Learning gone wrong…
26
Machine Learning gone wrong…
26
Machine Learning gone wrong…
26
Machine Learning and Ethics
Not everything that can be done, should be done
• Attributes in the data can encode information in an indirect way
• For example, home address and occupation can be used (perhaps with
other
seemingly-banal data) to infer age and social standing of an individual
• Potential legal exposure due to implicit “knowledge” used by a classifier
• Just because you didn’t realize doesn’t mean that you shouldn’t have
realized, or at least, made reasonable efforts to check
Questions to Ask
• Who is permitted to access the data?
• For what purpose was the data collected?
• What kinds of conclusions are legitimate?
• If our conclusions defy common sense, are there confounding factors?
• Could my research / application be abused (dual use)?
27
Summary
Today
• COMP90049 Overview
• What is machine learning?
• Why is it important? Some use cases.
• What can go wrong?
Next lecture: Concepts in machine learning
28
References i
Jacob Eisenstein. Natural Language Processing. MIT Press (2019)
Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong. Mathematics
for Machine Learning. Cambridge University Press (forthcoming)
Chris Bishop. Pattern Rechognition and Machine Learning. Springer (2009)
Tom Mitchell. Machine Learning. McGraw-Hill, New York, USA (1997).
29
References ii
Microsoft’s AI robot goes dark.
https:
//www.reuters.com/article/us-microsoft-twitter-bot-idUSKCN0WQ2LA
Amazon scraps secret recruiting tool.
https://www.reuters.com/article/
us-amazon-com-jobs-automation-insight-idUSKCN1MK08G
Predictive policing algorithms are biased.
https://www.bbc.com/news/technology-53165286
30
https://www.reuters.com/article/us-microsoft-twitter-bot-idUSKCN0WQ2LA
https://www.reuters.com/article/us-microsoft-twitter-bot-idUSKCN0WQ2LA
https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK 08G
https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK 08G
https://www.bbc.com/news/technology-53165286
Intros & Warm-up
About COMP90049
What and Why of Machine Learning?
ML Example Problem