Introduction to Machine Learning
Australian National University
Who Are We?
(郑良)
Course convener
Senior Lecturer
School of Computing
Office: N214, CSIT Building http://zheng-lab.cecs.anu.edu.au/
Tutors
Nutthadech Li
Shi Sun
Ruiqi Li Alexander La
Nikunj Li
Dian Lu Qingzheng Are you?
Undergraduate students Postgraduate students Graduate certificate students
Lectures
• 1:30pm – 3:00 Tuesday
• 3:00pm – 4:30pm Wednesday
• Week 1 to Week 12
•
• Zoom link: https://anu.zoom.us/j/83859790342?pwd=TWR2UnVlZ UREUXVEc2tzeWg3N1NpQT09
• Office hour (online)
• 1pm-2pm Friday
• Same Zoom link as lectures
Evaluation
• Homework (40 pts)
• 4 assignments, equally weighted
• Programming and theory • SubmittedtoWattle
• Honor Code
• You can form study groups to work on the homework
• Write-up solutions on your own
• List names of anyone you talked to
• Final Exam (60 pts)
• Assess your understanding of machine learning algorithms • You do not need to write codes or pseudo codes
To support hybrid learning
• Live streamed + in-person lectures
• Lecture recordings will be available
• Exercises in each lecture (e.g., last 5 minutes in each lecture)
• I will stay a while (10 – 20 minutes) after each lecture to answer individual questions
• Lab materials / lecture slides will be released as early as possible
• Group discussions in tutorials
• Instant feedback on assignments / exam / lectures on Piazza
• No hurdle
• We will have self-assessment in Week 2
• You may choose to drop the course if you feel the self- assessment questions are too difficult for you
How can you support us teaching?
• Try to show up in lectures
• Try to turn your camera on
• Actively participate in your online discussions • Try to show up in your labs/tutorials
• Be proactive
• Ask questions before your lab/tutorials • Give us feedback during/after class
Class representatives needed!
Assignment dates
Date when assignment is released Date when assignment is due
Date when mark is available. Feedback will be uploaded after that.
*Note: Mark release for A4 may be a few days after Week 12 but will be before the exam.
Policy
• Late policy
• No deadline extension unless
• accompanied by a doctor’s certificate
• A 100% penalty after the deadline – 0 mark
• A grace period of 5 minutes: it is fine if you are 5 minutes late.
• Other than that, your mark will be 0 if you are late by at least 5 min 1
second, as per time on Wattle
• We will send reminders 7 days, 3 days, 1day before the due date. • Test your internet connection & submit as you go
• For each assignment, if you think our marking is incorrect, you need to let us know in 30 days after the feedback is released.
• Note: after we recheck your assignment, you might have increased/decreased/same marks
• We reserve the right to ask you to orally explain your solutions (see ANU policy on plagiarism http://academichonesty.anu.edu.au/UniPolicy.html)
Plagiarism
• https://services.anu.edu.au/education-support/academic- integrity
• You must
• Work on your own solution, without taking a single look at others’ (you
can discuss though).
• Cite the uni ID of anyone you discussed with.
• Cite a (web) source when you get your idea from external sources.
• Work on your own solution even if you get the idea from the web
• Formal process (against plagiarism) will be taken if
• [Poor academic practice] Your solutions are highly similar to other
students
• [Poor academic practice] Your solutions are highly similar to a webpage (and potentially similar to other students who also use the same webpage as a source)
• Ifyouhavecitedthesource,i.e.,agenuinemistake,thepenaltywillbelighter
• [Minor breach] You fail to cite the external reference where you get your idea from (but your solution is sufficiently different from the external reference)
• [Minor breach] You fail to cite your peer who discussed with you (but your solutions are sufficiently different from your peers’)
• Other cases outlined in the ANU policy (link above).
Textbook
• Deisenroth, Faisal and Ong, “Mathematics for Machine Learning”, 2019. https://mml-book.github.io/book/mml-book.pdf
Syllabus
Week
Topic
Week
Topic
1
Intro & Linear algebra
7
Probability and distributions
2
Linear algebra & Analytic geometry
8
Gaussian Mixtures
3
Analytic geometry
& models meets data
9
Matrix decomposition
4
Clustering
10
Principal Component Analysis
5
Vector calculus
11
Classification
6
Linear Regression
12
Guest lectures
Machine Learning
What is machine learning?
Task Performance Experience
Algorithms that improve their performance at some task with experience
– (1998)
What is machine learning?
• A branch of artificial intelligence, concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data.
• As intelligence requires knowledge, it is necessary for the computers to acquire knowledge.
What is machine learning?
Hard-Coded Trained
Giving computers the ability to learn without being explicitly programmed – (1959)
Traditional Programming
Data Program
Machine Learning
Data
Output (Task)
Output
Computer
Computer
Program (Model)
What is machine learning?
• We have a model • We predict
• Given input
• Image classification
model input
Dog Building Cat ✔ Human Car
Training and testing
Data acquisition
Training set
(observed)
Testing set (unobserved)
Universal set (unobserved)
Practical usage
Training and tesBng
• Training is the process of making the system able to learn.
• No free lunch rule: a model that explains a certain situation
well may fail in another situation.
• Training set and testing set come from the same distribution • Before applying a model, check the assumptions!
Training data
Testing data
Types of machine learning
Supervised learning
Unsupervised learning
Semi-supervised learning
21
Supervised Learning
Regression (Linear)
Palm Beach
https://en.wikipedia.org/wiki/United_States_presidential_election_in_Florida,_2000 http://andrys.com/flballot.html
Learning a function
𝑦=𝑓𝑥 x∈R 𝑦∈R
2000 USA Presidential Elections.
Votes for Buchanan and Bush in counties of Florida on a log scale.
Supervised Learning
Regression (Non-linear)
Supervised Learning
Classification (Linear)
Malignant Benign
Supervised Learning
Classification (Non-linear)
Spam Filters
Bayesian Networks
Unsupervised Learning
Clustering
Unsupervised Learning
Dimensionality Reduction: Subspace Learning
Deep Learning
Image Classification
ImageNet dataset: 1,000 classes, 1.2 million images for training, 50k images for testing
Year
Top-1 error (%)
2010
47.1
2011
45.7
2012
37.5
2014
23.7
2014
21.99
Method
Sparse coding SIFT + FV AlexNet VGGNet GoogleNet ResNet
2016
19.38
Top-5 error (%)
28.2 25.7 17.0 6.8 4.82 3.57
Human: 5.1%
Alexnet
Alex Krizhevsky, , and . Hinton. “Imagenet classification with deep convolutional neural networks.” In Advances in neural information processing systems, pp. 1097-1105. 2012.
31
Image Classification
ImageNet dataset: 1,000 classes, 1.2 million images for training, 50k images for testing
Year
Top-1 error (%)
2010
47.1
2011
45.7
2012
37.5
2014
23.7
2014
21.99
Method
Sparse coding SIFT + FV AlexNet VGGNet GoogleNet ResNet
2016
19.38
Top-5 error (%)
28.2 25.7 17.0 6.8 4.82 3.57
Human: 5.1%
VGGNet
Karen Simonyan, and . “Very deep convolutional networks for large-scale image recognition.” ICLR 2015.
33
VGGNet
34
Image Classification
ImageNet dataset: 1,000 classes, 1.2 million images for training, 50k images for testing
Year
Top-1 error (%)
2010
47.1
2011
45.7
2012
37.5
2014
23.7
2014
21.99
Method
Sparse coding SIFT + FV AlexNet VGGNet GoogleNet ResNet
2016
19.38
Top-5 error (%)
28.2 25.7 17.0 6.8 4.82 3.57
Human: 5.1%
ResNet
Kaiming He, , , and Jian Sun. “Deep residual learning for image recognition.” In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778. 2016.
36
37
Vision Transformer, ICLR 2021
Object Detection
Generative Models
Generative Models
Exercises (unrelated to your mark)
• Which of the followings are machine learning applications?
(A) Timetabling at ANU
(B) Face recognition at airports
(C) Write machine learning assignment
(D) Use a camera to detect car speeding
(E) Use a GPS to track a player in a match.
(F) Crop a face from an image using photoshop
(G) automatically send alarm when water level is above a threshold
(H) Google translation (e.g., English -> French)
(I) Recommendation system in Facebook/Netflix