PowerPoint Presentation
COMP9517: Computer Vision
Introduction
Week 1 COMP9517 2021 T3 1
What is computer vision?
Week 1 COMP9517 2021 T3 2
What is computer vision?
Computer science perspective
Computer vision is the interdisciplinary field that develops
theories and methods to allow computers extract relevant
information from digital images or videos
Computer engineering perspective
Computer vision is the interdisciplinary field that develops
algorithms and tools to automate perceptual tasks normally
performed by the human visual system
Week 1 COMP9517 2021 T3 3
Computer vision
automates and integrates
many information processing
and representation approaches
useful for visual perception
Every picture tells a story
Week 1 COMP9517 2021 T3 4
Yes and no (but mostly no)
• Humans are much better at “hard” tasks
• Computers can be better at “easy” tasks
Can computers match (or beat) humans?
Week 1 COMP9517 2021 T3 5
Human vision has its limitations…
Which objects are brighter?
Week 1 COMP9517 2021 T3 6
Human vision has its limitations…
Which objects are brighter?
Week 1 COMP9517 2021 T3 7
Human vision has its limitations…
Which side of this object is brighter?
Week 1 COMP9517 2021 T3 8
Human vision has its limitations…
Week 1 COMP9517 2021 T3 9
Which side of this object is brighter?
Human vision has its limitations…
Week 1 COMP9517 2021 T3 10
Which side of this object is brighter?
Human vision has its limitations…
Are the cells popping in or out?
Week 1 COMP9517 2021 T3 11
Human vision has its limitations…
Are the cells popping in or out?
Week 1 COMP9517 2021 T3 12
180o
Human vision has its limitations…
What pattern do
the squares form?
Week 1 COMP9517 2021 T3 13
Human vision has its limitations…
What pattern do
the squares form?
Week 1 COMP9517 2021 T3 14
Human vision has its limitations…
What object do you
see in this image?
Week 1 COMP9517 2021 T3 15
Human vision has its limitations…
Week 1 COMP9517 2021 T3 16
What object do you
see in this image?
Human vision has its limitations…
How do the main lines
run with respect to
each other?
Week 1 COMP9517 2021 T3 17
Human vision has its limitations…
Week 1 COMP9517 2021 T3 18
How do the main lines
run with respect to
each other?
Human vision has its limitations…
Week 1 COMP9517 2021 T3 19
In which direction are these particles moving ?
Human vision has its limitations…
Week 1 COMP9517 2021 T3 20
Course rationale
Week 1 COMP9517 2021 T3 21
Human vision has its limitations
• Intensities, shapes, patterns, motions can be misinterpreted
• Is labor intensive, time-consuming, subjective, error-prone
Computer vision can potentially improve this
• Can work day and night without getting tired
• Analyses information quantitatively and objectively
• Is potentially more accurate, precise, reproducible
If the methods and tools are well designed!
Project VarCity recreates 3D city models using social media photos
Week 1 COMP9517 2021 T3 22
Application: 3D shape reconstruction
Google’s Show and Tell open-source image captioning model in TensorFlow
Week 1 COMP9517 2021 T3 23
Application: image classification and captioning
https://ai.googleblog.com/2016/09/show-and-tell-image-captioning-open.html
Iris Automation provides safer drone operation with intelligent collision avoidance
Week 1 COMP9517 2021 T3 24
Application: intelligent collision avoidance
Facebook’s DeepFace project nears human accuracy in identifying faces
Week 1 COMP9517 2021 T3 25
Application: face detection and recognition
https://research.fb.com/publications/deepface-closing-the-gap-to-human-level-performance-in-face-verification/
For improving image capture on digital cameras
Week 1 COMP9517 2021 T3 26
Application: face detection and recognition
Week 1 COMP9517 2021 T3 27
Application: vision-based biometrics
Who is she?
How the Afghan girl was identified by her iris patterns
The remarkable story of Sharbat Gula, first photographed in
1984 aged 12 in a refugee camp in Pakistan by National
Geographic photographer Steve McCurry, and traced 18
years later to a remote part of Afghanistan where she was
again photographed by McCurry…
http://www.cl.cam.ac.uk/%7Ejgd1000/afghan.html
Week 1 COMP9517 2021 T3 28
Application: logging in without a password
Fingerprint scanners on
modern laptops and
other devices
Windows Hello makes
logging in as easy as
looking at your PC
Converting scanned documents or number plates to processable text
Week 1 COMP9517 2021 T3 29
Application: optical character recognition (OCR)
Week 1 COMP9517 2021 T3 30
Application: object recognition in supermarkets
LaneHawk by Evolution Robotics Retail
provides a loss-prevention solution that
detects bottom-of-basket (BOB) items
in checkout lanes
Week 1 COMP9517 2021 T3 31
Application: object recognition in phones
Intel’s Mobileye makes cars safer and more autonomous
Week 1 COMP9517 2021 T3 32
Application: autonomous vehicles
https://www.mobileye.com/
NASA’s Mars Exploration Rover Spirit autonomously captured this picture in 2007
Week 1 COMP9517 2021 T3 33
Application: space exploration
Vision systems used for panorama stitching, 3D terrain modeling, obstacle detection, position tracking
See Computer Vision on Mars for more information
https://www.ri.cmu.edu/pub_files/pub4/matthies_larry_2007_1/matthies_larry_2007_1.pdf
Week 1 COMP9517 2021 T3 34
Application: machine vision in robotics
RoboCupNASA’s Mars Spirit Rover
http://www.robocup.org/
http://en.wikipedia.org/wiki/Spirit_rover
Week 1 COMP9517 2021 T3 35
Application: medical imaging
Image Guided SurgeryComputer Aided Diagnosis
Week 1 COMP9517 2021 T3 36
Application: video surveillance
• Traffic monitoring
• Person tracking
• Action recognition
• Speed estimation
• Object counting
• …
Week 1 COMP9517 2021 T3 37
Goals and challenges of computer vision
• Extract useful information from images, both metric and semantic
• Data heterogeneity, ambiguity, and complexity are a big challenge
• Significant progress in recent years due to improvements in
processing power, memory, storage capacity, data availability
• Computer vision workflow: images > measurements > models >
algorithms for learning and inference
Week 1 COMP9517 2021 T3 38
Computer vision tasks
• Obtain simple inferences from individual pixel values
• Group pixels to separate object regions or infer shape information
• Recognise objects using geometric or statistical pixel information
• Combine information from multiple images into a coherent whole
Requires understanding of the physics of imaging and the use of
mathematical and statistical models for information extraction
Week 1 COMP9517 2021 T3 39
Critical issues in computer vision
• Sensing: how do sensors obtain images of the world?
• Encoding: how do images encode information of the scene?
• Representation: what are appropriate representations of objects?
• Algorithmics: what are appropriate algorithms to process image
information and construct scene descriptions?
Week 1 COMP9517 2021 T3 40
Low-level computer vision
This is almost entirely digital image processing (image in > image out)
• Sensing: image capture and digitisation
• Preprocessing: suppress noise and enhance object features
• Segmentation: separate objects from background and partition them
• Description: compute features which differentiate objects
• Classification: assign labels to image segments (regions)
Week 1 COMP9517 2021 T3 41
High-level computer vision
This is about knowledge construction, representation, and inference
• Recognition: identify objects based on low-level information
• Interpretation: assign meaning to groups of recognized objects
• Scene analysis: complete understanding of the captured scene
Week 1 COMP9517 2021 T3 42
Assumed knowledge
To do this course successfully you should:
• Be able to program well in Python or willing to learn it independently
• Be familiar with data structures and algorithms and basic statistics
• Be able/learn to use and integrate software packages (OpenCV, Scikit-Learn, Keras)
• Be familiar with vector calculus and linear algebra or willing to learn it independently
Please self-assess before deciding to stay/enroll in the course
Week 1 COMP9517 2021 T3 43
Student learning outcomes
After completing this course, you will be able to:
• Explain basic scientific, statistical, and engineering approaches to computer vision
• Implement and test computer vision algorithms using existing software platforms
• Build larger computer vision applications by integrating software modules
• Interpret and comment on articles in the computer vision literature
Week 1 COMP9517 2021 T3 44
Course topics and lecturers
Week Topic Lecturer
1 Introduction & Image Formation Professor Erik Meijering
2 Image Processing Dr Wafa Johal
3 Feature Representation Dr Wafa Johal
4 Pattern Recognition Professor Erik Meijering
5 Image Segmentation Professor Erik Meijering
6 Flexible Week (No Lectures)
7 Motion and Tracking Professor Erik Meijering
8 Deep Learning Guest Lecturers TBC
9 Applications Guest Lecturers TBC
10 Project Demos Professor Erik Meijering
Week 1 COMP9517 2021 T3 45
Weekly class structure
• Lectures: Tuesdays 6-8pm & Thursdays 6-7pm (live stream via BB Colab)
All lectures will be live online with an opportunity to ask questions
• Lab consultations: Thursdays 7-8pm in weeks 1-5 (online via BB Colab)
Software demo in week 1 and lab consultations with your assigned tutor in weeks 2-5
• Project consultations: Thursdays 7-8pm in weeks 6-9 (online via BB Colab)
All project consultations will be live online with your assigned tutor
• Project demos: Tuesday 6-8pm & Thursday 6-8pm in week 10 (online via BB Colab)
Detailed schedule will be announced on WebCMS3 web page of the course
Week 1 COMP9517 2021 T3 46
Assessments
Late submission penalty: Unless you have received special dispensation from the Lecturer in Charge, work
that is submitted after the deadline during term will incur a penalty of 10% per day, up to a maximum of
100%. For the final examination, university exam rules will apply.
Assessment Marks Release Due
Assignment 10% Week 2 Week 4
Lab Work (4x) 10% Weeks 2, 3, 4, 5 Weeks 3, 4, 5, 6
Group Project 40% Week 5 Week 10
Exam 40% Exam Period Exam Period
Week 1 COMP9517 2021 T3 47
Communication modes and etiquette
• Online forum (Moodle) is your first port of call for queries of wider interest on lectures,
assignment, labs, project, and assessments
• Contact the LIC for late submission, absence, assessment deadlines, and specific questions
about the assignment, labs, project, and assessment contents
• Contact the course admin for issues with enrolment, file submission, group enrolment, or
other administration related matters
• Team is committed to respond quickly to queries with a maximum turnaround of 24 hours
• Do observe standards of equity and respect in dealing with all students and staff, in person,
emails, forum posts, and all other communication
• Language of communication is English
Week 1 COMP9517 2021 T3 48
Special Consideration
• If your work in this course is affected by unforeseen adverse circumstances, you should apply
for Special Consideration via the UNSW website
• UNSW handles Special Consideration requests centrally, so use the website and do not email
the Lecturer in Charge about Special Consideration requests
• Special Consideration requests must be accompanied by documentation
• Marks are calculated the same way as other students who sat the original assessment
• If you are awarded a Supplementary Exam and do not attend, your exam mark will be zero
See the course webpage on WebCMS3 for more detailed information and links
Week 1 COMP9517 2021 T3 49
Plagiarism Policy
READ the UNSW Policy and Procedure on this (links in the course outline on WebCMS3)
For the purposes of COMP9517, plagiarism includes copying or obtaining all, or a substantial
part, of the material for your assignment, whether written or graphical report material, or
software code, without written acknowledgement in your assignment from:
• A location on the internet
• A book, article or other written document (published or unpublished) whether electronic
or on paper or other medium
• Another student, whether in your class or another class
• Someone else (for example someone who writes assignments for money)
Week 1 COMP9517 2021 T3 50
Plagiarism Policy
• If you copy material from another student or non-student with acknowledgement, you will
not be penalised for plagiarism, but the marks you get for this will be at the marker’s
discretion and will reflect the marker’s perception of the amount of work you put into
finding and/or adapting the code/text.
• If you use text found in a publication (on the internet or otherwise), the marks you get for
this will be at the marker’s discretion and will reflect the marker’s perception of the
amount of work you put into finding and/or adapting the text.
Assessments provide opportunities for you to develop important skills!
Use these opportunities!
Week 1 COMP9517 2021 T3 51
Further information on WebCMS
Please be sure you are familiar with:
• Communication Etiquette
• Special Consideration
• Student Conduct
• Plagiarism Policy
• Academic Integrity
https://webcms3.cse.unsw.edu.au/COMP9517/20T2/outline
https://webcms3.cse.unsw.edu.au/COMP9517/20T2/outline#etiquette
https://webcms3.cse.unsw.edu.au/COMP9517/20T2/outline#special
https://webcms3.cse.unsw.edu.au/COMP9517/20T2/outline#conduct
https://webcms3.cse.unsw.edu.au/COMP9517/20T2/outline#plagiarism
https://webcms3.cse.unsw.edu.au/COMP9517/20T2/outline#integrity
Week 1 COMP9517 2021 T3 52
Further reading on lectured topics
In the lectures we will be referring to various online resources for further reading such as:
• Richard Szeliski, Computer Vision: Algorithms and Applications, Second Edition, Springer, 2021
• Dana H. Ballard and Christopher M. Brown, Computer Vision, Prentice Hall, 1982
• Ian Goodfellow, Yoshua Bengio, Aaron Courville, Deep Learning, MIT Press, 2016
• David A. Forsyth and Jean Ponce, Computer Vision: A Modern Approach, Prentice Hall, 2011
• Simon J. D. Prince, Computer Vision: Models, Learning and Inference, Cambridge University Press, 2012
And other books, scientific articles, and other resources available online or via the UNSW Library
http://szeliski.org/Book
http://homepages.inf.ed.ac.uk/rbf/BOOKS/BANDB/bandb.htm
https://www.deeplearningbook.org/
https://github.com/yihui-he/computer-vision-tutorial/blob/master/Computer%20Vision%20A%20Modern%20Approach%202nd%20Edition.pdf
http://www.computervisionmodels.com/
Week 1 COMP9517 2021 T3 53
Further reading on today’s topics
• Chapter 1 of Szeliski for a general introduction to computer vision
• Appendix A of Szeliski for a recap of linear algebra and numerical techniques
Week 1 COMP9517 2021 T3 54
Acknowledgements
• Some images on applications taken from Szeliski with original sources
credited where possible
• Other images and videos credited where possible
Slide Number 1
What is computer vision?
What is computer vision?
Every picture tells a story
Can computers match (or beat) humans?
Human vision has its limitations…
Human vision has its limitations…
Human vision has its limitations…
Human vision has its limitations…
Human vision has its limitations…
Human vision has its limitations…
Human vision has its limitations…
Human vision has its limitations…
Human vision has its limitations…
Human vision has its limitations…
Human vision has its limitations…
Human vision has its limitations…
Human vision has its limitations…
Human vision has its limitations…
Human vision has its limitations…
Course rationale
Application: 3D shape reconstruction
Application: image classification and captioning
Application: intelligent collision avoidance
Application: face detection and recognition
Application: face detection and recognition
Application: vision-based biometrics
Application: logging in without a password
Application: optical character recognition (OCR)
Application: object recognition in supermarkets
Application: object recognition in phones
Application: autonomous vehicles
Application: space exploration
Application: machine vision in robotics
Application: medical imaging
Application: video surveillance
Goals and challenges of computer vision
Computer vision tasks
Critical issues in computer vision
Low-level computer vision
High-level computer vision
Assumed knowledge
Student learning outcomes
Course topics and lecturers
Weekly class structure
Assessments
Communication modes and etiquette
Special Consideration
Plagiarism Policy
Plagiarism Policy
Further information on WebCMS
Further reading on lectured topics
Further reading on today’s topics
Acknowledgements