MULTIMEDIA RETRIEVAL
Course Review
Course Review
Course Survey
Copyright By PowCoder代写 加微信 powcoder
Feedback on Project Presentation Course Summary
Exam Preparation
School of Computer Science
Course Survey
Please complete the course survey to provide us your feedback
https://student-surveys.sydney.edu.au/
School of Computer Science
Course Content
Multimedia Basics
Multimedia Retrieval
Retrieval basics
Data representation
Similarity measurement
Advanced topics
School of Computer Science
Multimedia Basics
Digitization concept Sampling
Quantization Aliasing
Digital image acquisition and Representation
Digital video acquisition and representation
Digital audio acquisition and representation Waveform, frequency & spectrum
The content marked with (*) are for understanding only.
School of Computer Science
Retrieval Basics
Information retrieval
Motivations, challenges, and general paradigm
Document preprocessing
parsing/tokenization Stemming
Stopwordremoval
Indexing and index
Similarity Measurement
Relevance feedback
Query Expansion, Summarization, and visualization
Evaluation
Web Search
Characteristics of Web
Paradigm of web search systems
Crawling
PageRank:motivationandalgorithm HITS:motivationandalgorithm
School of Computer Science
Multimedia Retrieval Basics
Content-basedretrieval
Motivations and challenges Issues
Audioretrieval/classification Features, applications
Image retrieval
Feature extraction
Color: color space, techniques (color histogram, color moments, color coherence vector) and their properties
Texture: categorization, techniques and their properties (no calculating)
Shape: categorization, techniques and their properties (no calculating)
Spatial: 2D, 2D-G, 2D-C (*)
Others features: Compressed domain, graph (*)
Issues of feature extraction Case studies
School of Computer Science
Multimedia Retrieval Basics
Other issues of CBR
Feature combination Issues
Normalization: intra- & inter-
Similarity measure: distance functions and properties
Relevance feedback: concepts and techniques
Indexing: concepts, indexing techniques and their properties, issues
Performance evaluation Metrics
Benchmark
School of Computer Science
Multimedia Retrieval Basics
Video retrieval
Access video content: fundamental aspects Analysis, representation, browsing, retrieval
Segmentation
Shot detection
Key-frame extraction
Object segmentation
Scene/event detection
Story segmentation
Video abstract/skimming
Representation
Key-frame based
Shot-based: motion based, object based, …
Applications
Video retrieval: by motion/trajectories
Video annotation: events in soccer video, dialogue detection, news video analysis Video classification: movie genre classification, finding commercials in video
School of Computer Science
Social Media
Social multimedia
Attributes/properties Applications
User profile
User context
User interaction
School of Computer Science
Large Scale Retrieval
Semantic Gap
Image/Video Annotation/Tagging Co-occurrence approach Translation approach Classification approach
Bag-of-Visual-Words model
School of Computer Science
Sketch based Image Retrieval (SBIR)
Motivation of SBIR
Conventional Methods Edge representation Large scale SBIR
Deep Learning based Methods Categorical SBIR Fine-grained SBIR
School of Computer Science
Recommender Systems
Background
Recommendation algorithms Collaborative filtering
User based
Model based
Matrix factorization
Content-based
Product, document, image, video, audio
Learning based
Context Aware Recommendation Evaluation
School of Computer Science
Summarization
Text summarization TextRank/LexRank
Video summarization
Various categories of approaches
Strength and limitations Applications
School of Computer Science
Exam Preparation
Materials
Lecture notes
Master core concepts and techniques
Reference books, such as
Book1:Chapter1,2,3,5,6,7,8,12,14,20 Book2:Chapter1,2,4,6,7,8,19,20,21
Tutorials
Expectations
To understand
Concepts, contents, and principles
To be skilful
Practice algorithms/techniques with calculating
Solving problems Analyzing results Summarize ideas
School of Computer Science
Sample (Sub)Questions
Explain whether the indexing method employed in textual information retrieval can be similarly utilized for multimedia information retrieval. [6 marks]
Consider a collection made of the following 4 documents d1, d2, d3, and d4 (one document per line in italic):
d1: John gives a book to Mary
d2: John who reads a book loves Mary
d3: Who does John think Mary loves
d4: John thinks a book is a good gift
Perform a reasonable pre-processing (i.e. stop word removal and stemming) and build an index for these documents to support keyword based queries. [8 marks]
Explain one issue with collaborative filtering. [3 marks]
Consider the performance of following email anti-spam system against the ground truth. [3 marks]
We have a sample of 15 emails. The system reports that the following emails as spam: {1,4,7,8,11,13}, while the ground truth (labelled by human) denotes email {1,3,7,8,9,11,13,15} as spam. Calculate precision, recall and f-measure of the system
Suppose now you have an album of photos on your mobile taken from different trips, including photos of yourself and other people, photos of food dishes, as well as scenic photos without people. Design a fully automatic system using what you have learned in this unit, to organize the photo album by putting related photos together. [8 marks]
Note that Metadata and Geo-data will not be available in this case; your system should only consider the visual information in the photos.
School of Computer Science
Final Exam
2-hour + 10 min reading
Short-answer questions
No multiple choice questions
Problem solving
Each question may have multiple sub-questions
Writing/Drawing/Typing and submission
School of Computer Science
Final Exam
Open book (scenario 2): locally saved notes on the student’s computer and paper-based resources (including printed/handwritten notes and textbooks).
Exam Guide
https://canvas.sydney.edu.au/courses/27821/pages/sem-1-2022-process-
for-final-exams#step7
Resources on the Taking online exams
https://canvas.sydney.edu.au/courses/23380
1. NOonlineresources (including UoS Canvas resources and website search such as Google)
2. DoNOTusebrowser to open PDF files.
School of Computer Science
Capstone Projects on Multimedia Computing (12cpt/18cpt/Honours)
MultimediaInformationRetrieval
Video/Text/Multi-modal Summarization
Sketch Retrieval, Video Moment Retrieval, Cross-Modal/Multi-modal Retrieval, Conversational Retrieval
HumanMotionAnalysis,Modeling,Animation,andSynthesis
Human action recognition, 3D human/cloth animation, 3D human reconstruction Human emotion and affect analysis
ContentGeneration
Video prediction and generation (e.g., talking head), view synthesis, sketch/line-art
colorization, audio/speech generation, 3D scene reconstruction
HealthandMedicine
Surgical Video Analysis, Medical Image Analysis, EEG signal analysis
Agricultureandearthobservation Remote sensing image analysis
School of Computer Science
Wish You All Great Success!
Secrets of Success
http://www.ted.com/talks/richard_st_john_s_8_se
crets_of_success.html
Amazing Career Advice For College Grads From LinkedIn’s Billionaire
https://www.businessinsider.com.au/amazing- career-advice-for-college-grads-from-linkedins- billionaire-founder-2013-5
School of Computer Science
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com