School of Electronic Engineering and Computer Science
ECS797 Machine Learning for Visual Data Analysis Lab 4: Part-based Action localisation
Introduction
The outcomes from the lab are to be handed in as a .zip file that contains a report and programs that show that you have completed the steps of the lab successfully. Details are given at the end of this sheet.
1. Getting Started
1. Download the file “ECS797Lab4.tar.gz” from Qmplus and extract the contents in the directory ecs797/lab4.
2. Check first there are two sub-folders and one main file:
a. data -> stores data in .mat files
b. test_fea -> contains 30 test sequences (10 sequences/class) in the .mat
format
c. lab4.m-> main executable Matlab file
3. Start Matlab. Use “cd
4. In Matlab, load the “Lab4.m” file in the matlab editor. You will need to edit the code in the file in order to complete the parts of the lab that are in red with underline.
We use the terms codebook and dictionary interchangeably in this document
2. Creating Voting map for each of the codewords
1. Note the codebook creation by k-means clustering of the Spatiotemporal interest point descriptors (STIPs) in the ism.m.
2. The voting map creation for all codewords is performed in the file Votemap.m. Make note of how we store votes for the centre of the object at the current frame and votes for the start and end of action as well as votes for the size of the object’s bounding box. Note and understand in file Votemap.m the datastructure DataStructureVotemap. Make sure that you understand how we store the scale (both spatial and temporal) at which the space_time interest point has been detected.
3. Localising actions in image sequences
1. The function ism_test_voting calculates the voting maps for a single image sequence. Calculate the Euclidean matrix between the dictionary elements and the descriptors extracted in an image sequence.
2. Write a function that implements the voting scheme for the following properties: a) the spatial centre of the action at the current frame, b) the start and the end of action and c) the width and the height of the bounding box of the action in the current frame. The details are given in houghvoting.m
4. Evaluation
1. Using the provided code in ism_test_voting.m and recall_prec_curve.m, plot the Recall precision curves for each class.
2. Assign each sequence to a class according to which hypothesis received the higher number of votes (hint: use the values of the matrix TP_FP_mat). Report the misclassification error, or build the confusion matrix.
5. Dictionary size
1. Perform the localisation experiment using a very small dictionary and report the precision – recall curves. Hint: Cluster the descriptors into a small number of clusters (e.g. 20).
2. Explain the drop in the performance.
6. Handing In
Create a folder that will contain:
• A .pdf report that contains the answers to the exercises, the answers to specific questions, plots from experiments and program listings (including comments).
• The programs files
Create a .zip file and submit electronically.
IMPORTANT: Plagiarism (copying from other students, or copying the work of
others without proper referencing) is cheating, and will not be tolerated.
IF TWO “FOLDERS” ARE FOUND TO CONTAIN IDENTICAL MATERIAL, BOTH WILL BE GIVEN A MARK OF ZERO.