Lab 3
Lab 3
Objectives: This lab presents a revision of important concepts from weeks 4 and 5 and aims to make you familiar with implementing specific algorithms.
Materials: The sample images to be used in all the questions of this lab are available in WebCMS3. You are required to use OpenCV 3+ with Python 3 (see specific version to use for this lab in section 1.1).
Submission: Question 4 is assessable after the lab and is worth 2.5% of the total course marks. Submit your code and results as a Jupyter notebook in a zip file
1 Image Segmentation
Image segmentation can be thought of as labelling pixels in an image. It is an important research topic in computer vision and comes in many different flavours: interactive segmentation, semantic segmentation, instance segmentation, and many more. In this lab the MeanShift clustering algorithm and the Watershed algorithm will be used to solve unsupervised image segmentation.
1.1 MeanShift
It is a clustering algorithm that assigns pixels to clusters by iteratively shifting points towards the modes in the feature space, where a mode is a position with the locally highest number of data points (highest density). A visualisation can be seen here.
1.2 Watershed
It is a transformation that aims to segment the regions of interest in a grayscale image. This method is particularly useful when two regions of interest are close to each other; that is, their edges touch. It treats the image as a topographic map, with the intensity of each pixel
representing the height. For instance, dark areas are considered to be ¡®lower¡¯, and act as troughs. Bright areas are ¡®higher¡¯, acting as hills or as a mountain ridge.
QUESTION 1: MEANSHIFT
Given two images (imgQ12-1.png, imgQ12-2.png) use the MeanShift algorithm to segment the images.
Hint: Use MeanShift clustering from scikit-learn.
1. Once you have read the images into numpy arrays, extract each colour channel (R, G, B) so you can use each as a variable for classification. To do this you will need to convert the colour matrices into a flattened vector as depicted in the image below.
2. Then you can use the new flattened colour sample matrix (10,000 x 3 if your original image was 100 x 100) as your variable for classification.
3. Use the MeanShift fit_predict function to perform clustering and save the cluster labels, which we want to observe.
QUESTION 2: WATERSHED
Given two images (imgQ12-1.png, imgQ12-2.png) use Watershed transformation to segment grayscale versions of the images.
Hint: Use Watershed segmentation from scikit-learn.
1. Convert the image to grayscale.
2. Calculate the distance transform of the image. Note: this is a vital step of the
Watershed algorithm. Visualising this step may help you understand how the algorithm works! Plot the result of the distance transform to see what is happening under the hood.
3. Generate the Watershed markers as the ¡®clusters¡¯ furthest away from the background. This can be syntactically confusing so make sure to check the example code in the link above.
Visualising the Watershed: The left image can be topographically represented as the image on the right. Adopted from Agarwal 2015.
4. Perform Watershed on the image. This is the part where the image is ¡®flooded¡¯ and the water sinks to the ¡®catchment basins¡¯ based on the markers found in Step 3.
QUESTION 3: MOTION DETECTION
Given a background image of a scene from a video (imgQ31.jpg) and a subsequent frame (imgQ32.jpg) use Image Subtraction to detect change. Draw bounding boxes around the regions where change is detected.
(Hint: Use algorithm discussed in the Week 5 Motion Lecture (slide no: 13))
QUESTION 4: ASSESSMENT QUESTION (2.5 marks)
a) Given two images (imgQ41.jpg, imgQ42.jpg) compute the segmentation labels for the
two images using meanshift and watershed methods.
b) Display each image with its meanshift and watershed segmentation in the format
specified below.
c) Given the ground truth segmentation labels in a pickle file (mask.pickle) compute the
dice-coefficient and identify the best segmentation method for each image (Higher
the dice-coefficient better the segmentation accuracy).
d) Display each image along with its best segmentation mask and dice score in the
format given below.
Note: Dice calculation is removed from the assessment question.
Image
Watershed Meanshift