Low-Level Computer Vision with MATLAB
1. Introduction
This laboratory session provides the opportunity to experiment with a number of image processing techniques used for low-level computer vision. The hours timetabled for this laboratory are insufficient to complete it. You should therefore work on this in your own time, and use the timetabled hours to get help and clarification.
This laboratory is assessed. A type-written report needs to be submitted online through KEATS by the deadline specified on the module’s KEATS webpage.
I suggest you write a MATLAB script file, containing all your work, so that you can easily save your work and return to it at a later time.
2. Convolution
In MATLAB, 2-D (and 1-D) convolution is performed using the function conv2(I,H,shape), where I is a matrix representing an image, H is a matrix representing the mask, and shape is a parameter defining the size of the output. To perform convolution we need to convert an image into a standard, double precision floating-point, matrix.
MATLAB provides a command fspecial that can generate some masks commonly used in image processing.
3. Smoothing Masks
3.1 Box masks
A box, or average, mask is just an array of equal numbers that sum to one. Such a filter when convolved with an image provides smoothing.
Boxes like this one indicate instructions that you need to carry out, or commands you need to execute, in MATLAB.
1.0.1: Boxes like this one describe what is required for your report. Note that it is not necessary to write a formal report (one with an abstract, introduction, methods, results and discussion), you simply need to provide the information requested in boxes like this one. You also do not need to provide your code, except where you are specifically asked to do so.
Execute the command:
help conv2
Determine what value of parameter “shape” is required to generate an output image the same size as the input image.
Execute the command:
help fspecial
Determine what parameter values are required to generate box (or average) masks and Gaussian masks.
Copy the following two image files to the directory in which you are running MATLAB: rooster.jpg boxes.pgm
Load these images into MATLAB.
Convert these images to grayscale with double data type. Use these grayscale, double, images throughout the rest of the laboratory.
Using the command ones or the command fspecial to create a 5×5 box mask and a 25×25 box mask. Ensure these are correctly normalised.
Convolve both images with both these masks so that the resulting four images are the same size as the original images.
Display the output of each convolution as a subplot in the same window. Use a “gray” colormap and provide a colorbar for each image.
3.1.1: In your report include a print out of the figure with four subplots that you have just created. Briefly describe and explain the results you have obtained. [4 marks]
3.2 Gaussian mask
Repeat the preceding experiment using two Gaussian masks (generated using the fspecial command) with standard deviations 1.5 and 10. Ensure the size of each mask is sufficient to accurately represent the Gaussian.
Using the command fspecial create a 1-D Gaussian mask of size [1, 60] and with standard deviation 10.
Convolve this mask with the transpose of itself (using ‘full’ as the shape parameter in the conv2 command) in order to generate a 2-D Gaussian with standard deviation 10.
(a) Use the tic and toc commands to measure the total time taken to convolve the rooster image with the 1-D Gaussian, and to convolve the result of that convolution with the transpose of the 1-D Gaussian.
(b) Use the tic and toc commands to measure the time taken to convolve the rooster image with the 2-D Gaussian.
Determine whether or not the two images that result from (a) and (b) are equivalent.
3.2.1: In your report include a print out of the figure with four subplots that you have created using Gaussian masks. Briefly describe and explain the results you have obtained, and compare these results with those obtained using the box masks. [4 marks]
3.2.2: Report the times that you have measured in steps (a) and (b) and explain the reason for the difference. How did you check if the images that result from (a) and (b) are equivalent, and what was the result of this check? Briefly explain your results and their implications. [5 marks]
4. Difference Masks
4.1 Difference Masks (1-D)
To gain insight into the effects of difference masks we will first look at their effects on a simple 1-D signal.
4.2 Difference Masks (2-D)
In two dimensions a finite difference approximation to the second derivative is given by a Laplacian mask (strictly speaking the masks below are the additive inverse of the Laplacian, and hence approximate the minus of the second derivative).
4.3 Other Difference masks
A number of other difference masks for approximating the intensity-level gradient of an image have been proposed, and are in common usage for edge detection. Two of these are the Sobel and the Prewitt edge detectors.
Execute the following commands:
y=sin([0:0.01:2*pi]); subplot(3,1,1), plot(y); yd1=conv2(y,[-1,1],’valid’); subplot(3,1,2), plot(yd1) yd2=conv2(y,[-1,2,-1],’valid’); subplot(3,1,3), plot(yd2)
4.1.1: Plot the results in your report. Briefly describe and explain the results you have obtained
[2 marks]
Create two Laplacian masks with different amplitudes, i.e.:
-1/8 -1/8 -1/8 -1 -1 -1 -1/8 1 -1/8 and -1 8 -1 -1/8 -1/8 -1/8 -1 -1 -1
Convolve the boxes image with both these masks, and show the results.
4.2.1: In your report include the resulting images and note any difference in the two images that result from these two convolutions. [2 marks]
Use the command fspecial to generate a Sobel mask.
Convolve the boxes image once with the Sobel mask, and once with the transpose of the Sobel mask.
5. Edge Detection
To perform edge detection (i.e. to locate intensity discontinuities in an image), first and second order directional derivative masks are usually combined with a Gaussian mask to help suppress noise. Combining first derivative masks with a Gaussian results in Gaussian derivative masks, whereas combining the omni-directional second- derivative mask (the Laplacian) with a Gaussian results in a Laplacian of Gaussian mask.
5.1 Gaussian derivative masks
Create two Gaussian derivative masks (one for the derivative in the x direction, and one for the derivative in the y direction).
To do this convolve a 2-D Gaussian (with standard deviation 5) once with the mask [-1,1] and once with the transpose of this mask (use ‘same’ as the shape parameter in conv2).
Generate a mesh plots of the two Gaussian derivative masks you have created, put these plots as subplot(2,2,1) and subplot(2,2,2) in a figure.
Convolve the boxes image with the two Gaussian derivative masks you have created. Display images showing the output of these two convolutions as subplot(2,2,3) and
subplot(2,2,4) in the same window. Use a “jet” colormap and provide a colorbar.
Repeat the above steps using a Gaussian with standard deviation 1.5 (rather than 5).
To combine the horizontal and vertical edge images into a single image showing intensity-level discontinuities in any direction, we can calculate the L2-norm as follows: Ibdg=sqrt(Idgx.^2+Idgy.^2);
The above assumes that you have given your image convolved with the horizontal Gaussian derivative mask the variable name “Idgx”, and that you have given your image convolved with the vertical Gaussian derivative mask the variable name “Idgy”.
Create a image showing the L2-norm generated from the Gaussian derivative mask produced using a Gaussian with standard deviation 1.5 for (a) the boxes image, and (b) the rooster image. Use the “gray” colormap.
5.1.3: In your report include a figure with both these images as two subplots. [2 marks]
5.1.1: In your report include a print out of the figure with four subplots that you have created. What is the value of the convolved image at the locations of large intensity discontinuities (i.e. at edges)? [2 marks]
5.1.2: In your report include a print out of the second figure with four subplots that you have created . What is the value of the convolved image at the locations of large intensity discontinuities (i.e. at edges)? [2 marks]
5.2 Laplacian of Gaussian (LoG) mask
Display the output of these two convolutions as subplot(2,2,1) and subplot(2,2,2) in the same window. Use a “jet” colormap and provide a colorbar.
Use the command fspecial to generate a Prewitt mask.
Convolve the boxes image once with the Prewitt mask, and once with the transpose of the Prewitt
mask.
Display the output of these two convolutions as subplot(2,2,3) and subplot(2,2,4) in the same window. Use a “jet” colormap and provide a colorbar.
Create a Laplacian of Gaussian mask by convolving a 2-D Gaussian (with standard deviation 1.5) with the smaller amplitude Laplacian from section 4.2 (use ‘valid’ as the shape parameter in conv2).
Generate a mesh plot of the Laplacian of Gaussian mask you have created, put this as subplot(2,2,1) in a figure.
Convolve the boxes image with the Laplacian of Gaussian mask you have created.
Display images showing the output of this convolution as subplot(2,2,3) in the same window. Use a “jet” colormap and provide a colorbar.
5.3 Difference of Gaussians (DoG) mask
The LoG mask is usually approximated by using a Difference of Gaussians.
Create a Difference of Gaussians mask by subtracting one Gaussian (with standard deviation 6) from another Gaussian (with standard deviation 3).
Generate a mesh plot of the Difference of Gaussian mask you have created.
Compare the DoG mask created above with the LoG mask created in the previous section (NOT the
LoG mask generated by fspecial) using a Gaussian with standard deviation 5.
Calculate a numerical measure of their similarity using the following commands:
log=log./max(max(log));
dog=dog./max(max(dog));
sqrt(sum(sum((dog-log).^2)))
The above assumes that you have given your LoG mask the variable name “log”, that you have given your DoG mask the variable name “dog”, and that you have created two masks of the same size.
Write a programme that will systematically vary the standard deviations (in steps of 0.1) for the two Gaussians used to generate the DoG mask and use this code to search for the optimal values of these standard deviations which will generate a DoG mask that is almost identical to the LoG mask created (in the previous section) using a Gaussian with standard deviation 5.
5.3.1: Provide a listing of your code and report the two standard deviations. [3 marks]
6. Multi-Scale Representations
6.1 Gaussian Image Pyramid
In order to generate an Gaussian image pyramid, we can perform the following command:
Ia2 = imresize(conv2(Ia1,g,’same’), 0.5, ‘nearest’);
Where Ia1 is an image at one scale and Ia2 is the image at the next larger scale in the pyramid, and g is a 2D gaussian with standard deviation 1. This command convolves Ia1 with a Gaussian with standard deviation 1, and then down-samples the result by a factor of 2.
Use this command recursively to create a 4 level Gaussian pyramid of the rooster image, and display the outputs as four subplots in the same window.
6.1.1: In your report include a print out of the figure with four subplots that you have created.
[2 marks]
6.2 Laplacian Image Pyramid
With the aid of the lecture slides, create a 4 level Laplacian image pyramid, and display the outputs as four subplots in the same window.
6.2.1: In your report include a print out of the figure with four subplots that you have just created, and a listing of your code for generating this Laplacian pyramid. [4 marks] Comment briefly on this result in comparison to the results from section 5.2. [2 marks]
7. Edge Detection Challenge
Repeat the above steps using a Gaussian with standard deviation 5 (rather than 1.5), putting the mesh plot of the mask as subplot(2,2,2) and the convolved image as subplot(2,2,4)
Convolve the rooster image with each of the two Laplacian of Gaussian masks you created above, and display the resulting output images.
5.2.1: In your report include a print out of the figure with four subplots that you have created. What is the value of the convolved image at the locations of large intensity discontinuities (i.e. at edges)? What is the value of the convolved image near to image locations of large intensity discontinuities (i.e. near edges)? [2 marks]
5.2.2: What effect does changing the standard deviation of the Gaussian used to create the Laplacian of Gaussian mask have on the result? [2 marks]
MATLAB provides a function edge that will apply one of several edge detection methods to an image, and return a binary image in which only “edges” (i.e. significant intensity discontinuities) have a value of 1. The edge detection methods available include several we have looked at in this laboratory session (e.g. Prewitt, Sobel, LoG) and a popular method (canny) that is derived from the Gaussian derivative method explored earlier. Below are three grey-scale images. Next to each image are four binary images showing the edges identified by four different human observers. Notice that human observers do not agree on what constitutes an edge.
Load each of the grey-scale images below into MATLAB.
For one image, use the command edge to generate a binary edge image.
Spend 10 minutes experimenting with changing the parameters of the edge command to try to improve the edge image: try to find parameters that generate an edge image that includes true edges (ones that at least one human observer has marked as an edge), and excluded false edges (ones that no human observer has marked as an edge).
Without changing any parameters, perform the same edge detection on the other two images.
7.2.1: Include in your report the MATLAB command that you have used to process the images, and the three edge images that you have created. Comment on the ability of the edge command to find edges across different images. [2 marks]