Chapter 3 Spatial filtering
Objectives
• Applications
• Implementation via convolution
• Smoothing spatial filters (Remove noise) – Linear filters
– Order-statistic filters
• Sharpening spatial filters (Feature enhancement)
– First derivative filters
– Second derivative filters
2
Limitations of point operations
• They don’t know where they are in an image
• They don’t know anything about their neighbours.
• Most image features (edges, textures etc.) involve a spatial neighbourhood of pixels
• Must take into account neighbouring pixels if we want to manipulate these features
3
Applications
• Noise reduction
– Smoothing to remove high-frequency noise
Ultrasound Prostate Image
4
Applications
• Feature enhancement – Enhance edges
5
Definition of spatial filters
• Aspatialfilterisanimageoperationwhereeach pixel value I(i,j) is changed by a function of the intensities of pixels in a neighborhood of (i,j).
6
Implementation through convolution
• The convolution of an image I(i,j) with a kernel g(i,j) gives another signal Io(i,j):
where I(i,j) is the input image
Io(i,j) is the output image
g(i,j) is a convolution kernel
• Usually a small matrix (3×3, 5×5 etc.) • User-designed to achieve task
7
Convolution
• So, you need to understand the relation between g(m,n) and g(i-m, j-n).
8
Convolution
• So, what is Io(i,j)?
To obtain Io(i,j):
For each pixel where g(i-m, j-n) overlaps with I(m,n), (i) compute the product between corresponding
pixels and
(ii) sum the products associated with each pixel.
9
Implementation of convolution
Step 1: Flip the kernel g in horizontal and vertical directions to generate the computational molecule h.
10
Convolution vs. Correlation
• Note: the computation molecule h is therefore the flipped version of the convolution kernel g. We will mostly talk about the computation molecule h in this chapter.
•
• denotes the correlation operation in which the kernel does not need to be flipped.
11
Implementation of convolution
• Step 2: Overlay centre of h on top of a pixel (i,j) on I, multiply corresponding elements, sum the products and place result in (i,j) of output image Io.
• Step 3: Repeat Step 2 for each pixel.
Computational molecule
[Courtesy of http://www.songho.ca/dsp/convolution/convolution.html#convolution_2d] 12
Smoothing spatial filters
• Referred to as low-pass filters or averaging filters
• Reduce sharp transition in grey levels
• Uses:
– Noise reduction
– Blur out small details prior to large object detection
• Main drawback is that edges, as prominent features for image understanding, are also sharp transitions that are blurred out.
13
Smoothing spatial filters
• MxN boxcar filter (e.g., 3×3 boxcar filter)
• Weighted filter: give central pixel more weight than others
14
Example: NxN boxcar filter
• Original image (top left) contains squares of sizes 3, 5, 9, 15, 25, 35, 45, and 55 and are 25 pixels apart
• Vertical bars are 5 pixels wide and 100 pixels high
• Filter with boxcar filter of
size N = 3: slight blurring with objects about same size as filter being blurred the most (e.g., 3×3 and 5×5 squares and noise in rectangles)
• N=15andN=35:Extreme blurring. Used to eliminate small objects (e.g., 3 smallest squares, 2 circles and most noisy rectangles).
15
Example application
• Spatial smoothing is used to remove small objects prior to large object detection
• Small objects blended into background, making large objects easier to detect
Original image
15 x 15 boxcar filter applied
Threshold at 15% of maximum grey level
16
Python Implementation
17
Gaussian Smoothing Filter
• 1D
• 2D
• is the standard deviation
• In skimage, mask size is by default.
18
Gaussian Smoothing Filter
19
Border Handling in Python
• How do we do filtering if the computational molecule do not entirely cover the image?
20
Options for Border Handling
1. ‘reflect’(dcba|abcd|dcba)
– The input is extended by reflecting about the edge of the last pixel.
2. ‘constant’(kkkk|abcd|kkkk)
– The input is extended by filling all values beyond the edge with the same constant value, defined by the cval parameter (k above).
3. ‘nearest’ (a a a a | a b c d | d d d d)
– The input is extended by replicating the last pixel.
4. ‘mirror’ (d c b | a b c d | c b a)
– The input is extended by reflecting about the center of the last pixel.
5. ‘wrap’(abcd|abcd|abcd)
– The input is extended by wrapping around to the opposite edge.
21
Order-statistic filters
• Smoothing filters not related to convolution.
• Based on ordering (ranking) pixels in window and replacing central pixel with ranking result.
• Median filter: replace value at central pixel (i,j) by median in a MxN neighbourhood.
– e.g.,
3×3 neighbourhood
22
Example on median filter
• Medianfilterprovides excellent noise reduction for salt-and- pepper noise with less blurring than spatial averaging
Originalimage
10% salt-and-pepper noise
• Ina3x3neighborhood, median filter does well for cases when the number of corrupted pixels is lower than or equal to 4 (very unlikely to have that
15×15 boxcar filter
median filter
many corrupted pixels)
23
Python Implementation
24
Sharpening filters
• Highlight grey-level changes in the image
• Features such as edges and textures are characterized by grey-level changes
• Changes can be quantified by spatial differentiation (either first- or second-order derivatives).
25
First-order derivatives
• Discrete implementation of first-order derivative in x-direction (j-direction):
• Discrete implementation of first-order derivative in y-direction (i-direction):
h -1 0 1
h
-1 0 1
26
First-order derivatives
• Gradient of I(x,y)
• Magnitude of gradient:
– Tell us about strength of edge
27
First-order derivatives
• Simple first-order derivation may be sensitive to noise.
• Introduce 3×3 masks (Prewitt operators)
– In x-direction: consider derivatives along two neighbouring rows also:
– In y-direction:
28
First-order derivatives
• Variation: Sobel operators put heavier weight to middle row/column.
• Calculate strength of edges in I (either using Prewitt or Sobel operators):
29
Python Implementation
30
Second-order derivatives
• Firstderivativedetects ramp as thick edge → Not precise enough
• Secondderivative detects the start and end of the ramp → gives finer details
• Inadditiontoedge detection, the second derivative can also be used in image enhancement as it has the ability to capture fine details.
31
Second-order derivatives
• The x- and y-direction second order derivative are combined as expressed as the Laplacian operator:
32
Second-order derivatives
𝑥 1 𝑥 1
33
Second-order derivatives
• 2nd derivative along x direction
• Along y direction
34
Second-order derivatives
010 1 -4 1 010
• If second derivatives in the two diagonals are also considered, the mask becomes:
111 1 -8 1 111
35
Example
Original image
Laplacian-filtered Image
36
Application in image enhancement
• Subtract Laplacian-filtered image to the original image to highlight fine details.
2nd derivative
Inversion of 2nd derivative
37
Application in image enhancement
• What kernel do we use to generate I0?
– Could generate first, and then subtract it from I → Not efficient. Need two convolutions & subtraction.
– Could generate I0 by one convolution because of the distributive property of convolution, i.e.,
where h1, h2 are two computation molecules.
38
Application in image enhancement
000
010 000
010 1 ‐4 1 010
000‐010 0 1 0 1 ‐4 1 000 010
0 ‐1 0 ‐1 5 ‐1 0 ‐1 0
39
Application in image enhancement
• Computational Molecule to generate I0:
0 -1 0 -1 5 -1 0 -1 0
• Depending on the kernel used to compute the Laplacian-filtered image, this image enhancement operation can be implemented by:
-1 -1 -1 -1 9 -1 -1 -1 -1
40
Application in image enhancement
Original image
41
Unsharp Masking
• Subtracting an unsharp (smoothed) image from the original image.
• Algorithm:
1. Blur the original
image
2. Subtract the
blurred image from the original image, creating a mask
3. Add the mask to the original image
42
Unsharp Masking
43
Python Implementation
44
Summary
• Defined digital convolution – Convolution kernel
– Computational molecule
• Discussed the following filters: – Smoothing filters
– Order-statistic filters
– First-order derivative filters
– Second-order derivative filters – Unsharp masking
45