编程辅导 EBU7240 Computer Vision

EBU7240 Computer Vision
Restoration: spatial filtering
Semester 1
Changjae Oh

• Neighborhood in an image
• Convolution: review
̶ From ‘Signal Processing’ Lecture
• Spatial Filtering
Low-pass (or high-pass) filter
Gaussian (or Laplacian) filter
Mean, median, or mode filter
Advanced filters: Bilateral filter, Non-local means filter

Neighborhood in an image
• Spatial filtering is performed for each pixel (𝑖, 𝑗) using neighborhoods of (𝑖, 𝑗). ̶ 𝐼(𝑖, 𝑗): input image 𝑂(𝑖, 𝑗): output image
̶ 𝑤(𝑠, 𝑡): filtering kernel (a.k.a. mask)
𝑂 𝑖,𝑗 = ෍ ෍ 𝑤 𝑠,𝑡 𝐼(𝑖+𝑠,𝑗+𝑡) 𝑠=−𝑎 𝑡=−𝑏
• The filtering operation is defined according to what kind of 𝑤(𝑠, 𝑡) is used.
• The filter serves as an essential building block for many applications. ̶ Blurring, sharpening, image restoration, and so on

Spatial Filtering:
• Filter kernel should be defined accordingly, depending on applications.
𝑂 𝑖,𝑗 = ෍ ෍ 𝑤 𝑠,𝑡 𝐼(𝑖+𝑠,𝑗+𝑡) 𝑠=−𝑎 𝑡=−𝑏
Filter Kernel (Mask)

Filtering = SUM (Mask .
Neighborhood)
𝐼(𝑖 − 1, 𝑗 − 2)
𝐼(𝑖 − 1, 𝑗 − 1)
𝐼(𝑖 − 1, 𝑗)
𝐼(𝑖 − 1, 𝑗 + 1)
𝐼(𝑖 − 1, 𝑗 + 2)
𝐼(𝑖, 𝑗 − 2)
𝐼(𝑖, 𝑗 − 1)
𝐼(𝑖, 𝑗 + 1)
𝐼(𝑖, 𝑗 + 2)
𝐼(𝑖 + 1, 𝑗 − 2)
𝐼(𝑖 + 1, 𝑗 − 1)
𝐼(𝑖 + 1, 𝑗)
𝐼(𝑖 + 1, 𝑗 + 1)
𝐼(𝑖 + 1, 𝑗 + 2)

Filtering = SUM (Mask .
Neighborhood)

Convolution:
• Filtering = Convolution!
From the perspective of signal processing
̶ When the filter kernel is symmetric
• Let’s start with 1-D convolution (from signal processing)
Convolution integral
𝑦𝑡 =𝑥𝑡 ∗h𝑡 = න𝑥𝜏h𝑡−𝜏𝑑𝜏 −∞
𝑦[𝑛]=𝑥[𝑛]∗h[𝑛]= ෍ 𝑥𝑚h[𝑛−𝑚] 𝑚=−∞
For continuous signal
𝑦𝑛 =⋯+𝑥−1h𝑛+1 +𝑥0h𝑛 +𝑥1h𝑛−1 +⋯
For discrete signal

Convolution:
• From the perspective of signal processing
From the perspective of signal processing
An output signal 𝑦[𝑛] is obtained by passing an input signal 𝑥[𝑛] to a discrete system
with a response function h[𝑛].
𝑦[𝑛]=𝑥[𝑛]∗h[𝑛]= ෍ 𝑥𝑚h[𝑛−𝑚]
Note convolution in spatial domain = multiplication in frequency domain

Convolution Properties
• Associative property
̶ Impulse response of a cascade connection
= Convolution of the individual impulse responses
• Distributive property
̶ Impulse response of a parallel connection of LTI systems = Sum of the individual impulse responses.

• The convolution on the continuous domain ∞
𝑦𝑡 =𝑥𝑡 ∗h𝑡 = න𝑥𝜏h𝑡−𝜏𝑑𝜏 −∞
For illustration, let an input signal x(t) and the impulse response h(t) be the two functions below.

• The convolution on the continuous domain
𝑦𝑡 =𝑥𝑡 ∗h𝑡 = න𝑥𝜏h𝑡−𝜏𝑑𝜏 −∞
̶ The functional transformation from h(𝑡) to h(𝑡 − 𝜏)
h  ⎯ ⎯ ⎯→ h −  ⎯ ⎯ ⎯→ h −  − t = h t −  () →− ( ) →−t (( )) ( )

• The convolution on the continuous domain The convolution value at 𝑡:
The area under the product of 𝑥(𝑡) and h(𝑡 − 𝜏). For example, let 𝑡 = 5.
For 𝑡 = 5, the area under the product is zero. 𝑦5=𝑥5∗h5
= න𝑥𝜏h5−𝜏𝑑𝜏=0 −∞

• The convolution on the continuous domain
When 𝑡 = 0,
𝑦0 =𝑥0 ∗h0 = න𝑥𝜏h−𝜏𝑑𝜏=2 −∞

• The convolution on the continuous domain
The process of convolving to find 𝑦(𝑡) is illustrated below.

• The convolution on the discrete domain
𝑦[𝑛]=𝑥[𝑛]∗h[𝑛]= ෍ 𝑥𝑚h[𝑛−𝑚] 𝑚=−∞
What is 𝑦[𝑛]?
Note) 𝑥: 𝑀 points, h: 𝑁 points →𝑦: 𝑀 + 𝑁 − 1 points

Convolution vs. Filtering 1D convolution
∞ 𝑂[𝑖] = ෍ 𝑤[𝑠]𝐼 𝑖 + 𝑠
𝑦[𝑖] = 𝑥[𝑖] ∗ h[𝑖]
1D filtering
= ෍ h[𝑠]𝑥 𝑖 − 𝑠 𝑠=−∞
Suppose h[𝑖] exists for −𝑎 ≤ 𝑖 ≤ 𝑎 and is symmetric, i.e., h −𝑖 Then, 1D convolution = 1D filtering!
This also applies to 𝑁-D convolution and 𝑁-D filtering (𝑁 ≥ 2).
Remember 2D filtering
𝑂 𝑖,𝑗 = ෍ ෍ 𝑤 𝑠,𝑡 𝐼(𝑖+𝑠,𝑗+𝑡) 𝑠=−𝑎 𝑡=−𝑏
Note) This formulation is different from previous slides, but the result is identical. Namely, 𝑥 ∗ h = h ∗ 𝑥.

2D convolution
Source from D. Lowe

2D convolution

2D convolution
Source from D. Lowe

2D convolution
• Removing Blurred region means:
• Adding it back means:
Source from L. Fei-Fei

2D convolution
• Deep learning: Convolutional Neural Network
Credit: https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/

2D Image Filtering:
• Uniform mean filter
̶ The simplest low-pass filter
Uniform Mean Filter
EBU6018- remember?
Filter kernel

2D Image Filtering:
Uniform Mean Filter
• How to process pixels at image boundary? ̶ 1) Mirroring at an image boundary
𝑂 𝑖,𝑗 = ෍ ෍ 𝑤 𝑠,𝑡 𝐼′(𝑖+1+𝑠,𝑗+1+𝑡) 𝑠=−1 𝑡=−1
𝐼′: mirror image

2D Image Filtering:
• How to process pixels at image boundary?
Uniform Mean Filter
2) Zero padding at an image boundary Common way in convolutional neural networks
𝑂 𝑖,𝑗 = ෍ ෍ 𝑤 𝑠,𝑡 𝐼′(𝑖+1+𝑠,𝑗+1+𝑡) 𝑠=−1 𝑡=−1
𝐼′: zero-padded image

2D Image Filtering:
• How to process pixels at image boundary?
3) Adjusting filter kernel
Uniform Mean Filter
𝑂 𝑖,𝑗 = 𝑠=−1 𝑡=−1
𝑤′ 𝑠,𝑡 𝐼(𝑖+𝑠,𝑗+𝑡)
σ1 σ1 𝑤′ 𝑠,𝑡 𝑠=−1 𝑡=−1
= (0 ≤ 𝑖 + 𝑠 ≤ 𝐻 − 1 & 0 ≤ 𝑗 + 𝑡 ≤ 𝑊 − 1 ? 𝑤 𝑠, 𝑡 : 0)

2D Image Filtering:
Uniform Mean Filter
1111 11⋯1 (a)originalimage,(b)3×3filterwithzeropadding:9 1 1 1 ,(c)9×9filterwithzeropadding:81 ⋮ ⋱ ⋮ ,(d)25×25
1⋯1 111 1⋯1 1⋯1 filter with zero padding: 1 ⋮ ⋱ ⋮ , (e) 25 × 25 filter with mirroring: 1 ⋮ ⋱ ⋮
625 1 ⋯ 1 625 1 ⋯ 1

2D Image Filtering:
• Uniform mean filtering is separable.
Uniform Mean Filter
Separable filtering is VERY important in terms of runtime.
𝑤𝑠,𝑡= 111= 1 111=𝑤𝑠,0∗𝑤(0,𝑡) 9111313 𝑠 𝑠
Cascade Connection

2D Image Filtering: • Gaussian distribution
Gaussian Filter
One of the most commonly used parametric models
𝑭𝒐𝒖𝒓𝒊𝒆𝒓 𝒕𝒓𝒂𝒏𝒔𝒇𝒐𝒓𝒎 𝐺𝑎𝑢𝑠𝑠𝑖𝑎𝑛 𝑓𝑢𝑛𝑐. = 𝐺𝑎𝑢𝑠𝑠𝑖𝑎𝑛 𝑓𝑢𝑛𝑐.
Rotationally symmetric
Separable filter
𝐺𝑎𝑢𝑠𝑠𝑖𝑎𝑛 ∗ 𝐺𝑎𝑢𝑠𝑠𝑖𝑎𝑛 = 𝐺𝑎𝑢𝑠𝑠𝑖𝑎𝑛
1-D 1 (𝑥 − 𝜇)2 𝑓𝑥 = 2𝜋𝜎exp − 2𝜎2
2-D 1 𝑥−𝜇𝑥 2 𝑓𝑥,𝑦 =2𝜋𝜎𝜎 exp − 2𝜎2
− 2𝜎2 𝑥𝑦𝑥𝑦
N-D 1 𝑇 −1
𝑓 𝒎 =|2𝜋|𝑁/2 Σ1/2exp −(𝒎−𝝁) Σ (𝒎−𝝁)
Q: What if 𝜎 approaches infinite?

2D Image Filtering:
• Gaussian filter’s advantage
It considers spatial distances within neighborhoods
Blurred results looks more natural compared to mean filter.
Practical implementation
𝑂 𝑖,𝑗 = ෍ ෍ 𝑤 𝑠,𝑡 𝐼(𝑖+𝑠,𝑗+𝑡) 𝑠=−𝑎 𝑡=−𝑏
Zero mean Gaussian filter
σ𝑎σ𝑏exp−− 𝑠𝑡 𝑚=−𝑎 𝑛=−𝑏 2𝜎 2 2𝜎 2
Inresults,𝜇 =𝜇 =0and𝜎 =𝜎 =𝜎 𝑠𝑡 𝑠𝑡
Gaussian Filter
= 𝑚2 𝑛2 exp −2𝜎2−2𝜎2

2D Image Filtering:
• Gaussian filter’s advantage
̶ Separable
Practical implementation
𝑂 𝑖,𝑗 = ෍ ෍ 𝑤 𝑠,𝑡 𝐼(𝑖+𝑠,𝑗+𝑡) 𝑠=−𝑎 𝑡=−𝑏
Gaussian Filter
𝑂𝑖,𝑗 =෍𝑤 𝑠 ෍𝑤(𝑡)𝐼(𝑖+𝑠,𝑗+𝑡) 𝑠𝑡
1𝑠 𝑡2 𝑛2 exp −2𝜎2
2𝜋𝜎𝜎exp −2𝜎2−2𝜎2 𝑠𝑡𝑠𝑡
= 1 exp−𝑠2 1 exp−𝑡2
2𝜎 2 2𝜎 2 2𝜋𝜎 𝑠 2𝜋𝜎 𝑡
𝑛2 exp −2𝜎2−2𝜎2
σ𝑎σ𝑏exp−− 𝑠𝑡 𝑚=−𝑎 𝑛=−𝑏 2𝜎 2 2𝜎 2
σ𝑏exp− 𝑡 𝑛=−𝑏 2𝜎 2

Frequency: Low
– • Frequency in an image
pass Filters
̶ How often do intensity values vary in neighbourhoods?
• Low-pass filter: uniform average filter, Gaussian filter
• High-pass filter: Sobel filter, Laplacian filter (this estimates intensity change.)
Example of Low-pass filter: 9 Example of High-pass filter:
010 1 −4 1 010

2D Image Filtering:
• Using the first order gradient
Practical implementation
𝛻𝐼=𝐼,𝐼 = 𝑆∗𝐼,|𝑆∗𝐼| 𝑥𝑦𝑥𝑦

𝛻𝐼= 𝐼,𝐼 = 𝜕𝐼,𝜕𝐼
For color image,
𝑀 𝑥,𝑦 = 𝑀𝑅 𝑥,𝑦 +𝑀𝐺 𝑥,𝑦 +𝑀𝐵 𝑥,𝑦 /3
Sobel filter output
𝑀𝑥,𝑦= 𝐼2+𝐼2or𝐼 +|𝐼| 𝑥𝑦𝑥𝑦

2D Image Filtering:
• Using the second order gradient
• 1 derivative of an image 𝐼(𝑥, 𝑦)
𝛻𝐼𝑥,𝑦 = 𝜕𝐼,𝜕𝐼 𝜕𝑥 𝜕𝑦
𝐼 𝑥,𝑦 =𝜕𝐼=𝐼𝑥+1,𝑦 −𝐼(𝑥,𝑦) 𝑥 𝜕𝑥
• 2nd derivative of an image 𝐼(𝑥, 𝑦) 𝛻2𝐼𝑥,𝑦 = 𝜕2𝐼,𝜕2𝐼
Laplacian Filter
Laplacian Filtering output 𝐺 𝑥, 𝑦 = 𝜕 𝐼 + 𝜕 𝐼 𝜕𝑥2 𝜕𝑦2
𝐼 𝑥,𝑦 =𝜕𝐼=𝐼𝑥,𝑦+1 −𝐼(𝑥,𝑦) 𝑦 𝜕𝑦
𝜕2𝐼 =𝐼 𝑥,𝑦 −𝐼 𝑥−1,𝑦 =𝐼 𝑥+1,𝑦 +𝐼 𝑥−1,𝑦 −2𝐼(𝑥,𝑦)
𝜕2𝐼 =𝐼 𝑥,𝑦 −𝐼 𝑥,𝑦−1 =𝐼 𝑥,𝑦+1 +𝐼 𝑥,𝑦−1 −2𝐼(𝑥,𝑦)

2D Image Filtering:
• Using the second order gradient
Laplacian filter 𝐿
Laplacian Filter
More general form
For color image,
𝑂 𝑥,𝑦 = 𝑂𝑅 𝑥,𝑦 +𝑂𝐺 𝑥,𝑦 +𝑂𝐵 𝑥,𝑦 /3

• Makes an image look sharper by boosting high-frequency components.
Original image
Unsharp mask result

Scaling matters!
𝑂𝑢𝑡𝑝𝑢𝑡 = (𝐼 − 𝑘𝐿)/(1 − 𝑘)
x = double(imread(‘cameraman.tif’))/255;
f = fspecial(‘average’); xf = filter2(f,x); figure, imshow(xf)
Matlab Code
% k: parameter determining the amount of reducing low-frequency component
% As k increases, the output image looks sharper. But, it is recommended using k below 0.5; k = 0.5;
fi = zeros(3); fi(2,2)=1;
f2 = (fi – k*f)/(1-k);
xf2 = filter2(f2,x);
figure, imshow(xf2)
For color image, perform the operation for RGB channels independently.

𝑂𝑢𝑡𝑝𝑢𝑡 = (𝐼 − 𝑘𝐿)/(1 − 𝑘)
1/9 1/9 1/9 𝑤=0 1 0−𝑘1/9 1/9 1/9 0 0 0 1/9 1/9 1/9
1 0 0 0 𝑘 1/9 1/9 1/9 𝑤=1−𝑘0 1 0−1−𝑘1/9 1/9 1/9 0 0 0 1/9 1/9 1/9

2D Image Filtering:
• Linear vs. Nonlinear filters?
̶ Definition of linearity, and its advantages
• Some instances of nonlinear filter
̶ Max, Min, Median filter
Max filter
Min filter
Median filter
Nonlinear Filter

2D Image Filtering:
Nonlinear Filter
• Median Filter
1. Sort pixels within the window centered at reference pixel
2. Select the median value as an output.
Max filter Min filter Median filter
Q: Mean vs. Median?

• Spatial Filtering
Depends on how to define a filter kernel.
Linear vs. Nonlinear filter: Linear filter is separable!
Mean, median, maximum, minimum, low-pass, high-pass, Gaussian, Laplacian filter Edge Sharpening: combination of low-pass and high-pass filters.
• Convolution
̶ Same as filter
̶ Commutativity, Cascade property, Parallel property

Computer Vision
Restoration: noise and noise removal
Semester 1
Changjae Oh

• Image Degradation Model
̶ Image Noise
• Noise removal
Salt-and-Pepper Noise Removal Gaussian Noise Removal Periodic Noise Removal

Image Degradation Model
• Image restoration
̶ Aims to reduce the image degradation
• Types of image degradation
̶ Noise, our-of-focus blur, motion blur

Image Degradation Model
• When an input degraded image 𝑔(𝑥, 𝑦) is given, our goal is to estimate 𝑓(𝑥, 𝑦). 𝑔 𝑥,𝑦 =h 𝑥,𝑦 ∗𝑓 𝑥,𝑦 +𝑛(𝑥,𝑦)
• 𝑛(𝑥, 𝑦): Additive noise
• h 𝑥, 𝑦 : Blur kernel, which is the same as filtering mask.
• In frequency domain (using 2D DFT)?
𝐺 𝑢,𝑣 =𝐻 𝑢,𝑣 𝐹 𝑢,𝑣 +𝑁(𝑢,𝑣)
𝐹𝑢,𝑣 = 𝐺𝑢,𝑣 −𝑁𝑢,𝑣 /𝐻𝑢,𝑣
(𝑥, 𝑦): 2D image coordinate (𝑢, 𝑣): 2D frequency coord.
Is that it? It is not such a simple problem.
1) 𝑁(𝑢, 𝑣) is not known. 2) What if 𝐻(𝑢, 𝑣) = 0?

Image Noise
̶ Any kind of degradation in an image caused by external disturbance
̶ To make the problem simple, we pre-assume the noise models.
The most appropriate restoration may vary depending on the noise types.
• Noise types
̶ Periodic noise, Salt and Pepper noise, Gaussian noise, Speckle noise
• In the beginning, let’s think of the case using h(𝑥,𝑦)=1 𝑔𝑥,𝑦 =𝑓𝑥,𝑦 +𝑛(𝑥,𝑦)

Image Noise:
Salt and Pepper Noise
• Sharp and sudden disturbances
• Image is randomly scattered as white (salt) or black (pepper) pixels.

Image Noise: Gaussian Noise • Additive White Gaussian Noise (AWGN)
Additive noise: 𝐼 (𝑥, 𝑦) = 𝐼(𝑥, 𝑦) + 𝑁(𝑥, 𝑦) 𝐺
White noise: randomly fluctuated and normally distributed Most approaches assume this type of noise.
Usually, zero mean AWGN is assumed (𝜇 = 0).
The probability density function (PDF) 𝑃 of a Gaussian random variable 𝑧 is
𝑃(𝑧=𝑁)= 1 exp−(𝑧−𝜇)2 2𝜋𝜎 2𝜎2

Image Noise: Speckle Noise • Multiplicative Noise
𝐼 (𝑥,𝑦) = 𝐼(𝑥,𝑦) + 𝐼(𝑥,𝑦)𝑁(𝑥,𝑦) 1
𝑁(𝑥, 𝑦): zero mean uniform distributed function with 𝜎
This noise is usually in the active radar, synthetic aperture radar (SAR), medical
ultrasound and optical coherence tomography images.
The reduction of Speckle noise is MUCH more difficult due to the multiplication

Image Noise: Gaussian Noise vs. Speckle Noise
Gaussian Noise Speckle Noise

Image Noise: Periodic Noise • Periodic fluctuation
Electrical or electromechanical interference during image acquisition Spatially dependent noise
It can be modeled as sinusoid waves

• Image Degradation Model ̶ Image Noise
• Noise removal
Salt-and-Pepper Noise Removal Gaussian Noise Removal Periodic Noise Removal

Salt and Pepper Noise Removal
• Low-Pass Filtering
̶ For instance, uniform averaging filter or Gaussian averaging filter ̶ Not so effective
• Median Filtering
̶ Median filter works well in the salt-and-pepper noise removal. ̶ Rank-order Filtering is a general formulation of median filter.
• Outlier Rejection Method

Salt and Pepper Noise Removal • Low-Pass Filtering
̶ NOT effective in removing the salt and pepper noise.
Original Image 10% salt and pepper noise 3×3 average filter 7×7 average filter (salt: 5%, pepper: 5%)

Salt and Pepper Noise Removal:
• Median filtering procedure
1. Sort pixels within the window centered at reference pixel
2. Select the median value as an output.
̶ Good performance in the salt-and-pepper noise removal Why?
Median Filtering
3 × 3 median filter

Salt and Pepper Noise Removal: • More results using median filter
Median Filtering
20% salt and pepper noise Result using 3 × 3 median Result applying 3 × 3 (salt: 10%, pepper: 10%) filter median filter twice
Result applying 5 × 5 median filter

Salt and Pepper Noise Removal:
• The brute force implementation of median filtering is very slow.
̶ Different method was proposed to remove the salt-and-pepper noise efficiently.
• Key idea: Outlier detection→rejection
̶ Outliers usually tend to be different from neighboring pixels’ intensities.
• Outlier rejection method
1. Choose a threshold value 𝐷
2. For a given pixel, compare its value 𝑝 with the mean 𝑚 of the values of its eight neighbors
3. If |𝑝 − 𝑚| > 𝐷, then classify the pixel as noisy, otherwise not.
4. If the pixel is noisy, replace its value with m.
Outlier Rejection Method

Salt and Pepper Noise Removal:
Outlier Rejection Method

Gaussian Noise Removal
• Why does an image average filter work well for Gaussian Noise removal?
• Additive White Gaussian Noise (AWGN) 𝑁(𝑥, 𝑦) is added as below. ̶ 𝐼 (𝑥,𝑦)=𝐼(𝑥,𝑦)+𝑁(𝑥,𝑦)
1. Suppose we have 100 noisy images 𝐼 . 𝐺
Average 100 noisy images
𝑖 (𝑥, 𝑦) = 𝐼(𝑥, 𝑦) +
10×10 kernel
෍ 𝑁𝑖(𝑥, 𝑦) This will be close to 0, as AWGN 𝑁 is a
̶ 𝐼𝑖(𝑥,𝑦)=𝐼(𝑥,𝑦)+𝑁𝑖(𝑥,𝑦) 𝑖=1,…,100 𝐺
zero-mean Gaussian PDF.

Gaussian Noise Removal
• Why does an image average filter work well for Gaussian Noise removal?
Image averaging to remove Gaussian noise. (a) 10 images (b) 100 images

Gaussian Noise Removal:
• Simple Average Filtering: Uniform mean filter or Gaussian filter
̶ Using a small window: not so effective in noise removal.
̶ Using a large window: effective in noise removal, but the output is over-smoothed.
Simple Average Filtering
Uniform mean filter to remove Gaussian noise. (a) 3×3 averaging (b) 5×5 averaging

Gaussian Noise Removal: • Bilateral filter for grayscale image
Bilateral Filtering
One of the most popular filters with various applications Considers both spatial and intensity distances
𝑂 𝑖,𝑗 = ෍ ෍ 𝑤 𝑠,𝑡 𝐼(𝑖+𝑠,𝑗+𝑡) 𝑠=−𝑎 𝑡=−𝑏
1 𝑠2 𝑡2 𝐼 𝑖,𝑗 −𝐼(𝑖+𝑠,𝑗+𝑡) 2
𝑤𝑠,𝑡 =𝑊(𝑖,𝑗)exp −2𝜎2−2𝜎2 exp − 2𝜎2 𝑎𝑏𝑠𝑡𝑟
𝑚2 𝑛2 𝐼 𝑖,𝑗 −𝐼(𝑖+𝑚,𝑗+𝑛) 2 𝑊𝑖,𝑗 = ෍ ෍exp −2𝜎2−2𝜎2 exp − 2𝜎2
This can be rewritten as:
𝑂=1෍𝐺𝒑−𝒒𝐺𝐼−𝐼𝐼 𝒑𝜎𝜎𝒑𝒒𝒒
𝑊=෍𝐺 𝒑−𝒒𝐺 𝐼−𝐼 𝒑 𝜎𝑠 𝜎𝑟𝒑𝒒
𝒑 = (𝑖,𝑗) 𝒒 = (𝑖
+ 𝑠, 𝑗 + 𝑡)

Gaussian Noise Removal: • Bilateral filter for color image
Bilateral Filtering
Applying filter to each channel
𝑅=1෍𝐺𝒑−𝒒𝐺𝐶−𝐶𝑅 𝒑𝜎𝜎𝒑𝒒𝒒
𝐶 =(𝑅,𝐺,𝐵)T 𝒑 𝒑𝒑𝒑
𝐺=1෍𝐺𝒑−𝒒𝐺𝐶−𝐶𝐺 𝒑𝜎𝜎𝒑𝒒𝒒
𝐵=1෍𝐺𝒑−𝒒𝐺𝐶−𝐶𝐵 𝒑𝜎𝜎𝒑𝒒𝒒
𝒑 = (𝑖,𝑗) 𝒒 = (𝑖
+ 𝑠, 𝑗 + 𝑡)
𝑊=෍𝐺 𝒑−𝒒𝐺 𝐶−𝐶 𝒑 𝜎𝑠 𝜎𝑟𝒑𝒒

Gaussian filter vs. Bilateral filter • Gaussian filter
Weighted average of neighbors Depends only on spatial distance No edge term
intensity intensity

Gaussian filter vs. Bilateral filter • Bilateral filter
Weighted average of neighbors Depends on spatial and range difference
intensity intensity

𝐵𝐹[𝐼] = 1 ෍𝐺 ||𝐩−𝐪|| 𝐺 |𝐼−𝐼| 𝐼 𝐩𝜎𝜎𝐩𝐪𝐪
ilter on a
reproduced from [Durand 02]

Gaussian Noise Removal:
• Same goals:
̶ Smooth within Similar Regions
• KEY INSIGHT:
̶ Generalize, extend ‘Similarity’ • Bilateral:
̶ Averages neighbors with similar intensities; • NL-Means:
̶ Averages neighbors with similar neighborhoods!
local Means Filtering

Gaussian Noise Removal: • For each pixel p:
local Means Filtering
Define a small, simple fixed size neighborhood; Define vector Vp: a list of neighboring pixel values.
0.74 0.32 0.41 0.55 … … …

Gaussian Noise Removal: • For each pixel p:
local Means Filtering
Define a small, simple fixed size neighborhood;
Define vector Vp: a list of neighboring pixel values. q ‘Similar’ pixels p, q→SMALL distance || Vp – Vq ||2
‘Dissimilar’ pixels p, r→LARGE distance || Vp – Vr ||2
• Filtering with this neighboring pixels!
̶ No spatial terms,
̶ Measures the distance between patches (neighbor pixels)
𝑁𝐿𝑀𝐹[𝐼] = 1 ෍𝐺 ||𝐩−𝐪|| 𝐼 𝐩𝜎𝐪
𝐺 ||𝑉−𝑉||2

Gaussian Noise Removal: • Noisy source image:
local Means Filtering

Gaussian Noise Removal:
• Gaussian Filter ̶ Low noise
̶ Low detail
local Means Filtering

Gaussian Noise Removal:
• Bilateral Filter
̶ Better at noise removal ̶ but ‘stairsteps’
local Means Filtering

Gaussian Noise Removal:
• NL-Means ̶ Sharp
̶ Low noise
̶ Few artifacts
local Means Filtering

• Vision Transformer (in deep learning)
̶ A new architecture that shows state-of-the-art performance in computer vision tasks
local Means Filtering: Old
fashioned?
= 1 ෍𝐺 ||𝑉−𝑉||2 𝐼
𝜎r 𝐩 𝐪 𝐪 𝐩 𝐪∈𝑆

Many applications, not limited to denoising
Tone Mapping [Durand 02] Flash / No-Flash [Eisemann 04, Petschnigg 04]
Virtual Video Exposure [Bennett 05] Tone Management [Bae 06]

aware smoothing filters
[Shen 2015]

Periodic Noise Removal:
• Periodic fluctuation can be modeled as sinusoid waves
Frequency Domain Filtering

Periodic Noise Removal:
Frequency Domain Filtering Periodic fluctuation can be modeled as sinusoid waves
– Spatial domain
– Frequency domain
𝐴𝑀𝑁 −𝑗2𝜋 𝑢0𝐵𝑥+𝑣0𝐵𝑦
=𝑗 ൤𝑒 𝑀 𝑁𝛿𝑢+𝑢0,𝑣+𝑣0
=𝐴𝑠𝑖𝑛 0 𝑥+ 0 𝑦
𝐴 : amplitude
𝑢0,𝑣0 :sinusoidalfrequencies
𝐵 , 𝐵 : phase displacement w.r.t. the origin 𝑥𝑦
𝑀𝑁2𝑣𝐵 𝑗2𝜋 𝑢0𝐵𝑥+ 0 𝑦
𝛿 𝑢−𝑢0,𝑣−𝑣0
Sinusoidal wave in the spatial domain
Two symmetric spikes in the Fourier domain

Periodic Noise Removal:
• Periodic noise reduction by selective filters
̶ Notch filters can e

程序代写 CS代考加微信: powcoder QQ: 1823890830 Email: powcoder@163.com

Related Posts