18-793 Image and Video Processing
Submission instructions.
Fall 2020
• Submissions are due on Thursday 09/10 at 10 pm ET
• Please upload scans of your solution in GradeScope
Homework 1
• You are expected to upload solutions to each problem separately. Please ensure that your scans are readable.
Instructions
• Please solve all non-MATLAB problems without resorting to using MATLAB or other computing tools.
• Please show all necessary steps to get the final answer. However, there is no need to be overly elaborate. Crisp and complete answers.
• For all MATLAB problems, include all code written to generate solutions.
• Please post all questions on the discussion board on the Piazza course website, rather than email-ing the course staff. This will allow other students with the same question to see the response and any ensuing discussion.
• If you feel some information is missing, you are welcome to make reasonable assumptions and proceed. Sometimes the omissions are intentional. Needless to say, only reasonable assumptions will be accepted.
1. (5 pts) (Quantum efficiency and photon noise) Suppose that the average rate of arrival
of light at a pixel (x, y) and wavelength λ is given as P (x, y, λ) photons/second. The average electron count at the pixel is given as
average e(x, y) = T0
where T0 is the exposure time in seconds and Q(·) is the quantum efficiency (QE) of the
pixel.
However this is the average or expected number of photons. In practice, any actual measurement is a sampling of a Poisson random variable whose parameter is the average value. That is, a measurement of electron count at a pixel is given as
e(x, y) ∼ Poisson T0 P (x, y, λ)Q(λ)dλ .
P (x, y, λ)Q(λ)dλ,
2
Homework 1
When the pixel has a full well capacity of F electrons, the measurement morphs to
e(x, y) ∼ min F, Poisson T0 P (x, y, λ)Q(λ)dλ . In HW1 P1.mat you are given the following:
• P which provides P (x, y, λ) with has the average photon count per (nm × s),
• the QE for a RGB sensor in R response, G response, B response,
• Delta lambda, the spacing of wavelengths in nanometers that you can use in the integral in place of dλ.
Assume a full well capacity F = 1000e−. For the following exposure values, produce 3-color images observed by the sensor.
(a) T0 = 10μs (b) T0 = 100μs
(c) T0 = 1ms (d) T0 = 10ms
(e) T0 = 100ms (f) T0 = 1000ms
(g) T0 = 10s
Deliverables: 1) MATLAB code for implementing the solution. 2) Seven photographs,
one each for the seven subparts above. Notes:
(a) Make sure that the simulated image is of type double or single.
(b) Choose an ADC gain for each RGB image such that its maximum value is 1. This
can be done simply by dividing the RGB image by its maximum value.
(c) Since displays have a nonlinear response, you need to gamma correct the image for better visualization. Use the following command for that:
>> imshow( (max(0, img)).∧(1/2.2));
(d) Visualize your image using imshow command and imwrite to save them
(e) The commands poissrnd or random are useful for simulating Poisson measurements
2. (5 pts) (Signal to noise ratio)
There are two sources of noise when acquiring an image:
Homework 1 3 • the photon noise, that captures the inherent variability in photon arrivals, that is
modeled as a Poisson random variable,
• the readout noise, capturing thermal effects in the electronics, that is modeled with a Gaussian random variable.
You can assume that these two sources of noise are statistically independent.
For simplicity we are going to assume that there are no quantization related artifacts. We are going to assume an ideal sensor with a quantum efficiency of 1 and infinite well capacity.
Under these assumptions, the measurements made can be written as i ∼ Poisson(λT0) + n,
where λ is the expected arrival rate of photons, T0 is the exposure time, and n is the readout noise whose variance is σ2. We will measure the signal to noise ratio as
|E [i]|2 SNR = var(i) .
(Part a) Derive an expression for the SNR in terms of σ2, λ and T0.
(Partb)LetT0 =1s. Letσ=100e. Letsvaryλfrom1to105 inmultiplesof10(so 1, 10, 100, 103, 104, 105). Plot the variance of i as a function of λ (do use log scale for both axes using loglog command). Interpret and explain the plot.
(Part c) Repeat part b, except plot the SNR instead as a function of λ (do use log scale for both axes using loglog command). Interpret and explain the plot.