2/11/2021
CSE 473/573
‘-
Introduction to Computer Vision and Image Processing
1
IMAGE FORMATION Questions from Last Class?
‘-
2
1
2/11/2021
Physical parameters of image formation
• Geometric
• Type of projection • Camera pose
‘-
• Photometric
• Type, direction, intensity of light reaching sensor • Surfaces’ reflectance properties
• Sensor
• sampling, etc.
• Optical
• Sensor’s lens type
• focal length, field of view, aperture
3
Environment map
‘-
4
http://www.sparse.org/3d.html
2
2/11/2021
BDRF
‘-
5
Diffuse / Lambertian
‘-
6
3
2/11/2021
Physical parameters of image formation
• Geometric
• Type of projection • Camera pose
‘-
• Photometric
• Type, direction, intensity of light reaching sensor • Surfaces’ reflectance properties
• Sensor
• sampling, etc.
• Optical
• Sensor’s lens type
• focal length, field of view, aperture
9
Digital cameras
• Filmsensor array
• Often an array of charge coupled devices
‘-
• Each CCD is light sensitive diode that converts photons (light energy) to electrons
camera optics
CCD array
frame grabber
computer
10
K. Grauman
4
2/11/2021
Historical context
• Pinhole model: Mozi (470-390 BCE), Aristotle (384-322 BCE)
• Principles of optics (including lenses): Alhacen (965-1039 CE)
• Camera obscura: Leonardo da Vinci
(1452-1519), Johann Zahn (1631-1707) ‘-
• First photo: Joseph Nicephore Niepce (1822)
• Daguerréotypes (1839)
• Photographic film (Eastman, 1889)
• Cinema (Lumière Brothers, 1895)
• Color Photography (Lumière Brothers, 1908)
• Television (Baird, Farnsworth, Zworykin, 1920s)
• First consumer camera with CCD:
Sony Mavica (1981)
• First fully digital camera: Kodak DCS100 (1990)
Slide credit: L. Lazebnik
Alhacen’s notes
Niepce, “La Table Servie,” 1822
CCD chip
11
K. Grauman
‘-
12
5
2/11/2021
Digital Sensors
‘-
13
‘-
14
6
2/11/2021
Resolution
• sensor: size of real world scene element that images to a single pixel
• image: number of pixels
‘-
• Influences what analysis is feasible, affects best representation choice.
[fig from Mori et al] 15
Digital images
Think of images as matrices taken from CCD array.
‘-
16
K. Grauman
7
Digital images
j=1
‘-
im[194][203] has value 37 17
K. Grauman
2/11/2021
width 520
i=1
Intensity : [0,255]
500 height
im[176][201] has value 164
Color images, RGB color space
‘-
RGB
18
K. Grauman
8
2/11/2021
Sampling and Quantization
‘-
19
Color sensing in digital cameras
Estimate missing components from neighboring values (demosaicing)
Bayer grid
‘-
20
Source: Steve Seitz
9
2/11/2021
Color Image
R
G
B
21
‘-
Images in Python
• Imagesrepresentedasamatrix
• SupposewehaveaNxMRGBimagecalled“im”
– im(0,0,0) = top-left pixel value in R-channel
– im(y, x, b) = y pixels down, x pixels to right in the bth channel – im(N-1, M-1, 2) = bottom-right pixel in B-channel‘-
row column
R
0.92
0.95 0.89
0.96
0.71
0.49
0.86
0.96
0.69
0.79 0.91
0.93
0.89 0.72
0.95
0.81
0.62
0.84
0.67
0.49
0.73 0.94
0.94
0.82
0.51 0.92
0.88
0.95 0.81
0.89 0.60
0.96 0.74
0.71 0.54
0.49
0.56 0.86
0.90 0.96 0.89 0.69
0.79 0.91
0.97
0.89
0.55 0.93
0.94
0.89 0.87
0.72 0.58
0.95 0.58
0.81 0.85
0.62
0.66 0.84
0.67 0.67 0.49 0.49
0.73 0.94
0.62
0.56
0.51 0.94
0.56
0.82 0.57
0.51
0.92 0.50
0.88
0.95 0.51
0.81
0.89 0.48
0.60 0.96 0.43
0.74
0.71 0.33 0.54
0.49 0.41 0.56
0.86 0.90
0.96 0.89 0.69
0.79
0.91
0.37
0.31
0.42 0.97
0.46
0.89 0.37
0.55
0.93 0.60
0.94
0.89 0.39
0.87
0.72 0.37
0.58 0.95 0.42
0.58
0.81 0.61 0.85 0.62 0.78
0.66
0.84 0.67
0.67 0.49 0.49
0.73
0.94
0.85
0.75
0.57 0.62
0.91
0.56 0.80
0.51
0.94 0.58
0.56
0.82 0.73
0.57
0.51 0.88
0.50 0.88 0.77
0.51
0.81 0.69 0.48
0.60 0.78 0.43
0.74 0.33
0.54 0.41
0.56 0.90
0.89
0.97
0.92
0.41 0.37
0.87
0.31 0.88
0.42
0.97 0.50
0.46
0.89 0.92
0.37
0.55 0.90
0.60 0.94 0.73
0.39
0.87 0.79 0.37
0.58 0.77 0.42
0.58 0.61
0.85 0.78 0.66
0.67
0.49
0.93
0.81
0.49 0.85
0.90
0.75 0.89
0.57
0.62 0.61
0.91
0.56 0.91
0.80
0.51 0.94
0.58 0.56 0.71
0.73
0.57 0.73 0.88 0.50 0.89
0.77
0.51 0.69
0.48 0.78 0.43
0.33
0.41
0.92
0.95
0.91 0.97
0.97
0.92 0.79
0.41
0.37 0.45
0.87
0.31 0.49
0.88
0.42 0.82
0.50 0.46 0.90
0.92
0.37 0.93 0.90
0.60 0.99 0.73
0.39 0.79
0.37 0.77
0.42 0.61
0.78
0.99
0.91
0.92 0.93
0.95
0.81 0.85
0.49
0.85 0.33
0.90
0.75 0.74
0.89
0.57 0.93
0.61 0.91 0.99
0.91
0.80 0.97 0.94
0.58 0.93 0.71
0.73 0.73
0.88 0.89 0.77
0.69
0.78
0.92
0.95
0.91 0.97
0.97 0.92 0.79 0.41
0.45 0.87 0.49 0.88
0.82 0.50 0.90 0.92
0.93
0.90 0.99 0.73
0.79
0.77
0.99
0.91
0.92 0.93
0.95 0.81
0.85 0.49
0.33 0.90
0.74 0.89
0.93 0.61
0.99 0.91
0.97
0.94 0.93
0.71 0.73
0.89
G
B
22
0.92
0.95
0.91
0.97
0.79
0.45 0.49
0.82 0.90 0.93
0.99
0.99
0.91
0.92
0.95
0.85
0.33 0.74
0.93 0.99 0.97
0.93
10
2/11/2021
Color spaces
• How can we represent color?
‘-
http://en.wikipedia.org/wiki/File:RGB_illumination.jpg
23
Color spaces: RGB
0,1,0
Default color space
R
(G=0,B=0)
G
(R=0,B=0)
B
(R=0,G=0)
24
‘-
1,0,0
0,0,1
Some drawbacks
• Strongly correlated channels • Non-perceptual
Image from: http://en.wikipedia.org/wiki/File:RGB_color_solid_cube.png
11
2/11/2021
Color spaces: HSV
‘-
H
(S=1,V=1)
S
(H=1,V=1)
V
(H=1,S=0)
25
Intuitive color space
Color spaces: YCbCr
Y=0 Y=0.5
Cr
Fast to compute, good for compression, used by TV
‘-
Y
(Cb=0.5,Cr=0.5)
Cb
(Y=0.5,Cr=0.5)
Cr
(Y=0.5,Cb=05)
26
Cb
Y=1
12
2/11/2021
Color spaces: L*a*b*
“Perceptually uniform”* color space
‘-
L
(a=0,b=0)
a
(L=65,b=0)
b
(L=65,a=0)
27
If you had to choose, would you rather go
without luminance or chrominance?
‘-
28
13
2/11/2021
If you had to choose, would you rather go
without luminance or chrominance?
‘-
29
Most information in intensity
‘-
Only color shown – constant intensity
30
14
2/11/2021
Most information in intensity
‘-
Only intensity shown – constant color
31
Most information in intensity
‘-
Original image
32
15
2/11/2021
Summary
• Image formation affected by geometry, photometry, and optics.
• Projection equations express how world points mapped to 2d image.
‘-
• Lenses make pinhole model practical
• Photometry models: Lambertian, BRDF
• Digital imagers, Bayer demosaicing
Parameters (focal length, aperture, lens diameter, sensor sampling…) strongly affect image obtained.
33
K. Grauman
• Homogenous coordinates allow linear system for projection equations.
Quiz #0 Due Tuesday
• Proceed to UBLearns • UnderAssignments
• Take Quiz #0
‘-
34
16