2/9/2021
CSE 573/473
‘-
Introduction to Computer Vision and Image Processing
1
Questions from Last Class?
‘-
2
1
2/9/2021
Projection matrix
Extrinsic Assumptions • No rotation
• Camera at (0,0,0)
Intrinsic Assumptions • Optical center at (0,0)
• Unit aspect ratio • No skew
𝐱𝐊𝐈 𝟎 𝐗
Slide Credit: Savarese
X
x ‘-
u f 0 0 0x wv0 f 0 0y
1 0 0 1 0z 1
3
Remove assumption: known optical center
Extrinsic Assumptions • No rotation
• Camera at (0,0,0)
Intrinsic Assumptions • Optical center at (0,0)
• Unit aspect ratio • No skew
𝐱𝐊𝐈 𝟎 𝐗
‘-
u f 0 u 0x 0 y
wv 0 f v0 0z
1 0 0 1 0 1
4
2
2/9/2021
Remove assumption: square pixels
Extrinsic Assumptions • No rotation
• Camera at (0,0,0)
Intrinsic Assumptions • Optical center at (0,0)
• Unit aspect ratio • No skew
𝐱𝐊𝐈 𝟎 𝐗
X
x ‘-
u 0 u 0x 0 y
wv 0 v0 0z
1 0 0 1 0 1
5
Remove assumption: non-skewed pixels
Extrinsic Assumptions • No rotation
• Camera at (0,0,0)
Intrinsic Assumptions • Optical center at (0,0)
• Unit aspect ratio • No skew
𝐱𝐊𝐈 𝟎 𝐗
Note: different books use different notation for parameters
‘-
u s u 0x 0 y
wv 0 v0 0z
1 0 0 1 0 1
6
3
2/9/2021
Derive S? Given….
‘-
𝐱𝐊𝐈 𝟎 𝐗
u s u 0x 0 y
wv 0 v0 0z
1 0 0 1 0 1
7
Degrees of freedom
xKR tX
3 Extrinsic Rotation
2 Intrinsic Translation 2 Non-square pixels 1 Skew
3 Extrinsic Translation
5 6 ‘-
u s ur r r tx
011 12 13 xy wv0 v r r r t
0 21 22 23 yz
1 0 0 1r r r t 31 32 33 z1
12
4
2/9/2021
Weak perspective
• Approximation: treat magnification as constant • Assumes scene depth << average distance to
camera
Image plane
‘-
World points:
13
Orthographic projection
• Given camera at constant distance from scene
• World points projected along rays parallel to
optical access
‘-
14
5
2/9/2021
2D Planer Transformation
‘-
15
2D
‘-
16
6
2/9/2021
3D
‘-
17
line Things to remember Vanishing
Vertical vanishing point
(at infinity)
Vanishing point
• Vanishing points and vanishing lines
• Pinhole camera model and camera projection matrix
• Homogeneous coordinates
Vanishing point
‘-
xKR tX
18
7
2/9/2021
Other types of projection
• Lots of intriguing variants...
‘-
S. Seitz
19
360 degree field of view...
• Basic approach
• Takeaphotoofaparabolicmirrorwithanorthographiclens(Nayar) • Orbuyonealensfromavarietyofomnicammanufacturers...
‐ See http://www.cis.upenn.edu/~kostas/omni.html ‘-
20
S. Seitz
8
2/9/2021
Tilt-shift
http://www.northlight-images.co.uk/article_pages/tilt_and_shift_ts-e.html
‘-
Titlt-shift images from Olivo Barbieri and Photoshop imitations
21
S. Seitz
tilt, shift
‘-
http://en.wikipedia.org/wiki/Tilt-shift_photography
22
9
2/9/2021
Tilt-shift perspective correction
‘-
http://en.wikipedia.org/wiki/Tilt-shift_photography
23
normal lens tilt-shift lens
‘-
http://www.northlight-images.co.uk/article_pages/tilt_and_shift_ts-
e.html
24
10
2/9/2021
Rotating sensor (or object)
‘-
Rollout Photographs © Justin Kerr
http://research.famsi.org/kerrmaya.html
Also known as “cyclographs”, “peripheral images”
25
S. Seitz
Photofinish
‘-
26
S. Seitz
11
2/9/2021
IMAGE FORMATION
IMAGE PROCESSING
‘-
27
Creating an Image... Lets Drill Down...
Digital Camera
Image Processing Image Processing
‘-
SCENE
28
12
2/9/2021
Physical parameters of image formation
• Geometric
• Type of projection • Camera pose
• Optical ‘- • Sensor’s lens type
• focal length, field of view, aperture
• Photometric
• Type, direction, intensity of light reaching sensor • Surfaces’ reflectance properties
• Sensor
• sampling, etc.
29
The Eye
• The human eye is a camera!
• Iris - colored annulus with radial muscles
• Pupil - the hole (aperture) whose size is controlled by the iris
• What’sthe“film”?
– photoreceptor cells (rods and cones) in the retina
‘-
30
Slide by Steve Seitz
13
2/9/2021
Electromagnetic Spectrum
‘-
Human Luminance Sensitivity Function
31
http://www.yorku.ca/eye/photopik.htm
Visible Light
Why do we see light of these wavelengths?
10000 C
.
...because that’s where the
‘-
5000 C
Sun radiates EM energy
0
400 700 1000
700 C
2000 3000
2000 C
Visible Wavelength (nm) Region
© Stephen E. Palmer, 2002
32
14
Energy
2/9/2021
Pinhole size / aperture
How does the size of the aperture affect the image we’d get?
‘-
Larger
Smaller
33
K. Grauman
Adding a lens
focal point
‘-
f
• A lens focuses light onto the film
– Rays passing through the center are not deviated – All parallel rays converge to one point on a plane
located at the focal length f
34
Slide by Steve Seitz
15
2/9/2021
What are the physical benefits or challenges of adding lens?
• Light Concentration
• Change the Focus
• Change the Depth of Field • Change of the Field of View
• Vignetting • Aberration
‘-
35
Light Concentration
‘-
focal point
f
36
Slide by Steve Seitz
16
2/9/2021
Cameras with lenses
optical center (Center Of Projection)
F
‘- focal point
• A lens focuses parallel rays onto a single focal point
• Gather more light, while keeping focus; make pinhole perspective projection practical
Thin lens
Thin lens Left focus
Lens diameter d
37
K. Grauman
Right focus
Focal length f
Rays entering parallel on one side go through focus on other, and vice versa.
In ideal case – all rays from P imaged at P’.
38
K. Grauman
‘-
17
2/9/2021
Thin lens equation
uv
111 ‘- fuv
• Any object point satisfying this equation is in focus
39
K. Grauman
Focus and depth of field
‘-
40
Image credit: cambridgeincolour.com
18
2/9/2021
Focus and depth of field
• Depth of field: distance between image planes
where blur is tolerable
Thin lens: scene points at distinct depths come
‘-
(Real camera lens systems have greater depth of field.)
41
“circles of confusion”
in focus at different image planes.
Shapiro and Stockman
Focus and depth of field
• How does the aperture affect the depth of field?
‘-
• A smaller aperture increases the range in which the object is approximately in focus
Flower images from Wikipedia http://en.wikipedia.org/wiki/Depth_of_field Slide from S. Seitz
42
19
2/9/2021
Depth from focus
‘-
Images from same point of view, different camera parameters
3d shape / depth estimates
[figs from H. Jin and P. Favaro, 2002]
43
Field of view
• Angular measure of portion of 3D space seen by the camera
‘-
http://en.wikipedia.org/wiki/Angle_of_view
44
K. Grauman
20
2/9/2021
Field of view depends on focal length
• As f gets smaller, image becomes more wide angle
• more world points project onto the finite image plane
• As f gets larger, image becomes more telescopic
• smaller part of the world projects onto the finite image plane
‘-
45
from R. Duraiswami
Field of view depends on focal length
‘-
Smaller FOV = larger Focal Length 46 Slide by A. Efros
21
2/9/2021
Vignetting
http://www.ptgui.com/examples/vigntutorial.html
‘-
http://www.tlucretius.net/Photo/eHolga.html
47
Vignetting
• “natural”:
• “mechanical”: intrusion on optical path
‘-
48
22
2/9/2021
Chromatic aberration
‘-
49
Chromatic aberration
‘-
50
23
2/9/2021
Other Distortions
‘-
51
24