Week 1
1
COMP9517: Computer Vision
Image Formation
Image Formation
• « Image formation occurs when a sensor registers radiation
that has interacted with physical objects » Ballard & Brown
scene
?
2
Geometry of image formation
Mapping world coordinates to image coordinates • Pinhole camera model
• Projective geometry
• Projection matrix
3
Film
Object
Image formation
Idea 1: Put a piece of film in front of an object Do we get a reasonable image?
4
Film
Object
Image formation
Idea 1: Put a piece of film in front of an object Do we get a reasonable image?
5
Film
Object
Image formation
Idea 1: Put a piece of film in front of an object Do we get a reasonable image?
6
Film
Object
Image formation
Idea 1: Put a piece of film in front of an object Do we get a reasonable image?
7
Film
Image formation Barrier
Object
Idea 2: Add a barrier to block off most of the rays This reduces blurring significantly
Opening known as the pinhole or aperture
8
Pinhole camera model
f
c
f = focal length
c = centre of the camera
Figure from Forsyth COMP9517 2020 T3 9
Dimensionality reduction machine 3D world 2D image
Point of o bservation
10
Projection can be tricky…
11
Projection can be tricky…
12
Figure from Forsyth
13
Projective geometry
A’ C’
B’
Length and area are not preserved
Projective geometry
Parallel?
What is lost?
Length and angles are not preserved
Perpendicular?
14
Projective geometry
What is preserved?
Straight lines are still straight
Parallel?
Perpendicular?
15
Vanishing points and lines
Parallel lines in the world intersect in the image at a “vanishing point”
16
Vanishing points and lines
Vanishing Point Vanishing Point
Vanishing Line
17
Vanishing points and lines
Slide from Efros, photo from Criminisi 18
Vanishing points and lines
Vanishing Point
Slide from Efros, photo from Criminisi 19
Vanishing points and lines
Vanishing Point
Slide from Efros, photo from Criminisi 20
Vanishing Point
Vanishing points and lines
Vanishing Point (Infinity)
Vanishing Point
Slide from Efros, photo from Criminisi
21
Vanishing Point
Projection maths
world coordinates => image coordinates
fZ
X P Y
Z Y
X
V
U
p U
Camera
Center
(0, 0, 0)
V
22
Projection maths
world coordinates => image coordinates
X P Y
Z Y
fZ
X
V
U
p U
Camera
Center
(0, 0, 0)
If X = 2, Y = 3, Z = 5, and f = 2, what are U and V ?
V
23
Projection maths
world coordinates => image coordinates
fZ
V
U
p U
X P Y
Z Y
X
U X * Zf V Y * Zf
Camera
Center
(0, 0, 0)
V
24
Projection maths
world coordinates > image coordinates
X= 2 Y =3 Z =5
X P Y
Z Y
V
U
p U
f Z f= 2 X
U X * Zf V Y * Zf
U 2 * 25 V 3 * 25
Camera
Center
(0, 0, 0)
V
25
Perspective projection
• Apparent size of object depends on its distance: far objects appear smaller
• By similar triangles (x’,y’,z’)(f x,f y,f)
• Ignore the third coordinate
(x’,y’)(f x,f y) zz
zz
26
Affine projection
• Suitable when scene depth is small relative to the average distance from the camera
• Let magnification m f ‘ / z0 be a positive constant, treat all points in the scene as at constant distance z0 from camera
• Leads to weak perspective projection ( x ‘, y ‘) ( m x , m y )
27
Affine projection
• Camera always remains at roughly constant distance from the scene
• Orthographic projection when m is normalised to –1 (x ‘, y ‘) (x, y)
28
Beyond pinholes: radial distortions
Image from Martin Habbecke
29
Corrected barrel distortion
Comparing with human vision
• Cameras imitate the frequency response of the human eye, so it is good to know something about it
• Computer vision probably would not get as much attention if biological vision (especially human vision) had not proven that it is possible to make important judgements from 2D images
The Eye
30
Electromagnetic spectrum
https://sites.google.com/site/chempendix/em-spectrum Normalized responsivity spectra of human cone cells (S, M, L types)
31
Colour represented by RGB images Red
Green
Blue
32
Colour spaces: RGB Default colour space
1,0,0
0,0,1 Drawback: strongly correlated channels
G (R=0,B=0)
B (R=0,G=0)
R
0,1,0 (G=0,B=0)
33
Colour spaces: HSV Intuitive colour space
H (S=1,V=1)
S (H=1,V=1)
V (H=1,S=0)
34
Colour spaces: YCbCr
Fast to compute, good for compression, used by TV
Y=0
Y=0.5
Y (Cb=0.5,Cr=0.5)
Cb (Y=0.5,Cr=0.5)
Cr (Y=0.5,Cb=0.5)
Cr
Cb
Y=1
35
Colour spaces: L*a*b* “Perceptually uniform” colour space
L (a=0,b=0)
a (L=65,b=0)
b (L=65,a=0)
36
Digital image formation
Digital image
37
Digital image formation
38
Digitisation by spatial sampling
• Digitisation converts an analog image to a digital image by sampling the image space
• Sampling digitises the coordinates x and y:
– Spatial discretisation of a picture function F(x,y)
– Uses a (typically rectangular) grid of sampling points: x = jΔx, y = kΔy | j = 1…M, k = 1…N
– The Δx, Δy are called the sampling intervals
39
column row
R
G
Digital colour images
0.92
0.95
0.89
0.96
0.71 0.49
0.86
0.96
0.69
0.79
0.91
0.93
0.89
0.72
0.95
0.81 0.62
0.84
0.67
0.49
0.73
0.94
0.94
0.82
0.51
0.88 0.81
0.
89 0. 0.96
0.69
0.79
0.91
0.97
0.89
0.55
0.94 0.87
49 0. 0.67
0.49
0.73
0.94
0.62
0.56
0.51
0.56 0.57
0.
0.37
0.31
0.42
0.46 0.37
89 0. 0.86
0.96
0.69
0.79
0.91
0.85
0.75
0.57
0.91 0.80
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
49 0. 0.84
0.67
0.49
0.73
0.94
0.97
0.92
0.41
0.87 0.88
41 0. 0.74
0.54
0.56
0.90
0.89
0.93
0.81
0.49
0.90 0.89
78 0. 0.58
0.85
0.66
0.67
0.49
0.92
0.95
0.91
0.97 0.79
78 0. 0.51
0.48
0.43
0.33
0.41
0.99
0.91
0.92
0.95 0.85
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85 0.90 0.67 0.33 0..9621 0..9639 0..9749 0..9773 0..6923 0..397 0.85 0.97 0.93
0.49 0.62 0.86 0.84
0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
41 0..9758 0..8798 0..8727 0..89 0..5969 0..3913 0.75 0.92 0.81
0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74 0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49
0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93 0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90
0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99 0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89
0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97 0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61
77 0. 0.39
0.37
0.42
0.61
0.78
89 0. 0.73
0.88
0.77
0.69
0.78
0.92
0.95
0.91
0.97
99 0. 0.92
0.90
0.73
0.79
0.77
0.99
0.91
0.92
0.95
93 0.91
0.94
0.71
0.73
0.89
0.92
0.95
0.91
0.97
0.79
0.45 0.49
0.82
0.90
0.93
0.99
0.99
0.91
0.92
0.95
0.85
0.33 0.74
0.93
0.99
0.97
0.93
B
40
Spatial resolution
• Spatial resolution: number of pixels per unit of length
• Example: resolution decreases by one half each time (see right)
• Human faces can be recognized in 64 x 64 pixels images
• Appropriate resolution is essential:
– Toolittleresolution,poorrecognition
– Toomuchresolution,slowandwastesmemory
41
Quantisation
• Quantisation digitises the intensity or amplitude values F(x,y) – Called intensity or gray level quantisation
– Gray-level resolution to be chosen
• For example 16, 32, 64, …., 128, 256 levels
• Number of levels should be high enough for human perception of shading details… requires about
100 levels for a realistic image
42
Quantisation and bits/pixel
Pixel (picture element)
Levels per pixel:
8 bits = 28 = 256
12 bits = 212 = 4,096
16 bits = 216 = 65,536
24 bits = 224 = 16,777,216
43
Further reading
• Chapter 2 of Szeliski
• Chapter 2 of Shapiro and Stockman
44
Acknowledgements
• Several slides from Derek Hoiem, Alexei Efros, Steve Seitz, David Forsyth and Erik Meijering
• Image sources credited where possible
• Some material, including images and tables, were drawn from the referenced textbooks and associated online resources
45