COMP9517: Computer Vision
Image Formation
Week 1 COMP9517 2021 T1 1
Image Formation
• «Imageformationoccurswhenasensorregistersradiation
that has interacted with physical objects » Ballard & Brown
scene
Week 1 COMP9517 2021 T1 2
Geometry of image formation
Mapping world coordinates to image coordinates • Pinholecameramodel
• Projective geometry
• Projectionmatrix
Week 1 COMP9517 2021 T1 3
Film
Image formation
Object
Week 1
Idea 1: Put a piece of film in front of an object Do we get a reasonable image?
COMP9517 2021 T1 4
Film
Image formation
Object
Week 1
Idea 1: Put a piece of film in front of an object Do we get a reasonable image?
COMP9517 2021 T1 5
Film
Image formation
Object
Week 1
Idea 1: Put a piece of film in front of an object Do we get a reasonable image?
COMP9517 2021 T1 6
Film
Image formation
Object
Week 1
Idea 1: Put a piece of film in front of an object Do we get a reasonable image?
COMP9517 2021 T1 7
Film
Image formation Barrier
Object
Week 1
Idea 2: Add a barrier to block off most of the rays
This reduces blurring significantly
Opening known as the pinhole or aperture
COMP9517 2021 T1 8
Pinhole camera model
f
c
Week 1
f = focal length
c = centre of the camera
Figure from Forsyth COMP9517 2021 T1 9
Dimensionality reduction machine
3D world
2D image
Week 1
COMP9517 2021 T1
10
Projection can be tricky…
Week 1 COMP9517 2021 T1 11
Projection can be tricky…
Week 1 COMP9517 2021 T1 12
Week 1
Figure from Forsyth
COMP9517 2021 T1
13
Projective geometry
A’ C’
B’
Length and area are not preserved
Projective geometry
Parallel?
What is lost?
Length and angles are not preserved
Week 1
COMP9517 2021 T1
14
Perpendicular?
Projective geometry
Parallel?
What is preserved?
Straight lines are still straight
Perpendicular?
Week 1 COMP9517 2021 T1
15
Vanishing points and lines
Parallel lines in the world intersect in the image at a “vanishing point”
Week 1 COMP9517 2021 T1
16
Vanishing points and lines
Vanishing Point Vanishing Point
Week 1
COMP9517 2021 T1
17
Vanishing Line
Vanishing points and lines
Week 1 Slide from Efros, photo from Criminisi COMP9517 2021 T1 18
Vanishing points and lines
Vanishing Point
Week 1
Slide from Efros, photo from Criminisi COMP9517 2021 T1 19
Vanishing Point
Vanishing points and lines
Week 1
Slide from Efros, photo from Criminisi COMP9517 2021 T1 20
Vanishing Point
Vanishing points and lines
Vanishing Point (Infinity)
Vanishing Point
Week 1
Slide from Efros, photo from Criminisi
COMP9517 2021 T1
Vanishing Point
21
Projection maths
world coordinates => image coordinates
X P Y
Z Y
fZ
X
V
U
p U V
Camera
Center
(0, 0, 0)
Week 1
COMP9517 2021 T1
22
Projection maths
world coordinates => image coordinates
X P Y
Z Y
fZ
X
V
U
p U V
Camera
Center
(0, 0, 0)
If X = 2, Y = 3, Z = 5, and f = 2, what are U and V ?
Week 1
COMP9517 2021 T1
23
V U
p U V
Projection maths
world coordinates => image coordinates
X P Y
Z Y
fZ
X
Camera UX*f
Center
(0, 0, 0)
Z V Y * Zf
Week 1
COMP9517 2021 T1
24
Projection maths
world coordinates > image coordinates
X=2 Y=3 Z=5
X P Y
Z Y
V U
p U V
fZf=2 X
U X * f Z
V Y * Zf
U 2 * 2 5
V 3 * 25
Camera
Center
(0, 0, 0)
Week 1
COMP9517 2021 T1
25
Perspective projection
• Apparentsizeofobject depends on its distance: far objects appear smaller
• Bysimilartriangles (x’,y’,z’)(f x,f y,f)
• Ignore the third coordinate
zz
Week 1
(x’,y’)(f x,f y) zz
COMP9517 2021 T1 26
Affine projection
• Suitablewhenscenedepthissmallrelativetotheaverage distance from the camera
• Let magnification m f ‘ / z0 be a positive constant, treat all points in the scene as at constant distance z0 from camera
• Leads to weak perspective projection ( x ‘, y ‘) ( m x , m y )
Week 1
COMP9517 2021 T1 27
Affine projection
• Cameraalwaysremainsatroughlyconstantdistance from the scene
• Orthographicprojectionwhenmisnormalisedto–1 (x ‘, y ‘) (x, y)
Week 1 COMP9517 2021 T1 28
Beyond pinholes: radial distortions
Week 1 Image from Martin Habbecke COMP9517 2021 T1
29
Corrected barrel distortion
Comparing with human vision
• Cameras imitate the frequency response of the human eye, so it is good to know something about it
• Computer vision probably would not get as much attention if biological vision (especially human vision) had not proven that it is possible to make important judgements from 2D images
Week 1 COMP9517 2021 T1
The Eye
30
Electromagnetic spectrum
https://sites.google.com/site/chempendix/em-spectrum
Normalized responsivity spectra of human cone cells (S, M, L types)
Week 1
COMP9517 2021 T1
31
Colour represented by RGB images Red
Week 1 COMP9517 2021 T1
Green
Blue
32
Colour spaces: RGB Default colour space
0,1,0
R (G=0,B=0)
G (R=0,B=0)
B (R=0,G=0)
1,0,0
Drawback: strongly correlated channels
Week 1 http://en.wikipedia.org/wiki/File:RGB_color_solid_cube.png COMP9517 2021 T1
33
0,0,1
Colour spaces: HSV Intuitive colour space
H (S=1,V=1)
S (H=1,V=1)
V (H=1,S=0)
Week 1
COMP9517 2021 T1
34
Colour spaces: YCbCr
Fast to compute, good for compression, used by TV
Y=0
Y=0.5
Y (Cb=0.5,Cr=0.5)
Cb (Y=0.5,Cr=0.5)
Cr (Y=0.5,Cb=0.5)
Cr
Cb
Y=1
Week 1
COMP9517 2021 T1 35
Colour spaces: L*a*b* “Perceptually uniform” colour space
L (a=0,b=0)
a (L=65,b=0)
b (L=65,a=0)
Week 1
COMP9517 2021 T1
36
Digital image formation
Week 1 COMP9517 2021 T1
37
Digital image
Digital image formation
Week 1 COMP9517 2021 T1 38
Digitisation by spatial sampling
• Digitisationconvertsananalogimagetoadigitalimageby sampling the image space
• Samplingdigitisesthecoordinatesxandy:
– Spatial discretisation of a picture function F(x,y)
– Uses a (typically rectangular) grid of sampling points: x=jΔx,y=kΔy | j=1…M,k=1…N
– The Δx, Δy are called the sampling intervals
Week 1 COMP9517 2021 T1 39
row column
R
G
B
Digital colour images
0.92
0.95
0.89
0.96
0.71 0.49
0.86
0.96
0.69
0.79
0.91
0.93
0.89
0.72
0.95
0.81 0.62
0.84
0.67
0.49
0.73
0.94
0.94
0.82
0.51
0.88 0.81
0.
89 0. 0.96
0.69
0.79
0.91
0.97
0.89
0.55
0.94 0.87
49 0. 0.67
0.49
0.73
0.94
0.62
0.56
0.51
0.56 0.57
0.
0.37
0.31
0.42
0.46 0.37
89 0. 0.86
0.96
0.69
0.79
0.91
0.85
0.75
0.57
0.91 0.80
49 0. 0.84
0.67
0.49
0.73
0.94
0.97
0.92
0.41
0.87 0.88
41 0. 0.74
0.54
0.56
0.90
0.89
0.93
0.81
0.49
0.90 0.89
78 0. 0.58
0.85
0.66
0.67
0.49
0.92
0.95
0.91
0.97 0.79
78 0. 0.51
0.48
0.43
0.33
0.41
0.99
0.91
0.92
0.95 0.85
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89
0.90 0.67 0.33 0..9621 0..9639 0..9749 0..9773 0..6923 0..397 0.85 0.97 0.93
0.49 0.62 0.86 0.84
0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
41 0..9758 0..8798 0..8727 0..89 0..5969 0..3913 0.75 0.92 0.81
0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99 0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89
0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97 0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61
77 0. 0.39
0.37
0.42
0.61
0.78
89 0. 0.73
0.88
0.77
0.69
0.78
0.92
0.95
0.91
0.97
0.79 0.85
99 0. 0.92
0.90
0.73
0.79
0.77
0.99
0.91
0.92
0.95
0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74 0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49
0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93 0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90
93 0.91
0.94
0.71
0.73
0.89
0.92
0.95
0.91
0.97
0.79
0.45
0.49 0.82
0.90
0.93
0.99
0.99
0.91
0.92
0.95
0.85
0.33
0.74 0.93
0.99
0.97
0.93
Week 1
COMP9517 2021 T1
40
Spatial resolution
• Spatial resolution: number of pixels per unit of length
• Example: resolution decreases by one half each time (see right)
• Human faces can be recognized in 64 x 64 pixels images
• Appropriate resolution is essential:
– Toolittleresolution,poorrecognition
– Toomuchresolution,slowandwastesmemory
Week 1 COMP9517 2021 T1 41
Quantisation
• Quantisation digitises the intensity or amplitude values F(x,y) – Calledintensityorgraylevelquantisation
– Gray-levelresolutiontobechosen
Week 1
• •
For example 16, 32, 64, …., 128, 256 levels
Number of levels should be high enough for human perception of shading details… requires about
100 levels for a realistic image
COMP9517 2021 T1 42
Quantisation and bits/pixel
Pixel (picture element)
Week 1
COMP9517 2021 T1
43
Levels per pixel:
8bits=28 =256
12 bits = 212 = 4,096
16 bits = 216 = 65,536
24 bits = 224 = 16,777,216
Further reading • Chapter2ofSzeliski
• Chapter2ofShapiroandStockman
Week 1 COMP9517 2021 T1 44
Week 1
Acknowledgements
• SeveralslidesfromDerekHoiem,AlexeiEfros, Steve Seitz, and David Forsyth, Erik Meijering
• Imagesourcescreditedwherepossible
• Somematerial,includingimagesandtables, were drawn from the referenced textbooks and associated online resources
COMP9517 2021 T1 45