Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 1/24
Chapter 3
Perspective Projection
Multiple View Geometry
Summer 2021
Prof. Daniel Cremers
Chair for Computer Vision and Artificial Intelligence
Departments of Informatics & Mathematics
Technical University of Munich
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 2/24
Overview
1 Historic Remarks
2 Mathematical Representation
3 Intrinsic Parameters
4 Spherical Projection
5 Radial Distortion
6 Preimage and Coimage
7 Projective Geometry
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 3/24
Some Historic Remarks
The study of the image formation process has a long history.
The earliest formulations of the geometry of image formation
can be traced back to Euclid (4th century B.C.). Examples of a
partially correct perspective projection are visible in the
frescoes and mosaics of Pompeii (1 B.C.).
These skills seem to have been lost with the fall of the Roman
empire. Correct perspective projection emerged again around
1000 years later in early Renaissance art.
Among the proponents of perspective projection are the
Renaissance artists Brunelleschi, Donatello and Alberti. The
first treatise on the projection process, “Della Pittura” (1435)
was published by Leon Battista Alberti).
Apart from the geometry of image formation, the study of the
interaction of light with matter was propagated by artists like
Leonardo da Vinci in the 1500s and by Renaissance painters
such as Caravaggio and Raphael.
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 4/24
Perspective Projection in Art
Filippo Lippi, “The Feast of Herod: Salome’s Dance.”
Fresco, Cappella Maggiore, Duomo, Prato, Italy, c.1460-1464.
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 5/24
Perspective Projection in Art
Raphael, The School of Athens (1509)
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 6/24
Perspective Projection in Art
Dürer’s machine (1525)
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 7/24
Perspective Projection in Art
Satire by Hogarth 1753
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 8/24
Perspective Projection in Art
M.C. Escher, Another World 1947 Escher, Belvedere 1958
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 9/24
Mathematics of Perspective Projection
The above drawing shows the perspective projection of a point
P (observed through a thin lens) to its image p.
The point P has coordinates X = (X ,Y ,Z ) ∈ R3 relative to the
reference frame centered at Fl , where the z-axis is the optical
axis (of the lens). By comparing the similar triangles A and B,
we obtain the relation
Y
Z
= −
y
f
⇔ y = −f
Y
Z
. (1)
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 10/24
Mathematics of Perspective Projection
To simplify equations, one flips the signs of x- and y -axes,
which amounts to considering the image plane to be in front of
the center of projection (rather than behind it). The perspective
transformation π is therefore given by
π : R3 → R2; X 7→ x = π(X ) =
(
f XZ
f YZ
)
.
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 11/24
An Ideal Perspective Camera
In homogeneous coordinates, the perspective transformation is
given by:
Zx = Z
xy
1
=
f 0 0 00 f 0 0
0 0 1 0
X
Y
Z
1
= Kf Π0 X .
where we have introduced the two matrices
Kf ≡
f 0 00 f 0
0 0 1
and Π0 ≡
1 0 0 00 1 0 0
0 0 1 0
.
The matrix Π0 is referred to as the standard projection matrix.
Assuming Z to be a constant λ > 0, we obtain:
λx = Kf Π0 X .
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 12/24
An Ideal Perspective Camera
From the previous lectures, we know that due to the rigid
motion of the camera, the point X in camera coordinates is
given as a function of the point in world coordinates X 0 by:
X = RX 0 + T ,
or in homogeneous coordinates X = (X ,Y ,Z ,1)>:
X = gX 0 =
(
R T
0 1
)
X 0.
In total, the transformation from world coordinates to image
coordinates is therefore given by
λx = Kf Π0 g X 0.
If the focal length f is known, it can be normalized to 1 (by
changing the units of the image coordinates), such that:
λx = Π0 X = Π0 g X 0.
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 13/24
Intrinsic Camera Parameters
If the camera is not centered at the optical center, we have an
additional translation ox ,oy and if pixel coordinates do not have
unit scale, we need to introduce an additional scaling in x- and
y -direction by sx and sy . If the pixels are not rectangular, we
have a skew factor sθ.
The pixel coordinates (x ′, y ′,1) as a function of homogeneous
camera coordinates X are then given by:
λ
x ′y ′
1
=
sx sθ ox0 sy oy
0 0 1
︸ ︷︷ ︸
≡Ks
f 0 00 f 0
0 0 1
︸ ︷︷ ︸
≡Kf
1 0 0 00 1 0 0
0 0 1 0
︸ ︷︷ ︸
≡Π0
X
Y
Z
1
After the perspective projection Π0 (with focal length 1), we
have an additional transformation which depends on the
(intrinsic) camera parameters. This can be expressed by the
intrinsic parameter matrix K = Ks Kf .
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 14/24
The Intrinsic Parameter Matrix
All intrinsic camera parameters therefore enter the intrinsic
parameter matrix
K ≡ KsKf =
fsx fsθ ox0 fsy oy
0 0 1
.
As a function of the world coordinates X 0, we therefore have:
λx ′ = K Π0 X = K Π0 g X 0 ≡ Π X 0.
The 3× 4 matrix Π ≡ K Π0 g = (KR,KT ) is called a general
projection matrix.
Although the above equation looks like a linear one, we still
have the scale parameter λ. Dividing by λ gives:
x ′ =
π>1 X 0
π>3 X 0
, y ′ =
π>2 X 0
π>3 X 0
, z ′ = 1,
where π>1 , π
>
2 , π
>
3 ∈ R
4 are the three rows of the projection
matrix Π.
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 15/24
The Intrinsic Parameter Matrix
The entries of the intrinsic parameter matrix
K =
fsx fsθ ox0 fsy oy
0 0 1
,
can be interpreted as follows:
ox : x-coordinate of principal point in pixels,
oy : y -coordinate of principal point in pixels,
fsx = αx : size of unit length in horizontal pixels,
fsy = αy : size of unit length in vertical pixels,
αx/αy : aspect ratio σ,
fsθ: skew of the pixel, often close to zero.
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 16/24
Spherical Perspective Projection
The perspective pinhole camera introduced above considers a
planar imaging surface. Instead, one can consider a spherical
projection surface given by the unit sphere
S2 ≡ {x ∈ R3
∣∣ |x | = 1}. The spherical projection πs of a 3D
point X is given by:
πs : R3 → S2; X 7→ x =
X
|X |
.
The pixel coordinates x ′ as a function of the world coordinates
X 0 are:
λx ′ = K Π0 g X 0,
except that the scalar factor is now λ = |X | =
√
X 2 + Y 2 + Z 2.
One often writes x ∼ y for homogeneous vectors x and y if
they are equal up to a scalar factor. Then we can write:
x ′ ∼ Π X 0 = K Π0 g X 0.
This property holds for any imaging surface, as long as the ray
between X and the origin intersects the imaging surface.
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 17/24
Radial Distortion
bookshelf with regular lens bookshelf with short focal lens
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 18/24
Radial Distortion
The intrinsic parameters in the matrix K model linear
distortions in the transformation to pixel coordinates. In
practice, however, one can also encounter significant
distortions along the radial axis, in particular if a wide field of
view is used or if one uses cheaper cameras such as
webcams. A simple effective model for such distortions is:
x = xd (1 + a1r
2 + a2r
4), y = yd (1 + a1r
2 + a2r
4),
where xd ≡ (xd , yd ) is the distorted point, r2 = x2d + y
2
d . If a
calibration rig is available, the distortion parameters a1 and a2
can be estimated.
Alternatively, one can estimate a distortion model directly from
the images. A more general model (Devernay and Faugeras
1995) is
x = c + f (r)(xd − c), with f (r) = 1 + a1r + a2r2 + a3r3 + a4r4,
Here, r = |xd − c| is the distance to an arbitrary center of
distortion c and the distortion correction factor f (r) is an
arbitrary 4-th order expression. Parameters are computed from
distortions of straight lines or simultaneously with the 3D
reconstruction (Zhang ’96, Stein ’97, Fitzgibbon ’01).
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 19/24
Preimage of Points and Lines
The perspective transformation introduced above allows to
define images for arbitrary geometric entities by simply
transforming all points of the entity. However, due to the
unknown scale factor, each point is mapped not to a single
point x , but to an equivalence class of points y ∼ x . It is
therefore useful to study how lines are transformed.
A line L in 3-D is characterized by a base point
X 0 = (X0,Y0,Z0,1)> ∈ R4 and a vector
V = (V1,V2,V3,0)> ∈ R4:
X = X 0 + µV , µ ∈ R.
The image of the line L is given by
x ∼ Π0X = Π0(X 0 + µV ) = Π0X 0 + µΠ0V .
All points x treated as vectors from the origin o span a 2-D
subspace P. The intersection of this plane P with the image
plane gives the image of the line. P is called the preimage of
the line.
A preimage of a point or a line in the image plane is the largest
set of 3D points that give rise to an image equal to the given
point or line.
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 20/24
Preimage and Coimage
Preimage P of a line L
Preimages can be defined for curves or other more
complicated geometric structures. In the case of points and
lines, however, the preimage is a subspace of R3. This
subspace can also be represented by its orthogonal
complement, i.e. the normal vector in the case of a plane. This
complement is called the coimage. The coimage of a point or a
line is the subspace in R3 that is the (unique) orthogonal
complement of its preimage. Image, preimage and coimage
are equivalent because they uniquely determine oneanother:
image = preimage ∩ image plane, preimage = span(image),
preimage = coimage⊥, coimage = preimage⊥.
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 21/24
Preimage and Coimage of Points and Lines
In the case of the line L, the preimage is a 2D subspace,
characterized by the 1D coimage given by the span of its
normal vector ` ∈ R3. All points of the preimage, and hence all
points x of the image of L are orthogonal to `:
`> x = 0.
The space of all vectors orthogonal to ` is spanned by the row
vectors of ̂̀, thus we have:
P = span(̂̀).
In the case that x is the image of a point p, the preimage is a
line and the coimage is the plane orthogonal to x , i.e. it is
spanned by the rows of the matrix x̂ .
In summary we have the following table:
Image Preimage Coimage
Point span(x)∩ im. plane span(x) ⊂ R3 span(x̂) ⊂ R3
Line span(̂̀)∩ im. plane span(̂̀) ⊂ R3 span(`) ⊂ R3
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 22/24
Summary
In this part of the lecture, we studied the perspective projection
which takes us from the 3D (4D) camera coordinates to 2D
camera image coordinates and pixel coordinates. In
homogeneous coordinates, we have the transformations:
4D World coordinates
g∈SE(3)
−→ 4D Camera coordinates Kf Π0−→
3D image coordinates
Ks−→ 3D pixel coordinates.
In particular, we can summarize the (intrinsic) camera
parameters in the matrix
K = Ks Kf .
The full transformation from world coordinates X 0 to pixel
coordinates x ′ is given by:
λx ′ = K Π0 g X 0.
Moreover, for the images of points and lines we introduced the
notions of preimage (maximal point set which is consistent with
a given image) and coimage (its orthogonal complement). Both
can be used equivalently to the image.
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 23/24
Projective Geometry
In order to formally write transformations by linear operations,
we made extensive use of homogeneous coordinates to
represent a 3D point as a 4D-vector (X ,Y ,Z ,1) with the last
coordinate fixed to 1. This normalization is not always
necessary: One can represent 3D points by a general 4D
vector
X = (XW ,YW ,ZW ,W ) ∈ R4,
remembering that merely the direction of this vector is of
importance. We therefore identify the point in homogeneous
coordinates with the line connecting it with the origin. This
leads to the definition of projective coordinates.
An n-dimensional projective space Pn is the set of all
one-dimensional subspaces (i.e. lines through the origin) of the
vector space Rn+1. A point p ∈ Pn can then be assigned
homogeneous coordinates X = (x1, . . . , xn+1)>, among which
at least one x is nonzero. For any nonzero λ ∈ R, the
coordinates Y = (λx1, . . . , λxn+1)> represent the same point p.
Perspective Projection
Prof. Daniel Cremers
Historic Remarks
Mathematical
Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and
Coimage
Projective Geometry
updated April 12, 2021 24/24
Projective Geometry
If the two coordinate vectors X and Y differ by a scalar factor,
then they are said to be equivalent:
X ∼ Y .
The point p is represented by the equivalence class of all
multiples of X . Since all points are represented by lines
through the origin, there exist two alternative representations
for the two-dimensional projective space P2:
1 One can represent each point as a point on the 2D-sphere
S2, where any antipodal points represent the same line.
2 One can represent each point p either as a point on the
plane of R2 (homogeneous coordinates) modeling all
points with non-zero z-component, or as a point on the
circle S1 (again identifying antipodal points) which is
equivalent to P1.
Both representations hold for the n-dimensional projective
space Pn, which can be either seen as a an nD-sphere Sn or as
Rn with Pn−1 attached (to model lines at infinity).
Historic Remarks
Mathematical Representation
Intrinsic Parameters
Spherical Projection
Radial Distortion
Preimage and Coimage
Projective Geometry