Multiple View Geometry: Solution Sheet 5
Prof. Dr. Florian Bernard, Florian Hofherr, Tarun Yenamandra
Computer Vision Group, TU Munich
Link Zoom Room , Password: 307238
Exercise: May 26th, 2020
Part I: Theory
1. The Lucas-Kanade method
(a) Prove that the minimizer b of E(v) can be written as
b = −M−1q
where the entries of M and q are given by
mij = G ∗
(
Ixi · Ixj
)
and qi = G ∗ (Ixi · It)
We start by expanding the squared term in the energy E(v):
E(v) =
∫
W (x)
G(x− x′)
(
∇I(x′, t)>v
)2
dx′ +
∫
W (x)
G(x− x′)2∇I(x′, t)>v∂tI(x′, t)dx′+
+
∫
W (x)
G(x− x′)
(
∂tI(x
′, t)
)2
dx′
Now, for each term in the sum we take the derivative (gradient) with respect to v:
dE
dv
=
∫
W (x)
G(x− x′)2∇I(x′, t)
(
∇I(x′, t)>v
)
dx′+
+
∫
W (x)
G(x− x′)2∇I(x′, t)∂tI(x′, t)dx′ + 0 =
= 2
(
G ∗
(
∇I∇I>
))
v + 2 (G ∗ (∇I∂tI)) =: 2Mv + 2q
where M is defined as G ∗
(
∇I∇I>
)
and q as G ∗ (∇I∂tI). We further know ∇I∇I>
and ∇I∂tI:
∇I∇I> =
(
Ix
Iy
)(
Ix Iy
)
=
(
(Ix)
2 IxIy
IxIy (Iy)
2
)
and ∇I∂tI =
(
IxIt
IyIt
)
which proves that the entries of M and q are as stated. Since we want to find a minimizer
b of E(v), we require
dE(v)
dv
∣∣∣∣
v=b
= 0 ⇒ 2Mb+ 2q = 0 ⇒ b = −M−1q
1
https://tum-conf.zoom.us/s/62772800235?pwd=SUpZN2QrV0JpeXJyR2R1TWx5cHEwdz09
(b) Show that if the gradient direction is constant in W (x), i.e. ∇I(x′, t) = α(x′, t)u for a
scalar function α and a 2D vector u, M is not invertible.
u does not depend on x′, so it can be pulled out of the convolution integral. Thus,
M = G∗
(
∇I∇I>
)
=
(
G ∗ α2
)
uu> ⇒ detM =
(
G ∗ α2
)2 (
u21u
2
2 − (u1u2)
2
)
= 0.
Explain how this observation is related to the aperture problem.
The aperture problem states that it is impossible to determine the motion orthogonal to
the gradient direction in regions with constant gradient direction,. M not being invertible
means that there is no unique solution b, which is the mathematical formulation of “the
motion cannot be determined”.
(c) Write down explicit expressions for the two components b1 and b2 of the minimizer in
terms of mij and qi.
b = −M−1q where M−1 =
1
detM
(
m22 −m12
−m12 m11
)
⇒
(
b1
b2
)
=
−1
m11m22 −m212
(
m22 −m12
−m12 m11
)(
q1
q2
)
=
m22q1−m12q2
m212−m11m22
m11q2−m12q1
m212−m11m22
2. The Reconstruction Problem
The bundle adjustment (re-)projection error for N points X1,…,XN is
E(R,T,X1, …,XN ) =
N∑
j=1
(
‖xj1 − π(Xj)‖
2 + ‖xj2 − π(RXj +T)‖
2
)
(a) What dimension does the space of unknown variables have if …
– … R is restricted to a rotation about the camera’s y-axis? 4 + 3N
– … the camera is only rotated, not translated? 3 + 3N .
– … the points Xj are known to all lie on one plane? 9 + 2N .
In contrast to the projection error, the 8-point algorithm decouples the rigid body motion from
the coordinates Xj .
(b) Which constrained optimization problem does the 8-point algorithm solve? Write down a
cost function E8-pt(R,T) and according constraints using x
j
1, x
j
2, R and T.
E8-pt(R,T) =
N∑
j=1
(
(x
j
2)
>
(
T×Rxj1
))2
with ‖T‖ = 1
(c) Can the 8-point algorithm be used if …
– … R is restricted to a rotation about the camera’s y-axis? Yes.
– … the camera is only rotated, not translated? No.
– … the points Xj are known to all lie on one plane? No.
2