Multiple View Geometry: Solution Sheet 8
Prof. Dr. Florian Bernard, Florian Hofherr, Tarun Yenamandra
Computer Vision Group, TU Munich
Link Zoom Room , Password: 307238
Exercise: June 9th, 2021
Part I: Theory
1. Image Warping
(a) Look at the warping function τ(ξ,x) in Eq. (9). What do τ(ξ,x) and ri(ξ) look like at
ξ = 0?
For ξ = 0, we have
T (g(0),p) = T (Id4,p) = Id3p + 0 = p .
Thus, τ(0,x) becomes
τ(0,x) = π
(
π−1(x, Z1(x))
)
= x ,
where the last equality follows from inserting the formulas for π and π−1. Finally,
ri(0) = I2(τ(0,xi))− I1(xi) = I2(xi)− I1(xi) .
(b) Prove that the derivative of ri(ξ) w.r.t. ξ at ξ = 0 is
∂ri(ξ)
∂ξ
∣∣∣∣
ξ=0
=
1
z
(
Ixfx Iyfy
)(1 0 −x
z
−xy
z
z + x
2
z
−y
0 1 −y
z
−z − y
2
z
xy
z
x
)∣∣∣∣∣
(x,y,z)>=π−1(xi,Z1(xi))
To this end, apply the chain rule multiple times and use the following identity:
∂T (g(ξ),p)
∂ξ
∣∣∣∣
ξ=0
=
(
Id3 −p̂
)
∈ R3×6 .
Since I1(xi) does not depend on ξ, we only need to look at the first term in ri(ξ). It is a
composition of the functions I2, π and T . Applying the chain rule gives
∂ri(ξ)
∂ξ
∣∣∣∣
ξ=0
=
∂I2(y)
∂y
∣∣∣∣
y=π(T (g(0),π−1(xi,Z1(xi))))
·
∂π(p)
∂p
∣∣∣∣
p=T (g(0),π−1(xi,Z1(xi)))
·
·
∂T (g(ξ), π−1(xi, Z1(xi)))
∂ξ
∣∣∣∣
ξ=0
.
Now, we know that T (g(0),p) = p, so we can write
∂ri(ξ)
∂ξ
∣∣∣∣
ξ=0
=
[
(∇I2 (π(p)))> ·
∂π(p)
∂p
·
(
Id3 −p̂
)]∣∣∣∣
p=π−1(xi,Z1(xi))
1
https://tum-conf.zoom.us/s/62772800235?pwd=SUpZN2QrV0JpeXJyR2R1TWx5cHEwdz09
The second and third term are
∂π(p)
∂p
=
∂∂x
(
fxx
z
)
∂
∂y
(
fxx
z
)
∂
∂z
(
fxx
z
)
∂
∂x
(
fyy
z
)
∂
∂y
(
fyy
z
)
∂
∂z
(
fyy
z
)
= 1
z
(
fx 0
0 fy
)(
1 0 −x
z
0 1 −y
z
)
(
Id3 −p̂
)
=
1 0 0 0 z −y0 1 0 −z 0 x
0 0 1 y −x 0
Performing the matrix multiplication and using
π(p) = π
(
π−1(xi, Z1(xi))
)
= xi
(see (a)) as well as
(∇I2(xi))>
(
fx 0
0 fy
)
=
(
Ix(xi)fx Iy(xi)fy
)
leads to the desired result.
(c) Following the derivation in (b), determine the derivative for arbitrary ξ
∂ri(∆ξ ◦ ξ)
∂∆ξ
∣∣∣∣
∆ξ=0
where ◦ is defined by
ξ1 ◦ ξ2 := log
(
exp(ξ̂1) · exp(ξ̂2)
)∨
.
We observe that due to associativity of matrix multiplication
T (g(∆ξ ◦ ξ),p) = T (g(∆ξ)g(ξ),p) = T (g(∆ξ), T (g(ξ),p))
and thus
∂T (g(∆ξ ◦ ξ),p)
∂∆ξ
∣∣∣∣
∆ξ=0
=
∂T (g(∆ξ), T (g(ξ),p))
∂∆ξ
∣∣∣∣
∆ξ=0
=
(
Id3 −p̂′
)
∈ R3×6
with
p′ = T (g(ξ),p) .
Thus analogously to (b) we can derive
∂ri(∆ξ ◦ ξ)
∂∆ξ
∣∣∣∣
∆ξ=0
=
[(
∇I2
(
π(p′)
))> · ∂π(p′)
∂p′
·
(
Id3 −p̂′
)]∣∣∣∣
p′=T (g(ξ),π−1(xi,Z1(xi)))
.
Notice that the unlike in (b), the image gradient is now evaluated at the warped point
τ(ξ,xi). With this we get
∂ri(∆ξ ◦ ξ)
∂∆ξ
∣∣∣∣
∆ξ=0
=
1
z′
(
Ixfx Iyfy
)(1 0 −x′
z′
−x
′y′
z′
z′ + x
′2
z′
−y′
0 1 −y
′
z′
−z′ − y
′2
z′
x′y′
z′
x′
)∣∣∣∣∣
(x′,y′,z′)>=p′
,
p′ = T (g(ξ), π−1(xi, Z1(xi))) ,
2
which is very similar to the result in (b), just that everything (including the image gradient)
is evaluated at the transformed point p′. What follows are some additional remarks.
Note: In this exercise we use what is called “Forward Additive” direct image alignment,
whereas the 2013 ICRA paper describes “Forward Compositional” approach. The differ-
ence in the latter is that you always compute an intermediate “warped” image in every
iteration and you compute image gradients for that. For further explanation of the two
approaches see Christian Kerl’s Master thesis1 (Sections 4.2.1. and 4.2.2.).
Note: It is in principle also possible to linearize as
∂ri(∆ξ + ξ)
∂∆ξ
∣∣∣∣
∆ξ=0
,
however, the derivative becomes more complicated and slower to compute, since the
derivative of the exponential map for SE(3) for ξ 6= 0 is a bit more involved. More-
over, since with that we only use a linear approximation of the exponential map of SE(3),
convergence properties of the resulting optimization algorithm can be slower, as suggested
by some author’s experiments. Lastly, one needs to then also perform the update step as
ξnew = ∆ξ + ξold and take special care at rotations close to 180◦ (e.g. wrapping around to
−180◦).
2. Image Pyramids
How does the camera matrix K change from level l to l + 1? Write down f (l+1)x , f
(l+1)
y , c
(l+1)
x
and c(l+1)y in terms of f
(l)
x , f
(l)
y , c
(l)
x and c
(l)
y .
Looking at how each pixel coordinate transforms from one image level l to the next, l + 1, we
have
2x(l+1) + 1
2
= x(l) ⇒ x(l+1) = 1
2
x(l) − 1
4
.
Plugging into the latter the relations x(l+1) = 1
Z
K(l+1)X and x(l) = 1
Z
K(l)X results in
f (l+1)x =
1
2
f (l)x , f
(l+1)
y =
1
2
f (l)y , c
(l+1)
x =
1
2
c(l)x −
1
4
, c(l+1)y =
1
2
c(l)y −
1
4
.
Note: Remember that we define the (0, 0) continuous image coordinate to lie in the center of
the top-left pixel.
3. Optimization for Normally Distributed p(ri)
(a) Confirm that a normally distributed p(ri) with a uniform prior on the camera motion leads
to normal least squares minimization. To this end, use
p(ri|ξ) = p(ri) = A exp
(
−
r2i
σ2
)
to show that with a constant prior p(ξ), the maximum a posteriori estimate is given by
ξMAP = arg min
ξ
∑
i
ri(ξ)
2 .
p(ri|ξ) = p(ri) = A exp
(
−
r2i
σ2
)
⇒ − log p(ri|ξ) = − logA+
r2i
σ2
1https://vision.in.tum.de/_media/spezial/bib/kerl2012msc.pdf
3
https://vision.in.tum.de/_media/spezial/bib/kerl2012msc.pdf
Inserting into Eq. (15) gives
ξMAP = arg min
ξ
(
−N logA+
1
σ2
∑
i
ri(ξ)
2 − log p(ξ)
)
= arg min
ξ
∑
i
ri(ξ)
2 ,
since −N logA and − log p(ξ) are just constant shifts and 1
σ2
is only a scaling, and none
of them changes the argmin.
(b) Explicitly show that the weights
w(ri) =
1
ri
∂ log p(ri)
∂ri
are constant for normally distributed p(ri).
w(ri) =
1
ri
∂ log p(ri)
∂ri
=
1
ri
∂
(
logA− ri(ξ)
2
σ2
)
∂ri
=
1
ri
(
0−
2ri
σ2
)
= −
2
σ2
= const(ri)
(c) Show that in the case of normally distributed p(ri) the update step ∆ξ can be computed as
∆ξ = −
(
J>J
)−1
J>r(0) .
Eq. (21) reads
J>WJ∆ξ = −J>Wr(0) ,
with W a diagonal matrix with constant diagonal entries Wii = w(ri) = − 2σ2 .
⇒ W = −
2
σ2
Id ⇒ −
2
σ2
J>J∆ξ =
2
σ2
J>r(0) ⇒ claim .
4