代写代考 Perception for Autonomous Systems 31392:

Perception for Autonomous Systems 31392:
Epipolar Geometry and the Fundamental Matrix
Lecturer: —PhD
10 Feb. 2020 DTU Electrical Engineering 2

Epipolar Geometry: General Case
• Assuming:
– 2 Camera Views
– A ray passing through the camera center
10 Feb. 2020 DTU Electrical Engineering 3

Epipolar Geometry: General Case
• Assuming:
– 2 Camera Views
– A ray passing through the camera center
• We can find the:
– baseline and
– the epipolar lines
10 Feb. 2020 DTU Electrical Engineering 3

Epipolar Geometry: General Case
• Assuming:
– 2 Camera Views
– A ray passing through the camera center
• We can find the:
– baseline and
– the epipolar lines
– Through the epipolar plane
10 Feb. 2020 DTU Electrical Engineering 3

Epipolar Geometry
• What about this case?
10 Feb. 2020 DTU Electrical Engineering 6

Epipolar Geometry
• What about this case?
• What’s the epipolar plane?
10 Feb. 2020 DTU Electrical Engineering 6

Epipolar Geometry
• What about this case?
• What’s the epipolar plane?
• Can we find the epipole of I1?
10 Feb. 2020 DTU Electrical Engineering 6

Epipolar Geometry
• What about this case?
• What’s the epipolar plane?
• Can we find the epipole of I1? • What about the epipole of I2?
10 Feb. 2020 DTU Electrical Engineering 6

Epipolar Geometry:
• How about this case?
10 Feb. 2020 DTU Electrical Engineering 10

Epipolar Geometry:
• How about this case?
• Where are the epipoles?
10 Feb. 2020 DTU Electrical Engineering 10

Epipolar Geometry:
• How about this case?
• Where are the epipoles?
• How would such a case be usefull
– The scanline is used in Disparity calculation
10 Feb. 2020 DTU Electrical Engineering 10

Example of Epipolar Lines “in the wild”
• Let’s think of this example:
• Where should the epipole be?
10 Feb. 2020 DTU Electrical Engineering 13

Example of Epipolar Lines “in the wild”
• What about this:
• How should the epipolar Lines look like?
10 Feb. 2020 DTU Electrical Engineering 15

Essential Matrix
• Assuming two calibrated stereo pairs:
• We can express the points x, y
from the image plane to homogeneous
coordinates 𝑥ො and 𝑥′ using the inverse
of the camera matrix
x = K −1 x = X
ˆ −1   x=K x=X
10 Feb. 2020 DTU Electrical Engineering 17

Essential Matrix
• We can express the left homogeneous point to the right one:
• 𝑥ො = R ∗ 𝑥′ + 𝑇, where R is the Rotation Matrix
and T the translation vector
• The we can prove that there is a Matrix
෡ connecting the two points 𝑥ො, 𝑥′
• Trying to eliminate the left side by
Applying cross product and then dot product
T 𝐱 𝑥ො = T 𝐱 R ∗ 𝑥 ′ + 𝑇 𝐱 𝑇
𝑥ො . T 𝐱 𝑥ො = 𝑥ො . T 𝐱 R ∗ 𝑥 ′ + 0
෡ 0 = 𝑥ො . T 𝐱 R ∗ 𝑥 ′
• Or we can write it in matrix form:
xˆ T E xˆ  = 0 with
E = t  R , where [t] is the skew symmetric matrix x
10 Feb. 2020 DTU Electrical Engineering 18

Essential Matrix
• Thˆe esˆsential matrix
xEx=0 with E=tR
for which:
– E x’ is the epipolar line associated with x’ (l = E x’) – ETx is the epipolar line associated with x (l’ = ETx) – E e’ = 0 and ETe = 0
– E is singular (rank two)
– E has five degrees of freedom
is a 3×3 matrix,
10 Feb. 2020 DTU Electrical Engineering 19

Fundamental Matrix
• We know how to get from a homogeneous point in one camera to another
• How can we get directly from one image to another?
ˆ ˆ xTEx =0
x = K −1 x
ˆ −1  x=Kx
xTFx=0 with F=K−TEK−1 Which is the fundamental matrix
10 Feb. 2020 DTU Electrical Engineering 20

Fundamental Matrix
xTFx=0 with F=K−TEK−1
• The fundamental matrix is a 3×3 matrix,
for which:
– F x’ is the epipolar line associated with x’ – FTx is the epipolar line associated with x – F e’ = 0 and FTe = 0
– F is singular (rank two): det(F)=0
– F has seven degrees of freedom:
10 Feb. 2020 DTU Electrical Engineering 21

How to compute the _H_o_m__o_g_ra_p__h_y Fundamental Matrix • Similar to DLT method as before
It’s called the 8 point algorithm as we need 8 points to solve it xT Fx=0
𝑥𝑥′𝑓 +𝑥𝑦′𝑓 +𝑥𝑓 +𝑦𝑥′𝑓 +𝑦𝑦′𝑓 +𝑦𝑓 +𝑥′𝑓 +𝑦′𝑓 +𝑓 =0
12 13 21 22 23 31 32 33
𝑥𝑥′𝑥𝑦′𝑥𝑦𝑥′𝑦𝑦′𝑦𝑥′𝑦′1 111111111111
𝑓 A𝒇= ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ 13=0
𝑥𝑦′ 𝑥𝑦′ 𝑥 𝑦𝑥′ 𝑦𝑦′ 𝑦 𝑥′ 𝑦′ 1 21 𝑛𝑣𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 ⋮
10 Feb. 2020 DTU Electrical Engineering 22

How to compute the Fundamental Matrix
• Homography (No Translation)
• Fundamental Matrix (Translation) • Correspondence Relation
xT Fx = 0
1. Normalize image coordinates
• 1. 2. 3.
Correspondence Relation
x’ = Hx  x’Hx = 0 Normalize image coordinates
x=Tx x=Tx RANSAC with 4 points
– Solution via SVD De-normalize:
−1 ~ H=T HT
  x=Tx x=Tx
RANSAC with 8 points
– Initial solution via SVD
– Enforce det(F)= 0 by SVD
De-normalize:
 ~ F = T T FT
10 Feb. 2020 DTU Electrical Engineering 23

Epipolar Geometry to Rectified Epipolar Geometry
• To finally go full circle:
– To be able to use the stereo in a block matching algorithm to produce 3D points we must first
convert to rectified stereo
– How would you do that?
10 Feb. 2020 DTU Electrical Engineering 24

Rectified Epipolar Geometry
10 Feb. 2020 DTU Electrical Engineering 25

An Example
• Assume these two images
10 Feb. 2020 DTU Electrical Engineering 26

An Example
• First step is to match some points
• Next, calculate the fundamental matrix
10 Feb. 2020 DTU Electrical Engineering 27

An Example
• Finally calculate the rectified images
• Notice how the images are now scan line
10 Feb. 2020 DTU Electrical Engineering 29

Perception for Autonomous Systems 31392:
Epipolar Geometry and the Fundamental Matrix
Lecturer: —PhD
10 Feb. 2020 DTU Electrical Engineering 29

程序代写 CS代考加微信: powcoder QQ: 1823890830 Email: powcoder@163.com

Related Posts