Fundamentals of Computer Vision
Lecture
• Epipolar geometry.
• Essential matrix.
Overview of today’s lecture
Slide credits
Most of these slides were adapted from:
• Kris Kitani (15-463, Fall 2016, Fall 2017), Ioannis Gkioulekas (16-385, Spring 2019), Robert Colin (454, Fall 2019s).
Some slides were inspired or taken from: • Fredo Durand (MIT).
Epipolar geometry
Epipolar geometry
Image plane
Epipolar geometry
Image plane
Baseline
Epipolar geometry
Image plane
Baseline
Epipole
(projection of o’ on the image plane)
Epipolar geometry
Image plane
Epipolar plane
Baseline
Epipole
(projection of o’ on the image plane)
Epipolar line (intersection of Epipolar plane and image plane)
Epipolar geometry
Baseline
Epipole
(projection of o’ on the image plane)
Epipolar plane
Image plane
What is this?
Quiz
Epipolar plane
Quiz
What is this?
Epipolar plane
Quiz
Epipolar line (intersection of Epipolar plane and image plane)
Epipolar plane
Quiz
Epipolar line (intersection of Epipolar plane and image plane)
Epipolar plane
Quiz
What is this?
Epipolar line (intersection of Epipolar plane and image plane)
Epipolar plane
Quiz
Epipole
(projection of o’ on the image plane)
Epipolar line (intersection of Epipolar plane and image plane)
Epipolar plane
Quiz
What is this?
Epipole
(projection of o’ on the image plane)
Epipolar line (intersection of Epipolar plane and image plane)
Epipolar plane
Quiz
Baseline
Epipole
(projection of o’ on the image plane)
Epipolar constraint
Potential matches for lie on the epipolar line
Epipolar constraint
Potential matches for lie on the epipolar line
Epipolar constraint
P
P P
x
x’ x’
x’
Potential matches for x have to lie on the corresponding line l’. Potential matches for x’ have to lie on the corresponding line l.
The point x (left image) maps to a ___________ in the right image The baseline connects the ___________ and ____________
An epipolar line (left image) maps to a __________ in the right image
An epipole e is a projection of the ______________ on the image plane All epipolar lines in an image intersect at the ______________
Converging cameras
Where is the epipole in this image?
Converging cameras
here!
Where is the epipole in this image? It’s not always in the image
Parallel cameras
Where is the epipole?
Parallel cameras
epipole at infinity
The epipolar constraint is an important concept for stereo vision Task: Match point in left image to point in right image
Left image Right image
How would you do it?
Recall:Epipolar constraint
Potential matches for lie on the epipolar line
The epipolar constraint is an important concept for stereo vision Task: Match point in left image to point in right image
Left image Right image
Want to avoid search over entire image
Epipolar constraint reduces search to a single line
The epipolar constraint is an important concept for stereo vision Task: Match point in left image to point in right image
Left image Right image
Want to avoid search over entire image
Epipolar constraint reduces search to a single line
How do you compute the epipolar line?
The Essential Matrix
Recall:Epipolar constraint
Potential matches for lie on the epipolar line
Given a point in one image,
multiplying by the essential matrix will tell us the epipolar line in the second view.
Motivation
The Essential Matrix is a 3 x 3 matrix that encodes epipolar geometry
Given a point in one image,
multiplying by the essential matrix will tell us the epipolar line in the second view.
Recall: Cross and Dot Products
Vector (cross) product
takes two vectors and returns a vector perpendicular to both
dot product of two orthogonal vectors is zero
Cross product
Can also be written as a matrix multiplication
Skew symmetric
Representing the …
Epipolar Line
in vector form
If the point is on the epipolar line then
Epipolar Line
in vector form
If the point is on the epipolar line then
vector representing the line is normal (orthogonal) to the plane
vector representing the point x is inside the plane
Therefore:
So if and then
So if and then
Essential Matrix vs Homography
What’s the difference between the essential matrix and a homography?
Essential Matrix vs Homography
What’s the difference between the essential matrix and a homography?
They are both 3 x 3 matrices but …
Essential matrix maps a Homography maps a point to a line point to a point
Where does the Essential matrix come from?
Does this look familiar?
Camera-camera transform just like world-camera transform
These three vectors are coplanar
If these three vectors are coplanar then
If these three vectors are coplanar then
If these three vectors are coplanar then
If these three vectors are coplanar then
putting it together
rigid motion coplanarity
putting it together
rigid motion coplanarity
putting it together
rigid motion coplanarity
putting it together
rigid motion coplanarity
putting it together
rigid motion coplanarity
Essential Matrix
[Longuet-Higgins 1981]
Properties of the E matrix
Longuet-Higgins equation
(points in normalized coordinates)
Properties of the E matrix
Longuet-Higgins equation
Epipolar lines
(points in normalized coordinates)
Properties of the E matrix
Longuet-Higgins equation
Epipolar lines
Epipoles
(points in normalized coordinates)
Recall:Epipolar constraint
Potential matches for lie on the epipolar line
Given a point in one image,
multiplying by the essential matrix will tell us the epipolar line in the second view.
Assumption:
points aligned to camera coordinate axis (calibrated camera)
How do you generalize to uncalibrated cameras?
The Fundamental matrix
The
Fundamental matrix
is a
generalization
of the
Essential matrix,
where the assumption of
calibrated cameras
is no longer valid