程序代写代做代考 C algorithm html graph Fundamentals of Computer Vision

Fundamentals of Computer Vision
Lecture

Overview of today’s lecture
• Other camera models.
• Pose estimation.
• Leftover from previous lecture: Other types of cameras, calibration.
• Triangulation.
• Epipolar geometry.
• Essential matrix.
• Fundamental matrix.
• 8-point algorithm.

Slide credits
Most of these slides were adapted from:
• Kris Kitani (15-463, Fall 2016, Fall 2017), Ioannis Gkioulekas (16-385, Spring 2019), Robert Colin (454, Fall 2019s).
Some slides were inspired or taken from: • Fredo Durand (MIT).

Geometric camera calibration

Pose Estimation
Given a single image,
estimate the exact position of the photographer

Geometric camera calibration
Given a set of matched points
point in 3D point in the space image
and camera model
Where do we get these matched points from?
projection model
parameters
Camera matrix
Find the (pose) estimate of
We’ll use a perspective camera model for pose estimation

Calibration using a reference object
Given n points with known 3D coordinates Xi and known image projections xi, estimate the camera parameters
Place a known object in the scene:
• identify correspondences between image and scene
• compute mapping from scene to
image
P?
Xi xi
Issues:
• must know geometry very accurately
• must know 3D->2D correspondence

Same setup as homography estimation (slightly different derivation here)

Mapping between 3D point and image points
What are the unknowns?

Mapping between 3D point and image points
Heterogeneous coordinates
(non-linear relation between coordinates)
How can we make these relations linear?

How can we make these relations linear?
Make them linear with algebraic manipulation…
Now we can setup a system of linear equations with multiple point correspondences

How do we proceed?

In matrix form …
How do we proceed?

In matrix form …
For N points …
𝑨𝒙 = 𝟎
How do we solve this system?

Solve for camera matrix by
SVD!

Solve for camera matrix by
Solution x is the column of V corresponding to smallest singular value of

Now we have:
𝒑&’ 𝒑'( 𝒑’)
Are we done?

Almost there …
How do you get the intrinsic and extrinsic parameters from the projection matrix?

Decomposition of the Camera Matrix

Decomposition of the Camera Matrix
Find the camera center C Find intrinsic K and rotation R
What is the projection of the camera center?

Decomposition of the Camera Matrix
Find the camera center C Find intrinsic K and rotation R
How do we compute the camera center from this?

Decomposition of the Camera Matrix
Find the camera center C Find intrinsic K and rotation R
SVD of P!
c is the Eigenvector corresponding to the smallest Eigenvalue

Decomposition of the Camera Matrix
Find the camera center C
SVD of P!
c is the Eigenvector corresponding to smallest Eigenvalue
Find intrinsic K and rotation R
Any useful properties of K and R we can use?

Decomposition of the Camera Matrix
Recall?
K
Find the camera center C
SVD of P!
c is the Eigenvector corresponding to smallest Eigenvalue
Find intrinsic K and rotation R
How do we find K and R?
right upper triangle
orthogonal

Decomposition of the Camera Matrix
Find the camera center C Find intrinsic K and rotation R
SVD of P!
c is the Eigenvector corresponding to smallest Eigenvalue
QR decomposition

Geometric camera calibration
Advantages:
• Very simple to formulate.
• Analytical solution.
Disadvantages:
• Doesn’t model radial distortion.
• Hard to impose constraints (e.g., known f).
• Doesn’t minimize the correct error function.
For these reasons, nonlinear methods are preferred
• Define error function E between projected 3D points and image positions
– E is nonlinear function of intrinsics, extrinsics, radial distortion • Minimize E using nonlinear optimization techniques

Radial distortion
What causes this distortion?
no distortion barrel pincushion distortion distortion

Minimizing reprojection error
(u1,v1)
MPi
Pi
P
1
(ui,vi)
ççui- 1 i÷÷+ççvi- 2 i÷÷
æ m ×P ö2 æ
m ×P ö2
m×P m×P è3iøè3iø

Radial distortion model

Minimizing reprojection error with radial distortion
(u1,v1)
MPi
Pi
P
1
(ui,vi) reprojection error:
1𝑚&⋅𝑃+ ( 𝑢+−λ𝑚)⋅𝑃+
1𝑚(⋅𝑃+ ( + 𝑣+−λ𝑚)⋅𝑃+
Adddistortionsto

Correcting radial distortion
before after

Alternative: Multi-plane calibration
Advantages:
• Only requires a plane
• Don’t have to know positions/orientations
• Great code available online!
– Matlab version: http://www.vision.caltech.edu/bouguetj/calib_doc/index.html
– AlsoavailableonOpenCV.
Disadvantage: Need to solve non-linear optimization problem.

Step-by-step demonstration

Many different ways to calibrate a camera:
• Radiometric calibration.
• Color calibration.
• Geometric calibration.
• Noise calibration.
• Lens (or aberration) calibration.

Vanishing points and lines
Parallel lines in the world intersect in the image at a “vanishing point”
Slide source: Derek Hoiem

Vanishing points and lines
Vanishing Point
Vanishing Point oo
Vanishing Line
• The projections of parallel 3D lines intersect at a vanishing point
• The projection of parallel 3D planes intersect at a vanishing line
• If a set of parallel 3D lines are also parallel to a particular plane,
their vanishing point will lie on the vanishing line of the plane
• Not all lines that intersect are parallel
Slide source: Derek Hoiem

Vanishing points and lines
Vertical vanishing point
(at infinity)
Vanishing line
Vanishing point
Vanishing point
Slide from Efros, Photo from Criminisi

Two-view geometry

Triangulation

Triangulation
image 1
image 2
Given
camera 1 with matrix
camera 2 with matrix

Triangulation
Which 3D points map to x?
image 1 image 2
camera 1 with matrix camera 2 with matrix

Triangulation
How can you compute this ray?
image 1 image 2
camera 1 with matrix camera 2 with matrix

Create two points on the ray:
1) find the camera center;; and
2) apply the pseudo-inverse of P on x. Then connect the two points.
+
Why does this
point map to x?
Triangulation
image 1
image 2
camera 1 with matrix
camera 2 with matrix

Triangulation
How do we find the exact point on the ray?
+
image 1
image 2
camera 1 with matrix
camera 2 with matrix

Triangulation
Find 3D object point
Will the lines intersect?
image 1 image 2
camera 1 with matrix camera 2 with matrix

Triangulation
Find 3D object point (no single solution due to noise)
image 1 image 2
camera 1 with matrix camera 2 with matrix

Triangulation
Given a set of (noisy) matched points
and camera matrices
Estimate the 3D point

(homogeneous coordinate)
Also, this is a similarity relation because it involves homogeneous coordinates
(homogeneous coordinate)
Same ray direction but differs by a scale factor
How do we solve for unknowns in a similarity relation?
Remove scale factor, convert to linear system and solve with SVD.

Recall: Cross Product
Vector (cross) product
takes two vectors and returns a vector perpendicular to both
cross product of two vectors in the same direction is zero
remember this!!!

or
Same direction but differs by a scale factor
Cross product of two vectors of same direction is zero (this equality removes the scale factor)

Do you remember this?

Using the fact that the cross product should be zero
Third line is a linear combination of the first and second lines.
(x times the first line plus y times the second line), rank 2 matrix!
One 2D to 3D point correspondence give you 2 equations

Using the fact that the cross product should be zero
Third line is a linear combination of the first and second lines. (x times the first line plus y times the second line)
One 2D to 3D point correspondence give you 2 equations

Now we can make a system of linear equations (two lines for each 2D point correspondence)

Concatenate the 2D points from both images
sanity check! dimensions?
How do we solve homogeneous linear system?

Concatenate the 2D points from both images
How do we solve homogeneous linear system?
SVD!

minimize subject to
Solution is the eigenvector
corresponding to the smallest eigenvalue of

Related Posts