CMT107 Visual Computing
Stereo Vision
Copyright By PowCoder代写 加微信 powcoder
School of Computer Science and Informatics
Cardiff University
• Stereo Vision
• Multi-view geometry problems
• Triangulation
• Epipolar Geometry
• The epipolar constraint
• Essential matrix and fundamental matrix
• Eight-point algorithm
Acknowledgement
The majority of the slides in this section are from at University of Illinois at Urbana-Champaign
Stereo Vision
• Geometric problem formulation: given several images of the same object or
scene, compute a representation of its 3D shape.
Goal: Recovery of 3D Structure
• Geometric problem formulation: given several images of the same object or
scene, compute a representation of its 3D shape.
• “Images of the same object or scene”
• Arbitrary number of images (from two to thousands)
• Arbitrary camera positions (camera network or video sequence)
• Calibration may be initially unknown
• “Representation of 3D shape”
• Depth maps
• Point clouds
• Patch clouds
• Volumetric models
• Layered models
Goal: Recovery of 3D Structure
• Recovery of structure from one image is inherently ambiguous
Goal: Recovery of 3D Structure
• Recovery of structure from one image is inherently ambiguous
Goal: Recovery of 3D Structure
• Recovery of structure from one image is inherently ambiguous
Goal: Recovery of 3D Structure
• Recovery of structure from one image is inherently ambiguous
http://en.wikipedia.org/wiki/Ames_room
http://en.wikipedia.org/wiki/Ames_room
Goal: Recovery of 3D Structure
• We will need multi-view geometry
Multiview Geometry Problems
• Structure: Given projections of the same 3D point in two or more images,
compute the 3D coordinates of that point
Multiview Geometry Problems
• Motion: Given a set of corresponding points in two or more images, compute
the camera parameters
R1,t1 ? Camera 2
Triangulation
• Given projects of a 3D point in two or more images (with known camera
matrices), find the coordinates of the point
Triangulation
• We want to intersect the two visual rays corresponding to x1 and x2, but
because of noise and numerical errors, they don’t meet exactly
Triangulation: Geometric Approach
• Find shortest segment connecting the two viewing rays and let X be the
midpoint of that segment.
Triangulation: Linear Approach
Cross product as matrix multiplication:
Two independent equations each in terms of
three unknown entries of X
Triangulation: Non-Linear Approach
• Find X that minimizes
XPxdXPxd +
Two-view Geometry
Epipolar Geometry
• Baseline – line connecting the two camera centres
• Epipolar Plane – plane containing baseline (1D family)
• Epipoles: intersections of baseline with image planes; projections of the
other camera centres
• Epipolar Lines: intersections of epipolar plane with image planes (always
come in corresponding pairs)
Example: Converging Cameras
Example: Motion Parallel to Image Plane
Example: Motion Perpendicular to Image Plane
Example: Motion Perpendicular to Image Plane
Example: Motion Perpendicular to Image Plane
• Epipoles have same
coordinates in both
• Points move along lines
radiating from e: “focus
of expansion”
Epipolar Constraint
• If we observe a point x in one image, where can the corresponding point x’
be in the other image?
Epipolar Constraint
• Potential matches for x have to lie on the corresponding epipolar line 𝑙′
• Potential matches for x’ have to lie on the corresponding epipolar line 𝑙
Epipolar Constraint Example
Epipolar Constraint: Calibrated Case
• Assume that the intrinsic and extrinsic parameters of the cameras are known
• We can multiply the projection matrix of each camera (and the image points)
by the inverse of the calibration matrix to get normalised image coordinates.
• We can also set the global coordinate system to the coordinate system of the
first camera. Then the projection matrix of the first camera is [I | 0].
Epipolar Constraint: Calibrated Case
• The vectors x, t, and Rx’ are coplanar
Epipolar Constraint: Calibrated Case
• The vectors x, t, and Rx’ are coplanar
0)]([ = xRtx RtExEx
Essential Matrix
(Longuet-Higgins, 1981)
• Ex’ is the epipolar line associated with x’ (𝑙 = Ex’)
• ETx is the epipolar line associated with x (𝑙′ = ETx)
• Ee’ = 0 and ETe = 0
• E is singular (rank two), and E has five degrees of freedom
Epipolar Constraint: Calibrated Case
0)]([ = xRtx RtExEx
Epipolar Constraint: Uncalibrated Case
• The calibration matrices K and K’ of the two cameras are unknown
• We can write the epipolar constraint in terms of unknown normalized
coordinates
xKxxKx == ˆ,ˆ
• Fx’ is the epipolar line associated with x’ (𝑙 = Fx’)
• FTx is the epipolar line associated with x (𝑙′ = FTx)
• Fe’ = 0 and FTe = 0
• F is singular (rank two), and F has seven degrees of freedom
Epipolar Constraint: Uncalibrated Case
−− == KEKFxFx
The Eight-point Algorithm
x = (u, v, 1)T, x’ = (u’, v’, 1)T
under the constraint
The Eight-point Algorithm
• Meaning of error
Sum of Euclidean distances between points xi and epipolar line Fxi’ (or
points xi’ and epipolar line F
Txi) multiplied by a scale factor
• Non-linear approach: minimize
Problem with Eight-point Algorithm
Problem with Eight-point Algorithm
• Poor numerical conditioning
• Can be fixed by rescaling the data
The Normalized Eight-point Algorithm
• Centre the image data at the origin, and scale it so that the mean squared
distance between the origin and the data points is 2 pixels
• Use the eight-point algorithm to compute F from the normalized points
• Enforce the rank 2 constraint (for example, take SVD of F and throw out the
smallest singular value)
• Transform fundamental matrix back to the original units: if T and T’ are the
normalized transformations in the two images, then the fundamental matrix
in the original coordinates is TTFT’
(Hartley, 1995)
Comparison of Estimation Algorithms
Av. Dist. 1 Av. Dist. 2
8-point 2.33 pixels 2.18 pixels
Normalized
0.92 pixel 0.85 pixel
least squares
0.86 pixel 0.80 pixel
From Epipolar Geometry to Camera Calibration
• Estimating the fundamental matrix is known as “weak” calibration
• If we know the calibration matrices of the two cameras, we can estimate the
essential matrix: E = KTFK’
• The essential matrix gives us the relative rotation and translation between
the cameras, or their extrinsic parameters
• What is the problem of stereo vision?
• What is baseline? What are epipole, epipolar line, and epipolar plane? How
to determine epipolar lines?
• What is essential matrix? What is fundamental matrix?
• Describe eight-point algorithm
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com