CS代考 CMT107 Visual Computing

CMT107 Visual Computing

Stereo Vision

Copyright By PowCoder代写 加微信 powcoder

School of Computer Science and Informatics

Cardiff University

• Stereo Vision
• Multi-view geometry problems

• Triangulation

• Epipolar Geometry
• The epipolar constraint

• Essential matrix and fundamental matrix

• Eight-point algorithm

Acknowledgement
The majority of the slides in this section are from at University of Illinois at Urbana-Champaign

Stereo Vision

• Geometric problem formulation: given several images of the same object or
scene, compute a representation of its 3D shape.

Goal: Recovery of 3D Structure

• Geometric problem formulation: given several images of the same object or
scene, compute a representation of its 3D shape.

• “Images of the same object or scene”
• Arbitrary number of images (from two to thousands)

• Arbitrary camera positions (camera network or video sequence)

• Calibration may be initially unknown

• “Representation of 3D shape”
• Depth maps

• Point clouds

• Patch clouds

• Volumetric models

• Layered models

Goal: Recovery of 3D Structure

• Recovery of structure from one image is inherently ambiguous

Goal: Recovery of 3D Structure

• Recovery of structure from one image is inherently ambiguous

Goal: Recovery of 3D Structure

• Recovery of structure from one image is inherently ambiguous

Goal: Recovery of 3D Structure

• Recovery of structure from one image is inherently ambiguous

http://en.wikipedia.org/wiki/Ames_room

http://en.wikipedia.org/wiki/Ames_room

Goal: Recovery of 3D Structure

• We will need multi-view geometry

Multiview Geometry Problems

• Structure: Given projections of the same 3D point in two or more images,
compute the 3D coordinates of that point

Multiview Geometry Problems

• Motion: Given a set of corresponding points in two or more images, compute
the camera parameters

R1,t1 ? Camera 2

Triangulation

• Given projects of a 3D point in two or more images (with known camera
matrices), find the coordinates of the point

Triangulation

• We want to intersect the two visual rays corresponding to x1 and x2, but
because of noise and numerical errors, they don’t meet exactly

Triangulation: Geometric Approach

• Find shortest segment connecting the two viewing rays and let X be the
midpoint of that segment.

Triangulation: Linear Approach

Cross product as matrix multiplication:

Two independent equations each in terms of
three unknown entries of X

Triangulation: Non-Linear Approach

• Find X that minimizes

XPxdXPxd +

Two-view Geometry

Epipolar Geometry

• Baseline – line connecting the two camera centres

• Epipolar Plane – plane containing baseline (1D family)

• Epipoles: intersections of baseline with image planes; projections of the
other camera centres

• Epipolar Lines: intersections of epipolar plane with image planes (always
come in corresponding pairs)

Example: Converging Cameras

Example: Motion Parallel to Image Plane

Example: Motion Perpendicular to Image Plane

Example: Motion Perpendicular to Image Plane

Example: Motion Perpendicular to Image Plane

• Epipoles have same
coordinates in both

• Points move along lines
radiating from e: “focus
of expansion”

Epipolar Constraint

• If we observe a point x in one image, where can the corresponding point x’
be in the other image?

Epipolar Constraint

• Potential matches for x have to lie on the corresponding epipolar line 𝑙′

• Potential matches for x’ have to lie on the corresponding epipolar line 𝑙

Epipolar Constraint Example

Epipolar Constraint: Calibrated Case

• Assume that the intrinsic and extrinsic parameters of the cameras are known

• We can multiply the projection matrix of each camera (and the image points)
by the inverse of the calibration matrix to get normalised image coordinates.

• We can also set the global coordinate system to the coordinate system of the
first camera. Then the projection matrix of the first camera is [I | 0].

Epipolar Constraint: Calibrated Case

• The vectors x, t, and Rx’ are coplanar

Epipolar Constraint: Calibrated Case

• The vectors x, t, and Rx’ are coplanar

0)]([ = xRtx RtExEx

Essential Matrix
(Longuet-Higgins, 1981)

• Ex’ is the epipolar line associated with x’ (𝑙 = Ex’)

• ETx is the epipolar line associated with x (𝑙′ = ETx)

• Ee’ = 0 and ETe = 0

• E is singular (rank two), and E has five degrees of freedom

Epipolar Constraint: Calibrated Case

0)]([ = xRtx RtExEx

Epipolar Constraint: Uncalibrated Case

• The calibration matrices K and K’ of the two cameras are unknown

• We can write the epipolar constraint in terms of unknown normalized
coordinates

xKxxKx == ˆ,ˆ

• Fx’ is the epipolar line associated with x’ (𝑙 = Fx’)

• FTx is the epipolar line associated with x (𝑙′ = FTx)

• Fe’ = 0 and FTe = 0

• F is singular (rank two), and F has seven degrees of freedom

Epipolar Constraint: Uncalibrated Case

−− == KEKFxFx

The Eight-point Algorithm

x = (u, v, 1)T, x’ = (u’, v’, 1)T

under the constraint

The Eight-point Algorithm

• Meaning of error

Sum of Euclidean distances between points xi and epipolar line Fxi’ (or
points xi’ and epipolar line F

Txi) multiplied by a scale factor

• Non-linear approach: minimize

Problem with Eight-point Algorithm

Problem with Eight-point Algorithm

• Poor numerical conditioning

• Can be fixed by rescaling the data

The Normalized Eight-point Algorithm

• Centre the image data at the origin, and scale it so that the mean squared
distance between the origin and the data points is 2 pixels

• Use the eight-point algorithm to compute F from the normalized points

• Enforce the rank 2 constraint (for example, take SVD of F and throw out the
smallest singular value)

• Transform fundamental matrix back to the original units: if T and T’ are the
normalized transformations in the two images, then the fundamental matrix
in the original coordinates is TTFT’

(Hartley, 1995)

Comparison of Estimation Algorithms

Av. Dist. 1 Av. Dist. 2

8-point 2.33 pixels 2.18 pixels

Normalized

0.92 pixel 0.85 pixel

least squares

0.86 pixel 0.80 pixel

From Epipolar Geometry to Camera Calibration

• Estimating the fundamental matrix is known as “weak” calibration

• If we know the calibration matrices of the two cameras, we can estimate the
essential matrix: E = KTFK’

• The essential matrix gives us the relative rotation and translation between
the cameras, or their extrinsic parameters

• What is the problem of stereo vision?

• What is baseline? What are epipole, epipolar line, and epipolar plane? How
to determine epipolar lines?

• What is essential matrix? What is fundamental matrix?

• Describe eight-point algorithm

程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com