Field of view
FOV
Common FOV
O
One camera can only take an image of those locations within its field of view (FOV)
OL
OR
Computer Vision / Mid-Level Vision / Stereo and Depth
31
A stereo pair of cameras can find depth for those locations within the common FOV of the two cameras
Stereo: coplanar cameras
cFOV
cFOV
OLOR OLOR
Short baseline
• large common FOV • large depth error (changes in depth cause only small changes in disparity)
Long baseline
• small common FOV • small depth error
z= f Computer Vision / Mid-Level Vision / Stereo and Depth
B
d (changes in depth
cause larger changes in disparity)
32
Stereo: non-coplanar cameras
FOV
OL OR OL OR
Intersecting optical axes • large common FOV
• small depth error
Convergence angle θ (“vergence”)
Computer Vision / Mid-Level Vision / Stereo and Depth
33
Fixation point θ
Stereo: non-coplanar cameras
Disparity measured using angles instead of distance
Disparity = αL- αR
Disparity = zero at fixation point
Disparity = zero at all points on a curve where rays at equal angles intersect.
Curve with zero-disparity called the “horopter”
Location of horopter depends on vergence angle.
Fixation point
θ
Horopter
Computer Vision / Mid-Level Vision / Stereo and Depth
34
OL
OR
αL
αR
Stereo: non-coplanar cameras
Magnitude of disparity increases with the distance of objects from the horopter
(αL- αR) > 0 : outside of the horopter (αL- αR) < 0 : inside the horopter
Need to be consistent with signs of angles, and order of subtraction.
Fixation point
θ
Horopter
Computer Vision / Mid-Level Vision / Stereo and Depth
35
OL
OR
αL
αR
Stereo: non-coplanar eyes Humans employ a non-coplanar imaging system.
When point F is fixated:
• The images of F fall on the foveas of each retina.
D
• The images of points on the
horopter (e.g. H) fall an equal
distance from the foveas of each
retina. N
• The images of points nearer than the horopter (e.g. N) are displaced outwards (“crossed” disparties)
• The images of points more distant than the horopter (e.g. D) are displaced inwards (“uncrossed disparities)
• The further the point from the H
horopter, the greater the N left displacement. F D eye
Computer Vision / Mid-Level Vision / Stereo and Depth
F
H
right eye
H
D
FN 36
Horopter
Stereo: non-coplanar eyes
Some cortical neurons are tuned to retinal disparity, and hence, can signal the depth of image points [see lecture 4].
left eye
right eye
disparity tuned neurons
2 0 -2
Computer Vision / Mid-Level Vision / Stereo and Depth
37
Epipolar geometry
For coplanar cameras, corresponding points in the two images are on corresponding rows of each image.
pl
Ol
pr
P
Computer Vision / Mid-Level Vision / Stereo and Depth
38
Or
Epipolar Lines
Epipolar geometry
For non-coplanar cameras, corresponding points still occur on lines, but these are no longer horizontal lines.
A point in the image plane of the left camera corresponds to a line in the image plane of the right camera (and vice versa).
P
pl Epipolar Lines pr
Ol el er Or
These lines which correspond to points in the other image are called epipolar lines.
2D search for correspondence can be reduced to a 1D search along the “epipolar” line.
Computer Vision / Mid-Level Vision / Stereo and Depth 39
Epipolar geometry: terminology
Baseline: the line through the camera projection centres Ol, Or
Epipole: projection of the optic centre of one camera on the image plane of the other camera.
The right (left) epipole er
(el ) is the image of the left
(right) camera projection centre Ol (Or) in the right
(left) image plane.
Equivalently, the epipoles are the intersection of the baseline with each image plane.
P
Computer Vision / Mid-Level Vision / Stereo and Depth
40
pl
Epipoles
pr
Ol el er Or
Baseline
Epipolar geometry: terminology
Epipolar plane: a plane going through a particular 3D point and the optic centres of both cameras.
Epipolar lines: intersection of the epipolar plane (for a particular 3D point) and each image plane.
All epipolar lines in the left image go through el and all
epipolar lines in the right e image go through er. Ol
P
pl Epipolar Lines pr
Conjugated epipolar lines:
the epipolar lines generated by the same 3D point on the left and right image planes.
Epipolar plane
l
e
r Or
Epipolar Constraint: Corresponding points must lie on conjugated epipolar lines.
Computer Vision / Mid-Level Vision / Stereo and Depth 41
Rectification
Epipolar lines can be made parallel to the rows of the image via a transform called rectification.
Left Right
Epipolar Lines
Rectification warps both images so that all epipolar lines are horizontal.
Computer Vision / Mid-Level Vision / Stereo and Depth 43
Stereo geometry: summary
simple case P general case P
yp yp pryr lzl xll z
pr yr zr
l lzlr
xlO xrO lr
T
coplanar image planes (cameras translated along x-axis).
epipolar lines are horizontal scan lines
disparity measured in pixels
disparity inversely proportional to depth
Computer Vision / Mid-Level Vision / Stereo and Depth
Ol
xr Or
T+R
noncoplanar image planes (cameras related by a translation and rotation)
epipolar lines are lines radiating from the epipolar point
disparity measured using angles
disparity increases with distance from horopter 44
Stereo geometry: summary
simple case P general case P
yp yp pryr lzl xll z
l lzlr
xlO xrO lr
T
Ol
xr Or
1. triangulation (i.e. determining intersection of corresponding viewlines)
2. rectification of images and then treating as simple case
Both require knowledge of intrinsic and extrinsic camera parameters. 45
depth = fB/d
depth found by, either:
Computer Vision / Mid-Level Vision / Stereo and Depth
pr yr zr
T+R