2812ICT Perceptual Computing
Image Formation
Outline
• Pinhole camera
• Perspective effects
• Homogeneous coordinates
• Relating a real-world point to a point on the image
Image formation
• Let’s design a camera
– Idea 1: put a piece of film in front of an object – Do we get a reasonable image?
Slide by Steve Seitz
Pinhole camera
• Add a barrier to block off most of the rays – This reduces blurring
– The opening is known as the aperture – How does this transform the image?
Slide by Steve Seitz
Pinhole camera
• Pinhole camera is a simple model to approximate imaging process, perspective projection.
Virtual image
pinhole
If we treat pinhole as a point, only one ray from any given point can enter the camera.
Fig from Forsyth and Ponce
Image plane
Pinhole size / aperture
How does the size of the aperture affect the image we’d get?
Larger
Smaller
Slide credit: K. Grauman
Adding a lens
focal point
f
– Rays passing through the center are not deviated
– All parallel rays converge to one point on a plane located at the focal length f
• A lens focuses light onto the film
Slide credit: Steve Seitz
Pinhole vs. lens
Slide credit: K. Grauman
Cameras with lenses
F
focal point
optical center (Center Of Projection)
• A lens focuses parallel rays onto a single focal point
• Gather more light, while keeping focus; make pinhole perspective projection practical
Slide credit: K. Grauman
Perspective effects
Perspective effects
Projection can be tricky…
Slide source: Seitz
Projective Geometry
What is lost? • Length
• Angles
What is preserved?
• Straight lines are still straight
Slide credit: J. Hayes
Distant objects are smaller: length is not preserved
Computer Vision – A Modern Approach Slides by D.A. Forsyth
Vanishing points
• Each set of parallel lines meets at a different point
• The vanishing point for this direction
• Coplanar sets of parallel lines have a horizon
• The vanishing points lie on a line • Good way to spot faked images
Perspective projection
• 3d world mapped to 2d projection in image plane Image
plane
Focal length
Camera frame
Scene / world points
Optical axis
𝑋,𝑌,𝑍 → 𝑓𝑋,𝑓𝑌 𝑍𝑍
Scene point P
Image coordinates p
We have, by similar triangles, that (X, Y, Z) -> (f X/Z, f Y/Z, -f ):
Forsyth and Ponce
Pinhole Camera
• Common to draw image plane in front of the focal point.
• Moving the image plane merely scales the image.
𝑥 = 𝑓 𝑋 𝑍
𝑦=𝑓𝑌 𝑍
Slide credit: Fei-Fei Li
Relating real-world point to a point on an image
• P = (X, Y, Z)→ p = (f X/Z, f Y/Z) not a linear transformation • Homogeneouscoordinates
• Convertingfromhomogeneouscoordinates
Homogenous coordinates
• Add an extra coordinate and use an equivalence relation • For 2D
• equivalence relation
k*(X,Y,W) is the same as (X,Y,W)
• For 3D
• equivalence relation
k*(X,Y,Z,W) is the same as (X,Y,Z,W) • Basic notion
• Possible to represent points “at infinity”
• Where parallel lines intersect
• Where parallel planes intersect
• Possible to write the action of a perspective camera as a matrix
Computer Vision – A Modern Approach Slides by D.A. Forsyth
Relating a real-world point to a point on an image
In Cartesian coordinates:
In homogeneous coordinates: Projection is a matrix multiplication
1 0 0 0x x
0 1 0 0y= y (f’x,f’y)
0 0 1/f’ 0z z/f’ z z
1 divide by the third coordinate to convert
back to non-homogeneous coordinates
Why projection matter?
Object Recognition (CVPR 2006)
Slide credit: J. Hayes
Why projection matter?
Inserting photographed objects into images (SIGGRAPH 2007))
Slide credit: J. Hayes
Perspective projection & calibration
• Perspective equations so far in terms of camera’s reference frame….
• Camera’s intrinsic and extrinsic parameters needed to calibrate geometry.
Camera frame
Slide credit: K. Grauman
Perspective projection & calibration
World frame
Extrinsic:
Camera frame→World frame
Intrinsic:
Image coordinates relative to camera →Pixel coordinates
Camera frame
2D point (3×1)
=
Camera to pixel coord. trans. matrix
(3×3)
Perspective projection matrix (3×4)
World to camera coord. trans. matrix
(4×4)
3D point (4×1)
Slide credit: K. Grauman
Intrinsic parameters: from idealized world coordinates to pixel values
Forsyth&Ponce
Perspective projection
u=f v=f
x
z y
z
Slide credit: W. Freeman
Relating a real-world point to a point on the image
Slide credit: Fei-Fei Li
Relating a real-world point to a point on the image
Slide credit: Fei-Fei Li
Relating a real-world point to a point on the image
Slide credit: Fei-Fei Li
Relating a real-world point to a point on the image
Slide credit: Fei-Fei Li
Skew image plane
Relating a real-world point to a point on the image
Relating a real-world point to a point on the image
Note:For90,welets=-cot and1/sin1
Slide credit: Fei-Fei Li
Relating a real-world point to a point on the image
Slide credit: Fei-Fei Li
Relating a real-world point to a point on the image
Slide credit: Fei-Fei Li
Relating a real-world point to a point on the image
Slide credit: Fei-Fei Li
Relating a real-world point to a point on the image
Slide credit: Fei-Fei Li
Slide credit: Fei-Fei Li
Camera Calibration: estimate all intrinsic and extrinsic parameters
Slide credit: Fei-Fei Li
5
6
Orthographic Projection
Slide credit: S. Seitz
Scaled Orthographic Projection
Slide credit: S. Seitz
Field of View (Zoom)
Slide credit: Fei-Fei Li
Next week:
• Image processing and representation