程序代写代做代考 C html graph Fundamentals of Computer Vision

Fundamentals of Computer Vision
Lecture

• Camera matrix. • Perspective.
• Other camera models.
• Pose estimation.
Overview of today’s lecture

Slide credits
Most of these slides were adapted from:
• Kris Kitani (15-463, Fall 2016), Ioannis Gkioulekas (16-385, Spring 2019), Robert Colin (454, Fall 2019s).
Some slides were inspired or taken from: • Fredo Durand (MIT).

The (rearranged) pinhole camera
image plane
camera center
principal axis
• Principal axis: line from the camera center perpendicular to the image plane
• Normalized (camera) coordinate system: camera center is at the origin and the principal axis is the z-axis

The 2D view of the (rearranged) pinhole camera
image plane

The pinhole camera matrix
Relationship from similar triangles: General camera model:
𝑓𝑋 𝑓𝑌 𝑍
What does the pinhole camera projection look like?

World-to-camera coordinate system transformation
𝒙 = 𝑷 ⋅ 𝐗& 𝐜
𝑪&
Translate by – 𝑪& (align origins)
𝐗& 𝐜 = 𝑹 ⋅ 𝑿& 𝒘 − 𝑪&
translate
𝑿& 𝒄
points now coincide
Translate by – R (align points)
𝑿&𝒘
Camera coordinate system
Coordinate of the camera center in the world coordinate frame
World coordinate system
rotate

Modeling the coordinate system transformation
In heterogeneous coordinates, we have:
𝐗& 𝐜 = 𝐑 ⋅ 𝐗& 𝐰 − 𝐂5
In homogeneous coordinates, we have:
𝐗&𝐜 X&7 = R −RC5 X&< 1011 𝐗&𝐰 3x3 3x1 or 𝑿 𝒄 = R − R C5 𝑿 𝒘 01 1x3 Incorporating the transform in the camera matrix The previous camera matrix is for homogeneous 3D coordinates in camera coordinate system: We also just derived: 𝒙=𝐏𝐗𝐜 =𝐊 𝐈|𝟎 𝐗𝐜 𝐗 𝐜 = 𝐑 − 𝐑 𝐂5 𝐗 𝐰 𝟎1 We can write everything into a single projection: 𝒙 = 𝐊 𝐈 | 𝟎 𝐑 − 𝐑 𝐂5 𝐗 𝐰 𝟎1 𝒙= 𝐏 𝐗𝐰 new P Putting it all together We can write everything into a single projection: 𝒙 = 𝐊 𝐈 | 𝟎 𝐑 − 𝐑 𝐂5 𝐗 𝐰 𝟎1 𝒙= 𝐏 𝐗𝐰 The camera matrix now looks like: intrinsic parameters (3 x 3): correspond to camera internals correspond to camera externals (sensor not at f = 1 and origin shift) extrinsic parameters (3 x 4): (world-to-image transformation) General pinhole camera matrix We can decompose the camera matrix like this: 3x3 identity K 3x3 intrinsics 3x1 3x3 3D translation 3D rotation General pinhole camera matrix Old notation for ‘P’ 3x3 3D rotation projection 𝐏= projection rotation translation 3x1 3D translation Extrinsics • How do we get the camera to “canonical form”? • (Center of projection at the origin, x-axis points right, y-axis points up, z-axis points backwards) Step 1: Translate by -c 0 Extrinsics • How do we get the camera to “canonical form”? • (Center of projection at the origin, x-axis points right, y-axis points up, z-axis points backwards) Step 1: Translate by -c How do we represent translation as a matrix multiplication? 0 Extrinsics • How do we get the camera to “canonical form”? • (Center of projection at the origin, x-axis points right, y-axis points up, z-axis points backwards) Step 1: Translate by -c Step 2: Rotate by R 0 General pinhole camera matrix Another way to write the mapping: where 𝐏=𝐊 intrinsic parameters extrinsic parameters 3D rotation 3D translation What is the size and meaning of each term in the camera matrix? Recap 3x3 3x3 3x3 3x1 intrinsics 3D rotation identity 3D translation Question The camera matrix relates what two quantities? Question The camera matrix relates what two quantities? homogeneous 3D points to 2D image points Question The camera matrix relates what two quantities? homogeneous 3D points to 2D image points The camera matrix can be decomposed into? Question The camera matrix relates what two quantities? homogeneous 3D points to 2D image points The camera matrix can be decomposed into? intrinsic and extrinsic parameters More general camera matrices The following is the standard camera matrix we saw. What if pixels are not square? More general camera matrices CCD camera: pixels may not be square. How many degrees of freedom? More general camera matrices CCD camera: pixels may not be square. How many degrees of freedom? 10 DOF What if sensors are skewed? More general camera matrices Finite projective camera: sensor be skewed. How many degrees of freedom? More general camera matrices Finite projective camera: sensor be skewed. How many degrees of freedom? 11 DOF Finite projective camera What does this matrix look like if the camera and world have the same coordinate system? Finite projective camera The pinhole camera and all of the more general cameras we have seen so far have “perspective distortion”. Perspective projection from (homogeneous) 3D to 2D coordinates The (rearranged) pinhole camera image plane camera center Perspective projection in 3D principal axis The 2D view of the (rearranged) pinhole camera image plane Perspective distortion: magnification changes with depth Perspective projection in 2D Forced perspective The Ames room illusion Projection Projection Müller-Lyer Illusion http://www.michaelbach.de/ot/sze_muelue/index.html The arrow illusion Magnification depends on depth What happens as we change the focal length? real-world object depth Z depth 2 Z Magnification depends on focal length real-world object focal length f focal length 2 f What if... real-world object 1. Set focal length to half depth 2 Z focal length f focal length 2 f What if... real-world object Is this the same image as the one I had at focal length 2f and distance 2Z? 1. 2. Set focal length to half Set depth to half depth Z focal length f Perspective distortion long focal length mid focal length short focal length Vertigo effect Named after Alfred Hitchcock’s movie, also known as “dolly zoom” How would you create this effect? Other camera models real-world object ... we continue increasing Z and f while maintaining same magnification? 𝑓 → ∞ a n d 𝑍𝑓 = c o n s t a n t What if... depth Z focal length f camera is close to object and has small focal length camera is far from object and has large focal length Different cameras perspective camera weak perspective camera Weak perspective vs perspective camera image plane • magnification does not change with depth • constant magnification depending on f and Zo 𝑍L magnification changes with depth Object dimensions are small compared to the distance to camera Comparing camera matrices Let’s assume that the world and camera coordinate systems are the same. • The perspective camera matrix can be written as: • What would the matrix of the weak perspective camera look like? Comparing camera matrices Let’s assume that the world and camera coordinate systems are the same. • The perspective camera matrix can be written as: • The weak perspective camera matrix can be written as: Comparing camera matrices Let’s assume that the world and camera coordinate systems are the same. • The finite projective camera matrix can be written as: where we now have the more general intrinsic matrix • The affine camera matrix can be written as: In both cameras, we can incorporate extrinsic parameters same as we did before. When can we assume a weak perspective camera? When can we assume a weak perspective camera? 1. When the scene (or parts of it) is very far away. Weak perspective projection applies to the mountains. When can we assume a weak perspective camera? 2. When we use a telecentric lens. Place a pinhole at focal length, so that only rays parallel to primary ray pass through. object distance D focal length f sensor distance D’ Orthographic camera Also called “parallel projection” Special case of weak perspective camera where: • Constant magnification is equal to 1. • There is no shift between camera and image origins. • The world and camera coordinate systems are the same. image world Distance from the COP to the PP is infinite What is the camera matrix in this case? Orthographic camera Also called “parallel projection” Special case of weak perspective camera where: • constant magnification is equal to 1. • there is no shift between camera and image origins. • the world and camera coordinate systems are the same. image world Orthographic projection using a telecentric lens How do we make the telecentric lens act as an orthographic camera? We set the sensor distance as: 𝐷N = 2𝑓 in order to achieve unit magnification. object distance D focal length f sensor distance D’ Many other types of cameras Geometric camera calibration Pose Estimation Given a single image, estimate the exact position of the photographer Geometric camera calibration Given a set of matched points point in 3D point in the space image and camera model Where do we get these matched points from? projection model parameters Camera matrix Find the (pose) estimate of We’ll use a perspective camera model for pose estimation Calibration using a reference object Given n points with known 3D coordinates Xi and known image projections xi, estimate the camera parameters Place a known object in the scene: • identify correspondences between image and scene • compute mapping from scene to image P? Xi xi Issues: • must know geometry very accurately • must know 3D->2D correspondence

Basic reading:
• Szeliski textbook, Section 2.1.5, 6.2.
References
Additional reading:
• Hartley and Zisserman, “Multiple View Geometry in Computer Vision,” Cambridge University
Press 2004.
chapter 6 of this book has a very thorough treatment of camera models.
• Torralba and Freeman, “Accidental Pinhole and Pinspeck Cameras,” CVPR 2012. the eponymous paper discussed in the slides.