程序代写 EBU7240 Computer Vision

EBU7240 Computer Vision
Changjae Oh
Calibration
Semester 1, 2021

Objectives
• Understanding the concept of camera calibration
• Understanding the relationship between image coordinate, camera coordinate, and world coordinate
• Understanding a linear method for camera calibration

Our goal: Recovery of 3D structure
J. Vermeer, Music Lesson, 1662
A. Criminisi, M. Kemp, and A. Zisserman,Bringing Pictorial Space to Life: computer techniques for the analysis of painti ngs, Proc. Computers and the History of Art, 2002

Things aren’t always as they appear…
http://en.wikipedia.org/wiki/Ames_room

view ambiguity

view ambiguity
shadow sculptures

Our goal: Recovery of 3D structure
• When certain assumptions hold, we ca •In general, we need multi-view geometry n recover structure from a single view
• But first, we need to understand the geometry of a single camera…
Image source

Camera calibration • Camera calibration:
̶ figuring out transformation from world coordinate system to image coordinate system world coordinate system
• Normalized (camera) coordinate system: camera center is at the origin, the principal axis is the z-axis;
x and y axes of the image plane are parallel to x and y axes of the world

Review: Pinhole camera model
(X,Y,Z)(f X /Z, fY /Z) X X
  fX f Y  fY =
0  f 0Y 
Z Z 1   
1  

Principal point
• Principal point (𝒑): point where principal axis intersects the image plane
• Normalized coordinate system: origin of the image is at the principal point
• Image coordinate system: origin is in the corner

Principal point offset
We want the principal point to map to (px, py) instead of (0,0)
(X,Y,Z)  ( f X / Z + px , f Y / Z + py ) X X
  fX+Zp f p 0  Y x x Y
Z fY+Zpy = f py 0Z  Z  10
1  1  

Principal point offset
principal point:
( px , py ) X
px 0 Y  py 0 Z 

Principal point offset
fX+Zp f  x 
p 0  x Y py 0Z
 fY+Zpy =
1 0   1 
principal point: ( px , py ) X X
calibration matrix
projection matrix   [I | 0]
K P=KI|0

Pixel coordinates
Pixel size: 1  1 mx my
• mx pixels per meter in horizontal direction, my pixels per meter in vertical direction
 x  x  x x
K =  my  f py  =  y y   1 1 1

Camera rotation and translation
Conversion from world to camera coordinate system (in non-homogeneous coordinates):
camera coordinate system
• In general, the camera coordin ate frame will be related to th e world coordinate frame by a rotation and a translation
world coordinate system
= R ( X~ − C~ ) coords. of a point
in world frame
coords. of point in camera frame
coords. of camera center in world frame

Camera rotation and translation
camera coordinate system world coordinate system
= R ( X~ − C~ )
3D transformation matrix (4 x 4)

Camera rotation and translation
camera coordinate system world coordinate system
2D transformatio 3D transformation n matrix (3 x 3) perspective project matrix (4 x 4)
ion matrix (3 x 4)

Camera rotation and translation
camera coordinate system world coordinate system

Camera rotation and translation
camera coordinate system world coordinate system
t = − R C~

Camera parameters • Intrinsic parameters
P = KR t
Principal point coordinates
Focal length
Pixel magnification factors
Skew (non-rectangular pixels), Radial distortion
px x x
K =  my  f
 1 1 1    

Camera parameters P = KR t • Intrinsic parameters
Principal point coordinates
Focal length
Pixel magnification factors
Skew (non-rectangular pixels), Radial distortion
coords. of camera center in world frame
• Extrinsic parameters
to world coordinate system
P=KR −RC
Rotation and translation relative ~
What is the projection of the camera center?
PC=KR −RC =0  1 
The camera center is the null space of the projection matrix!

Camera calibration
x****  
  y=****
     ****

Camera calibration
• Given n points with known 3D coordinates Xi and known image projections xi, estimate the camera parameters

Camera calibration: Linear method
• 𝑷 has 11 degrees of freedom
• One 2D/3D correspondence gives us two linearly independent equations
6 correspondences needed for a minimal solution
x * * * *X
 Y y=****
 * * * *   1

Camera calibration: Linear method
• 𝑷 has 11 degrees of freedom
• One 2D/3D correspondence gives us two linearly independent equations ̶ 6 correspondences needed for a minimal solution

Recall: Week1 quiz

Camera calibration: Linear method
• Directly estimate 11 unknowns in the P matrix using known 3D points
(𝑋, 𝑌, 𝑍) and measured (𝑋 , 𝑌 , 𝑍 ) and measured feature positions (𝑢 , 𝑣 ) 𝑖𝑖𝑖 𝑖𝑖

Camera calibration: Linear method
• Solve for Projection Matrix 𝑷 using least-square techniques

Camera calibration: linear vs. nonlinear
• Linear calibration is easy to formulate and solve, but it doesn’t directly tell us the camera parameters
Write down objective function in terms of intrinsic and extrinsic parameters
Define error as sum of squared distances between measured 2D points and estimated projections of 3D points
Minimize error using Newton’s method or other non-linear optimization
Can model radial distortion and impose constraints such as known focal length and or thogonality
x * * * *X
 Y y=****
x=KR tX • In practice, non-linear methods are preferred
 Zvs.  * * * * 

Application?
• Calibration is fundamental task for various computer vision tasks
https://ch.mathworks.com/help/vision/ug/single-camera-calibrator-app.html

Application?
• Calibration is fundamental task for various computer vision tasks
Xiang et al. “PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes.” RSS, 2018

EBU7240 Computer Vision
view Modeling
Semester 1
Changjae Oh

Objectives
• To understand calibration from vanishing points
• To understand measuring height without ruler

Application: Single
view modelling
A. Criminisi, I. Reid, and A. Zisserman, Sin gle View Metrology, IJCV 2000

Camera calibration revisited
• What if world coordinates of reference 3D points are not known?
• We can use scene features such as vanishing points

Camera calibration revisited
• What if world coordinates of reference 3D points are not known?
• We can use scene features such as vanishing points
Vertical vanishing point
(at infinity)
Vanishing line
Vanishing point
Vanishing point
Slide from Efros, Photo from Criminisi

Recall: Vanishing points
image plane
camera center
vanishing point v
line in the scene
• All lines having the same direction share the same vanishing point

Computing vanishing points
X∞ is a point at infinity, v is its projection: v = PX∞ The vanishing point depends only on line direction All lines having direction d intersect at X∞
x0 +td1 x0 /t+d1
X =y0 +td2 =y0 /t+d2
t z0 +td3 z0 /t+d3
 1   1/t  
X =d2  d3 0

Calibration from vanishing points
• Consider a scene with three orthogonal vanishing directions:
• Note: v1, v2 are finite vanishing points and v3 is an infinite vanishing point

Calibration from vanishing points
• Consider a scene with three orthogonal vanishing directions:
• We can align the world coordinate system with these directions

Calibration from vanishing points
P =   *   = p 1 p 2 p 3 p 4 
• p1 = P(1,0,0,0)T – the vanishing point in the x direction
• Similarly, p2 and p3 are the vanishing points in the y and z directions
• p4 = P(0,0,0,1)T – projection of the origin of the world coordinate system
• Problem: we can only know the four columns up to independent scale factors, addition al constraints needed to solve for them

Calibration from vanishing points
• Let us align the world coordinate system with three orthogonal vanishing directions in the scene:
1 e1 = 0,
0 e2 = 1,
0 e3 = 0
0 

Calibration from vanishing points
• Let us align the world coordinate system with three orthogonal vanishing directions in the scene:
1 e1 = 0,
0 e2 = 1,
 ei=iRTK−1vi, eTiej=0
• Orthogonality constraint:

Calibration from vanishing points
• Let us align the world coordinate system with three orthogonal vanishing directions in the scene:
1 e1 = 0,
0 e2 = 1,
• Orthogonality constraint:
 ei=iRTK−1vi, eTiej=0
• Rotation disappears, each pair of vanishing points gives constraint on focal length and principal point

Calibration from vanishing points
Cannot recover focal length, principal Can solve for focal length, principal point point is the third vanishing point

Rotation from vanishing points
• Constraints on vanishing points:
• After solving for the calibration matrix:
• Get λi by using the constraint ||ri||2 = 1.

Calibration from vanishing points: Summary
• Solve for K (focal length, principal point) using three orthogonal vanishing points
• Get rotation directly from vanishing points once calibration matrix is known
• Advantages
̶ No need for calibration chart, 2D-3D correspondences ̶ Could be completely automatic
• Disadvantages
Only applies to certain kinds of scenes Inaccuracies in computation of vanishing points Problems due to infinite vanishing points

Making measurements from a single image
http://en.wikipedia.org/wiki/Ames_room

Recall: Measuring height
Camera height

Measuring height without a ruler
ground plane
Compute Z from image measurements
• Need more than vanishing points to do this

Projective invariant
• We need to use a projective invariant: a quantity that does not change u nder projective transformations (including perspective projection)
̶ What are some invariants for similarity, affine transformations?

Projective invariant
• We need to use a projective invariant: a quantity that does not change u nder projective transformations (including perspective projection)
The cross-ratio of four points:
P−PP−P 3142
P−PP−P 3241

Measuring height
T−B −R R−B −T
T (top of object)
R (referencepoint)
= HR scene cross ratio
t−bvZ−r =H r−b vZ −t R
image cross ratio
B (bottom of object) ground plane

Measuring height without a ruler

vanishing line (horizon)
= r−bvZ−t R
image cross ratio

A. Criminisi, I. Reid, and A. Zisserman, Single View Metrology, IJCV 2000 Figure from UPenn CIS580 slides

Measurements on planes
Approach: unwarp then measure What kind of warp is this?

Image rectification
• To unwarp (rectify) an image
̶ solve for homography H given p and p′
̶ how many points are necessary to solve for H?

Image rectification: example
Piero della Francesca, Flagellation, ca. 1455

Application: 3D modeling from a single image
A. Criminisi, M. Kemp, and A. Zisserman,Bringing Pictorial Space to Life: computer techniques for the analysis of paintings, Proc. Computers and the History of Art, 2002

Application: 3D modeling from a single image
J. Vermeer, Music Lesson, 1662
A. Criminisi, M. Kemp, and A. Zisserman,Bringing Pictorial Space to Life: computer techniques for the analysis of paintings, Proc. Computers and the History of Art, 2002

Application: Fully automatic
sky vertical
D. Hoiem, A.A. Efros, and M. Hebert, Automatic Photo Pop-up, SIGGRAPH 2005. http://dhoiem.cs.illinois.edu/projects/popup/popup_movie_450_250.mp4

Application: Object detection
D. Hoiem, A.A. Efros, and M. Hebert, Putting Objects in Perspective, CVPR 2006

Next Topic
• How about using two cameras? ̶ Prerequisite
• Review Part2-3: Calibration (this content!) • Review Part1-3: Bilateral filtering

程序代写 CS代考加微信: powcoder QQ: 1823890830 Email: powcoder@163.com

Related Posts