CS计算机代考程序代写 matlab [06-30213][06-30241][06-25024]

[06-30213][06-30241][06-25024]
Computer Vision and Imaging &
Robot Vision
Dr Hyung Jin Chang Dr Yixing Gao
h.j.chang@bham.ac.uk y.gao.8@bham.ac.uk
School of Computer Science

Camera Parameters
Hyung Jin Chang, Yixing Gao
08/02/2021

Camera parameters (INTRINSIC AND EXTRINSIC)
• Digital images are spatially discrete, and divided into (usually) rectangular picture elements, or pixels.
• This is an aspect of the image formation process that we have neglected so far, assuming instead that the image domain is spatially continuous.
• Likewise, the perspective equation derived in the previous section is valid only when all distances are measured in the camera’s reference frame, and when image coordinates have their origin at the image center where the axis of symmetry of the camera pierces its retina.
• In practice, the world and camera coordinate systems are related by a set of physical parameters, such as
– the focal length of the lens
– the size of the pixels
– the position of the image center
– the position and orientation of the camera.
Computer Vision – A Modern Approach Set: Introduction to Vision
Slides by D.A. Forsyth
Hyung Jin Chang, Yixing Gao
08/02/2021

Camera parameters
• Issue
– one unit in camera coordinates may not be the same as one unit in world coordinates
• intrinsic parameters – focal length, principal point, aspect ratio, angle between axes, etc.
– Camera’s coordinate system to a fixed world coordinate system and specify its position and orientation in space
• extrinsic parameters
æöæ öæ öæ ö
U Transformation 1
ç÷ç ÷ç ÷ç ÷Y
0 0
V = representing 0 1 0 0 representing ç ÷ ç÷ç ÷ç ÷ç ÷
çZ÷ ç÷ç ÷ç ÷ç ÷
èW ø
èintrinsic parametersøè0
0 1
ç÷ 0øèextrinsic parametersø
æXö 0 Transformation ç ÷
ç÷ èTø
Computer Vision – A Modern Approach Set: Introduction to Vision
Slides by D.A. Forsyth
Hyung Jin Chang, Yixing Gao
08/02/2021

Machine perception Camera geometry
Matej Kristan
Laboratorij za Umetne Vizualne Spoznavne Sisteme, Fakulteta za računalništvo in informatiko,
Univerza v Ljubljani

Single
• Points in a world 3D coordinate system (c.s.)
• Project to image plane into 2D pixels
Two kinds of projection:
1. “Extrinsic” projection
3D World→3D Camera
2. “Intrinsic” projection 3D Camera→2D Image
–
view geometry
Camera 3D c.s.
World 3D coordinate system

Consider
Recall image formation:
“Intrinsic” projection
first
c
Point in image
• Apointwrittenincamera3Dcoordinatesystem(meters)
• Projected to camera image plane (meters)
• Projected to discretized image (pixels)
• Let’s have a look at transformations for a pinhole camera!
Point in 3D

Homogeneous coordinates
•
•
Euclidean geometry uses Cartesian coordinate system
But for a projective geometry, homogenous coordinates are
much more appropriate
Cartesian Homogeneous Multiplying by a scalar (≠ 0) value
form form does not change a point!
From homogeneous system to Euclidian:
Simply divide by the last coordinate to make it 1.
•

Camera coordinate system
(meters)
• Principal axis:
A line from camera center perpendicular to image plane.
• Principal point (p):
A point where the principal axis punctures the image plane.
• Normalized (camera) coordinate system: A system with origin at the principal point.

A pinhole camera revisited
“Sideways” view
3D point in world c.s. 2D projection to image plane
(X,Y,Z)(f X /Z, fY /Z) Rewrite in homogeneous coordinates
• Projection as vector-matrix multiplication: fX f 0X
  Y fY= f 0
In 3D camera c.s.
 Z    
1 0Z 1 𝑷
x=PX In 2D image 0
plane c.s.
0
Images from Hartley & Zisserman

A pinhole camera revisited
•
Rewrite the projection matrix to separate the focal lengths:
fX f  f Y  = 
1
 1
X 0 
f
0 Y  1 0Z
x = P X 0
Z 
  1
1

“Sideways” view
P0 =diag( f,f,1) I|0 
Images from Hartley & Zisserman

From image plane to image pixels
• Change of coordinate system to image corner
• Normalized camera coordinate system: Origin in principal point
• Imagecoordinatesystem:
Origin in the corner of the image sensor.
p=[px,py]T
𝒑=𝒑,𝒑 𝑇. 𝑥𝑦

From image plane to image pixels (1/3)
• Change the c.s. origin by the principal point 𝒑.
• Write the transformation:
(X,Y,Z)  ( f X / Z + px , f Y / Z + py )
• Rewriteinvector-matrixmultiplication:
fX+Zp f 0 p 0X
 x x Y fY+Zp=0 f p 0   yx y Z
Z 0 0 1 0    1

From image plane to image pixels (2/3)
• Projection to a sensor of size
• Pixels are arranged into a rectangular
𝑀 × 𝑀 pixels matrix. 𝑥𝑦
• Let 𝑚 = 𝑀 /𝑊 and 𝑚 = 𝑀 /𝐻 . 𝑥𝑥𝑆𝑦𝑦𝑆
(in meters).
• Construct projection to pixels:
Just multiply by another matrix:  X  Abbreviated form:
0X   Y x 0 Y
x mx 0  y  =  0 my
0f 0
px 0  x  0 x
py 0  Z   y  =  0 
   
pixel/m m 1  1
0  0 f z0010010 y0 Z
𝑊W s ×x H𝐻 s 𝑆𝑆
 z 0 0 1 0 
y 0  

From image plane to image pixels (3/3)
• In general difficult to guarantee a rectangular sensor. Rectangular Skewed
Rectangular
Skewed
x0x0X x 0 Y
xsx0X x 0 Y
y=0y0  y 0 Z
y=0y0  y 0 Z
z0010  1
z0010  1
Projection matrix 𝑃P 0
Projection matrix 𝑃   P
00

Calibration matrix
• Expandtheprojectionmatrix x  s x1 0 0 0X
P0 x 0 Y
y=0  y0 1 0 0  y0Z
z 0 0 10 0 1 0    1
P0 =KI|0 • Calibrationmatrix𝑲:
“Prescribes projection of 3D point in camera c.s. into pixles!”
Q: What are the elements of a calibration matrix?
x s x0  K=0y
y0
001 
𝑷0

Single
• Points in a world 3D coordinate system (c.s.)
• Project to image plane into 2D pixels
–
view geometry
Two kinds of projection:
1. “Extrinsic” projection ?
3D World→3D Camera
2. “Intrinsic” projection 3D Camera→2D Image
World 3D coordinate system
Camera 3D c.s.

From world
• The 3D camera c.s. is related to 3D world c.s. by a
෩ rotation matrix 𝑹 and translation 𝒕 = 𝑪.
c.s
. to 3D camera
c.s
.
C
World c.s.
X~
cam
X
෤
… How to rotate the world c.s. about its own origin to align it with the camera c.s.
… Camera origin in world c.s. … Point in 3D world c.s.
Same point, but written in 3D camera c.s.
X … cam
World-to-camera c.s. transformation:
= R ( X~ – C~ ) (Euclidean)

Putting it all together
• Cameraisspecifiedbyacalibrationmatrix𝑲,the ෩
projection center in world c.s. 𝑪 and rotation matrix 𝑹. • A3Dpointinworldcoordinates(homogeneous)𝑿,
is projected into pixels 𝒙 by the following relation:
x = K I | 0X = K R |  X=PX cam  
t = − R C~
Note the structure of the projection matrix!
P=KR|t,
Q: What needs to be known to construct the projection matrix?
−RC
t

Lens adds a nasty nonlinearity
Straight lines are no longer straight!
Nonlinearity should be removed to apply a pinhole model!
radially distorted image radial distortion removed
http://vipbase.net/doc/img_trans_radial.htm

Lens adds a nasty nonlinearity
• Lensdistortionassumedradiallysymmetric • Radially expand an image to un-distort
distorted
undistorted
(,)→(,)
• In this transformation, only the radius of transformed
point changes, but the angle remains unchanged.

Summary: camera parameters
• Intrinsic parameters:
• Principal point coordinates
• Focal length
• Pixel scaling factor (rectangle)
• Shear (non-rectangular pixels)
• Radial distortion
• Extrinsic parameters
• Camera rotation R
• Camera translation t
• ==Cameraprojectionmatrix: P=K R|t =P P P P  21 22 23 24
P P P P  31 32 33 34 
sx x 0
K = 0y y0
001 
P P P P  11 12 13 14
Slide credit: Svetlana Lazebnik

Summary: camera parameters
• Intrinsic parameters:
• Principal point coordinates 2
• Focal length 1
• Pixel scaling factor (rectangle) 1
• Shear (non-rectangular pixels)
• Radial distortion
• Extrinsic parameters
• Rotation R
• Translation t
• Camera projection matrix
1
3 3
DoF
x K = 
s

x0  y 

y
0  1 
 

P = KR | t
 A pinhole camera: 9 DoF  Camera with rectangular pixels: 10 DoF  General camera: 11 DoF
Slide credit: Bastan Leibe

Vanishing points
• Where in image do sets of 3D parallel lines intersect? A 3D point:
Camera projection center
Vanishing point (VP)!
𝒙
𝑿
Ground plane Parallel lines
Perspective projection:
• Note that this image shows a special case with lines parallel with principal axis.
• But our derivation of VP will be general.

Vanishing points
• Consider a point on one of parallel lines
A 3D point A and vector D:
Camera projection center
Vanishing point!
XA  
XD  
A=YA  Z
D=YD 
𝒙(𝝀)
Ground plane
A
A point on a line:
Z D
Perspective projection:
𝑨𝑫
𝑿(𝝀)
X() = A + D

Vanishing points
• Now push the point far away from the camera…
Camera projection center
Vanishing point!
As the point is pushed towards infinity, the projection approaches the vanishing point 𝒗!
𝒙(𝝀) 𝑨𝑫
𝑿(𝝀) ∞
Projection of a point at infinity, i.e., 𝑿(∞): f XA +XD 
Vanishing point!
 Z +Z  v=limx()=lim A D
fX /Z  v= D D
→ → Y +Y  fAD
fY /Z DD
 ZA+ZD

Vanishing points
• VP depends on direction 𝑫, not on point 𝑨.
• A different set of parallel lines correspond to a different VP!
v=fXD /ZD fY /Z 
DD
• Horizon is formed by connecting VPs of a plane
𝒙(𝝀)
𝑨𝑫
𝑿(𝝀)

Looking at flat objects
• A camera looking at the some planar object
• How would it look if the camera changed position?
• A plane-to-plane projection is called a Homography

Very useful in practice
• E.g., Image stitching, Augmented reality, etc.

Estimate the
• Example of four corresponding points
homography
𝒙′ 𝒙
𝑤𝒙′=𝑯𝒙   1
21 22 23 H H H1
• The elements of 𝑯 can be estimated by applying a direct linear transform (DLT)!
x’ wy’ =
H11 H12 H13x H H H y

31 32 33

Matrix form of a
• Before we continue…
vector product
The vector cross product also can be expressed as the product of a skew-symmetric matrix and a vector:
Slide credit: Kristen Grauman

Homography
estimation by DLT
wx’i =Hxi
x’  H H H x  hT  hTX 
i 11 12 13i 1 1 i wy’=H H H y=hTX=hTX
 i  21 22 23i 2 i 2 i
1 H H H 1 hT hTX 31 32 333 3 i
x’iHxi =
Change the vector product into vector-matrix:
0
hTx xTh 0 −1 y’xTh
1i i1 ii1 x’hTx=x’ xTh= 1 0 x’xTh
i2i ii2 ii2
hTx xTh −y’ x’ 0xTh 3i i3i i i3

Homography
x’iHxi =0
Multiply in the matrix terms…
estimation by DLT
0 −1 y’xTh −xTh+y’xTh  ii1i2 ii3
x’Hx = 1 0 x’xTh= xTh−x’xTh  iiii2i1ii3
−y’ x’ 0  xTh −y’ xTh +x’ xTh i i i3ii1 ii2
Expose the homography terms h1, h2, h3into a single vector:
 0T −xT y’xT h iii1
x’Hx= xT 0T
iii ii2
h  ii ii 3
−y’ xT x’ xT 0T
−x’xTh=0
A single point contains three coordinates, but yields only two linearly independent equations (First equation *xi + Second equation *yi = Third equations)

Homography
estimation by DLT
n Correspondences…
x’1 x1 x’2 x2

SVD
The n points yields a system of equations:
0T -x’T y’xT
 1 xT 0T
1 1 
−x’ xT h1 
Homogeneoussystem!
 1 2
1 1 h =0 Ah=0 
h=v ,,v  19 99
v99
h 0T-xT y’xT
 n nn3 xT 0T −x’ xT 
d T  11
n nn
 d v  v T
19  11 19  A=UDV =U    
d  d v  v   91 99  91 99 
Minimizes the mean squared error.
Reshape h into H.

Preconditioning
• DLT works well if the corresponding points are normalized separately in each view!
• Transformation 𝑇 : 𝑝𝑟𝑒
• Subtract the average
• Scale to average distance 1.
𝒙
a 0 c T =0 b d
x෤=T x 𝑝𝑟𝑒
pre   0 0 1

• Set [a,b,c,d] such that the mean of the points 𝒙෥𝑖 is zero and their variance is 1.

Homography
• Applypreconditioning(i.e.,multiplybythetransform matrices) to points in each image separately:
estimation
x෤′=T′ x′ x෤=T x 𝑝𝑟𝑒 𝑝𝑟𝑒
෩
• Apply DLT to estimate the homography 𝐻:
• Transform back the solution to remove preconditioning:
H =T’−1HT
pre pre
x’=Hx

Application: Pattern analysis
Compute a homography
to this rectangle
Zoom-in of the floor
Slide credit: Antonio Criminisi

Camera calibration from a known scene
• Projectiongovernedby: x=PX
• Assumethe3Dcoordinates of 𝑿 (in meters) are known.
• Assumethe2Dcoordinates of 𝒙 (in pixels) are known.
x
X
• How to compute the 𝑷? – Resectioning/Calibration (compute intrinsic and extrinsic parameters)
World c.s.

Camera calibration from a known scene
Main idea of approach
• Put a calibration object with known
geometry in front of the camera.
• Determine the correspondences between
image coordinates and 3D object points.
• Solve mapping of 3D scene to 2D image:
Estimate P=PintPext
P?
Xi xi
Slide credit: Kristen Grauman

Camera calibration from a known scene
• Usingnpointswithknown3Dcoordinates{Xi}and known 2D image coordinates of their projections {xi},
estimate the camera parameters.
Xi xi
P?
Slide credit: Svetlana Lazebnik

Camera calibration: point detection
• Propercalibrationrequiresmeasuringthepointsat sub-pixel accuracy.
• Highlydependsonthecalibrationpattern.
Gives better results
• How many point correspondences are required?
• A rule of thumb:
• Number of constraints exceeds the number of unknowns by a factor 5. •  For 11 parameters in P, use at least 28 points.

Camera calibration
• Once the projection matrix P is known, we need to figure out its external and internal parameters, i.e., P=PintPext=K[R|t].
• This is a matrix decomposition problem.
• Inner and outer matrix have a particular form, that
makes such a decomposition possible.
• SolutioncanbefoundinForsyth&Ponce,Chapter1. for those who are interested to learn more about camera calibration.
Slide credit: Svetlana Lazebnik

Camera calibration by DLT
0T XT −yXT  111
XT 0T −xXT P P? 1 111
Xi xi
 P=0 AP=0
0T XT −yXT 2 P
 n nn3 XT 0T − x XT
n nn
• P has 11 DoF (12 parameters, but the scale is arbitrary).
• A single 2D-3D correspondence gives two linearly independent equations.
• Homogeneous system is solved by SVD of A.
• Solution requires at least 5 1⁄2 correspondences.
• Caution: coplanar points yield degenerate solutions.
• Apply preconditioning as with Homography estimation.
Slide adapted from Svetlana Lazebnik

Camera calibration:
practical advices
• The DLT implementation is pretty simple, but…
• A better cost function to optimize is the reprojection error:
E(p)=N εT1/2 0 ε ε=xi= (x−PX)  x  i i   Eucledian i i
i=1i 01/2
 y   yi 

• But requires nonlinear optimization (Hartley&Zisserman, Chapter 7.2)
• In practice, initialize by (preconditioned) DLT.
• For practical applications you will need to first remove the radial distortion (H&Z sec. 7.4, or F&P sec. 3.3.).
• Recently published approach allows a fast and easy estimation of
the camera projection matrix 𝑷 along with the radial distortion
parameters:
http://cmp.felk.cvut.cz/~kukelova/publications/Kukelova-Bujnak-ICCV-2013.pdf

Multiplane
• Widely-usedapproach
• Requiresonlymany
images of a single plane
• Doesnotrequireknowing positions/orientations
• Good code available online!
– Intel’s OpenCV library:
http://www.intel.com/research/mrl/research/opencv/
– Matlab version by Jean-Yves Bouget:
http://www.vision.caltech.edu/bouguetj/calib_doc/index.html
– Zhengyou Zhang’s web site: http://research.microsoft.com/~zhang/Calib/
camera calibration
Images courtesy Jean-Yves Bouguet, Intel Corp.

Summary
• Cameraparameters(intrinsic,extrinsic)
• A perspective projection matrix (3×4)
• Camera calibration (discrepancy between the observed image features and their theoretical position (as predicted by the perspective projection equation) is minimized with
respect to the camera’s intrinsic and extrinsic parameters). – obtain camera matrix by minimizing error
– obtain intrinsic parameters from camera matrix
Hyung Jin Chang, Yixing Gao
08/02/2021

Related Posts