Structure from motion
What is Point Cloud
• A collection of Un-ordered points with – Geometry: expressed as [x, y, z]
Copyright By PowCoder代写 加微信 powcoder
– Color Attributes: [r g b], or [y u v]
– Additional info: normal, timestamp, …etc.
• Key difference from mesh: no order info
Point Cloud Capture
• Passive: Camera array stereo depth sensor
• Active: LiDAR, mmWave, TOF sensors
Readings • Szeliski, Chapter 7.1 – 7.4
Shape From X
• Recovery of 3D (shape) from one or two 2D images
Structure from motion
• Given many images, how can we
a) figure out where they were all taken from? b) build a 3D model of the scene?
This is (roughly) the structure from motion problem
Applications
• Object Recognition • Robotics
• Computer Graphics • Image Retrieval
• Localization https://www.youtube.com/watch?v=p16frKJLVi0
Structure from motion
Reconstruction (side)
• Input: images with points in correspondence
pi,j = (ui,j,vi,j)
• structure: 3D location xi for each point pi • motion: camera parameters Rj , tj
• Objectivefunction:minimizereprojectionerror
Two scenarios • Suppose we know 3D points
– and have matches between these points and an image
– How can we compute the camera parameters?
• Suppose we have known camera parameters,
each of which observes a point
– How can we compute the 3D location of that point?
Structure from motion
• SfM solves both of these problems at once
• A kind of chicken-and-egg problem – (but solvable)
First step: how to get correspondence? • Feature detection and matching
Feature detection Detect features using SIFT [Lowe, IJCV 2004]
Feature detection Detect features using SIFT [Lowe, IJCV 2004]
Feature matching Match features between each pair of images
Correspondence estimation
• Link up pairwise matches to form connected components of matches across several images
Image 1 Image 2 Image 3 Image 4
Structure from motion
minimize g(R, T, X)
Re-projection Error
Problem size
• What are the variables need to be solved? R t P
• Trevi Fountain collection 466 input photos
+ > 100,000 3D points
= very large optimization problem
Constraints vs #Unknowns
argmin(uj − f(K,R ,T ,P))2 +(vj −g(K,R ,T ,P))2
{P},K,{R },{T } j=1 i=1 ijj
M camera poses
2MN point constraints
Structure from motion
• Minimize sum of squared reprojection errors:
predicted observed
image location image location is point i visible in image j ?
• Minimizing this function is called bundle adjustment
– e.g. Levenberg-Marquardt
indicator variable:
SfM applications
• 3D modeling
• Robot navigation and mapmaking
• Visual effects (“Match moving”)
– https://www.youtube.com/watch?v=RdYWp70P_kY
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com