代写代考 CVPR 2012

Computer Vision
Camera: Perspective projection
Semester 1
Changjae Oh

Copyright By PowCoder代写 加微信 powcoder

Announcement of
Pope Benedict XVI (2005),
Pope Francis (2013)

• The pinhole projection model ̶ Geometric properties
̶ Perspective projection matrix
• Cameras with lenses
Depth of focus Field of view Lens aberrations
• Digital sensors

• The pinhole projection model ̶ Geometric properties
̶ Perspective projection matrix
• Cameras with lenses
Depth of focus Field of view Lens aberrations
• Digital sensors

Let’s design a camera
• Idea 1: put a piece of film in front of an object • Do we get a reasonable image?

Pinhole camera
• Add a barrier to block off most of the rays
̶ Tiny aperture without lens
̶ Light from a scene passes through the aperture and projects an inv erted image on the film

Pinhole camera
Captures pencil of rays
All rays through a single point: aperture, center of projection, optic al center, focal point, camera center
The image is formed on the image plane (film)

Pinhole camera
f = focal length
c = center of the camera
Slide from Forsyth

Pinhole cameras are everywhere
Tree shadow during a solar eclipse
photo credit: Nils van der Burg
http://www.physicstogo.org/index.cfm

• Basic principle known to Mozi (470-390 BCE), Aristotle (384-322 BCE)
• Drawing aid for artists: described by Le onardo da Vinci (1452-1519)
Image source
Slide from Lazebnik

Turning a room into a camera
• A. Torralba and W. Freeman, Accidental Pinhole and Pinspeck Cameras, CVPR 2012

Turning a room into a camera obscura
Accidental Pinhole and Pinspeck Cameras Revealing the scene outside the picture. , . Freeman

Turning a room into a camera obscura

Pinhole projection model
• To compute the projection P’ of a scene point P,
̶ Form the visual ray connecting P to the camera center O and find where it intersects the image plane

Pinhole projection model
The coordinate system
The optical center (O) is at the origin
The image plane is parallel to xy-plane or perpendicular to the z-axis, which is the optical axis

Pinhole projection model
• Projection equations
• Derived using similar triangles
( x , y , z ) → ( f xz , f yz )

Dimensionality reduction: from 3D to 2D
3D world 2D image
Point of observation
What properties of the world are preserved? • Straightlines,incidence
What properties are not preserved? • Angles,lengths
Slide by A. Efros

Properties of projection
Who is taller?
Which is closer?
Slide by Derek Hoiem

Properties of projection
Perpendicular?
Slide by Derek Hoiem

What happens to the projection of a pattern on a plane parallel to the image plane?
All points on that plane are at a fixed depth z
The pattern gets scaled by a factor of f / z, but angles and ratios of lengths/areas are
parallel planes
Point of observation
( x , y , z ) → ( f xz , f yz )

parallel planes
What happens to the projection of a pattern on a plane parallel to the image plane?
All points on that plane are at a fixed depth z
The pattern gets scaled by a factor of f / z, but angles and ratios of lengths/areas are
Piero della Francesca, Flagellation of Christ, 1455-1460 , The Music Lesson, 1662-1665
Slide from S. Lazebnik

Vanishing points
• All parallel lines converge to a vanishing point
̶ Each direction in space is associated with its own vanishing point ̶ Exception: directions parallel to the image plane

Constructing the vanishing point of a line
image plane
vanishing point
camera center
line in the scene
Slide from S. Lazebnik

Vanishing lines of planes
How do we construct the vanishing line of a plane?
Image source

Vanishing lines of planes
• Horizon: vanishing line of the ground plane
– All points at the same height as the camera project to the horizon
– Points higher (resp. lower) than the camera project above (resp. b elow) the horizon
– Provides way of comparing height of objects
camera center
plane in the scene
Slide by S. Seitz

Vanishing lines of planes
Is the parachutist above or below the camera?
Slide by S. Seitz

Comparing heights
Vanishing Point
Slide by S. Seitz

Measuring height
Camera height
What is the height of the camera?
Slide by S. Seitz

Perspective geometry
Illusion Credit: RN Shepard, Mind Sights: Original Visual Illusions, Ambiguities, and other Anomalies

Perspective cues in art
• Masaccio, Trinity, Santa Maria N ovella, Florence, 1425-28
• One of the first consistent uses o f perspective in Western art
Slide by S. Seitz

Perspective distortion
• What is the shape of the projection of a sphere?
Image source: F. Durand

Perspective distortion
• What is the shape of the projection of a sphere?
Slide from S. Lazebnik

Perspective distortion
• Are the widths of the projected columns equal?
The exterior columns are wider
This is not an optical illusion, and is not due to lens flaws Phenomenon pointed out by
Source: F. Durand

Perspective distortion: People
Slide from S. Lazebnik

Modelling projection:
world to image
• Projection equation:
Note: instead of dealing with an image that is upside down, most of the time we will pretend that the image plane is in front of the camera center.
( x , y , z ) → ( f xz , f yz )
Source: J. Ponce, S. Seitz

Homogeneous coordinate
• Nonlinearity in the projection equation
(x, y, z) → ( f xz , f yz ) : division by z is nonlinear
• Add one more coordinate for linear transformation
homogeneous image homogeneous scene coordinates coordinates
Converting from homogeneous coordinates

Homogeneous coordinate • Why does this matter?
Homogeneous coordinates can handle general cases Invariant to scaling
Point in Cartesian is ray in Homogeneous
x kx ky=ky kw = w
    ky y
w kw kw w 
Homogeneous Cartesian Coordinates Coordinates
Credit: J. Hays

Homogeneous coordinate
• EBU6230 Image and Video Processing
̶ Week 2 day 1: image_transformations
• https://pages.mtu.edu/~shene/COURSES/cs3621/NOTES/geometry/ho mo-coor.html

Perspective projection matrix • Projection:
a matrix multiplication with homogeneous coordinates
f 0 0 0x fx
0 f 0 0y=fy (fx,fy)
0 0 1 0z z z z
 1   divide by the third co
ordinate In practice: lots of coordinate transformations…
2D point (3×1)
Camera to pixel coord. trans. matrix
Perspective projection matrix (3×4)
World to camera coord. trans. matrix
3D point (4×1)
Slide from S. Lazebnik

Orthographic Projection
• Special case of perspective projection
̶ Distance from center of projection to image plane is infinite ̶ Also called “parallel projection”

Orthographic Projection
• Special case of perspective projection
̶ Distance from center of projection to image plane is infinite ̶ Also called “parallel projection”
Slide from S. Lazebnik

Orthographic Projection
• Special case of perspective projection
Distance from center of projection to image plane is infinite Also called “parallel projection”
Image World
What’s the projection matrix?

• When the 3D world dimension reduces to 2D, what properties are preserved? ̶ Straight lines
̶ Incidence

• In the figure below, there are two blue parallel lines, perpendicular to the image plane. Draw how the lines look like in the image plan and discuss the result
Constructing the vanishing point of a line
image plane
camera center
lines in the scene

• Perform Orthograpic projection to each 3D cube in 3D below and show the result in 2D
03) Orthographic projection
(0,1,0) (0,0,0) (-1,1,0)

04) Projection matrix (2D
̶ 𝒙= 𝒙,…,𝒙 = 𝑢,𝑣 ,…,(𝑢,𝑣) = 0,260, 640,260, 0,400, 640,400 141144
̶ 𝒙′ = 𝒙′1,…,𝒙′4 = 𝑢′1,𝑣′1 ,…,(𝑢′4,𝑣′4) = 0,0 , 400,0 , 0,640 , 400,640 Find the perspective projection matrix 𝑷 such that 𝒙′ = 𝑷𝒙
Given two point sets:

Computer Vision
Camera: lenses & digital sensors
Semester 1
Changjae Oh

• The pinhole projection model ̶ Geometric properties
̶ Perspective projection matrix
• Cameras with lenses
Depth of focus Field of view Lens aberrations
• Digital sensors

Building a Real Camera

made pinhole camera
http://www.debevec.org/Pinhole/
Slide by A. Efros 52

Shrinking the aperture
• Images with varying the aperture size
• Why not make the aperture as small as possible? ̶ Less light gets through
̶ Diffraction effects…

Shrinking the aperture
• Images with varying the aperture size

Adding a lens
• A lens focuses light onto the film
̶ Thin lens model:
• Rays passing through the center are not deviated (pinhole projection model still holds)

Adding a lens
• A lens focuses light onto the film
̶ Thin lens model:
• Rays passing through the center are not deviated (pinhole projection model still holds)
• All rays parallel to the optical axis pass through the focal point
• All parallel rays converge to points on the focal plane
focal point

Thin lens formula
• Where does the lens focus the rays coming from a given point in the scene?
image plane
lens object
Slide by F. Durand

Thin lens formula
• What is the relation between the focal length ( f ),
the distance of the object from the optical center ( D ), and the distance at which the object will be in focus ( D′ )?
image plane
lens object
Slide by F. Durand

Thin lens formula
• Similar triangles everywhere!
y′/y = D′/D
image plane
lens object
Slide by F. Durand

Thin lens formula
• Similar triangles everywhere!
y′/y = D′/D y′/y = (D′−f )/f
image plane
lens object
Slide by F. Durand

Thin lens formula
• Any point satisfying the thin lens equation is in focus.
• What happens when D is very large?
y′/y = D′/D y′/y = (D′−f )/f
1+1=1 D′ D f
image plane
lens object
Slide by F. ′ D f

Depth of Field
• For a fixed focal length, there is a specific distance at which objects are “in focus”
̶ Other points project to a “circle of confusion” in the image
“circle of confusion”

Depth of Field
http://www.cambridgeincolour.com/tutorials/depth-of-field.htm
Slide by A. Efros

Controlling depth of field
• Changing the aperture size affects depth of field
̶ A smaller aperture increases the range in which the object is approximately in focus ̶ But small aperture reduces amount of light – need to increase exposure
http://en.wikipedia.org/wiki/File:Depth_of_field_illustration.svg

Varying the aperture
Large aperture = small DOF Small aperture = large DOF
Slide by A. Efros

Field of View
Slide by A. Efros

Field of View
Slide by A. Efros

Field of View
FOV depends on focal length and size of the camera retina
Larger focal length = smaller FOV
Slide by A. Efros

Field of View / Focal Length
Large FOV, small f Camera close to car
Small FOV, large f
Camera far from the car Sources: A. Efros, F. Durand

Same effect for faces
wide-angle standard telephoto
Source: F. Durand

Approximating an orthographic camera
Source: Hartley & Zisserman

The dolly zoom
• Continuously adjusting the focal length while the camera moves away fr om (or towards) the subject
http://en.wikipedia.org/wiki/Dolly_zoom

The dolly zoom
• Continuously adjusting the focal length while the camera moves away fr om (or towards) the subject
Example of dolly zoom from Goodfellas (YouTube)

Real lenses

Lens flaws: Vignetting

Radial Distortion
• Caused by imperfect lenses
• Deviations are most noticeable near the edge of the lens
No distortion Pin cushion Barrel

Lens flaws: Spherical aberration
• Spherical lenses don’t focus light perfectly
• Rays farther from the optical axis focus closer

Lens flaws: Chromatic Aberration
• Lens has different refractive indices for different wavelengths: causes co lor fringing
Near Lens Center Near Lens Outer Edge

• The pinhole projection model ̶ Geometric properties
̶ Perspective projection matrix
• Cameras with lenses
Depth of focus Field of view Lens aberrations
• Digital sensors

First digitally scanned photograph • 1957, 176×176 pixels
http://listverse.com/history/top-10-incredible-early-firsts-in-photography/

Camera sales over time

Camera sales over time • The full chart…

Digital cameras
• Digital cameras are not designed to be light measuring devices
• They are designed to produce visually pleasing photographs
• There is a great deal of processing (photo finishing) applied in the camera h ardware

Modern photography pipeline
Source: M. Brown

Digital camera sensors
• Each cell in a sensor array is a light-sensitive diode that converts photon s to electrons
̶ Dominant in the past: Charge Coupled Device (CCD)
̶ Dominant now: Complementary Metal Oxide Semiconductor (CMOS)
http://electronics360.globalspec.com/article/9464/ccd-vs-cmos-the-shift-in-image-sensor-technology

What does a raw image look like?

Color filter arrays
Bayer grid
Demosaicing:
Estimation of missing components from nei ghboring values
Why more green?
Human Luminance Sensitivity Function

Virtual Background

Filming With Green Screen: Everything You Need To Know

Demosaicing
• Producing full RGB image from mosaiced sensor output
Interpolate from neighbors:
Bilinear interpolation (needs 4 neighbors).
Bicubic interpolation (needs more neighbors, may overblur).
• Edge-awareinterpolation. • Large area of research.
Slide credit: I. Gkioulekas

Demosaicing
• Bilinear interpolation: Simply average your 4 neighbors.
• Neighborhood changes for different channels:
Slide credit: I. Gkioulekas

Digital camera artifacts • Noise
low light is where you most notice noise light sensitivity (ISO) / noise tradeoff stuck pixels
• In-camera processing
̶ oversharpening can produce halos
• Compression
̶ JPEG artifacts, blocking
• Blooming
̶ CCD charge overflowing into neighboring pixels
• Color artifacts ̶ Color moire
̶ Purple fringing from microlenses

Walking through the pipeline

Walking through the pipeline

Walking through the pipeline

Walking through the pipeline

• A Bayer filter mosaic is a color filter array (CFA) for arranging RGB color filters on a square grid of photose nsors. Figure 1 shows a raw image obtained by the Bayer pattern CFA where each pixel includes the R or G or B value. Perform image demosaicing to the image region in orange box using bilinear interpolation and provide each pixel’s RGB value, (R, G, B) of the 3 × 3 image.

Next topic
• How can we remove artifacts in images? ̶ Prerequisite
• Review EBU6230 Image/Video Processing – Week2: Image Filtering

程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com