程序代写 CVPR 2012 - PowCoder代写

PowerPoint 프레젠테이션

Changjae Oh

Computer Vision
– Camera: Perspective projection –

Semester 1, 22/23

Announcement of
Pope Benedict XVI (2005),
Pope Francis (2013)

• The pinhole projection model

̶ Geometric properties

̶ Perspective projection matrix

• Cameras with lenses

̶ Depth of focus

̶ Field of view

̶ Lens aberrations

• Digital sensors

• The pinhole projection model

̶ Geometric properties

̶ Perspective projection matrix

• Cameras with lenses

̶ Depth of focus

̶ Field of view

̶ Lens aberrations

• Digital sensors

Let’s design a camera

• Idea 1: put a piece of film in front of an object

• Do we get a reasonable image?

Pinhole camera

• Add a barrier to block off most of the rays

̶ Tiny aperture without lens

̶ Light from a scene passes through the aperture and projects an inv
erted image on the film

Pinhole camera

• Captures pencil of rays

̶ All rays through a single point: aperture, center of projection, optic
al center, focal point, camera center

• The image is formed on the image plane (film)

Pinhole camera

Slide from Forsyth

f = focal length
c = center of the camera

Pinhole cameras are everywhere

Tree shadow during a solar eclipse
photo credit: Nils van der Burg

http://www.physicstogo.org/index.cfm

• Basic principle known to Mozi (470-390
BCE), Aristotle (384-322 BCE)

Camera obscura

• Drawing aid for artists: described by Le
onardo da Vinci (1452-1519)

Image source

Slide from Lazebnik

https://en.wikipedia.org/wiki/Camera_obscura#/media/File:Camera_Obscura_box18thCentury.jpg

Turning a room into a camera obscura

• A. Torralba and W. Freeman, Accidental Pinhole and Pinspeck Cameras, CVPR 2012

http://people.csail.mit.edu/torralba/research/accidentalcameras/

Turning a room into a camera obscura

Accidental Pinhole and Pinspeck Cameras
Revealing the scene outside the picture.

, . Freeman

Turning a room into a camera obscura

Pinhole projection model

• To compute the projection P’ of a scene point P,
̶ Form the visual ray connecting P to the camera center O and find

where it intersects the image plane

Pinhole projection model

• The coordinate system
̶ The optical center (O) is at the origin

̶ The image plane is parallel to xy-plane or perpendicular to the z-axis,
which is the optical axis

• Projection equations
• Derived using similar triangles ),(),,(

Pinhole projection model

Dimensionality reduction: from 3D to 2D

Point of observation

3D world 2D image

• What properties of the world are preserved?

• Straight lines, incidence

• What properties are not preserved?

• Angles, lengths
Slide by A. Efros

Properties of projection

Which is closer?

Who is taller?

Slide by Derek Hoiem

Properties of projection

Perpendicular?

Slide by Derek Hoiem

Point of observation

Fronto-parallel planes

• What happens to the projection of a pattern on a plane parallel to
the image plane?

• All points on that plane are at a fixed depth z

• The pattern gets scaled by a factor of f / z, but angles and ratios of lengths/areas are

Fronto-parallel planes

• What happens to the projection of a pattern on a plane parallel to
the image plane?

• All points on that plane are at a fixed depth z

• The pattern gets scaled by a factor of f / z, but angles and ratios of lengths/areas are

, The Music Lesson, 1662-1665Piero della Francesca, Flagellation of Christ, 1455-1460 Slide from S. Lazebnik

Vanishing points

• All parallel lines converge to a vanishing point

̶ Each direction in space is associated with its own vanishing point

̶ Exception: directions parallel to the image plane

Constructing the vanishing point of a line

image plane

line in the scene

vanishing point

Slide from S. Lazebnik

Vanishing lines of planes

Image source
How do we construct the vanishing line of a plane?

First World War Centenary Cathedral Repairs Fund helps Peterborough Cathedral carry out essential work

Vanishing lines of planes

Slide by S. Seitz

plane in the scene

• Horizon: vanishing line of the ground plane

– All points at the same height as the camera project to the horizon

– Points higher (resp. lower) than the camera project above (resp. b
elow) the horizon

– Provides way of comparing height of objects

Vanishing lines of planes

Is the parachutist above or below the camera?

Slide by S. Seitz

Comparing heights

Slide by S. Seitz

Measuring height

Camera height

What is the height of the camera?

Slide by S. Seitz

Perspective geometry

Illusion Credit: RN Shepard, Mind Sights: Original Visual Illusions, Ambiguities, and other Anomalies

Perspective cues in art

• Masaccio, Trinity, Santa Maria N
ovella, Florence, 1425-28

• One of the first consistent uses o
f perspective in Western art

Slide by S. Seitz

Perspective distortion

• What is the shape of the projection of a sphere?

Image source: F. Durand

Perspective distortion

• What is the shape of the projection of a sphere?

Slide from S. Lazebnik

Perspective distortion

• Are the widths of the projected columns equal?

̶ The exterior columns are wider

̶ This is not an optical illusion, and is not due to lens flaws

̶ Phenomenon pointed out by

Source: F. Durand

Perspective distortion: People

Slide from S. Lazebnik

Modelling projection: world to image

• Projection equation:

Note: instead of dealing with an image that is upside down, most of the time we
will pretend that the image plane is in front of the camera center.

Source: J. Ponce, S. Seitz

Homogeneous coordinate

• Nonlinearity in the projection equation

• Add one more coordinate for linear transformation

fzyx → : division by z is nonlinear

homogeneous image
coordinates

homogeneous scene
coordinates

Converting from homogeneous coordinates

Homogeneous coordinate

• Why does this matter?

̶ Homogeneous coordinates can handle general cases

̶ Invariant to scaling

̶ Point in Cartesian is ray in Homogeneous

Homogeneous
Coordinates

Coordinates

Credit: J. Hays

Homogeneous coordinate

• EBU6230 Image and Video Processing

̶ Week 2 day 1: image_transformations

• https://pages.mtu.edu/~shene/COURSES/cs3621/NOTES/geometry/ho
mo-coor.html

https://pages.mtu.edu/~shene/COURSES/cs3621/NOTES/geometry/homo-coor.html

Perspective projection matrix

• Projection:
̶ a matrix multiplication with homogeneous coordinates

divide by the third co

In practice: lots of coordinate transformations…

camera coord.

trans. matrix

Perspective
projection matrix

pixel coord.

trans. matrix

Slide from S. Lazebnik

Orthographic Projection

• Special case of perspective projection

̶ Distance from center of projection to image plane is infinite

̶ Also called “parallel projection”

Image World

Orthographic Projection

• Special case of perspective projection

̶ Distance from center of projection to image plane is infinite

̶ Also called “parallel projection”

Slide from S. Lazebnik

• Special case of perspective projection

̶ Distance from center of projection to image plane is infinite

̶ Also called “parallel projection”

̶ What’s the projection matrix?

Orthographic Projection

Image World

Quiz-01) 3D-to-2D

• When the 3D world dimension reduces to 2D, what properties are preserved?

̶ Straight lines

̶ Incidence

Quiz-02) Constructing the vanishing point of a line

• In the figure below, there are two blue parallel lines, perpendicular to the image plane.
Draw how the lines look like in the image plan and discuss the result

image plane

lines in the scene

Quiz-03) Orthographic projection

• Perform Orthograpic projection to each 3D cube in 3D below and show the result in 2D

Quiz-04) Projection matrix (2D-to-2D)

• Given two point sets:
̶ 𝒙 = 𝒙1, … , 𝒙4 = 𝑢1, 𝑣1 , … , (𝑢4, 𝑣4) = 0,260 , 640,260 , 0,400 , 640,400

̶ 𝒙′ = 𝒙′1, … , 𝒙′4 = 𝑢′1, 𝑣′1 , … , (𝑢′4, 𝑣′4) = 0, 0 , 400, 0 , 0,640 , 400,640

Find the perspective projection matrix 𝑷 such that 𝒙′ = 𝑷𝒙

Changjae Oh

Computer Vision
– Camera: lenses & digital sensors –

Semester 1, 22/23

• The pinhole projection model

̶ Geometric properties

̶ Perspective projection matrix

• Cameras with lenses

̶ Depth of focus

̶ Field of view

̶ Lens aberrations

• Digital sensors

Building a Real Camera

Home-made pinhole camera

http://www.debevec.org/Pinhole/

Slide by A. Efros

http://www.debevec.org/Pinhole/35mm-pinhole-camera.jpg
http://www.debevec.org/Pinhole/

Shrinking the aperture

• Images with varying the aperture size

• Why not make the aperture as small as possible?

̶ Less light gets through

̶ Diffraction effects…

Shrinking the aperture

• Images with varying the aperture size

Adding a lens

• A lens focuses light onto the film

̶ Thin lens model:
• Rays passing through the center are not deviated (pinhole projection model still holds)

Adding a lens

• A lens focuses light onto the film

̶ Thin lens model:
• Rays passing through the center are not deviated (pinhole projection model still holds)

• All rays parallel to the optical axis pass through the focal point

• All parallel rays converge to points on the focal plane

focal point

Thin lens formula

• Where does the lens focus the rays coming from a given point
in the scene?

objectimage plane lens
Slide by F. Durand

Thin lens formula

• What is the relation between the focal length ( f ),
the distance of the object from the optical center ( D ),
and the distance at which the object will be in focus ( D′ )?

objectimage plane lens

Slide by F. Durand

Thin lens formula

• Similar triangles everywhere!

objectimage plane lens

y′/y = D′/D

Slide by F. Durand

Thin lens formula

• Similar triangles everywhere!

objectimage plane lens

y′/y = D′/D

y′/y = (D′−f )/f

Slide by F. Durand

Thin lens formula

• Any point satisfying the thin lens equation is in focus.

• What happens when D is very large?

objectimage plane lens

y′/y = D′/D

y′/y = (D′−f )/f

Slide by F. Durand

Depth of Field

• For a fixed focal length, there is a specific distance at which objects are
“in focus”

̶ Other points project to a “circle of confusion” in the image

“circle of
confusion”

Depth of Field

http://www.cambridgeincolour.com/tutorials/depth-of-field.htm

Slide by A. Efros

http://www.cambridgeincolour.com/tutorials/depth-of-field.htm

Controlling depth of field

• Changing the aperture size affects depth of field

̶ A smaller aperture increases the range in which the object is approximately in focus

̶ But small aperture reduces amount of light – need to increase exposure

http://en.wikipedia.org/wiki/File:Depth_of_field_illustration.svg

Varying the aperture

Large aperture = small DOF Small aperture = large DOF Slide by A. Efros

Field of View

Slide by A. Efros

Field of View

Slide by A. Efros

Field of View

Larger focal length = smaller FOV

FOV depends on focal length and size of the camera retina

Slide by A. Efros

Field of View / Focal Length

Large FOV, small f

Camera close to car

Small FOV, large f

Camera far from the car Sources: A. Efros, F. Durand

Same effect for faces

standardwide-angle telephoto

Source: F. Durand

The dolly zoom

• Continuously adjusting the focal length while the camera moves away fr
om (or towards) the subject

http://en.wikipedia.org/wiki/Dolly_zoom

The dolly zoom

• Continuously adjusting the focal length while the camera moves away fr
om (or towards) the subject

Example of dolly zoom from Goodfellas (YouTube)

Real lenses

Lens flaws: Vignetting

Radial Distortion

• Caused by imperfect lenses

• Deviations are most noticeable near the edge of the lens

No distortion Pin cushion Barrel

Lens flaws: Spherical aberration

• Spherical lenses don’t focus light perfectly

• Rays farther from the optical axis focus closer

Lens flaws: Chromatic Aberration

• Lens has different refractive indices for different wavelengths: causes co
lor fringing

Near Lens Center Near Lens Outer Edge

• The pinhole projection model

̶ Geometric properties

̶ Perspective projection matrix

• Cameras with lenses

̶ Depth of focus

̶ Field of view

̶ Lens aberrations

• Digital sensors

First digitally scanned photograph

• 1957, 176×176 pixels

http://listverse.com/history/top-10-incredible-early-firsts-in-photography/

Camera sales over time

https://www.dpreview.com/news/9398648371/2016-cipa-data-shows-compact-digital-camera-sales-lower-than-ever

Camera sales over time

• The full chart…

https://www.dpreview.com/news/9398648371/2016-cipa-data-shows-compact-digital-camera-sales-lower-than-ever

Digital cameras

• Digital cameras are not designed to be light measuring devices

• They are designed to produce visually pleasing photographs

• There is a great deal of processing (photo finishing) applied in the camera h

Modern photography pipeline

Source: M. Brown

Digital camera sensors

• Each cell in a sensor array is a light-sensitive diode that converts photon
s to electrons

̶ Dominant in the past: Charge Coupled Device (CCD)

̶ Dominant now: Complementary Metal Oxide Semiconductor (CMOS)

http://electronics360.globalspec.com/article/9464/ccd-vs-cmos-the-shift-in-image-sensor-technology

What does a raw image look like?

Color filter arrays

Demosaicing:

Estimation of missing
components from nei
ghboring values

Why more green?

Bayer grid

Human Luminance Sensitivity Function

Virtual Background

Filming With Green Screen: Everything You Need To Know

Demosaicking

• Producing full RGB image from mosaiced sensor output

̶ Interpolate from neighbors:
• Bilinear interpolation (needs 4 neighbors).

• Bicubic interpolation (needs more neighbors, may overblur).

• Edge-aware interpolation.

• Large area of research.

Slide credit: I. Gkioulekas

Demosaicking

• Bilinear interpolation: Simply average your 4 neighbors.

• Neighborhood changes for different channels:

Slide credit: I. Gkioulekas

Digital camera artifacts

̶ low light is where you most notice noise

̶ light sensitivity (ISO) / noise tradeoff

̶ stuck pixels

• In-camera processing
̶ oversharpening can produce halos

• Compression
̶ JPEG artifacts, blocking

• Blooming
̶ CCD charge overflowing into neighboring pixels

• Color artifacts
̶ Color moire

̶ Purple fringing from microlenses

Walking through the pipeline

• A Bayer filter mosaic is a color filter array (CFA) for arranging RGB color filters on a square grid of photose
nsors. Figure 1 shows a raw image obtained by the Bayer pattern CFA where each pixel includes the R or G
or B value. Perform image demosaicing to the image region in orange box using bilinear interpolation and
provide each pixel’s RGB value, (R, G, B) of the 3 × 3 image.

4 2 8 10 6 0 2

8 8 8 2 6 0 8

0 2 2 4 2 6 0

8 0 8 10 8 0 6

0 2 0 2 0 0 0

10 0 8 0 8 0 6

0 2 0 2 0 4 0

Next topic

• How can we remove artifacts in images?

̶ Prerequisite

• Review EBU6230 Image/Video Processing – Week2: Image Filtering

程序代写 CS代考加微信: powcoder QQ: 1823890830 Email: powcoder@163.com

Related Posts