代写代考 EBU6230 content)

PowerPoint 프레젠테이션

Changjae Oh

Computer Vision
– Tracking: Image Alignment –

Semester 1, 22/23

• Motion Estimation (Review of EBU6230 content)

• Image Alignment

• Kanade-Lucas-Tomasi (KLT) Tracking

• Mean-shift Tracking

Objectives

• To review Lucas-Kanade optical flow in EBU6230

• To understand Lucas-Kanade image alignment

• To understand the relationship between Lucas-Kanade optical flow and im
age alignment

• To understand Kanade-Lucas-Tomasi tracker

• Motion Estimation (Review of EBU6230 content)

• Image Alignment

• Kanade-Lucas-Tomasi (KLT) Tracking

• Mean-shift Tracking

Motion Estimation: Gradient method

• Brightness consistency constraint

• small motion: (Δx and Δy are less than 1 pixel)
– suppose we take the Taylor series expansion of I:

),,(),,( ttyyxxItyxH +++=

tyxIttyyxI

sorder termhigher

Gradient method

• Spatio-temporal constraint

• This equation introduces one constraint only
– Where the motion vector of a pixel has 2 components (parameters)

– A second constraints is necessary to solve the system

Aperture problem

• The aperture problem
– stems from the need to solve one equation with two unknowns,

which are the two components of optical flow

– it is not possible to estimate both components of the optical flow
from the local spatial and temporal derivatives

• By applying a constraint
– the optical flow field changes smoothly in a small neighborhood

it is possible to estimate both components of the optical flow
if the spatial and temporal derivatives of the image
intensity are available

Solving the aperture problem

• How to get more equations for a pixel?

• By applying a constraint
– the optical flow field changes smoothly in a small neighborhood

it is possible to estimate both components of the optical flow
if the spatial and temporal derivatives of the image
intensity are available

• Lucas–Kanade method

Gradient method

• The Lucas–Kanade method assumes that the displacement of the image contents
between two nearby instants (frames) is small

• Matrix form

… …

Gradient method

• Prob: we have more equations than unknowns

• Solution: solve least squares problem

• minimum least squares solution given by solution of:

• The summations are over all n pixels in the K x K window

• This technique was first proposed by Lukas & Kanade (1981)

minimize bAv −

Lucas-Kanade flow

• When is this solvable?
• ATA should be invertible

• ATA should not be too small due to noise

– eigenvalues l1 and l2 of A
TA should not be too small

• ATA should be well-conditioned

– l1/ l2 should not be too large (l1 = larger eigenvalue)

– Recall the Harris corner detector: M = ATA is the second moment

Interpreting the eigenvalues

l1 and l2 are large,

l1 and l2 are small “Edge”

Classification of image points using eigenvalues

of the second moment matrix:

Uniform region

– gradients have small magnitude

– small l1, small l2
– system is ill-conditioned

– gradients have one dominant direction

– large l1, small l2
– system is ill-conditioned

High-texture or corner region

– gradients have different directions, large magnitudes

– large l1, large l2
– system is well-conditioned

• Motion Estimation (Review of EBU6230 content)

• Image Alignment

• Kanade-Lucas-Tomasi (KLT) Tracking

• Mean-shift Tracking

How can I find in the image?

Idea #1: Template Matching

Slow, global solution

Idea #2: Pyramid Template Matching

Faster, locally optimal

Idea #3: Model refinement

Fastest, locally optimal

(when you have a good initial solution)

Some notation before we get into the math…

2D image transformation

2D image coordinate

Parameters of the transformation

Translation

transform coordinate

Pixel value at a coordinate

affine transform
coordinate

Warped image

can be written in matrix form when linear
affine warp matrix can also be 3×3 when last row is [0 0 1]

Image alignment

• Problem definition

warped image template image

Find the warp parameters p such that the SSD is minimized

Image alignment

Find the warp parameters p such that the SSD is minimized

Image alignment

• Problem definition

warped image template image

Find the warp parameters p such that the SSD is minimized

How could you find a solution to this problem?

Image alignment

This is a non-linear (quadratic) function of a
non-parametric function!

(Function I is non-parametric)

Hard to optimize

What can you do to make it easier to solve?

assume good initialization,
linearized objective and update incrementally

Lucas-Kanade alignment

If you have a good initial guess p…

can be written as …

(a small incremental adjustment)

(pretty strong assumption)

(this is what we are solving for now)

Lucas-Kanade alignment

How can we linearize the function I for a really small perturbation of p?

Taylor series approximation!

This is still a non-linear (quadratic) function of
a non-parametric function!

(Function I is non-parametric)

Lucas-Kanade alignment

Multivariable Taylor Series Expansion
(First order approximation)

chain rule

short-hand

Lucas-Kanade alignment

By linear approximation,

Now, the function is a linear function of the unknowns

Lucas-Kanade alignment

Affine transform

Rate of change of the warp

The Jacobian
(A matrix of partial derivatives)

Lucas-Kanade alignment

pixel coordinate

image intensity

warp function

warp parameters
(6 for affine)

image gradient

Partial derivatives of warp function

incremental warp

template image intensity

Summary: Lucas-Kanade alignment

Taylor series approximation

Difficult non-linear optimization problem

Assume known approximate solution
Solve for increment

then solve for

warped image template image