COMP9517: Computer Vision
Motion and Tracking Applications in Biomedical Imaging
Week 7 COMP9517 2021 T1 1
Week 7
COMP9517 2021 T1 2
• •
• •
Topics
Examples of change detection
– Patient motion correction in angiography
Examples of template matching
– Cell motion correction in microscopy
– Monomodal brain image registration
– Multimodal medical image registration
Examples of optical flow
– Heart tissue motion estimation
Examples of object tracking
– Particle tracking in molecular biology
– Bayesian multitarget tracking method – Heart motion tracking and analysis
– Tracking for neuron reconstruction
– Object tracking in cell biology
Example of Change Detection
Week 7 COMP9517 2021 T1 3
Digital Subtraction Angiography
X-ray at time t0
X-ray at time t0 + ∆t
Week 7
COMP9517 2021 T1
4
Mask Image
Live Image
Digital Subtraction Angiography Live – Mask Contrast Stretched
Meijering et al., Radiology, 2001
Week 7 COMP9517 2021 T1
5
Digital Subtraction Angiography Contrast Stretched Motion Corrected
Automatic motion correction here is a form of template matching
Week 7 COMP9517 2021 T1 6
Examples of Template Matching
Week 7 COMP9517 2021 T1 7
Cell Motion Correction
Week 7 COMP9517 2021 T1
8
Cell fixation by image post- processing allows analysis of the internal changes over time
Brain Image Registration
To understand how the human brain develops from childhood to adulthood and to study developmental disorders we can use magnetic resonance imaging (MRI) at different ages and match the images to a template using automatic image registration techniques
Week 7 COMP9517 2021 T1 9
Multimodal Image Registration
Computed Tomography (CT)
Magnetic Resonance (MR)
Joint Visualization
Registration (alignment) of images from multiple imaging modalities (devices) allows joint visualisation which may provide additional information to the physician
Week 7
COMP9517 2021 T1
10
Example of Optical Flow
Week 7 COMP9517 2021 T1 11
Heart Tissue Motion Estimation
• Heart tissue cultured 6 days
• Mono-layer cardiomyocytes
• Phase-contrast microscopy
• Real-time imaging 24 fps
Since the images contain rich information it is easy to estimate local gradients with high accuracy so this is a perfect case for the optical flow method
∇f⋅v=− ft
Week 7
COMP9517 2021 T1
12
Heart Tissue Motion
Motion vectors visualised by direction (color) and magnitude (intensity)
Week 7 COMP9517 2021 T1 13
Examples of Object Tracking
Week 7 COMP9517 2021 T1 14
Particle Tracking Problem
time
????
Week 7 COMP9517 2021 T1 15
Bayesian Tracking
Computing the degree of belief in the object state by taking into account all available evidence up to the current time point
• State: X =(r,v,a,s,I ,) expressedasprobabilitydensityP(X ) tttttt t
Position, velocity, acceleration, shape, intensity, …
• Evidence: a set of images or extracted features Y = {y ,, y }
Prior
t0t Transition Model Posterior
=∫D(X|X )P(X |Y )dX
t t−1 t−1 t−1 t−1
∝ L(Y | X ) tt
Observation Model Prior
• Prediction: • Correction:
P(X|Y ) t t−1
P(X |Y) tt
P(X|Y ) t t−1
Week 7
COMP9517 2021 T1
16
Posterior
Bayesian Multitarget Tracking • Extend the state space to include the states of all targets
Xt =(X1;t,X2;t,,XN;t)
X =(r ,v ,a ,s ,I ,) X =(r ,v ,a ,s ,I ,)
1;t 1;t 1;t 1;t 1;t 1;t N;t N;t N;t N;t N;t N;t Computational cost grows exponentially with the number of targets
• Use a mixture model of single-target probability densities N
∑ n=1
P(X|Y)= w P(X|Y)
n;t n t t
Requires heuristics to keep track of number of targets and identities
tt
Week 7 COMP9517 2021 T1 17
Smal & Meijering 2014
Tracking Heart Motion in MRI
Week 7 COMP9517 2021 T1 20
Tracking Heart Motion in MRI Tracks Strain
Week 7
Smal & Meijering, Medical Image Analysis, 2012
COMP9517 2021 T1 21
Neuron Reconstruction
Week 7 COMP9517 2021 T1 22
Neuron Reconstruction
v2 v3 v1
Week 7
COMP9517 2021 T1 23
Ixy Ixz
H=I I I=VT⋅ΛV⋅
Ixx
yx yy yz
III zx zy zz
Seed points: λ << λ ≈ λ 321
Neuron Reconstruction Target states
x =(x ,y ,z ,vx ,vy ,vz ) 1;k 1;k 1;k 1;k 1;k 1;k 1;k
x =(x ,y ,z ,vx ,vy ,vz ) 2;k 2;k 2;k 2;k 2;k 2;k 2;k
x =(x ,y ,z ,vx ,vy ,vz ) 3;k 3;k 3;k 3;k 3;k 3;k 3;k
Week 7
COMP9517 2021 T1 24
x =(x ,y ,z ,vx ,vy ,vz ) N;k N;k N;k N;k N;k N;k N;k
Tracking for Neuron Reconstruction
Radojevic & Meijering, Neuroinformatics, 2019
Week 7 COMP9517 2021 T1 25
Neuron Reconstruction Results
Week 7 COMP9517 2021 T1 26
Popular segmentation methods
• Intensity thresholding
• Watershed segmentation
• Active contour fitting
• Level-set segmentation
zero-level set
Cell Tracking
Week 7
COMP9517 2021 T1
27
level-set function
Model: Fitting:
C(r) = ∑P B(r n) n
n
C = arg min E (C )
ˆ
−
Cell Tracking Linking by contour model evolution
Dzyubachyk & Meijering, IEEE Transactions on Medical Imaging, 2010
Week 7 COMP9517 2021 T1 28
Cell Tracking
Coloured contours indicate the results of cell segmentation and indentification
Week 7 COMP9517 2021 T1 29
Cell Lineage Reconstruction
Week 7 COMP9517 2021 T1 31
Keller et al. 2014
Drosophila embryogenesis
Week 7 COMP9517 2021 T1 32
Cell Lineage Reconstruction Tracking each cell during Drosophila embryonic development
Keller et al., Nature Methods, 2014
Week 7 COMP9517 2021 T1 33
References and Acknowledgements
Further information on the presented applications can be found in the following papers:
• Image Registration for Digital Subtraction Angiography
• Advanced Level-Set Based Cell Tracking in Time-Lapse Fluorescence Microscopy
• Multimodal Volume Registration by Maximization of Mutual Information
• Optical-Flow Based Non-Invasive Analysis of Cardiomyocyte Contractility
• Multiple Object Tracking in Molecular Bioimaging by RBM Particle Filtering
• Objective Comparison of Particle Tracking Methods
• Reversible Jump MCMC Methods for Fully Automatic Motion Analysis in Tagged MRI
• Automated Neuron Tracing Using Probability Hypothesis Density Filtering
• An Objective Comparison of Cell-Tracking Algorithms
• Methods for Cell and Particle Tracking
• Reconstruction of Cell Lineages From Large-Scale Fluorescence Microscopy Data
• A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking
Week 7 COMP9517 2021 T1 34