[06-30213][06-30241][06-25024]
Computer Vision and Imaging &
Robot Vision
Dr Hyung Jin Chang Dr Yixing Gao
h.j.chang@bham.ac.uk y.gao.8@bham.ac.uk
School of Computer Science
Binary image analysis – Blobs and regions
Reading: (Szeliski 4.2)
Hyung Jin Chang
Lecture 1 – 2 14/02/2021
Binary images
3
Slide credit: Kristen Grauman
Binary images
• Advantages
– Easy to acquire
– Low storage: no more than 1 bit/pixel – Simple processing
• Disadvantages
– Limited application
– Does not extend to 3D
– Specialised lighting is required for silhouettes
Slide credit: Kristen Grauman
Binary image analysis: basic steps
• Converttheimageintobinaryform – Thresholding
• Cleanupthethresholdedimage – Morphological operators
• Extractseparateblobs – Connected components
• Describetheblobswithregionproperties
5
Slide credit: Kristen Grauman
Binary images
• Two-pixel values
– Foreground and background – Mark region(s) of interest
6
Slide credit: Kristen Grauman
Thresholding
• Grayscale->binarymask
• Usefulifobjectofinterest’sintensitydistribution is distinct from background
• Example http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FITZGIBBON/ simplebinary.html 7
Slide credit: Kristen Grauman
Thresholding
• Given a grayscale image or an intermediate matrix à threshold to create a binary output.
Example: edge detection
Gradient magnitude
Looking for pixels where gradient is strong.
fg_pix = find(gradient_mag > t);
8
Slide adapted from Kristen Grauman
Thresholding
• Given a grayscale image or an intermediate matrix à threshold to create a binary output.
Example: background subtraction
=
–
Looking for pixels that differ significantly from the “empty” background.
fg_pix = find(diff > t);
9
Slide credit: Kristen Grauman
Thresholding
• Given a grayscale image or an intermediate matrix à threshold to create a binary output.
Example: intensity-based detection
fg_pix = find(im < 65);
Looking for dark pixels
10
Slide credit: Kristen Grauman
Thresholding
• Given a grayscale image or an intermediate matrix à threshold to create a binary output.
Example: color-based detection
fg_pix = find(hue > t1 & hue < t2);
Looking for pixels within a certain hue range.
11
Slide credit: Kristen Grauman
A nice case: bimodal intensity histograms
Ideal histogram, light object on dark background
Images: http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/OWENS/LECT2/node3.html
12
Actual observed histogram with noise
Slide credit: Shapiro and Stockman
Not so nice cases
13
Issues
• Whattodowith“noisy”binary outputs?
– Holes
– Extra small fragments
• Howtodefineaboundaryon multiple regions of interest?
– Count objects
– Compute further features per object
14
Slide credit: Kristen Grauman
Issues
• Whattodowith“noisy”binary outputs?
– Holes
– Extra small fragments
• Howtodefineaboundaryon multiple regions of interest?
– Count objects
– Compute further features per object
15
Slide credit: Kristen Grauman
Morphological operators
• Changetheshapeoftheforegroundregionsvia intersection/union operations between a scanning structuring element and binary image
• Usefultocleanupresultfromthresholding
• Basicoperatorsare:
– Dilation – Erosion
16
Slide credit: Kristen Grauman
Dilation
• Expandsconnectedcomponents • Growfeatures
• Fillholes
Before dilation After dilation
17
Slide credit: Kristen Grauman
Erosion
• Erodeconnectedcomponents
• Shrinkfeatures
• Removebridges,branches,noise
Before erosion After erosion
18
Slide credit: Kristen Grauman
Structuring elements
• Masksofvaryingshapesandsizesusedto perform morphology, for example:
• Scanmaskacrossforegroundpixelsto transform the binary image
19
Slide credit: Kristen Grauman
Dilation vs. Erosion At each position:
• Dilation:ifcurrentpixelis1,thensetallthe output pixels corresponding to structuring element to 1.
20
Example for Dilation
Input image Structuring Elem1ent
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
1
Slide credit: Adapted by Kristen Grauman from T. Moeslund
21
Example for Dilation
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
1
1
Slide credit: Kristen Grauman
22
Example for Dilation
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
1
1
0
Slide credit: Kristen Grauman
23
Example for Dilation
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
1
1
0
0
Slide credit: Kristen Grauman
24
Example for Dilation
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
1
1
0
1
1
1
Slide credit: Kristen Grauman
25
Example for Dilation
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
1
1
0
1
1
1
1
Slide credit: Kristen Grauman
26
Example for Dilation
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
1
1
0
1
1
1
1
1
Slide credit: Kristen Grauman
27
Example for Dilation
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
1
1
0
1
1
1
1
1
Slide credit: Kristen Grauman
28
Example for Dilation
Input image Structuring Element
Output Image
Note that the object gets bigger and holes are filled.
>> help imdilate
29
1
0
0
0
1
1
1
0
1
1
1
1
1
1
1
0
1
1
1
1
1
1
1
Slide credit: Kristen Grauman
2D example for dilation
Slide credit: Shapiro & Stockman
30
Dilation vs. Erosion At each position:
• Dilation:ifcurrentpixelis1,thensetallthe output pixels corresponding to structuring element to 1.
• Erosion:ifeverypixelunderthestructuring element is 1, then set the output pixel corresponding to the current pixel to 1.
31
Example for Erosion (1D)
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
0
Slide credit: Kristen Grauman
32
Example for Erosion (1D)
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
0
0
Slide credit: Kristen Grauman
33
Example for Erosion (1D)
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
0
0
0
Slide credit: Kristen Grauman
34
Example for Erosion (1D)
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
0
0
0
0
Slide credit: Kristen Grauman
35
Example for Erosion (1D)
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
0
0
0
0
0
Slide credit: Kristen Grauman
36
Example for Erosion (1D)
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
0
0
0
0
0
1
Slide credit: Kristen Grauman
37
Example for Erosion (1D)
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
0
0
0
0
0
1
0
Slide credit: Kristen Grauman
38
Example for Erosion (1D)
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
0
0
0
0
0
1
0
0
Slide credit: Kristen Grauman
39
Example for Erosion (1D)
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
1
1
1
0
0
0
0
0
1
0
0
0
Slide credit: Kristen Grauman
40
Example for Erosion (1D)
Input image Structuring Element
Output Image
1
0
0
0
1
1
1
0
1
1
0
0
0
0
0
1
0
0
0
1
Slide credit: Kristen Grauman
Note that the object gets smaller
>> help imerode
41
1
1
1
2D example for erosion
Slide credit: Shapiro & Stockman
42
Opening • Erode,thendilate
• Removesmallobjects,keeporiginalshape
Before opening After opening
44
Slide credit: Kristen Grauman
Closing • Dilate,thenerode
• Fillholes,butkeeporiginalshape
Slide credit: Kristen Grauman
Before closing
After closing
45
Issues
• Whattodowith“noisy”binary outputs?
– Holes
– Extra small fragments
• Howtodefineaboundaryon multiple regions of interest?
– Count objects
– Compute further features per object
Slide credit: Kristen Grauman
46
Connected components
• Identifydistinctregionsof“connectedpixels”
>> L = bwlabel(BW,conn)
47
Connectedness
• Definingwhichpixelsareconsideredneighbors
Slide credit: Chaitanya Chandra
4-connected
8-connected
48
Slide credit: Pinar Duygulu
Connected components
49
Region properties
• Givenconnectedcomponents,cancompute
simple features per blob, such as:
– Area (num pixels in the region)
– Centroid (average x and y position of pixels in the region) – Bounding box (min and max coordinates)
A2=170
A1=200 50
Slide credit: Kristen Grauman
Binary image analysis: basic steps (recap)
• Converttheimageintobinaryform – Thresholding
• Cleanupthethresholdedimage – Morphological operators
• Extractseparateblobs – Connected components
• Describetheblobswithregionproperties
51
Slide credit: Kristen Grauman
Matlab
• L = bwlabel (BW,8);
• STATS = regionprops(L,PROPERTIES) ;
– ‘Area’
– ‘Centroid’
– ‘BoundingBox’
– ‘Orientation‘, … • IM2 = imerode(IM,SE);
• IM2 = imdilate(IM,SE); • IM2 = imclose(IM,SE); • IM2 = imopen(IM,SE);
Slide adapted from Kristen Grauman
52
Example using binary image analysis: segmentation of a liver
53
Slide credit: Li Shen
• Pros
Binary images
– Can be fast to compute, easy to store
– Simple processing techniques available
– Lead to some useful compact shape descriptors
• Cons
– Hard to get “clean” silhouettes
– Noise common in realistic scenarios – Can be too coarse of a representation
Slide credit: Kristen Grauman
54
• Operations, tools
Derivative filters Smoothing, morphology Thresholding Connected components Matching filters Histograms
Summary
• Features, representations
Edges, gradients Blobs/regions Local patterns Textures (next) Color distributions
57
Thank you
Hyung Jin Chang
14/02/2021