Image Representation and Description
Faraz Janan
Lecture 9: Image Processing
About me
What do we mean by Image Representation
After segmentation we need regions -> Region of Intrest
Extract data from that region, i.e. texture, gradient, shape colour –or atleast have to store its margins in a computer for later use
Useful in comparing regions/images, classification, recognition, interpretation; as well as when trying to reduce the amount of data
Inside and outside information
As the example, a region may be represented by its boundary, and the boundary is described by features such as its length, the orientation of the straight line joining its extreme points, and the number of concavities in the boundary.
An external representation is chosen when the primary focus is on shape characteristics.
An internal representation is selected when the primary focus is on regional properties, such as color and texture.
Preview
Sometimes it may be necessary to use both types of representation.
In either case, the features selected as descriptors should be as insensitive as possible to variations in size, translation, and rotation.
For most part, the descriptors discussed in this lecture satisfy one or more of these properties.
Preview
Classically there are atleast 3 ways to represent boundary (there are many more)
◦ 1. Representing a boundary as x-y locations
◦ 2. Representing a boundary as image pixel index
◦ 3. Representation using advance coding techniques and interpretation techniques such as chain codes, Fourier measures, shape signatures etc – this is what we are interested in
A region gives information about: ◦ Texture
◦ Shape
◦ Colour
◦ Motion
◦ Structure
We can call all of the above as features
Invariance?
Lesion Segmentation Segmented boundary
Segmented boundary Segmented boundary
Ground Truth
Overlaping region 140
Segmented boundary Ground Truth Overlaping region
Segmented boundary Ground Truth Overlaping region
Segmented boundary Ground Truth Overlaping region
Ground Truth Overlaping region
Ground Truth Overlaping region
120
120
Segmented boundary Ground Truth
Overlaping region
160
180
200
220
Segmented boundary Ground Truth Overlaping region
160 180 200 220 240 260 280 300
150
Segmented boundary Ground Truth Overlaping region
200
Segmented boundary Ground Truth 250 Overlaping region
300 350 400 450 500 550
280
300
320
340
360
380
400
420
90 100 110 120 130 140 150 160 170 180 190
Segmented boundary Ground Truth Overlaping region
140
Segmented boundary Ground Truth Overlaping region
100
150
200
250
300
200
250 300
350
400
450
500
550 600
Segmented boundary Ground Truth Overlaping region
240
100
Segmented boundary 110
44G0rou4n6d0 Tru4t8h0 Overlaping region
120 130 140 150 160 170 180
Segmented boundary Ground Truth Overlaping region
300
400
500
600
700 18900 200
20 40 60 80 100 120 140 160 180
Interested in texture inside and shape of boundary
440
350
300 320 340 360 380 400 420
400
100 150
200
250 300 350
Advance Boundary Descriptors
Boundary Descriptors
◦ Chain codes
◦ Shape Numbers and signatures ◦ Fourier Descriptors
◦ Statistical Moments
Regional Descriptors
◦ Topological Descriptors ◦ Texture
Statistical approaches Spectral approaches
Structural approaches
Boundary Descriptors – Example Algo
boundary of a region ordered in a clockwise direction.
Lets try to trace a boundary
we assume that:
◦ 1. we are working with binary images in which object and background
points are labeled 1 and 0, respectively.
◦ 2. images are padded with a border of 0s to eliminate the possibility of an object merging with image border.
◦ 3. for convenience, we limit the discussion to single regions.The approach is extended to multiple, disjoint regions by processing the regions individually.
Boundary Descriptors – Example Algo
Given a binary region R or its boundary, an algorithm for following the border of R, or the given boundary, consists of the following:
◦ 1- let the starting point, b0, be the uppermost, leftmost point in the image that is labeled 1. denote by c0 the west neighbor of b0 [see Fig. 11.1 (b)]. Clearly, c0 always is a background point. Examine the 8-neighbor of b0 , starting a c0 and proceeding in a clockwise direction. Let b1 denote the first neighbor encountered whose value is 1, and let c1 be the (background) point immediately preceding b1 in the sequence. Store the locations of b0 and b1 for use in step 5.\
◦ 2-letb=b1 andc=c1 [Fig.11(c)].
◦ 3- let the 8-neighbors of b, starting at c and proceeding in a clockwise direction, be
denoted by n1, n2, … , n8. Find the first nk labeled 1.
◦ 4-letb=nk andc=nk-1
◦ 5- repeat steps 3 and 4 until b = b0 and the next boundary point found is b1. the sequence of b points found when the algorithm stops constitutes the set of ordered boundary points.
Boundary Descriptors – Example Algo
Boundary Description – Chain Codes
Boundary Description – Chain codes
Boundary Description – Shape Number
Polygons
Minimum-Perimeter Polygon (MPP)
Foundation: an approach for generating an algorithm to compute MPPs is to enclose a boundary [Fig. 11.6(a)] by a set of concatenated cells, as in (b).
Think of the boundary as a rubber band, as it is allowed to shrink, the rubber band will be constrained by the inner and outer walls of the bounding region defined by the cells.
Minimum-Perimeter Polygon (MPP)
More on Polygons – Merging
More on Polygons – Splitting
More on Polygons – Splitting Techniques
More on Polygons – Splitting
Boundary Description – Signature
A signature is a 1-D functional representation of a boundary and may be generated in various ways.
One of the simplest ways is to plot the distance from the centroid to the boundary.
Signatures depends on rotation and scaling.
Representing signatures depend on normalization with respect to rotation, by selecting the starting point which is the farthest from the centroid.
Boundary Description – Signature
We can represent boundary as a 2-D shape signature using angles
Boundary Description – Segments
Or using one shape as a matric over the other: here we use circle
Boundary Description – Segments
Decomposition a boundary into segments is often useful.
Decomposition reduces the boundary’s complexity and thus simplifies the
description process.
This process is particularly attractive when the boundary contains one or more significant concavities that carry shape information.
This approach depends o the convex hull algorithm : you count the number, length and distance between convex regions in a shape
Boundary Description – Eccentricity
Boundary Description – Min Rectangle Fitting
Boundary Description – Min Rectangle Fitting
Boundary Description – Matlab
Regional Descriptors
In this section we consider several approaches for describing image regions. Keep in mind that it is a common practice to use both boundary and regional descriptors combined.
Some Simple Descriptors: the area of a region is described as the number of pixels in the region.The perimeter of a region is the length of its boundary.Although area and perimeter are sometimes used as descriptors, they apply primarily to situations in which the size of the regions of interest is invariant.
Regional Descriptors
The shape of boundary segments (and of signature waveforms) can be described quantitatively by using statistical moments, such as the mean, variance, and higher order moments.
Regional Descriptors– Histograms
Regional Descriptors – Histograms
Regional Descriptors – Topological Measures
Topological properties are useful for global descriptions of regions in the image plane.
Simply defined, topology is the study of properties of a figure
For example Fig. 11.23 shows a region with two holes.Thus if a topological descriptor is defined by the number of holes in the region, this property obviously will not be affected by stretching or rotation transformation. In general, however, the number of holes will change if the region is torn or folded.
Regional Descriptors -Topological Measures
Regional Descriptors – Topological Measures
Another topological property useful for region description is the number of connected components. Fig.11.24 shows a region with three connected components.The number of holes H and connected components C in a figure can be used to define to Euler number E:
E =C–H
The Euler formula is used for computing connected components
Another example in Fig.11.26 shows a polygonal network. Classifying interior regions of such a network into faces and holes is often important. Denoting the number of vertices by V, the number of edges is Q, and the number of faces is F gives the following relationship, called the Euler formula:
V–Q+F =C-H
Regional Descriptors – Topological Measures
Regional Descriptors – Topological Measures
Regional Descriptors – Texture
An important approach to region description is to quantify its texture content.Although no formal definition of texture exists, intuitively, this descriptor provides measures of properties such as smoothness, coarseness, and regularity.
The three principal approaches used in image processing to describe the texture of a region are: statistical, structural, and spectral.
◦ Statistical approaches: yield characterizations of textures as smooth, coarse, grainy, and so on.
◦ Structural techniques: deal with the arrangement of image primitives, such as the description of texture
based on regularity spaced parallel lines.
◦ Spectral techniques: are based on the properties of the Fourier spectrum and are used primarily to detect global periodicity in an image by identifying high-energy, narrow peaks in the spectrum.
Regional Descriptors – Texture
Regional Descriptors – Structural and Spectral approaches
Structuralapproaches:Dealwiththearrangementofimageprimitives,such as the description of texture based on regularity spaced parallel lines.
Spectral: are based on the properties of the Fourier spectrum and are used primarily to detect global periodicity in an image by identifying high-energy, narrow peaks in the spectrum.
Regional Descriptors – Spectral Approaches
Structural Descriptors – Skeletons
Structural Descriptors – Skeletons
Structural Descriptors – Geodesic
Structural Descriptors – Geodesic
Summary
The Representation and description of objects or regions that have been segmented out of an image are early steps in the operation of most automated involving images.
These description for example, constitute the input to the object recognition methods developed in the following chapter.
Hints!