3/16/2021
CSE 473/573
Introduction to Computer Vision and Image Processing
‘-
TEXTURES
Questions regarding anything? (Other then what textures are…)
All remaining homework’s will be delayed one week for assignment and due date
No office hours today. Wed 8-9 and 6-7
‘-
1
3/16/2021
Texture
‘-
What defines a texture?
What is Texture? • No Formal Definition
• There will be signficant variation in intensity levels between pixels
• these variations perform repetitive patterns-homogeneous at
some scale
• local statistics are constant and slowly varying
• Human visual systems perceived textures as homogeneous region even though they don’t have her intensity
‘-
4
2
3/16/2021
Includes: regular patterns • We interpret this first image as a brick wall
‘-
Includes: random patterns
‘-
3
3/16/2021
Texture
• Is a property of a “group of pixels”/Area
• a single pixel does not have texture
• It is scale dependent
• at different scales texture will have different properties ‘-
• It contains a large number of possibly countless primitive objects
• it involves the spatial distribution of intensities • 2D histograms
• Co-occurrence matrices
Source: Gaborski
7
Scale
‘-
8
4
3/16/2021
How would you describe a Texture?
• Coarseness
• Roughness
• Direction
• Frequency
• Uniformity
• Density
‘-
• How would you describe dog fur or cat fur or wood grain or cloth?
9
When are two textures similar?
‘-
All these images are different instances of the same texture We can differentiate between them, but they seem generated by the same process
10
5
3/16/2021
The Goal of Texture Synthesis
input image
SYN‘- THESIS True (infinite) texture generated image
• Given a finite sample of some texture, the goal is to synthesize other samples from that same texture
• The sample needs to be “large enough“
11
The Goal of Texture Analysis
input image
ANALYSIS
“Same” or “different”
True (infinite) texture
generated image
‘-
Compare textures and decide if they’re made of the same “stuff”.
6
3/16/2021
The trivial texture synthesis algorithm
‘-
13
Texture synthesis and representation
Set of equivalent textures
‘-
Space of all images
Set of equivalent textures: generated by exactly the same physical process
14
7
3/16/2021
Texture synthesis and representation
Set of equivalent textures
‘-
Set of perceptually equivalent textures
Space of all images
Set of equivalent textures: generated by exactly the same physical process
Set of perceptually equivalent textures: “well, they just look the same to m1e5”
Scale: objects vs. texture
‘-
Often the same thing in the world can occur as texture or an object, depending on the scale we are considering.
8
3/16/2021
Why analyze texture?
Importance to perception:
• Often indicative of a material’s properties
• Can be important appearance cue, especially if shape is
similar across objects
• Aim to distinguish between shape, boundaries, and texture
Technically:
• Representation-wise, we want a feature one step above “building blocks” of filters, edges.
‘-
‘-
9
3/16/2021
Texture-related tasks • Shape from texture
• Estimate surface orientation or shape from image texture ‘-
Shape from texture
• Use deformation of texture from point to point to
estimate surface shape
‘-
Pics from A. Loh: http://www.csse.uwa.edu.au/~angie/phdpics1.html
10
3/16/2021
‘-
Texture-related tasks
• Shape from texture
• Estimate surface orientation or shape from image texture
• Segmentation/classification from texture cues ‘-
• Analyze, represent texture
• Group image regions with consistent texture
• Synthesis
• Generate new texture patches/images given some examples
11
3/16/2021
‘-
23
‘-
http://animals.nationalgeographic.com/
24
12
3/16/2021
Color vs. texture
‘-
Recall: These looked very similar in terms of their color distributions (when our features were R-G-B)
But how would their texture distributions compare?
Psychophysics of texture
• Some textures distinguishable with preattentive perception– without scrutiny, eye movements [Julesz 1975]
‘-
Watch the screen very carefully. Are the left and right sides:
Same or different?
13
3/16/2021
‘-
27
‘-
28
14
3/16/2021
‘-
Julesz
• Textons: analyze the texture in terms of statistical relationships between fundamental texture elements, called “textons”.
‘-
• It generally required a human to look at the texture in order to decide what those fundamental units were…
15
3/16/2021
Texture representation
• Textures are made up of repeated local patterns, so: • Find the patterns
‐ Use filters that look like patterns (spots, bars, raw patches…)
‐ Consider magnitude of response ‘-
• Describe their statistics within each local window
‐ Mean, standard deviation
‐ Histogram
‐ Histogram of “prototypical” feature occurrences
‘-
32
16
3/16/2021
Texture representation:
mean d/dx value
mean d/dy value
Win. #1
4
10
‘-
original image
derivative filter responses, squared
statistics to summarize patterns in small windows
mean d/dx value
mean d/dy value
Win. #1
4
10
‘W- in.#2
18
7
original image
derivative filter responses, squared
statistics to summarize patterns in small windows
17
……
3/16/2021
mean d/dx value
mean d/dy value
Win. #1
4
10
‘W- in.#2
18
7
Win.#9
20
20
original image
derivative filter responses, squared
statistics to summarize patterns in small windows
Texture representation: example
mean d/dx value
mean d/dy value
Win. #1
4
10
‘W- in.#2
18
7
Win.#9
20
20
Dimension 1 (mean d/dx value)
statistics to summarize patterns in small windows
18
Dimension 2 (mean d/dy value)
…… ……
3/16/2021
Windows with primarily horizontal edges
Both
Dimension 2 (mean d/dy value)
Win. #1 ‘ W- i n . # 2 Win.#9
mean mean d/dx d/dy value value
4 10 1 8 7 20 20
statistics to summarize patterns in small windows
… …
Dimension 1 (mean d/dx value)
Windows with small gradient in both directions
Windows with primarily vertical edges
‘-
original image
visualization of the assignment to texture “types”
derivative filter responses, squared
19
3/16/2021
Texture representation: example
Dimension 2 (mean d/dy value)
Far: dissimilar textures
Close: similar textures
Dimension 1 (mean d/dx value)
Win. #1 ‘ W- i n . # 2 Win.#9
mean mean d/dx d/dy value value
4 10 1 8 7 20 20
statistics to summarize patterns in small windows
… …
Texture representation: window scale
• We’re assuming we know the relevant window size
for which we collect these statistics.
‘-
Possible to perform scale selection by looking for window scale where texture description not changing.
20
3/16/2021
Filter banks
• Our previous example used two filters, and resulted in a 2-dimensional feature vector to describe texture in a window.
• x and y derivatives revealed something about local structure.
• We can generalize to apply a collection of multiple (d) filters: a “filter bank”
• Then our feature vectors will be d-dimensional. • still can think of nearness, farness in feature space
‘-
d-dimensional features
‘-
…
2d
3d
21
3/16/2021
Filter banks
• What filters to put in the bank?
• Typically we want a combination of scales and orientations, different types of patterns.
orientations
‘-
scales
http://www.robots.ox.ac.uk/~vgg/research/texclass/filters.html
Pixel Neighborhood-based Feature
• The most important for texture analysis is to describe the spatial behavior of intensity values in any given neighborhood.
‘-
• Different methodologies have been proposed.
• Local binary pattern (LBP) is one of the most-widely
used approach – mainly for face recognition.
• LBP is used for texture analysis too.
44
22
3/16/2021
Local Binary Pattern (LBP)
For each PIXEL of an image, a BINARY CODE is produced to make a new matrix with the new value (binary to decimal value).
𝑃1 ‘-𝑝 𝐿𝐵𝑃𝑝,𝑟𝑁𝑐𝑔𝑁𝑝 𝑁𝑐2
𝑝0
where,
neighborhood pixels 𝑁 in each block
is thresholded by its center pixel value 𝑁
p sampling points (e.g., p = 0, 1, …, 7 for a 3×3 cell, where P = 8) rradius (for 3×3 cell, it is 1).
Binary threshold function 𝑔 𝑥 is,
𝑔𝑥 0, 𝑥0 1, 𝑥0
Computation of Local Binary Pattern
Binary code for > Nc
0
1
0
2
0
128 ‘-
?
0
0
0
16
3
7
2
8
4
1
2
3
5
Neighborhood of a gray-scale image
Representation
LBP
146
46
1
64
1
Nc
2
LBP Pattern
0
0
0
0
1
Component-wise multiplication
4
128
32
8
16
Example of how the LBP operator works
∑
Sum
23
3/16/2021
Computation of LBP
(LSB)
0
‘-
Binary Pattern:
1 (MSB)
1
1
1
0
0
0
1
(LSB)
Code/Weight 2 :
1 x 27
1 x 26
1 x 25
1 x 24
0 x 23
0x 22
0x 21
1 x 20
= 128
= 64
= 32
= 16
=0
=0
=0
=1
LBP:
1+0+0+0+16+32+64+128 = 241
47
LBP
• One the best performing texture descriptors
• A label is assigned to every pixel
• Use center pixel value to threshold the 3×3 neighborhood • Result in binary number ‘-
• Histogram of the labels is used as a texture descriptor
The Basic LBP operator
48
24
3/16/2021
LBP
• LBP is extended to use different sizes of neighborhoods.
• Local neighborhoods is defined as a set of sampling points.
• points evenly spaced on a circle centered at the labeled pixel.
• (P,R) , P = number of sampling points , R = radius ‘-
• Bilinear interpolation is used if sampling point is not in the center of the pixel.
The circular (8,1), (16,2) and (8,2) neighborhoods. The pixel values are bilinearly interpolated whenever the sampling point is not in the center of a pixel.
49
‘-
25
3/16/2021
‘-
LBP Example from OpenCV
• Notice how LBP feature are illumination invariant
‘-
52
26
3/16/2021
Uniform LBP
• Uniform patterns to further improve LBP.
• Uniform patterns has at most 2 bitwise transitions in
‘-
binary pattern.
• Histogram assigns separate bin for every uniform pattern.
• Histogram assigns a single bin for all non-uniform pattern.
53
Uniform LBP (2)
• Uniform patterns examples 00000000 (0 transitions) 01110000 (2 transitions) 11001111 (2 transitions)
• Non-uniformpatternsexamples 11001001 (4 transitions) 01010011 (5 transitions)
• 59 bins histogram Length of feature vector
‘-
54
27
3/16/2021
Other Measures of Texture (frequency)
‘-
55
‘-
56
28