3/18/2021
CSE 473/573
Introduction to Computer Vision and Image Processing
‘-
OBJECTS AND SCENES
Questions regarding anything?
‘-
1
3/18/2021
The texture
The object
‘-
The scene
Why be concerned about the difference? • Features to be extracted?
• Context for Recognition?
• Various categories have features that can be essential for interpretation.
‘-
4
2
3/18/2021
An example of categorical perception • Continuous perception: graded response
• Categorical perception: “sharp” boundaries
Many perceptual phenomena are a mixture of the two: categorical at an everyday level of magnification, but continuous at a more microscopic level. It can also depend on cultural aspects, expertise, task, attention, …
‘-
Bar Trivia
• How many colors can humans see?
• 3 types of cones
• Sensitive to red, green, and blue wavelengths of light.
‘-
• Each can detect about 100 different shades • A million (103) different color possibilities
• Some people have a 4th cone (tetrachromats)
3/18/2021 6
3
3/18/2021
Colors are perceivable, but often linguistic
• Note: Name of Colors are Cultural
• Named for things: “ripe fruit”, “blood”
• Single word for green/blue, or red/yellow
‘-
Why do we care about categories?
Perception of function:
• We can perceive the 3D shape, texture, material
properties, without knowing about objects.
• But, the concept of category encapsulates also
‘-
information about what can we do with those objects.
“We therefore include the perception of function as a proper –indeed, crucial- subject for vision science”, from Vision Science, chapter 9, Palmer.
4
3/18/2021
Why do we care about categories?
When we recognize an object we can make predictions about its behavior in the future, beyond of what is immediately perceived.
‘-
The perception of function • Direct perception (affordances): Gibson
‘-
Flat surface Horizontal Knee-high …
Sittable upon
• Mediated perception (Categorization)
One caveat of this comparison: deciding that something is a chair might require access to more features than the ones needed to decide that we can sit on something… (it is a different level of categorization)
Chair
Chair?
Flat surface Horizontal Knee-high …
Sittable upon
Chair
Chair
5
3/18/2021
Direct perception
Some aspects of an object function can be perceived directly
• Functional form: Some forms clearly indicate to a function (“sittable-upon”, container, cutting device, …)
‘-
Sittable-upon Sittable-upon It does not seem easy to sit-upon this…
Sittable-upon
Limitations of Direct Perception
Objects of similar structure might have very different functions
‘-
Not all functions seem to be available from direct visual information only.
The functions are the same at some level of description: we can put things inside in both and somebody will come later to empty them. However, we are not expected to put inside the same kinds of things…
6
3/18/2021
Indirect perception of function by categorization
Well… this requires object recognition (Later….)
‘-
Which level of categorization is the right one?
Car is an object composed of:
a few doors, four wheels (not all visible at all times), a roof, front lights, windshield
‘-
If you are thinking in buying a car, you might want to be a bit more specific about your categorization.
7
3/18/2021
Entry-level categories
• Typical member of a basic-level category are
categorized at the expected level
• Atypical members tend to be classified at a subordinate
level.
A bird (Jolicoeur, Gluck, Kosslyn 1984)
‘-
An ostrich
Object recognition: Is it really so hard?
Find the chair in this image
Output of normalized correlation
‘-
This is a chair
8
3/18/2021
Object recognition Is it really so hard?
Find the chair in this image
‘-
Pretty much garbage
Template matching is not going to make it
The texture
The object
‘-
The scene
9
3/18/2021
What do we perceive?
‘-
‘-
10
3/18/2021
PERSON
LAKE
PATH
PERSON
SKY
BENCH
DUCK DUCK
TREE PERSON
‘-
DUCK PERSON
GRASS
SIGN
A VIEW OF A PARK ON A NICE SPRING DAY
‘-
11
3/18/2021
PEOPLE WALKING IN THE PARK
‘-
Do not PERSON FEEDING feed DUCKS IN THE PARK
the ducks DUCKS LOOKING FOR FOOD sign
PEOPLE UNDER THE SHADOW OF THE TREES
‘-
DUCKS ON TOP OF THE GRASS
12
3/18/2021
Scene views vs. objects
‘-
Scene: a place in which a human can act within,
or a place to which a human being could navigate.
A lot more than just a combination of objects (just as objects are more than the combinations of their parts). Like objects, scenes are associated with specific functions and behaviors, such as eating in a restaurant, drinking in a pub, reading in a library, and sleeping in a bedroom.
Scene views vs. objects
‘-
13
3/18/2021
Mary Potter (1976)
Mary Potter (1975, 1976) demonstrated that during a rapid sequential visual presentation (100 msec per image), a novel picture is instantly understood and observers seem to comprehend a lot of visual information
‘-
Demo : Rapid image understanding
By Aude Oliva
Instructions: 9 photographs will be shown for half a second each. Your task is to memorize these pictures
‘-
14
3/18/2021
GET READY
GET
‘-
Memory Test
Which of the following pictures have you seen ?
Poll will be displayed for each image
‘-
3/18/2021
30
15
3/18/2021
‘-
Have you seen this picture ?
‘-
NO
16
3/18/2021
‘-
Have you seen this picture ?
NO
‘-
17
3/18/2021
‘-
Have you seen this picture ?
Yes
‘-
18
3/18/2021
‘-
Have you seen this picture ?
NO
‘-
19
3/18/2021
‘-
Have you seen this picture ?
NO
‘-
20
3/18/2021
‘-
Have you seen this picture ?
NO
‘-
21
3/18/2021
You have seen these pictures
You were tested with these pictures
‘-
The gist of the scene
In a glance, we remember the meaning of an image and its global layout but some objects and details are forgotten
‘-
22
3/18/2021
Summary of Results
Responses
Yes
No
I Don’t Know
‘-
3/18/2021 45
Recognition Challenges / Overview
‘-
23
3/18/2021
Object Categorization • How to recognize ANY car
• How to recognize ANY cow
‘-
K. Grauman, B. Leibe
47
Challenges: robustness
Illumination
Occlusions
‘-
Object pose
Intra-class appearance
Clutter
Viewpoint
24
3/18/2021
Challenges: robustness
• Detection in Crowded Scenes • Learn object variability
‘-
• Changes in appearance, scale, and articulation • Compensate for clutter, overlap, and occlusion
K. Grauman, B. Leibe
Challenges: context and human experience
‘-
K. Grauman, B. Leibe
25
3/18/2021
‘-
3/18/2021
51
Challenges: context and human experience
‘-
Context cues
Image credit: D. Hoeim
26
3/18/2021
Challenges: learning with minimal supervision
Less
More
‘
–
K. Grauman, B. Leibe
‘-
Slide from Pietro Perona, 2004 Object Recognition workshop
27
3/18/2021
‘-
Slide from Pietro Perona, 2004 Object Recognition workshop
Rough evolution of focus in recognition research
‘-
1980s 1990s to early 2000s
2000-2010…
28
3/18/2021
Today
‘-
Inputs/outputs/assumptions
What is the goal?
• Say yes/no as to whether an object present in image
And/or:
• Determine pose of an object, e.g. for robot to grasp • Categorize all objects
• Forced choice from pool of categories
• Bounding box on object
• Full segmentation
• Build a model of an object category
‘-
29
3/18/2021
Next Time • Segmentation
‘-
3/18/2021
59
30