CS计算机代考程序代写 The University of Sydney Page 70

The University of Sydney Page 70

Extension

The University of Sydney Page 71

Zeiler and Fergus, ECCV 2014
– A non-parametric view of invariance, showing which patterns from the

training set activate the feature map
– Using deconvolution to project the feature activations back to the input

pixel space

Visualize features

Input image

Feedforward (classification)

Pick a single intermediate neuron, e.g. one
value in 128 x 13 x 13 conv5 feature map

Deconvolution (visualization)
Compute gradient of neuron value with respect to image pixels

The University of Sydney Page 72

Deconvolutional Method

(Zeiler and Fergus, 2014)

– For one convolutional layer.
F

eed
-forw

ard

B
ack

w
ard

The University of Sydney Page 73

Deconvolutional Method

– Conv and Deconv
– Use learned filters {-} to convolve forward signal, but use transposed

versions {-⊺} for backward signal.

(Zeiler and Fergus, 2014)

)!,# )!,$
)#,! )#,# )#,$
)$,! )$,# )$,$

)!,!
*!,!% *!,#% *!,$% *!,&%

*#,!% *#,#% *#,$% *#,&%

*$,!% *$,#% *$,$% *$,&%

*&,!% *&,#% *&,$% *&,&%

*!,!%’! *!,#%’!

*#,!%’! *#,#%’!

+⊺×(*!,!%)!, ⋯ , *#,#%)!) ⊺= (*!,!% , ⋯ , *&,&% ) ⊺

+%)!×(*!,!%)!, ⋯ , *&,&%)!) ⊺= (*!,!% , ⋯ , *#,#% ) ⊺

The University of Sydney Page 74

Deconvolutional Method

– Activation function
– Both feedforward and backward processes employ ReLU as activation

functions to ensure feature maps are always positive.

(Zeiler and Fergus, 2014)

EFGH 8 = max(0, 8)

The University of Sydney Page 75
(Zeiler and Fergus, 2014)

Deconvolutional Method
– Pooling and Unpooling

– Despite the non-invertibility of max pooling, an approximate inverse
could be obtained by recording the locations of the maxima within each
pooling region in a set of switch variables.

The University of Sydney Page 76

Visualize features

– Instead of showing the single strongest activation for a given
feature map, the article shows the top 9 activations layer 2-5.

– Alongside these visualizations are the corresponding image
patches.

(Zeiler and Fergus, 2014)

The University of Sydney Page 77

Visualize features

(Zeiler and Fergus, 2014)

– Layer 2 responds to corners and other edge/color conjunctions.
– Layer 3 has more complex invariances, capturing similar textures (e.g.

mesh patterns).
– Layer 4 shows significant variation, but is more class-specific.
– Layer 5 shows entire objects with significant pose variation.

The University of Sydney Page 78

– Occlude images at different locations and visualize feature
activations and classifier confidence

Occlusion Experiments

(Zeiler and Fergus, 2014)

(a) Input Image
(b) Layer 5,

strongest feature map
(c) Classier, probability

of correct class

The University of Sydney Page 79

Variations of CNNs

The University of Sydney Page 80

– Transposed Convolution
– Also named “Deconvolution”, “up-convolution” …
– Usually, the output is an image
– Different from the above ‘Deconvolutional method’ for

visualizing CNNs learned features

Transposed Convolution

Input images 3 Transformed 1(3)

Convolution Transposed Convolution

The University of Sydney Page 81

– Applications
– Generative model, encoder-decoder, image editing, image

transformation …

Transposed Convolution

(Chaoyue et al, 2017)

The University of Sydney Page 82

– Compare to aforementioned convolution operation, can
we go the other way around, i.e., map from a 4-dim space
to a 16-dim space, while keeping the connectivity pattern
of the original convolution.

Transposed Convolution

Convolution operation

%!,# %!,$

%#,! %#,# %#,$

%$,! %$,# %$,$

%!,!
&!,!%&! &!,#%&! &!,$%&! &!,’%&!

&#,!%&! &#,#%&! &#,$%&! &#,’%&!

&$,!%&! &$,#%&! &$,$%&! &$,’%&!

&’,!%&! &’,#%&! &’,$%&! &’,’%&!

&!,!% &!,#%

&#,!% &#,#%

3,-% 4,-% (6,-%) 3,

The University of Sydney Page 83

– Compare to aforementioned convolution operation, can
we go the other way around, i.e., map from a 4-dim space
to a 16-dim space, while keeping the connectivity pattern
of the original convolution.

Transposed Convolution

&%,# &%,$

&#,% &#,# &#,$

&$,% &$,# &$,$

&%,%
8%,%
, 8%,#

, 8%,$
, 8%,.

,

8#,%
, 8#,#

, 8#,$
, 8#,.

,

8$,%
, 8$,#

, 8$,$
, 8$,.

,

8.,%
, 8.,#

, 8.,$
, 8.,.

,

8%,%
,-% 8%,#

,-%

8#,%
,-% 8#,#

,-%

Transposed Convolution

The University of Sydney Page 84

Transposed Convolution

1 0 −1

−1 0 2

0 0 1

0 −1

−1 0 2

0 0 1

1
1 3

-2 -1

1 0 −1

−1 0 2

0 0 1

The University of Sydney Page 85

Transposed Convolution

1 0 −1

−1 0 2

0 0 1

0 −1

−1 0 2

0 0 1

1
1 3

-2 -1

1 0 −1

−1 0 2

0 0 1

3 0 −3

−3 0 6

0 0 3

−3

6

3

1 3 −1

−1 −3 2

0 0 1

The University of Sydney Page 86

Transposed Convolution

1 0 −1

−1 0 2

0 0 1

0 −1

−1 0 2

0 0 1

1
1 3

-2 -1

3 0 −3

−3 0 6

0 0 3

−2 0 2

2 0 −4

0 0 −2

−3

6

3

1 3 −1

−1 −3 2

0 0 1

−3

6

3

1 3 −1

−3 −3 4

2 0 −3

0 0 −1

The University of Sydney Page 87

Transposed Convolution

1 0 −1

−1 0 2

0 0 1

0 −1

−1 0 2

0 0 1

1
1 3

-2 -1

3 0 −3

−3 0 6

0 0 3

−2 0 2

2 0 −4

0 0 −2

−1 0 1

1 0 −2

0 0 −1

−3

6

3

1 3 −1

−3 −3 4

2 0 −3

0 0 −1

−3

7

1

1 3 −1

−3 −4 4

2 1 −3

0 0 −1 2

The University of Sydney Page 88

Stride = 1

Transposed Convolution
– Stride

0 −1

−1 0 2

0 0 1

1
1 3

-2 -1

1 0 −1

−1 0 2

0 0 1

−3

6

3

1 3 −1

−1 −3 2

0 0 1

The University of Sydney Page 89

Stride = 2

Transposed Convolution
– Stride

0 −1

−1 0 2

0 0 1

1
1 3

-2 -1

1 0 −1

−1 0 2

0 0 1

0

0

0

1 0 2

−1 0 −1

0 0 1

−3

6

3

The University of Sydney Page 90

Transposed Convolution
– Cropping

0 −1

−1 0 2

0 0 1

1
1 3

-2 -1

0

0

0

1 0 2

−1 0 −1

−2 0 2

−3

6

4

0

0

1 0 −4

0 0 −1

4

2

The University of Sydney Page 91

Crop = 1

Transposed Convolution
– Cropping

0 −1

−1 0 2

0 0 1

1
1 3

-2 -1

0

0

0

1 0 2

−1 0 −1

−2 0 2

−3

6

4

0

0

1 0 −4

0 0 −1

4

2

−1 0

0 2 0

0 −4 0

0

The University of Sydney Page 92

Transposed Convolution
– Output Size

Output Size = ; − 1 < + = − 2>

N

# Outpu
t Siz

e

C

C

The University of Sydney Page 93

%!,# %!,$

%#,! %#,# %#,$

%$,! %$,# %$,$

%!,!

Convolution as a matrix operation

&!,!%&! &!,#%&! &!,$%&! &!,’%&!

&#,!%&! &#,#%&! &#,$%&! &#,’%&!

&$,!%&! &$,#%&! &$,$%&! &$,’%&!

&’,!%&! &’,#%&! &’,$%&! &’,’%&!

&!,!% &!,#%

&#,!% &#,#%

-()”×(%”,”
()”, ⋯ , %+,+

()”) ⊺= (%”,”
( , ⋯ , %#,#

( ) ⊺

– Recall the matrix operation of original convolution

(!”# =

*#,# *#,% *#,& 0 *%,# *%,% *%,& 0
0 *#,# *#,% *#,& 0 *%,# *%,% *%,&

*&,# *&,% *&,& 0 0 0 0 0
0 *&,# *&,% *&,& 0 0 0 0

0 0 0 0 *#,# *#,% *#,& 0
0 0 0 0 0 *#,# *#,% *#,&

*%,# *%,% *%,& 0 *&,# *&,% *&,& 0
0 *%,# *%,% *%,& 0 *&,# *&,% *&,&

3,-% 4,-% (6,-%) 3,

The University of Sydney Page 94

Transposed Convolution as a matrix operation

4 =

6!,! 6!,# 6!,$ 0 6#,! 6#,# 6#,$ 0
0 6!,! 6!,# 6!,$ 0 6#,! 6#,# 6#,$

6$,! 6$,# 6$,$ 0 0 0 0 0
0 6$,! 6$,# 6$,$ 0 0 0 0

0 0 0 0 6!,! 6!,# 6!,$ 0
0 0 0 0 0 6!,! 6!,# 6!,$

6#,! 6#,# 6#,$ 0 6$,! 6$,# 6$,$ 0
0 6#,! 6#,# 6#,$ 0 6$,! 6$,# 6$,$

– Given the same matrix ?, transposed convolution can be represented as:

Forward pass of transposed convolution

-⊺×(%”,”
()”, ⋯ , %#,#

()”) ⊺= (%”,”
( , ⋯ , %+,+

( ) ⊺

&%,# &%,$

&#,% &#,# &#,$

&$,% &$,# &$,$

&%,%
8%,%
, 8%,#

, 8%,$
, 8%,.

,

8#,%
, 8#,#

, 8#,$
, 8#,.

,

8$,%
, 8$,#

, 8$,$
, 8$,.

,

8.,%
, 8.,#

, 8.,$
, 8.,.

,

8%,%
,-% 8%,#

,-%

8#,%
,-% 8#,#

,-%

The University of Sydney Page 95

Max pooling & Up-pooling
Max pooling Up-pooling

– Pooling ops: max(:) – Fill numbers to pre −define
d position, others fill zeros.

-1 4 1 2

0 1 3 -2

1 5 -2 6

3 -1 -2 -2

4 3

5 6

Input

O
utput

4 0 3 0

0 0 0 0

5 0 6 0

0 0 0 0

4 3

5 6

O
utput

Input

The University of Sydney Page 96

– From features to image
– Kernel: 1 → K×L
– Cropping
– ⋯ → Filtering → Activate

function → Up−pooling → ⋯

Transposed Convolution

C
onvolution

T
ransposed

C
onvolution

Convolution Transposed convolution

– From image to features
– Kernel: K×L → 1
– Padding
– ⋯ → Filtering →Activate

function → Pooling → ⋯

https://github.com/vdumoulin/conv_arithmetic

The University of Sydney Page 97

Thank you!