2020/8/14 COMP9444 Exercise 8 Solutions
COMP9444 Neural Networks and Deep Learning Term 2, 2020
Solutions to Exercise 8: Hopfield Networks This page was last updated: 07/27/2020 11:29:21
1.
a. Compute the weight matrix for a Hopfield network with the two memory vectors
[1, –1, 1, –1, 1, 1] and [1, 1, 1, –1, –1, –1] stored in it. The outer product W1 of [1, –1, 1, –1, 1, 1] with itself is
0–1 1–1 1 1 –1 0–1 1–1–1 1–1 0–1 1 1 –1 1–1 0–1–1 1–1 1–1 0 1 1–1 1–1 1 0
The outer product W2 of [1, 1, 1, –1, –1, –1] with itself is
0 1 1–1–1–1 1 0 1–1–1–1 1 1 0–1–1–1
–1–1–1 0 1 1 –1–1–1 1 0 1 –1–1–1 1 1 0
The weight matrix W is (1/6)×(W1 + W2) = (1/3)×
0 0 1–1 0 0 0 0 0 0–1–1 1 0 0–1 0 0
–1 0–1 0 0 0 0–1 0 0 0 1 0–1 0 0 1 0
b. Confirm that both these vectors are stable states of this network.
sgn(W.[1, –1, 1, –1, 1, 1]) = sgn((2/3)×[1, –1, 1, –1, 1, 1]) = [1, –1, 1, –1, 1, 1]
so this one is stable. Similarly,
sgn(W.[1, 1, 1, –1, –1, –1]) = sgn((2/3)×[1, 1, 1, –1, –1, –1])
https://www.cse.unsw.edu.au/~cs9444/20T2/tut/sol/Ex8_Hopfield_sol.html 1/2
2020/8/14 COMP9444 Exercise 8 Solutions
= [1, 1, 1, –1, –1, –1]
so this one is stable too.
2. Consider the following weight matrix W:
0.0 –0.2 0.2 –0.2 –0.2 –0.2 0.0 –0.2 0.2 0.2 0.2 –0.2 0.0 –0.2 –0.2 –0.2 0.2 –0.2 0.0 0.2 –0.2 0.2 –0.2 0.2 0.0
a. Starting in the state [1, 1, 1, 1, –1], compute the state flow to the stable state using asynchronous updates.
W.[1, 1, 1, 1, –1] = [0, -0.4, 0, -0.4, 0]. Hence:
If neuron 1, 3, or 5 updates first, its total net input is 0, so it does not change state;
If neuron 2 updates first, its total net input is -0.4, and it’s current value is +1, so it changes state to –1, and the new state is [1, –1, 1, 1, –1]. Call this Case A.
If neuron 4 updates first, its total net input is -0.4, and it’s current value is one, so it changes state to –1, and the new state is [1, 1, 1, –1, –1]. Call this Case B.
Case A: W.[1, –1, 1, 1, –1] = [0.4, –0.4, 0.4, –0.8, –0.4]. Hence:
If neurons 1, 2, 3, or 5 update first, there is no state change.
If neuron 4 updates first, it flips, and the new state is [1, –1, 1, –1, –1]. W.[1, –1, 1, –1, –1] = [0.8, –0.8, 0.8, –0.8, –0.8]. So no matter which neuron updates, there is no change. This is a stable state.
Case B: W.[1, 1, 1, –1, –1] = [0.4, –0.8, 0.4, –0.4, –0.4]. Hence:
If neurons 1, 3, 4 or 5 update first, there is no state change.
If neuron 2 updates first, it flips, and the new state is [1, –1, 1, –1, –1].
This is the same state as that reached in case A, and as seen in case A, it is a stable state.
b. Starting in the (same) state [1, 1, 1, 1, –1], compute the next state using synchronous updates.
W.[1, 1, 1, 1, –1] = [0, -0.4, 0, -0.4, 0], so neurons 2 and 4 flip, resulting in a state of
[1, –1, 1, –1, –1]. (We know from the previous part that this is a stable state.)
https://www.cse.unsw.edu.au/~cs9444/20T2/tut/sol/Ex8_Hopfield_sol.html 2/2