INFO:Output for tiny dataset:Epoch 0
INFO:Output for tiny dataset: SGD with respect to sample 0
INFO:Output for tiny dataset: Begin forward pass
INFO:Output for tiny dataset: Value of a (before sigmoid):
[[0. 0. 0. 0.]]
INFO:Output for tiny dataset: Value of z (after sigmoid):
[[1. 0.5 0.5 0.5 0.5]]
INFO:Output for tiny dataset: Value of b (before softmax):
[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]
INFO:Output for tiny dataset: Value of y_hat (after softmax):
[[0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1]]
INFO:Output for tiny dataset: Cross entropy: 2.3025850929940455
INFO:Output for tiny dataset: Begin backward pass
INFO:Output for tiny dataset: d(loss)/d(b):
[[ 0.1 0.1 0.1 0.1 0.1 0.1 0.1 -0.9 0.1 0.1]]
INFO:Output for tiny dataset: d(loss)/d(beta):
[[ 0.1 0.05 0.05 0.05 0.05]
[ 0.1 0.05 0.05 0.05 0.05]
[ 0.1 0.05 0.05 0.05 0.05]
[ 0.1 0.05 0.05 0.05 0.05]
[ 0.1 0.05 0.05 0.05 0.05]
[ 0.1 0.05 0.05 0.05 0.05]
[ 0.1 0.05 0.05 0.05 0.05]
[-0.9 -0.45 -0.45 -0.45 -0.45]
[ 0.1 0.05 0.05 0.05 0.05]
[ 0.1 0.05 0.05 0.05 0.05]]
INFO:Output for tiny dataset: d(loss)/d(z):
[[0. 0. 0. 0.]]
INFO:Output for tiny dataset: d(loss)/d(a):
[[0. 0. 0. 0.]]
INFO:Output for tiny dataset: d(loss)/d(alpha):
[[0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0.]]
INFO:Output for tiny dataset: Update weights
INFO:Output for tiny dataset: New alpha:
[[0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0.]]
INFO:Output for tiny dataset: New beta:
[[-0.01 -0.005 -0.005 -0.005 -0.005]
[-0.01 -0.005 -0.005 -0.005 -0.005]
[-0.01 -0.005 -0.005 -0.005 -0.005]
[-0.01 -0.005 -0.005 -0.005 -0.005]
[-0.01 -0.005 -0.005 -0.005 -0.005]
[-0.01 -0.005 -0.005 -0.005 -0.005]
[-0.01 -0.005 -0.005 -0.005 -0.005]
[ 0.09 0.045 0.045 0.045 0.045]
[-0.01 -0.005 -0.005 -0.005 -0.005]
[-0.01 -0.005 -0.005 -0.005 -0.005]]
INFO:Output for tiny dataset: SGD with respect to sample 1
INFO:Output for tiny dataset: Begin forward pass
INFO:Output for tiny dataset: Value of a (before sigmoid):
[[0. 0. 0. 0.]]
INFO:Output for tiny dataset: Value of z (after sigmoid):
[[1. 0.5 0.5 0.5 0.5]]
INFO:Output for tiny dataset: Value of b (before softmax):
[[-0.02 -0.02 -0.02 -0.02 -0.02 -0.02 -0.02 0.18 -0.02 -0.02]]
INFO:Output for tiny dataset: Value of y_hat (after softmax):
[[0.09783393 0.09783393 0.09783393 0.09783393 0.09783393 0.09783393
0.09783393 0.11949463 0.09783393 0.09783393]]
INFO:Output for tiny dataset: Cross entropy: 2.324483831536847
INFO:Output for tiny dataset: Begin backward pass
INFO:Output for tiny dataset: d(loss)/d(b):
[[ 0.09783393 0.09783393 0.09783393 0.09783393 0.09783393 0.09783393
0.09783393 0.11949463 0.09783393 -0.90216607]]
INFO:Output for tiny dataset: d(loss)/d(beta):
[[ 0.09783393 0.04891696 0.04891696 0.04891696 0.04891696]
[ 0.09783393 0.04891696 0.04891696 0.04891696 0.04891696]
[ 0.09783393 0.04891696 0.04891696 0.04891696 0.04891696]
[ 0.09783393 0.04891696 0.04891696 0.04891696 0.04891696]
[ 0.09783393 0.04891696 0.04891696 0.04891696 0.04891696]
[ 0.09783393 0.04891696 0.04891696 0.04891696 0.04891696]
[ 0.09783393 0.04891696 0.04891696 0.04891696 0.04891696]
[ 0.11949463 0.05974732 0.05974732 0.05974732 0.05974732]
[ 0.09783393 0.04891696 0.04891696 0.04891696 0.04891696]
[-0.90216607 -0.45108304 -0.45108304 -0.45108304 -0.45108304]]
INFO:Output for tiny dataset: d(loss)/d(z):
[[0.00597473 0.00597473 0.00597473 0.00597473]]
INFO:Output for tiny dataset: d(loss)/d(a):
[[0.00149368 0.00149368 0.00149368 0.00149368]]
INFO:Output for tiny dataset: d(loss)/d(alpha):
[[0.00149368 0. 0. 0. 0.00149368 0. ]
[0.00149368 0. 0. 0. 0.00149368 0. ]
[0.00149368 0. 0. 0. 0.00149368 0. ]
[0.00149368 0. 0. 0. 0.00149368 0. ]]
INFO:Output for tiny dataset: Update weights
INFO:Output for tiny dataset: New alpha:
[[-0.00014937 0. 0. 0. -0.00014937 0. ]
[-0.00014937 0. 0. 0. -0.00014937 0. ]
[-0.00014937 0. 0. 0. -0.00014937 0. ]
[-0.00014937 0. 0. 0. -0.00014937 0. ]]
INFO:Output for tiny dataset: New beta:
[[-0.01978339 -0.0098917 -0.0098917 -0.0098917 -0.0098917 ]
[-0.01978339 -0.0098917 -0.0098917 -0.0098917 -0.0098917 ]
[-0.01978339 -0.0098917 -0.0098917 -0.0098917 -0.0098917 ]
[-0.01978339 -0.0098917 -0.0098917 -0.0098917 -0.0098917 ]
[-0.01978339 -0.0098917 -0.0098917 -0.0098917 -0.0098917 ]
[-0.01978339 -0.0098917 -0.0098917 -0.0098917 -0.0098917 ]
[-0.01978339 -0.0098917 -0.0098917 -0.0098917 -0.0098917 ]
[ 0.07805054 0.03902527 0.03902527 0.03902527 0.03902527]
[-0.01978339 -0.0098917 -0.0098917 -0.0098917 -0.0098917 ]
[ 0.08021661 0.0401083 0.0401083 0.0401083 0.0401083 ]]