程序代写代做 Keras 7CCMFM18 Machine Learning¶

7CCMFM18 Machine Learning¶
King’s College London 
Academic year 2019-2020 
Lecturer: Blanka Horvath
Example: Regression¶
24th February 2020

Let us first import the necessary libraries and functions, and set plotting style.
In [1]:
import numpy as np
import numpy.random as npr
import tensorflow.keras as keras
import matplotlib.pyplot as plt
plt.style.use(‘ggplot’)

We first define the function $g(x) := \sin(10x)$, $x \in \mathbb{R}$.
In [2]:
a = 10
def g(x):
return np.sin(a*x)

We then generate the data $x^0,x^1,\ldots,x^{N-1}$ (we switch here to the convention of Python of starting indexation from $0$) from $\mathrm{Uniform}(0,1)$ and \begin{equation} y^i = g(x^i) + \varepsilon^i, \quad i=0,1\ldots,N-1, \end{equation} with iid $\varepsilon^i \sim N(0,0.1)$, $i = 0,1,\ldots,N-1$, for $N = 1\,000\,000$.
In [3]:
N = 1000000
var = 0.1
x = npr.uniform(0,1, (N,1))
eps = npr.normal(0, np.sqrt(var), (N,1))
y = g(x)+eps

Let us plot the first $1000$ samples:
In [4]:
N_sub = 1000
plt.plot(x[0:N_sub,], y[0:N_sub,], “bo”, markersize=3)
plt.xlabel(r”$x^i$”)
plt.ylabel(r”$y^i$”)
plt.show()



Now we specify the neural network \begin{equation} \widehat{g} \in \mathcal{N}_4(1,100,100,100,1; \mathrm{ReLU},\mathrm{ReLU},\mathrm{ReLU},\mathrm{Id}) \end{equation} using the Sequential model of Keras, and inspect the specification.
In [4]:
g_hat = keras.Sequential([
keras.layers.Dense(100, activation=”relu”, input_shape=(1,)),
keras.layers.Dense(100, activation=”relu”),
keras.layers.Dense(100, activation=”relu”),
keras.layers.Dense(1, activation=”linear”)
]
)
g_hat.summary()

WARNING:tensorflow:From /Users/blanka/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense (Dense) (None, 100) 200
_________________________________________________________________
dense_1 (Dense) (None, 100) 10100
_________________________________________________________________
dense_2 (Dense) (None, 100) 10100
_________________________________________________________________
dense_3 (Dense) (None, 1) 101
=================================================================
Total params: 20,501
Trainable params: 20,501
Non-trainable params: 0
_________________________________________________________________

We compile $\widehat{g}$ so that it will be trained using Adam to minimise squared loss.
In [5]:
g_hat.compile(optimizer=”adam”, loss=”mean_squared_error”)

WARNING:tensorflow:From /Users/blanka/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/utils/losses_utils.py:170: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.

Then we can train $\widehat{g}$, with minibatches of size $100$, over $5$ epochs.
In [ ]:
g_hat.fit(x, y, batch_size=100, epochs=5)

WARNING:tensorflow:From /Users/blanka/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Epoch 1/5

Once training has finished, we compare $g$ with the trained $\widehat{g}$.
In [8]:
x_grid = np.linspace(0,1,num=1001)
y_hat = g_hat.predict(x_grid)
g_grid = g(x_grid)
plt.plot(x_grid, y_hat, label=r”$\widehat{g}(x)$”)
plt.plot(x_grid, g_grid, “b–“, label=”$g(x)$”)
plt.xlabel(“$x$”)
plt.ylabel(“Value”)
plt.legend()
plt.show()



The function $g$ is not so hard to learn when we have $N=1\,000\,000$ samples!

To demonstrate the potential problem of overfitting, we make now the problem harder: we are going to use only the first $200$ samples. In fact, we are going to even split these $200$ samples into a training set of $100$ samples and validation set of $100$ samples.
In [0]:
N_small = 200
x_sub = x[0:N_small]
y_sub = y[0:N_small]

We now specify a blatantly overparameterised network \begin{equation} \widehat{g}_{\mathrm{over}} \in \mathcal{N}_4(1,1000,1000,1000,1;\mathrm{ReLU},\mathrm{ReLU},\mathrm{ReLU},\mathrm{Id}). \end{equation}
In [10]:
g_hat_over = keras.Sequential([
keras.layers.Dense(1000, activation=”relu”, input_shape=(1,)),
keras.layers.Dense(1000, activation=”relu”),
keras.layers.Dense(1000, activation=”relu”),
keras.layers.Dense(1, activation=”linear”)
]
)
g_hat_over.summary()

Model: “sequential_1″
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_4 (Dense) (None, 1000) 2000
_________________________________________________________________
dense_5 (Dense) (None, 1000) 1001000
_________________________________________________________________
dense_6 (Dense) (None, 1000) 1001000
_________________________________________________________________
dense_7 (Dense) (None, 1) 1001
=================================================================
Total params: 2,005,001
Trainable params: 2,005,001
Non-trainable params: 0
_________________________________________________________________

We train it with Adam using full batches over $500$ epochs.
In [11]:
g_hat_over.compile(optimizer=”adam”, loss=”mean_squared_error”)
g_hat_over.fit(x_sub, y_sub, batch_size=100, epochs=500, validation_split=0.5)

Train on 100 samples, validate on 100 samples
Epoch 1/500
100/100 [==============================] – 0s 2ms/sample – loss: 0.5375 – val_loss: 0.6777
Epoch 2/500
100/100 [==============================] – 0s 54us/sample – loss: 0.5457 – val_loss: 0.6626
Epoch 3/500
100/100 [==============================] – 0s 47us/sample – loss: 0.5284 – val_loss: 0.6591
Epoch 4/500
100/100 [==============================] – 0s 50us/sample – loss: 0.5271 – val_loss: 0.6600
Epoch 5/500
100/100 [==============================] – 0s 43us/sample – loss: 0.5298 – val_loss: 0.6589
Epoch 6/500
100/100 [==============================] – 0s 44us/sample – loss: 0.5281 – val_loss: 0.6581
Epoch 7/500
100/100 [==============================] – 0s 42us/sample – loss: 0.5259 – val_loss: 0.6593
Epoch 8/500
100/100 [==============================] – 0s 52us/sample – loss: 0.5264 – val_loss: 0.6600
Epoch 9/500
100/100 [==============================] – 0s 47us/sample – loss: 0.5271 – val_loss: 0.6586
Epoch 10/500
100/100 [==============================] – 0s 44us/sample – loss: 0.5259 – val_loss: 0.6570
Epoch 11/500
100/100 [==============================] – 0s 48us/sample – loss: 0.5245 – val_loss: 0.6558
Epoch 12/500
100/100 [==============================] – 0s 56us/sample – loss: 0.5239 – val_loss: 0.6548
Epoch 13/500
100/100 [==============================] – 0s 49us/sample – loss: 0.5236 – val_loss: 0.6533
Epoch 14/500
100/100 [==============================] – 0s 52us/sample – loss: 0.5226 – val_loss: 0.6516
Epoch 15/500
100/100 [==============================] – 0s 50us/sample – loss: 0.5212 – val_loss: 0.6496
Epoch 16/500
100/100 [==============================] – 0s 51us/sample – loss: 0.5194 – val_loss: 0.6472
Epoch 17/500
100/100 [==============================] – 0s 55us/sample – loss: 0.5174 – val_loss: 0.6443
Epoch 18/500
100/100 [==============================] – 0s 47us/sample – loss: 0.5153 – val_loss: 0.6403
Epoch 19/500
100/100 [==============================] – 0s 48us/sample – loss: 0.5126 – val_loss: 0.6353
Epoch 20/500
100/100 [==============================] – 0s 64us/sample – loss: 0.5091 – val_loss: 0.6296
Epoch 21/500
100/100 [==============================] – 0s 51us/sample – loss: 0.5050 – val_loss: 0.6228
Epoch 22/500
100/100 [==============================] – 0s 47us/sample – loss: 0.5000 – val_loss: 0.6150
Epoch 23/500
100/100 [==============================] – 0s 40us/sample – loss: 0.4939 – val_loss: 0.6059
Epoch 24/500
100/100 [==============================] – 0s 42us/sample – loss: 0.4869 – val_loss: 0.5951
Epoch 25/500
100/100 [==============================] – 0s 51us/sample – loss: 0.4784 – val_loss: 0.5826
Epoch 26/500
100/100 [==============================] – 0s 45us/sample – loss: 0.4688 – val_loss: 0.5677
Epoch 27/500
100/100 [==============================] – 0s 49us/sample – loss: 0.4572 – val_loss: 0.5505
Epoch 28/500
100/100 [==============================] – 0s 45us/sample – loss: 0.4437 – val_loss: 0.5308
Epoch 29/500
100/100 [==============================] – 0s 50us/sample – loss: 0.4282 – val_loss: 0.5083
Epoch 30/500
100/100 [==============================] – 0s 67us/sample – loss: 0.4107 – val_loss: 0.4832
Epoch 31/500
100/100 [==============================] – 0s 54us/sample – loss: 0.3916 – val_loss: 0.4560
Epoch 32/500
100/100 [==============================] – 0s 42us/sample – loss: 0.3714 – val_loss: 0.4269
Epoch 33/500
100/100 [==============================] – 0s 48us/sample – loss: 0.3506 – val_loss: 0.3972
Epoch 34/500
100/100 [==============================] – 0s 59us/sample – loss: 0.3302 – val_loss: 0.3682
Epoch 35/500
100/100 [==============================] – 0s 60us/sample – loss: 0.3113 – val_loss: 0.3404
Epoch 36/500
100/100 [==============================] – 0s 105us/sample – loss: 0.2954 – val_loss: 0.3166
Epoch 37/500
100/100 [==============================] – 0s 60us/sample – loss: 0.2844 – val_loss: 0.2979
Epoch 38/500
100/100 [==============================] – 0s 76us/sample – loss: 0.2793 – val_loss: 0.3045
Epoch 39/500
100/100 [==============================] – 0s 69us/sample – loss: 0.2941 – val_loss: 0.3060
Epoch 40/500
100/100 [==============================] – 0s 74us/sample – loss: 0.3213 – val_loss: 0.2867
Epoch 41/500
100/100 [==============================] – 0s 51us/sample – loss: 0.2743 – val_loss: 0.3211
Epoch 42/500
100/100 [==============================] – 0s 75us/sample – loss: 0.3081 – val_loss: 0.2812
Epoch 43/500
100/100 [==============================] – 0s 53us/sample – loss: 0.2679 – val_loss: 0.2879
Epoch 44/500
100/100 [==============================] – 0s 58us/sample – loss: 0.2934 – val_loss: 0.2703
Epoch 45/500
100/100 [==============================] – 0s 63us/sample – loss: 0.2635 – val_loss: 0.2804
Epoch 46/500
100/100 [==============================] – 0s 73us/sample – loss: 0.2730 – val_loss: 0.2685
Epoch 47/500
100/100 [==============================] – 0s 55us/sample – loss: 0.2654 – val_loss: 0.2581
Epoch 48/500
100/100 [==============================] – 0s 58us/sample – loss: 0.2526 – val_loss: 0.2688
Epoch 49/500
100/100 [==============================] – 0s 77us/sample – loss: 0.2619 – val_loss: 0.2661
Epoch 50/500
100/100 [==============================] – 0s 66us/sample – loss: 0.2481 – val_loss: 0.2657
Epoch 51/500
100/100 [==============================] – 0s 83us/sample – loss: 0.2508 – val_loss: 0.2627
Epoch 52/500
100/100 [==============================] – 0s 51us/sample – loss: 0.2502 – val_loss: 0.2529
Epoch 53/500
100/100 [==============================] – 0s 114us/sample – loss: 0.2404 – val_loss: 0.2604
Epoch 54/500
100/100 [==============================] – 0s 100us/sample – loss: 0.2474 – val_loss: 0.2529
Epoch 55/500
100/100 [==============================] – 0s 54us/sample – loss: 0.2410 – val_loss: 0.2478
Epoch 56/500
100/100 [==============================] – 0s 73us/sample – loss: 0.2389 – val_loss: 0.2499
Epoch 57/500
100/100 [==============================] – 0s 64us/sample – loss: 0.2398 – val_loss: 0.2474
Epoch 58/500
100/100 [==============================] – 0s 75us/sample – loss: 0.2337 – val_loss: 0.2531
Epoch 59/500
100/100 [==============================] – 0s 70us/sample – loss: 0.2364 – val_loss: 0.2481
Epoch 60/500
100/100 [==============================] – 0s 54us/sample – loss: 0.2334 – val_loss: 0.2411
Epoch 61/500
100/100 [==============================] – 0s 71us/sample – loss: 0.2302 – val_loss: 0.2404
Epoch 62/500
100/100 [==============================] – 0s 75us/sample – loss: 0.2315 – val_loss: 0.2369
Epoch 63/500
100/100 [==============================] – 0s 66us/sample – loss: 0.2272 – val_loss: 0.2407
Epoch 64/500
100/100 [==============================] – 0s 69us/sample – loss: 0.2277 – val_loss: 0.2392
Epoch 65/500
100/100 [==============================] – 0s 73us/sample – loss: 0.2257 – val_loss: 0.2356
Epoch 66/500
100/100 [==============================] – 0s 76us/sample – loss: 0.2230 – val_loss: 0.2352
Epoch 67/500
100/100 [==============================] – 0s 77us/sample – loss: 0.2233 – val_loss: 0.2309
Epoch 68/500
100/100 [==============================] – 0s 70us/sample – loss: 0.2201 – val_loss: 0.2293
Epoch 69/500
100/100 [==============================] – 0s 53us/sample – loss: 0.2195 – val_loss: 0.2261
Epoch 70/500
100/100 [==============================] – 0s 79us/sample – loss: 0.2177 – val_loss: 0.2231
Epoch 71/500
100/100 [==============================] – 0s 77us/sample – loss: 0.2156 – val_loss: 0.2231
Epoch 72/500
100/100 [==============================] – 0s 78us/sample – loss: 0.2153 – val_loss: 0.2216
Epoch 73/500
100/100 [==============================] – 0s 89us/sample – loss: 0.2125 – val_loss: 0.2216
Epoch 74/500
100/100 [==============================] – 0s 53us/sample – loss: 0.2118 – val_loss: 0.2192
Epoch 75/500
100/100 [==============================] – 0s 60us/sample – loss: 0.2097 – val_loss: 0.2164
Epoch 76/500
100/100 [==============================] – 0s 49us/sample – loss: 0.2080 – val_loss: 0.2147
Epoch 77/500
100/100 [==============================] – 0s 56us/sample – loss: 0.2069 – val_loss: 0.2132
Epoch 78/500
100/100 [==============================] – 0s 115us/sample – loss: 0.2047 – val_loss: 0.2131
Epoch 79/500
100/100 [==============================] – 0s 103us/sample – loss: 0.2037 – val_loss: 0.2110
Epoch 80/500
100/100 [==============================] – 0s 59us/sample – loss: 0.2015 – val_loss: 0.2094
Epoch 81/500
100/100 [==============================] – 0s 60us/sample – loss: 0.2004 – val_loss: 0.2070
Epoch 82/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1983 – val_loss: 0.2050
Epoch 83/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1967 – val_loss: 0.2028
Epoch 84/500
100/100 [==============================] – 0s 87us/sample – loss: 0.1952 – val_loss: 0.2002
Epoch 85/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1935 – val_loss: 0.1989
Epoch 86/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1919 – val_loss: 0.1979
Epoch 87/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1900 – val_loss: 0.1968
Epoch 88/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1885 – val_loss: 0.1946
Epoch 89/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1866 – val_loss: 0.1923
Epoch 90/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1849 – val_loss: 0.1901
Epoch 91/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1829 – val_loss: 0.1884
Epoch 92/500
100/100 [==============================] – 0s 45us/sample – loss: 0.1811 – val_loss: 0.1863
Epoch 93/500
100/100 [==============================] – 0s 49us/sample – loss: 0.1791 – val_loss: 0.1843
Epoch 94/500
100/100 [==============================] – 0s 44us/sample – loss: 0.1771 – val_loss: 0.1826
Epoch 95/500
100/100 [==============================] – 0s 43us/sample – loss: 0.1751 – val_loss: 0.1806
Epoch 96/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1731 – val_loss: 0.1779
Epoch 97/500
100/100 [==============================] – 0s 49us/sample – loss: 0.1709 – val_loss: 0.1756
Epoch 98/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1687 – val_loss: 0.1738
Epoch 99/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1666 – val_loss: 0.1718
Epoch 100/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1645 – val_loss: 0.1700
Epoch 101/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1628 – val_loss: 0.1700
Epoch 102/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1626 – val_loss: 0.1755
Epoch 103/500
100/100 [==============================] – 0s 80us/sample – loss: 0.1674 – val_loss: 0.1801
Epoch 104/500
100/100 [==============================] – 0s 66us/sample – loss: 0.1712 – val_loss: 0.1676
Epoch 105/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1602 – val_loss: 0.1608
Epoch 106/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1542 – val_loss: 0.1691
Epoch 107/500
100/100 [==============================] – 0s 36us/sample – loss: 0.1615 – val_loss: 0.1629
Epoch 108/500
100/100 [==============================] – 0s 40us/sample – loss: 0.1554 – val_loss: 0.1566
Epoch 109/500
100/100 [==============================] – 0s 37us/sample – loss: 0.1494 – val_loss: 0.1609
Epoch 110/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1548 – val_loss: 0.1563
Epoch 111/500
100/100 [==============================] – 0s 38us/sample – loss: 0.1490 – val_loss: 0.1525
Epoch 112/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1454 – val_loss: 0.1557
Epoch 113/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1488 – val_loss: 0.1515
Epoch 114/500
100/100 [==============================] – 0s 86us/sample – loss: 0.1440 – val_loss: 0.1488
Epoch 115/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1413 – val_loss: 0.1513
Epoch 116/500
100/100 [==============================] – 0s 77us/sample – loss: 0.1436 – val_loss: 0.1481
Epoch 117/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1398 – val_loss: 0.1456
Epoch 118/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1374 – val_loss: 0.1464
Epoch 119/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1389 – val_loss: 0.1452
Epoch 120/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1367 – val_loss: 0.1422
Epoch 121/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1337 – val_loss: 0.1422
Epoch 122/500
100/100 [==============================] – 0s 49us/sample – loss: 0.1340 – val_loss: 0.1433
Epoch 123/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1341 – val_loss: 0.1409
Epoch 124/500
100/100 [==============================] – 0s 71us/sample – loss: 0.1319 – val_loss: 0.1392
Epoch 125/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1297 – val_loss: 0.1391
Epoch 126/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1295 – val_loss: 0.1387
Epoch 127/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1298 – val_loss: 0.1385
Epoch 128/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1291 – val_loss: 0.1371
Epoch 129/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1274 – val_loss: 0.1357
Epoch 130/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1258 – val_loss: 0.1351
Epoch 131/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1251 – val_loss: 0.1358
Epoch 132/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1250 – val_loss: 0.1364
Epoch 133/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1253 – val_loss: 0.1371
Epoch 134/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1258 – val_loss: 0.1370
Epoch 135/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1265 – val_loss: 0.1386
Epoch 136/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1270 – val_loss: 0.1372
Epoch 137/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1269 – val_loss: 0.1376
Epoch 138/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1258 – val_loss: 0.1347
Epoch 139/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1236 – val_loss: 0.1340
Epoch 140/500
100/100 [==============================] – 0s 79us/sample – loss: 0.1217 – val_loss: 0.1334
Epoch 141/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1209 – val_loss: 0.1334
Epoch 142/500
100/100 [==============================] – 0s 50us/sample – loss: 0.1212 – val_loss: 0.1353
Epoch 143/500
100/100 [==============================] – 0s 44us/sample – loss: 0.1223 – val_loss: 0.1351
Epoch 144/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1237 – val_loss: 0.1385
Epoch 145/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1249 – val_loss: 0.1360
Epoch 146/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1250 – val_loss: 0.1369
Epoch 147/500
100/100 [==============================] – 0s 78us/sample – loss: 0.1236 – val_loss: 0.1330
Epoch 148/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1213 – val_loss: 0.1325
Epoch 149/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1195 – val_loss: 0.1325
Epoch 150/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1194 – val_loss: 0.1325
Epoch 151/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1203 – val_loss: 0.1353
Epoch 152/500
100/100 [==============================] – 0s 66us/sample – loss: 0.1213 – val_loss: 0.1331
Epoch 153/500
100/100 [==============================] – 0s 89us/sample – loss: 0.1218 – val_loss: 0.1355
Epoch 154/500
100/100 [==============================] – 0s 75us/sample – loss: 0.1215 – val_loss: 0.1330
Epoch 155/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1204 – val_loss: 0.1334
Epoch 156/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1192 – val_loss: 0.1319
Epoch 157/500
100/100 [==============================] – 0s 79us/sample – loss: 0.1185 – val_loss: 0.1318
Epoch 158/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1184 – val_loss: 0.1329
Epoch 159/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1189 – val_loss: 0.1315
Epoch 160/500
100/100 [==============================] – 0s 75us/sample – loss: 0.1197 – val_loss: 0.1350
Epoch 161/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1207 – val_loss: 0.1332
Epoch 162/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1214 – val_loss: 0.1370
Epoch 163/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1216 – val_loss: 0.1336
Epoch 164/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1210 – val_loss: 0.1349
Epoch 165/500
100/100 [==============================] – 0s 88us/sample – loss: 0.1199 – val_loss: 0.1317
Epoch 166/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1185 – val_loss: 0.1320
Epoch 167/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1177 – val_loss: 0.1327
Epoch 168/500
100/100 [==============================] – 0s 44us/sample – loss: 0.1179 – val_loss: 0.1320
Epoch 169/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1186 – val_loss: 0.1350
Epoch 170/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1194 – val_loss: 0.1328
Epoch 171/500
100/100 [==============================] – 0s 50us/sample – loss: 0.1200 – val_loss: 0.1355
Epoch 172/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1200 – val_loss: 0.1320
Epoch 173/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1193 – val_loss: 0.1337
Epoch 174/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1183 – val_loss: 0.1314
Epoch 175/500
100/100 [==============================] – 0s 89us/sample – loss: 0.1175 – val_loss: 0.1319
Epoch 176/500
100/100 [==============================] – 0s 92us/sample – loss: 0.1171 – val_loss: 0.1324
Epoch 177/500
100/100 [==============================] – 0s 84us/sample – loss: 0.1172 – val_loss: 0.1315
Epoch 178/500
100/100 [==============================] – 0s 78us/sample – loss: 0.1176 – val_loss: 0.1340
Epoch 179/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1181 – val_loss: 0.1322
Epoch 180/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1186 – val_loss: 0.1357
Epoch 181/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1190 – val_loss: 0.1324
Epoch 182/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1191 – val_loss: 0.1353
Epoch 183/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1188 – val_loss: 0.1317
Epoch 184/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1183 – val_loss: 0.1339
Epoch 185/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1176 – val_loss: 0.1312
Epoch 186/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1171 – val_loss: 0.1325
Epoch 187/500
100/100 [==============================] – 0s 111us/sample – loss: 0.1167 – val_loss: 0.1317
Epoch 188/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1164 – val_loss: 0.1316
Epoch 189/500
100/100 [==============================] – 0s 80us/sample – loss: 0.1163 – val_loss: 0.1322
Epoch 190/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1164 – val_loss: 0.1315
Epoch 191/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1165 – val_loss: 0.1337
Epoch 192/500
100/100 [==============================] – 0s 71us/sample – loss: 0.1168 – val_loss: 0.1317
Epoch 193/500
100/100 [==============================] – 0s 71us/sample – loss: 0.1174 – val_loss: 0.1360
Epoch 194/500
100/100 [==============================] – 0s 38us/sample – loss: 0.1183 – val_loss: 0.1331
Epoch 195/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1198 – val_loss: 0.1404
Epoch 196/500
100/100 [==============================] – 0s 82us/sample – loss: 0.1219 – val_loss: 0.1350
Epoch 197/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1238 – val_loss: 0.1425
Epoch 198/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1239 – val_loss: 0.1340
Epoch 199/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1210 – val_loss: 0.1346
Epoch 200/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1172 – val_loss: 0.1320
Epoch 201/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1158 – val_loss: 0.1319
Epoch 202/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1175 – val_loss: 0.1390
Epoch 203/500
100/100 [==============================] – 0s 91us/sample – loss: 0.1202 – val_loss: 0.1331
Epoch 204/500
100/100 [==============================] – 0s 116us/sample – loss: 0.1210 – val_loss: 0.1376
Epoch 205/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1192 – val_loss: 0.1317
Epoch 206/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1164 – val_loss: 0.1322
Epoch 207/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1155 – val_loss: 0.1351
Epoch 208/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1167 – val_loss: 0.1327
Epoch 209/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1182 – val_loss: 0.1372
Epoch 210/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1183 – val_loss: 0.1310
Epoch 211/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1168 – val_loss: 0.1322
Epoch 212/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1154 – val_loss: 0.1329
Epoch 213/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1154 – val_loss: 0.1321
Epoch 214/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1164 – val_loss: 0.1366
Epoch 215/500
100/100 [==============================] – 0s 48us/sample – loss: 0.1172 – val_loss: 0.1323
Epoch 216/500
100/100 [==============================] – 0s 46us/sample – loss: 0.1171 – val_loss: 0.1349
Epoch 217/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1163 – val_loss: 0.1319
Epoch 218/500
100/100 [==============================] – 0s 106us/sample – loss: 0.1153 – val_loss: 0.1329
Epoch 219/500
100/100 [==============================] – 0s 45us/sample – loss: 0.1150 – val_loss: 0.1342
Epoch 220/500
100/100 [==============================] – 0s 45us/sample – loss: 0.1154 – val_loss: 0.1315
Epoch 221/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1160 – val_loss: 0.1354
Epoch 222/500
100/100 [==============================] – 0s 75us/sample – loss: 0.1164 – val_loss: 0.1317
Epoch 223/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1162 – val_loss: 0.1337
Epoch 224/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1155 – val_loss: 0.1318
Epoch 225/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1149 – val_loss: 0.1331
Epoch 226/500
100/100 [==============================] – 0s 79us/sample – loss: 0.1147 – val_loss: 0.1339
Epoch 227/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1148 – val_loss: 0.1313
Epoch 228/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1153 – val_loss: 0.1349
Epoch 229/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1155 – val_loss: 0.1322
Epoch 230/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1155 – val_loss: 0.1348
Epoch 231/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1153 – val_loss: 0.1315
Epoch 232/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1150 – val_loss: 0.1337
Epoch 233/500
100/100 [==============================] – 0s 71us/sample – loss: 0.1146 – val_loss: 0.1320
Epoch 234/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1143 – val_loss: 0.1322
Epoch 235/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1142 – val_loss: 0.1325
Epoch 236/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1141 – val_loss: 0.1320
Epoch 237/500
100/100 [==============================] – 0s 50us/sample – loss: 0.1141 – val_loss: 0.1338
Epoch 238/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1143 – val_loss: 0.1320
Epoch 239/500
100/100 [==============================] – 0s 79us/sample – loss: 0.1145 – val_loss: 0.1351
Epoch 240/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1149 – val_loss: 0.1318
Epoch 241/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1155 – val_loss: 0.1379
Epoch 242/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1162 – val_loss: 0.1331
Epoch 243/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1170 – val_loss: 0.1397
Epoch 244/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1178 – val_loss: 0.1334
Epoch 245/500
100/100 [==============================] – 0s 87us/sample – loss: 0.1179 – val_loss: 0.1392
Epoch 246/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1172 – val_loss: 0.1327
Epoch 247/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1159 – val_loss: 0.1354
Epoch 248/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1146 – val_loss: 0.1325
Epoch 249/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1137 – val_loss: 0.1329
Epoch 250/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1135 – val_loss: 0.1344
Epoch 251/500
100/100 [==============================] – 0s 49us/sample – loss: 0.1139 – val_loss: 0.1318
Epoch 252/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1148 – val_loss: 0.1383
Epoch 253/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1160 – val_loss: 0.1332
Epoch 254/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1172 – val_loss: 0.1408
Epoch 255/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1179 – val_loss: 0.1338
Epoch 256/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1176 – val_loss: 0.1399
Epoch 257/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1167 – val_loss: 0.1326
Epoch 258/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1151 – val_loss: 0.1351
Epoch 259/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1137 – val_loss: 0.1337
Epoch 260/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1132 – val_loss: 0.1327
Epoch 261/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1135 – val_loss: 0.1366
Epoch 262/500
100/100 [==============================] – 0s 48us/sample – loss: 0.1144 – val_loss: 0.1324
Epoch 263/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1155 – val_loss: 0.1396
Epoch 264/500
100/100 [==============================] – 0s 50us/sample – loss: 0.1163 – val_loss: 0.1335
Epoch 265/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1165 – val_loss: 0.1398
Epoch 266/500
100/100 [==============================] – 0s 80us/sample – loss: 0.1156 – val_loss: 0.1337
Epoch 267/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1144 – val_loss: 0.1354
Epoch 268/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1133 – val_loss: 0.1335
Epoch 269/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1129 – val_loss: 0.1332
Epoch 270/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1130 – val_loss: 0.1358
Epoch 271/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1135 – val_loss: 0.1324
Epoch 272/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1145 – val_loss: 0.1400
Epoch 273/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1155 – val_loss: 0.1342
Epoch 274/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1158 – val_loss: 0.1397
Epoch 275/500
100/100 [==============================] – 0s 71us/sample – loss: 0.1156 – val_loss: 0.1336
Epoch 276/500
100/100 [==============================] – 0s 81us/sample – loss: 0.1146 – val_loss: 0.1380
Epoch 277/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1137 – val_loss: 0.1328
Epoch 278/500
100/100 [==============================] – 0s 48us/sample – loss: 0.1128 – val_loss: 0.1331
Epoch 279/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1125 – val_loss: 0.1349
Epoch 280/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1128 – val_loss: 0.1323
Epoch 281/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1134 – val_loss: 0.1383
Epoch 282/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1142 – val_loss: 0.1340
Epoch 283/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1153 – val_loss: 0.1412
Epoch 284/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1159 – val_loss: 0.1335
Epoch 285/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1159 – val_loss: 0.1398
Epoch 286/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1149 – val_loss: 0.1332
Epoch 287/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1136 – val_loss: 0.1353
Epoch 288/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1126 – val_loss: 0.1340
Epoch 289/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1122 – val_loss: 0.1330
Epoch 290/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1124 – val_loss: 0.1362
Epoch 291/500
100/100 [==============================] – 0s 66us/sample – loss: 0.1130 – val_loss: 0.1327
Epoch 292/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1138 – val_loss: 0.1400
Epoch 293/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1145 – val_loss: 0.1341
Epoch 294/500
100/100 [==============================] – 0s 86us/sample – loss: 0.1147 – val_loss: 0.1400
Epoch 295/500
100/100 [==============================] – 0s 81us/sample – loss: 0.1143 – val_loss: 0.1342
Epoch 296/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1135 – val_loss: 0.1368
Epoch 297/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1125 – val_loss: 0.1338
Epoch 298/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1119 – val_loss: 0.1341
Epoch 299/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1118 – val_loss: 0.1358
Epoch 300/500
100/100 [==============================] – 0s 49us/sample – loss: 0.1122 – val_loss: 0.1324
Epoch 301/500
100/100 [==============================] – 0s 66us/sample – loss: 0.1130 – val_loss: 0.1403
Epoch 302/500
100/100 [==============================] – 0s 87us/sample – loss: 0.1142 – val_loss: 0.1346
Epoch 303/500
100/100 [==============================] – 0s 78us/sample – loss: 0.1153 – val_loss: 0.1423
Epoch 304/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1158 – val_loss: 0.1347
Epoch 305/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1152 – val_loss: 0.1405
Epoch 306/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1140 – val_loss: 0.1332
Epoch 307/500
100/100 [==============================] – 0s 48us/sample – loss: 0.1126 – val_loss: 0.1345
Epoch 308/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1117 – val_loss: 0.1351
Epoch 309/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1117 – val_loss: 0.1333
Epoch 310/500
100/100 [==============================] – 0s 46us/sample – loss: 0.1123 – val_loss: 0.1389
Epoch 311/500
100/100 [==============================] – 0s 50us/sample – loss: 0.1133 – val_loss: 0.1333
Epoch 312/500
100/100 [==============================] – 0s 78us/sample – loss: 0.1140 – val_loss: 0.1407
Epoch 313/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1143 – val_loss: 0.1343
Epoch 314/500
100/100 [==============================] – 0s 83us/sample – loss: 0.1140 – val_loss: 0.1400
Epoch 315/500
100/100 [==============================] – 0s 38us/sample – loss: 0.1130 – val_loss: 0.1343
Epoch 316/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1120 – val_loss: 0.1356
Epoch 317/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1114 – val_loss: 0.1350
Epoch 318/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1113 – val_loss: 0.1336
Epoch 319/500
100/100 [==============================] – 0s 49us/sample – loss: 0.1116 – val_loss: 0.1376
Epoch 320/500
100/100 [==============================] – 0s 71us/sample – loss: 0.1122 – val_loss: 0.1334
Epoch 321/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1130 – val_loss: 0.1419
Epoch 322/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1142 – val_loss: 0.1347
Epoch 323/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1147 – val_loss: 0.1421
Epoch 324/500
100/100 [==============================] – 0s 50us/sample – loss: 0.1145 – val_loss: 0.1351
Epoch 325/500
100/100 [==============================] – 0s 47us/sample – loss: 0.1135 – val_loss: 0.1394
Epoch 326/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1124 – val_loss: 0.1331
Epoch 327/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1115 – val_loss: 0.1354
Epoch 328/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1110 – val_loss: 0.1352
Epoch 329/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1108 – val_loss: 0.1338
Epoch 330/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1111 – val_loss: 0.1381
Epoch 331/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1116 – val_loss: 0.1339
Epoch 332/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1122 – val_loss: 0.1408
Epoch 333/500
100/100 [==============================] – 0s 93us/sample – loss: 0.1130 – val_loss: 0.1346
Epoch 334/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1139 – val_loss: 0.1447
Epoch 335/500
100/100 [==============================] – 0s 118us/sample – loss: 0.1151 – val_loss: 0.1345
Epoch 336/500
100/100 [==============================] – 0s 49us/sample – loss: 0.1157 – val_loss: 0.1444
Epoch 337/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1153 – val_loss: 0.1359
Epoch 338/500
100/100 [==============================] – 0s 48us/sample – loss: 0.1143 – val_loss: 0.1406
Epoch 339/500
100/100 [==============================] – 0s 49us/sample – loss: 0.1126 – val_loss: 0.1342
Epoch 340/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1111 – val_loss: 0.1366
Epoch 341/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1106 – val_loss: 0.1359
Epoch 342/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1106 – val_loss: 0.1337
Epoch 343/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1110 – val_loss: 0.1406
Epoch 344/500
100/100 [==============================] – 0s 78us/sample – loss: 0.1122 – val_loss: 0.1340
Epoch 345/500
100/100 [==============================] – 0s 80us/sample – loss: 0.1132 – val_loss: 0.1428
Epoch 346/500
100/100 [==============================] – 0s 84us/sample – loss: 0.1137 – val_loss: 0.1359
Epoch 347/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1138 – val_loss: 0.1431
Epoch 348/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1133 – val_loss: 0.1337
Epoch 349/500
100/100 [==============================] – 0s 43us/sample – loss: 0.1124 – val_loss: 0.1387
Epoch 350/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1111 – val_loss: 0.1358
Epoch 351/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1103 – val_loss: 0.1356
Epoch 352/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1103 – val_loss: 0.1379
Epoch 353/500
100/100 [==============================] – 0s 71us/sample – loss: 0.1105 – val_loss: 0.1343
Epoch 354/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1112 – val_loss: 0.1405
Epoch 355/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1119 – val_loss: 0.1350
Epoch 356/500
100/100 [==============================] – 0s 95us/sample – loss: 0.1126 – val_loss: 0.1436
Epoch 357/500
100/100 [==============================] – 0s 50us/sample – loss: 0.1131 – val_loss: 0.1347
Epoch 358/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1133 – val_loss: 0.1448
Epoch 359/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1135 – val_loss: 0.1355
Epoch 360/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1130 – val_loss: 0.1405
Epoch 361/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1119 – val_loss: 0.1355
Epoch 362/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1106 – val_loss: 0.1386
Epoch 363/500
100/100 [==============================] – 0s 89us/sample – loss: 0.1101 – val_loss: 0.1369
Epoch 364/500
100/100 [==============================] – 0s 82us/sample – loss: 0.1099 – val_loss: 0.1339
Epoch 365/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1102 – val_loss: 0.1394
Epoch 366/500
100/100 [==============================] – 0s 84us/sample – loss: 0.1111 – val_loss: 0.1346
Epoch 367/500
100/100 [==============================] – 0s 86us/sample – loss: 0.1117 – val_loss: 0.1427
Epoch 368/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1119 – val_loss: 0.1370
Epoch 369/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1120 – val_loss: 0.1427
Epoch 370/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1116 – val_loss: 0.1348
Epoch 371/500
100/100 [==============================] – 0s 123us/sample – loss: 0.1112 – val_loss: 0.1388
Epoch 372/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1104 – val_loss: 0.1357
Epoch 373/500
100/100 [==============================] – 0s 75us/sample – loss: 0.1098 – val_loss: 0.1384
Epoch 374/500
100/100 [==============================] – 0s 91us/sample – loss: 0.1097 – val_loss: 0.1378
Epoch 375/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1095 – val_loss: 0.1357
Epoch 376/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1096 – val_loss: 0.1378
Epoch 377/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1096 – val_loss: 0.1365
Epoch 378/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1098 – val_loss: 0.1395
Epoch 379/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1098 – val_loss: 0.1353
Epoch 380/500
100/100 [==============================] – 0s 48us/sample – loss: 0.1102 – val_loss: 0.1422
Epoch 381/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1108 – val_loss: 0.1358
Epoch 382/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1115 – val_loss: 0.1437
Epoch 383/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1123 – val_loss: 0.1374
Epoch 384/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1130 – val_loss: 0.1485
Epoch 385/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1141 – val_loss: 0.1358
Epoch 386/500
100/100 [==============================] – 0s 80us/sample – loss: 0.1148 – val_loss: 0.1484
Epoch 387/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1157 – val_loss: 0.1354
Epoch 388/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1151 – val_loss: 0.1466
Epoch 389/500
100/100 [==============================] – 0s 71us/sample – loss: 0.1131 – val_loss: 0.1374
Epoch 390/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1109 – val_loss: 0.1387
Epoch 391/500
100/100 [==============================] – 0s 48us/sample – loss: 0.1093 – val_loss: 0.1374
Epoch 392/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1091 – val_loss: 0.1359
Epoch 393/500
100/100 [==============================] – 0s 86us/sample – loss: 0.1101 – val_loss: 0.1437
Epoch 394/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1115 – val_loss: 0.1350
Epoch 395/500
100/100 [==============================] – 0s 40us/sample – loss: 0.1126 – val_loss: 0.1472
Epoch 396/500
100/100 [==============================] – 0s 35us/sample – loss: 0.1132 – val_loss: 0.1368
Epoch 397/500
100/100 [==============================] – 0s 75us/sample – loss: 0.1125 – val_loss: 0.1433
Epoch 398/500
100/100 [==============================] – 0s 94us/sample – loss: 0.1108 – val_loss: 0.1383
Epoch 399/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1093 – val_loss: 0.1377
Epoch 400/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1088 – val_loss: 0.1406
Epoch 401/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1096 – val_loss: 0.1365
Epoch 402/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1109 – val_loss: 0.1458
Epoch 403/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1123 – val_loss: 0.1349
Epoch 404/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1128 – val_loss: 0.1450
Epoch 405/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1116 – val_loss: 0.1376
Epoch 406/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1100 – val_loss: 0.1396
Epoch 407/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1091 – val_loss: 0.1393
Epoch 408/500
100/100 [==============================] – 0s 50us/sample – loss: 0.1087 – val_loss: 0.1364
Epoch 409/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1093 – val_loss: 0.1419
Epoch 410/500
100/100 [==============================] – 0s 50us/sample – loss: 0.1103 – val_loss: 0.1365
Epoch 411/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1105 – val_loss: 0.1437
Epoch 412/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1104 – val_loss: 0.1361
Epoch 413/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1097 – val_loss: 0.1409
Epoch 414/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1089 – val_loss: 0.1384
Epoch 415/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1085 – val_loss: 0.1374
Epoch 416/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1084 – val_loss: 0.1410
Epoch 417/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1088 – val_loss: 0.1371
Epoch 418/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1094 – val_loss: 0.1440
Epoch 419/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1101 – val_loss: 0.1367
Epoch 420/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1105 – val_loss: 0.1457
Epoch 421/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1106 – val_loss: 0.1367
Epoch 422/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1103 – val_loss: 0.1445
Epoch 423/500
100/100 [==============================] – 0s 78us/sample – loss: 0.1097 – val_loss: 0.1377
Epoch 424/500
100/100 [==============================] – 0s 66us/sample – loss: 0.1090 – val_loss: 0.1398
Epoch 425/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1083 – val_loss: 0.1389
Epoch 426/500
100/100 [==============================] – 0s 66us/sample – loss: 0.1080 – val_loss: 0.1380
Epoch 427/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1081 – val_loss: 0.1407
Epoch 428/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1085 – val_loss: 0.1369
Epoch 429/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1090 – val_loss: 0.1444
Epoch 430/500
100/100 [==============================] – 0s 71us/sample – loss: 0.1096 – val_loss: 0.1370
Epoch 431/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1099 – val_loss: 0.1455
Epoch 432/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1100 – val_loss: 0.1375
Epoch 433/500
100/100 [==============================] – 0s 92us/sample – loss: 0.1098 – val_loss: 0.1443
Epoch 434/500
100/100 [==============================] – 0s 77us/sample – loss: 0.1095 – val_loss: 0.1384
Epoch 435/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1090 – val_loss: 0.1415
Epoch 436/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1084 – val_loss: 0.1383
Epoch 437/500
100/100 [==============================] – 0s 87us/sample – loss: 0.1079 – val_loss: 0.1412
Epoch 438/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1078 – val_loss: 0.1396
Epoch 439/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1076 – val_loss: 0.1377
Epoch 440/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1077 – val_loss: 0.1421
Epoch 441/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1080 – val_loss: 0.1383
Epoch 442/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1081 – val_loss: 0.1431
Epoch 443/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1084 – val_loss: 0.1385
Epoch 444/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1088 – val_loss: 0.1464
Epoch 445/500
100/100 [==============================] – 0s 80us/sample – loss: 0.1096 – val_loss: 0.1369
Epoch 446/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1104 – val_loss: 0.1478
Epoch 447/500
100/100 [==============================] – 0s 50us/sample – loss: 0.1112 – val_loss: 0.1391
Epoch 448/500
100/100 [==============================] – 0s 50us/sample – loss: 0.1117 – val_loss: 0.1519
Epoch 449/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1122 – val_loss: 0.1386
Epoch 450/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1119 – val_loss: 0.1492
Epoch 451/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1116 – val_loss: 0.1381
Epoch 452/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1104 – val_loss: 0.1462
Epoch 453/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1088 – val_loss: 0.1397
Epoch 454/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1078 – val_loss: 0.1406
Epoch 455/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1072 – val_loss: 0.1415
Epoch 456/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1073 – val_loss: 0.1383
Epoch 457/500
100/100 [==============================] – 0s 66us/sample – loss: 0.1077 – val_loss: 0.1441
Epoch 458/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1084 – val_loss: 0.1386
Epoch 459/500
100/100 [==============================] – 0s 77us/sample – loss: 0.1089 – val_loss: 0.1489
Epoch 460/500
100/100 [==============================] – 0s 81us/sample – loss: 0.1097 – val_loss: 0.1379
Epoch 461/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1100 – val_loss: 0.1476
Epoch 462/500
100/100 [==============================] – 0s 66us/sample – loss: 0.1099 – val_loss: 0.1401
Epoch 463/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1093 – val_loss: 0.1459
Epoch 464/500
100/100 [==============================] – 0s 82us/sample – loss: 0.1084 – val_loss: 0.1393
Epoch 465/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1076 – val_loss: 0.1423
Epoch 466/500
100/100 [==============================] – 0s 47us/sample – loss: 0.1070 – val_loss: 0.1404
Epoch 467/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1070 – val_loss: 0.1420
Epoch 468/500
100/100 [==============================] – 0s 46us/sample – loss: 0.1069 – val_loss: 0.1419
Epoch 469/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1068 – val_loss: 0.1401
Epoch 470/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1069 – val_loss: 0.1430
Epoch 471/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1069 – val_loss: 0.1395
Epoch 472/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1069 – val_loss: 0.1423
Epoch 473/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1069 – val_loss: 0.1406
Epoch 474/500
100/100 [==============================] – 0s 77us/sample – loss: 0.1070 – val_loss: 0.1443
Epoch 475/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1072 – val_loss: 0.1387
Epoch 476/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1075 – val_loss: 0.1467
Epoch 477/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1080 – val_loss: 0.1399
Epoch 478/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1086 – val_loss: 0.1498
Epoch 479/500
100/100 [==============================] – 0s 66us/sample – loss: 0.1095 – val_loss: 0.1408
Epoch 480/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1105 – val_loss: 0.1512
Epoch 481/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1110 – val_loss: 0.1396
Epoch 482/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1111 – val_loss: 0.1532
Epoch 483/500
100/100 [==============================] – 0s 98us/sample – loss: 0.1115 – val_loss: 0.1376
Epoch 484/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1114 – val_loss: 0.1528
Epoch 485/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1108 – val_loss: 0.1403
Epoch 486/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1095 – val_loss: 0.1456
Epoch 487/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1081 – val_loss: 0.1416
Epoch 488/500
100/100 [==============================] – 0s 49us/sample – loss: 0.1066 – val_loss: 0.1427
Epoch 489/500
100/100 [==============================] – 0s 50us/sample – loss: 0.1062 – val_loss: 0.1437
Epoch 490/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1068 – val_loss: 0.1408
Epoch 491/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1075 – val_loss: 0.1473
Epoch 492/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1079 – val_loss: 0.1383
Epoch 493/500
100/100 [==============================] – 0s 66us/sample – loss: 0.1085 – val_loss: 0.1509
Epoch 494/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1092 – val_loss: 0.1392
Epoch 495/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1090 – val_loss: 0.1486
Epoch 496/500
100/100 [==============================] – 0s 48us/sample – loss: 0.1084 – val_loss: 0.1427
Epoch 497/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1081 – val_loss: 0.1445
Epoch 498/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1068 – val_loss: 0.1398
Epoch 499/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1064 – val_loss: 0.1452
Epoch 500/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1067 – val_loss: 0.1423
Out[11]:

We note that training loss becomes lower than validation loss, in fact lower than the model error variance 0.1. This is a clear sign of overfitting. Visual assessment confirms that this is indeed true.
In [12]:
y_hat = g_hat_over.predict(x_grid)
plt.plot(x_grid, y_hat, label=r”$\widehat{g}_{\mathrm{over}}(x)$”)
plt.plot(x_grid, g_grid, “b–“, label=”$g(x)$”)
plt.plot(x_sub, y_sub, “go”, markersize=3, label=”$(x^i,y^i)$”)
plt.xlabel(“$x$”)
plt.ylabel(“Value”)
plt.title(“Without dropout”, loc=’center’, fontsize=13)
plt.legend()
plt.show()



We can mitigate overfitting by a regularisation technique called dropout. In a dropout layer, inputs are randomly replaced by zeros with fixed probability during training. We introduce now a dropout layer with dropout probability $0.5$ after each hidden layer:
In [13]:
g_hat_dropout = keras.Sequential([
keras.layers.Dense(1000, activation=”relu”, input_shape=(1,)),
keras.layers.Dropout(0.5),
keras.layers.Dense(1000, activation=”relu”),
keras.layers.Dropout(0.5),
keras.layers.Dense(1000, activation=”relu”),
keras.layers.Dropout(0.5),
keras.layers.Dense(1, activation=”linear”)
]
)
g_hat_dropout.summary()

Model: “sequential_2″
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_8 (Dense) (None, 1000) 2000
_________________________________________________________________
dropout (Dropout) (None, 1000) 0
_________________________________________________________________
dense_9 (Dense) (None, 1000) 1001000
_________________________________________________________________
dropout_1 (Dropout) (None, 1000) 0
_________________________________________________________________
dense_10 (Dense) (None, 1000) 1001000
_________________________________________________________________
dropout_2 (Dropout) (None, 1000) 0
_________________________________________________________________
dense_11 (Dense) (None, 1) 1001
=================================================================
Total params: 2,005,001
Trainable params: 2,005,001
Non-trainable params: 0
_________________________________________________________________

We compile and train the new network $\widehat{g}_{\mathrm{dropout}}$ with other settings unchanged.
In [14]:
g_hat_dropout.compile(optimizer=”adam”, loss=”mean_squared_error”)
g_hat_dropout.fit(x_sub, y_sub, batch_size=100, epochs=500, validation_split=0.5)

Train on 100 samples, validate on 100 samples
Epoch 1/500
100/100 [==============================] – 0s 3ms/sample – loss: 0.5392 – val_loss: 0.6590
Epoch 2/500
100/100 [==============================] – 0s 45us/sample – loss: 0.5374 – val_loss: 0.6596
Epoch 3/500
100/100 [==============================] – 0s 50us/sample – loss: 0.5289 – val_loss: 0.6583
Epoch 4/500
100/100 [==============================] – 0s 68us/sample – loss: 0.5251 – val_loss: 0.6569
Epoch 5/500
100/100 [==============================] – 0s 52us/sample – loss: 0.5295 – val_loss: 0.6565
Epoch 6/500
100/100 [==============================] – 0s 58us/sample – loss: 0.5152 – val_loss: 0.6565
Epoch 7/500
100/100 [==============================] – 0s 59us/sample – loss: 0.5397 – val_loss: 0.6564
Epoch 8/500
100/100 [==============================] – 0s 59us/sample – loss: 0.5211 – val_loss: 0.6556
Epoch 9/500
100/100 [==============================] – 0s 56us/sample – loss: 0.5280 – val_loss: 0.6556
Epoch 10/500
100/100 [==============================] – 0s 55us/sample – loss: 0.5125 – val_loss: 0.6553
Epoch 11/500
100/100 [==============================] – 0s 42us/sample – loss: 0.5429 – val_loss: 0.6546
Epoch 12/500
100/100 [==============================] – 0s 48us/sample – loss: 0.5365 – val_loss: 0.6545
Epoch 13/500
100/100 [==============================] – 0s 42us/sample – loss: 0.5181 – val_loss: 0.6552
Epoch 14/500
100/100 [==============================] – 0s 42us/sample – loss: 0.5259 – val_loss: 0.6542
Epoch 15/500
100/100 [==============================] – 0s 44us/sample – loss: 0.5283 – val_loss: 0.6531
Epoch 16/500
100/100 [==============================] – 0s 41us/sample – loss: 0.5401 – val_loss: 0.6526
Epoch 17/500
100/100 [==============================] – 0s 47us/sample – loss: 0.5203 – val_loss: 0.6530
Epoch 18/500
100/100 [==============================] – 0s 42us/sample – loss: 0.5567 – val_loss: 0.6524
Epoch 19/500
100/100 [==============================] – 0s 53us/sample – loss: 0.5518 – val_loss: 0.6512
Epoch 20/500
100/100 [==============================] – 0s 51us/sample – loss: 0.5230 – val_loss: 0.6519
Epoch 21/500
100/100 [==============================] – 0s 39us/sample – loss: 0.5335 – val_loss: 0.6525
Epoch 22/500
100/100 [==============================] – 0s 57us/sample – loss: 0.5130 – val_loss: 0.6519
Epoch 23/500
100/100 [==============================] – 0s 54us/sample – loss: 0.5340 – val_loss: 0.6499
Epoch 24/500
100/100 [==============================] – 0s 58us/sample – loss: 0.5382 – val_loss: 0.6482
Epoch 25/500
100/100 [==============================] – 0s 53us/sample – loss: 0.5353 – val_loss: 0.6472
Epoch 26/500
100/100 [==============================] – 0s 72us/sample – loss: 0.5286 – val_loss: 0.6460
Epoch 27/500
100/100 [==============================] – 0s 68us/sample – loss: 0.5397 – val_loss: 0.6450
Epoch 28/500
100/100 [==============================] – 0s 111us/sample – loss: 0.5423 – val_loss: 0.6447
Epoch 29/500
100/100 [==============================] – 0s 90us/sample – loss: 0.5198 – val_loss: 0.6444
Epoch 30/500
100/100 [==============================] – 0s 109us/sample – loss: 0.5345 – val_loss: 0.6423
Epoch 31/500
100/100 [==============================] – 0s 85us/sample – loss: 0.4987 – val_loss: 0.6397
Epoch 32/500
100/100 [==============================] – 0s 65us/sample – loss: 0.4940 – val_loss: 0.6380
Epoch 33/500
100/100 [==============================] – 0s 63us/sample – loss: 0.5267 – val_loss: 0.6362
Epoch 34/500
100/100 [==============================] – 0s 72us/sample – loss: 0.5099 – val_loss: 0.6342
Epoch 35/500
100/100 [==============================] – 0s 87us/sample – loss: 0.5395 – val_loss: 0.6309
Epoch 36/500
100/100 [==============================] – 0s 77us/sample – loss: 0.5226 – val_loss: 0.6287
Epoch 37/500
100/100 [==============================] – 0s 55us/sample – loss: 0.5121 – val_loss: 0.6272
Epoch 38/500
100/100 [==============================] – 0s 54us/sample – loss: 0.5010 – val_loss: 0.6243
Epoch 39/500
100/100 [==============================] – 0s 55us/sample – loss: 0.5043 – val_loss: 0.6190
Epoch 40/500
100/100 [==============================] – 0s 66us/sample – loss: 0.5025 – val_loss: 0.6127
Epoch 41/500
100/100 [==============================] – 0s 76us/sample – loss: 0.5012 – val_loss: 0.6069
Epoch 42/500
100/100 [==============================] – 0s 71us/sample – loss: 0.4904 – val_loss: 0.6008
Epoch 43/500
100/100 [==============================] – 0s 50us/sample – loss: 0.4745 – val_loss: 0.5932
Epoch 44/500
100/100 [==============================] – 0s 42us/sample – loss: 0.4586 – val_loss: 0.5858
Epoch 45/500
100/100 [==============================] – 0s 48us/sample – loss: 0.4640 – val_loss: 0.5798
Epoch 46/500
100/100 [==============================] – 0s 52us/sample – loss: 0.4961 – val_loss: 0.5735
Epoch 47/500
100/100 [==============================] – 0s 50us/sample – loss: 0.4962 – val_loss: 0.5625
Epoch 48/500
100/100 [==============================] – 0s 74us/sample – loss: 0.4653 – val_loss: 0.5507
Epoch 49/500
100/100 [==============================] – 0s 56us/sample – loss: 0.4485 – val_loss: 0.5442
Epoch 50/500
100/100 [==============================] – 0s 50us/sample – loss: 0.4419 – val_loss: 0.5326
Epoch 51/500
100/100 [==============================] – 0s 71us/sample – loss: 0.4272 – val_loss: 0.5176
Epoch 52/500
100/100 [==============================] – 0s 64us/sample – loss: 0.4120 – val_loss: 0.5108
Epoch 53/500
100/100 [==============================] – 0s 78us/sample – loss: 0.4242 – val_loss: 0.5004
Epoch 54/500
100/100 [==============================] – 0s 46us/sample – loss: 0.4325 – val_loss: 0.4818
Epoch 55/500
100/100 [==============================] – 0s 63us/sample – loss: 0.4023 – val_loss: 0.4628
Epoch 56/500
100/100 [==============================] – 0s 55us/sample – loss: 0.3829 – val_loss: 0.4502
Epoch 57/500
100/100 [==============================] – 0s 51us/sample – loss: 0.4132 – val_loss: 0.4338
Epoch 58/500
100/100 [==============================] – 0s 62us/sample – loss: 0.3962 – val_loss: 0.4193
Epoch 59/500
100/100 [==============================] – 0s 80us/sample – loss: 0.3637 – val_loss: 0.4170
Epoch 60/500
100/100 [==============================] – 0s 48us/sample – loss: 0.3544 – val_loss: 0.4059
Epoch 61/500
100/100 [==============================] – 0s 83us/sample – loss: 0.3627 – val_loss: 0.3791
Epoch 62/500
100/100 [==============================] – 0s 53us/sample – loss: 0.3537 – val_loss: 0.3658
Epoch 63/500
100/100 [==============================] – 0s 57us/sample – loss: 0.3616 – val_loss: 0.3554
Epoch 64/500
100/100 [==============================] – 0s 65us/sample – loss: 0.3526 – val_loss: 0.3427
Epoch 65/500
100/100 [==============================] – 0s 52us/sample – loss: 0.3221 – val_loss: 0.3361
Epoch 66/500
100/100 [==============================] – 0s 94us/sample – loss: 0.3309 – val_loss: 0.3287
Epoch 67/500
100/100 [==============================] – 0s 76us/sample – loss: 0.3675 – val_loss: 0.3210
Epoch 68/500
100/100 [==============================] – 0s 86us/sample – loss: 0.3154 – val_loss: 0.3147
Epoch 69/500
100/100 [==============================] – 0s 59us/sample – loss: 0.3141 – val_loss: 0.3089
Epoch 70/500
100/100 [==============================] – 0s 55us/sample – loss: 0.2906 – val_loss: 0.3038
Epoch 71/500
100/100 [==============================] – 0s 75us/sample – loss: 0.3157 – val_loss: 0.2995
Epoch 72/500
100/100 [==============================] – 0s 61us/sample – loss: 0.3377 – val_loss: 0.2953
Epoch 73/500
100/100 [==============================] – 0s 78us/sample – loss: 0.3485 – val_loss: 0.2924
Epoch 74/500
100/100 [==============================] – 0s 87us/sample – loss: 0.2927 – val_loss: 0.2905
Epoch 75/500
100/100 [==============================] – 0s 69us/sample – loss: 0.3271 – val_loss: 0.2875
Epoch 76/500
100/100 [==============================] – 0s 104us/sample – loss: 0.3028 – val_loss: 0.2869
Epoch 77/500
100/100 [==============================] – 0s 49us/sample – loss: 0.3246 – val_loss: 0.2853
Epoch 78/500
100/100 [==============================] – 0s 71us/sample – loss: 0.2936 – val_loss: 0.2903
Epoch 79/500
100/100 [==============================] – 0s 58us/sample – loss: 0.3019 – val_loss: 0.2890
Epoch 80/500
100/100 [==============================] – 0s 52us/sample – loss: 0.3162 – val_loss: 0.2824
Epoch 81/500
100/100 [==============================] – 0s 47us/sample – loss: 0.2807 – val_loss: 0.2811
Epoch 82/500
100/100 [==============================] – 0s 40us/sample – loss: 0.2612 – val_loss: 0.2822
Epoch 83/500
100/100 [==============================] – 0s 74us/sample – loss: 0.2838 – val_loss: 0.2760
Epoch 84/500
100/100 [==============================] – 0s 81us/sample – loss: 0.2948 – val_loss: 0.2722
Epoch 85/500
100/100 [==============================] – 0s 64us/sample – loss: 0.2884 – val_loss: 0.2721
Epoch 86/500
100/100 [==============================] – 0s 66us/sample – loss: 0.2773 – val_loss: 0.2712
Epoch 87/500
100/100 [==============================] – 0s 54us/sample – loss: 0.2797 – val_loss: 0.2683
Epoch 88/500
100/100 [==============================] – 0s 66us/sample – loss: 0.2753 – val_loss: 0.2625
Epoch 89/500
100/100 [==============================] – 0s 72us/sample – loss: 0.2942 – val_loss: 0.2590
Epoch 90/500
100/100 [==============================] – 0s 71us/sample – loss: 0.2999 – val_loss: 0.2578
Epoch 91/500
100/100 [==============================] – 0s 81us/sample – loss: 0.2822 – val_loss: 0.2582
Epoch 92/500
100/100 [==============================] – 0s 75us/sample – loss: 0.2522 – val_loss: 0.2560
Epoch 93/500
100/100 [==============================] – 0s 59us/sample – loss: 0.2854 – val_loss: 0.2552
Epoch 94/500
100/100 [==============================] – 0s 67us/sample – loss: 0.2483 – val_loss: 0.2589
Epoch 95/500
100/100 [==============================] – 0s 66us/sample – loss: 0.2904 – val_loss: 0.2603
Epoch 96/500
100/100 [==============================] – 0s 55us/sample – loss: 0.2686 – val_loss: 0.2564
Epoch 97/500
100/100 [==============================] – 0s 62us/sample – loss: 0.3039 – val_loss: 0.2505
Epoch 98/500
100/100 [==============================] – 0s 60us/sample – loss: 0.2672 – val_loss: 0.2460
Epoch 99/500
100/100 [==============================] – 0s 60us/sample – loss: 0.2691 – val_loss: 0.2443
Epoch 100/500
100/100 [==============================] – 0s 54us/sample – loss: 0.2483 – val_loss: 0.2422
Epoch 101/500
100/100 [==============================] – 0s 55us/sample – loss: 0.2532 – val_loss: 0.2408
Epoch 102/500
100/100 [==============================] – 0s 66us/sample – loss: 0.2626 – val_loss: 0.2418
Epoch 103/500
100/100 [==============================] – 0s 48us/sample – loss: 0.2504 – val_loss: 0.2438
Epoch 104/500
100/100 [==============================] – 0s 75us/sample – loss: 0.2462 – val_loss: 0.2409
Epoch 105/500
100/100 [==============================] – 0s 71us/sample – loss: 0.2391 – val_loss: 0.2359
Epoch 106/500
100/100 [==============================] – 0s 83us/sample – loss: 0.2381 – val_loss: 0.2311
Epoch 107/500
100/100 [==============================] – 0s 56us/sample – loss: 0.2460 – val_loss: 0.2277
Epoch 108/500
100/100 [==============================] – 0s 73us/sample – loss: 0.2477 – val_loss: 0.2262
Epoch 109/500
100/100 [==============================] – 0s 50us/sample – loss: 0.2530 – val_loss: 0.2266
Epoch 110/500
100/100 [==============================] – 0s 73us/sample – loss: 0.2294 – val_loss: 0.2284
Epoch 111/500
100/100 [==============================] – 0s 67us/sample – loss: 0.2505 – val_loss: 0.2305
Epoch 112/500
100/100 [==============================] – 0s 53us/sample – loss: 0.2527 – val_loss: 0.2310
Epoch 113/500
100/100 [==============================] – 0s 57us/sample – loss: 0.2474 – val_loss: 0.2283
Epoch 114/500
100/100 [==============================] – 0s 67us/sample – loss: 0.2612 – val_loss: 0.2251
Epoch 115/500
100/100 [==============================] – 0s 68us/sample – loss: 0.2414 – val_loss: 0.2245
Epoch 116/500
100/100 [==============================] – 0s 53us/sample – loss: 0.2437 – val_loss: 0.2247
Epoch 117/500
100/100 [==============================] – 0s 66us/sample – loss: 0.2299 – val_loss: 0.2253
Epoch 118/500
100/100 [==============================] – 0s 67us/sample – loss: 0.2406 – val_loss: 0.2276
Epoch 119/500
100/100 [==============================] – 0s 59us/sample – loss: 0.2722 – val_loss: 0.2300
Epoch 120/500
100/100 [==============================] – 0s 51us/sample – loss: 0.2339 – val_loss: 0.2293
Epoch 121/500
100/100 [==============================] – 0s 67us/sample – loss: 0.2481 – val_loss: 0.2281
Epoch 122/500
100/100 [==============================] – 0s 52us/sample – loss: 0.2405 – val_loss: 0.2250
Epoch 123/500
100/100 [==============================] – 0s 67us/sample – loss: 0.2407 – val_loss: 0.2229
Epoch 124/500
100/100 [==============================] – 0s 55us/sample – loss: 0.2263 – val_loss: 0.2211
Epoch 125/500
100/100 [==============================] – 0s 73us/sample – loss: 0.2331 – val_loss: 0.2207
Epoch 126/500
100/100 [==============================] – 0s 67us/sample – loss: 0.2342 – val_loss: 0.2207
Epoch 127/500
100/100 [==============================] – 0s 48us/sample – loss: 0.2226 – val_loss: 0.2200
Epoch 128/500
100/100 [==============================] – 0s 57us/sample – loss: 0.2140 – val_loss: 0.2182
Epoch 129/500
100/100 [==============================] – 0s 75us/sample – loss: 0.2396 – val_loss: 0.2148
Epoch 130/500
100/100 [==============================] – 0s 82us/sample – loss: 0.2511 – val_loss: 0.2119
Epoch 131/500
100/100 [==============================] – 0s 57us/sample – loss: 0.2568 – val_loss: 0.2107
Epoch 132/500
100/100 [==============================] – 0s 71us/sample – loss: 0.2419 – val_loss: 0.2100
Epoch 133/500
100/100 [==============================] – 0s 74us/sample – loss: 0.2250 – val_loss: 0.2093
Epoch 134/500
100/100 [==============================] – 0s 57us/sample – loss: 0.2136 – val_loss: 0.2088
Epoch 135/500
100/100 [==============================] – 0s 73us/sample – loss: 0.2260 – val_loss: 0.2090
Epoch 136/500
100/100 [==============================] – 0s 72us/sample – loss: 0.2241 – val_loss: 0.2105
Epoch 137/500
100/100 [==============================] – 0s 64us/sample – loss: 0.2308 – val_loss: 0.2124
Epoch 138/500
100/100 [==============================] – 0s 67us/sample – loss: 0.2213 – val_loss: 0.2114
Epoch 139/500
100/100 [==============================] – 0s 79us/sample – loss: 0.2229 – val_loss: 0.2103
Epoch 140/500
100/100 [==============================] – 0s 62us/sample – loss: 0.2343 – val_loss: 0.2097
Epoch 141/500
100/100 [==============================] – 0s 76us/sample – loss: 0.2163 – val_loss: 0.2094
Epoch 142/500
100/100 [==============================] – 0s 63us/sample – loss: 0.2405 – val_loss: 0.2090
Epoch 143/500
100/100 [==============================] – 0s 60us/sample – loss: 0.2280 – val_loss: 0.2071
Epoch 144/500
100/100 [==============================] – 0s 67us/sample – loss: 0.2322 – val_loss: 0.2059
Epoch 145/500
100/100 [==============================] – 0s 69us/sample – loss: 0.2277 – val_loss: 0.2050
Epoch 146/500
100/100 [==============================] – 0s 61us/sample – loss: 0.2123 – val_loss: 0.2045
Epoch 147/500
100/100 [==============================] – 0s 68us/sample – loss: 0.2000 – val_loss: 0.2039
Epoch 148/500
100/100 [==============================] – 0s 83us/sample – loss: 0.2103 – val_loss: 0.2020
Epoch 149/500
100/100 [==============================] – 0s 59us/sample – loss: 0.2221 – val_loss: 0.1993
Epoch 150/500
100/100 [==============================] – 0s 124us/sample – loss: 0.2316 – val_loss: 0.1974
Epoch 151/500
100/100 [==============================] – 0s 112us/sample – loss: 0.2039 – val_loss: 0.1936
Epoch 152/500
100/100 [==============================] – 0s 53us/sample – loss: 0.2124 – val_loss: 0.1915
Epoch 153/500
100/100 [==============================] – 0s 71us/sample – loss: 0.2228 – val_loss: 0.1917
Epoch 154/500
100/100 [==============================] – 0s 62us/sample – loss: 0.2265 – val_loss: 0.1975
Epoch 155/500
100/100 [==============================] – 0s 58us/sample – loss: 0.2196 – val_loss: 0.1985
Epoch 156/500
100/100 [==============================] – 0s 111us/sample – loss: 0.2197 – val_loss: 0.1930
Epoch 157/500
100/100 [==============================] – 0s 70us/sample – loss: 0.2103 – val_loss: 0.1895
Epoch 158/500
100/100 [==============================] – 0s 67us/sample – loss: 0.2056 – val_loss: 0.1884
Epoch 159/500
100/100 [==============================] – 0s 98us/sample – loss: 0.2238 – val_loss: 0.1874
Epoch 160/500
100/100 [==============================] – 0s 119us/sample – loss: 0.2156 – val_loss: 0.1874
Epoch 161/500
100/100 [==============================] – 0s 56us/sample – loss: 0.2090 – val_loss: 0.1883
Epoch 162/500
100/100 [==============================] – 0s 85us/sample – loss: 0.2256 – val_loss: 0.1878
Epoch 163/500
100/100 [==============================] – 0s 59us/sample – loss: 0.2180 – val_loss: 0.1865
Epoch 164/500
100/100 [==============================] – 0s 84us/sample – loss: 0.2135 – val_loss: 0.1837
Epoch 165/500
100/100 [==============================] – 0s 81us/sample – loss: 0.2060 – val_loss: 0.1821
Epoch 166/500
100/100 [==============================] – 0s 79us/sample – loss: 0.2257 – val_loss: 0.1819
Epoch 167/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1962 – val_loss: 0.1823
Epoch 168/500
100/100 [==============================] – 0s 72us/sample – loss: 0.2262 – val_loss: 0.1820
Epoch 169/500
100/100 [==============================] – 0s 69us/sample – loss: 0.2130 – val_loss: 0.1820
Epoch 170/500
100/100 [==============================] – 0s 64us/sample – loss: 0.2243 – val_loss: 0.1844
Epoch 171/500
100/100 [==============================] – 0s 78us/sample – loss: 0.2163 – val_loss: 0.1900
Epoch 172/500
100/100 [==============================] – 0s 53us/sample – loss: 0.2290 – val_loss: 0.1930
Epoch 173/500
100/100 [==============================] – 0s 48us/sample – loss: 0.2169 – val_loss: 0.1890
Epoch 174/500
100/100 [==============================] – 0s 85us/sample – loss: 0.2080 – val_loss: 0.1860
Epoch 175/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1989 – val_loss: 0.1880
Epoch 176/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1853 – val_loss: 0.1872
Epoch 177/500
100/100 [==============================] – 0s 48us/sample – loss: 0.2132 – val_loss: 0.1858
Epoch 178/500
100/100 [==============================] – 0s 60us/sample – loss: 0.2237 – val_loss: 0.1883
Epoch 179/500
100/100 [==============================] – 0s 69us/sample – loss: 0.2131 – val_loss: 0.1872
Epoch 180/500
100/100 [==============================] – 0s 58us/sample – loss: 0.2156 – val_loss: 0.1779
Epoch 181/500
100/100 [==============================] – 0s 73us/sample – loss: 0.2122 – val_loss: 0.1749
Epoch 182/500
100/100 [==============================] – 0s 76us/sample – loss: 0.2070 – val_loss: 0.1741
Epoch 183/500
100/100 [==============================] – 0s 64us/sample – loss: 0.2256 – val_loss: 0.1751
Epoch 184/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1665 – val_loss: 0.1755
Epoch 185/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1935 – val_loss: 0.1760
Epoch 186/500
100/100 [==============================] – 0s 48us/sample – loss: 0.2049 – val_loss: 0.1745
Epoch 187/500
100/100 [==============================] – 0s 58us/sample – loss: 0.2006 – val_loss: 0.1767
Epoch 188/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1721 – val_loss: 0.1794
Epoch 189/500
100/100 [==============================] – 0s 54us/sample – loss: 0.2084 – val_loss: 0.1777
Epoch 190/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1801 – val_loss: 0.1728
Epoch 191/500
100/100 [==============================] – 0s 46us/sample – loss: 0.1871 – val_loss: 0.1751
Epoch 192/500
100/100 [==============================] – 0s 68us/sample – loss: 0.2123 – val_loss: 0.1747
Epoch 193/500
100/100 [==============================] – 0s 81us/sample – loss: 0.2061 – val_loss: 0.1710
Epoch 194/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1889 – val_loss: 0.1689
Epoch 195/500
100/100 [==============================] – 0s 46us/sample – loss: 0.1857 – val_loss: 0.1697
Epoch 196/500
100/100 [==============================] – 0s 84us/sample – loss: 0.1962 – val_loss: 0.1675
Epoch 197/500
100/100 [==============================] – 0s 87us/sample – loss: 0.2099 – val_loss: 0.1644
Epoch 198/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1839 – val_loss: 0.1677
Epoch 199/500
100/100 [==============================] – 0s 80us/sample – loss: 0.1971 – val_loss: 0.1726
Epoch 200/500
100/100 [==============================] – 0s 57us/sample – loss: 0.2242 – val_loss: 0.1679
Epoch 201/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1866 – val_loss: 0.1641
Epoch 202/500
100/100 [==============================] – 0s 78us/sample – loss: 0.1915 – val_loss: 0.1653
Epoch 203/500
100/100 [==============================] – 0s 65us/sample – loss: 0.2057 – val_loss: 0.1661
Epoch 204/500
100/100 [==============================] – 0s 68us/sample – loss: 0.2082 – val_loss: 0.1668
Epoch 205/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1822 – val_loss: 0.1706
Epoch 206/500
100/100 [==============================] – 0s 86us/sample – loss: 0.1723 – val_loss: 0.1732
Epoch 207/500
100/100 [==============================] – 0s 75us/sample – loss: 0.1900 – val_loss: 0.1703
Epoch 208/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1808 – val_loss: 0.1675
Epoch 209/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1961 – val_loss: 0.1670
Epoch 210/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1813 – val_loss: 0.1692
Epoch 211/500
100/100 [==============================] – 0s 82us/sample – loss: 0.1765 – val_loss: 0.1701
Epoch 212/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1932 – val_loss: 0.1751
Epoch 213/500
100/100 [==============================] – 0s 59us/sample – loss: 0.2053 – val_loss: 0.1803
Epoch 214/500
100/100 [==============================] – 0s 55us/sample – loss: 0.2185 – val_loss: 0.1694
Epoch 215/500
100/100 [==============================] – 0s 90us/sample – loss: 0.2025 – val_loss: 0.1608
Epoch 216/500
100/100 [==============================] – 0s 107us/sample – loss: 0.1912 – val_loss: 0.1614
Epoch 217/500
100/100 [==============================] – 0s 82us/sample – loss: 0.1822 – val_loss: 0.1660
Epoch 218/500
100/100 [==============================] – 0s 78us/sample – loss: 0.1809 – val_loss: 0.1594
Epoch 219/500
100/100 [==============================] – 0s 84us/sample – loss: 0.2018 – val_loss: 0.1560
Epoch 220/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1818 – val_loss: 0.1619
Epoch 221/500
100/100 [==============================] – 0s 72us/sample – loss: 0.2294 – val_loss: 0.1610
Epoch 222/500
100/100 [==============================] – 0s 58us/sample – loss: 0.2059 – val_loss: 0.1589
Epoch 223/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1718 – val_loss: 0.1620
Epoch 224/500
100/100 [==============================] – 0s 42us/sample – loss: 0.1634 – val_loss: 0.1648
Epoch 225/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1990 – val_loss: 0.1613
Epoch 226/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1890 – val_loss: 0.1593
Epoch 227/500
100/100 [==============================] – 0s 71us/sample – loss: 0.1972 – val_loss: 0.1586
Epoch 228/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1924 – val_loss: 0.1594
Epoch 229/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1925 – val_loss: 0.1603
Epoch 230/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1718 – val_loss: 0.1621
Epoch 231/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1912 – val_loss: 0.1619
Epoch 232/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1807 – val_loss: 0.1603
Epoch 233/500
100/100 [==============================] – 0s 71us/sample – loss: 0.1816 – val_loss: 0.1559
Epoch 234/500
100/100 [==============================] – 0s 83us/sample – loss: 0.1973 – val_loss: 0.1548
Epoch 235/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1865 – val_loss: 0.1572
Epoch 236/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1742 – val_loss: 0.1586
Epoch 237/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1804 – val_loss: 0.1562
Epoch 238/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1896 – val_loss: 0.1561
Epoch 239/500
100/100 [==============================] – 0s 77us/sample – loss: 0.1843 – val_loss: 0.1584
Epoch 240/500
100/100 [==============================] – 0s 42us/sample – loss: 0.1739 – val_loss: 0.1606
Epoch 241/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1849 – val_loss: 0.1595
Epoch 242/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1644 – val_loss: 0.1576
Epoch 243/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1730 – val_loss: 0.1513
Epoch 244/500
100/100 [==============================] – 0s 77us/sample – loss: 0.1796 – val_loss: 0.1479
Epoch 245/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1844 – val_loss: 0.1456
Epoch 246/500
100/100 [==============================] – 0s 59us/sample – loss: 0.2058 – val_loss: 0.1465
Epoch 247/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1832 – val_loss: 0.1476
Epoch 248/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1637 – val_loss: 0.1484
Epoch 249/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1703 – val_loss: 0.1486
Epoch 250/500
100/100 [==============================] – 0s 62us/sample – loss: 0.2128 – val_loss: 0.1466
Epoch 251/500
100/100 [==============================] – 0s 75us/sample – loss: 0.1856 – val_loss: 0.1426
Epoch 252/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1892 – val_loss: 0.1405
Epoch 253/500
100/100 [==============================] – 0s 67us/sample – loss: 0.2034 – val_loss: 0.1442
Epoch 254/500
100/100 [==============================] – 0s 44us/sample – loss: 0.1922 – val_loss: 0.1448
Epoch 255/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1857 – val_loss: 0.1408
Epoch 256/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1695 – val_loss: 0.1381
Epoch 257/500
100/100 [==============================] – 0s 83us/sample – loss: 0.1704 – val_loss: 0.1425
Epoch 258/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1553 – val_loss: 0.1439
Epoch 259/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1845 – val_loss: 0.1466
Epoch 260/500
100/100 [==============================] – 0s 77us/sample – loss: 0.1776 – val_loss: 0.1541
Epoch 261/500
100/100 [==============================] – 0s 46us/sample – loss: 0.1550 – val_loss: 0.1598
Epoch 262/500
100/100 [==============================] – 0s 83us/sample – loss: 0.1974 – val_loss: 0.1573
Epoch 263/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1880 – val_loss: 0.1531
Epoch 264/500
100/100 [==============================] – 0s 44us/sample – loss: 0.1720 – val_loss: 0.1558
Epoch 265/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1790 – val_loss: 0.1558
Epoch 266/500
100/100 [==============================] – 0s 94us/sample – loss: 0.1719 – val_loss: 0.1482
Epoch 267/500
100/100 [==============================] – 0s 86us/sample – loss: 0.1584 – val_loss: 0.1502
Epoch 268/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1898 – val_loss: 0.1547
Epoch 269/500
100/100 [==============================] – 0s 47us/sample – loss: 0.1479 – val_loss: 0.1523
Epoch 270/500
100/100 [==============================] – 0s 115us/sample – loss: 0.1806 – val_loss: 0.1483
Epoch 271/500
100/100 [==============================] – 0s 91us/sample – loss: 0.2071 – val_loss: 0.1463
Epoch 272/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1924 – val_loss: 0.1437
Epoch 273/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1728 – val_loss: 0.1411
Epoch 274/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1778 – val_loss: 0.1439
Epoch 275/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1754 – val_loss: 0.1451
Epoch 276/500
100/100 [==============================] – 0s 89us/sample – loss: 0.1745 – val_loss: 0.1450
Epoch 277/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1836 – val_loss: 0.1450
Epoch 278/500
100/100 [==============================] – 0s 83us/sample – loss: 0.1669 – val_loss: 0.1570
Epoch 279/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1984 – val_loss: 0.1566
Epoch 280/500
100/100 [==============================] – 0s 87us/sample – loss: 0.1739 – val_loss: 0.1471
Epoch 281/500
100/100 [==============================] – 0s 88us/sample – loss: 0.2104 – val_loss: 0.1498
Epoch 282/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1767 – val_loss: 0.1503
Epoch 283/500
100/100 [==============================] – 0s 66us/sample – loss: 0.2063 – val_loss: 0.1453
Epoch 284/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1812 – val_loss: 0.1423
Epoch 285/500
100/100 [==============================] – 0s 73us/sample – loss: 0.2400 – val_loss: 0.1487
Epoch 286/500
100/100 [==============================] – 0s 95us/sample – loss: 0.1791 – val_loss: 0.1537
Epoch 287/500
100/100 [==============================] – 0s 76us/sample – loss: 0.2043 – val_loss: 0.1508
Epoch 288/500
100/100 [==============================] – 0s 93us/sample – loss: 0.1748 – val_loss: 0.1496
Epoch 289/500
100/100 [==============================] – 0s 80us/sample – loss: 0.1785 – val_loss: 0.1495
Epoch 290/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1947 – val_loss: 0.1517
Epoch 291/500
100/100 [==============================] – 0s 85us/sample – loss: 0.1796 – val_loss: 0.1536
Epoch 292/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1643 – val_loss: 0.1554
Epoch 293/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1566 – val_loss: 0.1560
Epoch 294/500
100/100 [==============================] – 0s 81us/sample – loss: 0.1547 – val_loss: 0.1566
Epoch 295/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1732 – val_loss: 0.1562
Epoch 296/500
100/100 [==============================] – 0s 91us/sample – loss: 0.1705 – val_loss: 0.1549
Epoch 297/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1651 – val_loss: 0.1536
Epoch 298/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1809 – val_loss: 0.1491
Epoch 299/500
100/100 [==============================] – 0s 82us/sample – loss: 0.1662 – val_loss: 0.1448
Epoch 300/500
100/100 [==============================] – 0s 89us/sample – loss: 0.1551 – val_loss: 0.1385
Epoch 301/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1769 – val_loss: 0.1351
Epoch 302/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1611 – val_loss: 0.1363
Epoch 303/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1697 – val_loss: 0.1347
Epoch 304/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1868 – val_loss: 0.1346
Epoch 305/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1744 – val_loss: 0.1355
Epoch 306/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1941 – val_loss: 0.1390
Epoch 307/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1848 – val_loss: 0.1428
Epoch 308/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1852 – val_loss: 0.1439
Epoch 309/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1615 – val_loss: 0.1432
Epoch 310/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1684 – val_loss: 0.1445
Epoch 311/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1743 – val_loss: 0.1516
Epoch 312/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1776 – val_loss: 0.1584
Epoch 313/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1715 – val_loss: 0.1595
Epoch 314/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1905 – val_loss: 0.1508
Epoch 315/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1892 – val_loss: 0.1476
Epoch 316/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1905 – val_loss: 0.1472
Epoch 317/500
100/100 [==============================] – 0s 79us/sample – loss: 0.1679 – val_loss: 0.1471
Epoch 318/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1612 – val_loss: 0.1414
Epoch 319/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1605 – val_loss: 0.1381
Epoch 320/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1952 – val_loss: 0.1403
Epoch 321/500
100/100 [==============================] – 0s 88us/sample – loss: 0.1560 – val_loss: 0.1426
Epoch 322/500
100/100 [==============================] – 0s 79us/sample – loss: 0.1578 – val_loss: 0.1404
Epoch 323/500
100/100 [==============================] – 0s 79us/sample – loss: 0.1605 – val_loss: 0.1395
Epoch 324/500
100/100 [==============================] – 0s 102us/sample – loss: 0.1563 – val_loss: 0.1404
Epoch 325/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1639 – val_loss: 0.1442
Epoch 326/500
100/100 [==============================] – 0s 85us/sample – loss: 0.1589 – val_loss: 0.1436
Epoch 327/500
100/100 [==============================] – 0s 80us/sample – loss: 0.1693 – val_loss: 0.1411
Epoch 328/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1778 – val_loss: 0.1393
Epoch 329/500
100/100 [==============================] – 0s 81us/sample – loss: 0.1735 – val_loss: 0.1411
Epoch 330/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1807 – val_loss: 0.1433
Epoch 331/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1698 – val_loss: 0.1423
Epoch 332/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1581 – val_loss: 0.1420
Epoch 333/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1599 – val_loss: 0.1450
Epoch 334/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1693 – val_loss: 0.1440
Epoch 335/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1715 – val_loss: 0.1408
Epoch 336/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1588 – val_loss: 0.1391
Epoch 337/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1637 – val_loss: 0.1395
Epoch 338/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1918 – val_loss: 0.1402
Epoch 339/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1664 – val_loss: 0.1397
Epoch 340/500
100/100 [==============================] – 0s 66us/sample – loss: 0.1974 – val_loss: 0.1398
Epoch 341/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1459 – val_loss: 0.1419
Epoch 342/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1788 – val_loss: 0.1453
Epoch 343/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1861 – val_loss: 0.1411
Epoch 344/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1718 – val_loss: 0.1385
Epoch 345/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1475 – val_loss: 0.1372
Epoch 346/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1608 – val_loss: 0.1383
Epoch 347/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1818 – val_loss: 0.1378
Epoch 348/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1773 – val_loss: 0.1380
Epoch 349/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1527 – val_loss: 0.1389
Epoch 350/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1664 – val_loss: 0.1439
Epoch 351/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1736 – val_loss: 0.1468
Epoch 352/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1690 – val_loss: 0.1479
Epoch 353/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1660 – val_loss: 0.1488
Epoch 354/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1811 – val_loss: 0.1467
Epoch 355/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1578 – val_loss: 0.1440
Epoch 356/500
100/100 [==============================] – 0s 77us/sample – loss: 0.1571 – val_loss: 0.1408
Epoch 357/500
100/100 [==============================] – 0s 48us/sample – loss: 0.2046 – val_loss: 0.1397
Epoch 358/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1787 – val_loss: 0.1389
Epoch 359/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1653 – val_loss: 0.1389
Epoch 360/500
100/100 [==============================] – 0s 71us/sample – loss: 0.1499 – val_loss: 0.1409
Epoch 361/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1493 – val_loss: 0.1410
Epoch 362/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1384 – val_loss: 0.1400
Epoch 363/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1786 – val_loss: 0.1375
Epoch 364/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1555 – val_loss: 0.1373
Epoch 365/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1866 – val_loss: 0.1399
Epoch 366/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1912 – val_loss: 0.1401
Epoch 367/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1855 – val_loss: 0.1396
Epoch 368/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1761 – val_loss: 0.1392
Epoch 369/500
100/100 [==============================] – 0s 83us/sample – loss: 0.1703 – val_loss: 0.1375
Epoch 370/500
100/100 [==============================] – 0s 145us/sample – loss: 0.1587 – val_loss: 0.1379
Epoch 371/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1628 – val_loss: 0.1399
Epoch 372/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1991 – val_loss: 0.1395
Epoch 373/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1452 – val_loss: 0.1398
Epoch 374/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1551 – val_loss: 0.1420
Epoch 375/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1855 – val_loss: 0.1427
Epoch 376/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1682 – val_loss: 0.1411
Epoch 377/500
100/100 [==============================] – 0s 77us/sample – loss: 0.1621 – val_loss: 0.1415
Epoch 378/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1805 – val_loss: 0.1428
Epoch 379/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1682 – val_loss: 0.1431
Epoch 380/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1466 – val_loss: 0.1422
Epoch 381/500
100/100 [==============================] – 0s 78us/sample – loss: 0.1529 – val_loss: 0.1403
Epoch 382/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1515 – val_loss: 0.1392
Epoch 383/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1945 – val_loss: 0.1389
Epoch 384/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1709 – val_loss: 0.1387
Epoch 385/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1447 – val_loss: 0.1388
Epoch 386/500
100/100 [==============================] – 0s 97us/sample – loss: 0.1930 – val_loss: 0.1385
Epoch 387/500
100/100 [==============================] – 0s 133us/sample – loss: 0.1558 – val_loss: 0.1385
Epoch 388/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1455 – val_loss: 0.1393
Epoch 389/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1648 – val_loss: 0.1389
Epoch 390/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1455 – val_loss: 0.1370
Epoch 391/500
100/100 [==============================] – 0s 78us/sample – loss: 0.1681 – val_loss: 0.1355
Epoch 392/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1687 – val_loss: 0.1351
Epoch 393/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1728 – val_loss: 0.1349
Epoch 394/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1747 – val_loss: 0.1336
Epoch 395/500
100/100 [==============================] – 0s 78us/sample – loss: 0.1740 – val_loss: 0.1338
Epoch 396/500
100/100 [==============================] – 0s 78us/sample – loss: 0.1520 – val_loss: 0.1343
Epoch 397/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1530 – val_loss: 0.1343
Epoch 398/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1739 – val_loss: 0.1348
Epoch 399/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1575 – val_loss: 0.1377
Epoch 400/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1613 – val_loss: 0.1382
Epoch 401/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1652 – val_loss: 0.1375
Epoch 402/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1547 – val_loss: 0.1357
Epoch 403/500
100/100 [==============================] – 0s 113us/sample – loss: 0.1597 – val_loss: 0.1358
Epoch 404/500
100/100 [==============================] – 0s 95us/sample – loss: 0.1769 – val_loss: 0.1370
Epoch 405/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1506 – val_loss: 0.1375
Epoch 406/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1580 – val_loss: 0.1376
Epoch 407/500
100/100 [==============================] – 0s 82us/sample – loss: 0.1506 – val_loss: 0.1372
Epoch 408/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1780 – val_loss: 0.1384
Epoch 409/500
100/100 [==============================] – 0s 86us/sample – loss: 0.1490 – val_loss: 0.1410
Epoch 410/500
100/100 [==============================] – 0s 51us/sample – loss: 0.1474 – val_loss: 0.1426
Epoch 411/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1808 – val_loss: 0.1440
Epoch 412/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1914 – val_loss: 0.1437
Epoch 413/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1765 – val_loss: 0.1401
Epoch 414/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1817 – val_loss: 0.1364
Epoch 415/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1485 – val_loss: 0.1384
Epoch 416/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1765 – val_loss: 0.1412
Epoch 417/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1694 – val_loss: 0.1395
Epoch 418/500
100/100 [==============================] – 0s 49us/sample – loss: 0.2022 – val_loss: 0.1370
Epoch 419/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1534 – val_loss: 0.1391
Epoch 420/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1697 – val_loss: 0.1389
Epoch 421/500
100/100 [==============================] – 0s 59us/sample – loss: 0.2027 – val_loss: 0.1383
Epoch 422/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1380 – val_loss: 0.1408
Epoch 423/500
100/100 [==============================] – 0s 47us/sample – loss: 0.1726 – val_loss: 0.1429
Epoch 424/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1476 – val_loss: 0.1451
Epoch 425/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1669 – val_loss: 0.1412
Epoch 426/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1740 – val_loss: 0.1377
Epoch 427/500
100/100 [==============================] – 0s 48us/sample – loss: 0.1653 – val_loss: 0.1376
Epoch 428/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1611 – val_loss: 0.1395
Epoch 429/500
100/100 [==============================] – 0s 47us/sample – loss: 0.1867 – val_loss: 0.1415
Epoch 430/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1691 – val_loss: 0.1402
Epoch 431/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1631 – val_loss: 0.1402
Epoch 432/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1622 – val_loss: 0.1401
Epoch 433/500
100/100 [==============================] – 0s 48us/sample – loss: 0.1849 – val_loss: 0.1408
Epoch 434/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1859 – val_loss: 0.1419
Epoch 435/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1816 – val_loss: 0.1432
Epoch 436/500
100/100 [==============================] – 0s 66us/sample – loss: 0.1698 – val_loss: 0.1438
Epoch 437/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1486 – val_loss: 0.1427
Epoch 438/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1636 – val_loss: 0.1401
Epoch 439/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1560 – val_loss: 0.1384
Epoch 440/500
100/100 [==============================] – 0s 52us/sample – loss: 0.1643 – val_loss: 0.1357
Epoch 441/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1604 – val_loss: 0.1338
Epoch 442/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1663 – val_loss: 0.1330
Epoch 443/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1664 – val_loss: 0.1333
Epoch 444/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1727 – val_loss: 0.1337
Epoch 445/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1256 – val_loss: 0.1331
Epoch 446/500
100/100 [==============================] – 0s 54us/sample – loss: 0.1480 – val_loss: 0.1341
Epoch 447/500
100/100 [==============================] – 0s 80us/sample – loss: 0.1709 – val_loss: 0.1371
Epoch 448/500
100/100 [==============================] – 0s 124us/sample – loss: 0.1778 – val_loss: 0.1386
Epoch 449/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1490 – val_loss: 0.1399
Epoch 450/500
100/100 [==============================] – 0s 90us/sample – loss: 0.1642 – val_loss: 0.1404
Epoch 451/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1621 – val_loss: 0.1424
Epoch 452/500
100/100 [==============================] – 0s 66us/sample – loss: 0.1572 – val_loss: 0.1424
Epoch 453/500
100/100 [==============================] – 0s 81us/sample – loss: 0.1636 – val_loss: 0.1383
Epoch 454/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1636 – val_loss: 0.1351
Epoch 455/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1696 – val_loss: 0.1365
Epoch 456/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1625 – val_loss: 0.1407
Epoch 457/500
100/100 [==============================] – 0s 64us/sample – loss: 0.1607 – val_loss: 0.1433
Epoch 458/500
100/100 [==============================] – 0s 48us/sample – loss: 0.1593 – val_loss: 0.1454
Epoch 459/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1691 – val_loss: 0.1434
Epoch 460/500
100/100 [==============================] – 0s 75us/sample – loss: 0.1750 – val_loss: 0.1424
Epoch 461/500
100/100 [==============================] – 0s 49us/sample – loss: 0.1662 – val_loss: 0.1438
Epoch 462/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1885 – val_loss: 0.1422
Epoch 463/500
100/100 [==============================] – 0s 78us/sample – loss: 0.1526 – val_loss: 0.1403
Epoch 464/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1440 – val_loss: 0.1394
Epoch 465/500
100/100 [==============================] – 0s 48us/sample – loss: 0.1544 – val_loss: 0.1396
Epoch 466/500
100/100 [==============================] – 0s 58us/sample – loss: 0.1528 – val_loss: 0.1397
Epoch 467/500
100/100 [==============================] – 0s 82us/sample – loss: 0.1650 – val_loss: 0.1392
Epoch 468/500
100/100 [==============================] – 0s 91us/sample – loss: 0.1587 – val_loss: 0.1402
Epoch 469/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1647 – val_loss: 0.1418
Epoch 470/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1438 – val_loss: 0.1408
Epoch 471/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1433 – val_loss: 0.1379
Epoch 472/500
100/100 [==============================] – 0s 60us/sample – loss: 0.1543 – val_loss: 0.1348
Epoch 473/500
100/100 [==============================] – 0s 55us/sample – loss: 0.1887 – val_loss: 0.1335
Epoch 474/500
100/100 [==============================] – 0s 53us/sample – loss: 0.1862 – val_loss: 0.1332
Epoch 475/500
100/100 [==============================] – 0s 69us/sample – loss: 0.1903 – val_loss: 0.1330
Epoch 476/500
100/100 [==============================] – 0s 50us/sample – loss: 0.1433 – val_loss: 0.1332
Epoch 477/500
100/100 [==============================] – 0s 91us/sample – loss: 0.1514 – val_loss: 0.1357
Epoch 478/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1333 – val_loss: 0.1371
Epoch 479/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1644 – val_loss: 0.1359
Epoch 480/500
100/100 [==============================] – 0s 74us/sample – loss: 0.1524 – val_loss: 0.1323
Epoch 481/500
100/100 [==============================] – 0s 56us/sample – loss: 0.1944 – val_loss: 0.1317
Epoch 482/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1775 – val_loss: 0.1326
Epoch 483/500
100/100 [==============================] – 0s 61us/sample – loss: 0.1607 – val_loss: 0.1340
Epoch 484/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1803 – val_loss: 0.1356
Epoch 485/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1708 – val_loss: 0.1393
Epoch 486/500
100/100 [==============================] – 0s 68us/sample – loss: 0.1584 – val_loss: 0.1423
Epoch 487/500
100/100 [==============================] – 0s 42us/sample – loss: 0.1886 – val_loss: 0.1423
Epoch 488/500
100/100 [==============================] – 0s 76us/sample – loss: 0.1706 – val_loss: 0.1385
Epoch 489/500
100/100 [==============================] – 0s 73us/sample – loss: 0.1727 – val_loss: 0.1366
Epoch 490/500
100/100 [==============================] – 0s 63us/sample – loss: 0.1443 – val_loss: 0.1370
Epoch 491/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1739 – val_loss: 0.1369
Epoch 492/500
100/100 [==============================] – 0s 62us/sample – loss: 0.1701 – val_loss: 0.1367
Epoch 493/500
100/100 [==============================] – 0s 87us/sample – loss: 0.1665 – val_loss: 0.1388
Epoch 494/500
100/100 [==============================] – 0s 67us/sample – loss: 0.1709 – val_loss: 0.1436
Epoch 495/500
100/100 [==============================] – 0s 85us/sample – loss: 0.1716 – val_loss: 0.1438
Epoch 496/500
100/100 [==============================] – 0s 65us/sample – loss: 0.1788 – val_loss: 0.1412
Epoch 497/500
100/100 [==============================] – 0s 59us/sample – loss: 0.1486 – val_loss: 0.1381
Epoch 498/500
100/100 [==============================] – 0s 72us/sample – loss: 0.1554 – val_loss: 0.1368
Epoch 499/500
100/100 [==============================] – 0s 57us/sample – loss: 0.1641 – val_loss: 0.1367
Epoch 500/500
100/100 [==============================] – 0s 70us/sample – loss: 0.1560 – val_loss: 0.1366
Out[14]:

We do not see a major discrepancy between training and validation losses. Also training loss tends to stay above the model error variance $0.1$. Let us visually study the fit.
In [15]:
y_hat = g_hat_dropout.predict(x_grid)
plt.plot(x_grid, y_hat, label=r”$\widehat{g}_{\mathrm{dropout}}(x)$”)
plt.plot(x_grid, g_grid, “b–“, label=”$g(x)$”)
plt.plot(x_sub, y_sub, “go”, markersize=3, label=”$(x^i,y^i)$”)
plt.xlabel(“$x$”)
plt.ylabel(“Value”)
plt.title(“With dropout”, loc=’center’, fontsize=13)
plt.legend()
plt.show()



While the fit is not perfect (due to the small amount of samples), the overfit problem has disappeared.
In [0]: