CS计算机代考程序代写 python chain deep learning Keras cache COSC2779LabExercises_W5

COSC2779LabExercises_W5

¶
COSC 2779 | Deep Learning
¶
Week 5 Lab Exercises: **Data loader and Augmentation**
¶

Introduction¶
This lab is aimed at understanding how to write your own data loader and learn the importance of data augmentation. During this lab you will:

Learn to do data augmentation
Explore different data loading mechanisms in TensorFlow
Implement CNN
Write your own dataloader
Explore more functionality in TensorBoard

This notebook is designed to run on Google Colab. If you like to run this on your local machine, make sure that you have installed TensorFlow version 2.0.

Setting up the Notebook¶
Lets first load the packages we need.

In [ ]:

import tensorflow as tf
AUTOTUNE = tf.data.experimental.AUTOTUNE
import numpy as np
import pandas as pd

import tensorflow_datasets as tfds
import pathlib
import shutil
import tempfile

from IPython import display
from matplotlib import pyplot as plt

We can use the tensor board to view the learning curves. Lets first set it up.

In [ ]:

logdir = pathlib.Path(tempfile.mkdtemp())/”tensorboard_logs”
shutil.rmtree(logdir, ignore_errors=True)

# Load the TensorBoard notebook extension
%load_ext tensorboard

# Open an embedded TensorBoard viewer
%tensorboard –logdir {logdir}/models

We can also write our own function to plot the models training history ones training has completed.

In [ ]:

from itertools import cycle
def plotter(history_hold, metric = ‘binary_crossentropy’, ylim=[0.0, 1.0]):
cycol = cycle(‘bgrcmk’)
for name, item in history_hold.items():
y_train = item.history[metric]
y_val = item.history[‘val_’ + metric]
x_train = np.arange(0,len(y_val))

c=next(cycol)

plt.plot(x_train, y_train, c+’-‘, label=name+’_train’)
plt.plot(x_train, y_val, c+’–‘, label=name+’_val’)

plt.legend()
plt.xlim([1, max(plt.xlim())])
plt.ylim(ylim)
plt.xlabel(‘Epoch’)
plt.ylabel(metric)
plt.grid(True)

Data Augmentation¶
Image Augmentation is the process of generating new images for training our deep learning model. These new images are generated using the existing training images and hence we don’t have to collect them manually.

Data augmentation is a common technique to improve results and avoid overfitting in neural networks.

Common augmentation techniques used in deep learning are discussed in lecture 4. To understand the importance of augmentation let’s train a simple model with and without augmentation and observe.

Simple Example¶
We will use the MNIST dataset for this task.

In [ ]:

dataset, info = tfds.load(‘mnist’, as_supervised=True, with_info=True)
train_dataset, test_dataset = dataset[‘train’], dataset[‘test’]

num_train_examples= info.splits[‘train’].num_examples

Write a function to augment the images. Here we first convert the images to range [0,1] and then do the following augmetnations:

resize the image to 34 x 34 by padding
Crop it back to 28 x 28. The combined effect would be translation of the image in both x, y directions.
Change the brightness of the image pixels randomly.

In [ ]:

def convert(image, label):
image = tf.image.convert_image_dtype(image, tf.float32) # Cast and normalize the image to [0,1]
return image, label

def augment(image,label):
image,label = convert(image, label)
image = tf.image.resize_with_crop_or_pad(image, 34, 34) # Add 6 pixels of padding
image = tf.image.random_crop(image, size=[28, 28, 1]) # Random crop back to 28×28
image = tf.image.random_brightness(image, max_delta=0.5) # Random brightness

return image,label

Next step is to apply the above augmentations to the training dataset and create augmented datasets. Here we are going to limit the data to 2048 instances to highlight the affect of augmentation in preventing overfitting. the original MNIST dataset has many examples and it is unlikely to overfit.

In [ ]:

BATCH_SIZE = 64
# Only use a subset of the data so it’s easier to overfit, for this tutorial
NUM_EXAMPLES = 2048

augmented_train_batches = (
train_dataset
# Only train on a subset, so you can quickly see the effect.
.take(NUM_EXAMPLES)
.cache()
.shuffle(num_train_examples//4)
# The augmentation is added here.
.map(augment, num_parallel_calls=AUTOTUNE)
.batch(BATCH_SIZE)
.prefetch(AUTOTUNE)
)

For comparisons we will generate a non augmented dataset as well.

In [ ]:

non_augmented_train_batches = (
train_dataset
# Only train on a subset, so you can quickly see the effect.
.take(NUM_EXAMPLES)
.cache()
.shuffle(num_train_examples//4)
# No augmentation.
.map(convert, num_parallel_calls=AUTOTUNE)
.batch(BATCH_SIZE)
.prefetch(AUTOTUNE)
)

Validation batches are not usually augmented.

In [ ]:

validation_batches = (
test_dataset
.map(convert, num_parallel_calls=AUTOTUNE)
.batch(BATCH_SIZE)
)

Create callbacks for tensorboard. histogram_freq=1 writes the histogram of weights while training. This enables us to look at the evolution of weights during the training process.

In [ ]:

m_histories = {}

def get_callbacks(name):
return [
tf.keras.callbacks.TensorBoard(logdir/name, histogram_freq=1),
]

**TODO:** Create a model with two fully connected hidden layers with 4096 `ReLU` units.

Train the model without augmentation

In [ ]:

model_no_aug = tf.keras.Sequential([
# Complete the model
])
model_no_aug.compile(optimizer = ‘adam’,
loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=[tf.losses.SparseCategoricalCrossentropy(from_logits=True, name=’SparseCategoricalCrossentropy’), ‘accuracy’])

m_histories[‘No_augmentation’] = model_no_aug.fit(non_augmented_train_batches, epochs=50, validation_data=validation_batches, verbose=0, callbacks=get_callbacks(‘models/noAug’))

**TODO:** Create a model with two fully connected hidden layers with 4096 `ReLU` units. Same model as above.

Train the model with augmentation

In [ ]:

model_with_aug = tf.keras.Sequential([
# Complete the model
])
model_with_aug.compile(optimizer = ‘adam’,
loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=[tf.losses.SparseCategoricalCrossentropy(from_logits=True, name=’SparseCategoricalCrossentropy’), ‘accuracy’])

m_histories[‘With_augmentation’] = model_with_aug.fit(augmented_train_batches, epochs=50, validation_data=validation_batches, verbose=0, callbacks=get_callbacks(‘models/Aug’))

Plot the results

In [ ]:

plotter(m_histories, ylim=[0.0, 1.1], metric = ‘SparseCategoricalCrossentropy’)

In [ ]:

plotter(m_histories, ylim=[0.0, 1.1], metric = ‘accuracy’)

Did you observes overfitting in the model trained without augmentation?

Has augmentation prevented overfitting?

What are the other types of augmentation you can use for this task?

Explore other available options in TensorFlow Image Documentation](https://www.tensorflow.org/api_docs/python/tf/image)

Detailed Example¶
In this demo we will use CNN to predict the orientation of a face in a 32 by 32 pixel image. The data set used is from: Chapter 4 of Machine Learning book by Tom Mitchell

The faces of 20 different people are captured at 4 orientations: Left, Right, Up, Straight. Images from each individual is in a separate folder and the label (orientation) for a specific image is given by the image name. glickman_up_neutral_sunglasses.pgm -> up

In [ ]:

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

print(“Tensorflow version is: “, tf.__version__)
assert tf.__version__[0] == ‘2’

The image names and the labels are converted to CSV files and are available on canvas. Upload the data files to the google drive and then get it to the colab instance using the commands below.

In [ ]:

from google.colab import drive
drive.mount(‘/content/drive’)

!cp /content/drive/’My Drive’/COSC2779/COSC2779lab5/faces_TM.zip .
!unzip -q -o faces_TM.zip
!rm faces_TM.zip

Load the dataset names and labels into pandas data frames.

When developing machine learning models one would usually do some data exploration at this point. Check how the images look like and identify the properties of the task. This information should be used to justify your design choices

In [ ]:

import pandas as pd
import numpy as np

train_data_df = pd.read_csv(‘./faces_TM/TrainData_faces.csv’)
val_data_df = pd.read_csv(‘./faces_TM/ValData_faces.csv’)
train_data_df.head(5)

We can now read the data and organise them to a tensoflow dataset. You could use the data generators to directly train the model. However since We like to use the same code structure from the previous simple example, the data is organised to a dataset.

In [ ]:

from tensorflow.keras.preprocessing.image import ImageDataGenerator

BATCH_SIZE = 16

def make_train_generator():
train_datagen = ImageDataGenerator(data_format=’channels_last’)
train_generator = train_datagen.flow_from_dataframe(
dataframe=train_data_df,
directory=’./’,
x_col=”image_path”,
y_col=”label”,
target_size=(32, 32),
batch_size=1,
color_mode=”grayscale”,
shuffle=False,
class_mode=’categorical’)
return train_generator

def make_val_generator():
datagen = ImageDataGenerator(data_format=’channels_last’)
generator = datagen.flow_from_dataframe(
dataframe=val_data_df,
directory=’./’,
x_col=”image_path”,
y_col=”label”,
target_size=(32, 32),
batch_size=1,
color_mode=”grayscale”,
shuffle=False,
class_mode=’categorical’)
return generator

train_dataset = tf.data.Dataset.from_generator(make_train_generator,output_types=(tf.float32, tf.float32), output_shapes=([1,32,32,1], [1,4]))
vaidation_dataset = tf.data.Dataset.from_generator(make_val_generator,output_types=(tf.float32, tf.float32), output_shapes=([1,32,32,1], [1,4]))

VAL_DATA_LEN = 155
TRAIN_DATA_LEN = 469

Print the shapes of the output tensor to check if the dataset generation is working.

In [ ]:

for image, label in train_dataset.take(5):
print(image.shape, label.shape)

for image, label in vaidation_dataset.take(5):
print(image.shape, label.shape)

Write a function to augment the images. Here we first convert the image to range [0,1] and then do the following augmentations:

resize the image to 34 x 34 by padding
Rotate in the range [0,15] degrees
Crop it back to 28 x 28. the combined effect would be translation of the image in both x, y directions.
Change the brightness of the image pixels randomly.

In [ ]:

!pip install tfa-nightly

In [ ]:

import tensorflow_addons as tfa
import numpy as np

@tf.function
def rotate_tf(image,ang_deg=15):

random_angles = tf.random.uniform(shape = (), minval = -np.deg2rad(ang_deg), maxval = np.deg2rad(ang_deg))
return tfa.image.rotate(image,random_angles)

def convert(image, label):
image = tf.image.convert_image_dtype(image, tf.float32) # Cast and normalize the image to [0,1]
image = image[0,:]
label = label[0,:]
return image, label

def augment(image,label):
image,label = convert(image, label)
image = tf.image.resize_with_crop_or_pad(image, 37, 37) # Add 5 pixels of padding
image = rotate_tf(image, 5) # Rorate in the range [0,10]
image = tf.image.random_crop(image, size=[32, 32, 1]) # Random crop back to 28×28
image = tf.image.random_brightness(image, max_delta=0.5) # Random brightness

return image,label

Next step is to apply the above augmentations to the training dataset and create augmented datasets.

In [ ]:

augmented_train_batches = train_dataset.take(TRAIN_DATA_LEN).cache()
augmented_train_batches = augmented_train_batches.shuffle(TRAIN_DATA_LEN).map(augment, num_parallel_calls=AUTOTUNE).batch(BATCH_SIZE, drop_remainder=True)

In [ ]:

non_augmented_train_batches = train_dataset.take(TRAIN_DATA_LEN).cache()
non_augmented_train_batches = non_augmented_train_batches.shuffle(TRAIN_DATA_LEN).map(convert, num_parallel_calls=AUTOTUNE).batch(BATCH_SIZE, drop_remainder=True)

In [ ]:

validation_batches = vaidation_dataset.take(VAL_DATA_LEN).cache()
validation_batches =validation_batches.map(convert, num_parallel_calls=AUTOTUNE).batch(BATCH_SIZE, drop_remainder=True)

Setup callback for tensorboard

In [ ]:

m_histories = {}

def get_callbacks(name):
return [
tf.keras.callbacks.TensorBoard(logdir/name),
]

Write a function to create a CNN model. This model has several conv+pooling layers followed by a MLP classifier. L2 regularisation and dropout is also employed. Note that this function is not optimised for this task. You are encouraged to find the best model as a self study exercise.

In [ ]:

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Lambda, Input
from tensorflow.keras.layers import Activation, Dropout, Flatten, Dense
from tensorflow.keras.metrics import categorical_accuracy
from tensorflow.keras import regularizers, optimizers

def get_model():
model_cnn = Sequential()

# input
model_cnn.add(Input(shape=(32, 32, 1)))

# Conv Layer 1
model_cnn.add(Conv2D(32, (3, 3),kernel_regularizer=regularizers.l2(0.001)))
model_cnn.add(Activation(‘relu’))
model_cnn.add(MaxPooling2D(pool_size=(2, 2)))

# Conv Layer 2 (no pooling)
model_cnn.add(Conv2D(32, (3, 3),kernel_regularizer=regularizers.l2(0.001)))
model_cnn.add(Activation(‘relu’))

# Conv Layer 3
model_cnn.add(Conv2D(64, (3, 3)))
model_cnn.add(Activation(‘relu’))
model_cnn.add(MaxPooling2D(pool_size=(2, 2)))

# MLP
model_cnn.add(Flatten()) # this converts our 3D feature maps to 1D feature vectors
model_cnn.add(Dropout(0.5))
model_cnn.add(Dense(64))
model_cnn.add(Activation(‘relu’))
#model_cnn.add(Dropout(0.5))
model_cnn.add(Dense(4))
model_cnn.add(Activation(‘softmax’))

return model_cnn

In [ ]:

STEPS_PER_EPOCH = TRAIN_DATA_LEN//BATCH_SIZE
lr_schedule = tf.keras.optimizers.schedules.InverseTimeDecay(
0.001,
decay_steps=STEPS_PER_EPOCH*1000,
decay_rate=1,
staircase=False)

Train the model with dataset without augmentation

In [ ]:

model_with_aug = get_model()
model_with_aug.compile(optimizer = tf.keras.optimizers.Adam(),
loss=tf.losses.CategoricalCrossentropy(),
metrics=[tf.losses.CategoricalCrossentropy(name=’CategoricalCrossentropy’), ‘accuracy’])

m_histories[‘No_augmentation_face’] = model_with_aug.fit(non_augmented_train_batches, epochs=250, validation_data=validation_batches, verbose=0, callbacks=get_callbacks(‘models/face_NoAug’))

Train the model using dataset with augmentation

In [ ]:

model_with_aug = get_model()
model_with_aug.compile(optimizer = tf.keras.optimizers.Adam(),
loss=tf.losses.CategoricalCrossentropy(),
metrics=[tf.losses.CategoricalCrossentropy(name=’CategoricalCrossentropy’), ‘accuracy’])

m_histories[‘With_augmentation_face’] = model_with_aug.fit(augmented_train_batches, epochs=250, validation_data=validation_batches, verbose=0, callbacks=get_callbacks(‘models/face_Aug’))

Plot the results

In [ ]:

plotter(m_histories, ylim=[0.0, 1.1], metric = ‘CategoricalCrossentropy’)

In [ ]:

plotter(m_histories, ylim=[0.5, 1.1], metric = ‘accuracy’)

What other augmentation techniques are applicable to this task?

What are some of the common augmentation techniques that are not applicable to this task?

Writing a dataloader¶
Reading the data is a fundamental part of training neural networks and there are many tools provided with tensorflow and keras to do this.

The tf.data API enables you to build complex input pipelines from simple, reusable pieces. The tf.data API introduces a tf.data.Dataset abstraction that represents a sequence of elements, in which each element consists of one or more components. For example, in an image pipeline, an element might be a single training example, with a pair of tensor components representing the image and its label. To create an input pipeline, you must start with a data source. For example, to construct a Dataset from data in memory, you can use tf.data.Dataset.from_generator() or tf.data.Dataset.from_tensor_slices(). Once you have a Dataset object, you can transform it into a new Dataset by chaining method calls on the tf.data.Dataset object. For example, you can apply per-element transformations such as Dataset.map(), and multi-element transformations such as Dataset.batch(). See the documentation for tf.data.Dataset for a complete list of transformations. You might have noticed by now that we have used this methods in the previous section.

The most common tool used for reading the data is ImageDataGenerator provided with keras. This ImageDataGenerator Generate batches of tensor image data with real-time data augmentation. If you wanted to do the last part on augmentation using this interface the code would look like below.

Using ImageDataGenerator¶

In [ ]:

from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Train data with augmentation
train_datagen = ImageDataGenerator(rescale=1./255, data_format=’channels_last’,
rotation_range=5, width_shift_range=0.15,
height_shift_range=0.15)
val_datagen = ImageDataGenerator(rescale=1./255, data_format=’channels_last’)

train_generator = train_datagen.flow_from_dataframe(
dataframe=train_data_df,
directory=’./’,
x_col=”image_path”,
y_col=”label”,
target_size=(32, 32),
batch_size=BATCH_SIZE,
color_mode=”grayscale”,
class_mode=’categorical’)

validation_generator = val_datagen.flow_from_dataframe(
dataframe=val_data_df,
directory=’./’,
x_col=”image_path”,
y_col=”label”,
target_size=(32, 32),
batch_size=BATCH_SIZE,
color_mode=”grayscale”,
class_mode=’categorical’)

m_histories = {}
model_with_aug_dg = get_model()
model_with_aug_dg.compile(optimizer = tf.keras.optimizers.Adam(lr_schedule),
loss=tf.losses.CategoricalCrossentropy(),
metrics=[tf.losses.CategoricalCrossentropy(name=’CategoricalCrossentropy’), ‘categorical_accuracy’])

m_histories[‘With_augm_face_datagen’] = model_with_aug_dg.fit(train_generator, epochs=250, validation_data=validation_generator, verbose=0, callbacks=get_callbacks(‘models/face_Aug_datagen’))

plotter(m_histories, ylim=[0.0, 1.1], metric = ‘CategoricalCrossentropy’)

In [ ]:

plotter(m_histories, ylim=[0.0, 1.1], metric = ‘categorical_accuracy’)

Writing your own data generator¶
There might be situations were the out of the box solutions provided with tensorflow are not applicable for you. What would you do in this case?

We can always develop our own data loader. This can be done by inheriting the properties of keras.utils.Sequence so that we can leverage nice functionalities such as multiprocessing. Lets see how this is done.

This part of the Lab is base on the blog A detailed example of how to use data generators with Keras by Afshine Amidi and Shervine Amidi

A data generator that is done by inheriting the properties of keras.utils.Sequence. needs to have four functions:

Initialization function of the class: def __init__(self, ) This function will set the global variables of the class. We set relevant information about the data, such as data dimensions, number of classes, batch size, or decide whether we want to shuffle our data at generation (If the shuffle parameter is set to True, we will get a new order of exploration at each pass). We also store important information such as labels and the list of IDs that we wish to generate at each pass.

Function that returns the number of batches per epoch: def __len__(self) Each call to the data loader requests a batch index between 0 and the total number of batches, where the total number of batches is specified in this method.

Function evoked at the end of each epoch: def on_epoch_end(self) Shuffling the order in which examples are fed to the classifier is helpful so that batches between epochs do not look alike. Doing so will eventually make our model more robust.

A Function that returns the batch corresponding to a given index: def __getitem__(self, index)

There might be other functions that supports the four above. The extra functions are not required but will improve the code readability.

First let us write the code for reading the data for the above task. We are assuming that the data filenames and labels are provided to the dataloader at the start as a pandas dataframe. These fields will be read into the class at the initialization stage.

In [ ]:

import tensorflow.keras as keras
from scipy.interpolate import interp1d
import numpy as np
from scipy.ndimage.interpolation import rotate, shift
from PIL import Image

class DataGenerator(keras.utils.Sequence):
‘Generates data for Keras’
def __init__(self, data_frame, batch_size=8, dim=(32, 32, 1), n_classes=2, data_mean=0, data_std=1, data_prefix=”, shuffle=True, Augment=True):
‘Initialization’
self.dim = dim # Dimentions of the input
self.batch_size = batch_size
self.n_classes = n_classes # Number of classes. This is for a classification task

self.shuffle = shuffle # Flag to shuffle data at the end of epoch
self.Augment = Augment # Falg to augmetn the data

# The data is input as a pandas dataframe, we need to read the relevent fields
self.data_frame = data_frame
self.image_label = data_frame[‘labels_num’].values.tolist()
self.image_ids = np.arange(len(self.image_label)).tolist()
self.data_prefix = data_prefix

# Data normalization parameters
self.data_mean = data_mean
self.data_std = data_std

self.on_epoch_end()

def __len__(self):
‘Denotes the number of batches per epoch’
return int(np.floor(len(self.image_ids) / self.batch_size))

def __getitem__(self, index):
‘Generate one batch of data for the given index’
# Generate indexes of the batch
indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]

# Find list of IDs
data_ids_temp = [self.image_ids[k] for k in indexes]
image_label_temp = [self.image_label[k] for k in indexes]

# Generate data
X, y = self.__data_generation(data_ids_temp, image_label_temp)

return X, y

def on_epoch_end(self):
‘Updates indexes after each epoch’
self.indexes = np.arange(len(self.image_ids))
if self.shuffle == True:
np.random.shuffle(self.indexes)

# Support function

def __data_generation(self, data_ids_temp, image_label_temp):
‘Generates data containing batch_size samples’

# Initialization
X = np.empty((self.batch_size, *self.dim))
y = np.empty((self.batch_size), dtype=int)

# Generate data
for i, ids in enumerate(data_ids_temp):

X[i,] = self.__read_data_instance(data_ids_temp[i])
y[i] = image_label_temp[i]

return X, keras.utils.to_categorical(y, num_classes=self.n_classes)

def __read_data_instance(self, pid):
# Read an image
filepath = self.data_prefix + self.data_frame.iloc[pid][‘image_path’]

data = Image.open(filepath)
data = data.resize((32,32))
data = np.asarray(data)
data = np.expand_dims(data, axis=-1)

if self.Augment:
rot = np.random.rand(1) < 0.5 if rot: rot = np.random.randint(-10,10, size=1) data = rotate(data, angle=rot[0], reshape=False) shift_val = np.random.randint(-5, high=5, size=2, dtype=int).tolist() + [0,] data = shift(data, shift_val, order=0, mode='constant', cval=0.0, prefilter=False) X = data # Input normalization X = (X - self.data_mean)/self.data_std return X Initialize the Data Generators i.e train and test. In [ ]: data_mean = 0. data_std = 255.0 prefix='' training_generator = DataGenerator(train_data_df, batch_size=BATCH_SIZE, data_mean=data_mean, data_std=data_std, n_classes=4, Augment=True, data_prefix=prefix) validation_generator = DataGenerator(val_data_df, batch_size=BATCH_SIZE, data_mean=data_mean, data_std=data_std, n_classes=4, Augment=False, data_prefix=prefix) nEpochs = 250 It is good practice to plot the data generated by the code to see if everything is working as expected. In [ ]: print(training_generator.__len__()) for x,y in training_generator.__iter__(): print(x.shape, y.shape) # plt.imshow(x[0,:,:,0], cmap='gray') # plt.pause(.1) Train the model and plot results. In [ ]: model2 = get_model() model2.compile(optimizer = tf.keras.optimizers.Adam(), loss=tf.losses.CategoricalCrossentropy(), metrics=[tf.losses.CategoricalCrossentropy(name='CategoricalCrossentropy'), 'categorical_accuracy']) m_histories = {} m_histories['With_augm_face_datagen'] = model2.fit(training_generator, epochs=250, validation_data=validation_generator, verbose=0, callbacks=get_callbacks('models/face_Aug_datagen')) plotter(m_histories, ylim=[0.0, 1.1], metric = 'CategoricalCrossentropy') In [ ]: plotter(m_histories, ylim=[0.0, 1.1], metric = 'categorical_accuracy')