Approximate Computing for Deep Learning in
TensorFlow
Abstract
Nowadays, many machine learning techniques are applied on the smart phone to
do things like image classificatin, audio recognization and object detection to
make smart phone even smarter. Since deep learning has achieved the best result
in many fields. More and more people want to use deep neural netowrk model
in the smart phone. However, deep neural netowrk model can be large, need
large amount of computation that takes too much time and power. There are
a few methods of approximate computing proposed to address this problem in
recent years. The method I use in this paper is mobilenet model using tensorflow
which is just published by Google in this year. I will conduct experiments to
show whether mobilenet can decrease model size, increase speed while at the
same time keep decent accuracy. I will compare metrics of the mobilenet with
other traditional models such as VGG model. I will also how the parameters of
width multiplier and resolution multiplier impact the trade off between model
size, speed and accuracy.
Introduction
Motivation
Breakthroughs of Deep Learning
In recent years, machine learning technique, especially deep learning which
uses multiple layers of artificial neural networks has achieved remarkable break-
throughs in many fields. From image classification to Game AI (AlphaGo), deep
learning all has the best performance.
At the same time, more and more people use smart phone. With no doubt, AI
techniques such as deep learning will make smart phone even smarter. Functions
such as face recognization, audio recognization and image classification will be
added to many mobile apps.
Shortcomings of Inference done by server
Deep learning model training part can be done offline in the server clusters. For
the inference part, although we can send the data through network to the server,
and the server does the prediction and reply with the result. In some cases, if the
data is sensitive, the client may wish not to send out to servers. One example is
the bank card number recognization application. Even without security concern,
network traffic can be slow and expensive, building reliable servers increase the
operation cost.
1
Shortcomings of Inference done by mobile phone
So if we can do prediction on the smart phone, then there is no data security
concern, no network traffic delay and cost, no need to maintain a potentially
expensive server cluster. But this approach also has its drawbacks. It needs to
store the model in the smart phone’s limited storage and inference computing in
the mobile can be slow and cost battery power.
Deep neural network typically has many layers with large number of parameters.
It needs large storage and large number of math operations. For example, one
traditional image classification model VGG has about 100 million parameters,
need more than 1GB to store the model and takes more than 10000 million
Mult-Add operations. Thus it is not fit in the mobile phone.
Objective
To use deep learning models in the mobile phone, we must find a way to
significantly decrease the model size and the number of computing operations to
make the model file resonable small and computing fast with less power. In the
mean time, we don’t want the performance too bad. We need to find a suitable
trade-off between them.
Achieved results
Dissertation outline
2
Approximate Computing for Deep Learning in TensorFlow
Abstract
Introduction
Motivation