Table of Contents

Abstract............................................................................................2

1. Introduction..................................................................................3

2. Data Visualization in Android.......................................................5

3. Machine Learning Environments and Algorithms......................13

4. Machine Learning Models for Android.......................................21

5. Deep Learning for Android.........................................................25

6. About the Author........................................................................34

Abstract
Machine learning (ML) is the groundbreaking subfield of artificial intelligence which
is garnering a lot of attention these days. There are two primary factors driving the
surge in popularity:

Unlimited access to computing power. Algorithms existed in the past, but were
infeasible to execute at scale.

Explosive growth of data and the emergence of big data sets. The additional future
impact of IoT (Internet of Things) and the collection of sensory data will further
advance this trend.

As mobile developers participating in the app economy, we can now begin to think
about how to integrate ML into our mobile apps. In this paper, we will try to go beyond
the hype and investigate how we can leverage our Java skills and new developments
in the field to build and deploy ML apps.

The paper will be divided into the following sections:

1. Introduction
2. Data Visualization for Android
3. Machine Learning Environments and Algorithms
4. Machine Learning Models for Android
5. Deep Learning for Android

The paper will focus on solutions which leverage the Java language because it is
familiar to us as Android developers. Other popular ML environments and libraries
which are based on Python and the C-based languages (such as Tensorflow), will not
be covered.

2
1. Introduction
Artificial Intelligence (AI), Data Mining (DM), and Machine Learning (ML). These terms
are thrown around frequently these days, and their usage is inconsistent and confusing.

AI is not new. The term has been around since the 70s. It can loosely be defined as
anything which tries to act intelligent. DM and ML are subfields of AI which are based
on statistics. For this reason, DM is also sometimes referred to as computational
statistics.

When we build solutions, it can be helpful to think of DM as the infrastructure, and ML

as the algorithms. Using ML is really all about solving our problems by collecting and
preprocessing the relevant data, and then applying the most appropriate algorithm to
the data.

Figure 1. AI, Then and Now

Machine learning algorithms also are not new. What is new is our access to nearly
unlimited processing power in the cloud. This has led to enhancement in one special
category of algorithms called neural networks (NN). NN algorithms are also often
referred to by the term Deep Learning (DL). There have been many exciting recent
developments which use DL algorithms, including in the fields of computer vision
(CV) and natural language processing (NLP).

Next we will explore the importance of data for ML apps, and how we can visualize
data in Android.

4
2. Data Visualization for Android
Data is the single most important ingredient for a successful ML project. We need
high quality data, and we need lots of it. Data mining is all about working with our
data to identify hidden patterns, while machine learning takes the additional step of
applying algorithms to the processing of the data.

In both DM and ML, we are normally working with large, loosely structured data sets.
Being able to visualize our data is important. We need a good understanding of our
data before we can construct machine learning models that effectively process our
data. Nate Silver said it best in his book, The Signal and the Noise:

The numbers have no way of speaking for themselves. We speak for

them. Data-driven predictions can succeed- and they can fail. It is
when we deny our role in the process that the odds of failure rise.
Before we demand more of our data, we need to demand more of
ourselves.

A good starting point, as we demand more from ourselves, is to think about how we
represent and visualize our data.

As Android developers, you are probably familiar with JSON (Javascript Object Notation).
JSON is a data exchange format that widely used between servers and client devices.
It uses a simple but powerful collection of arrays and objects to represent data. Data
is represented within objects by name/value pairs. The flexible structure of JSON
enables it to represent very complex data relationships. In JSON, the placement of
parentheses and brackets to represent arrays and objects is very important. You can
learn more about how JSON works at www.json.org.

Figure 2 shows a partial listing of a JSON file called flare.json. You can view the complete
file at https://github.com/wickapps/machinelearningpaper. We will use this JSON file
to implement a visualization application for Android:
 {name: flare,
children: [
{name: analytics,
children: [
{name: cluster,
children: [
{name: AgglomerativeCluster, size: 3938},
{name: MergeEdge, size: 743}]},
{name: optimization,
children: [
{name: AspectRatioBanker, size: 7074}]}
]},
{name: animate,
children: [
{name: Easing, size: 17010},
{name: interpolate,
children: [
{name: ArrayInterpolator, size: 1983},
{name: ISchedulable, size: 1041},
{name: Tween, size: 6006}]}]

Figure 2. Sample JSON File

One of the best approaches to implement data visualization in Android is to use third
party open source graphic libraries in conjunction with the Android WebView. With
this approach we can generate some pretty amazing visualizations with minimal coding.

Figure 3 shows a partial list of some visualization libraries which can be used to
visualize data:

D3.JS Leaflet Timeline JS Highcharts

d3plus.org leaflet.js timeline.knightlab.com Highcharts.com

FusionCharts Dygraphs Plotly Raw

fustioncharts.com dygraphs.com plot.ly rawgraphs.io

Chart.js Datawrapper ChartBlocks

chart.js datawrapper.de chartblocks.com

Google Charts Tableau Infogram

developer.google.com/chart tableau.com infogr.am

Figure 3. Popular Javascript Data Visualization Libraries

6
In the following example, we are going to use D3.js. Like many of the libraries above,
D3 is a based on Javascript, which provides an impressive, interactive user experience.
All of the modern browsers can render Javascript, so it makes a good solution for
Android apps. We wont cover the underlying Javascript code in this paper, but links
will be provided to the Javascript code required if you wish to see how the charting
magic works.

You can explore the huge library of visualizations available for D3.js at github.com/
d3/d3/wiki/Gallery.

Visualization is all about choosing the best graphical style to represent our data.
Since we are working with JSON, Figures 4 and 5 below show two popular graphs
which I like to use to represent JSON data- the cluster dendogram, and the radial
dendogram:

Figure 4. JSON Visualizations: Cluster Dendogram

 Figure 5. JSON Visualizations: Radial Dendogram

These pictures can be beautiful, and the power of visualization is obvious. Each of
these visualizations give us a much better feel for the structure of our data than the
JSON text file provided.

Each of the myriad of visualizations available in the D3.js library can be achieved with
a minimal amount of Javascript code. Within the Javascript wrappers for these visu-
alizations, there are a couple of external references which need to be set up:

Reference to the D3 library within the <script> tags. Local copy or remote lookup.

8
 Point to our JSON file using the d3.json assignment statement.

Figure 6 shows an excerpt of the D3 HTML/Javascript code for the dendogram visual-
ization which highlights these required external references:

<!DOCTYPE html>
. . .
<script src=https://d3js.org/d3.v3.min.js></script>
. . .
d3.json(flare2.json, function(error, root) {
. . .
});

Figure 6. HTML/Javascript except for D3 Dendogram wrapper

The complete code for cluster-dendo.html can be found at https://github.com/wick-

apps/machinelearningpaper. In this example, the JSON file and the HTML file are
stored on the server in the same directory so no additional path information is re-
quired.

Once we have this visualization working for our brows er, setting it up for Android
is straightforward. Rather than importing a visualization or charting library into our
app, well take a shortcut and use Androids WebView to render out the Javascript.
Figure 7 shows how to set up a full screen WebView layout. A progress dialog pro-
vides an indication to the user that network content is being loaded. The

setContentView(R.layout.activity_fullscreen);
webView = (WebView) findViewById(R.id.wb_webview);
webView.setScrollbarFadingEnabled(false);
webView.setHorizontalScrollBarEnabled(true);
webView.setVerticalScrollBarEnabled(true);
webView.getSettings().setJavaScriptEnabled(true);
webView.clearCache(true);
webView.setFadingEdgeLength(10);
final Activity activity = this;
final ProgressDialog progressDialog = new ProgressDialog(activity);
progressDialog.setProgressStyle(ProgressDialog.STYLE_SPINNER);
progressDialog.setProgressStyle(ProgressDialog.THEME_HOLO_LIGHT);
progressDialog.setCancelable(true);
webView.setWebChromeClient(new WebChromeClient() {
public void onProgressChanged(WebView view, int progress) {
progressDialog.setCanceledOnTouchOutside(true);
progressDialog.setTitle(Loading visualization ...);
progressDialog.setButton(Cancel, new DialogInterface.OnClickListener() {
public void onClick(DialogInterface dialog, int which) {
webView.destroy();
finish();
} });
progressDialog.show();
progressDialog.setProgress(0);
activity.setProgress(progress * 1000);
 progressDialog.incrementProgressBy(progress);
if(progress == 100 && progressDialog.isShowing())
progressDialog.dismiss();
}
});
String postData = ;
webView.postUrl(http://www.yourserver.com/cluster-dendo.html,
EncodingUtils.getBytes(postData, base64));

Figure 7. Android WebView example for Cluster-Dendogram Visualization

The Android layout file, fullscreen.xml, is shown next in Figure 8. It includes the An-
droid WebView which is contained within a FrameLayout.

<FrameLayout xmlns:android=http://schemas.android.com/apk/res/android
xmlns:tools=http://schemas.android.com/tools
android:layout_width=match_parent
android:layout_height=match_parent
tools:context=android.wickham.com.datavis.FullscreenActivity>
<WebView
android:id=@+id/wb_webview
android:layout_width=fill_parent
android:layout_height=fill_parent />
</FrameLayout>

Figure 8. Android WebView XML Layout file

This Android code, when executed, downloads the HTML/Javascript file and then dis-
plays the visualization on your device, as shown in Figure 9.

10
Figure 9. Data Visualization in Android Javascript D3 Library WebView
Using WebView to display visualizations on Android can be considered is a bit of a
shortcut, but when set up properly, the simplicity cant be beat, and it provides a
clean, integrated user experience for your app.

Having a Javascript solution for visualizing data allows us to see what the data looks
like. This is a first step in understanding the data and being able to identify hidden
patterns in the data.

Inside your Android app, you may not visualize (pun intended) a business case to
display visualizations of your internal JSON structures to your users. However, there
may be times when this technique could be helpful as a back-door debugging function
for your users. I have used this, and find it feasible to troubleshoot problems when
users can look at a simple visualization to identify issues. This certainly could not be
possible if they needed to look at a JSON text file.

With JSON visualization, we have begun to demand more of ourselves with respect to
our data. Now, we are ready to take a look at Java-based ML environments and ML
algorithms.

12
3. Machine Learning Environments and
Algorithms
Like everything in life, becoming a machine learning expert requires practice. There
is no better way of practicing machine learning that working with one of the free
open source machine learning environments on the desktop. Figure 10 shows a
summary of the Java-based ML environments:

Waikato Environment for Knowledge Analysis. ML platform developed

Weka in Waikato University in New Zealand. Written in Java. Includes GUI,
Command Line Interface, Java API. Most popular ML environment. Great
environment to start or practice ML.

Konstanz Information Miner. Developed by Konstanz University in Ger-

KNIME many. Focus on pharma research and business intelligence.

Developed by Technical University of Dortmund. GUI and Java API. Data

Rapid Miner handling, visualization, modeling, algorithms.

Environment for Developing KDD Applications. Data mining workbench

ELKI developed at Ludwig University in Munich. Focus on data. Provides a
mini-GUI, Command Line Interface, Java API.

The Java Machine Learning Library. A collection of ML Algorithms. No

Java-ML GUI support.

Figure 10. Java Machine Learning Environments

Comparing these Java solutions, Weka excels because it is a complete workbench

as shown in Figure 11. Not all of the packages mentioned above include a GUI, which
makes things much easier.
 Figure 11. Web Explorer GUI Environment

Using ML environments via command line interfaces (CLI) does come with a badge of
honor, but not really an effective way to learn ML as we need to visualize the data.
Weka also has a large number of tools for data conversion. As Android mobile devel-
opers, we typically have a lot of JSON data, and Weka makes it easy for us to convert
these data into its ARFF file format.

Elki and Rapid Miner tend to have more of a focus on unsupervised learning and do
not have the broad coverage as Weka.

There are quite a few ML environment packages based on Python and the R data
science language. While Python is easy to learn, these packages typically do not have
the performance advantages of Java.

Weka is available for many platforms and can be downloaded at cs.waikato.ac.nz.

WEKA is a great way to get familiar with ML for the following reasons:

14
 Weka is Java-based and cross platform. You can download a version for the plat-
form of your choice.

Weka is easy to use. It contains a full desktop GUI environment with many tools
and ML algorithms.

Weka is extensible. It supports external add-ons, or we can even modify the exist-
ing Java algorithms to tailor our own solutions.

Weka includes many sample datasets that we can use to experiment with ML algo-
rithms, train models, and test models for accuracy.

Weka is portable. We can build models in the desktop environment, and then use
the model within our Android app by including a GUI-less Weka library in the An-
droid project.

The main disadvantage of Weka is that it is not well suited for some of the deep
learning algorithms which are increasingly popular today. We will discuss this short-
coming a bit later. However, Weka does give us access to many machine learning
algorithms and the Weka GUI environment makes it easy to explore ML and the many
algorithms available.

Machine learning algorithms can be categorized into the following different styles:
Supervised - Input training data has known labels. A model is prepared by cor-
recting wrong predictions. Training process continues until a desired accuracy is
reached. Supervised algorithms are good for Classification and Regression.

Unsupervised - Input data is not labeled and does not have a known result. A
model is prepared by deducing structures present in the input data. This may be
to extract rules, or to organize the data by similarity. Unsupervised algorithms are
good for Clustering, Dimension Reduction, Association Rule Learning.

Semi-supervised - Input data is a mixture of labeled and unlabeled. There is a
desired prediction, but the model must learn the structures and make the prediction.
Semi-supervised algorithms are good for Classification and Regression.

Deep Learning (DL) - A special type of neural network which includes hidden layers
and back propagation (memory). These can be applied to supervised or unsupervised
learning tasks. Deep Learning is good for Pattern Recognition, Language Process-
ing, Image Classification.
 Deep Belief
AODE Conditional Expectation- Maximization Generative Networks
Random Field Learning
Aritficial Neural Low-Density Deep Belief
Network ANOVA Low-Density Separation Separation Networks

Deep Recurrent
Bayesian Statistics Linear Classifiers Generative Topographic Map Graph-based Neural Network
Methods (RNN)
Instance-Based Quadratic Deep Belief
Learning Classifiers Information Bottleneck Co-training Networks

Lazy Learning K-Nearest Self-Organizing Map (ANN) Reinforce Hierarchical

Neighbor (Temporal Diff) Temporal Memory

Reinforce
Learning Automata Boosting Apriori Algorithm (Q-Learning)
Learning Vector Decision Trees
Quant (C45) Eclat Algorithm Reinforce (SARSA)
Decision Trees Reinforce
Logistic Model Tree (Random Forests) FP-Growth Algorithm (Automata)

Minimum Message Decision Trees Single-linkage Clustering

Length (CART)

PAC Learning Decision Trees Conceptual Clustering

(SLIQ)

Support Vector Decision Trees K-means Algorithm

Machines (SPRINT)

Random Forests Bayesian Fuzzy Clustering

Networks

Ensembles of Hidden Markov DBSCAN

Classifiers Models

Ordinal Classification OPTICS Algorithm

Fuzzy Networks Local Outlier Factor

Figure 12. Summary of Popular Machine Learning Algorithms

16
The key to effective ML programming is to identify the algorithm which is most
compatible with the dataset we are applying to the given problem. Its all about defining
your data and purpose, and then choosing an algorithm to create and train your model.

I found that Microsoft Azure ML has created arguably the most useful method for
choosing the correct algorithm. Figure 13 shows the Azure ML approach. They start
by asking the basic question, What do you want to find out?, and then lead you
through real-world examples before finally showing which algorithms are most suited
for your problem. Rather than the somewhat confusing supervised/unsupervised
categorization, they break the algorithms down into the five categories as shown in in
Figure 13.

Figure 13. Microsoft Azure ML Algorithm Selection

(Source: Summarized from- azuremlsimpleds.azurewebsites.net/simpleds/)

Choosing the right algorithm for your data requires practice to gain experience, and
that is where an environment like Weka comes in.

The Weka workbench is a collection of state-of-the-art ML algorithms and data

processing tools. It provides extensive support for the entire process of experimental
machine learning. To demonstrate how we can use Weka for machine learning, we
will set up a typical classification problem. The following steps are the general process
for experimenting with machine learning in Weka:
1. Load a data set. You can use your own data, or one of the built-in Weka data sets.

2. Choose your ML algorithm. Weka divides its algorithms into classification and
clustering categories, which roughly correspond to the supervised and unsupervised
ML styles.

3. Run the algorithm on your data set. Weka will train your model, and then test your
model the model using a subset of the data (called a fold)

4. Review the results for accuracy. You can then tweak your data or algorithm and
try again for improved accuracy.

Running a classifier in Weka is not difficult. In this example, we will make use of one
of the built-in Weka data sets called iris.arff. This data set contains 150 instances
(rows) and 4 attributes (columns). It is comprised of 50 samples from each of the 3
species of iris.You can read more about this dataset at
wikipedia.org/wiki/Iris_flower_data_set. After you Open File in Weka, Figure 14 shows
the iris data set after it has been loaded.

Figure 14. Weka Workbench - Iris.arff data set

You can see that Weka has loaded 150 instances and the data has 5 attributes. If you
click on each of the attributes, you can see a histogram on the right side showing the
distribution for this attribute across the 3 distinct species of iris we are trying to classify.

18
These represent the output labels for this classification.

After loading the data, we need to select and run an algorithm. Two of the most popu-
lar ML algorithms for classification are C4.5 and RandomForest. Weka has a version
of the C4.5 algorithm called J48 (J is for Java). Weka also includes the RandomForest
algorithm. In the Weka workbench, each of these can be found under the Classify tab
by selecting the Choose button.

To classify the iris data we will first try the RandomForest algorithm. Under Choose,
you simply select the Classifier -> Trees -> RandomForest algorithm. There are many
options for each of the algorithms in Weka, but in this case, we will keep the default
options.

One of the options under Test options is number of Cross-validation Folds. This is set
to 10 by default, which instructs Weka to divide the data into 10 sections, using the
first 9 for training the model, and the last section for testing the model. Press Start to
run the algorithm. Congratulations, you just run your first machine learning classifier
in Weka! Figure 15 shows the results.

Figure 15. Weka Classification Results - RandomForest Algorithm

You can see this model was able to predict 143 of the 150 instances correctly for
95.3% accuracy. The Confusion Matrix at the bottom of the results shows that 3 of the
type B iris were misclassified as type C while 4 of type C were misclassified as type B.
With Weka, we can easily choose another algorithm, such as J48 and run the model
again. Note that both RandomForest and J48 are both tree-based algorithms, and the
results are similar at about 96% accuracy using the default options
.
If you are interested in the documentation for the Weka algorithms, the summary
page for all of the Weka Java classes can be found at the following location after you
install Weka:

file:///YourWekaPath/Weka-3-8/doc/overview-summary.html

Each of the algorithms has its own documentation. The documentation page for the
RandomForest algorithm can be found here:

file:///YourWekaPath/Weka-3-8/doc/weka/classifiers/trees/RandomForest.html

The Weka workbench gives you a lot of power to experiment with machine learning.
And the best news of all, once you have a model you are happy with, you can export
the model. To save a model in Weka, simply right-click on the specific model results
in the Results list on the left hand side of the Classify tab in the Weka workbench.

The model object file can then be saved to your desktop as a Java binary serialized
object. This is very useful, as next we will explore how to import the pre-trained model
into Android.
ght side

20
4. Machine Learning Models for Android
Now that we know how to create ML models from our own datasets, lets take a look
at how we can use those precomputed Weka models inside our Android apps. This is
a useful technique because our devices do not really have the processing power or
storage to hold large data sets, or to train machine learning models. One huge rea-
son this is such a compelling approach is that the size of the model is NOT linearly
related to the amount of data that is used to train the model. Creating models in Ja-
va-based Weka and using them in Android is simpler than if we were using Python or
C-based technologies such as Tensorflow.

To use a Weka model in Android, there are a few points to consider:

When we export the model from Weka, the model is a serialized Java object. The
file will be saved by default with the .model extension.

In the Android app, we need to include the Weka library named wekaSTRIPPED.
jar into the Android project. The name is derived from the fact that the library has
the GUI aspects of the Weka workbench removed. You can download the file from
here: https://github.com/rjmarsan/Weka-for-Android/blob/master/wekaSTRIPPED.
jar. The jar file needs to be copied into the libs/ directory and included in the build.
gradle file.

We are going to load the model programmatically in the Android app.

Inside your Android app, you will be able to train, test and use the Weka model to
predict the target variable programmatically.

The Weka documentation contains two of articles which describe how to use a Weka
model in Android:

http://weka.wikispaces.com/Use+WEKA+in+your+Java+code
http://weka.wikispaces.com/Programmatic+Use

It is possible to create and train a model programmatically in Android. However, since

we have already accomplished these tasks using the Weka workbench on the desktop
environment, next we will simply review the loading and usage of the model in An-
droid.

Once your .model file is copied into onto your device, you can open the model using
the Classifier object as shown in Figure 16.
 // Define a Weka Classifier Object
private Classifier mClassifier = null;

// Load the Classifier from local storage

AssetManager assetManager = getAssets();
try {
mClassifier = (Classifier)
weka.core.SerializationHelper.read(assetManager.open(yourWekaModel.model));
} catch (IOException e) {
// Handle Weka model failed to load
e.printStackTrace();
} catch (Exception e) {
// Handle Weka model still failed to load
e.printStackTrace();
}

Figure 16. Opening a Precomputed Weka Model in Android

Once the model is successfully loaded, we can define a new untested sample. This
involves setting up the attributes and classes which correspond to our model defini-
tion, in this case, the iris dataset. Figure 17 shows how to setup the Android variables
to match the dataset for the precompiled model.

// Set up the Attributes. These need to match the Weka Iris.arff dataset
final Attribute attributeSepalLength = new Attribute(sepallength);
final Attribute attributeSepalWidth = new Attribute(sepalwidth);
final Attribute attributePetalLength = new Attribute(petallength);
final Attribute attributePetalWidth = new Attribute(petalwidth);

// Set up the Classes. For this dataset se have 3. These also needs to match the Iris.arff
dataset
final List<String> classes = new ArrayList<String>() {{
add(Iris-setosa);
add(Iris-versicolor);
add(Iris-virginica);}
};

// Set up an ArrayList to hold the values for the untested samples we wish to classify.
// In this example, we will just have a single untested sample.
ArrayList<Attribute> attributeList = new ArrayList<Attribute>(2) {{
add(attributeSepalLength);
add(attributeSepalWidth);
add(attributePetalLength);
add(attributePetalWidth);

Attribute attributeClass = new Attribute(@@class@@, classes);

add(attributeClass);}
};

Figure 17. Setting up Attributes and Class Labels

Now that everything is set up, we can ask the model to classify an untested sample.

22
Figure 18 shows the Android code. In this case, we are specifying the attributes for
the untested sample directly in the newInstance object. The get the prediction, we
just need to invoke the classifyInstance method on the mClassifier object. The result
will be returned as a double which can be converted to an integer which we can then
use to lookup the name of the predicted iris type.

// Create a new Instance which we will classify

Instances dataUnpredicted = new Instances(TestInstances,attributeList, 1);

// The last feature is the target variable

dataUnpredicted.setClassIndex(dataUnpredicted.numAttributes() - 1);

DenseInstance newInstance = new DenseInstance(dataUnpredicted.numAttributes()) {{

setValue(attributeSepalLength, 4.95);
setValue(attributeSepalWidth, 3.50);
setValue(attributePetalLength, 2.00);
setValue(attributePetalWidth, 0.45);}
};

// Define the dataset

newInstance.setDataset(dataUnpredicted);

// Predict the new sample

try {
double result = mClassifier.classifyInstance(newInstance);
String prediction = classes.get(new Double(result).intValue());
Toast.makeText(this, prediction, Toast.LENGTH_SHORT).show();
} catch (Exception e) {
// Oops, need to handle the Weka prediction exception
e.printStackTrace();
}

Figure 18. Classifying an Untested Sample

With just a small amount of code, we were able to use a precomputed model in our
Android app to make predictions. This is a powerful construct, and the possibilities
are endless. For instance, consider the following potential uses:

Stock market assistant- We could build a document classifier in Weka. We could

train the model with large amounts of financial news data, such as the business
news feed from Bloomberg or other financial providers. The classifier could be
trained to identify buy or sell recommendations on a particular stock/bond/option
or futures contract. The trained model could then be included in your Android app,
where it can monitor real-time headlines or push messages and notify you when a
trading opportunity arises.

Sports forecasting- We could train a Weka model with sports event data, such as
soccer or football game statistics. Such data is readily available. This model could
then be used to make final score predictions for upcoming games.
 Indoor location tracking- We could create and train a model with mobile phone
sensor data, such as Wifi signal strength. After the model is trained by exploring
all areas of the space, real-time data from the device sensors could be used for
indoor location identification.

Sensor data- Perhaps you have implemented an Arduino project which gives you
access to sensor data. Or you may have other sources of sensor data, such as
sprinkler systems, home climate control, remote vehicle fleets, industrial machine,
etc. By training an ML model with this data, you might be able to predict failures of
remote devices before they happen.

As mentioned earlier, one of the weaknesses of Weka is the lack of availability of

neural network algorithms which are typically used for deep learning. Weka has a
MultilayerPerceptron and you can stack this filter to create a network with multiple
layers. There is also a Convolutional Neural Network included in the Weka unofficial
package downloads. There is also a draft wrapper for the Deeplearning4J library. We
will talk more about this library and other approaches for implementing deep learning
in the next section.

24
5. Deep Learning for Android
If you have been watching the news recently, you may have heard about some
startling developments related to the field of machine learning:

The ancient Chinese game of Go was mastered by a deep learning algorithm. In

2016, British-based Deepmind (acquired by Google), created AlphaGo, which was
able to defeat Korean Go master Lee Sedol. Although with simpler rules, Go is
considered to be much more complicated than chess due to its massive number
of permutations. Most people thought machines could not beat the Go masters,
but AlphaGo proved them wrong. Recently, some of the Chinese Go masters have
declined to accept challenges to play AlphaGo, for fear that the algorithm will learn
their styles.

IBM Watson seems to be able to do anything, including helping farmers with
irrigation decisions, helping basketball scouts identify talent, and helping doctors
in remote locations diagnose patients.

An AI poker algorithm has whipped the pros. The program called Libratus, developed
at Carnegie Mellon, recently won a Texas holdem tournament against top players.
Poker requires reasoning and intelligence that is hard for machines to imitate.
In poker, hands are hidden so players must make decisions based on imperfect
information. Libratus won all the chips ($1.8 million). This represented a seminal
moment for machine learning.

Traders are being replaced by ML algorithms at the worlds largest money manager.
Black Rock, with assets under management of over $5 trillion, recently announced
that it would be migrating towards machine learning algorithms for a small percentage
of assets in its quantitative trading division.

These amazing developments are the result of success of the special category
of machine learning algorithms referred to as deep learning (DL). Deep learning
architectures differ from normal neural networks because they have more hidden
layers. They can be applied to supervised and unsupervised data sets. They are
popular for unsupervised learning tasks because an abundant amount of data is
unlabeled.

The renewed focus on DL algorithms can be partially attributed to the recent success
they have had in classification contests, most notably the Imagenet competition
which took place in 2012 (www.image-net.org). The 2012 Imagenet competition was a
large scale visual recognition challenge which was won by a deep convolutional neural
net receiving an error rate of 16%. This rate would be reduced to less than 2 percent
by 2014. Machine learning site Kaggle.com frequently hosts machine learning compe-
titions, and most of the winning algorithms use deep learning techniques.

Figure 19 shows a mostly complete summary of the neural networks architec-

tures being used today. Pay particular attention to the green and blue markers. The
green markers represent hidden cells, which are essentially the key characteristic
for deep learning. The blue markers represent memory cells. Algorithms which
use these types of algorithms, such as Recurrent Neural Networks (RNN) and Long
Short Term Memory (LSTM), are responsible for the amazing accomplishments listed
above. What makes these algorithms so powerful is their ability to remember, to
make decisions not just based on the input data, but to also consider prior results.

Figure 19. The Mostly Complete Guide to Neural Networks

(Source: Asimov Institute)

Deep learning can be intimidating. Many books which explore the topic are filled with
frightening math equations. However, there are two approaches which allow us to
make use of deep learning in Android without worrying about all the complicated
mathematical details:

Use a REST service to interface with commercial Machine Learning APIs. There

26
 are many such APIs available, but it should come as no surprise that the best
ones come from the large companies which host so much are our data.

Google Cloud ML Platform, including the Google Cloud Vision API

Microsoft Cortana Intelligence Suite, including Azure Machine Learning
IBM Watson
Amazon Machine Learning

Use a deep learning library to build models (externally or on the device) and make
predictions in our apps.

The easiest way to get started in Android with deep learning is to use commercially
available APIs. These services provide simple to use APIs which provide easy access
to large scale complex machine learning models.

Figure 20 shows a summary of the ML/DL APIs available from Google and IBM Wat-
son arranged by the most popular deep learning categories.

Language Alchemy Language

Cloud Translation Conversation
Dialog
Document Conversion
Language Language Translator
Processing Natural Language Classifier
Natural Language Understanding
Personality Insights
Retrieve and Rank
Tone Analyzer
Cloud Vision Visual Recognition
Computer Vision Video Intelligence
Label Detection
Alchemy Data News
Data Insights Discovery
Discovery News
Tradeoff Analytics
Speech Processing Cloud Speech Speech to Text
Text to Speech
Other Cloud Jobs
Figure 20. API Summary (Google and Watson)
Using these APIs is very straightforward and is very consistent regardless of which
of the providers you choose. If you find a suitable API which meets your requirements
from IBM Watson, Microsoft Azure ML, or Amazon Machine Learning, the implementation
will be very similar to the Google vision example shown next. From your Android app,
you simply perform a REST API call, and then process the returned JSON response.
The machine learning service does all the work.

Figure 21 shows an example of the JSON request which must be constructed in

order to use the Google Vision Label Detection API. This API will allow you to classify
images sent from your mobile device. You will need to specify your Google API Key,
along with the image details and the type, which is LABEL_DETECTION.

POST https://vision.googleapis.com/v1/images:annotate?key=YOUR_API_KEY
{
requests: [
{
images: {
content: ...base64-encoded-image-content ... see code in Figure 22
},
features: [
{
type: LABEL_DETECTION
}
]
}
]
}

Figure 21. JSON Request Header for Google Vision Label Detection

Figure 22 shows the Java code which can be used to construct this JSON request in
your Android app. This code is an excerpt from the Google Vision Label Detection API
example which makes use of Googles TensorFlow deep learning classification model.
You can view the full project code at:

https://cloud.google.com/vision/docs/detecting-labels

After including the Vision API in your project, the code makes use of the
VisionRequestInitializer object to build the JSON request. The code also uses the
AnnotateImageRequest object to provide the details of the image you wish to classify.
The image(s) is encoded to .jpg before it is sent. The request is sent up to the API by
the last line which includes the annotateRequest.execute();.

VisionRequestInitializer requestInitializer = new VisionRequestInitializer(CLOUD_VISION_API_

KEY) {};
BatchAnnotateImagesRequest batchAnnotateImagesRequest = new BatchAnnotateImagesRequest();
batchAnnotateImagesRequest.setRequests(new ArrayList<AnnotateImageRequest>() {{
AnnotateImageRequest annotateImageRequest = new AnnotateImageRequest();
// Add the encoded image

28
 Image base64EncodedImage = new Image();
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 90, byteArrayOutputStream);
byte[] imageBytes = byteArrayOutputStream.toByteArray();
base64EncodedImage.encodeContent(imageBytes);
annotateImageRequest.setImage(base64EncodedImage);
// add the features we want
annotateImageRequest.setFeatures(new ArrayList<Feature>() {{
Feature labelDetection = new Feature();
labelDetection.setType(LABEL_DETECTION);
labelDetection.setMaxResults(10);
add(labelDetection);
}});
// Add to the request
add(annotateImageRequest);
}});
Vision.Images.Annotate annotateRequest = vision.images().annotate(batchAnnotateImagesRe-
quest);
BatchAnnotateImagesResponse response = annotateRequest.execute();

Figure 22. Android Code for Google Vision Label Detection API

The request should return an HTTP response code of 200 OK and a body which
looks like Figure 23.

{
responses: [
{
labelAnnotations: [
{
mid: /m/01yrx,
description: cat,
score: 0.92562944
},
{
mid: /m/04rky,
description: tiger,
score: 0.90815818
},
{
mid: /m/01l7qd,
description: whiskers,
score: 0.79939437
},
{
mid: /m/07k6w8,
description: small to medium sized cats,
score: 0.66373962
},
{
mid: /m/0307l,
description: cat like mammal,
score: 0.65950978
}
]
}
]
}

Figure 23. Response from the Google Vision Label Detection API
Each of the various APIs has its own JSON response format, so you need to check
the documentation for all of the available content which can be discovered. In this
case, the Label_Detection API is providing us a text description and a probability
score for the most likely matches of the image submitted in ranked order.

The APIs available from the big providers give us access to powerful DL capabilities
for our Android apps with minimal effort. However, there are a few reasons we may
wish to seek alternative implementations to the REST APIs:

Usage of the ML APIs from the big providers is not free. While most of them allow
you to experiment for free, once you begin any type of commercial use, they all
have a pricing structure depending on traffic volume and model complexity.

You may want to develop and train your own proprietary models without reliance
on the third party vendors. Keep in mind, you needs lots of data, but once you
overcome this hurdle, you could be better off going with your own exclusive inde-
pendent solution.

I mentioned earlier that Weka was not the best environment for creating deep learn-
ing neural networks. Figure 24 shows a summary of the most popular deep learning
libraries.

Powerful general purpose tool for mathematical programming. De-

Theano veloped to facilitate deep learning. High level language and compiler Python
for GPU.
Library for all types of numerical computation associated with deep
TensorFlow learning. Heavily inspired by Theano. Data flow graphs represent the C++ and Python
ways multi-dimensional arrays (tensors) communicate.
Computational Network Toolkit. Released by Microsoft under a per-
CNTK missive license. C++
Clean and extensible language. Based on the AlexNet which won the
Caffe 2012 ImageNet challenge. Java
Java-based open source deep learning library (Apache 2.0 license).
DeepLearning4J Uses a multi-dimensional array class with linear algebra and matrix Java
manipulation. Has some impressive example projects to get you
started.
C
Torch Scientific computing framework. GPU focus GPU Support
Apache MXNet Open source Apache project. Used by AWS. Has many state of the
art models including CNN and LSTM. Very scalable. Founded by UW, Multi
CMU.

30
 High-level neural network API. Deep learning library for Theano and
Keras TensorFlow Python
Open source Computer Vision library which can be integrated for Java Wrapper
OpenCV Android

Figure 24. AI Engines with Deep Learning Libraries

You will notice that many of these libraries are C++ and Python-based, including the
popular solution Google has adopted from TensorFlow. Some of the most powerful
solutions are C/C++ based. There is some agreement in the field that the Java-based
solutions have a performance advantage over Python. Because we are focused on
Java and Android development, we will take a closer look at the DeepLearning4J
library or DL4J.

DL4J is an open-source, distributed deep learning project written in Java and Scala.
It was created by Skymind, a well-capitalized, San Francisco-based business intel-
ligence firm. DL4J is powered by the Java scientific computing engine called ND4J
(http://nd4j.org/), which is the N-Dimensions for Java library that internally uses
another Java library called JavaCPP. Because the framework is Java-based, it is
relatively easy to get deep learning apps up and running on Android.

According to Skymind, the DL4J platform excels at the following:

Integrates with other Java frameworks, such as Spark, Hadoop, and Kafka at scale.

Optimized to run on distributed CPUs and GPUs.

Serving the Java and Scala communities.

Commercial support for enterprise deployments.

There are some important setup points to consider when using DL4J on Android:

If you are going to build and train your model on your mobile device, processing
time could take longer due to CPU and memory limitations of the device.

The following three libraries need to be added to your project and build.gradle file:

Add the DL4J core library (deeplearning4j-core:0.7.2) to your project.

Add the ND4J library (nd4j-native:0.7.2) to your project. This is the N-Di
mensions scientific library which does all the matrix calculations required
 for neural networks.

Add your platform specific ND4J library (nd4j-native:0.7.2:android-x86).

There is a tutorial for DeepLearning4J on Android available at https://deeplearn-
ing4j.org/android

Ug the DeepLearning4J library, you can create, train, and use neural networks with
a small amount of code. One of the best features of DL4J is that it has a very intui-
tive API. If you recall, Figure 19 displayed a mostly complete summary of neural net-
works type. The DL4J API uses the same terminology to create any of these neural
networks, namely: input cells (layers), output cells, and hidden cells. Figure 25 shows
what the code looks like to create a neural network with 1 input layer, 1 output later,
and 1 hidden layer.

// Define the layer structure of the Neural Network

DenseLayer inputLayer = new DenseLayer.Builder()
.nIn(2)
.nOut(3)
.name(Input)
.build();

DenseLayer hiddenLayer = new DenseLayer.Builder()

.nIn(3)
.nOut(2)
.name(Hidden)
.build();

OutputLayer outputLayer = new OutputLayer.Builder()

.nIn(2)
.nOut(1)
.name(Output)
.build();

// Create a Builder object so we can configure the Neural Network

NeuralNetConfiguration.Builder nncBuilder = new NeuralNetConfiguration.Builder();
nncBuilder.iterations(10000);
nncBuilder.learningRate(0.01);

// Connect the layers

listBuilder.layer(0, inputLayer);
listBuilder.layer(1, hiddenLayer);
listBuilder.layer(2, outputLayer);

// Enable backpropagation and initialize the Neural Network

listBuilder.backprop(true);
MultiLayerNetwork myNetwork = new MultiLayerNetwork(listBuilder.build());
myNetwork.init();

Figure 25. Creating a Neural Network with DeepLearning4J API

The DL4J library brings sophisticated deep learning capabilities to any Android device.
A scan of the github repositories reveals many examples broken down into all of the

32
deep learning algorithms mentioned earlier in the algorithm summary, and more! The
repositories contain the following Deeplearning4J neural net example categories:

MLP Neural Nets

Convolutional Neural Nets

Recurrent Neural Nets (RNNs)

TSNE

Word2Vec & GloVe

Anomaly Detection

User interface examples

These many Java code examples within these categories include: solving MNIST- the
hello World of machine learning, building a recommendation engine, using RNNs
for best in class image classification, natural language processing, generating music
from MIDI files, recreating Shakespeare, etc. The possibilities are endless with
machine learning, and with some of the technologies discussed in this paper, hopefully
we are now able to bring these solutions to our mobile devices!
6. About the Author

Mark Wickham is a long-time software developer who has lived and worked in Beijing
most of the time since 2000. As an independent developer, Mark has been
teaching Android at the popular AnDevCon series of conferences since 2013. His
newest class, Machine Learning: Exploring the Possibilities with Android and Java
will debut at Andevcon in Washington, D.C. this July. Come to the class to learn more
about machine learning for Android and how you can apply this exciting new technology
to your mobile app development.

Mark is also the author of the new book, Android Software Development: A Collection
of Practical Projects, available at Amazon. An overview of the book can be found at
https://bit.ly/2dGOPHk

34