1/2/2019 Keeping Your Machine Learning Models Up-To-Date – The Data Lab – Medium

The bene t to a continuous learning system is that it can be completely

automated.

It’s probably a still a good idea to review your process on a regular basis. As
I mentioned before, you may nd a di erent algorithm or a new set of
features that improves your predictions, and this isn’t necessarily
something a continuous learning system is good at.

Continuous learning with Watson ML

If you are using the IBM Watson Data Platform or the Watson Machine
Learning service, you will be happy to know that there is built-in
support for continuous learning.

You can use the Model Builder to train your model and enable
continuous learning without writing a single line of code, but if you
want more control over the life cycle of your machine learning models
(like I do) you can use the Watson Machine Learning libraries and APIs.

To run continuous learning with Watson ML you will need to have

instances of the following services:

1. Watson Machine Learning manages your models and the

continuous learning process.

2. Db2 Warehouse on Cloud stores your training and feedback data.

3. Apache Spark is used by Watson ML to test and train your models.

The following diagram shows how the continuous learning process

works when you build your models in Jupyter notebooks:

https://medium.com/ibm-watson-data-lab/keeping-your-machine-learning-models-up-to-date-f1ead546591b 3/11
1/2/2019 Keeping Your Machine Learning Models Up-To-Date – The Data Lab – Medium

The ML continuous learning process on the IBM Cloud.

Let’s break this down:

1. You start by storing your training data in a table in Db2 Warehouse

on Cloud. When you are ready to train your model, pull your training
data into a Jupyter Notebook.

1. Load training data.

2. In your notebook, build your model using Spark ML. Spark ML is

currently the only machine learning library that Watson ML supports
for continuous learning.

https://medium.com/ibm-watson-data-lab/keeping-your-machine-learning-models-up-to-date-f1ead546591b 4/11
1/2/2019 Keeping Your Machine Learning Models Up-To-Date – The Data Lab – Medium

2. Build model.

3. Deploy your model to Watson ML. When deploying your model tell
Watson ML where to nd the training data (a table in the data
warehouse) as well as where it will nd feedback data (another table in
the warehouse) later for evaluation.

It’s also important to tell Watson ML how to measure model accuracy

and what value represents the current model’s accuracy.

Finally, instruct Watson ML at what accuracy threshold a model should

be retrained. This means if your model is performing below a certain
threshold (i.e., it is not performing well) then Watson ML will retrain
your model and determine if it can improve predictions.

https://medium.com/ibm-watson-data-lab/keeping-your-machine-learning-models-up-to-date-f1ead546591b 5/11
1/2/2019 Keeping Your Machine Learning Models Up-To-Date – The Data Lab – Medium

3. Deploy model and learning con guration.

4. For continuous learning to be e ective you need to have some type of

automated process for consuming new data. This could be a REST API,
a script that downloads data nightly, or any other process that gathers
new data. This is referred to as feedback data. When new feedback data
is received, send it to Watson ML, and Watson ML will store it in the
warehouse.

https://medium.com/ibm-watson-data-lab/keeping-your-machine-learning-models-up-to-date-f1ead546591b 6/11
1/2/2019 Keeping Your Machine Learning Models Up-To-Date – The Data Lab – Medium

4. Provide feedback for future model training.

5. When you determine you have amassed enough feedback data to

test, instruct Watson ML, via a REST API, to start feedback evaluation.

5. Start feedback evaluation.

6. Watson ML pulls any new feedback data and runs predictions against
the current model. (Watson ML will not include previously evaluated
feedback data.)

https://medium.com/ibm-watson-data-lab/keeping-your-machine-learning-models-up-to-date-f1ead546591b 7/11
1/2/2019 Keeping Your Machine Learning Models Up-To-Date – The Data Lab – Medium

6. Get feedback data.

7. After feedback evaluation completes, the accuracy of the data is

measured against the accuracy threshold. If the accuracy is below that
threshold then retraining is triggered.

8. Watson ML then pulls in all training data and all feedback data to
build a new model and measures its accuracy.

7–8. Evaluate feedback data. Train all feedback data and training data.

https://medium.com/ibm-watson-data-lab/keeping-your-machine-learning-models-up-to-date-f1ead546591b 8/11